My take is that Java -> Kotlin was a very natural progression, as they are very similar languages with an identical runtime in the jvm. Kotlin is just a better Java, which is incredibly easy to adopt into an existing Java ecosystem.

But then they also wanted a more bare metal environment in addition to that. So something like C, C++, or Rust.

It's always been unclear whether Go belongs in that list. It was conceived or at least positioned as a "systems" language like those I listed, and there was noise early on about it replacing C++ codebases at Google, but with over a decade of hindsight now, I think in practice it's pretty clearly a Java replacement rather than a C++ replacement. (I guess there's a fun symmetry here in that Java was also conceived as a C++ replacement and also wasn't.) But Kotlin is just a much more natural way to fill the Java replacement niche on Android.

I dont have first hand knowledge, but I've been in the space for 10 years and wrote Go, Java, Kotlin, so here is my take.

Kotlin made a huge effort to stay compatible with Java. You can call Kotlin code from Java and Java code from Kotlin, so orgs could migrate their apps piece by piece. The only real incompatibilities that I've experienced were when going from C code via JNI into Kotlin. All sorts of really weird issues creep up.

There was no boiling the ocean in any of the large apps I worked on at Amazon and Meta - just add Kotlin and see how it goes, it's a really easy sell to tech VPs. First we added Kotlin just for the unit tests, then for small features and eventually with Google releasing Jetpack with Kotlin extensions, for all new apps and features.

Kotlin also improved upon existing painpoints in Java and provided a ton of syntactic sugar for things that were tedious in Java. For example, proper get/set syntax for fields, data classes with autogenerated equals/hashcode, explicit nullability for types, val with 'final' semantics to get rid of all the final variables, fields in interfaces, top level functions, etc. All the functional stuff in it is nice too.

Go is much leaner than Kotlin. The VM or performance is not an issue - Kotlin generates entire classes for simple syntax, to make things convenient and compatible, but the bytecode is pretty bloated.

I imagine that Google had sufficient resources to make a Go<->Java translation layer (though Go would have needed generics) but transitioning a huge API like Android is already a large effort, and if the language isn't fully compatible with the existing code, it would have added an order of magnitude of complexity, so Kotlin was a much easier sell.

In my opinion, migrating to Dart would have been more practical than Go (as evidenced by Flutter), so thats the real head-scratcher. It would have been easy for Java devs to learn Dart because the OOP concepts translate directly.

Other factors:

Google already had a relationship with JetBrains because of Android Studio.

The Oracle Google lawsuit was happening at the time and I speculate that Google needed to show to Oracle they can move off of Java. Kotlin happened to be available.

The lawsuit excuse has always been quite a lame one, because regardless of Kotlin or Java, the whole Android development stack depend on the Java ecosystem for the IDE, Gradle, Maven Central based libraries, and Android team never made any effort to port anything on top of Kotlin/Native, or even improve it.

Most system APIs are only exposed in the JVM environment without native equivalents. You do need a JVM language to do anything meaningful. Either that, or you'll have to use JNI.

because some people in charge of android mistakenly believed that because some stats claimed Java was the most popular language that chosing Java would lead to the most developers for android (and then they chose Kotlin later beacuse it's a better Java and compatible with the JVM)

IMO, choosing Java was one of the worst decisions they ever made. There's a reason why no other platform (PS5, XBox, iPhone, Mac, Linux, etc..) don't choose a language like Java. Android has so much jank because they chose Java

Choosing Java made a lot of sense when Android got started. The binary architecture for mobile devices was not standard in any way, ARM, MIPS, x86 were all in common use at some time or another. And nowadays AArch64 (which is not the same as older ARM) is quite widespread. Plus there was nothing like WASM back then, so if you wanted binary independence for apps and the like you were stuck with something very much like Java.

Not sure why they picked Kotlin (probably JVM interop) or Rust, but Go's thread safety is closer to C++ than Rust. I think of it as an excellent python replacement, not a C++ replacement.