AssholeDesign

It's not impossible, but there are limitations to that alternative approach. For example, ARM processors are really heterogeneous with respect to exactly which extensions are supported. If you compile on the server, you might not be able to take advantage of features that only exist on a few phones like the latest flagships. It also requires the device to trust that the pre-compiled code is actually consistent and won't cause bad memory access for example that can lead to a crash. If the device compiles the code, it knows there's a true one-to-one correspondence with the actual Java byte code, closing an opportunity for inconsistency. The device knows the size of the CPU caches and might make different instruction scheduling decisions in the actual binary. Generally, it's easier to be more efficient when you have all the information about the target processor available on device. But, you could have done it a different way that's true. Google decided not to do that.

We should also think about what's to be gained by moving it all to the server? On the server, you still don't know the actual user patterns. It's not space efficient to compile the entire application even if the compile time is free on the server. You would have to download unnecessary native code that may only be used by an extremely small fraction of users, wasting storage space on user devices. What if I don't use that whole area in the application? What if most users don't use it? Why should we have to download an optimized version of it onto every device? Perhaps, the best optimization is to not ship any native code for the feature nobody uses.

If you want the best balance of high performance on the user device while not wasting extra space for binary code that isn't frequently used, I think this design makes more sense. You're never getting away from having a full compiler on every device because Google can't assume that you aren't using other app stores, and it's perfectly valid for Android apps to generate code dynamically at runtime that can never be compiled in advance.

I've done a dive of the source code and I can provide a partial answer.

The execution logging functionality is publicly available source code right here.. The code reveals how Android decides what and when to save hot methods.

Dex2oat uses the profile data. You can see references here to the profile directory.

Now, where is the code that ships and downloads these profiles? I can't find it. I suspect it's part of Play services. Thus, AOSP provides all the machinery, but Play probably handles the actual profile data delivery.

Google cannot do this because the code that matters to performance depends on all the inputs which includes the user's actual behavior. For example, I could write an app that contains a big feature that users overwhelmingly care nothing about, wasting time in compiling it and resources in storing the translated code for this unused feature.

Building on the server means giving up access to user behavior, which is important to what code really runs. It's tough to strike a good balance between not storing code that won't be used, install times, and runtime performance.

It's in Settings > General > App install optimisation. The last item in the list.