Uncategorized

Evolution of Android Update System | by Ivan Kuten | Jan, 2021

So, how can you update Android on mobile devices? While developing software for Smart TVs and Android-based set-top boxes, we’ve narrowed it down to four ways, discarding some very exotic options:

Let’s take a closer look at each of the options.

The option with JTAG allows you to update the device on the spot only and requires connecting the Android device to the host, for example, via the USB interface. Since we reflash a flash memory, a new Android version can be put on the firmware with other security keys. You can also choose almost any Android OS version, the assembled firmware, or reconfiguration of flash memory partitions.

However, the JTAG interface is usually available on dev kits only, significantly narrowing this update option’s scope.

The Android bootloader allows you to reflash the flash memory of your device with prepared partition images. This is done using the fastboot protocol or its analogue (in Amlogic’s case, the WorldCup Device protocol). Fastboot, like its counterpart WorldCup Device, is a protocol for communicating with the bootloader via the USB interface or local Ethernet network.

To reflash it, you need to connect your device via USB to the host (there is an option to use LAN Ethernet), put the bootloader into a special update mode, and in this mode, reflash the flash memory of the device.

This method’s advantages and disadvantages are the same as for JTAG: since the update takes place without the Android system’s participation, there are no restrictions related to the version of the system/build or security keys when reflashing.

If the first two updating options have remained unchanged throughout the evolution of Android, the next two options, such as updating via Recovery Mode and OTA, are implemented employing Android itself and have evolved with this OS.

It’s worth mentioning that Recovery Mode and OTA are two different options for calling the Android update engine.

Recovery and the updater engine (bootable/recovery/updater) are the things that started the Android update system (located in bootable/recovery in the AOSP source tree).

An example of flash memory partitioning on a device with Android 6.0:

Android 6.0.1, partition map:

The updating process involves two stages:

When upgrading using the updater engine, the first step is to check the version and digital signature of the image, so you cannot roll back to an older version of the operating system.

With the Recovery system, you can run the updating process either locally, by selecting Recovery Mode in the bootloader and running the updater engine in the Recovery Mode menu, or remotely using OTA when the Android application calls the same updater from Java. And this remote start makes possible a mass update of a whole series of devices. This option is used by operators to update OS for their digital TV solutions on set-top boxes.

The Recovery partition within the non-A/B update scheme is actually a physical partition of the flash memory. With the advent of the A/B system, the Recovery partition was moved to the RAM disk in the device’s random access memory, but the ability to make it a separate physical partition remained.

There is no clear distinction between “old” and “new” in the Android system; it’s more like adding additional features in the Android build configuration while maintaining compatibility with older solutions where possible. However, not all configurations work.

A critical disadvantage of the Recovery or non-A/B System Updates is that with any failure during the update process or with any invalid firmware, a device will require recovery. We will not get a “brick” because you can still run the device in the Recovery Mode from the Recovery section, but the device will not be working properly.

Apparently, the Android developers decided to do something about this because the next step in the evolution of the update system was seamless updates or A/B-scheme updates.

This feature appeared in Android 7.0; it is implemented in the new update_engine, located in the system/update_engine in the AOSP source tree.

So, that is how the upgrade works:

In case of any problems with the update, the bootloader will revert to the old firmware version after several unsuccessful attempts to boot from the new one.

On the official website for developers — Android Source — this process is described in more detail within 9 steps; it also explains how everything works after the reboot.

Android P partition map (recovery):

Android P partition map (A/B scheme):

If to compare these two configurations, you will notice that the data partition in the A/B scheme is 1.6 GB smaller, and this is the price of duplicate system partitions. Whether this is a lot or not is up to everyone to decide, based on their device/project characteristics.

The next changes to the update system took place in Android 8.0. Starting with Android O (8.0) and continuing with Android P (9.0), Google has been implementing its Treble project. The idea of the project is to simplify the technological process of creating an update for Android devices. Google suggested separating parts of the firmware created by different companies with the help of unchangeable interfaces. The process of developing firmware for a particular device can be simplified into the following steps:

The Treble project distinguishes between the Android OS with add-ons from Silicon Manufacturers and code added by Vendors so that the operating system can now receive updates without implementing changes from the device manufacturer.

The separation takes place both through the software interface (switching from Hardware Abstraction Layer 1.0 to HAL2.0) and by allocating separate partitions on the flash memory for Silicon Manufacturer and Vendor (you can see the odm, vendor, product partitions in the Android 9.0 partition map above).

The transition from HAL1.0 to HAL2.0 eliminates the direct linkage with system libraries. Instead, using the IPC Binder, you can connect to system services.

And one more small but useful change: starting from Android 8.0, update_engine added support for the A/B streaming updates, during which it writes directly to the B slot without the need for intermediate data storage in /data. These streaming updates require almost no temporary storage, just 100 kilobytes to store metadata.

This requires that the http server used to download the update should support HTTP range requests, or in other words, download resumption.

The next major step in the development of the Android update system was the Mainline project. It began with Android 10.0 and continued with the current Android 11.0.

The Mainline project allows you to upgrade individual system components without upgrading the entire OS. The necessary data is downloaded via Google Play separately from the OTA firmware update from the manufacturer. It is expected that the direct delivery of updates, which is not tied to hardware parts of Android, will significantly reduce the time it takes to receive updates, increase the speed of vulnerability fixes and reduce dependence on device manufacturers to support the security of the OS.

For the implementation of the Mainline project, selected components of the Android system are converted into modules. Some of these modules have the old APK format, while others are converted to the new APEX format, which differs from APK in its ability to be used in an early system’s boot phase. In case of possible failures, there is a mode of changes rollback.

APEX packets are handled by the APEX manager system service (apexd). It is a native service that, after verification, unpacks the APEX package into the user’s disk space and adds a record of it to its database. On the next system boot, the APEX manager checks all packages from the database, creates the ext4 image for each APEX package and mounts it at /apex/[email protected]

Modules with updates will initially come with open source code; they will be immediately available in the AOSP repositories (Android Open Source Project) and can include improvements and fixes prepared by third parties.

The Mainline project added 13 updatable modules to Android 10, and 11 more modules to Android 11.

One last thing. Rumors of the end of 2020 as a cherry on a cake. Google is converting the Android Runtime to a Mainline module. The Android Runtime or ART is an execution environment for Android applications that includes the compilation of the app bytecode into machine instructions. So there is a chance that in Android 12, you will be able to update ART via Google Play by installing the APEX package.

It is also likely that the Android update system will migrate to Fuchsia, a new OS by Google, which is currently under development. They traditionally copy successful solutions to their other software products. For example, the update_engine for the A/B-scheme, which is currently used in Android, is also used in Chrome, another OS designed by Google. Or one more example: Fuchsia offers the Machina library, which allows you to run Linux programs in a special isolated virtual machine, similar to the way Linux applications are run in Chrome OS.

So, may a successful update be with you and your Android devices!

P. S. Do you remember what they said about it in Indiana Jones? Elsa: Dr. Jones? I knew it was you, you have your father’s eyes. Indiana Jones: And my mother’s ears but the rest belongs to you.

Keep Reading