Overview of the Android system

As we can see from the architecture diagram, the architecture layers of Android include Application Framework, Android System Services, HAL, and Kernel. Binder IPC is used as a mechanism for inter-process communication. We will cover each of them in this section. Since recovery is also part of the system programming scope, we will also give an overview of recovery in this section.

You can find more information about key porting layers and system architecture internals at the following Google website:
http://source.android.com/devices/index.html

Kernel

As we know, Android uses the Linux kernel. Linux was developed by Linus Torvalds in 1991. The current Linux kernel is maintained by the Linux Kernel Organization Inc. The latest mainline kernel releases can be found at https://www.kernel.org.

Android uses a slightly customized Linux kernel. The following is a concise list of the changes to the Linux kernel:

• ashmem (Android Shared Memory): A file-based shared memory system to user space
• Binder: An interprocess communication (IPC) and remote procedure call (RPC) system
• logger: A high-speed in-kernel logging mechanism optimized for writes
• paranoid networking: A mechanism to restrict network I/O to certain processes
• pmem (physical memory): A driver for mapping large chunks of physical memory into user space
• Viking Killer: A replacement OOM killer that implements Android's "kill least recently used process" logic under low-memory conditions
• wakelocks: Android's unique power management solution, in which the default state of the device is sleep and explicit action is required (via a wakelock) to prevent that

Most of the changes were implemented as device drivers with little or no changes necessary to the core kernel code. The only significant subsystem-spanning change is wakelocks.

There are many Android patches accepted by the mainline Linux kernel today. The mainline kernel can even boot up Android directly. There is a blog from Linaro about how to boot Nexus 7 running a mainline kernel. If you want to try it, you can find the instructions at https://wiki.linaro.org/LMG/Kernel/FormFactorEnablement.

If a Linux device driver is available for a hardware device, it usually can work on Android as well. The development of device drivers is the same as the development of a typical Linux device driver. If you want to find out the merges on the mainline kernel related to Android, you can check the kernel release notes at https://kernelnewbies.org/LinuxVersions.

The Android kernel source code is usually provided by SoC vendors, such as Qualcomm or MTK. The kernel source code for Google devices can be found at https://android.googlesource.com/kernel/.

Google devices use SoC from various vendors so that you can find kernel source code from different vendors here. For example, the kernel source of QualComm SoC is under kernel/msm and the kernel source of Mediatek is under kernel/mediatek. The general Android kernel source code is in the folder kernel/common, which looks much like the Vanilla kernel.

The default build of AOSP is for various devices from Google, such as Nexus or Pixel. It started to include some reference boards from silicon vendors as well recently, such as hikey-linaro, and so on. If you need a vendor-specific Android kernel for your reference platform, you should get the kernel source code from your platform vendors.

There are also open source communities maintaining Android kernels. For example, the kernel for the ARM architecture can be found at Linaro for many reference boards. For Intel x86 architecture, you can find various versions of kernels in the Android-x86 project. As you can see from the following Linaro Linux Kernel status website, the linaro-android tree is a forward port of the out-of-tree AOSP patches. It provides a preview of what Google's next AOSP kernel/common.git tree "might" look like.

The Linaro Android kernel tree can be found at https://android.git.linaro.org/gitweb/kernel/linaro-android.git.
The status of this kernel tree can be seen at https://wiki.linaro.org/LMG/Kernel/Upstreaming.

HAL

HAL defines a standard interface for hardware vendors to implement and allows Android to be agnostic about lower-level driver implementations. HAL allows you to implement functionality without affecting or modifying the higher level system. HAL implementations are packaged into module (.so) files and loaded by the Android system at the appropriate time. This is one of the focuses for porting Android systems to a new platform. We will discover more about HAL in Chapter 3, Discovering Kernel, HAL, and Virtual Hardware. Throughout this book, I will give a very detailed analysis of the HAL layer for various hardware interfaces.

Android system services

Functionality exposed by application framework APIs communicates with system services to access the underlying hardware. There are two groups of services that application developers may interact mostly with. They are system (services such as window manager and notification manager) and media (services involved in playing and recording media). These are the services that provide application interfaces as part of the Android framework.

Besides these services, there are also native services supporting these system services, such as SurfaceFlinger, netd, logcatd, rild, and so on. Many of them are very similar to Linux daemons that you may find in a Linux distribution. In a complicated hardware module, such as graphic, both system services and native services need to access HAL in order to provide the framework API to the application layer. We will talk about system services when we debug the init process in Chapter 6, Enabling Wi-Fi on the Android Emulator to Chapter 9, Booting Up x86vbox Using PXE/NFS.

Binder IPC

The Binder IPC mechanism allows the application framework to cross process boundaries and call into the Android system services code. This enables high-level framework APIs to interact with Android system services. An Android application usually runs in its own process space. It doesn't have the ability to access system resources or the underlying hardware directly. It has to talk to system services through Binder IPC to access the system resource. Since applications and system services run in different processes, the Binder IPC provides a mechanism for this purpose.

The Binder IPC proxies are the channel by which the application framework can access system services in different process spaces. It does not mean it is a layer between the application framework and system services. Binder IPC is the inter-process communication mechanism that can be used by any process that wants to talk to another process. For example, system services can use Binder IPC to talk to each other as well.

Application framework

The application framework provides APIs to the applications. It is used most often by application developers. After an interface is invoked by the applications, application frameworks talk to the system services through the Binder IPC mechanism. An Android application framework is not just a set of libraries for the application developers to use. It provides much more than that.

The break-through technology that the Android application framework brought to the developer community is a very nice separation between application layers and system layers. As we know Android application development uses the Java language and Android applications run in an environment similar to the Java virtual machine. In this kind of setup, the application layer is separated from the system layer very clearly.

The Android application framework also provides a unique programming model together with a tight integration with the integrated development environment (IDE) from Google. With this programming model and related tools, Android developers can work on application development with great efficiency and productivity. All these are key reasons why Android has gained so much traction in the mobile device world.

I have given an overall introduction to all the layers in the previous Android system architecture diagram. As I mentioned about the scope of Android system programming before, we can consider all programming in Android systems as within the scope of system programming other than application programming. With this concept in mind, we actually missed one piece in the previous architecture diagram, which is recovery.

Recovery

In this chapter, we want to have a brief look at recovery as well, since we have three chapters about it in the second part of this book.

Recovery is a tool that can be used to upgrade or reinstall Android systems. It is part of the AOSP source code. The source code for recovery can be found at $AOSP/bootable/recovery.

The unique point about recovery compared to the other parts of Android is that it is a self-contained system by itself. We can look at recovery using the following diagram, and compare it to the Android and Brillo architectures that we talked about before:

Recovery is a separate system from Android that shares the same kernel with the Android system that it supports. We can treat it as a mini operating system or an embedded application that we can find in many embedded devices. It is a dedicated application running on top of the same Linux kernel as Android and it performs a single task, which is to update the current Android system.

When the system boots to recovery mode, it boots from a dedicated partition in the flash. This partition includes the recovery image that includes a Linux kernel and a special ramdisk image. If we look at Nexus 5 partitions, we will see the following list:

# parted /dev/block/mmcblk0
parted /dev/block/mmcblk0
GNU Parted 1.8.8.1.179-aef3
Using /dev/block/mmcblk0
Welcome to GNU Parted! Type 'help' to view a list of commands.
(parted) print
print
print
Model: MMC SEM32G (sd/mmc)
Disk /dev/block/mmcblk0: 31.3GB
Sector size (logical/physical): 512B/512B
Partition Table: gpt

Number  Start   End     Size    File system  Name      Flags
 1      524kB   67.6MB  67.1MB  fat16        modem
 2      67.6MB  68.7MB  1049kB               sbl1
 3      68.7MB  69.2MB  524kB                rpm
 4      69.2MB  69.7MB  524kB                tz
 5      69.7MB  70.3MB  524kB                sdi
 6      70.3MB  70.8MB  524kB                aboot
7      70.8MB  72.9MB  2097kB               pad
8      72.9MB  73.9MB  1049kB               sbl1b
9      73.9MB  74.4MB  524kB                tzb
10      74.4MB  75.0MB  524kB                rpmb
11      75.0MB  75.5MB  524kB                abootb
12      75.5MB  78.6MB  3146kB               modemst1
13      78.6MB  81.8MB  3146kB               modemst2
14      81.8MB  82.3MB  524kB                metadata
15      82.3MB  99.1MB  16.8MB               misc
16      99.1MB  116MB   16.8MB  ext4         persist
17      116MB   119MB   3146kB               imgdata
18      119MB   142MB   23.1MB               laf
19      142MB   165MB   23.1MB               boot

20      165MB   188MB   23.1MB               recovery

21      188MB   191MB   3146kB               fsg
22      191MB   192MB   524kB                fsc
23      192MB   192MB   524kB                ssd
24      192MB   193MB   524kB                DDR
25      193MB   1267MB  1074MB  ext4         system
26      1267MB  1298MB  31.5MB               crypto
27      1298MB  2032MB  734MB   ext4         cache
28      2032MB  31.3GB  29.2GB  ext4         userdata
29      31.3GB  31.3GB  5632B                grow

The list includes 29 partitions and recovery partition is one of them. The recovery ramdisk of recovery, it has a similar directory structure to the normal ramdisk. In the init script of recovery ramdisk, init starts the recovery program and it is the main process of the recovery mode. The recovery itself is the same as other native daemons in the Android system. The programming for recovery is part of the scope of Android system programming. The programming language and debug method for recovery is also the same as native Android applications. We will discuss this in more depth in the second part of this book.