Book Image

Android System Programming

By : Roger Ye, Shen Liu
Book Image

Android System Programming

By: Roger Ye, Shen Liu

Overview of this book

Android system programming involves both hardware and software knowledge to work on system level programming. The developers need to use various techniques to debug the different components in the target devices. With all the challenges, you usually have a deep learning curve to master relevant knowledge in this area. This book will not only give you the key knowledge you need to understand Android system programming, but will also prepare you as you get hands-on with projects and gain debugging skills that you can use in your future projects. You will start by exploring the basic setup of AOSP, and building and testing an emulator image. In the first project, you will learn how to customize and extend the Android emulator. Then you’ll move on to the real challenge—building your own Android system on VirtualBox. You’ll see how to debug the init process, resolve the bootloader issue, and enable various hardware interfaces. When you have a complete system, you will learn how to patch and upgrade it through recovery. Throughout the book, you will get to know useful tips on how to integrate and reuse existing open source projects such as LineageOS (CyanogenMod), Android-x86, Xposed, and GApps in your own system.
Table of Contents (15 chapters)

The third-party open source projects derived from AOSP

As we know, AOSP source code is the major source that we can start to work with in system-level programming. Various silicon vendors usually work with Google to enable their reference platforms. This is a huge effort and they won't publish everything to the world except for their customers. This brings a limitation to the open source world. Since the AOSP source code is mainly for Google devices, such as emulator, Nexus, or Pixel series, there is no problem for developers who use Nexus devices as hardware reference platforms. How about other devices? Manufacturers may release the kernel source code for their devices, but nothing else. In the open source world, several third-party organizations provide solutions for this situation. We will have a brief look at the ones that we used in this book in the following sections.

LineageOS (CyanogenMod)

LineageOS is a community providing aftermarket firmware distribution for many popular Android devices. It is the successor to the highly popular CyanogenMod. If you cannot build the ROM for your devices from AOSP source code, you may look at LineageOS source code. Because there are many devices supported by LineageOS, many major third-party ROM images are built on top of its predecessor CyanogenMod. From the famous MIUI in China to the latest OnePlus device, they all use CyanogenMod source code as the base start from. The major contributions of LineageOS/CyanogenMod to the open source world are the adaptation of the Linux kernel and HAL to various Android devices.

The source code of LineageOS is maintained in GitHub and you can find it at https://github.com/LineageOS.

To build LineageOS source code for your device, the overall build process is similar to the AOSP build. The key difference is the large number of devices supported by LineageOS. For each device, there is a web page to give information about how to build for a device. We use Nexus 5 as an example. You can go to the following page for detailed information:

https://wiki.lineageos.org/devices/hammerhead

In the information page, you can find information about how to download the ROM image, how to install the image, and how to build the image. There is a build guide for devices and we can find the build guide for Nexus 5 at https://wiki.lineageos.org/devices/hammerhead/build.

To build LineageOS for Nexus 5, the two key elements are Kernel and Device. The Kernel includes the Linux kernel and Nexus 5-specific device drivers, while the Device includes the major part of the device-specific HAL code. The naming convention for both the Kernel and Device folder is android_kernel/device_{manufacturer}_{code name}.

The code name for Nexus 5 is hammerhead and the manufacturer is lge, which is LG.

Other than the Kernel and Device, other important information is the LineageOS version. You may find it on the same device information page. For Nexus 5, the versions that can be used are 11, 12, 12.1, 13, and 14.1. You may be wondering how to match LineageOS versions to AOSP versions.

The information can be found at the following two pages at Wikipedia about CyanogenMod and LineageOS:
https://en.wikipedia.org/wiki/CyanogenMod#Version_history
https://en.wikipedia.org/wiki/LineageOS#Version_history

The LineageOS/CyanogenMod and AOSP versions supported for Nexus 5 are CM11 (Android 4.4), CM 12 (Android 5.0), CM 12.1 (Android 5.1), CM 13 (Android 6.0), and CM 14.1 (Android 7.1.1).

You will not be able to access the links related to CyanogenMod while you read this book, since the infrastructure behind CyanogenMod has been shut down recently. You can read the following post to find out more:
https://plus.google.com/+CyanogenMod/posts/RYBfQ9rTjEH

Nevertheless, the idea from the preceding configuration is that the key pieces of code to differentiate one device from another are the Kernel and Device. It is possible to share the rest of the code across devices. This is one of the goals for the projects in this book. We try to keep the changes for different hardware platforms within the Kernel and the Device, while keeping the rest of the AOSP source code untouched. This is not 100% possible, but we can try to do it as much as possible. The benefit is that we can keep our code separated from AOSP code and it is much easier to update to a new AOSP version.

Android-x86

While LineageOS/CyanogenMod provides excellent support for a large number of Android devices, many of these devices are ARM-based devices from various silicon vendors, such as Qualcomm, Samsung, MTK, and so on. Similarly, there is an open source community for Intel-based Android devices as well. This is another famous open source project, Android-x86. Even though the number of Intel x86-based Android devices on the market cannot compare to the number of ARM-based devices, there is another market using the Intel x86 Android build extensively. This is the Android emulator market. For commercial Android emulator products, you can find AMI DuOS, Genymotion, Andy, and so on.

The project Android-x86 uses a very different approach to support various Intel x86-based devices compared to LineageOS/CyanogenMod. Its goal is to provide Board Support Package (BSP) for any Intel x86 devices. It is similar to how you install Microsoft Windows or Linux on your PC. You have only one copy of the release and you can install it on any Intel PCs. There is no special build of Windows or Linux for each different PC or laptop.

To achieve this goal on Android, Android-x86 customized the Android boot up process significantly. There are two stages of boot up process in Android-x86. The first stage is booting up a minimal Linux environment using a special ramdisk--initrd.img. After the system can boot up to this Linux environment, it starts the second stage through the chroot or switch_root command. In this stage, it will boot up the actual Android system.

This is a very smart way to resolve the new challenge using existing technology. Essentially, we try to resolve the problem in two steps. In the first stage of the boot up process, since both Windows and Linux can boot on Intel x86 PCs without a dedicated build, you should be able to boot Linux on an Intel device without too much effort. This is exactly what the first stage of Android-x86 boot up does. After the minimal Linux system can run properly, this means the minimum set of hardware devices is initialized and you are able to debug or boot the rest of the system using this minimal Linux environment. In the second stage, a common Android image for Intel x86 can be started with limited hardware initialization. This approach can be used in the debugging of hardware devices as well. We will show how we can do the same thing on the Android emulator in this book.

The official website of the Android-x86 project is http://www.android-x86.org/. You can find the information about the Android-x86 project there. To build Android-x86, it is a little tricky to get the source code. The original source code was hosted at http://git.android-x86.org and it was maintained by volunteers from Taiwan Linux User Group (TLUG). It was valid for several years. However, it ceased to work from April 2015.

You can always find the latest status from the Google discussion group at https://groups.google.com/forum/#!forum/android-x86. There is an official announcement about the issue of git.android-x86.org at the discussion group from the maintainer Chih-Wei Huang. Later, the hosting was moved to SourceForge for a short period. However, issues retrieving source code from SourceForge have been reported again since July 2016. Currently, the source code is hosted at OSDN and you can search the announcement from Chih-Wei Huang on September 8, 2016 at the Android-x86 discussion group. Since most open source projects are maintained by volunteers, they may be up and down from time to time. It is always good to keep your own mirror of the projects that you work on. We will discuss this issue in this book as well so that you can have full control of your own work.

We know that many open source projects are related to each other and this is true for both Android-x86 and LineageOS/CyanogenMod as well. Starting from January 2016, Jaap Jan Meijer did the initial porting of CyanogenMod to Android-x86 and this makes CyanogenMod available on most Intel devices. If you are interested in this topic, you can search for CM porting plan in the Android-x86 discussion group.

CWM/CMR/TWRP

As a part of system-level programming, we introduced recovery in the previous section. The original recovery from AOSP only supports very limited functionalities so there are many third-party recovery projects.

ClockworkMod recovery (CWM) is one of the famous open source recovery projects, written by Koushik Dutta. Even though many people still use ClockworkMod recovery now, this project ceased development some time ago.

Another recovery project is CyanogenMod recovery (CMR). CMR is maintained by the CyanogenMod team and it is quite similar to ClockworkMod recovery.

TWRP or TeamWin Recovery Project is another very widely used custom recovery. It is fully touch-driven and has one of the most complete feature sets available. TWRP is the default recovery of OmniROM and its source code is hosted in GitHub as part of OmniROM at https://github.com/omnirom/android_bootable_recovery/.