Poky represents the reference build system for the metadata and tools of the Yocto Project, which are used as starting points for anyone interested in interacting with the Yocto Project. It is platform-independent and provides the tools and mechanisms to build and customize the end result, which is in fact a Linux software stack. Poky is used as the central piece of interaction with the Yocto Project.
When working with the Yocto Project as a developer, it is very important to have information about mailing lists and an Internet Relay Chat (IRC) channel. Also, Project Bugzilla can be a source of inspiration in terms of a list of available bugs and features. All of these elements would need a short introduction, so the best starting point would be the Yocto Project Bugzilla. It represents a bug tracking application for the users of the Yocto Project and is the place where problems are reported. The next component is represented by the available channels of IRC. There are two available components on a freenode, one used for Poky and the other for discussions related to the Yocto Project, such as #poky and #yocto, respectively. The third element is represented by the Yocto Project mailing lists, which are used to subscribe to these mailing lists of the Yocto Project:
- http://lists.yoctoproject.org/listinfo/yocto: This refers to the mailing list where the Yocto Project discussions take place
- http://lists.yoctoproject.org/listinfo/poky: This refers to the mailing list where discussions regarding the Poky build of the Yocto Project system take place
- http://lists.yoctoproject.org/listinfo/yocto-announce: This refers to the mailing list where official announcements of the Yocto Project are made, as well as where milestones of the Yocto Project are presented
With the help of http://lists.yoctoproject.org/listinfo, more information can be gathered regarding general and project-specific mailing lists. It contains a list of all the mailing lists available at https://www.yoctoproject.org/tools-resources/community/mailing-lists.
In order to initiate development using the Yocto Project in general, and Poky in particular, you should not only use the previously mentioned components; some information regarding these tolls should also be made available. A very good explanation of the Yocto Project is available on their documentation page at https://www.yoctoproject.org/documentation. Those of you interested in reading a shorter introduction, it may be worth checking out the Embedded Linux Development with Yocto Project, Otavio Salvador and Daiane Angolini, by Packt Publishing.
To use the Yocto Project, a number of specific requirements are needed:
- A host system: Let's assume that this is a Linux-based host system. However, it is not just any host system; Yocto has certain requirements. The supported operating systems are available inside the
poky.conf
file, available inside directorymeta-yocto/conf/distro
. The supported operating systems are defined in theSANITY_TESTED_DISTROS
variable, and a few of these systems are as follows:- Ubuntu-12.04
- Ubuntu-13.10
- Ubuntu-14.04
- Fedora-19
- Fedora-20
- CentOS-6.4
- CentOS-6.5
- Debian-7.0
- Debian-7.1
- Debian-7.2
- Debian-7.3
- Debian-7.4
- Debian-7.5
- Debian-7.6
- SUSE-LINUX-12.2
- openSUSE-project-12.3
- openSUSE-project-13.1
- Required packages: This contains a list of the minimum requirements for the packages available on the host system, besides the ones already available. Of course, this is different from one host system to another and the systems vary according to their purposes. However, for the Ubuntu host, we need the following requirements:
- Essentials: This refers to
sudo apt-get install gawk wget git-core diffstat unzip texinfo gcc-multilib build-essential chrpath socat
- Graphical and Eclipse Plug-in extras: This refers to
sudo apt-get install libsdl1.2-dev xterm
- Documentation: This refers to
sudo apt-get install make xsltproc docbook-utils fop dblatex xmlto
- ADT Installer Extras: This refers to
sudo apt-get install autoconf automake libtool libglib2.0-dev
- Essentials: This refers to
- Yocto Project release: Before staring any work, one of the available Poky releases should be chosen. This book is based on the dizzy branch, which is the Poky 1.7 version, but a developer can chose whatever fits him or her best. Of course, since the interaction with the project is done using the
git
versioning system, the user will first need to clone the Poky repository, and any contributions to the project should be submitted as a patch to the open source community. There is also a possibility of getting a tar archive, but this method has some limitations due to the fact that any changes done on the source are harder to trace, and it also limits the interaction with the community involved in the project.
- Custom Yocto Project kernel interaction: If a developer decides that the kernel source Yocto Projects are maintained and are not suitable for their needs, they could get one of the local copies of the Yocto Project supported by kernel versions, available at http://git.yoctoproject.org/cgit.cgi under the Yocto Linux Kernel section, and modify it according to their needs. These changes, of course, along with the rest of the kernel sources, will need to reside in a separate repository, preferably
git
, and it will be introduced to the Yocto world through a kernel recipe. - The meta-yocto-kernel-extras git repository: Here the metadata needed is gathered when building and modifying kernel images. It contains a bunch of
bbappend
files that can be edited to indicate to the local that the source code has changed, which is a more efficient method to use when you are working on the development of features of the Linux kernel. It is available under the Yocto Metadata Layers section at http://git.yoctoproject.org/cgit.cgi. - Supported Board Support Packages (BSPs): There are a large number of BSP layers that are available and supported by the Yocto Project. The naming of each BSP layer is very simple,
meta-<bsp-name>
, and can be found at http://git.yoctoproject.org/cgit.cgi under the Yocto Metadata Layers section. Each BSP layer is, in fact, a collection of recipes that define the behavior and minimum requirements offered by the BSP provider. More information regarding the development of BSP can be found at http://www.yoctoproject.org/docs/1.7/dev-manual/dev-manual.html#developing-a-board-support-package-bsp. - Eclipse Yocto Plug-ins: For developers who are interested in writing applications, an Eclipse Integrated Development Environment (IDE) is available with Yocto-specific plug-ins. You can find more information on this at http://www.yoctoproject.org/docs/1.7/dev-manual/dev-manual.html#setting-up-the-eclipse-ide.
The development process inside the Yocto Project has many meanings. It can refer to the various bugs and features that are available inside the Yocto Project Bugzilla. The developer can assign one of them to his or her account and solve it. Various recipes can be upgraded, and this process also requires the developer's involvement; new features can also be added and various recipes need to be written by developers. All these tasks need to have a well defined process in place that also involves git
interaction.
To send changes added in the recipes back into the community, the available create-pull-request and send-pull request scripts can be used. These scripts are available inside the poky repository in the scripts directory. Also, in this section, there are also a bunch of other interesting scripts available, such as the create-recipe
script, and others that I will let you discover on your own. The other preferred method to send the changes upstream would be to use the manual method, which involves interaction with git
commands, such as git add
, git commit –s
, git format-patch
, git send-email
, and others.
- System development: This covers the development of the BSP, kernel development, and its configurations. Each of them has a section in the Yocto Project documentation describing respective development processes, as shown at http://www.yoctoproject.org/docs/1.7/bsp-guide/bsp-guide.html#creating-a-new-bsp-layer-using-the-yocto-bsp-script and http://www.yoctoproject.org/docs/1.7/kernel-dev/kernel-dev.html.
- User application development: This covers the development of applications for a targeted hardware device. The information regarding the necessary setup for the application development on the host system is available at http://www.yoctoproject.org/docs/1.7/adt-manual/adt-manual.html. This component will also be discussed in the Eclipse ADT Plug-ins section of this chapter.
- Temporary modification of source code: This covers the temporary modifications that appear in the development process. This involves the solution for various implementation problems that are available in a project's source code. After the problem is solved, the changes need to be available upstream and applied accordingly.
- Development of a Hob image: The Hob build system can be used for operating and customizing system images. It is a graphical interface developed in Python as a more efficient interface with the Bitbake build system.
- Devshell development: This is a method of development that uses the exact environment of the Bitbake build system's tasks. It is one of the most efficient methods used for debugging or package editing. It is also one of the quickest ways to set up the build environment when writing various components of a project.
For operating systems where the provided components are too old to satisfy the requirements of the Yocto Project, a buildtools toolchain is recommended for providing the required versions of the software. There are two methods used for installing a buildtools
tarball. The first method implies the use of an already available prebuilt tarball, and the second one involves building it using the Bitbake build system. More information about this option can be found in the subsections under the Required Git, tar, and Python Versions section of the Yocto documentation mega manual available at http://www.yoctoproject.org/docs/1.7/mega-manual/mega-manual.html#required-git-tar-and-python-versions.
The Application Development Toolkit, also called ADT, provides a cross-development platform suitable for custom build and user-targeted applications. It is comprised of the following elements:
- A cross-toolchain: It is associated with the
sysroot
, both of them being automatically generated using Bitbake, and the target-specific metadata is made available by the target hardware supplier. - The Quick Emulator environment (Qemu): It is used to simulate the target hardware.
- User-space tools: It improves the overall experience of development of an application
- Eclipse IDE: It contains Yocto Project-specific plug-ins
The toolchain can be generated using multiple methods. The most common one is to download the toolchain from http://downloads.yoctoproject.org/releases/yocto/yocto-1.7/toolchain/, and get the appropriate toolchain installer for your host and target. One such example is the poky-glibc-x86_64-core-image-sato-armv7a-vfp-neon-toolchain-1.7.sh
script, which when executed will install the toolchain in the default location of the /opt/poky/1.7/
directory. This location can be changed if proper arguments are offered in the script before starting the execution of the script.
Another method I prefer to use when generating a toolchain involves the use of the Bitbake build system. Here, I am referring to meta-ide-support
. When running bitbake meta-ide-support
, the cross-toolchain is generated and it populates the build directory. After this task is finished, the same result is obtained as in the previously mentioned solution, but in this case, a build directory that is already available is used. The only remaining task for both solutions would be to set up the environment using the script that contains the environment-setup
string and start using it.
The Qemu emulator offers the possibility to simulate one hardware device when this one is not available. There are multiple ways of making it available in the development process:
- Install the ADT using the adt-installer generated script. One of the steps available in this script offers the possibility to enable or disable the use of Qemu in the development process.
- A Yocto Project release is downloaded and in the development process, the environment is set up by default. Then, the Qemu is installed and available for use.
- A
git
clone of the Poky repository is created and the environment is set up. In this case, the Qemu is installed and available also. - The
cross-toolchain
tarball was downloaded, installed, and the environment was set up. This also, by default, enables the use of Qemu and installs it for later use.
The user-space tools are included into the distribution and are used during the development process. They are very common on a Linux platform and can include the following:
- Perf: It is a Linux performance counter that measures certain hardware and software events. More information about this is available at https://perf.wiki.kernel.org/, and also on the profiling and tracing manual of Yocto, where a whole section is devoted to this tool.
- PowerTop: It is a power measurement tool that is used to determine the amount of power a software consumes. More information about it is available at https://01.org/powertop/.
- LatencyTop: It is a similar tool to PowerTop, the difference being that this one focuses on the latency measurement from audio skips and stutters on the desktop to server overload; it has measurement for these kind of scenarios and answers for the latency problems. Although it seems that no commit has been done inside this project since 2009, it is still used today due to the fact that it is very useful.
- OProfile: It represents a system-wide profiler for the Linux ecosystem with a low overhead. More information about it is available at http://oprofile.sourceforge.net/about/. It also has a section available in the profiling and tracing manual of Yocto.
- SystemTap: It offers information on the infrastructure of a running Linux system, as well as the performance and functional problems of the system. It is not available though as an Eclipse extension, but only as a tool inside the Linux distribution. More information about it can be found at http://sourceware.org/systemtap. It also has a section defined in the profiling and tracing manual of Yocto.
- Lttng-ust: It is the user-space tracer for the
lttng
project and offers information related to user-space activities. More information is available at http://lttng.org/.
The last element of the ADT platform is represented by the Eclipse IDE. It is, in fact, the most popular development environment, and it offers full support for the development of the Yocto Project. With the installation of the Yocto Project Eclipse Plug-ins into the Eclipse IDE, the Yocto Project experience is complete. These plugins offer the possibility to cross-compile, develop, deploy, and execute the resultant binary in a Qemu emulated environment. Activities, such as cross-debugging, tracing, remote profiling, and power data collection, are also possible. More information about the activities that appear related to working with Eclipse Plug-ins for the Yocto Project can be found at http://www.yoctoproject.org/docs/1.7/mega-manual/mega-manual.html#adt-eclipse.
The application development process can also be done with other tools that are different from the ones already presented. However, all these options involve the use of a Yocto Project component, most notably the Poby reference system. Therefore, ADT is the suggested, tested, and recommended option by the open source community.
The project—Hob—represents a graphical user interface for the Bitbake build system. Its purpose was to simplify the interaction with the Yocto Project and create a leaner learning curve for the project, allowing users to perform daily tasks in a simpler manner. Its primary focus was the generation of a Linux operating system image. With time, it evolved and can now be considered a tool suitable for both experienced and nonexperienced users. Although I mostly prefer using the command line interaction, this statement does not hold true for all Yocto Project users.
It might seem, though, that Hob development stopped with the release of Daisy 1.6. The development activity somewhat moved to the new project—Toaster—, which will be explained shortly; the Hob project is still in use today and its functionalities should be mentioned. So, the current available version of Hob is able to do the following:
- Visibility for the executed and reused tasks during the build process
- Visibility for build components, such as recipes and packages of an image - this is done in a manner similar to Hob
- Offering information about recipes, such as dependencies, licenses, and so on
- Offering performance-related information, such as disk I/O , CPU usage, and so on
- Presenting errors, warnings, and trace reports for the purpose of debugging
Although it might not seem much, this project promises to offer the possibility to build and customize builds the same way that Hob did, along with many other goodies. You can find useful information about this tool at: https://wiki.yoctoproject.org/wiki/Toaster.
Autobuilder is a project that facilitates the build test automation and conducts quality assurance. Through this internal project, the Yocto community tries to set a path on which embedded developers are able to publish their QA tests and testing plans, develop new tools for automatic testing, continuous integration, and develop QA procedures to demonstrate and show them for the benefit of all involved parties.
These points are already achieved by a project that publishes its current status using this Autobuilder platform, which is available at http://autobuilder.yoctoproject.org/. This link is accessible to everyone and testing is performed on all the changes related to the Yocto Project, as well as nightly builds for all supported hardware platforms. Although started from the Buildbot project, from which it borrowed components for continuous integration, this project promises to move forward and offer the possibility of performing runtime testing and other must-have functionalities.
You can find some useful information about this project at: https://wiki.yoctoproject.org/wiki/AutoBuilder and https://wiki.yoctoproject.org/wiki/QA, which offers access to the QA procedures done for every release, as well as some extra information.
The Lava project is not an internal work of the Yocto Project, but is, in fact, a project developed by Linaro, which is an automated validation architecture aimed towards testing the deployments of Linux systems on devices. Although its primary focus is the ARM architecture, the fact that it is open source does not make it a disincentive. Its actual name is Linaro Automation and Validation Architecture (LAVA).
This is developed with the idea of a continuous evolving architecture that allows test performing along with automation and quality control. At the same time, it offers validation for gathered data. Tests can be anything from compiling a boot test to a change on the kernel scheduler that may or may not have reduced power consumption.
Wic is more of a feature then a project per se. It is the least documented, and if a search is conducted for it, you may find no results. I have decided to mention it here because in the development process, some special requirements could appear, such as generating a custom root
filesystem from available packages (such as .deb
, .rpm
, or .ipk
). This job is the one that is best suited for the wic tool.
A list with all the available projects developed around the Yocto Project can be found at https://www.yoctoproject.org/tools-resources/projects. Some of the projects available there were not discussed in the context of this chapter, but I will let you discover each one of them. There are also other external projects that did not make the list. I encourage you to find out and learn about them on your own.