Guidelines to follow for Yocto Project

This section aims to gather some guidelines for aspects of the Yocto Project metadata and project organization tips that make our life easier in terms of short- and long-term maintenance.

Managing layers

As our journey in product development advances, we will naturally use multiple repositories to meet the needs we face. Keeping track of the repositories is a complex challenge as we need to do the following:

• Make sure we can reproduce a previous build in the future
• Allow multiple team members to work in the same code base
• Validate the changes we make using Continuous Integration tools
• Avoid subtle changes in the layers we use

Those goals are intimidating, but a few tools are in use, with different strategies to overcome those challenges.

The simplest solution uses the image-buildinfo class (https://docs.yoctoproject.org/4.0.4/ref-manual/classes.html#image-buildinfo-bbclass), which writes a plain text file containing build information and layers revisions to the target filesystem at ${sysconfdir}/buildinfo by default. Some tools have been developed that can help this process. These tools are discussed as follows:

• Google developed the repo (https://source.android.com/docs/setup/download#repo) tool for Android development. It has been adopted for use in other projects. A critical aspect of repo is that it requires some tooling to integrate with Yocto Project-based projects to automate the build directory and environment configuration. See the O.S. Systems Embedded Linux project (https://github.com/OSSystemsEmbeddedLinux/ossystems-embedded-linux-platform) as inspiration for using repo in your projects.
• Siemens developed kas (https://github.com/siemens/kas) to provide an easy mechanism for downloading sources, automating the build directory and environment configuration, and so on.
• Garmin developed Whisk (https://github.com/garmin/whisk) to manage complex product configurations using OpenEmbedded and the Yocto Project. The key features are a single source tree, multiple axes of configuration, multiple product builds, isolated layer configuration, and so on.
• Agilent developed Yocto Buddy (https://github.com/Agilent/yb). The design aims to ease the setup and keep Yocto Project-based environments synchronized. Yocto Buddy was inspired by all previously mentioned tools and is still early in development.

This is a subset of existing tools and shouldn’t be considered a complete list. Ideally, you should play with them before deciding, as the choice depends on the project use case and team expertise.

Avoid creating too many layers

A significant advantage of the Yocto Project is that it has the ability to use and create multiple layers. It allows us to do the following:

• Reuse BSP layers from semiconductor vendors
• Reduce duplication of work by sharing reusable blocks to enable the use of new or specific applications, programming languages, and so on.

However, creating multiple layers may be unproductive when developing a project or a set of products. For example, the development of BSP-only layers makes sense in the following situations:

• The board is the product, as in the System on Module (SoM) vendors’ case
• When external access to the layer is critical, however, we want to limit the access for the non-BSP source

Using a single layer for the product, or even the company, has many advantages, such as the following:

• Facilitating the development of reusable components such as a packagegroup package for development tools or network utilities shared by multiple products
• Reducing the risk of unexpected side effects due to changes for a specific product or board
• Increasing the reuse of bug fixes across multiple products and reuse of BSP low-level components such as the Linux kernel or bootloader
• Boosting standardization across multiple products, reducing the learning curve for new team members

The decision to use one or more layers depends on several aspects; however, we recommend starting simple and, in the future, splitting the layer if required.

Prepare the product metadata for new Yocto Project releases

As our product grows, so does our metadata and the need for good organization. Some use cases commonly seen during product development are as follows:

• The need to backport a new recipe version due to a bug fix or a feature
• A missing package configuration or bug fix is not yet available in the Yocto Project recipe

We use two recipe directories to organize this kind of content:

• recipes-backport: Backports of recipes coming from new Yocto Project releases
• recipes-staging: New recipes or bbappend files adding missing package configurations or bug fixes

We continuously send new recipes or bug fixes from recipes-staging to the respective upstream project (for example, OpenEmbedded Core). Then, when the patch is accepted, we move this change from recipes-staging to the recipes-backport directory. This approach allows us to keep track of pending upstreaming tasks and easily upgrade our meta layer to a new Yocto Project release. Furthermore, we can quickly act on the backport directory and remove it.

Create your custom distro

When using the Yocto Project, we usually add many configurations in build/conf/local.conf. However, as discussed in the book, this is bad as it is not at source control management and is likely to differ among developers. Using a custom distribution has many benefits, and some of them are highlighted here:

• Allows consistent use among multiple developers
• Provides a clear view of the different DISTRO_FEATURES we use when compared to our base distribution (for example, poky)
• Provides a central place where we can have a global view of all the required recipe configurations we need for our product, reducing the number of bbappend files required to configure our recipes (for example, PACKAGECONFIG:pn-<myrecipe>:append = " myfeature")

Besides those more technical aspects, using a custom distro also allows the proper branding of SDK or other Yocto Project-generated artifacts.

We learned how to create a custom distribution in the Using a custom distribution sectiown in Chapter 12, Creating Custom Layers.

Avoid reusing existing images for your product

Images are where everything fits together. When we are developing a product, it is important to minimize the number of packages we have installed in our images for multiple reasons:

• Reducing the rootfs size
• Reducing the build time
• Reducing the number of licenses to deal with
• Reducing the surface of attack for security breaches

A typical starting point is copying the core-image-base.bb file to our custom layer as myproduct-image.bb and extending it, adding what we need for the product’s image. In addition, we create an image called myproduct-image-dev.bb for use during development and make sure it requires myproduct-image.bb along with the artifacts used only for development, avoiding code duplication. This way, we have two images for production and development, but they share the same core features and packages.

Standard SDK is commonly undervalued

Application development implies an interactive process, mainly because we usually continuously build the application until we accomplish what we aim for. This use case is not well suited for the Yocto Project, mainly for the following reasons:

• Every time we start the build of a recipe, it discards the previous build objects
• The time needed for deploying the application or image is much longer
• A lack of proper integration in the IDE environment

There are alternatives for a few of those topics, such as using devtool to reuse the build objects and helping to deploy the application. We saw how to use devtool in the Deploying to the target using devtool and Building a recipe using devtool sections from Chapter 9, Developing with the Yocto Project, but the development experience is still cumbersome.

Using Standard SDK for application and other components’ development, such as the Linux kernel and bootloader, is still preferable. This way, we focus on faster development, postponing or parallelizing the Yocto Project integration task.

Avoid too many patches for Linux kernel and bootloader modifications

The need for patches in the Linux kernel and bootloader is inherent to embedded Linux development, as we rarely use the hardware without any changes. The level of modification on those components is related to your hardware design, for example:

• Using a Single-Board Computer (SBC), the number of changes should be minimal
• In the use of System-On-Module (SOM) with a custom baseboard, the number of changes could vary depending on the number of modifications from the vendor baseboard hardware design
• Ultimately, the use of custom hardware design implies the development of a custom BSP and, consequently, a considerable number of modifications

Those are not set in stone. So, for example, consider starting the project using an SBC. Later, we find out that the vendor does not provide a good reference BSP, so the number of modifications and amount of work for the BSP will increase considerably.

When we have small changes, it is better to tackle the changes as patch files added to the component recipe. But when the effort to maintain the component increases, it justifies having a separate fork of that component to keep all the changes in place. Using a repository fork gives us the following advantages:

• The history of the changes
• Different branches or tags for development and production
• The possibility of merging with other providers
• It allows the use of much simpler recipes, as we don’t need to carry on individual patches

In summary, we should use the strategy that makes sense for the project. Eventually, this will change, but using the right approach reduces the total effort to support the hardware in use properly.

Avoid using AUTOREV as SRCREV

The use of AUTOREV as SRCREV is usually applied when developing a product. We must interactively change the code and try that code inside the Yocto Project. That said, this comes with a couple of drawbacks:

• It is hard to reproduce the previous build as every time we rebuild our image, it may use a different revision for our recipe.
• The AUTOREV value is only applied when BitBake invalidates the cache of a specific recipe. That happens when we modify the recipe itself or when we change something that triggers the BitBake cache rebuild, such as changing any .conf file.

Those drawbacks make AUTOREV very fragile, and other alternatives can cover the interactive code change more consistently. Typically, devtool is used as it allows us to change the code directly in the workspace and forces the recipe to use this as the source. Another alternative is to use the externalsrc.bbclass class (https://docs.yoctoproject.org/4.0.4/index.html#ref-classes-externalsrc), which allows us to configure a recipe to use a directory as the source for the build.

Create a Software Bill of Materials

The Poky build system can describe all the components used in an image from the licenses for each software component. This description is generated as a Software Bill of Materials (SBOM) using the Software Package Data Exchange (SPDX) standard (https://spdx.dev/). Using the SPDX format has the advantage of leveraging existing tooling, allowing extra automation, which is impossible using Poky’s standard license output format.

The SBOM is critical to ensure open source license compliance. However, the SBOM is not generated by default. You can refer to the Creating a Software Bill of Materials section from The Yocto Project Development Tasks Manual (https://docs.yoctoproject.org/4.0.4/dev-manual/common-tasks.html#creating-a-software-bill-of-materials).