Setting up a package feed

An embedded system project seldom has the need to introduce changes to the Yocto build system. Most of the time and effort is spent in application development, followed by a lesser amount in system development, maybe kernel and bootloader work.

As such, a whole system rebuild is probably done very few times. A new project is usually built from a prebuilt shared state cache, and application development work only needs to be done to perform full or incremental builds of a handful of packages.

Once the packages are built, they need to be installed on the target system for testing. Emulated machines are fine for application development, but most hardware-dependent work needs to be done on embedded hardware.

Getting ready

An option is to manually copy the build binaries to the target's root filesystem, either copying it to the NFS share on the host system the target is mounting its root filesystem from (as explained in the Configuring network booting for a development setup recipe earlier) or using any other method such as SCP, FTP, or even a microSD card.

This method is also used by IDEs such as Eclipse when debugging an application you are working on, and by the devtool Yocto command-line tool which will be introduced later on. However, this method does not scale well when you need to install several packages and dependencies.

The next option would be to copy the packaged binaries (that is, the RPM, DEB, or IPK packages) to the target's filesystem and then use the target's package management system to install them. For this to work, your target's filesystem needs to be built with package management tools. Doing this is as easy as adding the package-management feature to your root filesystem; for example, you may add the following line to your project's conf/local.conf file:

EXTRA_IMAGE_FEATURES += "package-management"

The default package type in Yocto is RPM, and for an RPM package, you will copy it to the target and use the rpm or dnf utilities to install it. In Yocto 2.4, the default RPM package manager is Dandified Yum (DNF). It is the next generation version of the Yellodog Updater Modified (YUM) and licensed under the General Public License v2.

However, the most convenient way to do this is to convert your host system package's output directory into a package feed. For example, if you are using the default RPM package format, you may convert tmp/deploy/rpm in your build directory into a package feed that your target can use to update.

For this to work, you need to configure an HTTP server on your computer that serves the packages.

Versioning packages

You also need to make sure that the generated packages are correctly versioned, and that means updating the recipe revision, PR, with every change. It is possible to do this manually, but the recommended and compulsory way if you want to use package feeds is to use a PR server.

However, the PR server is not enabled by default. The packages generated without a PR server are consistent with each other but offer no update guarantees for a system that is already running.

The simplest PR server configuration is to run it locally on your host system. To do this, you add the following to your conf/local.conf file:

PRSERV_HOST = "localhost:0" 

With this setup, update coherency is guaranteed for your feed.

If you want to share your feed with other developers, or you are configuring a build server or package server, you would run a single instance of the PR server by running the following command:

$ bitbake-prserv --host <server_ip> --port <port> --start  

And you will update the project's build configuration to use the centralized PR server, editing conf/local.conf as follows:

PRSERV_HOST = "<server_ip>:<port>"  

Also, if you are using a shared state cache as described before, all of the contributors to the shared state cache need to use the same PR server.

Once the feed's integrity is guaranteed, we need to configure an HTTP server to serve the feed.

How to do it...

We will use lighttpd for this example, as it is lightweight and easy to configure. Follow these steps:

1. Install the web server:

$ sudo apt-get install lighttpd  

2. By default, the document root specified in the /etc/lighttpd/lighttpd.conf configuration file is /var/www/html, so we only need a symlink to our package feed:

$ sudo ln -s /opt/yocto/fsl-community-bsp/wandboard/tmp/deploy/rpm /var/www/html/rpm 

3. Next, reload the configuration as follows:

$ sudo service lighttpd reload 

For development, you can also launch a Python HTTP server from the feeds directory as follows:
$ cd /opt/yocto/fsl-community-bsp/wandboard/tmp/deploy/rpm
$ sudo python -m SimpleHTTPServer 80

4. Refresh the package index. This needs to be done manually to update the package feed after every build:

$ bitbake package-index

5. If you want to serve the packages from a different directory instead of directly from your build directory:
   1. You will need to copy the packages:

$ rsync -r -u /opt/yocto/fsl-community-bsp/wandboard/tmp/deploy/rpm/* <new_dir>/

1. 2. Then add the corresponding metadata to the repositories. For that, you will need to install the createrepo tool:

$ sudo apt-get install createrepo

1. 3. And direct it to the new feed directory:

$ createrepo <new_dir>

The createrepo tool will create XML-based metadata from the RPM packages:

You can also build and use the createrepo-c utility from your Yocto build system, a C implementation of createrepo, as follows:
$ bitbake createrepo-c-native -c addto_recipe_sysroot
$ oe-run-native createrepo-c-native createrepo_c <new_dir>

Then we need to configure our target filesystem with the new package feeds:

1. Log in to the target and create a new directory to contain the repository configuration:

$ mkdir -p /etc/yum.repos.d

The repository configuration files will have the following format:

[<repo name>]
name=<Repository description>
baseurl=<url://path/to/repo>
enabled=<0 (disable) or 1 (enabled)>
gpgcheck=<0 (disable signature check) or 1 (enabled)>
gpgkey=<url://path/to/gpg-file if gpgcheck is enabled>  

The previously mentioned baseurl is the complete URL for the repositories, with a http://, https://, ftp://, or file:// prefix.

An example repository configuration file is as follows:

$ vi /etc/yum.repos.d/yocto.repo
[yocto-rpm]                                                       
name=Yocto 2.4: rpm                                                 
baseurl=http://<server-ip>/rpm/                                                                                                                                              

2. Once the setup is ready, we will be able to query and update packages from the target's root filesystem with the following:

# dnf --nogpgcheck makecache
# dnf --nogpgcheck search <package_name>
# dnf --nogpgcheck install <package_name>

By default, dnf is built to use sign package feeds so we need to either configure the preceding repository with:

gpgcheck=0

Or use the --nogpgcheck command line argument as shown previously.

3. To make this change persistent in the target's root filesystem, we can configure the package feeds at compilation time by using the PACKAGE_FEED_* variables in conf/local.conf, as follows:

PACKAGE_FEED_URIS = "http://<server_ip>/"
PACKAGE_FEED_BASE_PATHS = "rpm"

The package feed's base URL is composed as shown next:

${PACKAGE_FEED_URIS}/${PACKAGE_FEED_BASE_PATHS}/${PACKAGE_FEED_ARCHS}. 

By default, the package feed is prepared as a single repository so there is no need to use the PACKAGE_FEED_ARCHS variable.

The variables shown previously will configure the filesystem for any of the supported package formats.

There's more...

The Yocto build system can both generate signed packages and configure target images to use a signed package feed.

The build system will use the GNU privacy guard (GNUPG), an RFC 4880-compliant cryptographic software suite licensed under the GNU General Public License GPLv3.

Generating signed packages

To configure the project for RPM package signing, add the following to your conf/local.conf configuration file:

INHERIT += "sign_rpm"  

For IPK package signing, do the following instead:

INHERIT += "sign_ipk"  

You will then need to define the name of the GPG key to use for signing, and its passphrase:

RPM_GPG_NAME = "<key ID>"
RPM_GPG_PASSPHRASE = "<key passphrase>"  

Or for the IPK package format:

IPK_GPG_NAME = "<key ID>"
IPK_GPG_PASSPHRASE_FILE = "<path/to/passphrase/file>"  

See the Creating a GNUPG key pair next section in this same recipe to find the generated key ID.

The Yocto build system will locate the private GPG key in the host and use it to sign the generated packages.

The Yocto 2.4 release supports signing RPM and IPK packages, but not DEB packages.

Using signed package feeds

To enable your target image to use a signed package feed, you will need to add the following configuration to your conf/local.conf configuration file:

INHERIT += "sign_package_feed"
PACKAGE_FEED_GPG_NAME = "<key name>"
PACKAGE_FEED_GPG_PASSPHRASE_FILE = "<path/to/passphrase/file>" 

The <path/to/passphrase/file> shown previously is the absolute path to a text file containing the passphrase.

The dnf package manager will use the configured public key to verify the authenticity of the package feed.

Creating a GNUPG key pair

In the Setting up the host system recipe in this same chapter, you installed the gnupg package in your host machine; if you didn't, you can do so now with:

$ sudo apt-get install gnupg 

To generate a key, type the following command:

$ gpg --gen-key  

Follow the instructions, keeping the default values. You may need to generate random data with mouse movements and disk activity.

You can check your key with:

$ gpg --list-keys
/home/alex/.gnupg/pubring.gpg
-----------------------------
pub   2048R/4EF0ECE0 2017-08-13
uid                  Alex Gonzalez <[e-mail]>
sub   2048R/298446F3 2017-08-13  

The GPG key ID in the previous example is 4EF0ECE0.

And export it with the following command:

$ gpg --output rpm-feed.gpg --export <id>  

The ID may be the key ID or any part of the user ID, such as the email address. The exported public key may now be moved to its final destination, such as the package feed web server.

An example conf/local.conf configuration would be:

INHERIT += "sign_rpm"
RPM_GPG_NAME = "4EF0ECE0"
RPM_GPG_PASSPHRASE = "<very-secure-password>"
INHERIT += "sign_package_feed"
PACKAGE_FEED_GPG_NAME = "4EF0ECE0"
PACKAGE_FEED_GPG_PASSPHRASE_FILE = "/opt/yocto/passphrase.txt"  

Remember to run the following after rebuilding the image so that the repository feed is signed:
$ bitbake package-index
If you are preparing a repository manually, you will have to sign it too.

Backing up your keys

You can move your key pair to a secure location with:

$ gpg --output rpm-feed.pub --armor --export <key id>
$ gpg --output rpm-feed.sec --armor --export-secret-key <key id>  

Copy them securely to a new location and import them with:

$ gpg --import  rpm-feed.pub
$ gpg --allow-secret-key-import --import  rpm-feed.sec 

See also

• More information and a user manual for the dnf utility can be found at http://dnf.readthedocs.io/en/latest/index.html
• The GNUPG documentation can be accessed at https://www.gnupg.org/documentation/