Using build history
When maintaining software for an embedded product, you need a way to know what has changed and how it is going to affect your product.
On a Yocto system, you may need to update a package revision (for instance, to fix a security vulnerability), and you need to make sure what the implications of this change are; for example, in terms of package dependencies and changes to the root filesystem.
Build history enables you to do just that, and we will explore it in this recipe.
How to do it...
To enable build history, add the following to your conf/local.conf file:
INHERIT += "buildhistory"
The following enables information gathering, including dependency graphs:
BUILDHISTORY_COMMIT = "1"
The preceding line of code enables the storage of build history in a local Git repository.
The Git repository location can be set by the BUILDHISTORY_DIR variable, which by default is set to a buildhistory directory on your build directory.
By default, buildhistory tracks changes to packages, images, and SDKs. This is configurable using the BUILDHISTORY_FEATURES variable. For example, to track only image changes, add the following to your conf/local.conf:
BUILDHISTORY_FEATURES = "image"
It can also track specific files and copy them to the buildhistory directory. By default, this includes only /etc/passwd and /etc/groups, but it can be used to track any important files like security certificates. The files need to be added with the BUILDHISTORY_IMAGE_FILES variable in your conf/local.conf file as follows:
BUILDHISTORY_IMAGE_FILES += "/path/to/file"
Build history will slow down the build, increase the build size, and may also grow the Git directory to an unmanageable size. The recommendation is to enable it on a build server for software releases, or in specific cases, such as when updating production software.
How it works...
When enabled, it will keep a record of the changes to each package and image in the form of a Git repository in a way that can be explored and analyzed.
For a package, it records the following information:
- Package and recipe revision
- Dependencies
- Package size
- Files
For an image, it records the following information:
- Build configuration
- Dependency graphs
- A list of files that includes ownership and permissions
- List of installed packages
And for an SDK, it records the following information:
- SDK configuration
- List of both host and target files, including ownership and permissions
- Dependency graphs
- A list of installed packages
Looking at the build history
Inspecting the Git directory with the build history can be done in several ways:
- Using Git tools like gitk or git log.
- Using the buildhistory-diff command-line tool, which displays the differences in a human-readable format.
- Using a Django-1.4-based web interface. You will need to import the build history data to the application's database after every build. The details are available at http://git.yoctoproject.org/cgit/cgit.cgi/buildhistory-web/tree/README.
There's more...
To maintain the build history, it's important to optimize it and avoid it from growing over time. Periodic backups of the build history and clean-ups of older data are important to keep the build history repository at a manageable size.
Once the buildhistory directory has been backed up, the following process will trim it and keep only the most recent history:
- Copy your repository to a temporary RAM filesystem (
tmpfs) to speed things up. Check the output of thedf -hcommand to see which directories aretmpfsfilesystems and how much space they have available, and use one. For example, in Ubuntu, the/run/shmdirectory is available. - Add a graft point for a commit one month ago with no parents:
$ git rev-parse "HEAD@{1 month ago}" > .git/info/grafts - Make the graft point permanent:
$ git filter-branch - Clone a new repository to clean up the remaining Git objects:
$ git clone file://${tmpfs}/buildhistory buildhistory.new - Replace the old
buildhistorydirectory with the new cleaned one:$ rm -rf buildhistory $ mv buildhistory.new buildhistory
