Working with build statistics

The build system can collect build information per task and image. The data may be used to identify areas of optimization of build times and bottlenecks, especially when new recipes are added to the system. This recipe will explain how the build statistics work.

How to do it...

To enable the collection of statistics, your project needs to inherit the buildstats class by adding it to USER_CLASSES in your conf/local.conf file. By default, the fsl-community-bsp build project is configured to enable them.

USER_CLASSES ?= "buildstats"

You can configure the location of these statistics with the BUILDSTATS_BASE variable, and by default it is set to the buildstats folder in the tmp directory under the build directory (tmp/buildstats).

The buildstats folder contains a folder per image with the build stats under a timestamp folder. Under it will be a subdirectory per package in your built image, and a build_stats file that contains:

• Host system information
• Root filesystem location and size
• Build time
• Average CPU usage
• Disk statistics

How it works...

The accuracy of the data depends on the download directory, DL_DIR, and the shared state cache directory, SSTATE_DIR, existing on the same partition or volume, so you may need to configure them accordingly if you are planning to use the build data.

An example build-stats file looks like the following:

Host Info: Linux agonzal 3.13.0-35-generic #62-Ubuntu SMP Fri Aug 15 01:58:42 UTC 2014 x86_64 x86_64
Build Started: 1411486841.52
Uncompressed Rootfs size: 6.2M  /opt/yocto/fsl-community- bsp/wandboard-quad/tmp/work/wandboard_quad-poky-linux- gnueabi/core-image-minimal/1.0-r0/rootfs
Elapsed time: 2878.26 seconds
CPU usage: 51.5%
EndIOinProgress: 0
EndReadsComp: 0
EndReadsMerged: 55289561
EndSectRead: 65147300
EndSectWrite: 250044353
EndTimeIO: 14415452
EndTimeReads: 10338443
EndTimeWrite: 750935284
EndWTimeIO: 816314180
EndWritesComp: 0
StartIOinProgress: 0
StartReadsComp: 0
StartReadsMerged: 52319544
StartSectRead: 59228240
StartSectWrite: 207536552
StartTimeIO: 13116200
StartTimeReads: 8831854
StartTimeWrite: 3861639688
StartWTimeIO: 3921064032
StartWritesComp: 0

These disk statistics come from the Linux kernel disk I/O stats (https://www.kernel.org/doc/Documentation/iostats.txt). The different elements are explained here:

• ReadsComp: This is the total number of reads completed
• ReadsMerged: This is the total number of adjacent reads merged
• SectRead: This is the total number of sectors read
• TimeReads: This is the total number of milliseconds spent reading
• WritesComp: This is the total number of writes completed
• SectWrite: This is the total number of sectors written
• TimeWrite: This is the total number of milliseconds spent writing

IOinProgress: This is the total number of I/Os in progress when reading /proc/diskstats

• TimeIO: This is the total number of milliseconds spent performing I/O
• WTimeIO: This is the total number of weighted time while performing I/O

And inside each package, we have a list of tasks; for example, for ncurses-5.9-r15.1, we have the following tasks:

• do_compile
• do_fetch
• do_package
• do_package_write_rpm
• do_populate_lic
• do_rm_work
• do_configure
• do_install
• do_packagedata
• do_patch
• do_populate_sysroot
• do_unpack

Each one of them contain, in the same format as earlier, the following:

• Build time
• CPU usage
• Disk stats

There's more...

You can also obtain a graphical representation of the data using the pybootchartgui.py tool included in the Poky source. From your project's build folder, you can execute the following command to obtain a bootchart.png graphic in /tmp:

$ ../sources/poky/scripts/pybootchartgui/pybootchartgui.py tmp/buildstats/core-image-minimal-wandboard-quad/ -o /tmp