Testing the recipes

The recipes are tested on state-of-the-art continuous integration (CI) services: Travis (https://travis-ci.org) for GNU/Linux and macOS, Appveyor (https://www.appveyor.com) for Windows, and CircleCI (https://circleci.com) for additional GNU/Linux testing with commercial compilers. The configuration files for the CI services can be found in the repository (https://github.com/dev-cafe/cmake-cookbook/): .travis.yml for Travis, .appveyor.yml for Appveyor, and .circleci/config.yml for CircleCI. Additional installation scripts for Travis and Appveyor can be found in the folder testing/dependencies.

We test the recipes with CMake 3.5.2 and CMake 3.12.1 on the Travis GNU/Linux infrastructure. CMake 3.12.1 is used on the Travis macOS infrastructure. On Appveyor, testing uses CMake 3.11.3. On Circle, CMake 3.12.1 is used.

The testing machinery is a set of Python scripts that are also contained in the testing folder. The script collect_tests.py will run tests and report their statuses. Recipes can be tested in isolation, or in batches; collect_tests.py accepts a regular expression as command-line input, for example:

$ pipenv run python testing/collect_tests.py 'chapter-0[1,7]/recipe-0[1,2,5]'

This command will run tests for Recipes 1, 2, and 5, in Chapters 1 and 7. A sample of the output looks as follows:

To get more verbose output, set VERBOSE_OUTPUT=ON:

$ env VERBOSE_OUTPUT=ON pipenv run python testing/collect_tests.py 'chapter-*/recipe-*'