Developing cookbooks and making sure your nodes converge to the desired state is a complex endeavor. You need transparency about what is happening. This chapter will cover a lot of ways to see what's going on and make sure that everything is working as it should. From running basic checks on your cookbooks to a fully test-driven development approach, we'll see what the Chef ecosystem has to offer.

You know how annoying this is: you tweak a cookbook, run Test Kitchen, and, boom! it fails. What's even more annoying is that it fails only because you missed a mundane comma in the default recipe of the cookbook you just tweaked. Fortunately, there's a very quick and easy way to find such simple glitches before you go all in and try to run your cookbooks on Test Kitchen.

mma@laptop:~/chef-repo $ knife cookbook site install ntp
Installing ntp to /Users/mma/work/chef-repo/cookbooks
…TRUNCATED OUTPUT…
Cookbook ntp version 3.2.0 successfully installed

mma@laptop:~/chef-repo $ cookstyle cookbooks/ntp
Inspecting 5 files
...C.

Offenses:

cookbooks/ntp/recipes/default.rb:25:1: C: Extra blank line detected.

5 files inspected, 1 offense detected 

You might wonder what the proven ways to write cookbooks are. Foodcritic tries to identify possible issues with the logic and style of your cookbooks.

In this section, you'll learn how to use Foodcritic on some existing cookbooks.

mma@laptop:~/chef-repo $ knife cookbook site install mysql 6.0.0
Installing mysql to /Users/mma/work/chef-repo/cookbooks
…TRUNCATED OUTPUT…
Cookbook mysql version 6.0.0 successfully installed

Test-driven development (TDD) is a way to write unit tests before writing any recipe code. By writing the test first, you design what your recipe should do. Then, you ensure that your test fails, while you haven't written your recipe code.

As soon as you've completed your recipe, your unit tests should pass. You can be sure that your recipe works as expected – even if you decide to refactor it later to make it more readable.

ChefSpec is built on the popular RSpec framework and offers a tailored syntax to test Chef recipes.

Let's develop a very simple recipe using the TDD approach with ChefSpec.

Verifying that servers and applications you install are configured correctly and fulfill all compliance requirements by hand is tedious and error-prone. Chef comes with InSpec, a human-readable language for compliance auditing and testing your infrastructure. With InSpec, you can write automated tests to verify a host of criteria on your servers: from the contents of certain files to applications running on certain ports, you can make sure that your servers and applications are configured correctly.

Verifying that your cookbooks actually work when converging a node is essential. Only when you know that you can rely on your cookbooks, are you ready to run them anytime on your production servers.

Test Kitchen is Chef's integration testing framework. It enables you to write tests, which run after a VM is instantiated and converged, using your cookbook. Your tests run in that VM and can verify that everything works as expected.

This is in contrast to ChefSpec, which only simulates a Chef run. Test Kitchen boots up a real node and runs Chef on it. Your InSpec tests run by Test Kitchen see the real thing.

Let's see how you can write such integration tests for your cookbooks.

You tweak a cookbook to support your new server and upload it to your Chef server. Your new node converges just fine and you're happy. Well, until your older production server picks up your modified cookbook during an automated Chef client run and throws a fit. Obviously, you forgot that your old production server was still using the cookbook you tweaked. Luckily, there is the knife preflight command, which can show you all the nodes using a certain cookbook before you upload it to your Chef server.

mma@laptop:~/chef-repo $ chef gem install knife-preflight
Fetching gem metadata from https://rubygems.org/
...TRUNCATED OUTPUT...
Installing knife-preflight (0.1.8)

Even though we do not want to do a full Chef client run, we might need to run, for example, the users cookbook, in order to add a new colleague to a server. This is where the Chef client's feature to override a run list in order to execute a single recipe comes in very handy.

While writing cookbooks, being able to try out parts of a recipe interactively and using breakpoints helps you to understand how your recipes work.

Chef comes with chef-shell, which is essentially an interactive Ruby session with Chef. In chef-shell, you can create attributes, write recipes, and initialize Chef runs, among other things. Chef-shell allows you to evaluate parts of your recipes on-the-fly before uploading them to your Chef server.

why-run mode lets each resource tell you what it would do during a Chef client run, assuming certain prerequisites. This is great because it gives you a glimpse of what might really happen on your node when you run your recipe for real.

However, because Chef converges a lot of resources to a desired state, why-run will never be accurate for a complete run. Nevertheless, it might help you during development while you're adding resources step-by-step to build the final recipe.

In this section, we'll try out why-run mode to see what it tells us about our Chef client runs.

Sometimes you get obscure error messages when running the Chef client and you have a hard time finding any clue about where to look for the error. Is your cookbook broken? Do you have a networking issue? Is your Chef server down? Only by looking at the most verbose log output do you have a chance to find out.

user@server:~$ sudo chef-client -l debug
...TRUNCATED OUTPUT...
[2016-11-14T07:57:36+00:00] DEBUG: Sleeping for 0 seconds
[2016-11-14T07:57:36+00:00] DEBUG: Running Ohai with the following configuration: {:log_location=>#<IO:<STDOUT>>, :log_level=>:debug, ...TRUNCATED OUTPUT...
 [2016-11-14T07:57:37+00:00] DEBUG: Plugin C: ran 'cc -V -flags' and returned 1
[2016-11-14T07:57:37+00:00] DEBUG: Plugin C 'cc -V -flags' failed. Skipping data.
[2016...

When developing new cookbooks, we need to know what exactly went wrong when a Chef client run fails.

Even though Chef prints all the details to stdout, you might want to look at it again, for example, after clearing your shell window.

Nil.each {}

You need to know what exactly happened on your servers. If you want to record every Chef client run and want to see statistics about successful and failed runs, Chef Reporting is your tool of choice. You can even dive into each individual run across your whole organization if you have Reporting enabled for your Chef clients.

mma@laptop:~/chef-repo $ chef gem install knife-reporting
Successfully installed knife-reporting-0.5.0
1 gem installed