Chapter 2. Evaluating and Troubleshooting Cookbooks and Chef Runs

"Most people spend more time and energy going around problems than in trying to solve them."
Henry Ford

In this chapter, we'll cover the following recipes:

Testing your Chef cookbooks with cookstyle and Rubocop
Flagging problems in your Chef cookbooks with Foodcritic
Test-driven development for cookbooks using ChefSpec
Compliance testing with InSpec
Integration-testing your Chef cookbooks with Test Kitchen
Showing affected nodes before uploading cookbooks
Overriding a node's run list to execute a single recipe
Using chef-shell
Using why-run mode to find out what a recipe might do
Debugging Chef client runs
Inspecting the results of your last Chef run
Using Reporting to keep track of all your Chef client runs
Raising and logging exceptions in recipes
Diff-ing cookbooks with knife
Using community exception and...

Introduction

Developing cookbooks and making sure your nodes converge to the desired state is a complex endeavor. You need transparency about what is happening. This chapter will cover a lot of ways to see what's going on and make sure that everything is working as it should. From running basic checks on your cookbooks to a fully test-driven development approach, we'll see what the Chef ecosystem has to offer.

Testing your Chef cookbooks with cookstyle and Rubocop

You know how annoying this is: you tweak a cookbook, run Test Kitchen, and, boom! it fails. What's even more annoying is that it fails only because you missed a mundane comma in the default recipe of the cookbook you just tweaked. Fortunately, there's a very quick and easy way to find such simple glitches before you go all in and try to run your cookbooks on Test Kitchen.

Getting ready

Install the ntp cookbook by running the following command:

mma@laptop:~/chef-repo $ knife cookbook site install ntp
Installing ntp to /Users/mma/work/chef-repo/cookbooks
…TRUNCATED OUTPUT…
Cookbook ntp version 3.2.0 successfully installed

How to do it…

Carry out the following steps to test your cookbook; run cookstyle on the ntp cookbook:

mma@laptop:~/chef-repo $ cookstyle cookbooks/ntp
Inspecting 5 files
...C.

Offenses:

cookbooks/ntp/recipes/default.rb:25:1: C: Extra blank line detected.

5 files inspected, 1 offense detected...

Flagging problems in your Chef cookbooks with Foodcritic

You might wonder what the proven ways to write cookbooks are. Foodcritic tries to identify possible issues with the logic and style of your cookbooks.

In this section, you'll learn how to use Foodcritic on some existing cookbooks.

Getting ready

Install version 6.0.0 of the mysql cookbook by running the following code:

mma@laptop:~/chef-repo $ knife cookbook site install mysql 6.0.0
Installing mysql to /Users/mma/work/chef-repo/cookbooks
…TRUNCATED OUTPUT…
Cookbook mysql version 6.0.0 successfully installed

How to do it…

Let's see how Foodcritic reports findings:

Run foodcritic on your cookbook:

mma@laptop:~/chef-repo $ foodcritic ./cookbooks/mysql
...TRUNCATED OUTPUT...
FC001: Use strings in preference to symbols to access node attributes: ./cookbooks/mysql/libraries/helpers.rb:273
FC005: Avoid repetition of resource declarations: ./cookbooks/mysql/libraries/provider_mysql_service.rb:77
...TRUNCATED OUTPUT...

Introduction

Developing cookbooks and making sure your nodes converge to the desired state is a complex endeavor. You need transparency about what is happening. This chapter will cover a lot of ways to see what's going on and make sure that everything is working as it should. From running basic checks on your cookbooks to a fully test-driven development approach, we'll see what the Chef ecosystem has to offer.

Testing your Chef cookbooks with cookstyle and Rubocop

You know how annoying this is: you tweak a cookbook, run Test Kitchen, and, boom! it fails. What's even more annoying is that it fails only because you missed a mundane comma in the default recipe of the cookbook you just tweaked. Fortunately, there's a very quick and easy way to find such simple glitches before you go all in and try to run your cookbooks on Test Kitchen.

Getting ready

Install the ntp cookbook by running the following command:

mma@laptop:~/chef-repo $ knife cookbook site install ntp
Installing ntp to /Users/mma/work/chef-repo/cookbooks
…TRUNCATED OUTPUT…
Cookbook ntp version 3.2.0 successfully installed

How to do it…

Carry out the following steps to test your cookbook; run cookstyle on the ntp cookbook:

mma@laptop:~/chef-repo $ cookstyle cookbooks/ntp
Inspecting 5 files
...C.

Offenses:

cookbooks/ntp/recipes/default.rb:25:1: C: Extra blank line detected.

5 files inspected, 1 offense detected

How it works…

Cookstyle is...

Flagging problems in your Chef cookbooks with Foodcritic

You might wonder what the proven ways to write cookbooks are. Foodcritic tries to identify possible issues with the logic and style of your cookbooks.

In this section, you'll learn how to use Foodcritic on some existing cookbooks.

Getting ready

Install version 6.0.0 of the mysql cookbook by running the following code:

mma@laptop:~/chef-repo $ knife cookbook site install mysql 6.0.0
Installing mysql to /Users/mma/work/chef-repo/cookbooks
…TRUNCATED OUTPUT…
Cookbook mysql version 6.0.0 successfully installed

How to do it…

Let's see how Foodcritic reports findings:

Run foodcritic on your cookbook:

mma@laptop:~/chef-repo $ foodcritic ./cookbooks/mysql
...TRUNCATED OUTPUT...
FC001: Use strings in preference to symbols to access node attributes: ./cookbooks/mysql/libraries/helpers.rb:273
FC005: Avoid repetition of resource declarations: ./cookbooks/mysql/libraries/provider_mysql_service.rb:77
...TRUNCATED OUTPUT...

Get a detailed list of the...

Test-driven development for cookbooks using ChefSpec

Test-driven development (TDD) is a way to write unit tests before writing any recipe code. By writing the test first, you design what your recipe should do. Then, you ensure that your test fails, while you haven't written your recipe code.

As soon as you've completed your recipe, your unit tests should pass. You can be sure that your recipe works as expected – even if you decide to refactor it later to make it more readable.

ChefSpec is built on the popular RSpec framework and offers a tailored syntax to test Chef recipes.

Let's develop a very simple recipe using the TDD approach with ChefSpec.

Getting ready

Make sure you have a cookbook called my_cookbook and the run_list of your node includes my_cookbook, as described in the Creating and using cookbooks recipe in Chapter 1, Chef Infrastructure.

How to do it…

Let's write a failing test first. Then we will write a recipe that satisfies the test:

Create your spec file:
```
mma@laptop:~/chef-repo ...
```

Compliance testing with InSpec

Verifying that servers and applications you install are configured correctly and fulfill all compliance requirements by hand is tedious and error-prone. Chef comes with InSpec, a human-readable language for compliance auditing and testing your infrastructure. With InSpec, you can write automated tests to verify a host of criteria on your servers: from the contents of certain files to applications running on certain ports, you can make sure that your servers and applications are configured correctly.

Getting ready

Make sure you have ChefDK installed, as described in the Installing the Chef Development Kit on your workstation recipe in Chapter 1, Chef Infrastructure.

How to do it…

Let's create a very simple compliance requirement as code and run it on your local workstation:

Create a new profile for your InSpec tests:

mma@laptop:~/chef-repo $ inspec init profile my_profile
Create new profile at /Users/mma/work/chef-repo/my_profile
 * Create directory controls
 * Create...

Integration-testing your Chef cookbooks with Test Kitchen

Verifying that your cookbooks actually work when converging a node is essential. Only when you know that you can rely on your cookbooks, are you ready to run them anytime on your production servers.

Test Kitchen is Chef's integration testing framework. It enables you to write tests, which run after a VM is instantiated and converged, using your cookbook. Your tests run in that VM and can verify that everything works as expected.

This is in contrast to ChefSpec, which only simulates a Chef run. Test Kitchen boots up a real node and runs Chef on it. Your InSpec tests run by Test Kitchen see the real thing.

Let's see how you can write such integration tests for your cookbooks.

Getting ready

Make sure you have a cookbook named my_cookbook, as described in the Creating and using cookbooks recipe in Chapter 1, Chef Infrastructure.

Make sure you have Vagrant installed on your workstation, as described in the Managing virtual machines with Vagrant...

Showing affected nodes before uploading cookbooks

You tweak a cookbook to support your new server and upload it to your Chef server. Your new node converges just fine and you're happy. Well, until your older production server picks up your modified cookbook during an automated Chef client run and throws a fit. Obviously, you forgot that your old production server was still using the cookbook you tweaked. Luckily, there is the knife preflight command, which can show you all the nodes using a certain cookbook before you upload it to your Chef server.

Getting ready

For the following example, we assume that you have multiple servers with the ntp cookbook in their run list (either directly or via roles).

Use Chef to install the knife-preflight gem. It contains the preflight plugin extending knife with additional commands:

mma@laptop:~/chef-repo $ chef gem install knife-preflight
Fetching gem metadata from https://rubygems.org/
...TRUNCATED OUTPUT...
Installing knife-preflight (0.1.8)

How to do it...

Overriding a node's run list to execute a single recipe

Even though we do not want to do a full Chef client run, we might need to run, for example, the users cookbook, in order to add a new colleague to a server. This is where the Chef client's feature to override a run list in order to execute a single recipe comes in very handy.

Note

Only use this feature when you absolutely must! It is bad practice because it breaks the principles of desired state config and single source of truth.

Getting ready

To follow along with the following example, you'll need a node hooked up to your Chef server having multiple recipes and/or roles in its run list.

How to do it...

Let's see how to run a single recipe out of a bigger run list on your node:

Show the data for your node. In this example, the node has the role base in its run list. Depending on your setup, you'll find other data here:
```
mma@laptop:~/chef-repo $ knife node show www.example.com
...TRUNCATED OUTPUT…
Run List:    role[base]
Roles:       base
Recipes...
```

Using chef-shell

While writing cookbooks, being able to try out parts of a recipe interactively and using breakpoints helps you to understand how your recipes work.

Chef comes with chef-shell, which is essentially an interactive Ruby session with Chef. In chef-shell, you can create attributes, write recipes, and initialize Chef runs, among other things. Chef-shell allows you to evaluate parts of your recipes on-the-fly before uploading them to your Chef server.

How to do it…

Running chef-shell is straightforward:

Start chef-shell in standalone mode:

mma@laptop:~/chef-repo $ chef-shell
loading configuration: none (standalone chef-shell session)
Session type: standalone
Loading......done.

This is the chef-shell.
 Chef Version: 12.14.89
 http://www.chef.io/
 http://docs.chef.io/

run `help' for help, `exit' or ^D to quit.

Ohai2u mma@laptop!
chef (12.14.89)>

Switch to the attributes mode in chef-shell:
```
chef (12.14.89)> attributes_mode
```
Set an attribute value to be used inside the recipe later...

Using why-run mode to find out what a recipe might do

why-run mode lets each resource tell you what it would do during a Chef client run, assuming certain prerequisites. This is great because it gives you a glimpse of what might really happen on your node when you run your recipe for real.

However, because Chef converges a lot of resources to a desired state, why-run will never be accurate for a complete run. Nevertheless, it might help you during development while you're adding resources step-by-step to build the final recipe.

In this section, we'll try out why-run mode to see what it tells us about our Chef client runs.

Getting ready

To try out why-run mode, you need a node where you can execute the Chef client and at least one cookbook available on that node.

How to do it…

Let's try to run the ntp cookbook in why-run mode:

Override the current run list to run the ntp recipe in why-run mode on a brand new box:

user@server:~$ sudo chef-client -o 'recipe[ntp]' --why-run
...TRUNCATED OUTPUT......

Debugging Chef client runs

Sometimes you get obscure error messages when running the Chef client and you have a hard time finding any clue about where to look for the error. Is your cookbook broken? Do you have a networking issue? Is your Chef server down? Only by looking at the most verbose log output do you have a chance to find out.

Getting ready

You need a Chef client hooked up to the hosted Chef or your own Chef server.

How to do it…

To see how we can ask the Chef client to print debug messages, run the Chef client with debug output:

user@server:~$ sudo chef-client -l debug
...TRUNCATED OUTPUT...
[2016-11-14T07:57:36+00:00] DEBUG: Sleeping for 0 seconds
[2016-11-14T07:57:36+00:00] DEBUG: Running Ohai with the following configuration: {:log_location=>#<IO:<STDOUT>>, :log_level=>:debug, ...TRUNCATED OUTPUT...
 [2016-11-14T07:57:37+00:00] DEBUG: Plugin C: ran 'cc -V -flags' and returned 1
[2016-11-14T07:57:37+00:00] DEBUG: Plugin C 'cc -V -flags' failed. Skipping data.
[2016...

Inspecting the results of your last Chef run

When developing new cookbooks, we need to know what exactly went wrong when a Chef client run fails.

Even though Chef prints all the details to stdout, you might want to look at it again, for example, after clearing your shell window.

Getting ready

You need to have a broken cookbook in your node's run list; any invalid piece of Ruby code will do:

Nil.each {}

How to do it...

Carry out the following steps:

Run the Chef client with your broken cookbook:

user@server:~$ sudo chef-client
================================================================================
Recipe Compile Error in /var/chef/cache/cookbooks/my_cookbook/recipes/default.rb
================================================================================
NoMethodError
-------------
undefined method `each' for nil:NilClass

Cookbook Trace:
---------------
  /var/chef/cache/cookbooks/my_cookbook/recipes/default.rb:7:in `from_file'

Relevant File Content:
----------------------
/var/chef...

Using Reporting to keep track of all your Chef client runs

You need to know what exactly happened on your servers. If you want to record every Chef client run and want to see statistics about successful and failed runs, Chef Reporting is your tool of choice. You can even dive into each individual run across your whole organization if you have Reporting enabled for your Chef clients.

Getting ready

Make sure you have Vagrant installed, as described in the Managing virtual machines with Vagrant recipe in Chapter 1, Chef Infrastructure.

Note

Reporting is a premium feature. If you're running your own Chef server you need a Chef Automate license to use it.

Install the reporting knife plugin by running the following command:

mma@laptop:~/chef-repo $ chef gem install knife-reporting
Successfully installed knife-reporting-0.5.0
1 gem installed

How to do it…

Carry out the following steps to see how Reporting tracks your Chef client runs:

Configure Vagrant to send reporting data to your Chef server by editing...

Chef Cookbook: Achieve powerful IT infrastructure management and automation , Third Edition

What do you get with Print?

Contact Details

Shipping Address

Billing Address

Key benefits

Description

Who is this book for?

What you will learn

Product Details

What do you get with Print?

Contact Details

Shipping Address

Billing Address

Product Details

Packt Subscriptions

Frequently bought together

Table of Contents

Recommendations for you

People who bought this also bought

About the author

FAQs