You'll need Git installed on your local workstation. Either use your operating system's package manager (such as Apt on Ubuntu or Homebrew on OS X, or simply download the installer from www.git-scm.org.

Git is a distributed version control system. This means that you don't necessarily need a central host to store your repositories. However, in practice, using GitHub as your central repository has proven to be very helpful. In this book, I'll assume that you're using GitHub. Therefore, you need to go to www.github.com and create an account (which is free) to follow the instructions given in this book. Make sure that you upload your Secure Shell (SSH) key by following the instructions at https://help.github.com/articles/generating-ssh-keys, so that you're able to use the SSH protocol to interact with your GitHub account.

As soon as you have created your GitHub account, you should create your repository by visiting https://github.com and using chef-repo as the repository name.

Make sure you have wget installed on your local workstation, to be able to download the required files from public servers.

Before you can write any cookbooks, you need to set up your initial Git repository on your development box. Chef Software, Inc. provides an empty Chef repository to get you started. Let's see how you can set up your own Chef repository with Git, using Chef's skeleton:

You have downloaded a tarball containing Chef's skeleton repository. Then, you initialized chef-repo and connected it to your own repository on GitHub.

After that, you added all the files from the tarball to your repository and committed them. This makes Git track your files and the changes you make later.

Finally, you pushed your repository to GitHub, so that your co-workers can use your code too.

Let's assume you're working on the same chef-repo repository, together with your co-workers. They cloned the repository, added a new cookbook called other_cookbook, committed their changes locally, and pushed to GitHub. Now, it's time for you to get the new cookbook downloaded to your own laptop.

Pull your co-workers' changes from GitHub. This will merge their changes into your local copy of the repository. Use the pull subcommand:

mma@laptop:~/chef-repo $ git pull --rebase
From github.com:mmarschall/chef-repo
 * branch            master     -> FETCH_HEAD
...TRUNCATED OUTPUT...
create mode 100644 cookbooks/other_cookbook/recipes/default.rb

In the event of any conflicting changes, Git will help you merge and resolve them.

Let's see how to install the Chef DK on your local workstation using Chef's omnibus installer:

The omnibus installer will download Ruby and all required Ruby gems into /opt/chefdk.

Before being able to use hosted Chef, you need to sign up for the service. There is a free account for up to five nodes.

Visit http://manage.chef.io/signup and register for a free account.

I registered as the user webops with an organization short name of awo. An organization is the top-level entity for role-based access control in the Chef server.

After registering your account, it is time to prepare your organization to be used with your chef-repo repository.

Carry out the following steps to interact with the hosted Chef:

The following line of code in your knife.rb file tells knife where to find your user's private key. It is used to authenticate you with the Chef server:

client_key               "#{current_dir}/webops.pem"

Also, the following line of code in your knife.rb file tells knife that you are using hosted Chef. You will find your organization name as the last part of the URL:

chef_server_url          "https://api.chef.io/organizations/awo"

Using the knife.rb file and your user's key, you can now connect to your organization hosted by Chef Software, Inc.

This setup is good for you if you do not want to worry about running, scaling, and updating your own Chef server and if you're happy with saving all your configuration data in the Cloud (under the control of Chef Software, Inc.).

Let's create and boot a virtual node by using Vagrant:

The Vagrantfile is written in a Ruby Domain Specific Language (DSL) to configure the Vagrant virtual machines. We want to boot a simple Ubuntu VM. Let's go through the Vagrantfile step by step.

First, we create a config object. Vagrant will use this config object to configure the VM:

Vagrant.configure("2") do |config|
...
end

Inside the config block, we tell Vagrant which VM image to use, in order to boot the node:

config.vm.box = "opscode-ubuntu-16.04"
config.vm.box_url = "http://opscode-vm-bento.s3.amazonaws.com/vagrant/virtualbox/opscode_ubuntu-16.04_chef-provisionerless.box"

We want to boot our VM using a so-called Bento Box, provided by Chef. We use Ubuntu Version 16.04 here.

As we want our VM to have the Chef client installed, we tell the omnibus vagrant plugin to use the latest version of the Chef client:

  config.omnibus.chef_version = :latest

After selecting the VM image to boot, we configure how to provision the box by using Chef. The Chef configuration happens in a nested Ruby block:

  config.vm.provision :chef_client do |chef|
  ...
  end

Inside this chef block, we need to instruct Vagrant on how to hook up our virtual node to the Chef server. First, we need to tell Vagrant where to store all the Chef stuff on your node:

    chef.provisioning_path = "/etc/chef"

Vagrant needs to know the API endpoint of your Chef server. If you use hosted Chef, it is https://api.chef.io/organizations/<YOUR_ORG>. You need to replace <YOUR_ORG> with the name of the organization that you created in your account on hosted Chef. If you are using your own Chef server, change the URL accordingly:

    chef.chef_server_url = "https://api.chef.io/organizations/<YOUR_ORG>"

While creating your user on hosted Chef, you must have downloaded your private key. Tell Vagrant where to find this file:

    chef.validation_key_path = ".chef/<YOUR_USER>.pem"

Also, you need to tell Vagrant which client it should use to validate itself against in the Chef server:

    chef.validation_client_name = "<YOUR_USER>"

Finally, you should tell Vagrant how to name your node:

    chef.node_name = "server"

After configuring your Vagrantfile, all you need to do is run the basic Vagrant commands such as vagrant up, vagrant provision, and vagrant ssh. To stop your VM, just run the vagrant halt command.

If you want to start from scratch again, you will have to destroy your VM and delete both the client and the node from your Chef server by running the following command:

mma@laptop:~/chef-repo $ vagrant destroy
mma@laptop:~/chef-repo $ knife node delete server -y && knife client delete server -y

Alternatively, you may use the Vagrant Butcher plugin found at https://github.com/cassianoleal/vagrant-butcher.

Make sure you have Chef DK installed and a node available for testing. Check out the installation instructions at http://learn.chef.io if you need help here.

Edit your knife.rb file (usually found in the hidden .chef directory) and add the following three lines to it, filling in your own values:

cookbook_copyright "your company"
cookbook_license "apachev2"
cookbook_email "your email address"

Chef will use the preceding values as the defaults whenever you create a new cookbook. We assume that you have a node called server registered with your Chef server, as described in the Managing virtual machines with Vagrant section in this chapter.

Carry out the following steps to create and use cookbooks:

The chef executable helps you to manage your local Chef Development environment. We used it here to generate the cookbook.

Knife is the command-line interface for the Chef server. It uses the RESTful API exposed by the Chef server to do its work and helps you to interact with the Chef server.

The knife command supports a host of commands structured as follows:

knife <subject> <command>

The <subject> used in this section is either cookbook or node. The commands we use are upload for the cookbook, and run_list add for the node.

Before uploading your cookbook to the Chef server, it's a good idea to run it in Test Kitchen first. Test Kitchen will spin up a virtual machine, execute your cookbook, and destroy the virtual machine again. That way you can evaluate what your cookbook does before you upload it to the Chef server and run it on real nodes.

To run your cookbook with Test Kitchen on an Ubuntu 16.04 virtual machine, execute the following steps:

Make sure you have a cookbook named my_cookbook, and the run_list of your node includes my_cookbook, as described in the Creating and using cookbooks recipe in this chapter.

Edit the metadata of your cookbook in the file cookbooks/my_cookbook/metadata.rb to add a dependency to the build-essential cookbook:

mma@laptop:~/chef-repo $ subl cookbooks/my_cookbook/metadata.rb
...
depends 'build-essential', '>= 7.0.3'

If you want to use a feature of another cookbook inside your cookbook, you will need to include the other cookbook in your recipe using the include_recipe directive:

include_recipe 'build-essential'

To tell the Chef server that your cookbook requires the build-essential cookbook, you need to declare that dependency in the metadata.rb file. If you uploaded all the dependencies to your Chef server either using knife cookbook upload my_cookbook --include-dependencies or berks install and berks upload, as described in the Managing cookbook dependencies with Berkshelf recipe in this chapter, the Chef server will then send all the required cookbooks to the node.

The depends function call tells the Chef server that your cookbook depends on a version greater than or equal to 7.0.3 of the build-essential cookbook.

You may use any of these version constraints with depends calls:

If you include another recipe inside your recipe, without declaring the cookbook dependency in your metadata.rb file, Foodcritic will warn you:

mma@laptop:~/chef-repo $ foodcritic cookbooks/my_cookbook
FC007: Ensure recipe dependencies are reflected in cookbook metadata: cookbooks/my_cookbook/recipes/default.rb:9

Additionally, you can declare conflicting cookbooks through the conflicts call:

conflicts "nginx"

Of course, you can use version constraints exactly the same way you did with depends.

Make sure you have a cookbook named my_cookbook and the run_list of your node includes my_cookbook, as described in the Creating and using cookbooks recipe.

Berkshelf helps you to keep those utility cookbooks out of your Chef repository. This makes it much easier to maintain the important cookbooks.

Let's see how to write a cookbook by running a bunch of utility recipes and manage the required cookbooks with Berkshelf:

Berkshelf comes with the Chef DK.

We edit our cookbook and tell it to use a few basic cookbooks.

Instead of making us manually install all the cookbooks using knife cookbook site install, Chef generates a Berksfile, besides the metadata.rb file.

The Berksfile is simple. It tells Berkshelf to use the Chef supermarket as the default source for all cookbooks:

source "https://supermarket.chef.io"

And the Berksfile tells Berkshelf to read the metadata.rb file to find all the required cookbooks. This is the simplest way when working inside a single cookbook. Please see the following There's more… section to find an example of a more advanced usage of the Berksfile.

After telling Berkshelf where to find all the required cookbook names, we use it to install all those cookbooks:

berks install

Berkshelf stores cookbooks in ~/.berkshelf/cookbooks, by default. This keeps your Chef repository clutter-free. Instead of having to manage all the required cookbooks inside your own Chef repository, Berkshelf takes care of them. You simply need to check in Berksfile with your cookbook, and everyone using your cookbook can download all the required cookbooks by using Berkshelf.

To make sure that there's no mix-up with different cookbook versions when sharing your cookbook, Berkshelf creates a file called Berksfile.lock alongside Berksfile.

Here, you'll find the exact versions of all the cookbooks that Berkshelf installed:

DEPENDENCIES
  my_cookbook
    path: .
    metadata: true

GRAPH
  apt (4.0.2)
    compat_resource (>= 12.10)
  chef-client (6.0.0)
    cron (>= 1.7.0)
    logrotate (>= 1.9.0)
    windows (>= 1.42.0)
  compat_resource (12.14.7)
  cron (2.0.0)
  logrotate (2.1.0)
    compat_resource (>= 0.0.0)
  my_cookbook (0.1.1)
    apt (>= 0.0.0)
    chef-client (>= 0.0.0)
    ntp (>= 0.0.0)
  ntp (3.2.0)
  windows (2.0.2)

Berkshelf will only use the exact versions specified in the Berksfile.lock file, if it finds this file.

Finally, we use Berkshelf to upload all the required cookbooks to the Chef server:

berks upload

Berkshelf integrates tightly with Vagrant via the vagrant-berkshelf plugin. You can set up Berkshelf and Vagrant in such a way that Berkshelf installs and uploads all the required cookbooks on your Chef server whenever you execute vagrant up or vagrant provision. You'll save all the work of running berks install and berks upload manually before creating your node with Vagrant.

Let's see how you can integrate Berkshelf and Vagrant: