Managing cookbook dependencies with Berkshelf

It's a pain to manually ensure that you have installed all the cookbooks that another cookbook depends on. You have to download each and every one of them manually only to find out that with each downloaded cookbook, you inherit another set of dependent cookbooks.

And even if you use knife cookbook site install, which installs all the dependencies locally for you, your cookbook directory and your repository get cluttered with all those cookbooks. Usually, you don't really care about all those cookbooks and don't want to see or manage them.

This is where Berkshelf comes into play. It works like Bundler for Ruby gems, managing cookbook dependencies for you. Berkshelf downloads all the dependencies you defined recursively and helps you to upload all cookbooks to your Chef server.

Instead of polluting your Chef repository, it stores all the cookbooks in a central location. You just commit your Berkshelf dependency file (called Berksfile) to your repository, and every colleague or build server can download and install all those dependent cookbooks based on it.

Let's see how to use Berkshelf to manage the dependencies of your cookbook.

Getting ready

Make sure you have a cookbook named my_cookbook and the run_list of your node includes my_cookbook, as described in the Creating and using cookbooks recipe.

How to do it...

Berkshelf helps you to keep those utility cookbooks out of your Chef repository. This makes it much easier to maintain the cookbooks, which really matter.

Let's see how to write a cookbook by running a bunch of utility recipes and manage the required cookbooks with Berkshelf:

1. Edit your cookbook's metadata:

   mma@laptop:~/chef-repo $ subl cookbooks/my_cookbook/metadata.rb
   

   ...
   depends "chef-client"
   depends "apt"
   depends "ntp"

2. Edit your cookbook's default recipe:

   mma@laptop:~/chef-repo $ subl cookbooks/my_cookbook/recipes/default.rb
   

   ...
   include_recipe "chef-client"
   include_recipe "apt"
   include_recipe "ntp"

3. Run Berkshelf to install all the required cookbooks:

   mma@laptop:~/chef-repo $ cd cookbooks/my_cookbook
   mma@laptop:~/chef-repo/cookbooks/my_cookbook $ berks install
   

   Resolving cookbook dependencies...
   Fetching 'my_cookbook' from source at .
   Fetching cookbook index from https://supermarket.chef.io...
   Installing apt (2.6.0) from https://supermarket.chef.io ([opscode] https://supermarket.chef.io/api/v1)
   ...TRUNCATED OUTPUT...

4. Upload all the cookbooks on the Chef server:

   mma@laptop:~/chef-repo/cookbooks/my_cookbook $ berks upload
   

   Using my_cookbook (0.1.0)
   ...TRUNCATED OUTPUT...
   Uploading windows (1.34.8) to: 'https://api.chef.io:443/organizations/awo

How it works...

Berkshelf comes with the Chef DK.

We create our cookbook and tell it to use a few basic cookbooks.

Instead of making us manually install all the cookbooks using knife cookbook site install, chef generate creates a Berksfile, besides the metadata.rb file.

The Berksfile is pretty simple. It tells Berkshelf to use the Chef supermarket as the default source for all cookbooks:

source "https://supermarket.chef.io"

It tells Berkshelf to read the metadata.rb file to find all the required cookbooks. This is the simplest way when working inside a single cookbook. Please see the following There's more… section to find an example of a more advanced usage of the Berksfile.

After telling Berkshelf where to find all the required cookbook names, we use it to install all those cookbooks:

berks install

Berkshelf stores cookbooks in ~/.berkshelf/cookbooks, by default. This keeps your Chef repository clutter-free. Instead of having to manage all the required cookbooks inside your own Chef repository, Berkshelf takes care of them. You simply need to check in Berksfile with your recipe, and everyone using your recipe can download all the required cookbooks by using Berkshelf.

To make sure that there's no mix-up with different cookbook versions when sharing your cookbook, Berkshelf creates a file called Berksfile.lock alongside Berksfile. Here, you'll find the exact versions of all the cookbooks that Berkshelf installed:

DEPENDENCIES
  my_cookbook
    path: .
    metadata: true

GRAPH
  apt (2.6.0)
  chef-client (3.9.0)
    cron (>= 1.2.0)
    logrotate (>= 1.2.0)
    windows (~> 1.11)
  chef_handler (1.1.6)
  cron (1.6.1)
  logrotate (1.7.0)
  my_cookbook (0.1.0)
    apt (>= 0.0.0)
    chef-client (>= 0.0.0)
    ntp (>= 0.0.0)
  ntp (1.6.8)
  windows (1.34.8)
    chef_handler (>= 0.0.0)

Berkshelf will only use the exact versions specified in the Berksfile.lock file, if it finds this file.

Finally, we use Berkshelf to upload all the required cookbooks on the Chef server:

berks upload

There's more...

Berkshelf integrates tightly with Vagrant via the vagrant-berkshelf plugin. You can set up Berkshelf and Vagrant in such a way that Berkshelf installs and uploads all the required cookbooks on your Chef server whenever you execute vagrant up or vagrant provision. You'll save all the work of running berks install and berks upload manually before creating your node with Vagrant.

Let's see how you can integrate Berkshelf and Vagrant.

First, you need to install the Berkshelf plugin for Vagrant:

mma@mma-mbp:~/work/chef-repo (master)$ vagrant plugin install vagrant-berkshelf

Installing the 'vagrant-berkshelf' plugin. This can take a few minutes...
Installed the plugin 'vagrant-berkshelf (4.0.1)'!

Then, you need to tell Vagrant that you want to use the plugin. You do this by enabling the plugin in Vagrantfile:

mma@mma-mbp:~/work/chef-repo (master)$ subl Vagrantfile

config.berkshelf.enabled = true

Then, you need a Berksfile in the root directory of your Chef repository to tell Berkshelf which cookbooks to install on each Vagrant run:

source 'https://supermarket.chef.io'

cookbook 'my_cookbook', path: 'cookbooks/my_cookbook'

Eventually, you can start your VM using Vagrant. Berkshelf will first download and then install all the required cookbooks in the Berkshelf, and upload them on the Chef server. Only after all the cookbooks are made available on the Chef server by Berkshelf, will Vagrant go on:

mma@mma-mbp:~/work/chef-repo $ vagrant up

Bringing machine 'server' up with 'virtualbox' provider...

==> default: Loading Berkshelf datafile...
==> default: Updating Vagrant's Berkshelf...
==> default: Resolving cookbook dependencies...
==> default: Fetching 'my_cookbook' from source at cookbooks/my_cookbook
==> default: Fetching cookbook index from https://supermarket.getchef.com...
...TRUNCATED OUTPUT...

This way, using Berkshelf together with Vagrant, saves a lot of manual steps and gets faster cycle times for your cookbook development.

See also

• For the full documentation of Berkshelf, please visit http://berkshelf.com/
• Please find the Berkshelf source code at https://github.com/RiotGames/berkshelf
• Please find the Vagrant Berkshelf plugin source code at https://github.com/riotgames/vagrant-berkshelf
• The Managing virtual machines with Vagrant recipe in this chapter