Setting up a work environment
As we saw earlier, the Chef ecosystem comprises of three components: chef-server, chef-client, and a developer workstation.
We'll be developing all our beautiful Chef codes on our workstation. As we are developing a code, it's good practice to keep our code in some version control system such as git/svn/mercurial and so on. We'll choose Git for our purpose and I'll presume you've a repository called chef-repo that is being tracked by Git.
The following software should be installed on your machine before you try to set up your workstation:
- Ruby (Preferably, 1.9.x).
- We need Chef and Knife installed on our workstation and it's pretty easy to go about installing Chef along with Knife using the Ruby gems. Just open up a terminal and issue the command:
#gem install chef - Once Chef is installed, create a
.cheffolder in your home directory and create aknife.rbfile in it.
Knife is a tool using which we'll use to communicate with a chef-server. Knife can be used for lots of purposes such as managing cookbooks, nodes, API clients, roles, environments, and so on. Knife also comes with plugins that allow it to be used for various other useful purposes. We'll learn more about them in later chapters.
Knife needs the knife.rb file present in the $HOME/.chef folder. The following is a sample knife.rb file:
log_level :info log_location STDOUT node_name 'NAME_OF_YOUR_CHOICE' client_key '~/.chef/NAME_OF_YOUR_CHOICE.pem' validation_client_name 'chef-validator' validation_key '~/.chef/validation.pem' chef_server_url 'http://chef-server.sychonet.com:4000' cache_type 'BasicFile' cache_options (:path => '~/.chef/checksums') cookbook_path [ '~/code/chef-repo/cookbooks' ]
Connect to your chef-server web interface and visit the client section and create a new client with a name of your choice (ensure that no client with the same name exists on the chef-server):
Once you've created the client, a chef-server will respond with a public/private key pair as shown in the following screenshot:
Copy the contents of the private key and store them in ~/.chef/<NAME_OF_YOUR_CHOICE>.pem
Also, copy the private key for the chef-validator (/etc/chef/validation.pem) from the chef-server to ~/.chef/validation.pem.
Specify NAME_OF_YOUR_CHOICE as the node name.
As you can see, we've specified cookbook_path to be ~/code/chef-repo/cookbooks. I'm presuming that you'll be storing your Chef cookbooks inside this folder.
Create the following directory structure inside ~/code/chef-repo:
chef-repo
├── cookbooks
├── data_bags
├── environments
└── rolesThe cookbooks directory will hold our cookbooks, the data_bags directory will contain data bags, the environments directory will contain configuration files for different environments, and the roles directory will contain files associated with different roles.
Once you've created these directories, commit them to your Git repository.
Now, let's try to see if we are able to make use of the Knife executable and query the Chef server:
$knife client list chef-validator chef-webui chef-eg01
This command will list all the available API clients registered with the chef-server. As you can see, chef-eg01 is a newly created client and it's now registered with the chef-server.
Knife caches the checksum of Ruby and ERB files when performing a cookbook syntax check with knife cookbook test or knife cookbook upload. The cache_type variable defines which type of cache to make use of. The most used type is BasicFile and it's probably best to leave it at that.
The cache_options is a hash for options related to caching. For BasicFile, :path should be the location on the filesystem where Knife has write access.
If you want the Knife cookbook to create a command to prefill values for copyright and e-mail in comments, you can also specify the following options in your knife.rb file:
cookbook_copyright "Company name" cookbook_email "Email address"
With this setup, now we are ready to start creating new cookbooks, roles, and environments, and manage them along with nodes and clients using Knife from our workstation.
Before we jump into cookbook creation and other exciting stuff, we need to ensure that we follow a test-driven approach to our Chef development. We will make use of test-kitchen to help us write Chef cookbooks that are tested thoroughly before being pushed to a chef-server.
test-kitchen can be installed as a gem:
$ gem install test-kitchen
Also, download Vagrant from http://www.vagrantup.com and install it.
If you want some help, use the help option of the kitchen command:
$ kitchen help Commands: kitchen console # Kitchen Console! kitchen converge [INSTANCE|REGEXP|all] # Converge one or more instances kitchen create [INSTANCE|REGEXP|all] # Create one or more instances kitchen destroy [INSTANCE|REGEXP|all] # Destroy one or more instances kitchen diagnose [INSTANCE|REGEXP|all] # Show computed diagnostic configuration kitchen driver # Driver subcommands kitchen driver create [NAME] # Create a new Kitchen Driver gem project kitchen driver discover # Discover Test Kitchen drivers published on RubyGems kitchen driver help [COMMAND] # Describe subcommands or one specific subcommand kitchen help [COMMAND] # Describe available commands or one specific command kitchen init # Adds some configuration to your cookbook so Kitchen can rock kitchen list [INSTANCE|REGEXP|all] # Lists one or more instances kitchen login INSTANCE|REGEXP # Log in to one instance kitchen setup [INSTANCE|REGEXP|all] # Setup one or more instances kitchen test [INSTANCE|REGEXP|all] # Test one or more instances kitchen verify [INSTANCE|REGEXP|all] # Verify one or more instances kitchen version # Print Kitchen's version information
Now, let's create a new cookbook called passenger-nginx:
$knife cookbook create passenger-nginx
Now, we'll add test-kitchen to our project using the init subcommand:
$ kitchen init create .kitchen.yml create test/integration/default run gem install kitchen-vagrant from "." Fetching: kitchen-vagrant-0.14.0.gem (100%) Successfully installed kitchen-vagrant-0.14.0 Parsing documentation for kitchen-vagrant-0.14.0 Installing ri documentation for kitchen-vagrant-0.14.0 Done installing documentation for kitchen-vagrant after 0 seconds 1 gem installed
The kitchen init command has created a configuration file called .kitchen.yml, along with a test/integration/default directory.
It also went on to install a gem called kitchen-vagrant. kitchen needs a virtual machine to test run the chef code, and drivers are responsible for managing virtual machines. By default, kitchen makes use of Vagrant to manage the virtual machine.
Let's see what we have in our configuration file, kitchen.yml:
$ cat .kitchen.yml --- driver: name: vagrant provisioner: name: chef_solo platforms: - name: ubuntu-12.04 - name: centos-6.4 suites: - name: default run_list: - recipe[cb-test1::default] attributes:
The file is divided into four sections:
- Driver: This is where we set up basic stuff such as the SSH username and credentials. Under this section, we've a
nameproperty with avagrantvalue. This tells kitchen to make use of the kitchen-vagrant driver. - Provisioner: This tells kitchen to make use of a chef-solo to apply the cookbook to a newly created virtual machine.
- Platforms: This lists the operating systems on which we want to run our code.
- Suites: Here we describe what we wish to test.
Now, let's see what we have on our hands:
$ kitchen list Instance Driver Provisioner Last Action default-ubuntu-1204 Vagrant ChefSolo <Not Created> default-centos-64 Vagrant ChefSolo <Not Created>
As you can see, it's listing two instances: default-ubuntu-1204 and default-centos-64. These names are a combination of the suite name and the platform name.
Now, let's spin up one instance to see what happens:
$ kitchen create default-ubuntu-1204 -----> Starting Kitchen (v1.2.1) -----> Creating <default-ubuntu-1204>... Bringing machine 'default' up with 'virtualbox' provider... ==> default: Box 'opscode-ubuntu-12.04' could not be found. Attempting to find and install... default: Box Provider: virtualbox default: Box Version: >= 0 ==> default: Adding box 'opscode-ubuntu-12.04' (v0) for provider: virtualbox default: Downloading: https://opscode-vm-bento.s3.amazonaws.com/vagrant/virtualbox/opscode_ubuntu-12.04_chef-provisionerless.box ==> default: Successfully added box 'opscode-ubuntu-12.04' (v0) for 'virtualbox'! ==> default: Importing base box 'opscode-ubuntu-12.04'... ==> default: Matching MAC address for NAT networking... ==> default: Setting the name of the VM: default-ubuntu-1204_default_1398006642518_53572 ==> default: Clearing any previously set network interfaces... ==> default: Preparing network interfaces based on configuration... default: Adapter 1: nat ==> default: Forwarding ports... default: 22 => 2222 (adapter 1) ==> default: Running 'pre-boot' VM customizations... ==> default: Booting VM... ==> default: Waiting for machine to boot. This may take a few minutes... default: SSH address: 127.0.0.1:2222 default: SSH username: vagrant default: SSH auth method: private key default: Warning: Connection timeout. Retrying... ==> default: Machine booted and ready! ==> default: Checking for guest additions in VM... ==> default: Setting hostname... Vagrant instance <default-ubuntu-1204> created. Finished creating <default-ubuntu-1204> (4m4.17s). -----> Kitchen is finished. (4m4.71s)
So, this leads to the downloading of a virtual machine image for Ubuntu 12.04 and, eventually, the machine boots up. The default username for SSH connection is vagrant.
Let us check the status of our instance again:
$ kitchen list Instance Driver Provisioner Last Action default-ubuntu-1204 Vagrant ChefSolo Created default-centos-64 Vagrant ChefSolo <Not Created>
So, our Ubuntu instance is up and running. Now, let's add some meat to our recipe:
# # Cookbook Name:: cb-test1 # Recipe:: default # # Copyright 2014, Sychonet # # All rights reserved - Do Not Redistribute # package "nginx" log "Cool. So we have nginx installed"
So, now we've got our recipe ready, let's let test-kitchen run it in our instance now:
$ kitchen converge default-ubuntu-1204 -----> Starting Kitchen (v1.2.1) -----> Converging <default-ubuntu-1204>... Preparing files for transfer Preparing current project directory as a cookbook Removing non-cookbook files before transfer -----> Installing Chef Omnibus (true) downloading https://www.getchef.com/chef/install.sh to file /tmp/install.sh trying wget... Downloading Chef for ubuntu... downloading https://www.getchef.com/chef/metadata?v=&prerelease=false&nightlies=false&p=ubuntu&pv=12.04&m=x86_64 to file /tmp/install.sh.1144/metadata.txt trying wget... url https://opscode-omnibus-packages.s3.amazonaws.com/ubuntu/12.04/x86_64/chef_11.12.2-1_amd64.deb md5 cedd8a2df60a706e51f58adf8441971b sha256 af53e7ef602be6228dcbf68298e2613d3f37eb061975992abc6cd2d318e4a0c0 downloaded metadata file looks valid... downloading https://opscode-omnibus-packages.s3.amazonaws.com/ubuntu/12.04/x86_64/chef_11.12.2-1_amd64.deb to file /tmp/install.sh.1144/chef_11.12.2-1_amd64.deb trying wget... Comparing checksum with sha256sum... Installing Chef installing with dpkg... Selecting previously unselected package chef. (Reading database ... 56035 files and directories currently installed.) Unpacking chef (from .../chef_11.12.2-1_amd64.deb) ... Setting up chef (11.12.2-1) ... Thank you for installing Chef! Transfering files to <default-ubuntu-1204> [2014-04-20T15:50:31+00:00] INFO: Forking chef instance to converge... [2014-04-20T15:50:31+00:00] WARN: * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * SSL validation of HTTPS requests is disabled. HTTPS connections are still encrypted, but chef is not able to detect forged replies or man in the middle attacks. To fix this issue add an entry like this to your configuration file: ``` # Verify all HTTPS connections (recommended) ssl_verify_mode :verify_peer # OR, Verify only connections to chef-server verify_api_cert true ``` To check your SSL configuration, or troubleshoot errors, you can use the `knife ssl check` command like so: ``` knife ssl check -c /tmp/kitchen/solo.rb ``` * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * Starting Chef Client, version 11.12.2 [2014-04-20T15:50:31+00:00] INFO: *** Chef 11.12.2 *** [2014-04-20T15:50:31+00:00] INFO: Chef-client pid: 1225 [2014-04-20T15:50:39+00:00] INFO: Setting the run_list to ["recipe[cb-test1::default]"] from CLI options [2014-04-20T15:50:39+00:00] INFO: Run List is [recipe[cb-test1::default]] [2014-04-20T15:50:39+00:00] INFO: Run List expands to [cb-test1::default] [2014-04-20T15:50:39+00:00] INFO: Starting Chef Run for default-ubuntu-1204 [2014-04-20T15:50:39+00:00] INFO: Running start handlers [2014-04-20T15:50:39+00:00] INFO: Start handlers complete. Compiling Cookbooks... Converging 2 resources Recipe: cb-test1::default * package[nginx] action install[2014-04-20T15:50:39+00:00] INFO: Processing package[nginx] action install (cb-test1::default line 10) - install version 1.1.19-1ubuntu0.6 of package nginx * log[Cool. So we have nginx installed] action write[2014-04-20T15:50:52+00:00] INFO: Processing log[Cool. So we have nginx installed] action write (cb-test1::default line 12) [2014-04-20T15:50:52+00:00] INFO: Cool. So we have nginx installed [2014-04-20T15:50:52+00:00] INFO: Chef Run complete in 12.923797655 seconds Running handlers: [2014-04-20T15:50:52+00:00] INFO: Running report handlers Running handlers complete [2014-04-20T15:50:52+00:00] INFO: Report handlers complete Chef Client finished, 2/2 resources updated in 21.14983058 seconds Finished converging <default-ubuntu-1204> (2m10.10s). -----> Kitchen is finished. (2m10.41s)
So, here is what happened under the hood when kitchen converge was executed:
- Chef was installed on an Ubuntu instance
- Our
cb-test1cookbook and a chef-solo configuration were uploaded to an Ubuntu instance. - The Chef run was initiated using
run_listand attributes defined in.kitchen.yml
If the exit code of the kitchen command is 0, then the command run was successful. If it's not 0, then any part of the operation associated with the command was not successful.
Let's check the status of our instance once more:
$ kitchen list Instance Driver Provisioner Last Action default-ubuntu-1204 Vagrant ChefSolo Converged default-centos-64 Vagrant ChefSolo <Not Created>
So, our instance is converged, but we still don't know if nginx was installed successfully or not. One way to check this is to log in to the instance using the following command:
$ kitchen login default-ubuntu-1204
Once you've logged in to the system, you can now go ahead and check for the presence of the binary named nginx:
vagrant@default-ubuntu-1204:~$ which nginx /usr/sbin/nginx
So, Nginx is indeed installed.
However, with kitchen, we no longer need to take the pain of logging in to the system and verifying the installation. We can do this by writing a test case.
We'll make use of bash automated testing system (bats), called for this purpose.
Create a directory using the following command:
$ mkdir -p test/integration/default/bats
Create a new file package test.bats under the bats directory:
#!/usr/bin/env bats
@test "nginx binary is found in PATH"
{
run which nginx
[ "$status" -eq 0 ]
}Now, let's run our test using kitchen verify:
$ kitchen verify default-ubuntu-1204 -----> Starting Kitchen (v1.2.1) -----> Setting up <default-ubuntu-1204>... Fetching: thor-0.19.0.gem (100%) Fetching: busser-0.6.2.gem (100%) Successfully installed thor-0.19.0 Successfully installed busser-0.6.2 2 gems installed -----> Setting up Busser Creating BUSSER_ROOT in /tmp/busser Creating busser binstub Plugin bats installed (version 0.2.0) -----> Running postinstall for bats plugin Installed Bats to /tmp/busser/vendor/bats/bin/bats Finished setting up <default-ubuntu-1204> (1m41.31s). -----> Verifying <default-ubuntu-1204>... Suite path directory /tmp/busser/suites does not exist, skipping. Uploading /tmp/busser/suites/bats/package-test.bats (mode=0644) -----> Running bats test suite ✓ nginx binary is found in PATH 1 test, 0 failures Finished verifying <default-ubuntu-1204> (0m1.03s). -----> Kitchen is finished. (0m1.51s)
So, we see that our test has successfully passed verification, and we can proudly go ahead and upload our cookbook to the chef-server and trigger a chef-client run on the concerned instance.
