Puppet 3 Cookbook

Chapter 1. Puppet Infrastructure

	Computers in the future may have as few as 1,000 vacuum tubes and weigh only 1.5 tons.
	--Popular Mechanics, 1949

In this chapter, we will cover:

Installing Puppet
Creating a manifest
Managing your manifests with Git
Creating a decentralized Puppet architecture
Writing a papply script
Running Puppet from cron
Deploying changes with Rake
Bootstrapping Puppet with Rake
Automatic syntax checking with Git hooks

Installing Puppet

If you already have a working Puppet installation, you can skip this section. If not, or if you want to upgrade or re-install Puppet, we'll go through the installation process step by step.

I'm using an Amazon EC2 cloud instance to demonstrate setting up Puppet, though you may prefer to use a physical server, a Linux workstation, or a virtual machine such as Vagrant, VMWare, or VirtualBox (with Internet access). I'll log in as the ubuntu user and use sudo to run commands that need root privileges (the default setup on Ubuntu).

Note

On EC2 Ubuntu images, the ubuntu user is already set up with the sudo permissions to run any commands as root. If you're using a different Linux distribution or you're not on EC2, you'll need to configure this yourself in the /etc/sudoers file.

Getting ready...

To prepare the machine for Puppet, we need to set its hostname.

Set a suitable hostname for your server (ignore any warning from sudo):

ubuntu@domU-12-31-39-09-51-23:~$ sudo hostname cookbook
ubuntu@domU-12-31-39-09-51-23:~$ sudo su -c 'echo cookbook
  >/etc/hostname'
sudo: unable to resolve host cookbook

Log out and log back in to check the hostname is now correctly set:
```
ubuntu@cookbook:~$
```

Find out the local IP address of the server:

ubuntu@cookbook:~$ ip addr show |grep eth0
    inet 10.96.247.132/23 brd 10.96.247.255 scope global eth0

Copy the IP address of your server (here it's 10.96.247.132) and add this to the /etc/hosts file so that it looks something like this (use your own hostname and domain):
```
10.96.247.132 cookbook cookbook.example.com
```

How to do it...

Puppet packages for most Linux distributions, including Ubuntu, are available from Puppet Labs. Here's how to install the version for Ubuntu 12.04 Precise:

Download the Puppet Labs repo package:

ubuntu@cookbook:~$ wget http://apt.puppetlabs.com/puppetlabs-
  release-precise.deb

Install the repo package:

ubuntu@cookbook:~$ sudo dpkg -i puppetlabs-release-precise.deb
Selecting previously unselected package puppetlabs-release.
(Reading database ... 33975 files and directories currently 
  installed.)
Unpacking puppetlabs-release (from puppetlabs-release-
    precise.deb)
Setting up puppetlabs-release (1.0-5)

Update your APT configuration:
```
ubuntu@cookbook:~$ sudo apt-get update
```
Note
If you're not using Ubuntu 12.04 Precise, you can find out how to add the Puppet Labs repos package to your system here:
http://docs.puppetlabs.com/guides/puppetlabs_package_repositories.html
Install Puppet:
```
ubuntu@cookbook:~$ sudo apt-get -y install puppet
```
Note
If you're on Mac, you can download and install suitable DMG images from Puppet Labs available at:
https://downloads.puppetlabs.com/mac/
If you're using Windows, you can download MSI packages from the Puppet Labs website available at:
https://downloads.puppetlabs.com/windows/
Run the following command to check that Puppet is properly installed:
```
ubuntu@cookbook:~$ puppet --version
3.2.2
```

If the version of Puppet you've installed is not exactly the same, it doesn't matter; you'll get whatever is the latest version made available by Puppet Labs. So long as your version is at least 3.0, you should have no trouble running the examples in this book.

If you have an older version of Puppet, you may find that some things don't work or work differently to the way you'd expect. I recommend that you upgrade to Puppet 3.x or later if at all possible.

Now that Puppet is set up, you can use it to make some configuration changes by creating a manifest. We'll see how to do this in the next section.

Managing your manifests with Git

It's a great idea to put your Puppet manifests in a version control system such as Git or Subversion (I recommend Git) and give all Puppet-managed machines a checkout from your repository. This gives you several advantages:

You can undo changes and revert to any previous version of your manifest
You can experiment with new features using a branch
If several people need to make changes to the manifests, they can make them independently, in their own working copies, and then merge their changes later
You can use the git log feature to see what was changed, and when (and by whom)

Getting ready...

In this section we'll import your existing manifest files into Git. If you have created a puppet directory in the previous section, use that, otherwise use your existing manifest directory.

I'm going to use the popular GitHub service as my Git server. You don't have to do this, it's easy to run your own Git server but it does simplify things. If you already use Git and have a suitable server, feel free to use that instead.

Note

Note that GitHub currently only offers free repository hosting for public repositories (that is, everyone will be able to see and read your Puppet manifests). This isn't a good idea if your manifest contains secret data such as passwords. It's fine for playing and experimenting with the recipes in this book, but for production use, consider a private GitHub repo instead.

Here's what you need to do to prepare for importing your manifest:

First, you'll need Git installed on your machine:

ubuntu@cookbook:~/puppet$ sudo apt-get install git

Next, you'll need a GitHub account (free for open-source projects, or you'll need to pay a small fee to create private repositories) and a repository. Follow the instructions at github.com to create and initialize your repository (from now on, just "repo" for short). Make sure you tick the box that says, Initialize this repository with a README.
Authorize your SSH key for read/write access to the repo (see the GitHub site for instructions on how to do this).

How to do it...

You're now ready to add your existing manifests to the Git repo. We're going to clone the repo, and then move your manifest files into it, as follows:

First, move your puppet directory to a different name:
```
mv puppet puppet.import
```

Clone the repo onto your machine into a directory named puppet (use your own repo URL, as shown on GitHub):

ubuntu@cookbook:~$ git clone 
  git@github.com:bitfield/cookbook.git puppet
Cloning into 'puppet'...
remote: Counting objects: 3, done.
remote: Total 3 (delta 0), reused 0 (delta 0)
Receiving objects: 100% (3/3), done.

Move everything from puppet.import to puppet:

ubuntu@cookbook:~$ mv puppet.import/* puppet/

Add and commit the new files to the repo, setting your Git identity details if necessary:

ubuntu@cookbook:~$ cd puppet
ubuntu@cookbook:~/puppet$ git status
# On branch master
# Untracked files:
#   (use "git add <file>..." to include in what will be
  committed)
#
#       manifests/
nothing added to commit but untracked files present (use "git
  add" to track)
ubuntu@cookbook:~/puppet$ git add manifests/
ubuntu@cookbook:~/puppet$ git config --global user.name "John
  Arundel"
ubuntu@cookbook:~/puppet$ git config --global user.email 
  "john@bitfieldconsulting.com"
ubuntu@cookbook:~/puppet$ git commit -m "Importing"
[master a063a5b] Importing
Committer: John Arundel <john@bitfieldconsulting.com>
2 files changed, 6 insertions(+)
create mode 100644 manifests/nodes.pp
create mode 100644 manifests/site.pp

Finally, push your changes back to GitHub:

ubuntu@cookbook:~/puppet$ git push -u origin master
Counting objects: 6, done.
Compressing objects: 100% (4/4), done.
Writing objects: 100% (5/5), 457 bytes, done.
Total 5 (delta 0), reused 0 (delta 0)
To git@github.com:bitfield/cookbook.git
   6d6aa51..a063a5b  master -> master

How it works...

Git tracks changes to files, and stores a complete history of all changes. The history of the repo is made up of commits. A commit represents the state of the repo at a particular point in time, which you create with the git commit command and annotate with a message.

You've added your Puppet manifest files to the repo and created your first commit. This updates the history of the repo, but only in your local working copy. To synchronize the changes with GitHub's copy, the git push command pushes all changes made since the last sync.

There's more...

Now that you have a central Git repo for your Puppet manifests, you can check out multiple copies of it in different places and work on them, before committing your changes. For example, if you're working in a team, each member can have her own local copy of the repo and synchronize changes with the others via GitHub.

Now that you've taken control of your manifests with Git, you can use it as a simple, scalable way to distribute manifest files to lots of machines. We'll see how to do this in the next section.

Creating a decentralized Puppet architecture

Some systems work best when they're decentralized. The Mafia is a good example, although, of course, there is no Mafia.

A common way to use Puppet is to run a Puppet Master server, which Puppet clients can then connect to and receive their manifests. However, you don't need a Puppet Master to use Puppet. You can run the puppet apply command directly on a manifest file to have Puppet apply it:

ubuntu@cookbook:~/puppet$ puppet apply manifests/site.pp
Notice: Finished catalog run in 0.08 seconds

In other words, if you can arrange to distribute a suitable manifest file to a client machine, you can have Puppet execute it directly without the need for a central Puppet Master. This removes the performance bottleneck of a single master server, and also eliminates a single point of failure. It also avoids having to sign and exchange the SSL certificates when provisioning a new client machine.

There are many ways you could deliver the manifest file to the client, but Git (or any version control system) does most of the work for you. You can edit your manifests in a local working copy, commit them to Git, and push them to a central repo, and from there they can be automatically distributed to the client machines.

Getting ready

If your Puppet manifests aren't already in Git, follow the steps in Managing your manifests with Git.

You'll need a second machine to check out a copy of your Puppet repo. If you're using EC2 instances, create another instance, and call it something like cookbook2.

How to do it...

Follow these steps:

Check out your GitHub repo on the new machine:

ubuntu@cookbook2:~$ git clone 
  git@github.com:bitfield/cookbook.git puppet
Cloning into 'puppet'...
remote: Counting objects: 8, done.
remote: Compressing objects: 100% (5/5), done.
remote: Total 8 (delta 0), reused 5 (delta 0)
Receiving objects: 100% (8/8), done.

Modify your manifests/nodes.pp file, as follows:

node 'cookbook', 'cookbook2' {
  file { '/tmp/hello':
    content => "Hello, world\n",
  }
}

Run the following command:

ubuntu@cookbook2:~/puppet$ sudo puppet apply manifests/site.pp
Notice: /Stage[main]//Node[cookbook2]/File[/tmp/hello]/ensure:
defined content as '{md5}a7966bf58e23583c9a5a4059383ff850'
Notice: Finished catalog run in 0.05 seconds

How it works...

We've created a new working copy of the Puppet repo on the new machine:

ubuntu@cookbook2:~$ git clone git@github.com:bitfield/cookbook.git puppet

However, before we can run Puppet, we have to create a node declaration for the cookbook2 node:

node 'cookbook', 'cookbook2' {
  ...
}

Now, we apply the manifest:

ubuntu@cookbook2:~/puppet$ sudo puppet apply manifests/site.pp

Puppet finds the node declaration for cookbook2 and applies the same manifest that we used before on cookbook:

Notice: /Stage[main]//Node[cookbook2]/File[/tmp/hello]/ensure:
  defined content as '{md5}a7966bf58e23583c9a5a4059383ff850'

There's more...

Having scaled your Puppet infrastructure from one machine to the other, you can now extend it to as many as you like! All you need to do is check out the Git repo on a machine, and run puppet apply.

This is a great way to add Puppet management to your existing machines without lots of complicated setup, or using an extra machine to serve as a Puppet Master. Many of my clients have switched to using a Git-based infrastructure because it's simpler, easier to scale, and easier to maintain.

A refinement which you might like to consider is having each machine automatically pull changes from GitHub and apply them with Puppet. Then all you need to do is push a change to GitHub, and it will roll out to all your Puppet-managed machines within a certain time. We'll see how to do this in the following sections.

Writing a papply script

We'd like to make it as quick and easy as possible to apply Puppet on a machine, so I usually write a little script that wraps the puppet apply command with the parameters it needs. And to deploy the script where it's needed, what better tool than Puppet itself?

How to do it...

Follow these steps:

In your Puppet repo, create the directories needed for a puppet module:

ubuntu@cookbook:~/puppet$ mkdir modules
ubuntu@cookbook:~/puppet$ mkdir modules/puppet
ubuntu@cookbook:~/puppet$ mkdir modules/puppet/manifests
ubuntu@cookbook:~/puppet$ mkdir modules/puppet/files

Create the file modules/puppet/files/papply.sh with the following contents (change the path /home/ubuntu/puppet to where your Puppet repo is located). The sudo puppet apply command should all be on one line:
```
#!/bin/sh
sudo puppet apply /home/ubuntu/puppet/manifests/site.pp
  --modulepath=/home/ubuntu/puppet/modules/ $*
```

Create the file modules/puppet/manifests/init.pp with the following contents:

class puppet {
  file { '/usr/local/bin/papply':
    source => 'puppet:///modules/puppet/papply.sh',
    mode   => '0755',
  }
}

Modify your manifests/nodes.pp file as follows:
```
node 'cookbook' {
  include puppet
}
```

Apply your changes:

ubuntu@cookbook:~/puppet$ sudo puppet apply manifests/site.pp
  --modulepath=/home/ubuntu/puppet/modules
Notice: /Stage[main]/Puppet/File[/usr/local/bin/papply]
  /ensure: defined content as '{md5}
    171896840d39664c00909eb8cf47a53c'
Notice: Finished catalog run in 0.07 seconds

Test that the script works:

ubuntu@cookbook:~/puppet$ papply
Notice: Finished catalog run in 0.07 seconds

Now whenever you need to run Puppet, you can simply run papply. In future, when we apply Puppet changes, I'll ask you to run papply instead of the full puppet apply command.

How it works...

As you've seen, to run Puppet on a machine and apply a specified manifest file, we use the puppet apply command:

puppet apply manifests/site.pp

When you're using modules (such as the puppet module we just created) you also need to tell Puppet where to search for modules, using the modulepath argument:

puppet apply manifests/nodes.pp --
  modulepath=/home/ubuntu/puppet/modules

In order to run Puppet with the root privileges it needs, we have to put sudo before everything:

sudo puppet apply manifests/nodes.pp --
  modulepath=/home/ubuntu/puppet/modules

Finally, any additional arguments passed to papply will be passed through to Puppet itself, by adding the $* parameter:

sudo puppet apply manifests/nodes.pp --
  modulepath=/home/ubuntu/puppet/modules $*

That's a lot of typing, so putting this in a script makes sense. We've added a Puppet file resource that will deploy the script to /usr/local/bin and make it executable:

file { '/usr/local/bin/papply':
  source => 'puppet:///modules/puppet/papply.sh',
  mode   => '0755',
}

Finally, we include the puppet module in our node declaration for cookbook:

node 'cookbook' {
  include puppet
}

You can do the same for any other nodes managed by Puppet.

Running Puppet from cron

You can do a lot with the setup you already have: work on your Puppet manifests as a team, communicate changes via GitHub, and manually apply them on a machine using the papply script.

However, you still have to log into each machine to update the Git repo and re-run Puppet. It would be helpful to have each machine update itself and apply any changes automatically. Then all you need to do is to push a change to the repo, and it will go out to all your machines within a certain time.

The simplest way to do this is with a cron job that pulls updates from the repo at regular intervals and then runs Puppet if anything has changed.

Getting ready...

You'll need the Git repo we set up in Managing your manifests with Git and Creating a decentralized Puppet architecture, and the papply script from Writing a papply script.

You'll also need to create an SSH key that each machine can use to pull changes from the Git repo. To create this, follow these steps:

Run the following command to generate the keyfile:

ubuntu@cookbook:~/puppet$ ssh-keygen -f ubuntu
Generating public/private rsa key pair.
Enter passphrase (empty for no passphrase):
Enter same passphrase again:
Your identification has been saved in ubuntu.
Your public key has been saved in ubuntu.pub.
The key fingerprint is:
ae:80:48:1c:14:51:d6:b1:73:4f:60:e2:cf:3d:ce:f1 ubuntu@cookbook
The key's randomart image is:
+--[ RSA 2048]----+
| ++o.o.o         |
|      +          |
|     +           |
|      = +        |
| o     oS=       |
|        o +      |
|         o E     |
|                 |
|                 |
+-----------------+

Print the contents of the ubuntu.pub file:

ubuntu@cookbook:~/puppet$ cat ubuntu.pub
ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC8EsdLAZHIg1nnMzJuIQ5jEcFL1WI5AVhml6Z3Gw4zc4xw6F1Citomc+3DexcaD+y3VrD3WEOGcXweCsxJF0EGyJoc4RbPAJaP3D4V/+9FQVZcH90GukasvtIrfJYy2KFfRBROKtrfckMbBlWF7U2U+FwaalMOtgLzZeECSDU4eYuheN3UVcyg9Zx87zrLYU5EK1JH2WVoZd3UmdH73/rwPJWtSEQ3xs9A2wMr0lJsCF4CcFCVwrAIoEf5WzIoHbhWyZaVyPR4gHUHd3wNIzC0rmoRiYwE5uYvVBObLN10uZhn7zGPWHEc5tYU7DMbz61iTe4NLtauwJkZxmXUiPJh ubuntu@cookbook

Copy this and add it to your GitHub repo as a deploy key (refer to the GitHub site for instructions on how to do this). This will authorize the key to clone the Puppet repo from GitHub.

How to do it...

Follow these steps:

Move the public key file into your puppet module:

ubuntu@cookbook:~/puppet$ mv ubuntu.pub
  modules/puppet/files/ubuntu.pub

Keep the private key file somewhere separate from your Puppet repo (you'll distribute this via some other channel to machines which need to check out the repo).
Create the file modules/puppet/files/pull-updates.sh with the following contents:
```
#!/bin/sh
cd /home/ubuntu/puppet
git pull && /usr/local/bin/papply
```

Modify the file modules/puppet/manifests/init.pp to look like this:

class puppet {  
  file { '/usr/local/bin/papply':
    source => 'puppet:///modules/puppet/papply.sh',
    mode   => '0755',
  }

  file { '/usr/local/bin/pull-updates':
    source => 'puppet:///modules/puppet/pull-updates.sh',
    mode   => '0755',
  }
  
  file { '/home/ubuntu/.ssh/id_rsa':
    source => 'puppet:///modules/puppet/ubuntu.priv',
    owner  => 'ubuntu',
    mode   => '0600',
  }

  cron { 'run-puppet':
    ensure  => 'present',
    user    => 'ubuntu',
    command => '/usr/local/bin/pull-updates',
    minute  => '*/10',
    hour    => '*',
  }
}

Run Puppet:

ubuntu@cookbook:~/puppet$ papply
Notice: /Stage[main]/Puppet/Cron[run-puppet]/ensure: created
Notice: /Stage[main]/Puppet/File[/usr/local/bin/pull-
  updates]/ensure: defined content as 
    '{md5}20cfc6cf2a40155d4055d475a109137d'
Notice:
  /Stage[main]/Puppet/File[/home/ubuntu/.ssh/id_rsa]/ensure:
    defined content as '{md5}db19f750104d3bf4e2603136553c6f3e'
Notice: Finished catalog run in 0.27 seconds

Test that the new SSH key is authorized to GitHub correctly:

ubuntu@cookbook:~/puppet$ ssh git@github.com
PTY allocation request failed on channel 0
Hi bitfield/cookbook! You've successfully authenticated, but
  GitHub does not provide shell access.
Connection to github.com closed.

Check that the pull-updates script works properly:

ubuntu@cookbook:~/puppet$ pull-updates
Already up-to-date.
Notice: Finished catalog run in 0.16 seconds

How it works...

Up to now, you've been using your own SSH credentials to access GitHub from the managed machine (using SSH agent forwarding), but that won't work if we want the machine to be able to pull updates unattended, while you're not logged in. So we've created a new SSH keypair and added the public part of it as a deploy key on GitHub, which gives repo access to anyone who has the private half of the key.

We've added this private key as the ubuntu user's default SSH key:

file { '/home/ubuntu/.ssh/id_rsa':
  source => 'puppet:///modules/puppet/ubuntu.priv',
  owner  => 'ubuntu',
  mode   => '0600',
}

This enables the ubuntu user to run git pull in the puppet directory. We've also added the pull-updates script, which does this and runs Puppet if any changes were pulled:

#!/bin/sh
cd /home/ubuntu/puppet
git pull && papply

We deploy this script to the box with Puppet:

file { '/usr/local/bin/pull-updates':
  source => 'puppet:///modules/puppet/pull-updates.sh',
  mode   => '0755',
}

Finally, we've created a cron job that runs pull-updates at regular intervals (every 10 minutes, but feel free to change this if you need to):

cron { 'run-puppet':
  ensure  => 'present',
  command => '/usr/local/bin/pull-updates',
  minute  => '*/10',
  hour    => '*',
}

There's more...

Congratulations, you now have a fully-automated Puppet infrastructure! Once you have checked out the repo on a new machine and applied the manifest, the machine will be set up to pull any new changes and apply them automatically.

So, for example, if you wanted to add a new user account to all your machines, all you have to do is add the account in your working copy of the manifest, and commit and push the changes to GitHub. Within 10 minutes it will automatically be applied to every machine that's running Puppet.

That's very handy, but sometimes we'd like to be able to apply the changes to a specific machine right away, without waiting for them to be picked up by the cron job. We can do this using the Rake tool, and we'll see how to do that in the next section.

Deploying changes with Rake

Rake is a useful tool from the Ruby world which you can use to help automate your Puppet workflow. Although there are lots of other ways to run commands on remote servers, this happens to be the one I use, and it's easily extensible to whatever you need it to do.

The first helpful thing we can have Rake perform for us is log into the remote machine and run the pull-updates script to apply any new Puppet manifest changes. This is fairly simple to do, as you'll see in the following sections.

Getting ready

You may already have Rake installed (try running rake), but if not, here's how to install it:

Run the following command:
```
sudo apt-get install rake
```

How to do it...

Perform the following steps:

In your Puppet repo, create the file, Rakefile with the following contents. Replace ssh... with the correct ssh command line for you to log into your server:

SSH = 'ssh -A -i ~/git/bitfield/bitfield.pem -l ubuntu'

desc "Run Puppet on ENV['CLIENT']"
task :apply do
  client = ENV['CLIENT']
  sh "git push"
  sh "#{SSH} #{client} pull-updates"
end

Add, commit, and push the change to the Git repo:

ubuntu@cookbook:~/puppet$ git add Rakefile
ubuntu@cookbook:~/puppet$ git commit -m "adding Rakefile"
[master 63bb0c1] adding Rakefile
1 file changed, 8 insertions(+)
create mode 100644 Rakefile
ubuntu@cookbook:~/puppet$ git push
Counting objects: 31, done.
Compressing objects: 100% (22/22), done.
Writing objects: 100% (28/28), 4.67 KiB, done.
Total 28 (delta 1), reused 0 (delta 0)
To git@github.com:bitfield/cookbook.git
   a063a5b..63bb0c1  master -> master

On your own computer, check out a working copy of the Puppet repo, if you haven't already got one (replace the Git URL with the URL to your own repo):

[john@Susie:~/git]$ git clone 
  git@github.com:bitfield/cookbook.git
Cloning into 'cookbook'...
remote: Counting objects: 36, done.
remote: Compressing objects: 100% (26/26), done.
remote: Total 36 (delta 1), reused 33 (delta 1)
Receiving objects: 100% (36/36), 5.28 KiB, done.
Resolving deltas: 100% (1/1), done.

Run the following command, replacing cookbook with the address of your server:

[john@Susie:~/git]$ cd cookbook
[john@Susie:~/git/cookbook(master)]$ rake CLIENT=
  cookbook apply
(in /Users/john/git/cookbook)
git push
Everything up-to-date
ssh -A -i ~/git/bitfield/bitfield.pem -l ubuntu cookbook
  pull-updates
Already up-to-date.
Notice: Finished catalog run in 0.18 seconds
Connection to cookbook closed.

How it works...

What you'd do manually to update your server is log into it using SSH, and then run pull-updates. The Rakefile simply automates this for you. First, we set up the correct SSH command line:

SSH = 'ssh -A -i ~/git/bitfield/bitfield.pem -l ubuntu'

The arguments to ssh are as follows:

-A: Forward your SSH key to the remote server, so that you can use it for further authentication
-i KEYFILE: Set the SSH private key to use (in this case, it's the Amazon AWS keyfile that I'm using. You may not need this argument if you're already set up for SSH access to the server with your default key.)
-l ubuntu: Log in as user ubuntu (the standard arrangement on EC2 servers; you may not need this argument if you log into servers using the same username as on your local machine.)

We then define a Rake task named apply:

desc "Run Puppet on ENV['CLIENT']"
task :apply do
     
end

The desc is just a helpful description, which you'll see if you run the command, rake -T, which lists available tasks:

$ rake –T
(in /Users/john/git/cookbook)
rake apply      # Run puppet on ENV['CLIENT']

The code between task and end will be run when you run rake apply. Here's what it does:

client = ENV['CLIENT']

This captures the value of the CLIENT environment variable, which tells the script the address of the remote servers to connect to.

The next line is as follows:

sh "git push"

sh simply runs a command in your local shell, in this case to make sure any changes to the Puppet repo have been pushed to GitHub. If they weren't, they wouldn't be picked up on the remote machine.

sh "#{SSH} #{client} pull-updates"

This is the line which actually connects to the client, using the ssh command line we defined at the start of the script, and the client address. Having logged in, it will then run the pull-updates command as the remote user.

Since we already set up the pull-updates script to do everything necessary to get the latest changes from GitHub and apply Puppet, that's all we need to do.

There's more...

You can now make and apply Puppet changes to your remote servers without ever explicitly logging into them. Once you've installed Puppet on a machine, checked out a copy of the manifest repo, and run Puppet for the first time, you can then do all the administration for that machine remotely.

If you're as lazy as I am, you're already asking, "Couldn't we use a Rake task to do the initial Puppet install and checkout, as well as just applying changes?"

We certainly can, and we'll see how to do that in the next section.

Bootstrapping Puppet with Rake

To make a newly provisioned machine part of our Puppet infrastructure, we just need to run a few commands on it, so let's make this process even easier by adding a new bootstrap task to the Rakefile.

Getting ready...

To get ready for the recipe, do the following:

Add the following line to the top of your Rakefile:
```
REPO = 'git@github.com:bitfield/cookbook.git'
```

Add the following task anywhere in the Rakefile:

desc "Bootstrap Puppet on ENV['CLIENT'] with
  hostname ENV['HOSTNAME']"
task :bootstrap do
  client = ENV['CLIENT']
  hostname = ENV['HOSTNAME'] || client
  commands = <<BOOTSTRAP
sudo hostname #{hostname} && \
sudo su - c 'echo #{hostname} >/etc/hostname' && \
wget http://apt.puppetlabs.com/puppetlabs-release-precise.deb && \
sudo dpkg -i puppetlabs-release-precise.deb && \
sudo apt-get update && sudo apt-get -y install git
  puppet && \
git clone #{REPO} puppet && \
sudo puppet apply --modulepath=/home/ubuntu/puppet
  /modules /home/ubuntu/puppet/manifests/site.pp
BOOTSTRAP
  sh "#{SSH} #{client} '#{commands}'"
end

How to do it...

You'll need a freshly provisioned server (one that you can log in to, but that doesn't have Puppet installed or any other config changes made on it). If you're using EC2, create a new EC2 instance. Get the public instance address from the AWS control panel; it'll be something like:

ec2-107-22-22-159.compute-1.amazonaws.com

Here are the steps to bootstrap the new server using Rake:

Add a node declaration to your nodes.pp file for the hostname you'll be using on the new server. For example, if you wanted to call it cookbook-test, you could use
```
node 'cookbook-test' {
  include puppet
}
```
Run the following command in the Puppet repo on your own machine (substitute the address of the new server as the value of CLIENT, and the hostname you want to use as the value of HOSTNAME). The command should all be on one line:
```
$ rake CLIENT=ec2-107-22-22-159.compute-1.amazonaws.com HOSTNAME=cookbook-test bootstrap
```

You'll see output something like the following:

(in /Users/john/git/cookbook)
ssh -A -i ~/git/bitfield/bitfield.pem -l ubuntu ec2-107-22-22-159.compute-1.amazonaws.com 'sudo hostname cookbook-test && sudo su -c 'echo cookbook-test >/etc/hostname' && wget http://apt.puppetlabs.com/puppetlabs-release-precise.deb && sudo dpkg -i puppetlabs-release-precise.deb && sudo apt-get update && sudo apt-get -y install git puppet && git clone git@github.com:bitfield/cookbook.git puppet && sudo puppet apply --modulepath=/home/ubuntu/puppet/modules /home/ubuntu/puppet/manifests/site.pp'
The authenticity of host 'ec2-107-22-22-159.compute-1.amazonaws.com (107.22.22.159)' can't be established.
RSA key fingerprint is 23:c5:06:ad:58:f3:8d:e5:75:bd:94:6e:1e:a0:a3:a4.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added 'ec2-107-22-22-159.compute-1.amazonaws.com,107.22.22.159' (RSA) to the list of known hosts.
sudo: unable to resolve host cookbook-test
--2013-03-15 15:53:44--  http://apt.puppetlabs.com/puppetlabs-release-precise.deb
Resolving apt.puppetlabs.com (apt.puppetlabs.com)... 96.126.116.126, 2600:3c00::f03c:91ff:fe93:711a
Connecting to apt.puppetlabs.com (apt.puppetlabs.com)|96.126.116.126|:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: 3392 (3.3K) [application/x-debian-package]
Saving to: `puppetlabs-release-precise.deb'
     0K                                                       100%  302M=0s
2013-03-15 15:53:44 (302 MB/s) - `puppetlabs-release-precise.deb' saved [3392/3392]
Selecting previously unselected package puppetlabs-release.
(Reading database ... 25370 files and directories currently installed.)
Unpacking puppetlabs-release (from puppetlabs-release-precise.deb) ...
Setting up puppetlabs-release (1.0-5) ...
Processing triggers for initramfs-tools ...
update-initramfs: Generating /boot/initrd.img-3.2.0-29-virtual
Ign http://us-east-1.ec2.archive.ubuntu.com precise InRelease
[ ... apt output redacted ... ]
Setting up hiera (1.1.2-1puppetlabs1) ...
Setting up puppet-common (3.2.2-1puppetlabs1) ...
Setting up puppet (3.2.2-1puppetlabs1) ...
* Starting puppet agent
puppet not configured to start, please edit /etc/default/puppet to enable
   ...done.
Processing triggers for libc-bin ...
ldconfig deferred processing now taking place
Cloning into 'puppet'...
Warning: Permanently added 'github.com,207.97.227.239' (RSA) to the list of known hosts.
Notice: /Stage[main]/Puppet/Cron[run-puppet]/ensure: created
Notice: /Stage[main]/Puppet/File[/usr/local/bin/pull-updates]/ensure: defined content as '{md5}20cfc6cf2a40155d4055d475a109137d'
Notice: /Stage[main]/Puppet/File[/usr/local/bin/papply]/ensure: defined content as '{md5}171896840d39664c00909eb8cf47a53c'
Notice: /Stage[main]/Puppet/File[/home/ubuntu/.ssh/id_rsa]/ensure: defined content as '{md5}db19f750104d3bf4e2603136553c6f3e'
Notice: Finished catalog run in 0.11 seconds

How it works...

Here's a line by line breakdown of what the Rake task does. In order to make the machine ready to run Puppet, we need to set its hostname to the name you've chosen:

sudo hostname #{hostname}
sudo echo #{hostname} >/etc/hostname

Next, we download and install the Puppet Labs repo package, and install Puppet and Git:

wget http://apt.puppetlabs.com/puppetlabs-release-precise.deb
sudo dpkg -i puppetlabs-release-precise.deb
sudo apt-get update && sudo apt-get -y install git puppet

We need to disable the SSH StrictHostKeyChecking option to avoid being prompted when the script clones the Git repo:

echo -e \"Host github.com\n\tStrictHostKeyChecking no\n\" 
  >> ~/.ssh/config

We check out the repo:

git clone #{REPO} puppet

And finally, run Puppet:

sudo puppet apply --modulepath=/home/ubuntu/puppet/modules
  /home/ubuntu/puppet/manifests/site.pp

The new machine will now pull and apply Puppet changes automatically, without you ever having to log into it interactively. You can use this Rake task to bring lots of new servers under Puppet control quickly.

Automatic syntax checking with Git hooks

It would be nice if we knew there was a syntax error in the manifest before we even committed it. You can have Puppet check the manifest using the puppet parser validate command:

ubuntu@cookbook:~/puppet$ puppet parser validate manifests/nodes.pp
Error: Could not parse for environment production: Syntax error at end of file; expected '}' at /home/ubuntu/puppet/manifests/nodes.pp:3
Error: Try 'puppet help parser validate' for usage

This is especially useful because a mistake anywhere in the manifest will stop Puppet from running on any node, even on nodes that don't use that particular part of the manifest. So checking in a bad manifest can cause Puppet to stop applying updates to production for some time, until the problem is discovered, and this could potentially have serious consequences. The best way to avoid this is to automate the syntax check, by using a pre-commit hook in your version control repo.

How to do it…

Follow these steps:

In your Puppet repo, create a new hooks directory:
```
ubuntu@cookbook:~/puppet$ mkdir hooks
```

Create the file hooks/check_syntax.sh with the following contents (based on a script by Puppet Labs):

#!/bin/sh

syntax_errors=0
error_msg=$(mktemp /tmp/error_msg.XXXXXX)

if git rev-parse --quiet --verify HEAD > /dev/null
then
    against=HEAD
else
    # Initial commit: diff against an empty tree object
    against=4b825dc642cb6eb9a060e54bf8d69288fbee4904
fi

# Get list of new/modified manifest and template files
  to check (in git index)
for indexfile in `git diff-index --diff-filter=AM --
  name-only --cached $against | egrep '\.(pp|erb)'`
do
    # Don't check empty files
    if [ `git cat-file -s :0:$indexfile` -gt 0 ]
    then
        case $indexfile in
            *.pp )
                # Check puppet manifest syntax
                git cat-file blob :0:$indexfile | 
                  puppet parser validate > $error_msg ;;
            *.erb )
                # Check ERB template syntax
                git cat-file blob :0:$indexfile | 
                  erb -x -T - | ruby -c 2> $error_msg >
                    /dev/null ;;
        esac
        if [ "$?" -ne 0 ]
        then
            echo -n "$indexfile: "
            cat $error_msg
            syntax_errors=`expr $syntax_errors + 1`
        fi
    fi
done

rm -f $error_msg

if [ "$syntax_errors" -ne 0 ]
then
    echo "Error: $syntax_errors syntax errors found,
      aborting commit."
    exit 1
fi

Set execute permission for the hook script with the following command:
```
ubuntu@cookbook:~/puppet$ chmod a+x .hooks/check_syntax.sh
```

Add the following task to your Rakefile:

desc "Add syntax check hook to your git repo"
task :add_check do
  here = File.dirname(__FILE__)
  sh "ln -s #{here}/hooks/check_syntax.sh
    #{here}/.git/hooks/pre-commit"
  puts "Puppet syntax check hook added"
end

Run the following command:

ubuntu@cookbook:~/puppet$ rake add_check
ln -s /home/ubuntu/puppet/hooks/check_syntax.sh
  /home/ubuntu/puppet/.git/hooks/pre-commit
Puppet syntax check hook added

How it works…

The check_syntax.sh script will prevent you from committing any files with syntax errors:

ubuntu@cookbook:~/puppet$ git commit -m "test commit"
Error: Could not parse for environment production: Syntax error at
  '}' at line 3
Error: Try 'puppet help parser validate' for usage
manifests/nodes.pp: Error: 1 syntax errors found, aborting commit.

If you add the hooks directory to your Git repo, anyone who has a checkout can run the rake add_check task and get this syntax checking behavior.

Filter reviews by

All

Amazon verified reviews

Alon Becker Feb 09, 2014

I wanted to enhance my Puppet knowledge and this book does just that. From using tags to encrypting sensitive data or with HieraGPG or creating your own custom function this book covers a tremendous amount of material. The book covers masterless puppet installation and really covers the topic well. It goes in to detail of how to set up a masterless ecosystem with Rake, Puppet and Git. My only complaints about this book is it does not cover enough of when to use masterless Puppet or when to have a Puppet master. Thus it does not cover PuppetDB, which is something we use. All in all a great book.I learned a lot from this book and have a bunch of new tools to use in my day to day use of Puppet.

Amazon Verified review

Stephen Collier Mar 22, 2015

Great book to learn puppet easy to read, well written. Has helped me understand how and why puppet works.

ad_amazon Dec 07, 2013

Wer schon ein paar Grundlagen hat, kann in dem Buch viele interessante Beispiele finden.Ich schaue immer mal wieder in das Buch.

Thomas Dao Jan 04, 2014

This book is very well-structured for the beginner to intermediate puppet and devops person.Starts off with overview and goes thru step by step examples and problems with helpfulalong the way. Using this book was able to setup and create recipes at the workplace. Got the PDFversion and read it both on my iPad and Android tablet.[...]

carl Dec 02, 2013

John Arundel wrote another book - Puppet 2.7 cookbook in 2011. You can consider this one is the revised version with additional puppet 3 features added. Through this book, John shared his hands-on puppet development experience and gave his approach to deploy puppet infrastructure in distributed environment. Although each environment differs, reader will be able to get some idea from the usage described in book and apply the guideline within their own infrastructure.Many people like me, we are not coding puppet every day. Instead we build the framework, we developed module to meet business request. Then we move on to the next task. We don't spend lots of time to tune our code for best efficiency, nor we study public module to learn tricks/patterns on daily basis. This book happens to be a good resource for such readers and it uses many user cases to demonstrate what is the most efficient way.Author is not trying to cover every aspect in a less than 300 pages book. This book is more or less the summary of author's puppet working experience. He tried to list the most important concepts/features, wished readers can benefit from them and can have a more advanced start line.I definitely recommend this book to people who start programming puppet and developers, who are doing intermittent puppet development.

Puppet 3 Cookbook: An essential book if you have responsibility for servers. Real-world examples and code will give you Puppet expertise, allowing more control over servers, cloud computing, and desktops. A time-saving, career-enhancing tutorial , Second Edition

What do you get with Print?

Contact Details

Shipping Address

Billing Address

Key benefits

Description

Who is this book for?

What you will learn

Product Details

What do you get with Print?

Contact Details

Shipping Address

Billing Address

Product Details

Packt Subscriptions

Frequently bought together

Table of Contents

Recommendations for you

Customer reviews

Filter reviews by

People who bought this also bought

About the author

FAQs