Configuring Distributed Rails Applications with Chef: Part 2

In my Part 1 post, I gave you the low down about Chef. I covered what it’s for and what it’s capable of. Now let’s get into some real code and take a look at how you install and run Chef Solo and Chef Server.

What we want to accomplish

First let’s make a list of some goals. What are we trying to get out of deploying and provisioning with Chef?

• Once we have it set up, provisioning a new server should be simple; no more than a few simple commands.
• We want it to be platform-agnostic so we can deploy any VPS provider we choose with the same scripts.
• We want it to be easy to follow and understand. Any new developer coming later should have no problem figuring out what’s going on.
• We want the server to be nearly automated. It should take care of itself as much as possible, and alert us if anything goes wrong.

Before we start, let’s decide on a stack. You should feel free to run any stack you choose. This is just what I’m using for this post setup:

• Ubuntu 12.04 LTS
• RVM
• Ruby 1.9.3+
• Rails 3.2+
• Postgres 9.3+
• Redis 3.1+
• Chef
• Git

Now that we’ve got that out of the way, let’s get started!

Step 1: Install the tools

First, make sure that all of the packages we download to our VPS are up to date:

~$ sudo apt-get update

Next, we'll install RVM (Ruby Version Manager). RVM is a great tool for installing Ruby. It allows you to use several versions of Ruby on one server. Don't get ahead of yourself though; at this point, we only care about one version.

To install RVM, we’ll need curl:

~$ sudo apt-get install curl

We also need to install Git. Git is an open source distributed version control system, primarily used to maintain software projects. (If you didn't know that much, you're probably reading the wrong post. But I digress!):

~$ sudo apt-get install git

Now install RVM with this curl command:

~$ curl -sSL https://get.rvm.io | bash -s stable

You’ll need to source RVM (you can add this to your bash profile):

~$ source ~/.rvm/scripts/rvm

In order for it to work, RVM has some of its own dependencies that need to be installed. To automatically install them, use the following command:

~$ rvm requirements

Once we have RVM set up, installing Ruby is simple:

~$ rvm install 1.9.3

Ruby 1.9.3 is now installed! Since we'll be accessing it through a tool that can potentially have a variety of Ruby versions loaded, we need to tell the system to use this version as the default:

~$ rvm use 1.9.3 --default

Next we'll make sure that we can install any Ruby Gem we need into this new environment. We'll stick with RVM for installing gems as well. This'll ensure they get loaded into our Ruby version properly. Run this command:

~$ rvm rubygems current

Don’t worry if it seems like you’re setting up a lot of things manually now. Once Chef is set up, all of this will be part of your cookbooks, so you’ll only have to do this once.

Step 2: Install Chef and friends

First, we'll start off by cloning the Opscode Chef repository:

~$ git clone git://github.com/opscode/chef-repo.git chef

With Ruby and RubyGems set up, we can install some gems! We’ll start with a gem called Librarian-Chef. Librarian-Chef is sort of a Rails Bundler for Chef cookbooks. It'll download and manage cookbooks that you specify in Cheffile. Many useful cookbooks are published by different sources within the Chef community. You'll want to make use of them as you build out your own Chef environment.

~$ gem install librarian-chef

 Initialize Librarian in your Chef repository with this command:

~$ cd chef
~/chef$ librarian-chef init

This command will create a Cheffile in your Chef repository. All of your dependencies should be specified in that file. To deploy the stack we just built, your Cheffile should look like this:

1  site 'http://community.opscode.com/api/v1'
2  cookbook 'sudo'
3  cookbook 'apt'
4  cookbook 'user'
5  cookbook 'git'
6  cookbook 'rvm'
7  cookbook 'postgresql'
8  cookbook 'rails'
~

Now use Librarian to pull these community cookbooks:

~/chef$ librarian-chef install

Librarian will pull the cookbooks you specify, along with their dependencies, to the cookbooks folder and create a Cheffile.lock file. Commit both Cheffile and Cheffile.lock to your repo:

 ~/chef$ git add Cheffile Cheffile.lock
 ~/chef$ git commit -m “updated cookbooks list”

There is no need to commit the cookbooks folder, because you can always use the install command and Librarian will pull the same group of cookbooks with the correct versions. You should not touch the cookbooks folder—let Librarian manage it for you. Librarian will overwrite any changes you make inside that folder. If you want to manually create and manage cookbooks, outside of Librarian, add a new folder, like local-cookbooks, for instance.

Step 3: Cooking up somethin’ good!

Now that you see how to get the cookbooks, you can create your roles. You use roles to determine what role a server instance would have in you server stack, and you specify what that role would need. For instance, your Database Server role would most likely need a Postgresql server (or you DB of choice), a DB client, user authorization and management, while your Web Server would need Apache (or Nginx), Unicorn, Passenger, and so on. You can also make base roles, to have a basic provision that all your servers would have.

Given what we’ve installed so far, our basic configuration might look something like this:

  name "base"
  description "Basic configuration for all nodes"
  run_list(
    'recipe[git]',
    'recipe[sudo]',
    'recipe[apt]',
    'recipe[rvm::user]',
    'recipe[postgresql::client]'
  )
  
  override_attributes(
    authorization: {
      sudo: {
        users: ['ubuntu'],
        passwordless: true
      }
    },
    rvm: {
      rubies: ['ruby-1.9.3-p125'],
      default_ruby: 'ruby-1.9.3-p125',
      global_gems: ['bundler', 'rake']
    }
  )
  
~

Deploying locally with Chef Solo:

Chef Solo is a Ruby gem that runs a self-contained Chef instance. Solo is great for running your recipes locally to test them, or to provision development machines. If you don’t have a hosted Chef Server set up, you can use Chef Solo to set up remote servers too. If your architecture is still pretty small, this might be just what you need.

We need to create a Chef configuration file, so we’ll call it deploy.rb:

 root  = File.absolute_path(File.dirname(__FILE__))
 roles = File.join(root, 'cookbooks')
 books = File.join(root, 'roles')
 file_cache_path root
 cookbook_path books
 role_path roles
  
~

We’ll also need a JSON-formatted configuration file. Let’s call this one deploy.json:

  { 
     "run_list": ["recipe[base]"]
  }    
~

Now run Chef with this command:

~/chef$ sudo chef-solo -j deploy.json -c deploy.rb

Deploying to a new Amazon EC2 instance:

You’ll need the Chef server for this step.

First you need to create a new VPS instance for your Chef server and configure it with a static IP or a domain name, if possible. We won’t go through that here, but you can find instructions for setting up a server instance on EC2 with a public IP and configuring a domain name in the documentation for your VPS.

Once you have your server instance set up, SSH onto the instance and install Chef server. Start by downloading the dep package using the wget tool:

 ~$ wget https://opscode-omnibus-packages.s3.amazonaws.com/
ubuntu/12.04/x86_64/chef-server_11.0.10-1.ubuntu.12.04_amd64.deb

Once the dep package has downloaded, install Chef server like so:

~$ sudo dpkg -i chef-server*

When it completes, it will print to the screen an instruction that you need to run this next command to actually configure the service for your specific machine. This command will configure everything automatically:

~$ sudo chef-server-ctl reconfigure

Once the configuration step is complete, the Chef server should be up and running. You can access the web interface immediately by browsing to your server's domain name or IP address.

Now that you’ve got Chef up and running, install the knife EC2 plugin. This will also install the knife gem as a dependency:

~$ gem install knife-ec2

You now have everything you need! So create another VPS to provision with Chef. Once you do that, you’ll need to copy your SSH keys over:

~$ ssh-copy-id root@yourserverip

You can finally provision your server! Start by installing Chef on your new machine:

~$ knife solo prepare root@yourserverip

This will generate a file, nodes/yourserverip.json. You need to change this file to add your own environment settings. For instance, you will need to add username and password for monit.

You will also need to add a password for postgresql to the file. Run the openssl command again to create a password for postgresql. Take the generated password, and add it to the file.

Now, you can finally provision your server! Start the Chef command:

~$ knife solo cook root@yourserverip

Now just sit back, relax and watch Chef cook up your tasty app server. This process may take a while. But once it completes, you’ll have a server ready for a Rails, Postgres, and Redis!

I hope these posts helped you get an idea of how much Chef can simplify your life and your deployments. Here’s a couple of links with more information and references about Chef:

Chef community site:
http://cookbooks.opscode.com/

Chef Wiki:
https://wiki.opscode.com/display/chef/Home

Chef Supermarket:
https://community.opscode.com/cookbooks?utf8=%E2%9C%93&q=user

Chef cookbooks for busy Ruby developers:
http://teohm.com/blog/2013/04/17/chef-cookbooks-for-busy-ruby-developers/

Deploying Rails apps with Chef and Capistrano:
http://www.slideshare.net/SmartLogic/guided-exploration-deploying-rails-apps-with-chef-and-capistrano

About the author

Rahmal Conda is a Software Development Professional and Ruby aficionado from Chicago. After 10 years working in web and application development, he moved out to the Bay Area, eager to join the startup scene. He had a taste of the startup life in Chicago working at a small personal finance company. After that he knew it was the life he had been looking for. So he moved his family out west. Since then he's made a name for himself in the social space at some high profile Silicon Valley startups. Right now he's the one of the Co-founders and Platform Architect of Boxes, a mobile marketplace for the world's hidden treasures.