Using Vagrant remotely with AWS EC2 and Docker

Another powerful usage of Vagrant can be with remote IaaS resources such as Amazon EC2. Amazon Web Services Elastic Compute Cloud (EC2) and similar Infrastructure-as-a-Service providers like Google Cloud, Azure or Digital Ocean, to name a few, are selling virtual machines with varying compute power and network bandwidth for a fee. You don't always have all the necessary CPU and memory you need on your laptop, or you need to have some specific computing power for a task, or you just want to replicate part of an existing production environment: here's how you can leverage the power of Vagrant using Amazon EC2.

Here, we'll deploy a Ghost blog with an NGINX reverse proxy, all on Docker, using an Ubuntu Xenial 16.04 on AWS EC2! This is to simulate a real deployment of an application, so you can see if it is working in real conditions.

Getting ready

To step through this recipe, you will need the following:

• A working Vagrant installation (no hypervisor needed)
• An Amazon EC2 account (or create one for free at https://aws.amazon.com/ if you don't have one already), with valid Access Keys, a keypair named iac-lab, a security group named iac-lab allowing at least HTTP ports, and SSH access.
• An Internet connection

How to do it…

Begin by installing the plugin:

$ vagrant plugin install vagrant-aws

A requirement of this plugin is the presence of a dummy Vagrant box that does nothing:

$ vagrant box add dummy https://github.com/mitchellh/vagrant-aws/raw/master/dummy.box

Remember how we configured the Docker provider in the previous recipes? This is no different:

config.vm.provider :aws do |aws, override|
  # AWS Configuration
  override.vm.box = "dummy"
end

Then, defining an application VM will consist of specifying which provider it's using (AWS in our case), the Amazon Machine Image (AMI) (Ubuntu 16.04 LTS in our case), and a provisioning script that we creatively named script.sh.

You can find other AMI IDs at http://cloud-images.ubuntu.com/locator/ec2/:

config.vm.define "srv-1" do |config|
    config.vm.provider :aws do |aws|
      aws.ami = "ami-c06b1eb3"
    end
    config.vm.provision :shell, :path => "script.sh"
end

So what is the AWS-related information we need to fill in so Vagrant can launch servers on AWS?

We need the AWS Access Keys, preferably from environment variables so you don't hardcode them in your Vagrantfile:

aws.access_key_id = ENV['AWS_ACCESS_KEY_ID']
aws.secret_access_key = ENV['AWS_SECRET_ACCESS_KEY']

Indicate the region and availability zone where you want the instance to start:

aws.region = "eu-west-1"
aws.availability_zone = "eu-west-1a"

Include the instance type; here, we've chosen the one included in the AWS free tier plan so it won't cost you a dime with a new account:

aws.instance_type = "t2.micro"

Indicate in which security group this instance will live (it's up to you to adapt the requirements to your needs):

aws.security_groups = ['iac-lab']

Specify the AWS keypair name, and override the default SSH username and keys:

aws.keypair_name = "iac-lab"
override.ssh.username = "ubuntu"
override.ssh.private_key_path = "./keys/iac-lab.pem"

Under some circumstances, you can experience a bug with NFS while using Vagrant and AWS EC2, so I choose to disable this feature:

override.nfs.functional = false

Finally, it's a good practice to tag the instances, so you can later find out where they come from:

aws.tags = {
  'Name'   => 'Vagrant'
}

Add a simple shell script that will install Docker and docker-compose, then execute the docker-compose file:

#!/bin/sh
# install Docker
curl -sSL https://get.docker.com/ | sh
# add ubuntu user to docker group
sudo usermod -aG docker ubuntu
# install docker-compose
curl -L https://github.com/docker/compose/releases/download/1.8.0/docker-compose-`uname -s`-`uname -m` > /usr/local/bin/docker-compose
sudo chmod +x /usr/local/bin/docker-compose
# execute the docker compose file
cd /vagrant
docker-compose up -d

Include both NGINX configuration and docker-compose.yml files from the previous recipe and you're good to go:

$ vagrant up
Bringing machine 'srv-1' up with 'aws' provider...
[…]
==> srv-1: Launching an instance with the following settings...
==> srv-1:  -- Type: t2.micro
==> srv-1:  -- AMI: ami-c06b1eb3
==> srv-1:  -- Region: eu-west-1
[…]
==> srv-1: Waiting for SSH to become available...
==> srv-1: Machine is booted and ready for use!
[…]
==> srv-1:  docker version
[…]
==> srv-1: Server:
==> srv-1:  Version:      1.12.1
[…]
==> srv-1: Creating vagrant_app_1
==> srv-1: Creating vagrant_front_1

Open your browser at http://a.b.c.d/ (using the EC2 instance public IP) and you'll see your Ghost blog behind an NGINX reverse proxy, using Docker containers, using Vagrant on Amazon EC2.

A common usage for such a setup is for the developer to test the application in close to real production conditions, maybe to show a new feature to a remote product owner, replicate a bug seen only in this setup, or at some point in the CI. Once Docker containers have been built, smoke test them on EC2 before going any further.