Amazon EC2 Cookbook

Chapter 1. Selecting and Configuring Amazon EC2 Instances

In this chapter, we will cover recipes for:

Choosing the right AWS EC2 instance types
Preparing AWS CLI tools
Launching EC2 instances using EC2-Classic and EC2-VPC
Allocating Elastic IP addresses
Creating an instance with multiple NIC cards and a static private IP address
Selecting the right storage for your EC2 instance
Creating tags for consistency
Configuring security groups
Creating an EC2 key pair
Grouping EC2 instances using placement groups
Configuring Elastic Load Balancing
Architecting for high availability
Creating instances for AWS Marketplace

Choosing the right AWS EC2 instance types

An EC2 instance is a virtual machine hosted on the AWS Cloud. As an instance creator, you have root privileges on any instances you started. An EC2 instance can be used to host one or more of web servers, application servers, database servers, or backend processes/services requiring heavy compute or graphics processing. Depending on your application architecture, you can choose to host various components distributed across multiple EC2 instances.

AWS offers different types of storage attachments viz. SSD and magnetic. If you require higher storage performance, then ensure that the EC2 instance type you choose supports SSD.

There are three distinct purchasing options available for provisioning the AWS EC2 instances:

On-demand instances: These instances are billed on an hourly basis and no upfront payments are required. Applications with unpredictable workloads or short-duration requirements are best handled using on-demand instances. This is the default purchasing option in AWS.
Spot instances: There are no upfront costs for provisioning spot instances, and the costs are typically much lower than the on-demand instances. The provisioning is done through a bidding process. If you lose the bid, you will not get the EC2 instances. Usually, applications that are viable only at very low compute prices are a good use case for using spot instances.
Reserved instances: These instances can be 50–60% cheaper than on-demand instances. This option is available for 1 and 3 year plans. Applications with predictable workloads that require compute instances for longer durations are a good fit for using reserved instances.

There are several AWS EC2 instance families available for different types of application workloads. These include general purpose, memory optimized, compute optimized, storage optimized, and GPU instances. Choosing the right instance type is a key decision in provisioning EC2 instances.

Note

Refer to http://aws.amazon.com/ec2/instance-types/ for descriptions and typical use cases for each of these EC2 instance types.

We recommend that you start with a minimum required instance type that meets your requirements. In many cases, choosing a general-purpose EC2 instance is a good starting point. You can then load test your application on this instance for overall performance and stability. If your applications are not meeting your performance objectives on the current instance type, you can easily upgrade the size or choose a more specialized instance type, though this process does require a reboot of your instance. This approach can help you optimize your instance sizes and types.

To achieve high performance or meet compliance requirements or to just avoid noisy neighbors, the type of tenancy chosen is a critical decision. On AWS, there are two types of tenancy, dedicated and shared. In the case of dedicated tenancy, AWS provisions your instance on dedicated hardware. These instances are isolated from instances created using the shared tenancy option and instances created by other tenants. Tenancy can be configured at the instance level or at the VPC level. Once the option is selected, changing the tenancy type (instance or VPC level) is not allowed. There are cost implications of using dedicated tenancy versus shared tenancy.

In addition, if we want to set the Provisioned IOPS parameter, then we have to use the EBS-optimized instance types. Amazon EBS-optimized instances deliver dedicated throughput to Amazon EBS, with options ranging between 500 Mbps and 2,000 Mbps (depending on the instance type selected). EBS-optimized flag provides dedicated and more consistent link between EC2 and EBS. EBS optimized EC2 instances also allocate dedicated bandwidth to its attached volumes.

How to do it…

In this recipe, we will create and launch an EC2 instance.

After you log in to the AWS console, choose Services, and then select EC2 from the list of AWS services. At this stage, the EC2 Dashboard will appear, then perform the following operations:
1. Press the Launch Instance button.
2. AWS supports two types of virtualization paravirtual (PV) and hardware virtual machine (HVM). For Windows-based instances, HVM is the only option available to you. For Linux-based instances, you can use either PV or HVM. The I/O drivers, which help PV to get rid of the network and hardware emulation, are now available on HVM. Hence, HVM can give better performance than PV. Choose an AMI from the list according to your requirement.
3. Filter instance type:
Choose Columns for more details:
Choose EBS-Optimized Available instance type in the Choose an Instance Type wizard to avail this performance benefit:
Note
In EBS-backed instances, the root device for an instance launched using an AMI is an Amazon EBS volume created from an Amazon EBS snapshot. If we use an EBS-backed instance type, then we may or may not choose to use the instance's storage devices. We can also change the instance size, subsequently, or stop the instances to stop billing.
In case, we choose to use the instance's storage, any data stored on it will be lost after a restart of the instance. The root device for an instance launched from the AMI is an instance store volume created from a template stored in Amazon S3. We can't stop these instances—we can only terminate them. In addition, we can't change the size of instance, once created.
Next, we configure the VPC, subnet, and tenancy details for the instance:
If you don't want to customize any further then review and launch the instance.

Preparing AWS CLI tools

AWS CLI is a set of unified command-line tools to work with multiple AWS services. Using AWS CLI tools you can manage EC2 resources (such as instances, security groups, and volumes) and your VPC resources (such as VPCs, subnets, route tables, and Internet gateways).

How to do it…

In the following two sections, we list the set of instructions required to accomplish this on Linux and Windows/Mac platforms.

Getting access key ID and secret access key

You need AWS access key ID and AWS secret access key to access AWS services. Instead of generating these credentials from the root account, it's always best practice to use IAM users. You should save these credentials in a secure location. If you lose these keys, you must delete the access key and then create a new key.

You can get the AWS credentials from AWS management portal by following these steps:

Log in to the AWS management portal using your AWS username and password.
Select account name from top menu at the right corner in the console.
Select security credentials.
Click on access keys (access key ID and secret access key).
Click on the Create New Access Key button.
Click on Download Key File, which will download the file. If you do not download the key file now, you will not be able to retrieve your secret access key again.
Copy this key file to a secure location.
Note
Don't upload your code base with AWS security credentials to public code repositories such as GitHub. Attackers are scraping GitHub for AWS credentials. If anyone gets access to these credentials, they can misuse your AWS account.

Installing AWS CLI using pip in Linux

We can use the pip tool to install the Python packages.

Before installing Python, please check whether Python is already installed on your machine or not using the following command. If Python is already installed on your machine, then skip to the pip installation step.
```
$ python --help
```
Start by installing Python. Download the compressed TAR archive file from the Python site, and then install it using the commands listed below. The following steps target the apt-based Linux distributions:
```
$ sudo apt-get install gcc
$ wget https://www.python.org/ftp/python/2.7.8/Python-2.7.8.tgz
$ tar -zxvf Python-2.7.8.tgz
$ cd Python-2.7.8
$ ./configure
$ make
$ sudo make install
```
Next, check the Python installation:
```
$ python –help
```
Before installing pip, please check whether pip is already installed on your machine or not by using the following command. If pip is already installed on your machine, then skip to the awscli installation step:
```
$ pip –help
```
Move on to installing pip:
```
$ sudo apt-get install pip
```
Then install AWS CLI. If you have already installed awscli, you can upgrade the installation using the –upgrade option.
```
$ sudo pip install awscli
```
Next, configure AWS CLI.
On the command prompt, type the following command, which will prompt for the AWSAccessKey ID, AWSSecretKey, default AWS region, and default output format.
```
$ sudo aws configure
```
Finally, check the installation by getting regions list:
```
$ sudo aws ec2 describe-regions
```

Installing AWS CLI using pip in Windows/Mac

We can use the pip tool to install the Python packages.

Before installing Python, please check whether Python is already installed on your machine or not by using the following command. If Python is already installed on your machine, then skip to the pip installation step.
```
$ python –help
```
Start by installing Python. Download the installer from the following URL and install Python by using that installer: https://www.python.org/downloads/.
Check your Python installation:
```
$ python –help
```
Before installing pip, check whether pip is already installed on your machine or not by using the following command. If pip is already installed on your machine, skip to the awscli installation step.
```
$ pip –help
```
In the next step, we install pip. Download and run the installation script from https://bootstrap.pypa.io/get-pip.py. After that, run the following command:
```
$ python get-pip.py
```
Install AWS CLI. If you have already installed awscli, you can upgrade the installation using the –upgrade option.
```
$ pip install awscli
```
Next, we configure AWS CLI. Execute the following command from the command prompt.
```
$ aws configure
```
This command will then prompt you for the AWSAccessKey ID, AWSSecretKey, default AWS region, and default output format.
Check the installation by getting the regions list:
```
$ aws ec2 describe-regions
```

Launching EC2 instances using EC2-Classic and EC2-VPC

Your EC2 instance receives a private IP address from the EC2-Classic range each time it's started, whereas your instance receives a static private IP address from the address range in EC2-VPC. You can only have one private IP address in EC2-Classic, but in EC2-VPC, we have multiple private IP addresses. If you attach an EIP (Elastic IP) to EC2-Classic instance, it will get dissociated when you stop the instance. But for VPC EC2 instance, it remains associated even after you stop it. We can create subnets, routing tables, and Internet gateways in VPC. For on-premise connectivity, we need VPC.

Note

There are different VPC options available, depending on whether you created your AWS account before or after 2013-12-04.

If you created your AWS account after 2013-12-04, then only EC2-VPC is supported. In this case, a default VPC is created in each AWS region. Therefore, unless you create your own VPC and specify it when you launch an instance, your instances are launched in your default VPC.

If you created your AWS account before 2013-03-18, then both EC2-Classic and EC2-VPC are supported in the regions you used before, and only EC2-VPC in regions that you didn't use. In this case, a default VPC is created in each region in which you haven't created any AWS resources. Therefore, unless you create your own VPC and specify it when you launch an instance in a region (that you haven't used before), the instance is launched in your default VPC for that region. However, if you launch an instance in a region that you've used before, the instance is launched in EC2-Classic.

In this recipe, we will launch EC2 instances using EC2-Classic and EC2-VPC.

Getting started…

Before we launch the EC2 instances, we need the image ID.

Run the following command to get the list of images. We can apply the filter to identify a specific image. Record the image ID for later use:

$ aws ec2 describe-images
--filter [Filter]

Note

You can specify one or more filters in this command.

By executing the following command, you obtain the image ID of a 64-bit version of Ubuntu 12.04 image:

$ aws ec2 describe-images
--filter
"Name=virtualization-type,Values=paravirtual"
"Name=root-device-type,Values=ebs" "Name=architecture,Values=x86_64"
"Name=name,Values=ubuntu/images/ebs/ubuntu-precise-12.04-amd64-server-20130204"

How to do it…

We will see the EC2 instances being launched, one by one:

Launching the EC2 instance in EC2-Classic

Using the following command, we can launch instances in EC2-Classic. You can specify the number of instances to launch using the count parameter.

$ aws ec2 run-instances 
--image-id [ImageId] 
--count [InstanceCount] 
--instance-type [InstanceType] 
--key-name [KeyPairName] 
--security-group-ids [SecurityGroupIds]

The parameters used in this command are described as follows:

[ImageId]: This is the ID of the image
[InstanceCount]: This gives number of instances to be created
[InstanceType]: This gives the type of EC2 instance
[KeyPairName]: This parameter provides the key/pair name for authentication
[SecurityGroupIds]: This one provides security group IDs

The following command will create a micro instance in EC2-Classic (in the Singapore region):

$ aws ec2 run-instances 
--image-id ami-7e2c612c 
--count 1 
--instance-type t1.micro 
--key-name WebServerKeyPair 
--security-group-ids sg-ad70b8c9

Launching the EC2 instance in VPC

Run the following command to launch instances in EC2-VPC. We need to specify the subnet ID while creating an instance in EC2-VPC. Before creating the instance in EC2-VPC, you have to create the VPC and subnets inside it.

$ aws ec2 run-instances 
--image-id [ImageId] 
--count [InstanceCount] 
--instance-type [InstanceType] 
--key-name [KeyPairName] 
--security-group-ids [SecurityGroupIds]
--subnet-id [SubnetId]

Here, SubnetId specifies the subnet where you want to launch your instance.

Next, run the following command to create a micro instance in EC2-VPC (in the Singapore region):

$ aws ec2 run-instances 
--image-id ami-7e2c612c 
--count 1 
--instance-type t1.micro 
--key-name WebServerKeyPair
--security-group-ids sg-ad70b8c8 
--subnet-id subnet-aed11acb

Allocating Elastic IP addresses

Elastic IP (EIP) address is the static public IP address. You can attach and detach the EIP from EC2 instance at any time. Instances in EC2-Classic support only one private IP address and corresponding EIP. Instances in EC2-VPC support multiple private IP addresses, and each one can have a corresponding EIP. If you stop the instance in EC2-Classic the EIP is disassociated from instance, and you have to associate it again when you start the instance. But if you stop the instance in EC2-VPC, the EIP remains associated with the EC2 instance.

In this recipe, we list the commands for allocating an Elastic IP address in a VPC and associating it with the network interface.

How to do it…

For allocating EIP addresses, perform the following steps:

Run the following command to allocate the EIP:
```
$ aws ec2 allocate-address 
--domain [Domain]
```
You have to specify whether domain is standard or VPC. Record the allocation ID for further use.
Domain value indicates whether the EIP address is used with instances in EC2-Classic (standard) or instances in a EC2-VPC (VPC).
Next, run the following command to create the EIP in VPC:
```
$ aws ec2 allocate-address --domain vpc
```
Then, run the following command to associate the EIP to the Elastic Network Interface (ENI):
```
$ aws ec2 associate-address 
--network-interface-id [NetworkInterfaceId]
--allocation-id [AllocationId]
```
You need to provide the network interface ID of the ENI and allocation ID of the EIP you obtained in step 1. If you don't specify the private IP address, then the Elastic IP address is associated with the primary IP address.
The parameters used in this command are described here:
- [NetworkInterfaceId]: This gives the ENI ID to attach
- [AllocationId]: This provides the allocation ID of the EIP for EC2-VPC

Finally, run the following command to associate the EIP to ENI:

$ aws ec2 associate-address 
--network-interface-id eni-d68df2b3 
--allocation-id eipalloc-82e0ffe0

Creating an instance with multiple NIC cards and a static private IP address

With multiple NICs, you can better manage your network traffic. Multiple NICs is one of the prerequisite for high availability. The number of NICs attached to the EC2 instance will depend on the type of EC2 instance. ENI's and multiple private IP addresses are only available for instances running in a VPC. In cases of instance failure, we can detach and then re-attach the ENI to a standby instance, where DNS changes are not required for achieving business continuity. We can attach multiple ENIs from different subnets to an instance, but they both should be in the same availability zone. This enables us to separate the public-facing traffic from the management traffic.

We can have one primary address and one or more secondary addresses for an NIC. We can detach and then attach NIC from one instance to another. We can attach one Elastic IP to each private address. When you launch an instance, a public IP address can be autoassigned to the network interface for eth0. This is possible only when you create a network interface for eth0 instead of using an existing network interface. You can detach secondary NIC (ethN) when an instance is running or stopped. However, you can't detach the primary (eth0) interface. In addition, you can attach security groups to NIC. If you set the instance termination policy to delete on termination, then the NIC will automatically be deleted, if you delete the EC2 instance.

How to do it…

Creating an instance with multiple NIC cards requires us to create a network interface, attach it to an instance, and finally associate the EIP to the ENI.

Creating a network interface

Use the following steps to create a network interface:

Run the following command to create the ENI. You will need to provide the subnet ID, security group IDs, and one or more private IP addresses.
```
$ aws ec2 create-network-interface 
--subnet-id [SubnetId] 
--groups [SecurityGroupIds]
--private-ip-addresses [PrivateIpAddressList] 
```
The parameters used in this command are described as follows:
- [SubnetId]: This gives the ID of the subnet to associate with the network interface
- [SecurityGroupIds]: This parameter provides IDs of one or more security groups
- [PrivateIpAddressList]: This is used to show list of private IP addresses
  Syntax:
```
PrivateIpAddress=string,Primary=boolean 
```

Next, run the following command to create the ENI with private IP addresses 10.0.0.26 and 10.0.0.27:

$ aws ec2 create-network-interface 
--subnet-id subnet-aed11acb 
--groups sg-ad70b8c8 
--private-ip-addresses PrivateIpAddress=10.0.0.26,Primary=true PrivateIpAddress=10.0.0.27,Primary=false

In the next step, we attach the network interface to the instance.

Attaching the network interface to an instance

By running the following command, we can attach the ENI to an EC2 instance. You will need to provide the ENI ID, EC2 instance ID, and the device index.

$ aws ec2 attach-network-interface 
--network-interface-id [NetworkInterfaceId]
--instance-id [InstanceId]
--device-index [DeviceIndex]

The parameters used in this command are described as follows:

[NetworkInterfaceId]: This parameter provides the network interface ID to attach to an EC2 instance
[InstanceId]: This one provides an EC2 instance ID
[DeviceIndex]: This parameter provides the index of the device for the network interface attachment

Then, run the following command to attach the ENI to the EC2 instance:

$ aws ec2 attach-network-interface 
--network-interface-id eni-5c88f739 
--instance-id i-2e7dace3 
--device-index 1

Associating the EIP to the ENI

By running the following command, we can associate the EIP to the ENI. You have to provide the ENI ID, EIP allocation ID, and the private address.

$ aws ec2 associate-address
--network-interface-id [NetworkInterfaceId]
--allocation-id [AllocationId]
--private-ip-address [PrivateIpAddress]

The parameters used in this command are described as follows:

[NetworkInterfaceId]: This parameter provides the network interface ID to attach to an EC2 instance
[AllocationId]: This gives the allocation ID of EIP, which is required for EC2-VPC
[PrivateIpAddress]: If no private IP address is specified, the Elastic IP address is associated with the primary private IP address

Next, run the following command to associate the EIP to 10.0.0.26 (the private IP address of the ENI):

$ aws ec2 associate-address
--network-interface-id eni-5c88f739
--allocation-id eipalloc-d59f80b7
--private-ip-address 10.0.0.26

Selecting the right storage for your EC2 instance

Instance storage consists of disks that are physically attached to the host computer. Data on these disks is lost once the instance restarts. For persistence across restarts, we need to use EBS volumes.

EBS volumes are automatically replicated within its availability zone to protect against component failures.

AWS EBS volumes are persisted independently from your EC2 instances. These are connected through Network Attached Storage (NAS). If you lose the EC2 instance, then the data stored on EBS will still be available to a newly provisioned EC2 instance. You can attach as many EBS volumes as you want. However, an EBS volume can only be attached to one EC2 instance at a time. You can detach EBS volume from one EC2 instance, and then attach to a different EC2 instance. An I/O request of up to 256 Kilobytes is counted as a single I/O operation (IOP).

If we use standard EBS volumes as the boot device volume, then the boot process of a Windows or Linux machine is fast. We can have storage up to 16 TB and 10,000 IOPS per volume. General purpose SSD is best for boot device volumes, and small and medium sized databases. These SSD volumes can deliver a maximum throughput of 160 Mbps when attached to EBS-optimized instances.

Provisioned IOPS (SSD) volumes deliver within 10% of the IOPS performance 99.9% of the time over a given year. If we have a 200 GB volume with 1,000 IOPS, then 99.9% of the time, actual I/O on this volume will be at 900 IOPS or higher. Many database workloads need provisioned IOPS for consistent performance. We can configure storage up to 16 TB and 20,000 IOPS per volume. Provisioned IOPS volumes can deliver 320 Mbps when attached to EBS-optimized instances.

Magnetic disks are a lower cost option for EBS volumes. If data read frequency is low then this type of EBS volume is a good option.

Note

If you want more IOPS than what single EBS volume provides, configure the RAID array on multiple EBS volumes.

Encryption is also possible while using the EBS volumes. Encryption is done for data at rest, data in transit, and disk I/O. Using encrypted EBS volumes have a minor effect on I/O latency, but the performance remains the same. To encrypt EBS volume, you just need to select the Encrypt this volume checkbox when creating EBS volume from AWS console. In this recipe, we list the commands for creating an EBS volume, and then attaching it to an EC2 instance.

How to do it…

Run the following command to list the availability zones in a selected region. If the command is run in the ap-southeast-1 region, you get the list of availability zones in the Singapore region.

$ aws ec2 describe-availability-zones

Creating an EBS volume

Run the following command to create an Amazon EBS volume that can be attached to an instance in the same availability zone. Record the volume ID for further usage.

$ aws ec2 create-volume 
--availability-zone [AvailabilityZone] 
--volume-type [VolumeType]
--iops [IOPS]
--size [Size]

The parameters used in this command are described as follows:

[AvailabilityZone]: This specifies the availability zone in which to create the volume. Use the describe-availability-zones command to list the availability zones.
[VolumeType]: This gives the volume type. This can be gp2 for General Purpose (SSD) volumes, io1 for Provisioned IOPS (SSD) volumes, or standard for Magnetic volumes.
[IOPS]: This is only valid for Provisioned IOPS (SSD) volumes. This parameter specifies the number of IOPS to provision for the volume.
[Size]: This one gives the size of the volume, in GiBs.

Use the following command to create a 90 GiB Provisioned IOPS (SSD) volume with 1000 Provisioned IOPS in availability zone ap-southeast-1b:

$ aws ec2 create-volume 
--availability-zone ap-southeast-1b 
--volume-type io1 
--iops 1000 
--size 90

Attaching the volume

Run the following command to attach an EBS volumes to an EC2 instance. You will need to provide the EC2 instance ID, EBS volume ID, and the device name.

$ aws ec2 attach-volume 
--volume-id [VolumeId]
--instance-id [InstanceId]
--device [Device]

The parameters used in this command are described as follows:

[VolumeId]: This provides the volume ID
[InstanceId]: This parameter gives an EC2 instance ID
[Device]: This one is used to mention the device name to expose to the instance (for example, /dev/sdh or xvdh)

Run the following command to attach the EBS volume to an EC2 instance as /dev/sdf:

$ aws ec2 attach-volume 
--volume-id vol-64e54f6a 
--instance-id i-2e7dace3 
--device /dev/sdf

Configuring security groups

Security groups are like firewalls for your EC2 instances. If you don't specify the security group while creating instance in EC2-VPC, then AWS automatically assigns the default security group of the EC2-VPC to the instance. We can configure the inbound and outbound rules for security groups. We can also change these inbound and outbound rules while the instance is running. These changes are automatically applied.

For every VPC, we get a default security group, which we can't delete. You can't use a security group that you created for EC2-VPC when you launch an instance in EC2-Classic. You also can't use security group that you created for EC2-Classic, when you launch an instance in EC2-VPC. After you launch an instance in EC2-Classic, you can't change its security group but you can add and delete rules, which are then applied, automatically. But after you launch an instance in EC2-VPC, you can change its security groups, and add and remove rules, which are then applied, automatically.

When you specify a security group as the source or destination for a rule, the rule affects all instances associated with the security group The security groups created for EC2-Classic can only have inbound rules, but security groups created for EC2-VPC can have both inbound and outbound rules.

The limit to create security groups for each region is 500. You can create up to 100 security groups per VPC. You can also assign an unlimited number of security groups to the instance launched in EC2-Classic, whereas only 5 security groups can be assigned to an instance launched in VPC. The number of rules that can be added to each security group on EC2-Classic is 100 and for VPC it is 50.

How to do it…

In this recipe, we first list the commands for creating a security group for EC2-Classic and EC2-VPC. Then, we see how to create inbound and outbound rules. Finally, we list the command for adding the security group to an instance.

Creating a security group for EC2-Classic

By running the following command, you can create the security group in EC2-Classic. You have to provide the security group name and security group description for the security group.

$ aws ec2 create-security-group 
--group-name [SecurityGroupName]
--description [Description]

The parameters used in this command are described as follows:

[SecurityGroupName]: This provides the security group name
[Description]: This gives the description of the security group

Next, run the following command to create a security group with the WebServerSecurityGroup name in EC2-Classic:

$ aws ec2 create-security-group 
--group-name WebServerSecurityGroup 
--description "Web Server Security Group"

Creating a security group for EC2-VPC

By running the following command, you can create a security group in EC2-VPC. You have to provide the security group name, security group description, and VPC ID for the security group:

$ aws ec2 create-security-group 
--group-name [SecurityGroupName]
--description [Description] 
--vpc-id [VPCId]

The parameters used in this command are described as follows:

[SecurityGroupName]: This parameter provides the security group name
[Description]: This one gives the description of the security group
[VPCId]: This option provides a VPC ID

The following command will create a security group named WebServerSecurityGroup in VPC (vpc-1f33c27a). You can get your VPC IDs by running the aws ec2 describe-vpcs command.

$ aws ec2 create-security-group 
--group-name WebServerSecurityGroup 
--description "Web Server Security Group" 
--vpc-id vpc-1f33c27a

Adding an inbound rule

Run the following command to add an inbound rule to your security group. You will need to provide the security group ID, protocol (TCP/UDP/ICMP), port, and the CIDR IP range.

$ aws ec2 authorize-security-group-ingress 
--group-id [SecurityGroupId] 
--protocol [Protocol]
--port [Port]
--cidr [CIDR]

The parameters used in this command are described as follows:

[SecurityGroupId]: This is used to provide the security group ID
[Protocol]: This one provides the IP protocol of this permission
[Port]: This is used to specify the range of ports to allow
[CIDR]: This one gives the CIDR IP range

Next, run the following command to create the inbound rule that allows SSH traffic from IP address 123.252.223.114 in the security group (sg-c6b873a3):

$ aws ec2 authorize-security-group-ingress 
--group-id sg-c6b873a3 
--protocol tcp 
--port 22 
--cidr 123.252.223.114/32

Adding an outbound rule

Run the following command to add an outbound rule to your security group. You will need to specify the security group ID, protocol (TCP/UDP/ICMP), port, and the CIDR IP range.

$ aws ec2 authorize-security-group-egress 
--group-id [SecurityGroupId] 
--protocol [Protocol] 
--port [Port]
--cidr [CIDR]

The parameters used in this command are described as follows:

[SecurityGroupId]: This parameter provides the security group ID
[Protocol]: This option specifies the IP protocol of this permission
[Port]: This is used to give the range of ports to allow
[CIDR]: This one gives the CIDR IP range

Then, run the following command to create the outbound rule that allows MySQL traffic from your instance to IP address 123.252.223.114 in the security group (sg-c6b873a3):

$ aws ec2 authorize-security-group-egress 
--group-id sg-c6b873a3 
--protocol tcp 
--port 3866 
--cidr 123.252.223.114/24

Adding the security group to an instance

By running the following command, you can attach the security group to your EC2 instance. You have to provide the EC2 instance ID, and one or more security group IDs:

$ aws ec2 modify-instance-attribute 
--instance-id [InstanceId] 
--groups [SecurityGroupIds]

The parameters used in this command are described here:

[InstanceId]: This option gives an EC2 instance ID
[SecurityGroupIds]: This option provides the IDs of one or more security groups

Then, run the following command to add the security groups sg-c6b873a3 and sg-ccb873a9 to EC2 instance i-2e7dace3:

$ aws ec2 modify-instance-attribute 
--instance-id i-2e7dace3 
--groups sg-c6b873a3 sg-ccb873a9

Creating an EC2 key pair

AWS can authenticate using the public-private key mechanism. The recommended authentication mechanism is public-private key authentication instead of passwords to remotely log in to your instances with SSH. We upload the public key to AWS, and store the private key on our local machine. If anyone has your private key, then they can easily log in to your EC2 instances. It's a best practice to store these private keys in a secure place. We can create the public and private key from our machine using tools like PuTTY Key Generator.

You should include a passphrase with the private key to prevent unauthorized persons from logging in to your EC2 instance. When you include a passphrase, you have to enter the passphrase whenever you log in to the EC2 instance. A passphrase on a private key is an extra layer of protection. If you lost your private key for an EBS-backed instance, you can regain access to your instance by executing the following steps:

Stop the EBS-backed EC2 instance.
Detach the root volume from EC2 instance.
Launch the new EC2 instance for recovery.
Attach the EC2 root volume as data volume to the previously created instance.
Modify the authorized_keys file.
Detach the root volume from recovery instance.
Attach the root volume back to the EC2 instance.
Start the instance.

How to do it…

Here, we list the commands to create a key pair and then launching the EC2 instance (using the key pair).

Creating a key pair

Use the following steps to create a key pair:

Run the following command to create the key pair.
You have to provide the key pair name. You can explicitly specify the text output for this command using the –output argument for easy cut and paste.
```
$ aws ec2 create-key-pair 
--key-name [KeyPairName]
```
Note
The [KeyPairName] parameter in this command is used to specify a name for the key pair.
After executing the create-key-pair command, copy the entire output key into file including the following lines:
```
----BEGIN RSA PRIVATE KEY----
-----END RSA PRIVATE KEY-----
```
Save the file with ASCII encoding.
Run the following command to create the key pair with name WebServerKeyPair.
```
$ aws ec2 create-key-pair 
--key-name WebServerKeyPair
```

Grouping EC2 instances using placement groups

EC2 instances can be grouped using placement groups. For example, instances requiring low latency and high bandwidth communication can be placed in the same placement group. When instances are placed in this placement group, they have access to low latency, non-blocking 10 Gbps networking when communicating with other instances in the placement group (within a single availability zone). AWS recommends launching all the instances within the cluster placement group at the same time.

How to do it…

In order to group EC2 instances using placement groups, first we create a placement group, and then add our EC2 instances in it.

Creating a placement group

Run the following command to create placement groups. You have to provide the placement group name and the placement strategy.

$ aws ec2 create-placement-group 
--group-name [GroupName] 
--strategy [Strategy]

Here, the GroupName parameter specifies a name for the placement group and the Strategy parameter specifies the placement strategy.

Next, run the following command to create a placement group with the name WebServerGroup:

$ aws ec2 create-placement-group 
--group-name WebServerGroup 
--strategy cluster

Placing instances in the placement group

Run the following command to launch instances in a placement group. You will need to specify the placement group name along with the EC2 instance properties.

$ aws ec2 run-instances 
--image-id [ImageId] 
--count [Count]
--instance-type [InstanceType]
--key-name [KeyPairName]
--security-group-ids [SecurityGroupIds]
--subnet-id [SubnetId]
--placement [Placement]

The parameters used in this command are described as follows:

[ImageId]: This gives the ID of the image from which you want to create the EC2 instance
[Count]: This one provides the number of instances to create
[InstanceType]: This option gives the type of EC2 instance
[KeyPairName]: This parameter provides the key pair name for the authentication
[SecurityGroupIds]: This parameter gives one or more security group IDs
[SubnetId]: This option provides the ID of the subnet where you want to launch your instance

[Placement]: This gives the placement for the instance.

Syntax:

--placement AvailabilityZone=value,GroupName=value,Tenancy=value

Next, execute the following command to launch a c3.large EC2 instance in the WebServerGroup placement group:

$ aws ec2 run-instances 
--image-id ami-7e2c612c 
--count 1 
--instance-type c3.large 
--key-name WebServerKeyPair
--security-group-ids sg-ad70b8c8 
--subnet-id subnet-aed11acb 
--placement GroupName= WebServerGroup

Configuring Elastic Load Balancing

The Elastic Load Balancer (ELB) works within a single AWS region. You can scale both horizontally (adding more EC2 instances) and vertically (increasing EC2 instance size) within AWS, but it's best practice to scale horizontally. It can, however, load balance across several instances in multiple availability zones. If you don't want to load balance instances across multiple availability zones, then you can also disable it. If we want to load balance the instances across multiple regions, then we have to use Route 53 (instead of an ELB). ELB continuously checks the health of the instances, and only routes traffic to healthy instances. The health check frequency and the URL parameters are configurable.

If a healthy instance comes online, then the ELB recognizes the instance and routes traffic to it. ELB can be used to implement high-availability application architectures. If we use Route 53 with ELB, we can enable failover to a different region. ELB can also be configured with autoscaling, thereby enabling load balancing across new instances created by auto-scaling groups.

ELB can work with instances in EC2-Classic and VPC. There are two types of load balancers we can create internal or internet facing. We can't create internal load balancer without VPC. We can create both internal and internet facing load balancers within VPC. You can also enable sticky sessions on ELB using either application generated cookies or ELB generated cookies. In addition, you can assign security groups to ELBs. If you don't assign any security group while creating the ELB in VPC, it uses the default security group of the VPC. SSL termination is also supported in ELB, using this obviates the need to install SSL certificate on each and every EC2 instance.

How to do it…

Here, we list the commands for creating an ELB, configuring the same for performing health checks, and finally associating specific EC2 instances with it.

Creating an Internet-facing ELB with listeners

Run the following command to create an Internet-facing ELB. You will have to provide the listeners, subnet IDs, and security group IDs.

$ aws elb create-load-balancer 
--load-balancer-name [LoanBalancerName] 
--listeners [Listeners]
--subnets [SubnetIds] 
--security-groups [SecurityGroups]

The parameters used in this command are described as follows:

[LoanBalancerName]: This option provides the name of the load balancer.
[Listeners]: This parameter gives a list of the following tuples: Protocol, LoadBalancerPort, InstanceProtocol, InstancePort, and SSLCertificateId.
[SubnetIds]: This option gives a list of subnet IDs in your VPC to attach to your load balancer. You can get a list of subnet IDs by running the aws ec2 describe-subnets command.
[SecurityGroups]: This option provides the security groups to assign to your load balancer within your VPC. You can get security group ID by running the aws ec2 describe-security-groups command. You should provide the security group name in the preceding command.

Run the following command to create an ELB that receives traffic on port 80, and the load balances across instances listening on port 8080:

$ aws elb create-load-balancer
--load-balancer-name WebLoadBalancer
--listeners Protocol=HTTP,LoadBalancerPort=80,InstanceProtocol=HTTP,InstancePort=8080 
--subnets subnet-aed11acb 
--security-groups sg-c6b873a3

Configuring health checks on ELB

Run the following command to add health check configuration to an ELB. You have to provide the load balancer name and health check configuration:

$ aws elb configure-health-check 
--load-balancer-name [LoanBalancerName] 
--health-check [HealthCheckup]

The parameters used in this command are described as follows:

[LoanBalancerName]: This option provides the name of the load balancer

[HealthCheckup]: This parameter provides the health check configuration

Syntax:

Target=HTTP:8080/index.html,Interval=30,UnhealthyThreshold= 2,HealthyThreshold=2,Timeout=3

The following command will add the health check configuration to an ELB. The ELB checks the instance health at <URL>:8080/index.html. ELB health check interval is set to 30 seconds. UnhealthyThreshold specifies the number of consecutive unsuccessful URL probes before the ELB changes the instance health status to unhealthy. HealthyThreshold specifies the number of consecutive successful URL probes before ELB changes the instance health status to healthy.

$ aws elb configure-health-check 
--load-balancer-name WebLoadBalancer
--health-check Target=HTTP:8080/index.html,Interval=30,UnhealthyThreshold=2,HealthyThreshold=2,Timeout=3

Adding instances to the ELB

By running the following command, you can add instances to the ELB. You have to provide the ELB name and the list of instance IDs.

$ aws elb register-instances-with-load-balancer
--load-balancer-name [LoanBalancerName] 
--instances [Instances]

The parameters used in this command are described as follows:

[LoanBalancerName]: This option gives the name of the load balancer
[Instances]: This option gives a list of instances for the load balancer

The following command will add ELB to EC2 instances with IDs i-d3ff2c1e and i-2e7dace3.

$ aws elb register-instances-with-load-balancer 
--load-balancer-name WebLoadBalancer 
--instances i-d3ff2c1e i-2e7dace3

Architecting for high availability

Application and network errors can render the system unavailable to the user. Multi-availability zone deployments are used for building high-availability applications at the AWS region level. For implementing fault tolerance for region level failures, we have to deploy our application in availability zones spanning across different regions. If we use multiple regions, we have to use Route 53 for failover. If the primary region goes down, Route 53 fails over to the secondary region.

Increasing load on system can also cause system availability issues, but the autoscaling feature can help us solve the problem by autoscaling the number of servers during a spike in load. The number of servers is automatically reduced when the load comes back to normal levels. Detailed explanation on autoscaling is in Chapter 3, Managing AWS Resources Using AWS CloudFormation.

Building loosely coupled applications can also help avoid single points of failure. We can use Simple Queue Service (SQS) to build loosely coupled applications. Using the SQS queue size as a parameter, we can auto-scale our EC2 instances. For RDS high availability, we can configure a multi availability zone-deployment option. This will deploy the primary and secondary database instances in two different availability zones.

How to do it…

Here, we list the commands required for configuring high availability across two different regions using Route 53:

Create an instance in the first region. Before launching the EC2 instance, create the required VPC, subnets, key pairs, and security groups in this region.
```
$ aws ec2 run-instances 
--image-id [ImageId] 
--count [InstanceCount] 
--instance-type [InstanceType] 
--key-name [KeyPairName] 
--security-group-ids [SecurityGroupIds]
--subnet-id [SubnetId]
```
The parameters used in this command are described as follows:
- [ImageId]: This option gives the ID of the image
- [InstanceCount]: This parameter provides the number of instances to create
- [InstanceType]: This parameter provides the type of EC2 instance
- [KeyPairName]: This gives a key/pair name for authentication
- [SecurityGroupIds]: This option provides the security group ID
- [SubnetId]: This parameter provides the ID of subnet where you want to launch your instance
Create an instance in the second region. Before launching the EC2 instance, create the required VPC, subnets, key pairs, and security groups in this region:
```
$ aws ec2 run-instances 
--image-id [ImageId] 
--count [InstanceCount] 
--instance-type [InstanceType] 
--key-name [KeyPairName] 
--security-group-ids [SecurityGroupIds]
--subnet-id [SubnetId] 
```
The parameters used in this command are described as follows:
- [ImageId]: This parameter provides the ID of the image
- [InstanceCount]: This option gives the number of instances to create
- [InstanceType]: This one gives the type of EC2 instance
- [KeyPairName]: This parameter provides a key/pair name for authentication
- [SecurityGroupIds]: This option gives a security group ID
- [SubnetId]: This parameter provides the ID of the subnet where you want to launch your instance
Create an AWS hosted zone in Route 53 service.
The following command will return the name server records. Record the name server records and the hosted zone ID for the further usage.
```
$ aws route53 create-hosted-zone 
--name [Name] 
--caller-reference [CallReference]
```
The parameters used in this command are described as follows:
- [Name]: This parameter gives the name of the domain
- [CallReference]: This parameter gives a unique string that identifies the request and that allows failed create-hosted-zone requests to be retried without the risk of executing the operation twice
Change the name servers records with your domain registrar.
Note
Use the following link to understand how to change name servers with GoDaddy:
https://support.godaddy.com/help/article/664/setting-nameservers-for-your-domain-names
Create health checks for previously created instances in the first region by performing the following steps:
1. First create a virginiahc.json file with the following JSON. The IP address used is the public IP address of EC2 instance.
```
{
"IPAddress":"54.173.200.169",
"Port":8080,
"Type":"HTTP",
"ResourcePath":"/index.html",
"RequestInterval":30,
"FailureThreshold":3
}
```
2. Execute the following command for the first region:
```
$ aws route53 create-health-check 
--caller-reference [CallReference] 
--health-check-config [HealthCheckConfig]
```
  The parameters used in this command are described as follows:
  - [CallReference]: This is a unique string that identifies the request and that allows failed create-health-check requests to be retried without the risk of executing the operation twice
  - [HealthCheckConfig]: This option gives the health check configuration
    Syntax:
    file://virginiahc.json
3. Create health check by running the following command. Record the health check ID for further usage.
```
$ aws route53 create-health-check 
--caller-reference 2014-11-29-17:03 
--health-check-config file://virginiahc.json
```
Create health checks for previously created instances in second region by performing the following steps:
1. Create a second singaporehc.json file with the following JSON. The IP address used is the public IP address of EC2 instance.
```
{
"IPAddress":"54.169.85.163", 
"Port":8080, 
"Type":"HTTP", 
"ResourcePath":"/index.html", 
"RequestInterval":30, 
"FailureThreshold":3 
}
```
2. Execute the following command for the second region:
```
$ aws route53 create-health-check 
--caller-reference [CallReference] 
--health-check-config [HealthCheckConfig]
```
  The parameters used in this command are described as follows:
  - [CallReference]: A unique string that identifies the request and that allows failed create-health-check requests to be retried without the risk of executing the operation twice
  - [HealthCheckConfig]: This option provides the health check configuration
    Syntax:
    file:// singaporehc.json
3. Create health check by running the following command. Record the health check ID for further usage.
```
$ aws route53 create-health-check 
--caller-reference 2014-11-29-17:04 
--health-check-config file://singaporehc.json
```

Add a primary and secondary record set to the Route 53-hosted zone by performing the following steps:

Create a recordset.json file with the following JSON. In primary record set, replace health check ID and IP address with first region health check ID and EC2 public IP address accordingly. In secondary record set, replace health check ID and IP address with second region health check ID and EC2 public IP address accordingly.

{
    "Comment":"Creating Record Set",
    "Changes":[ 
        {
            "Action":"CREATE", 
            "ResourceRecordSet":{ 
                "Name":"DNS Domain Name",
                "Type":"A", 
                "SetIdentifier":"PrimaryRecordSet", 
                "Failover":"PRIMARY",  
                "TTL":300, 
                "ResourceRecords":[
                    {
                        "Value":"54.173.200.169"
                    }
                ],
                "HealthCheckId":"<your first region's 
                 health check id>"
            }
        },
        {
            "Action":"CREATE", 
            "ResourceRecordSet":{ 
                "Name":" DNS Domain Name", 
                "Type":"A", 
                "SetIdentifier":"SecondaryRecordSet",  
                "Failover":"SECONDARY", 
                "TTL":300, 
                "ResourceRecords":[
                    {
                        "Value":"54.169.85.163"
                    }
                ],
                "HealthCheckId":"<your second region's 
                 health check id>"
            }
        }
    ]
}

Execute the following command to add record set:
```
$ aws route53 change-resource-record-sets 
--hosted-zone-id [HostedZoneId] 
--change-batch [ChangeBatch] 
```
The parameters used in this command are described as follows:
- [HostedZoneId]: This option provides the Route 53-hosted zone ID
- [ChangeBatch]: A complex type that contains an optional comment and the changes element
  Syntax:
```
file://recordset.json
```

Add the record set to the hosted zone by running the following command:

$ aws route53 change-resource-record-sets 
--hosted-zone-id Z3DYG8V5Z07JP8 
--change-batch file://recordset.json

Test the failover configuration by stopping the server in the primary region. You can stop your first region EC2 instance by running the aws ec2 stop-instances command.