Introducing Amazon Web Services

Now, that you clearly understand what cloud computing is all about and what it can do for you, let's get to know the main topic of this book—Amazon Web Services—a little better.

Amazon Web Services or AWS is a comprehensive public cloud computing platform that offers a variety of web-based products and services on an on-demand and pay-per-use basis. AWS was earlier a part of the e-commerce giant Amazon.com, and it wasn't until 2006 that AWS became a separate entity of its own. Today, AWS operates globally with data centers located in USA, Europe, Brazil, Singapore, Japan, China, and Australia. AWS provides a variety of mechanisms, using which the end users can connect to and leverage its services, the most common form of interaction being the web-based dashboard also called as AWS Management Console.

So how does the whole thing work? Well, it is very easy to understand when you compare the way AWS works with a power and utilities company. AWS offers its customers certain services just as a power company would to its consumers. You as an end user simply consume the electricity without having to worry about the underlying necessities such as generator costs, cabling, and so on. At the end of the month, all you get is a bill based on your electricity consumption and that's it! In a similar way, AWS provides its products such as compute, storage, and networking all as a service, and you only have to pay for the amount of service that you use. No upfront costs or heavy investments whatsoever!

The other important thing worth mentioning here is that AWS allows organizations to use their own operating systems, databases, and programming/architecting models as well, without requiring any major re-engineering. This provides a lot of flexibility and cost optimization to organizations as they get to operate and work with platforms that they are familiar with. This, accompanied with AWS's massively scalable and highly available infrastructure, ensures that your applications and data remain secure and available for use no matter what.

AWS architecture and components

Before we begin with the actual signup process, it is important to take a look at some of the key architecture and core components of services offered by AWS.

Regions and availability zones

We do know that AWS is spread out globally and has its presence across USA, Europe, Asia, Australia, and so on. Each of these areas is termed as a region. AWS currently has about 10 regions, each containing multiple data centers within themselves. So what's with all these regions and why do they matter? In simple terms, the resources that are geographically close to your organization are served much faster! For example, an organization running predominantly from USA can leverage the USA's regions to host their resources and gain access to them must faster.

For most of the AWS services that you use, you will be prompted to select a region in which you want to deploy the service. Each region is completely isolated from the other and runs independently as well.

Note

AWS does not replicate resources across regions automatically. It is up to the end user to set up the replication process.

A list of regions and their corresponding codes is provided here for your reference. The code is basically how AWS refers to its multiple regions:

Region	Name	Code
North America	US East (N. Virginia)	us-east-1
	US West (N. California)	us-west-1
	US West (Oregon)	us-west-2
South America	Sao Paulo	sa-east-1
Europe	EU (Frankfurt)	eu-central-1
	EU (Ireland)	eu-west-1
Asia	Asia Pacific (Tokyo)	ap-northeast-1
	Asia Pacific (Singapore)	ap-southeast-1
	Asia Pacific (Sydney)	ap-southeast-2
	Asia Pacific (Beijing)	cn-north-1

Each region is split up into one or more Availability Zones (AZs) and pronounced as A-Zees. An A Z is an isolated location inside a region. AZs within a particular region connect to other AZs via low-latency links. What do these AZs contain? Well, ideally they are made up of one or more physical data centers that host AWS services on them. Just as with regions, even AZs have corresponding codes to identify them, generally they are regional names followed by a numerical value. For example, if you select and use us-east-1, which is the North Virginia region, then it would have AZs listed as us-east-1b, us-east-1c, us-east-1d, and so on:

AZs are very important from a design and deployment point of view. Being data centers, they are more than capable of failure and downtime, so it is always good practice to distribute your resources across multiple AZs and design your applications such that they can remain available even if one AZ goes completely offline.

An important point to note here is that AWS will always provide the services and products to you as a customer; however, it is your duty to design and distribute your applications so that they do not suffer any potential outages or failures.

RULE OF THUMB: Design for failure and nothing will fail! This is what we will be sticking with for the remainder of this book as we go along the different AWS services and products; so keep this in mind, always!

Note

AWS provides a health dashboard of all its services running across each of the regions. You can view the current status and availability of each AWS service by visiting the following link: http://status.aws.amazon.com/.

AWS platform overview

The AWS platform consists of a variety of services that you can use either in isolation or in combination based on your organization's needs. This section will introduce you to some of the most commonly used services as well as some newly launched ones. To begin with, let's divide the services into three major classes:

• Foundation services: This is generally the pillars on which the entire AWS infrastructure commonly runs on, including the compute, storage, network, and databases.
• Application services: This class of services is usually more specific and generally used in conjunction with the foundation services to add functionality to your applications. For example, services such as distributed computing, messaging and Media Transcoding, and other services fall under this class.
• Administration services: This class deals with all aspects of your AWS environment, primarily with identity and access management tools, monitoring your AWS services and resources, application deployments, and automation.

Let's take a quick look at some of the key services provided by AWS. However, do note that this is not an exhaustive list:

We will discuss each of the foundation services.

Compute

This includes the following services:

• Elastic Compute Cloud (EC2): When it comes to brute computation power and scalability, there must be very few cloud providers out there in the market that can match AWS's EC2 service. EC2 or Elastic Compute Cloud is a web service that provides flexible, resizable, and secure compute capacity on an on-demand basis. AWS started off with EC2 as one of its core services way back in 2006 and has not stopped bringing changes and expanding the platform ever since. The compute infrastructure runs on a virtualized platform that predominantly consists of the open sourced Xen virtualization engine. We will be exploring EC2 and its subsequent services in detail in the coming chapters.
• EC2 Container Service: A recently launched service, the EC2 Container Service, allows you to easily run and manage docker containers across a cluster of specially created EC2 instances.
• Amazon Virtual Private Cloud (VPC): VPC enables you to create secure, fully customizable, and isolated private clouds within AWS's premises. They provide additional security and control than your standard EC2 along with connectivity options to on premise data centers.

Storage

This includes the following services:

• Simple Storage Service (S3): S3 is a highly reliable, fault tolerant, and fully redundant data storage infrastructure provided by AWS. It was one of the first services offered by AWS way back in 2006, and it has not stopped growing since. As of April 2013, an approximate 2 trillion objects have been uploaded to S3, and these numbers are growing exponentially each year.
• Elastic Block Storage (EBS): EBS is a raw block device that can be attached to your compute EC2 instances to provide them with persistent storage capabilities.
• Amazon Glacier: It is a similar service offering to S3. Amazon Glacier offers long-term data storage, archival, and backup services to its customers.
• Amazon Elastic File System: Yet another very recent service offering introduced by AWS, Elastic File System (EFS) provides scalable and high-performance storage to EC2 compute instances in the form of an NFS filesystem.

Databases

This includes the following services:

• Amazon Relational Database Service (RDS): RDS provides a scalable, high-performance relational database system such as MySQL, SQL Server, PostgreSQL, and Oracle in the cloud. RDS is a completely managed solution provided by AWS where all the database heavy lifting work is taken care of by AWS.
• Amazon DynamoDB: DynamoDB is a highly scalable NoSQL database as a service offering provided by AWS.
• Amazon Redshift: Amazon Redshift is a data warehouse service that is designed to handle and scale to petabytes of data. It is primarily used by organizations to perform real-time analytics and data mining.

Networking

This includes the following services:

• Elastic Load Balancer (ELB): ELB is a dynamic load balancing service provided by AWS used to distribute traffic among EC2 instances. You will be learning about ELB a bit more in detail in subsequent chapters.
• Amazon Route 53: Route 53 is a highly scalable and available DNS web service provided by AWS. Rather than configuring DNS names and settings for your domain provider, you can leverage Route 53 to do the heavy lifting work for you.

These are just few of the most commonly used AWS foundational services that we listed out here. There are a lot more services and products that you can leverage to add functionality or use to manage your applications. A few of these important services are briefly described in the next section.

Distributed computing and analytics

This includes the following services:

• Amazon Elastic MapReduce (EMR): As the name suggests, this service provides users with a highly scalable and easy way to distribute and process large amounts of data using Apache's Hadoop. You can integrate the functionalities of EMR with Amazon S3 to store your large data or with Amazon DynamoDB as well.
• Amazon Redshift: This is a massive data warehouse that users can use to store, analyze, and query petabytes of data.

Content distribution and delivery

Amazon CloudFront is basically a content delivery web service that can be used to distribute various types of content, such as media, files, and so on, with high data transfer speeds to end users globally. You can use CloudFront in conjunction with other AWS services such as EC2 and ELB as well.

Workflow and messaging

This includes the following services:

• Amazon Simple Notification Service (SNS): SNS is a simple, fully managed push messaging service provided by AWS. You can use it to push your messages to mobile devices (SMS service) and even to other AWS services as API calls to trigger or notify certain activities.
• Amazon Simple Email Service (SES): As the name suggests, SES is used to send bulk e-mails to various recipients. These e-mails can be anything, from simple notifications to transactions messages, and so on. Think of it as a really large mail server that can scale as per your requirements and is completely managed by AWS! Awesome, isn't it!

Monitoring

Amazon CloudWatch is a monitoring tool provided by AWS that you can use to monitor any and all aspects of your AWS environment, from EC2 instances to your RDS services to the load on your ELBs, and so on. You can even create your own metrics, set thresholds, create alarms, and a whole lot of other activities as well.

Identity and access management

AWS provides a rich set of tools and services to secure and control your infrastructure on the cloud. The most important and commonly used service for this is identity and access management (IAM). Using IAM, you can, as an organizational administrator, create and manage users, assign them specific roles and permissions, and manage active directory federations as well. We will be using a lot of IAM in the next chapter, which covers this topic in greater depth.

Getting started with AWS

So far, you have learned a lot about AWS, its architecture, and core components. Now, let's get started with the fun part—the signup process.

For all first time users, signing up for AWS is a very simple and straightforward process. We will go through this shortly, but first let's take a quick look at something called as a Free Tier! Yes, you heard it right… FREE!

So, AWS basically offers usage of certain of its products at no charge for a period of 12 months from the date of the actual signup. A brief list of a few products along with their description is listed here for your reference. Note that some of the description text may not make much sense now, but that's ok as this is just for your reference, and we will be bringing this up from time to time as we progress through the book.

AWS Product	What's free?
Amazon EC2	750 hours per month of Linux micro instance usage 750 hours per month of Windows micro instance usage
Amazon S3	5 GB of standard storage 20,000 get requests 2,000 put requests
Amazon RDS	750 Hours of Amazon RDS Single-AZ micro instance usage 20 GB of DB Storage: any combination of general purpose (SSD) or magnetic 20 GB for backups 10,000,000 I/Os
Amazon ELB	750 hours per month 15 GB of data processing

Note

For a complete insight into the free tier usage, check http://aws.amazon.com/free/.

Awesome! So when we have free stuff for us right from the word go, why wait? Let's sign up for AWS. To begin with, launch your favorite web browser and type in the following URL in the address bar: http://aws.amazon.com/.

You should see the AWS landing page similar to one shown here. Here, select either the Create an AWS Account option or the Create a Free Account option to get started:

The next screen will help you with the initial signup process. Provide a suitable e-mail address or your contact number in the E-mail or mobile number field. Select the I am a new user option and select the Sign in using our secure server button to proceed:

Note

You can alternatively sign in using your Amazon.com credentials as well; however, its best to use separate credentials for working with AWS.

The next couple of screens will be used to provide your basic details along with the billing information. In the Login Credentials page, enter your Name, your E-mail address along with a suitable Password. This password will be used by you to login to the AWS Management Console, so ideally provide a strong password here. Click on Create account when done.

The next screen is the Contact Information page. Provide your Full Name, Company Name, Country, Address, City, Postal Code, and Phone Number as requested. Check the Amazon Internet Service Pvt. Ltd. Customer Agreement checkbox and select the Create Account and continue options.

Enter a suitable Cardholder's Name and your Credit/Debit Card Number in the Payment Information page as shown:

The last part of the signup process is the Identity Verification process where you will receive an automated call from AWS as a part of the verification process. You will have to enter the displayed four digit PIN code on your telephone's keypad during the call. Once the verification is completed, you can click on the Continue to select your Support Plan tab:

The final step in the signup process involves the selection of the Support Plan. AWS provides four support options to customers, each having their own SLAs and costs associated with it. Here is a quick look at the support plans provided by AWS:

• Basic Support: As the name suggests, this is the most basic level of support provided by AWS. This support level provides you with access to the AWS community forums. You can additionally contact customer services for any queries related to your account and bill generation.

  Note

  The Basic Support plan is free of charge and all customers are entitled to it.

• Developer Support: This is a paid support service ($49 per month). You can create and raise tickets for your support case, which is generally answered within 12 working hours.
• Business Support: This is a paid support service as well and is generally meant for enterprise-level customers running production workloads on AWS. The SLAs for this support are much higher as a case has to be answered within an hour from its creation. Additional support is provided 24/7, 365 days a year via phone and chat.
• Enterprise Support: A paid support service with the highest SLA available (15 minutes); these cases are generally handled by a separate team at AWS called the Technical Account Manager (TAM) who are subject matter experts in their own fields.

In our case, we opted to go with Basic Support for the time being. You can change the support levels later on as well according to your needs. Click on Continue to complete the signup process. You should receive a couple of e-mails on your supplied e-mail address as well. These are introductory e-mails that will provide you with important links such as how to get started with AWS, billing page, account information, and so on.

With these steps completed, you are now ready to sign in to the AWS Management Console!

Introducing the AWS Management Console

So here we are, all ready to get started with the AWS Management Console! This is the most commonly used method to access, manage, and work with AWS services. We shall be looking more closely at the different AWS access mechanisms in the next chapter; however for now, let's quickly look at what the AWS Management Console is all about.

First off, sign in to the Management Console by launching your favorite browser and typing in http://aws.amazon.com. Click on the Sign in to the Console option and provide your Email Address and password as set during our signup process. Once you sign in, you will be welcomed to the AWS Management Console main landing page as shown here. Wow! That's a lot of products and services, right? The products are classified into their main classes such as compute, storage and content delivery, administration and security, and so on so forth. Take a moment and just browse through the dashboard. Get a good feel for it.

Navigating through the dashboard is also pretty straightforward. Let's look at the top navigation bar first. To the right-hand side you should be able to see three drop-down menus. The first should display your name as an end user. This option consists of three submenus that will help you with configuring your account details, security credentials, and billing management. The next tab lists the Region from where you will currently be operating. In our case, we have been placed in the US West (Oregon) region. Remember you can change these regions as and when you require, so feel free to change as per your current global presence:

Note

The US East (North Virginia) region is the cheapest region in AWS as it was one of the first regions to get set up and started.

The final tab in the list is the Support tab, and you can use it to login to the Support Center, AWS Forums, and view the latest set of AWS Documentation as well. Moving to the left-hand side of the tool bar, you will see four main icons listed there. Among these is a Home Screen icon, which when clicked on will bring you back to the AWS dashboard screen irrespective of where you currently are.

The next drop-down option in the list is named as AWS, but what it really contains is called as Resource Groups. These are a collection of AWS resources that can be organized and viewed as per your requirements. Think of these resource groups as a customized console where you as an end user can view all your required information about various AWS services in a single pane. How do resource groups work? Don't worry. We will be looking into this in more detail in the upcoming chapter.

Adjoining the Resource Groups is the Services tab, which lists the AWS services according to their class. It also has a history option that can be used to list and view your recently used AWS services.

The final tab is the Edit tab. This tab is used to customize your toolbar by filling it with those AWS services that you use frequently, kind of like a quick access bar. To add a particular AWS service to the toolbar, simply select the AWS service, drag and place it on the toolbar:

You can add multiple services as you see fit, and arrange them according to your needs as well. To save the changes to the toolbar, simply select the Edit option once again. You should see your AWS services listed out on the toolbar now. Feel free to dig around and check out the various options under each menu.

Getting started with AWS CLI

Now that you have a hang of the AWS Management Console, this would be a good time to take a quick look at the AWS CLI as well. Yes, you heard it right, apart from the standard web user interface, AWS provides a host of other mechanisms as well to help you gain access and use the various AWS resources. But why use a CLI in the first place? Isn't the AWS Management Console more than enough? Well, no. CLIs are more than just simple access and management tools. Using CLIs, you can automate the deployment and management of your AWS services using simple code and script, much like how you would use bash and shell scripting. This provides you with a lot of flexibility and customizability that a standard GUI simply won't provide!

The AWS CLI can be either installed on a Windows or a Linux machine. In case of Windows, AWS provides an easy-to-use installer, which can be downloaded directly from the AWS site. Once downloaded, all you need to do is run the installer, and voila, your Windows server should have the CLI installed and ready for use. But I'm not a Windows guy, so we will be walking you through the installation procedure on a standard Linux system.

Note

The 64-bit AWS CLI installer for Windows can be downloaded from https://s3.amazonaws.com/aws-cli/AWSCLI64.msi. The 32-bit installer can be downloaded from https://s3.amazonaws.com/aws-cli/AWSCLI32.msi.

In this case, we will be installing the AWS CLI on a CentOS 6.5 64-bit OS. The Linux distribution can be anything, from a Debian such as Ubuntu to a RedHat system; so long as it has Python installed and running off the latest version.

Note

Python versions supported are Python 2 version 2.6.5 and above or Python 3 version 3.3 and above.

You will also need sudo or root privileges to install and execute the commands, so make sure you have an appropriate user already created on your Linux system.

The installation of the CLI involves two major steps; the first involves the installation of Python setuptools, which is a prerequisite of installing Python's pip.

Run the following commands from your Linux terminal:

1. Download the setuptools tar file from the Python source repo:

   wget https://pypi.python.org/packages/source/s/setuptools/setuptools-7.0.tar.gz

2. Next, untar the setuptools installer using the tar command:

   tar xvf setuptools-7.0.tar.gz

3. Once the contents of the tar file are extracted, change the directory to the setuptools directory:

   cd setuptools-7.0

4. Finally, run the setup.py script to install the setuptools package:

   python setup.py install

The following is the screenshot of preceding commands of the install process:

This completes the first part of the install process. The next process is very simple as well. We now install the Python pip package. Python pip is generally recommended when installing Python packages.

Run the following commands from your Linux terminal to install the Python pip package:

1. Download the Python pip installer script from Python's repo:

   wget https://bootstrap.pypa.io/get-pip.py

2. Install the pip package:

   python get-pip.py

3. Once pip is installed, you can now easily install the AWS CLI by executing the following command:

   pip install awscli

   Refer to the following screenshot showing the output of the installation process:

   
4. Simple, wasn't it! You can test your AWS CLI by executing few simple commands, for example, check the AWS CLI version using the following command:

   aws –version

That's just for starters! There is a whole lot more that you can achieve with the AWS CLI, and we will make sure to utilize it in each of our chapters, just to get a good feel for it.