Setting up the environment

The AWS CLI is a significant way to automate the AWS infrastructure. Its features are as follows:

• Single unified tool for managing all AWS resources
• Supports Linux, macOS, and Windows
• Supports 200+ top-level commands

For the AWS CLI to interact with Amazon's API, it uses an AWS access key and a secret access key. These keys are used to authenticate and authorize any request sent to AWS. The steps to create an IAM user and retrieve the keys are as follows:

1. In order to generate these credentials, go to the Identity and Access Management (IAM) console (https://aws.amazon.com/console/) and log in with your credentials, and search for IAM, as illustrated in the following screenshot:
   

   Figure 1.1 – AWS Management Console

2. Click on the Users tab: https://console.aws.amazon.com/iam/home?#/users.
3. Create a new user or use an existing user.
4. If you are creating a new user, click on Add user, which will take you to the following screen:
   

   Figure 1.2 – IAM Add user screen

   Important note

   Please make sure you click on Programmatic access (as this will enable/create an access key and a secret access key).

5. Click Next: Permissions, and in the next screen, assign the AdministratorAccess policy to the user and click Next: Tags, as illustrated in the following screenshot:
   

   Figure 1.3 – IAM Set permissions screen

   Important note

   As an AWS security best practice, never give admin access to any user. Please follow the principle of least privilege. In the next chapter, we will tighten security and only assign the necessary privileges to the user.

6. The tag field is optional. I am leaving it blank, but please feel free to add tags to the newly created user depending upon your requirements. The field is shown in the following screenshot:
   

   Figure 1.4 – IAM tags (optional field)

7. Review all the settings such as User name, AWS access type, and Permissions boundary, and click Create user, as illustrated in the following screenshot:
   

   Figure 1.5 – Review user creation

8. Please take a note of the Access key ID and Secret access key values, illustrated in the following screenshot:

Figure 1.6 – The newly created IAM user

Important note

This is your only chance to see/retrieve the secret access key. There is no way to retrieve this key in the future. Keep this file confidential and never share this key, and never ever accidentally commit these keys to the GitHub/public code repository.

Installing the AWS CLI

The AWS CLI package works on Python and supports the following Python versions:

• 2.7.x and greater
• 3.4.x and greater

The AWS CLI installation is pretty straightforward. Run the following command to download, unzip, and install the AWS CLI:

curl "https://awscli.amazonaws.com/awscli-exe-linux-x86_64.zip" -o "awscliv2.zip"
unzip awscliv2.zip
sudo ./aws/install -i /usr/local/aws-cli -b /usr/local/bin

Note

The AWS CLI v2 is still not available in the Python Package Index (PyPI) repository. Please check the bug at the following link for more info: https://github.com/aws/aws-cli/issues/4947.

Run the following command to verify the installation:

aws --version
aws-cli/2.0.24 Python/3.7.3 Linux/4.15.0-1065-aws botocore/2.0.0dev28

Note

Throughout this book, we're going to discuss and use the AWS CLI version 2, which comes with its own set of features (for example: auto-prompt; wizard; YAML Ain't Markup Language (YAML) support). Please make sure to update or uninstall the AWS CLI v1 before continuing. See the following page for more information: https://docs.aws.amazon.com/cli/latest/userguide/install-cliv2-linux.html#cliv2-linux-upgrade.

Configuring command-line completion

To enable command-line completion, run the following command from the shell (for example: bash) that we are using:

$ complete -C '/usr/local/bin/aws_completer' aws

This command connects aws_completer to the aws command. As we execute these commands in the current shell, these changes will be lost as soon as we log out of this shell. To make this change permanent, add the preceding entry in ~/.bashrc.

Once the command-line completion is done, we can type any partial command and press the Tab key on the keyboard to see all the available commands, as illustrated in the following code snippet:

aws s<TAB>
s3                      sagemaker-runtime       securityhub   ses                     snowball                sso-oidc     

We have configured the command-line completion, so let's go ahead and configure the AWS CLI.

Configuring the AWS command line

With command-line completion in place, our next step is to see how the AWS CLI will interact with the AWS API, and the fastest way to achieve this is via the aws configure command, as illustrated in the following code snippet:

aws configure
AWS Access Key ID [None]: XXXXXXXXXXXX
AWS Secret Access Key [None]: XXXXXXXXXXXX
Default region name [None]: us-west-2
Default output format [None]: json

As you can see, when we run this command, the AWS CLI asks for the following four sets of information:

• Access key ID/secret access key ID: Think of the access key and the secret key as a username/password. To access the AWS console, you need your username and password, but to access the AWS API, you need your access/secret keys. We already created an access key and a secret access key earlier in this chapter.
• AWS region: The location where we set up the AWS infrastructure (for example, us-west-2 if we set up our infrastructure in Oregon).
• Output format: Specifies how the result is formatted (supported formats: JavaScript Object Notation (JSON) (default), YAML, text, and table).

  Note

  Please make sure that the computer date and time is set correctly, because if it is not in sync or is way off, AWS will reject the request.

These credentials (access/secret key, region, and output) are stored in ~/.aws/credentials, and the default region and output format are stored in ~/.aws/config, as illustrated in the following code snippet:

cat ~/.aws/credentials 
[default]
aws_access_key_id = XXXXXXXX
aws_secret_access_key = XXXXXXXXXXXXX
cat ~/.aws/config 
[default]
region = us-west-2
output = json

The AWS CLI stores this information (access/secret key, region, and output) in a default profile and the configuration file. In the next section, let's explore more about the location of the configuration file.

Understanding the AWS CLI command structure

The AWS CLI command is split into four parts and we need to specify these parts in order, as illustrated in the following code snippet:

aws <command> <subcommand> [options and parameters]

As you can see in the preceding command, the following apply:

• Everything starts with the aws program.
• The top-level command is the service supported by the AWS CLI (for example: s3 in the following example).
• The sub command specifies the operation to perform (ls in the following example).
• Options or parameters required by the operation are provided (s3://example-bucket).

Examples of the preceding syntax commands are shown here:

$ aws s3 ls
2020-04-26 15:59:11 my-test-s3-bucket-XXXXXXX
$ aws s3 ls s3://example-bucket
2020-06-07 18:28:47        166 testfile

Other commands that can be used to verify the AWS CLI are listed here:

• aws ec2 describe-instances: This command describes the specified instances or all instances.
• aws s3 mb s3://mytestbucket1235334: This is used to create a Simple Storage Service (S3) bucket.
• aws iam list-users: This is used to list the IAM users.

We now have the AWS CLI configured and ready to use. In the next section, we will see how to install and configure Boto3.