Using the command-line interface (CLI)

The AWS command-line interface (CLI) tool is an important piece of the AWS administrator's toolkit.

The CLI tool is often one of the quickest and easiest ways to interact with the API. As a text-based tool, it scales much easier than using the web console. Unlike the console, it can be automated, for example, via scripts. The AWS application programming interface (API) represents all the functionality that's available to you as an AWS administrator. It is also easier to keep track of through your command-line history. Like all good CLI tools, simple individual commands can be chained (or piped) together to perform complex tasks.

The CLI tool is open source software, and is maintained on GitHub (https://github.com/aws/aws-cli). For more detailed documentation, refer to the AWS CLI home page at https://aws.amazon.com/cli.

Installation

The CLI tool requires Python 2.6.5 or greater.

The easiest way to install it is to use the Python package manager, pip:

pip install awscli

This will make the aws command available on your system.

Upgrade

AWS frequently releases new services and functionality. To use these new features, you will need to upgrade the CLI tool.

To upgrade the CLI tool, run the following pip command periodically:

pip install --upgrade awscli

Configuration

Authentication between the CLI tool and the AWS API is done via two pieces of information:

• Access key ID
• Secret access key

As its name suggests, you should keep your secret access key a secret! Be careful where you store or send it.

Once you have created a user, you can configure the tool so that you can use it for authentication purposes.

While you can configure the CLI tool with access keys directly, this should be avoided. Instead, you should use profiles to store your credentials. Using profiles gives you a more consistent and manageable centralized location to secure your secret keys.

Default profile

Without any additional configuration or options, your CLI tool commands will use the default profile.

To set up the default profile, you can use the following command:

aws configure

This will prompt you for an access key ID, secret access key, region, and output format.

Named profiles

In addition to the default profile, you can configure other, named profiles. This is useful for switching between users with different levels of access (for example, read-only and administrator) or even between users in different accounts:

aws configure --profile <profile-name>

Once you have responded to these prompts, you can reference the named profile by passing the --profile <profile-name> option with your command.

Environment variables

You can also configure the CLI via the use of environment variables:

export AWS_PROFILE=<profile-name>

While you should prefer to use profiles over setting your access ID and secret keys directly, sometimes you may have to do so. If you must set your keys directly, do so via environment variables so that you don't need to pass your keys around or hardcode them:

export AWS_ACCESS_KEY_ID=<access-key-id>
export AWS_SECRET_ACCESS_KEY=<secret-access-key>

Instance roles

When running the CLI tool on an EC2 instance, you can leverage the instance's IAM role to make calls. This means that you don't need to configure credentials or set environment variables (manually).

Behind the scenes, the instance will retrieve and set its own AWS environment variables that allow API calls. You need to ensure that the instance has the appropriate permissions.

The AWS CLI tool comes preinstalled on AWS Linux-based instances.

Usage

All CLI tool commands are service-based. By using service commands and subcommands, you can make calls directly to the AWS API.

Commands

Each command represents an AWS service. While most services have one command associated with them, some services have multiple commands (for example, S3 has s3 and s3api).

Run aws help to see all the commands/services that are available – they will have probably changed by the time this book is printed!

Subcommands

Each command has a selection of subcommands to perform service-specific actions.

Run aws <command> help to see all the available subcommands.

Options

Subcommands take options and start with --.

You can view all the options and their purposes by running aws <command> <subcommand> help.

While most are optional (hence the name), those that are not surrounded by square brackets ([]) are required. You will get an error message (with the appropriate details) if you don't include them.

The built-in documentation is the best place to start looking for answers. There are usually examples after all of the options have been described. Otherwise, there are plenty of examples available online.

Some options are available for all or most commands, so they are particularly useful to know.

Output

The CLI tool can be configured to output in JSON, table, or text format. To control the output type, use the --output option.

To set a default output type for all your commands, set the output parameter for your profile.

JSON

JavaScript Object Notation (JSON) (http://json.org/) is a standard machine- and human-readable information interchange format. Here's what the AZs in the us-east-1 (North Virginia) region look like, represented as JSON:

aws ec2 describe-availability-zones --output json 
{
    "AvailabilityZones": [
        {
            "State": "available",
            "ZoneName": "us-east-1a",
            "Messages": [],
            "RegionName": "us-east-1"
        },
        {
            "State": "available",
            "ZoneName": "us-east-1b",
            "Messages": [],
            "RegionName": "us-east-1"
        },
        ...
    ]
}

Note that a portion of the output was elided for space.

Table

The table format displays a text/ASCII table of results. This can be useful for generating printable reports:

Table format

Text

The text output format only displays the resulting key/value response. No additional formatting or display characters are added:

Text format

The text format is the default and is suitable for most routine CLI tasks.

Querying

The CLI tool supports transforming the response from the API with the --query option. This option takes a JMESPath query as a parameter and returns the query result.

JMESPath is a query language for JSON. For more information, visit http://jmespath.org/.

As the query is processed as part of the command, it takes place on the server, not the client. By offloading work to the server, you can reduce the size of the resulting payload and improve response times.

JMESPath can be used to transform the response that you receive:

$ aws ec2 describe-availability-zones \
  --output json \
  --query "AvailabilityZones[].ZoneName"
  [
  "us-east-1a",
  "us-east-1c",
  "us-east-1d",
  "us-east-1e"
  ]

It can also be used to filter the data that is received:

$ aws ec2 describe-availability-zones 
  --output json 
  --query "AvailabilityZones[?ZoneName == 'us-east-1a'].State"
  [
  "available"
  ]

Using the --query option can open up a number of possibilities to give you flexible options for solving problems with the AWS CLI.

Generating a CLI skeleton

When performing complex tasks with the CLI tool, it may be easier to pass a JSON object of options. This kind of interaction may signify that you should use one of the AWS software development kits (SDKs).

Input

To generate a sample JSON object that will be accepted, run any command with the --generate-cli-skeleton option:

$ aws ec2 describe-availability-zones --generate-cli-skeleton 
{ 
"DryRun": true, 
"ZoneNames": [ 
"" 
    ], 
"Filters": [ 
        { 
"Name": "", 
"Values": [ 
"" 
           ] 
        } 
    ] 
}

You can then copy, edit, and use this object to define your command options without passing lots of individual options. It works best for commands with arrays of options or a variable number of options.

Output

You can also get a preview of the output of a command by calling the command with the --generate-cli-skeleton output option. This can speed up the process of combining CLI commands as you can see a response without actually calling the API:

$ aws ec2 describe-availability-zones --generate-cli-skeleton output 
{ 
  "AvailabilityZones": [ 
    { 
      "ZoneName": "ZoneName", 
      "State": "State", 
      "RegionName": "RegionName", 
      "Messages": [ 
        { 
          "Message": "Message" 
        } ] 
    } ] 
}

Pagination

The results that are returned by the CLI tool are limited to 1,000 resources by default.

This is not normally an issue, but at a certain scale, you may run into pagination issues. A common example is a list of files in an S3 bucket.

If you are absolutely sure you should be seeing a particular resource in the response but cannot, check your pagination. The resource may be included in the matching resources, just not in the part of the response that was returned to you.

The following options allow you to control the number and starting point of the results that are returned to you from the API:

• --page-size: This limits how many resources will be displayed to you, but does not actually limit the number that's returned. The default number of items (that is, 1,000) will still be processed and returned to you.
• --max-items: This sets an upper limit on how many items will actually be returned in the response. You may receive fewer items, but you will not receive more than this number.
• --starting-token: This changes where the response starts. Use this to display subsequent results, beyond the first page:

aws s3api list-objects --bucket bucket-name --max-items 100 --starting-token [TOKEN]

Use a token that's been returned by a previous CLI command to continue where you left off.

Autocomplete

You can enable tab completion of commands, subcommands, and options by configuring the completer included with the CLI tool.

On macOS, Linux, and Windows systems with a bash shell, you can load the completer with the following command:

complete -C 'which aws_completer'aws

By default, the aws_completer program is installed in /usr/local/bin. If your tool is installed to a non-standard location, you will need to find it and change the which aws_completer command to the relevant path.

There's more...

At the time of writing, AWS is previewing a new tool called aws-shell. You can check it out at https://github.com/awslabs/aws-shell. When using aws-shell, you can use all the same commands offered by the CLI, but without the aws prefix. It also offers robust auto-completion, including the ability to autocomplete resources such as EC2 instance names.

See also

• Chapter 10, Advanced CloudFormation, will dive into more complex scenarios and features, such as custom resources