The command-line interface tool

The AWS command-line interface (CLI) tool is an important piece of the AWS administrator's toolkit.

The CLI tool is often one of the quickest and easiest ways to interact with the API. As a text-based tool, it scales much easier than using the web console. Unlike the console, it can be automated, for example, via scripts. The AWS application programming interface (API) represents all the functionality available to you as an AWS administrator. It is also easier to keep a track of through your command-line history. Like all good CLI tools, simple individual commands can be chained (or piped) together to perform complex tasks.

The CLI tool is open source software, maintained on GitHub https://github.com/aws/aws-cli. For more detailed documentation, refer to the AWS CLI homepage https://aws.amazon.com/cli.

Installation

The CLI tool requires Python 2.6.5 or greater.

The easiest way to install it is to use the Python package manager, pip:

pip install awscli

This will make the command aws available on your system.

Upgrade

AWS frequently releases new services and functionality. In order to use the new features, you will need to upgrade the CLI tool.

To upgrade, run the following pip command periodically:

pip install --upgrade awscli

Configuration

Authentication between the CLI tool and the AWS API is done via two pieces of information:

• Access key ID
• Secret access key

As the name suggests, you should keep your secret access key a secret! Be careful where you store or send it.

Once you have created a user, you can configure the tool to use it for authentication purposes.

While you can configure the CLI tool with access keys directly, this should be avoided. Instead, you should use profiles to store your credentials. Using profiles gives you a more consistent and manageable centralized location to secure your secret keys.

Default profile

Without any additional configuration or options, your CLI tool commands will use the default profile.

To set up the default profile, you can use the following command:

aws configure

This will prompt you for an access key ID, secret access key, region, and output format.

Named profiles

In addition to the default profile, you can configure other, named profiles. This is useful for switching between users with different levels of access (for example, read-only and administrator) or even between users in different accounts.

aws configure --profile <profile-name>

Once you have responded to the prompts, you can reference the named profile by passing the --profile <profile-name> option with your command.

Environment variables

You can also configure the CLI via the use of environment variables:

export AWS_PROFILE=<profile-name>

While you should prefer to use profiles over setting your access ID and secret keys directly, sometimes you may have to do it. If you must set your keys directly, do so via environment variables so that you do not need to pass your keys around or hardcode them:

export AWS_ACCESS_KEY_ID=<access-key-id>
export AWS_SECRET_ACCESS_KEY=<secret-access-key>

Instance roles

When running the CLI tool on an EC2 instance, you can leverage the instance's IAM role to make calls. This means you do not need to configure credentials or set environment variables (manually).

Behind the scenes, the instance will retrieve and set its own AWS environment variables that allow API calls. You do need to ensure the instance has appropriate permissions.

The AWS CLI tool comes preinstalled on AWS Linux-based instances.

Usage

All CLI tool commands are service based. Using service commands and subcommands, you can make calls directly to the AWS API.

Commands

Each command represents an AWS service. While most services have one command associated with them, some services have multiple commands (for example, S3 has s3 and s3api).

Run aws help to see all the commands/services that are available—they will probably have changed by the time this book prints!

Subcommands

Each command has a selection of subcommands to perform service-specific actions.

Run aws <command> help to see all subcommands.

Options

Subcommands take options, which start with --.

See all options and their purpose with aws <command> <subcommand> help.

While most are optional (hence the name), those that are not surrounded by square brackets ([]) are required. You will get an error message (with appropriate details) if you do not include them.

The built-in documentation is the best place to start looking for answers. There are usually examples after all of the options have been described. Otherwise, there are plenty of examples available online.

Some options are available to all or most commands, so they are particularly useful to know.

Output

The CLI tool can be configured to output in JSON, table, or text format. To control the output type, use the --output option.

To set a default output type for all your commands, set the output parameter for your profile.

JSON

JavaScript Object Notation (JSON) (http://json.org/), a standard, machine- and human-readable information interchange format. Here's what the AZs in the us-east-1 (North Virginia) region look like represented as JSON:

aws ec2 describe-availability-zones --output json 
{ 
"AvailabilityZones": [ 
        { 
"State": "available", 
"RegionName": "us-east-1", 
"Messages": [], 
"ZoneName": "us-east-1a" 
        }, 
        { 
"State": "available", 
"RegionName": "us-east-1", 
"Messages": [], 
"ZoneName": "us-east-1c" 
        }, 
        { 
"State": "available", 
"RegionName": "us-east-1", 
"Messages": [], 
"ZoneName": "us-east-1d" 
        }, 
        { 
"State": "available", 
"RegionName": "us-east-1", 
"Messages": [], 
"ZoneName": "us-east-1e" 
        } 
    ] 
}

Table

The table format displays a text/ASCII table of results. This can be useful for generating printable reports:

Text

The text output format only displays the resulting key/value response. No additional formatting or display characters are added.

Querying

The CLI tool supports transforming the response from the API with the --query option. This option takes a JMESPath query as a parameter and returns the query result.

JMESPath is a query language for JSON. For more information, visit http://jmespath.org/.

As the query is processed as part of the command, it takes place on the server, not the client. By offloading work to the server, you can reduce the size of the resulting payload and improve response times.

JMESPath can be used to transform the response that you receive:

$ aws ec2 describe-availability-zones \
  --output json \
  --query 'AvailabilityZones[].ZoneName'
  [
  "us-east-1a",
  "us-east-1c",
  "us-east-1d",
  "us-east-1e"
  ]

It can also be used to filter the data that is received:

$ aws ec2 describe-availability-zones 
  --output json 
  --query "AvailabilityZones[?ZoneName == 'us-east-1a'].State"
  [
  "available"
  ]

Generate CLI skeleton

When performing complex tasks with the CLI tool, it may be easier to pass a JSON object of options. This kind of interaction may signify that you should use one of the AWS software development kits (SDKs).

Input

To generate a sample JSON object that will be accepted, run the command with the --generate-cli-skeleton option:

$ aws ec2 describe-availability-zones --generate-cli-skeleton 
{ 
"DryRun": true, 
"ZoneNames": [ 
"" 
    ], 
"Filters": [ 
        { 
"Name": "", 
"Values": [ 
"" 
           ] 
        } 
    ] 
}

You can then copy, edit, and use this object to define your command options without passing lots of individual options. It works best for commands with arrays of options or a variable number of options.

Output

You can also get a preview of the output of a command by calling the command with the --generate-cli-skeleton output option. This can speed up the process of combining CLI commands as you can see a response without actually calling the API:

$ aws ec2 describe-availability-zones --generate-cli-skeleton output 
{ 
"AvailabilityZones": [ 
        { 
"ZoneName": "ZoneName", 
"State": "State", 
"RegionName": "RegionName", 
"Messages": [ 
                  { 
"Message": "Message" 
                } 
            ] 
        } 
    ] 
}

Pagination

Results returned by the CLI tool are limited to 1,000 resources by default.

This is not normally an issue, but at a certain scale, you may run into pagination issues. A common example is files in an S3 bucket.

If you are absolutely sure you should be seeing a particular resource in a response but cannot, check your pagination. The resource may be included in the matching resources, just not in the part of the response that was returned to you.

The following options allow you to control the number and starting point of the results returned to you from the API:

• --page-size: This limits how many resources will be displayed to you, but does not actually limit the number returned. The default number of items (that is, 1,000) will still be processed and returned to you.
• --max-items: This sets an upper limit on how many items will actually be returned in the response. You may receive fewer items, but you will not receive more than this number.
• --starting-token: This changes where the response starts. Use this to display subsequent results, beyond the first page.

aws s3 ls --bucket bucket-name --max-items 100 --starting-token None___100

Autocomplete

You can enable tab-completion of commands, subcommands, and options by configuring the completer included with the CLI tool.

On OS X, Linux, and Windows systems with a bash shell, you can load the completer with the following command:

complete -C 'which aws_completer'aws

By default, the aws_completer program is installed in /usr/local/bin. If your tool is installed to a non-standard location, you will need to find it and change the which aws_completer command to the relevant path.

Related tools

The following program work nicely with the AWS CLI tool, and may come in handy.

jq

jq is a lightweight tool for processing and transforming JSON. It follows the Unix philosophy of doing one thing and doing it well. It can be found at https://stedolan.github.io/jq/.

While jq and JMESPath are similar, jq is a lot easier to get started with. It also supports transforming JSON into plaintext; JMESPath queries will always return more JSON.

You can pipe JSON results from the CLI tool to it, and easily transform the results for use elsewhere. This example uses jq's property name selectors to convert JSON output to text:

$ aws ec2 describe-availability-zones --output json | jq '.AvailabilityZones[].ZoneName'
"us-east-1a"
"us-east-1c"
"us-east-1d"
"us-east-1e"