Creating an S3 bucket policy

In this recipe, we will learn to create bucket policies for our S3 buckets. Whenever possible, it is preferable to use a bucket policy or IAM policy instead of ACLs. The choice between bucket and IAM policies is mostly a personal preference. We can also create bucket policies using prefixes. S3 is an object store with no concept of folders, but prefixes can be used to imitate folders. Prefixes can represent objects as well.

Getting ready

We need a working AWS account with following resources configured:

1. A bucket and a file in it: I will be using a bucket name awsseccookbook with a file named image-heartin-k.png. Replace them with your bucket name and filename.
2. A user with no permission and a user with administrator permission: Configure CLI profiles for these users. I will be calling users and their profiles testuser and awssecadmin, respectively.

3. Uncheck the two Block all public access settings related to bucket policies. Leave the the other settings checked, as shown in the following screenshot, and click Save:

4. Verify that your bucket does not allow listing for everyone by going to the bucket URL from the browser.

Next, we will use bucket policies to give permissions to everyone to list the contents of our bucket and then retry this step.

How to do it...

We will first generate a policy from the console using the policy generator. Later, we will execute the policy from the CLI.

Bucket public access with a bucket policy from the console

We can give public access to list the contents of a bucket as follows:

1. Go to the S3 service in the console, click on your bucket's name, go to the Permissions tab, and then go to Bucket Policy.
2. Click on Policy generator in the lower-left corner.
3. Within Policy generator, select/enter data as follows:
   • Select Type of Policy as Bucket Policy.
   • Select Principal as *.
   • Select AWS Service as Amazon S3.
   • Select Actions as ListBucket.
   • Select Amazon Resource Name (ARN) as arn:aws:s3:::awsseccookbook.
4. Click on Add Conditions (Optional).
5. Click Add Condition and enter the following:
   • Condition as DateLessThan
   • Key as aws:EpochTime
   • Value as a future date in epoch format (for example, 1609415999)
6. Click Add Condition.
7. Click Add Statement.
8. Click Generate Policy. The policy should look similar to the following. I have changed the Sid to a meaningful name:

{
     "Id": "Policy1560413644620",
     "Version": "2012-10-17",
     "Statement": [
         {
             "Sid": "ListBucketPermissionForAll",
             "Action": [
                 "s3:ListBucket"
               ],
             "Effect": "Allow",
             "Resource": "arn:aws:s3:::awsseccookbook",
             "Condition": {
                 "DateLessThan": {
                     "aws:EpochTime": "1609415999"
                 }
             },
             "Principal": "*"
        }
     ]
}

9. Copy and paste the policy from the policy generator into the bucket policy editor, and click Save. The contents of the bucket should now be listed:

10. In the bucket policy, change the value of Action to s3:GetObject and Resource to arn:aws:s3:::awsseccookbook/*, and then click Save. Access any object from within the bucket from the browser. We should be able to successfully retrieve the object:

If we change the resource to arn:aws:s3:::awsseccookbook/* without an object operation such as s3:GetObject, we will get an error stating that the action does not apply to any resource. This is because, when we add any prefix to the bucket, it is considered an object operation and we have not defined any object operations yet.

Bucket list access with a bucket policy from the CLI

In this section, we will see how to add a bucket policy from the CLI:

1. If you are following along from the previous section, remove the bucket policy that was added. Verify that you do not have access to list the bucket or get the object from the browser.
2. Create a bucket policy to allow our test user to access it and save it as bucket-policy-allow-test-user.json:

{
    "Id": "Policy1560416549842",
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "ListAllBuckets",
            "Action": [
                "s3:ListBucket"
            ],
            "Effect": "Allow",
            "Resource": "arn:aws:s3:::awsseccookbook",
            "Principal": {
                "AWS": "arn:aws:iam::135301570106:user/testuser"
            }
        }
    ]
}

The condition element is an optional element.

3. Attach the policy to the bucket:

aws s3api put-bucket-policy \
 --bucket awsseccookbook \
 --policy file://resources/bucket-policy-allow-test-user.json \
 --profile awssecadmin

4. List the contents of the bucket using the testuser user from the command line, as shown in the following screenshot:

Now that you have seen how to create policies from the console and the CLI, practice more scenarios with each of the available actions and conditions.

How it works...

In this recipe, we created S3 bucket policies. A bucket policy statement can have the following components: Sid, Principal, Effect, Action, Resource, and Condition. All of these except Principal are the same as an IAM policy and we explored them in the Creating IAM policies recipe in Chapter 1, Managing AWS Accounts with IAM and Organizations.

Principal for a bucket policy can be an account, user, or everyone (denoted by *). Principals can contain an ARN for a resource (specified using the ARN element) or a canonical ID (specified using the CanonicalUser element).

Resource in the case of a bucket policy is a bucket or object and is denoted using a bucket ARN. The bucket ARN should be in the form: arn:aws:s3:::bucket_name. An object resource is represented in the form: arn:aws:s3:::bucket_name/key_name. To denote all objects within a bucket, we can use arn:aws:s3:::bucket_name/*. We can denote every resource in every bucket as arn:aws:s3:::*.

Conditions allow us to conditionally execute policies. We used conditions in one of the examples. We will see further practical uses of conditions in the next recipe.

There's more...

Bucket policies follow the same JSON document structure as IAM policies, but have an additional principal field. The principal is the user or entity for which a policy statement is applicable. There is no principal for an IAM policy as it is attached to an IAM user. The IAM user who executes that policy is the principal in the case of an IAM policy.

Consider the following examples when using Principal in bucket policies:

• A root user can be represented as follows:

"Principal" : {
    "AWS": "arn:aws:iam::135301570106:root"
}

• An IAM user can be represented as follows:

"Principal" : {
    "AWS": "arn:aws:iam::135301570106:user/testuser"
}

• A canonical user ID can be represented as follows:

"Principal" : {
   "CanonicalUser":"5df5b6014ae606808dcb64208aa09e4f19931b3123456e152c4dfa52d38bf8fd"
}

Canonical IDs were used in the previous recipe, Creating S3 access control lists.

• An anonymous user can be represented as follows:

"Principal" : "*"

Let's quickly go through some more important details relating to S3 bucket policies:

• Currently, we have around 50 bucket policy actions, including those that work on an object (for example, s3:PutObject), a bucket (for example, s3:CreateBucket), or a bucket sub-resource (for example, PutBucketAcl).
• The current list of bucket sub-resources with permissions includes BucketPolicy, BucketWebsite, AccelerateConfiguration, BucketAcl, BucketCORS, BucketLocation, BucketLogging, BucketNotification, BucketObjectLockConfiguration, BucketPolicyStatus, BucketPublicAccessBlock, BucketRequestPayment, BucketTagging, BucketVersioning, EncryptionConfiguration, InventoryConfiguration, LifecycleConfiguration, MetricsConfiguration, ReplicationConfiguration, and AnalyticsConfiguration.
• We cannot specify an IAM group as a principal in an S3 bucket policy. If we add a group instead of a user, we will get an error: Invalid principal in policy.
• Here are some S3-specific condition keys available for use in conditions within a policy: s3:x-amz-acl, s3:x-amz-copy-source, s3:x-amz-metadata-directive, s3:x-amz-server-side-encryption, s3:VersionId, s3:LocationConstraint, s3:delimiter, s3:max-keys, s3:prefix, s3:x-amz-server-side-encryption-aws-kms-key-id, s3:ExistingObjectTag/<tag-key>, s3:RequestObjectTagKeys, s3:RequestObjectTag/<tag-key>, s3:object-lock-remaining-retention-days, s3:object-lock-mode, s3:object-lock-retain-until-date, and s3:object-lock-legal-hold.

See also

• You can read about IAM policies in the Creating IAM policies recipe in Chapter 1, Managing AWS Accounts with IAM and Organizations.
• For a detailed comparison of ACLs, bucket policies, and IAM policies, refer to the There's more section in the Creating S3 access control lists recipe.