You're reading from Data Engineering with AWS Acquire the skills to design and build AWS-based data transformation pipelines like a pro

Product type Paperback

Published in Oct 2023

Publisher Packt

ISBN-13 9781804614426

Length 636 pages

Edition 2nd Edition

Tools

AWS

Concepts

Data Engineering

Author (1):

Gareth Eagar

View More author details

Table of Contents (24) Chapters

Preface

1. Section 1: AWS Data Engineering Concepts and Trends

2. An Introduction to Data Engineering FREE CHAPTER

3. Data Management Architectures for Analytics

4. The AWS Data Engineer’s Toolkit

5. Data Governance, Security, and Cataloging

6. Section 2: Architecting and Implementing Data Engineering Pipelines and Transformations

7. Architecting Data Engineering Pipelines

8. Ingesting Batch and Streaming Data

9. Transforming Data to Optimize for Analytics

10. Identifying and Enabling Data Consumers

11. A Deeper Dive into Data Marts and Amazon Redshift

12. Orchestrating the Data Pipeline

13. Section 3: The Bigger Picture: Data Analytics, Data Visualization, and Machine Learning

14. Ad Hoc Queries with Amazon Athena

15. Visualizing Data with Amazon QuickSight

16. Enabling Artificial Intelligence and Machine Learning

17. Section 4: Modern Strategies: Open Table Formats, Data Mesh, DataOps, and Preparing for the Real World

18. Building Transactional Data Lakes

19. Implementing a Data Mesh Strategy

20. Building a Modern Data Platform on AWS

21. Wrapping Up the First Part of Your Learning Journey

22. Other Books You May Enjoy

23. Index

Hands-on – Using the AWS Command Line Interface (CLI) to create S3 buckets

In Chapter 1, An Introduction to Data Engineering, you created an AWS account and an AWS administrative user, and then ensured you could access your account. Console access allows you to access AWS services and perform most functions, however it can also be useful to interact with AWS services via the Command Line Interface (CLI) at times. In this hands-on section, you learn how to access the AWS CLI, and then use the CLI to create Amazon S3 buckets (a storage container in the Amazon S3 service).

Accessing the AWS CLI

The AWS CLI can be installed on your personal computer / laptop, or can be accessed from the AWS Console. To access the CLI on your personal computer, you need to generate a set of access keys.Your access keys consist of an Access Key ID (which is comparable to a user name), and a Secret Access Key (which is comparable to a password). With these two pieces of information, you can authenticate...