You're reading from Data Engineering with AWS Learn how to design and build cloud-based data transformation pipelines using AWS

Product type Paperback

Published in Dec 2021

Publisher Packt

ISBN-13 9781800560413

Length 482 pages

Edition 1st Edition

Languages

Python

Tools

AWS

Concepts

Data Analysis

Author (1):

Gareth Eagar

View More author details

Table of Contents (19) Chapters

Preface

1. Section 1: AWS Data Engineering Concepts and Trends

2. Chapter 1: An Introduction to Data Engineering FREE CHAPTER

3. Chapter 2: Data Management Architectures for Analytics

4. Chapter 3: The AWS Data Engineer's Toolkit

5. Chapter 4: Data Cataloging, Security, and Governance

6. Section 2: Architecting and Implementing Data Lakes and Data Lake Houses

7. Chapter 5: Architecting Data Engineering Pipelines

8. Chapter 6: Ingesting Batch and Streaming Data

9. Chapter 7: Transforming Data to Optimize for Analytics

10. Chapter 8: Identifying and Enabling Data Consumers

11. Chapter 9: Loading Data into a Data Mart

12. Chapter 10: Orchestrating the Data Pipeline

13. Section 3: The Bigger Picture: Data Analytics, Data Visualization, and Machine Learning

14. Chapter 11: Ad Hoc Queries with Amazon Athena

15. Chapter 12: Visualizing Data with Amazon QuickSight

16. Chapter 13: Enabling Artificial Intelligence and Machine Learning

17. Chapter 14: Wrapping Up the First Part of Your Learning Journey

18. Other Books You May Enjoy

Chapter 9: Loading Data into a Data Mart

While the data lake enables a significant amount of analytics to happen inside it, there are several use cases where a data engineer may need to load data into an external data warehouse, or data mart, to enable a set of data consumers.

As we reviewed in Chapter 2, Data Management Architectures for Analytics, a data lake is a single source of truth across multiple lines of business, while a data mart contains a subset of data of interest to a particular group of users. A data mart could be a relational database, a data warehouse, or a different kind of data store.

Data marts serve two primary purposes. First, they provide a database with a subset of the data in the data lake, optimized for specific types of queries (such as for a specific business function). In addition, they also provide a higher-performing, lower latency query engine, which is often required for specific analytic use cases (such as for powering business intelligence...