You're reading from Building ETL Pipelines with Python Create and deploy enterprise-ready ETL pipelines by employing modern methods

Product type Paperback

Published in Sep 2023

Publisher Packt

ISBN-13 9781804615256

Length 246 pages

Edition 1st Edition

Languages

Python

Tools

AWS

Concepts

Data Streaming

Authors (2):

Brij Kishore Pandey

Emily Ro Schoof

View More author details

Table of Contents (22) Chapters

Preface

1. Part 1:Introduction to ETL, Data Pipelines, and Design Principles

2. Chapter 1: A Primer on Python and the Development Environment FREE CHAPTER

3. Chapter 2: Understanding the ETL Process and Data Pipelines

4. Chapter 3: Design Principles for Creating Scalable and Resilient Pipelines

5. Part 2:Designing ETL Pipelines with Python

6. Chapter 4: Sourcing Insightful Data and Data Extraction Strategies

7. Chapter 5: Data Cleansing and Transformation

8. Chapter 6: Loading Transformed Data

9. Chapter 7: Tutorial – Building an End-to-End ETL Pipeline in Python

10. Chapter 8: Powerful ETL Libraries and Tools in Python

11. Part 3:Creating ETL Pipelines in AWS

12. Chapter 9: A Primer on AWS Tools for ETL Processes

13. Chapter 10: Tutorial – Creating an ETL Pipeline in AWS

14. Chapter 11: Building Robust Deployment Pipelines in AWS

15. Part 4:Automating and Scaling ETL Pipelines

16. Chapter 12: Orchestration and Scaling in ETL Pipelines

17. Chapter 13: Testing Strategies for ETL Pipelines

18. Chapter 14: Best Practices for ETL Pipelines

19. Chapter 15: Use Cases and Further Reading

20. Index

Why subscribe?

21. Other Books You May Enjoy

Computing and automation with AWS

AWS Glue, AWS Lambda, and AWS Step Functions are three cloud services offered by AWS that provide serverless computing and workflow automation capabilities. Let’s look at them in more detail.

AWS Glue

AWS Glue (https://docs.aws.amazon.com/glue/latest/dg/what-is-glue.html) is a fully consolidated data integration tool for end-to-end use, from data sourcing to analytic dashboards. It contains a fully managed ETL service that allows you to create and provide a number of features such as automatic schema discovery, data cataloging, job scheduling, error handling, and monitoring. Since Glue is an AWS service, it’s integrated directly with over 70 types of source data formats as well as popular AWS data target locations, such as Amazon S3, Amazon RDS, and Amazon Redshift. Most importantly, Glue uses a serverless architecture that provides built-in high availability (HA) and pay-as-you-go billing for increased agility.

Within its infrastructure...

The rest of the chapter is locked

You're reading from Building ETL Pipelines with Python Create and deploy enterprise-ready ETL pipelines by employing modern methods

Table of Contents (22) Chapters

Computing and automation with AWS

AWS Glue

Unlock this book and the full library FREE for 7 days

Authors (2)

Personalised recommendations for you