You're reading from Building ETL Pipelines with Python Create and deploy enterprise-ready ETL pipelines by employing modern methods

Product type Paperback

Published in Sep 2023

Publisher Packt

ISBN-13 9781804615256

Length 246 pages

Edition 1st Edition

Languages

Python

Tools

AWS

Concepts

Data Streaming

Authors (2):

Brij Kishore Pandey

Emily Ro Schoof

View More author details

Table of Contents (22) Chapters

Preface

1. Part 1:Introduction to ETL, Data Pipelines, and Design Principles

2. Chapter 1: A Primer on Python and the Development Environment FREE CHAPTER

3. Chapter 2: Understanding the ETL Process and Data Pipelines

4. Chapter 3: Design Principles for Creating Scalable and Resilient Pipelines

5. Part 2:Designing ETL Pipelines with Python

6. Chapter 4: Sourcing Insightful Data and Data Extraction Strategies

7. Chapter 5: Data Cleansing and Transformation

8. Chapter 6: Loading Transformed Data

9. Chapter 7: Tutorial – Building an End-to-End ETL Pipeline in Python

10. Chapter 8: Powerful ETL Libraries and Tools in Python

11. Part 3:Creating ETL Pipelines in AWS

12. Chapter 9: A Primer on AWS Tools for ETL Processes

13. Chapter 10: Tutorial – Creating an ETL Pipeline in AWS

14. Chapter 11: Building Robust Deployment Pipelines in AWS

15. Part 4:Automating and Scaling ETL Pipelines

16. Chapter 12: Orchestration and Scaling in ETL Pipelines

17. Chapter 13: Testing Strategies for ETL Pipelines

18. Chapter 14: Best Practices for ETL Pipelines

19. Chapter 15: Use Cases and Further Reading

20. Index

Why subscribe?

21. Other Books You May Enjoy

Summary

In this chapter, we went through a series of examples to demonstrate how to build robust ETL pipelines in Python using various frameworks and libraries.

By understanding the different frameworks and libraries available for building ETL pipelines in Python, data engineers and analysts can make informed decisions about how to optimize their workflows for efficiency, reliability, and maintainability. With the right tools and practices, ETL can be a powerful and streamlined process that enables organizations to leverage the full potential of their data assets.

In the next chapter, we will continue to dig deeper into creating robust data pipelines using external resources. More specifically, we will introduce the AWS ecosystem and demonstrate how you can leverage AWS to create exceptional, scalable, and cloud-based ETL pipelines.

The rest of the chapter is locked

You're reading from Building ETL Pipelines with Python Create and deploy enterprise-ready ETL pipelines by employing modern methods

Table of Contents (22) Chapters

Summary

Unlock this book and the full library FREE for 7 days

Authors (2)

Personalised recommendations for you