Part 1:Introduction to ETL, Data Pipelines, and Design Principles

For the first part of this book, we will introduce the fundamentals of data pipelines in Python and set up your local development environment with Integrated Development Environments (IDEs), virtual environments, and Git version control. We will provide you with an overview of what Extract-Load-Transform (ETL) data pipelines are and how to design them yourself. As a word of caution, Python is at the core of this book; you must have a basic familiarity with Python in order to follow along accordingly.

This section contains the following chapters:

• Chapter 1, A Primer on Python and the Development Environment
• Chapter 2, Understanding the ETL Process and Data Pipelines
• Chapter 3, Design Principles for Creating Scalable and Resilient Pipelines