You're reading from Cracking the Data Engineering Interview Land your dream job with the help of resume-building tips, over 100 mock questions, and a unique portfolio

Product type Paperback

Published in Nov 2023

Publisher Packt

ISBN-13 9781837630776

Length 196 pages

Edition 1st Edition

Languages

Python

Tools

Core Data

Concepts

Data Engineering

Authors (2):

Kedeisha Bryan

Taamir Ransome

View More author details

Table of Contents (23) Chapters

Preface

1. Part 1: Landing Your First Data Engineering Job

2. Chapter 1: The Roles and Responsibilities of a Data Engineer FREE CHAPTER

3. Chapter 2: Must-Have Data Engineering Portfolio Projects

4. Chapter 3: Building Your Data Engineering Brand on LinkedIn

5. Chapter 4: Preparing for Behavioral Interviews

6. Part 2: Essentials for Data Engineers Part I

7. Chapter 5: Essential Python for Data Engineers

8. Chapter 6: Unit Testing

9. Chapter 7: Database Fundamentals

10. Chapter 8: Essential SQL for Data Engineers

11. Part 3: Essentials for Data Engineers Part II

12. Chapter 9: Database Design and Optimization

13. Chapter 10: Data Processing and ETL

14. Chapter 11: Data Pipeline Design for Data Engineers

15. Chapter 12: Data Warehouses and Data Lakes

16. Part 4: Essentials for Data Engineers Part III

17. Chapter 13: Essential Tools You Should Know

18. Chapter 14: Continuous Integration/Continuous Development (CI/CD) for Data Engineers

19. Chapter 15: Data Security and Privacy

20. Chapter 16: Additional Interview Questions

21. Index

Why subscribe?

22. Other Books You May Enjoy

Portfolio data engineering project

In this section, we will look at an example Azure data engineering project on sports analytics that involves creating a pipeline that ingests, cleans, and visualizes data.

Scenario

You were recently employed by a company (Connect) that has rendered all data-related services to its clients for the past 2 months. You have been attached to a team, but today, you have been given your first job sole project.

A new season of the English Premier League just commenced and your company assigned you to a data engineering job posted by a client. They provided several website links to acquire the data from. Your job is to have all this data extracted, transformed, and loaded into cloud storage and a PostgreSQL database every Saturday and Sunday until the end of the season.

As a data engineer, you should be able to assess the requirements to properly decide which tool to use for each task that would make the process efficient. For example, using Spark...