You're reading from Data Engineering with Databricks Cookbook Build effective data and AI solutions using Apache Spark, Databricks, and Delta Lake

Product type Paperback

Published in May 2024

Publisher Packt

ISBN-13 9781837633357

Length 438 pages

Edition 1st Edition

Tools

Apache Spark

Concepts

Data Engineering

Author (1):

Pulkit Chadha

View More author details

Table of Contents (16) Chapters

Preface

1. Part 1 – Working with Apache Spark and Delta Lake FREE CHAPTER

2. Chapter 1: Data Ingestion and Data Extraction with Apache Spark

3. Chapter 2: Data Transformation and Data Manipulation with Apache Spark

4. Chapter 3: Data Management with Delta Lake

5. Chapter 4: Ingesting Streaming Data

6. Chapter 5: Processing Streaming Data

7. Chapter 6: Performance Tuning with Apache Spark

8. Chapter 7: Performance Tuning in Delta Lake

9. Part 2 – Data Engineering Capabilities within Databricks

10. Chapter 8: Orchestration and Scheduling Data Pipeline with Databricks Workflows

11. Chapter 9: Building Data Pipelines with Delta Live Tables

12. Chapter 10: Data Governance with Unity Catalog

13. Chapter 11: Implementing DataOps and DevOps on Databricks

14. Index

Why subscribe?

15. Other Books You May Enjoy

Leveraging GitHub Actions with Databricks Asset Bundles (DABs)

CI/CD are software development practices that aim to automate the building, testing, and deployment of code on various platforms. CI/CD can help developers deliver high-quality software faster and more reliably, as well as reduce the risks and costs associated with manual and error-prone processes. Using CI/CD with Databricks can enable developers to leverage the power and flexibility of the platform while ensuring that their code is consistent, tested, and deployable across different environments and teams. CI/CD can also help developers integrate their Databricks workflows with other tools and services, such as source control, issue tracking, monitoring, and security. By using DABs, you can leverage popular CI/CD tools, such as GitHub Actions, Azure DevOps, Jenkins, and CircleCI, to automate the testing and deployment of your bundle projects. This way, you can reduce errors, promote software best practices, and iterate...