Search icon Close icon
Search icon CANCEL
Subscription
0
Cart icon
Cart
Close icon
You have no products in your basket yet
Save more on your purchases!
Savings automatically calculated. No voucher code required
Arrow left icon
All Products
Best Sellers
New Releases
Books
Videos
Audiobooks
Learning Hub
Newsletters
Free Learning
Arrow right icon
Mex$179.99 | ALL EBOOKS & VIDEOS
Save more on purchases! Buy 2 and save 10%, Buy 3 and save 15%, Buy 5 and save 20%
Home > Data > Data Engineering > Mastering Azure Databricks for Data Engineers [Video]
Mastering Azure Databricks for Data Engineers [Video]
Mastering Azure Databricks for Data Engineers [Video]

Mastering Azure Databricks for Data Engineers: Learn Azure Databricks for professional data engineers using PySpark and Spark SQL [Video]

By ScholarNest
Mex$2,256.99 Mex$179.99
Video Jun 2024 17 hours 31 minutes 1st Edition
Video
Mex$2,256.99 Mex$179.99
Subscription
Free Trial
Video
Mex$2,256.99 Mex$179.99
Subscription
Free Trial

What do you get with a video?

Product feature icon Download this video in MP4 format
Product feature icon Access this title in our online reader with advanced features
Product feature icon DRM FREE - Read whenever, wherever and however you want
Buy Now
Table of content icon View table of contents Download code icon Download Code

Key benefits

  • Step-by-step guide to setting up and navigating the Azure Databricks platform
  • Detailed lessons on working with Spark clusters, notebooks, and DBFS
  • Real-world capstone project to apply your skills in a practical scenario

Description

Embark on a transformative journey through the Azure Databricks platform with this expertly crafted course. We start by laying a solid foundation, guiding you through the course prerequisites and familiarizing you with the resources at your disposal. Our introduction section covers the essentials of data engineering and how Apache Spark integrates with Databricks, setting the stage for a deep dive into the platform. As you progress, you’ll create an Azure cloud account and Databricks workspace, gaining insights into the platform's architecture. Hands-on sessions will enable you to create Spark clusters, work with Databricks notebooks, and utilize magic commands and utilities effectively. We then delve into the Databricks File System (DBFS), teaching you how to manage and mount data storage efficiently. The course further explores Unity Catalog for secure data management, Delta Lake for robust data processing, and incremental ingestion tools for real-time data handling. You'll also master Databricks Delta Live Tables (DLT), enhancing your skills in building scalable data pipelines. Our final sections cover automation features, including working with Databricks Repos, Workflows, REST API, and CLI, ensuring you can automate and streamline your data projects.

What you will learn

Create and manage Databricks workspaces and Spark clusters Utilize Databricks notebooks and magic commands for efficient data processing Implement secure data management with Unity Catalog Perform advanced data operations with Delta Lake and Delta Tables Automate data workflows using Databricks Repos, Workflows, REST API, and CLI Execute a comprehensive data engineering project from start to finish

Product Details

Country selected
Publication date : Jun 25, 2024
Length 17 hours 31 minutes
Edition : 1st Edition
Language : English
ISBN-13 : 9781836200437
Category :
Data
Languages :
Pyspark (Intermediate)
Concepts :
Data Engineering
Tools :
Azure Databricks (Intermediate)

What do you get with a video?

Product feature icon Download this video in MP4 format
Product feature icon Access this title in our online reader with advanced features
Product feature icon DRM FREE - Read whenever, wherever and however you want
Buy Now

Product Details

Publication date : Jun 25, 2024
Length 17 hours 31 minutes
Edition : 1st Edition
Language : English
ISBN-13 : 9781836200437
Category :
Data
Languages :
Pyspark (Intermediate)
Concepts :
Data Engineering
Tools :
Azure Databricks (Intermediate)

Table of Contents

12 Chapters
1. Before you start Chevron down icon Chevron up icon
Course Prerequisites
About the Course
How to access Course Material and Resources
Note for Students - Before Start
2. Introduction Chevron down icon Chevron up icon
Introduction to Data Engineering
Apache Spark to Data Engineering Platform
Introduction to Databricks Platform
3. Getting Started Chevron down icon Chevron up icon
What will you learn in this section
Creating Azure Cloud Account
Azure Portal Overview
Creating Databricks Workspace Service
Introduction to Databricks Workspace
Azure Databricks Platform Architecture
4. Working in Databricks Workspace Chevron down icon Chevron up icon
What will you learn in this section
How to create Spark Cluster
Working with Databricks Notebook
Notebook Magic Commands
Databricks Utilities Package
5. Working with Databricks File System - DBFS Chevron down icon Chevron up icon
What will you learn in this section
Introduction to DBFS
Working with DBFS Root
Mounting ADLS to DBFS
6. Working with Unity Catalog Chevron down icon Chevron up icon
What will you learn in this section
Introduction to Unity Catalog
Setup Unity Catalog
Unity Catalog User Provisioning
Working with Securable Objects
7. Working with Delta Lake and Delta Tables Chevron down icon Chevron up icon
What will you learn in this section
Introduction to Delta Lake
Creating Delta Table
Sharing data for External Delta Table
Reading Delta Table
Delta Table Operations
Delta Table Time Travel
Convert Parquet to Delta
Delta Table Schema Validation
Delta Table Schema Evolution
Look Inside Delta Table
Delta Table Utilities and Optimization
8. Working with Databricks Incremental Ingestion Tools Chevron down icon Chevron up icon
What will you learn in this section
Architecture and Need for Incremental Ingestion
Using Copy Into with Manual Schema Evolution
Using Copy Into with Automatic Schema Evolution
Streaming Ingestion with Manual Schema Evolution
Streaming Ingestion with Automatic Schema Evolution
Introduction to Databricks Autoloader
Autoloader with Automatic Schema Evolution
9. Working with Databricks Delta Live Tables (DLT) Chevron down icon Chevron up icon
What will you learn in this section
Introduction to Databricks DLT
Understand DLT Use Case Scenario
Setup DLT Scenario Dataset
Creating DLT Workload in SQL
Creating DLT Pipeline for your Workload
Creating DLT Workload in Python
10. Databricks Project and Automation Features Chevron down icon Chevron up icon
What will you learn in this section
Working with Databricks Repos
Working with Databricks Workflows
Working with Databricks Rest API
Working with Databricks CLI
11. Capstone Project Chevron down icon Chevron up icon
Project Scope and Background
Taking out the operational requirement
Storage Design
Implement Data Security
Implement Resource Policies
Decouple Data Ingestion
Design Bronze Layer
Design Silver and Gold Layer
Setup your environment
Create a workspace
Create and Storage Layer
Setup Unity Catalog
Create Metadata Catalog and External Locations
Setup your source control
Start Coding
Test your code
Load historical data
Ingest into bronze layer
Process the silver layer
Handling multiple updates
Implementing Gold Layer
Creating a run script
Preparing for Integration testing
Creating Test Data Producer
Creating Integration Test for Batch mode
Creating Integration Test for Stream mode
Implementing CI CD Pipeline
Develop Build Pipeline
Develop Release Pipeline
Creating Databricks CLI Script
12. Final Word Chevron down icon Chevron up icon
Congratulations

Recommendations for you

Left arrow icon
Building LLM Powered Applications
Building LLM Powered Applications
Read more
May 2024 342 pages
ebook
eBook
  • ebook eBook Mex$179.99
  • print Print Mex$1,025.99
Mex$179.99 Mex$820.99
Mex$1,025.99
Machine Learning with PyTorch and Scikit-Learn
Machine Learning with PyTorch and Scikit-Learn
Read more
Feb 2022 774 pages
Full star icon 5
ebook
eBook
  • ebook eBook Mex$179.99
  • print Print Mex$1,128.99
Mex$179.99 Mex$902.99
Mex$1,128.99
Transformers for Natural Language Processing and Computer Vision
Transformers for Natural Language Processing and Computer Vision
Read more
Feb 2024 728 pages
ebook
eBook
  • ebook eBook Mex$179.99
  • print Print Mex$1,128.99
Mex$179.99 Mex$902.99
Mex$1,128.99
The Definitive Guide to Power Query (M)
The Definitive Guide to Power Query (M)
Read more
Mar 2024 758 pages
ebook
eBook
  • ebook eBook Mex$902.99
  • print Print Mex$1,128.99
Mex$902.99
Mex$1,128.99
Building Smart Chatbots with LangChain
Building Smart Chatbots with LangChain
Read more
Jun 2024 4h 15m
video Video
Mex$179.99 Mex$6,133.99
The Machine Learning Solutions Architect Handbook
The Machine Learning Solutions Architect Handbook
Read more
Apr 2024 602 pages
ebook
eBook
  • ebook eBook Mex$179.99
  • print Print Mex$840.99
Mex$179.99 Mex$672.99
Mex$840.99
Microsoft Azure AI Fundamentals AI-900 Exam Guide
Microsoft Azure AI Fundamentals AI-900 Exam Guide
Read more
May 2024 288 pages
ebook
eBook
  • ebook eBook Mex$179.99
  • print Print Mex$922.99
Mex$179.99 Mex$738.99
Mex$922.99
Generative AI with LangChain
Generative AI with LangChain
Read more
Dec 2023 360 pages
Full star icon 3
ebook
eBook
  • ebook eBook Mex$179.99
  • print Print Mex$1,025.99
Mex$179.99 Mex$820.99
Mex$1,025.99
Mastering NLP from Foundations to LLMs
Mastering NLP from Foundations to LLMs
Read more
Apr 2024 340 pages
ebook
eBook
  • ebook eBook Mex$179.99
  • print Print Mex$1,086.99
Mex$179.99 Mex$869.99
Mex$1,086.99
Python Data Cleaning Cookbook
Python Data Cleaning Cookbook
Read more
May 2024 486 pages
ebook
eBook
  • ebook eBook Mex$179.99
  • print Print Mex$1,025.99
Mex$179.99 Mex$820.99
Mex$1,025.99
Right arrow icon

Customer reviews

Top Reviews
  • Top Reviews
  • Most Recent
Rating distribution
Empty star icon Empty star icon Empty star icon Empty star icon Empty star icon 0
(0 Ratings)
5 star 0%
4 star 0%
3 star 0%
2 star 0%
1 star 0%
Top Reviews
  • Top Reviews
  • Most Recent
No reviews found
Get free access to Packt library with over 7500+ books and video courses for 7 days!
Start Free Trial

FAQs

How can I download a video package for offline viewing? Chevron down icon Chevron up icon
  1. Login to your account at Packtpub.com.
  2. Click on "My Account" and then click on the "My Videos" tab to access your videos.
  3. Click on the "Download Now" link to start your video download.
How can I extract my video file? Chevron down icon Chevron up icon

All modern operating systems ship with ZIP file extraction built in. If you'd prefer to use a dedicated compression application, we've tested WinRAR / 7-Zip for Windows, Zipeg / iZip / UnRarX for Mac and 7-Zip / PeaZip for Linux. These applications support all extension files.

How can I get help and support around my video package? Chevron down icon Chevron up icon

If your video course doesn't give you what you were expecting, either because of functionality problems or because the content isn't up to scratch, please mail customercare@packt.com with details of the problem. In addition, so that we can best provide the support you need, please include the following information for our support team.

  1. Video
  2. Format watched (HTML, MP4, streaming)
  3. Chapter or section that issue relates to (if relevant)
  4. System being played on
  5. Browser used (if relevant)
  6. Details of support
Why can’t I download my video package? Chevron down icon Chevron up icon

In the even that you are having issues downloading your video package then please follow these instructions:

  1. Disable all your browser plugins and extensions: Some security and download manager extensions can cause issues during the download.
  2. Download the video course using a different browser: We've tested downloads operate correctly in current versions of Chrome, Firefox, Internet Explorer, and Safari.