Search icon CANCEL
Arrow left icon
Explore Products
Best Sellers
New Releases
Books
Videos
Audiobooks
Learning Hub
Conferences
Free Learning
Arrow right icon
Arrow up icon
GO TO TOP
Azure Data and AI Architect Handbook

You're reading from   Azure Data and AI Architect Handbook Adopt a structured approach to designing data and AI solutions at scale on Microsoft Azure

Arrow left icon
Product type Paperback
Published in Jul 2023
Publisher Packt
ISBN-13 9781803234861
Length 284 pages
Edition 1st Edition
Tools
Arrow right icon
Authors (2):
Arrow left icon
Olivier Mertens Olivier Mertens
Author Profile Icon Olivier Mertens
Olivier Mertens
Breght Van Baelen Breght Van Baelen
Author Profile Icon Breght Van Baelen
Breght Van Baelen
Arrow right icon
View More author details
Toc

Table of Contents (18) Chapters Close

Preface 1. Part 1: Introduction to Azure Data Architect
2. Chapter 1: Introduction to Data Architectures FREE CHAPTER 3. Chapter 2: Preparing for Cloud Adoption 4. Part 2: Data Engineering on Azure
5. Chapter 3: Ingesting Data into the Cloud 6. Chapter 4: Transforming Data on Azure 7. Chapter 5: Storing Data for Consumption 8. Part 3: Data Warehousing and Analytics
9. Chapter 6: Data Warehousing 10. Chapter 7: The Semantic Layer 11. Chapter 8: Visualizing Data Using Power BI 12. Chapter 9: Advanced Analytics Using AI 13. Part 4: Data Security, Governance, and Compliance
14. Chapter 10: Enterprise-Level Data Governance and Compliance 15. Chapter 11: Introduction to Data Security 16. Index 17. Other Books You May Enjoy

Batch ingestion architectures

The simplest form of ingestion architecture is a use case where data is only ingested in batches from other cloud-based sources (no sources residing on-premises). In this case, we will use data pipelines to periodically fetch large amounts of data and write them to the bronze layer in the data lake. Note that we restrain from performing any kind of transformation in this initial pipeline.

We will look at ingesting data from the following sources:

  • Cloud sources
  • On-premises sources

Let’s first look at how to ingest data from cloud-based sources.

Ingesting data from cloud sources

When ingesting data from other cloud sources, the connection is often more convenient, Also, we can make use of Azure-hosted integration runtimes (IRs). This will serve as the compute for the pipeline orchestration in either Azure Data Factory or Azure Synapse pipelines. Other Data Factory components will be more elaborately discussed in the next...

lock icon The rest of the chapter is locked
Register for a free Packt account to unlock a world of extra content!
A free Packt account unlocks extra newsletters, articles, discounted offers, and much more. Start advancing your knowledge today.
Unlock this book and the full library FREE for 7 days
Get unlimited access to 7000+ expert-authored eBooks and videos courses covering every tech area you can think of
Renews at €18.99/month. Cancel anytime