Search icon CANCEL
Subscription
0
Cart icon
Your Cart (0 item)
Close icon
You have no products in your basket yet
Save more on your purchases now! discount-offer-chevron-icon
Savings automatically calculated. No voucher code required.
Arrow left icon
Explore Products
Best Sellers
New Releases
Books
Videos
Audiobooks
Learning Hub
Conferences
Free Learning
Arrow right icon
Arrow up icon
GO TO TOP
Data Lakehouse in Action

You're reading from   Data Lakehouse in Action Architecting a modern and scalable data analytics platform

Arrow left icon
Product type Paperback
Published in Mar 2022
Publisher Packt
ISBN-13 9781801815932
Length 206 pages
Edition 1st Edition
Languages
Tools
Concepts
Arrow right icon
Author (1):
Arrow left icon
Pradeep Menon Pradeep Menon
Author Profile Icon Pradeep Menon
Pradeep Menon
Arrow right icon
View More author details
Toc

Table of Contents (14) Chapters Close

Preface 1. PART 1: Architectural Patterns for Analytics
2. Chapter 1: Introducing the Evolution of Data Analytics Patterns FREE CHAPTER 3. Chapter 2: The Data Lakehouse Architecture Overview 4. PART 2: Data Lakehouse Component Deep Dive
5. Chapter 3: Ingesting and Processing Data in a Data Lakehouse 6. Chapter 4: Storing and Serving Data in a Data Lakehouse 7. Chapter 5: Deriving Insights from a Data Lakehouse 8. Chapter 6: Applying Data Governance in the Data Lakehouse 9. Chapter 7: Applying Data Security in a Data Lakehouse 10. PART 3: Implementing and Governing a Data Lakehouse
11. Chapter 8: Implementing a Data Lakehouse on Microsoft Azure 12. Chapter 9: Scaling the Data Lakehouse Architecture 13. Other Books You May Enjoy

Ingesting and processing streaming data

The following diagram depicts the components required for stream data ingestion and processing:

Figure 3.7 – The streaming data ingestion and processing pattern

Now, let's discuss how to stream data processing through the lens of the ELTL process.

Streaming data sources

Streaming data is a data source that continuously emanates data. Social media feeds, IoT devices, and event-driven processes such as swiping a credit card are examples of streaming data. The data is continually produced, and the goal of stream processing is to tap into that stream of data and gain insights as quickly as possible. Stream data ingestion and processing facilitate real-time analytics. This implies that analytics is performed on the data without the data being persisted on disk.

Extraction-load

Stream data is extracted using an event publishing-subscribing service. An event publishing-subscribing service enables creating...

lock icon The rest of the chapter is locked
Register for a free Packt account to unlock a world of extra content!
A free Packt account unlocks extra newsletters, articles, discounted offers, and much more. Start advancing your knowledge today.
Unlock this book and the full library FREE for 7 days
Get unlimited access to 7000+ expert-authored eBooks and videos courses covering every tech area you can think of
Renews at $19.99/month. Cancel anytime