Real-Time Stream Processing Using Apache Spark 3 for Python Developers: Learn to create real-time stream processing applications using Apache Spark

ScholarNest

€93.99

Video Feb 2022 4hrs 34mins 1st Edition

ScholarNest

€93.99

Video Feb 2022 4hrs 34mins 1st Edition

What do you get with a video?

Download this video in MP4 format

Access this title in our online reader with advanced features

DRM FREE - Read whenever, wherever and however you want

View table of contents

Download Code

Key benefits

Learn real-time stream processing concepts
Understand Spark structured streaming APIs and architecture
Work with file streams, Kafka source, and integrating Spark with Kafka

Description

Take your first steps towards discovering, learning, and using Apache Spark 3.0. We will be taking a live coding approach in this carefully structured course and explaining all the core concepts needed along the way. In this course, we will understand the real-time stream processing concepts, Spark structured streaming APIs, and architecture. We will work with file streams, Kafka source, and integrating Spark with Kafka. Next, we will learn about state-less and state-full streaming transformations. Then cover windowing aggregates using Spark stream. Next, we will cover watermarking and state cleanup. After that, we will cover streaming joins and aggregation, handling memory problems with streaming joins. Finally, learn to create arbitrary streaming sinks. By the end of this course, you will be able to create real-time stream processing applications using Apache Spark. All the resources for the course are available at https://github.com/PacktPublishing/Real-time-stream-processing-using-Apache-Spark-3-for-Python-developers

Who is this book for?

This course is designed for software engineers and architects who are willing to design and develop big data engineering projects using Apache Spark. It is also designed for programmers and developers who are aspiring to grow and learn data engineering using Apache Spark. For this course, you need to know Spark fundamentals and should be exposed to Spark Dataframe APIs. Also, you should know Kafka fundamentals and have a working knowledge of Apache Kafka. One should also have programming knowledge of Python programming.