Search icon CANCEL
Subscription
0
Cart icon
Your Cart (0 item)
Close icon
You have no products in your basket yet
Arrow left icon
Explore Products
Best Sellers
New Releases
Books
Videos
Audiobooks
Learning Hub
Conferences
Free Learning
Arrow right icon
Learning PySpark
Learning PySpark

Learning PySpark: Data Processing with Python and Apache Spark

Arrow left icon
Profile Icon Tomasz Drabas
Arrow right icon
AU$180.99
Video Feb 2018 2hrs 28mins 1st Edition
Video
AU$180.99
Subscription
Free Trial
Renews at AU$24.99p/m
Arrow left icon
Profile Icon Tomasz Drabas
Arrow right icon
AU$180.99
Video Feb 2018 2hrs 28mins 1st Edition
Video
AU$180.99
Subscription
Free Trial
Renews at AU$24.99p/m
Video
AU$180.99
Subscription
Free Trial
Renews at AU$24.99p/m

What do you get with a video?

Product feature icon Download this video in MP4 format
Product feature icon Access this title in our online reader with advanced features
Product feature icon DRM FREE - Read whenever, wherever and however you want
OR
Modal Close icon
Payment Processing...
tick Completed

Billing Address

Key benefits

  • Practical techniques to help you combine the power of Python and Apache Spark to process your data efficiently
  • Overcome any challenge when it comes to developing and deploying efficient, scalable, real-time Spark solutions
  • Take your understanding of using Spark with Python to the next level with this hands-on video

Description

Apache Spark is an open-source distributed engine for querying and processing data. In this tutorial, we provide a brief overview of Spark and its stack. This tutorial presents effective, time-saving techniques on how to leverage the power of Python and put it to use in the Spark ecosystem. You will start by getting a firm understanding of the Apache Spark architecture and how to set up a Python environment for Spark. You'll learn about different techniques for collecting data, and distinguish between (and understand) techniques for processing data. Next, we provide an in-depth review of RDDs and contrast them with DataFrames. We provide examples of how to read data from files and from HDFS and how to specify schemas using reflection or programmatically (in the case of DataFrames). The concept of lazy execution is described and we outline various transformations and actions specific to RDDs and DataFrames. Finally, we show you how to use SQL to interact with DataFrames. By the end of this tutorial, you will have learned how to process data using Spark DataFrames and mastered data collection techniques by distributed data processing.

Who is this book for?

If you are a Python developer keen to master hands-on techniques using the Apache Spark 2.x ecosystem in the best possible manner, this video is for you. A firm understanding of Python is expected to get the best out of the tutorial. Familiarity with Spark would also be helpful.

What you will learn

  • Learn about Apache Spark and the Spark 2.0 architecture.
  • Understand schemas for RDD, lazy executions, and transformations.
  • Explore the sorting and saving elements of RDD.
  • Build and interact with Spark DataFrames using Spark SQL
  • Create and explore various APIs to work with Spark DataFrames.
  • Learn how to change the schema of a DataFrame programmatically.
  • Explore how to aggregate, transform, and sort data with DataFrames.

Product Details

Country selected
Publication date, Length, Edition, Language, ISBN-13
Publication date : Feb 27, 2018
Length: 2hrs 28mins
Edition : 1st
Language : English
ISBN-13 : 9781788396592
Category :
Languages :

What do you get with a video?

Product feature icon Download this video in MP4 format
Product feature icon Access this title in our online reader with advanced features
Product feature icon DRM FREE - Read whenever, wherever and however you want
OR
Modal Close icon
Payment Processing...
tick Completed

Billing Address

Product Details

Publication date : Feb 27, 2018
Length: 2hrs 28mins
Edition : 1st
Language : English
ISBN-13 : 9781788396592
Category :
Languages :

Packt Subscriptions

See our plans and pricing
Modal Close icon
AU$24.99 billed monthly
Feature tick icon Unlimited access to Packt's library of 7,000+ practical books and videos
Feature tick icon Constantly refreshed with 50+ new titles a month
Feature tick icon Exclusive Early access to books as they're written
Feature tick icon Solve problems while you work with advanced search and reference features
Feature tick icon Offline reading on the mobile app
Feature tick icon Simple pricing, no contract
AU$249.99 billed annually
Feature tick icon Unlimited access to Packt's library of 7,000+ practical books and videos
Feature tick icon Constantly refreshed with 50+ new titles a month
Feature tick icon Exclusive Early access to books as they're written
Feature tick icon Solve problems while you work with advanced search and reference features
Feature tick icon Offline reading on the mobile app
Feature tick icon Choose a DRM-free eBook or Video every month to keep
Feature tick icon PLUS own as many other DRM-free eBooks or Videos as you like for just AU$5 each
Feature tick icon Exclusive print discounts
AU$349.99 billed in 18 months
Feature tick icon Unlimited access to Packt's library of 7,000+ practical books and videos
Feature tick icon Constantly refreshed with 50+ new titles a month
Feature tick icon Exclusive Early access to books as they're written
Feature tick icon Solve problems while you work with advanced search and reference features
Feature tick icon Offline reading on the mobile app
Feature tick icon Choose a DRM-free eBook or Video every month to keep
Feature tick icon PLUS own as many other DRM-free eBooks or Videos as you like for just AU$5 each
Feature tick icon Exclusive print discounts

Frequently bought together


Stars icon
Total AU$ 309.97
Learning PySpark
AU$180.99
Learning PySpark
AU$67.99
PySpark Cookbook
AU$60.99
Total AU$ 309.97 Stars icon

Table of Contents

5 Chapters
A Brief Primer on PySpark Chevron down icon Chevron up icon
Resilient Distributed Datasets Chevron down icon Chevron up icon
Resilient Distributed Datasets and Actions Chevron down icon Chevron up icon
DataFrames and Transformations Chevron down icon Chevron up icon
Data Processing with Spark DataFrames Chevron down icon Chevron up icon
Get free access to Packt library with over 7500+ books and video courses for 7 days!
Start Free Trial

FAQs

How can I download a video package for offline viewing? Chevron down icon Chevron up icon
  1. Login to your account at Packtpub.com.
  2. Click on "My Account" and then click on the "My Videos" tab to access your videos.
  3. Click on the "Download Now" link to start your video download.
How can I extract my video file? Chevron down icon Chevron up icon

All modern operating systems ship with ZIP file extraction built in. If you'd prefer to use a dedicated compression application, we've tested WinRAR / 7-Zip for Windows, Zipeg / iZip / UnRarX for Mac and 7-Zip / PeaZip for Linux. These applications support all extension files.

How can I get help and support around my video package? Chevron down icon Chevron up icon

If your video course doesn't give you what you were expecting, either because of functionality problems or because the content isn't up to scratch, please mail customercare@packt.com with details of the problem. In addition, so that we can best provide the support you need, please include the following information for our support team.

  1. Video
  2. Format watched (HTML, MP4, streaming)
  3. Chapter or section that issue relates to (if relevant)
  4. System being played on
  5. Browser used (if relevant)
  6. Details of support
Why can’t I download my video package? Chevron down icon Chevron up icon

In the even that you are having issues downloading your video package then please follow these instructions:

  1. Disable all your browser plugins and extensions: Some security and download manager extensions can cause issues during the download.
  2. Download the video course using a different browser: We've tested downloads operate correctly in current versions of Chrome, Firefox, Internet Explorer, and Safari.