Packt+ | Advance your knowledge in tech

You're reading from Fast Data Processing with Spark 2 Accelerate your data for rapid insight

Product type Paperback

Published in Oct 2016

Publisher Packt

ISBN-13 9781785889271

Length 274 pages

Edition 3rd Edition

Languages

Scala

Tools

Apache Spark

Concepts

Data Processing

Authors (2):

Krishna Sankar

Holden Karau

View More author details

Table of Contents (13) Chapters

Preface

What this book covers
What you need for this book
Who this book is for
Conventions
Reader feedback
Customer support

1. Installing Spark and Setting Up Your Cluster

Directory organization and convention
Installing the prebuilt distribution
Building Spark from source
Spark topology
A single machine
Running Spark on EC2
Deploying Spark with Chef (Opscode)
Deploying Spark on Mesos
Spark on YARN
Spark standalone mode
References
Summary

2. Using the Spark Shell FREE CHAPTER

The Spark shell
Loading a simple text file
Interactively loading data from S3
Summary

3. Building and Running a Spark Application

Building Spark applications
Data wrangling with iPython
Developing Spark with Eclipse
Developing Spark with other IDEs
Building your Spark job with Maven
Building your Spark job with something else
References
Summary

4. Creating a SparkSession Object

SparkSession versus SparkContext
Building a SparkSession object
SparkContext - metadata
Shared Java and Scala APIs
Python
iPython
Reference
Summary

5. Loading and Saving Data in Spark

Spark abstractions
Data modalities
Data modalities and Datasets/DataFrames/RDDs
Loading data into an RDD
Saving your data
References
Summary

6. Manipulating Your RDD

Manipulating your RDD in Scala and Java
Manipulating your RDD in Python
References
Summary

7. Spark 2.0 Concepts

Code and Datasets for the rest of the book
The data scientist and Spark features
Spark v2.0 and beyond
Apache Spark - evolution
Apache Spark - the full stack
The art of a big data store - Parquet
References
Summary

8. Spark SQL

The Spark SQL architecture
Spark SQL how-to in a nutshell
Spark SQL programming
References
Summary

9. Foundations of Datasets/DataFrames – The Proverbial Workhorse for DataScientists

Datasets - a quick introduction
Dataset APIs - an overview
Dataset interfaces and functions
References
Summary

10. Spark with Big Data

Parquet - an efficient and interoperable big data format
HBase
Reference
Summary

11. Machine Learning with Spark ML Pipelines

Spark's machine learning algorithm table
Spark machine learning APIs - ML pipelines and MLlib
ML pipelines
Spark ML examples
The API organization
Basic statistics
Linear regression
Classification
Clustering
Recommendation
Hyper parameters
The final thing
References
Summary

12. GraphX

Graphs and graph processing - an introduction
Spark GraphX
GraphX - computational model
The first example - graph
Building graphs
The GraphX API landscape
Structural APIs
Community, affiliation, and strengths
Algorithms
Partition strategy
Case study - AlphaGo tweets analytics
References
Summary

References

Some references are as follows:

http://www.scala-lang.org/api/current/index.html#scala.collection.immutable.List
http://spark.apache.org/docs/latest/api/scala/index.html#org.apache.spark.api.java.JavaRDD
http://spark.apache.org/docs/latest/api/scala/index.html#org.apache.spark.api.java.JavaPairRDD
http://spark.apache.org/docs/latest/api/scala/index.html#org.apache.spark.api.java.JavaDoubleRDD
https://spark.apache.org/docs/latest/api/scala/index.html#org.apache.spark.SparkContext
http://abshinn.github.io/python/apache-spark/2014/10/11/using-combinebykey-in-apache-spark/
Good examples of RDD transformations: (https://github.com/JerryLead/SparkLearning/tree/master/src)

The rest of the chapter is locked

You're reading from Fast Data Processing with Spark 2 Accelerate your data for rapid insight

Table of Contents (13) Chapters

References

Unlock this book and the full library FREE for 7 days

Authors (2)

Personalised recommendations for you