Search icon CANCEL
Subscription
0
Cart icon
Your Cart (0 item)
Close icon
You have no products in your basket yet
Arrow left icon
Explore Products
Best Sellers
New Releases
Books
Videos
Audiobooks
Learning Hub
Free Learning
Arrow right icon
Arrow up icon
GO TO TOP
Fast Data Processing with Spark 2

You're reading from   Fast Data Processing with Spark 2 Accelerate your data for rapid insight

Arrow left icon
Product type Paperback
Published in Oct 2016
Publisher Packt
ISBN-13 9781785889271
Length 274 pages
Edition 3rd Edition
Languages
Arrow right icon
Authors (2):
Arrow left icon
Krishna Sankar Krishna Sankar
Author Profile Icon Krishna Sankar
Krishna Sankar
Holden Karau Holden Karau
Author Profile Icon Holden Karau
Holden Karau
Arrow right icon
View More author details
Toc

Table of Contents (13) Chapters Close

Preface 1. Installing Spark and Setting Up Your Cluster 2. Using the Spark Shell FREE CHAPTER 3. Building and Running a Spark Application 4. Creating a SparkSession Object 5. Loading and Saving Data in Spark 6. Manipulating Your RDD 7. Spark 2.0 Concepts 8. Spark SQL 9. Foundations of Datasets/DataFrames – The Proverbial Workhorse for DataScientists 10. Spark with Big Data 11. Machine Learning with Spark ML Pipelines 12. GraphX

Structural APIs


Armed with the knowledge of how to create a graph and the knowledge of how to navigate the APIs, let's focus on the structural APIs. The code is available in graph-03.scala, graph-04.scala, and graph-05.scala files:

You can get the number of edges and vertices by numEdges and numVertices.

graph.numEdges 
graph.numVertices 
// 
val vertices = graph.vertices 
vertices.collect.foreach(println) 
// 
val edges = graph.edges 
edges.collect.foreach(println) 
// 
val triplets = graph.triplets 
triplets.take(3) 
triplets.map(t=>t.toString).collect().foreach(println) 

The triplets are interesting. They actually encompass an edge and two vertices that the edge connects to and all the user-defined objects in one neat object. They are very useful when we want to write algorithms:

Let's look at the code. It is not that complex:

val inDeg = graph.inDegrees // Followers 
inDeg.collect() 
val outDeg = graph.outDegrees...
lock icon The rest of the chapter is locked
Register for a free Packt account to unlock a world of extra content!
A free Packt account unlocks extra newsletters, articles, discounted offers, and much more. Start advancing your knowledge today.
Unlock this book and the full library FREE for 7 days
Get unlimited access to 7000+ expert-authored eBooks and videos courses covering every tech area you can think of
Renews at $19.99/month. Cancel anytime
Banner background image