Summary

In this chapter, we've learned about graphs: a useful abstraction to model a huge variety of phenomena. We started the chapter using the Clojure library Loom to visualize and traverse small graphs of Twitter followers. We learned about two different methods of graph traversal, depth-first and breadth-first search, and the effect of changing edge weights on the paths discovered by Dijkstra's algorithm and Prim's algorithm. We also looked at the density of the whole graph and plotted the degree distributions to observe the difference between random and scale-free graphs.

We introduced GraphX and the Clojure library Glittering as a means of processing large graphs in a scalable way using Spark. In addition to providing several built-in graph algorithms, Glittering also exposes GraphX's Pregel API: a set of three symbiotic functions to express graph algorithms in a vertex-centric way. We showed that this alternative model of computation could be used to express triangle...