Summary
In this chapter we introduced you to the Apache Spark engine for fast Big Data processing. We explained how to launch a multi-node HDInsight cluster with Hadoop, Spark, and the Hive database installed and how to connect all these resources to RStudio Server.
We then used Bay Area Bike Share open data to guide you through the numerous functions of the SparkR package for data management, transformations, and analysis on data stored in Hive tables directly from the R console.
In Chapter 8, Machine Learning Methods for Big Data in R, we will explore another powerful dimension of Big Data analytics using R language: we will apply a variety of predictive analytics algorithms to large-scale data sets using the H2O platform for distributed machine learning of Big Data.
