Advantages and limitations

The R language has long been the lingua franca of data scientists. Its simple-to-understand DataFrame abstraction, expressive APIs, and vibrant package ecosystem are exactly what the analysts needed. The main challenge was with the scalability. SparkR bridges that gap by providing distributed in-memory DataFrames without leaving the R eco-system. Such a symbiotic relationship allows users to gain the following benefits:

• There is no need for the analyst to learn a new language

• The SparkR APIs are similar to R APIs

• You can access SparkR from R studio, along with the autocomplete feature

• Performing interactive, exploratory analysis of a very large dataset is no longer hindered by memory limitations or long turnaround times

• Accessing data from different types of data sources becomes a lot easier. Most of the tasks which were imperative before have become declarative. Check Chapter 4, Unified Data Access, to learn more

• You can freely mix dplyr such as Spark functions, SQL...