The Livy REST job server and Hue Notebooks
Hadoop User Experience (Hue) introduced Spark-based notebooks, which are inspired by IPython Notebooks. The Spark Notebook works on the Livy REST job server backend to provide Spark as a service to end users.
Hue Notebooks are similar to IPython Notebooks or Zeppelin Notebooks, which are integrated well with Hadoop ecosystem components. Notebooks support Scala, Python, and R languages, and Hive and Impala queries. They can be shared with import/export and sharing features. Visualizations can be done with a variety of charts and graphs. Hue supports a multiuser environment with impersonation.
Livy is a Spark REST job server that can submit and interact with Spark jobs from anywhere. It is inspired by the popular IPython/Jupyter and Spark REST job server, but implemented to better integrate the Hadoop ecosystem with multiple users. With the Livy server, Spark can be offered as a service to users in the following two ways:
Instead of every user creating...
