Packt+ | Advance your knowledge in tech

You're reading from Apache Spark 2.x Machine Learning Cookbook Over 100 recipes to simplify machine learning model implementations with Spark

Product type Paperback

Published in Sep 2017

Publisher Packt

ISBN-13 9781783551606

Length 666 pages

Edition 1st Edition

Languages

Scala

Tools

Apache Spark

Concepts

Machine Learning

Authors (5):

Broderick Hall

Meenakshi Rajendran

Shuen Mei

Mohammed Guller

Siamak Amirghodsi

+1 more

View More author details

Table of Contents (14) Chapters

Preface

1. Practical Machine Learning with Spark Using Scala FREE CHAPTER

2. Just Enough Linear Algebra for Machine Learning with Spark

3. Spark's Three Data Musketeers for Machine Learning - Perfect Together

4. Common Recipes for Implementing a Robust Machine Learning System

5. Practical Machine Learning with Regression and Classification in Spark 2.0 - Part I

6. Practical Machine Learning with Regression and Classification in Spark 2.0 - Part II

7. Recommendation Engine that Scales with Spark

8. Unsupervised Clustering with Apache Spark 2.0

9. Optimization - Going Down the Hill with Gradient Descent

10. Building Machine Learning Systems with Decision Tree and Ensemble Models

11. Curse of High-Dimensionality in Big Data

12. Implementing Text Analytics with Spark 2.0 ML Library

13. Spark Streaming and Machine Learning Library

Exploring the movies data details for the recommendation system in Spark 2.0

In this recipe, we will begin to explore the data file by parsing data into a Scala case class and generating a simple metric. The key here is to acquire an understanding of our data, so in the later stages, if nebulous results arise, we will have some insight to make an informed conclusion about the correctness of our results.

This is the first of the two recipes which explore the movie dataset. Data exploration is an important first step in statistical analysis and machine learning.

One of the best ways to understand the data quickly is to generate a data visualization of it, and we will use JFreeChart to do that. It is very important to make sure you feel comfortable with the data and understand firsthand what is in each file, and the story it tries to tell.

We must always explore, understand, and visualize the data before we do anything else. Most performances and misses with ML and others systems can be traced...