LabeledPoint data structure for Spark ML
LabeledPoint is a data structure that has been since the early days for a feature vector along with a label so it can be used in unsupervised learning algorithms. We demonstrate a short recipe that uses LabeledPoint, the Seq data structure, and DataFrame to run a logistic regression for binary classification of the data. The emphasis here is on LabeledPoint, and the regression algorithms are covered in more depth in Chapter 5, Practical Machine Learning with Regression and Classification in Spark 2.0 - Part I and Chapter 6, Practical Machine Learning with Regression and Classification in Spark 2.0 - Part II.
How to do it...
- Start a new project in IntelliJ or in an IDE of your choice. Make sure that the necessary JAR files are included.
- Set up the package location where the program will reside:
package spark.ml.cookbook.chapter4- Import the necessary packages for SparkContext to get access to the cluster:
import org.apache.spark.ml.feature.LabeledPoint...
