Creating DenseVector and setup with Spark 2.0
In this recipe, we explore DenseVectors
using the Spark 2.0 library.
Spark provides two types of vector facilities (dense and sparse) for storing and manipulating feature vectors that are going to be used in learning or optimization algorithms.
How to do it...
- In this section, we examine
DenseVector
examples that you would most likely use for implementing/augmenting existing machine learning programs. These examples also help to better understand Spark ML or MLlib source code and the underlying implementation (for example, Single Value Decomposition). - Here we look at creating an ML vector feature (with independent variables) from arrays, which is a common use case. In this case, we have three almost fully populated Scala arrays corresponding to customer and product feature sets. We convert these arrays to the corresponding
DenseVectors
in Scala:
val CustomerFeatures1: Array[Double] = Array(1,3,5,7,9,1,3,2,4,5,6,1,2,5,3,7,4,3,4,1) val CustomerFeatures2...