Performing matrix arithmetic using Spark 2.0
In this recipe, we explore matrix such as addition, transpose, and in Spark. The more complex operations such as inverse, SVD, and so on, will be covered in future sections. The native sparse and dense matrices for the Spark ML library provide multiplication operators so there is no need to convert to Breeze
explicitly.
Matrices are the workhorses of distributed computing. ML features that are collected can be arranged in a matrix configuration and operated at scale. Many of the ML methods such as ALS (Alternating Least Square) and SVD (Singular Value Decomposition) rely on efficient matrix and vector operations to achieve large-scale machine learning and training.
How to do it...
- Start a new project in IntelliJ or in an IDE of your choice. Make sure that the necessary JAR files are included.
- Import the necessary packages for vector and matrix manipulation:
import org.apache.spark.mllib.linalg.distributed.RowMatrix import org.apache.spark.mllib...