Model estimation
Once the feature sets get finalized in our last section, what follows is the estimation of parameters of the selected models, for which we will use MLlib. As earlier, we need to arrange distributed computing, especially for this case with various cars for various customer segments.
As MLlib is a built-in package for Apache Spark, the computation is a straightforward process for which the readers may consult Chapter 1, Spark for Machine Learning
One of the main reasons for our client utilizing Apache Spark is to take advantage of its computation speed and the ease of implementing parallel computing. For this project, as we need to build models for more than 40 products and many customer segments, we will perform machine learning only against segments by age.
For updated information about implementing parallel computing with Spark and especially about submitting and monitoring application jobs, users should always consult Apache Spark's updated and detailed guidelines at http...
