Linear regression

Linear regression involves a little more work than statistics. We need the data in a vector form along with a few more parameters; such as the learning rate, that is, the step size. We will also split the Dataset into training and test, as shown in the later part of this chapter.

Data transformation and feature extraction

The ml.feature library has a class vector assembler that transforms the data into a vector of features:

    // 
    // Linear Regression 
    // 
    // Transformation to a labeled data that Linear Regression Can use            
val cars1 = cars.na.drop()        
val assembler = new VectorAssembler() 
assembler.setInputCols(Array("displacement","hp","torque","CRatio","RARatio","CarbBarrells","NoOfSpeed","length","width","weight","automatic")) 
assembler.setOutputCol("features") 
val cars2 = assembler.transform(cars1) 
cars2.show(40) 

The result is a Dataset with a new column features, which contains vectorized...