Results explanation

After we have passed our model evaluation stage and decided to select the estimated and evaluated model as our final model, our next task is to interpret results to the university leaders and technicians.

In terms of explaining the machine learning results, the university is particularly interested in, firstly, understanding how their designed interventions affect student attrition, and, secondly, among the common reasons of finances, academic performance, social/emotional encouragement, and personal adjustment, which has the biggest impact.

We will work on results explanation with our focus on big influencing variables in the following sections.

Calculating the impact of interventions

The following summarizes some of the result samples briefly, for which we can use some functions from randomForest and decision tree to produce.

With Spark 1.5, you can use the following code to obtain a vector of feature importance:

val importances: Vector = model.featureImportances

With the...