Building evaluation metrics for the PredictionIO systems

Let's discuss the methods for performance improvement in general for machine learning systems. The following diagram assumes that we are using the classification algorithm and walks us through the process followed in machine learning systems to evaluate a model. A true machine learning system should use these three sets of data for this purpose:

• Training dataset
• Testing dataset
• Validation dataset

Let's discuss the process of tuning the machine learning accuracy step by step:

1. We will split the dataset into Training Data and Testing Data. The following diagram demonstrates this concept:

2. Then, we will build a model on the training set. The following diagram shows the process of training and building a model with Training Data and Testing Data:

3. Perform an evaluation on the training set, and retrain if needed...