Analyzing the logs
For every operation it carries out, Amazon ML gives us access to the related logs. We can download and analyze the model training logs and infer a few things on how Amazon ML trains and selects the best model.
Go back to the last Titanic model, and in the summary part, click on the Download Log link. The log file is too long to be reproduced here but is available at https://github.com/alexperrier/packt-aml/blob/master/ch5/titanic_training.log:
Amazon ML launches five versions of the SGD algorithm in parallel. Each version is called a learner and corresponds to a different value for the learning rate: 0.01, 0.1,1, 10, and 100. The following five metrics are calculated at each new pass of the algorithm:
- Accuracy
- Recall
- Precision
- F1-score
- AUC
The negative-log-likelihood is also calculated to assess whether the last iterations have brought significant improvement in reducing the residual error.
Optimizing the learning rate
If you recall from Chapter 2, Machine Learning Definitions and...
