Tracking machine learning experiments
Keeping track of our machine learning experiments will help us reduce the risks of invalid conclusions and selecting unreliable models. Experiment tracking in machine learning is about saving the information about the experiments – for instance, the data that has been used – the testing performance and the metric used for performance assessment, and the algorithms and the hyperparameters used for modeling. Here are some of the important considerations for using a machine learning experiment tracking tool:
- Can you integrate the tool with your continuous integration/continuous development (CI/CD) pipeline and machine learning modeling frameworks?
- Can you reproduce your experiments?
- Can you easily search through the experiments to find the best models or models with bad or unexpected behaviors?
- Does it cause any security or privacy issues?
- Does the tool help you better collaborate in your machine learning projects...