Validating the code, data, and models
When implementing a CI/CD pipeline, you need to make sure you have all the necessary tests in place to deploy your newly created code with ease and confidence. Once you are running a CI or CI/CD pipeline, the power of automated tests will become immediately visible. It not only helps you to detect failures in your code, but it also helps to detect future issues in the whole ML process, including the environment setup, build dependencies, data requirements, model initialization, optimization, resource requirements, and deployment.
When implementing a validation pipeline for our ML process, we can take inspiration from traditional software development principles (for example, unit testing, integration testing, and end-to-end testing). We can translate these techniques directly to steps during the ML process, such as input data, models, and the application code of the scoring service. Let's understand how we can adapt these testing techniques...