Evaluating Forecasts – Validation Strategies

Throughout the last few chapters, we have been looking at a few relevant, but seldom discussed, aspects of time series forecasting. While we learned about different forecasting metrics in the previous chapter, we now move on to the final piece of the puzzle – validation strategies. This is another integral part of evaluating forecasts.

In this chapter, we try to answer the question How do we choose the validation strategy to evaluate models from a time series forecasting perspective? We will look at different strategies and their merits and demerits so that by the end of the chapter, you can make an informed decision to set up the validation strategy for your time series problem.

In this chapter, we will be covering these main topics:

• Model validation
• Holdout strategies
• Cross-validation strategies
• Choosing a validation strategy
• Validation strategies for datasets with multiple time series

...