Bayesian deep learning

We now have a whole set of models that can make forecasts on time series. But are the point estimates that these models give sensible estimates or just random guesses? How certain is the model? Most classic probabilistic modeling techniques, such as Kalman filters, can give confidence intervals for predictions, whereas regular deep learning cannot do this. The field of Bayesian deep learning combines Bayesian approaches with deep learning to enable models to express uncertainty.

The key idea in Bayesian deep learning is that there is inherent uncertainty in the model. Sometimes this is done by learning a mean and standard deviation for weights instead of just a single weight value. However, this approach increases the number of parameters required, so it did not catch on. A simpler hack that allows us to turn regular deep networks into Bayesian deep networks is to activate dropout during prediction time and then make multiple predictions.

In this section, we will be...