Understanding TensorFlow probability, variational inference, and Monte Carlo methods

TensorFlow Probability (tfp in code – https://www.tensorflow.org/probability/overview#layer_2_model_building) was recently released by Google to perform probabilistic reasoning in a scalable manner. It provides tools and functionalities to define distributions, build neural networks with prior on weights, and perform probabilistic inference tasks such as Monte Carlo or Variational Inference.

Let's take a look at some of the functions/utilities we will be using for building our model:

• Tfp.distributions.categorical: This is a standard categorical distribution that's characterized by probabilities or log-probabilities over K classes. In this project, we have Traffic Sign images from 43 different traffic signs. We will define a categorical distribution over 43 classes in this project.
• Probabilistic layers: Built on top of the TensorFlow layers implementation, probabilistic layers incorporate uncertainty over the...