8. Estimating the Mutual Information of a bivariate Gaussian

In this section, we validate MINE on a bivariate Gaussian distribution. Figure 13.8.1 shows a bivariate Gaussian distribution with mean and covariance:

(Equation 13.8.1)

(Equation 13.8.2)

Figure 13.8.1 A two dimensional Gaussian distribution with mean and covariance as shown in Equation 13.8.1 and Equation 13.8.2

Our goal is to estimate MI by approximating Equation 13.1.3. The approximation can be done by obtaining a huge number of samples (such as 1 million) and creating a histogram with a large number of bins (such as 100 bins). Listing 13.8.1 shows the manual computation of the MI of a bivariate Gaussian distribution using binning.

Listing 13.8.1: mine-13.8.1.py:

def sample(joint=True,
           mean=[0, 0],
           cov=[[1, 0.5], [0.5, 1]],
           n_data=1000000):
    """Helper function to obtain samples 
        fr a bivariate Gaussian distribution

...