9. Unsupervised clustering using continuous random variables in Keras

In the unsupervised classification of MNIST digits, we used IIC since the MI can be computed using discrete joint and marginal distributions. We obtained good accuracy with a linear assignment algorithm.

In this section, we will attempt to use MINE to perform clustering. We'll use the same key ideas from IIC: from a pair of images and their transformed versions , maximize the MI of the corresponding encoded latent vectors . By maximizing the MI, we perform clustering of the encoded latent vectors. The difference with MINE is that the encoded latent vectors are continuous and not in one-hot vector format, as used in IIC. Since the output of clustering is not in one-hot vector format, we will use a linear classifier. A linear classifier is an MLP without a non-linear activation layer such as ReLU. A linear classifier is used as an alternative to the linear assignment algorithm in the case of...