Unsupervised Learning Using Mutual Information

Many machine learning tasks such as classification, detection, and segmentation are dependent on labeled data. The performance of a network on these tasks is directly affected by the quality of labeling and the amount of data. The problem is that producing a sufficient amount of good-quality annotated data is costly and time-consuming.

To continue the progress of development in machine learning, new algorithms should be less dependent on human labelers. Ideally, a network should learn from unlabeled data, which is abundant due to the growth of the internet and the popularity of sensing devices such as smartphones and the Internet of Things (IoT). Learning from unlabeled data is a field of unsupervised learning. In some cases, unsupervised learning is also called self-supervised learning to emphasize the use of pure unlabeled data for training and the absence of human supervision. In this text, we will use the term...