Visualizing the newsgroups data with t-SNE
We can answer these questions easily by visualizing those representation vectors. If we can see that the document vectors from the same topic form a cluster, we did a good job mapping the documents into vectors. But how? They are of 500 dimensions, while we can visualize data of, at most, three dimensions. We can resort to t-SNE for dimensionality reduction.
What is dimensionality reduction?
Dimensionality reduction is an important machine learning technique that reduces the number of features and, at the same time, retains as much information as possible. It is usually performed by obtaining a set of new principal features.
As mentioned before, it is difficult to visualize data of high dimensions. Given a three-dimensional plot, we sometimes don’t find it straightforward to observe any findings, not to mention 10, 100, or 1,000 dimensions. Moreover, some of the features in high-dimensional data may be correlated and, as...
