Evaluating clusters

We defined machine learning as the design and study of systems that learn from experience to improve their performance of a task as measured by some metric. K-means is an unsupervised learning algorithm; there are no labels or ground truth to compare with the clusters. However, we can still evaluate the performance of the algorithm using intrinsic measures. We have already discussed measuring the distortions of clusters. In this section, we will discuss another performance measure for clustering called silhouette coefficient. The silhouette coefficient is a measure of compactness and separation of clusters. It increases as the quality of clusters increases; it is large for compact clusters that are far from each other and small for large, overlapping clusters. The silhouette coefficient is calculated per instance; for a set of instances, it is calculated as...