Chapter 2: Hierarchical Clustering

Activity 2: Applying Linkage Criteria

Solution:

1. Visualize the x dataset that we created in Exercise 7, Building a Hierarchy:

   from scipy.cluster.hierarchy import linkage, dendrogram, fcluster
   from sklearn.datasets import make_blobs
   import matplotlib.pyplot as plt
   %matplotlib inline
   # Generate a random cluster dataset to experiment on. X = coordinate points, y = cluster labels (not needed)
   X, y = make_blobs(n_samples=1000, centers=8, n_features=2, random_state=800)
   # Visualize the data
   plt.scatter(X[:,0], X[:,1])
   plt.show()

   The output is as follows:

   

   Figure 2.20: A scatter plot of the generated cluster dataset

2. Create a list with all the possible linkage method hyperparameters:

   methods = ['centroid', 'single', 'complete', 'average', 'weighted']

3. Loop through each of the methods in the list that you just created and display the effect that they have on the same dataset:

   for method in methods:
       distances = linkage(X, method=method, metric="euclidean")
       clusters = fcluster(distances, 3, criterion="distance") 
       plt.title('linkage: ' + method)
       plt.scatter(X[:,0], X[:,1], c=clusters, cmap='tab20b')
       plt.show()

   The output is as follows:

   

   Figure 2.21: A scatter plot for all the methods

   As you can see from the preceding plots, by simply changing the linkage criteria, you can dramatically change the efficacy of your clustering. In this dataset, centroid and average linkage work best at finding discrete clusters that make sense. This is clear from the fact that we generated a dataset of eight clusters, and centroid and average linkage are the only ones that show the clusters that are represented using eight different colors. The other linkage types fall short – most noticeably, single linkage.

Analysis:

Activity 3: Comparing k-means with Hierarchical Clustering

Solution:

1. Import the necessary packages from scikit-learn (KMeans, AgglomerativeClustering, and silhouette_score), as follows:

   from sklearn.cluster import KMeans
   from sklearn.cluster import AgglomerativeClustering
   from sklearn.metrics import silhouette_score
   import pandas as pd
   import matplotlib.pyplot as plt

2. Read the wine dataset into the pandas DataFrame and print a small sample:

   wine_df = pd.read_csv("wine_data.csv")
   print(wine_df.head)

   The output is as follows:

   

   Figure 2.22: The output of the wine dataset

3. Visualize the wine dataset to understand the data structure:

   plt.scatter(wine_df.values[:,0], wine_df.values[:,1])
   plt.title("Wine Dataset")
   plt.xlabel("OD Reading")
   plt.ylabel("Proline")
   plt.show()

   The output is as follows:

   

   Figure 2.23: A plot of raw wine data

4. Use the sklearn implementation of k-means on the wine dataset, knowing that there are three wine types:

   km = KMeans(3)
   km_clusters = km.fit_predict(wine_df)

5. Use the sklearn implementation of hierarchical clustering on the wine dataset:

   ac = AgglomerativeClustering(3, linkage='average')
   ac_clusters = ac.fit_predict(wine_df)

6. Plot the predicted clusters from k-means, as follows:

   plt.scatter(wine_df.values[:,0], wine_df.values[:,1], c=km_clusters)
   plt.title("Wine Clusters from Agglomerative Clustering")
   plt.xlabel("OD Reading")
   plt.ylabel("Proline")
   plt.show()

   The output is as follows:

   

   Figure 2.24: A plot of clusters from k-means clustering

7. Plot the predicted clusters from hierarchical clustering, as follows:

   plt.scatter(wine_df.values[:,0], wine_df.values[:,1], c=ac_clusters)
   plt.title("Wine Clusters from Agglomerative Clustering")
   plt.xlabel("OD Reading")
   plt.ylabel("Proline")
   plt.show()

   The output is as follows:

   

   Figure 2.25: A plot of clusters from agglomerative clustering

8. Compare the silhouette score of each clustering method:

   print("Silhouette Scores for Wine Dataset:\n")
   print("k-means Clustering: ", silhouette_score(X[:,11:13], km_clusters))
   print("Agg Clustering: ", silhouette_score(X[:,11:13], ac_clusters))

   The output will be as follows:

   

   Figure 2.26: Silhouette scores for the wine dataset

As you can see from the preceding silhouette metric, agglomerative clustering narrowly beats k-means clustering when it comes to separating the clusters by mean intra-cluster distance. This is not the case for every version of agglomerative clustering, however. Instead, try different linkage types and examine how the silhouette score and clustering changes between each!