Applied Unsupervised Learning with Python: Discover hidden patterns and relationships in unstructured data with Python

In this chapter, we will expand on the basic ideas that we built in Chapter 1, Introduction to Clustering, by surrounding clustering with the concept of similarity. Once again, we will be implementing forms of the Euclidean distance to capture the notion of similarity. It is important to bear in mind that the Euclidean distance just happens to be one of the most popular distance metrics and not the only one! Through these distance metrics, we will expand on the simple neighbor calculations that we explored in the previous chapter by introducing the concept of hierarchy. By using hierarchy to convey clustering information, we can build stronger groupings that make more logical sense.

Chapter 1, Introduction to Clustering, covered both the high-level intuition and in-depth details of one of the most basic clustering algorithms: k-means. While it is indeed a simple approach, do not discredit it; it will be a valuable addition to your toolkit as you continue your exploration of the unsupervised learning world. In many real-world use cases, companies experience groundbreaking discoveries through the simplest methods, such as k-means or linear regression (for supervised learning). As a refresher, let's quickly walk through what clusters are and how k-means works to find them:

Figure 2.1: The attributes that separate supervised and unsupervised problems

If you were given a random collection of data without any guidance, you would likely start your exploration using basic statistics – for example, what the mean, median, and mode values are of each of the features. Remember that, from a high-level data model that simply exists, knowing whether it is supervised...

Both the natural and human-made world contain many examples of organizing systems into hierarchies and why, for the most part, it makes a lot of sense. A common representation that is developed from these hierarchies can be seen in tree-based data structures. Imagine that you had a parent node with any number of child nodes that could subsequently be parent nodes themselves. By organizing concepts into a tree structure, you can build an information-dense diagram that clearly shows how things are related to their peers and their larger abstract concepts.

An example from the natural world to help illustrate this concept can be seen in how we view the hierarchy of animals, which goes from parent classes to individual species:

Figure 2.2: Navigating the relationships of animal species in a hierarchical tree structure

In Figure 2.2, you can see an example of how relational information between varieties of animals can be easily mapped out in a way that both saves space...

Until this point, we have shown that hierarchies can be excellent structures in which to organize information that clearly show nested relationships among data points. While this is helpful in gaining an understanding of the parent/child relationships between items, it can also be very handy when forming clusters. Expanding on the animal example of the prior section, imagine that you were simply presented with two features of animals: their height (measured from the tip of the nose to the end of the tail) and their weight. Using this information, you then have to recreate the same structure in order to identify which records in your dataset correspond to dogs or cats, as well as their relative subspecies.

Since you are only given animal heights and weights, you won't be able to extrapolate the specific names of each species. However, by analyzing the features that you have been provided, you can develop a structure within the data that serves as an...

In Exercise 7, Building a Hierarchy, you implemented hierarchical clustering using what is known as Centroid Linkage. Linkage is the concept of determining how you can calculate the distances between clusters and is dependent on the type of problem you are facing. Centroid linkage was chosen for the first activity as it essentially mirrors the new centroid search that we used in k-means. However, this is not the only option when it comes to clustering data points together. Two other popular choices for determining distances between clusters are single linkage and complete linkage.

Single Linkage works by finding the minimum distance between a pair of points between two clusters as its criteria for linkage. Put simply, it essentially works by combining clusters based on the closest points between the two clusters. This is expressed mathematically as follows:

dist(a,b) = min( dist( a[i]), b[j] ) )

Complete Linkage is the opposite of single linkage and it works by finding the maximum distance...

Our instances of hierarchical clustering so far have all been agglomerative – that is, they have been built from the bottom up. While this is typically the most common approach for this type of clustering, it is important to know that it is not the only way a hierarchy can be created. The opposite hierarchical approach, that is, built from the top up, can also be used to create your taxonomy. This approach is called Divisive Hierarchical Clustering and works by having all the data points in your dataset in one massive cluster. Many of the internal mechanics of the divisive approach will prove to be quite similar to the agglomerative approach:

Figure 2.15: Agglomerative versus divisive hierarchical clustering

As with most problems in unsupervised learning, deciding the best approach is often highly dependent on the problem you are faced with solving.

Imagine that you are an entrepreneur who has just bought a new grocery store and needs to stock it with...

Now that we have expanded our understanding of how k-means clustering works, it is important to explore where hierarchical clustering fits into the picture. As mentioned in the linkage criteria section, there is some potential direct overlap when it comes to grouping data points together using centroids. Universal to all of the approaches mentioned so far, is also the use of a distance function to determine similarity. Due to our in-depth exploration in the previous chapter, we have kept using the Euclidean distance, but we understand that any distance function can be used to determine similarity.

In practice, here are some quick highlights for choosing one clustering method over another:

In this chapter, we discussed how hierarchical clustering works and where it may be best employed. In particular, we discussed various aspects of how clusters can be subjectively chosen through the evaluation of a dendrogram plot. This is a huge advantage compared to k-means clustering if you have absolutely no idea of what you're looking for in the data. Two key parameters that drive the success of hierarchical clustering were also discussed: the agglomerative versus divisive approach and linkage criteria. Agglomerative clustering takes a bottom-up approach by recursively grouping nearby data together until it results in one large cluster. Divisive clustering takes a top-down approach by starting with the one large cluster and recursively breaking it down until each data point falls into its own cluster. Divisive clustering has the potential to be more accurate since it has a complete view of the data from the start; however, it adds a layer of complexity that can decrease the stability...