Using Ganglia reports for cluster health
Ganglia is one of the most popular cluster monitoring tools available. An Azure Databricks cluster's performance and health can be monitored using Ganglia. It is a scalable distributed monitoring system. Azure Databricks uses many clusters for operating in the backend, as well as for performing tasks, so it's very important to monitor the performance and health of the clusters for troubleshooting reasons, as well as to capture live metrics.
Getting ready
Before getting started, ensure you have completed Chapter 3, Understanding Spark Query Execution, where you learned how to create Azure Databricks jobs. In this recipe, we will run the Notebook and monitor the cluster utilization percentage (%) using the Ganglia UI.
How to do it…
Let's learn how to use the Ganglia UI to monitor Azure Databricks clusters by going through the following steps:
- Before checking the Ganglia UI, let's get the worker and...
