Integrating a Log Analytics workspace with Azure Databricks
In this recipe, you will learn how to integrate a Log Analytics workspace with Azure Databricks. You will also learn how to send Databricks service metrics and Databricks application metrics to the Log Analytics workspace.
Getting ready
Before starting, we need to ensure we have access to a subscription and have contributor access to that subscription. To send Databricks service metrics to a Log Analytics workspace, we will need the Azure Databricks Premium tier.
You can follow along by running the steps in the 5_1.Using Azure Key Vault Secrets in Notebooks notebook. This will allow you to send Databricks log data to a Log Analytics workspace (https://github.com/PacktPublishing/Azure-Databricks-Cookbook/tree/main/Chapter05/).
How to do it…
To send metrics to the Log Analytics workspace, we need to turn on diagnostics in the Azure Databricks service. Following this, we must send the service or application...
