Using Azure Blob storage with Azure Databricks
In the same way that we can access objects stored in AWS S3, we can access objects in Azure Blob storage. Both options allow us to have a redundant data storage that can be accessed from anywhere. Their differences lie in the tools that they will be used with and certain characteristics that may make them more suitable to use in a certain project. Azure Blob storage is more cost-efficient and has high redundancy, while S3 is extensively used by several organizations and has a small learning curve. We will see how to set up an Azure Blob storage account, upload the file that we were using, and read it from our notebook.
Setting up Azure Blob storage
The first step is to create an Azure Blob storage account, as follows:
- Search for Storage account in the Azure portal and select Create a new storage account.
- In the Create storage account options, fill out the details and set the Replication option to Locally-redundant storage...
