Search icon CANCEL
Arrow left icon
Explore Products
Best Sellers
New Releases
Books
Videos
Audiobooks
Learning Hub
Conferences
Free Learning
Arrow right icon
Arrow up icon
GO TO TOP
Azure Data Engineering Cookbook

You're reading from   Azure Data Engineering Cookbook Get well versed in various data engineering techniques in Azure using this recipe-based guide

Arrow left icon
Product type Paperback
Published in Sep 2022
Publisher Packt
ISBN-13 9781803246789
Length 608 pages
Edition 2nd Edition
Languages
Tools
Arrow right icon
Authors (3):
Arrow left icon
Ahmad Osama Ahmad Osama
Author Profile Icon Ahmad Osama
Ahmad Osama
Nagaraj Venkatesan Nagaraj Venkatesan
Author Profile Icon Nagaraj Venkatesan
Nagaraj Venkatesan
Luca Zanna Luca Zanna
Author Profile Icon Luca Zanna
Luca Zanna
Arrow right icon
View More author details
Toc

Table of Contents (16) Chapters Close

Preface 1. Chapter 1: Creating and Managing Data in Azure Data Lake 2. Chapter 2: Securing and Monitoring Data in Azure Data Lake FREE CHAPTER 3. Chapter 3: Building Data Ingestion Pipelines Using Azure Data Factory 4. Chapter 4: Azure Data Factory Integration Runtime 5. Chapter 5: Configuring and Securing Azure SQL Database 6. Chapter 6: Implementing High Availability and Monitoring in Azure SQL Database 7. Chapter 7: Processing Data Using Azure Databricks 8. Chapter 8: Processing Data Using Azure Synapse Analytics 9. Chapter 9: Transforming Data Using Azure Synapse Dataflows 10. Chapter 10: Building the Serving Layer in Azure Synapse SQL Pool 11. Chapter 11: Monitoring Synapse SQL and Spark Pools 12. Chapter 12: Optimizing and Maintaining Synapse SQL and Spark Pools 13. Chapter 13: Monitoring and Maintaining Azure Data Engineering Pipelines 14. Index 15. Other Books You May Enjoy

Securing an Azure storage account with SAS using PowerShell

A Shared Access Signature (SAS) provides more granular access to blobs by specifying an expiry limit, specific permissions, and IPs.

Using an SAS, we can specify different permissions to users or applications on different blobs, based on the requirement. For example, if an application needs to read one file/blob from a container, instead of providing access to all the files in the container, we can use an SAS to provide read access on the required blob.

In this recipe, we'll learn to create and use an SAS to access blobs.

Getting ready

Before you start, go through the following steps:

  1. Make sure you have an existing Azure storage account. If not, create one by following the Provisioning an Azure storage account using PowerShell recipe in Chapter 1, Creating and Managing Data in Azure Data Lake.
  2. Make sure you have an existing Azure storage container. If not, create one by following the Creating containers and uploading files to Azure Blob storage using PowerShell recipe.
  3. Make sure you have existing blobs/files in an Azure storage container. If not, you can upload blobs by following the previous recipe.
  4. Log in to your Azure subscription in PowerShell. To log in, run the Connect- AzAccount command in a new PowerShell window and follow the instructions.

How to do it…

Let's begin by securing blobs using an SAS.

Securing blobs using an SAS

Perform the following steps:

  1. Execute the following command in the PowerShell window to get the storage context:
    $resourcegroup = "packtadestorage"
    $storageaccount = "packtadestoragev2"
    #get storage context
    $storagecontext = (Get-AzStorageAccount -ResourceGroupName $resourcegroup -Name $storageaccount). Context
  2. Execute the following commands to get the SAS token for the logfile1.txt blob in the logfiles container with list and read permissions:
    #set the token expiry time
    $starttime = Get-Date
    $endtime = $starttime.AddDays(1)
    # get the SAS token into a variable
    $sastoken = New-AzStorageBlobSASToken -Container "logfiles" -Blob "logfile1.txt" -Permission lr -StartTime $starttime -ExpiryTime $endtime -Context $storagecontext
    # view the SAS token.
     $sastoken
  3. Execute the following commands to list the blob using the SAS token:
    #get storage account context using the SAS token
    $ctx = New-AzStorageContext -StorageAccountName $storageaccount -SasToken $sastoken
    #list the blob details
    Get-AzStorageBlob -blob "logfile1.txt" -Container "logfiles" -Context $ctx

You should get output as shown in the following screenshot:

Figure 2.40 – Listing blobs using an SAS

Figure 2.40 – Listing blobs using an SAS

  1. Execute the following command to write data to logfile1.txt. Ensure you have the Logfile1.txt file in the C:\ADECookbook\Chapter1\ Logfiles\ folder in the machine you are running the script from:
    Set-AzStorageBlobContent -File C:\ADECookbook\Chapter1\ Logfiles\Logfile1.txt -Container logfiles -Context $ctx

You should get output as shown in the following screenshot:

Figure 2.41 – Uploading a blob using an SAS

Figure 2.41 – Uploading a blob using an SAS

The write fails, as the SAS token was created with list and read access.

Securing a container with an SAS

Perform the following steps:

  1. Execute the following command to create a container stored access policy:
    $resourcegroup = "packtadestorage"
    $storageaccount = "packtadestoragev2"
    #get storage context 
    $storagecontext = (Get-AzStorageAccount -ResourceGroupName $resourcegroup -Name $storageaccount). Context
    $starttime = Get-Date
    $endtime = $starttime.AddDays(1)
    New-AzStorageContainerStoredAccessPolicy -Container logfiles -Policy writepolicy -Permission lw -StartTime $starttime -ExpiryTime $endtime -Context $storagecontext
  2. Execute the following command to create the SAS token:
    #get the SAS token
    $sastoken = New-AzStorageContainerSASToken -Name logfiles -Policy writepolicy -Context
  3. Execute the following commands to list all the blobs in the container using the SAS token:
    #get the storage context with SAS token
    $ctx = New-AzStorageContext -StorageAccountName $storageaccount -SasToken $sastoken
    #list blobs using SAS token
    Get-AzStorageBlob -Container logfiles -Context $ctx

How it works…

To generate a shared access token for a blob, use the New-AzStorageBlobSASToken command. We need to provide the blob name, container name, permission (l = list, r = read, and w = write), and storage context to generate an SAS token. We can additionally secure the token by providing IPs that can access the blob.

We then use the SAS token to get the storage context using the New-AzStorageContext command. We use the storage context to access the blobs using the Get-AzStorageBlob command. Note that we can only list and read blobs and can't write to them, as the SAS token doesn't have write permissions.

To generate a shared access token for a container, we first create an access policy for the container using the New-AzStorageContainerStoredAccessPolicy command. The access policy specifies the start and expiry time, permission, and IPs. We then generate the SAS token by passing the access policy name to the New-AzStorageContainerSASToken command.

We can now access the container and the blobs using the SAS token.

You have been reading a chapter from
Azure Data Engineering Cookbook - Second Edition
Published in: Sep 2022
Publisher: Packt
ISBN-13: 9781803246789
Register for a free Packt account to unlock a world of extra content!
A free Packt account unlocks extra newsletters, articles, discounted offers, and much more. Start advancing your knowledge today.
Unlock this book and the full library FREE for 7 days
Get unlimited access to 7000+ expert-authored eBooks and videos courses covering every tech area you can think of
Renews at $19.99/month. Cancel anytime