Search icon CANCEL
Subscription
0
Cart icon
Your Cart (0 item)
Close icon
You have no products in your basket yet
Arrow left icon
Explore Products
Best Sellers
New Releases
Books
Videos
Audiobooks
Learning Hub
Free Learning
Arrow right icon
Arrow up icon
GO TO TOP
Hands-On Data Warehousing with Azure Data Factory

You're reading from   Hands-On Data Warehousing with Azure Data Factory ETL techniques to load and transform data from various sources, both on-premises and on cloud

Arrow left icon
Product type Paperback
Published in May 2018
Publisher Packt
ISBN-13 9781789137620
Length 284 pages
Edition 1st Edition
Languages
Tools
Concepts
Arrow right icon
Authors (3):
Arrow left icon
Christian Cote Christian Cote
Author Profile Icon Christian Cote
Christian Cote
Giuseppe Ciaburro Giuseppe Ciaburro
Author Profile Icon Giuseppe Ciaburro
Giuseppe Ciaburro
Michelle Gutzait Michelle Gutzait
Author Profile Icon Michelle Gutzait
Michelle Gutzait
Arrow right icon
View More author details
Toc

Using the data factory to manipulate data in the Data Lake

In the previous section, we created the Data Lake Analytics Resource for the U-SQL task:

  • Even though possible, it is not at all straightforward to run U-SQL to connect directly to an SQL database. It involves tweaking firewalls and permissions. This is why we do not cover this part in the next section, which describes how to run a U-SQL job directly from the Data Lake Analytics resource.
  • It is much simpler to copy data from an SQL Server database to a file on Azure Blob Storage via the Azure Data Factory.
  • In this section, we show how to do this and then how to manipulate the copied data with U-SQL using the Azure Data Factory.

We will now create a pipeline in Azure Data Factory that will do the following:

  • Task 1: Import data from SQL Server (from a view) into a file on blob storage
  • Task 2: Use U-SQL to export summary...
lock icon The rest of the chapter is locked
Register for a free Packt account to unlock a world of extra content!
A free Packt account unlocks extra newsletters, articles, discounted offers, and much more. Start advancing your knowledge today.
Unlock this book and the full library FREE for 7 days
Get unlimited access to 7000+ expert-authored eBooks and videos courses covering every tech area you can think of
Renews at $19.99/month. Cancel anytime
Banner background image