Search icon CANCEL
Arrow left icon
Explore Products
Best Sellers
New Releases
Books
Videos
Audiobooks
Learning Hub
Conferences
Free Learning
Arrow right icon
Arrow up icon
GO TO TOP
Azure Data Factory Cookbook

You're reading from   Azure Data Factory Cookbook Build and manage ETL and ELT pipelines with Microsoft Azure's serverless data integration service

Arrow left icon
Product type Paperback
Published in Dec 2020
Publisher Packt
ISBN-13 9781800565296
Length 382 pages
Edition 1st Edition
Tools
Arrow right icon
Authors (4):
Arrow left icon
Dmitry Anoshin Dmitry Anoshin
Author Profile Icon Dmitry Anoshin
Dmitry Anoshin
Roman Storchak Roman Storchak
Author Profile Icon Roman Storchak
Roman Storchak
Xenia Ireton Xenia Ireton
Author Profile Icon Xenia Ireton
Xenia Ireton
Dmitry Foshin Dmitry Foshin
Author Profile Icon Dmitry Foshin
Dmitry Foshin
Arrow right icon
View More author details
Toc

Table of Contents (12) Chapters Close

Preface 1. Chapter 1: Getting Started with ADF 2. Chapter 2: Orchestration and Control Flow FREE CHAPTER 3. Chapter 3: Setting Up a Cloud Data Warehouse 4. Chapter 4: Working with Azure Data Lake 5. Chapter 5: Working with Big Data – HDInsight and Databricks 6. Chapter 6: Integration with MS SSIS 7. Chapter 7: Data Migration – Azure Data Factory and Other Cloud Services 8. Chapter 8: Working with Azure Services Integration 9. Chapter 9: Managing Deployment Processes with Azure DevOps 10. Chapter 10: Monitoring and Troubleshooting Data Pipelines 11. Other Books You May Enjoy

Rerunning activities

When our data transfers fail for one reason or another, we frequently need to rerun affected pipelines. This ensures the appropriate data movement is performed, albeit delayed. If our design is complex, or if we are moving large volumes of data, it is useful to be able to repeat the run from the point of failure, to minimize the time lost in the failed pipeline.

In this section, we will look at two features of Data Factory that help us to troubleshoot our pipelines and rerun them with maximum efficiency. The first feature is breakpoints, which allow us to execute a pipeline up to an activity of our choice. The second feature is rerunning from the point of failure, which helps to minimize the time lost due to a failed execution.

Getting ready

Preparing your environment for this recipe is identical to the preparation required for the previous recipe in this chapter, Investigating failures – running in debug mode. We will be using the same Azure Data...

lock icon The rest of the chapter is locked
Register for a free Packt account to unlock a world of extra content!
A free Packt account unlocks extra newsletters, articles, discounted offers, and much more. Start advancing your knowledge today.
Unlock this book and the full library FREE for 7 days
Get unlimited access to 7000+ expert-authored eBooks and videos courses covering every tech area you can think of
Renews at €18.99/month. Cancel anytime