Configuring Error Handling for the Transformation
In all the examples of ADF pipelines you used in this chapter, you have only seen success cases. However, ADF also provides a separate flow to handle errors and failures.
Note
This section primarily focuses on the Configure error handling for a transformation data concept of the DP-203: Data Engineering on Microsoft Azure exam.
In fact, ADF supports four different flows—Success
, Failure
, Completion
, and Skipped
—as shown in Figure 4.45:
![Figure 4.45 – The image illustrates an Azure Data Factory (ADF) interface, illustrating a data flow diagram. It features a “Data flow” box labeled “ADFTransformations” connected to an “Add activity on” box. The latter includes options for “Success,” “Failure,” “Completion,” and “Skipped,” each with a corresponding-colored square. This setup suggests a workflow where subsequent activities are contingent on the outcomes of the data flow.](https://static.packt-cdn.com/products/9781805124689/graphics/image/B21126_04_45.jpg)
Figure 4.45 – ADF supporting four activity flows
If any errors are encountered at any step of the pipeline, you can build an error-handling branch that can be used to either fix the errors or store them for future actions. Figure 4.46 shows one such pipeline. You will have to connect the indicated line to the error-handling activity:
![Figure 4.46 – The image illustrates an error-handling pipeline in a data processing environment. It shows a data flow with labeled boxes for “Data flow,” “Lookup,” and “Execute Pipeline,” connected by arrows to indicate the sequence of processes. The “Data flow” box is connected to the “Lookup” box, which in turn is connected to the “Execute Pipeline” box. A red arrow loops back from “Execute Pipeline” to “Data flow,” suggesting a retry mechanism in case of errors.](https://static.packt-cdn.com/products/9781805124689/graphics/image/B21126_04_46.jpg)
Figure 4.46 – Creating an error-handling pipeline
Select the Execute...