Configuring Exception Handling
In the Creating Tests for Data Pipelines section, you learned that a pipeline run can terminate with a failure due to an unhandled error. Just like, when developing code, you tend to attempt to prevent exception errors such as expected file missing
from happening, and handle them before they result in a failure, in pipelines, you connect one activity to the next using one of the four available dependency conditions: On Success
(green arrow), On Fail
(red arrow), On Completion
(blue arrow), and On Skip
(gray arrow).
In particular, you can use On Fail
to connect your action with another action that can provide a workaround or a limitation strategy for that possible failure. If no workaround is possible, your exception-handling action can just be to log the problem and maybe send an alert. In any case, the exception will be considered handled, and the pipeline will not appear as a failure, provided, of course, that the handling action is in turn executed...