Creating Tests for Data Pipelines

You have now created your ADF or Synapse pipeline. You have seen many examples in ADF, which were explained in the Creating Data Pipelines section of this chapter. Now you need to run the pipeline by assigning a trigger to it.

Note

This section primarily focuses on the Create tests for data pipelines concept of the DP-203: Data Engineering on Microsoft Azure exam.

If you want to test its execution, you can choose the Trigger now option, which will result in an immediate execution of the pipeline. At that point, you can switch to the Monitor hub, where you can see the status of your pipeline and check whether it is running or completed and, in the latter case, how long it took and whether it completed successfully or failed with an error.

Note

You will find a reference to the testing process for data pipelines in Chapter 6, Developing a Stream Processing Solution, within the section titled Developing Testing Processes for Data Pipelines...