Scheduling data pipelines in Data Factory/Synapse pipelines
Scheduling pipelines refers to the process of defining when and how a pipeline needs to be started. The process is the same between ADF and Synapse pipelines. ADF and Synapse pipelines have a button named Add Trigger in the Pipelines tab that can be used to schedule the pipelines, as shown in the following screenshot:
Figure 11.7 – Adding a trigger from ADF/Synapse pipelines
The following screenshot shows the details that are required to configure a Schedule trigger:
Figure 11.8 – Defining the trigger in ADF
ADF and Synapse pipeline services support four types of triggers:
- Schedule trigger: This triggers a pipeline once or regularly based on the wall clock time.
- Tumbling window trigger: This triggers a pipeline based on periodic intervals while maintaining state; that is, the trigger understands which window of data was processed last and restarts...
