Data factory creation
This is our first recipe in this chapter. We will create and explore the various Data Factory components. The following recipes will use the same data factory to move and transform data.
As shown in the following diagram, a data factory contains the following components. Azure Storage and SQL Database are not part of the factory; they are just an example of a simple copy activity that Data Factory can do:
Figure 6.1 – An overview of Azure Data Factory
The two main components are triggers and pipelines.
A trigger is essentially a mechanism that starts a pipeline execution. There are three types of triggers:
- Scheduler: Can be based on a wall clock schedule or based on tumbling windows. A tumbling window essentially triggers the pipeline execution every n time: for example, every 5 minutes, hours, and so on.
- Event: Detects the presence of a file in an Azure Storage account. Once the file is detected, the pipeline...
