Integrating Azure Functions with Data Factory pipelines

In many enterprise applications, the need to work with data is definitely there, especially when there are a variety of heterogeneous data sources. In such cases, we need to identify tools that help us to extract the raw data, transform it, and then load the processed data into other persistent media to generate reports.

Azure assists organizations in carrying out the preceding scenarios by using a service called Azure Data Factory (ADF).

Azure Data Factory is another cloud-native serverless solution from Microsoft Azure. ADF can be used as an Extract, Transform, and Load (ETL) tool to process the data from various data sources, transform it, and load the processed data into a wide variety of data destinations. Before we start working with the recipe, I would recommend that you learn more about Azure Data Factory and its concepts at https://docs.microsoft.com/azure/data-factory/introduction.

When we have complex processing...