Moving and transforming using a data flow
Azure data flows are a low-code/no-code visual transformation tool that provides data engineers with a simple interface to develop the transformation logic without writing code. Azure data flows provide flexibility to ETL developers so that they can use the drag-and-drop user experience to build the data flow from one source to another, applying various transformations.
In this recipe, we will discuss and create a data flow considering the following data flow components:
- Select
- Optimize partition type
- Derived column
- Sink to Azure Synapse
Getting ready
We will be using the same dataset that we used in a previous recipe, Choosing a data loading option. Access the New York yellow taxi trip data dataset from the ADLS2 storage container named taxistagingdata. The code can be downloaded from the GitHub repository here: https://github.com/PacktPublishing/Analytics-in-Azure-Synapse-Simplified.
Also, we will be using...
