Loading multiple files using Data Factory
We are going to use BIMLStudio 2019 to build a more elaborate solution. Let's say we want to load a set of files available online into our on-premises SQL Server database. We want to use the data from these files to do reports by joining them with on-premises data. In this recipe, we will look at creating a Data Factory solution that loads a set of data files in parallel from the website FiveThirtyEight.com into Azure Data Lake Storage Gen2, and then into an on-premises SQL Server database. FiveThirtyEight.com is a data journalism organization that provides opinions and analysis on politics, economics, and sport, based on data science and analytics. The site provides access to a collection of datasets easily available for download. For our recipe, we will be downloading a set of CSV files containing statistical weather data, such as actual temperature, maximum temperature, minimum temperature, and levels of precipitation for every city...