Using Airflow 2.x in orchestrating API endpoints
Airflow 2.x is an open source platform that provides workflow authorization, monitoring, scheduling, and maintenance with its easy-to-use UI dashboard. It can manage extract, transform, load (ETL) workflows and data analytics.
Airflow uses Flask Blueprints internally and allows customization just by adding custom Blueprints in its Airflow directory. However, the main goal of this chapter is to use Airflow as an API orchestration tool to run sets of workflow activities that consume API services for resources.
Let us begin with the installation of the Airflow 2.x platform.
Installing and configuring Airflow 2.x
There is no direct Airflow 2.x installation for the Windows platform yet. But there is a Docker image that can run Airflow on Windows and operating systems with low memory resources. Our approach was to install Airflow directly on WSL2 (Ubuntu) through Windows PowerShell and also use Ubuntu to implement our Flask application...