Using notification operators

So far, we have focused on ensuring that code is well logged and has enough information to provide valid monitoring. Nevertheless, the purpose of having mature and structured pipelines is to avoid the necessity of manual intervention. With busy agendas and other projects, it is hard to constantly look at monitoring dashboards to check whether everything is fine.

Thankfully, Airflow also has native operators to trigger alerts depending on their configured situation. In this recipe, we will configure an email operator to trigger a message every time a pipeline succeeds or fails, allowing us to remediate the problem rapidly.

Getting ready

Refer to the Technical requirements section for this recipe, since we will handle it with the same technology.

In addition to that, you need to create an app password for your Google account. This password will allow our application to authenticate and use the Simple Mail Transfer Protocol (SMTP) host from...