Conditional branching in Databricks Workflows
Tasks in a Databricks workflow can run based on the run status of their dependencies or the outcome of a Boolean expression. You can use these methods to run tasks conditionally in a Databricks workflow:
- Run if dependencies: This method allows you to run a task only when certain conditions are met by the task’s dependencies. For instance, this method can help your job resume running and recover from failures by running a task even when some or all of its dependencies have failed.
- If/else condition task: This method lets you run a part of a job DAG based on a Boolean expression. The if/else task allows you to incorporate branching logic into your job. For instance, you can use this method to run data processing tasks only if the upstream ingestion task has added new data.
To run a part of a job DAG based on a Boolean expression, use the if/else task. The expression consists of a Boolean operator and two operands...
