Databricks Workflows

Now that we’ve gone through the YAML deployment of workflows in dbx, next, we will look at the web console. Here, we have the main page for workflows. We can create a new workflow by clicking the Create job button at the top left:

Figure 9.1: Create job

When you create a workflow, you will be presented with a diagram of the workflow and a menu for each step:

Figure 9.2: My_workflow

Be sure to match the package name and entry point with what is defined in setup.py if you’re using a package:

Figure 9.3: Workflow diagram

When you run your workflow, you will see each instance run, its status, and its start time:

Figure 9.4: Workflow run

Here is an example of a two-step workflow that has failed:

Figure 9.5: Workflow flow

You can see your failed runs individually in the console:

...