Workflows

Question 1

Workflows vs Delta Live Tables

Answer 1

• DLT is notebooks only, Workflows can run DLT plus ad-hoc code
• Workflows are for task automation and orchestration

Question 2

Workflows vs Delta Live Tables (use cases)

Answer 2

Jobs Workflows
- Orchestration of dependent jobs (e.g. on schedule, contains dependent tasks/steps)
- Machine Learning tasks
- Arbitrary Code, External API Calls, Custom Tasks

Delta Live Tables
- Data Ingestion and Transformation (ELT)

Question 3

How are predecessor tasks set up in Workflows?

Answer 3

Define the order of execution of tasks in a job using the “Depends on” dropdown menu
- Can be set to one or more tasks in the job

Question 4

What sort of alerts can be sent from Workflows?

Answer 4

Specific alerts can be sent when a job beings, completes or fails. Notifications can be sent via email, Slack or AWS SNS

Question 5

What features enables you to debug a Workflow?

Answer 5

The Repair feature allows you to re-run only the failed task and sub-tasks

Question 6

Task Configuration options

Answer 6

• Cluster
• Dependent libraries
• Task notifications
• Task parameter variables
• Timeout
• Retries