COURSE 20 OF 22
Orchestration and Data Quality
Coordinate repeatable tasks and publish only after the checks pass.
Background reading
Introduction to Orchestration and Data Quality
The extraction job finishes, but the source file contains duplicate deliveries. If the dashboard refreshes immediately, the pipeline can be green while the business numbers are wrong.
An orchestrator coordinates dependencies, schedules and retries. Data quality checks decide whether the output is safe to publish. Both are needed: finishing a task does not prove its result is correct.
Loading course…