Data Contracts and Change Capture

Schema and data types

A schema describes fields and their types. Inspect it before building ingestion: the pipeline must know what it accepts and how to interpret it.

A field’s name alone is not enough. Check whether it is optional, which values its type permits, and what a missing value means to downstream logic.

Inspect actual payloads as well as documentation. A pipeline that accepts JSON syntax may still receive records with shapes its transformations cannot handle.

Semi-structured data can change shape between records. Unstructured files still have technical properties, such as encoding and format, that readers must handle.