Data Flow Spec Block
Document data flows with source, destination, transformations, frequency, and validation rules.
Overview
The Data Flow Spec block documents data movement between systems, including source and destination details, data formats, transformations, frequency, volume, and validation rules. Use this block to create complete data flow documentation for data engineering and integration projects.
Real-time Data Flow
A real-time data flow for customer events.
Batch Data Flow
A daily batch data flow for financial reporting.
Block Properties
| Property | Required | Description |
|---|---|---|
| flowName | Yes | Data flow name |
| description | No | Flow description and purpose |
| source | No | Source system object with name, type, and format |
| destination | No | Destination system object with name, type, and format |
| transformations | No | Array of transformation steps |
| frequency | No | How often data flows (real-time, hourly, daily, etc.) |
| volume | No | Data volume expectations |
| latency | No | Acceptable latency |
| errorHandling | No | How errors are handled |
| validations | No | Array of validation rules |