Data Flow Specification Block
Document data flows between systems with source, destination, transformations, and performance characteristics.
The Data Flow Specification block provides a structured way to document data movement between systems. It captures source and destination details, transformations, performance characteristics, and validation rules, making it ideal for integration documentation and data architecture.
When to Use
Block Properties
| Property | Required | Description |
|---|---|---|
| Flow Name | Yes | Identifier for the data flow |
| Description | No | Overview of the data flow purpose |
| Source | No | Origin system with name, type, and format |
| Destination | No | Target system with name, type, and format |
| Transformations | No | Array of ordered data transformation steps |
| Frequency | No | How often data flows (real-time, hourly, daily) |
| Volume | No | Expected data volume (records per time period) |
| Latency | No | Maximum acceptable delay |
| Error Handling | No | Strategy for handling failures |
| Validations | No | Array of data quality rules applied |
Source/Destination Properties
| Property | Required | Description |
|---|---|---|
| name | Yes | Name of the source or destination system |
| type | No | System type: database, api, file, message-queue, data-stream, service, or storage |
| format | No | Data format: json, parquet, csv, xml, avro, etc. |
Source/Destination Type Values
Frequency Values
Example: Order Processing Pipeline
A real-time data flow from orders database to analytics warehouse.
Example: API Integration
A scheduled data sync between external API and internal service.