Data platform preview
Redgold applications store typed records in owner-scoped datasets and process them through declared pipelines. Checked-in Rust types with serde derives are the schema source. CBOR is the binary storage and HTTP encoding.
Supported contract
| Capability | Access | Stability |
|---|---|---|
| Query records owned by a workspace | MCP data tools and configured applications | Owner scope and bounded reads are required |
| Address a typed dataset | DatasetReference | Field names and semantic identity are reviewed |
| Address one logical record | PrimaryKeyId | Content identity and source time remain explicit |
| Carry generic field values | TypedValue | Prefer a domain Rust record when the schema is known |
| Author a pipeline | Git-authored TOML | The pipeline preview subset is supported |
| Publish or deploy a pipeline | Repository automation | Managed preview |
Data model
A dataset has a stable type name and owner. Every row has a PrimaryKeyId.
The key distinguishes ingestion time from the producer's source time, which
supports late events and deterministic reprocessing.
The authoritative definitions are checked in under the focused schema crates:
redgold_primary_key::{DatasetReference, PrimaryKeyId}redgold_schema_base::base::typed_value::TypedValue- the domain record module used by the dataset adapter
Queries and writes
Queries are owner-scoped and bounded. Results may come from the hot record tier or the columnar tier; storage placement is an implementation detail. Typed writes use CBOR-encoded domain records, a deterministic primary key, and a narrow application route that validates ownership and input.
General arbitrary file upload is outside the preview. Dataset onboarding defines the Rust record, source-time mapping, primary-key construction, and ingestion route together.
Versioning and operational boundary
Pin a pipeline-backed application to an exact pipeline revision when a client
needs reproducible behavior. A new behavior or persisted schema requires a new
reviewed revision and migration guidance; the managed installation may retain a
previous compatible revision for rollback. The latest pointer is a registry
convenience, not a client compatibility guarantee.
The public V1 route inventory is the authority for self-service routes. It currently lists the model API; application routes remain onboarding/managed preview routes until their dataset, publication, and environment are approved. The public inventory has no generic pipeline-app health endpoint, so managed users should use the deployment status surface supplied during onboarding plus a bounded owner-scoped smoke request. There is no documented promise of zero-downtime migration for preview applications.
Compatibility
Released Rust schema fields evolve additively where persisted data requires it. New fields are optional or have serde defaults, retired fields receive migration guidance, and route additions require the explicit public API tag. The full internal schema and route inventory remains an implementation detail.