diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index 62f7cf6..3c36a3c 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -79,7 +79,7 @@ When the drift check fails, or forge-cli releases a new schema version ```bash COMMIT=... # the commit from step 1, which is also upstream.commit - for v in 0.7.5 0.7.6; do + for v in 0.7.6 0.7.7; do curl -fsSL "https://raw.githubusercontent.com/Agenticstiger/forge-cli/$COMMIT/fluid_build/schemas/fluid-schema-$v.json" \ -o "schema/fluid-schema-$v.json" done diff --git a/GENERATOR_DOCS.md b/GENERATOR_DOCS.md index 8132f9b..95ca239 100644 --- a/GENERATOR_DOCS.md +++ b/GENERATOR_DOCS.md @@ -48,7 +48,7 @@ pip install --require-hashes -r scripts/requirements-docs.lock python3 generate-docs.py # every synced version python3 generate-docs.py --all # the same, spelled out -python3 generate-docs.py --version 0.7.6 # one version; repeat the flag for more +python3 generate-docs.py --version 0.7.7 # one version; repeat the flag for more python3 generate-docs.py --check # render to a temporary directory and # exit 1 if specs/ differs, or if a # file under schema/ or specs/ is not @@ -76,8 +76,8 @@ pip-compile --generate-hashes --strip-extras \ --output-file=scripts/requirements-docs.lock scripts/requirements-docs.txt ``` -A preview's page is labelled by its schema's own `title`; for 0.7.6 that reads -"FLUID 0.7.6 β Declarative Packaging Modes (preview)". +A preview's page is labelled by its schema's own `title`; for 0.7.7 that reads +"FLUID 0.7.7 β Preview: recursive streaming types". `--version` also renders a version that is not synced, if its schema is in `schema/`. The pages for versions before 0.7.2 were rendered by earlier diff --git a/README.md b/README.md index e7cc494..60e391b 100644 --- a/README.md +++ b/README.md @@ -12,7 +12,7 @@ π§ Schema Anatomy Β· π Examples Β· π vs ODCS / ODPS Β· - β¨ What's New + β¨ What's New
--- @@ -23,7 +23,7 @@ FLUID is one YAML file that describes a data product end to end β **schema, bu ```mermaid flowchart LR - F["FLUID v0.7.5/locations/ FLUID contract schema 0.7.6 β the current PREVIEW version, opened when 0.7.5 was promoted to stable. FLUID contract schema 0.7.6 β opened as the preview when 0.7.5 was promoted to stable, and stable itself since 0.7.7 opened. π₯ NEW in v0.7.6: Contract schema version. Accepts '0.7.3' (source-aligned data products + acquisition pattern + ingestion engines) or '0.7.4' (adds the MCP output-port gateway + runtime agentPolicy enforcement). 0.7.4 is fully backward-compatible with 0.7.3 β every 0.7.3 contract validates as 0.7.4 unchanged. '0.7.6' is the current preview. Contract schema version. Accepts '0.7.3' (source-aligned data products + acquisition pattern + ingestion engines) or '0.7.4' (adds the MCP output-port gateway + runtime agentPolicy enforcement). 0.7.4 is fully backward-compatible with 0.7.3 β every 0.7.3 contract validates as 0.7.4 unchanged. NEW in v0.7.6: Either a plain string β unchanged meaning, it becomes Prose description. Exported as NEW in v0.7.6: The structured form of the root What this data product is for. How a consumer is expected to use it. What it must not be used for, and what it does not cover. NEW in v0.7.6: The team behind this product, as a roster rather than a single contact. The people on the team, each with the role they hold on this product. NEW in v0.7.6: One member of The member's identifier in the organisation's directory β the one field Bitol requires. Display name. Must contain a minimum of The role this member holds on this product, e.g. 'owner', 'steward', 'data engineer'. Free-form: Bitol does not constrain it. Semantic version (semver.org). Business alignment information for data mesh organization. Business domain this product belongs to. Expose-level labels for metadata and automation. Business capability this product supports. Value stream this product participates in. Must contain a minimum of Hash of canonical schema/OpenAPI. Used by contract-tests gates. Stable interface handle (used by consumers & build.outputs). NEW in v0.7.3: Schema evolution policy. strict = fail on any schema change; discoverandfreeze = capture from source on first run, fail thereafter; evolvesafe = accept additive changes; evolveall = accept all changes (cast on type narrow, route failures to DLQ). Detailed description of the exposed resource. Semantic version (semver.org). Must be at least Column data type. Accepts a canonical FLUID/SQL type name (case-insensitive) or a parameterized form such as decimal(18,4) or varchar(255). The enum lists the canonical bare types; the pattern additionally permits case variants and parameter suffixes. Intentionally generous β it rejects obvious garbage (e.g. MYSTERY_TYPE) while admitting the cross-dialect superset. Expose-level labels for metadata and automation. NEW in v0.7.6: An explicit, stable contract id, used verbatim as the ODCS Must be at least Hash of canonical schema/OpenAPI. Used by contract-tests gates. Field-level sensitivity classification for improved policy ergonomics. NEW in v0.7.3: Schema evolution policy. strict = fail on any schema change; discoverandfreeze = capture from source on first run, fail thereafter; evolvesafe = accept additive changes; evolveall = accept all changes (cast on type narrow, route failures to DLQ). Tabular schema (for table/view/file). Must be at least Field-level validation constraints. Column data type. Accepts a canonical FLUID/SQL type name (case-insensitive) or a parameterized form such as decimal(18,4) or varchar(255). The enum lists the canonical bare types; the pattern additionally permits case variants and parameter suffixes. Intentionally generous β it rejects obvious garbage (e.g. MYSTERY_TYPE) while admitting the cross-dialect superset. Semantic meaning (e.g., 'email', 'phone', 'identifier'). OpenAPI document URL (for APIs). Business-friendly name for this field. Business definition and meaning of this field. Compatibility promise for non-breaking evolution. Field-level validation constraints. Enhanced data quality rules including anomaly detection. NEW in v0.7.6: Column-level data quality rules, in ODCS's own shape β the array that is written straight into Stable rule identifier, unique within this column's rules. The pattern is ODCS ODCS quality check type. text = prose; library = one of ODCS's predefined metrics; sql = a query whose result is compared; custom = a named engine's own check. type: library only. The five predefined metrics ODCS v3.1.0 defines; a name outside this list is not a library rule. type: library only. Extra arguments for the metric β type: sql only. The query whose result the comparison operator is applied to. type: custom only. The engine that runs the check, e.g. soda, great-expectations, dbt. DQ rule labels for metadata and routing. The data quality dimension the rule measures. Severity of a failure. ODCS types this as a free string and documents info / warning / error by example. ISO-8601 duration (e.g., P1D, PT15M, P2Y6M). Unit the comparison is in, usually rows (the default) or percent. Field-level tags for categorization and data governance. Simplified SQL-expression data quality rules (alternate form of dq.rules for inline assertions). OpenAPI document URL (for APIs). Compatibility promise for non-breaking evolution. Severity when the rule fails. Enhanced data quality rules including anomaly detection. Value must be greater or equal to Value must be greater or equal to ISO-8601 duration (e.g., P1D, PT15M, P2Y6M). Column-level access control. A column named in any restriction is restricted. 'deny': the principal may not read the columns. 'allow': the columns are readable only by the principals an allow names. A deny beats an allow, and a restriction never grants access: the readers are the expose's readers. Principals are logical and resolve through binding.principals. NEW in v0.7.6 (the field itself is older): If the conditions in the "If" tab are respected, then the conditions in the "Then" tab should be respected.
- Otherwise, the conditions in the "Else" tab should be respected. Simplified SQL-expression data quality rules (alternate form of dq.rules for inline assertions). If the conditions in the "If" tab are respected, then the conditions in the "Then" tab should be respected.
- Otherwise, the conditions in the "Else" tab should be respected. Rule identifier. SQL or boolean expression that must evaluate to true. Severity when the rule fails. Logical principal the restriction applies to, written as in accessPolicy (e.g. group:analysts@company.example); binding.principals maps it per cloud. Availability percentage (e.g., 99.9%). e.g., '0 rows' Target completeness ratio (0.0 to 1.0). Value must be greater or equal to deny: the principal may not read the columns. allow: only the principals an allow names may read them. mask: the principal reads the columns masked by the platform, by the rule in mask, and never their raw values (NEW in v0.7.6). Acceptable error rate (0.0 to 1.0). Value must be greater or equal to The masking rule an access: mask restriction applies, required with it and refused without it. lastfour: 'XXXXX' and the value's last four characters; firstfour: its first four characters and 'XXXXX' (a value of four characters or fewer is hashed instead, by each platform); nullify: NULL. GCP BigQuery: a data policy (DATAMASKINGPOLICY, LASTFOURCHARACTERS, FIRSTFOURCHARACTERS or ALWAYS_NULL) on the columns' policy tag, and roles/bigquerydatapolicy.maskedReader for the masked principals, who read the same column masked. AWS Glue: Lake Formation cannot mask, so the masked principals' grant on the table excludes the columns and they read a protected Glue Data Catalog view (SECURITY DEFINER, binding.governance.lakeFormation.maskedViews) that returns them masked by the same rule (NEW in v0.7.6). Row-level access control (NEW in v0.7.6). Each filter lets its principal read only the rows its where predicate selects; a reader no filter names reads every row, and a principal may have one filter per expose. The predicate is one SQL boolean expression over this expose's columns, with no statement separator or comment. Principals are logical and resolve through binding.principals. GCP BigQuery: a row access policy (filterpredicate, grantees) per filter, plus one that selects every row for every other reader and writer, since a principal no row access policy names reads no row at all. AWS Glue: a Lake Formation data cells filter (rowfilter and the principal's excluded columns), and the principal's SELECT is granted on the filter instead of the table. Logical principal the filter applies to, written as in accessPolicy; binding.principals maps it per cloud. The filter's name on the platform (the row access policy or data cells filter); derived from the principal when omitted. Column-level access control. A column named in any restriction is restricted. 'deny': the principal may not read the columns. 'allow': the columns are readable only by the principals an allow names. A deny beats an allow, and a restriction never grants access: the readers are the expose's readers. Principals are logical and resolve through binding.principals. NEW in v0.7.6 (the field itself is older): One SQL boolean expression over the expose's columns, e.g. consentdataanalytics = true AND is_suppressed = false. Must be at least Columns to treat before they land. The DuckDB acquisition runner rewrites each named column inside the write, so the file, S3 object or BigQuery load and the DLQ receive the treated value; quality gates judge the source value first. A treated column lands as a string, so declare it with a string type (string, or VARCHAR under a schemaPolicy that compares source type names). A rule the build cannot apply (an unset salt or key, k_anonymity, a column the stream lacks, a non-string declared type, the incremental cursor) refuses the build rather than landing the column untreated. fluid verify checks every non-null value of a masked column for its strategy's shape, on a local file and, through Athena, on an S3+Glue table. The MCP output-port gateway drops masked columns from what it serves. hash: lowercase hex SHA-256 of salt || value, deterministic so joins work (64 hex characters). mask: every character but the first params.keepFirst and the last params.keepLast replaced by '*', length preserved. tokenize: HMAC-SHA256 token, 32 hex characters, the token the tokenizepii preLand hook makes. encrypt: AES-GCM, written as 'aesgcm:v1:' + base64url(nonce || ciphertext || tag) with the column name as associated data; reversible with the key, not deterministic. kanonymity: accepted by the schema but refused at landing: it is a property of a whole table over its quasi-identifiers, not of one value. If the conditions in the "If" tab are respected, then the conditions in the "Then" tab should be respected.
+ Otherwise, the conditions in the "Else" tab should be respected. Strategy parameters. The runner accepts exactly these keys and refuses any other, including a literal salt or key: secrets come only from the environment, never the contract, which is published with the plan, the bundle and the Glue table's fluid_contract parameter. Contracts on 0.7.5 and earlier may use the same keys; their schemas leave params untyped, and the runner checks them at build time. NEW in v0.7.6: hash only. The environment variable holding the salt, at least 16 bytes. Unset or empty, the build is refused: an unsalted SHA-256 of a phone number is reversed by enumerating the numbering plan. NEW in v0.7.6: tokenize and encrypt. The environment variable holding the key. tokenize defaults to FLUIDPIITOKENIZATIONKEY and needs at least 32 bytes; encrypt defaults to FLUIDPIIENCRYPTIONSECRET_KEY and needs the base64 of a 16, 24 or 32-byte AES key (openssl rand -base64 32). Unset or empty, the build is refused. NEW in v0.7.6: mask only. Leading characters left as they are. Logical principal the restriction applies to, written as in accessPolicy (e.g. group:analysts@company.example); binding.principals maps it per cloud. Value must be greater or equal to NEW in v0.7.6: mask only. Trailing characters left as they are, so +46701234567 lands as ********4567. A value no longer than keepFirst + keepLast is masked entirely. Columns of this expose's schema. Value must be greater or equal to Additional Properties of any type are allowed. Access control rule tags. Access control rule labels. The masking rule an access: mask restriction applies, required with it and refused without it. lastfour: 'XXXXX' and the value's last four characters; firstfour: its first four characters and 'XXXXX' (a value of four characters or fewer is hashed instead, by each platform); nullify: NULL. GCP BigQuery: a data policy (DATAMASKINGPOLICY, LASTFOURCHARACTERS, FIRSTFOURCHARACTERS or ALWAYS_NULL) on the columns' policy tag, and roles/bigquerydatapolicy.maskedReader for the masked principals, who read the same column masked. AWS Glue: Lake Formation cannot mask, so the masked principals' grant on the table excludes the columns and they read a protected Glue Data Catalog view (SECURITY DEFINER, binding.governance.lakeFormation.maskedViews) that returns them masked by the same rule (NEW in v0.7.6). Logical principal the filter applies to, written as in accessPolicy; binding.principals maps it per cloud. NEW in v0.7.1: AI/LLM usage governance - controls which AI models can consume this data and for what purposes. The filter's name on the platform (the row access policy or data cells filter); derived from the principal when omitted. Whitelist of AI models permitted to consume this data. Examples: ['gpt-4', 'claude-3-opus', 'gemini-pro']. Empty array = no AI access. One SQL boolean expression over the expose's columns, e.g. consentdataanalytics = true AND is_suppressed = false. Must be at least All items must be unique Blacklist of AI models explicitly prohibited from consuming this data. Takes precedence over allowedModels. Columns to treat before they land. The DuckDB acquisition runner rewrites each named column inside the write, so the file, S3 object or BigQuery load and the DLQ receive the treated value; quality gates judge the source value first. A treated column lands as a string, so declare it with a string type (string, or VARCHAR under a schemaPolicy that compares source type names). A rule the build cannot apply (an unset salt or key, k_anonymity, a column the stream lacks, a non-string declared type, the incremental cursor) refuses the build rather than landing the column untreated. fluid verify checks every non-null value of a masked column for its strategy's shape, on a local file and, through Athena, on an S3+Glue table. The MCP output-port gateway drops masked columns from what it serves. All items must be unique Maximum tokens per AI request. Prevents excessive data exposure in single queries. Value must be greater or equal to Daily token limit for AI consumption. Enforces usage quotas. hash: lowercase hex SHA-256 of salt || value, deterministic so joins work (64 hex characters). mask: every character but the first params.keepFirst and the last params.keepLast replaced by '*', length preserved. tokenize: HMAC-SHA256 token, 32 hex characters, the token the tokenizepii preLand hook makes. encrypt: AES-GCM, written as 'aesgcm:v1:' + base64url(nonce || ciphertext || tag) with the column name as associated data; reversible with the key, not deterministic. kanonymity: accepted by the schema but refused at landing: it is a property of a whole table over its quasi-identifiers, not of one value. Value must be greater or equal to Permitted AI use cases. Controls whether data can be used for training, inference, reasoning, etc. Strategy parameters. The runner accepts exactly these keys and refuses any other, including a literal salt or key: secrets come only from the environment, never the contract, which is published with the plan, the bundle and the Glue table's fluid_contract parameter. Contracts on 0.7.5 and earlier may use the same keys; their schemas leave params untyped, and the runner checks them at build time. All items must be unique NEW in v0.7.6: hash only. The environment variable holding the salt, at least 16 bytes. Unset or empty, the build is refused: an unsalted SHA-256 of a phone number is reversed by enumerating the numbering plan. Prohibited AI use cases. Example: Allow reporting but deny model training. All items must be unique NEW in v0.7.6: tokenize and encrypt. The environment variable holding the key. tokenize defaults to FLUIDPIITOKENIZATIONKEY and needs at least 32 bytes; encrypt defaults to FLUIDPIIENCRYPTIONSECRET_KEY and needs the base64 of a 16, 24 or 32-byte AES key (openssl rand -base64 32). Unset or empty, the build is refused. Whether AI agents can use this data for multi-step reasoning/chain-of-thought. False = simple retrieval only. NEW in v0.7.6: mask only. Leading characters left as they are. Value must be greater or equal to Whether AI systems can cache/store this data. False = ephemeral processing only. NEW in v0.7.6: mask only. Trailing characters left as they are, so +46701234567 lands as ********4567. A value no longer than keepFirst + keepLast is masked entirely. Value must be greater or equal to Additional Properties of any type are allowed. Data retention requirements for AI systems consuming this data. Whether AI system must delete data after use. Privacy masking rule labels. Free-text description of allowed purposes. Example: 'Customer service chatbot only, not for marketing'. Provider-specific predicate If the conditions in the "If" tab are respected, then the conditions in the "Then" tab should be respected.
- Otherwise, the conditions in the "Else" tab should be respected. NEW in v0.7.1: AI/LLM usage governance - controls which AI models can consume this data and for what purposes. Whitelist of AI models permitted to consume this data. Examples: ['gpt-4', 'claude-3-opus', 'gemini-pro']. Empty array = no AI access. All items must be unique Blacklist of AI models explicitly prohibited from consuming this data. Takes precedence over allowedModels. All items must be unique Maximum tokens per AI request. Prevents excessive data exposure in single queries. Value must be greater or equal to Daily token limit for AI consumption. Enforces usage quotas. Value must be greater or equal to Permitted AI use cases. Controls whether data can be used for training, inference, reasoning, etc. All items must be unique Each additional property must conform to the following schema Prohibited AI use cases. Example: Allow reporting but deny model training. All items must be unique Whether AI agents can use this data for multi-step reasoning/chain-of-thought. False = simple retrieval only. Whether AI systems can cache/store this data. False = ephemeral processing only. Data retention requirements for AI systems consuming this data. Maximum days AI system can retain this data. 0 = no retention allowed. Value must be greater or equal to If the conditions in the "If" tab are respected, then the conditions in the "Then" tab should be respected.
- Otherwise, the conditions in the "Else" tab should be respected. Whether AI system must delete data after use. Whether AI consumption must be logged/audited. Free-text description of allowed purposes. Example: 'Customer service chatbot only, not for marketing'. Agent policy labels for automation. If the conditions in the "If" tab are respected, then the conditions in the "Then" tab should be respected.
+ Otherwise, the conditions in the "Else" tab should be respected. Each additional property must conform to the following schema Target platform. NEW in v0.7.5: 'pgvector' for the vector/embeddings output port (pgvector-on-PostgreSQL RAG target β emits the embeddings table + ANN index DDL for the ai-embeddable columns). NEW in v0.7.5: 'confluent' for the Confluent Cloud Tableflow managed KafkaβIceberg emitter. NEW in v0.7.4: 'postgres' for the MCP output-port PostgreSQL driver (AWS Athena uses platform 'aws'). Binding format. NEW in v0.7.5: 'pgvectortable' β a pgvector embeddings table target for the vector output port. NEW in v0.7.4: 'postgrestable' (PostgreSQL driver), 'athenatable' / 'gluetable' (AWS Athena driver) for the MCP output port. Each additional property must conform to the following schema Apache Iceberg table format configuration Iceberg table format version Underlying file format for data files Partition field name Partition transform function If the conditions in the "If" tab are respected, then the conditions in the "Then" tab should be respected.
+ Otherwise, the conditions in the "Else" tab should be respected. Column to sort by Sort direction NULL value ordering Each additional property must conform to the following schema Iceberg table properties Each additional property must conform to the following schema NEW in v0.7.5: vector / embeddings output-port configuration (platform 'pgvector', format 'pgvectortable'). Consumed by fluidbuild.output_ports.vector to emit the pgvector embeddings-table + ANN-index DDL and a RAG provenance manifest for the schema's ai-embeddable columns. Embedding vector width (e.g. 1536 for OpenAI text-embedding-3-small, 1024 for Cohere embed-english-light). Maps to the pgvector column type vector(<dimensions>). Value must be greater or equal to Provenance: the embedding model that produces the vectors (e.g. 'text-embedding-3-small'). Recorded in the RAG manifest and the embeddings-table's embedding_model column so a retriever can verify model parity. pgvector column type: 'vector' (float32, up to 2000 dims) or 'halfvec' (float16, half the storage, up to 4000 dims). Target platform. NEW in v0.7.5: 'pgvector' for the vector/embeddings output port (pgvector-on-PostgreSQL RAG target β emits the embeddings table + ANN index DDL for the ai-embeddable columns). NEW in v0.7.5: 'confluent' for the Confluent Cloud Tableflow managed KafkaβIceberg emitter. NEW in v0.7.4: 'postgres' for the MCP output-port PostgreSQL driver (AWS Athena uses platform 'aws'). ANN index built on the embedding column. 'hnsw' (higher recall / lower query latency, buildable before data lands), 'ivfflat' (lower build cost, needs data first), or 'none' (no index β exact search only). Binding format. NEW in v0.7.5: 'pgvectortable' β a pgvector embeddings table target for the vector output port. NEW in v0.7.4: 'postgrestable' (PostgreSQL driver), 'athenatable' / 'gluetable' (AWS Athena driver) for the MCP output port. Distance function the index optimises for β pgvector operator class: cosineβvectorcosineops, l2βvectorl2ops, innerproductβvectoripops, l1βvectorl1_ops. Apache Iceberg table format configuration Target embeddings table name. Defaults to '<exposeId>_embeddings' when omitted. Iceberg table format version Provenance: the source primary-key column that ties each embedding row back to its origin row (written to the embeddings-table's source_key column). Enables update/delete propagation from the source product to the vector store. Underlying file format for data files HNSW index tuning (used when indexType is 'hnsw'). Partition transformation specifications Value must be greater or equal to Build-time dynamic candidate-list size (pgvector default 64). Partition field name Value must be greater or equal to IVFFlat index tuning (used when indexType is 'ivfflat'). Source column for partitioning Number of inverted lists (pgvector guidance: rows/1000 for < 1M rows). Partition transform function Value must be greater or equal to Provider-native addressing with tightened structure for better interop. Number of buckets (for bucket transform) Truncate width (for truncate transform) Sort order specifications for data files Column to sort by Sort direction NULL value ordering Iceberg table properties Each additional property must conform to the following schema File or object path NEW in v0.7.5: vector / embeddings output-port configuration (platform 'pgvector', format 'pgvectortable'). Consumed by fluidbuild.output_ports.vector to emit the pgvector embeddings-table + ANN-index DDL and a RAG provenance manifest for the schema's ai-embeddable columns. Embedding vector width (e.g. 1536 for OpenAI text-embedding-3-small, 1024 for Cohere embed-english-light). Maps to the pgvector column type vector(<dimensions>). Value must be greater or equal to Provenance: the embedding model that produces the vectors (e.g. 'text-embedding-3-small'). Recorded in the RAG manifest and the embeddings-table's embedding_model column so a retriever can verify model parity. pgvector column type: 'vector' (float32, up to 2000 dims) or 'halfvec' (float16, half the storage, up to 4000 dims). ANN index built on the embedding column. 'hnsw' (higher recall / lower query latency, buildable before data lands), 'ivfflat' (lower build cost, needs data first), or 'none' (no index β exact search only). Distance function the index optimises for β pgvector operator class: cosineβvectorcosineops, l2βvectorl2ops, innerproductβvectoripops, l1βvectorl1_ops. NEW in v0.7.5: Confluent Cloud environment id (env-xxxxx) for the Tableflow emitter. Target embeddings table name. Defaults to '<exposeId>_embeddings' when omitted. NEW in v0.7.5: Confluent Cloud Kafka cluster id (lkc-xxxxx) the Tableflow topic belongs to. Provenance: the source primary-key column that ties each embedding row back to its origin row (written to the embeddings-table's source_key column). Enables update/delete propagation from the source product to the vector store. NEW in v0.7.5: AWS Kinesis Data Stream name β maps to awskinesisstream in the streaming IaC emitter. HNSW index tuning (used when indexType is 'hnsw'). NEW in v0.7.5: Amazon Redshift Serverless workgroup name β maps to awsredshiftserverlessworkgroup. Build-time dynamic candidate-list size (pgvector default 64). Value must be greater or equal to NEW in v0.7.5: ARN of the IAM role attached to the Redshift Serverless namespace / used by the redshift-data CREATE EXTERNAL SCHEMA bridge. IVFFlat index tuning (used when indexType is 'ivfflat'). NEW in v0.7.5: Glue Data Catalog database that backs the Redshift external schema. Provider-native addressing with tightened structure for better interop. NEW in v0.7.5: ARN of the pre-created AWS IAM role Confluent Tableflow assumes (byobaws). Its trust policy is updated post-apply with the provider integration's externalid (two-phase IAM). Cloud account ID NEW in v0.7.5: The Iceberg catalog that owns this table: glue, rest, lakekeeper, polaris, unity, nessie, bigquery, hive, jdbc, hadoop, dynamodb or snowflake-managed. Spellings fold case and '-'/'', so icebergrest == rest and snowflake == snowflake-managed; fluid validate refuses any other value on an Iceberg expose. When absent: glue on platform aws, snowflake-managed on snowflake, rest elsewhere. The REST catalogs (rest, lakekeeper, polaris, unity) need location.uri plus location.warehouse as the catalog's warehouse NAME, not an object-store URI; Lakekeeper serves its Iceberg REST API under <host>/catalog (uri: http://lakekeeper:8181/catalog). On platform aws, a kind other than glue gets no Glue table and no Glue grant: that catalog creates and governs the table. Every emitter (streaming sinks, dbt catalogs.yml, the IaC, policy compile) reads one table, providers/icebergcatalog.py. GCP project ID NEW in v0.7.5: Iceberg warehouse β the canonical Iceberg REST 'warehouse' property (maps to iceberg.catalog.warehouse): an object-store URI (s3://|gs://|abfss://) for object-store catalogs, or the catalog/warehouse name for a REST catalog. BigQuery dataset or database name NEW in v0.7.5: REST Iceberg catalog endpoint URL β the canonical Iceberg REST 'uri' property (maps to iceberg.catalog.uri). Database name NEW in v0.7.5: Iceberg partition columns β a FLUID abstraction over the Iceberg PARTITIONED BY / PartitionSpec (maps to iceberg.tables.default-partition-by). On a gcp BigQuery table with exposes[].lifecycle.expire: true (fluid-schema 0.7.6), one DATE, TIMESTAMP or DATETIME column to partition by (time_partitioning.field); without it the table is partitioned by ingestion time. With a column, retention counts from the date in that column, not from landing: BigQuery deletes a partition Schema name Binding configuration labels for automation. File or object path Additional Properties of any type are allowed. NEW in v0.7.6: Host or endpoint; for a Kafka binding, the bootstrap servers. The ODCS server mapper already reads Per-resource governance: registerLocation, principal grants, tag associations, row/column filters. Account-wide governance (admins, tag definitions) lives at the contract top-level governance block. Base URL for API endpoints Per-resource LF settings: location registration, principal grants, LF-tag associations, row/column filters. Applies only to formats backed by S3 + Glue catalog (iceberg/parquet/csv/json/avro/orc/delta). Pub/Sub or Kafka topic name When true, emit awslakeformationresource that registers the binding's S3 path with Lake Formation. Required before LF can manage access to objects under that path. Pub/Sub subscription name NEW in v0.7.6: Which grants[] principals get a statement in the awss3bucketpolicy emitted beside the grants (s3:ListBucket and s3:GetBucketLocation on the bucket, s3:GetObject under location.path). A statement lets its principal read the objects straight from S3, which skips Lake Formation's column, row and cell filters and its revocations. 'cross-account' (the default): only principals whose ARN names an AWS account other than the one running the apply, decided at plan time against the caller identity; same-account grantees get no statement, since on a registered location Lake Formation vends them credentials. 'none': no bucket policy at all; the tightest setting when the location is registered (registerLocation) and every reader, cross-account ones included, queries through a Lake Formation integrated engine, or when the bucket policy is managed elsewhere. 'all-grantees': every grantee, same-account included, which is the output before this field existed and reopens the direct S3 read path. An emitted awss3bucketpolicy is authoritative: it replaces every other statement on the bucket. When no grantee is left none is emitted. Cloud region_masked, that returns the columns masked by the restriction's rule and applies its row filter; its grant on the table excludes the columns. The definer is granted SELECT with grant option on the table and DESCRIBE on the database; the principal SELECT and DESCRIBE on the view.",
+ "properties": {
+ "definer": {
+ "type": "string",
+ "description": "ARN of the IAM role the views are defined as: Glue and Lake Formation must be allowed to assume it (sts:AssumeRole, sts:SetContext), and the applying principal to pass it (iam:PassRole)."
+ },
+ "validationConnection": {
+ "type": "string",
+ "description": "The Glue connection of type VIEW_VALIDATION_ATHENA (WORKGROUP_NAME) Glue validates each view's SQL through; the applying principal and the definer need glue:PassConnection on it."
+ }
+ }
+ }
+ }
+ }
+ }
+ },
+ "consumerRef": {
+ "type": "object",
+ "additionalProperties": false,
+ "required": [
+ "name",
+ "type"
+ ],
+ "$comment": "dbt-exposures-shaped (name/label/type/owner/url/maturity/description); exposeIds is the FLUID-native refinement tying a consumer to specific output ports.",
+ "properties": {
+ "name": {
+ "$ref": "#/$defs/identifier",
+ "description": "Stable identifier for this consumer (unique within the contract)."
+ },
+ "label": {
+ "type": "string",
+ "description": "Human-facing display name, e.g. \"Weekly Revenue Dashboard\"."
+ },
+ "type": {
+ "type": "string",
+ "enum": [
+ "dashboard",
+ "notebook",
+ "analysis",
+ "ml",
+ "application"
+ ],
+ "description": "What kind of artifact consumes this product."
+ },
+ "owner": {
+ "type": "object",
+ "additionalProperties": false,
+ "$comment": "Mirrors metadata.owner's vocabulary so 'owner' means one thing across the standard: team is the routable identity, email the contact. A dbt exposure's owner.name maps to team on import.",
+ "properties": {
+ "team": {
+ "type": "string",
+ "description": "Owning team for this consumer (routable identity, as in metadata.owner)."
+ },
+ "email": {
+ "type": "string",
+ "format": "email",
+ "description": "Contact address for the consumer's owner."
+ }
+ }
+ },
+ "url": {
+ "type": "string",
+ "description": "Link to the live artifact (dashboard URL, notebook, app)."
+ },
+ "maturity": {
+ "type": "string",
+ "enum": [
+ "high",
+ "medium",
+ "low"
+ ],
+ "description": "How production-hardened the consumer is (dbt exposures semantics)."
+ },
+ "description": {
+ "type": "string"
+ },
+ "exposeIds": {
+ "type": "array",
+ "items": {
+ "$ref": "#/$defs/identifier"
+ },
+ "description": "Which output ports this consumer reads. Absent = all exposes."
+ }
+ }
+ }
+ }
+}
\ No newline at end of file
diff --git a/scripts/schema-versions.json b/scripts/schema-versions.json
index 45e5153..54c50b7 100644
--- a/scripts/schema-versions.json
+++ b/scripts/schema-versions.json
@@ -11,12 +11,12 @@
"repository": "Agenticstiger/forge-cli",
"schemaDir": "fluid_build/schemas",
"previewSource": "fluid_build/schema_manager.py",
- "ref": "v0.21.0",
- "commit": "91e071e2c2273e2ba06fd0deabfe42d78de82f2e"
+ "ref": "v0.27.0",
+ "commit": "c0e97c076f735205f4767f67f0bd05bcd78a47cd"
},
- "latestStable": "0.7.5",
- "preview": ["0.7.6"],
- "synced": ["0.7.2", "0.7.3", "0.7.4", "0.7.5", "0.7.6"],
+ "latestStable": "0.7.6",
+ "preview": ["0.7.7"],
+ "synced": ["0.7.2", "0.7.3", "0.7.4", "0.7.5", "0.7.6", "0.7.7"],
"notSynced": {
"0.7.1": "The reference implementation bundles a different document under the same $id (it adds orchestration, icebergConfig, binding.properties and more). Re-vendoring it would change which documents the published 0.7.1 accepts, so it is a decision for the maintainers rather than a sync, and the published copy is left as it is.",
"0.0.1-0.5.7": "Predate the compatibility promise; the reference implementation no longer bundles them."
diff --git a/specs/0.7.6/fluid-spec.html b/specs/0.7.6/fluid-spec.html
index 5a43501..43d2378 100644
--- a/specs/0.7.6/fluid-spec.html
+++ b/specs/0.7.6/fluid-spec.html
@@ -13,16 +13,18 @@
-
FLUID 0.7.6 β Declarative Packaging Modes (preview)
Type: object
+ FLUID 0.7.6 β Declarative Packaging Modes
Type: object
β’ Declarative packaging modes β a top-level packaging block (plus a per-exposure binding.packaging override) declares per-container ownership for the IaC emit: isolated (this product creates and owns the container) vs shared (a pre-existing, platform-owned pool referenced via an OpenTofu data source), with a per-kind containers override map for hybrid tiers and a pool id for tenancy/cost attribution (RFC-packaging-modes.md).
+β’ Optional Bitol-shaped fields the exporters already wanted, each one opt-in: metadata.team (a roster β name plus members, every member a $defs.teamMember whose only required field is username), the root description as {purpose, usage, limitations} as well as a plain string, consumes[].contractId (an upstream that has a contract id of its own), consumes[].contractVersion (the exact upstream version this product was built against, inside that range), exposes[].contract.id (used verbatim as the ODCS id and the ODPS output port contractId instead of the derived <product id>.<exposeId>; two exposes may not share one) and binding.location.host plus binding.serialization.{value, key} for Kafka. A contract that sets none of them validates and exports exactly as it does today.
+β’ exposes[].contract.schema[].quality β column-level data quality rules in ODCS's own shape. The ODCS exporter always replayed this key verbatim and read it back on import, but $defs.column is closed and did not define it, so the one way to publish a hand-written column rule was to write a contract fluid validate refuses. Optional: a column without it exports exactly as it does today.
β’ Backward compatible with v0.7.5 β every 0.7.5 contract validates as 0.7.6 unchanged; an ABSENT packaging block keeps the legacy byte-identical emit.
/>
fluidVersionType: enum (of string)
-Must be one of:
@@ -359,8 +361,264 @@
d="M4 8a.5.5 0 0 1 .5-.5h5.793L8.146 5.354a.5.5 0 1 1 .708-.708l3 3a.5.5 0 0 1 0 .708l-3 3a.5.5 0 0 1-.708-.708L10.293 8.5H4.5A.5.5 0 0 1 4 8z"
/>
- description
+ description
+purpose on export β or the structured object. While this was a string, an export of a valid contract could only ever carry description.purpose: usage and limitations had no home a contract could declare, so every ODCS and ODPS export left them empty. fluid odps import restores the object to the root description when the schema version the imported contract declares accepts an object, and also keeps a verbatim copy in extensions.odps.metadata.description so the ODPS leg stays an exact fixed point. All three fields therefore survive a FLUID β ODPS β FLUID cycle and reach both the ODCS and the ODPS export afterwards. Importing against a version whose root description is a string still roots purpose alone, with the object recoverable from that bucket.
+
+
+
+description.purpose.
+description. The three field names are Bitol's β ODPS v1.1.0 $defs.Description and the ODCS v3.1.0 root description both use exactly purpose/usage/limitations β so import and export are a pass-through rather than a translation.
+
+
+
+
+
+
+
+
+
+
+
+
-
-
+
+metadata.owner holds one team name and one mailbox and carries no role, so a product with a data owner, a steward and two engineers had nowhere to say so and the exported ODCS/ODPS team block came out empty. The field names are Bitol's, so an imported team round-trips without a translation. Optional: a contract without it exports exactly as it does today.
-
+
-
+
+Each item of this array must be:
-
-
-
+metadata.team.members. The field names and the username-only required list are Bitol's ODCS v3.1.0 / ODPS v1.1.0 $defs.TeamMember, copied verbatim so an imported team exports back unchanged.
-
+
+
-
+
+
-
+
-
-
-
-
-
-
-
-
-
- 1 itemsEach item of this array must be:
-
+
-{
- "exposeId": "customer_profiles",
- "title": "Customer Profiles Table",
- "kind": "table",
- "tags": [
- "pii",
- "customer-facing",
- "analytical"
- ],
- "labels": {
- "sensitivity": "high",
- "retention": "7-years",
- "team": "customer-analytics"
- },
- "contract": {
- "schema": [
- {
- "name": "customer_id",
- "type": "STRING",
- "required": true,
- "sensitivity": "cleartext",
- "tags": [
- "identifier",
- "primary-key"
- ],
- "labels": {
- "business-name": "Customer ID",
- "data-category": "identifier"
- }
- }
- ]
- },
- "binding": {
- "platform": "gcp",
- "format": "bigquery_table",
- "location": {
- "project": "company-data",
- "dataset": "customer",
- "table": "profiles"
- }
- }
-}
-
-
+
-
-
-
-
+
+
@@ -1968,18 +2144,18 @@
-
+
-^(0|[1-9]\d*)\.(0|[1-9]\d*)\.(0|[1-9]\d*)(?:-[0-9A-Za-z.-]+)?(?:\+[0-9A-Za-z.-]+)?$
+ businessContext
+
-
+
-Must be one of:
-
-
+Must be one of:
-
+
-
+
-
+
-
+
+
-
-
-
+
+
+
+
+
+ 1 itemsEach item of this array must be:
+
-
-
-
-
+{
+ "exposeId": "customer_profiles",
+ "title": "Customer Profiles Table",
+ "kind": "table",
+ "tags": [
+ "pii",
+ "customer-facing",
+ "analytical"
+ ],
+ "labels": {
+ "sensitivity": "high",
+ "retention": "7-years",
+ "team": "customer-analytics"
+ },
+ "contract": {
+ "schema": [
+ {
+ "name": "customer_id",
+ "type": "STRING",
+ "required": true,
+ "sensitivity": "cleartext",
+ "tags": [
+ "identifier",
+ "primary-key"
+ ],
+ "labels": {
+ "business-name": "Customer ID",
+ "data-category": "identifier"
+ }
+ }
+ ]
+ },
+ "binding": {
+ "platform": "gcp",
+ "format": "bigquery_table",
+ "location": {
+ "project": "company-data",
+ "dataset": "customer",
+ "table": "profiles"
+ }
+ }
+}
+
-
+
-^sha256:[0-9a-fA-F]{64}$
-
-
-
-
-
-
+ exposeId
+
-
+
-Must be one of:
-
-
+
@@ -2423,18 +2607,18 @@ Must be one of:
-
+
+Each item of this array must be:
+
+
+
+
+
+^(0|[1-9]\d*)\.(0|[1-9]\d*)\.(0|[1-9]\d*)(?:-[0-9A-Za-z.-]+)?(?:\+[0-9A-Za-z.-]+)?$
-
-
+
+Must be one of:
+
+ 1 characters long
-
+
+
+
+
-
-
-
-
+
+
+
+
+
+
+ No Additional Properties
+
+
+
-Must be one of:
-
-
+
+ The following properties are required:
+
+
-Must match regular expression: (?i)^\s*(string|text|varchar|varchar2|nvarchar|char|nchar|character|clob|int|integer|int2|int4|int8|int16|int32|int64|tinyint|smallint|mediumint|bigint|long|longint|serial|bigserial|float|float4|float8|float32|float64|double|real|double\s+precision|decimal|dec|numeric|number|bignumeric|money|boolean|bool|bit|date|time|datetime|datetime2|smalldatetime|timestamp|timestamptz|timestamp_tz|timestamp_ntz|timestamp_ltz|timestampntz|timestamp\s+with(out)?\s+time\s+zone|interval|year|variant|object|array|struct|map|record|row|json|jsonb|super|binary|varbinary|bytes|blob|bytea|raw|geography|geometry|geom|point|uuid|uniqueidentifier|guid|enum|hll)\s*(\(\s*[0-9A-Za-z_,\s'\"-]*\s*\))?\s*$
-
+ item 1
+
+ The following properties are required:
+
+ Must be one of:
-
-
+
+id and as the ODPS output port contractId. Without it the id is derived as <product id>.<exposeId>, so renaming the product or the expose silently changes the contract id and breaks every consumer reference to it, and a catalog that requires an id format of its own (a UUID, say) cannot be served at all. Each expose needs its own: two exposes sharing one id is refused by fluid validate.1 characters long
-
+
+^sha256:[0-9a-fA-F]{64}$
@@ -2948,18 +3107,18 @@
-
+
-
+Must be one of:
-
+ Must be one of:
-
+
+Each item of this array must be:
-
-
-
+
+
+ No Additional Properties
-
-
+
+
+ 1 characters long
-
+
-Each item of this array must be:
-
+
+
+
+
+Must be one of:
+
+
-
-
-
+Must match regular expression: (?i)^\s*(string|text|varchar|varchar2|nvarchar|char|nchar|character|clob|int|integer|int2|int4|int8|int16|int32|int64|tinyint|smallint|mediumint|bigint|long|longint|serial|bigserial|float|float4|float8|float32|float64|double|real|double\s+precision|decimal|dec|numeric|number|bignumeric|money|boolean|bool|bit|date|time|datetime|datetime2|smalldatetime|timestamp|timestamptz|timestamp_tz|timestamp_ntz|timestamp_ltz|timestampntz|timestamp\s+with(out)?\s+time\s+zone|interval|year|variant|object|array|struct|map|record|row|json|jsonb|super|binary|varbinary|bytes|blob|bytea|raw|geography|geometry|geom|point|uuid|uniqueidentifier|guid|enum|hll)\s*(\(\s*[0-9A-Za-z_,\s'\"-]*\s*\))?\s*$
+
+
+
+
+
+
+ Must be one of:
-
+
-
+
-
+
-
+
+
-
+
-
+
-
+
+
-
+
-Must be one of:
-
-
+Each item of this array must be:
-
-
-
+ No Additional Properties
- No Additional ItemsEach item of this array must be:
+
+
+
+
+
-Type: enum (of string)
+Must be one of:
-
+ Must be one of:
-
-
+
-
-
-
-
- No Additional ItemsEach item of this array must be:
+
+
+
+
+
- No Additional Properties
-
-
+
+schema[].properties[].quality[] of the exported ODCS v3.1.0 contract, and the array an imported ODCS column's rules are read back into, inline on the column rather than in the importer's pass-through bucket: the ODPS importer merges a resolved port's contract block into its output-port stub and discards the sub-import's root extensions, so rules left in that bucket were dropped outright by a FLUID β ODPS β FLUID cycle. The ODCS exporter has always replayed this key verbatim and always read it back on import, but $defs.column is additionalProperties: false and did not define it, so a column written the way the exporter supports was REFUSED by fluid validate: the only way to get a hand-written quality rule into a published contract was to write an invalid contract. Optional and additive β a column without it exports exactly as it does today, with the rules forge derives from required, the primary-key tag and validationRules. Setting it REPLACES those derived rules for that column rather than adding to them, which is what makes an imported ODCS column round-trip byte-for-byte. The authority for an item's contents is ODCS v3.1.0 $defs.DataQuality, deliberately not re-declared here (a copy would drift from the vendored schema, and fluid odcs validate already checks an emitted document against it); the keys below are the common ones. Every rule should carry an id: it is optional in ODCS, but a DQ engine that labels its results by rule id rejects a contract whose rules have none.Each item of this array must be:
-
-
-
-Must be one of:
-
-
+
-
-
+
+$defs.StableId. Rules forge derives itself are named <column>-<metric>.^[A-Za-z0-9_-]+$
@@ -4430,18 +4568,18 @@
-
+
+Must be one of:
+
+
-
+
-
+Must be one of:
-
+ Must be one of:
-
+
+pattern (a regex) or validValues (a list) for invalidValues, as DataHub's ODCS ingestion source reads them.^P(?!$)(\d+Y)?(\d+M)?(\d+W)?(\d+D)?(T(\d+H)?(\d+M)?(\d+S)?)?$
@@ -4661,18 +4823,18 @@
-
+
-Must be one of:
-
-
+Must be one of:
-
+
+
-
+
-
+
-
+Must be one of:
+
-
+
- No Additional Properties
+ schema items
+
+ quality
+
+ quality items
+
+ description
+
-
-
+
+
-
+
-
+
-
+
+
-
+
-
-
-
-
+
-
+Each item of this array must be:
+
+
+
+
+
+ No Additional Properties
-
-
+
+Must be one of:
+
+
-
+
+
-
- Each item of this array must be:
-
-
-
-
+Must be one of:
-
+ Must be one of:
-
-
+
+
-
+
-
-
-
+
+
+
+
+
+
+ No Additional ItemsEach item of this array must be:
+
-
-
- No Additional Properties
-
-
-
-
-
-
-
-
-
-
- Must match regular expression: ^(100(\.0+)?|\d{2}(\.\d+)?|\d{1}\d(\.\d+)?)%$
+
+ No Additional Properties
-
-
+
-
-
-
-
+
-
-
-
+Must be one of:
+
+ 0 and lesser or equal to 1
-
+
+
- 0 and lesser or equal to 1
-
+
-
-
- No Additional Properties
-
-
-
-
-
-
-
-
-
-
-
+Must be one of:
-
+ Must be one of:
-
+
+^P(?!$)(\d+Y)?(\d+M)?(\d+W)?(\d+D)?(T(\d+H)?(\d+M)?(\d+S)?)?$
-
-
+
+Must be one of:
+
+ Each item of this array must be:
+
+
+
+
+
-
+
+
+
+
-
+
-fluid apply enforces it. GCP BigQuery: a Data Catalog taxonomy per product and dataset with fine-grained access control, a policy tag per set of restricted columns that share their readers, attached through the table schema's policyTags, and roles/datacatalog.categoryFineGrainedReader for exactly the expose's readers allowed to read them: the accessPolicy read grantees and the expose's own policy.authz.readers (a denied principal gets an access error on the columns). A restriction on an expose with no reader on the binding is refused, as it would lock the columns for everyone. A restriction's tags and labels are written into the policy tag's description; AWS has no place for them. AWS: each governance.lakeFormation grant with SELECT excludes the restricted columns its principal may not read (tablewithcolumns.excludedcolumnnames); a grant's hand-written excludedColumns must agree, and a binding with no Lake Formation grants is refused. fluid verify checks the policy tags and their readers on GCP, and the principals' Lake Formation permissions on AWS. NEW in v0.7.6: access: mask lets the principal read the columns masked by the platform (see mask): a deny beats a mask, and a mask beats an allow for the principal it names.
+ No Additional Properties
- No Additional ItemsEach item of this array must be:
+
+
+
+
+
+
+
+
+
+
+
-
-
-
-
+
+
+
+
+
+
+
+
+
+
+
-
+
+
+
+
+Each item of this array must be:
+
-
-
+
+
+
-
-
-
-
+
+
+
+
+
+
+
+
+Must be one of:
+
+
-
+
+
+
+
-
+
+
+
+ No Additional Properties
+
+
+
+
+
+
+
+
+
+
-
+^(100(\.0+)?|\d{2}(\.\d+)?|\d{1}\d(\.\d+)?)%$
@@ -7505,18 +7656,18 @@
-
+
+
+
+
+Each item of this array must be:
+
+
+
+
+
+
+
+
+
+
+
+0 and lesser or equal to 1
-
+
-Must be one of:
-
-
+0 and lesser or equal to 1
-
+
-
+
-
+
+
+
+ No Additional Properties
+
+
+
+
+
+
+
+
+
+
-
+Must be one of:
-
+ Must be one of:
-
+
+ No Additional Properties
+
+
+
+
+
+
+
-
+
-
-
-
- Each item of this array must be:
+
-
- Each item of this array must be:
+
+
+
+
+
+
-
-Each item of this array must be:
-
-
-
-
+
+
-
+
-^[a-z][a-z0-9_]{0,62}$
+ columnRestrictions
+fluid apply enforces it. GCP BigQuery: a Data Catalog taxonomy per product and dataset with fine-grained access control, a policy tag per set of restricted columns that share their readers, attached through the table schema's policyTags, and roles/datacatalog.categoryFineGrainedReader for exactly the expose's readers allowed to read them: the accessPolicy read grantees and the expose's own policy.authz.readers (a denied principal gets an access error on the columns). A restriction on an expose with no reader on the binding is refused, as it would lock the columns for everyone. A restriction's tags and labels are written into the policy tag's description; AWS has no place for them. AWS: each governance.lakeFormation grant with SELECT excludes the restricted columns its principal may not read (tablewithcolumns.excludedcolumnnames); a grant's hand-written excludedColumns must agree, and a binding with no Lake Formation grants is refused. fluid verify checks the policy tags and their readers on GCP, and the principals' Lake Formation permissions on AWS. NEW in v0.7.6: access: mask lets the principal read the columns masked by the platform (see mask): a deny beats a mask, and a mask beats an allow for the principal it names.Each item of this array must be:
-
-
-
-1 characters long
-
-
-
+ No Additional Properties
-
-
-
-
- No Additional Properties
-
-
-
-
-
-
-
-
-
-
-
- Each item of this array must be:
+
+
+
+
+
- No Additional Properties
+ columnRestrictions items
+
+ allOf
+
+ item 0
+
+ if
+
+ access
+Specific value: "mask"
-
-
-
+
-
-
-
-Must be one of:
-
-
+
+
+
+
-
-
-
-
+
+ The following properties are required:
+
+
-
-
-
-^[A-Z_][A-Z0-9_]{0,127}$
+ item 1
+
+ then
+
-
-
+
-^[A-Z_][A-Z0-9_]{0,127}$
+ item 1
+
+ then
+
+ access
+Specific value: "mask"
@@ -9074,18 +9217,32 @@
-
+
-
+0 and lesser or equal to 64
-
+
-
+0 and lesser or equal to 64
-
-
- Each item of this array must be:
+
-
+
-
+
+
-
+
- No Additional Properties
-
-
-
-
-
-
-Type: object
+
-
+
+Must be one of:
+
+
-
+
-
+
-
+
-
-
+
+
- Each item of this array must be:
+
+ No Additional Properties
+
+
+
+
+
+
+
-
+
-Must be one of:
-
-
+Must be one of:
-
+
-
+^[a-z][a-z0-9_]{0,62}$
-
-
+
-
+1 characters longEach item of this array must be:
+
+
+
+
+
-Must match regular expression: ^[a-z0-9][a-z0-9-_.]*[a-z0-9]$|^[a-z0-9]$
+ rowFilters items
+
+ description
+
-
-"gpt-4"
-"claude-3-opus"
-"claude-3-sonnet"
-"gemini-pro"
-"gemini-ultra"
-"llama-3"
-"mixtral-8x7b"
-
-
+
+ No Additional Properties
+
+
+
+
+
+
+
+
+
+
-
+Each item of this array must be:
+ No Additional ItemsEach item of this array must be:
-
+
-
+
- 1
-4096
-8192
-16384
-
-
+
-
+Must be one of:
+
+ 1
-
+
-
+Each item of this array must be:
+
+
+
+
+
-Must be one of:
-
-
-[
- "inference",
- "reasoning",
- "summarization"
-]
-
[
- "search",
- "qa",
- "rag"
-]
-
+^[A-Z_][A-Z0-9_]{0,127}$
+
+
+
+
+
+
-
+
-Each item of this array must be:
-
-Must be one of:
-
-
+^[A-Z_][A-Z0-9_]{0,127}$
-
-[
- "training",
- "fine_tuning"
-]
-
-
+
-
+0 and lesser or equal to 64
-
+
-
+0 and lesser or equal to 64
-
+
-
+
-
-
+
-
+
-
+
-
+
+ No Additional Properties
-
-
+
-
+
-
+
-
+
-
+
-
+
-
-
- No Additional Properties
-
-
-
+Must be one of:
+
+
+
+
+
-
-
-
-
+
+
+
+
+Each item of this array must be:
-
-
-
+Must match regular expression: ^[a-z0-9][a-z0-9-_.]*[a-z0-9]$|^[a-z0-9]$
-
+
+"gpt-4"
+"claude-3-opus"
+"claude-3-sonnet"
+"gemini-pro"
+"gemini-ultra"
+"llama-3"
+"mixtral-8x7b"
+
+
+
+
+Each item of this array must be:
-
-
-
-
+
-Must not be:
-
+1
+4096
+8192
+16384
+
+
+
+
-Must match regular expression: ^(alias/|arn:)
-
-
-
-
-
-
-
+1
-
+
+Each item of this array must be:
-
-
-
+Must be one of:
+
+
+[
+ "inference",
+ "reasoning",
+ "summarization"
+]
+
[
+ "search",
+ "qa",
+ "rag"
+]
+
+
+
+
-
-
-
-
+Each item of this array must be:
+
-Must match regular expression: ^(user|group|serviceAccount|domain):.+$
+ deniedUseCases items
+Must be one of:
+
+
+[
+ "training",
+ "fine_tuning"
+]
+
+
+
+
+
+
+
+
+Each item of this array must be:
+
+
+
+
+
+
+
+
+
-Must match regular expression: ^(user|group|serviceAccount|domain):.+$
-
-
-
-
-
-
-
+0
+
+
-
-
-
-
+
+
+
+
+
-
+
+
+
+
+
-
+
-
-
-
-
-
-
-
-
+
-
+
+
+
+
-Must not be:
-
+
+
+
+
+
+
+ No Additional Properties
+
+
+
-Must match regular expression: ^projects/
-
-
-
-
-
-
-
+
+
+
+
-
-
-
@@ -12797,19 +13239,19 @@
-
-
+
-
-
-
-
+
+
+
+
+
+
+
+
+
+
+
+
-Must match regular expression: ^arn:aws[a-z0-9-]*:iam::
-
-
-
-
-
-
-
+
+
+
+
+
+
+
+
+
+
+
-
-
-
-
-
-
- No Additional ItemsEach item of this array must be:
-
+Must not be:
+
+Must match regular expression: ^(alias/|arn:)
-
-
+
-Must be one of:
-
-
-
-
-
-Must be one of:
-
-
+
-
-
+
-
-
-
-
-Must be one of:
-
-
-
-
-
+
+
+
+
-Must be one of:
-
-
-
-
-
+Must match regular expression: ^(user|group|serviceAccount|domain):.+$
- No Additional ItemsEach item of this array must be:
-
-
-
-
-
-
-
-
-
-
-
-
+
+
+
+
+
+
+ No Additional ItemsEach item of this array must be:
+
-
-
-
-
+Must match regular expression: ^(user|group|serviceAccount|domain):.+$
+
+
+
+
+
+
+
-
-
+
-Must be one of:
-
-
+
+
+
+
-
-
-
-
+
+Specific value: "aws"
@@ -13957,25 +14234,13 @@
-
-
-
+
- No Additional ItemsEach item of this array must be:
+
+
+
+
+
@@ -14075,18 +14363,18 @@
-
-
+
-
-
-
-
+Must not be:
+
-Must be one of:
-
-
+Must match regular expression: ^projects/
+
+
+
+
+
+
+ Must be one of:
-
+
-Must be one of:
-
-
+
-
-
+
-
-
-
-
+
+
+
+
-
+ binding
+
+ allOf
+
+ item 1
+
+ then
+
+ principals
+
+ additionalProperties
+
+ anyOf
+
+ item 0
+Must match regular expression: ^arn:aws[a-z0-9-]*:iam::
-
-
-
-
-
+
-
-Each item of this array must be:
-
-
-
-1 and lesser or equal to 16000
-
-
-
-
+Must match regular expression: ^arn:aws[a-z0-9-]*:iam::
+
+
+
+
+
+
+
-
+
-
+Must be one of:
-
+ Must be one of:
-
+
-
+Must be one of:
-
+ Must be one of:
-
+
-Must be one of:
-
-
+
-
+
-
+Must be one of:
+
+
-
+
-
+Must be one of:
+
+
-
+
-
+Each item of this array must be:
-
-
-
+
- 1
-
+
-
+1
-
+
-
+
-
+
-
+Must be one of:
+
+ 1
-
+
-
+
-
+
+
-
+
+Each item of this array must be:
-
-
-
+
-
-
+
+
-
+
+Must be one of:
+
+
-
+
+Must be one of:
+
+
-
+
+
-
-
+
-
+
@@ -15827,18 +16243,26 @@
-
+
+
-
+
+1 and lesser or equal to 16000
-
+
+
-
+
+Must be one of:
+
+
-
+
+Must be one of:
+
+
-
+
+Must be one of:
+
+
-
+
-
+
-
+
-
+
-
+
-
+
-
+
-
+1
-
+
-
+
-
+
-
+
-
+
-
+
-
+
-
+
-
+
-
+
-
+
-
+
-
+
-
+
-retention after its date, so a backfill of older rows is deleted at once. Leave it out for the S3 rule's semantics (age since written).
+Each item of this array must be:
+
+
+
+
+
-
+
-
+
-
+
-
+
-
+
+host and falls back to account, so until now the only valid way to set the bootstrap servers was a field named for something else β and with neither set, the exported ODCS server is the placeholder ${KAFKA_BOOTSTRAP_SERVERS}. Usually set per environment in an overlay rather than in the base contract.
-
+
-
+
-
+
-
+
-
+
-
+
-
+
-Must be one of:
-
-
+Must be one of:
-
+