Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 5 additions & 2 deletions docs/advanced/error-codes.md
Original file line number Diff line number Diff line change
Expand Up @@ -347,7 +347,7 @@ The state refusals (`state_shared_with_another_provider`, `state_migration_ambig
| Kind | Triggered by | Fix |
|---|---|---|
| `databricks-tag-collision` | Two labels that become one tag key once `-` is read as `_` (`cost-center` and `cost_center`), or a label that becomes one of the product's own tag keys *(since 0.28.0, [forge-cli #831](https://github.com/Agenticstiger/forge-cli/pull/831))*. Only a key the contract states counts: with no `metadata.productType`, a label `fluid_product_type` is no collision | Rename the label so its key differs |
| `databricks-tag-value-too-long` | A tag value over 256 characters: the contract's `id`, `name`, `domain`, `metadata.layer`, `metadata.productType`, `metadata.owner.team`, `fluidVersion` or the expose's `version`, or a column's `sensitivity` | Shorten it to 256 characters or fewer. forge-cli refuses rather than cuts, since a cut id names another product |
| `databricks-tag-value-too-long` | A tag value over 256 characters: the contract's `id`, `name`, `domain`, `metadata.layer`, `metadata.productType`, `metadata.owner.team`, `fluidVersion` or the expose's `version`, or a column's `sensitivity`. *([forge-cli #868](https://github.com/Agenticstiger/forge-cli/pull/868), since 0.30.0)* Also a governance label's value, which is no longer cut to 63 characters | Shorten it to 256 characters or fewer. forge-cli refuses rather than cuts, since a cut id names another product |
| `databricks-tags-too-many` | More than 50 tags on a table or schema, counting the product's own tags with the governance labels, or more than 1,000 column tags on a table *(since 0.28.0, [forge-cli #831](https://github.com/Agenticstiger/forge-cli/pull/831))* | Remove labels until 50 or fewer remain, or classify 1,000 columns or fewer |
| `databricks-comment-unsafe` | A table or column description that holds a backslash (the provider escapes only quotes in `COMMENT '...'`, and Databricks reads a backslash in a literal as an escape, so a trailing one, or one before a quote, would break the literal) or `${` (Spark may substitute a variable there) | Remove the backslash, or write `$ {` for `${` |
| `databricks-certification-invalid` | The label `certification` has any value but `certified` (case ignored), on a product that is not `deprecated`. Unity Catalog's `system.certification_status` takes `certified` or `deprecated` | Set `labels.certification: certified` or remove it. Mark a deprecated product with `lifecycle.state: deprecated` |
Expand Down Expand Up @@ -423,7 +423,10 @@ The `fluid generate iac` and `fluid apply` runs print the refusal and exit befor

- Move the offending binding into an allowed region, or widen 'sovereignty.allowedRegions' in the contract
- Run 'fluid validate <contract>' to see the same findings with full detail
- Set 'sovereignty.enforcementMode: advisory' to report without blocking
- Set 'sovereignty.enforcementMode: advisory' to report a region outside 'allowedRegions' without blocking; a region in 'deniedRegions' blocks in every mode
- Pass --no-check-sovereignty to write the plan anyway (logged at WARNING; fluid apply keeps its own checks)

*([forge-cli #868](https://github.com/Agenticstiger/forge-cli/pull/868), since 0.30.0)* `fluid plan` raises it by default, before it writes a plan, when the contract's own sovereignty policy blocks; see [Sovereignty gate](../cli/plan.md#sovereignty-gate-since-0-15-0). The third suggestion is reworded and the fourth is new. Up to 0.29.0 the third read "Set 'sovereignty.enforcementMode: advisory' to report without blocking", although a region in `deniedRegions` blocks in every mode.

## Product authoring

Expand Down
4 changes: 2 additions & 2 deletions docs/advanced/governance.md
Original file line number Diff line number Diff line change
Expand Up @@ -220,7 +220,7 @@ A contract's `sovereignty` block declares where its data may live. `enforcementM

| `enforcementMode` | Severity | Effect |
|---|---|---|
| `strict` *(the schema default)* | ❌ error | [`fluid validate`](../cli/validate.md) exits 1; [`fluid plan --check-sovereignty`](../cli/plan.md#sovereignty-gate-since-0-15-0) blocks. |
| `strict` *(the schema default)* | ❌ error | [`fluid validate`](../cli/validate.md) exits 1; [`fluid plan --check-sovereignty`](../cli/plan.md#sovereignty-gate-since-0-15-0) blocks, and *([forge-cli #868](https://github.com/Agenticstiger/forge-cli/pull/868), since 0.30.0)* so does plain `fluid plan`. |
| `advisory` | ⚠️ warning | Reported, does not block, though `fluid validate --strict` promotes warnings to errors. |
| `audit` | ℹ️ info | Logged only. |

Expand Down Expand Up @@ -292,7 +292,7 @@ A Cloud KMS key ring and a Data Catalog taxonomy are placed at their dataset's m
| Stage | What it does |
|---|---|
| [`fluid validate`](../cli/validate.md) | Runs the policy engine on every contract; severity follows `enforcementMode`. |
| [`fluid plan --check-sovereignty`](../cli/plan.md#sovereignty-gate-since-0-15-0) | Opt-in when you run `plan` yourself. Since 0.17.0 every pipeline that `fluid generate ci` emits runs stage 6 as `fluid plan ... --out runtime/plan.json --check-sovereignty`, so a strict violation fails before any stage reaches a cloud. A contract with no `sovereignty` block prints `NOT CHECKED` and passes. |
| [`fluid plan`](../cli/plan.md#sovereignty-gate-since-0-15-0) | *([forge-cli #868](https://github.com/Agenticstiger/forge-cli/pull/868), since 0.30.0)* Runs the policy engine by default and refuses what `fluid validate` refuses, before it writes a plan; `--no-check-sovereignty` skips it with a warning. `--check-sovereignty` also asks the provider's hook and prints the verdict; up to 0.29.0 only that flag checked. Since 0.17.0 every pipeline that `fluid generate ci` emits runs stage 6 as `fluid plan ... --out runtime/plan.json --check-sovereignty`, so a strict violation fails before any stage reaches a cloud. With that flag, a contract with no `sovereignty` block prints `NOT CHECKED` and passes. |
| [`fluid generate iac`](../cli/generate-iac.md) / [`fluid apply`](../cli/apply.md) | The AWS provider, and since 0.17.0 the GCP provider, refuse a placement outside the policy and exit 1, and **no module is written**. GCP checks every location the plan emits, including one a resource inherits by default. |
| Embedded-SQL builds on BigQuery | The read and the landing are checked against the policy too (`EmbeddedSqlSovereigntyError`). |
| [`fluid mcp output-port serve`](mcp.md#caller-jurisdiction-enforcement-since-0-15-0) | A pinned `jurisdiction` becomes a query-time gate on the **verified** caller jurisdiction, enforced by default, with the escape hatches in the contract and not in a flag. |
Expand Down
63 changes: 59 additions & 4 deletions docs/cli/plan.md
Original file line number Diff line number Diff line change
Expand Up @@ -25,7 +25,8 @@ fluid apply runtime/plan.json --mode amend-and-build --yes
```bash
fluid plan CONTRACT [--env ENV] [--mode MODE] [--out PATH]
[--provider NAME] [--project PROJECT] [--region REGION]
[--validate-actions] [--estimate-cost] [--check-sovereignty]
[--validate-actions] [--estimate-cost]
[--check-sovereignty | --no-check-sovereignty]
[--html [PATH]] [--verbose]
```

Expand All @@ -41,7 +42,8 @@ fluid plan CONTRACT [--env ENV] [--mode MODE] [--out PATH]
| `--verbose`, `-v` | Show detailed action information |
| `--validate-actions` | Validate generated provider actions against the ProviderAction SDK schema |
| `--estimate-cost` | Ask the provider to estimate cost |
| `--check-sovereignty` | *(since 0.15.0)* Resolve a real sovereignty verdict and name which source produced it — the provider's `validate_sovereignty` hook when it has one (the `gcp` provider has one since 0.17.0), otherwise the built-in policy engine (the same checker [`fluid validate`](./validate.md) runs), otherwise `NOT CHECKED`. **Exits 1 when the check fails.** Opt-in, off by default. See [Sovereignty gate](#sovereignty-gate-since-0-15-0). |
| `--check-sovereignty` | *(since 0.15.0)* Resolve a real sovereignty verdict and name which source produced it — the provider's `validate_sovereignty` hook when it has one (the `gcp` provider has one since 0.17.0), otherwise the built-in policy engine (the same checker [`fluid validate`](./validate.md) runs), otherwise `NOT CHECKED`. **Exits 1 when the check fails.** *([forge-cli #868](https://github.com/Agenticstiger/forge-cli/pull/868), since 0.30.0)* `plan` checks the policy without this flag too, and refuses before it writes a plan; the flag adds the provider's hook and the printed verdict. See [Sovereignty gate](#sovereignty-gate-since-0-15-0). |
| `--no-check-sovereignty` | *([forge-cli #868](https://github.com/Agenticstiger/forge-cli/pull/868), since 0.30.0)* Skip the sovereignty check `plan` runs by default, and write the plan whatever the policy says. Prints a warning and logs a WARNING (`sovereignty_check_skipped`). `fluid apply` keeps its own checks. |
| `--provider` | Override the provider from the contract |
| `--project` | Override the project / account from the contract |
| `--region` | Override the region / location from the contract |
Expand Down Expand Up @@ -95,7 +97,44 @@ A bundle is never re-overlaid: `fluid plan bundle.tgz --env prod` on a bundle bu

## Sovereignty gate (since 0.15.0)

`--check-sovereignty` stays **opt-in and off by default**, but on `0.15.0` it is a gate rather than a label. The verdict resolves in a fixed order, and the output always names which source answered:
*([forge-cli #868](https://github.com/Agenticstiger/forge-cli/pull/868), since 0.30.0)* `fluid plan` checks the contract's sovereignty policy by default, before it plans. It asks the built-in policy engine, the checker [`fluid validate`](./validate.md) runs, and refuses what `validate` refuses:

- Under `enforcementMode: strict`, the schema default, the findings that block include a binding outside `allowedRegions`, one in a region of another jurisdiction, and one on AWS, GCP, Azure or Databricks that declares no region (the platform would pick it).
- In every mode, a binding in a region listed in `deniedRegions` blocks.

A refusal exits 1 with `ERR_SOVEREIGNTY_VIOLATION` and writes no plan. A plan file an earlier run left at `--out` stays as it was. The check builds no provider, so it needs no cloud SDK and no credentials. When nothing blocks it prints nothing, and the plan is written as before: for a contract with no `sovereignty` block, and for one whose findings are only warnings or info lines (under `advisory` or `audit`, every finding but a denied region).

This is the refusal on 0.30.0 for a contract with `enforcementMode: strict`, `allowedRegions: [eu-central-1, eu-west-1]`, `deniedRegions: [us-west-2]` and one binding in `us-west-2`:

```text
Sovereignty check: 3 finding(s) — source: built-in policy engine, enforcementMode=strict
- ❌ [customer_events] Region 'us-west-2' is explicitly denied by sovereignty policy
💡 Use one of the allowed regions: eu-central-1, eu-west-1
- ❌ [customer_events] Region 'us-west-2' not in allowed regions list
💡 Allowed regions: eu-central-1, eu-west-1
- ❌ [customer_events] Region 'us-west-2' (jurisdiction: US) does not match required jurisdiction: EU
💡 Consider using regions in EU jurisdiction

❌ Sovereignty check FAILED — enforcementMode is 'strict'.
❌ sovereignty_violation [ERR_SOVEREIGNTY_VIOLATION]
command: plan
contract: .../contract.fluid.yaml
errors: ["[customer_events] Region 'us-west-2' is explicitly denied by sovereignty policy", "[customer_events] Region 'us-west-2' not in allowed regions list", "[customer_events] Region 'us-west-2' (jurisdiction: US) does not match required jurisdiction: EU"]
message: The contract's sovereignty policy (enforcementMode strict) blocks where it puts data, so fluid plan wrote no plan.
...
```

Each finding names its expose in brackets, and the error carries the findings as `errors`. The suggestions printed after it are listed under [`sovereignty_violation`](../advanced/error-codes.md#sovereignty-violation).

Up to 0.29.0 the check was opt-in: plain `fluid plan` wrote a plan for this contract and exited 0. To write the plan anyway now, pass [`--no-check-sovereignty`](#no-check-sovereignty).

::: warning Behavior change in 0.30.0
A script or pipeline that runs `fluid plan` on a contract `fluid validate` refuses for sovereignty now stops at plan, with exit 1 and no plan file. See [Changes in 0.30.0](../upgrading.md#changes-in-0-30-0).
:::

### `--check-sovereignty`

`--check-sovereignty` runs the same check first. If nothing blocks, plan then resolves a verdict and prints it, naming which source answered, in a fixed order:

1. The target provider's `validate_sovereignty` hook, when it returns a verdict.
2. Otherwise the **built-in policy engine** — the same sovereignty checker `fluid validate` runs, so the two stages cannot reach opposite verdicts on the same contract.
Expand Down Expand Up @@ -124,16 +163,32 @@ Sovereignty check: 10 violation(s) — source: gcp provider hook
❌ sovereignty_violation [ERR_SOVEREIGNTY_VIOLATION]
```

*([forge-cli #868](https://github.com/Agenticstiger/forge-cli/pull/868), since 0.30.0)* The second output predates 0.30.0. Now the default check runs first and refuses that contract under `strict` before the hook is asked: 0.30.0 prints the built-in engine's two findings for the `us-central1` binding (`not in allowed regions list` and the jurisdiction mismatch) and writes no plan. The hook's verdict is printed when the engine finds no error.

The pipeline `fluid generate ci --system github` writes runs `fluid plan ... --check-sovereignty` in its plan job, and the Jenkins pipeline's stage 6 does the same.

A failing check prints a `Sovereignty check: N finding(s)` header naming the source, then its findings as a bulleted list, then `❌ Sovereignty check FAILED`, and **exits 1**. The plan file is still written; the non-zero exit is what makes the flag usable as a CI gate.
A failing check prints a `Sovereignty check: N finding(s)` header naming the source, then its findings as a bulleted list, then `❌ Sovereignty check FAILED`, and **exits 1**. *([forge-cli #868](https://github.com/Agenticstiger/forge-cli/pull/868), since 0.30.0)* It writes no plan, as the default check does. Up to 0.29.0 the plan file was still written, and only the non-zero exit stopped a pipeline.

On `0.14.1` and earlier none of that happened. No shipped provider implemented `validate_sovereignty`, the hook helper returned an empty violation list for a hook that was absent (and for one that raised, because the invoker swallows the exception and hands back its first argument), and an empty list rendered as `Sovereignty check: PASS` with exit 0 — printed on contracts `fluid validate` rejects with two residency errors. The flag was also skipped outright when the provider failed to build. `PASS` is now printed only when a check actually ran and found nothing.

::: warning Behavior change in 0.15.0
A pipeline that already passes `--check-sovereignty` moves from a step that could only ever be green to one that can fail. Two rules decide whether it blocks: a region named in `deniedRegions` is an **error in every mode**, and everything else follows the contract's own `sovereignty.enforcementMode`, whose default is `strict`. Full table: [Governance → Sovereignty enforcement modes](../advanced/governance.md#sovereignty-enforcement-modes-since-0-15-0).
:::

### `--no-check-sovereignty`

*([forge-cli #868](https://github.com/Agenticstiger/forge-cli/pull/868), since 0.30.0)* `--no-check-sovereignty` skips the check and writes the plan whatever the policy says:

```bash
fluid plan contract.fluid.yaml --no-check-sovereignty --out runtime/plan.json
```

```text
⚠️ Sovereignty check skipped (--no-check-sovereignty): the plan is written whatever the contract's sovereignty policy says.
```

It also logs a WARNING with the event `sovereignty_check_skipped`, so the skip is on record. It changes nothing in `fluid apply`, which keeps its own checks; [Where sovereignty is enforced](../advanced/governance.md#where-sovereignty-is-enforced) lists them.

## Examples

### Basic plan
Expand Down
4 changes: 2 additions & 2 deletions docs/concepts/sovereignty.md
Original file line number Diff line number Diff line change
Expand Up @@ -138,7 +138,7 @@ This is the part most worth getting straight, because the two gates share a bloc
|---|---|---|
| **Question** | May the data *sit* here? | May this *caller* read it? |
| **Compares** | each expose's `binding.location.region` → jurisdiction, against `sovereignty.jurisdiction` | the caller's **verified** jurisdiction claim, against `sovereignty.jurisdiction` |
| **Runs on** | `fluid validate` (always); `fluid plan --check-sovereignty` (run by stage 6 of a generated pipeline); `fluid generate iac` and `fluid apply` on AWS and GCP | each `tools/call` that reaches the policy check at `fluid mcp output-port serve` |
| **Runs on** | `fluid validate` (always); `fluid plan` (by default *([forge-cli #868](https://github.com/Agenticstiger/forge-cli/pull/868), since 0.30.0)*; up to 0.29.0 only with `--check-sovereignty`, which stage 6 of a generated pipeline passes); `fluid generate iac` and `fluid apply` on AWS and GCP | each `tools/call` that reaches the policy check at `fluid mcp output-port serve` |
| **Since** | `0.7.1`, blocking by default since `0.15.0` | `0.15.0` |
| **Relaxed by** | `enforcementMode: advisory` / `audit` | `crossBorderTransfer: true` |

Expand Down Expand Up @@ -237,7 +237,7 @@ fluid apply plan.json --yes # after editing plan.json by hand

The two digests it prints are yours, not ours. A `planDigest` is a function of your plan, so the property to rely on is *stable across re-runs, different after an edit*, never a particular hex string.

So the division of labour is: `validate` (and `plan --check-sovereignty` in CI) decides whether the contract's residency claim is consistent; on AWS and GCP the provider refuses to create a resource outside the policy, including where it fills in a default; and the digest guarantees the artifact reaching your cloud is the one that decision was made about.
So the division of labour is: `validate` (and `plan`, which checks by default since 0.30.0) decides whether the contract's residency claim is consistent; on AWS and GCP the provider refuses to create a resource outside the policy, including where it fills in a default; and the digest guarantees the artifact reaching your cloud is the one that decision was made about.

## When `policy-compile` or `policy-apply` fails

Expand Down
Loading
Loading