feat(test): Databricks live checks, a Unity Catalog results table and an AI/BI dashboard

AGENTS.md

@@ -768,6 +768,33 @@ onprem = "my_pkg.iac:OnPremIacPlugin"
 - **Tests.** `tests/iac/test_iac_entrypoint_lazy_plugin.py` builds a real
   throwaway `.dist-info` and is the pattern to copy.
 
+## `fluid test` on Databricks: results table and dashboard (2026-11-12)
+
+- **Live checks.** `providers/databricks_validation.py` reads columns from the Unity
+  Catalog tables API and runs the row count and the rules on `DATABRICKS_WAREHOUSE_ID`
+  through `providers/databricks/statements.py` (Statement Execution API, `:name`
+  parameters, credential-scrubbed errors). Contract types go through
+  `engines/dbt/_types.sql_type(..., "databricks")` before they are compared.
+- **One reading of the rules.** `providers/quality_engine.py` is the native engine, and
+  `quality_engine.value_domain` is what a `valid_values` rule allows: the rule's own list,
+  else every declaration its column makes (`enum`, `regex`, `range` validationRules, and
+  `labels.references`, resolved by the provider's `resolve_reference`; Databricks finds the
+  table by its `fluid_product_id` tag). The SodaCL exporter renders the same reading (a
+  reference or a strict bound stays unmapped); the dbt test mapping
+  (`engines/dbt/_test_mapping.py`) still has its own. Identifiers are quoted per dialect
+  (backticks on Databricks and BigQuery, where `"x"` is a string), string literals through
+  `_sql_safety.quote_string_literal_for`. `schema` rules are decided in
+  `cli/contract_validation.py::_decide_schema_rules` from the live column comparison.
+- **Same cases everywhere.** `cli/_test_cases.py` lists the run's test cases (each rule, then
+  each structural category). `--output junit` and `--results-table`
+  (`cli/_test_results_databricks.py`) both read it, so Jenkins and the table agree.
+- **Ownership.** The results table is created if missing; the catalog and schema never are,
+  a product's own schema is refused, and a table without the result columns is never written.
+  A run is appended in one statement (its rows bound as one JSON parameter), so it is never
+  half-written. Databricks schemas are never served from the validation cache.
+  The dashboard (`cli/_test_dashboard_databricks.py`) is found by its workspace path, created,
+  PATCHed only when its definition differs, and published without embedded credentials.
+
 ## Links
 
 - **Documentation**: [https://agenticstiger.github.io/forge_docs/](https://agenticstiger.github.io/forge_docs/)

CHANGELOG.md

@@ -51,6 +51,30 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
   script needs a SQL warehouse that runs SQL scripting. A dbt build that
   `--mode amend-and-build` runs locally is followed by the same re-declaration.
 
+- **`fluid test` checks a Databricks product live, and can keep its results in Unity Catalog.**
+  `fluid test` used to answer "Live validation against Unity Catalog is not available" for
+  `platform: databricks`. It now reads the table's columns from the Unity Catalog tables API
+  (no compute), and runs the row count and every `dq.rules[]` entry as SQL on
+  `DATABRICKS_WAREHOUSE_ID`, signing in as `fluid apply` and `fluid verify` do (the
+  `[databricks]` extra). Bronze products count like dbt-built ones. Two new flags:
+  - `--results-table CATALOG.SCHEMA.TABLE` appends one row per check of the run (each rule, then
+    each structural check, the same cases `--output junit` reports, which still works beside
+    it): run id and time, product id, version and layer, expose and table, rule id, type and
+    severity, passed, observed value, threshold, message, forge version. The table is created if
+    missing in a schema the operator already has; forge never creates the catalog or the schema,
+    refuses the product's own schema (before any check runs), and writes nothing into an
+    existing table without these columns. A run is appended in one statement, so it is never
+    half-written, and every value is a bound parameter. It refuses `--no-data`, which runs no
+    live check to record; Databricks schemas are read live, never from the validation cache.
+  - `--results-dashboard WORKSPACE_PATH` keeps an AI/BI (Lakeview) dashboard over that table:
+    the pass rate of the latest runs, the pass rate per product (coloured by layer), the failing
+    checks and the daily trend. It is found by its workspace path (no state file, so ephemeral CI
+    agents never duplicate it), created on first use, updated only when its definition changed,
+    and published without embedded credentials.
+  The row shape follows Databricks Labs DQX's summary-metrics table and Elementary's test
+  results; the dashboard lookup follows databricks-solutions/ai-dev-kit. No code is taken from
+  either.
+
 ### Changed
 
 - **`fluid apply --mode amend-and-build` runs a Databricks dbt build now.** A dbt build with
@@ -78,9 +102,38 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
   path exits 1 where it used to write a template instead. A `fluid ai setup`
   choice and a single keyring key still come first, and `GEMINI_API_KEY` with `GOOGLE_API_KEY` is
   one provider. Pick one with `--llm-provider` or `FLUID_LLM_PROVIDER`.
+- **The native quality engine runs the rules a contract already declares in full.** A
+  `valid_values` rule that names no values takes them from its column: its `enum`
+  validationRule (`"A | B"` or `"A,B"`), its `regex` validationRule, its `range`
+  validationRule (`">= 0"`, `">= 0 and <= 100"`, `"between 0 and 100"`), or its
+  `labels.references: <product id>.<column>` (on Databricks, the referenced product's table is
+  found by the `fluid_product_id` tag `fluid apply` puts on it, the tested table's catalog
+  first, and every non-null value must be in its column). A `completeness`, `accuracy` or
+  `valid_values` rule may list several comma-separated columns, each of which must hold (a
+  `valid_values` column against its own declaration); a `uniqueness` or `freshness` rule over
+  several columns is reported as not run, never guessed. A `schema` rule passes when the live
+  table has every contract column with its type, and fails, at the rule's severity, when it
+  does not or could not be read. An `accuracy` rule with no threshold is reported as not run
+  (it used to pass whenever the column held a value). Every rule's result, passed or not, now
+  carries what was measured (`--output json` adds `rule_id`, `rule_type` and `expose_id` to
+  each check). `--engine soda` renders the same column declarations (`valid values`,
+  `valid regex`, `valid min` / `valid max`, one check per listed column); a reference or a
+  strict bound stays unmapped there. On the tower-management industry pack `fluid test` ran
+  38 of its 168 rules before; it now runs every rule of every deployed product.
 
 ### Fixed
 
+- **Quality checks measured a constant on BigQuery (and would have on Databricks).** The
+  completeness, uniqueness, accuracy and valid_values checks quoted their column the ANSI way,
+  `"col"`, which BigQuery and Spark SQL read as a string literal, and cast ratios to
+  `DOUBLE PRECISION`, which neither has. Columns are now backtick-quoted and ratios cast to
+  `FLOAT64` / `DOUBLE` on those two dialects; ANSI, DuckDB and Snowflake SQL is unchanged.
+- **A valid value containing a quote never matched on BigQuery or Databricks.** String literals
+  doubled the quote (`'it''s'`), which Databricks reads as the adjacent literals `'it'` and `'s'`
+  (checked on a SQL warehouse: the value becomes `its`) and BigQuery rejects. The new
+  `_sql_safety.quote_string_literal_for(dialect, value)` escapes it as `\'` there, after
+  doubling every backslash, so no value can end the literal early.
+
 - **`fluid ai models --provider` works, and takes the names `fluid forge --llm-provider`
   takes.** The flag stored its value where the global infrastructure `--provider` keeps its
   own, so the global check read `gemini` as an infrastructure provider: every

HONESTLY_TESTED.md

@@ -98,6 +98,8 @@ Markers declared in `pyproject.toml`.
 | Stage 1 real tofu, stand-in workspace: Genie space | `tests/iac/test_iac_databricks_genie_fake_e2e.py` | ✅ 3. The real `fluid` CLI and `tofu`: validate, dry-run, apply (the space after the tables, shared with the readers), a second apply that changes nothing, the block removed (trashed); a Genie failure fails the apply after the tables with its fix and the next apply converges; an invalid block refused before `tofu` | The stand-in is modelled on the measured API, not the service itself |
 | Manual live, Free Edition, Genie space, 2026-11-12 | the real `fluid` CLI, no test file: branch `feat/databricks-genie-space`, databricks-sdk 0.162.0, a copy of industry-models tower-management `dp019_site_360` in the scratch catalog `fluid_iactest_genie` (no dbt build, no MENA sovereignty block: the workspace is in us-west-2), 6 synthetic rows | ✅ dry-run (would create, would share), apply `+73` then the space created and shared `CAN_RUN` with two persona service principals, a second apply `+0 ~0 -0` and "no change", the question "Which on-air sites in Amman are colocation-ready with hybrid power and no standing risk?" answered through the Genie Conversation API with the 2 expected sites (its SQL took the latest `snapshot_date` and the inverse of the `at_risk_site_count` filter from the instruction), a metric view created by hand picked up ("updated: metric views", then "no change"), the block removed (trashed). Two defects found and fixed in the run: the metric-view read failed on a schema not created yet, and the service keeps a metric view among the tables. A re-check after the security review (the plugin's own hook over `samples.nyctaxi.trips`, nothing created in Unity Catalog) found the list shows a new space only ~9 s later (a run straight after made a second space) and that a taken title is suffixed on create and 409 on PATCH; fixed, then create, an immediate second run ("no change", same space), and the block removed (trashed, via the folder check) passed live. Swept: the space trashed, `DROP CATALOG fluid_iactest_genie CASCADE`, 0 left | A reader without `SELECT` asking (no persona credential here; the access model is Genie's documented one); a service principal as the deploying identity (its folder is `/Users/<application id>`, the one Databricks Asset Bundles build from `current_user.userName`; a create that lands anywhere else is trashed and stops the apply); a space over several products |
 | Stage 2 emulator (nightly, non-gating) | `tests/iac/test_iac_databricks_minilake.py` (job `nightly-databricks-minilake`) | ⚠️ 1 passed, 4 strict xfails (manual dispatch, run 42267640334, 2026-11-06, minilake v1.9.0). Schema, table and grant creates work, the replace gate holds, adding a column plans no replace, and a grant revoke is classified not gated. The four tests about those resources apply the module without its tag assignments (the `untagged` fixture). An xfail is satisfied only by its own gap's failure (`MinilakeGap`), and a pass fails the run, so an image that closes a gap turns the job red until the mark comes off. Earlier: 1/4 from the first run (2026-10-30) to the 2026-11-04 nightly, then 0/4 after #831, every apply stopping at the first tag. | A second plan is not empty and no in-place table update applies: minilake drops a CREATE TABLE's `TBLPROPERTIES` and `COMMENT`, and DuckDB cannot parse the `ALTER TABLE ... SET TBLPROPERTIES` the provider sends on every update of a table that sets properties. No tag is applied: minilake has no entity-tag-assignments API (it answers 501). v1.9.3 (2026-11-05) has both gaps, so the pin stays; an upstream issue for each is drafted, not filed. Jobs and workspace files are not in this flow yet. |
+| Stage 1 unit: `fluid test` on Databricks, results table, dashboard | `tests/providers/test_databricks_validation.py`, `tests/providers/test_databricks_statements.py`, `tests/providers/test_quality_engine_column_rules.py`, `tests/cli/test_contract_validation_schema_rules.py`, `tests/cli/test_fluid_test_results_table.py`, `tests/cli/test_fluid_test_dashboard.py` | ✅ Against a stubbed `api_client.do`: the Unity Catalog columns and the contract's types through the emitter's type table (`VARCHAR` is `string`), a missing table (404 → `missing_resource`, and every rule reported as not run with no statement), the row count, the warehouse check; the statements over `databricks_enforce.SqlRunner` with `:name` parameters, polling, timeout cancel, chunks, type conversion and scrubbed errors; every rule kind in the Databricks dialect (backticked identifiers, `\'` literals, `timestampdiff` freshness, `RLIKE`), comma-listed columns, values from the column's enum, regex or range rule, references resolved by the `fluid_product_id` tag (same catalog first, ambiguity refused, cached); `schema` rules from the column comparison; the result rows (one per JUnit case, `rule_type` `structure` for the structural ones, expose version, UTC run time), the batched parameterised `INSERT`, the refusals (bad name, a product's own schema, a table without the columns), the fix-its; the dashboard definition's structure and the create, unchanged, re-publish, update-with-etag and create-race paths. | What Databricks accepts: the manual live row below. |
+| Manual live, Free Edition (founder-approved), 2026-11-12: `fluid test` results | the real `fluid` CLI, no test file: `fluid test <contract> --env databricks --output junit --results-table fluid_iactest_dqr.dq.fluid_test_results --results-dashboard '/Workspace/Shared/fluid_iactest_dqr/Data quality'` on the 19 tower-management products in `northwind_towers` (read only: `SELECT` and metadata reads), contracts derived from industry-models main | ✅ 19 runs, 17-34 s each, 18 exit 0; `dp019_site_360` exits 1 because its table is not deployed in the demo workspace (`missing_resource`; its rules are reported as not run, and no statement is sent). 168 rule cases: the 155 on deployed tables all ran (146 pass, 9 freshness rules fail at `warn`: the bronze data is hours old against a 15-minute window), every rule kind the pack uses among them: enum, regex and range values (#4's range rule over five columns included), 23 references (3 of them bronze references outside `consumes[]`, resolved by the `fluid_product_id` tag), `schema` rules and comma-listed completeness rules. 396 rows appended, one per JUnit case; the dashboard was created by the first run, found unchanged by every later one, and updated (PATCH, re-publish) once when its definition changed; in a browser it shows the counters (88.1% of rules passing, 19 products, 20 failing rules at that point), the per-product bars by layer, the trend and the failing-checks table. Afterwards the catalog `fluid_iactest_dqr` was dropped CASCADE and the dashboard deleted. | Lakeview validates widget specs only when the dashboard renders, so a widget shape Databricks changes would show as "Invalid widget definition" without failing the command (the table widget did, until it carried every column key). |
 | Stage 3 live | — | ❌ not automated: manual maintainer runs only, never CI. Two have run: the manual rows above (2026-11-01 and 2026-11-03). | Everything the rows above leave to the live tier. |
 
 ## What's NOT tested (deliberately deferred)

fluid_build/_error_catalog.py

@@ -638,6 +638,82 @@ def slug_for(event: str) -> str:
         ["Pass the product id to roll back: fluid rollback <product-id>"],
         None,
     ),
+    # ── fluid test results table / dashboard (Databricks) ──────────────────
+    "test_results_table_invalid": (
+        [
+            "Name the table as catalog.schema.table, e.g. --results-table ops.dq.fluid_test_results",
+            "Each part may hold letters, digits and underscores only",
+        ],
+        None,
+    ),
+    "test_results_table_in_product_schema": (
+        [
+            "Put the results table in a schema of its own: every product writes to it, "
+            + "and fluid apply owns each product's schema",
+            "Create one with CREATE SCHEMA <catalog>.dq, then pass "
+            + "--results-table <catalog>.dq.fluid_test_results",
+        ],
+        None,
+    ),
+    "test_results_table_mismatch": (
+        [
+            "The table exists without the fluid test results columns, so nothing was written",
+            "Point --results-table at a new table name; forge creates it with the right columns",
+        ],
+        None,
+    ),
+    "test_results_write_failed": (
+        [
+            "Check DATABRICKS_HOST, the token or OAuth client, and DATABRICKS_WAREHOUSE_ID",
+            "The schema must exist and this identity needs USE CATALOG, USE SCHEMA, "
+            + "CREATE TABLE and MODIFY on it",
+            "The test report above is unaffected: rerun fluid test once the write can succeed",
+        ],
+        None,
+    ),
+    "test_results_dashboard_needs_table": (
+        [
+            "--results-dashboard draws the --results-table table: pass both",
+        ],
+        None,
+    ),
+    "test_results_table_needs_native_engine": (
+        [
+            "--results-table records the native engine's per-rule results; drop --engine soda",
+            "Or keep --engine soda and its JUnit or JSON output, without --results-table",
+        ],
+        None,
+    ),
+    "test_results_table_needs_live_checks": (
+        [
+            "--results-table records the live checks of a run, and --no-data runs none",
+            "Drop --no-data to test and record, or drop --results-table",
+        ],
+        None,
+    ),
+    "test_results_dashboard_invalid": (
+        [
+            "Give a workspace path whose last part is the name, e.g. "
+            + "--results-dashboard '/Workspace/Shared/fluid/Data quality'",
+            "The path must start with / and have no empty, . or .. parts",
+        ],
+        None,
+    ),
+    "test_results_dashboard_path_taken": (
+        [
+            "Something other than a dashboard already sits at that workspace path",
+            "Choose another --results-dashboard path, or move that object away",
+        ],
+        None,
+    ),
+    "test_results_dashboard_failed": (
+        [
+            "The results were written; only the dashboard step failed",
+            "This identity needs CAN MANAGE on the dashboard (or write access to its folder) "
+            + "and CAN USE on DATABRICKS_WAREHOUSE_ID",
+        ],
+        None,
+    ),
     # ── apply safety gates ─────────────────────────────────────────────────
     "iceberg_catalog_move_blocked": (
         [