Skip to content

docs(databricks): data quality monitoring from freshness and completeness SLOs (forge-cli#864) - #208

Merged
fas89 merged 1 commit into
mainfrom
docs/databricks-data-quality-864
Nov 12, 2026
Merged

fas89 merged 1 commit into
mainfrom
docs/databricks-data-quality-864

Conversation

@fas89

@fas89 fas89 commented Nov 12, 2026

Copy link
Copy Markdown
Collaborator

Documents Agenticstiger/forge-cli#864: a Databricks expose that states a freshness or completeness SLO turns on Databricks data quality monitoring for the schema the product owns.

What changed

  • docs/providers/databricks.md: a new "Data quality monitoring" section. It covers what turns monitoring on and the opt-out, the monitor fluid apply creates, why the SLO values are not thresholds (and the dq.monitoring.window warning), where the results show, the cost and the Free Edition quota, adoption, the ungated removal and a refused monitor. Also an Overview bullet, a row in "What fluid apply provisions", a row in "Changes the apply gates", a sentence in "What validate refuses", a Limitations bullet and a sentence in the live row of "How it is tested".
  • docs/upgrading.md: a bullet under "Not in a release yet". A contract that already states such an SLO gets a monitor at its next apply, that apply can now fail if Databricks refuses the monitor, and the bullet says how to opt out.

Markers

forge-cli #864 merged at 22:49Z as 215b51d7, so each new passage carries the main form: "On forge-cli main, not yet in a release (forge-cli #864)." That is 8 markers on 2 pages. The pin bump for the next release turns them into "since" markers.

forge-cli #861 merged at 22:38Z, so its branch markers from #207 on these two pages are stale now. This PR does not touch them.

Checked against

forge-cli 215b51d7, the squash of #864. The checks ran at the PR head (39e692ce, then befe92ea). Against the squash, the PR's files differ only by #861's CHANGELOG entry and its applied_state plumbing in the apply engine, neither of which this page documents.

  • The text blocks are real output: fluid validate and fluid generate iac run at the PR head, the plan note from plan_notices called offline (as the Certification section does), and the removal and refusal output from the PR's six stand-in e2e tests (real tofu and provider 1.140.0 against the in-process stand-in), all six passing.
  • Live facts come from the 2026-11-12 Free Edition row of HONESTLY_TESTED.md. Databricks facts (the indicator's statuses, who can read table_results, the billing product, MANAGE) come from Databricks' anomaly detection page.
  • A separate verifier pass checked about 90 claims against the code, the CLI and tofu providers schema. It found nothing wrong and 7 imprecise claims, all fixed here.

Where the page differs from the PR's own wording:

  • The CHANGELOG says a refused monitor "names the opt-out first". In the printed error, the monitor's two hints follow three generic suggestions, so the page says the suggestions name it and shows the output.
  • HONESTLY_TESTED.md says notification_settings is not in provider 1.140.0. It is, but under data_profiling_config (on_failure only). anomaly_detection_config holds only excluded_table_full_names, so the page says the anomaly detection settings have no alert setting.
  • The CHANGELOG says Free Edition's daily quota covers the scans. That was not measured (no billing row came within the live session), so the page says to expect them to count.

Gates, run locally (Python 3.11, data-product-forge 0.29.0)

  • scripts/check_cli_docs.py (full, --version-only, --markers-only): OK
  • scripts/check_providers.py: OK
  • scripts/gen_contract_reference.py --check: OK
  • npm ci && npm run docs:build && node scripts/check-dist-links.mjs: OK, and the six new anchors are in the built page
  • lychee --offline on both pages over CI's route mirror: 0 errors

…ness SLOs (forge-cli#864)

A Databricks expose that states a freshness or completeness SLO now turns
on Databricks data quality monitoring for the schema the product owns.
Document what turns it on and the opt-out, the monitor the apply creates,
why the SLO values are not thresholds, where the results show, the cost
and the Free Edition quota, adoption, the ungated removal and a refused
monitor, plus an upgrading note. The markers use the main form: #864
merged as 215b51d7.
@fas89
fas89 merged commit 3c8eae2 into main Nov 12, 2026
9 checks passed

This branch was previously deployed

1 inactive deployment
github-pages — 4c48a76c Deployed Nov 12, 2026 by fas89 via deploy #301
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant