Repository navigation
docs(databricks): contract DQ rules run as dbt tests in the dbt job (forge-cli#871) - #210
Merged
Merged
Conversation
…forge-cli#871) A "Contract rules as dbt tests" part in the dbt jobs section: where the test files go and that the project's own files upload unchanged, the dbt test each rule type runs, severity and thresholds, what a valid_values rule allows, anomaly_detection left to data quality monitoring, what fluid validate, generate iac and apply refuse (each with its fix), and what fluid apply prints for each rule after a run, with two live captures. Short pointers in the overview, the project upload, what is refused, what validate refuses, limitations and how it is tested, and an upgrading.md entry. Checked against forge-cli d50084b1, the last head of #871; the squash commit 1e20c739 has the same tree.
Merged
1 of 3 tasks
This branch was successfully deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Documents forge-cli #871: a Databricks dbt job now runs its contract's data-quality rules as dbt tests.
docs/providers/databricks.md, in the dbt jobs section, gets a "Contract rules as dbt tests" part:<test path>/fluid_dq/<model>/<name>.sql), how they are named, and that the project's own files upload unchanged;criticalanderrorfail the run,warnandinfodo not) and a threshold below 1.0 read as a share;valid_valuesrule allows, read asfluid testreads it, andanomaly_detectionbeing left to data quality monitoring;fluid validate,fluid generate iacandfluid applyrefuse, each with its fix (two tables);fluid apply --mode amend-and-buildprints for each rule, with the two live captures from the implementing session (workspace host and ids replaced).Short pointers go into the overview, "Where the dbt project comes from", "What is refused", "What validate refuses", Limitations and "How it is tested".
docs/upgrading.mdgets an entry under "Not in a release yet".Status
#871 merged on 2026-11-13 as
1e20c739a132829446c2872848700ffa889bb3c3, so every added sentence about it uses themainmarker:*On forge-cli `main`, not yet in a release (forge-cli #871)*. The pin bump that follows the next forge-cli release turns these into release statements.Checked against forge-cli
d50084b17bab84f83ab50c8ddbbdc4e7d5598985, the last head of the #871 branch. The squash commit has the same tree (git diff d50084b1 1e20c739is empty), so what was checked is what is onmain.What the CLI does that differs from #871's description
fluid plandoes not refuse these rules and does not list the tests, although #871's description and its CHANGELOG entry say it does. On a 0.7.x contract it writes the abstract actions and never reaches the Databricks planner.fluid validate,fluid generate iacandfluid applyrefuse. The page says what the CLI does. Iffluid planis changed to refuse, one sentence in "What the rules refuse" changes with it.Verification
<name>and<model>rules, the range, threshold and operator rows of the refusal table, "one file per rule", the byte limits, two scoping fixes in the upgrading entry, share thresholds, and "when the output can be read".scripts/check_cli_docs.py(full,--version-only,--markers-only),scripts/check_providers.pyandscripts/gen_contract_reference.py --check, with the pinned CLI 0.29.0 on Python 3.11: all pass.npm ci && npm run docs:build && node scripts/check-dist-links.mjs: pass. Every#anchorthe diff adds resolves to a heading id in the built page. The source link check (lychee, offline, with CI's route mirror) reports 0 errors.Left alone
fluid_keystask now runs withrun_if: ALL_DONE(#871). Nothing on this site describes that task, so nothing here does.docs/cli/test.mdstill says avalid_valuesrule with no list checks nothing. That has been stale since #867; its own docs TODO is open. The new text describes the current reading.