Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
112 changes: 112 additions & 0 deletions packages/opencode/src/altimate/review/dbt-patterns.ts
Original file line number Diff line number Diff line change
Expand Up @@ -926,6 +926,117 @@ const detectMultiCountDistinct: Detector = (c) => {
)
}

const JOIN_TYPE_GAP = /^\s*(?:(?:natural|inner|left|right|full|cross)\s+)?(?:outer\s+)?$/i
const JOIN_CLAUSE_END = /^(?:join|where|group\s+by|order\s+by|qualify|having|union|limit|window|left|right|inner|full|cross|natural|lateral|select|from)\b|^;/i

/** Normalised equality/comparison predicate: no whitespace, lowercase, `a=b` with sides sorted. */
function predKey(p: string): string {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[MINOR] predKey lowercases and strips whitespace inside string literals, so different predicates compare equal

const t = p.replace(/\s+/g, "").toLowerCase()

from a inner join b on a.flag = 'X' inner join c on b.id = c.id and a.flag = 'x' flags a.flag = 'x' as redundant (reproduced), but with 'X' filtering first it is not: removing it as suggested changes results. The doc comment says "Literals are compared as written", which this contradicts.

Fix: normalise only outside quoted literals, keep literal contents exact.

const t = p.replace(/\s+/g, "").toLowerCase()

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2: predKey removes whitespace and lowercases SQL string literals, so distinct predicates such as p.platform = 'a b' and p.platform = 'ab' compare equal and produce a false redundant-join comment. Preserve quoted literal contents while normalizing only SQL syntax and identifiers.

Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. When an issue isn't valid or won't be fixed in this PR, reply in its thread with the reason and then resolve the thread. At packages/opencode/src/altimate/review/dbt-patterns.ts, line 899:

<comment>`predKey` removes whitespace and lowercases SQL string literals, so distinct predicates such as `p.platform = 'a b'` and `p.platform = 'ab'` compare equal and produce a false redundant-join comment. Preserve quoted literal contents while normalizing only SQL syntax and identifiers.</comment>

<file context>
@@ -891,6 +891,117 @@ const detectMultiCountDistinct: Detector = (c) => {
+
+/** Normalised equality/comparison predicate: no whitespace, lowercase, `a=b` with sides sorted. */
+function predKey(p: string): string {
+  const t = p.replace(/\s+/g, "").toLowerCase()
+  const eq = /^([^=<>!]+)=([^=<>!]+)$/.exec(t)
+  return eq ? [eq[1], eq[2]].sort().join("=") : t
</file context>

const eq = /^([^=<>!]+)=([^=<>!]+)$/.exec(t)
return eq ? [eq[1], eq[2]].sort().join("=") : t
}

/**
* Predicates in a JOIN's ON clause that an earlier INNER join of the same join chain already states
* and that don't reference the newly joined relation: the inner join already dropped every row they
* would reject, so they filter nothing. (After a LEFT join the same predicate is not redundant: the
* left join keeps the rows it rejects.) Literals are compared as written. Returns each such
* predicate as written (whitespace collapsed).
*/
export function redundantJoinPredicates(sql: string): string[] {
// Literal text stays (`= 'a'` and `= 'b'` differ); `and`/parens inside a literal are rare enough here.
const s = stripComments(sql).replace(/\{\{[\s\S]*?\}\}/g, "__jinja__").replace(/\{%[\s\S]*?%\}/g, " ")

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

SUGGESTION: literal text now feeds the structural join scan

stripLiterals was dropped here so = 'a' and = 'b' no longer compare equal, but the same s is also used as SQL structure: matchAll(/\bjoin\b/gi) (two lines below) and JOIN_CLAUSE_END now match keywords inside string literals. A model that adds a line such as {% set q = 'select * from a join b on a.id = b.id and y = 1' %} gets a phantom join parsed out of the string: when it precedes the real chain, chained is false at the next real join (its end lands mid-literal), so seen is reset and redundant predicates are missed; when it follows one, the phantom ON clause repeats the earlier predicate key and detectRedundantJoinPredicate finds the predicate text in the literal's own added line and reports "Remove it" against it. Mask literals in the copy used for the structural scan (e.g. replace each with a same-length placeholder) and keep the literal-preserving text only for predKey/the message.

const joins: { alias?: string; preds: { raw: string; key: string }[]; start: number; end: number; inner: boolean }[] = []
for (const m of s.matchAll(/\bjoin\b/gi)) {
const inner = !/\b(left|right|full|cross|natural|outer)\s+(?:outer\s+)?$/i.test(s.slice(Math.max(0, m.index! - 20), m.index))
let i = m.index! + 4
const rest = s.slice(i)
let rel = /^\s*(\S+)/.exec(rest)
if (rest.trimStart().startsWith("(")) {
const open = i + rest.indexOf("(")
let depth = 0
for (i = open; i < s.length; i++) {
if (s[i] === "(") depth++
else if (s[i] === ")" && --depth === 0) break
}
i++
rel = null
} else if (rel) i += rel[0].length
const tail = /^\s+(?:as\s+)?(\w+)\s+on\b/i.exec(s.slice(i)) ?? /^\s+on\b/i.exec(s.slice(i))

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P3: The join parser accepts only unquoted word aliases, so AS "b" makes this rule skip the entire join and miss repeated predicates. Parse quoted aliases before treating a join as having no analyzable ON clause.

Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. When an issue isn't valid or won't be fixed in this PR, reply in its thread with the reason and then resolve the thread. At packages/opencode/src/altimate/review/dbt-patterns.ts, line 930:

<comment>The join parser accepts only unquoted word aliases, so `AS "b"` makes this rule skip the entire join and miss repeated predicates. Parse quoted aliases before treating a join as having no analyzable `ON` clause.</comment>

<file context>
@@ -891,6 +891,117 @@ const detectMultiCountDistinct: Detector = (c) => {
+      i++
+      rel = null
+    } else if (rel) i += rel[0].length
+    const tail = /^\s+(?:as\s+)?(\w+)\s+on\b/i.exec(s.slice(i)) ?? /^\s+on\b/i.exec(s.slice(i))
+    if (!tail) {
+      joins.push({ preds: [], start: m.index!, end: i, inner })
</file context>

if (!tail) {
joins.push({ preds: [], start: m.index!, end: i, inner })
continue
}
const alias = tail[1] && tail[1].toLowerCase() !== "on" ? tail[1].toLowerCase() : rel?.[1].split(".").pop()?.toLowerCase()
let j = i + tail[0].length
const onStart = j
let depth = 0
for (; j < s.length; j++) {
const ch = s[j]
if (ch === "(") depth++
else if (ch === ")") {
if (depth === 0) break
depth--
} else if (depth === 0 && /\W/.test(s[j - 1]) && JOIN_CLAUSE_END.test(s.slice(j, j + 12))) break
}
const clause = s.slice(onStart, j)
const preds = /\bor\b/i.test(clause)
? []
: topLevelAnd(clause).map((raw) => ({ raw: raw.replace(/\s+/g, " ").trim(), key: predKey(raw) })).filter((p) => p.key)
joins.push({ alias, preds, start: m.index!, end: j, inner })
}
const out: string[] = []
let seen: Set<string> = new Set()
for (let k = 0; k < joins.length; k++) {
const prev = joins[k - 1]
const chained = prev && JOIN_TYPE_GAP.test(s.slice(prev.end, joins[k].start))
if (!chained) seen = new Set()
const { alias, preds } = joins[k]
for (const p of preds)
if (chained && seen.has(p.key) && /=/.test(p.key) && !(alias && new RegExp(`\\b${alias}\\.`, "i").test(p.raw))) out.push(p.raw)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[MAJOR] Unescaped join relation text used in new RegExp crashes the whole review

!(alias && new RegExp(`\\b${alias}\\.`, "i").test(p.raw))

When a join has no alias, alias falls back to rel[1].split(".").pop() (line 935), which is raw text. For ... join b on a.id = b.id join unnest(a.arr) on a.id = b.id (also Snowflake join table(flatten(a.arr)) on ...) redundantJoinPredicates throws SyntaxError: Invalid regular expression: unmatched parentheses. Reproduced directly. The model detectors run without a try/catch, so runReview rejects and the PR gets no verdict at all. Rare (needs a repeated predicate too), but the failure is total on valid SQL.

Fix: use escapeRe(alias) (or only accept the fallback when it matches /^\w+$/), and add a test with an unaliased unnest(...) join.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[MINOR] Predicates stay in seen across RIGHT/FULL joins

if (joins[k].inner) for (const p of preds) seen.add(p.key)

from a inner join b on a.id = b.id right join c on b.id = c.id inner join d on c.id = d.id and a.id = b.id flags a.id = b.id (reproduced). The RIGHT join re-introduces rows where it is false, so removing it changes results ([1] vs [1, 2] in a SQLite check).

Fix: clear seen when a RIGHT/FULL join is crossed.

if (joins[k].inner) for (const p of preds) seen.add(p.key)

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2: redundantJoinPredicates keeps an inner join's predicates in seen across a RIGHT or FULL join. Those joins NULL-extend earlier relations, so a later join repeating the predicate is not redundant. Reset seen after a right or full join.

Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. When an issue isn't valid or won't be fixed in this PR, reply in its thread with the reason and then resolve the thread. At packages/opencode/src/altimate/review/dbt-patterns.ts, line 962:

<comment>`redundantJoinPredicates` keeps an inner join's predicates in `seen` across a RIGHT or FULL join. Those joins NULL-extend earlier relations, so a later join repeating the predicate is not redundant. Reset `seen` after a right or full join.</comment>

<file context>
@@ -891,6 +891,117 @@ const detectMultiCountDistinct: Detector = (c) => {
+    const { alias, preds } = joins[k]
+    for (const p of preds)
+      if (chained && seen.has(p.key) && /=/.test(p.key) && !(alias && new RegExp(`\\b${alias}\\.`, "i").test(p.raw))) out.push(p.raw)
+    if (joins[k].inner) for (const p of preds) seen.add(p.key)
+  }
+  return out
</file context>
Suggested change
if (joins[k].inner) for (const p of preds) seen.add(p.key)
if (joins[k].inner) for (const p of preds) seen.add(p.key)
else if (/\b(right|full)\s+(?:outer\s+)?$/i.test(s.slice(Math.max(0, joins[k].start - 20), joins[k].start))) seen = new Set()

}
return out
}

/** Split a predicate list on top-level AND. */
function topLevelAnd(clause: string): string[] {
const parts: string[] = []
let depth = 0
let last = 0
for (const m of clause.matchAll(/[()]|\band\b/gi)) {

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2: topLevelAnd treats AND and parentheses inside quoted SQL text as syntax, so different literals containing and can yield the same malformed fragment and trigger a false redundant-predicate finding. Tokenize quoted literals before splitting the ON clause.

Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. When an issue isn't valid or won't be fixed in this PR, reply in its thread with the reason and then resolve the thread. At packages/opencode/src/altimate/review/dbt-patterns.ts, line 972:

<comment>`topLevelAnd` treats `AND` and parentheses inside quoted SQL text as syntax, so different literals containing `and` can yield the same malformed fragment and trigger a false redundant-predicate finding. Tokenize quoted literals before splitting the ON clause.</comment>

<file context>
@@ -891,6 +891,117 @@ const detectMultiCountDistinct: Detector = (c) => {
+  const parts: string[] = []
+  let depth = 0
+  let last = 0
+  for (const m of clause.matchAll(/[()]|\band\b/gi)) {
+    if (m[0] === "(") depth++
+    else if (m[0] === ")") depth--
</file context>

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

SUGGESTION: splitting on and inside literals creates a false "redundant" match

topLevelAnd sees the raw clause (literals are deliberately kept), so p.segment = 'smb and enterprise' and p.segment = 'smb and midmarket' both split into the fragment p.segment = 'smb, compare equal, and are reported as redundant even though the joins filter different values. Split on and in a literal-masked copy and keep the raw predicate for the message.

if (m[0] === "(") depth++
else if (m[0] === ")") depth--
else if (depth === 0) {
parts.push(clause.slice(last, m.index))
last = m.index! + m[0].length
}
}
parts.push(clause.slice(last))
return parts.map((p) => p.trim()).filter(Boolean)
}

// 46. A join predicate an earlier join of the same chain already states.
const detectRedundantJoinPredicate: Detector = (c) => {
if (!c.added.some((l) => /=/.test(l))) return null
const norm = (t: string) => stripComments(t).replace(/\s+/g, "").toLowerCase()
for (const pred of redundantJoinPredicates(c.newSql)) {
const key = norm(pred)
const hit = c.added.find((l) => norm(l).includes(key))

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

WARNING: the anchor search can point at the legitimate earlier join

key keeps operand order and c.added.find(...) returns the first added line containing the normalised predicate. When the whole chain is added (a new model, or a new CTE inside an existing model) the needed predicate in the earlier join matches before the repeated one, so the comment is posted on the correct join's line and says "Remove it" about a predicate that is required. Return the predicate together with its join's line range and match c.added only inside that range.

if (!hit) continue
return pattern(
c,
"sql_quality",
"suggestion",
"redundant-join-predicate",
"join repeats a predicate an earlier join already applies",
`\`${pred}\` doesn't involve the relation this join adds and is already in an earlier join's ON clause, so it filters nothing. Remove it, or compare the joined relation's own column if that was the intent.`,
hit,
)
}
return null
}

const MODEL_DETECTORS: Detector[] = [
detectClock,
detectCrossJoin,
Expand Down Expand Up @@ -960,6 +1071,7 @@ const MODEL_DETECTORS: Detector[] = [
detectBooleanPrecedence,
detectOffsetNoOrder,
detectMultiCountDistinct,
detectRedundantJoinPredicate,
]

/**
Expand Down
102 changes: 99 additions & 3 deletions packages/opencode/src/altimate/review/orchestrate.ts
Original file line number Diff line number Diff line change
Expand Up @@ -41,7 +41,23 @@ import { detectUnitTestFixtures } from "./unit-tests"
import { finalSelectNames } from "./select-list"
import { detectTestAdequacy, detectThresholdLoosening } from "./test-adequacy"
import { incrementalFindings, snapshotFindings, snapshotYmlFindings } from "./incremental-semantics"
import { dbtProjectFindings, jinjaFindings, joinTargets, missingRefFindings, modelConfigFindings, packagesFindings, testsOnNullColumns } from "./project-rules"
import {
dbtProjectFindings,
dispatchKwargFindings,
dispatchVariants,
docRefFindings,
jinjaFindings,
joinTargets,
missingRefFindings,
modelConfigFindings,
isPackageProject,
packageLockFindings,
packagesFindings,
quickstartModelFindings,
testWindowFindings,
testsOnNullColumns,
yamlParseFindings,
} from "./project-rules"
import { type AiContextFile, type AiLaneStatus, type AiReviewFile, type AiReviewInput, type AiReviewResult, safeReason } from "./ai-review"
import { ruleTier } from "./rule-tiers"
import { type AiVerifyEvidence, numberedSlice } from "./ai-verify"
Expand Down Expand Up @@ -2187,6 +2203,18 @@ export async function runReview(input: OrchestrateInput): Promise<VerdictEnvelop
for (const findings of sets) all.push(findings)
}

// An `adapter.dispatch` call that now passes a keyword some `<adapter>__` variant in the repo doesn't take.
const dispatchVariantFindings = async (file: ChangedFile, newSql: string): Promise<Finding[]> => {
if (!/\badapter\.dispatch\b/.test(newSql)) return []
const texts = new Map<string, string>([[file.path, newSql]])
for (const name of new Set([...newSql.matchAll(/\badapter\.dispatch\s*\(\s*['"](?:\w+\.)?(\w+)['"]/g)].map((m) => m[1]))) {

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2: dispatchVariantFindings searches the repo and reads every hit file for each adapter.dispatch name in the macro, even names whose calls add no keyword. Those results are discarded. Limit the loop to the names addedDispatchKwargs(file, newSql) returns.

Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. When an issue isn't valid or won't be fixed in this PR, reply in its thread with the reason and then resolve the thread. At packages/opencode/src/altimate/review/orchestrate.ts, line 2124:

<comment>`dispatchVariantFindings` searches the repo and reads every hit file for each `adapter.dispatch` name in the macro, even names whose calls add no keyword. Those results are discarded. Limit the loop to the names `addedDispatchKwargs(file, newSql)` returns.</comment>

<file context>
@@ -2101,6 +2117,18 @@ export async function runReview(input: OrchestrateInput): Promise<VerdictEnvelop
+  const dispatchVariantFindings = async (file: ChangedFile, newSql: string): Promise<Finding[]> => {
+    if (!/\badapter\.dispatch\b/.test(newSql)) return []
+    const texts = new Map<string, string>([[file.path, newSql]])
+    for (const name of new Set([...newSql.matchAll(/\badapter\.dispatch\s*\(\s*['"](?:\w+\.)?(\w+)['"]/g)].map((m) => m[1]))) {
+      const hits = (await input.grepProject?.(`__${name}(`).catch(() => [])) ?? []
+      for (const path of new Set(hits.map((h) => h.path)))
</file context>

const hits = (await input.grepProject?.(`__${name}(`).catch(() => [])) ?? []
for (const path of new Set(hits.map((h) => h.path)))
if (!texts.has(path)) texts.set(path, (await getContent?.(path, "new").catch(() => undefined)) ?? "")
}
return dispatchKwargFindings(file, newSql, (name) => [...texts.values()].flatMap((t) => dispatchVariants(t, name)))
}

// Macros: the catalog rules that hold in a SQL template, and vars with no default.
if (lanes.has("dbt_patterns")) {
const macros = reviewable.filter((f) => f.kind === "macro" && f.status !== "deleted")
Expand All @@ -2202,6 +2230,7 @@ export async function runReview(input: OrchestrateInput): Promise<VerdictEnvelop
return [
...detectMacroPatterns(file, newSql, input.rubric, projectVars, { dialect, dialectKnown: !!input.dialectKnown }),
...jinjaFindings(file, oldSql, newSql),
...(await dispatchVariantFindings(file, newSql)),
]
} catch (err) {
failedFile(file, err)
Expand Down Expand Up @@ -2233,7 +2262,7 @@ export async function runReview(input: OrchestrateInput): Promise<VerdictEnvelop
file.status === "added" ? Promise.resolve(undefined) : getContent?.(file.oldPath ?? file.path, "old")?.catch(() => undefined),
])
if (file.kind === "project_config") return [...packagesFindings(file, newContent), ...dbtProjectFindings(file, oldContent, newContent)]
return jinjaFindings(file, oldContent, newContent)
return [...jinjaFindings(file, oldContent, newContent), ...(file.kind === "test" ? testWindowFindings(file) : [])]
} catch (err) {
failedFile(file, err)
return []
Expand All @@ -2243,6 +2272,56 @@ export async function runReview(input: OrchestrateInput): Promise<VerdictEnvelop
for (const findings of sets) all.push(findings)
}

// Files checked whatever their dbt role (a `.quickstart/quickstart.yml` is "other"): YAML that
// no longer parses, and new models missing from the quickstart `public_models`.
const otherChecked = new Set<string>()
const checkedOther = (file: ChangedFile) => {
otherChecked.add(file.path)
touch([file])
}
if (lanes.has("dbt_patterns")) {
const read = (p: string, side: "old" | "new") => getContent?.(p, side)?.catch(() => undefined) ?? Promise.resolve(undefined)
const ymls = reviewable.filter((f) => f.status !== "deleted" && /\.ya?ml$/i.test(f.path))
const sets = await Promise.all(
ymls.map(async (file) => {
const [newContent, oldContent] = await Promise.all([

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

SUGGESTION: every changed yml now costs two extra git shows before the schema lane re-reads the same file

getContent (makeContentResolver) has no cache, and ylms includes all schema_yml files, so each one is read old+new here and again in the schema block (around line 2282), on top of the contract lane's reads. The Promise.all over ylms also launches 2 × yml count subprocesses at once. On a schema-heavy PR this is a large share of the review's process time; memoising getContent per (path, side) or reusing these contents in the schema block would keep it bounded.

read(file.path, "new"),
file.status === "added" ? Promise.resolve(undefined) : read(file.oldPath ?? file.path, "old"),
])
if (file.kind === "other") checkedOther(file)
return yamlParseFindings(file, oldContent, newContent)

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P3: For added YAML files, oldContent is undefined, so yamlParseFindings skips the "parsed before" gate. A new file with a syntax error then gets a finding that says it parsed before this change. Either skip added files here or change the message for files with no base.

Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. When an issue isn't valid or won't be fixed in this PR, reply in its thread with the reason and then resolve the thread. At packages/opencode/src/altimate/review/orchestrate.ts, line 2206:

<comment>For added YAML files, `oldContent` is `undefined`, so `yamlParseFindings` skips the "parsed before" gate. A new file with a syntax error then gets a finding that says it parsed before this change. Either skip added files here or change the message for files with no base.</comment>

<file context>
@@ -2157,6 +2186,56 @@ export async function runReview(input: OrchestrateInput): Promise<VerdictEnvelop
+          file.status === "added" ? Promise.resolve(undefined) : read(file.oldPath ?? file.path, "old"),
+        ])
+        if (file.kind === "other") checkedOther(file)
+        return yamlParseFindings(file, oldContent, newContent)
+      }),
+    )
</file context>

}),
)
for (const findings of sets) all.push(findings)

for (const file of reviewable.filter((f) => f.status === "added" && /(^|\/)package-lock\.yml$/.test(f.path))) {
const dir = file.path.slice(0, file.path.length - "package-lock.yml".length)
const [projectYml, testsYml, nested] = await Promise.all([
read(`${dir}dbt_project.yml`, "new"),
read(`${dir}integration_tests/dbt_project.yml`, "new"),
// `integration_tests/<name>/dbt_project.yml` (grepProject globs from the project root;
// a git pathspec `*` crosses directories).
(dir === (input.layout?.root ? input.layout.root + "/" : "") && input.grepProject?.("name", "integration_tests/*dbt_project.yml").then((h) => h ?? []).catch(() => [])) ||
Promise.resolve([]),
])
if (projectYml === undefined && testsYml === undefined) continue
checkedOther(file)
all.push(packageLockFindings(file, isPackageProject(projectYml, !!testsYml || nested.length > 0)))
}

const newModels = reviewable.filter((f) => f.kind === "model_sql" && f.status === "added")
if (newModels.length) {
const QUICKSTART = /(^|\/)\.quickstart\/quickstart\.ya?ml$/
const changed = reviewable.find((f) => QUICKSTART.test(f.path) && f.status !== "deleted")
const qsPath = changed?.path ?? `${input.layout?.root ? input.layout.root + "/" : ""}.quickstart/quickstart.yml`
const content = await read(qsPath, "new")
if (content) {
if (changed) checkedOther(changed)
all.push(quickstartModelFindings(changed ?? newModels[0], qsPath, content, newModels.map((f) => modelNameFromPath(f.path))))
}
}
}

// schema.yml-level detectors (test removal) — run on changed YAML files
// regardless of tier, since deleting a guardrail test is always worth flagging.
// Pass old + new content when available so the detector diffs structural YAML
Expand All @@ -2264,6 +2343,22 @@ export async function runReview(input: OrchestrateInput): Promise<VerdictEnvelop
//
// Fetches run in parallel across schema files (a schema-heavy PR could touch
// dozens of yml files; serial `git show` per file adds up).
// A `{% docs name %}` block in any of the project's .md files at head (cached per name).
const docLookups = new Map<string, Promise<boolean>>()
const docExists = (name: string) => {
if (!docLookups.has(name)) {
const block = new RegExp(`\\{%-?\\s*docs\\s+${name}\\s*-?%\\}`)
docLookups.set(
name,
(input.grepProject?.(`docs ${name}`, "*.md") ?? Promise.resolve([]))
// An unavailable search (undefined) is no evidence the block is missing.
.then((hits) => hits === undefined || hits.some((h) => block.test(h.text)))
// A failed search is no evidence the block is missing.
.catch(() => true),
)
}
return docLookups.get(name)!
}
const schemaFiles = reviewable.filter((f) => f.kind === "schema_yml")
touch(schemaFiles)
const sqlByModel = new Map([...ctxByPath.values()].filter((c) => c.newSql).map((c) => [modelNameFromPath(c.file.path).toLowerCase(), c.newSql!]))
Expand All @@ -2282,6 +2377,7 @@ export async function runReview(input: OrchestrateInput): Promise<VerdictEnvelop
...detectUnitTestFixtures(file, oldContent, newContent),
...snapshotYmlFindings(file, oldContent, newContent),
...detectTestAdequacy({ file, oldContent, newContent, projectYml, layout: input.layout }),
...(lanes.has("dbt_patterns") && input.grepProject ? await docRefFindings(file, oldContent, newContent, docExists) : []),
]
} catch (err) {
// One malformed schema file degrades that file only.
Expand Down Expand Up @@ -2523,7 +2619,7 @@ export async function runReview(input: OrchestrateInput): Promise<VerdictEnvelop
...(input.scope?.notReviewed ?? []),
...dropped,
...reviewable
.filter((f) => f.kind === "other" || f.kind === "analysis")
.filter((f) => (f.kind === "other" && !otherChecked.has(f.path)) || f.kind === "analysis")
.map((f) => ({ path: f.path, reason: "not a dbt model/macro/test/yml/seed file" })),
...reviewable
.filter((f) => !touched.has(f.path))
Expand Down
Loading
Loading