Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions plugins/altimate-code/.claude-plugin/plugin.json
Original file line number Diff line number Diff line change
Expand Up @@ -30,8 +30,8 @@
},
"maxScanBytes": {
"type": "number",
"title": "Largest BigQuery scan without asking",
"description": "A dbt run, build or show that a BigQuery dry run estimates would scan more bytes than this is held until you approve it with /allow-scan; 0 never holds, and other warehouses are never held. Optional; default: 10737418240 (10 GiB).",
"title": "Largest dbt scan without asking",
"description": "A dbt run, build or show estimated to scan more bytes than this is held until you approve it with /allow-scan; 0 never holds. BigQuery's estimate is its dry run; Snowflake's and Databricks' is the query planner's (EXPLAIN), which can differ from what the run scans. A query with no estimate, and other warehouses, are never held. Optional; default: 10737418240 (10 GiB).",

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Minor (wording, real): on Databricks the estimate is an EXPLAIN COST on the SQL warehouse the profile connects to. altimate-code #1768's tool description says this starts the warehouse if it is stopped, and it is billed while up. Neither this description nor the README row tells the user.

It doesn't block. The estimate runs only just before a dbt run, build or show of one named model (or show --inline) on the default target, which would start the same warehouse anyway. Only a hold, or an explicit /scan-cost, starts it without the command following. It runs one at a time per project, within a 120 s budget. Base already makes the same call once the new CLI is installed.

Suggest adding a short clause here and in the README row, e.g. "On Databricks the estimate runs on the SQL warehouse, which starts it if stopped." Codex rated this major at first, then accepted minor on that evidence.

"default": 10737418240,
"min": 0
},
Expand Down
4 changes: 2 additions & 2 deletions plugins/altimate-code/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -154,12 +154,12 @@ Compile and validate runs a query on your warehouse after every model edit (`dbt

## Settings

The plugin has seven settings, shown when you enable it (`userConfig` in `.claude-plugin/plugin.json`). All are optional: Claude Code may report them as "not set", and each then takes the default below. `dbt_mods` is used by the dbt helpers, `maxScanBytes` by the BigQuery scan check, `healthBlocksPush` and `healthConfigPath` by the project health check, the two comments settings by team comments, and `show_edit_impact` by the per-edit impact note.
The plugin has seven settings, shown when you enable it (`userConfig` in `.claude-plugin/plugin.json`). All are optional: Claude Code may report them as "not set", and each then takes the default below. `dbt_mods` is used by the dbt helpers, `maxScanBytes` by the scan check (BigQuery, Snowflake, Databricks), `healthBlocksPush` and `healthConfigPath` by the project health check, the two comments settings by team comments, and `show_edit_impact` by the per-edit impact note.

| Setting | Default | What it controls |
| --- | --- | --- |
| `dbt_mods` | on | Turns all of the dbt helpers (above) off when false. |
| `maxScanBytes` | 10737418240 (10 GiB) | A dbt run, build or show that a BigQuery dry run estimates would scan more bytes than this is held until you approve it with `/allow-scan` (0 never holds; at least 0). Other warehouses are never held. |
| `maxScanBytes` | 10737418240 (10 GiB) | A dbt run, build or show estimated to scan more bytes than this is held until you approve it with `/allow-scan` (0 never holds; at least 0). On BigQuery the estimate is a dry run; on Snowflake and Databricks it is the query planner's (`EXPLAIN`): Snowflake's bytes after pruning, an upper bound, and Databricks' table sizes, which a run usually reads less of. A query with no estimate (a Databricks table without statistics) runs, with a note saying why; other warehouses are never held. |
| `healthBlocksPush` | off | When on, `git push` is refused while the project health check finds new issues in the models you changed, and while the check cannot run (altimate-code missing, or without `dbt_health`) or has not finished. Off, the issues are only shown. If the first check of a session lands after you changed a model, that model's existing issues count as new (the baseline then lacks them). The reverse also holds: a model you drop from a schema file before that first check lands has its new issues taken as existing, so a push is not held for them. |
| `healthConfigPath` | empty | The DataPilot health check config file, relative to the dbt project. Empty uses the default checks. |
| `commentsPollSeconds` | 900 | How often, in seconds, the team's comments on shared docs are fetched again in an interactive session (30 to 86400). |
Expand Down
156 changes: 154 additions & 2 deletions plugins/altimate-code/hooks/features/cost.test.tsx
Original file line number Diff line number Diff line change
Expand Up @@ -409,13 +409,13 @@ test('commands that scan nothing are not looked at', async ($, on) => {
})

test('another warehouse is told once, never held, and not asked again', async ($, on) => {
const { clock, estimates, ran } = await start($, on, { fallback: { success: true, supported: false, adapter: 'snowflake' } })
const { clock, estimates, ran } = await start($, on, { fallback: { success: true, supported: false, adapter: 'duckdb' } })
const first: any = await $.tool.call(bash('dbt run -s orders'))
const second: any = await $.tool.call(bash('dbt run -s customers'))
await clock.settle()

expect(first.deny).toBeUndefined()
expect(first.context).toEqual(['cost · no estimate on snowflake: only BigQuery has a dry run'])
expect(first.context).toEqual(['cost · no estimate on duckdb: scans are estimated on BigQuery, Snowflake and Databricks only'])
expect(second.deny).toBeUndefined()
expect(estimates).toEqual(['orders'])
expect(ran).toEqual(['dbt run -s orders', 'dbt run -s customers'])
Expand Down Expand Up @@ -1524,3 +1524,155 @@ test('a parallel call whose estimate fails is held by the hold another call reco
expect(results.every(r => typeof r?.deny === 'string')).toBe(true)
expect(ran).toEqual([])
})

// Snowflake and Databricks: dbt_cost_estimate answers the planner's EXPLAIN estimate (`source: "explain"`), with no rounding bound.
const planned = (adapter: string, bytes: number, display: string, bound = adapter === 'snowflake' ? 'upper' : 'estimate') => ({
success: true,
supported: true,
adapter,
source: 'explain',
bound,
bytes,
display,
})

for (const [adapter, name] of [
['snowflake', 'Snowflake'],
['databricks', 'Databricks'],
] as const) {
test(`on ${adapter}, a model whose plan estimate is over the limit is held, with the estimate and where it came from`, async ($, on) => {
const { clock, estimates, ran } = await start($, on, { answers: { orders: planned(adapter, 20 * GIB, '20.0 GiB') } })
const result: any = await $.tool.call(bash('dbt run -s orders'))
await clock.settle()

expect(estimates).toEqual(['orders'])
expect(ran).toEqual([])
expect(result.deny).toBe(`≈ 20.0 GiB would be scanned (${name} plan estimate), over the 10.0 GiB limit. Narrow the query, or ask the user to run /allow-scan and then run the same command again.`)
})

test(`on ${adapter}, a model whose plan estimate is under the limit runs without a hold, saying where the estimate came from`, async ($, on) => {
const { clock, ran } = await start($, on, { answers: { orders: planned(adapter, 2 * GIB, '2.0 GiB') } })
const result: any = await $.tool.call(bash('dbt run -s orders'))
await clock.settle()

expect(result.deny).toBeUndefined()
expect(ran).toEqual(['dbt run -s orders'])
expect(result.context).toEqual([`cost · orders would scan 2.0 GiB (${name} plan estimate)`])
})

test(`on ${adapter}, an altimate-code that estimates BigQuery only says a newer one is needed, and holds nothing`, async ($, on) => {
const { clock, ran } = await start($, on, { fallback: { success: true, supported: false, adapter } })
const result: any = await $.tool.call(bash('dbt run -s orders'))
await clock.settle()

expect(result.deny).toBeUndefined()
expect(ran).toEqual(['dbt run -s orders'])
expect(result.context).toEqual([`cost · no estimate on ${adapter}: this altimate-code estimates BigQuery only; ${name} needs a newer altimate-code (the release with Snowflake and Databricks estimates)`])
})
}

test('on Snowflake, a held command is allowed once by /allow-scan, and its note says the estimate is the plan\'s', async ($, on) => {
const { clock, ran } = await start($, on, { answers: { orders: planned('snowflake', 20 * GIB, '20.0 GiB') } })
expect(((await $.tool.call(bash('dbt build -s orders'))) as any).deny).toBeDefined()
expect((await $.command.run({ command: 'allow-scan', args: '' } as any)).text).toContain('orders (≈ 20.0 GiB, Snowflake plan estimate)')
const allowed: any = await $.tool.call(bash('dbt build -s orders'))
await clock.settle()

expect(ran).toEqual(['dbt build -s orders'])
expect(allowed.context).toEqual(['cost · orders would scan 20.0 GiB (Snowflake plan estimate), over the 10.0 GiB limit: allowed once'])
})

test('on Databricks, a table without statistics gives no estimate: the command runs, with the reason', async ($, on) => {
const reason = "Databricks' planner has no size for a table this query reads (it shows Spark's default for a table without statistics). ANALYZE TABLE <table> COMPUTE STATISTICS gives it one."
const { clock, ran } = await start($, on, { answers: { orders: { success: false, adapter: 'databricks', source: 'explain', error: reason } } })
const result: any = await $.tool.call(bash('dbt run -s orders'))
await clock.settle()

expect(result.deny).toBeUndefined()
expect(ran).toEqual(['dbt run -s orders'])
expect(result.context).toEqual([`cost · no estimate for orders: ${reason}`])
})

test('on Databricks, a model with no estimate is asked about again on the next command: one table without statistics does not stop the check', async ($, on) => {
const { clock, estimates, ran } = await start($, on, {
answers: { orders: { success: false, adapter: 'databricks', source: 'explain', error: 'no size' }, customers: planned('databricks', 20 * GIB, '20.0 GiB') },
})
await $.tool.call(bash('dbt run -s orders'))
const second: any = await $.tool.call(bash('dbt run -s customers'))
await clock.settle()

expect(estimates).toEqual(['orders', 'customers'])
expect(ran).toEqual(['dbt run -s orders'])
expect(second.deny).toContain('(Databricks plan estimate)')
})

test('/scan-cost on Snowflake names the plan estimate and the verdict', async ($, on) => {
await start($, on, { answers: { orders: planned('snowflake', 20 * GIB, '20.0 GiB') } })
expect((await $.command.run({ command: 'scan-cost', args: 'orders' } as any)).text).toBe(
'orders would scan 20.0 GiB (Snowflake plan estimate), over the 10.0 GiB limit: a dbt run of it is held until /allow-scan.',
)
})

test('/scan-cost on an older altimate-code over Databricks says a newer one is needed', async ($, on) => {
await start($, on, { fallback: { success: true, supported: false, adapter: 'databricks' } })
expect((await $.command.run({ command: 'scan-cost', args: 'orders' } as any)).text).toBe(
'No estimate: this altimate-code estimates BigQuery only; Databricks needs a newer altimate-code (the release with Snowflake and Databricks estimates).',
)
})

test('a BigQuery dry run is shown as before, with no estimate source', async ($, on) => {
const { clock } = await start($, on, { answers: { orders: UNDER } })
const result: any = await $.tool.call(bash('dbt run -s orders'))
await clock.settle()
expect(result.context).toEqual(['cost · orders would scan 2.0 GiB'])
})

test('on Snowflake, a hold kept while the estimate fails on the retry still names the plan estimate, and so does /allow-scan', async ($, on) => {
const { clock, ran } = await start($, on, {
sequence: { orders: [planned('snowflake', 20 * GIB, '20.0 GiB'), { success: false, adapter: 'snowflake', source: 'explain', error: 'EXPLAIN failed' }] },
})
expect(((await $.tool.call(bash('dbt run -s orders'))) as any).deny).toContain('(Snowflake plan estimate)')
const retry: any = await $.tool.call(bash('dbt run -s orders'))
await clock.settle()

expect(ran).toEqual([])
expect(retry.deny).toBe('≈ 20.0 GiB would be scanned (Snowflake plan estimate), over the 10.0 GiB limit. Narrow the query, or ask the user to run /allow-scan and then run the same command again.')
expect((await $.command.run({ command: 'allow-scan', args: '' } as any)).text).toContain('orders (≈ 20.0 GiB, Snowflake plan estimate)')
})

/** The band above the prompt, as health's tests mount it. */
const BAND = {
plugin: 'altimate-code',
component: 'AbovePrompt',
props: { hasSurvey: false, isCollapsed: false, isWorking: false, maxRows: 10, bodyColumns: 120, scroll: { offset: 0, bodyRows: 10 }, view: {} },
} as const

/** Whether the mounted band shows this exact line. */
async function bandShows($: any, text: string): Promise<boolean> {
const band = await $.ui.mount({ ...BAND, surface: 'terminal' })
const found = await band.find({ type: 'Text', text })
await band.unmount()
return found !== undefined && found !== null
}

for (const [adapter, name] of [
['snowflake', 'Snowflake'],
['databricks', 'Databricks'],
] as const) {
test(`on ${adapter}, the band's held line names the plan estimate, and still does once the retry's estimate fails`, async ($, on) => {
const { clock } = await start($, on, { sequence: { orders: [planned(adapter, 20 * GIB, '20.0 GiB'), { success: false, adapter, source: 'explain', error: 'EXPLAIN failed' }] } })
await $.tool.call(bash('dbt run -s orders'))
await clock.settle()
expect(await bandShows($, `⛔ held: orders would scan 20.0 GiB (${name} plan estimate) (limit 10.0 GiB) · /allow-scan`)).toBe(true)
await $.tool.call(bash('dbt run -s orders'))
await clock.settle()
expect(await bandShows($, `⛔ held: orders would scan 20.0 GiB (${name} plan estimate) (limit 10.0 GiB) · /allow-scan`)).toBe(true)
})
}

test("on BigQuery, the band's held line is as before", async ($, on) => {
const { clock } = await start($, on, { answers: { orders: OVER } })
await $.tool.call(bash('dbt run -s orders'))
await clock.settle()
expect(await bandShows($, '⛔ held: orders would scan 20.0 GiB (limit 10.0 GiB) · /allow-scan')).toBe(true)
})
Loading
Loading