Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
36 commits
Select commit Hold shift + click to select a range
d0f9aef
feat: `finops_databricks_savings`: Databricks SQL warehouse savings
shreyastelkar Nov 17, 2026
3cbff41
feat: `finops_databricks_savings`: idle clusters, jobs on all-purpose…
shreyastelkar Nov 18, 2026
022b90c
fix: hand core the job inventory's trigger; window, limit and blank-n…
shreyastelkar Nov 18, 2026
44ed9d8
fix: keep the warehouse findings when core refuses cluster or job facts
shreyastelkar Nov 18, 2026
bcb5af2
feat: read every workspace the connection's system tables cover
shreyastelkar Nov 18, 2026
3052130
docs: savings tool header, refusal text and example match every-works…
shreyastelkar Nov 18, 2026
c253ac7
test: the live savings check must succeed; docs: only cluster and job…
shreyastelkar Nov 18, 2026
c4511d2
fix: assessment mode: the packaged CLI contacts only the customer's w…
shreyastelkar Nov 18, 2026
19d5cdd
docs: state the savings cost window as the queries and core compute it
shreyastelkar Nov 18, 2026
ffd8be6
fix: assessment mode also covers sign-in lookups, remote config, plug…
shreyastelkar Nov 18, 2026
3301c32
fix: assessment mode keeps the well-known token for substitution; fla…
shreyastelkar Nov 18, 2026
6e0c010
fix: assessment mode keeps local sign-out cleanup and makes no active…
shreyastelkar Nov 18, 2026
4e3799c
fix: assessment mode installs no npm plugin in the plugin loader
shreyastelkar Nov 18, 2026
cd5ed30
docs: serverless SQL warehouses get the over-sized check
shreyastelkar Nov 18, 2026
db658f4
fix: assessment mode switches core's own telemetry off before the sav…
shreyastelkar Nov 18, 2026
41fce2a
fix: assessment mode silences altimate-core's own telemetry for every…
shreyastelkar Nov 18, 2026
2ab21ec
fix: assessment mode silences core before `mcp serve` returns from th…
shreyastelkar Nov 18, 2026
73683e1
test: the middleware test passes the assessment flag and core switch,…
shreyastelkar Nov 18, 2026
6d1ee99
feat: the savings check runs only with `confirm: true`, and stops aft…
shreyastelkar Nov 18, 2026
a3338f7
fix: the savings check's limit covers connecting and the run's time b…
shreyastelkar Nov 18, 2026
a80b2f1
fix: the jobs savings check finds jobs through the compute their task…
shreyastelkar Nov 18, 2026
b468212
fix: job facts skip rows with no job, keep every task, and count job …
shreyastelkar Nov 18, 2026
48545c1
Merge origin/main into feat/databricks-savings-tool
shreyastelkar Nov 19, 2026
20b89b3
fix: close a Databricks statement whose fetch fails, and test the mis…
shreyastelkar Nov 19, 2026
16d0af7
feat: the Snowflake driver cancels on `cancelAfterMs` or abort, and s…
shreyastelkar Nov 18, 2026
db06540
feat: `finops_snowflake_savings`, the Snowflake warehouse savings check
shreyastelkar Nov 18, 2026
ade5a6b
fix: Snowflake savings counts restart gaps in whole elapsed seconds
shreyastelkar Nov 18, 2026
89567af
feat: the Databricks savings check reports how many workspaces it che…
shreyastelkar Nov 20, 2026
f59833a
docs: assessment mode claims no Altimate contact only for the savings…
shreyastelkar Nov 20, 2026
7400c9b
fix: a Snowflake query with no cloud services credits keeps its compu…
shreyastelkar Nov 20, 2026
211c191
fix: Databricks and Snowflake cancel an aborted statement without a d…
shreyastelkar Nov 20, 2026
f5ef7a1
fix: the savings checks start no work after a stop and read the rate …
shreyastelkar Nov 20, 2026
7187e47
fix: assessment mode leaves the user's workspace skills and rules in …
shreyastelkar Nov 20, 2026
78dc8e3
fix: Databricks waits for the close on ordinary queries; savings outp…
shreyastelkar Nov 20, 2026
a75b3ed
fix: a client-side query timeout abandons the statement; only a stop …
shreyastelkar Nov 20, 2026
f7d74f2
Merge origin/main into the savings tools branch
shreyastelkar Nov 20, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -26,6 +26,7 @@ env:
PROCESS_SUITES: >-
test/altimate/learn/bootstrap-cli.test.ts
test/altimate/tool-cli/cli.test.ts
test/altimate/assessment-zero-egress.test.ts
test/altimate/rules/host-cli.test.ts
test/upstream/bridge-merge-e2e.test.ts

Expand Down
66 changes: 66 additions & 0 deletions docs/docs/data-engineering/tools/finops-tools.md
Original file line number Diff line number Diff line change
Expand Up @@ -178,6 +178,72 @@ Recommendations:

---

## finops_databricks_savings

Check a Databricks workspace for savings with Altimate's rules, on a Databricks connection whose `http_path` is a SQL warehouse (`/sql/1.0/warehouses/<id>`). It reads seven days of system tables (`system.billing`, `system.compute`, `system.query`, `system.lakeflow`) on that warehouse, which Databricks starts if it is stopped. System tables cover every workspace of the account they can see, so one connection checks all of them; each finding names its workspace. Nothing is written. Four checks:
Comment thread
shreyastelkar marked this conversation as resolved.
Resolved
Comment thread
shreyastelkar marked this conversation as resolved.
Resolved
Comment thread
shreyastelkar marked this conversation as resolved.
Resolved

- **Migrate Classic/Pro to Serverless:** a Classic or Pro warehouse that bills for time it sits idle.
- **Over-Sized Warehouse:** a warehouse whose queries never use the cores it pays for.
Comment thread
shreyastelkar marked this conversation as resolved.
Resolved
Comment thread
shreyastelkar marked this conversation as resolved.
Resolved
- **Idle Cluster Detection:** an all-purpose cluster with idle workers and auto-termination off or an hour or more.
- **Job Running on All-Purpose Compute:** a job with at least one scheduled run in the seven days, or at least 24 hours of continuous running in one task, on an all-purpose cluster. Its DBUs would bill at the jobs-compute rate on a job cluster.

The tool measures each resource's usage; altimate-core decides what qualifies and what it saves. Prices are list prices. Each resource is counted once in the total, and each flagged warehouse also gets what it would cost at every smaller size and as serverless. If the connection cannot read the cluster or job tables, that check is named as not checked and the warehouse checks still report; the warehouse tables themselves are required.

**What it does not assess.** Databricks Apps spend is not covered, and a serverless SQL warehouse gets only the Over-Sized Warehouse check. A SQL warehouse's cost is its billed usage for the six full days before the window's last day, counting only records Databricks had ingested before that day, spread over the seven-day window as Altimate's pipeline does, so it reads lower than seven days of billing (on one measured workspace, about 7% lower). A cluster's cost is its billed usage for the seven full days before the window's last day. A job's cost is estimated, because Databricks billing does not name the job on all-purpose compute: its task hours from the job timeline times the DBUs per hour its cluster billed. When the jobs on one cluster would add up to more DBUs than it billed, their estimates are scaled down to fit. The connection needs the `@databricks/sql` driver installed.

**Asks first, and stops long queries.** The check runs only when called with `confirm: true`; without it, the agent is told to ask you first, since a run can start a stopped warehouse that then bills. It runs three queries and stops after 9 minutes, connecting included, cancelling any query still running on the warehouse (or sooner, at an `altimate run --time-budget` cutoff); stopping the session cancels the running query too. The Claude Code `/savings` command passes `confirm: true`, since typing it is the go-ahead.

**Assessment mode.** Set `ALTIMATE_ASSESSMENT_MODE=true` to keep a savings check from contacting anything but your warehouse. It sends no telemetry (not even the one-time opt-out event), treats you as signed out of Altimate (no workspace or project-binding lookups; workspace skills already on disk stay as they are, and pulled team rules are not applied while it is on), fetches no remote config and installs no plugin dependencies, and downloads neither the model catalog (models.dev) nor ripgrep (github.com). Any value but empty, `0` or `false` turns it on, and a tool daemon started without it is not reused. It is measured for `tool run` of the two savings tools. Other tools and commands are not covered: a signed-in user's other Altimate tools can still reach Altimate, and so can the update check and language-server downloads. Core's compiled analysis is not audited for network use. Without it, `ALTIMATE_TELEMETRY_DISABLED=true` alone still lets the CLI send that opt-out event and fetch those. The Claude Code `/savings` command sets it for you. Tool results shown in a Claude Code conversation reach its model provider; assessment mode does not change that.

```
> finops_databricks_savings --warehouse prod_dbx --confirm

Databricks savings: 7 SQL warehouses, 3 all-purpose clusters, 12 jobs checked, the 7 days to 2026-11-17, at list price.
Total: $85.17–$106 a month on 1 resource, each counted once at its largest saving.

Classic (sql warehouse, workspace 525779153077873) · Migrate Classic/Pro to Serverless · $85.17–$106 a month (critical)
Comment thread
shreyastelkar marked this conversation as resolved.
Resolved
CLASSIC SMALL warehouse, 95% idle over 10.8 running hours (2,988 queries)
$29.29 in DBUs over the window; serverless would bill $4.44 for the busy time alone

What Classic (now SMALL (CLASSIC), $125 a month) would cost instead:
SERVERLESS: $19.03 a month, saving $106
X-SMALL: $62.60 a month, saving $62.60
2X-SMALL: $41.73 a month, saving $83.47
```


---

## finops_snowflake_savings

Check a Snowflake account for savings with Altimate's rules. It lists warehouses with `SHOW WAREHOUSES` and, to report on seven days, reads up to eleven days of `SNOWFLAKE.ACCOUNT_USAGE` (`WAREHOUSE_METERING_HISTORY`, `WAREHOUSE_EVENTS_HISTORY`, `QUERY_HISTORY`), read-only, on the connection's warehouse. The role needs `IMPORTED PRIVILEGES` on the `SNOWFLAKE` database, and sees only the warehouses it has access to; warehouses that billed credits but cannot be listed are named as not checked, and so is usage an earlier warehouse billed under a name a newer one now has.

- **High Idle Time:** on days a warehouse sat idle for more than 40% of its active time, the idle share of that day's cost.
- **Downsize Warehouse:** nearly all queries finish within 5 seconds, or nearly all active periods last 80 seconds or less, so one size smaller would do.
- **Underutilized Cluster:** the highest cluster of a multi-cluster warehouse is barely needed. It is reported without a dollar estimate.
- **Frequent Restart Overcharge:** a warehouse that suspends and resumes within one billed minute pays the overlap twice.

The tool measures each warehouse's usage; altimate-core decides what qualifies and what it saves. Credits are priced from `ORGANIZATION_USAGE.RATE_SHEET_DAILY` when the role can read it in dollars, otherwise at $3.00 (also for an account billed in another currency), and the output says which. Each warehouse is counted once in the total.

**What it does not assess.** These are four of Altimate's warehouse checks. Storage, serverless features and individual query tuning are not covered. As in Altimate's hosted product, idle time is measured within active periods (resume to the next suspend or resume) that ran at least one query, and a period counts on the day it started: a warehouse that never suspended in the window has no completed active period, and one resumed and left running with no query at all adds no idle time, so neither gets an idle finding for that time. Days are UTC, and the window ends at 00:00 UTC on the latest day at least six hours past, because `ACCOUNT_USAGE` lags by up to that long.

**Asks first, and stops long queries.** The check runs only when called with `confirm: true`; without it, the agent is told to ask you first, since a run can resume a suspended warehouse that then bills. It runs six queries and stops after 9 minutes, connecting included, cancelling any query still running on Snowflake (or sooner, at an `altimate run --time-budget` cutoff); stopping the session cancels the running query too.

**Assessment mode.** `ALTIMATE_ASSESSMENT_MODE=true` works as it does for `finops_databricks_savings`: the run contacts your Snowflake account, and turns off the Snowflake driver's cloud-platform detection. If a result is large enough for Snowflake to deliver it from its own cloud storage, the driver fetches it from there too.

```
> finops_snowflake_savings --warehouse prod_sf --confirm

Snowflake savings: 12 warehouses checked, the 7 days to 2026-11-18, at $3.00 a credit from the account's rate sheet.
Total: $54.00–$90.00 a month on 1 warehouse, each counted once at its largest saving.

BI_WH (warehouse) · Downsize Warehouse · $54.00–$90.00 a month (medium)
Large could be Medium: 95.0% of its active periods lasted 80 seconds or less
The saving is a share of $42.00 over the window
```

---

## finops_role_grants

Analyze role permissions and access patterns (RBAC).
Expand Down
11 changes: 9 additions & 2 deletions packages/core/src/flag/flag.ts
Original file line number Diff line number Diff line change
Expand Up @@ -97,7 +97,13 @@ export const Flag = {
OPENCODE_DISABLE_TERMINAL_TITLE: truthy("OPENCODE_DISABLE_TERMINAL_TITLE"),
OPENCODE_SHOW_TTFD: truthy("OPENCODE_SHOW_TTFD"),
OPENCODE_DISABLE_AUTOCOMPACT: truthy("OPENCODE_DISABLE_AUTOCOMPACT"),
OPENCODE_DISABLE_MODELS_FETCH: truthy("OPENCODE_DISABLE_MODELS_FETCH"),
// altimate_change start — assessment mode: a savings check contacts nothing but the customer's own warehouse
// Fails closed like the other kill switches: `yes` or `on` must not leave the CLI contacting hosts.
get ALTIMATE_ASSESSMENT_MODE() {
return killSwitchSet(process.env["ALTIMATE_ASSESSMENT_MODE"])
},
OPENCODE_DISABLE_MODELS_FETCH: truthy("OPENCODE_DISABLE_MODELS_FETCH") || killSwitchSet(process.env["ALTIMATE_ASSESSMENT_MODE"]),
// altimate_change end
OPENCODE_DISABLE_MOUSE: truthy("OPENCODE_DISABLE_MOUSE"),
// altimate_change start — documented ALTIMATE_CLI_ name read first (see `env`)
OPENCODE_FAKE_VCS: env("OPENCODE_FAKE_VCS"),
Expand Down Expand Up @@ -131,7 +137,8 @@ export const Flag = {
// runtime `--` middleware can flip it between plugin activation and command execution.
// Unrelated to OPENCODE_EXPERIMENTAL_WORKSPACES (multi-instance control plane).
get ALTIMATE_DISABLE_WORKSPACE() {
return killSwitchSet(env("ALTIMATE_DISABLE_WORKSPACE"))
// Assessment mode also turns the SaaS workspace paths off: they send the project path and sign-in to Altimate.
return killSwitchSet(env("ALTIMATE_DISABLE_WORKSPACE")) || killSwitchSet(process.env["ALTIMATE_ASSESSMENT_MODE"])
Comment thread
shreyastelkar marked this conversation as resolved.
Resolved
},
/**
* Workspace precedence escape hatch, set by `--integrations=local`. When on, the
Expand Down
3 changes: 3 additions & 0 deletions packages/core/src/models-dev.ts
Original file line number Diff line number Diff line change
Expand Up @@ -217,6 +217,9 @@ export const layer = Layer.effect(
const get = (): Effect.Effect<Record<string, Provider>> => cachedGet

const refresh = Effect.fn("ModelsDev.refresh")(function* (force = false) {
// altimate_change start — assessment mode contacts no catalog host, even when asked to refresh
if (Flag.ALTIMATE_ASSESSMENT_MODE) return
// altimate_change end
if (!force && (yield* fresh())) return
yield* Effect.scoped(
Effect.gen(function* () {
Expand Down
4 changes: 4 additions & 0 deletions packages/core/src/ripgrep/binary.ts
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,7 @@ import { LayerNode } from "../effect/layer-node"
import { httpClient } from "../effect/layer-node-platform"
import { FSUtil } from "../fs-util"
import { Global } from "../global"
import { Flag } from "../flag/flag"
import { which } from "../util/which"
// altimate_change start — upstream_fix: unzip in-process instead of shelling out to PowerShell.
import { randomUUID } from "node:crypto"
Expand Down Expand Up @@ -173,6 +174,9 @@ export namespace RipgrepBinary {
const config = PLATFORM[platformKey]
if (!config) throw new Error(`unsupported platform for ripgrep: ${platformKey}`)

// altimate_change start — assessment mode downloads nothing: a missing ripgrep fails the search
if (Flag.ALTIMATE_ASSESSMENT_MODE) return yield* Effect.fail(new Error("ripgrep is not installed and downloads are off in assessment mode"))
// altimate_change end
const filename = `ripgrep-${VERSION}-${config.platform}.${config.extension}`
const url = `https://github.com/BurntSushi/ripgrep/releases/download/${VERSION}/${filename}`

Expand Down
86 changes: 83 additions & 3 deletions packages/drivers/src/databricks.ts
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,8 @@
*/

import type { ConnectionConfig, Connector, ConnectorResult, ExecuteOptions, SchemaColumn } from "./types"
import { QueryTimeoutError } from "./types"
import { isStopRequest, untilStopped, type Stopped } from "./stop"
import { loadOptionalDriver } from "./resolve"

export async function connect(config: ConnectionConfig): Promise<Connector> {
Expand All @@ -17,6 +19,86 @@ export async function connect(config: ConnectionConfig): Promise<Connector> {
return value.replace(/`/g, "``")
}

// How long to wait for the server to acknowledge a cancel before reporting the statement as
// possibly still running.
const CANCEL_WAIT_MS = 30_000

/** Frees the statement on the server. A failed close never replaces the query's own outcome. */
const closeQuietly = (operation: any): Promise<void> => operation.close().catch(() => {})

async function cancelAndClose(operation: any): Promise<boolean> {
let timer: ReturnType<typeof setTimeout> | undefined
const cancelled = await Promise.race([
operation.cancel().then(
() => true,
() => false,
),
new Promise<boolean>((resolve) => {
timer = setTimeout(() => resolve(false), CANCEL_WAIT_MS)
}),
]).finally(() => clearTimeout(timer))
void closeQuietly(operation)
return cancelled
}

// Past `cancelAfterMs`, or when `signal` is stopped, the statement is cancelled on the warehouse: SQL
// warehouses ignore the client's server-side `queryTimeout`. Otherwise, a client-side query timeout
// included, it runs to the end.
async function runStatement(
query: string,
stmtOptions: Record<string, any>,
options?: ExecuteOptions,
): Promise<any[]> {
const deadlineMs = options?.cancelAfterMs && options.cancelAfterMs > 0 ? options.cancelAfterMs : undefined
const signal = options?.signal
if (deadlineMs === undefined && !signal) {
const operation = await session.executeStatement(query, stmtOptions)
let rows: any[]
try {
rows = await operation.fetchAll()
} catch (error) {
// A statement that failed while fetching is still open on the session: free it, and
// report the fetch's own error rather than any from the close.
void closeQuietly(operation)
throw error
}
await closeQuietly(operation)
return rows
}
signal?.throwIfAborted()
let stopped = false
let operation: any
const work = (async () => {
const started = await session.executeStatement(query, stmtOptions)
// A statement the server accepted only after the stop is cancelled as soon as it exists.
if (stopped) {
await cancelAndClose(started)
return [] as any[]
}
operation = started
return operation.fetchAll() as Promise<any[]>
})()
let outcome: Stopped<any[]>
try {
outcome = await untilStopped(work, { deadlineMs, signal, stopsOnAbort: isStopRequest, onStop: () => (stopped = true) })
} catch (error) {
// The fetch failed before any stop: the statement is still open on the session.
if (operation) void closeQuietly(operation)
throw error
}
if (!outcome.stopped) {
// Closing frees the server's result, and a failed close must not fail a query that already
// returned. Without a deadline it is awaited as on the plain path; under one it runs in the
// background, so a slow close cannot push the query past it.
if (deadlineMs === undefined) await closeQuietly(operation)
else void closeQuietly(operation)
return outcome.value
}
const cancelled = operation ? await cancelAndClose(operation) : false
if (outcome.stopped === "aborted") throw signal!.reason
throw new QueryTimeoutError({ engine: "databricks", timeoutMs: deadlineMs!, cancelled })
}

return {
async connect() {
const DBSQLClient = databricksModule.DBSQLClient ?? databricksModule
Expand Down Expand Up @@ -55,9 +137,7 @@ export async function connect(config: ConnectionConfig): Promise<Connector> {
if (binds?.length) {
stmtOptions.ordinalParameters = binds
}
const operation = await session.executeStatement(query, stmtOptions)
const rows = await operation.fetchAll()
await operation.close()
const rows = await runStatement(query, stmtOptions, options)

const columns = rows.length > 0 ? Object.keys(rows[0]) : []
const truncated = effectiveLimit > 0 && rows.length > effectiveLimit
Expand Down
Loading
Loading