Skip to content

Add opt-in OpenCode local history with incremental rowid-window ingestion - #4744

Closed
Danielweng0902 wants to merge 3 commits into
steipete:mainfrom
Danielweng0902:feat/opencode-shared-history
Closed

Danielweng0902 wants to merge 3 commits into
steipete:mainfrom
Danielweng0902:feat/opencode-shared-history

Conversation

@Danielweng0902

Copy link
Copy Markdown

Third attempt at the OpenCode local history source (#3273), replacing #4574 and #4702 with the incremental-store shape from their close comments. The opt-in contract is unchanged; the storage is new.

The store: incremental rowid windows, not a blob

OpenCodeLocalHistoryStore keeps its own durable SQLite cache whose entries are individual turns keyed by stable source ID (part ID for step-finish parts, message ID for fallback rows, session-message ID for v2 rows), with an index on createdMs:

  • Publications are windowed SQL queries. Every read — cached publications included — is WHERE createdMs >= ? AND createdMs <= ? over the indexed cache. A publication materializes only the rows in its window; nothing scales with total history.
  • Ingestion is incremental. Each source table keeps a [low, high] rowid window of ingested rows — the SQLite analogue of the OpenCodex usage store's byte cursors (a database has no byte offsets, but it does have monotonically assigned rowids). A scan costs one fingerprint stat; when the fingerprint moved, the pass reads only what the window is missing: new rows above high (the common append case), and, for a first-ever pass on a database larger than the row budget, newest-first continuation below low so partial passes still prioritize recent history. Rows persist by upsert, so no pass rewrites rows it did not change.
  • Late step-finish parts are handled: a part rowid cursor re-evaluates the messages new parts belong to, purging the message fallback row so a message that gained a part is never counted twice.
  • Deletions are detected by raw table counts: a count that shrank restarts the window fresh, so removed rows cannot linger. Recorded costs are append-only event data upstream, so growth never requires re-reading what the window already holds.
  • Bounded and cancellable: a 100k-row soft budget per pass with convergence across scans, Task.isCancelled checks every 512 rows in both decode and write loops (a cancelled pass rolls back whole), fingerprint captured before the read and verified after — a moving source never marks itself complete, and cursors only advance to rows actually read.
  • Cache-only publication stays cache-only: the persisted source path is compared as a string (no stat, no open) before the windowed query, so a cache built for another database is never served, and the OpenCode database is never touched by cached reads. A damaged cache file is dropped and rebuilt rather than silently omitting history.
  • Multiple step-finish parts on one message count as separate turns, one entry per part, matching the OpenCode Go reader.

The 129 MB question

Measured with this branch's CLI against a synthetic 100,000-row OpenCode database (22 MB, 9.7k rows inside the reporting window), /usr/bin/time -l, ambient Codex history present:

Scenario Wall Peak RSS
Native only (setting off, refresh) 4.96 s 228 MB
First-ever ingest of 100k rows (one-time, --refresh) 10.88 s 225 MB
Cached publication (fingerprint unchanged) 3.60 s 211 MB
One appended row (fingerprint changed, --refresh) 5.04 s 209 MB

The native Codex scan dominates all rows. The store's marginal cost on a cached publication and on a one-row append is within run-to-run noise of the native-only baseline — the previously reported store-alone 129 MB peak on a cached refresh is gone, and a changed source no longer re-reads and rewrites history (one row changed, one row read: the appended $0.25 turn appeared in Today exactly).

The #3273 contract (unchanged)

Opt-in default-off (codexIncludeOpenCodeLocalHistory); reads usage metadata only with SQLITE_OPEN_READONLY; exact providerID == "openai" assistant records; recorded cost surfaced as-is (OpenCode records $0 for subscription usage); sourceRecorded/mixed provenance with labelled hints; permanent exclusion from scoped CODEX_HOME scans, managed accounts, and the Spend Dashboard; OPENCODE_DB/XDG_DATA_HOME selection follows upstream's single-database contract.

Wiring

  • Foreground scan merges the report behind the ambient gate; background catch-up captures the flag in its context and converges on the same rows (loadCachedCodexTokenSnapshotResult).
  • codexbar cost reads the same default-off setting; session-grouped text excludes the merge (like Pi) with a stderr notice; JSON and the default view include it; all Codex cost views render provenance-aware hints.
  • The menu chart and both inline dashboard hint call sites receive costProvenance.
  • "Clear cost cache" also removes the opencode-local-history cache directory.

Tests

46 tests across the three OpenCode suites, including the incremental behaviors: multi-part messages, append-only ingestion with cursor state, deletion-triggered fresh re-ingest, late-part fallback replacement, multi-pass budget convergence, cross-cache source validation, damaged-cache recovery, and the usable-legacy v1/v2 projection. Related catch-up, CLI cost, and chart suites (217 tests) pass. The full suite has the two known parallel-load-flaky dashboard timing tests that fail identically on the base commit.

Closes nothing — #3273 stays the tracker.

Reimplements steipete#4574 in the shape requested in its close comment: one
shared, fingerprint-gated OpenCodeLocalHistoryStore replaces the
separate full-load reader.

- Bounded memory: the store retains compact per-turn entries and
  windows them into day rows at read time; nothing pre-windowed is
  held.
- Cache-only publication: decoded entries persist in the store's own
  SQLite database (never Codex cost-usage.sqlite, never the OpenCode
  Go reader's in-memory cache), and cached publications read that
  durable cache without touching the OpenCode database at all.
- Fingerprint gating: scans re-read the database only when its
  size/mtime/WAL fingerprint changed, keeping the tightened scan I/O
  budget intact.

The ambient merge keeps the steipete#3273 contract: opt-in and default-off,
OpenAI-provider assistant records only, stable-ID v1/v2 deduplication,
recorded cost surfaced as-is, provenance-labelled sourceRecorded/mixed,
and permanent exclusion from scoped CODEX_HOME scans, managed
accounts, and the Spend Dashboard. The lane/scope/account-switch
behavior from steipete#4574 carries over unchanged.
The claudeConfigDirectory restoration prepared account-specific
projects roots in `options` but the call still passed a fresh
scannerOptions lookup, so a mapped Claude account validated its
directory while the scan read ambient history (ClawSweeper P1 on
steipete#4702). Pass the prepared options through as main did.
Replace the blob cache with the incremental-store shape from steipete#4702's
close comment: per-table [low, high] rowid windows append new rows only,
late step-finish parts re-evaluate their messages, a shrinking raw table
count restarts fresh, and entries persist by part/message/session-message
source ID so every pass is an idempotent upsert. Publications stay
windowed SQL queries over the durable cache.

Also wires the catch-up, CLI, chart, and inline hint call sites through
the opt-in flag and provenance, and extends clear-cost-cache to the
opencode-local-history directory.
@clawsweeper

clawsweeper Bot commented Oct 23, 2026 •

Copy link
Copy Markdown

🦞👀
ClawSweeper picked this up.

Pull request received. I will update this pull request when review starts.

ClawSweeper review complete

ClawSweeper finished reviewing this revision. The review result is being finalized.

View the workflow run.

@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Oct 23, 2026 •

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ⚠️ Failed 2026-10-23T14:44:19.281250Z
We couldn’t complete this review. Comment @codex review to try again.
6877fb5 PR opened
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@clawsweeper clawsweeper Bot added clawsweeper:intake:feature Preliminary submission classification; full source review pending. P2 Normal priority bug or improvement with limited blast radius. merge-risk: 🚨 availability 🚨 Merging this PR could cause crashes, hangs, restart loops, stalls, or process outages. rating: 🧂 unranked krab Not merge-ready due to missing proof or serious correctness/safety concerns. status: 📣 needs proof The PR needs real behavior proof before ClawSweeper can clear the contributor ask. and removed clawsweeper:intake:feature Preliminary submission classification; full source review pending. labels Oct 23, 2026
@clawsweeper

clawsweeper Bot commented Oct 23, 2026

Copy link
Copy Markdown

Codex review: needs real behavior proof before merge.

What this changes

Adds a default-off setting that merges OpenCode’s recorded OpenAI usage into ambient Codex history through a persistent incremental SQLite cache.

Example: An OpenCode OpenAI assistant turn records 150 tokens and $0 subscription cost.

  • Before: Ambient Codex history excludes the turn.
  • After: With the setting enabled, ambient history includes 150 tokens and the recorded $0, with OpenCode provenance.

Review scores

Measure Result What it means
Overall readiness 🧂 unranked krab (1/6) The feature direction is useful, but the state machine has multiple accounting defects and the supplied narrative does not clear the behavior or upgrade proof gates.
Proof confidence 🧂 unranked krab (1/6) Real behavior proof is necessary before merge. See Before merge.
Patch quality 🦪 silver shellfish (2/6) 12 actionable review findings remain.

Product

Kind: Feature · Worth it: Yes · Fix scope: Partial
User problem: Users cannot include ordinary OpenCode OpenAI sessions in ambient Codex token and cost history.
Reason: The maintainer welcomed this default-off, account-isolated direction in the earlier PR discussions; the approved ambient subset has clear value without attempting unsupported historical account attribution.

Merge readiness

⛔ Blocked before merge - 14 items remain

Keep open: this PR implements a useful, maintainer-welcomed feature, but ingestion and cache-consistency defects can omit or miscount history.

Priority: P2
Reviewed head: 6877fb5f9493e200a693a974afe6280c1e2c8715

Before merge

  • Add real behavior proof - The body reports CLI performance measurements and a $0.25 append result, but provides no captured CLI text/JSON, logs, screenshots, or linked artifact exercising this replacement store; supply changed-path output covering ingestion, cached publication, and scoped exclusion, with private paths, endpoints, IPs, and credentials redacted. After adding proof, update the PR body, then request a new review with @clawsweeper re-review. The PR author or someone with repository write access can request it.
  • Revisit existing v2 rows when their usage becomes complete (P2) - OpenCode creates an assistant session_message at step start and UPDATEs that same row with cost and tokens at step end. If a scan runs during the step, this cursor advances past the row even when its cost is absent or still zero. Completion changes the fingerprint but neither the rowid nor table count, so subsequent scans never import the completed usage. Track pending or changed projections rather than treating session_message as append-only.
  • Advance cursors within the source read snapshot (P2) - Each usage SELECT finishes before a separate MAX(rowid) query advances the cursor, and no source read transaction spans them. A turn committed between those statements is absent from the decoded entries but included in the new maximum. Marking the fingerprint unstable does not repair this: the next pass begins beyond the skipped row. Read the boundary and rows from one snapshot or retain a boundary proven to have been consumed.
  • Retain late-part work until message reevaluation finishes (P2) - partHigh advances according to the raw part-ID query while reevaluated.exhausted and its cursor are ignored. With a two-row budget, cache a message with two parts, then append a third: reevaluation can return the two old parts and omit the new one, yet partHigh advances beyond it. The same failure occurs when earlier ingestion consumes most of the production budget. Persist unfinished reevaluation work before advancing the part cursor or purging fallback rows.
  • Measure SQL exhaustion using consumed rows (P2) - The query has a SQL LIMIT, but exhaustion is detected using successfully decoded entries. With budget two and descending rows consisting of one negative-cost row followed by three valid rows, SQL returns only the first three rows; the invalid row leaves entries.count below two until the last returned row, so SQLITE_DONE is mistaken for complete exhaustion. messageLow becomes zero and the remaining valid row is permanently skipped. Account for raw rows and preserve continuation whenever the limited query may have more input.
  • Reconcile deletions even when new rows offset their count (P2) - A smaller table count is the only same-path invalidation condition. Deleting an older session and recording a new turn between refreshes can leave the count unchanged or larger; ingestion then adds the new turn but retains the deleted session’s cached costs indefinitely. Replacing the selected database at the same path can produce the same stale-cache problem. Validate source identity and reconcile removals without relying on a net count decrease.
  • Validate the durable base state under the cache write lock (P2) - The app and CLI share this cache, but cached state is read before source parsing and never revalidated after BEGIN IMMEDIATE. Loader A can prepare an incremental batch for database A while loader B replaces the cache with database B; A then appends its delta to B’s rows and labels the combined cache as A. Compare the durable source identity and cursors against the state used to prepare the batch, and retry a stale write as the existing OpenCodex store does.
  • Preserve a valid cache on transient SQLite failures (P2) - Both cache readers remove the database for every error. If another app or CLI connection holds a lock beyond the 250 ms busy timeout, SQLITE_BUSY is treated as corruption and a healthy cache is unlinked, potentially while its writer still owns the open file. Restrict destructive recovery to corruption or invalid-schema failures and retain the cache on lock contention and other transient errors.
  • Propagate OpenCode ingestion completeness to publications (P2) - The store persists ingestComplete but returns only a daily report, and the merged snapshot’s coverage depends solely on native and Pi history. A complete native scan combined with a budget-limited OpenCode scan therefore presents an incomplete OpenCode subtotal as fully scanned; cached catch-up can likewise accept it with requireCompleteHistory enabled. Return source coverage alongside the report and apply it to foreground and cached publication.
  • Deduplicate model names during daily accumulation (P2) - Every turn appends its model name to the previous array and sorts the entire result. Ten thousand same-model turns on one day create ten thousand identical names and repeatedly copy and sort the growing array before the later merge can deduplicate it. Cached publication repeats this work, defeating the bounded-refresh goal for dense histories. Accumulate unique model names with a set or keyed daily accumulator and sort once.
  • Exclude invalid parts before suppressing the message fallback (P2) - The NOT EXISTS predicate treats any numeric step-finish cost as usable, although stepEntries later rejects negative costs and invalid timestamps. A valid $5 message with a negative-cost part consequently contributes nothing: the part is discarded and the message fallback is suppressed. Use the same usability predicate for selecting parts and deciding whether they replace the fallback.
  • Keep source filesystem resolution out of cached publication (P2) - matchesSelectedSource resolves symlinks on every cached publication, which performs source-path filesystem resolution rather than the promised string-only comparison. If OPENCODE_DB names a symlink that disappears after ingestion, the resolved value changes and previously usable cached history is rejected. Persist the selected lexical path separately from physical scan identity so cache-only publication can compare the selection without accessing the source.
  • Make the CLI heading reflect recorded-cost provenance (P3) - The new merge supplies sourceRecorded snapshots to renderCostText, but its heading still unconditionally calls every Codex amount an “API-equivalent estimate.” An OpenCode-only subscription history therefore displays its recorded $0 under a claim that public API prices were applied, contradicting the provenance-aware footer. Select the heading from the snapshot provenance while preserving the subscription-bill disclaimer.
  • Add data-model compatibility proof - The review found that existing stored data may not work after upgrade. Show that existing data still loads and works with this change.

Findings

  • [P2] Revisit existing v2 rows when their usage becomes complete — Sources/CodexBarCore/Providers/OpenCode/OpenCodeLocalHistoryStore.swift:477-486
  • [P2] Advance cursors within the source read snapshot — Sources/CodexBarCore/Providers/OpenCode/OpenCodeLocalHistoryStore.swift:438-440
  • [P2] Retain late-part work until message reevaluation finishes — Sources/CodexBarCore/Providers/OpenCode/OpenCodeLocalHistoryStore.swift:457-469

Tests

  • Low-value test Tests/CodexBarTests/OpenCodeLocalHistoryStoreTests.swift: Remove or consolidate “a changed database fingerprint forces one re-read”: its append-and-total assertions duplicate the incremental-append test and do not establish its claimed read count.
  • Missing end-to-end proof: Provide captured shipped-CLI output for completed-row updates, budgeted convergence and coverage, plus an upgrade from v0.80.0 preserving settings and native history; the supplied tests do not demonstrate base-fail/head-pass behavior for the findings.
Agent review details

How this fits together

The local-history store reads OpenCode usage metadata, caches individual turns, and supplies windowed reports to Codex menu and CLI aggregation while excluding scoped homes and account-owned spend views.

flowchart LR
  A[OpenCode SQLite] --> B[Incremental ingestion]
  B --> C[CodexBar history cache]
  C --> D[Windowed report]
  E[Native Codex history] --> F[Ambient aggregation]
  D --> F
  F --> G[Menu and cost CLI]
Loading

Technical review

Best possible solution:

Use a bounded, transactionally validated ingestion adapter that observes completed updates, reconciles deletions, preserves cursor progress, and exposes coverage through the existing reporting owners.

Do we have a high-confidence way to reproduce the issue?

The reviewed source establishes the failing cursor, update, deletion, and aggregation paths; no target code was executed during this read-only review.

Is this the best way to solve the issue?

Indexed window queries improve the previous implementation, but rowid-only progress is insufficient for OpenCode’s mutable projections and needs transactional consistency and explicit coverage.

Full review comments:

  • [P2] Revisit existing v2 rows when their usage becomes complete — Sources/CodexBarCore/Providers/OpenCode/OpenCodeLocalHistoryStore.swift:477-486
    OpenCode creates an assistant session_message at step start and UPDATEs that same row with cost and tokens at step end. If a scan runs during the step, this cursor advances past the row even when its cost is absent or still zero. Completion changes the fingerprint but neither the rowid nor table count, so subsequent scans never import the completed usage. Track pending or changed projections rather than treating session_message as append-only.
    Confidence: 1
  • [P2] Advance cursors within the source read snapshot — Sources/CodexBarCore/Providers/OpenCode/OpenCodeLocalHistoryStore.swift:438-440
    Each usage SELECT finishes before a separate MAX(rowid) query advances the cursor, and no source read transaction spans them. A turn committed between those statements is absent from the decoded entries but included in the new maximum. Marking the fingerprint unstable does not repair this: the next pass begins beyond the skipped row. Read the boundary and rows from one snapshot or retain a boundary proven to have been consumed.
    Confidence: 0.99
  • [P2] Retain late-part work until message reevaluation finishes — Sources/CodexBarCore/Providers/OpenCode/OpenCodeLocalHistoryStore.swift:457-469
    partHigh advances according to the raw part-ID query while reevaluated.exhausted and its cursor are ignored. With a two-row budget, cache a message with two parts, then append a third: reevaluation can return the two old parts and omit the new one, yet partHigh advances beyond it. The same failure occurs when earlier ingestion consumes most of the production budget. Persist unfinished reevaluation work before advancing the part cursor or purging fallback rows.
    Confidence: 0.99
  • [P2] Measure SQL exhaustion using consumed rows — Sources/CodexBarCore/Providers/OpenCode/OpenCodeLocalHistoryStore.swift:1103-1111
    The query has a SQL LIMIT, but exhaustion is detected using successfully decoded entries. With budget two and descending rows consisting of one negative-cost row followed by three valid rows, SQL returns only the first three rows; the invalid row leaves entries.count below two until the last returned row, so SQLITE_DONE is mistaken for complete exhaustion. messageLow becomes zero and the remaining valid row is permanently skipped. Account for raw rows and preserve continuation whenever the limited query may have more input.
    Confidence: 0.99
  • [P2] Reconcile deletions even when new rows offset their count — Sources/CodexBarCore/Providers/OpenCode/OpenCodeLocalHistoryStore.swift:382-386
    A smaller table count is the only same-path invalidation condition. Deleting an older session and recording a new turn between refreshes can leave the count unchanged or larger; ingestion then adds the new turn but retains the deleted session’s cached costs indefinitely. Replacing the selected database at the same path can produce the same stale-cache problem. Validate source identity and reconcile removals without relying on a net count decrease.
    Confidence: 0.99
  • [P2] Validate the durable base state under the cache write lock — Sources/CodexBarCore/Providers/OpenCode/OpenCodeLocalHistoryStore.swift:572-575
    The app and CLI share this cache, but cached state is read before source parsing and never revalidated after BEGIN IMMEDIATE. Loader A can prepare an incremental batch for database A while loader B replaces the cache with database B; A then appends its delta to B’s rows and labels the combined cache as A. Compare the durable source identity and cursors against the state used to prepare the batch, and retry a stale write as the existing OpenCodex store does.
    Confidence: 0.99
  • [P2] Preserve a valid cache on transient SQLite failures — Sources/CodexBarCore/Providers/OpenCode/OpenCodeLocalHistoryStore.swift:140-143
    Both cache readers remove the database for every error. If another app or CLI connection holds a lock beyond the 250 ms busy timeout, SQLITE_BUSY is treated as corruption and a healthy cache is unlinked, potentially while its writer still owns the open file. Restrict destructive recovery to corruption or invalid-schema failures and retain the cache on lock contention and other transient errors.
    Confidence: 0.97
  • [P2] Propagate OpenCode ingestion completeness to publications — Sources/CodexBarCore/CostUsageFetcher.swift:1064-1066
    The store persists ingestComplete but returns only a daily report, and the merged snapshot’s coverage depends solely on native and Pi history. A complete native scan combined with a budget-limited OpenCode scan therefore presents an incomplete OpenCode subtotal as fully scanned; cached catch-up can likewise accept it with requireCompleteHistory enabled. Return source coverage alongside the report and apply it to foreground and cached publication.
    Confidence: 1
  • [P2] Deduplicate model names during daily accumulation — Sources/CodexBarCore/Providers/OpenCode/OpenCodeLocalHistoryStore.swift:218-220
    Every turn appends its model name to the previous array and sorts the entire result. Ten thousand same-model turns on one day create ten thousand identical names and repeatedly copy and sort the growing array before the later merge can deduplicate it. Cached publication repeats this work, defeating the bounded-refresh goal for dense histories. Accumulate unique model names with a set or keyed daily accumulator and sort once.
    Confidence: 1
  • [P2] Exclude invalid parts before suppressing the message fallback — Sources/CodexBarCore/Providers/OpenCode/OpenCodeLocalHistoryStore.swift:1193-1196
    The NOT EXISTS predicate treats any numeric step-finish cost as usable, although stepEntries later rejects negative costs and invalid timestamps. A valid $5 message with a negative-cost part consequently contributes nothing: the part is discarded and the message fallback is suppressed. Use the same usability predicate for selecting parts and deciding whether they replace the fallback.
    Confidence: 0.99
  • [P2] Keep source filesystem resolution out of cached publication — Sources/CodexBarCore/Providers/OpenCode/OpenCodeLocalHistoryStore.swift:303-306
    matchesSelectedSource resolves symlinks on every cached publication, which performs source-path filesystem resolution rather than the promised string-only comparison. If OPENCODE_DB names a symlink that disappears after ingestion, the resolved value changes and previously usable cached history is rejected. Persist the selected lexical path separately from physical scan identity so cache-only publication can compare the selection without accessing the source.
    Confidence: 0.97
  • [P3] Make the CLI heading reflect recorded-cost provenance — Sources/CodexBarCLI/CLICostCommand.swift:225-227
    The new merge supplies sourceRecorded snapshots to renderCostText, but its heading still unconditionally calls every Codex amount an “API-equivalent estimate.” An OpenCode-only subscription history therefore displays its recorded $0 under a claim that public API prices were applied, contradicting the provenance-aware footer. Select the heading from the snapshot provenance while preserving the subscription-bill disclaimer.
    Confidence: 1

Overall correctness: patch is incorrect
Overall confidence: 0.99

AGENTS.md: found and applied where relevant.

Codex review notes: model internal, reasoning medium; reviewed against d6c651ca26c4.

Provenance checked

Testing

Proof path: in-process harness.

Security

None.

Evidence

What I checked:

  • Maintainer direction: The maintainer welcomed the default-off, account-isolated feature and requested bounded incremental storage with windowed cached publication in Add opt-in OpenCode local history for the ambient Codex ledger #4574 (comment) and Add opt-in OpenCode local history on the shared incremental store #4702 (comment).
  • Current-main capability boundary: Current-main documentation excludes ordinary OpenCode OpenAI/Codex sessions from local cost history; the existing SQLite reader selects opencode-go records instead. (docs/opencode.md:63, d6c651ca26c4)
  • Introduced implementation: The pinned introduced diff adds indexed per-turn cache rows and ambient-only merge gates, but advances cursors using later table maxima and publishes reports without the store’s completeness state. (Sources/CodexBarCore/Providers/OpenCode/OpenCodeLocalHistoryStore.swift:438, 6877fb5f9493)
  • Authoritative source lifecycle: This dependency contract applies because the PR consumes OpenCode’s message/session_message schema and claims append-only usage ingestion: session.next.step.ended updates an existing assistant’s cost and tokens, and the projector performs UPDATE on session_message rather than inserting a new row. (packages/core/src/session/projector.ts:115, 10765ff2a9da)
  • Proof inspected: The full current body and comments contain a CLI benchmark table, a reported $0.25 append result, and test claims, but no captured changed-path output or upgrade artifact; body source identity is a5f33638b42b009252a19f9d5876df7353b507649e119191e4703defde460fd1. (6877fb5f9493)
  • Checks and inspection limits: The reviewed head has a successful GitGuardian check and no recorded build/test checks; tests were not executed during this read-only review, the checkout stayed clean, and some deeper historical blobs were unavailable despite readable scoped commit history. (6877fb5f9493)

Likely related people:

  • Yuxin Qiao: Raw commit 81c8a1c adds Sources/CodexBar/UsageStore+CodexCostCatchUp.swift:275 relative to its recorded parents. This identifies author metadata, not feature responsibility or a PR merger. (role: source-line author; confidence: high; commits: 81c8a1cbf9bb; files: Sources/CodexBar/UsageStore+CodexCostCatchUp.swift)

Root-cause cluster

Relationship: partial_overlap
Canonical: #3273
Summary: This PR implements the ambient-history portion of the original request and explicitly replaces two closed implementations; historical account attribution remains outside its scope.

Members:

Proposal only: this assessment does not dispatch repair, suppress jobs, mutate sibling items, close, or merge anything.

Labels

Label changes:

  • add P2: This is an approved optional history feature with accounting defects, without an established emergency in currently shipped behavior.
  • add merge-risk: 🚨 availability: Daily accumulation repeatedly copies and sorts a model-name array that grows once per turn, allowing dense histories to stall the shared cost scan even on cached publication.
  • add rating: 🧂 unranked krab: Overall readiness is 🧂 unranked krab; proof is 🧂 unranked krab and patch quality is 🦪 silver shellfish.
  • add status: 📣 needs proof: The PR needs real behavior proof before ClawSweeper can clear the contributor ask.

Label justifications:

  • P2: This is an approved optional history feature with accounting defects, without an established emergency in currently shipped behavior.
  • merge-risk: 🚨 availability: Daily accumulation repeatedly copies and sorts a model-name array that grows once per turn, allowing dense histories to stall the shared cost scan even on cached publication.
  • rating: 🧂 unranked krab: Overall readiness is 🧂 unranked krab; proof is 🧂 unranked krab and patch quality is 🦪 silver shellfish.
  • status: 📣 needs proof: The PR needs real behavior proof before ClawSweeper can clear the contributor ask.

Rating scale

6/6 🦀 challenger crab · 5/6 🦞 diamond lobster · 4/6 🐚 platinum hermit · 3/6 🦐 gold shrimp · 2/6 🦪 silver shellfish · 1/6 🧂 unranked krab. Overall follows the weaker of proof and patch quality; ✨ marks media proof (a screenshot, video, or linked artifact) that directly shows the changed behavior.

Workflow

ClawSweeper edits this one comment on every review. Comment @clawsweeper re-review for a fresh review only. Repair and merge execution lanes are retired.

Reviewed October 23, 2026, 11:02 AM ET / 15:02 UTC.

@steipete

Copy link
Copy Markdown
Owner

Thanks @Danielweng0902 for sticking with this, and cached reads really did improve: in a synthetic check at 1k, 10k and 100k history rows, the indexed 100-row window stayed flat at about 1,333 SQLite instructions. I'm closing this version anyway, because it still adds a separate ~1,300-line SQLite store (its own schema, connections and persistence) instead of an adapter onto the existing incremental history store, so the existing history performance gates don't cover it. The checks also turned up three cursor problems: an invalid row can mark an unfinished range complete and skip an older valid row, a message with more parts than the per-pass budget never converges, and an append between the read and the later MAX(rowid) can be skipped for good. Initial ingestion also still scales with history (about 37k / 370k / 3.7M instructions for a two-row budget).

The shape we'd take is a source adapter that feeds OpenCode rows into the existing store and reuses its persistence, bounded publication and completion tracking. I know that's a different structure from what you've built, so if you'd like to try it, a short sketch in an issue first would let us confirm the approach before you invest in the full implementation.

@steipete steipete closed this Oct 23, 2026
Danielweng0902 added a commit to Danielweng0902/Win-CodexBar that referenced this pull request Oct 25, 2026
Stage 1+2 of the steipete#4752 checklist. The adapter decodes OpenCode assistant
turns inside short read-only source transactions and commits usage rows,
per-file day aggregates, global deltas and its typed checkpoint
atomically through applyOpenCodeLocalHistoryPass (baseline
revalidation, rollback on stale). Turns keep hash-stable identities
over <table>:<sourceID>; windows walk by exclusive rowid boundaries
pinned by their own SELECT with raw-lookahead exhaustion.

Explicit regressions for the three steipete#4744 cursor failures: invalid rows
consume budget without starving older valid rows, oversized messages
checkpoint a strict (messageRowid, partRowid) continuation and resume
mid-message without re-emitting, and appends after a snapshot stay
eligible because boundaries never come from a later MAX(rowid). Tests:
5 adapter tests + 4 checkpoint seam tests, all passing.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

merge-risk: 🚨 availability 🚨 Merging this PR could cause crashes, hangs, restart loops, stalls, or process outages. P2 Normal priority bug or improvement with limited blast radius. rating: 🧂 unranked krab Not merge-ready due to missing proof or serious correctness/safety concerns. status: 📣 needs proof The PR needs real behavior proof before ClawSweeper can clear the contributor ask.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants