Repository navigation
Add opt-in OpenCode local history on the shared incremental store - #4702
Danielweng0902 wants to merge 2 commits into
Conversation
Reimplements steipete#4574 in the shape requested in its close comment: one shared, fingerprint-gated OpenCodeLocalHistoryStore replaces the separate full-load reader. - Bounded memory: the store retains compact per-turn entries and windows them into day rows at read time; nothing pre-windowed is held. - Cache-only publication: decoded entries persist in the store's own SQLite database (never Codex cost-usage.sqlite, never the OpenCode Go reader's in-memory cache), and cached publications read that durable cache without touching the OpenCode database at all. - Fingerprint gating: scans re-read the database only when its size/mtime/WAL fingerprint changed, keeping the tightened scan I/O budget intact. The ambient merge keeps the steipete#3273 contract: opt-in and default-off, OpenAI-provider assistant records only, stable-ID v1/v2 deduplication, recorded cost surfaced as-is, provenance-labelled sourceRecorded/mixed, and permanent exclusion from scoped CODEX_HOME scans, managed accounts, and the Spend Dashboard. The lane/scope/account-switch behavior from steipete#4574 carries over unchanged.
|
🦞👀 Pull request received. I will update this pull request when review starts. ClawSweeper review completeClawSweeper finished reviewing this revision. The review result is being finalized. |
Codex Review SummaryThis comment shows the latest Codex review activity on this pull request.
ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings. |
|
Codex review: needs real behavior proof before merge. What this changesAdds a default-off setting that merges OpenAI-provider OpenCode history into ambient Codex history using a new persistent cache. Example: OpenCode records 150 tokens and $0 for an OpenAI assistant turn today.
Review scores
ProductKind: Feature · Worth it: Yes · Fix scope: Partial Merge readiness⛔ Blocked before merge - 13 items remain Keep this PR open for revision: the maintainer welcomed this feature, but the implementation still lacks bounded ingestion and introduces an unrelated Claude account-isolation regression. Priority: P2 Before merge
Findings
Provenance
Tests
Agent review detailsHow this fits togetherThe local cost pipeline reads provider history, caches decoded usage, and publishes daily totals to app menus, charts, and CLI consumers; account-owned dashboard scans must exclude ambient OpenCode records. flowchart LR
A[OpenCode SQLite history] --> B[OpenCode history store]
B --> C[Persistent decoded cache]
C --> D[Codex cost fetcher]
E[Native Codex history] --> D
D --> F[Ambient menus and charts]
D --> G[CLI consumers]
Technical reviewBest possible solution: Integrate bounded, source-scoped incremental ingestion with the existing cost-store owners and every intended history consumer while preserving account isolation. Do we have a high-confidence way to reproduce the issue? Source inspection establishes the caller omissions, discarded Claude options, and cache defects; no target code was executed under the read-only review constraint. Is this the best way to solve the issue? The ambient-only boundary follows the maintainer's direction, but wrapping a complete decoded history array in SQLite does not satisfy the requested bounded incremental design. Full review comments:
Overall correctness: patch is incorrect AGENTS.md: found and applied where relevant. Codex review notes: model internal, reasoning medium; reviewed against 8de11fffdf1a. Provenance checked
TestingProof path: in-process harness. SecurityNone. EvidenceWhat I checked:
Likely related people:
Root-cause clusterRelationship: Members:
Proposal only: this assessment does not dispatch repair, suppress jobs, mutate sibling items, close, or merge anything. LabelsLabel changes:
Label justifications:
Rating scale6/6 🦀 challenger crab · 5/6 🦞 diamond lobster · 4/6 🐚 platinum hermit · 3/6 🦐 gold shrimp · 2/6 🦪 silver shellfish · 1/6 🧂 unranked krab. Overall follows the weaker of proof and patch quality; ✨ marks media proof (a screenshot, video, or linked artifact) that directly shows the changed behavior. WorkflowClawSweeper edits this one comment on every review. Comment Reviewed October 22, 2026, 5:12 AM ET / 09:12 UTC. |
The claudeConfigDirectory restoration prepared account-specific projects roots in `options` but the call still passed a fresh scannerOptions lookup, so a mapped Claude account validated its directory while the scan read ambient history (ClawSweeper P1 on steipete#4702). Pass the prepared options through as main did.
|
Thanks @Danielweng0902 for coming back to this, and the off-by-default setting and account isolation are exactly right. I'm closing this version, though, because the core storage issue from #4574 is still there: the new store keeps all of history in one cached blob and decodes the whole thing on cached publication, and a changed source re-reads and rewrites everything. In a synthetic history with 100k records and one requested day, the store alone peaked around 129 MB on a cached refresh. The shape that fits is an adapter onto the existing incremental history store, so cached publication only touches windowed data and work stays bounded as history grows. If you want to try that, it would be very welcome. |
The claudeConfigDirectory restoration prepared account-specific projects roots in `options` but the call still passed a fresh scannerOptions lookup, so a mapped Claude account validated its directory while the scan read ambient history (ClawSweeper P1 on steipete#4702). Pass the prepared options through as main did.
Replace the blob cache with the incremental-store shape from steipete#4702's close comment: per-table [low, high] rowid windows append new rows only, late step-finish parts re-evaluate their messages, a shrinking raw table count restarts fresh, and entries persist by part/message/session-message source ID so every pass is an idempotent upsert. Publications stay windowed SQL queries over the durable cache. Also wires the catch-up, CLI, chart, and inline hint call sites through the opt-in flag and provenance, and extends clear-cost-cache to the opencode-local-history directory.
|
Thanks for the clear close note — the incremental shape is up in #4744. The store now keeps per-table |
Reimplements #4574 in the shape from its close comment: one shared, fingerprint-gated
OpenCodeLocalHistoryStorereplaces the separate full-load reader.The three close-comment concerns, addressed
opencode-local-history-v1.sqlite, nevercost-usage.sqlite); cached publications callcachedDailyReportwhich reads that durable cache without touching the OpenCode database at allCacheBox— it has its own durable cache, mirroring the OpenCodex usage store's layoutScans re-read the OpenCode database only when its size/mtime/WAL fingerprint changed, so unchanged databases cost one stat per scan.
What carries over from #4574
The #3273 contract is unchanged: opt-in default-off, OpenAI-provider assistant records only (exact
providerID == "openai"), stable-ID v1/v2 deduplication, recorded cost surfaced as-is,sourceRecorded/mixedprovenance with the labelled list-price hints, and permanent exclusion from scoped CODEX_HOME scans, managed accounts, and the Spend Dashboard. The lane activation, ambient scope selection, and account-switch survival behavior also carry over.Tests
OpenCodeLocalHistoryStoreTests(20): provider selection, step-finish precedence, v1/v2 dedup, conservative day totals, windowing calendars, XDG/OPENCODE_DBcontract, and the new fingerprint gating / cache-only publication / cache removal behaviorsOpenCodeLocalHistoryMergeTests: foreground merge, seeded cached publication, ambient-only gating, provenance windowsOpenCodeLocalHistorySettingsTests: lane activation, ambient scope over managed homes, publication survival across account switchesGatekeeper passes (two suppression anchors follow the renamed lane helper). Local run: 39/39 in the feature suites; the two parallel-load-flaky dashboard timing tests fail identically on the base commit and pass in isolation.
Closes nothing — #3273 stays the tracker. Happy to adjust the store's cache location or fingerprint semantics if the tightened budgets want something different.