Skip to content

fix(codex): read Zstandard-compressed rollouts - #1786

Open
iamtoruk wants to merge 1 commit into
mainfrom
fix/codex-zstd-rollouts
Open

iamtoruk wants to merge 1 commit into
mainfrom
fix/codex-zstd-rollouts

Conversation

@iamtoruk

Copy link
Copy Markdown
Member

Summary

  • Codex now moves cold rollouts to rollout-*.jsonl.zst, which CodeBurn skipped, so most Codex history disappeared from reports. Every Codex reader (discovery, parser, fork lookup, codex-tps, context view) now decodes them.
  • A session present in both forms counts once, from the plain file. Compressed files are parsed whole and never resumed by byte offset.
  • On Node below 22.15 (no node:zlib zstd), compressed rollouts are skipped with one notice. Results there match current main.
  • DAILY_CACHE_VERSION goes to 76 so settled days pick up the recovered usage.

Testing / Verification

  • I have tested this locally against real data (not just unit tests)
  • npm test passes
  • npm run build succeeds
  • Not a UI change

Real ~/.codex with 1,436 compressed and 6 plain rollouts, 2026-01-01 to 2026-10-21:

Sessions Calls Cost
main 3 54 $0.16
this branch 1,358 51,564 $4,842.39
  • The same files decompressed and read by main give an identical export in every section.
  • With zstd hidden from node:zlib, the export matches main exactly and prints the notice once.
  • A daily cache written after compression goes from $0.16 to $4,842.38 on the first run. A cache written before compression keeps its totals, with 0 days changed.

Codex now moves cold rollouts to rollout-*.jsonl.zst. Discovery, the
first-line probe, the parser, fork parent lookup, codex-tps and the context
view read them through a zstd decode; a session present in both forms
counts once from the plain file. Compressed files are parsed whole and never
resumed by byte offset. Without node:zlib zstd (Node < 22.15) they are
skipped with one notice. DAILY_CACHE_VERSION 76 so settled days pick up the
recovered Codex usage.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant