Skip to content

Attribute native Codex tool events - #1767

Open
uttamdeb wants to merge 8 commits into
getagentseal:mainfrom
uttamdeb:fix/codex-native-tool-events
Open

uttamdeb wants to merge 8 commits into
getagentseal:mainfrom
uttamdeb:fix/codex-native-tool-events

Conversation

@uttamdeb

Copy link
Copy Markdown
Contributor

Summary

  • Attribute native Codex web_search_call, tool_search_call, and image_generation_call items as WebSearch, ToolSearch, and ImageGeneration. Also recognize completed WebSearch items and the observed Extension / kind: "web.search" format.
  • Deduplicate stable item identities across status updates, mirrors, append checkpoints, and fork replay. A readable parent's exact native-item identities suppress copied activity while preserving a fork's new activity before the usage replay anchor.
  • Refresh stale Codex/session/daily attribution caches, preserving the existing usage accounting and provider migration thresholds. Daily cache v76 includes a regression upgrading a literal shipped v75 cache.

Real-session reproduction

I generated one native search using installed codex-cli 0.162.0-alpha.17.2, existing ChatGPT authentication, gpt-6-luna with max reasoning, and a read-only sandbox. The CLI emitted one web_search start/completion pair. Its stable identity matches the persisted event_msg/item_completed item with type: "Extension", kind: "web.search", and action.type: "search".

Parsing that same unmodified session with parseCodexFileFull on clean upstream 0322472d and this branch 4ed13312 gives:

Recorded/parsed value Main This branch
Sessions 1 1
Usage calls 2 2
WebSearch tool activity 0 1
Existing Bash activity 1 1
Input excluding cached input 11,374 11,374
Cache-read input 32,128 32,128
Output 204 204
Cache-write input 0 0
Reasoning output 87 87
CodeBurn-calculated cost $0.00156068 $0.00156068
webSearchRequests 0 0

The normalized total input is 43,502, matching the CLI's recorded usage. Assertions confirm unchanged accounting, no duplicate call keys, exactly one recovered WebSearch, and the stable-identity match. No provider billing amount was recorded; the cost above is CodeBurn's token-price calculation. Native activity does not establish an additional billable search request.

This live check covers the observed Extension transport. The other native response forms are covered by controlled fixtures and Codex's protocol model. The full local session stays outside Git; this description contains no prompt, query, result, argument, credential, raw session ID, or source path.

Regression coverage

  • Stable/missing identities, mirrored updates, trailing activity, and full/append/resume equivalence.
  • A canonical parent/fork fixture reproduces new native activity being lost inside the replay burst. Its regression failed with an empty tool list before the fix and now passes.
  • A large-record fixture reproduces nested metadata.payload being mistaken for ImageGeneration. Its regression also failed before the fix and now passes.
  • Bounded direct-field decoding through oversized results, leading whitespace, unknown extensions, and parent image/search records.
  • Worker/sequential parity with exact parent-native masking and a single parent-index request per fork.
  • Warm cache refresh and finalized daily categories after a literal v75 upgrade, with accounting and the original v75 file retained.

Testing / Verification

  • Tested locally against the real session described above.
  • All 118 focused tests across Codex, resume, cache invalidation, daily rederivation, and worker suites passed using the current main dependency installation.
  • tsc --noEmit, npm run build, and git diff --check passed.
  • Independent read-only agent review of final head 4ed13312 completed with no unresolved findings; its concrete findings were fixed and covered by regressions.
  • Full npm test runs on unchanged main and final branch completed and were compared: zero new failing cases. Initial shared failure and serial reruns are disclosed below.
  • Not a UI change or a new provider.

Full-suite comparison

Windows, Node 24.9.0, npm 11.6.0, clean committed trees, identical lockfiles and installed dependency versions:

Checkout Passed Failed Skipped
Main 0322472d 5,188 1 30
Branch 4ed13312 5,199 1 30

Each run covered 389 test files: 386 passed, one failed, two skipped. The branch adds 11 test cases and introduces zero new failing cases.

The sole failure on both checkouts was tests/cli-budget.test.ts, keeps overview budget lines only on unfiltered overviews: the original 30-second timeout, at 32.2s on main and 31.4s on this branch. After both full runs finished, I reran only that case on each checkout, one after the other, with the original timeout unchanged. Main passed in 24.1s and this branch passed in 24.0s (one passed / six unselected cases skipped per rerun). No assertion or timeout was weakened. The initial full-run failures are retained and are not described as all-green full runs.

Both full runs used the repository's normal test scope, concurrently with one worker each:

npm test -- --configLoader runner --maxWorkers 1 --reporter=default --reporter=json --outputFile.json=<report>

Serial rerun selection:

node node_modules/vitest/vitest.mjs run tests/cli-budget.test.ts --configLoader runner --maxWorkers 1 -t 'keeps overview budget lines only on unfiltered overviews'

The four serial refresh-lock suites remain excluded by the normal npm test script. No Electron or native-app code is changed.

@uttamdeb
uttamdeb requested a review from iamtoruk as a code owner October 18, 2026 06:45
@ozymandiashh ozymandiashh added area: cli The codeburn CLI and its core parsing/reporting engine area: docs README and docs/ content enhancement New feature or request provider: codex Codex CLI provider validated Validated against real-world usage, ready for review labels Oct 21, 2026
@uttamdeb
uttamdeb force-pushed the fix/codex-native-tool-events branch from 4ed1331 to b2c3c36 Compare October 21, 2026 08:16
@uttamdeb

Copy link
Copy Markdown
Contributor Author

@iamtoruk — I have rebased this PR onto the latest main and pushed the update. The five Codex-focused suites pass (118 tests). When you have a chance, could you review the native Codex tool attribution changes? Thanks.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area: cli The codeburn CLI and its core parsing/reporting engine area: docs README and docs/ content enhancement New feature or request provider: codex Codex CLI provider validated Validated against real-world usage, ready for review

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants