Make braintrust.json the single source of tracing configuration - #27
Merged
Stephen Belanger (Qard) merged 3 commits intoAug 20, 2026
Merged
Conversation
Every tracing plugin previously mixed persistent config with direct environment-variable reads, and Claude/Codex resolved that conflict differently than OpenCode/Pi. Standardize on one rule across all four agents: braintrust.json is the only source of persistent routing and enablement, and `bt trace run` is the only per-invocation override. - bt-daemon: drop the clap `env = "BRAINTRUST_*"` fallbacks on hook, import, setup, and run arguments, and stop falling back to `TRACE_TO_BRAINTRUST` when a persisted setting is absent. - OpenCode and Pi plugins: stop overlaying BRAINTRUST_PROFILE, BRAINTRUST_ORG_NAME, BRAINTRUST_PROJECT, TRACE_TO_BRAINTRUST, and BRAINTRUST_ADDITIONAL_METADATA onto the loaded config file; only `bt trace run` invocation settings can override it, matching Claude and Codex. - Update docs and examples across all four plugins to point at `bt trace setup`/`bt trace run` instead of environment variables. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
|
You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard. |
The test wrote the global braintrust.json config under $HOME/.config/opencode, but index.ts resolves that directory from XDG_CONFIG_HOME first. The GitHub Actions Linux runner sets that variable, so the plugin looked for config in the wrong place and CI failed while the same test passed locally. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Only the live hook dispatcher should be file-only, since it fires automatically on every event with no user in the loop. `bt trace setup`, `bt trace run`, and `bt trace import` are explicit, user-typed commands, so restore their BRAINTRUST_PROFILE, BRAINTRUST_ORG_NAME, BRAINTRUST_PROJECT, BRAINTRUST_DESTINATION, and BRAINTRUST_ADDITIONAL_METADATA environment-variable fallbacks alongside their flags. - bt-daemon: re-add the clap `env = "BRAINTRUST_*"` attributes to SetupArgs, RunArgs, and ImportArgs. HookArgs keeps none. Split the standalone binary's route args into a hook-only RouteArgs (flags only) and a RunRouteArgs (flags + environment), sharing one `build_route` helper. - Update docs across bt-daemon and the plugin READMEs to describe the corrected split. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Stephen Belanger (Qard)
requested review from
Abhijeet Prasad (AbhiPrasad) and
Andrew Kent (realark)
August 20, 2026 13:19
Abhijeet Prasad (AbhiPrasad)
approved these changes
Aug 20, 2026
Stephen Belanger (Qard)
deleted the
t3code/fix-tracing-plugin-configuration
branch
August 20, 2026 15:13
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Fixes the configuration precedence problem described in Tracing plugin configuration: problems and proposal.
Previously, precedence between
braintrust.jsonand environment variables depended on which agent plugin you used:bt trace hook): a route inbraintrust.jsonmade routing environment variables inert.braintrust.json, the opposite precedence.BRAINTRUST_ADDITIONAL_METADATAwas read straight from the environment regardless of the above.This implements the proposed fix:
braintrust.jsonis now the only source of persistent tracing configuration (routing + enablement) for all four agents, andbt trace runis the only mechanism for a per-invocation override — with no plugin readingBRAINTRUST_PROFILE,BRAINTRUST_ORG_NAME,BRAINTRUST_PROJECT,TRACE_TO_BRAINTRUST, orBRAINTRUST_ADDITIONAL_METADATAdirectly.Changes
env = "BRAINTRUST_*"fallbacks onhook,import,setup, andrunarguments, and removed theTRACE_TO_BRAINTRUSTenvironment fallback inAgentSettings::tracing_enabled.profile,org_name,project,trace_to_braintrust, andadditional_metadatainconfig.ts; only the file andbt trace run'sBT_TRACE_INVOCATION_SETTINGScan set these now. Plugin-local behavior toggles unrelated to routing (BRAINTRUST_DEBUG,BRAINTRUST_OPENCODE_ENABLE_TOOLS,BRAINTRUST_SHOW_UI,BRAINTRUST_SHOW_TRACE_LINK) are unchanged..env.example, andinstall.shacross all four plugins to point atbt trace setup/bt trace runinstead of environment variables for routing and metadata..env_remove("BRAINTRUST_ADDITIONAL_METADATA")in the managed-run launcher and a hook-forwarder test block that encoded the old (now incorrect) env-based metadata contract.Test plan
cargo test --all-featuresinbt-daemon(all passing, including pipeline/replay/translator integration suites)cargo clippy --all-features --all-targetscleanpnpm vitest runfor both theopencodeandpiplugins (all passing, including updated precedence tests)pnpm run check(format + lint + typecheck) clean for both pluginsmake build-claude build-codex+bash scripts/test-hook-forwarders.sh distpassingmake build-opencode build-pi+validate.sh(includes apnpm publish --dry-run) passing for bothagent_integration.rstests (claude_session_emits_traces,codex_session_emits_traces,opencode_session_emits_traces,pi_session_emits_traces) require the actual coding-agent CLIs on PATH and are#[ignore]d in this sandbox — verified by inspection that their test harness already drives routing viabraintrust.json/bt trace runrather than the removed env vars, so behavior is unaffected, but CI should confirm.🤖 Generated with Claude Code