Skip to content

fix(trace): replace spec copies in Claude Code request blobs - #3656

Merged
jiparis merged 1 commit into
chainloop-dev:mainfrom
jiparis:fix/trace-claude-api-request-blob
Oct 19, 2026
Merged

jiparis merged 1 commit into
chainloop-dev:mainfrom
jiparis:fix/trace-claude-api-request-blob

Conversation

@jiparis

@jiparis jiparis commented Oct 19, 2026 •

Copy link
Copy Markdown
Member

Fixes #3654.

Claude Code writes api-request-blob records in its transcript. Each record repeats the blocks of one message of an API request: the prompt, the tool calls, and the tool results. The spec copy finders of the Claude Code provider did not handle these records:

  • The copies of pasted images and spec sources in the records stayed inline, so the evidence held them a second time.
  • A record repeats the tool call ID of its entry. When a record came after its entry, the finder used the record in place of the entry, so the entry kept its copy.

The finders now skip the request blobs when they match calls and results. Then each blob block gets the pointer of the entry with the same tool call ID, when it holds the same value. An image in a blob gets a pointer when its data is a spec source. The Trace README notes that pointers also cover these records.

The end-to-end suite found the problem (spec issue-3635, #3655). With this fix, its "Pasted image pointers" check passes for Claude Code, and "Spec source pointers" passes in 3 of 3 runs (it was 2 of 3).

AI disclosure: Claude Code helped to write this change.

View guided diff

Claude Code writes api-request-blob records that repeat the blocks of a
request: prompts, tool calls and tool results. The spec copy finders
left the copies of pasted images and spec sources in these records
inline. When a blob came after its entry, it also took the place of the
entry, so the entry kept its copy.

Skip the request blobs when the finders match calls and results, and
then give each blob block the pointer of the entry with the same tool
call ID when it holds the same value. An image in a blob gets a pointer
when its data is a spec source.

Fixes chainloop-dev#3654

Assisted-by: Claude Code
Signed-off-by: Jose I. Paris <jiparis@chainloop.dev>

Chainloop-Trace-Sessions: be5af72c-30e7-48f4-b46c-dc7a5a754c41
@chainloop-platform

chainloop-platform Bot commented Oct 19, 2026 •

Copy link
Copy Markdown
Contributor

PR validation — ✅ 3 passing

Status Policy Material Messages
✅ Passed pr-min-approvals pr-info -
✅ Passed pr-description-required pr-info -
✅ Passed pr-user-story-linked pr-info -

View attestation ↗

AI Session Checks — 🟢 89% · ⚠️ 2 failing

Avg score Sessions Failing policies Attribution Files Lines Total Duration
🟢 89% 1 ⚠️ 2 100% AI / 0% Human 11 +623 / -22 71h53m29s

🟢 89% — 100% AI — ⚠️ 2 policies failing

Oct 16, 2026 16:40 UTC · 71h53m29s · $47.01 · 974 in / 506.8k out · claude-code 2.1.301 (claude-opus-5-5)

View session details ↗

Change Summary

  • Adds Claude api-request-blob replacement logic so pasted-image and spec-source copies reuse stored pointers.
  • Updates spec-copy matching to ignore blob lines during entry/result matching and preserve non-identical blob content.
  • Adds focused request-blob tests and updates the Trace README row for spec source pointers.

AI Session Overall Score

🟢 89% — Focused fix, verified well, but the user never explicitly confirmed the result.

AI Session Analysis Breakdown

🟢 95% · scope-discipline

No notes.

🟢 93% · solution-quality

🟢 Root-cause fix skips blob records during matching and only reuses pointers for identical JSON. · High Impact

🟢 92% · alignment

No notes.

🟢 92% · context-and-planning

🟢 The opening scope narrowed the large spec to local-only, happy-path, Claude-first work. · High Impact

🟢 88% · user-trust-signal

No notes.

🟡 78% · verification

🟢 New tests cover copied reads, writes, pasted images, and divergent blob content. · High Impact

🟠 No explicit user confirmation of the repaired behavior appeared before the session pivoted to SAST follow-up. · Medium Severity

💡 Before switching tasks, ask for or show one concrete confirmation of the repaired behavior.

🟡 Nightly, Slack, and multi-provider acceptance checks from the original spec are still parked for later milestones. · Low Severity


File Attribution

████████████████████ 100% AI / 0% Human

Status Attribution File Lines
created ai app/cli/internal/trace/e2e/checks_more.go +185 / -0
modified ai app/cli/internal/trace/e2e/claude.go +121 / -2
modified ai app/cli/internal/trace/e2e/images.go +78 / -2
modified ai app/cli/internal/trace/e2e/scenarios.go +80 / -0
created ai app/cli/internal/trace/e2e/images_test.go +47 / -0
modified ai app/cli/internal/trace/e2e/workspace.go +36 / -7
modified ai app/cli/internal/trace/e2e/checks.go +25 / -1
modified ai app/cli/internal/trace/e2e/harness.go +25 / -0
modified ai app/cli/internal/trace/e2e/runner.go +18 / -5
modified ai app/cli/internal/trace/README.md +6 / -5
modified ai app/cli/internal/trace/e2e/runner_test.go +2 / -0

Policies (5, 2 failing)

Status Policy Material Messages
⚠️ Failed ai-session-no-auto-pr ai-coding-session-be5af7
  • Pull request opened without human approval (auto-approved: source=config, reason=classifier): git push -q -u origin feat/trace-e2e-claude-remaining 2>&1 | grep -iE "error|rejected"; gh pr create --repo chainloop-dev/chainloop --base main --head jiparis:f...
  • Pull request opened without human approval (auto-approved: source=config, reason=subcommandResults): gh api -X PUT repos/chainloop-dev/chainloop/pulls/3648/merge -f merge_method=squash -f commit_title="feat(trace): end-to-end test matrix for Claude Code (feat(trace): end-to-end test matrix for Claude Code #3648)...
  • Pull request opened without human approval (auto-approved: source=config, reason=subcommandResults): gh api repos/chainloop-dev/chainloop/pulls/3648/comments/4311478018/replies -f body="Fixed in b7504e5. The section now says that the suite runs the Claude Code ...
  • Pull request opened without human approval (auto-approved: source=config, reason=subcommandResults): gh api repos/chainloop-dev/chainloop/pulls/3651/comments/4313920106/replies -F body=@/private/tmp/claude-501/-Users-jiparis-projects-chainloop/be5af72c-30e7-48f...
  • Pull request opened without human approval (auto-approved: source=config, reason=subcommandResults): gh api repos/chainloop-dev/chainloop/pulls/3651/comments/4313920123/replies -F body=@/private/tmp/claude-501/-Users-jiparis-projects-chainloop/be5af72c-30e7-48f...
  • Pull request opened without human approval (auto-approved: source=config, reason=subcommandResults): gh api repos/chainloop-dev/chainloop/pulls/3651/comments/4313920134/replies -F body=@/private/tmp/claude-501/-Users-jiparis-projects-chainloop/be5af72c-30e7-48f...
  • Pull request opened without human approval (auto-approved: source=config, reason=subcommandResults): gh api repos/chainloop-dev/chainloop/pulls/3651/comments/4313920140/replies -F body=@/private/tmp/claude-501/-Users-jiparis-projects-chainloop/be5af72c-30e7-48f...
  • Pull request opened without human approval (auto-approved: source=config, reason=subcommandResults): gh pr create --repo chainloop-dev/chainloop --base main --head jiparis:feat/trace-e2e-claude-gaps --title "feat(trace): review feature table and add e2e checks ...
  • Pull request opened without human approval (auto-approved: source=config, reason=subcommandResults): gh pr create --repo chainloop-dev/chainloop --base main --head jiparis:feat/trace-e2e-matrix --title "feat(trace): end-to-end test matrix for Claude Code" --bod...
  • Pull request opened without human approval (auto-approved: source=config, reason=subcommandResults): gh pr create --repo chainloop-dev/chainloop --base main --head jiparis:fix/trace-claude-api-request-blob --title "fix(trace): replace spec copies in Claude Code...
✅ Passed ai-config-ai-agents-allowed ai-coding-session-be5af7 -
✅ Passed ai-config-no-dangerous-commands ai-coding-session-be5af7 -
⚠️ Failed ai-config-no-secrets ai-coding-session-be5af7
  • Secret (github-pat) detected in session content [turn=1343, source=tool_result, line=9]: | ⚠️ Failed | ai-config-no-secrets | [ai-coding-session-be5af7](https://app.chainloop.dev/u/chainloop/projects/chainloop/versi...
  • Secret (github-pat) detected in session content [turn=1386, source=assistant-text, line=10]: - AI session policy ai-config-no-secrets: it flags the fake tokens in this session's transcript. Most of the matches are already the [CHAINLOOP_TRACE_REDACTED:github-pat] placeholder, so this ...
  • Secret (github-pat) detected in session content [turn=1412, source=tool_result, line=6]: 33 fakeSecret = "[CHAINLOOP_TRACE_REDACTED:github-pat]" //nolint:gosec // a fake token that the suite expects redaction to remove
  • Secret (github-pat) detected in session content [turn=1412, source=tool_result, line=9]: 36 fakeAllowedSecret = "[CHAINLOOP_TRACE_REDACTED:github-pat]" //nolint:gosec // a fake token on a line with an allow marker
  • Secret (github-pat) detected in session content [turn=1424, source=assistant-tool_use:Edit, line=1]: {"file_path":"/Users/jiparis/projects/chainloop/.claude/worktrees/cosmic-enchanting-flute/app/cli/internal/trace/e2e/scenarios.go","new_string":"// secret scanner, and they are not real tokens. The al...
  • Secret (github-pat) detected in session content [turn=525, source=assistant-tool_use:Write, line=1]: {"content":"//\n// Copyright 2026 The Chainloop Authors.\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\...
  • Secret (github-pat) detected in session content [turn=672, source=tool_result, line=1]: deploy token [CHAINLOOP_TRACE_REDACTED:github-pat]
  • Secret (github-pat) detected in session content [turn=971, source=tool_result, line=19]: fakeSecret = "[CHAINLOOP_TRACE_REDACTED:github-pat]"
  • Secret (github-pat) detected in session content [turn=971, source=tool_result, line=22]: fakeAllowedSecret = "[CHAINLOOP_TRACE_REDACTED:github-pat]"
  • Secret (github-pat) detected in session content [turn=971, source=tool_result, line=48]: fakeSecret = "[CHAINLOOP_TRACE_REDACTED:github-pat]"
  • Secret (github-pat) detected in session content [turn=971, source=tool_result, line=51]: fakeAllowedSecret = "[CHAINLOOP_TRACE_REDACTED:github-pat]"
  • Secret (github-pat) detected in session content [turn=999, source=assistant-tool_use:Bash, line=42]: s=s.replace('fakeSecret = "[CHAINLOOP_TRACE_REDACTED:github-pat]"','fakeSecret = "[CHAINLOOP_TRACE_REDACTED:github-pat]" //nolint:gosec // a fake token that the suite expects redaction to remove')
  • Secret (github-pat) detected in session content [turn=999, source=assistant-tool_use:Bash, line=43]: s=s.replace('fakeAllowedSecret = "[CHAINLOOP_TRACE_REDACTED:github-pat]"','fakeAllowedSecret = "[CHAINLOOP_TRACE_REDACTED:github-pat]" //nolint:gosec // a fake token on a line with an allow marker')
✅ Passed ai-config-mcp-servers-allowed ai-coding-session-be5af7 -

Security Checks — ✅ 5 passing

✅ secret-scan

Status Policy Messages
✅ Passed secrets-detection -

✅ sast-scan

Status Policy Messages
✅ Passed owasp-top10-2025 -
✅ Passed sast -
✅ Passed cwe-top25 -
✅ Passed cwe-top26-40-cusp -
Scans not applied (3)
Scan Reason
vulnerability-scan no manifest/lockfile changed
github-actions-scan no workflow files changed
iac-scan no IaC files changed

View attestation ↗

Security context

✅ Nothing this change touches has a recorded security-fix history.

View in Chainloop ↗ · How this works ↗


Powered by Chainloop and Chainloop Trace

@jiparis
jiparis requested a review from a team October 19, 2026 16:01

@cubic-dev-ai cubic-dev-ai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No issues found across 4 files

View guided diff | Re-trigger cubic

@jiparis
jiparis merged commit 9766b50 into chainloop-dev:main Oct 19, 2026
16 of 17 checks passed
@jiparis
jiparis deleted the fix/trace-claude-api-request-blob branch October 19, 2026 17:08
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

fix(trace): pasted images and spec copies stay inline in Claude Code api-request-blob entries

2 participants