Skip to content

fix(trace): stop crediting finished sessions on unrelated commits - #3439

Merged
jiparis merged 4 commits into
chainloop-dev:mainfrom
jiparis:jiparis/pfm-7263-scope-trace-session-attribution
Sep 14, 2026
Merged

jiparis merged 4 commits into
chainloop-dev:mainfrom
jiparis:jiparis/pfm-7263-scope-trace-session-attribution

Conversation

@jiparis

@jiparis jiparis commented Sep 14, 2026 •

Copy link
Copy Markdown
Member

The Chainloop-Trace-Sessions trailer was chosen by intersecting the staged files with every ai-lines ledger on disk, with no scoping in time, branch or session liveness. Those ledgers never expire — WipeTraceDir preserves them and GCOrphans only drops one when no surviving commit references it — so a session pinned itself into the repository after its first commit and kept being stamped onto every later commit that touched any file it had ever edited.

The consequences compounded downstream: a session's evidence file set comes from the diffs of the commits credited to it, so a foreign commit contributed its whole diff, and every file the session had no ranges for was reported as human-written. A single session accumulated commits from unrelated branches and surfaced on pull requests it never touched.

What changed

Attribution is now consumed when it lands:

  • Each ai-lines entry carries a recorded_at, and the post-commit hook appends a consumed_by marker for the files the commit contained.
  • Matching runs against the pending set instead of the full range history. Ranges are retained for push-time enrichment, and per-file consumption keeps credit intact for a session committing its work in chunks.
  • Entries written before recorded_at existed are never pending, so ledgers already on disk stop matching immediately.

Two supporting changes:

  • session-end persists active: false and session-start revives it on resume, which lets a push with no commits to go on skip sessions that have ended. active was previously written once and never read.
  • GCOrphans retires sessions older than a week once everything they produced has been attested, since reachability alone never expires them.

active remains unsuitable as an attribution signal — session-end is not installed for trace run and never fires on SIGKILL or a closed terminal.

Review in cubic

The Chainloop-Trace-Sessions trailer was chosen by intersecting the staged
files with every ai-lines ledger on disk, with no scoping in time, branch or
session liveness. Those ledgers never expire — WipeTraceDir preserves them and
GCOrphans only drops one when no surviving commit references it — so a session
pinned itself into the repository after its first commit and kept being stamped
onto every later commit that touched any file it had ever edited.

The consequences compounded downstream: a session's evidence file set comes from
the diffs of the commits credited to it, so a foreign commit contributed its
whole diff, and every file the session had no ranges for was reported as
human-written. A single session accumulated commits from unrelated branches and
surfaced on pull requests it never touched.

Attribution is now consumed when it lands. Each ai-lines entry carries a
recorded_at, the post-commit hook appends a consumed_by marker for the files the
commit contained, and matching runs against the pending set instead of the full
history. Ranges are retained for push-time enrichment, and per-file consumption
keeps credit intact for a session committing its work in chunks. Entries written
before recorded_at existed are never pending, so ledgers already on disk stop
matching immediately.

Two supporting changes: session-end now persists active: false and session-start
revives it on resume, which lets a push with no commits to go on skip sessions
that have ended; and GCOrphans retires sessions older than a week once
everything they produced has been attested, since reachability alone never
expires them.

Active remains unsuitable as an attribution signal — session-end is not
installed for `trace run` and never fires on SIGKILL or a closed terminal.

Signed-off-by: Jose I. Paris <jiparis@chainloop.dev>
Consuming attribution on commit closed the case where a finished session had
already committed its work, but not the one where it had not. Two sessions in a
row over the same file, neither having committed, both stayed pending, so the
commit carried both IDs — attaching a session that had moved on to that commit
and to the pull request holding it, with the whole commit diff attributed to it.

Each staged file is now owned by the session that edited it last: the earlier
edit no longer describes the file, since the later session rewrote those lines.
Ownership is per file, so a session still keeps credit for the other files it
has pending.

Ordering this needs better than the second resolution NowTimestamp provides —
two sessions editing one file within the same second were indistinguishable, and
the tie-break decided it. Attribution entries now record RFC3339Nano through a
new NowTimestampPrecise, and Pending carries the parsed edit time per file.
Timestamps are compared as times, never as strings: a fractional-second form
does not sort lexicographically against a whole-second one.

The tie-break stays for genuinely equal timestamps, falling to the lower session
ID so the trailer does not follow map iteration order.

Signed-off-by: Jose I. Paris <jiparis@chainloop.dev>
@jiparis
jiparis requested a review from a team September 14, 2026 14:11
@jiparis
jiparis marked this pull request as ready for review September 14, 2026 14:11
@chainloop-platform

chainloop-platform Bot commented Sep 14, 2026 •

Copy link
Copy Markdown
Contributor

Security Checks — ✅ 8 passing

✅ secret-scan

Status Policy Messages
✅ Passed secrets-detection -

✅ sast-scan

Status Policy Messages
✅ Passed owasp-top10-2025 -
✅ Passed sast -
✅ Passed cwe-top25 -
✅ Passed cwe-top26-40-cusp -

PR info

Status Policy Messages
✅ Passed pr-min-approvals -
✅ Passed pr-description-required -
✅ Passed pr-user-story-linked -

⏭️ 3 scans not applied

Scan Reason
vulnerability-scan no manifest/lockfile changed
github-actions-scan no workflow files changed
iac-scan no IaC files changed

View attestation ↗


Powered by Chainloop and Chainloop Trace

@cubic-dev-ai cubic-dev-ai Bot left a comment •

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All reported issues were addressed across 7 files

Reply with feedback, questions, or to request a fix.

Re-trigger cubic

Comment thread app/cli/internal/trace/state/state.go
Comment thread app/cli/internal/trace/state/state_test.go Outdated
Comment thread app/cli/pkg/action/trace_agent_hook.go Outdated
Comment thread app/cli/internal/trace/state/ailines.go
Comment thread app/cli/internal/trace/state/ailines.go Outdated
…test clocks

Only session-start revives an ended session. The revival was in
ensureSessionTracked, which the pre- and post-tool-use hooks also call, so one
of those arriving after session-end resurrected a session that really had
finished and the next no-commit push would attest it.

A consumption marker now names the edit it retires rather than the time it was
written. MarkConsumed reads the pending set and then appends, so a concurrent
RecordLineRanges could land in between and leave the marker behind an edit the
commit never contained; replay then cleared that edit's pending state and the
session lost credit for work still in the working tree. Replay compares the
pending time against the marker's, so a later edit survives. The snapshot is now
an explicit parameter, since it is the boundary this has to be correct across.

Ordering tests no longer race the wall clock. They asserted strict ordering
between two consecutive RecordLineRanges calls, which ties on a host whose clock
advances in coarse ticks; the new RecordLineRangesAt lets them set the times
instead, and the same seam makes the two-session lifecycle test deterministic.

Left as is, with the reasoning recorded in staleSessionIDs: the GC does not
spare a week-old session's uncommitted attribution. Keeping it means a session
nothing has heard from in a week can still claim a file and pull the whole
commit's diff into its evidence, and by then its transcript is usually gone, so
the attestation it wins yields nothing. Attributing that edit to nobody is the
safe direction.

Signed-off-by: Jose I. Paris <jiparis@chainloop.dev>
@jiparis

jiparis commented Sep 14, 2026

Copy link
Copy Markdown
Member Author

Thanks — four of the five were real. Addressed in 79a3764.

Fixed

  • Session revived from any hook (trace_agent_hook.go). Correct: ensureSessionTracked is also called from pre- and post-tool-use, so a late hook resurrected a genuinely ended session. Revival moved to HandleAgentSessionStart only, with a test asserting a post-tool-use hook leaves Active false.
  • MarkConsumed race (ailines.go). Correct, and the sharper failure is that the session loses credit for work still in the working tree. Rather than lock the ledger, each marker now names the edit it retires instead of the time it was written, and replay only clears the file when the pending time is not after the marker's. A concurrent edit's later timestamp keeps it pending. The pending snapshot is an explicit parameter now, since it is the boundary this has to be correct across. Test verified to fail without the change.
  • Clock-dependent ordering tests (state_test.go). Correct — those would tie on a coarse-clock host. Added RecordLineRangesAt so the tests set the times, and used the same seam to make the two-session lifecycle test deterministic; it was racing the wall clock too.
  • Legacy entry after a valid one (ailines.go). Folded into the marker change above: the parse-failure path is now only about markers, and an unreadable marker retires the file outright.

Not changed — deliberate, reasoning now recorded in staleSessionIDs

The GC not sparing a week-old session's uncommitted attribution is the intended trade-off, not an oversight. Keeping it means a session nothing has heard from in a week can still win ownership of a file and pull the whole commit's diff into its evidence — the exact bug this PR exists to fix. By that point its transcript is usually gone (raw/ is wiped on every push), so ParseSession fails and the attestation it wins yields no evidence anyway, leaving a commit record pointing at nothing. Attributing that edit to nobody under-credits the AI; the alternative credits the wrong session. Note also that a session with no commits at all was already dropped by the pre-existing reachability rule, independent of this change.

@cubic-dev-ai cubic-dev-ai Bot left a comment •

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All reported issues were addressed across 6 files (changes from recent commits).

Reply with feedback, questions, or to request a fix.

Re-trigger cubic

Comment thread app/cli/internal/trace/state/ailines.go Outdated
Bounding a consumption marker with the retired edit's timestamp left the race it
was meant to close open at the edges: a racing edit that landed in the same
clock tick compared equal, so the marker swept it up and the session lost credit
for work still in the working tree. Timestamps cannot settle this — that is the
same coarse-clock tie the ordering tests had to stop relying on.

Pending entries now carry their position in the ledger alongside the edit time,
and a marker records the position it retires. Positions come from an append-only
file, so they are unique and strictly increasing no matter what the clock does,
and an edit appended after the snapshot sits past the marker's bound and stays
pending. A marker without a position retires the file outright, matching the
behaviour of the markers written before the field existed.

Ownership between sessions still goes by edit time: positions from different
ledgers are not comparable.

Signed-off-by: Jose I. Paris <jiparis@chainloop.dev>
@jiparis

jiparis commented Sep 14, 2026

Copy link
Copy Markdown
Member Author

Good catch, and valid — fixed in 5f2d38d.

You were right that a timestamp cannot bound the marker: a racing edit landing in the same clock tick compared equal, so the marker swept it up and the session lost credit for work still in the working tree. That is the same coarse-clock tie that forced the ordering tests off the wall clock, so keying retirement to a time was never going to hold.

Rather than add a separate ordering token, the bound is now the entry's position in the ledger. PendingEdit carries {At, Seq}, and a marker records the consumed_seq it retires; replay clears the file only when the pending position is at or below it. The ledger is append-only, so positions are unique and strictly increasing whatever the clock does, and an edit appended after the snapshot sits past the bound and stays pending. A marker with no position retires the file outright, matching the markers written before the field existed.

Ownership between sessions still goes by At — positions from different ledgers are not comparable, so only the time can order two sessions' edits to the same file.

Two tests added, both verified to fail against the timestamp-bounded version: a racing edit that shares the snapshot's exact timestamp survives, and a marker with no position retires the file.

@jiparis
jiparis merged commit 06ec131 into chainloop-dev:main Sep 14, 2026
16 checks passed
@jiparis
jiparis deleted the jiparis/pfm-7263-scope-trace-session-attribution branch September 14, 2026 20:21
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants