Skip to content

feat(audit-trail): record the id a primary key moved from [PRD-1321] - #1946

Open
bexchauveto wants to merge 4 commits into
mainfrom
feature/prd-1321-agent-nodejs-carry-the-id-a-primary-key-move-came-from
Open

bexchauveto wants to merge 4 commits into
mainfrom
feature/prd-1321-agent-nodejs-carry-the-id-a-primary-key-move-came-from

Conversation

@bexchauveto

@bexchauveto bexchauveto commented Sep 30, 2026 •

Copy link
Copy Markdown
Member

Stacked on #1911 (PRD-1257): its base retargets to main once #1911 merges.

Closes PRD-1321. The Ruby twin is agent-ruby#394, already merged.

What was lossy

An update carries two states but is filed under one id. When a primary key is both writable (so the capture records it) and redacted, both sides hold the placeholder, the packed id is the only value left — and it speaks for the side the row was filed under, not the other one.

#1909 chose to withhold rather than answer the previous side from the id the record ended up with. That is correct, and it costs a caller squarely in scope the values they should have seen: /state loses the most, since it rebuilds from a delete row carrying the whole writable column set.

What this does

previous_record_id on the audit row, written on every confirmed update, and withhold.ts judges the previous side against it.

Written whether or not the key moved — this is the one place it departs from agent-ruby#394. That table had never shipped, so a null there can only mean the key held still. This one has: a null has to keep meaning written before this column existed, or a row from an older agent whose key did move would be judged by the id it moved to, which is exactly the leak #1909 closed. For the same reason the column arrives as 002-add-previous-record-id rather than an edit to 001, which deployed databases have already recorded as applied.

Ported from Ruby unchanged:

  • a pending update's new side answers only with what it captured — the row is filed under the id the record had before the write, which says nothing about the state it was moving to;
  • the column never reaches a client. Both response paths strip it.

Tests

  • the previous side of a moved key is now released to a scope that covers the id it came from, and the new side withheld — the case that was lost;
  • a pending update's new side stays withheld;
  • the capture records the previous id on a move and on a no-move alike;
  • the column is absent from the served payload;
  • migration 002 applies, is idempotent, and the schema assertions cover the new column.

442 tests pass across test/audit-trail and test/routes/access.

Two things the existing tests caught while I wrote this, both worth knowing: an absent column read as known until I normalised undefined to null (which released a previous side it should have withheld), and decoding per side rather than per row made a bad id warn twice.

Also in this PR: the timeline's authors

Second commit, here because it needs #1911's timeline route.

The project-level author filter could only offer activity-log authors: GET /forest/_audit-trail answered no meta.availableUsers. It now does, on the first page only (no before), like the per-record route; later pages omit the key rather than send [].

  • New optional store method listTimelineUsers (optional like listTimeline): the distinct authors under the timeline's filters and readable collections, ignoring the cursor. The SQL store shares the MAX()/GROUP BY query with listDistinctUsers. A store without it serves the rows with no availableUsers.
  • Stacking fix: rows from listTimeline now carry previousRecordId too, so the timeline route strips it like the per-record paths do.
  • Rebase conflict: feat(audit-trail): cross-collection timeline route [PRD-1257] #1911 changed scanServedValues to return { page, count, authors } and build the body in its caller. I kept that shape and strip previousRecordId where the caller builds the response.

Front reader: ForestAdmin/forestadmin#10033. Ruby contract added to PRD-1258.

Tests: route (first page announces with the filters and collections, later page omits, store without the method omits, previousRecordId absent) and SQL store (filters and collections honoured, empty collections match nothing). 484 tests pass across test/audit-trail and test/routes/access; lint clean.

🤖 Generated with Claude Code

Note

Record previous primary-key id in audit trail and add timeline route

  • Confirmed update audit rows now store the pre-update packed record id as previousRecordId in instrument.ts, even when the key did not change
  • Adds a nullable previous_record_id column and a (timestamp, id) index via new migrations in migrations.ts, with retry handling for concurrent startup races
  • Adds a cross-collection GET /_audit-trail timeline route in audit-trail-timeline.ts with newest-first paging, shared filters, and author metadata. It is mounted only when the store exposes listTimeline
  • Adds canUseAuditTrailTimeline to the capabilities response in capabilities.ts, set from whether the audit-trail store has listTimeline
  • Permission-scope withholding in withhold.ts now evaluates update snapshots against their pre- and post-update ids
  • Behavioral Change: previousRecordId is internal and stripped from all client responses in both the timeline and history handlers in audit-trail-timeline.ts and audit-trail.ts; existing audit tables get the new column via migration 003

Macroscope summarized 0e91320.

@linear-code

linear-code Bot commented Sep 30, 2026

Copy link
Copy Markdown

PRD-1321

@qltysh

qltysh Bot commented Sep 30, 2026 •

Copy link
Copy Markdown

1 new issue

Tool Category Rule Count
qlty Structure Function with high complexity (count = 16): buildMigrations 1

Comment thread packages/agent/src/audit-trail/migrations.ts Outdated
Comment thread packages/agent/src/audit-trail/sql-store.ts
@qltysh

qltysh Bot commented Sep 30, 2026 •

Copy link
Copy Markdown

Qlty


Coverage Impact

This PR will not change total coverage.

Modified Files with Diff Coverage (5)

RatingFile% DiffUncovered Line #s
Coverage rating: A Coverage rating: A
packages/agent/src/audit-trail/withhold.ts100.0%
Coverage rating: A Coverage rating: A
packages/agent/src/audit-trail/migrations.ts90.9%273
Coverage rating: A Coverage rating: A
packages/agent/src/routes/access/audit-trail-timeline.ts100.0%
Coverage rating: A Coverage rating: A
packages/agent/src/routes/access/audit-trail.ts100.0%
Coverage rating: A Coverage rating: A
packages/agent/src/audit-trail/sql-store.ts100.0%
Total96.3%
🤖 Increase coverage with AI coding...
In the `feature/prd-1321-agent-nodejs-carry-the-id-a-primary-key-move-came-from` branch, add test coverage for this new code:

- `packages/agent/src/audit-trail/migrations.ts` -- Line 273

🚦 See full report on Qlty Cloud »

🛟 Help
  • Diff Coverage: Coverage for added or modified lines of code (excludes deleted files). Learn more.

  • Total Coverage: Coverage for the whole repository, calculated as the sum of all File Coverage. Learn more.

  • File Coverage: Covered Lines divided by Covered Lines plus Missed Lines. (Excludes non-executable lines including blank lines and comments.)

    • Indirect Changes: Changes to File Coverage for files that were not modified in this PR. Learn more.

@bexchauveto
bexchauveto force-pushed the feature/prd-1321-agent-nodejs-carry-the-id-a-primary-key-move-came-from branch from 05cba83 to f068dbc Compare October 7, 2026 13:50
@bexchauveto
bexchauveto changed the base branch from main to feature/prd-1257-agent-nodejs-cross-collection-audit-trail-route-for-the October 7, 2026 13:50
@bexchauveto
bexchauveto added this pull request to stack #1915 October 7, 2026 13:50
@bexchauveto
bexchauveto force-pushed the feature/prd-1321-agent-nodejs-carry-the-id-a-primary-key-move-came-from branch 4 times, most recently from c19f05f to fca32f5 Compare October 9, 2026 07:06
Base automatically changed from feature/prd-1257-agent-nodejs-cross-collection-audit-trail-route-for-the to main October 9, 2026 12:33
@bexchauveto
bexchauveto force-pushed the feature/prd-1321-agent-nodejs-carry-the-id-a-primary-key-move-came-from branch from fca32f5 to a9158cc Compare October 9, 2026 12:33
bexchauveto and others added 4 commits October 9, 2026 14:43
…each side by it

An update carries two states but files under one id, so a writable primary key
that is also redacted left the previous side with nothing to answer a permission
scope: #1909 withheld it rather than judging it by the id the record ended up
with. Correct, and lossy — a caller squarely in scope saw nothing.

The row now carries `previous_record_id`, written on every confirmed update, and
each side is judged against the id it actually had.

Written whether or not the key moved, which is where this departs from
agent-ruby#394: that table had never shipped, so a null there can only mean the
key held still. Here a null has to keep meaning "written before this column
existed", or a row from an older agent whose key did move would be judged by the
id it moved to — the leak #1909 closed. For the same reason the column arrives
as its own migration rather than an edit to 001.

A pending update's new side still answers with what it captured: the row is
filed under the id the record had before the write, which says nothing about the
state it was moving to.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The project-level author filter could only offer activity-log authors:
the cross-collection route answered no availableUsers. It now carries
them on the first page (no before), like the per-record route, through
an optional store method, listTimelineUsers. Authors are not detail
values, so the admin gate leaves them in.

Stacking on the timeline route also exposed previousRecordId on its
rows; it is stripped there as on the per-record paths.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…d, and keep it at insert

Migration 002 checked for the column, then added it. Without Postgres's
advisory lock, a second agent booting at the same moment could add it in
between, and the duplicate-column error failed this agent's startup. A
failed addColumn now re-reads the table: a column that is there means
the work is done, anything else is rethrown.

toRow also keeps a previousRecordId given at insert, as the in-memory
store does, so a confirm that omits it cannot make the row read as older
than the column.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The admin gate is gone, so authors need no exemption from it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@bexchauveto
bexchauveto force-pushed the feature/prd-1321-agent-nodejs-carry-the-id-a-primary-key-move-came-from branch from a9158cc to 0e91320 Compare October 9, 2026 12:43

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant