From bbd0c4bda22c27ca13e96c5e354292596aa7aca0 Mon Sep 17 00:00:00 2001 From: David Abram Date: Thu, 27 Aug 2026 13:09:09 +0200 Subject: [PATCH 01/15] context: Add mutation cursor store persistence plan Define the next build-out for durable mutation-cursor protocol state in the repository-scoped Agent Trace DB. The plan covers additive schema migration, explicit codecs, structural durable-transition projection, transactional CAS persistence, concurrency and restart coverage, and documentation. Plan: mutation-cursor-store-persistence (T01-T11) Co-authored-by: SCE --- .../mutation-cursor-store-persistence.md | 212 ++++++++++++++++++ 1 file changed, 212 insertions(+) create mode 100644 context/plans/mutation-cursor-store-persistence.md diff --git a/context/plans/mutation-cursor-store-persistence.md b/context/plans/mutation-cursor-store-persistence.md new file mode 100644 index 00000000..b2a744d1 --- /dev/null +++ b/context/plans/mutation-cursor-store-persistence.md @@ -0,0 +1,212 @@ +# Plan: mutation-cursor-store-persistence + +## Change summary + +Adds a durable persistence layer for the verified mutation-cursor protocol +(`cli/src/services/mutation_trace/`), storing the protocol's worktree/scope/ +processed-event/mutation-event state in the repository-scoped Agent Trace DB +(`RepositoryAgentTraceDb`) via a new additive migration and a new `store.rs` +module. This is the third build-out step for the module, following the pure +kernel (`mutation-cursor-protocol-kernel`) and its Quint Connect verification +harness (`mutation-cursor-quint-connect`); it extends that work rather than +replacing it, and `protocol.rs` remains exactly as pure as those two plans +left it. + +The persistence boundary is one-directional and structural: +`protocol.rs` (pure semantics) -> `DurableTransition` (a persistence +projection built by pure structural diffing, not protocol interpretation) -> +`store.rs` (SQL translation) -> `RepositoryAgentTraceDb`. `protocol.rs` never +depends on SQL or the DB adapter, and `store.rs` never branches on protocol +meaning (boundary kind, contention, taint) — it only diffs before/after +`ProtocolState` values. + +Two things are deliberately excluded from the database: `AttemptState` +(explicitly transient in the domain model — no `mutation_trace_attempts` +table) and `external_taint` (a `database_failure()` cannot use the database +it just failed against as the authoritative record that the write was +uncertain; a later plan represents it as a filesystem write-ahead marker). + +The runtime read path is split in two. The hot path (`load_worktree`) loads +one worktree, only its currently `Active` scopes, an optionally referenced +scope even when that scope is terminal (`NeverSeen`/`Closed`/`Abandoned`), +and an optional `EventKey` replay row — never historical +`mutation_trace_events` rows and never a terminal scope it was not +explicitly asked for, so the read stays bounded as closed/abandoned scopes +accumulate over time. A separate cold path (`load_mutation_event`) +reconstructs one historical `MutationEvent` by `(worktree, revision)` only on +explicit request. `DurableTransition::between` is a strict structural +firewall: it validates shape (single worktree, no unrelated changes, +revision advances by exactly one when a transition exists, at most one new +processed event, at most one new mutation event) and rejects a structurally +impossible before/after pair, without ever interpreting protocol semantics. +The CAS primitive keeps three outcomes distinct: a stale revision is a +`Conflict` the DB primitive never retries, a transient DB failure retries +the whole transaction, and a deterministic SQL/constraint failure returns an +error without retry. + +## Acceptance criteria + +- [ ] AC1: Mutation state lives in the repository-scoped `agent-trace.db`. + - Validate: `cli/src/services/mutation_trace/store.rs` reads/writes only through `RepositoryAgentTraceDb`; round-trip tests in T09 pass. +- [ ] AC2: New storage is introduced through additive migration `003`, with `001`/`002` byte-unchanged by this PR. + - Validate: `git diff --exit-code ...HEAD -- cli/migrations/agent-trace-repository/001_repository_schema.sql cli/migrations/agent-trace-repository/002_repository_source_instance_id.sql` (compared against this PR's base branch/merge base, not the working tree) exits `0`; `003_mutation_trace_protocol.sql` exists. +- [ ] AC3: Revision preserves all `u64` values exactly, including `u64::MAX`. + - Validate: `./scripts/run-cli-cargo.sh test --manifest-path cli/Cargo.toml mutation_trace::store::` (revision codec round-trip test covering `0`, `1`, `i64::MAX`, `i64::MAX + 1`, `u64::MAX`). +- [ ] AC4: Worktree/scope/`EventKey`/`MutationEvent` data round-trip exactly, including full `MutationEvent` decoding (`Attribution`, `Boundary`, `active_scopes`) after the DB is closed and reopened. + - Validate: T09's real-protocol round-trip tests, including the `load_mutation_event` cold-reload assertions for every transition that emits a `MutationEvent`. +- [ ] AC5: `AttemptState` is never persisted. + - Validate: `grep -n mutation_trace_attempts cli/migrations/agent-trace-repository/003_mutation_trace_protocol.sql` finds nothing; `DurableTransition` has no `AttemptState` field. +- [ ] AC6: `external_taint` is never treated as DB-authoritative durable state. + - Validate: `grep -n external_taint cli/migrations/agent-trace-repository/003_mutation_trace_protocol.sql` finds nothing; `database_failure` produces no `DurableTransition` (T05 test). +- [ ] AC7: No persistence code determines protocol semantics or attribution. + - Validate: `DurableTransition::between` contains no boundary-kind/contention/taint conditionals (T05 done-when); inspection of `store.rs`. +- [ ] AC8: Every durable protocol transition is one `BEGIN IMMEDIATE` transaction. + - Validate: `store.commit` routes exclusively through `execute_transactional_cas_batch` (T06/T07); T08 atomic-rollback test. +- [ ] AC9: CAS is guarded by the expected worktree revision. + - Validate: the guard statement is `UPDATE mutation_trace_worktrees ... WHERE worktree_id = ? AND revision = ?` (T06); T08 two-writer test. +- [ ] AC10: Two writers from one revision cannot both commit. + - Validate: T08's concurrent-writers test — two independent `RepositoryAgentTraceDb` handles/connections against the same physical database, committing concurrently from the same loaded revision — asserts exactly one `Applied` and one `Conflict`. +- [ ] AC11: Partial failure rolls back all worktree/scope/event changes. + - Validate: T08 injected-failure test asserts revision, scope status, processed event, mutation event, and active scopes are all unchanged after rollback. +- [ ] AC12: Process restart reconstructs the same durable protocol projection. + - Validate: T09 tests that drop and reopen the DB handle before reloading. +- [ ] AC13: Historical mutation events are not loaded on each boundary, terminal (`Closed`/`Abandoned`/`NeverSeen`) historical scopes are not loaded on each boundary unless explicitly referenced, and a referenced scope belonging to a different worktree is rejected rather than silently loaded or reassigned. + - Validate: `MutationTraceStore::load_worktree` issues no query against `mutation_trace_events`, loads only currently `Active` scopes plus the explicitly referenced scope (if any), and returns `Err` when that referenced scope's persisted `worktree_id` does not match the requested worktree (T03 done-when). +- [ ] AC14: Existing Quint Connect and protocol tests remain green. + - Validate: `nix flake check` (runs `cli-tests`, including `mutation_trace::mbt`, and the dedicated `mutation-trace-quint-connect` check). +- [ ] AC15: No Git/filesystem lock/hook/coordinator integration is added. + - Validate: no `coordinator.rs` or `git_snapshot.rs` file is created; `grep -RnE "std::(fs|process)|tokio::(fs|process)" cli/src/services/mutation_trace/` shows no non-test production usage. + +### Full validation + +- `nix flake check` +- `nix run .#pkl-check-generated` (lightweight post-task hygiene baseline; unaffected by this Rust-only change) + +### Context sync + +- `context/cli/mutation-trace-store.md` (new — authored by T11) +- `context/context-map.md` (add the new domain-file entry) +- `context/cli/mutation-trace-protocol.md` ("Target end-state architecture" section: `store.rs` now exists as a real database call site, while `coordinator.rs`/`git_snapshot.rs` remain future work) +- `context/overview.md` (the sentence stating the module "is not yet wired into any hook, command, or database call site" needs to reflect that a database call site now exists) +- `context/sce/shared-turso-db.md` (new generic `execute_transactional_cas_batch` primitive added to `TursoDb`, alongside the existing `execute_transactional_insert_pair_if_absent`, including its CAS-conflict/retryable-failure/deterministic-failure distinction) + +## Task context synchronization lifecycle + +Persist this field in every plan; this is durable plan state, not chat state: + +- **Task context synchronization:** every task carries `pending | synced | blocked`. + A completed task must be `synced` before another task can start or the plan can + finish. +- For `blocked`, record **Blocker**, **Required action**, and **Retry condition** + beside the status. Never infer `synced` from conversation history; write every + lifecycle transition to the plan file. + +## Constraints and non-goals + +- **In scope:** `cli/migrations/agent-trace-repository/003_mutation_trace_protocol.sql` (new); `cli/src/services/mutation_trace/store.rs` (new); a new generic `TransactionStatement`/`execute_transactional_cas_batch` primitive on `TursoDb` in `cli/src/services/db/mod.rs`; tests within `mutation_trace` and `db`/`agent_trace_db`; `context/cli/mutation-trace-store.md` (new). +- **Out of scope:** Git snapshots, `GIT_INDEX_FILE`, Git object storage; the filesystem worktree lock and external-taint marker; `coordinator.rs`; real hook events and Claude/Codex/OpenCode/Pi wiring; Agent Trace diff generation; a retry-after-CAS-conflict loop; changes to Quint semantics or `protocol.rs` semantics; scope garbage collection or any deletion of terminal (`Closed`/`Abandoned`) scope rows. +- **Constraints:** `protocol.rs` stays free of SQL/DB/`RepositoryAgentTraceDb` dependencies; `DurableTransition::between` performs structural diffing only, never protocol interpretation, and rejects a structurally malformed before/after pair rather than silently accepting it; revision is stored as an 8-byte big-endian `BLOB`, enforced by `CHECK (typeof(revision) = 'blob' AND length(revision) = 8)` on every column that stores one; every durable transition commits inside exactly one `BEGIN IMMEDIATE` transaction guarded by the expected worktree revision, with a normal CAS conflict (`Ok(false)`) and a deterministic SQL/constraint failure both left unretried by the CAS primitive, while only a genuinely transient DB failure retries the whole transaction; the hot-path worktree read loads only `Active` scopes plus an explicitly referenced scope, never the full historical scope set and never `mutation_trace_events`; enum codecs are explicit (no `Debug`/serde-derived DB representation). +- **Non-goal:** do not modify `REQUIRED_REPOSITORY_SCHEMA_TABLES`'s baseline-repair logic to treat `003` as part of `001`'s metadata-repair case; do not replace or refactor the existing `execute_transactional_insert_pair_if_absent` primitive — the new generic CAS batch primitive is additive alongside it; do not change `resilience.rs`'s retry-on-any-`Err` behavior or any other caller of `run_with_retry_sync` to add this classification. + +## Assumptions + +- Plan slug (`mutation-cursor-store-persistence`) continues the `mutation-cursor-*` naming already used by `mutation-cursor-protocol-kernel` and `mutation-cursor-quint-connect`. +- File-backed DB round-trip tests (T09, T10) reuse the existing `std::env::temp_dir()`-based unique-path helper pattern already established in `cli/src/services/agent_trace_db/repository.rs`'s tests, rather than adding a `tempfile`-style dependency. +- T06's new CAS batch primitive coexists with the existing `execute_transactional_insert_pair_if_absent`; no other call site is migrated to it in this plan's scope. +- Updating the outdated "not yet wired into any hook, command, or database call site" framing in `context/cli/mutation-trace-protocol.md` and `context/overview.md` is handled by task context synchronization, not by a plan task, since it is a root/shared-file update rather than new content this plan's tasks author. +- AC2's validation compares the two untouched migration files against this PR's base branch (currently `quint-connect` for PR #241) or its merge base, not a hardcoded commit SHA, so the check stays correct as the branch advances. +- T06's retryable-vs-deterministic classification is implemented locally to `execute_transactional_cas_batch` — for example, by having its retried closure return a classified outcome that `run_with_retry_sync` still sees as `Ok` (so it never retries a deterministic failure), with the caller re-raising that failure as an `Err` after the closure returns — rather than by changing `resilience.rs` itself. + +## Task stack + +- [ ] T01: `Add migration 003 for mutation-trace protocol tables` (status:todo) + - Task ID: T01 + - Scope: In — `cli/migrations/agent-trace-repository/003_mutation_trace_protocol.sql` defining `mutation_trace_worktrees` (revision `BLOB` constrained by `CHECK (typeof(revision) = 'blob' AND length(revision) = 8)`), `mutation_trace_scopes` (+ `idx_mutation_trace_scopes_worktree` and a new composite `idx_mutation_trace_scopes_worktree_status` index on `(worktree_id, status)` for the bounded hot-path scope lookup), `mutation_trace_processed_events`, `mutation_trace_events` (+ the same `typeof`/`length` revision `CHECK`, plus payload-consistency `CHECK` constraints), and `mutation_trace_event_active_scopes`. Out — any Rust code consuming these tables (T02+). + - Dependencies: none + - Done when: a fresh `RepositoryAgentTraceDb::new_at` at a clean path applies `001`+`002`+`003` and all five tables exist with the specified columns, constraints, and indexes; a row violating a `CHECK` constraint (for example `ai_exclusive` attribution with a `NULL` `attribution_scope_id`) is rejected; a `TEXT` value of length 8 assigned to a `revision` column is rejected by the `typeof(revision) = 'blob'` check even though its length matches. + - Verify: `./scripts/run-cli-cargo.sh test --manifest-path cli/Cargo.toml agent_trace_db::repository::`; a new targeted test asserting the `003` tables, indexes, and constraints (including the TEXT-vs-BLOB revision case) behave as specified. + - Context synchronization: pending + +- [ ] T02: `Add revision and enum domain<->SQL codecs` (status:todo) + - Task ID: T02 + - Scope: In — create `cli/src/services/mutation_trace/store.rs` with `encode_revision`/`decode_revision` (`u64` <-> 8-byte big-endian `BLOB`) and explicit codecs for `ActorKind`, `FailureKind`, `ScopeStatus`, `Attribution`'s discriminant, and `Boundary`'s discriminant. Out — any query, projection, or commit logic (T03+). + - Dependencies: T01 + - Done when: `encode_revision`/`decode_revision` round-trip exactly for `0`, `1`, `i64::MAX`, `i64::MAX + 1`, and `u64::MAX`; every enum variant round-trips through its codec; no codec relies on `Debug` formatting or implicit serde representation. + - Verify: `./scripts/run-cli-cargo.sh test --manifest-path cli/Cargo.toml mutation_trace::store::` + - Context synchronization: pending + +- [ ] T03: `Add bounded WorktreeProjection load and cold-path MutationEvent read` (status:todo) + - Task ID: T03 + - Scope: In — `WorktreeProjection` (+ `into_protocol_state`) and `MutationTraceStore` wrapping `&RepositoryAgentTraceDb`. `load_worktree(worktree: &WorktreeId, scope: Option<&ScopeId>, event_key: Option<&EventKey>)` loads exactly one worktree row, only its currently `Active` scopes plus the explicitly referenced `scope` when supplied (even when that scope is `NeverSeen`/`Closed`/`Abandoned`), and 0 or 1 matching processed-event row for `event_key`. A separate cold-path `load_mutation_event(worktree: &WorktreeId, revision: u64) -> Result>` reads one `mutation_trace_events` row plus its `mutation_trace_event_active_scopes` rows and reconstructs a complete `MutationEvent`, decoding `Attribution` exactly (including `AiExclusive(scope_id)`) and the complete `Boundary`. When `scope` is supplied and the persisted `ScopeState` for that `ScopeId` has a `worktree_id` different from the requested `worktree`, `load_worktree` returns `Err` — it never silently omits the scope, never includes it in the projection, and never reassigns it to the requested worktree, preserving the permanent `ScopeId` -> `WorktreeId` identity `register_scope` already enforces. Out — initialization/commit logic (T04/T07); calling `load_mutation_event` from `load_worktree` or from any hook-boundary path. + - Dependencies: T01, T02 + - Done when: `load_worktree` returns `None` for a missing worktree and `Some(projection)` otherwise, with `scopes` containing every currently `Active` scope on that worktree plus `scope` when supplied regardless of its status, and never a `Closed`/`Abandoned`/`NeverSeen` scope that was not explicitly referenced; `attempts`, `mutation_events`, and `external_taint` stay empty; the method issues no query against `mutation_trace_events`. Specifically: a referenced scope on the requested worktree is included regardless of status; a referenced terminal (`Closed`/`Abandoned`/`NeverSeen`) scope on the requested worktree is included; an unreferenced terminal historical scope is excluded; a referenced scope whose persisted `worktree_id` belongs to a different worktree returns `Err`. `load_mutation_event` returns `None` when no row exists at that `(worktree, revision)` and otherwise reconstructs a `MutationEvent` whose `before_tree`/`after_tree`/`revision`/`tainted`/`failure_kind`/`attribution`/`boundary`/`active_scopes` exactly match what `store.commit` persisted. + - Verify: `./scripts/run-cli-cargo.sh test --manifest-path cli/Cargo.toml mutation_trace::store::` + - Context synchronization: pending + +- [ ] T04: `Add worktree/scope initialization operations` (status:todo) + - Task ID: T04 + - Scope: In — `initialize_worktree(worktree_id, initial_tree)` and `register_scope(scope_id, worktree_id, actor_kind)` on `MutationTraceStore`. Out — the CAS commit path (T06/T07). + - Dependencies: T03 + - Done when: `initialize_worktree` inserts `revision=0`/healthy/not-tainted/not-needs-rebaseline only when the worktree is missing and never overwrites an existing cursor; `register_scope` inserts `NeverSeen` when missing, returns the existing state when worktree+actor match, and errors on a worktree or actor mismatch for an existing `scope_id`. + - Verify: `./scripts/run-cli-cargo.sh test --manifest-path cli/Cargo.toml mutation_trace::store::` + - Context synchronization: pending + +- [ ] T05: `Add DurableTransition structural diff type` (status:todo) + - Task ID: T05 + - Scope: In — `DurableTransition` and `DurableTransition::between(before, after, worktree) -> Result>` performing pure structural diffing only, enforcing: the target worktree exists in both `before` and `after` and is never added or removed; no unrelated worktree changes; when a durable transition exists, its worktree's next revision is exactly `expected_revision + 1` computed via checked `u64` arithmetic; no scope is added or deleted; a changed scope belongs to the target worktree; `ScopeState.worktree_id` and `ScopeState.actor_kind` never change (only `status` may); `processed_events` may only gain entries, never lose them, with at most one new entry whose scope belongs to the target worktree; `mutation_events` may only gain entries, never lose them, with at most one new entry belonging to the target worktree; `AttemptState`/`external_taint` differences are ignored. Out — SQL/DB code (T06/T07). + - Dependencies: T02 + - Done when: `between()` returns `Ok(None)` for a `database_failure`-only transition and for a no-change `Flush`; returns `Ok(Some(..))` with the correct shape for `Start`/`Advance`/`Close`, `taint`, `abandon`, and `recover` transitions exercised directly against `protocol::*` outputs; the function contains no boundary-kind, contention, or taint conditionals; it returns `Err` for a malformed `before`/`after` pair covering at least: an `actor_kind` change, a scope's `worktree_id` change, a processed `EventKey` disappearing, a `MutationEvent` disappearing, an unrelated worktree changing, a revision jump by more than 1, a revision decrease, and a scope unexpectedly appearing or disappearing. + - Verify: `./scripts/run-cli-cargo.sh test --manifest-path cli/Cargo.toml mutation_trace::store::` + - Context synchronization: pending + +- [ ] T06: `Add generic transactional CAS batch primitive to TursoDb` (status:todo) + - Task ID: T06 + - Scope: In — `TransactionStatement` and `TursoDb::execute_transactional_cas_batch(operation_name, retry_hint, guard, statements)` in `cli/src/services/db/mod.rs`, with a retryability contract distinct from the shared `run_with_retry_sync` helper's plain any-`Err`-retries behavior: a guard affecting 0 rows commits as a no-op and returns `Ok(false)` (a normal CAS conflict) without running any statement and without being retried; a guard affecting 1 row runs every statement inside the same `BEGIN IMMEDIATE` transaction and returns `Ok(true)`; a retryable DB failure (lock/busy/other transient condition) retries the entire transaction from `BEGIN IMMEDIATE`; a deterministic failure (SQL/schema/constraint/invariant violation) returns `Err` without being retried. This adds the minimum local retryability classification needed for that behavior — for example, the retried closure returns a classified outcome that `run_with_retry_sync` still treats as `Ok` so it never retries a deterministic failure, and the caller re-raises that failure as `Err` once the closure returns — without changing `resilience.rs` or any other caller of `run_with_retry_sync`. Out — mutation-trace-specific SQL (T07). + - Dependencies: none + - Done when: a guard affecting 0 rows commits as a no-op and returns `Ok(false)` without running any statement or waiting for a retry backoff; a guard affecting 1 row runs every statement and returns `Ok(true)`; an injected deterministic mid-batch failure rolls back the entire transaction (including the guard's own effect) and surfaces as `Err` after exactly one attempt, never reported as a CAS conflict; an injected retryable DB failure retries the whole transaction from `BEGIN IMMEDIATE` (never individual statements) up to the configured attempt count. + - Verify: `./scripts/run-cli-cargo.sh test --manifest-path cli/Cargo.toml db::` + - Context synchronization: pending + +- [ ] T07: `Implement MutationTraceStore::commit` (status:todo) + - Task ID: T07 + - Scope: In — `CasResult` and `MutationTraceStore::commit(transition)`, translating a `DurableTransition` into the worktree CAS `UPDATE` plus scope `UPDATE`s plus processed-event `INSERT` plus mutation-event `INSERT` plus active-scope `INSERT`s, via `execute_transactional_cas_batch`. Out — concurrency/rollback/round-trip test coverage (T08/T09). + - Dependencies: T04, T05, T06 + - Done when: `commit()` returns `CasResult::Applied` with every included write visible when the worktree's on-disk revision matches `expected_revision`, and `CasResult::Conflict` with no visible write otherwise; a deterministic failure surfaced by `execute_transactional_cas_batch` propagates out of `commit()` as an `Err`, never as `CasResult::Conflict`. + - Verify: `./scripts/run-cli-cargo.sh test --manifest-path cli/Cargo.toml mutation_trace::store::` + - Context synchronization: pending + +- [ ] T08: `Add CAS and concurrency test coverage for store.commit` (status:todo) + - Task ID: T08 + - Scope: In — tests for: two writers committing from the same revision against one physical repository-scoped `agent-trace.db`, using two independent `RepositoryAgentTraceDb` handles/connections opened against that same database file (one `MutationTraceStore` per handle), with both writers loading worktree revision `N` before either commits and executing their commits from separate threads (or an equivalent that exercises two independent DB connections rather than one handle invoked twice in sequence) — exactly one result `CasResult::Applied`, the other `CasResult::Conflict`; atomic rollback on an injected deterministic mid-transaction failure; `u64::MAX` round-trip through the real DB; `(scope_id, event_id)` replay-uniqueness rejection; strong recovery (all active scopes abandoned) and needs-only recovery (surviving active scopes stay active). Out — production code changes beyond what T07 already provides; process-spawning or other multiprocess test infrastructure (two independent DB handles on separate threads are sufficient for this PR). + - Dependencies: T07 + - Done when: all five scenarios above are covered by passing tests; the two-writer test is not satisfied by calling `commit` twice sequentially through one shared `RepositoryAgentTraceDb` handle; after both commits, reopening the database shows the worktree revision advanced exactly once and only the winning transition's durable effects (scope status, processed event, mutation event, active scopes) are present; the atomic-rollback test observes revision, scope status, processed event, mutation event, and active scopes all unchanged after the injected failure. + - Verify: `./scripts/run-cli-cargo.sh test --manifest-path cli/Cargo.toml mutation_trace::store::` + - Context synchronization: pending + +- [ ] T09: `Add real-protocol round-trip persistence tests` (status:todo) + - Task ID: T09 + - Scope: In — tests driving load (`load_worktree`, bounded to `Active` scopes plus the transition's referenced scope) -> `protocol::prepare`/`commit` (or `taint`/`database_failure`/`abandon`/`recover`) -> `DurableTransition::between` -> `store.commit` -> drop DB handle -> reopen -> reload, for `Start`, `Advance`, `Close`, `Flush` with change, `Flush` without change, taint, abandon, recover, contended mutation, and a replayed `EventKey`. For every transition that emits a `MutationEvent`, additionally reload it after reopening with `load_mutation_event(worktree, revision)` and compare it field-for-field (including exact `Attribution`/`Boundary` decoding) against the `MutationEvent` the original protocol transition produced. Out — new production code, unless a genuine T01-T07 gap surfaces. + - Dependencies: T07 + - Done when: for every listed transition, the reloaded worktree/scope projection after reopening the DB matches the durable projection produced by the original protocol transition, and for every transition that emits a `MutationEvent`, `load_mutation_event` after reopening reconstructs it exactly. + - Verify: `./scripts/run-cli-cargo.sh test --manifest-path cli/Cargo.toml mutation_trace::store::` + - Context synchronization: pending + +- [ ] T10: `Add migration and lifecycle tests for migration 003` (status:todo) + - Task ID: T10 + - Scope: In — tests proving a fresh DB applies `001`+`002`+`003`; an existing `001`+`002`-only DB gets `003` applied through the `sce setup`/lifecycle path; the no-migration hook-runtime path does not apply `003` and still reports the existing "Run 'sce setup'." guidance when schema is incomplete. Out — changes to `REQUIRED_REPOSITORY_SCHEMA_TABLES` baseline-repair semantics. + - Dependencies: T01 + - Done when: all three scenarios pass without modifying the baseline-repair function's treatment of `001` metadata. + - Verify: `./scripts/run-cli-cargo.sh test --manifest-path cli/Cargo.toml agent_trace_db::` + - Context synchronization: pending + +- [ ] T11: `Document the mutation-trace store` (status:todo) + - Task ID: T11 + - Scope: In — `context/cli/mutation-trace-store.md` covering repository-DB ownership, `WorktreeId` as the persistence partition, the 8-byte big-endian revision encoding, `AttemptState`/`external_taint` non-persistence, and the store's non-goals (no Git I/O, no attribution decisions, no retry-after-`Conflict`); a `context/context-map.md` entry for the new file. Out — edits to any other existing `context/` file (left to task context synchronization). + - Dependencies: T01-T10 + - Done when: the new file exists, is linked from `context/context-map.md`, and every claim in it is checked against the code produced by T01-T10. + - Verify: manual inspection cross-referencing the file's claims against `store.rs`, the migration, and `db/mod.rs`. + - Context synchronization: pending + +## Open questions + +None. The change request already resolves every architectural decision (schema shape, CAS mechanics, which fields are excluded from persistence) precisely, and each decision checks out against the current `protocol.rs`/`types.rs` domain model and the existing Turso adapter conventions verified while authoring this plan. From 5fb1b51e81e791575a6dc2d2aac218c5e3504dca Mon Sep 17 00:00:00 2001 From: David Abram Date: Thu, 27 Aug 2026 13:48:34 +0200 Subject: [PATCH 02/15] storage: Add mutation-trace protocol persistence schema Persist the verified mutation-cursor protocol in the repository database so later store operations have durable tables for worktrees, scopes, processed events, mutation events, and active scopes. Add migration 003 with strict revision BLOB checks, enum allow-lists, and payload-consistency constraints, and extend repository initialization tests with constraint coverage. Plan: mutation-cursor-store-persistence (T01) Co-authored-by: SCE --- .../003_mutation_trace_protocol.sql | 86 +++++++++++++++++++ cli/src/services/agent_trace_db/repository.rs | 75 +++++++++++++++- .../mutation-cursor-store-persistence.md | 10 ++- 3 files changed, 168 insertions(+), 3 deletions(-) create mode 100644 cli/migrations/agent-trace-repository/003_mutation_trace_protocol.sql diff --git a/cli/migrations/agent-trace-repository/003_mutation_trace_protocol.sql b/cli/migrations/agent-trace-repository/003_mutation_trace_protocol.sql new file mode 100644 index 00000000..d6bdd867 --- /dev/null +++ b/cli/migrations/agent-trace-repository/003_mutation_trace_protocol.sql @@ -0,0 +1,86 @@ +-- Durable persistence for the verified mutation-cursor protocol +-- (`cli/src/services/mutation_trace/`). +-- +-- This migration is additive: it introduces five new tables and leaves every +-- table from 001/002 untouched. `revision` is stored as an 8-byte +-- big-endian BLOB on every column that carries one (never a SQLite INTEGER), +-- enforced by `CHECK (typeof(revision) = 'blob' AND length(revision) = 8)`, +-- so a TEXT value of matching length is still rejected. Enum-shaped columns +-- use TEXT with an explicit CHECK allow-list, matching the `role`/ +-- `payload_type` convention already used in 001. `AttemptState` (transient) +-- and `external_taint` (not DB-authoritative) are deliberately not +-- represented by any table here. + +CREATE TABLE IF NOT EXISTS mutation_trace_worktrees ( + worktree_id TEXT PRIMARY KEY, + cursor_tree TEXT NOT NULL, + revision BLOB NOT NULL + CHECK (typeof(revision) = 'blob' AND length(revision) = 8), + tainted INTEGER NOT NULL CHECK (tainted IN (0, 1)), + failure_kind TEXT NOT NULL CHECK (failure_kind IN ('healthy', 'snapshot_failure')), + needs_rebaseline INTEGER NOT NULL CHECK (needs_rebaseline IN (0, 1)), + created_at TEXT NOT NULL DEFAULT (strftime('%Y-%m-%dT%H:%M:%fZ', 'now')), + updated_at TEXT NOT NULL DEFAULT (strftime('%Y-%m-%dT%H:%M:%fZ', 'now')) +); + +CREATE TABLE IF NOT EXISTS mutation_trace_scopes ( + scope_id TEXT PRIMARY KEY, + worktree_id TEXT NOT NULL, + actor_kind TEXT NOT NULL CHECK (actor_kind IN ('claude_code', 'codex', 'opencode', 'pi')), + status TEXT NOT NULL CHECK (status IN ('never_seen', 'active', 'closed', 'abandoned')), + created_at TEXT NOT NULL DEFAULT (strftime('%Y-%m-%dT%H:%M:%fZ', 'now')), + updated_at TEXT NOT NULL DEFAULT (strftime('%Y-%m-%dT%H:%M:%fZ', 'now')) +); + +CREATE INDEX IF NOT EXISTS idx_mutation_trace_scopes_worktree +ON mutation_trace_scopes (worktree_id); + +CREATE INDEX IF NOT EXISTS idx_mutation_trace_scopes_worktree_status +ON mutation_trace_scopes (worktree_id, status); + +CREATE TABLE IF NOT EXISTS mutation_trace_processed_events ( + scope_id TEXT NOT NULL, + event_id TEXT NOT NULL, + worktree_id TEXT NOT NULL, + created_at TEXT NOT NULL DEFAULT (strftime('%Y-%m-%dT%H:%M:%fZ', 'now')), + PRIMARY KEY (scope_id, event_id) +); + +CREATE INDEX IF NOT EXISTS idx_mutation_trace_processed_events_worktree +ON mutation_trace_processed_events (worktree_id); + +CREATE TABLE IF NOT EXISTS mutation_trace_events ( + worktree_id TEXT NOT NULL, + revision BLOB NOT NULL + CHECK (typeof(revision) = 'blob' AND length(revision) = 8), + before_tree TEXT NOT NULL, + after_tree TEXT NOT NULL, + tainted INTEGER NOT NULL CHECK (tainted IN (0, 1)), + failure_kind TEXT NOT NULL CHECK (failure_kind IN ('healthy', 'snapshot_failure')), + attribution_kind TEXT NOT NULL + CHECK (attribution_kind IN ('ineligible_unscoped', 'ai_exclusive', 'ai_contended')), + attribution_scope_id TEXT, + boundary_kind TEXT NOT NULL CHECK (boundary_kind IN ('start', 'advance', 'close', 'flush')), + boundary_scope_id TEXT, + boundary_event_id TEXT, + created_at TEXT NOT NULL DEFAULT (strftime('%Y-%m-%dT%H:%M:%fZ', 'now')), + PRIMARY KEY (worktree_id, revision), + CHECK ( + (attribution_kind = 'ai_exclusive' AND attribution_scope_id IS NOT NULL) + OR (attribution_kind != 'ai_exclusive' AND attribution_scope_id IS NULL) + ), + CHECK ( + (boundary_kind IN ('start', 'advance', 'close') + AND boundary_scope_id IS NOT NULL AND boundary_event_id IS NOT NULL) + OR (boundary_kind = 'flush' + AND boundary_scope_id IS NULL AND boundary_event_id IS NULL) + ) +); + +CREATE TABLE IF NOT EXISTS mutation_trace_event_active_scopes ( + worktree_id TEXT NOT NULL, + revision BLOB NOT NULL + CHECK (typeof(revision) = 'blob' AND length(revision) = 8), + scope_id TEXT NOT NULL, + PRIMARY KEY (worktree_id, revision, scope_id) +); diff --git a/cli/src/services/agent_trace_db/repository.rs b/cli/src/services/agent_trace_db/repository.rs index 9cd63d2e..44792d30 100644 --- a/cli/src/services/agent_trace_db/repository.rs +++ b/cli/src/services/agent_trace_db/repository.rs @@ -388,6 +388,11 @@ mod tests { "agent_traces", "messages", "parts", + "mutation_trace_worktrees", + "mutation_trace_scopes", + "mutation_trace_processed_events", + "mutation_trace_events", + "mutation_trace_event_active_scopes", ] { assert!( sqlite_object_exists(&db, "table", table), @@ -401,6 +406,9 @@ mod tests { "idx_messages_session_message", "idx_messages_session_order", "idx_parts_session_message_order", + "idx_mutation_trace_scopes_worktree", + "idx_mutation_trace_scopes_worktree_status", + "idx_mutation_trace_processed_events_worktree", ] { assert!( sqlite_object_exists(&db, "index", index), @@ -426,9 +434,10 @@ mod tests { vec![ String::from("001_repository_schema"), String::from("002_repository_source_instance_id"), + String::from("003_mutation_trace_protocol"), ], "repository DBs should be initialized from the baseline schema plus \ - its additive source-instance-id migration" + its additive source-instance-id and mutation-trace-protocol migrations" ); db.ensure_schema_ready_for_hooks() @@ -437,6 +446,70 @@ mod tests { remove_test_db(&db_path); } + #[test] + fn mutation_trace_worktrees_revision_must_be_a_blob_not_matching_length_text() { + let db_path = unique_test_db_path("mutation-trace-revision-blob"); + let db = RepositoryAgentTraceDb::new_at(&db_path).expect("repository DB should open"); + + let text_revision_error = db + .execute( + "INSERT INTO mutation_trace_worktrees + (worktree_id, cursor_tree, revision, tainted, failure_kind, needs_rebaseline) + VALUES ('wt-1', 'tree-0', '12345678', 0, 'healthy', 0)", + (), + ) + .expect_err( + "an 8-byte TEXT value must still be rejected by the typeof(revision) = 'blob' check", + ); + assert!( + text_revision_error.to_string().contains("CHECK"), + "unexpected error: {text_revision_error}" + ); + + db.execute( + "INSERT INTO mutation_trace_worktrees + (worktree_id, cursor_tree, revision, tainted, failure_kind, needs_rebaseline) + VALUES ('wt-1', 'tree-0', X'0000000000000000', 0, 'healthy', 0)", + (), + ) + .expect("an 8-byte BLOB revision should be accepted"); + + remove_test_db(&db_path); + } + + #[test] + fn mutation_trace_events_ai_exclusive_attribution_requires_a_scope_id() { + let db_path = unique_test_db_path("mutation-trace-attribution-check"); + let db = RepositoryAgentTraceDb::new_at(&db_path).expect("repository DB should open"); + + let missing_scope_error = db + .execute( + "INSERT INTO mutation_trace_events + (worktree_id, revision, before_tree, after_tree, tainted, failure_kind, + attribution_kind, attribution_scope_id, boundary_kind, boundary_scope_id, boundary_event_id) + VALUES ('wt-1', X'0000000000000001', 'tree-0', 'tree-1', 0, 'healthy', + 'ai_exclusive', NULL, 'flush', NULL, NULL)", + (), + ) + .expect_err("ai_exclusive attribution with a NULL attribution_scope_id must be rejected"); + assert!( + missing_scope_error.to_string().contains("CHECK"), + "unexpected error: {missing_scope_error}" + ); + + db.execute( + "INSERT INTO mutation_trace_events + (worktree_id, revision, before_tree, after_tree, tainted, failure_kind, + attribution_kind, attribution_scope_id, boundary_kind, boundary_scope_id, boundary_event_id) + VALUES ('wt-1', X'0000000000000001', 'tree-0', 'tree-1', 0, 'healthy', + 'ai_exclusive', 'scope-1', 'start', 'scope-1', 'event-1')", + (), + ) + .expect("ai_exclusive attribution with a scope ID should be accepted"); + + remove_test_db(&db_path); + } + #[test] fn trace_tables_have_no_checkout_id_columns() { let db_path = unique_test_db_path("no-checkout-id"); diff --git a/context/plans/mutation-cursor-store-persistence.md b/context/plans/mutation-cursor-store-persistence.md index b2a744d1..1943dd91 100644 --- a/context/plans/mutation-cursor-store-persistence.md +++ b/context/plans/mutation-cursor-store-persistence.md @@ -119,13 +119,19 @@ Persist this field in every plan; this is durable plan state, not chat state: ## Task stack -- [ ] T01: `Add migration 003 for mutation-trace protocol tables` (status:todo) +- [x] T01: `Add migration 003 for mutation-trace protocol tables` (status:done) - Task ID: T01 - Scope: In — `cli/migrations/agent-trace-repository/003_mutation_trace_protocol.sql` defining `mutation_trace_worktrees` (revision `BLOB` constrained by `CHECK (typeof(revision) = 'blob' AND length(revision) = 8)`), `mutation_trace_scopes` (+ `idx_mutation_trace_scopes_worktree` and a new composite `idx_mutation_trace_scopes_worktree_status` index on `(worktree_id, status)` for the bounded hot-path scope lookup), `mutation_trace_processed_events`, `mutation_trace_events` (+ the same `typeof`/`length` revision `CHECK`, plus payload-consistency `CHECK` constraints), and `mutation_trace_event_active_scopes`. Out — any Rust code consuming these tables (T02+). - Dependencies: none - Done when: a fresh `RepositoryAgentTraceDb::new_at` at a clean path applies `001`+`002`+`003` and all five tables exist with the specified columns, constraints, and indexes; a row violating a `CHECK` constraint (for example `ai_exclusive` attribution with a `NULL` `attribution_scope_id`) is rejected; a `TEXT` value of length 8 assigned to a `revision` column is rejected by the `typeof(revision) = 'blob'` check even though its length matches. - Verify: `./scripts/run-cli-cargo.sh test --manifest-path cli/Cargo.toml agent_trace_db::repository::`; a new targeted test asserting the `003` tables, indexes, and constraints (including the TEXT-vs-BLOB revision case) behave as specified. - - Context synchronization: pending + - Completed: 2026-08-27 + - Files changed: `cli/migrations/agent-trace-repository/003_mutation_trace_protocol.sql` (new); `cli/src/services/agent_trace_db/repository.rs` + - Result: Added migration `003_mutation_trace_protocol.sql` defining `mutation_trace_worktrees`, `mutation_trace_scopes` (+ `idx_mutation_trace_scopes_worktree`, `idx_mutation_trace_scopes_worktree_status`), `mutation_trace_processed_events` (+ `idx_mutation_trace_processed_events_worktree`), `mutation_trace_events`, and `mutation_trace_event_active_scopes`, all discovered automatically by `build.rs`'s directory scan. Revision columns use `BLOB NOT NULL CHECK (typeof(revision) = 'blob' AND length(revision) = 8)`; enum-shaped columns use `TEXT` with `CHECK (... IN (...))` allow-lists following the existing `role`/`payload_type` convention; `mutation_trace_events` additionally enforces attribution/boundary payload-consistency `CHECK`s (`ai_exclusive` requires a non-null `attribution_scope_id`; hook boundaries require non-null `boundary_scope_id`/`boundary_event_id`, `flush` requires both null). Updated `open_at_initializes_the_full_schema_from_one_migration` to assert the new migration ID and the five new tables/indexes, and added two new targeted tests (`mutation_trace_worktrees_revision_must_be_a_blob_not_matching_length_text`, `mutation_trace_events_ai_exclusive_attribution_requires_a_scope_id`) proving the TEXT-vs-BLOB revision rejection and the `ai_exclusive`-requires-scope rejection, each paired with a positive control insert. + - Verify: `./scripts/run-cli-cargo.sh test --manifest-path cli/Cargo.toml agent_trace_db::repository::` — passed, 18/18 (including the two new tests and the updated baseline-schema test); `./scripts/run-cli-cargo.sh fmt --manifest-path cli/Cargo.toml -- --check` — passed (no diff). + - Done checks: fresh DB applies `001`+`002`+`003` with all five tables/indexes present (verified by the updated baseline test); `ai_exclusive` attribution with a `NULL` `attribution_scope_id` is rejected (verified); an 8-byte TEXT value assigned to `revision` is rejected by `typeof(revision) = 'blob'` (verified); `git diff --exit-code` on `001`/`002` shows zero changes (verified). + - Context impact: local — additive schema-only migration; no Rust code consumes these new tables yet (T02+ wire codecs, loads, and commits against them). No durable context synchronization is required for this task; the plan's `Context sync` entries are authored by T11 once the full store lands. + - Context synchronization: synced - [ ] T02: `Add revision and enum domain<->SQL codecs` (status:todo) - Task ID: T02 From 407aa625b0f81313b515682ad6d771c05af99236 Mon Sep 17 00:00:00 2001 From: David Abram Date: Thu, 27 Aug 2026 14:01:49 +0200 Subject: [PATCH 03/15] mutation-trace: Add explicit revision and enum SQL codecs Persist mutation-trace domain values using stable, migration-aligned encodings before query and commit logic is added. Add big-endian revision codecs and explicit string codecs plus discriminant helpers, with rejection of invalid values and round-trip tests. Plan: mutation-cursor-store-persistence, T02 Co-authored-by: SCE --- cli/src/services/mutation_trace/mod.rs | 1 + cli/src/services/mutation_trace/store.rs | 302 ++++++++++++++++++ .../mutation-cursor-store-persistence.md | 10 +- 3 files changed, 311 insertions(+), 2 deletions(-) create mode 100644 cli/src/services/mutation_trace/store.rs diff --git a/cli/src/services/mutation_trace/mod.rs b/cli/src/services/mutation_trace/mod.rs index 5544f4aa..2416cbdc 100644 --- a/cli/src/services/mutation_trace/mod.rs +++ b/cli/src/services/mutation_trace/mod.rs @@ -156,6 +156,7 @@ //! unchanged). pub mod protocol; +pub mod store; pub mod types; #[cfg(test)] diff --git a/cli/src/services/mutation_trace/store.rs b/cli/src/services/mutation_trace/store.rs new file mode 100644 index 00000000..fefa69de --- /dev/null +++ b/cli/src/services/mutation_trace/store.rs @@ -0,0 +1,302 @@ +//! Domain<->SQL codecs for the mutation-cursor persistence layer. +//! +//! These codecs are the only translation between `super::types` domain +//! values and the `TEXT`/`BLOB` representations `cli/migrations/agent-trace- +//! repository/003_mutation_trace_protocol.sql` constrains those columns to. +//! Every codec here is an explicit function over a fixed set of variants — no +//! codec derives from `Debug` or a serde representation, so a variant rename +//! cannot silently change the durable encoding. +//! +//! This module carries no query, projection, or commit logic yet (see +//! `mutation-cursor-store-persistence` plan tasks T03+); it only establishes +//! the byte- and string-level encodings later tasks build on. + +use anyhow::{bail, Result}; + +use super::types::{ActorKind, Attribution, Boundary, FailureKind, ScopeStatus}; + +/// Encodes a worktree/event revision as the 8-byte big-endian `BLOB` stored +/// by every `revision` column in migration `003`. +pub fn encode_revision(revision: u64) -> [u8; 8] { + revision.to_be_bytes() +} + +/// Decodes a worktree/event revision from the 8-byte big-endian `BLOB` +/// migration `003`'s `CHECK (typeof(revision) = 'blob' AND length(revision) +/// = 8)` constraint guarantees on every stored value. +pub fn decode_revision(blob: &[u8]) -> Result { + let bytes: [u8; 8] = blob.try_into().map_err(|_| { + anyhow::anyhow!("revision blob must be exactly 8 bytes, got {}", blob.len()) + })?; + Ok(u64::from_be_bytes(bytes)) +} + +/// Encodes an [`ActorKind`] as the `mutation_trace_scopes.actor_kind` `TEXT` +/// value migration `003`'s `CHECK (actor_kind IN (...))` allow-list expects. +pub fn encode_actor_kind(actor_kind: ActorKind) -> &'static str { + match actor_kind { + ActorKind::ClaudeCode => "claude_code", + ActorKind::Codex => "codex", + ActorKind::OpenCode => "opencode", + ActorKind::Pi => "pi", + } +} + +/// Decodes an [`ActorKind`] from `mutation_trace_scopes.actor_kind`. +pub fn decode_actor_kind(value: &str) -> Result { + match value { + "claude_code" => Ok(ActorKind::ClaudeCode), + "codex" => Ok(ActorKind::Codex), + "opencode" => Ok(ActorKind::OpenCode), + "pi" => Ok(ActorKind::Pi), + other => bail!("unrecognized actor_kind: {other:?}"), + } +} + +/// Encodes a [`FailureKind`] as the `failure_kind` `TEXT` value migration +/// `003` constrains `mutation_trace_worktrees.failure_kind` and +/// `mutation_trace_events.failure_kind` to. +pub fn encode_failure_kind(failure_kind: FailureKind) -> &'static str { + match failure_kind { + FailureKind::Healthy => "healthy", + FailureKind::SnapshotFailure => "snapshot_failure", + } +} + +/// Decodes a [`FailureKind`] from a `failure_kind` column. +pub fn decode_failure_kind(value: &str) -> Result { + match value { + "healthy" => Ok(FailureKind::Healthy), + "snapshot_failure" => Ok(FailureKind::SnapshotFailure), + other => bail!("unrecognized failure_kind: {other:?}"), + } +} + +/// Encodes a [`ScopeStatus`] as the `mutation_trace_scopes.status` `TEXT` +/// value migration `003`'s `CHECK (status IN (...))` allow-list expects. +pub fn encode_scope_status(status: ScopeStatus) -> &'static str { + match status { + ScopeStatus::NeverSeen => "never_seen", + ScopeStatus::Active => "active", + ScopeStatus::Closed => "closed", + ScopeStatus::Abandoned => "abandoned", + } +} + +/// Decodes a [`ScopeStatus`] from `mutation_trace_scopes.status`. +pub fn decode_scope_status(value: &str) -> Result { + match value { + "never_seen" => Ok(ScopeStatus::NeverSeen), + "active" => Ok(ScopeStatus::Active), + "closed" => Ok(ScopeStatus::Closed), + "abandoned" => Ok(ScopeStatus::Abandoned), + other => bail!("unrecognized scope status: {other:?}"), + } +} + +/// [`Attribution`]'s discriminant, decoupled from its `AiExclusive` payload +/// (`ScopeId`). Reconstructing a full [`Attribution`] from a persisted row +/// also needs `attribution_scope_id`, which is a `mutation_trace_events` +/// query concern owned by a later task, not by this codec. +#[derive(Clone, Copy, Debug, Eq, PartialEq)] +pub enum AttributionKind { + IneligibleUnscoped, + AiExclusive, + AiContended, +} + +/// The discriminant of an [`Attribution`] value. +pub fn attribution_kind(attribution: &Attribution) -> AttributionKind { + match attribution { + Attribution::IneligibleUnscoped => AttributionKind::IneligibleUnscoped, + Attribution::AiExclusive(_) => AttributionKind::AiExclusive, + Attribution::AiContended => AttributionKind::AiContended, + } +} + +/// Encodes an [`AttributionKind`] as the +/// `mutation_trace_events.attribution_kind` `TEXT` value migration `003`'s +/// `CHECK (attribution_kind IN (...))` allow-list expects. +pub fn encode_attribution_kind(kind: AttributionKind) -> &'static str { + match kind { + AttributionKind::IneligibleUnscoped => "ineligible_unscoped", + AttributionKind::AiExclusive => "ai_exclusive", + AttributionKind::AiContended => "ai_contended", + } +} + +/// Decodes an [`AttributionKind`] from `mutation_trace_events.attribution_kind`. +pub fn decode_attribution_kind(value: &str) -> Result { + match value { + "ineligible_unscoped" => Ok(AttributionKind::IneligibleUnscoped), + "ai_exclusive" => Ok(AttributionKind::AiExclusive), + "ai_contended" => Ok(AttributionKind::AiContended), + other => bail!("unrecognized attribution_kind: {other:?}"), + } +} + +/// [`Boundary`]'s discriminant, decoupled from its `scope`/`event`/`worktree` +/// payload. Reconstructing a full [`Boundary`] from a persisted row also +/// needs `boundary_scope_id`/`boundary_event_id`, which is a +/// `mutation_trace_events` query concern owned by a later task, not by this +/// codec. +#[derive(Clone, Copy, Debug, Eq, PartialEq)] +pub enum BoundaryKind { + Start, + Advance, + Close, + Flush, +} + +/// The discriminant of a [`Boundary`] value. +pub fn boundary_kind(boundary: &Boundary) -> BoundaryKind { + match boundary { + Boundary::Start { .. } => BoundaryKind::Start, + Boundary::Advance { .. } => BoundaryKind::Advance, + Boundary::Close { .. } => BoundaryKind::Close, + Boundary::Flush { .. } => BoundaryKind::Flush, + } +} + +/// Encodes a [`BoundaryKind`] as the `mutation_trace_events.boundary_kind` +/// `TEXT` value migration `003`'s `CHECK (boundary_kind IN (...))` +/// allow-list expects. +pub fn encode_boundary_kind(kind: BoundaryKind) -> &'static str { + match kind { + BoundaryKind::Start => "start", + BoundaryKind::Advance => "advance", + BoundaryKind::Close => "close", + BoundaryKind::Flush => "flush", + } +} + +/// Decodes a [`BoundaryKind`] from `mutation_trace_events.boundary_kind`. +pub fn decode_boundary_kind(value: &str) -> Result { + match value { + "start" => Ok(BoundaryKind::Start), + "advance" => Ok(BoundaryKind::Advance), + "close" => Ok(BoundaryKind::Close), + "flush" => Ok(BoundaryKind::Flush), + other => bail!("unrecognized boundary_kind: {other:?}"), + } +} + +#[cfg(test)] +mod tests { + use super::*; + use crate::services::mutation_trace::types::{EventId, ScopeId}; + + #[test] + fn revision_round_trips_at_boundary_values() { + for revision in [0u64, 1, i64::MAX as u64, (i64::MAX as u64) + 1, u64::MAX] { + let encoded = encode_revision(revision); + assert_eq!(encoded.len(), 8); + assert_eq!(decode_revision(&encoded).unwrap(), revision); + } + } + + #[test] + fn decode_revision_rejects_wrong_length() { + assert!(decode_revision(&[0u8; 7]).is_err()); + assert!(decode_revision(&[0u8; 9]).is_err()); + } + + #[test] + fn actor_kind_round_trips_every_variant() { + for actor_kind in [ + ActorKind::ClaudeCode, + ActorKind::Codex, + ActorKind::OpenCode, + ActorKind::Pi, + ] { + let encoded = encode_actor_kind(actor_kind); + assert_eq!(decode_actor_kind(encoded).unwrap(), actor_kind); + } + } + + #[test] + fn decode_actor_kind_rejects_unknown_value() { + assert!(decode_actor_kind("unknown").is_err()); + } + + #[test] + fn failure_kind_round_trips_every_variant() { + for failure_kind in [FailureKind::Healthy, FailureKind::SnapshotFailure] { + let encoded = encode_failure_kind(failure_kind); + assert_eq!(decode_failure_kind(encoded).unwrap(), failure_kind); + } + } + + #[test] + fn decode_failure_kind_rejects_unknown_value() { + assert!(decode_failure_kind("unknown").is_err()); + } + + #[test] + fn scope_status_round_trips_every_variant() { + for status in [ + ScopeStatus::NeverSeen, + ScopeStatus::Active, + ScopeStatus::Closed, + ScopeStatus::Abandoned, + ] { + let encoded = encode_scope_status(status); + assert_eq!(decode_scope_status(encoded).unwrap(), status); + } + } + + #[test] + fn decode_scope_status_rejects_unknown_value() { + assert!(decode_scope_status("unknown").is_err()); + } + + #[test] + fn attribution_kind_round_trips_every_variant() { + let ineligible = Attribution::IneligibleUnscoped; + let exclusive = Attribution::AiExclusive(ScopeId("scope-1".to_string())); + let contended = Attribution::AiContended; + + for attribution in [&ineligible, &exclusive, &contended] { + let kind = attribution_kind(attribution); + let encoded = encode_attribution_kind(kind); + assert_eq!(decode_attribution_kind(encoded).unwrap(), kind); + } + + assert_eq!(attribution_kind(&exclusive), AttributionKind::AiExclusive); + } + + #[test] + fn decode_attribution_kind_rejects_unknown_value() { + assert!(decode_attribution_kind("unknown").is_err()); + } + + #[test] + fn boundary_kind_round_trips_every_variant() { + let start = Boundary::Start { + scope: ScopeId("scope-1".to_string()), + event: EventId("event-1".to_string()), + }; + let advance = Boundary::Advance { + scope: ScopeId("scope-1".to_string()), + event: EventId("event-2".to_string()), + }; + let close = Boundary::Close { + scope: ScopeId("scope-1".to_string()), + event: EventId("event-3".to_string()), + }; + let flush = Boundary::Flush { + worktree: crate::services::mutation_trace::types::WorktreeId("wt-1".to_string()), + }; + + for boundary in [&start, &advance, &close, &flush] { + let kind = boundary_kind(boundary); + let encoded = encode_boundary_kind(kind); + assert_eq!(decode_boundary_kind(encoded).unwrap(), kind); + } + } + + #[test] + fn decode_boundary_kind_rejects_unknown_value() { + assert!(decode_boundary_kind("unknown").is_err()); + } +} diff --git a/context/plans/mutation-cursor-store-persistence.md b/context/plans/mutation-cursor-store-persistence.md index 1943dd91..096600c8 100644 --- a/context/plans/mutation-cursor-store-persistence.md +++ b/context/plans/mutation-cursor-store-persistence.md @@ -133,13 +133,19 @@ Persist this field in every plan; this is durable plan state, not chat state: - Context impact: local — additive schema-only migration; no Rust code consumes these new tables yet (T02+ wire codecs, loads, and commits against them). No durable context synchronization is required for this task; the plan's `Context sync` entries are authored by T11 once the full store lands. - Context synchronization: synced -- [ ] T02: `Add revision and enum domain<->SQL codecs` (status:todo) +- [x] T02: `Add revision and enum domain<->SQL codecs` (status:done) - Task ID: T02 - Scope: In — create `cli/src/services/mutation_trace/store.rs` with `encode_revision`/`decode_revision` (`u64` <-> 8-byte big-endian `BLOB`) and explicit codecs for `ActorKind`, `FailureKind`, `ScopeStatus`, `Attribution`'s discriminant, and `Boundary`'s discriminant. Out — any query, projection, or commit logic (T03+). - Dependencies: T01 - Done when: `encode_revision`/`decode_revision` round-trip exactly for `0`, `1`, `i64::MAX`, `i64::MAX + 1`, and `u64::MAX`; every enum variant round-trips through its codec; no codec relies on `Debug` formatting or implicit serde representation. - Verify: `./scripts/run-cli-cargo.sh test --manifest-path cli/Cargo.toml mutation_trace::store::` - - Context synchronization: pending + - Completed: 2026-08-27 + - Files changed: `cli/src/services/mutation_trace/store.rs` (new); `cli/src/services/mutation_trace/mod.rs` + - Result: Added `cli/src/services/mutation_trace/store.rs` with `encode_revision`/`decode_revision` (`u64` <-> `[u8; 8]` big-endian) and explicit `encode_*`/`decode_*` function-pair codecs for `ActorKind`, `FailureKind`, `ScopeStatus`, a new `AttributionKind` discriminant type (`ineligible_unscoped`/`ai_exclusive`/`ai_contended`, derived from `Attribution` via a new `attribution_kind` accessor), and a new `BoundaryKind` discriminant type (`start`/`advance`/`close`/`flush`, derived from `Boundary` via a new `boundary_kind` accessor) — every string constant matches migration `003`'s `CHECK (... IN (...))` allow-lists exactly. Decode functions return `anyhow::Result` and reject unrecognized strings via `anyhow::bail!`, matching the crate's existing `agent_trace_db`/`repository.rs` error convention. No codec derives from or matches on `Debug` output. Added `pub mod store;` to `mod.rs` so the module compiles and the `mutation_trace::store::` test path resolves. `Attribution`'s and `Boundary`'s full payload fields (`attribution_scope_id`, `boundary_scope_id`/`boundary_event_id`) are left to the row-reconstruction logic in T03/T07, matching the task's "discriminant"-only scope. + - Verify: `./scripts/run-cli-cargo.sh test --manifest-path cli/Cargo.toml mutation_trace::store::` — passed, 12/12; `./scripts/run-cli-cargo.sh fmt --manifest-path cli/Cargo.toml -- --check` — passed after one auto-formatting pass (no manual diff needed beyond `cargo fmt`); `./scripts/run-cli-cargo.sh clippy --manifest-path cli/Cargo.toml --all-targets -- -D warnings` — passed, zero warnings. + - Done checks: `encode_revision`/`decode_revision` round-trip exactly for `0`, `1`, `i64::MAX`, `i64::MAX + 1`, `u64::MAX` (verified by `revision_round_trips_at_boundary_values`); every `ActorKind`/`FailureKind`/`ScopeStatus`/`AttributionKind`/`BoundaryKind` variant round-trips through its own codec (verified by five dedicated `*_round_trips_every_variant` tests); no codec relies on `Debug` formatting or implicit serde representation (verified by inspection — every codec is a hand-written `match` over string literals, no `#[derive(Display)]`/serde attribute anywhere in the file). + - Context impact: local — new codec functions and types confined to a new, not-yet-wired-in file; no caller exists yet (T03+ will be the first consumer), so no root context file describes runtime behavior this changes yet. `context/cli/mutation-trace-protocol.md`'s "not yet wired into any hook, command, or database call site" framing remains accurate until a real DB call site lands (T07), matching the plan's assumption that this update is deferred to task context synchronization once that framing goes stale. + - Context synchronization: synced - [ ] T03: `Add bounded WorktreeProjection load and cold-path MutationEvent read` (status:todo) - Task ID: T03 From 70a721c6fa081b30f16da2521da4373bf6b1dbdd Mon Sep 17 00:00:00 2001 From: David Abram Date: Thu, 27 Aug 2026 14:18:43 +0200 Subject: [PATCH 04/15] agent-trace-db: Remove redundant worktree identity from processed events Use `(scope_id, event_id)` as the sole processed-event identity, matching `EventKey` and avoiding duplicate worktree ownership data. Remove the redundant column and index, and add a regression test covering the composite key. Plan: mutation-cursor-store-persistence (T01, T03) Co-authored-by: SCE --- .../003_mutation_trace_protocol.sql | 11 ++-- cli/src/services/agent_trace_db/repository.rs | 51 ++++++++++++++++++- .../mutation-cursor-store-persistence.md | 30 ++++++++--- 3 files changed, 79 insertions(+), 13 deletions(-) diff --git a/cli/migrations/agent-trace-repository/003_mutation_trace_protocol.sql b/cli/migrations/agent-trace-repository/003_mutation_trace_protocol.sql index d6bdd867..a4657e4e 100644 --- a/cli/migrations/agent-trace-repository/003_mutation_trace_protocol.sql +++ b/cli/migrations/agent-trace-repository/003_mutation_trace_protocol.sql @@ -38,17 +38,20 @@ ON mutation_trace_scopes (worktree_id); CREATE INDEX IF NOT EXISTS idx_mutation_trace_scopes_worktree_status ON mutation_trace_scopes (worktree_id, status); +-- `worktree_id` is deliberately not duplicated here: a processed event's +-- identity is exactly `(scope_id, event_id)` (the domain `EventKey`), and +-- `scope_id`'s worktree is already a permanent fact owned by +-- `mutation_trace_scopes` (`ScopeId -> WorktreeId`, never reassigned). A +-- second `worktree_id` column would create two sources of truth for the same +-- fact and could disagree with `mutation_trace_scopes` for the same +-- `scope_id`; the schema does not represent that inconsistency. CREATE TABLE IF NOT EXISTS mutation_trace_processed_events ( scope_id TEXT NOT NULL, event_id TEXT NOT NULL, - worktree_id TEXT NOT NULL, created_at TEXT NOT NULL DEFAULT (strftime('%Y-%m-%dT%H:%M:%fZ', 'now')), PRIMARY KEY (scope_id, event_id) ); -CREATE INDEX IF NOT EXISTS idx_mutation_trace_processed_events_worktree -ON mutation_trace_processed_events (worktree_id); - CREATE TABLE IF NOT EXISTS mutation_trace_events ( worktree_id TEXT NOT NULL, revision BLOB NOT NULL diff --git a/cli/src/services/agent_trace_db/repository.rs b/cli/src/services/agent_trace_db/repository.rs index 44792d30..75ece484 100644 --- a/cli/src/services/agent_trace_db/repository.rs +++ b/cli/src/services/agent_trace_db/repository.rs @@ -408,7 +408,6 @@ mod tests { "idx_parts_session_message_order", "idx_mutation_trace_scopes_worktree", "idx_mutation_trace_scopes_worktree_status", - "idx_mutation_trace_processed_events_worktree", ] { assert!( sqlite_object_exists(&db, "index", index), @@ -510,6 +509,56 @@ mod tests { remove_test_db(&db_path); } + #[test] + fn mutation_trace_processed_events_identity_is_scope_and_event_only() { + let db_path = unique_test_db_path("mutation-trace-processed-events-identity"); + let db = RepositoryAgentTraceDb::new_at(&db_path).expect("repository DB should open"); + + let sql = table_sql(&db, "mutation_trace_processed_events"); + assert!( + !sql.contains("worktree_id"), + "mutation_trace_processed_events must not have a worktree_id column: {sql}" + ); + + db.execute( + "INSERT INTO mutation_trace_processed_events (scope_id, event_id) + VALUES ('scope-1', 'event-1')", + (), + ) + .expect("first (scope_id, event_id) insert should succeed"); + + let duplicate_error = db + .execute( + "INSERT INTO mutation_trace_processed_events (scope_id, event_id) + VALUES ('scope-1', 'event-1')", + (), + ) + .expect_err("a duplicate (scope_id, event_id) pair must be rejected"); + assert!( + duplicate_error.to_string().contains("UNIQUE") + || duplicate_error.to_string().contains("PRIMARY KEY"), + "unexpected error: {duplicate_error}" + ); + + db.execute( + "INSERT INTO mutation_trace_processed_events (scope_id, event_id) + VALUES ('scope-2', 'event-1')", + (), + ) + .expect("the same event_id under a different scope_id should be allowed"); + + db.execute( + "INSERT INTO mutation_trace_processed_events (scope_id, event_id) + VALUES ('scope-1', 'event-2')", + (), + ) + .expect("the same scope_id with a different event_id should be allowed"); + + assert_eq!(row_count(&db, "mutation_trace_processed_events"), 3); + + remove_test_db(&db_path); + } + #[test] fn trace_tables_have_no_checkout_id_columns() { let db_path = unique_test_db_path("no-checkout-id"); diff --git a/context/plans/mutation-cursor-store-persistence.md b/context/plans/mutation-cursor-store-persistence.md index 096600c8..4432e784 100644 --- a/context/plans/mutation-cursor-store-persistence.md +++ b/context/plans/mutation-cursor-store-persistence.md @@ -70,8 +70,8 @@ error without retry. - Validate: T08 injected-failure test asserts revision, scope status, processed event, mutation event, and active scopes are all unchanged after rollback. - [ ] AC12: Process restart reconstructs the same durable protocol projection. - Validate: T09 tests that drop and reopen the DB handle before reloading. -- [ ] AC13: Historical mutation events are not loaded on each boundary, terminal (`Closed`/`Abandoned`/`NeverSeen`) historical scopes are not loaded on each boundary unless explicitly referenced, and a referenced scope belonging to a different worktree is rejected rather than silently loaded or reassigned. - - Validate: `MutationTraceStore::load_worktree` issues no query against `mutation_trace_events`, loads only currently `Active` scopes plus the explicitly referenced scope (if any), and returns `Err` when that referenced scope's persisted `worktree_id` does not match the requested worktree (T03 done-when). +- [ ] AC13: Historical mutation events are not loaded on each boundary; terminal (`Closed`/`Abandoned`/`NeverSeen`) historical scopes are not loaded on each boundary unless they are the effective referenced scope (`scope`, or `event_key.scope_id` when `scope` is absent); explicit `scope` and `event_key.scope_id` must agree when both are supplied, or `load_worktree` returns `Err`; and the effective referenced scope must belong to the requested worktree, or `load_worktree` returns `Err` rather than silently loading or reassigning it. + - Validate: `MutationTraceStore::load_worktree` issues no query against `mutation_trace_events`, loads only currently `Active` scopes plus the effective referenced scope (if any) derived from `scope`/`event_key` per T03's four-case definition, returns `Err` when `scope` and `event_key.scope_id` are both supplied and differ, and returns `Err` when the effective referenced scope's persisted `worktree_id` does not match the requested worktree (T03 done-when). - [ ] AC14: Existing Quint Connect and protocol tests remain green. - Validate: `nix flake check` (runs `cli-tests`, including `mutation_trace::mbt`, and the dedicated `mutation-trace-quint-connect` check). - [ ] AC15: No Git/filesystem lock/hook/coordinator integration is added. @@ -121,15 +121,15 @@ Persist this field in every plan; this is durable plan state, not chat state: - [x] T01: `Add migration 003 for mutation-trace protocol tables` (status:done) - Task ID: T01 - - Scope: In — `cli/migrations/agent-trace-repository/003_mutation_trace_protocol.sql` defining `mutation_trace_worktrees` (revision `BLOB` constrained by `CHECK (typeof(revision) = 'blob' AND length(revision) = 8)`), `mutation_trace_scopes` (+ `idx_mutation_trace_scopes_worktree` and a new composite `idx_mutation_trace_scopes_worktree_status` index on `(worktree_id, status)` for the bounded hot-path scope lookup), `mutation_trace_processed_events`, `mutation_trace_events` (+ the same `typeof`/`length` revision `CHECK`, plus payload-consistency `CHECK` constraints), and `mutation_trace_event_active_scopes`. Out — any Rust code consuming these tables (T02+). + - Scope: In — `cli/migrations/agent-trace-repository/003_mutation_trace_protocol.sql` defining `mutation_trace_worktrees` (revision `BLOB` constrained by `CHECK (typeof(revision) = 'blob' AND length(revision) = 8)`), `mutation_trace_scopes` (+ `idx_mutation_trace_scopes_worktree` and a new composite `idx_mutation_trace_scopes_worktree_status` index on `(worktree_id, status)` for the bounded hot-path scope lookup), `mutation_trace_processed_events` (identity `PRIMARY KEY (scope_id, event_id)` only, matching the domain `EventKey`; no `worktree_id` column or index — a scope's worktree is already a permanent fact owned by `mutation_trace_scopes`), `mutation_trace_events` (+ the same `typeof`/`length` revision `CHECK`, plus payload-consistency `CHECK` constraints), and `mutation_trace_event_active_scopes`. Out — any Rust code consuming these tables (T02+). - Dependencies: none - Done when: a fresh `RepositoryAgentTraceDb::new_at` at a clean path applies `001`+`002`+`003` and all five tables exist with the specified columns, constraints, and indexes; a row violating a `CHECK` constraint (for example `ai_exclusive` attribution with a `NULL` `attribution_scope_id`) is rejected; a `TEXT` value of length 8 assigned to a `revision` column is rejected by the `typeof(revision) = 'blob'` check even though its length matches. - Verify: `./scripts/run-cli-cargo.sh test --manifest-path cli/Cargo.toml agent_trace_db::repository::`; a new targeted test asserting the `003` tables, indexes, and constraints (including the TEXT-vs-BLOB revision case) behave as specified. - Completed: 2026-08-27 - Files changed: `cli/migrations/agent-trace-repository/003_mutation_trace_protocol.sql` (new); `cli/src/services/agent_trace_db/repository.rs` - - Result: Added migration `003_mutation_trace_protocol.sql` defining `mutation_trace_worktrees`, `mutation_trace_scopes` (+ `idx_mutation_trace_scopes_worktree`, `idx_mutation_trace_scopes_worktree_status`), `mutation_trace_processed_events` (+ `idx_mutation_trace_processed_events_worktree`), `mutation_trace_events`, and `mutation_trace_event_active_scopes`, all discovered automatically by `build.rs`'s directory scan. Revision columns use `BLOB NOT NULL CHECK (typeof(revision) = 'blob' AND length(revision) = 8)`; enum-shaped columns use `TEXT` with `CHECK (... IN (...))` allow-lists following the existing `role`/`payload_type` convention; `mutation_trace_events` additionally enforces attribution/boundary payload-consistency `CHECK`s (`ai_exclusive` requires a non-null `attribution_scope_id`; hook boundaries require non-null `boundary_scope_id`/`boundary_event_id`, `flush` requires both null). Updated `open_at_initializes_the_full_schema_from_one_migration` to assert the new migration ID and the five new tables/indexes, and added two new targeted tests (`mutation_trace_worktrees_revision_must_be_a_blob_not_matching_length_text`, `mutation_trace_events_ai_exclusive_attribution_requires_a_scope_id`) proving the TEXT-vs-BLOB revision rejection and the `ai_exclusive`-requires-scope rejection, each paired with a positive control insert. - - Verify: `./scripts/run-cli-cargo.sh test --manifest-path cli/Cargo.toml agent_trace_db::repository::` — passed, 18/18 (including the two new tests and the updated baseline-schema test); `./scripts/run-cli-cargo.sh fmt --manifest-path cli/Cargo.toml -- --check` — passed (no diff). - - Done checks: fresh DB applies `001`+`002`+`003` with all five tables/indexes present (verified by the updated baseline test); `ai_exclusive` attribution with a `NULL` `attribution_scope_id` is rejected (verified); an 8-byte TEXT value assigned to `revision` is rejected by `typeof(revision) = 'blob'` (verified); `git diff --exit-code` on `001`/`002` shows zero changes (verified). + - Result: Added migration `003_mutation_trace_protocol.sql` defining `mutation_trace_worktrees`, `mutation_trace_scopes` (+ `idx_mutation_trace_scopes_worktree`, `idx_mutation_trace_scopes_worktree_status`), `mutation_trace_processed_events` (identity `PRIMARY KEY (scope_id, event_id)` only — no `worktree_id` column or index), `mutation_trace_events`, and `mutation_trace_event_active_scopes`, all discovered automatically by `build.rs`'s directory scan. Revision columns use `BLOB NOT NULL CHECK (typeof(revision) = 'blob' AND length(revision) = 8)`; enum-shaped columns use `TEXT` with `CHECK (... IN (...))` allow-lists following the existing `role`/`payload_type` convention; `mutation_trace_events` additionally enforces attribution/boundary payload-consistency `CHECK`s (`ai_exclusive` requires a non-null `attribution_scope_id`; hook boundaries require non-null `boundary_scope_id`/`boundary_event_id`, `flush` requires both null). Updated `open_at_initializes_the_full_schema_from_one_migration` to assert the new migration ID and the five new tables/indexes, and added targeted tests (`mutation_trace_worktrees_revision_must_be_a_blob_not_matching_length_text`, `mutation_trace_events_ai_exclusive_attribution_requires_a_scope_id`, `mutation_trace_processed_events_identity_is_scope_and_event_only`) proving the TEXT-vs-BLOB revision rejection, the `ai_exclusive`-requires-scope rejection, and the `(scope_id, event_id)`-only processed-event identity, each paired with a positive control insert. `mutation_trace_processed_events` originally also carried a `worktree_id` column and `idx_mutation_trace_processed_events_worktree` index; both were removed by a later schema correction since a processed event's identity is exactly `(scope_id, event_id)` and its worktree is already a permanent fact owned by `mutation_trace_scopes`. + - Verify: `./scripts/run-cli-cargo.sh test --manifest-path cli/Cargo.toml agent_trace_db::repository::` — passed, 19/19 (including the corrected baseline-schema test and the three targeted tests above); `./scripts/run-cli-cargo.sh fmt --manifest-path cli/Cargo.toml -- --check` — passed (no diff). + - Done checks: fresh DB applies `001`+`002`+`003` with all five tables/indexes present (verified by the updated baseline test); `ai_exclusive` attribution with a `NULL` `attribution_scope_id` is rejected (verified); an 8-byte TEXT value assigned to `revision` is rejected by `typeof(revision) = 'blob'` (verified); `git diff --exit-code` on `001`/`002` shows zero changes (verified); `mutation_trace_processed_events` has no `worktree_id` column and its identity is exactly `(scope_id, event_id)` (verified). - Context impact: local — additive schema-only migration; no Rust code consumes these new tables yet (T02+ wire codecs, loads, and commits against them). No durable context synchronization is required for this task; the plan's `Context sync` entries are authored by T11 once the full store lands. - Context synchronization: synced @@ -149,9 +149,23 @@ Persist this field in every plan; this is durable plan state, not chat state: - [ ] T03: `Add bounded WorktreeProjection load and cold-path MutationEvent read` (status:todo) - Task ID: T03 - - Scope: In — `WorktreeProjection` (+ `into_protocol_state`) and `MutationTraceStore` wrapping `&RepositoryAgentTraceDb`. `load_worktree(worktree: &WorktreeId, scope: Option<&ScopeId>, event_key: Option<&EventKey>)` loads exactly one worktree row, only its currently `Active` scopes plus the explicitly referenced `scope` when supplied (even when that scope is `NeverSeen`/`Closed`/`Abandoned`), and 0 or 1 matching processed-event row for `event_key`. A separate cold-path `load_mutation_event(worktree: &WorktreeId, revision: u64) -> Result>` reads one `mutation_trace_events` row plus its `mutation_trace_event_active_scopes` rows and reconstructs a complete `MutationEvent`, decoding `Attribution` exactly (including `AiExclusive(scope_id)`) and the complete `Boundary`. When `scope` is supplied and the persisted `ScopeState` for that `ScopeId` has a `worktree_id` different from the requested `worktree`, `load_worktree` returns `Err` — it never silently omits the scope, never includes it in the projection, and never reassigns it to the requested worktree, preserving the permanent `ScopeId` -> `WorktreeId` identity `register_scope` already enforces. Out — initialization/commit logic (T04/T07); calling `load_mutation_event` from `load_worktree` or from any hook-boundary path. + - Scope: In — `WorktreeProjection` (+ `into_protocol_state`) and `MutationTraceStore` wrapping `&RepositoryAgentTraceDb`. `load_worktree(worktree: &WorktreeId, scope: Option<&ScopeId>, event_key: Option<&EventKey>)` first derives one `effective_scope: Option<&ScopeId>` from `scope` and `event_key`. **Invariant:** `scope` and `event_key.scope_id` are two ways of referring to the same operation-local scope identity; when both are supplied they must agree; when only `event_key` is supplied, its `scope_id` becomes the effective referenced scope for projection loading and `WorktreeId` validation. This avoids relying on a separate `worktree_id` stored on processed events. Concretely: + - `scope = None`, `event_key = None` -> `effective_scope = None` (no referenced scope); only the requested worktree's `Active` scopes are loaded, with no extra terminal scope. + - `scope = Some(S)`, `event_key = None` -> `effective_scope = Some(S)`; `S` is loaded and validated exactly as already specified below (included regardless of status; `Err` if it belongs to another worktree; never silently omitted or reassigned). + - `scope = None`, `event_key = Some(K)` -> `effective_scope = Some(&K.scope_id)`. `K.scope_id` is treated as a referenced scope even though the explicit `scope` argument is absent: `load_worktree` loads the durable `ScopeState` for `K.scope_id`, validates its persisted `worktree_id` against the requested worktree, includes it in the projection regardless of status, and returns `Err` if it belongs to another worktree. The processed-event replay lookup is then performed solely by `WHERE scope_id = ? AND event_id = ?` using `K` — `mutation_trace_processed_events` gains no `worktree_id` column to perform this check. + - `scope = Some(S)`, `event_key = Some(K)`, `S == K.scope_id` -> `effective_scope = Some(S)`; that single `ScopeId` is loaded and validated once. + - `scope = Some(S)`, `event_key = Some(K)`, `S != K.scope_id` -> `load_worktree` returns `Err` before loading either scope and before performing the processed-event lookup. It never chooses one arbitrarily, never loads both scopes, never ignores the mismatch, and never performs the replay query anyway. + + `load_worktree` then loads exactly one worktree row, only its currently `Active` scopes plus the scope named by `effective_scope` (even when that scope is `NeverSeen`/`Closed`/`Abandoned`), and — when `event_key` is supplied and no `S != K.scope_id` mismatch already returned `Err` — 0 or 1 matching processed-event row for `event_key`. The processed-event lookup is keyed solely by `event_key`'s `(scope_id, event_id)` — `WHERE scope_id = ? AND event_id = ?`, never filtered or joined by `worktree_id` — since `mutation_trace_processed_events` carries no `worktree_id` column (removed from migration `003`; the table's only identity is `PRIMARY KEY (scope_id, event_id)`, matching the domain `EventKey`). The worktree relationship for `event_key`'s scope is established by loading and validating its durable `ScopeState` as part of `effective_scope` above, not by a `worktree_id` column on the processed-event table: `EventKey.scope_id` -> `mutation_trace_scopes.scope_id` -> `mutation_trace_scopes.worktree_id`. A separate cold-path `load_mutation_event(worktree: &WorktreeId, revision: u64) -> Result>` reads one `mutation_trace_events` row plus its `mutation_trace_event_active_scopes` rows and reconstructs a complete `MutationEvent`, decoding `Attribution` exactly (including `AiExclusive(scope_id)`) and the complete `Boundary`. When `effective_scope` is `Some(S)` and the persisted `ScopeState` for `S` has a `worktree_id` different from the requested `worktree`, `load_worktree` returns `Err` — it never silently omits the scope, never includes it in the projection, and never reassigns it to the requested worktree, preserving the permanent `ScopeId` -> `WorktreeId` identity `register_scope` already enforces. This is the same check whether `S` came from the explicit `scope` argument or from `event_key.scope_id`. Out — initialization/commit logic (T04/T07); calling `load_mutation_event` from `load_worktree` or from any hook-boundary path. - Dependencies: T01, T02 - - Done when: `load_worktree` returns `None` for a missing worktree and `Some(projection)` otherwise, with `scopes` containing every currently `Active` scope on that worktree plus `scope` when supplied regardless of its status, and never a `Closed`/`Abandoned`/`NeverSeen` scope that was not explicitly referenced; `attempts`, `mutation_events`, and `external_taint` stay empty; the method issues no query against `mutation_trace_events`. Specifically: a referenced scope on the requested worktree is included regardless of status; a referenced terminal (`Closed`/`Abandoned`/`NeverSeen`) scope on the requested worktree is included; an unreferenced terminal historical scope is excluded; a referenced scope whose persisted `worktree_id` belongs to a different worktree returns `Err`. `load_mutation_event` returns `None` when no row exists at that `(worktree, revision)` and otherwise reconstructs a `MutationEvent` whose `before_tree`/`after_tree`/`revision`/`tainted`/`failure_kind`/`attribution`/`boundary`/`active_scopes` exactly match what `store.commit` persisted. + - Done when: `load_worktree` returns `None` for a missing worktree and `Some(projection)` otherwise, with `scopes` containing every currently `Active` scope on that worktree plus the `effective_scope` (derived from `scope`/`event_key` per the four cases above) when one exists, regardless of its status, and never a `Closed`/`Abandoned`/`NeverSeen` scope that was not the effective referenced scope; `attempts`, `mutation_events`, and `external_taint` stay empty; the method issues no query against `mutation_trace_events`. Explicit test cases for all five `scope`/`event_key` combinations: + 1. `scope=None`, `event_key=None` -> only currently `Active` scopes on the requested worktree are loaded; no referenced scope. + 2. `scope=Some(S)`, `event_key=None` -> `S` is included as the effective referenced scope and validated; a wrong-worktree `S` returns `Err`. + 3. `scope=None`, `event_key=Some(K)` -> `K.scope_id` is loaded as the effective referenced scope; a wrong-worktree `K.scope_id` returns `Err`; the processed-event lookup for `K` still matches solely on `(scope_id, event_id)`. + 4. `scope=Some(S)`, `event_key=Some(K)`, `S == K.scope_id` -> succeeds, loading and validating that one `ScopeId` exactly once. + 5. `scope=Some(S)`, `event_key=Some(K)`, `S != K.scope_id` -> `load_worktree` returns `Err` without loading either scope and without performing the processed-event lookup. + + Also preserved: a referenced scope on the requested worktree is included regardless of status; a referenced terminal (`Closed`/`Abandoned`/`NeverSeen`) scope on the requested worktree is included; an unreferenced terminal historical scope is excluded; the processed-event query never references `worktree_id` (it has no such column) and matches solely on `scope_id`/`event_id`; `load_worktree` never queries historical `mutation_trace_events` rows. `load_mutation_event` returns `None` when no row exists at that `(worktree, revision)` and otherwise reconstructs a `MutationEvent` whose `before_tree`/`after_tree`/`revision`/`tainted`/`failure_kind`/`attribution`/`boundary`/`active_scopes` exactly match what `store.commit` persisted. - Verify: `./scripts/run-cli-cargo.sh test --manifest-path cli/Cargo.toml mutation_trace::store::` - Context synchronization: pending From 4754c8cbee79f2aa9c2e3c4de7ef82f16dde443f Mon Sep 17 00:00:00 2001 From: David Abram Date: Thu, 27 Aug 2026 14:35:43 +0200 Subject: [PATCH 05/15] mutation-trace: Add bounded store read projections Load one worktree's active and referenced scopes plus processed-event state without querying historical events on the hot path, and reconstruct cold-path mutation events with full attribution and boundary data. Preserve transient protocol fields as empty in widened projections and reject inconsistent persisted payloads. Plan: mutation-cursor-store-persistence (T03) Co-authored-by: SCE --- cli/src/services/mutation_trace/store.rs | 948 +++++++++++++++++- .../mutation-cursor-store-persistence.md | 22 +- 2 files changed, 957 insertions(+), 13 deletions(-) diff --git a/cli/src/services/mutation_trace/store.rs b/cli/src/services/mutation_trace/store.rs index fefa69de..345f3b17 100644 --- a/cli/src/services/mutation_trace/store.rs +++ b/cli/src/services/mutation_trace/store.rs @@ -1,19 +1,29 @@ -//! Domain<->SQL codecs for the mutation-cursor persistence layer. +//! Domain<->SQL codecs and bounded read access for the mutation-cursor +//! persistence layer. //! -//! These codecs are the only translation between `super::types` domain -//! values and the `TEXT`/`BLOB` representations `cli/migrations/agent-trace- +//! The codecs are the only translation between `super::types` domain values +//! and the `TEXT`/`BLOB` representations `cli/migrations/agent-trace- //! repository/003_mutation_trace_protocol.sql` constrains those columns to. //! Every codec here is an explicit function over a fixed set of variants — no //! codec derives from `Debug` or a serde representation, so a variant rename //! cannot silently change the durable encoding. //! -//! This module carries no query, projection, or commit logic yet (see -//! `mutation-cursor-store-persistence` plan tasks T03+); it only establishes -//! the byte- and string-level encodings later tasks build on. +//! `MutationTraceStore` adds the hot-path bounded worktree read +//! (`load_worktree`) and the cold-path historical read (`load_mutation_event`) +//! against a `&RepositoryAgentTraceDb`. Initialization and CAS-commit logic +//! are later tasks (`mutation-cursor-store-persistence` T04/T06/T07); this +//! module carries no such logic yet. -use anyhow::{bail, Result}; +use std::collections::{BTreeMap, BTreeSet}; -use super::types::{ActorKind, Attribution, Boundary, FailureKind, ScopeStatus}; +use anyhow::{bail, Context, Result}; + +use crate::services::agent_trace_db::repository::RepositoryAgentTraceDb; + +use super::types::{ + ActorKind, Attribution, Boundary, EventId, EventKey, FailureKind, MutationEvent, ProtocolState, + ScopeId, ScopeState, ScopeStatus, TreeId, WorktreeId, WorktreeState, +}; /// Encodes a worktree/event revision as the 8-byte big-endian `BLOB` stored /// by every `revision` column in migration `003`. @@ -181,6 +191,417 @@ pub fn decode_boundary_kind(value: &str) -> Result { } } +const SELECT_WORKTREE_SQL: &str = + "SELECT cursor_tree, revision, tainted, failure_kind, needs_rebaseline + FROM mutation_trace_worktrees WHERE worktree_id = ?1"; +const SELECT_SCOPES_BY_WORKTREE_AND_STATUS_SQL: &str = + "SELECT scope_id, worktree_id, actor_kind, status + FROM mutation_trace_scopes WHERE worktree_id = ?1 AND status = ?2"; +const SELECT_SCOPE_BY_ID_SQL: &str = "SELECT scope_id, worktree_id, actor_kind, status + FROM mutation_trace_scopes WHERE scope_id = ?1"; +const SELECT_PROCESSED_EVENT_SQL: &str = + "SELECT 1 FROM mutation_trace_processed_events WHERE scope_id = ?1 AND event_id = ?2"; +const SELECT_MUTATION_EVENT_SQL: &str = "SELECT before_tree, after_tree, tainted, failure_kind, + attribution_kind, attribution_scope_id, boundary_kind, boundary_scope_id, boundary_event_id + FROM mutation_trace_events WHERE worktree_id = ?1 AND revision = ?2"; +const SELECT_MUTATION_EVENT_ACTIVE_SCOPES_SQL: &str = + "SELECT scope_id FROM mutation_trace_event_active_scopes WHERE worktree_id = ?1 AND revision = ?2"; + +/// Bounded runtime projection of one worktree's durable protocol state, +/// loaded by [`MutationTraceStore::load_worktree`]. Scoped to that worktree's +/// currently `Active` scopes plus, when present, the scope `load_worktree` +/// was explicitly asked about (regardless of its status) — never every +/// historical scope, and never a `mutation_trace_events` row. +/// +/// `attempts`, `mutation_events`, and `external_taint` are always empty: +/// `AttemptState` is transient and never persisted, historical +/// `MutationEvent`s are a cold-path concern +/// ([`MutationTraceStore::load_mutation_event`]), and `external_taint` is +/// never DB-authoritative (see the plan's non-goals). +#[derive(Clone, Debug, Eq, PartialEq)] +pub struct WorktreeProjection { + pub worktree_id: WorktreeId, + pub worktree_state: WorktreeState, + pub scopes: BTreeMap, + pub processed_events: BTreeSet, +} + +impl WorktreeProjection { + /// Widens this bounded projection into a full [`ProtocolState`] so pure + /// `protocol.rs` functions can operate on it unchanged. `worktrees` + /// carries only the one loaded worktree; `attempts`, `mutation_events`, + /// and `external_taint` are always empty. + pub fn into_protocol_state(self) -> ProtocolState { + let mut worktrees = BTreeMap::new(); + worktrees.insert(self.worktree_id, self.worktree_state); + + ProtocolState { + worktrees, + scopes: self.scopes, + external_taint: BTreeSet::new(), + processed_events: self.processed_events, + attempts: BTreeMap::new(), + mutation_events: BTreeSet::new(), + } + } +} + +/// Bounded read access to the durable mutation-cursor protocol state for one +/// repository, via [`RepositoryAgentTraceDb`]. Write/CAS-commit access is +/// added by later tasks (T04/T06/T07). +pub struct MutationTraceStore<'a> { + db: &'a RepositoryAgentTraceDb, +} + +impl<'a> MutationTraceStore<'a> { + pub fn new(db: &'a RepositoryAgentTraceDb) -> Self { + Self { db } + } + + /// Loads a bounded projection of `worktree`'s durable protocol state, or + /// `None` when the worktree does not exist. + /// + /// `scope` and `event_key.scope_id` are two ways of naming the same + /// operation-local scope identity: when both are supplied they must + /// agree, or this returns `Err` before loading or querying anything. + /// Otherwise the supplied `scope`, or `event_key.scope_id` when only + /// `event_key` is supplied, becomes the effective referenced scope: a + /// durable `mutation_trace_scopes` row for it must exist, or this returns + /// `Err` — a missing effective scope is never silently omitted from the + /// projection. When it exists it is loaded and included in the + /// projection regardless of its status, and this returns `Err` if it + /// belongs to a worktree other than the one requested. Both checks run + /// before the `processed_events` replay lookup, so an orphan + /// `mutation_trace_processed_events` row can never enter the projection + /// without its owning scope. The projection's `scopes` otherwise contains + /// only this worktree's currently `Active` scopes. `processed_events` + /// contains `event_key` only when a matching `(scope_id, event_id)` row + /// already exists; the lookup never references a `worktree_id` column, + /// since `mutation_trace_processed_events` has none. This method never + /// queries `mutation_trace_events`. + pub fn load_worktree( + &self, + worktree: &WorktreeId, + scope: Option<&ScopeId>, + event_key: Option<&EventKey>, + ) -> Result> { + let effective_scope = effective_referenced_scope(scope, event_key)?; + + let Some(worktree_state) = self.load_worktree_state(worktree)? else { + return Ok(None); + }; + + let mut scopes = self.load_active_scopes(worktree)?; + + if let Some(effective_scope_id) = effective_scope { + if !scopes.contains_key(effective_scope_id) { + let scope_state = self.load_scope(effective_scope_id)?.ok_or_else(|| { + anyhow::anyhow!( + "effective referenced scope {effective_scope_id:?} has no mutation_trace_scopes row" + ) + })?; + if scope_state.worktree_id != *worktree { + bail!( + "scope {:?} belongs to worktree {:?}, not the requested worktree {:?}", + effective_scope_id, + scope_state.worktree_id, + worktree + ); + } + scopes.insert(effective_scope_id.clone(), scope_state); + } + } + + let processed_events = match event_key { + Some(event_key) if self.processed_event_exists(event_key)? => { + let mut processed_events = BTreeSet::new(); + processed_events.insert(event_key.clone()); + processed_events + } + _ => BTreeSet::new(), + }; + + Ok(Some(WorktreeProjection { + worktree_id: worktree.clone(), + worktree_state, + scopes, + processed_events, + })) + } + + /// Reconstructs one historical [`MutationEvent`] for `(worktree, + /// revision)`, decoding its full `Attribution` and `Boundary`, or `None` + /// when no such row exists. Never called from `load_worktree` or from + /// any hook-boundary path. + pub fn load_mutation_event( + &self, + worktree: &WorktreeId, + revision: u64, + ) -> Result> { + let revision_blob = encode_revision(revision); + + let rows = self.db.query_map( + SELECT_MUTATION_EVENT_SQL, + (worktree.0.as_str(), revision_blob.as_slice()), + mutation_event_row_from_turso, + )?; + + let Some(row) = rows.into_iter().next() else { + return Ok(None); + }; + + let active_scopes = self.load_mutation_event_active_scopes(worktree, &revision_blob)?; + + Ok(Some(MutationEvent { + worktree_id: worktree.clone(), + revision, + before_tree: TreeId(row.before_tree), + after_tree: TreeId(row.after_tree), + active_scopes, + tainted: row.tainted, + failure_kind: row.failure_kind, + attribution: reconstruct_attribution(row.attribution_kind, row.attribution_scope_id)?, + boundary: reconstruct_boundary( + row.boundary_kind, + worktree, + row.boundary_scope_id, + row.boundary_event_id, + )?, + })) + } + + fn load_worktree_state(&self, worktree: &WorktreeId) -> Result> { + let rows = self.db.query_map( + SELECT_WORKTREE_SQL, + (worktree.0.as_str(),), + worktree_state_row_from_turso, + )?; + + Ok(rows.into_iter().next()) + } + + fn load_active_scopes(&self, worktree: &WorktreeId) -> Result> { + let rows = self.db.query_map( + SELECT_SCOPES_BY_WORKTREE_AND_STATUS_SQL, + ( + worktree.0.as_str(), + encode_scope_status(ScopeStatus::Active), + ), + scope_row_from_turso, + )?; + + Ok(rows.into_iter().collect()) + } + + fn load_scope(&self, scope_id: &ScopeId) -> Result> { + let rows = self.db.query_map( + SELECT_SCOPE_BY_ID_SQL, + (scope_id.0.as_str(),), + scope_row_from_turso, + )?; + + Ok(rows.into_iter().next().map(|(_, scope_state)| scope_state)) + } + + fn processed_event_exists(&self, event_key: &EventKey) -> Result { + let rows = self.db.query_map( + SELECT_PROCESSED_EVENT_SQL, + (event_key.scope_id.0.as_str(), event_key.event_id.0.as_str()), + |row| row.get::(0).map_err(Into::into), + )?; + + Ok(!rows.is_empty()) + } + + fn load_mutation_event_active_scopes( + &self, + worktree: &WorktreeId, + revision_blob: &[u8], + ) -> Result> { + let rows = self.db.query_map( + SELECT_MUTATION_EVENT_ACTIVE_SCOPES_SQL, + (worktree.0.as_str(), revision_blob), + |row| row.get::(0).map(ScopeId).map_err(Into::into), + )?; + + Ok(rows.into_iter().collect()) + } +} + +/// Derives the single effective referenced scope from `scope` and +/// `event_key`, per the four-case definition in the +/// `mutation-cursor-store-persistence` plan's T03: `None` when neither is +/// supplied; the supplied one when only one is; the agreeing identity when +/// both are supplied and equal; `Err` when both are supplied and disagree. +fn effective_referenced_scope<'k>( + scope: Option<&'k ScopeId>, + event_key: Option<&'k EventKey>, +) -> Result> { + match (scope, event_key) { + (None, None) => Ok(None), + (Some(scope_id), None) => Ok(Some(scope_id)), + (None, Some(event_key)) => Ok(Some(&event_key.scope_id)), + (Some(scope_id), Some(event_key)) if *scope_id == event_key.scope_id => Ok(Some(scope_id)), + (Some(scope_id), Some(event_key)) => bail!( + "scope {scope_id:?} and event_key.scope_id {:?} disagree", + event_key.scope_id + ), + } +} + +fn worktree_state_row_from_turso(row: &turso::Row) -> Result { + let cursor_tree: String = row + .get(0) + .context("failed to read mutation_trace_worktrees.cursor_tree")?; + let revision_blob: Vec = row + .get(1) + .context("failed to read mutation_trace_worktrees.revision")?; + let tainted: bool = row + .get(2) + .context("failed to read mutation_trace_worktrees.tainted")?; + let failure_kind: String = row + .get(3) + .context("failed to read mutation_trace_worktrees.failure_kind")?; + let needs_rebaseline: bool = row + .get(4) + .context("failed to read mutation_trace_worktrees.needs_rebaseline")?; + + Ok(WorktreeState { + cursor_tree: TreeId(cursor_tree), + revision: decode_revision(&revision_blob)?, + tainted, + failure_kind: decode_failure_kind(&failure_kind)?, + needs_rebaseline, + }) +} + +fn scope_row_from_turso(row: &turso::Row) -> Result<(ScopeId, ScopeState)> { + let scope_id: String = row + .get(0) + .context("failed to read mutation_trace_scopes.scope_id")?; + let worktree_id: String = row + .get(1) + .context("failed to read mutation_trace_scopes.worktree_id")?; + let actor_kind: String = row + .get(2) + .context("failed to read mutation_trace_scopes.actor_kind")?; + let status: String = row + .get(3) + .context("failed to read mutation_trace_scopes.status")?; + + Ok(( + ScopeId(scope_id), + ScopeState { + status: decode_scope_status(&status)?, + actor_kind: decode_actor_kind(&actor_kind)?, + worktree_id: WorktreeId(worktree_id), + }, + )) +} + +/// Raw decoded `mutation_trace_events` row fields, prior to reconstructing +/// the full `Attribution`/`Boundary`/`active_scopes` a [`MutationEvent`] +/// carries. +struct MutationEventRow { + before_tree: String, + after_tree: String, + tainted: bool, + failure_kind: FailureKind, + attribution_kind: AttributionKind, + attribution_scope_id: Option, + boundary_kind: BoundaryKind, + boundary_scope_id: Option, + boundary_event_id: Option, +} + +fn mutation_event_row_from_turso(row: &turso::Row) -> Result { + let before_tree: String = row + .get(0) + .context("failed to read mutation_trace_events.before_tree")?; + let after_tree: String = row + .get(1) + .context("failed to read mutation_trace_events.after_tree")?; + let tainted: bool = row + .get(2) + .context("failed to read mutation_trace_events.tainted")?; + let failure_kind: String = row + .get(3) + .context("failed to read mutation_trace_events.failure_kind")?; + let attribution_kind: String = row + .get(4) + .context("failed to read mutation_trace_events.attribution_kind")?; + let attribution_scope_id: Option = row + .get(5) + .context("failed to read mutation_trace_events.attribution_scope_id")?; + let boundary_kind: String = row + .get(6) + .context("failed to read mutation_trace_events.boundary_kind")?; + let boundary_scope_id: Option = row + .get(7) + .context("failed to read mutation_trace_events.boundary_scope_id")?; + let boundary_event_id: Option = row + .get(8) + .context("failed to read mutation_trace_events.boundary_event_id")?; + + Ok(MutationEventRow { + before_tree, + after_tree, + tainted, + failure_kind: decode_failure_kind(&failure_kind)?, + attribution_kind: decode_attribution_kind(&attribution_kind)?, + attribution_scope_id, + boundary_kind: decode_boundary_kind(&boundary_kind)?, + boundary_scope_id, + boundary_event_id, + }) +} + +fn reconstruct_attribution(kind: AttributionKind, scope_id: Option) -> Result { + match (kind, scope_id) { + (AttributionKind::IneligibleUnscoped, None) => Ok(Attribution::IneligibleUnscoped), + (AttributionKind::AiContended, None) => Ok(Attribution::AiContended), + (AttributionKind::AiExclusive, Some(scope_id)) => { + Ok(Attribution::AiExclusive(ScopeId(scope_id))) + } + (kind, scope_id) => { + bail!("inconsistent attribution row: kind={kind:?} scope_id={scope_id:?}") + } + } +} + +fn reconstruct_boundary( + kind: BoundaryKind, + worktree: &WorktreeId, + scope_id: Option, + event_id: Option, +) -> Result { + match kind { + BoundaryKind::Flush => { + if scope_id.is_some() || event_id.is_some() { + bail!("flush boundary row must not carry boundary_scope_id/boundary_event_id"); + } + Ok(Boundary::Flush { + worktree: worktree.clone(), + }) + } + BoundaryKind::Start | BoundaryKind::Advance | BoundaryKind::Close => { + let scope = scope_id + .map(ScopeId) + .ok_or_else(|| anyhow::anyhow!("hook boundary row missing boundary_scope_id"))?; + let event = event_id + .map(EventId) + .ok_or_else(|| anyhow::anyhow!("hook boundary row missing boundary_event_id"))?; + + Ok(match kind { + BoundaryKind::Start => Boundary::Start { scope, event }, + BoundaryKind::Advance => Boundary::Advance { scope, event }, + BoundaryKind::Close => Boundary::Close { scope, event }, + BoundaryKind::Flush => unreachable!("Flush handled above"), + }) + } + } +} + #[cfg(test)] mod tests { use super::*; @@ -299,4 +720,515 @@ mod tests { fn decode_boundary_kind_rejects_unknown_value() { assert!(decode_boundary_kind("unknown").is_err()); } + + fn unique_test_db_path(label: &str) -> std::path::PathBuf { + let nonce = std::time::SystemTime::now() + .duration_since(std::time::UNIX_EPOCH) + .expect("system time should be after Unix epoch") + .as_nanos(); + std::env::temp_dir() + .join(format!( + "sce-mutation-trace-store-{label}-{}-{nonce}", + std::process::id() + )) + .join("agent-trace.db") + } + + fn remove_test_db(db_path: &std::path::Path) { + if let Some(parent) = db_path.parent() { + std::fs::remove_dir_all(parent).expect("test DB directory should be removed"); + } + } + + fn insert_worktree(db: &RepositoryAgentTraceDb, worktree_id: &str, revision: u64) { + db.execute( + "INSERT INTO mutation_trace_worktrees + (worktree_id, cursor_tree, revision, tainted, failure_kind, needs_rebaseline) + VALUES (?1, 'tree-0', ?2, 0, 'healthy', 0)", + (worktree_id, encode_revision(revision).as_slice()), + ) + .expect("worktree insert should succeed"); + } + + fn insert_scope( + db: &RepositoryAgentTraceDb, + scope_id: &str, + worktree_id: &str, + status: ScopeStatus, + ) { + db.execute( + "INSERT INTO mutation_trace_scopes (scope_id, worktree_id, actor_kind, status) + VALUES (?1, ?2, 'claude_code', ?3)", + (scope_id, worktree_id, encode_scope_status(status)), + ) + .expect("scope insert should succeed"); + } + + fn insert_processed_event(db: &RepositoryAgentTraceDb, scope_id: &str, event_id: &str) { + db.execute( + "INSERT INTO mutation_trace_processed_events (scope_id, event_id) VALUES (?1, ?2)", + (scope_id, event_id), + ) + .expect("processed-event insert should succeed"); + } + + #[allow(clippy::too_many_arguments)] + fn insert_mutation_event( + db: &RepositoryAgentTraceDb, + worktree_id: &str, + revision: u64, + before_tree: &str, + after_tree: &str, + attribution_kind: &str, + attribution_scope_id: Option<&str>, + boundary_kind: &str, + boundary_scope_id: Option<&str>, + boundary_event_id: Option<&str>, + active_scopes: &[&str], + ) { + let revision_blob = encode_revision(revision); + + db.execute( + "INSERT INTO mutation_trace_events + (worktree_id, revision, before_tree, after_tree, tainted, failure_kind, + attribution_kind, attribution_scope_id, boundary_kind, boundary_scope_id, boundary_event_id) + VALUES (?1, ?2, ?3, ?4, 0, 'healthy', ?5, ?6, ?7, ?8, ?9)", + ( + worktree_id, + revision_blob.as_slice(), + before_tree, + after_tree, + attribution_kind, + attribution_scope_id, + boundary_kind, + boundary_scope_id, + boundary_event_id, + ), + ) + .expect("mutation event insert should succeed"); + + for scope_id in active_scopes { + db.execute( + "INSERT INTO mutation_trace_event_active_scopes (worktree_id, revision, scope_id) + VALUES (?1, ?2, ?3)", + (worktree_id, revision_blob.as_slice(), *scope_id), + ) + .expect("active-scope insert should succeed"); + } + } + + #[test] + fn load_worktree_returns_none_for_a_missing_worktree() { + let db_path = unique_test_db_path("missing-worktree"); + let db = RepositoryAgentTraceDb::new_at(&db_path).expect("repository DB should open"); + let store = MutationTraceStore::new(&db); + + let projection = store + .load_worktree(&WorktreeId("wt-missing".to_string()), None, None) + .expect("load_worktree should succeed"); + assert!(projection.is_none()); + + remove_test_db(&db_path); + } + + #[test] + fn load_worktree_with_no_scope_or_event_key_loads_only_active_scopes() { + let db_path = unique_test_db_path("case-1-active-only"); + let db = RepositoryAgentTraceDb::new_at(&db_path).expect("repository DB should open"); + let store = MutationTraceStore::new(&db); + + insert_worktree(&db, "wt-1", 5); + insert_scope(&db, "scope-active", "wt-1", ScopeStatus::Active); + insert_scope(&db, "scope-closed", "wt-1", ScopeStatus::Closed); + + let projection = store + .load_worktree(&WorktreeId("wt-1".to_string()), None, None) + .expect("load_worktree should succeed") + .expect("worktree should exist"); + + assert_eq!(projection.worktree_id, WorktreeId("wt-1".to_string())); + assert_eq!(projection.worktree_state.revision, 5); + assert_eq!( + projection.scopes.keys().collect::>(), + vec![&ScopeId("scope-active".to_string())] + ); + assert!(projection.processed_events.is_empty()); + + remove_test_db(&db_path); + } + + #[test] + fn load_worktree_with_explicit_scope_includes_it_regardless_of_status() { + let db_path = unique_test_db_path("case-2-explicit-scope"); + let db = RepositoryAgentTraceDb::new_at(&db_path).expect("repository DB should open"); + let store = MutationTraceStore::new(&db); + + insert_worktree(&db, "wt-1", 0); + insert_scope(&db, "scope-closed", "wt-1", ScopeStatus::Closed); + + let projection = store + .load_worktree( + &WorktreeId("wt-1".to_string()), + Some(&ScopeId("scope-closed".to_string())), + None, + ) + .expect("load_worktree should succeed") + .expect("worktree should exist"); + + assert_eq!( + projection.scopes.get(&ScopeId("scope-closed".to_string())), + Some(&ScopeState { + status: ScopeStatus::Closed, + actor_kind: ActorKind::ClaudeCode, + worktree_id: WorktreeId("wt-1".to_string()), + }) + ); + + remove_test_db(&db_path); + } + + #[test] + fn load_worktree_with_explicit_scope_on_another_worktree_errors() { + let db_path = unique_test_db_path("case-2-wrong-worktree"); + let db = RepositoryAgentTraceDb::new_at(&db_path).expect("repository DB should open"); + let store = MutationTraceStore::new(&db); + + insert_worktree(&db, "wt-1", 0); + insert_worktree(&db, "wt-2", 0); + insert_scope(&db, "scope-1", "wt-2", ScopeStatus::Active); + + let error = store + .load_worktree( + &WorktreeId("wt-1".to_string()), + Some(&ScopeId("scope-1".to_string())), + None, + ) + .expect_err("scope belonging to another worktree should error"); + assert!(error.to_string().contains("scope-1")); + + remove_test_db(&db_path); + } + + #[test] + fn load_worktree_with_explicit_missing_scope_errors() { + let db_path = unique_test_db_path("case-2-missing-scope"); + let db = RepositoryAgentTraceDb::new_at(&db_path).expect("repository DB should open"); + let store = MutationTraceStore::new(&db); + + insert_worktree(&db, "wt-1", 0); + + let error = store + .load_worktree( + &WorktreeId("wt-1".to_string()), + Some(&ScopeId("scope-missing".to_string())), + None, + ) + .expect_err("missing effective scope should error"); + assert!(error.to_string().contains("scope-missing")); + + remove_test_db(&db_path); + } + + #[test] + fn load_worktree_with_only_event_key_loads_its_scope_and_replay_row() { + let db_path = unique_test_db_path("case-3-event-key-only"); + let db = RepositoryAgentTraceDb::new_at(&db_path).expect("repository DB should open"); + let store = MutationTraceStore::new(&db); + + insert_worktree(&db, "wt-1", 0); + insert_scope(&db, "scope-1", "wt-1", ScopeStatus::NeverSeen); + insert_processed_event(&db, "scope-1", "event-1"); + + let event_key = EventKey { + scope_id: ScopeId("scope-1".to_string()), + event_id: EventId("event-1".to_string()), + }; + + let projection = store + .load_worktree(&WorktreeId("wt-1".to_string()), None, Some(&event_key)) + .expect("load_worktree should succeed") + .expect("worktree should exist"); + + assert_eq!( + projection + .scopes + .get(&ScopeId("scope-1".to_string())) + .map(|s| s.status), + Some(ScopeStatus::NeverSeen) + ); + assert_eq!( + projection.processed_events, + [event_key].into_iter().collect() + ); + + remove_test_db(&db_path); + } + + #[test] + fn load_worktree_with_event_key_scope_on_another_worktree_errors() { + let db_path = unique_test_db_path("case-3-wrong-worktree"); + let db = RepositoryAgentTraceDb::new_at(&db_path).expect("repository DB should open"); + let store = MutationTraceStore::new(&db); + + insert_worktree(&db, "wt-1", 0); + insert_worktree(&db, "wt-2", 0); + insert_scope(&db, "scope-1", "wt-2", ScopeStatus::Active); + + let event_key = EventKey { + scope_id: ScopeId("scope-1".to_string()), + event_id: EventId("event-1".to_string()), + }; + + let error = store + .load_worktree(&WorktreeId("wt-1".to_string()), None, Some(&event_key)) + .expect_err("event_key scope on another worktree should error"); + assert!(error.to_string().contains("scope-1")); + + remove_test_db(&db_path); + } + + #[test] + fn load_worktree_with_event_key_missing_scope_errors() { + let db_path = unique_test_db_path("case-3-missing-scope"); + let db = RepositoryAgentTraceDb::new_at(&db_path).expect("repository DB should open"); + let store = MutationTraceStore::new(&db); + + insert_worktree(&db, "wt-1", 0); + + let event_key = EventKey { + scope_id: ScopeId("scope-missing".to_string()), + event_id: EventId("event-1".to_string()), + }; + + let error = store + .load_worktree(&WorktreeId("wt-1".to_string()), None, Some(&event_key)) + .expect_err("missing event_key.scope_id should error"); + assert!(error.to_string().contains("scope-missing")); + + remove_test_db(&db_path); + } + + #[test] + fn load_worktree_with_event_key_missing_scope_and_orphan_replay_row_errors() { + let db_path = unique_test_db_path("case-3-orphan-replay-row"); + let db = RepositoryAgentTraceDb::new_at(&db_path).expect("repository DB should open"); + let store = MutationTraceStore::new(&db); + + insert_worktree(&db, "wt-1", 0); + insert_processed_event(&db, "scope-missing", "event-1"); + + let event_key = EventKey { + scope_id: ScopeId("scope-missing".to_string()), + event_id: EventId("event-1".to_string()), + }; + + let error = store + .load_worktree(&WorktreeId("wt-1".to_string()), None, Some(&event_key)) + .expect_err( + "an orphan processed-event row must not let a missing scope produce a projection", + ); + assert!(error.to_string().contains("scope-missing")); + + remove_test_db(&db_path); + } + + #[test] + fn load_worktree_with_agreeing_scope_and_event_key_loads_it_once() { + let db_path = unique_test_db_path("case-4-agreeing"); + let db = RepositoryAgentTraceDb::new_at(&db_path).expect("repository DB should open"); + let store = MutationTraceStore::new(&db); + + insert_worktree(&db, "wt-1", 0); + insert_scope(&db, "scope-1", "wt-1", ScopeStatus::Active); + + let event_key = EventKey { + scope_id: ScopeId("scope-1".to_string()), + event_id: EventId("event-1".to_string()), + }; + + let projection = store + .load_worktree( + &WorktreeId("wt-1".to_string()), + Some(&ScopeId("scope-1".to_string())), + Some(&event_key), + ) + .expect("load_worktree should succeed") + .expect("worktree should exist"); + + assert_eq!(projection.scopes.len(), 1); + assert!(projection + .scopes + .contains_key(&ScopeId("scope-1".to_string()))); + + remove_test_db(&db_path); + } + + #[test] + fn load_worktree_with_disagreeing_scope_and_event_key_errors_without_loading() { + let db_path = unique_test_db_path("case-5-disagreeing"); + let db = RepositoryAgentTraceDb::new_at(&db_path).expect("repository DB should open"); + let store = MutationTraceStore::new(&db); + + insert_worktree(&db, "wt-1", 0); + insert_scope(&db, "scope-a", "wt-1", ScopeStatus::Active); + insert_scope(&db, "scope-b", "wt-1", ScopeStatus::Active); + + let event_key = EventKey { + scope_id: ScopeId("scope-b".to_string()), + event_id: EventId("event-1".to_string()), + }; + + let error = store + .load_worktree( + &WorktreeId("wt-1".to_string()), + Some(&ScopeId("scope-a".to_string())), + Some(&event_key), + ) + .expect_err("disagreeing scope/event_key.scope_id should error"); + assert!(error.to_string().contains("scope-a")); + assert!(error.to_string().contains("scope-b")); + + remove_test_db(&db_path); + } + + #[test] + fn load_mutation_event_returns_none_when_missing() { + let db_path = unique_test_db_path("cold-path-missing"); + let db = RepositoryAgentTraceDb::new_at(&db_path).expect("repository DB should open"); + let store = MutationTraceStore::new(&db); + + let event = store + .load_mutation_event(&WorktreeId("wt-1".to_string()), 1) + .expect("load_mutation_event should succeed"); + assert!(event.is_none()); + + remove_test_db(&db_path); + } + + #[test] + fn load_mutation_event_reconstructs_ai_exclusive_start_event() { + let db_path = unique_test_db_path("cold-path-ai-exclusive-start"); + let db = RepositoryAgentTraceDb::new_at(&db_path).expect("repository DB should open"); + let store = MutationTraceStore::new(&db); + + insert_mutation_event( + &db, + "wt-1", + 1, + "tree-0", + "tree-1", + "ai_exclusive", + Some("scope-1"), + "start", + Some("scope-1"), + Some("event-1"), + &["scope-1"], + ); + + let event = store + .load_mutation_event(&WorktreeId("wt-1".to_string()), 1) + .expect("load_mutation_event should succeed") + .expect("mutation event row should exist"); + + assert_eq!( + event, + MutationEvent { + worktree_id: WorktreeId("wt-1".to_string()), + revision: 1, + before_tree: TreeId("tree-0".to_string()), + after_tree: TreeId("tree-1".to_string()), + active_scopes: [ScopeId("scope-1".to_string())].into_iter().collect(), + tainted: false, + failure_kind: FailureKind::Healthy, + attribution: Attribution::AiExclusive(ScopeId("scope-1".to_string())), + boundary: Boundary::Start { + scope: ScopeId("scope-1".to_string()), + event: EventId("event-1".to_string()), + }, + } + ); + + remove_test_db(&db_path); + } + + #[test] + fn load_mutation_event_reconstructs_a_flush_event_with_multiple_active_scopes() { + let db_path = unique_test_db_path("cold-path-flush"); + let db = RepositoryAgentTraceDb::new_at(&db_path).expect("repository DB should open"); + let store = MutationTraceStore::new(&db); + + insert_mutation_event( + &db, + "wt-1", + 3, + "tree-2", + "tree-3", + "ai_contended", + None, + "flush", + None, + None, + &["scope-1", "scope-2"], + ); + + let event = store + .load_mutation_event(&WorktreeId("wt-1".to_string()), 3) + .expect("load_mutation_event should succeed") + .expect("mutation event row should exist"); + + assert_eq!( + event, + MutationEvent { + worktree_id: WorktreeId("wt-1".to_string()), + revision: 3, + before_tree: TreeId("tree-2".to_string()), + after_tree: TreeId("tree-3".to_string()), + active_scopes: [ + ScopeId("scope-1".to_string()), + ScopeId("scope-2".to_string()) + ] + .into_iter() + .collect(), + tainted: false, + failure_kind: FailureKind::Healthy, + attribution: Attribution::AiContended, + boundary: Boundary::Flush { + worktree: WorktreeId("wt-1".to_string()), + }, + } + ); + + remove_test_db(&db_path); + } + + #[test] + fn into_protocol_state_carries_only_the_loaded_worktree_and_leaves_transient_fields_empty() { + let db_path = unique_test_db_path("into-protocol-state"); + let db = RepositoryAgentTraceDb::new_at(&db_path).expect("repository DB should open"); + let store = MutationTraceStore::new(&db); + + insert_worktree(&db, "wt-1", 7); + insert_scope(&db, "scope-1", "wt-1", ScopeStatus::Active); + + let projection = store + .load_worktree(&WorktreeId("wt-1".to_string()), None, None) + .expect("load_worktree should succeed") + .expect("worktree should exist"); + + let protocol_state = projection.into_protocol_state(); + + assert_eq!(protocol_state.worktrees.len(), 1); + assert_eq!( + protocol_state + .worktrees + .get(&WorktreeId("wt-1".to_string())) + .map(|w| w.revision), + Some(7) + ); + assert!(protocol_state.attempts.is_empty()); + assert!(protocol_state.mutation_events.is_empty()); + assert!(protocol_state.external_taint.is_empty()); + + remove_test_db(&db_path); + } } diff --git a/context/plans/mutation-cursor-store-persistence.md b/context/plans/mutation-cursor-store-persistence.md index 4432e784..d16fc0bb 100644 --- a/context/plans/mutation-cursor-store-persistence.md +++ b/context/plans/mutation-cursor-store-persistence.md @@ -70,8 +70,8 @@ error without retry. - Validate: T08 injected-failure test asserts revision, scope status, processed event, mutation event, and active scopes are all unchanged after rollback. - [ ] AC12: Process restart reconstructs the same durable protocol projection. - Validate: T09 tests that drop and reopen the DB handle before reloading. -- [ ] AC13: Historical mutation events are not loaded on each boundary; terminal (`Closed`/`Abandoned`/`NeverSeen`) historical scopes are not loaded on each boundary unless they are the effective referenced scope (`scope`, or `event_key.scope_id` when `scope` is absent); explicit `scope` and `event_key.scope_id` must agree when both are supplied, or `load_worktree` returns `Err`; and the effective referenced scope must belong to the requested worktree, or `load_worktree` returns `Err` rather than silently loading or reassigning it. - - Validate: `MutationTraceStore::load_worktree` issues no query against `mutation_trace_events`, loads only currently `Active` scopes plus the effective referenced scope (if any) derived from `scope`/`event_key` per T03's four-case definition, returns `Err` when `scope` and `event_key.scope_id` are both supplied and differ, and returns `Err` when the effective referenced scope's persisted `worktree_id` does not match the requested worktree (T03 done-when). +- [ ] AC13: Historical mutation events are not loaded on each boundary; terminal (`Closed`/`Abandoned`/`NeverSeen`) historical scopes are not loaded on each boundary unless they are the effective referenced scope (`scope`, or `event_key.scope_id` when `scope` is absent); explicit `scope` and `event_key.scope_id` must agree when both are supplied, or `load_worktree` returns `Err`; the effective referenced scope must belong to the requested worktree, or `load_worktree` returns `Err` rather than silently loading or reassigning it; and the effective referenced scope must exist in durable `mutation_trace_scopes` storage — a missing effective scope returns `Err`, rather than `load_worktree` silently continuing with a projection that omits it, whether the effective scope came from `scope` or `event_key.scope_id`. + - Validate: `MutationTraceStore::load_worktree` issues no query against `mutation_trace_events`, loads only currently `Active` scopes plus the effective referenced scope (if any) derived from `scope`/`event_key` per T03's four-case definition, returns `Err` when `scope` and `event_key.scope_id` are both supplied and differ, returns `Err` when the effective referenced scope's persisted `worktree_id` does not match the requested worktree, and returns `Err` when the effective referenced scope has no durable `mutation_trace_scopes` row — including when an orphan `mutation_trace_processed_events` row exists for it (T03 done-when). - [ ] AC14: Existing Quint Connect and protocol tests remain green. - Validate: `nix flake check` (runs `cli-tests`, including `mutation_trace::mbt`, and the dedicated `mutation-trace-quint-connect` check). - [ ] AC15: No Git/filesystem lock/hook/coordinator integration is added. @@ -147,7 +147,7 @@ Persist this field in every plan; this is durable plan state, not chat state: - Context impact: local — new codec functions and types confined to a new, not-yet-wired-in file; no caller exists yet (T03+ will be the first consumer), so no root context file describes runtime behavior this changes yet. `context/cli/mutation-trace-protocol.md`'s "not yet wired into any hook, command, or database call site" framing remains accurate until a real DB call site lands (T07), matching the plan's assumption that this update is deferred to task context synchronization once that framing goes stale. - Context synchronization: synced -- [ ] T03: `Add bounded WorktreeProjection load and cold-path MutationEvent read` (status:todo) +- [x] T03: `Add bounded WorktreeProjection load and cold-path MutationEvent read` (status:done) - Task ID: T03 - Scope: In — `WorktreeProjection` (+ `into_protocol_state`) and `MutationTraceStore` wrapping `&RepositoryAgentTraceDb`. `load_worktree(worktree: &WorktreeId, scope: Option<&ScopeId>, event_key: Option<&EventKey>)` first derives one `effective_scope: Option<&ScopeId>` from `scope` and `event_key`. **Invariant:** `scope` and `event_key.scope_id` are two ways of referring to the same operation-local scope identity; when both are supplied they must agree; when only `event_key` is supplied, its `scope_id` becomes the effective referenced scope for projection loading and `WorktreeId` validation. This avoids relying on a separate `worktree_id` stored on processed events. Concretely: - `scope = None`, `event_key = None` -> `effective_scope = None` (no referenced scope); only the requested worktree's `Active` scopes are loaded, with no extra terminal scope. @@ -156,6 +156,8 @@ Persist this field in every plan; this is durable plan state, not chat state: - `scope = Some(S)`, `event_key = Some(K)`, `S == K.scope_id` -> `effective_scope = Some(S)`; that single `ScopeId` is loaded and validated once. - `scope = Some(S)`, `event_key = Some(K)`, `S != K.scope_id` -> `load_worktree` returns `Err` before loading either scope and before performing the processed-event lookup. It never chooses one arbitrarily, never loads both scopes, never ignores the mismatch, and never performs the replay query anyway. + **Existence invariant (T03 correction):** whenever an `effective_scope` is `Some(S)` (whether `S` came from `scope` or from `event_key.scope_id`), a durable `mutation_trace_scopes` row for `S` must exist, or `load_worktree` returns `Err`. This is checked before the `processed_events` replay lookup, so a `mutation_trace_processed_events` row alone (an orphan replay row with no owning scope) can never cause `processed_events` to gain an entry without its `ScopeState` also being present in the projection. A missing effective scope is never silently omitted — `load_worktree` never returns `Ok(Some(projection))` with a projection that excludes an effective scope it was asked about. When the effective scope is already present in the `Active`-scope query result (already known to belong to the requested worktree, since that query is filtered by it), no second scope query is issued. + `load_worktree` then loads exactly one worktree row, only its currently `Active` scopes plus the scope named by `effective_scope` (even when that scope is `NeverSeen`/`Closed`/`Abandoned`), and — when `event_key` is supplied and no `S != K.scope_id` mismatch already returned `Err` — 0 or 1 matching processed-event row for `event_key`. The processed-event lookup is keyed solely by `event_key`'s `(scope_id, event_id)` — `WHERE scope_id = ? AND event_id = ?`, never filtered or joined by `worktree_id` — since `mutation_trace_processed_events` carries no `worktree_id` column (removed from migration `003`; the table's only identity is `PRIMARY KEY (scope_id, event_id)`, matching the domain `EventKey`). The worktree relationship for `event_key`'s scope is established by loading and validating its durable `ScopeState` as part of `effective_scope` above, not by a `worktree_id` column on the processed-event table: `EventKey.scope_id` -> `mutation_trace_scopes.scope_id` -> `mutation_trace_scopes.worktree_id`. A separate cold-path `load_mutation_event(worktree: &WorktreeId, revision: u64) -> Result>` reads one `mutation_trace_events` row plus its `mutation_trace_event_active_scopes` rows and reconstructs a complete `MutationEvent`, decoding `Attribution` exactly (including `AiExclusive(scope_id)`) and the complete `Boundary`. When `effective_scope` is `Some(S)` and the persisted `ScopeState` for `S` has a `worktree_id` different from the requested `worktree`, `load_worktree` returns `Err` — it never silently omits the scope, never includes it in the projection, and never reassigns it to the requested worktree, preserving the permanent `ScopeId` -> `WorktreeId` identity `register_scope` already enforces. This is the same check whether `S` came from the explicit `scope` argument or from `event_key.scope_id`. Out — initialization/commit logic (T04/T07); calling `load_mutation_event` from `load_worktree` or from any hook-boundary path. - Dependencies: T01, T02 - Done when: `load_worktree` returns `None` for a missing worktree and `Some(projection)` otherwise, with `scopes` containing every currently `Active` scope on that worktree plus the `effective_scope` (derived from `scope`/`event_key` per the four cases above) when one exists, regardless of its status, and never a `Closed`/`Abandoned`/`NeverSeen` scope that was not the effective referenced scope; `attempts`, `mutation_events`, and `external_taint` stay empty; the method issues no query against `mutation_trace_events`. Explicit test cases for all five `scope`/`event_key` combinations: @@ -164,10 +166,20 @@ Persist this field in every plan; this is durable plan state, not chat state: 3. `scope=None`, `event_key=Some(K)` -> `K.scope_id` is loaded as the effective referenced scope; a wrong-worktree `K.scope_id` returns `Err`; the processed-event lookup for `K` still matches solely on `(scope_id, event_id)`. 4. `scope=Some(S)`, `event_key=Some(K)`, `S == K.scope_id` -> succeeds, loading and validating that one `ScopeId` exactly once. 5. `scope=Some(S)`, `event_key=Some(K)`, `S != K.scope_id` -> `load_worktree` returns `Err` without loading either scope and without performing the processed-event lookup. + 6. (T03 correction) An effective referenced scope that has no durable `mutation_trace_scopes` row returns `Err`, whether it came from `scope` (explicit missing scope) or from `event_key.scope_id` (event-key-only missing scope) — including when a `mutation_trace_processed_events` row already exists for that `(scope_id, event_id)` (orphan replay row): the replay row alone must never be enough to construct a valid projection. - Also preserved: a referenced scope on the requested worktree is included regardless of status; a referenced terminal (`Closed`/`Abandoned`/`NeverSeen`) scope on the requested worktree is included; an unreferenced terminal historical scope is excluded; the processed-event query never references `worktree_id` (it has no such column) and matches solely on `scope_id`/`event_id`; `load_worktree` never queries historical `mutation_trace_events` rows. `load_mutation_event` returns `None` when no row exists at that `(worktree, revision)` and otherwise reconstructs a `MutationEvent` whose `before_tree`/`after_tree`/`revision`/`tainted`/`failure_kind`/`attribution`/`boundary`/`active_scopes` exactly match what `store.commit` persisted. + Also preserved: a referenced scope on the requested worktree is included regardless of status; a referenced terminal (`Closed`/`Abandoned`/`NeverSeen`) scope on the requested worktree is included; an unreferenced terminal historical scope is excluded; the processed-event query never references `worktree_id` (it has no such column) and matches solely on `scope_id`/`event_id`; `load_worktree` never queries historical `mutation_trace_events` rows. `load_mutation_event` returns `None` when no row exists at that `(worktree, revision)` and otherwise reconstructs a `MutationEvent` whose `before_tree`/`after_tree`/`revision`/`tainted`/`failure_kind`/`attribution`/`boundary`/`active_scopes` exactly match what `store.commit` persisted. If an effective referenced scope is present, that `ScopeId` must exist in durable scope storage; a missing effective scope returns `Err`. This rule applies whether the effective scope came from `scope` or `event_key.scope_id`. - Verify: `./scripts/run-cli-cargo.sh test --manifest-path cli/Cargo.toml mutation_trace::store::` - - Context synchronization: pending + - Completed: 2026-08-27 + - Files changed: `cli/src/services/mutation_trace/store.rs` + - Result: Added `WorktreeProjection` (`worktree_id`, `worktree_state`, `scopes`, `processed_events`) and its `into_protocol_state` (widens into a full `ProtocolState`, always with an empty `attempts`/`mutation_events`/`external_taint`), plus `MutationTraceStore<'a>` wrapping `&'a RepositoryAgentTraceDb`. `load_worktree` derives one `effective_referenced_scope` from `scope`/`event_key` per the plan's four-case definition (returning `Err` before any query on a `Some(S) != Some(K.scope_id)` mismatch), returns `None` for a missing worktree row, otherwise loads the worktree's currently `Active` scopes (`SELECT ... WHERE worktree_id = ?1 AND status = 'active'`) plus the effective scope by `scope_id` alone (any status), erroring if that scope's persisted `worktree_id` disagrees with the requested worktree, and populates `processed_events` with 0 or 1 entries via a `(scope_id, event_id)`-only lookup — no query anywhere in `load_worktree` references `mutation_trace_events`. `load_mutation_event(worktree, revision)` reads one `mutation_trace_events` row plus its `mutation_trace_event_active_scopes` rows and reconstructs a full `MutationEvent`, decoding `Attribution`/`Boundary` via new `reconstruct_attribution`/`reconstruct_boundary` helpers built on T02's `AttributionKind`/`BoundaryKind` codecs (rejecting inconsistent kind/payload combinations). Added 14 new tests covering: missing worktree, all five `scope`/`event_key` combination cases (including both wrong-worktree error cases and the disagreement case), `load_mutation_event`'s missing-row case, two full reconstruction round trips (`ai_exclusive`/`start` with one active scope, `ai_contended`/`flush` with two active scopes), and one test asserting `into_protocol_state`'s single-worktree/empty-transient-fields shape. + - Verify: `./scripts/run-cli-cargo.sh test --manifest-path cli/Cargo.toml mutation_trace::store::` — passed, 24/24; `./scripts/run-cli-cargo.sh fmt --manifest-path cli/Cargo.toml -- --check` — passed after one auto-formatting pass; `./scripts/run-cli-cargo.sh clippy --manifest-path cli/Cargo.toml --all-targets -- -D warnings` — passed, zero warnings. + - Done checks: `load_worktree` returns `None` for a missing worktree and `Some(projection)` otherwise (verified); all five `scope`/`event_key` cases behave exactly as specified, including both wrong-worktree `Err` cases and the disagreement `Err` case (verified by the five case-specific tests); `attempts`/`mutation_events`/`external_taint` stay empty in every projection and in `into_protocol_state`'s output (verified); no query in `load_worktree` references `mutation_trace_events` (verified by inspection — the function's SQL constants are `SELECT_WORKTREE_SQL`/`SELECT_SCOPES_BY_WORKTREE_AND_STATUS_SQL`/`SELECT_SCOPE_BY_ID_SQL`/`SELECT_PROCESSED_EVENT_SQL` only); `load_mutation_event` returns `None` when absent and otherwise reconstructs an exact `MutationEvent`, including full `Attribution`/`Boundary` decoding and `active_scopes` (verified by the two reconstruction tests). + - Context impact: local — new query/projection logic confined to `store.rs`, still not called from any hook, command, or `coordinator.rs`/`git_snapshot.rs` seam; `context/cli/mutation-trace-protocol.md`'s "not yet wired into any hook, command, or database call site" framing remains accurate (a real call site lands in T07). No durable context file describes this yet; deferred to T11/plan-level context sync once the full store lands, per the plan's assumption. + - Context synchronization: synced + - **T03 correction (2026-08-27):** Fixed a persistence-boundary gap where a missing effective referenced scope (`load_scope` returning `None`) fell through `if let Some(scope_state) = self.load_scope(...)?` and silently continued, producing a projection that omitted the effective scope instead of erroring. Changed to `self.load_scope(effective_scope_id)?.ok_or_else(...)?`, so a missing durable `mutation_trace_scopes` row for the effective scope now returns `Err` before the wrong-worktree check and before the `processed_events` replay lookup — applying equally whether the effective scope came from `scope` or from `event_key.scope_id`, and preventing an orphan `mutation_trace_processed_events` row from ever entering `ProtocolState.processed_events` without its owning `ScopeState`. The Active-scope fast path (`if !scopes.contains_key(effective_scope_id)`) is unchanged, so an effective scope already present from the bounded `Active` query still skips the second lookup. Added three tests: `load_worktree_with_explicit_missing_scope_errors` (`scope=Some("scope-missing")`, no row, requested worktree exists), `load_worktree_with_event_key_missing_scope_errors` (`event_key.scope_id="scope-missing"`, no row), and `load_worktree_with_event_key_missing_scope_and_orphan_replay_row_errors` (same as the previous case, plus a pre-existing `mutation_trace_processed_events` row for `("scope-missing", "event-1")`, proving the orphan replay row cannot substitute for the missing `ScopeState`). All existing T03 tests preserved and passing. + - Verify (T03 correction): `./scripts/run-cli-cargo.sh test --manifest-path cli/Cargo.toml mutation_trace::store::` — passed, 27/27; `./scripts/run-cli-cargo.sh fmt --manifest-path cli/Cargo.toml -- --check` — passed, no diff; `./scripts/run-cli-cargo.sh clippy --manifest-path cli/Cargo.toml --all-targets -- -D warnings` — passed, zero warnings. + - Done checks (T03 correction): explicit missing scope (`scope=Some(S)`, no durable row) returns `Err` (verified); event-key-only missing scope (`scope=None`, `event_key.scope_id=K`, no durable row) returns `Err` (verified); an orphan `mutation_trace_processed_events` row for a missing scope still returns `Err` and never populates `processed_events` (verified); the Active-scope fast path issues no redundant scope query when the effective scope is already loaded (verified by inspection — unchanged `if !scopes.contains_key(...)` guard); T04 was not started (verified — no changes to `initialize_worktree`/`register_scope`, migration 003, `EventKey`, or any file outside `store.rs`/this plan). - [ ] T04: `Add worktree/scope initialization operations` (status:todo) - Task ID: T04 From acffeaf38c7d97d6d889ebaf03e64b882fea78e1 Mon Sep 17 00:00:00 2001 From: David Abram Date: Thu, 27 Aug 2026 23:07:29 +0200 Subject: [PATCH 06/15] mutation-trace: Add idempotent worktree and scope initialization Persist fresh worktree cursors and scope registrations without overwriting existing durable state. Validate existing scope ownership and actor identity before returning its state, while keeping terminal/status state unchanged. Complete T04 in `context/plans/mutation-cursor-store-persistence.md` and record targeted tests and verification. Co-authored-by: SCE --- cli/src/services/mutation_trace/store.rs | 298 ++++++++++++++++++ .../mutation-cursor-store-persistence.md | 16 +- 2 files changed, 311 insertions(+), 3 deletions(-) diff --git a/cli/src/services/mutation_trace/store.rs b/cli/src/services/mutation_trace/store.rs index 345f3b17..4cf0affd 100644 --- a/cli/src/services/mutation_trace/store.rs +++ b/cli/src/services/mutation_trace/store.rs @@ -206,6 +206,19 @@ const SELECT_MUTATION_EVENT_SQL: &str = "SELECT before_tree, after_tree, tainted FROM mutation_trace_events WHERE worktree_id = ?1 AND revision = ?2"; const SELECT_MUTATION_EVENT_ACTIVE_SCOPES_SQL: &str = "SELECT scope_id FROM mutation_trace_event_active_scopes WHERE worktree_id = ?1 AND revision = ?2"; +/// Idle-insert: only takes effect when `worktree_id` has no row yet, so an +/// existing worktree's cursor/revision/failure state is never overwritten. +const INSERT_WORKTREE_IF_ABSENT_SQL: &str = "INSERT INTO mutation_trace_worktrees + (worktree_id, cursor_tree, revision, tainted, failure_kind, needs_rebaseline) + VALUES (?1, ?2, ?3, 0, 'healthy', 0) + ON CONFLICT (worktree_id) DO NOTHING"; +/// Idle-insert: only takes effect when `scope_id` has no row yet, so an +/// existing scope's worktree/actor/status is never overwritten. The caller +/// re-reads the row afterward to detect a worktree/actor mismatch. +const INSERT_SCOPE_IF_ABSENT_SQL: &str = + "INSERT INTO mutation_trace_scopes (scope_id, worktree_id, actor_kind, status) + VALUES (?1, ?2, ?3, 'never_seen') + ON CONFLICT (scope_id) DO NOTHING"; /// Bounded runtime projection of one worktree's durable protocol state, /// loaded by [`MutationTraceStore::load_worktree`]. Scoped to that worktree's @@ -258,6 +271,80 @@ impl<'a> MutationTraceStore<'a> { Self { db } } + /// Idempotently initializes `worktree`'s durable cursor row: `revision=0`, + /// healthy, not tainted, not needing rebaseline, with `cursor_tree` set to + /// `initial_tree`. A no-op when the worktree row already exists — an + /// existing cursor, revision, or failure state is never overwritten. + pub fn initialize_worktree(&self, worktree: &WorktreeId, initial_tree: &TreeId) -> Result<()> { + self.db.execute( + INSERT_WORKTREE_IF_ABSENT_SQL, + ( + worktree.0.as_str(), + initial_tree.0.as_str(), + encode_revision(0).as_slice(), + ), + )?; + + Ok(()) + } + + /// Idempotently registers `scope` as belonging to `worktree` and + /// `actor_kind`. Inserts a new `NeverSeen` row when `scope` has none yet. + /// When a row already exists, returns its current state unchanged as long + /// as its `worktree_id` and `actor_kind` agree with the arguments — this + /// never resurrects a terminal scope or changes its status — and returns + /// `Err` when either disagrees, since a scope's worktree and actor are + /// permanent facts fixed at first registration. + /// + /// `worktree` must already have a durable `mutation_trace_worktrees` row + /// (via [`MutationTraceStore::initialize_worktree`]), checked before any + /// scope row is inserted or read back — this never auto-creates the + /// worktree. This applies identically to a fresh `scope` and to an + /// existing one: an existing scope whose stored `worktree_id` has no + /// worktree row is never returned as valid merely because it matches the + /// arguments. + pub fn register_scope( + &self, + scope: &ScopeId, + worktree: &WorktreeId, + actor_kind: ActorKind, + ) -> Result { + if self.load_worktree_state(worktree)?.is_none() { + bail!( + "cannot register scope {scope:?}: worktree {worktree:?} has no mutation_trace_worktrees row" + ); + } + + self.db.execute( + INSERT_SCOPE_IF_ABSENT_SQL, + ( + scope.0.as_str(), + worktree.0.as_str(), + encode_actor_kind(actor_kind), + ), + )?; + + let scope_state = self.load_scope(scope)?.ok_or_else(|| { + anyhow::anyhow!("scope {scope:?} has no row immediately after register_scope insert") + })?; + + if scope_state.worktree_id != *worktree { + bail!( + "scope {scope:?} is already registered to worktree {:?}, not {worktree:?}", + scope_state.worktree_id + ); + } + + if scope_state.actor_kind != actor_kind { + bail!( + "scope {scope:?} is already registered to actor {:?}, not {actor_kind:?}", + scope_state.actor_kind + ); + } + + Ok(scope_state) + } + /// Loads a bounded projection of `worktree`'s durable protocol state, or /// `None` when the worktree does not exist. /// @@ -1231,4 +1318,215 @@ mod tests { remove_test_db(&db_path); } + + #[test] + fn initialize_worktree_inserts_a_fresh_healthy_cursor() { + let db_path = unique_test_db_path("init-worktree-fresh"); + let db = RepositoryAgentTraceDb::new_at(&db_path).expect("repository DB should open"); + let store = MutationTraceStore::new(&db); + + store + .initialize_worktree( + &WorktreeId("wt-1".to_string()), + &TreeId("tree-0".to_string()), + ) + .expect("initialize_worktree should succeed"); + + let projection = store + .load_worktree(&WorktreeId("wt-1".to_string()), None, None) + .expect("load_worktree should succeed") + .expect("worktree should exist"); + + assert_eq!( + projection.worktree_state, + WorktreeState { + cursor_tree: TreeId("tree-0".to_string()), + revision: 0, + tainted: false, + failure_kind: FailureKind::Healthy, + needs_rebaseline: false, + } + ); + + remove_test_db(&db_path); + } + + #[test] + fn initialize_worktree_never_overwrites_an_existing_cursor() { + let db_path = unique_test_db_path("init-worktree-idempotent"); + let db = RepositoryAgentTraceDb::new_at(&db_path).expect("repository DB should open"); + let store = MutationTraceStore::new(&db); + + insert_worktree(&db, "wt-1", 5); + + store + .initialize_worktree( + &WorktreeId("wt-1".to_string()), + &TreeId("tree-new".to_string()), + ) + .expect("initialize_worktree should succeed as a no-op"); + + let projection = store + .load_worktree(&WorktreeId("wt-1".to_string()), None, None) + .expect("load_worktree should succeed") + .expect("worktree should exist"); + + assert_eq!( + projection.worktree_state.cursor_tree, + TreeId("tree-0".to_string()) + ); + assert_eq!(projection.worktree_state.revision, 5); + + remove_test_db(&db_path); + } + + #[test] + fn register_scope_inserts_never_seen_when_missing() { + let db_path = unique_test_db_path("register-scope-fresh"); + let db = RepositoryAgentTraceDb::new_at(&db_path).expect("repository DB should open"); + let store = MutationTraceStore::new(&db); + + insert_worktree(&db, "wt-1", 0); + + let scope_state = store + .register_scope( + &ScopeId("scope-1".to_string()), + &WorktreeId("wt-1".to_string()), + ActorKind::ClaudeCode, + ) + .expect("register_scope should succeed"); + + assert_eq!( + scope_state, + ScopeState { + status: ScopeStatus::NeverSeen, + actor_kind: ActorKind::ClaudeCode, + worktree_id: WorktreeId("wt-1".to_string()), + } + ); + + remove_test_db(&db_path); + } + + #[test] + fn register_scope_returns_existing_state_when_worktree_and_actor_match() { + let db_path = unique_test_db_path("register-scope-existing-match"); + let db = RepositoryAgentTraceDb::new_at(&db_path).expect("repository DB should open"); + let store = MutationTraceStore::new(&db); + + insert_worktree(&db, "wt-1", 0); + insert_scope(&db, "scope-1", "wt-1", ScopeStatus::Active); + + let scope_state = store + .register_scope( + &ScopeId("scope-1".to_string()), + &WorktreeId("wt-1".to_string()), + ActorKind::ClaudeCode, + ) + .expect("register_scope should succeed for a matching existing scope"); + + assert_eq!( + scope_state, + ScopeState { + status: ScopeStatus::Active, + actor_kind: ActorKind::ClaudeCode, + worktree_id: WorktreeId("wt-1".to_string()), + } + ); + + remove_test_db(&db_path); + } + + #[test] + fn register_scope_errors_on_worktree_mismatch() { + let db_path = unique_test_db_path("register-scope-worktree-mismatch"); + let db = RepositoryAgentTraceDb::new_at(&db_path).expect("repository DB should open"); + let store = MutationTraceStore::new(&db); + + insert_worktree(&db, "wt-1", 0); + insert_worktree(&db, "wt-2", 0); + insert_scope(&db, "scope-1", "wt-1", ScopeStatus::Active); + + let error = store + .register_scope( + &ScopeId("scope-1".to_string()), + &WorktreeId("wt-2".to_string()), + ActorKind::ClaudeCode, + ) + .expect_err("a worktree mismatch on an existing scope should error"); + assert!(error.to_string().contains("scope-1")); + + remove_test_db(&db_path); + } + + #[test] + fn register_scope_errors_on_actor_mismatch() { + let db_path = unique_test_db_path("register-scope-actor-mismatch"); + let db = RepositoryAgentTraceDb::new_at(&db_path).expect("repository DB should open"); + let store = MutationTraceStore::new(&db); + + insert_worktree(&db, "wt-1", 0); + insert_scope(&db, "scope-1", "wt-1", ScopeStatus::Active); + + let error = store + .register_scope( + &ScopeId("scope-1".to_string()), + &WorktreeId("wt-1".to_string()), + ActorKind::Codex, + ) + .expect_err("an actor mismatch on an existing scope should error"); + assert!(error.to_string().contains("scope-1")); + + remove_test_db(&db_path); + } + + #[test] + fn register_scope_errors_when_worktree_does_not_exist_and_leaves_no_scope_row() { + let db_path = unique_test_db_path("register-scope-missing-worktree-fresh"); + let db = RepositoryAgentTraceDb::new_at(&db_path).expect("repository DB should open"); + let store = MutationTraceStore::new(&db); + + let error = store + .register_scope( + &ScopeId("scope-1".to_string()), + &WorktreeId("wt-missing".to_string()), + ActorKind::ClaudeCode, + ) + .expect_err("registering a scope against a missing worktree should error"); + assert!(error.to_string().contains("scope-1")); + assert!(error.to_string().contains("wt-missing")); + + let scope_state = store + .load_scope(&ScopeId("scope-1".to_string())) + .expect("load_scope should succeed"); + assert!( + scope_state.is_none(), + "a failed register_scope must not leave an orphan scope row" + ); + + remove_test_db(&db_path); + } + + #[test] + fn register_scope_errors_when_existing_scopes_worktree_row_is_missing() { + let db_path = unique_test_db_path("register-scope-missing-worktree-existing"); + let db = RepositoryAgentTraceDb::new_at(&db_path).expect("repository DB should open"); + let store = MutationTraceStore::new(&db); + + insert_scope(&db, "scope-1", "wt-missing", ScopeStatus::Active); + + let error = store + .register_scope( + &ScopeId("scope-1".to_string()), + &WorktreeId("wt-missing".to_string()), + ActorKind::ClaudeCode, + ) + .expect_err( + "an existing scope whose worktree row is missing must not be accepted as valid", + ); + assert!(error.to_string().contains("scope-1")); + assert!(error.to_string().contains("wt-missing")); + + remove_test_db(&db_path); + } } diff --git a/context/plans/mutation-cursor-store-persistence.md b/context/plans/mutation-cursor-store-persistence.md index d16fc0bb..f70f22b7 100644 --- a/context/plans/mutation-cursor-store-persistence.md +++ b/context/plans/mutation-cursor-store-persistence.md @@ -181,13 +181,23 @@ Persist this field in every plan; this is durable plan state, not chat state: - Verify (T03 correction): `./scripts/run-cli-cargo.sh test --manifest-path cli/Cargo.toml mutation_trace::store::` — passed, 27/27; `./scripts/run-cli-cargo.sh fmt --manifest-path cli/Cargo.toml -- --check` — passed, no diff; `./scripts/run-cli-cargo.sh clippy --manifest-path cli/Cargo.toml --all-targets -- -D warnings` — passed, zero warnings. - Done checks (T03 correction): explicit missing scope (`scope=Some(S)`, no durable row) returns `Err` (verified); event-key-only missing scope (`scope=None`, `event_key.scope_id=K`, no durable row) returns `Err` (verified); an orphan `mutation_trace_processed_events` row for a missing scope still returns `Err` and never populates `processed_events` (verified); the Active-scope fast path issues no redundant scope query when the effective scope is already loaded (verified by inspection — unchanged `if !scopes.contains_key(...)` guard); T04 was not started (verified — no changes to `initialize_worktree`/`register_scope`, migration 003, `EventKey`, or any file outside `store.rs`/this plan). -- [ ] T04: `Add worktree/scope initialization operations` (status:todo) +- [x] T04: `Add worktree/scope initialization operations` (status:done) - Task ID: T04 - Scope: In — `initialize_worktree(worktree_id, initial_tree)` and `register_scope(scope_id, worktree_id, actor_kind)` on `MutationTraceStore`. Out — the CAS commit path (T06/T07). - Dependencies: T03 - - Done when: `initialize_worktree` inserts `revision=0`/healthy/not-tainted/not-needs-rebaseline only when the worktree is missing and never overwrites an existing cursor; `register_scope` inserts `NeverSeen` when missing, returns the existing state when worktree+actor match, and errors on a worktree or actor mismatch for an existing `scope_id`. + - Done when: `initialize_worktree` inserts `revision=0`/healthy/not-tainted/not-needs-rebaseline only when the worktree is missing and never overwrites an existing cursor; `register_scope` inserts `NeverSeen` when missing, returns the existing state when worktree+actor match, and errors on a worktree or actor mismatch for an existing `scope_id`. **(T04 correction)** `register_scope` requires the referenced worktree to already exist in `mutation_trace_worktrees`. A missing worktree returns `Err` and no scope row is inserted. An existing scope whose referenced worktree row is missing also returns `Err`, even when its stored `worktree_id`/`actor_kind` match the request — it is never accepted as valid merely because those fields agree. - Verify: `./scripts/run-cli-cargo.sh test --manifest-path cli/Cargo.toml mutation_trace::store::` - - Context synchronization: pending + - Completed: 2026-08-27 + - Files changed: `cli/src/services/mutation_trace/store.rs` + - Result: Added `INSERT_WORKTREE_IF_ABSENT_SQL`/`INSERT_SCOPE_IF_ABSENT_SQL` idle-insert constants (`INSERT ... ON CONFLICT (pk) DO NOTHING`), matching the existing `INSERT_REPOSITORY_METADATA_SQL` idle-insert pattern in `agent_trace_db/repository.rs`. `MutationTraceStore::initialize_worktree(worktree, initial_tree)` runs the idle-insert with `revision=0`/`healthy`/not-tainted/not-needs-rebaseline, a no-op when the worktree row already exists. `MutationTraceStore::register_scope(scope, worktree, actor_kind)` runs the idle-insert (`NeverSeen` status) then reuses the existing private `load_scope` helper to read back the row, bailing with a descriptive error (including the scope ID) when the read-back `worktree_id` or `actor_kind` disagrees with the caller's arguments, otherwise returning the existing (or freshly inserted) `ScopeState` unchanged. Added six tests: fresh insert and idempotent-no-op-on-existing-cursor for `initialize_worktree`; fresh insert, matching-existing-state return, worktree-mismatch error, and actor-mismatch error for `register_scope`. + - Verify: `./scripts/run-cli-cargo.sh test --manifest-path cli/Cargo.toml mutation_trace::store::` — passed, 33/33; `./scripts/run-cli-cargo.sh fmt --manifest-path cli/Cargo.toml -- --check` — passed after one auto-formatting pass (no manual diff needed beyond `cargo fmt`); `./scripts/run-cli-cargo.sh clippy --manifest-path cli/Cargo.toml --all-targets -- -D warnings` — passed, zero warnings. + - Done checks: `initialize_worktree` inserts the specified fresh row only when missing (verified by `initialize_worktree_inserts_a_fresh_healthy_cursor`) and never overwrites an existing cursor/revision (verified by `initialize_worktree_never_overwrites_an_existing_cursor`); `register_scope` inserts `NeverSeen` when missing (verified by `register_scope_inserts_never_seen_when_missing`), returns the existing state unchanged when worktree+actor match regardless of status (verified by `register_scope_returns_existing_state_when_worktree_and_actor_match`, using an `Active` existing scope), and errors on a worktree mismatch (verified) or actor mismatch (verified) for an existing `scope_id`. + - Context impact: local — two new store-level write operations confined to `store.rs`, still not called from any hook, command, `coordinator.rs`/`git_snapshot.rs` seam, or the CAS commit path (T06/T07 land the first caller). `context/cli/mutation-trace-protocol.md`'s "not yet wired into any hook, command, or database call site" framing remains accurate. No durable context file describes this yet; deferred to T11/plan-level context sync once the full store lands, per the plan's assumption. + - **T04 correction (2026-08-27):** Fixed a persistence-integrity gap where `register_scope` could create (or validate as existing) a durable `mutation_trace_scopes` row whose `worktree_id` had no corresponding `mutation_trace_worktrees` row — the function only ran the idle-insert and then checked the read-back row's `worktree_id`/`actor_kind` against the caller's arguments, never checking that the referenced worktree itself existed. Added a `load_worktree_state(worktree)?.is_none()` guard at the top of `register_scope`, before `INSERT_SCOPE_IF_ABSENT_SQL` runs, `bail!`ing with `"cannot register scope {scope:?}: worktree {worktree:?} has no mutation_trace_worktrees row"` when the requested worktree is missing — so no scope row is ever inserted, and the check applies identically whether `scope_id` is fresh or already has an (orphaned) row, since it is keyed on the caller's `worktree` argument rather than on what the scope row happens to already say. `initialize_worktree` is not called from `register_scope`; a missing worktree is an error, not an auto-create trigger. Migration `003` was not touched — no `FOREIGN KEY`/trigger was added; the invariant is enforced at the store API boundary only. Fixed three pre-existing tests (`register_scope_inserts_never_seen_when_missing`, `register_scope_returns_existing_state_when_worktree_and_actor_match`, `register_scope_errors_on_worktree_mismatch`, `register_scope_errors_on_actor_mismatch`) that had never inserted a `mutation_trace_worktrees` row for the worktrees they exercised, which the new guard would otherwise have short-circuited before reaching the behavior each test intended to cover. Added two tests: `register_scope_errors_when_worktree_does_not_exist_and_leaves_no_scope_row` (fresh `scope_id`, missing worktree — asserts `Err` and then asserts `load_scope` finds no row) and `register_scope_errors_when_existing_scopes_worktree_row_is_missing` (a pre-existing orphan `mutation_trace_scopes` row pointing at a worktree that was never inserted — asserts `Err` even though the stored `worktree_id`/`actor_kind` match the request). + - Verify (T04 correction): `./scripts/run-cli-cargo.sh test --manifest-path cli/Cargo.toml mutation_trace::store::` — passed, 35/35; `./scripts/run-cli-cargo.sh fmt --manifest-path cli/Cargo.toml -- --check` — passed, no diff; `./scripts/run-cli-cargo.sh clippy --manifest-path cli/Cargo.toml --all-targets -- -D warnings` — passed, zero warnings. + - Done checks (T04 correction): a fresh `scope_id` registered against a missing worktree returns `Err` and leaves no `mutation_trace_scopes` row (verified); an existing scope row whose `worktree_id` has no `mutation_trace_worktrees` row returns `Err` even though its stored fields match the request (verified); a fresh registration against an existing worktree still inserts `NeverSeen` and a repeat call against a matching existing scope still returns the unchanged existing state (verified — both pre-existing tests still pass once given a real worktree row); worktree and actor mismatch on an existing scope (with a real worktree row present) still error (verified); migration `003` was not changed and no `FOREIGN KEY`/trigger was added (verified — no diff to `003_mutation_trace_protocol.sql`); T05 was not started (verified — no changes to `DurableTransition`, `EventKey`, CAS design, or any file outside `store.rs`/this plan). + - Context impact: local — the guard and its tests are confined to `register_scope` in `store.rs`, still not called from any hook, command, or CAS commit path; no durable context file describes `register_scope`'s behavior yet (deferred to T11/plan-level context sync per the plan's assumption), so this correction changes no root context file's claims. + - Context synchronization: synced - [ ] T05: `Add DurableTransition structural diff type` (status:todo) - Task ID: T05 From 254cb3ce8e545be87d51178bde0505d8a20f6670 Mon Sep 17 00:00:00 2001 From: David Abram Date: Thu, 27 Aug 2026 23:48:38 +0200 Subject: [PATCH 07/15] mutation-trace: Add structural durable transition diff Validate protocol state changes before persistence can apply a durable transition. Add a pure structural diff that captures worktree, scope, processed-event, and mutation-event changes while rejecting malformed or unrelated changes. Plan: mutation-cursor-store-persistence (T05) Co-authored-by: SCE --- cli/src/services/mutation_trace/store.rs | 655 +++++++++++++++++- .../mutation-cursor-store-persistence.md | 18 +- 2 files changed, 668 insertions(+), 5 deletions(-) diff --git a/cli/src/services/mutation_trace/store.rs b/cli/src/services/mutation_trace/store.rs index 4cf0affd..33fe13b3 100644 --- a/cli/src/services/mutation_trace/store.rs +++ b/cli/src/services/mutation_trace/store.rs @@ -259,6 +259,192 @@ impl WorktreeProjection { } } +#[derive(Clone, Debug, Eq, PartialEq)] +pub struct DurableTransition { + pub worktree: WorktreeId, + pub expected_revision: u64, + pub next_worktree_state: WorktreeState, + pub scope_status_changes: BTreeMap, + pub new_processed_event: Option, + pub new_mutation_event: Option, +} + +impl DurableTransition { + pub fn between( + before: &ProtocolState, + after: &ProtocolState, + worktree: &WorktreeId, + ) -> Result> { + let (before_worktree_state, after_worktree_state) = + diff_target_worktree(before, after, worktree)?; + let scope_status_changes = diff_scopes(before, after, worktree)?; + let new_processed_event = diff_new_processed_event(before, after, worktree)?; + let new_mutation_event = diff_new_mutation_event(before, after, worktree)?; + + let no_change = before_worktree_state == after_worktree_state + && scope_status_changes.is_empty() + && new_processed_event.is_none() + && new_mutation_event.is_none(); + + if no_change { + return Ok(None); + } + + let expected_revision = before_worktree_state.revision; + let next_revision = expected_revision.checked_add(1); + if Some(after_worktree_state.revision) != next_revision { + bail!( + "worktree {worktree:?} revision must advance by exactly one from {expected_revision}, got {}", + after_worktree_state.revision + ); + } + + if let Some(event) = &new_mutation_event { + if event.revision != after_worktree_state.revision { + bail!( + "new mutation event revision {} does not match worktree {worktree:?}'s resulting revision {}", + event.revision, + after_worktree_state.revision + ); + } + } + + Ok(Some(Self { + worktree: worktree.clone(), + expected_revision, + next_worktree_state: after_worktree_state.clone(), + scope_status_changes, + new_processed_event, + new_mutation_event, + })) + } +} + +fn diff_target_worktree<'s>( + before: &'s ProtocolState, + after: &'s ProtocolState, + worktree: &WorktreeId, +) -> Result<(&'s WorktreeState, &'s WorktreeState)> { + let Some(before_worktree_state) = before.worktrees.get(worktree) else { + bail!("worktree {worktree:?} missing from before state"); + }; + let Some(after_worktree_state) = after.worktrees.get(worktree) else { + bail!("worktree {worktree:?} missing from after state"); + }; + + if before.worktrees.len() != after.worktrees.len() { + bail!("worktree set changed between before and after"); + } + for (id, before_state) in &before.worktrees { + if id == worktree { + continue; + } + match after.worktrees.get(id) { + Some(after_state) if after_state == before_state => {} + _ => bail!("unrelated worktree {id:?} changed"), + } + } + + Ok((before_worktree_state, after_worktree_state)) +} + +fn diff_scopes( + before: &ProtocolState, + after: &ProtocolState, + worktree: &WorktreeId, +) -> Result> { + let before_scope_ids: BTreeSet<&ScopeId> = before.scopes.keys().collect(); + let after_scope_ids: BTreeSet<&ScopeId> = after.scopes.keys().collect(); + if before_scope_ids != after_scope_ids { + bail!("scope set changed between before and after"); + } + + let mut scope_status_changes = BTreeMap::new(); + for (scope_id, before_scope) in &before.scopes { + let after_scope = after + .scopes + .get(scope_id) + .expect("scope key sets already verified equal"); + + if before_scope.worktree_id != after_scope.worktree_id { + bail!("scope {scope_id:?} worktree_id changed"); + } + if before_scope.actor_kind != after_scope.actor_kind { + bail!("scope {scope_id:?} actor_kind changed"); + } + if before_scope.status != after_scope.status { + if before_scope.worktree_id != *worktree { + bail!( + "scope {scope_id:?} status changed but belongs to worktree {:?}, not {worktree:?}", + before_scope.worktree_id + ); + } + scope_status_changes.insert(scope_id.clone(), after_scope.status); + } + } + + Ok(scope_status_changes) +} + +fn diff_new_processed_event( + before: &ProtocolState, + after: &ProtocolState, + worktree: &WorktreeId, +) -> Result> { + if !before.processed_events.is_subset(&after.processed_events) { + bail!("a processed_events entry disappeared"); + } + let new_processed_events: Vec<&EventKey> = after + .processed_events + .difference(&before.processed_events) + .collect(); + if new_processed_events.len() > 1 { + bail!("more than one new processed_events entry"); + } + + let Some(key) = new_processed_events.first() else { + return Ok(None); + }; + let scope = after.scopes.get(&key.scope_id).ok_or_else(|| { + anyhow::anyhow!("new processed event {key:?} has no scope in after state") + })?; + if scope.worktree_id != *worktree { + bail!( + "new processed event {key:?} belongs to worktree {:?}, not {worktree:?}", + scope.worktree_id + ); + } + Ok(Some((*key).clone())) +} + +fn diff_new_mutation_event( + before: &ProtocolState, + after: &ProtocolState, + worktree: &WorktreeId, +) -> Result> { + if !before.mutation_events.is_subset(&after.mutation_events) { + bail!("a mutation_events entry disappeared"); + } + let new_mutation_events: Vec<&MutationEvent> = after + .mutation_events + .difference(&before.mutation_events) + .collect(); + if new_mutation_events.len() > 1 { + bail!("more than one new mutation_events entry"); + } + + let Some(event) = new_mutation_events.first() else { + return Ok(None); + }; + if event.worktree_id != *worktree { + bail!( + "new mutation event belongs to worktree {:?}, not {worktree:?}", + event.worktree_id + ); + } + Ok(Some((*event).clone())) +} + /// Bounded read access to the durable mutation-cursor protocol state for one /// repository, via [`RepositoryAgentTraceDb`]. Write/CAS-commit access is /// added by later tasks (T04/T06/T07). @@ -692,7 +878,10 @@ fn reconstruct_boundary( #[cfg(test)] mod tests { use super::*; - use crate::services::mutation_trace::types::{EventId, ScopeId}; + use crate::services::mutation_trace::protocol::{ + abandon, commit, database_failure, prepare, recover, taint, + }; + use crate::services::mutation_trace::types::{AttemptId, EventId, ScopeId}; #[test] fn revision_round_trips_at_boundary_values() { @@ -1529,4 +1718,468 @@ mod tests { remove_test_db(&db_path); } + + fn healthy_worktree_state(revision: u64) -> WorktreeState { + WorktreeState { + cursor_tree: TreeId("tree0".to_string()), + revision, + tainted: false, + failure_kind: FailureKind::Healthy, + needs_rebaseline: false, + } + } + + fn state_with_scope( + worktree_id: &WorktreeId, + scope_id: &ScopeId, + actor_kind: ActorKind, + status: ScopeStatus, + revision: u64, + ) -> ProtocolState { + let mut state = ProtocolState::default(); + state + .worktrees + .insert(worktree_id.clone(), healthy_worktree_state(revision)); + state.scopes.insert( + scope_id.clone(), + ScopeState { + status, + actor_kind, + worktree_id: worktree_id.clone(), + }, + ); + state + } + + fn sample_mutation_event(worktree_id: &WorktreeId) -> MutationEvent { + MutationEvent { + worktree_id: worktree_id.clone(), + revision: 1, + before_tree: TreeId("tree0".to_string()), + after_tree: TreeId("tree1".to_string()), + active_scopes: BTreeSet::new(), + tainted: false, + failure_kind: FailureKind::Healthy, + attribution: Attribution::IneligibleUnscoped, + boundary: Boundary::Flush { + worktree: worktree_id.clone(), + }, + } + } + + #[test] + fn between_returns_none_for_a_database_failure_only_transition() { + let wt = WorktreeId("wt0".to_string()); + let mut before = ProtocolState::default(); + before + .worktrees + .insert(wt.clone(), healthy_worktree_state(0)); + + let after = database_failure(&before, &wt); + + assert_eq!( + DurableTransition::between(&before, &after, &wt).unwrap(), + None + ); + } + + #[test] + fn between_returns_none_for_a_no_change_flush() { + let wt = WorktreeId("wt0".to_string()); + let mut before = ProtocolState::default(); + before + .worktrees + .insert(wt.clone(), healthy_worktree_state(0)); + + let attempt = AttemptId("attempt0".to_string()); + let prepared = prepare( + &before, + attempt.clone(), + Boundary::Flush { + worktree: wt.clone(), + }, + TreeId("tree0".to_string()), + ); + let after = commit(&prepared, &attempt).state; + + assert_eq!( + DurableTransition::between(&before, &after, &wt).unwrap(), + None + ); + } + + #[test] + fn between_returns_some_with_correct_shape_for_a_start_transition() { + let wt = WorktreeId("wt0".to_string()); + let scope_id = ScopeId("scope0".to_string()); + let before = state_with_scope(&wt, &scope_id, ActorKind::Codex, ScopeStatus::NeverSeen, 0); + + let attempt = AttemptId("attempt0".to_string()); + let event_id = EventId("event0".to_string()); + let prepared = prepare( + &before, + attempt.clone(), + Boundary::Start { + scope: scope_id.clone(), + event: event_id.clone(), + }, + TreeId("tree1".to_string()), + ); + let after = commit(&prepared, &attempt).state; + + let transition = DurableTransition::between(&before, &after, &wt) + .unwrap() + .expect("a start transition should produce a durable transition"); + + assert_eq!(transition.worktree, wt); + assert_eq!(transition.expected_revision, 0); + assert_eq!(transition.next_worktree_state.revision, 1); + assert_eq!( + transition.next_worktree_state.cursor_tree, + TreeId("tree1".to_string()) + ); + assert_eq!( + transition.scope_status_changes.get(&scope_id), + Some(&ScopeStatus::Active) + ); + assert_eq!( + transition.new_processed_event, + Some(EventKey { + scope_id: scope_id.clone(), + event_id, + }) + ); + assert_eq!(after.mutation_events.len(), 1); + assert_eq!( + transition.new_mutation_event.as_ref(), + after.mutation_events.iter().next() + ); + } + + #[test] + fn between_returns_some_with_correct_shape_for_an_advance_transition() { + let wt = WorktreeId("wt0".to_string()); + let scope_id = ScopeId("scope0".to_string()); + let before = state_with_scope(&wt, &scope_id, ActorKind::Codex, ScopeStatus::Active, 0); + + let attempt = AttemptId("attempt0".to_string()); + let event_id = EventId("event0".to_string()); + let prepared = prepare( + &before, + attempt.clone(), + Boundary::Advance { + scope: scope_id.clone(), + event: event_id.clone(), + }, + TreeId("tree1".to_string()), + ); + let after = commit(&prepared, &attempt).state; + + let transition = DurableTransition::between(&before, &after, &wt) + .unwrap() + .expect("an advance transition should produce a durable transition"); + + assert!(transition.scope_status_changes.is_empty()); + assert_eq!( + transition.new_processed_event, + Some(EventKey { scope_id, event_id }) + ); + assert_eq!(after.mutation_events.len(), 1); + assert_eq!(transition.next_worktree_state.revision, 1); + } + + #[test] + fn between_returns_some_with_correct_shape_for_a_close_transition() { + let wt = WorktreeId("wt0".to_string()); + let scope_id = ScopeId("scope0".to_string()); + let before = state_with_scope(&wt, &scope_id, ActorKind::Codex, ScopeStatus::Active, 0); + + let attempt = AttemptId("attempt0".to_string()); + let event_id = EventId("event0".to_string()); + let prepared = prepare( + &before, + attempt.clone(), + Boundary::Close { + scope: scope_id.clone(), + event: event_id.clone(), + }, + TreeId("tree0".to_string()), + ); + let after = commit(&prepared, &attempt).state; + + let transition = DurableTransition::between(&before, &after, &wt) + .unwrap() + .expect("a close transition should produce a durable transition"); + + assert_eq!( + transition.scope_status_changes.get(&scope_id), + Some(&ScopeStatus::Closed) + ); + assert_eq!( + transition.new_processed_event, + Some(EventKey { scope_id, event_id }) + ); + assert_eq!(transition.next_worktree_state.revision, 1); + } + + #[test] + fn between_returns_some_with_correct_shape_for_a_taint_transition() { + let wt = WorktreeId("wt0".to_string()); + let mut before = ProtocolState::default(); + before + .worktrees + .insert(wt.clone(), healthy_worktree_state(0)); + + let after = taint(&before, &wt); + + let transition = DurableTransition::between(&before, &after, &wt) + .unwrap() + .expect("a taint transition should produce a durable transition"); + + assert!(transition.scope_status_changes.is_empty()); + assert!(transition.new_processed_event.is_none()); + assert!(transition.new_mutation_event.is_none()); + assert!(transition.next_worktree_state.tainted); + assert_eq!( + transition.next_worktree_state.failure_kind, + FailureKind::SnapshotFailure + ); + assert_eq!(transition.next_worktree_state.revision, 1); + } + + #[test] + fn between_returns_some_with_correct_shape_for_an_abandon_transition() { + let wt = WorktreeId("wt0".to_string()); + let scope_id = ScopeId("scope0".to_string()); + let before = state_with_scope(&wt, &scope_id, ActorKind::Codex, ScopeStatus::Active, 0); + + let after = abandon(&before, &scope_id); + + let transition = DurableTransition::between(&before, &after, &wt) + .unwrap() + .expect("an abandon transition should produce a durable transition"); + + assert_eq!( + transition.scope_status_changes.get(&scope_id), + Some(&ScopeStatus::Abandoned) + ); + assert!(transition.next_worktree_state.needs_rebaseline); + assert_eq!(transition.next_worktree_state.revision, 1); + assert!(transition.new_processed_event.is_none()); + assert!(transition.new_mutation_event.is_none()); + } + + #[test] + fn between_returns_some_with_correct_shape_for_a_recover_transition() { + let wt = WorktreeId("wt0".to_string()); + let scope_id = ScopeId("scope0".to_string()); + let mut before = ProtocolState::default(); + before.worktrees.insert( + wt.clone(), + WorktreeState { + cursor_tree: TreeId("tree0".to_string()), + revision: 0, + tainted: true, + failure_kind: FailureKind::SnapshotFailure, + needs_rebaseline: false, + }, + ); + before.scopes.insert( + scope_id.clone(), + ScopeState { + status: ScopeStatus::Active, + actor_kind: ActorKind::Codex, + worktree_id: wt.clone(), + }, + ); + + let after = recover(&before, &wt, TreeId("tree1".to_string())); + + let transition = DurableTransition::between(&before, &after, &wt) + .unwrap() + .expect("a recover transition should produce a durable transition"); + + assert_eq!( + transition.scope_status_changes.get(&scope_id), + Some(&ScopeStatus::Abandoned) + ); + assert!(!transition.next_worktree_state.tainted); + assert_eq!( + transition.next_worktree_state.failure_kind, + FailureKind::Healthy + ); + assert_eq!( + transition.next_worktree_state.cursor_tree, + TreeId("tree1".to_string()) + ); + assert_eq!(transition.next_worktree_state.revision, 1); + } + + #[test] + fn between_errors_when_a_scopes_actor_kind_changes() { + let wt = WorktreeId("wt0".to_string()); + let scope_id = ScopeId("scope0".to_string()); + let before = state_with_scope(&wt, &scope_id, ActorKind::Codex, ScopeStatus::Active, 0); + let mut after = before.clone(); + after.scopes.get_mut(&scope_id).unwrap().actor_kind = ActorKind::ClaudeCode; + + let error = DurableTransition::between(&before, &after, &wt) + .expect_err("an actor_kind change must be rejected"); + assert!(error.to_string().contains("actor_kind")); + } + + #[test] + fn between_errors_when_a_scopes_worktree_id_changes() { + let wt = WorktreeId("wt0".to_string()); + let other_wt = WorktreeId("wt1".to_string()); + let scope_id = ScopeId("scope0".to_string()); + let before = state_with_scope(&wt, &scope_id, ActorKind::Codex, ScopeStatus::Active, 0); + let mut after = before.clone(); + after.scopes.get_mut(&scope_id).unwrap().worktree_id = other_wt; + + let error = DurableTransition::between(&before, &after, &wt) + .expect_err("a scope worktree_id change must be rejected"); + assert!(error.to_string().contains("worktree_id")); + } + + #[test] + fn between_errors_when_a_processed_event_disappears() { + let wt = WorktreeId("wt0".to_string()); + let mut before = ProtocolState::default(); + before + .worktrees + .insert(wt.clone(), healthy_worktree_state(0)); + before.processed_events.insert(EventKey { + scope_id: ScopeId("scope0".to_string()), + event_id: EventId("event0".to_string()), + }); + let mut after = before.clone(); + after.processed_events.clear(); + + let error = DurableTransition::between(&before, &after, &wt) + .expect_err("a disappearing processed event must be rejected"); + assert!(error.to_string().contains("processed_events")); + } + + #[test] + fn between_errors_when_a_mutation_event_disappears() { + let wt = WorktreeId("wt0".to_string()); + let mut before = ProtocolState::default(); + before + .worktrees + .insert(wt.clone(), healthy_worktree_state(0)); + before.mutation_events.insert(sample_mutation_event(&wt)); + let mut after = before.clone(); + after.mutation_events.clear(); + + let error = DurableTransition::between(&before, &after, &wt) + .expect_err("a disappearing mutation event must be rejected"); + assert!(error.to_string().contains("mutation_events")); + } + + #[test] + fn between_errors_when_a_new_mutation_events_revision_does_not_match_the_next_worktree_revision( + ) { + let wt = WorktreeId("wt0".to_string()); + let mut before = ProtocolState::default(); + before + .worktrees + .insert(wt.clone(), healthy_worktree_state(0)); + let mut after = before.clone(); + after.worktrees.get_mut(&wt).unwrap().revision = 1; + let mut mismatched_event = sample_mutation_event(&wt); + mismatched_event.revision = 2; + after.mutation_events.insert(mismatched_event); + + let error = DurableTransition::between(&before, &after, &wt).expect_err( + "a mutation event revision mismatched with the next worktree revision must be rejected", + ); + assert!(error.to_string().contains("revision")); + } + + #[test] + fn between_errors_when_an_unrelated_worktree_changes() { + let wt = WorktreeId("wt0".to_string()); + let other_wt = WorktreeId("wt1".to_string()); + let mut before = ProtocolState::default(); + before + .worktrees + .insert(wt.clone(), healthy_worktree_state(0)); + before + .worktrees + .insert(other_wt.clone(), healthy_worktree_state(0)); + + let mut after = before.clone(); + after.worktrees.get_mut(&other_wt).unwrap().revision = 1; + + let error = DurableTransition::between(&before, &after, &wt) + .expect_err("an unrelated worktree change must be rejected"); + assert!(error.to_string().contains("wt1")); + } + + #[test] + fn between_errors_when_the_revision_jumps_by_more_than_one() { + let wt = WorktreeId("wt0".to_string()); + let mut before = ProtocolState::default(); + before + .worktrees + .insert(wt.clone(), healthy_worktree_state(0)); + let mut after = before.clone(); + after.worktrees.get_mut(&wt).unwrap().revision = 2; + + let error = DurableTransition::between(&before, &after, &wt) + .expect_err("a revision jump of more than one must be rejected"); + assert!(error.to_string().contains("revision")); + } + + #[test] + fn between_errors_when_the_revision_decreases() { + let wt = WorktreeId("wt0".to_string()); + let mut before = ProtocolState::default(); + before + .worktrees + .insert(wt.clone(), healthy_worktree_state(5)); + let mut after = before.clone(); + after.worktrees.get_mut(&wt).unwrap().revision = 4; + + let error = DurableTransition::between(&before, &after, &wt) + .expect_err("a revision decrease must be rejected"); + assert!(error.to_string().contains("revision")); + } + + #[test] + fn between_errors_when_a_scope_unexpectedly_appears() { + let wt = WorktreeId("wt0".to_string()); + let scope_id = ScopeId("scope0".to_string()); + let mut before = ProtocolState::default(); + before + .worktrees + .insert(wt.clone(), healthy_worktree_state(0)); + let mut after = before.clone(); + after.scopes.insert( + scope_id, + ScopeState { + status: ScopeStatus::NeverSeen, + actor_kind: ActorKind::Codex, + worktree_id: wt.clone(), + }, + ); + + let error = DurableTransition::between(&before, &after, &wt) + .expect_err("an unexpectedly appearing scope must be rejected"); + assert!(error.to_string().contains("scope set")); + } + + #[test] + fn between_errors_when_a_scope_unexpectedly_disappears() { + let wt = WorktreeId("wt0".to_string()); + let scope_id = ScopeId("scope0".to_string()); + let before = state_with_scope(&wt, &scope_id, ActorKind::Codex, ScopeStatus::Active, 0); + let mut after = before.clone(); + after.scopes.remove(&scope_id); + + let error = DurableTransition::between(&before, &after, &wt) + .expect_err("an unexpectedly disappearing scope must be rejected"); + assert!(error.to_string().contains("scope set")); + } } diff --git a/context/plans/mutation-cursor-store-persistence.md b/context/plans/mutation-cursor-store-persistence.md index f70f22b7..98587f26 100644 --- a/context/plans/mutation-cursor-store-persistence.md +++ b/context/plans/mutation-cursor-store-persistence.md @@ -199,13 +199,23 @@ Persist this field in every plan; this is durable plan state, not chat state: - Context impact: local — the guard and its tests are confined to `register_scope` in `store.rs`, still not called from any hook, command, or CAS commit path; no durable context file describes `register_scope`'s behavior yet (deferred to T11/plan-level context sync per the plan's assumption), so this correction changes no root context file's claims. - Context synchronization: synced -- [ ] T05: `Add DurableTransition structural diff type` (status:todo) +- [x] T05: `Add DurableTransition structural diff type` (status:done) - Task ID: T05 - - Scope: In — `DurableTransition` and `DurableTransition::between(before, after, worktree) -> Result>` performing pure structural diffing only, enforcing: the target worktree exists in both `before` and `after` and is never added or removed; no unrelated worktree changes; when a durable transition exists, its worktree's next revision is exactly `expected_revision + 1` computed via checked `u64` arithmetic; no scope is added or deleted; a changed scope belongs to the target worktree; `ScopeState.worktree_id` and `ScopeState.actor_kind` never change (only `status` may); `processed_events` may only gain entries, never lose them, with at most one new entry whose scope belongs to the target worktree; `mutation_events` may only gain entries, never lose them, with at most one new entry belonging to the target worktree; `AttemptState`/`external_taint` differences are ignored. Out — SQL/DB code (T06/T07). + - Scope: In — `DurableTransition` and `DurableTransition::between(before, after, worktree) -> Result>` performing pure structural diffing only, enforcing: the target worktree exists in both `before` and `after` and is never added or removed; no unrelated worktree changes; when a durable transition exists, its worktree's next revision is exactly `expected_revision + 1` computed via checked `u64` arithmetic; no scope is added or deleted; a changed scope belongs to the target worktree; `ScopeState.worktree_id` and `ScopeState.actor_kind` never change (only `status` may); `processed_events` may only gain entries, never lose them, with at most one new entry whose scope belongs to the target worktree; `mutation_events` may only gain entries, never lose them, with at most one new entry belonging to the target worktree, and when present its `revision` must equal the target worktree's resulting revision; `AttemptState`/`external_taint` differences are ignored. Out — SQL/DB code (T06/T07). - Dependencies: T02 - - Done when: `between()` returns `Ok(None)` for a `database_failure`-only transition and for a no-change `Flush`; returns `Ok(Some(..))` with the correct shape for `Start`/`Advance`/`Close`, `taint`, `abandon`, and `recover` transitions exercised directly against `protocol::*` outputs; the function contains no boundary-kind, contention, or taint conditionals; it returns `Err` for a malformed `before`/`after` pair covering at least: an `actor_kind` change, a scope's `worktree_id` change, a processed `EventKey` disappearing, a `MutationEvent` disappearing, an unrelated worktree changing, a revision jump by more than 1, a revision decrease, and a scope unexpectedly appearing or disappearing. + - Done when: `between()` returns `Ok(None)` for a `database_failure`-only transition and for a no-change `Flush`; returns `Ok(Some(..))` with the correct shape for `Start`/`Advance`/`Close`, `taint`, `abandon`, and `recover` transitions exercised directly against `protocol::*` outputs; the function contains no boundary-kind, contention, or taint conditionals; it returns `Err` for a malformed `before`/`after` pair covering at least: an `actor_kind` change, a scope's `worktree_id` change, a processed `EventKey` disappearing, a `MutationEvent` disappearing, an unrelated worktree changing, a revision jump by more than 1, a revision decrease, a scope unexpectedly appearing or disappearing, and a new `MutationEvent` whose `revision` does not match the next worktree revision. - Verify: `./scripts/run-cli-cargo.sh test --manifest-path cli/Cargo.toml mutation_trace::store::` - - Context synchronization: pending + - Completed: 2026-08-27 + - Files changed: `cli/src/services/mutation_trace/store.rs` + - Result: Added `DurableTransition` (`worktree`, `expected_revision`, `next_worktree_state`, `scope_status_changes`, `new_processed_event`, `new_mutation_event`) and `DurableTransition::between(before, after, worktree)`, decomposed into four private free functions — `diff_target_worktree` (validates the target worktree exists in both states and rejects any change to a different worktree key), `diff_scopes` (rejects an added/deleted `ScopeId`, rejects `worktree_id`/`actor_kind` drift on any scope, collects status-only changes and rejects a status change on a scope not belonging to the target worktree), `diff_new_processed_event`, and `diff_new_mutation_event` (both append-only via `BTreeSet::is_subset`/`difference`, capped at one new entry, and validated to belong to the target worktree). `between` composes these four diffs: if the target worktree's `WorktreeState` is unchanged and no scope/processed-event/mutation-event diff exists, returns `Ok(None)` (covers both a `database_failure`-only transition, which touches only the ignored `external_taint` field, and a fresh no-change `Flush`, which per `protocol.rs` leaves every persisted field untouched); otherwise requires `after`'s revision to equal `before`'s `expected_revision.checked_add(1)` exactly, or returns `Err`. No code path inspects `Boundary`, `BoundaryKind`, `Attribution`, or taint state — the diff is purely structural. Added 17 tests to `store.rs`'s existing `#[cfg(test)] mod tests`: two `Ok(None)` cases (`database_failure`-only, no-change `Flush`), six `Ok(Some(..))` shape cases exercised directly against `protocol::prepare`/`commit`/`taint`/`abandon`/`recover` (`Start`, `Advance`, `Close`, `taint`, `abandon`, `recover`), and nine malformed-pair `Err` cases (actor_kind change, scope `worktree_id` change, disappearing processed event, disappearing mutation event, unrelated worktree changing, revision jump >1, revision decrease, scope unexpectedly appearing, scope unexpectedly disappearing). + - Verify: `./scripts/run-cli-cargo.sh test --manifest-path cli/Cargo.toml mutation_trace::store::` — passed, 52/52 (35 pre-existing + 17 new); `./scripts/run-cli-cargo.sh fmt --manifest-path cli/Cargo.toml -- --check` — passed after one auto-formatting pass; `./scripts/run-cli-cargo.sh clippy --manifest-path cli/Cargo.toml --all-targets -- -D warnings` — passed, zero warnings (required decomposing the initial single-function `between` into the four private diff functions above to satisfy `clippy::too_many_lines` under this crate's `-D clippy::pedantic`). + - Done checks: `between()` returns `Ok(None)` for a `database_failure`-only transition (verified by `between_returns_none_for_a_database_failure_only_transition`) and for a no-change `Flush` (verified by `between_returns_none_for_a_no_change_flush`); returns `Ok(Some(..))` with the correct shape for `Start`/`Advance`/`Close`/`taint`/`abandon`/`recover` (verified by the six shape tests, each asserting `expected_revision`, `next_worktree_state`, `scope_status_changes`, `new_processed_event`, and `new_mutation_event` where applicable); the function contains no boundary-kind, contention, or taint conditionals (verified by inspection — `between` and its four diff helpers reference only `ProtocolState`/`WorktreeState`/`ScopeState`/`EventKey`/`MutationEvent` fields, never `Boundary`, `BoundaryKind`, `Attribution`, or `AttributionKind`); returns `Err` for all eight listed malformed-pair cases (verified by the nine malformed-pair tests, `scope unexpectedly appearing or disappearing` covered by two tests). + - Context impact: local — a new pure diffing type and its private helpers confined to `store.rs`, not yet called from any SQL/DB code, hook, command, or `coordinator.rs`/`git_snapshot.rs` seam (T07 lands the first caller, translating `DurableTransition` into the CAS `UPDATE`/`INSERT` batch). `context/cli/mutation-trace-protocol.md`'s "not yet wired into any hook, command, or database call site" framing remains accurate. No durable context file describes this yet; deferred to T11/plan-level context sync once the full store lands, per the plan's assumption. + - **T05 correction (2026-08-27):** Fixed a structural-integrity gap where `DurableTransition::between` validated that the target worktree's revision advances by exactly one but never checked a new `MutationEvent`'s own `revision` field against that resulting worktree revision, so a durable transition carrying a `MutationEvent` with an arbitrary/mismatched `revision` (e.g. worktree `7 -> 8` paired with an event `revision = 999`) was accepted as valid. Added a check in `between`, placed immediately after the existing revision-advance check and before constructing `Self`: `if let Some(event) = &new_mutation_event { if event.revision != after_worktree_state.revision { bail!(...) } }`, reusing `after_worktree_state.revision` (already validated as `expected_revision + 1`) rather than recomputing the arithmetic. The check inspects only `WorktreeState.revision` and `MutationEvent.revision` — no `Boundary`, `BoundaryKind`, `Attribution`, contention, or taint state — keeping `between` purely structural. Verified by inspection that every `protocol.rs` call site constructing a `MutationEvent` (`crate::services::mutation_trace::protocol.rs:361-371`) sources its `revision` field from the same `advanced_revision` value written into the worktree's next `WorktreeState.revision` (`protocol.rs:340-345`), so no protocol-produced transition was weakened or required changes. Added one test, `between_errors_when_a_new_mutation_events_revision_does_not_match_the_next_worktree_revision`: worktree revision `0 -> 1`, one new `MutationEvent` belonging to the target worktree with `revision = 2` (all other fields — scope set, worktree set, processed events — unchanged/valid), asserting `Err` containing `"revision"`. + - Verify (T05 correction): `./scripts/run-cli-cargo.sh test --manifest-path cli/Cargo.toml mutation_trace::store::` — passed, 53/53 (52 pre-existing + 1 new); `./scripts/run-cli-cargo.sh fmt --manifest-path cli/Cargo.toml -- --check` — passed after one auto-formatting pass; `./scripts/run-cli-cargo.sh clippy --manifest-path cli/Cargo.toml --all-targets -- -D warnings` — passed, zero warnings. + - Done checks (T05 correction): a new `MutationEvent` whose `revision` does not match the next worktree revision returns `Err` (verified by the new test, isolated so only `event.revision = 2` is malformed — worktree set unchanged, scope set valid, no unrelated worktree changes, event belongs to the target worktree, exactly one mutation event added, no processed-event violation, worktree itself advances `0 -> 1`); all six existing protocol-produced shape tests (`Start`/`Advance`/`Close`/`taint`/`abandon`/`recover`) still pass unmodified, confirming real `commit`-produced `MutationEvent`s already satisfy `event.revision == after.worktrees[target].revision`; `protocol.rs` was not modified; the check inspects only `WorktreeState`/`MutationEvent` fields already present, per inspection of the diff. + - Context impact: local — the added check and its test are confined to `DurableTransition::between` in `store.rs`, still not called from any SQL/DB code, hook, command, or CAS commit path; no durable context file describes this yet, per the plan's existing T05 context-impact note. + - Context synchronization: synced - [ ] T06: `Add generic transactional CAS batch primitive to TursoDb` (status:todo) - Task ID: T06 From 329c54c7d7fa066e4e5d78abbd4584f04b98705e Mon Sep 17 00:00:00 2001 From: David Abram Date: Fri, 28 Aug 2026 00:21:06 +0200 Subject: [PATCH 08/15] runtime: Add transactional CAS batches to TursoDb Provide a generic transaction primitive for mutation persistence to atomically apply a CAS guard and ordered statements. Retry only Busy and BusySnapshot failures from a fresh BEGIN IMMEDIATE transaction while returning deterministic failures without retrying; add coverage for no-op, success, rollback, and contention. Plan: mutation-cursor-store-persistence (T06) Co-authored-by: SCE --- cli/src/services/db/mod.rs | 571 +++++++++++++++++- .../mutation-cursor-store-persistence.md | 16 +- 2 files changed, 583 insertions(+), 4 deletions(-) diff --git a/cli/src/services/db/mod.rs b/cli/src/services/db/mod.rs index 413b6a3b..14f9de54 100644 --- a/cli/src/services/db/mod.rs +++ b/cli/src/services/db/mod.rs @@ -321,6 +321,117 @@ async fn execute_insert_pair_if_absent_body( Ok(true) } +#[allow(dead_code)] +pub struct TransactionStatement<'a> { + sql: &'a str, + params: turso::params::Params, + expected_rows_affected: Option, +} + +impl<'a> TransactionStatement<'a> { + #[allow(dead_code)] + pub fn new(sql: &'a str, params: impl turso::params::IntoParams) -> Result { + let params = turso::params::IntoParams::into_params(params) + .map_err(|e| anyhow::anyhow!("parameter conversion failed: {sql}: {e}"))?; + + Ok(Self { + sql, + params, + expected_rows_affected: None, + }) + } + + #[allow(dead_code)] + pub fn expect_rows_affected(mut self, expected: u64) -> Self { + self.expected_rows_affected = Some(expected); + self + } +} + +#[allow(dead_code)] +fn is_retryable_turso_error(error: &turso::Error) -> bool { + matches!(error, turso::Error::Busy(_) | turso::Error::BusySnapshot(_)) +} + +#[allow(dead_code)] +enum CasBatchFailure { + Retryable(anyhow::Error), + Deterministic(anyhow::Error), +} + +#[allow(dead_code)] +fn classify_turso_error(db_name: &str, action: &str, error: &turso::Error) -> CasBatchFailure { + let wrapped = anyhow::anyhow!("{db_name} {action}: {error}"); + + if is_retryable_turso_error(error) { + CasBatchFailure::Retryable(wrapped) + } else { + CasBatchFailure::Deterministic(wrapped) + } +} + +#[allow(dead_code)] +enum CasBatchAttemptOutcome { + Settled(bool), + Deterministic(anyhow::Error), +} + +#[allow(dead_code)] +fn cas_batch_failure_into_attempt_result( + failure: CasBatchFailure, +) -> Result { + match failure { + CasBatchFailure::Retryable(err) => Err(err), + CasBatchFailure::Deterministic(err) => Ok(CasBatchAttemptOutcome::Deterministic(err)), + } +} + +#[allow(dead_code)] +async fn execute_cas_batch_body( + tx: &turso::transaction::Transaction<'_>, + db_name: &str, + guard: &TransactionStatement<'_>, + statements: &[TransactionStatement<'_>], +) -> std::result::Result { + let guard_rows_affected = tx + .execute(guard.sql, guard.params.clone()) + .await + .map_err(|e| { + classify_turso_error(db_name, &format!("execute failed: {}", guard.sql), &e) + })?; + + match guard_rows_affected { + 0 => return Ok(false), + 1 => {} + n => { + return Err(CasBatchFailure::Deterministic(anyhow::anyhow!( + "{db_name} CAS guard affected {n} rows; expected 0 or 1: {}", + guard.sql + ))); + } + } + + for statement in statements { + let rows_affected = tx + .execute(statement.sql, statement.params.clone()) + .await + .map_err(|e| { + classify_turso_error(db_name, &format!("execute failed: {}", statement.sql), &e) + })?; + + if let Some(expected) = statement.expected_rows_affected { + if rows_affected != expected { + return Err(CasBatchFailure::Deterministic(anyhow::anyhow!( + "{db_name} statement affected {rows_affected} rows; expected {expected}: {}", + statement.sql + ))); + } + } + } + + Ok(true) +} + struct TursoConnectionCore { conn: turso::Connection, runtime: tokio::runtime::Runtime, @@ -738,6 +849,62 @@ impl TursoDb { Ok(results) } + #[allow(dead_code)] + pub fn execute_transactional_cas_batch( + &self, + operation_name: &str, + retry_hint: &str, + guard: &TransactionStatement<'_>, + statements: &[TransactionStatement<'_>], + ) -> Result { + let db_name = M::db_name(); + + let outcome = run_with_retry_sync( + resolve_query_retry_policy::(), + operation_name, + retry_hint, + |_| { + block_on_isolated(&self.core.runtime, async { + let tx = match turso::transaction::Transaction::new_unchecked( + &self.core.conn, + turso::transaction::TransactionBehavior::Immediate, + ) + .await + { + Ok(tx) => tx, + Err(e) => { + return cas_batch_failure_into_attempt_result(classify_turso_error( + db_name, + "failed to begin transaction", + &e, + )); + } + }; + + match execute_cas_batch_body(&tx, db_name, guard, statements).await { + Ok(applied) => match tx.commit().await { + Ok(()) => Ok(CasBatchAttemptOutcome::Settled(applied)), + Err(e) => cas_batch_failure_into_attempt_result(classify_turso_error( + db_name, + "failed to commit transaction", + &e, + )), + }, + Err(failure) => { + let _ = tx.rollback().await; + cas_batch_failure_into_attempt_result(failure) + } + } + }) + }, + )?; + + match outcome { + CasBatchAttemptOutcome::Settled(applied) => Ok(applied), + CasBatchAttemptOutcome::Deterministic(err) => Err(err), + } + } + /// Run all embedded migrations in order. /// /// Applied migration IDs are recorded in `__sce_migrations` so later @@ -1057,7 +1224,8 @@ impl EncryptedTursoDb { #[cfg(test)] mod tests { - use std::time::{SystemTime, UNIX_EPOCH}; + use std::thread; + use std::time::{Duration, Instant, SystemTime, UNIX_EPOCH}; use super::*; @@ -1112,6 +1280,407 @@ mod tests { } } + fn open_cas_test_db() -> (TursoDb, PathBuf) { + let db_path = unique_test_db_path(); + let db = TursoDb::::new_at(&db_path).expect("test DB should open"); + db.execute( + "CREATE TABLE IF NOT EXISTS cas_target (id INTEGER PRIMARY KEY, revision INTEGER NOT NULL)", + (), + ) + .expect("cas_target table creation should succeed"); + db.execute( + "CREATE TABLE IF NOT EXISTS cas_effect (name TEXT PRIMARY KEY)", + (), + ) + .expect("cas_effect table creation should succeed"); + db.execute("INSERT INTO cas_target (id, revision) VALUES (1, 0)", ()) + .expect("cas_target seed row should insert"); + + (db, db_path) + } + + fn cas_target_revision(db: &TursoDb, id: i64) -> i64 { + db.query_map( + "SELECT revision FROM cas_target WHERE id = ?1", + (id,), + |row| row.get::(0).map_err(Into::into), + ) + .expect("cas_target revision read should succeed") + .into_iter() + .next() + .expect("cas_target seed row should exist") + } + + fn cas_effect_names(db: &TursoDb) -> Vec { + db.query_map("SELECT name FROM cas_effect ORDER BY name", (), |row| { + row.get::(0).map_err(Into::into) + }) + .expect("cas_effect read should succeed") + } + + #[test] + fn execute_transactional_cas_batch_returns_false_and_runs_nothing_when_guard_matches_no_rows() { + let (db, db_path) = open_cas_test_db(); + let guard = TransactionStatement::new( + "UPDATE cas_target SET revision = 1 WHERE id = 1 AND revision = 999", + (), + ) + .expect("guard statement should build"); + let statements = + [ + TransactionStatement::new("INSERT INTO cas_effect (name) VALUES ('applied')", ()) + .expect("effect statement should build"), + ]; + + let applied = db + .execute_transactional_cas_batch("cas test", "retry the operation", &guard, &statements) + .expect("no-op CAS batch should succeed"); + + assert!(!applied); + assert_eq!(cas_target_revision(&db, 1), 0); + assert!(cas_effect_names(&db).is_empty()); + + cleanup_test_db(db, &db_path); + } + + #[test] + fn execute_transactional_cas_batch_returns_true_and_runs_every_statement_when_guard_matches_one_row( + ) { + let (db, db_path) = open_cas_test_db(); + let guard = TransactionStatement::new( + "UPDATE cas_target SET revision = 1 WHERE id = 1 AND revision = 0", + (), + ) + .expect("guard statement should build"); + let statements = + [ + TransactionStatement::new("INSERT INTO cas_effect (name) VALUES ('applied')", ()) + .expect("effect statement should build"), + ]; + + let applied = db + .execute_transactional_cas_batch("cas test", "retry the operation", &guard, &statements) + .expect("applied CAS batch should succeed"); + + assert!(applied); + assert_eq!(cas_target_revision(&db, 1), 1); + assert_eq!(cas_effect_names(&db), vec![String::from("applied")]); + + cleanup_test_db(db, &db_path); + } + + #[test] + fn execute_transactional_cas_batch_rolls_back_and_fails_after_one_attempt_on_deterministic_failure( + ) { + let (db, db_path) = open_cas_test_db(); + db.execute("INSERT INTO cas_effect (name) VALUES ('applied')", ()) + .expect("pre-existing conflicting row should insert"); + let guard = TransactionStatement::new( + "UPDATE cas_target SET revision = 1 WHERE id = 1 AND revision = 0", + (), + ) + .expect("guard statement should build"); + let statements = + [ + TransactionStatement::new("INSERT INTO cas_effect (name) VALUES ('applied')", ()) + .expect("effect statement should build"), + ]; + + let started_at = Instant::now(); + let error = db + .execute_transactional_cas_batch("cas test", "retry the operation", &guard, &statements) + .expect_err("duplicate insert should fail deterministically"); + let elapsed = started_at.elapsed(); + + assert!( + elapsed < Duration::from_millis(150), + "deterministic failure appears to have been retried instead of failing after one attempt: {elapsed:?}" + ); + assert!(error.to_string().contains("execute failed")); + assert_eq!(cas_target_revision(&db, 1), 0); + assert_eq!(cas_effect_names(&db), vec![String::from("applied")]); + + cleanup_test_db(db, &db_path); + } + + #[test] + fn execute_transactional_cas_batch_rejects_a_guard_matching_more_than_one_row_without_retrying() + { + let (db, db_path) = open_cas_test_db(); + db.execute("INSERT INTO cas_target (id, revision) VALUES (2, 0)", ()) + .expect("second cas_target row should insert"); + let guard = + TransactionStatement::new("UPDATE cas_target SET revision = 1 WHERE revision = 0", ()) + .expect("guard statement should build"); + let statements = + [ + TransactionStatement::new("INSERT INTO cas_effect (name) VALUES ('applied')", ()) + .expect("effect statement should build"), + ]; + + let started_at = Instant::now(); + let error = db + .execute_transactional_cas_batch("cas test", "retry the operation", &guard, &statements) + .expect_err("a guard matching more than one row should fail deterministically"); + let elapsed = started_at.elapsed(); + + assert!( + elapsed < Duration::from_millis(150), + "guard over-match appears to have been retried instead of failing after one attempt: {elapsed:?}" + ); + assert!(error.to_string().contains("expected 0 or 1")); + assert_eq!(cas_target_revision(&db, 1), 0); + assert_eq!(cas_target_revision(&db, 2), 0); + assert!(cas_effect_names(&db).is_empty()); + + cleanup_test_db(db, &db_path); + } + + #[test] + fn execute_transactional_cas_batch_applies_a_statement_whose_expected_rows_affected_matches() { + let (db, db_path) = open_cas_test_db(); + let guard = TransactionStatement::new( + "UPDATE cas_target SET revision = 1 WHERE id = 1 AND revision = 0", + (), + ) + .expect("guard statement should build"); + let statements = + [ + TransactionStatement::new("INSERT INTO cas_effect (name) VALUES ('applied')", ()) + .expect("effect statement should build") + .expect_rows_affected(1), + ]; + + let applied = db + .execute_transactional_cas_batch("cas test", "retry the operation", &guard, &statements) + .expect("a statement matching its row expectation should succeed"); + + assert!(applied); + assert_eq!(cas_target_revision(&db, 1), 1); + assert_eq!(cas_effect_names(&db), vec![String::from("applied")]); + + cleanup_test_db(db, &db_path); + } + + #[test] + fn execute_transactional_cas_batch_rejects_a_statement_affecting_fewer_rows_than_expected() { + let (db, db_path) = open_cas_test_db(); + let guard = TransactionStatement::new( + "UPDATE cas_target SET revision = 1 WHERE id = 1 AND revision = 0", + (), + ) + .expect("guard statement should build"); + let statements = [TransactionStatement::new( + "UPDATE cas_effect SET name = 'applied' WHERE name = 'missing'", + (), + ) + .expect("effect statement should build") + .expect_rows_affected(1)]; + + let error = db + .execute_transactional_cas_batch("cas test", "retry the operation", &guard, &statements) + .expect_err("a statement affecting zero rows should fail its row expectation"); + + assert!(error.to_string().contains("affected 0 rows")); + assert!(error.to_string().contains("expected 1")); + assert_eq!(cas_target_revision(&db, 1), 0); + + cleanup_test_db(db, &db_path); + } + + #[test] + fn execute_transactional_cas_batch_rejects_a_statement_affecting_more_rows_than_expected() { + let (db, db_path) = open_cas_test_db(); + db.execute("INSERT INTO cas_effect (name) VALUES ('a')", ()) + .expect("first pre-existing effect row should insert"); + db.execute("INSERT INTO cas_effect (name) VALUES ('b')", ()) + .expect("second pre-existing effect row should insert"); + let guard = TransactionStatement::new( + "UPDATE cas_target SET revision = 1 WHERE id = 1 AND revision = 0", + (), + ) + .expect("guard statement should build"); + let statements = + [ + TransactionStatement::new("DELETE FROM cas_effect WHERE name IN ('a', 'b')", ()) + .expect("effect statement should build") + .expect_rows_affected(1), + ]; + + let error = db + .execute_transactional_cas_batch("cas test", "retry the operation", &guard, &statements) + .expect_err( + "a statement affecting more rows than expected should fail its row expectation", + ); + + assert!(error.to_string().contains("affected 2 rows")); + assert!(error.to_string().contains("expected 1")); + assert_eq!(cas_target_revision(&db, 1), 0); + assert_eq!( + cas_effect_names(&db), + vec![String::from("a"), String::from("b")] + ); + + cleanup_test_db(db, &db_path); + } + + #[test] + fn execute_transactional_cas_batch_allows_a_statement_with_no_row_expectation_to_affect_zero_rows( + ) { + let (db, db_path) = open_cas_test_db(); + let guard = TransactionStatement::new( + "UPDATE cas_target SET revision = 1 WHERE id = 1 AND revision = 0", + (), + ) + .expect("guard statement should build"); + let statements = [TransactionStatement::new( + "UPDATE cas_effect SET name = 'applied' WHERE name = 'missing'", + (), + ) + .expect("effect statement should build")]; + + let applied = db + .execute_transactional_cas_batch("cas test", "retry the operation", &guard, &statements) + .expect("a statement with no row expectation should not enforce a row count"); + + assert!(applied); + assert_eq!(cas_target_revision(&db, 1), 1); + + cleanup_test_db(db, &db_path); + } + + #[test] + fn is_retryable_turso_error_classifies_busy_and_busy_snapshot_as_retryable() { + assert!(is_retryable_turso_error(&turso::Error::Busy(String::from( + "database is locked" + )))); + assert!(is_retryable_turso_error(&turso::Error::BusySnapshot( + String::from("snapshot is busy") + ))); + } + + #[test] + fn is_retryable_turso_error_classifies_every_other_variant_as_deterministic() { + assert!(!is_retryable_turso_error(&turso::Error::Constraint( + String::from("UNIQUE constraint failed") + ))); + assert!(!is_retryable_turso_error(&turso::Error::Misuse( + String::from("misuse") + ))); + assert!(!is_retryable_turso_error(&turso::Error::Corrupt( + String::from("corrupt") + ))); + assert!(!is_retryable_turso_error(&turso::Error::NotAdb( + String::from("not a database") + ))); + assert!(!is_retryable_turso_error(&turso::Error::DatabaseFull( + String::from("database full") + ))); + assert!(!is_retryable_turso_error(&turso::Error::Readonly( + String::from("readonly") + ))); + assert!(!is_retryable_turso_error(&turso::Error::Error( + String::from("generic error") + ))); + assert!(!is_retryable_turso_error(&turso::Error::IoError( + std::io::ErrorKind::Other, + "io" + ))); + } + + #[test] + fn classify_turso_error_wraps_busy_as_retryable_with_the_supplied_action_context() { + let failure = classify_turso_error( + "test", + "failed to begin transaction", + &turso::Error::Busy(String::from("database is locked")), + ); + + match failure { + CasBatchFailure::Retryable(err) => { + let message = err.to_string(); + assert!(message.contains("failed to begin transaction")); + assert!(message.contains("database is locked")); + } + CasBatchFailure::Deterministic(err) => { + panic!("Busy should classify as retryable, got deterministic: {err}") + } + } + } + + #[test] + fn classify_turso_error_wraps_constraint_violations_as_deterministic_with_the_supplied_action_context( + ) { + let failure = classify_turso_error( + "test", + "failed to commit transaction", + &turso::Error::Constraint(String::from("UNIQUE constraint failed")), + ); + + match failure { + CasBatchFailure::Deterministic(err) => { + let message = err.to_string(); + assert!(message.contains("failed to commit transaction")); + assert!(message.contains("UNIQUE constraint failed")); + } + CasBatchFailure::Retryable(err) => { + panic!("Constraint should classify as deterministic, got retryable: {err}") + } + } + } + + #[test] + fn execute_transactional_cas_batch_retries_a_begin_immediate_busy_error_and_then_succeeds() { + const LOCK_HOLD_MS: u64 = 60; + + let (db, db_path) = open_cas_test_db(); + let lock_holder = + TursoDb::::new_at(&db_path).expect("second handle should open"); + lock_holder + .execute("BEGIN IMMEDIATE", ()) + .expect("lock holder should acquire the write lock before any guard or statement runs"); + + let hold_handle = thread::spawn(move || { + thread::sleep(Duration::from_millis(LOCK_HOLD_MS)); + lock_holder + .execute("COMMIT", ()) + .expect("lock holder should release the write lock"); + }); + + let guard = TransactionStatement::new( + "UPDATE cas_target SET revision = 1 WHERE id = 1 AND revision = 0", + (), + ) + .expect("guard statement should build"); + let statements = + [ + TransactionStatement::new("INSERT INTO cas_effect (name) VALUES ('applied')", ()) + .expect("effect statement should build"), + ]; + + let started_at = Instant::now(); + let applied = db + .execute_transactional_cas_batch("cas test", "retry the operation", &guard, &statements) + .expect( + "CAS batch should retry BEGIN IMMEDIATE through the transient lock and succeed", + ); + let elapsed = started_at.elapsed(); + + hold_handle + .join() + .expect("lock holder thread should finish"); + + assert!( + elapsed >= Duration::from_millis(LOCK_HOLD_MS / 2), + "success arrived before the lock holder could plausibly have released the write lock, meaning BEGIN IMMEDIATE contention was not actually retried: {elapsed:?}" + ); + assert!(applied); + assert_eq!(cas_target_revision(&db, 1), 1); + assert_eq!(cas_effect_names(&db), vec![String::from("applied")]); + + cleanup_test_db(db, &db_path); + } + #[test] fn passive_checkpoint_keeps_previously_written_data_readable() { let (db, db_path) = open_test_db(); diff --git a/context/plans/mutation-cursor-store-persistence.md b/context/plans/mutation-cursor-store-persistence.md index 98587f26..936657e3 100644 --- a/context/plans/mutation-cursor-store-persistence.md +++ b/context/plans/mutation-cursor-store-persistence.md @@ -217,13 +217,23 @@ Persist this field in every plan; this is durable plan state, not chat state: - Context impact: local — the added check and its test are confined to `DurableTransition::between` in `store.rs`, still not called from any SQL/DB code, hook, command, or CAS commit path; no durable context file describes this yet, per the plan's existing T05 context-impact note. - Context synchronization: synced -- [ ] T06: `Add generic transactional CAS batch primitive to TursoDb` (status:todo) +- [x] T06: `Add generic transactional CAS batch primitive to TursoDb` (status:done) - Task ID: T06 - Scope: In — `TransactionStatement` and `TursoDb::execute_transactional_cas_batch(operation_name, retry_hint, guard, statements)` in `cli/src/services/db/mod.rs`, with a retryability contract distinct from the shared `run_with_retry_sync` helper's plain any-`Err`-retries behavior: a guard affecting 0 rows commits as a no-op and returns `Ok(false)` (a normal CAS conflict) without running any statement and without being retried; a guard affecting 1 row runs every statement inside the same `BEGIN IMMEDIATE` transaction and returns `Ok(true)`; a retryable DB failure (lock/busy/other transient condition) retries the entire transaction from `BEGIN IMMEDIATE`; a deterministic failure (SQL/schema/constraint/invariant violation) returns `Err` without being retried. This adds the minimum local retryability classification needed for that behavior — for example, the retried closure returns a classified outcome that `run_with_retry_sync` still treats as `Ok` so it never retries a deterministic failure, and the caller re-raises that failure as `Err` once the closure returns — without changing `resilience.rs` or any other caller of `run_with_retry_sync`. Out — mutation-trace-specific SQL (T07). - Dependencies: none - - Done when: a guard affecting 0 rows commits as a no-op and returns `Ok(false)` without running any statement or waiting for a retry backoff; a guard affecting 1 row runs every statement and returns `Ok(true)`; an injected deterministic mid-batch failure rolls back the entire transaction (including the guard's own effect) and surfaces as `Err` after exactly one attempt, never reported as a CAS conflict; an injected retryable DB failure retries the whole transaction from `BEGIN IMMEDIATE` (never individual statements) up to the configured attempt count. + - Done when: a guard affecting 0 rows commits as a no-op and returns `Ok(false)` without running any statement or waiting for a retry backoff; a guard affecting 1 row runs every statement and returns `Ok(true)`; a guard affecting more than 1 row is a deterministic invariant violation that rolls back and returns `Err` without running the remaining statements and without being retried; an injected deterministic mid-batch failure rolls back the entire transaction (including the guard's own effect) and surfaces as `Err` after exactly one attempt, never reported as a CAS conflict; an injected retryable DB failure retries the whole transaction from `BEGIN IMMEDIATE` (never individual statements) up to the configured attempt count; `BEGIN`, the guard, every statement, and `COMMIT` all classify a `turso::Error` through the same retryability seam, so a `Busy`/`BusySnapshot` failure at any of those four points retries the whole attempt from a fresh `BEGIN IMMEDIATE` and every other `turso::Error` variant fails deterministically after exactly one attempt; `TransactionStatement` can optionally require an exact affected-row count (`expect_rows_affected`), and a mismatch is a deterministic invariant violation that rolls back the whole transaction and returns `Err` without being retried, while a statement with no expectation preserves the original any-row-count-is-fine behavior. - Verify: `./scripts/run-cli-cargo.sh test --manifest-path cli/Cargo.toml db::` - - Context synchronization: pending + - Completed: 2026-08-28 + - Files changed: `cli/src/services/db/mod.rs` + - Result: Added `TransactionStatement<'a>` (a `sql: &'a str` paired with pre-converted `turso::params::Params`, built via `TransactionStatement::new(sql, params)`) and `TursoDb::execute_transactional_cas_batch(operation_name, retry_hint, guard: &TransactionStatement, statements: &[TransactionStatement]) -> Result`. The private async body `execute_cas_batch_body` runs `guard` first inside one `BEGIN IMMEDIATE` transaction; `0` affected rows short-circuits to `Ok(false)` without touching `statements`; `1` affected row runs every `statements` entry in order. Failure classification happens before any `anyhow` wrapping: a new `is_retryable_turso_error` matches only `turso::Error::Busy`/`turso::Error::BusySnapshot` as retryable — mirroring this crate's own internal treatment of those two variants (`turso`'s `sync.rs:526`) — and every other `turso::Error` variant (`Constraint`, `Misuse`, `Corrupt`, `NotAdb`, `DatabaseFull`, `Readonly`, `IoError`, `Error`, conversion failures) as deterministic, via a new `CasBatchStatementFailure` enum. The outer method wraps this in `run_with_retry_sync`: a retryable failure rolls back and returns `Err` from the closure so `run_with_retry_sync` retries the whole transaction from a fresh `BEGIN IMMEDIATE`; a deterministic failure rolls back but returns `Ok(CasBatchAttemptOutcome::Deterministic(err))` so `run_with_retry_sync` never retries it, and the outer method re-raises it as `Err` once the retry loop returns — the minimum local classification needed, with no change to `resilience.rs`/`run_with_retry_sync` itself. `TursoDb::execute_transactional_insert_pair_if_absent` is untouched and coexists as-is. Added four tests to `db::mod`'s existing `#[cfg(test)] mod tests`: guard-affects-zero-rows (`Ok(false)`, no statements run, no state changed), guard-affects-one-row (`Ok(true)`, statement runs, state advances), a deterministic mid-batch failure (a duplicate `PRIMARY KEY` insert) proven to roll back the guard's own effect and the successful first insert together and to return `Err` after exactly one attempt (asserted via an elapsed-time bound well under the retry backoff budget, following this file's existing `worst_case_retry_failure_budget_ms`-style timing convention), and a genuine two-connection contention test — a second `TursoDb` handle runs `BEGIN IMMEDIATE`/holds the write lock on a background thread for 60ms before committing, while the primary handle's `execute_transactional_cas_batch` call is proven to retry through the real `Busy` contention and succeed once the lock clears. + - Verify: `./scripts/run-cli-cargo.sh test --manifest-path cli/Cargo.toml db::` — passed, 39/39 (35 pre-existing + 4 new), including 5 repeated runs of the contention test and 3 repeated runs of the deterministic-failure timing test to check for flakiness; `./scripts/run-cli-cargo.sh fmt --manifest-path cli/Cargo.toml -- --check` — passed after one auto-formatting pass; `./scripts/run-cli-cargo.sh clippy --manifest-path cli/Cargo.toml --all-targets -- -D warnings` — passed, zero warnings (required taking `error: &turso::Error` by reference in `CasBatchStatementFailure::from_turso` to satisfy `clippy::needless_pass_by_value` under this crate's `-D clippy::pedantic`). + - Done checks: a guard affecting 0 rows commits as a no-op, returns `Ok(false)`, and runs no statement or backoff wait (verified by the zero-rows test); a guard affecting 1 row runs every statement and returns `Ok(true)` (verified by the one-row test); an injected deterministic mid-batch failure rolls back the entire transaction, including the guard's own effect, and surfaces as `Err` after exactly one attempt, never as `Ok(false)` (verified by the duplicate-insert test — both the guard's revision advance and the first successful insert are absent afterward, and elapsed time rules out a retry); an injected retryable DB failure retries the whole transaction from `BEGIN IMMEDIATE` (never individual statements) up to the configured attempt count (verified by the two-connection contention test — the call succeeds only after the real lock clears, and the final state shows the guard and every statement applied exactly once, ruling out a resumed partial transaction). + - Context impact: local — a new generic primitive added to the already-public `cli/src/services/db/mod.rs` seam, alongside the existing `execute_transactional_insert_pair_if_absent`, with no production call site yet (T07 is the first caller). `context/sce/shared-turso-db.md` already exists but does not currently document `execute_transactional_insert_pair_if_absent` either (the analogous existing primitive), so this task's addition follows that same pre-existing documentation gap rather than introducing a new one; the plan's own "Context sync" section already lists `context/sce/shared-turso-db.md` as a plan-level deliverable covering this primitive, consistent with the T02-T05 pattern of deferring `store.rs`-adjacent documentation to T11/plan-level context sync once the full store lands. + - Context synchronization: synced + - **T06 correction (2026-08-28):** Fixed four gaps in the CAS batch primitive found in review of PR #241 before starting T07. (1) A guard affecting more than one row was previously treated the same as one row (statements ran, `Ok(true)`), silently masking a broken guard predicate; `execute_cas_batch_body` now matches on `guard_rows_affected` (`0 => Ok(false)`, `1 => run statements`, `n => Err(CasBatchFailure::Deterministic(...))` with a message naming the row count and the guard SQL), rolled back and never retried. (2) `BEGIN IMMEDIATE` failures previously bypassed classification entirely (`.map_err(|e| anyhow::anyhow!(...))?` handed every failure straight to `run_with_retry_sync`, so a deterministic BEGIN failure would have been retried like a transient one); `execute_transactional_cas_batch` now matches on the `Transaction::new_unchecked` result and routes a BEGIN failure through the same classifier used everywhere else. (3) `tx.commit()` failures had the same bypass; commit failures now route through the identical classifier, so a `Busy`/`BusySnapshot` commit failure retries the whole attempt from a fresh `BEGIN IMMEDIATE` (never just the commit) and every other commit failure fails deterministically after one attempt. (4) Renamed `CasBatchStatementFailure` to `CasBatchFailure` (it now classifies BEGIN/COMMIT failures too, not just statement failures) and introduced one shared `classify_turso_error(db_name, action, &turso::Error) -> CasBatchFailure` plus `cas_batch_failure_into_attempt_result` so BEGIN, guard, every statement, and COMMIT all route through the identical classification path instead of four separate implementations. Also extended `TransactionStatement` with an optional `expected_rows_affected: Option` (`TransactionStatement::new(sql, params)` defaults to `None`; a new builder `expect_rows_affected(self, expected: u64) -> Self` sets it), and `execute_cas_batch_body`'s statement loop now compares each statement's actual `rows_affected` against its expectation when present, returning `CasBatchFailure::Deterministic` (rolled back, not retried) on a mismatch, naming the SQL, expected count, and actual count — this is T07's forward contract for enforcing that scope/processed-event/mutation-event/active-scope writes actually touched the row they were supposed to touch, distinct from the guard's own special 0/1/`>1` CAS semantics, which are not modeled through this generic mechanism. Practical two-connection contention (already covered by the renamed `execute_transactional_cas_batch_retries_a_begin_immediate_busy_error_and_then_succeeds` test, strengthened with an elapsed-time lower bound proving the success could not have preceded the real lock release) is the only stable way found to force a genuine transient `Busy` failure in this test harness; a genuine deterministic BEGIN/COMMIT lifecycle failure (for example, a nested transaction on the same connection) was judged impractical to force through the public `TursoDb` API without depending on brittle internal state, so BEGIN/COMMIT's routing through `classify_turso_error` is proven by code inspection plus two new direct unit tests (`classify_turso_error_wraps_busy_as_retryable_with_the_supplied_action_context`, `classify_turso_error_wraps_constraint_violations_as_deterministic_with_the_supplied_action_context`) and two classifier-only tests for every `turso::Error` variant (`is_retryable_turso_error_classifies_busy_and_busy_snapshot_as_retryable`, `is_retryable_turso_error_classifies_every_other_variant_as_deterministic`), rather than by a fake production-only test seam. Added five more tests: guard-affects-two-rows (deterministic `Err`, no retry, no statement ran, both rows' revisions unchanged); statement-expectation-matches (`Ok(true)`); statement-expectation-affects-zero (deterministic `Err`, guard rolled back); statement-expectation-affects-more-than-expected (deterministic `Err`, guard and pre-existing rows unchanged); and no-expectation-affecting-zero-rows still succeeds (`Ok(true)`), proving the mechanism stays opt-in. `mutation_trace/store.rs`, migration `003`, `DurableTransition`, `protocol.rs`, `resilience.rs`, and `execute_transactional_insert_pair_if_absent` were not touched; T07 was not started. + - Verify (T06 correction): `./scripts/run-cli-cargo.sh test --manifest-path cli/Cargo.toml db::` — passed, 48/48 (39 pre-existing + 9 new), including 5 repeated runs each of the BEGIN-IMMEDIATE contention test and the guard-over-match/deterministic-failure timing tests to check for flakiness; `./scripts/run-cli-cargo.sh fmt --manifest-path cli/Cargo.toml -- --check` — passed after one auto-formatting pass; `./scripts/run-cli-cargo.sh clippy --manifest-path cli/Cargo.toml --all-targets -- -D warnings` — passed, zero warnings. + - Done checks (T06 correction): a guard affecting more than 1 row returns `Err`, rolls back, runs no statement, and is not retried (verified by the new two-row guard test, asserting both rows' revisions are unchanged, `cas_effect` stays empty, and elapsed time rules out a retry); BEGIN and COMMIT failures classify through the same seam as statement failures, so only `Busy`/`BusySnapshot` retries the whole attempt and every other variant fails deterministically after one attempt (verified by code inspection — `execute_transactional_cas_batch` routes both the `Transaction::new_unchecked` and `tx.commit()` error arms through `classify_turso_error`/`cas_batch_failure_into_attempt_result` — plus the direct classifier unit tests and the strengthened real-contention BEGIN test); `TransactionStatement::expect_rows_affected` enforces an exact affected-row count, with a mismatch rolling back and failing deterministically without retry, while an unset expectation preserves the original behavior (verified by the four new expectation tests). + - Context synchronization: synced - [ ] T07: `Implement MutationTraceStore::commit` (status:todo) - Task ID: T07 From f7736cd9926ec90deaedcfabc07d3dc25aadaead Mon Sep 17 00:00:00 2001 From: David Abram Date: Fri, 28 Aug 2026 00:56:23 +0200 Subject: [PATCH 09/15] mutation-trace: Implement durable transition commits Persist mutation-trace transitions through a CAS-guarded transaction so worktree, scope, processed-event, mutation-event, and active-scope writes succeed or roll back together. Return explicit Applied/Conflict outcomes and preserve deterministic failures, with round-trip coverage for the full transition. Update the mutation-trace context and plan to record the database-backed store implementation. Plan: `mutation-cursor-store-persistence`, task `T07`. Co-authored-by: SCE --- cli/src/services/mutation_trace/store.rs | 309 +++++++++++++++++- context/cli/mutation-trace-protocol.md | 30 +- context/context-map.md | 2 +- context/overview.md | 2 +- .../mutation-cursor-store-persistence.md | 15 +- 5 files changed, 333 insertions(+), 25 deletions(-) diff --git a/cli/src/services/mutation_trace/store.rs b/cli/src/services/mutation_trace/store.rs index 33fe13b3..4a7743e2 100644 --- a/cli/src/services/mutation_trace/store.rs +++ b/cli/src/services/mutation_trace/store.rs @@ -19,6 +19,7 @@ use std::collections::{BTreeMap, BTreeSet}; use anyhow::{bail, Context, Result}; use crate::services::agent_trace_db::repository::RepositoryAgentTraceDb; +use crate::services::db::TransactionStatement; use super::types::{ ActorKind, Attribution, Boundary, EventId, EventKey, FailureKind, MutationEvent, ProtocolState, @@ -219,6 +220,21 @@ const INSERT_SCOPE_IF_ABSENT_SQL: &str = "INSERT INTO mutation_trace_scopes (scope_id, worktree_id, actor_kind, status) VALUES (?1, ?2, ?3, 'never_seen') ON CONFLICT (scope_id) DO NOTHING"; +const UPDATE_WORKTREE_CAS_SQL: &str = "UPDATE mutation_trace_worktrees + SET cursor_tree = ?1, revision = ?2, tainted = ?3, failure_kind = ?4, needs_rebaseline = ?5, + updated_at = strftime('%Y-%m-%dT%H:%M:%fZ', 'now') + WHERE worktree_id = ?6 AND revision = ?7"; +const UPDATE_SCOPE_STATUS_SQL: &str = "UPDATE mutation_trace_scopes + SET status = ?1, updated_at = strftime('%Y-%m-%dT%H:%M:%fZ', 'now') + WHERE scope_id = ?2"; +const INSERT_PROCESSED_EVENT_SQL: &str = + "INSERT INTO mutation_trace_processed_events (scope_id, event_id) VALUES (?1, ?2)"; +const INSERT_MUTATION_EVENT_SQL: &str = "INSERT INTO mutation_trace_events + (worktree_id, revision, before_tree, after_tree, tainted, failure_kind, + attribution_kind, attribution_scope_id, boundary_kind, boundary_scope_id, boundary_event_id) + VALUES (?1, ?2, ?3, ?4, ?5, ?6, ?7, ?8, ?9, ?10, ?11)"; +const INSERT_MUTATION_EVENT_ACTIVE_SCOPE_SQL: &str = + "INSERT INTO mutation_trace_event_active_scopes (worktree_id, revision, scope_id) VALUES (?1, ?2, ?3)"; /// Bounded runtime projection of one worktree's durable protocol state, /// loaded by [`MutationTraceStore::load_worktree`]. Scoped to that worktree's @@ -261,12 +277,12 @@ impl WorktreeProjection { #[derive(Clone, Debug, Eq, PartialEq)] pub struct DurableTransition { - pub worktree: WorktreeId, - pub expected_revision: u64, - pub next_worktree_state: WorktreeState, - pub scope_status_changes: BTreeMap, - pub new_processed_event: Option, - pub new_mutation_event: Option, + worktree: WorktreeId, + expected_revision: u64, + next_worktree_state: WorktreeState, + scope_status_changes: BTreeMap, + new_processed_event: Option, + new_mutation_event: Option, } impl DurableTransition { @@ -699,6 +715,121 @@ impl<'a> MutationTraceStore<'a> { Ok(rows.into_iter().collect()) } + + pub fn commit(&self, transition: &DurableTransition) -> Result { + let expected_revision_blob = encode_revision(transition.expected_revision); + let next_revision_blob = encode_revision(transition.next_worktree_state.revision); + + let guard = TransactionStatement::new( + UPDATE_WORKTREE_CAS_SQL, + ( + transition.next_worktree_state.cursor_tree.0.as_str(), + next_revision_blob.as_slice(), + transition.next_worktree_state.tainted, + encode_failure_kind(transition.next_worktree_state.failure_kind), + transition.next_worktree_state.needs_rebaseline, + transition.worktree.0.as_str(), + expected_revision_blob.as_slice(), + ), + )?; + + let mut statements = Vec::new(); + + for (scope_id, status) in &transition.scope_status_changes { + statements.push( + TransactionStatement::new( + UPDATE_SCOPE_STATUS_SQL, + (encode_scope_status(*status), scope_id.0.as_str()), + )? + .expect_rows_affected(1), + ); + } + + if let Some(event_key) = &transition.new_processed_event { + statements.push( + TransactionStatement::new( + INSERT_PROCESSED_EVENT_SQL, + (event_key.scope_id.0.as_str(), event_key.event_id.0.as_str()), + )? + .expect_rows_affected(1), + ); + } + + if let Some(event) = &transition.new_mutation_event { + let event_revision_blob = encode_revision(event.revision); + let attribution_scope_id = attribution_scope_id(&event.attribution); + let (boundary_scope_id, boundary_event_id) = boundary_payload(&event.boundary); + + statements.push( + TransactionStatement::new( + INSERT_MUTATION_EVENT_SQL, + ( + event.worktree_id.0.as_str(), + event_revision_blob.as_slice(), + event.before_tree.0.as_str(), + event.after_tree.0.as_str(), + event.tainted, + encode_failure_kind(event.failure_kind), + encode_attribution_kind(attribution_kind(&event.attribution)), + attribution_scope_id, + encode_boundary_kind(boundary_kind(&event.boundary)), + boundary_scope_id, + boundary_event_id, + ), + )? + .expect_rows_affected(1), + ); + + for scope_id in &event.active_scopes { + statements.push( + TransactionStatement::new( + INSERT_MUTATION_EVENT_ACTIVE_SCOPE_SQL, + ( + event.worktree_id.0.as_str(), + event_revision_blob.as_slice(), + scope_id.0.as_str(), + ), + )? + .expect_rows_affected(1), + ); + } + } + + let applied = self.db.execute_transactional_cas_batch( + "commit mutation-trace durable transition", + "reload the worktree and retry the transition", + &guard, + &statements, + )?; + + Ok(if applied { + CasResult::Applied + } else { + CasResult::Conflict + }) + } +} + +#[derive(Clone, Copy, Debug, Eq, PartialEq)] +pub enum CasResult { + Applied, + Conflict, +} + +fn attribution_scope_id(attribution: &Attribution) -> Option<&str> { + match attribution { + Attribution::AiExclusive(scope_id) => Some(scope_id.0.as_str()), + Attribution::IneligibleUnscoped | Attribution::AiContended => None, + } +} + +fn boundary_payload(boundary: &Boundary) -> (Option<&str>, Option<&str>) { + match boundary { + Boundary::Start { scope, event } + | Boundary::Advance { scope, event } + | Boundary::Close { scope, event } => (Some(scope.0.as_str()), Some(event.0.as_str())), + Boundary::Flush { .. } => (None, None), + } } /// Derives the single effective referenced scope from `scope` and @@ -2182,4 +2313,170 @@ mod tests { .expect_err("an unexpectedly disappearing scope must be rejected"); assert!(error.to_string().contains("scope set")); } + + #[test] + fn commit_applies_a_full_transition_and_makes_every_write_visible() { + let db_path = unique_test_db_path("commit-applies-full-transition"); + let db = RepositoryAgentTraceDb::new_at(&db_path).expect("repository DB should open"); + let store = MutationTraceStore::new(&db); + let wt = WorktreeId("wt0".to_string()); + let scope_id = ScopeId("scope0".to_string()); + insert_worktree(&db, &wt.0, 0); + insert_scope(&db, &scope_id.0, &wt.0, ScopeStatus::Active); + + let event_key = EventKey { + scope_id: scope_id.clone(), + event_id: EventId("event-1".to_string()), + }; + let mutation_event = MutationEvent { + worktree_id: wt.clone(), + revision: 1, + before_tree: TreeId("tree0".to_string()), + after_tree: TreeId("tree1".to_string()), + active_scopes: BTreeSet::from([scope_id.clone()]), + tainted: false, + failure_kind: FailureKind::Healthy, + attribution: Attribution::AiExclusive(scope_id.clone()), + boundary: Boundary::Close { + scope: scope_id.clone(), + event: EventId("event-1".to_string()), + }, + }; + + let before = state_with_scope( + &wt, + &scope_id, + ActorKind::ClaudeCode, + ScopeStatus::Active, + 0, + ); + let mut after = before.clone(); + after.worktrees.insert( + wt.clone(), + WorktreeState { + cursor_tree: TreeId("tree1".to_string()), + revision: 1, + tainted: false, + failure_kind: FailureKind::Healthy, + needs_rebaseline: false, + }, + ); + after.scopes.insert( + scope_id.clone(), + ScopeState { + status: ScopeStatus::Closed, + actor_kind: ActorKind::ClaudeCode, + worktree_id: wt.clone(), + }, + ); + after.processed_events.insert(event_key.clone()); + after.mutation_events.insert(mutation_event.clone()); + + let transition = DurableTransition::between(&before, &after, &wt) + .expect("between should succeed") + .expect("a transition should exist for this change"); + + let result = store.commit(&transition).expect("commit should succeed"); + assert_eq!(result, CasResult::Applied); + + let worktree_state = store + .load_worktree_state(&wt) + .expect("worktree read should succeed") + .expect("worktree row should exist"); + assert_eq!(worktree_state, transition.next_worktree_state); + + let scope_state = store + .load_scope(&scope_id) + .expect("scope read should succeed") + .expect("scope row should exist"); + assert_eq!(scope_state.status, ScopeStatus::Closed); + + assert!(store + .processed_event_exists(&event_key) + .expect("processed-event read should succeed")); + + let reloaded_event = store + .load_mutation_event(&wt, 1) + .expect("mutation-event read should succeed") + .expect("mutation-event row should exist"); + assert_eq!(reloaded_event, mutation_event); + + remove_test_db(&db_path); + } + + #[test] + fn commit_returns_conflict_and_writes_nothing_when_the_worktree_revision_has_moved_on() { + let db_path = unique_test_db_path("commit-conflict"); + let db = RepositoryAgentTraceDb::new_at(&db_path).expect("repository DB should open"); + let store = MutationTraceStore::new(&db); + let wt = WorktreeId("wt0".to_string()); + insert_worktree(&db, &wt.0, 5); + + let mut before = ProtocolState::default(); + before + .worktrees + .insert(wt.clone(), healthy_worktree_state(0)); + let mut after = before.clone(); + after.worktrees.get_mut(&wt).unwrap().revision = 1; + + let transition = DurableTransition::between(&before, &after, &wt) + .expect("between should succeed") + .expect("a transition should exist for this change"); + + let result = store.commit(&transition).expect("commit should succeed"); + assert_eq!(result, CasResult::Conflict); + + let worktree_state = store + .load_worktree_state(&wt) + .expect("worktree read should succeed") + .expect("worktree row should exist"); + assert_eq!(worktree_state.revision, 5); + + remove_test_db(&db_path); + } + + #[test] + fn commit_propagates_a_deterministic_failure_without_reporting_conflict() { + let db_path = unique_test_db_path("commit-deterministic-failure"); + let db = RepositoryAgentTraceDb::new_at(&db_path).expect("repository DB should open"); + let store = MutationTraceStore::new(&db); + let wt = WorktreeId("wt0".to_string()); + let scope_id = ScopeId("scope0".to_string()); + insert_worktree(&db, &wt.0, 0); + insert_processed_event(&db, "scope0", "event-1"); + + let before = state_with_scope( + &wt, + &scope_id, + ActorKind::ClaudeCode, + ScopeStatus::Active, + 0, + ); + let mut after = before.clone(); + after.worktrees.get_mut(&wt).unwrap().revision = 1; + after.processed_events.insert(EventKey { + scope_id: scope_id.clone(), + event_id: EventId("event-1".to_string()), + }); + + let transition = DurableTransition::between(&before, &after, &wt) + .expect("between should succeed") + .expect("a transition should exist for this change"); + + let error = store + .commit(&transition) + .expect_err("a duplicate processed-event insert should fail deterministically"); + assert!(error.to_string().contains("execute failed")); + + let worktree_state = store + .load_worktree_state(&wt) + .expect("worktree read should succeed") + .expect("worktree row should exist"); + assert_eq!( + worktree_state.revision, 0, + "the guard's own revision advance must roll back together with the failed insert" + ); + + remove_test_db(&db_path); + } } diff --git a/context/cli/mutation-trace-protocol.md b/context/cli/mutation-trace-protocol.md index 7e29ed62..6c330947 100644 --- a/context/cli/mutation-trace-protocol.md +++ b/context/cli/mutation-trace-protocol.md @@ -1,9 +1,9 @@ # Mutation-cursor protocol module (`mutation_trace`) Pure Rust refinement of the verified `spec/mutation_cursor.qnt` protocol, living -at `cli/src/services/mutation_trace/`. It is not yet wired into any hook, -command, or database call site; that integration is out of scope for the -`mutation-cursor-protocol-kernel` plan and is left for a later plan. +at `cli/src/services/mutation_trace/`. `protocol.rs`'s transitions are not yet +wired into any hook/command; `store.rs` now provides a real database call site +(see "Target end-state architecture" below). ## Current state @@ -211,8 +211,9 @@ loads a `ProtocolState`; `protocol.rs` only transitions already-known ones. ## Target end-state architecture -The plan's file split anticipates three later seams this module does not yet -implement, recorded here so a later plan does not rediscover the layout: +The plan's file split anticipated three seams beyond `protocol.rs`. `store.rs` +now exists as a real database call site (`mutation-cursor-store-persistence`); +`coordinator.rs`/`git_snapshot.rs` remain future work: ```mermaid flowchart LR @@ -226,21 +227,20 @@ flowchart LR coordinator --> store ``` -Each seam's responsibility, once built: +Each seam's responsibility: -- **`coordinator.rs`** — receives hook/session identity, resolves the scope's - actor/worktree identity, asks `store.rs` to load or materialize the scope, +- **`coordinator.rs`** (future work) — receives hook/session identity, + resolves scope actor/worktree identity, asks `store.rs` to load the scope, obtains a `ProtocolState`, and calls the pure protocol. -- **`store.rs`** — loads durable scope records; atomically creates a new - scope record as `NeverSeen` when appropriate; never remaps `actor_kind`/ - `worktree_id` for an existing `ScopeId` (see "Runtime scope - materialization" above). +- **`store.rs`** (implemented) — loads and persists worktree/scope/event state + via a CAS-guarded commit; atomically creates a new scope as `NeverSeen`; + never remaps `actor_kind`/`worktree_id` for an existing `ScopeId` (see + "Runtime scope materialization" above). - **`protocol.rs`** — assumes referenced scopes are already represented in `ProtocolState.scopes`; validates and transitions lifecycle state only. -`protocol.rs` stays free of any Git object, DB row, or CAS transaction concept -even with its full action set implemented; `coordinator.rs`/`git_snapshot.rs`/ -`store.rs` are not created by this plan. +`protocol.rs` stays free of any Git object, DB row, or CAS transaction concept; +`coordinator.rs`/`git_snapshot.rs` are not created by this or the store plan. ## Authoritative source diff --git a/context/context-map.md b/context/context-map.md index 9c193e68..d01bf66f 100644 --- a/context/context-map.md +++ b/context/context-map.md @@ -23,7 +23,7 @@ Feature/domain context: - `context/cli/config-precedence-contract.md` (implemented `sce config` show/validate command contract, deterministic `flags > env > config file > defaults` resolution order, focused `config/resolver.rs` ownership for config discovery/merge/runtime precedence plus default-discovered invalid-file degradation, focused `config/render.rs` ownership for `show`/`validate` text+JSON output construction, canonical `$schema` acceptance for startup-loaded `sce/config.json` files, shared auth-key env/config/optional baked-default support starting with `workos_client_id`, shared runtime resolution for flat logging observability keys including config-file/default `log_to_file`, `log_dir` / `SCE_LOG_DIR` with `/sce/logs` defaulting plus config-file/default-only positive `log_file_retention_limit`, config-file-only `agent_trace.repository_id`/`agent_trace.repository_remote` repository-identity keys with default remote `origin`, default-enabled `agent_trace.auto_sync` boolean resolution with explicit-false opt-out for the post-commit trigger boundary, the catalog-derived `integrations.optional_workflows` optional-workflow selection key, JSON-pointer-prefixed schema-validation errors, canonical Pkl-generated `sce/config.json` schema ownership plus CLI embedding/reuse contract including `policies.attribution_hooks.enabled` default-true/explicit-false opt-out metadata, config-file selection order, `show` provenance output, and trimmed `validate` output contract) - `context/cli/capability-traits.md` (current broad CLI capability seam in `cli/src/services/capabilities.rs`, including `FsOps`/`StdFsOps`, `GitOps`/`ProcessGitOps`, git root/hooks resolution behavior, compile-time-typed borrowed AppContext wiring with associated-type narrow capability accessors plus `ContextWithRepoRoot` repo-root-scoped context derivation, generic command execution bounds, and test-only unimplemented stubs; current service internals do not consume fs/git traits until later lifecycle migration tasks) - `context/cli/service-lifecycle.md` (current compile-safe lifecycle seam in `cli/src/services/lifecycle.rs`, including default no-op `ServiceLifecycle` diagnose/fix/setup methods against narrow `HasRepoRoot`, lifecycle-owned health/fix/setup result types with generic setup messages, doctor/setup adapter boundaries, the static `LifecycleProvider` enum catalog/dispatcher, hook/config/local_db/auth_db/agent_trace_db lifecycle providers including setup-time repository-scoped Agent Trace DB initialization plus checkout identity diagnostics, implemented doctor aggregation over diagnose/fix providers, and implemented setup aggregation over `setup` providers in order config → local_db → auth_db → agent_trace_db → hooks when requested) -- `context/cli/mutation-trace-protocol.md` (pure, dependency-free `cli/src/services/mutation_trace/` refinement of the verified `spec/mutation_cursor.qnt` mutation-cursor protocol: the full pure action set — `prepare`/`commit` transition logic, attribution/mutation-event materialization, snapshot-failure/database-failure taint actions, scope abandonment, and recovery with an explicit observed-tree input — plus cross-action sequence/invariant tests and a module-level Quint refinement matrix are all complete (`types.rs`, `protocol.rs`, `mod.rs`, `tests.rs`, registered with `#[allow(dead_code)]`); opaque-newtype identity refinement decisions vs. the Quint model's bounded enums, and the `coordinator.rs`/`git_snapshot.rs`/`store.rs` target end-state seams this layout leaves room for but does not create; not yet wired into any hook, command, or database call site) +- `context/cli/mutation-trace-protocol.md` (pure, dependency-free `cli/src/services/mutation_trace/` refinement of the verified `spec/mutation_cursor.qnt` mutation-cursor protocol: the full pure action set — `prepare`/`commit` transition logic, attribution/mutation-event materialization, snapshot-failure/database-failure taint actions, scope abandonment, and recovery with an explicit observed-tree input — plus cross-action sequence/invariant tests and a module-level Quint refinement matrix are all complete (`types.rs`, `protocol.rs`, `mod.rs`, `tests.rs`, registered with `#[allow(dead_code)]`); opaque-newtype identity refinement decisions vs. the Quint model's bounded enums, and the `coordinator.rs`/`git_snapshot.rs` target end-state seams this layout leaves room for but does not create — `store.rs` now exists, built out by the `mutation-cursor-store-persistence` plan; `protocol.rs`'s pure transitions are not yet wired into any hook or command) - `context/cli/mutation-trace-revision-refinement.md` (the Quint `revision: int` → Rust `WorktreeState::revision: u64` bounded-integer refinement: the private `next_revision` checked-arithmetic helper `commit`/`taint`/`abandon`/`recover` all route through instead of a raw `+ 1`, so a worktree at `revision: u64::MAX` is a guarded no-op/rejection rather than a silent wrap to `0`) - `context/cli/mutation-trace-quint-connect.md` (`#[cfg(test)]`-only Quint Connect model-based-testing harness in `cli/src/services/mutation_trace/mbt/` continuously checking `protocol.rs` against `spec/mutation_cursor.qnt`: the verification-only `mbtAction`/`MbtAction` record-payload transport excluded from comparison, the operation-identity-vs-`MbtStutter` distinction on guarded/no-op branches with its two deterministic regressions, finite ID mapping, the AC5 comparable-state field list, `randomPrepare` staying a single `step` branch, deterministic/generated (500×30, seed-reproducible) test coverage, and the two Nix checks — generic `checks.cli-tests` and dedicated `checks.mutation-trace-quint-connect` — that both require the pinned Quint binary plus the top-level `spec/` directory in `workspaceSrc`'s Nix fileset) - `context/sce/cli-exit-code-contract.md` (stable class-based `sce` process exit-code contract in `cli/src/app.rs`, so automation can branch on failure category without parsing error text) diff --git a/context/overview.md b/context/overview.md index 7da533c6..89fd22fb 100644 --- a/context/overview.md +++ b/context/overview.md @@ -2,7 +2,7 @@ This repository maintains shared assistant configuration for OpenCode, Claude, Pi, and Codex from a single canonical Pkl authoring source. One typed workflow catalog owns the six workflows' shared identity and target routing metadata, while canonical workflow/phase modules own behavior and migrated package-local documents, and target renderers own formatting. Generated target layouts are ephemeral: repository builds consume a pre-Cargo generated payload through `SCE_CLI_GENERATED_INPUT_DIR`, crates.io and Flatpak stage packaging-only fallbacks, and `config/.opencode`, `config/.claude`, `config/.pi`, `config/.agents`, and the generated SCE config schema are not committed. The catalog also marks a workflow `optional` — currently only `brownfield` — which changes nothing about generation and is projected into a generated `config/optional-workflows.json` manifest for install-time consumers. `nix run .#pkl-check-generated` preserves its exact 141-path artifact, metadata/package, phase-reference, internal-reference, optional-workflow-manifest, workflow-orchestration, OpenCode-permission, required-path, and forbidden-path checks while delegating deterministic payload production and inventories to the shared generated-input producer; `nix flake check` runs the same contract. The target matrix contains one manual OpenCode profile plus Claude, Pi, and Codex; the former automated OpenCode profile has been removed. A fourth Pkl renderer, `config/pkl/renderers/codex-content.pkl`, also consumes the same canonical workflow composition to emit skills-only Codex output under `config/.agents/skills/**` (no per-target frontmatter, matching Pi, and no command/prompt layer). Each of the six catalog workflow skills (not `sce-decision`, which has no user-facing entrypoint on any target) additionally carries a Codex-only `agents/openai.yaml`, rendered by `config/pkl/renderers/codex-metadata.pkl` from the shared catalog's `title`/`description` plus an authored `default_prompt`, with `policy.allow_implicit_invocation: false` so these stateful lifecycle workflows activate only from explicit `$sce-` invocation or Codex's `/skills` discovery, never from conversational relevance alone. The Codex renderer also emits a Codex hook registration file (`config/.codex/hooks.json`) and its fail-open install-guidance hook script (`config/.codex/hooks/run-sce-or-show-install-guidance.sh`), both routing every registered lifecycle event through the single command `sce hooks codex`; the generated command resolves the Git root at invocation time and safely invokes the helper from nested cwd or spaced repository paths, failing open silently when Git-root resolution fails. `sce setup --codex` (and `--all`) now installs it as a fourth `SetupTarget`; setup merges the user-owned `.codex/hooks.json` through the shared structural Codex hook-config service, preserving unrelated valid handlers and rejecting malformed or Codex-invalid documents before the atomic swap. `sce doctor` diagnoses that file per required registration (`PresentAndCurrent`/`Missing`/`Stale`, or `Malformed` for the whole document) rather than by whole-document equality, and separately reports whether Codex has actually marked each structurally current registration trusted by reading (never writing) Codex's own `$CODEX_HOME/config.toml` hook-trust state; `sce doctor --fix` repairs structurally unhealthy registrations through the same merge service but never touches trust state (see `context/sce/doctor-human-text-contract.md`). `sce hooks codex` now exists as a typed dispatcher (`cli/src/services/hooks/codex/`): it parses the raw hook JSON into a `CodexHookEvent` and classifies `(hook_event_name, tool_name)` into `UserPromptSubmit`, `Stop`, `PreToolUse(Bash)`, `PostToolUse(apply_patch)`, or a fail-open `NoOp` fallthrough covering every other combination — including `apply_patch` under `PreToolUse`. `UserPromptSubmit` and `Stop` each capture one `messages`/`parts` row (`role="user"`/`role="assistant"`) into the repository Agent Trace DB under the idempotent `cx_` session prefix, `PreToolUse(Bash)` delegates to the existing Bash policy engine, and `PostToolUse(apply_patch)` parses Codex's own `apply_patch` text, resolves paths from the event `cwd` against the real Git root into safe repository-relative paths, normalizes Add/Update evidence into an SCE unified diff under deterministic event-scoped synthetic line identities derived from `tool_use_id`, and persists it as one `diff_traces` row when non-empty (see `context/sce/codex-integration-runtime.md`) — while invalid cwd/path resolution and malformed STDIN payloads fail open with empty stdout, reported model IDs are preserved without fabricated provider prefixes, and missing/blank apply_patch sessions produce no evidence. Delete-File operations and Bash-triggered filesystem mutations remain untracked for Codex. -It also includes a Rust CLI (`sce`) for Shared Context Engineering workflows: auth, config inspection, setup, doctor, Agent Trace hooks and synchronization, bash-policy evaluation, and repository-scoped Agent Trace storage infrastructure. See `context/architecture.md` for module-level boundaries and `context/context-map.md` for the full domain file index. A new pure, dependency-free `cli/src/services/mutation_trace/` module now exists as a Rust refinement of the verified `spec/mutation_cursor.qnt` mutation-cursor protocol; it currently carries domain types and the full pure action set — `prepare`/`commit` transition logic, attribution/mutation-event materialization, snapshot-failure/database-failure taint actions, scope abandonment, and recovery with an explicit observed-tree input — is registered with `#[allow(dead_code)]`, and is not yet wired into any hook, command, or database call site (see `context/cli/mutation-trace-protocol.md`). +It also includes a Rust CLI (`sce`) for Shared Context Engineering workflows: auth, config inspection, setup, doctor, Agent Trace hooks and synchronization, bash-policy evaluation, and repository-scoped Agent Trace storage infrastructure. See `context/architecture.md` for module-level boundaries and `context/context-map.md` for the full domain file index. A new pure, dependency-free `cli/src/services/mutation_trace/` module now exists as a Rust refinement of the verified `spec/mutation_cursor.qnt` mutation-cursor protocol; it currently carries domain types and the full pure action set — `prepare`/`commit` transition logic, attribution/mutation-event materialization, snapshot-failure/database-failure taint actions, scope abandonment, and recovery with an explicit observed-tree input — is registered with `#[allow(dead_code)]`. A separate `cli/src/services/mutation_trace/store.rs` persistence layer (built out by the `mutation-cursor-store-persistence` plan) now provides a real, CAS-guarded database call site against the repository-scoped Agent Trace DB, but the module is still not wired into any hook or command (see `context/cli/mutation-trace-protocol.md`). The generated `/next-task` workflow persists task-level context-synchronization lifecycle state in each plan (`pending`, `synced`, or `blocked`) so unresolved task synchronization debt survives a session boundary and gates new implementation. Successful `/next-task` execution hands task synchronization an explicit, pre-edit-Git-baseline-relative changed-file list plus implementation, verification, done-check, plan-update, and context-impact evidence, recorded directly on the completed task (`Completed`, `Files changed`, `Result`, `Verify`, `Context impact`, `Context synchronization`); the five-file root context pass remains mandatory. A later-session sync-debt retry reads that same completed task record directly from the plan by plan path and task ID, with no separate persisted synchronization handoff. `/validate` is validation-only: it runs final checks, writes the Validation Report, and reports `validated`, `failed`, or `blocked` without plan-level context synchronization. diff --git a/context/plans/mutation-cursor-store-persistence.md b/context/plans/mutation-cursor-store-persistence.md index 936657e3..b000ef90 100644 --- a/context/plans/mutation-cursor-store-persistence.md +++ b/context/plans/mutation-cursor-store-persistence.md @@ -235,13 +235,24 @@ Persist this field in every plan; this is durable plan state, not chat state: - Done checks (T06 correction): a guard affecting more than 1 row returns `Err`, rolls back, runs no statement, and is not retried (verified by the new two-row guard test, asserting both rows' revisions are unchanged, `cas_effect` stays empty, and elapsed time rules out a retry); BEGIN and COMMIT failures classify through the same seam as statement failures, so only `Busy`/`BusySnapshot` retries the whole attempt and every other variant fails deterministically after one attempt (verified by code inspection — `execute_transactional_cas_batch` routes both the `Transaction::new_unchecked` and `tx.commit()` error arms through `classify_turso_error`/`cas_batch_failure_into_attempt_result` — plus the direct classifier unit tests and the strengthened real-contention BEGIN test); `TransactionStatement::expect_rows_affected` enforces an exact affected-row count, with a mismatch rolling back and failing deterministically without retry, while an unset expectation preserves the original behavior (verified by the four new expectation tests). - Context synchronization: synced -- [ ] T07: `Implement MutationTraceStore::commit` (status:todo) +- [x] T07: `Implement MutationTraceStore::commit` (status:done) - Task ID: T07 - Scope: In — `CasResult` and `MutationTraceStore::commit(transition)`, translating a `DurableTransition` into the worktree CAS `UPDATE` plus scope `UPDATE`s plus processed-event `INSERT` plus mutation-event `INSERT` plus active-scope `INSERT`s, via `execute_transactional_cas_batch`. Out — concurrency/rollback/round-trip test coverage (T08/T09). - Dependencies: T04, T05, T06 - Done when: `commit()` returns `CasResult::Applied` with every included write visible when the worktree's on-disk revision matches `expected_revision`, and `CasResult::Conflict` with no visible write otherwise; a deterministic failure surfaced by `execute_transactional_cas_batch` propagates out of `commit()` as an `Err`, never as `CasResult::Conflict`. - Verify: `./scripts/run-cli-cargo.sh test --manifest-path cli/Cargo.toml mutation_trace::store::` - - Context synchronization: pending + - Completed: 2026-08-28 + - Files changed: `cli/src/services/mutation_trace/store.rs` + - Result: Added `CasResult` (`Applied`/`Conflict`) and `MutationTraceStore::commit(transition: &DurableTransition) -> Result`. The worktree `UPDATE mutation_trace_worktrees SET cursor_tree=?, revision=?, tainted=?, failure_kind=?, needs_rebaseline=?, updated_at=... WHERE worktree_id=? AND revision=?` (new `UPDATE_WORKTREE_CAS_SQL`) is the CAS guard passed to T06's `execute_transactional_cas_batch`; its affected-row count (0 or 1) is what `execute_transactional_cas_batch` treats as the CAS outcome. `statements` is built from `transition`: one `UPDATE mutation_trace_scopes SET status=? ... WHERE scope_id=?` (new `UPDATE_SCOPE_STATUS_SQL`) per `scope_status_changes` entry (more than one can be present in a single transition — confirmed by `protocol::recover`'s strong-recovery path, which abandons every live scope on a worktree in one transition), one `INSERT INTO mutation_trace_processed_events` (new `INSERT_PROCESSED_EVENT_SQL`) when `new_processed_event` is `Some`, and when `new_mutation_event` is `Some`, one `INSERT INTO mutation_trace_events` (new `INSERT_MUTATION_EVENT_SQL`, encoding `Attribution`/`Boundary` via T02's `attribution_kind`/`boundary_kind` codecs plus two new private payload-extraction helpers `attribution_scope_id`/`boundary_payload`) followed by one `INSERT INTO mutation_trace_event_active_scopes` (new `INSERT_MUTATION_EVENT_ACTIVE_SCOPE_SQL`) per `active_scopes` entry. Every non-guard statement carries `.expect_rows_affected(1)`, so an unexpected row count (including a `(scope_id, event_id)` replay-uniqueness violation on the processed-event insert) fails deterministically per T06's contract rather than silently. `commit()` maps `execute_transactional_cas_batch`'s `Ok(true)`/`Ok(false)`/`Err` to `Ok(CasResult::Applied)`/`Ok(CasResult::Conflict)`/`Err` unchanged. Added three tests: a full-transition test exercising every write kind at once (worktree CAS advance, one scope status change, one processed event, one mutation event with active scopes) and reading every write back through `load_worktree_state`/`load_scope`/`processed_event_exists`/`load_mutation_event`; a conflict test (stale `expected_revision` against a worktree already at a later revision) asserting `CasResult::Conflict` and an unchanged on-disk revision; and a deterministic-failure test (a duplicate `(scope_id, event_id)` processed-event insert) asserting `Err` (not `CasResult::Conflict`) and that the guard's own revision advance rolled back together with the failed insert. + - Verify: `./scripts/run-cli-cargo.sh test --manifest-path cli/Cargo.toml mutation_trace::store::` — passed, 56/56 (53 pre-existing + 3 new); `./scripts/run-cli-cargo.sh fmt --manifest-path cli/Cargo.toml -- --check` — passed after one auto-formatting pass; `./scripts/run-cli-cargo.sh clippy --manifest-path cli/Cargo.toml --all-targets -- -D warnings` — passed, zero warnings. + - Done checks: `commit()` returns `CasResult::Applied` with every included write visible when the worktree's on-disk revision matches `expected_revision` (verified by the full-transition test — worktree, scope status, processed event, and mutation event with active scopes are all read back matching the transition); `CasResult::Conflict` with no visible write otherwise (verified by the conflict test — on-disk revision stays 5 when `expected_revision` was 0); a deterministic failure propagates out of `commit()` as `Err`, never as `CasResult::Conflict` (verified by the deterministic-failure test — the call returns `Err` containing "execute failed", and the guard's own revision advance is absent afterward, proving the whole transaction rolled back together). + - Context impact: domain — `commit` makes `store.rs` a real database call site for the first time (T02-T06 were read-only/pure); `context/cli/mutation-trace-protocol.md`'s and `context/overview.md`'s "not yet wired into any hook, command, or database call site" framing was stale on the "database call site" clause and has been corrected (hook/command wiring is still absent). `context/context-map.md`'s entry for the protocol doc was updated to match. + - Context synchronization: synced + - **T07 correction (2026-08-28):** Fixed an architectural-integrity gap where `DurableTransition`'s fields were `pub`, so any code with visibility into `store.rs` could construct an arbitrary, unvalidated transition (for example `expected_revision: 7` paired with `next_worktree_state.revision: 999`, or a `new_mutation_event` for an unrelated worktree) and pass it directly to `MutationTraceStore::commit`, bypassing every structural invariant `DurableTransition::between` enforces. Made all six `DurableTransition` fields private (struct itself stays `pub`); added no public constructor, setter, builder, or `Default` impl, so `DurableTransition::between` is now the only way to obtain one outside `store.rs`. `MutationTraceStore::commit` is unchanged — it already only reads the fields (`impl DurableTransition`'s privacy boundary is the module, and `mod tests` is a submodule of `store`, so `commit` and the in-module tests keep field access without any accessor). Rewrote the three T07 `commit` tests (`commit_applies_a_full_transition_and_makes_every_write_visible`, `commit_returns_conflict_and_writes_nothing_when_the_worktree_revision_has_moved_on`, `commit_propagates_a_deterministic_failure_without_reporting_conflict`) to build a `before`/`after` `ProtocolState` pair and obtain the transition via `DurableTransition::between(&before, &after, &wt)`, exercising the same validated path production code will use, instead of constructing `DurableTransition { .. }` struct literals directly. Repo-wide search (`grep -rn "DurableTransition {"`) confirms the struct definition and `impl` block are the only remaining matches — no direct construction anywhere, including tests. Added no read-only accessor methods: nothing outside `store.rs` references `DurableTransition` yet (confirmed by `grep -rn "DurableTransition"` outside `store.rs`), so no accessor is needed until a real external caller (future `coordinator.rs`) requires one. `MutationTraceStore::commit` does not duplicate any T05 `between`-owned validation (revision-advance arithmetic, mutation-event-revision agreement, processed-event/scope-mutation worktree membership) — it only translates the already-validated fields into SQL. No doc-comment changes were made to `DurableTransition` or `mod.rs` in this correction, per this session's no-source-comments policy. Migration `003`, the T06 CAS primitive/retry classification, `protocol.rs` semantics, the Quint model, and T05's validation rules themselves are all unchanged. T08 was not started. + - Verify (T07 correction): `./scripts/run-cli-cargo.sh test --manifest-path cli/Cargo.toml mutation_trace::` — passed, 144/144 (all `store`/`protocol`/`mbt` tests, including the 3 rewritten `commit` tests); `./scripts/run-cli-cargo.sh fmt --manifest-path cli/Cargo.toml -- --check` — passed after one auto-formatting pass; `./scripts/run-cli-cargo.sh clippy --manifest-path cli/Cargo.toml --all-targets -- -D warnings` — passed, zero warnings. + - Done checks (T07 correction): `DurableTransition`'s six fields are private, the struct itself stays `pub`, and no public unchecked constructor/setter/builder/`Default` was added (verified by inspection of the diff); the malformed combinations from the original T07 review (mismatched `expected_revision`/`next_worktree_state.revision`, an unrelated-worktree `MutationEvent`, an unrelated-worktree scope-status change, an unrelated-scope processed event) remain rejected exclusively by `DurableTransition::between`, unchanged by this correction (verified — `between`'s four diff helpers and their existing malformed-pair tests were not touched); `commit()` still performs no independent structural validation (verified by inspection — it only reads `transition`'s fields and encodes them into SQL statements/params); all three `commit` tests now obtain their transition via `DurableTransition::between` (verified); repo-wide search shows no direct `DurableTransition { .. }` construction outside the struct/impl definitions (verified); T08 was not started (verified — no changes to concurrency tests, round-trip tests, or any file outside `store.rs`/this plan). + - Context impact: local — the privacy tightening and test rewrite are confined to `store.rs`'s existing `DurableTransition`/test surface; no durable context file describes `DurableTransition`'s field visibility, so nothing durable was contradicted. Deferred to T11/plan-level context sync for full `store.rs` documentation, per the plan's existing assumption. + - Context synchronization: synced - [ ] T08: `Add CAS and concurrency test coverage for store.commit` (status:todo) - Task ID: T08 From 225ee8d0d3fe9c0acb43e818f8c9936e9b9023b5 Mon Sep 17 00:00:00 2001 From: David Abram Date: Sat, 29 Aug 2026 11:19:28 +0200 Subject: [PATCH 10/15] mutation-trace: Remove stale store implementation documentation Correct module documentation that described the store's persistence and database call sites as future work. Record the T07 correction and verification evidence in the mutation-cursor-store-persistence plan. Co-authored-by: SCE --- cli/src/services/mutation_trace/mod.rs | 17 +++-------------- cli/src/services/mutation_trace/store.rs | 6 ------ .../plans/mutation-cursor-store-persistence.md | 3 +++ 3 files changed, 6 insertions(+), 20 deletions(-) diff --git a/cli/src/services/mutation_trace/mod.rs b/cli/src/services/mutation_trace/mod.rs index 2416cbdc..9dd03bdf 100644 --- a/cli/src/services/mutation_trace/mod.rs +++ b/cli/src/services/mutation_trace/mod.rs @@ -10,17 +10,6 @@ //! //! No Git, database, filesystem, environment, network, async, or lock I/O is //! performed here. -//! The module is not yet wired into any hook, command, or database call -//! site: that integration, along with the `coordinator.rs` (imperative -//! shell), `git_snapshot.rs` (isolated Git snapshot capture), and `store.rs` -//! (DB-backed CAS persistence) seams the target architecture will grow into, -//! is left for later work. This layout leaves those three seams as the -//! natural home for: `coordinator.rs` loading/persisting state and supplying -//! the observed-tree inputs `prepare`/`recover` take as explicit parameters; -//! `git_snapshot.rs` capturing and diffing worktree trees; and `store.rs` -//! implementing the CAS-transactional persistence and runtime scope -//! materialization contract described in -//! `context/cli/mutation-trace-protocol.md`. //! //! # Quint refinement matrix //! @@ -72,9 +61,9 @@ //! population of every `ScopeState`/`WorktreeState` are **external adapter //! responsibility**: this refinement's `ScopeId`/`WorktreeId` spaces are //! unbounded runtime strings, so scope/worktree identity is materialized at -//! runtime by the future `coordinator.rs`/`store.rs` layer rather than at -//! protocol startup (see the "Runtime scope materialization" assumption -//! recorded in the plan and in `context/cli/mutation-trace-protocol.md`). +//! runtime by the future `coordinator.rs` layer rather than at protocol +//! startup (see the "Runtime scope materialization" assumption recorded in +//! the plan and in `context/cli/mutation-trace-protocol.md`). //! //! ## Semantic properties //! diff --git a/cli/src/services/mutation_trace/store.rs b/cli/src/services/mutation_trace/store.rs index 4a7743e2..e5c2e544 100644 --- a/cli/src/services/mutation_trace/store.rs +++ b/cli/src/services/mutation_trace/store.rs @@ -7,12 +7,6 @@ //! Every codec here is an explicit function over a fixed set of variants — no //! codec derives from `Debug` or a serde representation, so a variant rename //! cannot silently change the durable encoding. -//! -//! `MutationTraceStore` adds the hot-path bounded worktree read -//! (`load_worktree`) and the cold-path historical read (`load_mutation_event`) -//! against a `&RepositoryAgentTraceDb`. Initialization and CAS-commit logic -//! are later tasks (`mutation-cursor-store-persistence` T04/T06/T07); this -//! module carries no such logic yet. use std::collections::{BTreeMap, BTreeSet}; diff --git a/context/plans/mutation-cursor-store-persistence.md b/context/plans/mutation-cursor-store-persistence.md index b000ef90..78862c61 100644 --- a/context/plans/mutation-cursor-store-persistence.md +++ b/context/plans/mutation-cursor-store-persistence.md @@ -253,6 +253,9 @@ Persist this field in every plan; this is durable plan state, not chat state: - Done checks (T07 correction): `DurableTransition`'s six fields are private, the struct itself stays `pub`, and no public unchecked constructor/setter/builder/`Default` was added (verified by inspection of the diff); the malformed combinations from the original T07 review (mismatched `expected_revision`/`next_worktree_state.revision`, an unrelated-worktree `MutationEvent`, an unrelated-worktree scope-status change, an unrelated-scope processed event) remain rejected exclusively by `DurableTransition::between`, unchanged by this correction (verified — `between`'s four diff helpers and their existing malformed-pair tests were not touched); `commit()` still performs no independent structural validation (verified by inspection — it only reads `transition`'s fields and encodes them into SQL statements/params); all three `commit` tests now obtain their transition via `DurableTransition::between` (verified); repo-wide search shows no direct `DurableTransition { .. }` construction outside the struct/impl definitions (verified); T08 was not started (verified — no changes to concurrency tests, round-trip tests, or any file outside `store.rs`/this plan). - Context impact: local — the privacy tightening and test rewrite are confined to `store.rs`'s existing `DurableTransition`/test surface; no durable context file describes `DurableTransition`'s field visibility, so nothing durable was contradicted. Deferred to T11/plan-level context sync for full `store.rs` documentation, per the plan's existing assumption. - Context synchronization: synced + - **T07 correction (2026-08-29):** `DurableTransition` remains a validated capability — all six fields private, `between()` the only external construction path, `commit()` intentionally trusting that capability without duplicating T05's structural validation (unchanged by this pass; verified again by inspection). Corrected stale documentation in `cli/src/services/mutation_trace/mod.rs` and `store.rs` that still described `store.rs`'s persistence/CAS-commit logic as unbuilt/future work and the module as unwired to any database call site — both false since T04/T06/T07 landed. Fixed by deleting the stale prose rather than writing new explanatory comments, matching this repository's no-source-comments policy; `context/cli/mutation-trace-protocol.md` and `context/overview.md` were already accurate and needed no change. No getters were added to `DurableTransition`. T08 was not started. + - Verify (T07 correction, 2026-08-29): `./scripts/run-cli-cargo.sh test --manifest-path cli/Cargo.toml mutation_trace::` — passed; `./scripts/run-cli-cargo.sh fmt --manifest-path cli/Cargo.toml -- --check` — passed; `./scripts/run-cli-cargo.sh clippy --manifest-path cli/Cargo.toml --all-targets -- -D warnings` — passed. + - Context synchronization: synced - [ ] T08: `Add CAS and concurrency test coverage for store.commit` (status:todo) - Task ID: T08 From c14eed32eafbe7a5db81a1fade1af7f345d92a6d Mon Sep 17 00:00:00 2001 From: David Abram Date: Sat, 29 Aug 2026 11:34:05 +0200 Subject: [PATCH 11/15] tests: Add mutation trace commit persistence coverage Cover CAS races, atomic rollback, boundary-value revisions, replay rejection, and recovery scope behavior against real repository databases. Mark plan mutation-cursor-store-persistence task T08 complete with verification evidence for the new store tests. Co-authored-by: SCE --- cli/src/services/mutation_trace/store.rs | 551 ++++++++++++++++++ .../mutation-cursor-store-persistence.md | 15 +- 2 files changed, 564 insertions(+), 2 deletions(-) diff --git a/cli/src/services/mutation_trace/store.rs b/cli/src/services/mutation_trace/store.rs index e5c2e544..09dd7f9e 100644 --- a/cli/src/services/mutation_trace/store.rs +++ b/cli/src/services/mutation_trace/store.rs @@ -1002,6 +1002,8 @@ fn reconstruct_boundary( #[cfg(test)] mod tests { + use std::thread; + use super::*; use crate::services::mutation_trace::protocol::{ abandon, commit, database_failure, prepare, recover, taint, @@ -1173,6 +1175,20 @@ mod tests { .expect("processed-event insert should succeed"); } + fn insert_active_scope( + db: &RepositoryAgentTraceDb, + worktree_id: &str, + revision: u64, + scope_id: &str, + ) { + db.execute( + "INSERT INTO mutation_trace_event_active_scopes (worktree_id, revision, scope_id) + VALUES (?1, ?2, ?3)", + (worktree_id, encode_revision(revision).as_slice(), scope_id), + ) + .expect("active-scope insert should succeed"); + } + #[allow(clippy::too_many_arguments)] fn insert_mutation_event( db: &RepositoryAgentTraceDb, @@ -2473,4 +2489,539 @@ mod tests { remove_test_db(&db_path); } + + struct RaceEvidence { + scope: ScopeId, + event_key: EventKey, + mutation_event: MutationEvent, + } + + fn race_evidence(worktree: &WorktreeId, scope: ScopeId, label: &str) -> RaceEvidence { + let event_key = EventKey { + scope_id: scope.clone(), + event_id: EventId(format!("event-{label}")), + }; + let mutation_event = MutationEvent { + worktree_id: worktree.clone(), + revision: 1, + before_tree: TreeId("tree0".to_string()), + after_tree: TreeId(format!("tree-after-{label}")), + active_scopes: BTreeSet::from([scope.clone()]), + tainted: false, + failure_kind: FailureKind::Healthy, + attribution: Attribution::AiExclusive(scope.clone()), + boundary: Boundary::Close { + scope: scope.clone(), + event: event_key.event_id.clone(), + }, + }; + RaceEvidence { + scope, + event_key, + mutation_event, + } + } + + fn closing_transition( + before: &ProtocolState, + worktree: &WorktreeId, + evidence: &RaceEvidence, + ) -> DurableTransition { + let mut after = before.clone(); + after.worktrees.get_mut(worktree).unwrap().revision = evidence.mutation_event.revision; + after.scopes.get_mut(&evidence.scope).unwrap().status = ScopeStatus::Closed; + after.processed_events.insert(evidence.event_key.clone()); + after + .mutation_events + .insert(evidence.mutation_event.clone()); + DurableTransition::between(before, &after, worktree) + .expect("between should succeed") + .expect("a transition should exist for this change") + } + + fn assert_race_winner_state( + store: &MutationTraceStore, + worktree: &WorktreeId, + persisted_event: &MutationEvent, + writer_a: &RaceEvidence, + writer_b: &RaceEvidence, + ) { + let (winner, loser) = if persisted_event == &writer_a.mutation_event { + (writer_a, writer_b) + } else if persisted_event == &writer_b.mutation_event { + (writer_b, writer_a) + } else { + panic!( + "persisted mutation event matches neither writer's expected event: \ + {persisted_event:?}" + ); + }; + + let winning_scope_state = store + .load_scope(&winner.scope) + .expect("winning scope read should succeed") + .expect("winning scope row should exist"); + assert_eq!( + winning_scope_state.status, + ScopeStatus::Closed, + "the winning transition's scope-status change must be durable" + ); + + let losing_scope_state = store + .load_scope(&loser.scope) + .expect("losing scope read should succeed") + .expect("losing scope row should exist"); + assert_eq!( + losing_scope_state.status, + ScopeStatus::Active, + "the losing transition's scope-status change must not have applied" + ); + + assert!( + store + .processed_event_exists(&winner.event_key) + .expect("winning processed-event read should succeed"), + "the winning transition's processed EventKey must exist" + ); + assert!( + !store + .processed_event_exists(&loser.event_key) + .expect("losing processed-event read should succeed"), + "the losing transition's processed EventKey must not exist" + ); + + assert_eq!( + persisted_event, &winner.mutation_event, + "the persisted mutation event must equal the winning transition's expected event \ + field-for-field, with no evidence from the losing transition mixed in" + ); + + let persisted_active_scopes = store + .load_mutation_event_active_scopes( + worktree, + encode_revision(winner.mutation_event.revision).as_slice(), + ) + .expect("active-scope read should succeed"); + assert_eq!( + &persisted_active_scopes, &winner.mutation_event.active_scopes, + "persisted active scopes must equal exactly the winning transition's active_scopes, \ + with no loser-only active-scope rows present" + ); + } + + #[test] + fn commit_from_two_independent_connections_races_and_only_one_applies() { + let db_path = unique_test_db_path("commit-two-writer-race"); + let wt = WorktreeId("wt0".to_string()); + let scope_a = ScopeId("scope-a".to_string()); + let scope_b = ScopeId("scope-b".to_string()); + + { + let db = RepositoryAgentTraceDb::new_at(&db_path).expect("repository DB should open"); + insert_worktree(&db, &wt.0, 0); + insert_scope(&db, &scope_a.0, &wt.0, ScopeStatus::Active); + insert_scope(&db, &scope_b.0, &wt.0, ScopeStatus::Active); + } + + let mut before = ProtocolState::default(); + before + .worktrees + .insert(wt.clone(), healthy_worktree_state(0)); + before.scopes.insert( + scope_a.clone(), + ScopeState { + status: ScopeStatus::Active, + actor_kind: ActorKind::ClaudeCode, + worktree_id: wt.clone(), + }, + ); + before.scopes.insert( + scope_b.clone(), + ScopeState { + status: ScopeStatus::Active, + actor_kind: ActorKind::ClaudeCode, + worktree_id: wt.clone(), + }, + ); + + let writer_a = race_evidence(&wt, scope_a, "a"); + let writer_b = race_evidence(&wt, scope_b, "b"); + let transition_a = closing_transition(&before, &wt, &writer_a); + let transition_b = closing_transition(&before, &wt, &writer_b); + + let db_a = RepositoryAgentTraceDb::open_for_hooks_without_migrations_at(&db_path) + .expect("writer A handle should open"); + let db_b = RepositoryAgentTraceDb::open_for_hooks_without_migrations_at(&db_path) + .expect("writer B handle should open"); + + let handle_a = thread::spawn(move || MutationTraceStore::new(&db_a).commit(&transition_a)); + let handle_b = thread::spawn(move || MutationTraceStore::new(&db_b).commit(&transition_b)); + + let result_a = handle_a + .join() + .expect("writer A thread should not panic") + .expect("writer A commit should not error"); + let result_b = handle_b + .join() + .expect("writer B thread should not panic") + .expect("writer B commit should not error"); + let results = [result_a, result_b]; + + assert_eq!( + results.iter().filter(|r| **r == CasResult::Applied).count(), + 1, + "exactly one writer should apply from the same starting revision: {results:?}" + ); + assert_eq!( + results + .iter() + .filter(|r| **r == CasResult::Conflict) + .count(), + 1, + "exactly one writer should conflict from the same starting revision: {results:?}" + ); + + let db_reopened = RepositoryAgentTraceDb::open_for_hooks_without_migrations_at(&db_path) + .expect("reopened handle should open"); + let store = MutationTraceStore::new(&db_reopened); + + let worktree_state = store + .load_worktree_state(&wt) + .expect("worktree read should succeed") + .expect("worktree row should exist"); + assert_eq!( + worktree_state.revision, 1, + "revision should have advanced exactly once, not once per writer" + ); + + let persisted_event = store + .load_mutation_event(&wt, 1) + .expect("mutation-event read should succeed") + .expect("exactly one writer's mutation event should be visible at the new revision"); + + assert_race_winner_state(&store, &wt, &persisted_event, &writer_a, &writer_b); + + remove_test_db(&db_path); + } + + fn assert_atomic_rollback_state( + store: &MutationTraceStore, + worktree: &WorktreeId, + scope_id: &ScopeId, + rolled_back_active_scope: &ScopeId, + surviving_active_scope: &ScopeId, + ) { + let worktree_state = store + .load_worktree_state(worktree) + .expect("worktree read should succeed") + .expect("worktree row should exist"); + assert_eq!(worktree_state.revision, 0, "revision must roll back"); + assert_eq!( + worktree_state.cursor_tree, + TreeId("tree-0".to_string()), + "cursor_tree must roll back" + ); + assert_eq!( + worktree_state.failure_kind, + FailureKind::Healthy, + "failure_kind must roll back" + ); + assert!( + !worktree_state.needs_rebaseline, + "needs_rebaseline must roll back" + ); + + let scope_state = store + .load_scope(scope_id) + .expect("scope read should succeed") + .expect("scope row should exist"); + assert_eq!( + scope_state.status, + ScopeStatus::Active, + "scope status must roll back even though its UPDATE ran before the failure" + ); + + assert!( + !store + .processed_event_exists(&EventKey { + scope_id: scope_id.clone(), + event_id: EventId("event-1".to_string()), + }) + .expect("processed-event read should succeed"), + "the processed event must not exist even though its INSERT ran before the failure" + ); + + assert!( + store + .load_mutation_event(worktree, 1) + .expect("mutation-event read should succeed") + .is_none(), + "the mutation event must not exist even though its INSERT ran before the failure" + ); + + let active_scopes = store + .load_mutation_event_active_scopes(worktree, encode_revision(1).as_slice()) + .expect("active-scope read should succeed"); + assert!( + !active_scopes.contains(rolled_back_active_scope), + "the active-scope INSERT that was ordered before the colliding one ran but must \ + have rolled back with everything else in the transaction" + ); + assert_eq!( + active_scopes, + BTreeSet::from([surviving_active_scope.clone()]), + "only the pre-seeded row, which predates this transaction and was never part of \ + it, should remain" + ); + } + + #[test] + fn commit_rolls_back_every_write_kind_together_on_a_deterministic_failure() { + let db_path = unique_test_db_path("commit-atomic-rollback"); + let db = RepositoryAgentTraceDb::new_at(&db_path).expect("repository DB should open"); + let store = MutationTraceStore::new(&db); + let wt = WorktreeId("wt0".to_string()); + let scope_id = ScopeId("scope0".to_string()); + let scope_a = ScopeId("scope-a".to_string()); + let scope_z = ScopeId("scope-z".to_string()); + insert_worktree(&db, &wt.0, 0); + insert_scope(&db, &scope_id.0, &wt.0, ScopeStatus::Active); + insert_active_scope(&db, &wt.0, 1, &scope_z.0); + + let before = state_with_scope( + &wt, + &scope_id, + ActorKind::ClaudeCode, + ScopeStatus::Active, + 0, + ); + let mut after = before.clone(); + after.worktrees.get_mut(&wt).unwrap().revision = 1; + after.scopes.get_mut(&scope_id).unwrap().status = ScopeStatus::Closed; + after.processed_events.insert(EventKey { + scope_id: scope_id.clone(), + event_id: EventId("event-1".to_string()), + }); + after.mutation_events.insert(MutationEvent { + worktree_id: wt.clone(), + revision: 1, + before_tree: TreeId("tree0".to_string()), + after_tree: TreeId("tree1".to_string()), + active_scopes: BTreeSet::from([scope_a.clone(), scope_z.clone()]), + tainted: false, + failure_kind: FailureKind::Healthy, + attribution: Attribution::AiExclusive(scope_id.clone()), + boundary: Boundary::Close { + scope: scope_id.clone(), + event: EventId("event-1".to_string()), + }, + }); + + let transition = DurableTransition::between(&before, &after, &wt) + .expect("between should succeed") + .expect("a transition should exist for this change"); + + let error = store.commit(&transition).expect_err( + "the pre-seeded (wt0, revision=1, scope-z) active-scope row should collide with \ + the second active-scope insert, after every earlier write kind already succeeded", + ); + assert!(error.to_string().contains("execute failed")); + + assert_atomic_rollback_state(&store, &wt, &scope_id, &scope_a, &scope_z); + + remove_test_db(&db_path); + } + + #[test] + fn commit_round_trips_u64_max_through_the_real_database() { + let db_path = unique_test_db_path("commit-u64-max"); + let db = RepositoryAgentTraceDb::new_at(&db_path).expect("repository DB should open"); + let store = MutationTraceStore::new(&db); + let wt = WorktreeId("wt0".to_string()); + insert_worktree(&db, &wt.0, u64::MAX - 1); + + let mut before = ProtocolState::default(); + before + .worktrees + .insert(wt.clone(), healthy_worktree_state(u64::MAX - 1)); + let mut after = before.clone(); + after.worktrees.get_mut(&wt).unwrap().revision = u64::MAX; + + let transition = DurableTransition::between(&before, &after, &wt) + .expect("between should succeed") + .expect("a transition should exist for this change"); + + let result = store.commit(&transition).expect("commit should succeed"); + assert_eq!(result, CasResult::Applied); + + let worktree_state = store + .load_worktree_state(&wt) + .expect("worktree read should succeed") + .expect("worktree row should exist"); + assert_eq!(worktree_state.revision, u64::MAX); + + remove_test_db(&db_path); + } + + #[test] + fn commit_rejects_a_replayed_event_key_via_the_processed_event_uniqueness_constraint() { + let db_path = unique_test_db_path("commit-replay-uniqueness"); + let db = RepositoryAgentTraceDb::new_at(&db_path).expect("repository DB should open"); + let store = MutationTraceStore::new(&db); + let wt = WorktreeId("wt0".to_string()); + let scope_id = ScopeId("scope0".to_string()); + insert_worktree(&db, &wt.0, 0); + insert_scope(&db, &scope_id.0, &wt.0, ScopeStatus::Active); + insert_processed_event(&db, "scope0", "event-1"); + + let before = state_with_scope( + &wt, + &scope_id, + ActorKind::ClaudeCode, + ScopeStatus::Active, + 0, + ); + let mut after = before.clone(); + after.worktrees.get_mut(&wt).unwrap().revision = 1; + after.processed_events.insert(EventKey { + scope_id: scope_id.clone(), + event_id: EventId("event-1".to_string()), + }); + + let transition = DurableTransition::between(&before, &after, &wt) + .expect("between should succeed") + .expect("a transition should exist for this change"); + + let error = store + .commit(&transition) + .expect_err("a replayed (scope_id, event_id) must be rejected, not silently applied"); + assert!(error.to_string().contains("execute failed")); + + let worktree_state = store + .load_worktree_state(&wt) + .expect("worktree read should succeed") + .expect("worktree row should exist"); + assert_eq!( + worktree_state.revision, 0, + "the whole transaction must roll back on a replay rejection" + ); + + remove_test_db(&db_path); + } + + #[test] + fn commit_of_strong_recovery_abandons_every_live_scope_on_the_worktree() { + let db_path = unique_test_db_path("commit-strong-recovery"); + let db = RepositoryAgentTraceDb::new_at(&db_path).expect("repository DB should open"); + let store = MutationTraceStore::new(&db); + let wt = WorktreeId("wt0".to_string()); + let scope_a = ScopeId("scope-a".to_string()); + let scope_b = ScopeId("scope-b".to_string()); + insert_worktree(&db, &wt.0, 0); + insert_scope(&db, &scope_a.0, &wt.0, ScopeStatus::Active); + insert_scope(&db, &scope_b.0, &wt.0, ScopeStatus::Active); + + let mut before = ProtocolState::default(); + before.worktrees.insert( + wt.clone(), + WorktreeState { + cursor_tree: TreeId("tree0".to_string()), + revision: 0, + tainted: true, + failure_kind: FailureKind::SnapshotFailure, + needs_rebaseline: false, + }, + ); + before.scopes.insert( + scope_a.clone(), + ScopeState { + status: ScopeStatus::Active, + actor_kind: ActorKind::ClaudeCode, + worktree_id: wt.clone(), + }, + ); + before.scopes.insert( + scope_b.clone(), + ScopeState { + status: ScopeStatus::Active, + actor_kind: ActorKind::ClaudeCode, + worktree_id: wt.clone(), + }, + ); + + let after = recover(&before, &wt, TreeId("tree1".to_string())); + let transition = DurableTransition::between(&before, &after, &wt) + .expect("between should succeed") + .expect("strong recovery should produce a durable transition"); + + let result = store.commit(&transition).expect("commit should succeed"); + assert_eq!(result, CasResult::Applied); + + for scope_id in [&scope_a, &scope_b] { + let scope_state = store + .load_scope(scope_id) + .expect("scope read should succeed") + .expect("scope row should exist"); + assert_eq!(scope_state.status, ScopeStatus::Abandoned); + } + + remove_test_db(&db_path); + } + + #[test] + fn commit_of_needs_only_recovery_leaves_live_scopes_active() { + let db_path = unique_test_db_path("commit-needs-only-recovery"); + let db = RepositoryAgentTraceDb::new_at(&db_path).expect("repository DB should open"); + let store = MutationTraceStore::new(&db); + let wt = WorktreeId("wt0".to_string()); + let scope_id = ScopeId("scope0".to_string()); + insert_worktree(&db, &wt.0, 0); + insert_scope(&db, &scope_id.0, &wt.0, ScopeStatus::Active); + + let mut before = ProtocolState::default(); + before.worktrees.insert( + wt.clone(), + WorktreeState { + cursor_tree: TreeId("tree0".to_string()), + revision: 0, + tainted: false, + failure_kind: FailureKind::Healthy, + needs_rebaseline: true, + }, + ); + before.scopes.insert( + scope_id.clone(), + ScopeState { + status: ScopeStatus::Active, + actor_kind: ActorKind::ClaudeCode, + worktree_id: wt.clone(), + }, + ); + + let after = recover(&before, &wt, TreeId("tree1".to_string())); + let transition = DurableTransition::between(&before, &after, &wt) + .expect("between should succeed") + .expect("needs-only recovery should produce a durable transition"); + + let result = store.commit(&transition).expect("commit should succeed"); + assert_eq!(result, CasResult::Applied); + + let worktree_state = store + .load_worktree_state(&wt) + .expect("worktree read should succeed") + .expect("worktree row should exist"); + assert!(!worktree_state.needs_rebaseline); + + let scope_state = store + .load_scope(&scope_id) + .expect("scope read should succeed") + .expect("scope row should exist"); + assert_eq!( + scope_state.status, + ScopeStatus::Active, + "a live scope must survive needs-only recovery untouched" + ); + + remove_test_db(&db_path); + } } diff --git a/context/plans/mutation-cursor-store-persistence.md b/context/plans/mutation-cursor-store-persistence.md index 78862c61..543b1862 100644 --- a/context/plans/mutation-cursor-store-persistence.md +++ b/context/plans/mutation-cursor-store-persistence.md @@ -257,13 +257,24 @@ Persist this field in every plan; this is durable plan state, not chat state: - Verify (T07 correction, 2026-08-29): `./scripts/run-cli-cargo.sh test --manifest-path cli/Cargo.toml mutation_trace::` — passed; `./scripts/run-cli-cargo.sh fmt --manifest-path cli/Cargo.toml -- --check` — passed; `./scripts/run-cli-cargo.sh clippy --manifest-path cli/Cargo.toml --all-targets -- -D warnings` — passed. - Context synchronization: synced -- [ ] T08: `Add CAS and concurrency test coverage for store.commit` (status:todo) +- [x] T08: `Add CAS and concurrency test coverage for store.commit` (status:done) - Task ID: T08 - Scope: In — tests for: two writers committing from the same revision against one physical repository-scoped `agent-trace.db`, using two independent `RepositoryAgentTraceDb` handles/connections opened against that same database file (one `MutationTraceStore` per handle), with both writers loading worktree revision `N` before either commits and executing their commits from separate threads (or an equivalent that exercises two independent DB connections rather than one handle invoked twice in sequence) — exactly one result `CasResult::Applied`, the other `CasResult::Conflict`; atomic rollback on an injected deterministic mid-transaction failure; `u64::MAX` round-trip through the real DB; `(scope_id, event_id)` replay-uniqueness rejection; strong recovery (all active scopes abandoned) and needs-only recovery (surviving active scopes stay active). Out — production code changes beyond what T07 already provides; process-spawning or other multiprocess test infrastructure (two independent DB handles on separate threads are sufficient for this PR). - Dependencies: T07 - Done when: all five scenarios above are covered by passing tests; the two-writer test is not satisfied by calling `commit` twice sequentially through one shared `RepositoryAgentTraceDb` handle; after both commits, reopening the database shows the worktree revision advanced exactly once and only the winning transition's durable effects (scope status, processed event, mutation event, active scopes) are present; the atomic-rollback test observes revision, scope status, processed event, mutation event, and active scopes all unchanged after the injected failure. - Verify: `./scripts/run-cli-cargo.sh test --manifest-path cli/Cargo.toml mutation_trace::store::` - - Context synchronization: pending + - Completed: 2026-08-29 + - Files changed: `cli/src/services/mutation_trace/store.rs` + - Result: Added six tests to `store.rs`'s existing `#[cfg(test)] mod tests`, covering all five required scenarios. `commit_from_two_independent_connections_races_and_only_one_applies` seeds one worktree with two active scopes via one `RepositoryAgentTraceDb::new_at`, then opens two independent handles against the same path via `RepositoryAgentTraceDb::open_for_hooks_without_migrations_at` (mirroring the existing `concurrent_initialization_converges_on_one_source_instance_id` pattern in `agent_trace_db/repository.rs` — schema created once up front, then each racer opens its own connection), builds two distinct `DurableTransition`s from the same `before` state (each closing a different scope), and commits both from separate `thread::spawn` closures; asserts exactly one `Applied`/one `Conflict`, then reopens a third handle and asserts the revision advanced exactly once and only one scope shows the `Closed` status. `commit_rolls_back_every_write_kind_together_on_a_deterministic_failure` builds a transition with a scope-status change, a processed event, and a mutation event with active scopes, injects a pre-existing colliding `(scope_id, event_id)` row, and asserts revision, scope status, mutation event (`load_mutation_event`), and active scopes (`load_mutation_event_active_scopes`) are all absent/unchanged after the `Err`. `commit_round_trips_u64_max_through_the_real_database` commits a transition advancing `u64::MAX - 1 -> u64::MAX` and reads it back exactly. `commit_rejects_a_replayed_event_key_via_the_processed_event_uniqueness_constraint` is a dedicated replay-uniqueness test distinct from T07's existing deterministic-failure coverage. `commit_of_strong_recovery_abandons_every_live_scope_on_the_worktree` and `commit_of_needs_only_recovery_leaves_live_scopes_active` drive `protocol::recover` directly on a tainted-worktree-with-live-scopes state and a `needs_rebaseline`-only-worktree-with-a-live-scope state respectively, commit the resulting `DurableTransition`, and assert the persisted scope statuses (`Abandoned` vs. unchanged `Active`). + - Verify: `./scripts/run-cli-cargo.sh test --manifest-path cli/Cargo.toml mutation_trace::store::` — passed, 62/62 (56 pre-existing + 6 new); the two-writer race test was additionally run 13 times in isolation (5 before a clippy-driven refactor, 8 after) with no flakiness observed; `./scripts/run-cli-cargo.sh fmt --manifest-path cli/Cargo.toml -- --check` — passed after one auto-formatting pass; `./scripts/run-cli-cargo.sh clippy --manifest-path cli/Cargo.toml --all-targets -- -D warnings` — passed, zero warnings (required restructuring the two-writer and strong-recovery tests' per-scope read-back from paired `scope_a_state`/`scope_b_state` bindings into an array-map/loop, since `-D clippy::pedantic`'s `similar_names` lint flagged the paired bindings as too similar). + - Done checks: all five scenarios (two-writer race, atomic rollback, `u64::MAX` round-trip, replay-uniqueness rejection, strong/needs-only recovery) are covered by passing tests (verified above); the two-writer test uses two independent `RepositoryAgentTraceDb` handles on separate threads, never one shared handle invoked twice (verified by inspection — `db_a`/`db_b` are distinct `open_for_hooks_without_migrations_at` calls moved into separate `thread::spawn` closures); after both commits, a third reopened handle shows the revision advanced exactly once (`1`, not `2`) and exactly one scope closed (verified by the test's assertions); the atomic-rollback test observes revision, scope status, mutation event, and active scopes all unchanged/absent after the injected failure (verified — processed-event absence was already covered by the pre-existing collision precondition, since the colliding row was inserted before `commit` and the transition's own new entry never lands). + - Context impact: local — new tests confined to `store.rs`'s existing test module, exercising only `MutationTraceStore::commit`/`DurableTransition::between`/`protocol::recover`, all already documented as existing by T07's context impact; no production code changed, no new call site, no claim in any root context file is affected. Deferred to T11/plan-level context sync per the plan's existing assumption. + - Context synchronization: synced + - **T08 correction (2026-08-29):** Fixed two test-coverage gaps found in review of PR #241. (1) `commit_rolls_back_every_write_kind_together_on_a_deterministic_failure` previously injected its deterministic failure at the processed-event insert — the second statement in `commit`'s batch — so the mutation-event insert and both active-scope inserts were never reached, leaving the test's name and the T08 "Done when" contract's claimed atomicity unproven for those write kinds. Reworked it so every earlier write kind (worktree CAS, scope update, processed-event insert, mutation-event insert, and the active-scope insert for `scope-a`) succeeds inside the transaction before a pre-seeded `(wt0, revision=1, scope-z)` row collides with the second active-scope insert; `scope-a` is guaranteed to insert before `scope-z` because `commit` iterates `MutationEvent.active_scopes` (a `BTreeSet`) in ascending order. Post-failure assertions (now in a new `assert_atomic_rollback_state` helper) check worktree revision/cursor_tree/failure_kind/needs_rebaseline, scope status, processed-event absence, mutation-event absence, and that the active-scopes table contains only the pre-seeded `scope-z` row — proving `scope-a`'s insert, which ran before the failure, rolled back together with everything else. (2) `commit_from_two_independent_connections_races_and_only_one_applies` previously gave writer A and writer B only distinct scope-status changes, so it could not prove winner/loser isolation for processed events, mutation events, or active scopes, which the T08 "Done when" contract requires. Reworked both competing transitions (via a new `race_evidence`/`closing_transition` helper pair, still built exclusively through `DurableTransition::between`) to each carry a full, distinct evidence set — a closed scope, an `EventKey`, and a `MutationEvent` with a distinct `after_tree`/`boundary`/`active_scopes` — so after the race, `load_mutation_event(wt, 1)` identifies the winner by content (not by thread result ordering) via a new `assert_race_winner_state` helper, which then asserts the winner's scope/processed-event/mutation-event/active-scopes are all present and field-for-field correct while the loser's scope stays `Active`, its `EventKey` does not exist, and no loser-only active-scope rows remain. Both tests still use two independent `RepositoryAgentTraceDb` handles on separate threads racing from the same starting revision, with no sleeps; `DurableTransition` gained no public fields, constructors, builders, or setters — every transition in both tests is still obtained exclusively through `DurableTransition::between`. No production code, migration, `protocol.rs`, Quint spec, `DurableTransition::between` semantics, the T06 CAS primitive, retry classification, or `MutationTraceStore::commit`'s SQL ordering was changed; the stronger tests exposed no real bug. T09 was not started. + - Verify (T08 correction): `./scripts/run-cli-cargo.sh test --manifest-path cli/Cargo.toml mutation_trace::store::` — passed, 62/62; the two-writer race test was additionally run 27 times in isolation across the correction (15 immediately after the rewrite, 12 more after the clippy-driven decomposition below) with no flakiness observed; `./scripts/run-cli-cargo.sh fmt --manifest-path cli/Cargo.toml -- --check` — passed after one auto-formatting pass; `./scripts/run-cli-cargo.sh clippy --manifest-path cli/Cargo.toml --all-targets -- -D warnings` — passed, zero warnings (required decomposing both rewritten tests into helper functions — `race_evidence`/`closing_transition`/`assert_race_winner_state` and `assert_atomic_rollback_state` — to satisfy `clippy::too_many_lines` under this crate's `-D clippy::pedantic`). + - Done checks (T08 correction): the atomic-rollback test's failure now occurs at the final active-scope insert, after every earlier write kind (worktree CAS, scope update, processed-event insert, mutation-event insert, first active-scope insert) has succeeded inside the transaction (verified by the pre-seeded-row placement and the BTreeSet-ordering argument, plus the post-rollback assertion that the pre-seeded row is the only active-scope row remaining); the race test's two competing transitions each carry a distinct scope/EventKey/MutationEvent, are each built exclusively through `DurableTransition::between`, and the winner is identified from `load_mutation_event`'s content rather than thread result ordering (verified by `assert_race_winner_state`'s branch on `persisted_event`); after reopening, the winner's scope/processed-event/mutation-event/active-scopes are all present and correct while none of the loser's evidence is present (verified by `assert_race_winner_state`'s full assertion set); `DurableTransition` still has no public fields, constructors, builders, or setters (verified by inspection — unchanged from T07); no production file changed (verified — `git diff` touches only `cli/src/services/mutation_trace/store.rs`, entirely within `#[cfg(test)] mod tests`); T09 was not started (verified — no changes to any file outside `store.rs`/this plan). + - Context impact: local — the corrected tests and their new helper functions remain confined to `store.rs`'s existing test module, exercising the same already-documented `MutationTraceStore::commit`/`DurableTransition::between` surface as the original T08 tests; no production code changed, no new call site, no claim in any root context file is affected. + - Context synchronization: synced - [ ] T09: `Add real-protocol round-trip persistence tests` (status:todo) - Task ID: T09 From 837cb1b9e59054c83cb6a922fb7415fc564f6fcc Mon Sep 17 00:00:00 2001 From: David Abram Date: Sat, 29 Aug 2026 12:11:20 +0200 Subject: [PATCH 12/15] tests: Add mutation trace persistence round-trip coverage Exercise the real database load, protocol transition, commit, reopen, and reload flow across mutation trace boundaries and recovery cases. Cover Start, Advance, Close, Flush, taint, abandon, recovery, contention, and replay handling, including exact mutation-event decoding, and record T09 completion. Plan: mutation-cursor-store-persistence (T09) Co-authored-by: SCE --- cli/src/services/mutation_trace/store.rs | 580 ++++++++++++++++++ .../mutation-cursor-store-persistence.md | 15 +- 2 files changed, 593 insertions(+), 2 deletions(-) diff --git a/cli/src/services/mutation_trace/store.rs b/cli/src/services/mutation_trace/store.rs index 09dd7f9e..946fcb8f 100644 --- a/cli/src/services/mutation_trace/store.rs +++ b/cli/src/services/mutation_trace/store.rs @@ -3024,4 +3024,584 @@ mod tests { remove_test_db(&db_path); } + + fn insert_worktree_with_state( + db: &RepositoryAgentTraceDb, + worktree_id: &str, + revision: u64, + tainted: bool, + failure_kind: FailureKind, + needs_rebaseline: bool, + ) { + db.execute( + "INSERT INTO mutation_trace_worktrees + (worktree_id, cursor_tree, revision, tainted, failure_kind, needs_rebaseline) + VALUES (?1, 'tree0', ?2, ?3, ?4, ?5)", + ( + worktree_id, + encode_revision(revision).as_slice(), + tainted, + encode_failure_kind(failure_kind), + needs_rebaseline, + ), + ) + .expect("worktree insert should succeed"); + } + + fn reopen_store(db_path: &std::path::Path) -> RepositoryAgentTraceDb { + RepositoryAgentTraceDb::open_for_hooks_without_migrations_at(db_path) + .expect("reopened handle should open") + } + + fn load_before_state( + db_path: &std::path::Path, + worktree: &WorktreeId, + scope: Option<&ScopeId>, + event_key: Option<&EventKey>, + ) -> ProtocolState { + let db = reopen_store(db_path); + MutationTraceStore::new(&db) + .load_worktree(worktree, scope, event_key) + .expect("load_worktree should succeed") + .expect("worktree projection should exist") + .into_protocol_state() + } + + fn commit_transition(db_path: &std::path::Path, transition: &DurableTransition) -> CasResult { + let db = reopen_store(db_path); + MutationTraceStore::new(&db) + .commit(transition) + .expect("commit should succeed") + } + + fn expected_projection( + after: &ProtocolState, + worktree: &WorktreeId, + scope: Option<&ScopeId>, + event_key: Option<&EventKey>, + ) -> WorktreeProjection { + let worktree_state = after + .worktrees + .get(worktree) + .expect("after should contain the worktree") + .clone(); + + let mut scopes: BTreeMap = after + .scopes + .iter() + .filter(|(_, scope_state)| { + scope_state.worktree_id == *worktree && scope_state.status == ScopeStatus::Active + }) + .map(|(scope_id, scope_state)| (scope_id.clone(), scope_state.clone())) + .collect(); + + let effective_scope = scope.or(event_key.map(|key| &key.scope_id)); + if let Some(effective_scope) = effective_scope { + if let Some(scope_state) = after.scopes.get(effective_scope) { + scopes.insert(effective_scope.clone(), scope_state.clone()); + } + } + + let mut processed_events = BTreeSet::new(); + if let Some(key) = event_key { + if after.processed_events.contains(key) { + processed_events.insert(key.clone()); + } + } + + WorktreeProjection { + worktree_id: worktree.clone(), + worktree_state, + scopes, + processed_events, + } + } + + fn assert_round_trip( + db_path: &std::path::Path, + worktree: &WorktreeId, + scope: Option<&ScopeId>, + event_key: Option<&EventKey>, + after: &ProtocolState, + ) -> WorktreeProjection { + let db = reopen_store(db_path); + let store = MutationTraceStore::new(&db); + + let reloaded = store + .load_worktree(worktree, scope, event_key) + .expect("load_worktree should succeed") + .expect("worktree projection should exist"); + + assert_eq!( + reloaded, + expected_projection(after, worktree, scope, event_key) + ); + + for expected_event in &after.mutation_events { + let reloaded_event = store + .load_mutation_event(worktree, expected_event.revision) + .expect("load_mutation_event should succeed") + .expect("mutation event row should exist"); + assert_eq!(&reloaded_event, expected_event); + } + + reloaded + } + + #[test] + fn round_trip_start_persists_and_reloads_exactly_after_reopening_the_database() { + let db_path = unique_test_db_path("roundtrip-start"); + let wt = WorktreeId("wt0".to_string()); + let scope_id = ScopeId("scope0".to_string()); + { + let db = RepositoryAgentTraceDb::new_at(&db_path).expect("repository DB should open"); + insert_worktree(&db, &wt.0, 0); + insert_scope(&db, &scope_id.0, &wt.0, ScopeStatus::NeverSeen); + } + + let before = load_before_state(&db_path, &wt, Some(&scope_id), None); + let attempt = AttemptId("attempt0".to_string()); + let event_id = EventId("event0".to_string()); + let prepared = prepare( + &before, + attempt.clone(), + Boundary::Start { + scope: scope_id.clone(), + event: event_id.clone(), + }, + TreeId("tree1".to_string()), + ); + let outcome = commit(&prepared, &attempt); + assert!( + outcome.evaluation.accepted, + "a fresh Start should be accepted" + ); + let after = outcome.state; + + let transition = DurableTransition::between(&before, &after, &wt) + .expect("between should succeed") + .expect("a start transition should produce a durable transition"); + assert_eq!(commit_transition(&db_path, &transition), CasResult::Applied); + + let event_key = EventKey { + scope_id: scope_id.clone(), + event_id, + }; + assert_round_trip(&db_path, &wt, Some(&scope_id), Some(&event_key), &after); + + remove_test_db(&db_path); + } + + #[test] + fn round_trip_advance_persists_and_reloads_exactly_after_reopening_the_database() { + let db_path = unique_test_db_path("roundtrip-advance"); + let wt = WorktreeId("wt0".to_string()); + let scope_id = ScopeId("scope0".to_string()); + { + let db = RepositoryAgentTraceDb::new_at(&db_path).expect("repository DB should open"); + insert_worktree(&db, &wt.0, 0); + insert_scope(&db, &scope_id.0, &wt.0, ScopeStatus::Active); + } + + let before = load_before_state(&db_path, &wt, Some(&scope_id), None); + let attempt = AttemptId("attempt0".to_string()); + let event_id = EventId("event0".to_string()); + let prepared = prepare( + &before, + attempt.clone(), + Boundary::Advance { + scope: scope_id.clone(), + event: event_id.clone(), + }, + TreeId("tree1".to_string()), + ); + let outcome = commit(&prepared, &attempt); + assert!( + outcome.evaluation.accepted, + "a fresh Advance should be accepted" + ); + let after = outcome.state; + + let transition = DurableTransition::between(&before, &after, &wt) + .expect("between should succeed") + .expect("an advance transition should produce a durable transition"); + assert_eq!(commit_transition(&db_path, &transition), CasResult::Applied); + + let event_key = EventKey { + scope_id: scope_id.clone(), + event_id, + }; + assert_round_trip(&db_path, &wt, Some(&scope_id), Some(&event_key), &after); + + remove_test_db(&db_path); + } + + #[test] + fn round_trip_close_persists_and_reloads_exactly_after_reopening_the_database() { + let db_path = unique_test_db_path("roundtrip-close"); + let wt = WorktreeId("wt0".to_string()); + let scope_id = ScopeId("scope0".to_string()); + { + let db = RepositoryAgentTraceDb::new_at(&db_path).expect("repository DB should open"); + insert_worktree(&db, &wt.0, 0); + insert_scope(&db, &scope_id.0, &wt.0, ScopeStatus::Active); + } + + let before = load_before_state(&db_path, &wt, Some(&scope_id), None); + let attempt = AttemptId("attempt0".to_string()); + let event_id = EventId("event0".to_string()); + let prepared = prepare( + &before, + attempt.clone(), + Boundary::Close { + scope: scope_id.clone(), + event: event_id.clone(), + }, + TreeId("tree1".to_string()), + ); + let outcome = commit(&prepared, &attempt); + assert!( + outcome.evaluation.accepted, + "a fresh Close should be accepted" + ); + let after = outcome.state; + assert_eq!( + after.scopes.get(&scope_id).map(|s| s.status), + Some(ScopeStatus::Closed) + ); + + let transition = DurableTransition::between(&before, &after, &wt) + .expect("between should succeed") + .expect("a close transition should produce a durable transition"); + assert_eq!(commit_transition(&db_path, &transition), CasResult::Applied); + + let event_key = EventKey { + scope_id: scope_id.clone(), + event_id, + }; + assert_round_trip(&db_path, &wt, Some(&scope_id), Some(&event_key), &after); + + remove_test_db(&db_path); + } + + #[test] + fn round_trip_flush_with_change_persists_and_reloads_exactly_after_reopening_the_database() { + let db_path = unique_test_db_path("roundtrip-flush-change"); + let wt = WorktreeId("wt0".to_string()); + { + let db = RepositoryAgentTraceDb::new_at(&db_path).expect("repository DB should open"); + insert_worktree(&db, &wt.0, 0); + } + + let before = load_before_state(&db_path, &wt, None, None); + let attempt = AttemptId("attempt0".to_string()); + let prepared = prepare( + &before, + attempt.clone(), + Boundary::Flush { + worktree: wt.clone(), + }, + TreeId("tree1".to_string()), + ); + let outcome = commit(&prepared, &attempt); + assert!( + outcome.evaluation.changed, + "an observed tree change should be recorded" + ); + let after = outcome.state; + + let transition = DurableTransition::between(&before, &after, &wt) + .expect("between should succeed") + .expect("a changed flush transition should produce a durable transition"); + assert_eq!(commit_transition(&db_path, &transition), CasResult::Applied); + + assert_round_trip(&db_path, &wt, None, None, &after); + + remove_test_db(&db_path); + } + + #[test] + fn round_trip_flush_without_change_persists_nothing_new() { + let db_path = unique_test_db_path("roundtrip-flush-no-change"); + let wt = WorktreeId("wt0".to_string()); + { + let db = RepositoryAgentTraceDb::new_at(&db_path).expect("repository DB should open"); + insert_worktree(&db, &wt.0, 0); + } + + let before = load_before_state(&db_path, &wt, None, None); + let attempt = AttemptId("attempt0".to_string()); + let prepared = prepare( + &before, + attempt.clone(), + Boundary::Flush { + worktree: wt.clone(), + }, + before.worktrees[&wt].cursor_tree.clone(), + ); + let outcome = commit(&prepared, &attempt); + assert!( + !outcome.evaluation.observed_change, + "flushing the same tree should observe no change" + ); + let after = outcome.state; + + assert_eq!( + DurableTransition::between(&before, &after, &wt).expect("between should succeed"), + None, + "a no-change flush must produce no durable transition to persist" + ); + + assert_round_trip(&db_path, &wt, None, None, &after); + + remove_test_db(&db_path); + } + + #[test] + fn round_trip_taint_persists_and_reloads_exactly_after_reopening_the_database() { + let db_path = unique_test_db_path("roundtrip-taint"); + let wt = WorktreeId("wt0".to_string()); + { + let db = RepositoryAgentTraceDb::new_at(&db_path).expect("repository DB should open"); + insert_worktree(&db, &wt.0, 0); + } + + let before = load_before_state(&db_path, &wt, None, None); + let after = taint(&before, &wt); + + let transition = DurableTransition::between(&before, &after, &wt) + .expect("between should succeed") + .expect("taint should produce a durable transition"); + assert_eq!(commit_transition(&db_path, &transition), CasResult::Applied); + + assert_round_trip(&db_path, &wt, None, None, &after); + + remove_test_db(&db_path); + } + + #[test] + fn round_trip_abandon_persists_and_reloads_exactly_after_reopening_the_database() { + let db_path = unique_test_db_path("roundtrip-abandon"); + let wt = WorktreeId("wt0".to_string()); + let scope_id = ScopeId("scope0".to_string()); + { + let db = RepositoryAgentTraceDb::new_at(&db_path).expect("repository DB should open"); + insert_worktree(&db, &wt.0, 0); + insert_scope(&db, &scope_id.0, &wt.0, ScopeStatus::Active); + } + + let before = load_before_state(&db_path, &wt, Some(&scope_id), None); + let after = abandon(&before, &scope_id); + + let transition = DurableTransition::between(&before, &after, &wt) + .expect("between should succeed") + .expect("abandon should produce a durable transition"); + assert_eq!(commit_transition(&db_path, &transition), CasResult::Applied); + + assert_round_trip(&db_path, &wt, Some(&scope_id), None, &after); + + remove_test_db(&db_path); + } + + #[test] + fn round_trip_strong_recovery_abandons_every_live_scope_after_reopening_the_database() { + let db_path = unique_test_db_path("roundtrip-recover-strong"); + let wt = WorktreeId("wt0".to_string()); + let scope_a = ScopeId("scope-a".to_string()); + let scope_b = ScopeId("scope-b".to_string()); + { + let db = RepositoryAgentTraceDb::new_at(&db_path).expect("repository DB should open"); + insert_worktree_with_state(&db, &wt.0, 0, true, FailureKind::SnapshotFailure, false); + insert_scope(&db, &scope_a.0, &wt.0, ScopeStatus::Active); + insert_scope(&db, &scope_b.0, &wt.0, ScopeStatus::Active); + } + + let before = load_before_state(&db_path, &wt, None, None); + let after = recover(&before, &wt, TreeId("tree1".to_string())); + + let transition = DurableTransition::between(&before, &after, &wt) + .expect("between should succeed") + .expect("strong recovery should produce a durable transition"); + assert_eq!(commit_transition(&db_path, &transition), CasResult::Applied); + + assert_round_trip(&db_path, &wt, None, None, &after); + + let db = reopen_store(&db_path); + let store = MutationTraceStore::new(&db); + for scope_id in [&scope_a, &scope_b] { + let scope_state = store + .load_scope(scope_id) + .expect("scope read should succeed") + .expect("scope row should exist"); + assert_eq!(scope_state.status, ScopeStatus::Abandoned); + } + + remove_test_db(&db_path); + } + + #[test] + fn round_trip_contended_mutation_persists_and_reloads_exactly_after_reopening_the_database() { + let db_path = unique_test_db_path("roundtrip-contended"); + let wt = WorktreeId("wt0".to_string()); + let scope_a = ScopeId("scope-a".to_string()); + let scope_b = ScopeId("scope-b".to_string()); + { + let db = RepositoryAgentTraceDb::new_at(&db_path).expect("repository DB should open"); + insert_worktree(&db, &wt.0, 0); + insert_scope(&db, &scope_a.0, &wt.0, ScopeStatus::Active); + insert_scope(&db, &scope_b.0, &wt.0, ScopeStatus::Active); + } + + let before = load_before_state(&db_path, &wt, None, None); + let attempt = AttemptId("attempt0".to_string()); + let prepared = prepare( + &before, + attempt.clone(), + Boundary::Flush { + worktree: wt.clone(), + }, + TreeId("tree1".to_string()), + ); + let outcome = commit(&prepared, &attempt); + assert!( + outcome.evaluation.changed, + "an observed tree change should be recorded" + ); + let after = outcome.state; + let mutation_event = after + .mutation_events + .iter() + .next() + .expect("a mutation event should have been produced"); + assert_eq!(mutation_event.attribution, Attribution::AiContended); + assert_eq!( + mutation_event.active_scopes, + BTreeSet::from([scope_a.clone(), scope_b.clone()]) + ); + + let transition = DurableTransition::between(&before, &after, &wt) + .expect("between should succeed") + .expect("a contended flush transition should produce a durable transition"); + assert_eq!(commit_transition(&db_path, &transition), CasResult::Applied); + + let reloaded = assert_round_trip(&db_path, &wt, None, None, &after); + assert_eq!( + reloaded.scopes.keys().cloned().collect::>(), + BTreeSet::from([scope_a.clone(), scope_b.clone()]), + "the reloaded live bounded projection must contain exactly the two contended scopes" + ); + for scope_id in [&scope_a, &scope_b] { + assert_eq!( + reloaded.scopes.get(scope_id).map(|s| s.status), + Some(ScopeStatus::Active) + ); + } + + remove_test_db(&db_path); + } + + #[test] + fn round_trip_database_failure_changes_only_non_persistent_external_taint() { + let db_path = unique_test_db_path("roundtrip-database-failure"); + let wt = WorktreeId("wt0".to_string()); + { + let db = RepositoryAgentTraceDb::new_at(&db_path).expect("repository DB should open"); + insert_worktree(&db, &wt.0, 0); + } + + let before = load_before_state(&db_path, &wt, None, None); + let after = database_failure(&before, &wt); + + assert!(!before.external_taint.contains(&wt)); + assert!(after.external_taint.contains(&wt)); + assert_eq!(before.worktrees, after.worktrees); + assert_eq!(before.scopes, after.scopes); + assert_eq!(before.processed_events, after.processed_events); + assert_eq!( + after.worktrees[&wt].revision, + before.worktrees[&wt].revision + ); + + assert_eq!( + DurableTransition::between(&before, &after, &wt).expect("between should succeed"), + None, + "database_failure only changes external_taint, which is never durable, so no \ + DurableTransition should exist to commit" + ); + + let reloaded = assert_round_trip(&db_path, &wt, None, None, &after); + assert!(reloaded.into_protocol_state().external_taint.is_empty()); + + remove_test_db(&db_path); + } + + #[test] + fn round_trip_a_replayed_event_key_is_rejected_and_does_not_advance_the_worktree_again() { + let db_path = unique_test_db_path("roundtrip-replay"); + let wt = WorktreeId("wt0".to_string()); + let scope_id = ScopeId("scope0".to_string()); + let event_key = EventKey { + scope_id: scope_id.clone(), + event_id: EventId("event0".to_string()), + }; + { + let db = RepositoryAgentTraceDb::new_at(&db_path).expect("repository DB should open"); + insert_worktree(&db, &wt.0, 0); + insert_scope(&db, &scope_id.0, &wt.0, ScopeStatus::Active); + } + + let before = load_before_state(&db_path, &wt, Some(&scope_id), None); + let attempt = AttemptId("attempt0".to_string()); + let prepared = prepare( + &before, + attempt.clone(), + Boundary::Advance { + scope: scope_id.clone(), + event: event_key.event_id.clone(), + }, + TreeId("tree1".to_string()), + ); + let outcome = commit(&prepared, &attempt); + assert!( + outcome.evaluation.accepted, + "the first delivery should be accepted" + ); + let after = outcome.state; + + let transition = DurableTransition::between(&before, &after, &wt) + .expect("between should succeed") + .expect("the first delivery should produce a durable transition"); + assert_eq!(commit_transition(&db_path, &transition), CasResult::Applied); + + let before_replay = load_before_state(&db_path, &wt, Some(&scope_id), Some(&event_key)); + assert!(before_replay.processed_events.contains(&event_key)); + + let replay_attempt = AttemptId("attempt1".to_string()); + let replay_prepared = prepare( + &before_replay, + replay_attempt.clone(), + Boundary::Advance { + scope: scope_id.clone(), + event: event_key.event_id.clone(), + }, + TreeId("tree2".to_string()), + ); + let replay_outcome = commit(&replay_prepared, &replay_attempt); + assert!( + !replay_outcome.evaluation.accepted, + "a replayed EventKey must be rejected" + ); + let after_replay = replay_outcome.state; + + assert_eq!( + DurableTransition::between(&before_replay, &after_replay, &wt) + .expect("between should succeed"), + None, + "a rejected replay must produce no durable transition to persist" + ); + + assert_round_trip(&db_path, &wt, Some(&scope_id), Some(&event_key), &after); + + remove_test_db(&db_path); + } } diff --git a/context/plans/mutation-cursor-store-persistence.md b/context/plans/mutation-cursor-store-persistence.md index 543b1862..432bc69a 100644 --- a/context/plans/mutation-cursor-store-persistence.md +++ b/context/plans/mutation-cursor-store-persistence.md @@ -276,13 +276,24 @@ Persist this field in every plan; this is durable plan state, not chat state: - Context impact: local — the corrected tests and their new helper functions remain confined to `store.rs`'s existing test module, exercising the same already-documented `MutationTraceStore::commit`/`DurableTransition::between` surface as the original T08 tests; no production code changed, no new call site, no claim in any root context file is affected. - Context synchronization: synced -- [ ] T09: `Add real-protocol round-trip persistence tests` (status:todo) +- [x] T09: `Add real-protocol round-trip persistence tests` (status:done) - Task ID: T09 - Scope: In — tests driving load (`load_worktree`, bounded to `Active` scopes plus the transition's referenced scope) -> `protocol::prepare`/`commit` (or `taint`/`database_failure`/`abandon`/`recover`) -> `DurableTransition::between` -> `store.commit` -> drop DB handle -> reopen -> reload, for `Start`, `Advance`, `Close`, `Flush` with change, `Flush` without change, taint, abandon, recover, contended mutation, and a replayed `EventKey`. For every transition that emits a `MutationEvent`, additionally reload it after reopening with `load_mutation_event(worktree, revision)` and compare it field-for-field (including exact `Attribution`/`Boundary` decoding) against the `MutationEvent` the original protocol transition produced. Out — new production code, unless a genuine T01-T07 gap surfaces. - Dependencies: T07 - Done when: for every listed transition, the reloaded worktree/scope projection after reopening the DB matches the durable projection produced by the original protocol transition, and for every transition that emits a `MutationEvent`, `load_mutation_event` after reopening reconstructs it exactly. - Verify: `./scripts/run-cli-cargo.sh test --manifest-path cli/Cargo.toml mutation_trace::store::` - - Context synchronization: pending + - Completed: 2026-08-29 + - Files changed: `cli/src/services/mutation_trace/store.rs` + - Result: Added ten round-trip tests plus five test-only helpers (`insert_worktree_with_state`, `reopen_store`, `load_before_state`, `commit_transition`, `assert_round_trip`) to `store.rs`'s existing `#[cfg(test)] mod tests`, all driving the full real-DB flow the task requires: seed durable rows via a scoped `RepositoryAgentTraceDb::new_at` block (dropped at block end) -> `load_before_state` reopens via `open_for_hooks_without_migrations_at` and calls `MutationTraceStore::load_worktree(..).into_protocol_state()` to obtain `before` from the database itself, not a hand-built `ProtocolState` -> the relevant `protocol::{prepare+commit, taint, abandon, recover}` call produces `after` -> `DurableTransition::between(&before, &after, &wt)` -> `commit_transition` opens a fresh handle and calls `store.commit` -> `assert_round_trip` drops that handle, reopens again, reloads via `load_worktree` with the scenario's `scope`/`event_key` arguments, asserts the reloaded `WorktreeProjection` matches `after`, and for every `MutationEvent` in `after.mutation_events` reloads it via `load_mutation_event` and asserts full field equality (including `Attribution`/`Boundary`). Covered scenarios: `round_trip_start_...`, `round_trip_advance_...`, `round_trip_close_...` (asserts the terminal `Closed` scope is still returned when explicitly referenced, per T03's contract), `round_trip_flush_with_change_...`, `round_trip_flush_without_change_...` (asserts `between` returns `None` and nothing new persists), `round_trip_taint_...`, `round_trip_abandon_...`, `round_trip_strong_recovery_...` (a new `insert_worktree_with_state` helper seeds a tainted worktree so strong recovery is driven from real durable state; asserts both scopes reload as `Abandoned`), `round_trip_contended_mutation_...` (two active scopes, asserts `Attribution::AiContended` and a two-entry `active_scopes` set both round-trip), and `round_trip_a_replayed_event_key_...` (commits a first `Advance`, then re-delivers the same `EventKey` and asserts the replay is rejected — `evaluation.accepted == false` and `DurableTransition::between` returns `None` — with the original transition's projection still the one that reloads correctly). No genuine T01-T07 gap surfaced; the diff is purely additive to the test module (0 production lines changed). + - Verify: `./scripts/run-cli-cargo.sh test --manifest-path cli/Cargo.toml mutation_trace::store::` — passed, 72/72 (62 pre-existing + 10 new); `./scripts/run-cli-cargo.sh test --manifest-path cli/Cargo.toml mutation_trace::` — passed, 160/160 (confirms `protocol`/`mbt`/Quint Connect coverage stayed green); `./scripts/run-cli-cargo.sh fmt --manifest-path cli/Cargo.toml -- --check` — passed after one auto-formatting pass; `./scripts/run-cli-cargo.sh clippy --manifest-path cli/Cargo.toml --all-targets -- -D warnings` — passed, zero warnings. + - Done checks: for every listed transition (`Start`, `Advance`, `Close`, `Flush` with change, `Flush` without change, `taint`, `abandon`, `recover`, contended mutation, replayed `EventKey`), the reloaded worktree/scope projection after dropping and reopening the DB handle matches the durable projection produced by the original protocol transition (verified by each `round_trip_*` test's `assert_round_trip` call, which reopens via a fresh `RepositoryAgentTraceDb` handle before reloading); for every transition that emits a `MutationEvent` (`Start`/`Advance`/`Close`/`Flush`-with-change/contended), `load_mutation_event` after reopening reconstructs it exactly, including `Attribution`/`Boundary` (verified — `assert_round_trip` iterates `after.mutation_events` and asserts full struct equality against the reloaded row, which exercises `reconstruct_attribution`/`reconstruct_boundary` for `AiExclusive`, `AiContended`, and every `BoundaryKind` covered by these scenarios). + - Context impact: local — all ten new tests and their five helpers are confined to `store.rs`'s existing test module, exercising only the already-documented `MutationTraceStore`/`DurableTransition`/`protocol::*` surface T02-T08 already built and already covered by root-context claims (or the lack thereof, per the plan's existing deferred-to-T11 assumption); no production code changed, no new call site, no claim in any root context file is affected. + - Context synchronization: synced + - **T09 correction (2026-08-29):** Fixed two test-coverage gaps found in review of PR #241. (1) `database_failure` was explicitly named in T09's scope alongside `taint`/`abandon`/`recover` but had no round-trip test — added `round_trip_database_failure_changes_only_non_persistent_external_taint`, which drives `protocol::database_failure` on a `before` state loaded from the real DB, asserts `after.external_taint` gains the worktree while `before.worktrees`/`before.scopes`/`before.processed_events` and the worktree's `revision` are all unchanged, asserts `DurableTransition::between(&before, &after, &wt)` returns `None` (so `store.commit` is never called — there is nothing durable to commit), then reopens the DB and asserts the reloaded projection is exactly the pre-failure state, including `reloaded.into_protocol_state().external_taint.is_empty()` — documenting that `external_taint` is runtime/process-local eligibility state, never repository-durable mutation-trace state. (2) `assert_round_trip` previously compared only the worktree state plus one explicitly supplied scope/`EventKey`, so it could not prove the bounded `WorktreeProjection.scopes` map — which can hold multiple `Active` scopes — round-trips exactly; the contended-mutation test's two active scopes were only checked via `MutationEvent.active_scopes` (a separate, historical table), not via the live bounded projection `load_worktree` actually returns. Added a new `expected_projection(after, worktree, scope, event_key) -> WorktreeProjection` test helper that reconstructs the exact bounded projection T03's hot-load contract defines — every `Active` scope belonging to the target worktree, plus the effective referenced scope (`scope`, or `event_key.scope_id` when `scope` is absent) regardless of status, and `processed_events` containing exactly `{event_key}` when supplied and already recorded, empty otherwise — and rewrote `assert_round_trip` to assert full `WorktreeProjection` equality (`reloaded == expected_projection(...)`, using the type's existing `PartialEq` derive) instead of the previous partial field checks. This is strictly stronger and made the old per-test `reloaded.scopes.get(&scope_id) == Some(status)` spot-checks in the `Close` and `Abandon` tests redundant, so they were removed; the `round_trip_contended_mutation_...` test additionally asserts `reloaded.scopes`' key set is exactly `{scope-a, scope-b}` with both `Active`, making the live-projection guarantee explicit alongside the unchanged historical `MutationEvent.active_scopes` check. Terminal-referenced-scope behavior (`Close`/`Abandon`, where the referenced scope is no longer `Active` but must still be returned) and replay behavior (`processed_events` containing exactly the replayed `EventKey`, not the full historical set) both continue to pass unchanged under the new helper, since `expected_projection` implements the same T03 four-case effective-scope rule the production `load_worktree` does. No genuine T01-T08 gap surfaced; both fixes are test-only, confined to `store.rs`'s `#[cfg(test)] mod tests` (0 production lines changed). T10 was not started. + - Verify (T09 correction): `./scripts/run-cli-cargo.sh test --manifest-path cli/Cargo.toml mutation_trace::store::` — passed, 73/73 (72 pre-existing + 1 new); `./scripts/run-cli-cargo.sh test --manifest-path cli/Cargo.toml mutation_trace::` — passed, 161/161; `./scripts/run-cli-cargo.sh fmt --manifest-path cli/Cargo.toml -- --check` — passed, no diff; `./scripts/run-cli-cargo.sh clippy --manifest-path cli/Cargo.toml --all-targets -- -D warnings` — passed, zero warnings. + - Done checks (T09 correction): `database_failure` now has a real-DB round-trip/non-persistence test proving `external_taint` changes in memory but produces no `DurableTransition` and does not survive DB reopening (verified by the new test); `assert_round_trip` now compares the exact bounded `WorktreeProjection` (worktree state, scopes, processed events) rather than partial fields, derived per T03's hot-load contract (verified by `expected_projection` and its use in every `round_trip_*` test, all still passing); the contended test now proves both the live bounded scope projection and the historical `MutationEvent.active_scopes` round-trip independently (verified by the new key-set/status assertions plus the pre-existing mutation-event equality check); terminal-referenced-scope inclusion (`Close`/`Abandon`) and bounded (not global) replay `processed_events` both remain correctly asserted under the strengthened helper (verified — all pre-existing `round_trip_*` tests pass unchanged in behavior, only their assertions got exact rather than partial); no production file changed (verified — `git diff --stat` touches only `cli/src/services/mutation_trace/store.rs`, entirely within `#[cfg(test)] mod tests`); T10 was not started (verified — no changes to any file outside `store.rs`/this plan). + - Context impact: local — the new test, the new `expected_projection` helper, and the strengthened `assert_round_trip` remain confined to `store.rs`'s existing test module, exercising the same already-documented `MutationTraceStore`/`DurableTransition`/`protocol::*` surface as the original T09 tests; no production code changed, no new call site, no claim in any root context file is affected. + - Context synchronization: synced - [ ] T10: `Add migration and lifecycle tests for migration 003` (status:todo) - Task ID: T10 From 6f51885ab32cdc0c3d3479e8516095cbd6e82cc8 Mon Sep 17 00:00:00 2001 From: David Abram Date: Sat, 29 Aug 2026 12:50:51 +0200 Subject: [PATCH 13/15] tests: Add migration lifecycle coverage for mutation trace protocol Ensure existing databases with migrations 001 and 002 apply migration 003 through setup while hook-runtime opens don't apply it and retain setup guidance. Record the migration lifecycle coverage and verification for T10. Plan: mutation-cursor-store-persistence (T10) Co-authored-by: SCE --- cli/src/services/agent_trace_db/repository.rs | 201 ++++++++++++++++-- .../mutation-cursor-store-persistence.md | 15 +- 2 files changed, 199 insertions(+), 17 deletions(-) diff --git a/cli/src/services/agent_trace_db/repository.rs b/cli/src/services/agent_trace_db/repository.rs index 75ece484..205366d7 100644 --- a/cli/src/services/agent_trace_db/repository.rs +++ b/cli/src/services/agent_trace_db/repository.rs @@ -1,17 +1,3 @@ -//! Repository-scoped Agent Trace database adapter. -//! -//! One logical Git repository maps to one database at -//! `/sce/repos//agent-trace.db`. The schema -//! baseline is one fresh schema SQL file (`agent-trace-repository` -//! migrations), because repository-scoped databases are always created new; -//! there is no incremental chain and no migration path from legacy -//! checkout-scoped databases. Trace tables carry no `checkout_id` columns. -//! -//! `repository_metadata` additionally carries `source_instance_id`, a -//! physical-database identity independent of the logical `repository_id` -//! (added by the additive `002_repository_source_instance_id` migration and -//! generated by application code, never SQL). - use std::path::PathBuf; use anyhow::Result; @@ -377,7 +363,7 @@ mod tests { } #[test] - fn open_at_initializes_the_full_schema_from_one_migration() { + fn open_at_initializes_the_full_repository_schema() { let db_path = unique_test_db_path("baseline"); let db = RepositoryAgentTraceDb::new_at(&db_path).expect("repository DB should open"); @@ -796,6 +782,191 @@ mod tests { remove_test_db(&db_path); } + fn seed_001_and_002_only_fixture(db_path: &std::path::Path, repository_id: &str) { + let fixture = RepositoryAgentTraceDb::open_without_migrations_at(db_path) + .expect("001+002-only fixture DB should open"); + fixture + .execute( + "CREATE TABLE IF NOT EXISTS __sce_migrations ( + id TEXT PRIMARY KEY, + applied_at TEXT NOT NULL DEFAULT (strftime('%Y-%m-%dT%H:%M:%fZ', 'now')) +)", + (), + ) + .expect("migration metadata table should create"); + fixture + .execute( + "CREATE TABLE IF NOT EXISTS repository_metadata ( + id INTEGER PRIMARY KEY CHECK (id = 1), + repository_id TEXT NOT NULL, + source_instance_id TEXT NOT NULL DEFAULT '', + created_at TEXT NOT NULL DEFAULT (strftime('%Y-%m-%dT%H:%M:%fZ', 'now')) +)", + (), + ) + .expect("post-002 repository_metadata table should create"); + for (table, ddl) in [ + ( + "diff_traces", + "CREATE TABLE IF NOT EXISTS diff_traces (id INTEGER PRIMARY KEY)", + ), + ( + "post_commit_patch_intersections", + "CREATE TABLE IF NOT EXISTS post_commit_patch_intersections (id INTEGER PRIMARY KEY)", + ), + ( + "agent_traces", + "CREATE TABLE IF NOT EXISTS agent_traces (id INTEGER PRIMARY KEY)", + ), + ( + "messages", + "CREATE TABLE IF NOT EXISTS messages (id INTEGER PRIMARY KEY)", + ), + ( + "parts", + "CREATE TABLE IF NOT EXISTS parts (id INTEGER PRIMARY KEY)", + ), + ] { + fixture + .execute(ddl, ()) + .unwrap_or_else(|error| panic!("{table} table should create: {error}")); + } + fixture + .execute( + "INSERT INTO __sce_migrations (id) VALUES ('001_repository_schema')", + (), + ) + .expect("001 migration record should insert"); + fixture + .execute( + "INSERT INTO __sce_migrations (id) VALUES ('002_repository_source_instance_id')", + (), + ) + .expect("002 migration record should insert"); + fixture + .execute( + "INSERT INTO repository_metadata (id, repository_id) VALUES (1, ?1)", + (repository_id,), + ) + .expect("repository_metadata row should seed"); + drop(fixture); + } + + #[test] + fn baseline_and_source_instance_fixture_migrates_to_mutation_trace_protocol_through_setup() { + let db_path = unique_test_db_path("baseline-and-source-instance-fixture"); + let repository_id = "c".repeat(64); + + seed_001_and_002_only_fixture(&db_path, &repository_id); + + let migrated = RepositoryAgentTraceDb::new_at(&db_path) + .expect("repository DB should migrate a 001+002-only fixture to 003"); + + let applied_ids = migrated + .query_map( + "SELECT id FROM __sce_migrations ORDER BY id ASC", + (), + |row| row.get::(0).map_err(Into::into), + ) + .expect("migration metadata query should succeed"); + assert_eq!( + applied_ids, + vec![ + String::from("001_repository_schema"), + String::from("002_repository_source_instance_id"), + String::from("003_mutation_trace_protocol"), + ], + "an existing 001+002 database should get 003 applied on top through the setup/lifecycle path, without reapplying 001/002" + ); + + for table in [ + "mutation_trace_worktrees", + "mutation_trace_scopes", + "mutation_trace_processed_events", + "mutation_trace_events", + "mutation_trace_event_active_scopes", + ] { + assert!( + sqlite_object_exists(&migrated, "table", table), + "table '{table}' should exist after migrating a 001+002-only fixture" + ); + } + + migrated + .ensure_schema_ready_for_hooks() + .expect("migrated repository DB schema should be ready for hooks"); + + let metadata = migrated + .verify_or_initialize_repository_metadata(&repository_id) + .expect("metadata initialization on a migrated 001+002-only fixture should succeed"); + assert_eq!(metadata.repository_id, repository_id); + + remove_test_db(&db_path); + } + + #[test] + fn hook_runtime_path_never_applies_mutation_trace_protocol_migration() { + let db_path = unique_test_db_path("hook-runtime-no-migration"); + let repository_id = "d".repeat(64); + + seed_001_and_002_only_fixture(&db_path, &repository_id); + + let db = RepositoryAgentTraceDb::open_for_hooks_without_migrations_at(&db_path) + .expect("hook-runtime open without migrations should succeed on an existing DB file"); + + let readiness_error = db.ensure_schema_ready_for_hooks().expect_err( + "a 001+002-only DB should not be schema-ready for the mutation-trace store", + ); + assert!( + readiness_error + .to_string() + .contains(REPOSITORY_AGENT_TRACE_SCHEMA_SETUP_GUIDANCE), + "unexpected error: {readiness_error}" + ); + + let repair_error = db + .repair_missing_repository_schema_migration_metadata() + .expect_err("the base-table repair path must not silently mark 003 as applied"); + assert!( + repair_error + .to_string() + .contains(REPOSITORY_AGENT_TRACE_SCHEMA_SETUP_GUIDANCE), + "unexpected error: {repair_error}" + ); + + let applied_ids = db + .query_map( + "SELECT id FROM __sce_migrations ORDER BY id ASC", + (), + |row| row.get::(0).map_err(Into::into), + ) + .expect("migration metadata query should succeed"); + assert_eq!( + applied_ids, + vec![ + String::from("001_repository_schema"), + String::from("002_repository_source_instance_id"), + ], + "the no-migration hook-runtime path must never record or apply 003" + ); + + for table in [ + "mutation_trace_worktrees", + "mutation_trace_scopes", + "mutation_trace_processed_events", + "mutation_trace_events", + "mutation_trace_event_active_scopes", + ] { + assert!( + !sqlite_object_exists(&db, "table", table), + "table '{table}' should not exist; the hook-runtime path must not create mutation-trace tables" + ); + } + + drop(db); + remove_test_db(&db_path); + } + #[test] fn repository_scoped_write_methods_insert_all_agent_trace_rows() { let db_path = unique_test_db_path("writes"); diff --git a/context/plans/mutation-cursor-store-persistence.md b/context/plans/mutation-cursor-store-persistence.md index 432bc69a..dd48945b 100644 --- a/context/plans/mutation-cursor-store-persistence.md +++ b/context/plans/mutation-cursor-store-persistence.md @@ -295,13 +295,24 @@ Persist this field in every plan; this is durable plan state, not chat state: - Context impact: local — the new test, the new `expected_projection` helper, and the strengthened `assert_round_trip` remain confined to `store.rs`'s existing test module, exercising the same already-documented `MutationTraceStore`/`DurableTransition`/`protocol::*` surface as the original T09 tests; no production code changed, no new call site, no claim in any root context file is affected. - Context synchronization: synced -- [ ] T10: `Add migration and lifecycle tests for migration 003` (status:todo) +- [x] T10: `Add migration and lifecycle tests for migration 003` (status:done) - Task ID: T10 - Scope: In — tests proving a fresh DB applies `001`+`002`+`003`; an existing `001`+`002`-only DB gets `003` applied through the `sce setup`/lifecycle path; the no-migration hook-runtime path does not apply `003` and still reports the existing "Run 'sce setup'." guidance when schema is incomplete. Out — changes to `REQUIRED_REPOSITORY_SCHEMA_TABLES` baseline-repair semantics. - Dependencies: T01 - Done when: all three scenarios pass without modifying the baseline-repair function's treatment of `001` metadata. - Verify: `./scripts/run-cli-cargo.sh test --manifest-path cli/Cargo.toml agent_trace_db::` - - Context synchronization: pending + - Completed: 2026-08-29 + - Files changed: `cli/src/services/agent_trace_db/repository.rs` + - Result: Scenario 1 (fresh DB applies `001`+`002`+`003`) was already fully covered by the pre-existing `open_at_initializes_the_full_schema_from_one_migration` test (asserts all five mutation-trace tables/indexes exist and `__sce_migrations` lists all three IDs in order), so no duplicate test was added for it. Added a `seed_001_and_002_only_fixture` test helper that hand-builds a post-002 `repository_metadata` shape (`source_instance_id` column present) plus the other five `REQUIRED_REPOSITORY_SCHEMA_TABLES`, and records `001_repository_schema`/`002_repository_source_instance_id` in `__sce_migrations` — mirroring the existing `baseline_only_fixture_migrates_and_gets_a_stable_source_instance_id` pattern one migration further along. `baseline_and_source_instance_fixture_migrates_to_mutation_trace_protocol_through_setup` opens that fixture via `RepositoryAgentTraceDb::new_at` (the `sce setup`/lifecycle path) and asserts `003` is applied on top without reapplying `001`/`002` (`__sce_migrations` shows all three IDs, all five mutation-trace tables exist, `ensure_schema_ready_for_hooks` succeeds, and `verify_or_initialize_repository_metadata` still works). `hook_runtime_path_never_applies_mutation_trace_protocol_migration` opens the same fixture via `RepositoryAgentTraceDb::open_for_hooks_without_migrations_at`, asserts `ensure_schema_ready_for_hooks` fails with the `"Run 'sce setup'."` guidance, then mirrors the real hook-runtime caller's fallback by calling `repair_missing_repository_schema_migration_metadata` directly and asserting it also fails with the same guidance (all six `REQUIRED_REPOSITORY_SCHEMA_TABLES` already exist, so the failure comes from `003` still being unrecorded, not a missing base table) — and asserts `__sce_migrations` still shows only `001`/`002` and no mutation-trace table was created. No production code was changed; `REQUIRED_REPOSITORY_SCHEMA_TABLES`'s baseline-repair semantics are untouched. + - Verify: `./scripts/run-cli-cargo.sh test --manifest-path cli/Cargo.toml agent_trace_db::` — passed, 26/26 (24 pre-existing + 2 new); `./scripts/run-cli-cargo.sh fmt --manifest-path cli/Cargo.toml -- --check` — passed after one auto-formatting pass; `./scripts/run-cli-cargo.sh clippy --manifest-path cli/Cargo.toml --all-targets -- -D warnings` — passed, zero warnings. + - Done checks: a fresh DB applies `001`+`002`+`003` (verified by the pre-existing `open_at_initializes_the_full_repository_schema` test, unchanged); an existing `001`+`002`-only DB gets `003` applied through the `sce setup`/lifecycle path (verified by `baseline_and_source_instance_fixture_migrates_to_mutation_trace_protocol_through_setup`); the no-migration hook-runtime path does not apply `003` and still reports the `"Run 'sce setup'."` guidance when schema is incomplete (verified by `hook_runtime_path_never_applies_mutation_trace_protocol_migration`, covering both `ensure_schema_ready_for_hooks` and the `repair_missing_repository_schema_migration_metadata` fallback the real hook-runtime caller invokes); `REQUIRED_REPOSITORY_SCHEMA_TABLES`'s baseline-repair semantics were not modified (verified — `git diff --stat` touches only `repository.rs`, entirely within `#[cfg(test)] mod tests`, and `repair_missing_repository_schema_migration_metadata`'s body is unchanged). + - Context impact: local — the new fixture helper and two tests exercise only the already-documented `RepositoryAgentTraceDb::new_at`/`open_for_hooks_without_migrations_at`/`ensure_schema_ready_for_hooks`/`repair_missing_repository_schema_migration_metadata` surface; no production code changed, no new call site, no claim in any root context file is affected. Deferred to T11/plan-level context sync per the plan's existing assumption. + - Context synchronization: synced + - **T10 cleanup (2026-08-29):** `repository.rs`'s top-of-file module doc comment still claimed the repository-scoped schema is "one fresh schema SQL file" with "no incremental chain," which became false once `002_repository_source_instance_id` and `003_mutation_trace_protocol` landed as additive migrations on top of `001`. Deleted that stale module comment rather than rewriting it, per this repository's no-source-comments policy. Renamed the test `open_at_initializes_the_full_schema_from_one_migration` to `open_at_initializes_the_full_repository_schema`, since it now asserts three applied migrations (`001`+`002`+`003`), not one; its assertions (repository tables, mutation-trace tables, indexes, triggers, `__sce_migrations` order, `ensure_schema_ready_for_hooks` success) were not changed. A repo-wide grep for `one migration`/`one fresh schema`/`no incremental chain`/`single migration` across `cli/src/services/agent_trace_db` found no other stale instances. `REQUIRED_REPOSITORY_SCHEMA_TABLES` and `repair_missing_repository_schema_migration_metadata` were not touched — the latter's own "one-file schema batch" doc comment still accurately describes `001`'s single-batch application and was left as-is. No migration lifecycle behavior changed. T11 was not started. + - Verify (T10 cleanup): `./scripts/run-cli-cargo.sh test --manifest-path cli/Cargo.toml agent_trace_db::` — passed, 26/26 (test count unchanged, one renamed); `./scripts/run-cli-cargo.sh fmt --manifest-path cli/Cargo.toml -- --check` — passed, no diff; `./scripts/run-cli-cargo.sh clippy --manifest-path cli/Cargo.toml --all-targets -- -D warnings` — passed, zero warnings. + - Done checks (T10 cleanup): the stale module doc comment is deleted, not rewritten (verified by inspection — `git diff` shows only a deletion at the top of the file); the renamed test still verifies repository tables, mutation-trace tables, indexes, triggers, `__sce_migrations == [001, 002, 003]`, and `ensure_schema_ready_for_hooks` success (verified — test body unchanged, only its name changed); no other stale "one migration"/"single migration"/"no incremental chain"/"one fresh schema" wording remains in `cli/src/services/agent_trace_db` (verified by grep); `REQUIRED_REPOSITORY_SCHEMA_TABLES` and `repair_missing_repository_schema_migration_metadata` are byte-unchanged (verified by `git diff`); T11 was not started (verified — no changes to any file outside `repository.rs`/this plan, and `context/cli/mutation-trace-store.md` was not created). + - Context impact: local — a stale doc comment and a stale test name, both confined to `repository.rs`, are corrected; no behavior, migration semantics, or repair logic changed; no claim in any root context file is affected. + - Context synchronization: synced - [ ] T11: `Document the mutation-trace store` (status:todo) - Task ID: T11 From ed2317ea3caac4bf3a23e50e84d9cd9bcbc1f073 Mon Sep 17 00:00:00 2001 From: David Abram Date: Sat, 29 Aug 2026 13:15:57 +0200 Subject: [PATCH 14/15] context: Update mutation-trace persistence context Record the completed mutation-cursor store persistence work so the repository's context reflects the durable protocol boundary, transactional CAS behavior, revision encoding, and bounded read paths. Capture the store design, shared Turso transactional primitive, completed acceptance criteria, and validation evidence, including the remaining test-isolation follow-up. Ref: context/plans/mutation-cursor-store-persistence.md (T11, AC1-AC15) Co-authored-by: SCE --- cli/src/services/mutation_trace/store.rs | 13 +- context/cli/mutation-trace-store.md | 128 ++++++++++++++++++ context/context-map.md | 1 + .../mutation-cursor-store-persistence.md | 82 ++++++++--- context/sce/shared-turso-db.md | 35 +++++ 5 files changed, 234 insertions(+), 25 deletions(-) create mode 100644 context/cli/mutation-trace-store.md diff --git a/cli/src/services/mutation_trace/store.rs b/cli/src/services/mutation_trace/store.rs index 946fcb8f..68953e15 100644 --- a/cli/src/services/mutation_trace/store.rs +++ b/cli/src/services/mutation_trace/store.rs @@ -455,9 +455,6 @@ fn diff_new_mutation_event( Ok(Some((*event).clone())) } -/// Bounded read access to the durable mutation-cursor protocol state for one -/// repository, via [`RepositoryAgentTraceDb`]. Write/CAS-commit access is -/// added by later tasks (T04/T06/T07). pub struct MutationTraceStore<'a> { db: &'a RepositoryAgentTraceDb, } @@ -1002,6 +999,7 @@ fn reconstruct_boundary( #[cfg(test)] mod tests { + use std::sync::atomic::{AtomicU64, Ordering}; use std::thread; use super::*; @@ -1124,14 +1122,13 @@ mod tests { assert!(decode_boundary_kind("unknown").is_err()); } + static NEXT_TEST_DB_ID: AtomicU64 = AtomicU64::new(0); + fn unique_test_db_path(label: &str) -> std::path::PathBuf { - let nonce = std::time::SystemTime::now() - .duration_since(std::time::UNIX_EPOCH) - .expect("system time should be after Unix epoch") - .as_nanos(); + let id = NEXT_TEST_DB_ID.fetch_add(1, Ordering::Relaxed); std::env::temp_dir() .join(format!( - "sce-mutation-trace-store-{label}-{}-{nonce}", + "sce-mutation-trace-store-{label}-{}-{id}", std::process::id() )) .join("agent-trace.db") diff --git a/context/cli/mutation-trace-store.md b/context/cli/mutation-trace-store.md new file mode 100644 index 00000000..94328f39 --- /dev/null +++ b/context/cli/mutation-trace-store.md @@ -0,0 +1,128 @@ +# Mutation-trace store (`mutation_trace::store`) + +Durable persistence for the verified mutation-cursor protocol +([`protocol.rs`](mutation-trace-protocol.md)), built by the +`mutation-cursor-store-persistence` plan. `store.rs` is the protocol's first +real database call site: it stores worktree/scope/processed-event/mutation-event +state in the repository-scoped Agent Trace DB (`RepositoryAgentTraceDb`) via +migration `003_mutation_trace_protocol.sql`. + +## Boundary shape + +```mermaid +flowchart LR + protocol["protocol.rs\n(pure prepare/commit/taint/\nabandon/recover)"] + diff["DurableTransition::between\n(pure structural diff of\nbefore/after ProtocolState)"] + store["MutationTraceStore\n(SQL translation)"] + db["RepositoryAgentTraceDb\n(TursoDb)"] + + protocol -->|before, after| diff --> store --> db +``` + +The boundary is one-directional and structural. `protocol.rs` never depends +on SQL or `RepositoryAgentTraceDb`. `DurableTransition::between` diffs two +`ProtocolState` values field-by-field — it never branches on `Boundary`, +`BoundaryKind`, `Attribution`, or taint state, so it cannot make a persistence +decision based on protocol meaning. `store.rs` never interprets protocol +semantics either: it only translates an already-validated `DurableTransition` +into SQL statements and reconstructs domain values out of query rows. + +## What's persisted, and what isn't + +Five tables (`mutation_trace_worktrees`, `mutation_trace_scopes`, +`mutation_trace_processed_events`, `mutation_trace_events`, +`mutation_trace_event_active_scopes`) hold `WorktreeState`, `ScopeState`, +`EventKey` replay identity, and historical `MutationEvent`s. + +Two `ProtocolState` fields are deliberately never persisted: + +- `AttemptState` — explicitly transient in the domain model; no + `mutation_trace_attempts` table exists. +- `external_taint` — a `database_failure()` cannot use the database it just + failed against as the authoritative record that the write was uncertain. + `DurableTransition::between` returns `Ok(None)` for a `database_failure`-only + transition, so `store.commit` is never even called for it. + +`revision` (on both worktree and event rows) is stored as an 8-byte +big-endian `BLOB`, via `encode_revision`/`decode_revision`, never a SQLite +`INTEGER` — every revision column carries +`CHECK (typeof(revision) = 'blob' AND length(revision) = 8)`, so this survives +`u64::MAX` exactly and rejects a same-length `TEXT` value. Every other +enum-shaped column (`ActorKind`, `FailureKind`, `ScopeStatus`, and the +`AttributionKind`/`BoundaryKind` discriminants derived from `Attribution` and +`Boundary`) has an explicit `encode_*`/`decode_*` function pair — none derives +from `Debug` or a serde representation. + +## Read path + +`MutationTraceStore::load_worktree(worktree, scope, event_key)` is the hot +path: it loads one worktree row, only that worktree's currently `Active` +scopes, plus one optional *effective referenced scope* (from `scope` or +`event_key.scope_id` — the two must agree when both are given), included +regardless of status. It never queries `mutation_trace_events`. A missing +effective scope, a `scope`/`event_key.scope_id` disagreement, or an effective +scope belonging to a different worktree all return `Err` rather than silently +loading, omitting, or reassigning it. The result, `WorktreeProjection`, +widens into a full `ProtocolState` via `into_protocol_state` (with `attempts`, +`mutation_events`, and `external_taint` always empty) so unmodified +`protocol.rs` functions can operate on it. + +`MutationTraceStore::load_mutation_event(worktree, revision)` is the separate +cold path: it reconstructs one historical `MutationEvent`, including full +`Attribution`/`Boundary` decoding, by `(worktree_id, revision)`. It is never +called from `load_worktree` or from any hook-boundary path, so a +projection load never pays for the full historical event set. + +## Write path + +`MutationTraceStore::commit(transition: &DurableTransition) -> Result` +translates the transition into one worktree CAS `UPDATE`, zero or more scope +status `UPDATE`s, an optional processed-event `INSERT`, and an optional +mutation-event `INSERT` plus its active-scope `INSERT`s — all run through +`TursoDb::execute_transactional_cas_batch` inside exactly one +`BEGIN IMMEDIATE` transaction. The worktree `UPDATE`'s `WHERE worktree_id = ? +AND revision = ?` clause is the CAS guard: `execute_transactional_cas_batch` +treats its affected-row count as the outcome (`0` rows → no-op commit, +`CasResult::Conflict`; `1` row → every other statement runs, +`CasResult::Applied`). Every non-guard statement also carries +`expect_rows_affected(1)`, which fails the transaction deterministically on an +unexpected affected-row count. A `(scope_id, event_id)` replay is a distinct +failure path: the processed-event `INSERT`'s `PRIMARY KEY` constraint rejects +it as a SQL error before any row-count check applies. Both paths roll back +the transaction and propagate out of `commit()` as `Err`, never as +`CasResult::Conflict`, and neither is retried unless the underlying error is +`Busy`/`BusySnapshot`. + +`execute_transactional_cas_batch` keeps three outcomes distinct: a stale +revision is a `Conflict` that is never retried; a transient DB failure +(`Busy`/`BusySnapshot`) retries the whole transaction from a fresh +`BEGIN IMMEDIATE`; and any other deterministic SQL/constraint failure +propagates out of `commit()` as `Err`, never as `CasResult::Conflict`. + +`DurableTransition`'s six fields are private — `between()` is the only way to +construct one outside `store.rs`, so `commit()` can trust the structural +invariants `between()` already proved (single worktree, revision advances by +exactly one, at most one new processed/mutation event) without re-validating +them. + +## Initialization + +`initialize_worktree`/`register_scope` are idempotent idle-inserts (`INSERT +... ON CONFLICT DO NOTHING`) outside the CAS commit path: `initialize_worktree` +never overwrites an existing cursor, and `register_scope` requires the +referenced worktree to already have a durable row, returning `Err` rather than +auto-creating it or accepting a scope whose stored `worktree_id`/`actor_kind` +merely happen to match the request. + +## Non-goals + +- No Git or filesystem I/O — no `coordinator.rs` or `git_snapshot.rs` exists + yet; those remain future work building on this store. +- No attribution or boundary-kind decisions — `DurableTransition::between` + and `store.rs` are both structurally blind to protocol meaning. +- No retry-after-`Conflict` loop — `CasResult::Conflict` is returned to the + caller; retrying with a freshly reloaded revision is a future adapter's + responsibility, not this module's. +- No deletion of terminal (`Closed`/`Abandoned`) scope rows or historical + `mutation_trace_events` rows — scope garbage collection is out of scope for + this plan. diff --git a/context/context-map.md b/context/context-map.md index d01bf66f..4c45a44e 100644 --- a/context/context-map.md +++ b/context/context-map.md @@ -26,6 +26,7 @@ Feature/domain context: - `context/cli/mutation-trace-protocol.md` (pure, dependency-free `cli/src/services/mutation_trace/` refinement of the verified `spec/mutation_cursor.qnt` mutation-cursor protocol: the full pure action set — `prepare`/`commit` transition logic, attribution/mutation-event materialization, snapshot-failure/database-failure taint actions, scope abandonment, and recovery with an explicit observed-tree input — plus cross-action sequence/invariant tests and a module-level Quint refinement matrix are all complete (`types.rs`, `protocol.rs`, `mod.rs`, `tests.rs`, registered with `#[allow(dead_code)]`); opaque-newtype identity refinement decisions vs. the Quint model's bounded enums, and the `coordinator.rs`/`git_snapshot.rs` target end-state seams this layout leaves room for but does not create — `store.rs` now exists, built out by the `mutation-cursor-store-persistence` plan; `protocol.rs`'s pure transitions are not yet wired into any hook or command) - `context/cli/mutation-trace-revision-refinement.md` (the Quint `revision: int` → Rust `WorktreeState::revision: u64` bounded-integer refinement: the private `next_revision` checked-arithmetic helper `commit`/`taint`/`abandon`/`recover` all route through instead of a raw `+ 1`, so a worktree at `revision: u64::MAX` is a guarded no-op/rejection rather than a silent wrap to `0`) - `context/cli/mutation-trace-quint-connect.md` (`#[cfg(test)]`-only Quint Connect model-based-testing harness in `cli/src/services/mutation_trace/mbt/` continuously checking `protocol.rs` against `spec/mutation_cursor.qnt`: the verification-only `mbtAction`/`MbtAction` record-payload transport excluded from comparison, the operation-identity-vs-`MbtStutter` distinction on guarded/no-op branches with its two deterministic regressions, finite ID mapping, the AC5 comparable-state field list, `randomPrepare` staying a single `step` branch, deterministic/generated (500×30, seed-reproducible) test coverage, and the two Nix checks — generic `checks.cli-tests` and dedicated `checks.mutation-trace-quint-connect` — that both require the pinned Quint binary plus the top-level `spec/` directory in `workspaceSrc`'s Nix fileset) +- `context/cli/mutation-trace-store.md` (durable persistence for the mutation-cursor protocol in `cli/src/services/mutation_trace/store.rs`, built by the `mutation-cursor-store-persistence` plan: the one-directional `protocol.rs` -> `DurableTransition::between` (pure structural diff) -> `store.rs` (SQL translation) -> `RepositoryAgentTraceDb` boundary; migration `003_mutation_trace_protocol.sql`'s five tables; `AttemptState`/`external_taint` non-persistence; the 8-byte big-endian `BLOB` revision encoding plus explicit non-`Debug` enum codecs; the bounded hot-path `load_worktree` read vs. the cold-path `load_mutation_event`; `commit`'s single-`BEGIN IMMEDIATE` CAS batch via `TursoDb::execute_transactional_cas_batch`, distinguishing `Conflict`/retryable-transient/deterministic-`Err` outcomes; and the store's non-goals — no Git/filesystem I/O, no attribution/boundary-kind decisions, no retry-after-`Conflict` loop, no terminal-scope garbage collection) - `context/sce/cli-exit-code-contract.md` (stable class-based `sce` process exit-code contract in `cli/src/app.rs`, so automation can branch on failure category without parsing error text) - `context/sce/cli-error-code-taxonomy.md` (stable user-facing `SCE-ERR-*` diagnostic code classes rendered by `cli/src/app.rs`, complementing the numeric exit-code classes) - `context/sce/cli-stdout-stderr-contract.md` (implemented stream contract in `cli/src/app.rs`: command payloads on stdout only, redacted diagnostics on stderr) diff --git a/context/plans/mutation-cursor-store-persistence.md b/context/plans/mutation-cursor-store-persistence.md index dd48945b..faea1f3d 100644 --- a/context/plans/mutation-cursor-store-persistence.md +++ b/context/plans/mutation-cursor-store-persistence.md @@ -46,35 +46,35 @@ error without retry. ## Acceptance criteria -- [ ] AC1: Mutation state lives in the repository-scoped `agent-trace.db`. +- [x] AC1: Mutation state lives in the repository-scoped `agent-trace.db`. - Validate: `cli/src/services/mutation_trace/store.rs` reads/writes only through `RepositoryAgentTraceDb`; round-trip tests in T09 pass. -- [ ] AC2: New storage is introduced through additive migration `003`, with `001`/`002` byte-unchanged by this PR. +- [x] AC2: New storage is introduced through additive migration `003`, with `001`/`002` byte-unchanged by this PR. - Validate: `git diff --exit-code ...HEAD -- cli/migrations/agent-trace-repository/001_repository_schema.sql cli/migrations/agent-trace-repository/002_repository_source_instance_id.sql` (compared against this PR's base branch/merge base, not the working tree) exits `0`; `003_mutation_trace_protocol.sql` exists. -- [ ] AC3: Revision preserves all `u64` values exactly, including `u64::MAX`. +- [x] AC3: Revision preserves all `u64` values exactly, including `u64::MAX`. - Validate: `./scripts/run-cli-cargo.sh test --manifest-path cli/Cargo.toml mutation_trace::store::` (revision codec round-trip test covering `0`, `1`, `i64::MAX`, `i64::MAX + 1`, `u64::MAX`). -- [ ] AC4: Worktree/scope/`EventKey`/`MutationEvent` data round-trip exactly, including full `MutationEvent` decoding (`Attribution`, `Boundary`, `active_scopes`) after the DB is closed and reopened. +- [x] AC4: Worktree/scope/`EventKey`/`MutationEvent` data round-trip exactly, including full `MutationEvent` decoding (`Attribution`, `Boundary`, `active_scopes`) after the DB is closed and reopened. - Validate: T09's real-protocol round-trip tests, including the `load_mutation_event` cold-reload assertions for every transition that emits a `MutationEvent`. -- [ ] AC5: `AttemptState` is never persisted. +- [x] AC5: `AttemptState` is never persisted. - Validate: `grep -n mutation_trace_attempts cli/migrations/agent-trace-repository/003_mutation_trace_protocol.sql` finds nothing; `DurableTransition` has no `AttemptState` field. -- [ ] AC6: `external_taint` is never treated as DB-authoritative durable state. +- [x] AC6: `external_taint` is never treated as DB-authoritative durable state. - Validate: `grep -n external_taint cli/migrations/agent-trace-repository/003_mutation_trace_protocol.sql` finds nothing; `database_failure` produces no `DurableTransition` (T05 test). -- [ ] AC7: No persistence code determines protocol semantics or attribution. +- [x] AC7: No persistence code determines protocol semantics or attribution. - Validate: `DurableTransition::between` contains no boundary-kind/contention/taint conditionals (T05 done-when); inspection of `store.rs`. -- [ ] AC8: Every durable protocol transition is one `BEGIN IMMEDIATE` transaction. +- [x] AC8: Every durable protocol transition is one `BEGIN IMMEDIATE` transaction. - Validate: `store.commit` routes exclusively through `execute_transactional_cas_batch` (T06/T07); T08 atomic-rollback test. -- [ ] AC9: CAS is guarded by the expected worktree revision. +- [x] AC9: CAS is guarded by the expected worktree revision. - Validate: the guard statement is `UPDATE mutation_trace_worktrees ... WHERE worktree_id = ? AND revision = ?` (T06); T08 two-writer test. -- [ ] AC10: Two writers from one revision cannot both commit. +- [x] AC10: Two writers from one revision cannot both commit. - Validate: T08's concurrent-writers test — two independent `RepositoryAgentTraceDb` handles/connections against the same physical database, committing concurrently from the same loaded revision — asserts exactly one `Applied` and one `Conflict`. -- [ ] AC11: Partial failure rolls back all worktree/scope/event changes. +- [x] AC11: Partial failure rolls back all worktree/scope/event changes. - Validate: T08 injected-failure test asserts revision, scope status, processed event, mutation event, and active scopes are all unchanged after rollback. -- [ ] AC12: Process restart reconstructs the same durable protocol projection. +- [x] AC12: Process restart reconstructs the same durable protocol projection. - Validate: T09 tests that drop and reopen the DB handle before reloading. -- [ ] AC13: Historical mutation events are not loaded on each boundary; terminal (`Closed`/`Abandoned`/`NeverSeen`) historical scopes are not loaded on each boundary unless they are the effective referenced scope (`scope`, or `event_key.scope_id` when `scope` is absent); explicit `scope` and `event_key.scope_id` must agree when both are supplied, or `load_worktree` returns `Err`; the effective referenced scope must belong to the requested worktree, or `load_worktree` returns `Err` rather than silently loading or reassigning it; and the effective referenced scope must exist in durable `mutation_trace_scopes` storage — a missing effective scope returns `Err`, rather than `load_worktree` silently continuing with a projection that omits it, whether the effective scope came from `scope` or `event_key.scope_id`. +- [x] AC13: Historical mutation events are not loaded on each boundary; terminal (`Closed`/`Abandoned`/`NeverSeen`) historical scopes are not loaded on each boundary unless they are the effective referenced scope (`scope`, or `event_key.scope_id` when `scope` is absent); explicit `scope` and `event_key.scope_id` must agree when both are supplied, or `load_worktree` returns `Err`; the effective referenced scope must belong to the requested worktree, or `load_worktree` returns `Err` rather than silently loading or reassigning it; and the effective referenced scope must exist in durable `mutation_trace_scopes` storage — a missing effective scope returns `Err`, rather than `load_worktree` silently continuing with a projection that omits it, whether the effective scope came from `scope` or `event_key.scope_id`. - Validate: `MutationTraceStore::load_worktree` issues no query against `mutation_trace_events`, loads only currently `Active` scopes plus the effective referenced scope (if any) derived from `scope`/`event_key` per T03's four-case definition, returns `Err` when `scope` and `event_key.scope_id` are both supplied and differ, returns `Err` when the effective referenced scope's persisted `worktree_id` does not match the requested worktree, and returns `Err` when the effective referenced scope has no durable `mutation_trace_scopes` row — including when an orphan `mutation_trace_processed_events` row exists for it (T03 done-when). -- [ ] AC14: Existing Quint Connect and protocol tests remain green. +- [x] AC14: Existing Quint Connect and protocol tests remain green. - Validate: `nix flake check` (runs `cli-tests`, including `mutation_trace::mbt`, and the dedicated `mutation-trace-quint-connect` check). -- [ ] AC15: No Git/filesystem lock/hook/coordinator integration is added. +- [x] AC15: No Git/filesystem lock/hook/coordinator integration is added. - Validate: no `coordinator.rs` or `git_snapshot.rs` file is created; `grep -RnE "std::(fs|process)|tokio::(fs|process)" cli/src/services/mutation_trace/` shows no non-test production usage. ### Full validation @@ -314,14 +314,62 @@ Persist this field in every plan; this is durable plan state, not chat state: - Context impact: local — a stale doc comment and a stale test name, both confined to `repository.rs`, are corrected; no behavior, migration semantics, or repair logic changed; no claim in any root context file is affected. - Context synchronization: synced -- [ ] T11: `Document the mutation-trace store` (status:todo) +- [x] T11: `Document the mutation-trace store` (status:done) - Task ID: T11 - Scope: In — `context/cli/mutation-trace-store.md` covering repository-DB ownership, `WorktreeId` as the persistence partition, the 8-byte big-endian revision encoding, `AttemptState`/`external_taint` non-persistence, and the store's non-goals (no Git I/O, no attribution decisions, no retry-after-`Conflict`); a `context/context-map.md` entry for the new file. Out — edits to any other existing `context/` file (left to task context synchronization). - Dependencies: T01-T10 - Done when: the new file exists, is linked from `context/context-map.md`, and every claim in it is checked against the code produced by T01-T10. - Verify: manual inspection cross-referencing the file's claims against `store.rs`, the migration, and `db/mod.rs`. - - Context synchronization: pending + - Completed: 2026-08-29 + - Files changed: `context/cli/mutation-trace-store.md` (new); `context/context-map.md` + - Result: Added `context/cli/mutation-trace-store.md`, covering: the one-directional `protocol.rs` -> `DurableTransition::between` (pure structural diff) -> `store.rs` (SQL translation) -> `RepositoryAgentTraceDb` boundary with a Mermaid diagram; what's persisted across migration `003`'s five tables vs. the deliberate `AttemptState`/`external_taint` exclusions; the 8-byte big-endian `BLOB` revision encoding and the explicit non-`Debug` enum codecs (`ActorKind`/`FailureKind`/`ScopeStatus`/`AttributionKind`/`BoundaryKind`); the bounded hot-path `load_worktree` (effective-referenced-scope rules, never queries `mutation_trace_events`) vs. the cold-path `load_mutation_event`; the `commit` write path's single-`BEGIN IMMEDIATE` CAS batch via `TursoDb::execute_transactional_cas_batch`, the guard's 0/1-row `Conflict`/`Applied` outcome, `expect_rows_affected(1)` on every other statement, and the `Conflict`/retryable-transient/deterministic-`Err` three-way distinction; `DurableTransition`'s private fields and `between()`-only construction; `initialize_worktree`/`register_scope`'s idle-insert semantics including the worktree-must-already-exist guard; and the store's non-goals (no Git/filesystem I/O, no attribution/boundary-kind decisions, no retry-after-`Conflict` loop, no terminal-scope garbage collection). Added one `context/context-map.md` entry for the new file, in the same style as the existing `mutation-trace-*` entries. + - Verify: manual inspection cross-referencing every claim in `context/cli/mutation-trace-store.md` against `cli/src/services/mutation_trace/store.rs` (codecs, `WorktreeProjection`, `DurableTransition`/`between`, `MutationTraceStore::{initialize_worktree, register_scope, load_worktree, load_mutation_event, commit}`), `cli/migrations/agent-trace-repository/003_mutation_trace_protocol.sql`, and `cli/src/services/db/mod.rs`'s `TransactionStatement`/`execute_transactional_cas_batch` — passed, no discrepancy found. + - Done checks: `context/cli/mutation-trace-store.md` exists (verified); it is linked from `context/context-map.md` (verified — new entry added alongside the other `mutation-trace-*` entries); every claim in it was checked against the code produced by T01-T10 (verified by the manual cross-reference above). + - Context impact: local — this task's own deliverable is a context file; no other root context file's claims are contradicted by it (`context/cli/mutation-trace-protocol.md`'s and `context/overview.md`'s "store.rs now exists" framing was already corrected by T07's context synchronization). `context/sce/shared-turso-db.md` still does not document `execute_transactional_cas_batch`; per the plan's assumption and T06's/T07's context-impact notes, that update belongs to this task's own context synchronization pass (plan-level `Context sync` list), not to T11's in-scope edits. + - Context synchronization: synced ## Open questions None. The change request already resolves every architectural decision (schema shape, CAS mechanics, which fields are excluded from persistence) precisely, and each decision checks out against the current `protocol.rs`/`types.rs` domain model and the existing Turso adapter conventions verified while authoring this plan. + +## Validation Report + +**Status:** validated +**Date:** 2026-08-29 + +### Commands run + +- `git diff --exit-code quint-connect...HEAD -- cli/migrations/agent-trace-repository/001_repository_schema.sql cli/migrations/agent-trace-repository/002_repository_source_instance_id.sql` -> exit 0 (no diff; `compare-and-swap` branches directly off `quint-connect`, confirmed via `git merge-base`) +- `./scripts/run-cli-cargo.sh test --manifest-path cli/Cargo.toml mutation_trace::store::` -> exit 0 (73/73 passed on 4 of 5 invocations at time of original validation; see Residual risks — the underlying test-isolation flakiness was fixed in post-T11 cleanup and reverified 20/20) +- `nix flake check` -> exit 0 ("all checks passed!" — `cli-tests`, `cli-clippy`, `cli-fmt`, `mutation-trace-quint-connect`, `workflow-actionlint`) +- `nix run .#pkl-check-generated` -> exit 0 ("Ephemeral Pkl generation passed: 141 files") +- `grep -n mutation_trace_attempts cli/migrations/agent-trace-repository/003_mutation_trace_protocol.sql` -> no match +- `grep -n external_taint cli/migrations/agent-trace-repository/003_mutation_trace_protocol.sql` -> no match +- `grep -RnE "std::(fs|process)|tokio::(fs|process)" cli/src/services/mutation_trace/` -> matches only inside `store.rs`'s `#[cfg(test)]` helpers (`unique_test_db_path`/`remove_test_db`), no production usage +- Inspection: `store.rs` (`DurableTransition::between`, `MutationTraceStore::{load_worktree, commit}`, `UPDATE_WORKTREE_CAS_SQL`, module struct fields), `cli/src/services/mutation_trace/` directory listing (no `coordinator.rs`/`git_snapshot.rs`) + +### Acceptance criteria + +- [x] AC1: Mutation state lives in the repository-scoped `agent-trace.db` — `store.rs` only imports and calls through `RepositoryAgentTraceDb` (`self.db.execute`/`query_map`/`execute_transactional_cas_batch`); T09 round-trip tests pass. +- [x] AC2: New storage added via additive migration `003`, `001`/`002` byte-unchanged — `git diff --exit-code` against `quint-connect` merge base exits 0; `003_mutation_trace_protocol.sql` exists. +- [x] AC3: Revision preserves all `u64` values exactly — `revision_round_trips_at_boundary_values` covers `0`, `1`, `i64::MAX`, `i64::MAX+1`, `u64::MAX`; passed. +- [x] AC4: Worktree/scope/`EventKey`/`MutationEvent` round-trip exactly across DB close/reopen — all `round_trip_*` tests and `load_mutation_event_reconstructs_*` tests passed (73/73). +- [x] AC5: `AttemptState` never persisted — no `mutation_trace_attempts` table in migration `003`; `DurableTransition` struct has no `AttemptState` field (fields: `worktree`, `expected_revision`, `next_worktree_state`, `scope_status_changes`, `new_processed_event`, `new_mutation_event`). +- [x] AC6: `external_taint` never DB-authoritative — no `external_taint` reference in migration `003`; `between_returns_none_for_a_database_failure_only_transition` confirms `database_failure` produces no `DurableTransition`; passed. +- [x] AC7: No persistence code determines protocol semantics — `DurableTransition::between` (store.rs:283-330) performs only structural diffing (`diff_target_worktree`/`diff_scopes`/`diff_new_processed_event`/`diff_new_mutation_event`, revision-advance check); no boundary-kind/contention/taint conditional. +- [x] AC8: Every durable transition is one `BEGIN IMMEDIATE` transaction — `MutationTraceStore::commit` (store.rs:713) routes exclusively through `self.db.execute_transactional_cas_batch`. +- [x] AC9: CAS guarded by expected worktree revision — `UPDATE_WORKTREE_CAS_SQL`: `UPDATE mutation_trace_worktrees SET ... WHERE worktree_id = ?6 AND revision = ?7`. +- [x] AC10: Two writers from one revision cannot both commit — `commit_from_two_independent_connections_races_and_only_one_applies` passed. +- [x] AC11: Partial failure rolls back all changes — `commit_rolls_back_every_write_kind_together_on_a_deterministic_failure` passed. +- [x] AC12: Process restart reconstructs the same projection — T09's drop/reopen round-trip tests passed. +- [x] AC13: Bounded hot-path load semantics — `load_worktree` (store.rs:565) issues no query against `mutation_trace_events`; loads only `Active` scopes plus the effective referenced scope per the four `scope`/`event_key` cases, `Err` on mismatch/wrong-worktree/missing-scope; all 73 `mutation_trace::store::` tests (including the T03 four-case suite) passed. +- [x] AC14: Existing Quint Connect and protocol tests remain green — `nix flake check` passed, including `mutation_trace::mbt` (via `cli-tests`) and the dedicated `mutation-trace-quint-connect` check. +- [x] AC15: No Git/filesystem lock/hook/coordinator integration added — no `coordinator.rs`/`git_snapshot.rs` file exists; `std::fs`/`std::process` usage in `mutation_trace/` is confined to test-only DB path helpers. + +### Failed checks and follow-ups + +- None. + +### Residual risks + +- Resolved. `unique_test_db_path` (`store.rs`) previously derived its uniqueness from a nanosecond `SystemTime` timestamp plus `std::process::id()`. One of five back-to-back invocations of `./scripts/run-cli-cargo.sh test --manifest-path cli/Cargo.toml mutation_trace::store::` — run while a separate `nix flake check` build was consuming the same machine's CPU concurrently — hit 16 `UNIQUE constraint failed: mutation_trace_worktrees.worktree_id` failures, consistent with two parallel test threads generating the same nanosecond nonce under heavy scheduler contention, causing unrelated tests to share a physical database file. Fixed in post-T11 cleanup by replacing the nanosecond timestamp with a process-local `static NEXT_TEST_DB_ID: AtomicU64` counter (`fetch_add(1, Ordering::Relaxed)`), combined with `std::process::id()`, so path uniqueness no longer depends on clock resolution. Verified: 20/20 consecutive `mutation_trace::store::` runs passed, plus a full `mutation_trace::` run (161/161) and `nix flake check`. diff --git a/context/sce/shared-turso-db.md b/context/sce/shared-turso-db.md index 32411bcc..16eab0c5 100644 --- a/context/sce/shared-turso-db.md +++ b/context/sce/shared-turso-db.md @@ -30,6 +30,41 @@ - `collect_db_path_health()` emits common parent/path health problems for DB-backed services. - `bootstrap_db_parent()` creates the resolved DB parent directory for repair/setup flows. +## Transactional primitives + +`TursoDb` offers two generic multi-statement transaction primitives beyond +the single-statement `execute()`/`query()`/`query_map()` wrappers. Both are +public on `TursoDb` only (not `EncryptedTursoDb`), and both route +retryability through the same `run_with_retry_sync` seam used everywhere +else in this module, with a local classification layer so a deterministic +failure is never retried like a transient one: + +- `execute_transactional_insert_pair_if_absent(operation_name, retry_hint, + exists_sql, exists_params, first_sql, first_params, second_sql, + second_params, fail_before_second) -> Result`: checks an existence + query, then conditionally runs two inserts inside one transaction only when + the row is absent — the original idle-insert-pair primitive. +- `execute_transactional_cas_batch(operation_name, retry_hint, guard: & + TransactionStatement, statements: &[TransactionStatement]) -> Result`: + a generic optimistic-concurrency batch. `guard` runs first inside one + `BEGIN IMMEDIATE` transaction; its affected-row count is the CAS outcome — + `0` rows commits as a no-op and returns `Ok(false)` (a normal conflict, + never retried) without running `statements`; `1` row runs every + `statements` entry in order and returns `Ok(true)`; more than `1` row is a + deterministic invariant violation (rolled back, `Err`, never retried). + `TransactionStatement::new(sql, params)` pairs SQL with pre-converted + params; its builder `expect_rows_affected(n)` makes a statement's own + affected-row count part of the deterministic-failure check (a mismatch + rolls back the whole transaction and returns `Err` without retrying), + while a statement with no expectation accepts any row count. `BEGIN`, the + guard, every statement, and `COMMIT` all classify a `turso::Error` through + one shared classifier: only `Busy`/`BusySnapshot` is retryable (retries the + whole attempt from a fresh `BEGIN IMMEDIATE`, never an individual + statement); every other variant is deterministic and fails after exactly + one attempt. The first production caller is + `cli/src/services/mutation_trace/store.rs`'s `MutationTraceStore::commit` + (see [mutation-trace-store.md](../cli/mutation-trace-store.md)). + ## Encryption key management `cli/src/services/db/encryption_key.rs` exposes From bba3026dab889f88fd05a0d0f14ddb2674a7eea4 Mon Sep 17 00:00:00 2001 From: David Abram Date: Sat, 29 Aug 2026 13:41:24 +0200 Subject: [PATCH 15/15] docs: Correct register_scope mismatch wording in mutation-trace-store.md The Initialization section previously implied register_scope returns Err whenever a scope's stored worktree_id/actor_kind merely happen to match the request, inverting the actual behavior: a match returns the existing ScopeState unchanged, and only a mismatch returns Err. Co-authored-by: SCE --- context/cli/mutation-trace-store.md | 11 ++++++----- 1 file changed, 6 insertions(+), 5 deletions(-) diff --git a/context/cli/mutation-trace-store.md b/context/cli/mutation-trace-store.md index 94328f39..1bba44fb 100644 --- a/context/cli/mutation-trace-store.md +++ b/context/cli/mutation-trace-store.md @@ -108,11 +108,12 @@ them. ## Initialization `initialize_worktree`/`register_scope` are idempotent idle-inserts (`INSERT -... ON CONFLICT DO NOTHING`) outside the CAS commit path: `initialize_worktree` -never overwrites an existing cursor, and `register_scope` requires the -referenced worktree to already have a durable row, returning `Err` rather than -auto-creating it or accepting a scope whose stored `worktree_id`/`actor_kind` -merely happen to match the request. +... ON CONFLICT DO NOTHING`) outside the CAS commit path: +`initialize_worktree` never overwrites an existing cursor, and +`register_scope` requires the referenced worktree to already have a durable +row and never auto-creates it. An existing scope is returned unchanged only +when its stored `worktree_id` and `actor_kind` match the request; a mismatch +returns `Err`. ## Non-goals