Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions delivery-graph.json
Original file line number Diff line number Diff line change
Expand Up @@ -3418,7 +3418,7 @@
"Crash injection at each switch phase recovers",
"Local HTTP manifest tests for every refusal case",
"Before a transparent Windows upgrade from an older holder without started.json, preserve known command-start/limit evidence or settle the STARTING attempt; validate node-gap and post-recovery starts with timeout and CPU-time limits across versions (MR-95, architecture 04 section 3).",
"Cross-version update/rollback with a live STOPPING attempt must preserve the pending termination_reason/detail interpretation and retry rules, or settle the attempt with the compatible node first; an unknown legacy cause must not be inferred from human text (MR-100, architecture 04 section 9)."
"Cross-version update/rollback with a live STOPPING or FINALIZING attempt must preserve the pending termination_reason/detail interpretation, durable completion phase and retry rules, or settle the attempt with the compatible node first; an unknown legacy cause must not be inferred from human text and a FINALIZING row without a pending stop cause must derive its outcome from exit/launch evidence, never inferred cancellation (MR-100/MR-99, architecture 04 sections 3 and 9)."
],
"verification": {
"author": [
Expand Down Expand Up @@ -3457,7 +3457,7 @@
}
],
"size": "L",
"notes": "MR-95 fixes current-version recovery using additive holder start evidence; it does not backfill older live holders. Cross-version update preservation/settlement remains this planned task's acceptance obligation; no updater or migration is implemented by maintenance. MR-100 uses existing reason/detail columns for pending STOPPING intent; update/rollback compatibility with older readers remains future acceptance, not implemented or verified by maintenance.",
"notes": "MR-95 fixes current-version recovery using additive holder start evidence; it does not backfill older live holders. Cross-version update preservation/settlement remains this planned task's acceptance obligation; no updater or migration is implemented by maintenance. MR-100 uses existing reason/detail columns for pending STOPPING intent; update/rollback compatibility with older readers remains future acceptance, not implemented or verified by maintenance. MR-99 persists FINALIZING before the final result and carries nullable pending stop intent through it; compatibility of live attempts across older/newer node versions remains planned acceptance, not an update implementation or a verified upgrade claim.",
"blocked_by": [],
"evidence": {
"prs": [],
Expand Down
3 changes: 3 additions & 0 deletions ledger/ledger.jsonl
Original file line number Diff line number Diff line change
Expand Up @@ -138,3 +138,6 @@
{"at": "2026-10-25T18:05:40Z", "task": "AF-EXEC-02", "kind": "note", "text": "MR-95 fixed current-version STARTING recovery: Design#28 merged 64e7c994 (reviewed 1683367; full Windows 3 steps/17 s, Linux 3 steps/46 s) before ArcForges#50 merged 6caf418af3eba19a5d4d7989e8ed11476b67736b, reviewed final head 265342e91fe69a53d263dce1eb013af7871af9fd tree 07fd93e4c67b33ec9a7d9a047c52bcf5186de4b6 base 5eaee92. Native full Windows 13 steps/948 s and Linux 16 steps/1128 s passed at final pushed head, all results/statuses checked; direct self-review 6400937846. Four real recovered-start limit cases failed before and passed after; new launch-step and evidence-write-failure guards passed. Initial Linux comm observer failures, incomplete MCP launch, erroneous Windows AOT help command, and superseded-head Windows pass retained distinctly in maintenance/2026-10-25-mr95-validation.md. No assertion weakened, SQL/RPC migration or updater; older Windows holders without started.json remain future AF-DIST-02 preservation/settlement and update validation, still planned."}
{"at": "2026-10-25T18:56:02Z", "task": "AF-DIST-02", "kind": "note", "text": "MR-100 future compatibility assignment: Design#29 merged c58c228fa160c34e91c126e1408c02a2e4efb478; current-version cause recovery is being repaired in ArcForges#51 at 6b4e8c4. Existing reason/detail fields now carry pending STOPPING intent; supported-version update/rollback must preserve that interpretation and retry rules or settle with the compatible node first. Unknown legacy causes keep cancellation fallback, never inferred from human text. Added acceptance/notes; AF-DIST-02 stays planned, no updater, migration, backfill or cross-version validation implemented."}
{"at": "2026-10-25T20:39:05Z", "task": "AF-EXEC-02", "kind": "note", "text": "MR-100 fixed: Design#29 merge c58c228fa160c34e91c126e1408c02a2e4efb478 and Design#30 merge 049cf20a7da7994d849c13dab2c7ac8944c20529 before ArcForges#51 merge dec32cb8ab48a575d563c5e4a1b10c80fc95c486; reviewed head c146ddebb8ce912d375ef5facd505d00525dba6b, tree6e9977f679f505c8b1ac4520a12ea5986e36406a, base6caf418. Final native Windows full13/1051s and Linux full16/1168s passed with exact-head required statuses; direct self-review by OpenAI Codex alone6402619710. STOPPING cause/detail persist atomically and restore permitted retry; pending intent is not a public final result; legacy null cause stays cancellation. Linux forced cleanup preserves bounded controls while killing remaining workload identities and directly thawing even timed-out freezes. Real crash4 Windows/3 Linux and actual end-step/lingering-descendant before-fail/after-pass evidence; Windows root-rundown fixture release corrected without weakening any assertion or bound. Failed/incomplete/diagnostic/superseded runs retained distinctly; no older-head gate reused. MR-95/MR-102 follow-ups are recorded. AF-DIST-02 cross-version acceptance remains future/planned; MR-16 stays blocked, no real credential-store/orphan-file operations; no later feature, LAN/cloud MCP or database migration. See maintenance/2026-10-25-mr100-validation.md."}
{"at": "2026-10-25T21:21:55Z", "task": "AF-DIST-02", "kind": "note", "text": "MR-99 future acceptance only: extend live-attempt update/rollback compatibility to the separately persisted FINALIZING phase and nullable pending stop cause/detail, including normal completion without a cause. AF-DIST-02 stays planned; no updater, database migration or cross-version validation is implemented or claimed."}
{"at": "2026-10-25T21:21:55Z", "task": "AF-EXEC-02", "kind": "note", "text": "MR-99 direct review observation: add open maintenance follow-up MR-105 for the existing synthetic ADOPTED initial RUNNING and direct CANCELLED/HAPPENED finish paths not named by the generic canonical machines. The managed executing-attempt repair does not claim those adoption paths repaired. Reconcile Design/model kinds and verify actual Linux adoption separately; successful Windows adoption remains deferred where every process is already in a job. No task activated and no new host adoption operation performed by this audit."}
{"at": "2026-10-25T22:00:50Z", "task": "AF-EXEC-02", "kind": "note", "text": "MR-99 fixed managed executing attempts: Design#31 merge376c6e9b3b895e75ef3e486623e0b16ad931161f (full Windows3/13s, Linux3/45s) before ArcForges#52 merge47ec7aa38b3c8a7431862ad98b6c0b3c6ddcf8e5; final reviewed headd86b89ce1a0670cfdc6b9dd3fa43260c967f9f6e tree9fd0e767d5f06e56647c0226b8bbf772cf84d8af base dec32cb8. Final native Windows full13/1023s and Linux full16/1183s passed with all results and required exact-head statuses read, fresh main/head checked; direct self-review6403660689 by OpenAI Codex alone under user authorization, no independent-review claim. STOPPING cause precedes platform action, FINALIZING is separately durable with no public final result, normal nullable causes differ from legacy cancellation, and initial STARTING has no fictional CREATED predecessor. Real crash3 plus causal cancel-reply regression passed on each native AOT host, MR100 legacy/retry controls retained; original bounds unchanged. Initial71b367d Linux CI failure136s and old Windows pass1055s are historical only, no stale gate reuse. Synthetic ADOPTED canonical mapping remains separate open MR-105; AF-DIST-02 cross-version acceptance stays planned. No AF-REC-01/later feature, real credential-store/orphan-file operation, LAN/cloud MCP or database migration. See maintenance/2026-10-25-mr99-validation.md."}
5 changes: 3 additions & 2 deletions maintenance/2026-10-24-repair-list.md
Original file line number Diff line number Diff line change
Expand Up @@ -30,7 +30,7 @@ managed tasks, no option that disables protection to enable elevation. The stop

| ID | Issue | Ledger | Repos | Evidence | Proposed disposition | Status |
|---|---|---|---|---|---|---|
| MR-04 | SM-ATTEMPT has no STARTING -> STOPPING although a cancel during a launch persists STOPPING and recovery continues it; the model test carries an open-question allowance. | 39 (1); ArcForges#12 | Design, ArcForges (test) | state-machines README rows 123-135 | fix: Design adds the transition (and `machines.json`); the model test's allowance becomes the specified rule | fixed: Design#25 (merge 724fa49, head a356ad0; retained Windows 3 steps/20 s, Linux 3 steps/44 s) before ArcForges#44 (merge e79f837, head a204daa; fresh Windows 13 steps/979 s, Linux 16 steps/1140 s); canonical transition added, model-test comment aligned without runtime/pair-set change; MR-99/MR-100 remain open; ledger AF-EXEC-02 note |
| MR-04 | SM-ATTEMPT has no STARTING -> STOPPING although a cancel during a launch persists STOPPING and recovery continues it; the model test carries an open-question allowance. | 39 (1); ArcForges#12 | Design, ArcForges (test) | state-machines README rows 123-135 | fix: Design adds the transition (and `machines.json`); the model test's allowance becomes the specified rule | fixed: Design#25 (merge 724fa49, head a356ad0; retained Windows 3 steps/20 s, Linux 3 steps/44 s) before ArcForges#44 (merge e79f837, head a204daa; fresh Windows 13 steps/979 s, Linux 16 steps/1140 s); canonical transition added, model-test comment aligned without runtime/pair-set change; MR-99 and MR-100 follow-ups tracked separately below; ledger AF-EXEC-02 note |
| MR-05 | 04 §7 does not state the effect of a stop during the launch steps (both OS record HAPPENED; Windows can prove the command never started). | 39 (1); ArcForges#12 | Design | 04 §7 effect rule | fix: Design states the rule the implementation follows (launch steps are part of the task, so HAPPENED), or a reviewed narrower rule if the review finds one is required | fixed: ArcForges-Design#26 (merge 8ec5735), reviewed head c5aa36d; 04 section 7 explicitly records HAPPENED for a stop during a running launch step even before the command starts, without a command-run or rollback guarantee; native Windows 3 steps/21 s and Linux 3 steps/45 s passed; existing outcome mappings/retry rules unchanged; ledger AF-EXEC-02 note |
| MR-06 | `StopNode` before READY (STARTING, RECOVERING) answers INTERNAL (`InvalidLifecycleTransitionException`); SM-NODE and the API catalogue give no specified answer. | 39 (1); ArcForges#12 | Design, ArcForges | SM-NODE rows 197-205 | fix: Design specifies the answer (a typed STATE error, or a stop accepted and applied once recovered); node implements and tests it | fixed: Design#27 (merge c36bf89, head bf86fc6; Windows 3 steps/21 s, Linux 3 steps/43 s) before ArcForges#49 (merge 5eaee92, head 8fe744f; Windows 13 steps/963 s, Linux 16 steps/1129 s); typed STATE/invalid_state inside idempotent writer, no deferred stop; real-RPC before-fail/after-pass and error-replay regression; [validation record](2026-10-25-mr06-validation.md); ledger AF-NODE-01 note |
| MR-07 | One unexplained `node stop` exit 75 ("No node listens on the pipe") in a supervisor test under load. | 39 (2); ArcForges#12 | ArcForges | Not reproduced in about 500 targeted stops; diagnostics in place *(confirm against the host logs since)* | fix if a cause is found; otherwise reported with its evidence and kept watched (no claim of resolution) | deferred validation (AR-20), unresolved: historical exit 75 still has no proven cause; current verbose stop/descriptor/supervisor/node diagnostics retained; read-only audit of 197 logs from 53 retained Windows runs (47 successful, 6 failed/incomplete) found no matching message or named supervisor-test failure; this is not a repair or a fresh stress pass; [observation](2026-10-25-mr07-observation.md); ledger AF-NODE-01 note |
Expand All @@ -41,9 +41,10 @@ managed tasks, no option that disables protection to enable elevation. The stop
| MR-12 | DesktopPlatform `LocalRpcClientChannel` CurrentUserOnly check refuses an elevated same-user client (the defect ArcForges.Client fixed locally). | 30 | DesktopPlatform | *(confirm in DesktopPlatform main)* | fix in DesktopPlatform; adoption by ArcForges recorded separately (MR-43) | fixed in producer: DesktopPlatform#200 (merge e02ebe4), reviewed head 66ffbc7; explicit user-SID client check and parent-child pipe owner, foreign/group/missing owners denied; native Windows full 17 steps/532 s and Linux full 24 steps/406 s passed; normal NuGet publication run 40213555173 completed successfully (publish job 121236288721, 2026-10-25); consumer cohort admission/adoption MR-43 separate; AR-21 real cross-integrity validation remains deferred; [evidence](2026-10-25-mr12-validation.md); ledger AF-DP-01 note |
| MR-92 | Lost `JobRemoved` signal: `SystemdManager`'s handler looked for the job's waiter and recorded the result only when it found none, while `WaitJob` registers its waiter and then reads the record; a wait registering in between missed the end and ran to its timeout although the job had finished (seen as a 10 s adoption refusal in `AdoptionTests`; an attempt's 90 s start wait could record 'did not start' for a running attempt). | found in MR-76 (Linux leg at bbd4ddb) | ArcForges | live-manager probe: 68 of 20,800 waits missed on main, 0 of 28,800 with the fix | fix | fixed: ArcForges#40 (merge 64e9f7c); ledger AF-EXEC-02 note |
| MR-95 | Timeouts and CPU-time limits never fire for an attempt recovered STARTING: the supervisor's timeout and CPU-time checks require `attempt.StartedTimestamp`, which only the change to RUNNING sets, and recovery reads `row.StartedAt`, so a command that a recovered launch starts (during the node gap or after) runs without `timeout_ms` or a CPU-time limit, contrary to 04 section 6 and section 9. | found in MR-89 review (ArcForges#42) | ArcForges | `ExecutionSupervisor.cs` timeout/CPU check and recovery (lines cited in #42's review record) | fix: give a recovered STARTING attempt a start time once its command is known to run; fault test with a restart in a launch step | fixed current-version recovery: Design#28 (merge 64e7c99; Windows full 3 steps/17 s, Linux 3 steps/46 s) before ArcForges#50 (merge 6caf418, reviewed head 265342e; Windows full 13 steps/948 s, Linux 16 steps/1128 s); durable Windows command-start evidence, refreshed Linux root and original recovered clock restore timeout/CPU polling; before-fail/after-pass real-process regressions; older-holder cross-version preservation/settlement assigned to planned AF-DIST-02, not claimed verified; [validation](2026-10-25-mr95-validation.md); ledger AF-EXEC-02 and AF-DIST-02 notes; MR-100 follow-up preserves Linux exit-record accounting during a late forced stop; [follow-up](2026-10-25-mr100-validation.md) |
| MR-99 | The persisted attempt states diverge from SM-ATTEMPT: `RecordFinished` writes FINISHED from whatever state the attempt is in, so FINALIZING is never persisted; timeout, CPU-limit, output, disk and shutdown stops persist no STOPPING (SM-ATTEMPT names timeout and CPU limit as RUNNING -> STOPPING triggers), so a crash during such a stop recovers the attempt as RUNNING; STARTING is persisted at admission, before the container exists, and CREATED is never persisted. The model test checks pairs and allows every state -> FINISHED, so it catches none of this. | found in MR-04 (worker and review) | ArcForges, Design | `ExecutionSupervisor.cs` RecordFinished, Stop, admission; `TaskLifecycleModelTests` | decide per divergence: persist the specified states (code) or amend the triggers (Design, reviewed); tighten the model test | open |
| MR-99 | The persisted attempt states diverge from SM-ATTEMPT: `RecordFinished` writes FINISHED from whatever state the attempt is in, so FINALIZING is never persisted; timeout, CPU-limit, output, disk and shutdown stops persist no STOPPING (SM-ATTEMPT names timeout and CPU limit as RUNNING -> STOPPING triggers), so a crash during such a stop recovers the attempt as RUNNING; STARTING is persisted at admission, before the container exists, and CREATED is never persisted. The model test checks pairs and allows every state -> FINISHED, so it catches none of this. | found in MR-04 (worker and review) | ArcForges, Design | `ExecutionSupervisor.cs` RecordFinished, Stop, admission; `TaskLifecycleModelTests` | decide per divergence: persist the specified states (code) or amend the triggers (Design, reviewed); tighten the model test | fixed managed executions: Design#31 (merge 376c6e9; native full Windows3/13 s, Linux3/45 s) before ArcForges#52 (merge 47ec7aa, final head d86b89c; native full Windows13/1023 s, Linux16/1183 s); durable STOPPING/FINALIZING, real initial event, trigger/effect model and awaited cancel initiation; real before-fail/after-pass and causal missing-await control, all original bounds retained; earlier 71b367d Linux CI failure and Windows pass retained separately; synthetic ADOPTED mapping is open MR-105 and cross-version acceptance stays planned AF-DIST-02; [validation](2026-10-25-mr99-validation.md); ledger AF-EXEC-02 note |
| MR-100 | A preempted attempt that lives through a node restart ends CANCELLED instead of PREEMPTED and loses its retry: recovery sets `StopReason = Cancelled` for every recovered STOPPING attempt, while Preempt and Requeue keep the PREEMPTED reason in memory only. | found in MR-04 review | ArcForges | `ExecutionSupervisor.cs` recovery of STOPPING | fix: persist the stop reason with STOPPING and restore it; fault test (preempt, restart, expect PREEMPTED and the retry) | fixed: Design#29 (merge c58c228) and Design#30 (merge 049cf20; native Windows full3/12 s, Linux3/38 s) before ArcForges#51 (merge dec32cb, reviewed head c146dde; native Windows full13/1051 s, Linux full16/1168 s); durable STOPPING cause/detail and permitted retry restored; pending result remains private until FINISHED; legacy null cause retains cancellation; Linux workload cleanup preserves bounded control helpers/exit accounting; real before-fail/after-pass crash and lingering-descendant tests; all original bounds retained; future cross-version acceptance stays in planned AF-DIST-02; [validation](2026-10-25-mr100-validation.md); ledger AF-EXEC-02 note |
| MR-103 | An end step whose `TimeoutMs` is -1 gets an unbounded wait in `RunEndSteps` and bypasses the 10 s end-step budget (04 section 3), delaying `TerminateLeftovers` and `exit.json` for as long as it runs (present on main before MR-101). | found in MR-101 review | ArcForges | `AttemptRunner.RunEndSteps` timeout choice | fix: treat `TimeoutMs <= 0` as the remaining budget, or validate `TimeoutMs > 0` where steps are built | open |
| MR-105 | A code-wide MR-99 audit found a separate canonical mapping gap for synthetic ADOPTED records: adoption inserts task and attempt directly RUNNING; cancellation finishes the attempt directly with CANCELLED/HAPPENED, and natural disappearance with LOST/UNKNOWN. SM-TASK/SM-ATTEMPT do not name the ADOPTED-specific initial state or cancellation shortcut, although 04 section 10 specifies the synthetic adoption behavior. | found during MR-99 direct review | ArcForges, Design | Adoptions.cs insertion/FinishAsync and 04 section 10; read-only code audit, existing suite regressions are separate evidence | fix: reconcile the canonical machines with the existing adoption semantics and guard the model by attempt kind; verify real Linux adoption and retain the deferred successful Windows adoption case | open; separate from MR-99 managed executing-attempt repair |

## 3. Secret purge and backup retention

Expand Down
Loading