diff --git a/ledger/ledger.jsonl b/ledger/ledger.jsonl index d93ae28..0d37faa 100644 --- a/ledger/ledger.jsonl +++ b/ledger/ledger.jsonl @@ -147,3 +147,4 @@ {"at": "2026-10-26T00:47:34Z", "task": "AF-NODE-02", "kind": "note", "text": "MR-13 fixed (original ledger26 item3): Design#33 merged as 6358f94ebf32f4e2968d591413d86f5e2939decc before ArcForges#55 merged as 448e28dc247fb4234607cc4e811f3723f53e9b31; reviewed final head fcd99c085bb8b5bba44e4be4cb43d97fc147979e, tree 5517871c08c62ec72e659a37ddf014cadc51caa3, base 54e0809. Design complete native Windows3/25s, Linux3/46s; source complete native Windows13/1135s and Linux16/1355s, all results and exact-head statuses read with fresh main/head/tree. Retained backups never carry sealed task input: sanitize new copies before publish and older copies before open, reuse tombstones, compact/check integrity, fail closed on corrupt/locked/unreadable copies; records/names/schema/live retention/key remain. Restored sealed inputs require new submission. Real SQLite/GCM before 5 failures/0 skips each native host (test exit2), final fixed regressions5/5 both; full affected store project Windows93-pass/2-platform-skips of95 and Linux92-pass/3-platform-skips of95; Host and Bench AOT zero warnings/errors both. Initial CS0111 duplicate-constructor compile-only failure removed, existing constructor reused; no weakened assertion. External key backend is isolated under policy, never real OS-store stress/credential/orphan-file operation; no new or deployed database schema migration, enum/RPC/package change, node-key deletion/rotation or later feature. Exact-head direct self-review by OpenAI Codex alone, no other model/agent. MR-16 blocked and MR-14 remains next separate page/WAL audit. See maintenance/2026-10-26-mr13-validation.md."} {"at": "2026-10-26T02:12:43Z", "task": "AF-NODE-02", "kind": "note", "text": "MR-14 fixed: Design#34 merge 3df9db7946e9af6fd1e43f97216adfc44fd55507 before ArcForges#56 merge 5fa2ad5fe77bef791b96db11129b3851fe55c940, reviewed head 0d261bbe1d48f3c1e9848c15f32a26132cf030da. Existing secure_delete plus manual post-commit/replay WAL TRUNCATE and periodic zero-new-row retry; busy/failed manual checkpoint returns existing INTERNAL/internal, effect HAPPENED, no rollback/dedup rewrite. Real before 6 failures per native host, after all 6 own-platform regressions pass; full Store Windows95/97 (2 platform skips), Linux94/97 (3), affected AOT zero warnings/errors. Final complete Windows13/1148s and Linux16/1364s passed at exact head; Design Windows3/15s Linux3/34s. Evidence maintenance/2026-10-26-mr14-validation.md. Direct work/review by OpenAI Codex alone. No new schema migration, restricted credential operation or task activation. MR-16 remains blocked; MR-07 and MR-11 deferred/unresolved. Owner's 2026-10-26 instruction: finish MR-14 then stop for context adjustment to 872000; MR-15 remains next and unstarted. Current handoff maintenance/2026-10-26-handoff.md; 35 disposed,67 open,2 deferred,1 blocked of105. Design#35 merge 2d9995e78c4aa0f6327371ab0b5639857718da5c resolved the common idempotency-summary review finding before source merge; native full Windows3/26s Linux3/45s passed, no source/profile input change."} {"at": "2026-10-26T02:27:22Z", "task": "AF-NODE-02", "kind": "note", "text": "MR-15 audited, unresolved/deferred AR-22, not fixed or validated: ArcForges#12 original Windows/WSL-concurrent CrashConsistencyTests(420) all20 unavailable has no retained unsealing reason; historical Windows selected real Credential Manager, whereas 6978b0c moved writer/reopen to file-backed isolated keys. Read-only Windows audit1940 text logs/11920238bytes across88 run directories/86 summaries (70win,16legacy host-unspecified;75success,11other) found only6 explicit isolated case passes, no original failure/cause; no read errors or size skips. Audit ba16cbe71de88665228e807460632a52c1915591f5d42fe36d28e6cd96f8c0c4. Existing native source profiles and isolated Store passes do not prove the old real-store path. Evidence maintenance/2026-10-26-mr15-observation.md, owning AF-NODE-02 AR-22 closure requires actual nonsecret causal evidence, no new OS-store operation/stress/cleanup authorized. MR-16 remains blocked; no runtime/Design patch, assertion relaxation, real-store test, migration or task-state activation. Owner resumed on2026-10-26 and requests remaining counts after each item:35 disposed,66open,3deferred,1blocked of105;70 unresolved total. Next MR-17, skip blocked MR-16. Direct OpenAI Codex work/review only."} +{"at": "2026-10-26T03:23:31Z", "task": "AF-EXEC-01", "kind": "note", "text": "MR-17 fixed: Design#36 merge9da4127003a90842ddafeb03a6f07e7c5877f34c before ArcForges#57 merge852d23d8b4daca78cef509cb7a6925b18795b6b1, reviewed head92235c31a30d7434a3abe78c09083c3bd507bf9e/tree8013bd2effc7df38b2c14000b073f2a5f643ccd0. Holder retains job_memory from own durable peak above actual enabled notification threshold after released-memory/query sequence; nonzero-exit condition and original hard fallback remain, not proof of allocation refusal. Initial4/4 and5/5 diagnostic batches retained; discriminating before9ec4939 two recovery reason failures plus3 controls, no skips,13.413s. Final author Windows new5/5 JIT+5/5 AOT holder, TaskControl15/15,Job8/8,Repo72/72,HostAOT0warnings/errors; LinuxRepo72/72,pureformat14/14,HostAOT0warnings/errors. Full final pushed-head Windows13/1163s and Linux16/1364s passed, all results/statuses read; Design Windows3/12s Linux3/45s. Evidence maintenance/2026-10-26-mr17-validation.md. Direct OpenAI Codex work/review alone; no new schema migration, real credential operation, altered assertions/steps, old-head pass or later task activation. MR-16 blocked;MR-07/11/15 deferred. Counts36 disposed,65open,3deferred,1blocked of105;69 unresolved total. Report counts and continue MR-18 under owner's resumed authorization."} diff --git a/maintenance/2026-10-24-repair-list.md b/maintenance/2026-10-24-repair-list.md index fe10708..b7b3de2 100644 --- a/maintenance/2026-10-24-repair-list.md +++ b/maintenance/2026-10-24-repair-list.md @@ -59,7 +59,7 @@ managed tasks, no option that disables protection to enable elevation. The stop | ID | Issue | Ledger | Repos | Evidence | Proposed disposition | Status | |---|---|---|---|---|---|---| -| MR-17 | Windows: an attempt that crossed `mem_ceiling` while the node was down is classified EXITED, not MEMORY_LIMIT (the exit record's violation query reads the job state at exit, empty once memory is released). | 39 (2); ArcForges#12 | Design, ArcForges | ArcForges#12 open question (3 workstation runs) | fix: the holder records the job's peak against the notification limit; Design 04 §7 states the node-gap rule | open | +| MR-17 | Windows: an attempt that crossed `mem_ceiling` while the node was down is classified EXITED, not MEMORY_LIMIT (the exit record's violation query reads the job state at exit, empty once memory is released). | 39 (2); ArcForges#12 | Design, ArcForges | ArcForges#12 open question (3 workstation runs) | fix: the holder records the job's peak against the notification limit; Design 04 §7 states the node-gap rule | fixed: Design#36 (merge9da4127; native full Windows3/12s,Linux3/45s) before ArcForges#57 (merge852d23d, reviewed head92235c3; native full Windows13/1163s,Linux16/1364s); durable peak compared with actual enabled notification threshold preserves job_memory after post-release queries; nonzero-exit guard/hard fallback retained, no allocation-refusal guarantee; real before2 failures/3 controls, after JIT5/5 and AOT holder5/5; [validation](2026-10-26-mr17-validation.md); ledger AF-EXEC-01 note | | MR-18 | MEMORY_LIMIT is recorded for a non-zero exit after the commit notification at `mem_ceiling` with no allocation refused, and when a workload's own nested job reports a limit. | 55 (1) | Design, ArcForges | *(confirm in the classification code)* | fix: classification follows the specified evidence; Design states it | open | | MR-19 | `windows_status` labels every code at or above 0xC0000000 as NTSTATUS, including customer-bit codes (0xE0434352, 0xE0000008). | 55 (2), 63 | ArcForges, Design | `ExecutionSupervisor.cs` 1994 | fix: customer-bit codes are application codes (not NTSTATUS); Design and node agree | open | | MR-20 | OOM_KILLED says "killed at its memory limit" for four causes (its own MemoryMax, the slice's, a machine OOM, systemd-oomd). | 56 (1) | Design, ArcForges | *(confirm)* | fix: classify from `memory.events.local` of the attempt and the slice and systemd-oomd's attribute; detail names the cause | open | diff --git a/maintenance/2026-10-26-handoff.md b/maintenance/2026-10-26-handoff.md index cf5e9f7..b5f5fe2 100644 --- a/maintenance/2026-10-26-handoff.md +++ b/maintenance/2026-10-26-handoff.md @@ -3,14 +3,14 @@ ## Latest owner instruction On 2026-10-26 the owner resumed the authorized maintenance list after adjusting context and asked for the remaining -counts after every item. Continue serially from MR-15, completing each item's review, required native checks, merge, +counts after every item. Continue serially in maintenance priority order, completing each item's review, required native checks, merge, maintenance list and CLI ledger record before reporting the new counts and automatically continuing. This supersedes the temporary finish-MR-14-and-stop instruction. Work alone; do not start AF-REC-01 or later features. This supersedes the resumption instruction and stale operational state in the [2026-10-25 handoff](2026-10-25-handoff.md). Scope/order and current dispositions are in the [repair list](2026-10-24-repair-list.md); read the append-only ledger too. -## Completed MR-14 and current maintenance position +## Completed MR-14 and MR-17; current maintenance position - MR-14: [Design#34](https://github.com/ArcForges/ArcForges-Design/pull/34), merge 3df9db7946e9af6fd1e43f97216adfc44fd55507, precedes [ArcForges#56](https://github.com/ArcForges/ArcForges/pull/56), merge 5fa2ad5fe77bef791b96db11129b3851fe55c940. @@ -22,17 +22,29 @@ Scope/order and current dispositions are in the [repair list](2026-10-24-repair- tombstones, live key and retention remain. Busy readers can defer physical erasure, and manual success is withheld then. - MR-13 is already complete: [source#55](https://github.com/ArcForges/ArcForges/pull/55) and [Plan#66](https://github.com/ArcForges/Plan/pull/66). MR-101, MR-04 and MR-102 are also merged; never resume their historical diagnostics as unfinished work. -- This Plan record updates the list, appends the AF-NODE-02 note only through the Plan CLI, and links the new handoff from - the old one. It needs its own fresh native full profiles and unchanged freshly fetched Design context before merge. +- The MR-14 Plan record updated the list, appended its AF-NODE-02 note through the Plan CLI, and linked this handoff from + the old one after its own fresh native full profiles passed. -- Additional [Design#35](https://github.com/ArcForges/ArcForges-Design/pull/35), merge 2d9995e78c4aa0f6327371ab0b5639857718da5c, aligned AD-0004 and the common replay summary before source merge; both native full legs passed (Windows 3/26 s, Linux 3/45 s). It is the Design context used by this Plan record. +- Additional [Design#35](https://github.com/ArcForges/ArcForges-Design/pull/35), merge 2d9995e78c4aa0f6327371ab0b5639857718da5c, aligned AD-0004 and the common replay summary before source merge; both native full legs passed (Windows 3/26 s, Linux 3/45 s). That was the Design context used by the MR-14 record; current verification fetches Design main afresh. + +## Latest completed repair: MR-17 + +[Design#36](https://github.com/ArcForges/ArcForges-Design/pull/36) merge9da4127003a90842ddafeb03a6f07e7c5877f34c +precedes [source#57](https://github.com/ArcForges/ArcForges/pull/57) merge852d23d8b4daca78cef509cb7a6925b18795b6b1. +Reviewed head92235c31a30d7434a3abe78c09083c3bd507bf9e/tree8013bd2effc7df38b2c14000b073f2a5f643ccd0; +native full Windows13/1163s and Linux16/1364s passed at that head. +[Evidence](2026-10-26-mr17-validation.md) distinguishes the initial4/4 and5/5 nondiscriminating batches from the final +before2-fail/3-control and after JIT/AOT-holder5/5 cases. Current job notification threshold, strict peak crossing and +nonzero-exit condition; no hard-allocation-refusal proof, new schema/key operation or later feature. +The owning AF-EXEC-01 completion note is appended only through Plan CLI after source merge. +MR-18 correlated/nested-limit classification/detail remains next; MR-16 and AR-20/21/22 remain unchanged. ## Remaining and boundaries -105 list items: 35 disposed, 66 open, 3 deferred, 1 blocked: 70 unresolved total. +105 list items: 36 disposed, 65 open, 3 deferred, 1 blocked: 69 unresolved total. MR-15 has been audited and recorded as unresolved/deferred AR-22, never repaired or passed; see [the observation](2026-10-26-mr15-observation.md). Current isolated-key passes cannot validate its historical real-store cause. -MR-17 is next eligible; MR-16 keeps its existing block. Check actual logs/processes/results before retrying a timed-out launch. +MR-18 is next eligible; MR-16 keeps its existing block. Check actual logs/processes/results before retrying a timed-out launch. MR-16 stays blocked: no real Windows Credential Manager cleanup/stress or orphan credential-file operations; preserve the recovery proposal. MR-07/AR-20, MR-11/AR-21 and MR-15/AR-22 stay unresolved/deferred. Other acceptance-register limitations, @@ -46,9 +58,9 @@ Refresh GitHub PRs/statuses, git refs, all applicable AGENTS/governance and Desi Recorded SHAs and PIDs are historical evidence, never live state. Preserve all worktrees, user changes and ignored caches. Primary checkouts intentionally remain stale; use fresh origin/main and task worktrees. -Windows source worktree: ArcForges/.worktree/mr-14 (clean detached final head). -Linux source worktree: ArcForges/.worktree/mr-14 (task/mr-14, clean final head). -Plan reusable Windows worktree: Plan/.worktree/maintenance-resume-2026-10-25, branch plan/maintenance-mr15-observation. -Formal logs remain under each repository's .git/arcforges-verify/logs; mr14-* wrapper logs are in the hosts' dev/logs or logs. +Windows source worktree: ArcForges/.worktree/mr-17 (clean detached final head). +Linux source worktree: ArcForges/.worktree/mr-17 (task/mr-17, clean final head). +Plan reusable Windows worktree: Plan/.worktree/maintenance-resume-2026-10-25, branch plan/maintenance-mr17-memory-gap. +Formal logs remain under each repository's .git/arcforges-verify/logs; mr14-*/mr17-* wrapper logs are in the hosts' dev/logs or logs. Never terminate the Linux user systemd manager, use pkill -f or daemon-reload. Keep formal hosts free of competing tests. Work alone under the owner's instruction; use truthful OpenAI Codex attribution and direct review. diff --git a/maintenance/2026-10-26-mr17-validation.md b/maintenance/2026-10-26-mr17-validation.md new file mode 100644 index 0000000..8577946 --- /dev/null +++ b/maintenance/2026-10-26-mr17-validation.md @@ -0,0 +1,86 @@ +# MR-17: Windows node-gap memory-notification evidence, 2026-10-26 + +MR-17 is fixed. [Design#36](https://github.com/ArcForges/ArcForges-Design/pull/36) merged first as +9da4127003a90842ddafeb03a6f07e7c5877f34c. [ArcForges#57](https://github.com/ArcForges/ArcForges/pull/57) merged as +852d23d8b4daca78cef509cb7a6925b18795b6b1, reviewed head92235c31a30d7434a3abe78c09083c3bd507bf9e, +tree8013bd2effc7df38b2c14000b073f2a5f643ccd0, base5fa2ad5fe77bef791b96db11129b3851fe55c940. + +## Diagnosed defect and repair + +A final live violation query is not durable history. If a different notification/accounting query runs after an allocator +released its memory, the holder can later see no job_memory even though the attempt crossed its enabled notification +threshold while the node was down. Existing fallback compares peak only with the higher hard commit cap, so a peak +between notification and hard thresholds is missed and recovery reports EXITED on a nonzero exit. + +JobObject now queries the actual job's enabled notification threshold (zero when disabled). At exit the holder compares +its existing durable peak with that threshold and adds the existing job_memory marker on a strict crossing. The original +hard-limit fallback and Outcome's nonzero-exit guard remain; exit0 is still EXITED. No new polling loop, enforcement action, +enum/RPC/package/schema/migration or data-key/credential operation. QUERY rights/interop types already exist. +Design AD-0005 and04 section7 specify the node-gap rule and same-user guardrail. Evidence is a notification crossing, +not proof of an allocation refused by a hard cap. MR-18's correlated/nested-limit classification/detail questions stay separate. + +## Discriminating native Windows before/after + +Three test-only stages keep production unchanged from5fa2ad5: +- 110d67cbc20690d4abc756ec8c461cd5bdbeff6b:4/4 pass with unread notifications still retained. This did not reproduce the defect. +- eee443d4d83904e1297829720edd05a093a74f3b:5/5 pass; a query while allocation continued could be followed by another notification. + For example one query saw168570880 bytes but the eventual peak was220999680. These are retained diagnostic iterations, + not discriminating failure evidence and not a repaired-code validation. +- 9ec49397236f7f792778d111688121fa7c1504fa (treefc5e0391c77801991e89a0ad81401eb20a9bde9d): a second conditional gate + confirms the allocator process exited before the observer queries and before the root exits. Release build0 warnings/errors + in11.87 s; testexit2 with2 failed recovery reasons (expected MEMORY_LIMIT, actual EXITED) and3 controls passed, + no skips,13.413 s; outer wrapper0/drained. No fixture-precondition failure was counted as the defect. + +The two failed cases had peaks307466240 and221958144 bytes, notification thresholds268435456 and167772160, hard +cap402653184. Both peaks were far below the original hard fallback; the changed-limit peak was below the startup +notification value. Post-release queries reported68698112/68698112 and64135168/64200704 bytes; flags and exit-record +job_memory were empty. The controls retained EXITED for zero exit or no crossing, and MEMORY_LIMIT when a notification +had not been consumed. Original bounds/assertions were retained in the fixed tests. + +The final5-case regression uses real in-process NodeHost stop/restart across generations, native jobs/holders, actual +committed allocation, real queries, file gates and durable exit records. Only external key storage is isolated under the +test policy. It deliberately avoids relying on allocation refusal: a controlled nonzero command exit plus threshold +evidence exercises the specified normalization. No fake samples/events/exceptions or fixed timing releases. +The new test is isolation=exclusive with DisableParallelization and runs with parallel none. +The AOT variant uses the published holder; the node remains in-process. No new standalone native-node crash claim. + +## Final-head author evidence + +| Check | Native Windows | Native Linux | +|---|---|---| +| Format, Release build, analyzers | pass,0 warnings/errors;13.21 s build | pass,0 warnings/errors;15.64 s build | +| New node-gap regression | JIT5/5,no skips14.204 s; AOT holder5/5,no skips14.094 s | Windows-only behavior; no inferred Windows pass | +| Affected TaskControl class |15/15,no skips77.887 s | unchanged Linux behavior | +| Affected JobTests class |8/8,no skips15.263 s | Windows-only behavior | +| Repository guards |72/72,no skips3.130 s |72/72,no skips0.982 s | +| Pure format/protocol tests | covered by full profiles |14/14,no skips0.628 s | +| Host Native AOT publish |0 warnings/errors42.22 s |0 warnings/errors48.56 s | +| Final format/diff/clean checkout, author wrapper |pass,exit0 |pass,exit0 | + +Exact scripts/logs: C:/MyFile/Projects/dev/logs/mr17-before{,2,3}.ps1/.out/.err/.exit, +mr17-author.ps1/.out/.err/.exit; Linux /home/sammiller/arcforges/logs/mr17-author.sh/.out/.err/.exit. +The affected Windows mixed classes were run alone for their exclusive/memory-sensitive methods; no repository +parallelism setting changed. Existing keys/fixtures clean up only their own isolated temporary data. + +## Final pushed-head gates and direct review + +Design headcd2ed1367d9479eaf70dab78bf20cd5b291e9045, treee0fab623ef594f531e9bc5293fc0309b4113eb10: +complete native Windows3 steps/12 s and Linux3 steps/45 s passed. All six results, runner exits0 and required exact-head +statuses were read with fresh main/head/tree before merge. [Design review/gate](https://github.com/ArcForges/ArcForges-Design/pull/36#issuecomment-6407178689). +Design summary IDs:20261026-025257-cd2ed1367d94-win and20261026-105249-cd2ed1367d94-linux. + +Source final head92235c31a30d7434a3abe78c09083c3bd507bf9e: complete native Windows13 steps/1163 s and +Linux16 steps/1364 s passed. All29 results, both wrapper exits0 and engine-reported exact-head success +statuses were read; owner-hosted checks skipped as expected. No competing same-host build/test, altered assertion/step/hook, +old-head pass or partial run substituted. Windows summary20261026-030042-92235c31a30d-win; +Linux summary20261026-105858-92235c31a30d-linux. Both hosts use native MCP, no SSH/WSL/container substitute. + +[Author evidence](https://github.com/ArcForges/ArcForges/pull/57#issuecomment-6407264108), +[direct self-review](https://github.com/ArcForges/ArcForges/pull/57#issuecomment-6407264417), +[final source gate](https://github.com/ArcForges/ArcForges/pull/57#issuecomment-6407498149). Full three-file review checked actual job threshold/rights, strict crossing, +disabled threshold, unchanged zero-exit/hard fallback, real released-memory and changed-threshold preconditions. +OpenAI Codex alone under explicit user instruction; no independent reviewer or other agent/model claimed. + +Only after actual source merge, update MR-17 to fixed and append the owning AF-EXEC-01 note through Plan CLI. +No task-state/later-feature activation. MR-16 remains blocked; MR-07,MR-11,MR-15 remain deferred/unresolved. +After MR-17:36 disposed,65 open,3 deferred,1 blocked of105,69 unresolved total. Report those counts and continue MR-18.