Skip to content

docs: record MR-17 memory-notification recovery repair #69

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
deku2026 merged 1 commit into from
Oct 26, 2026
Merged

docs: record MR-17 memory-notification recovery repair #69

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions ledger/ledger.jsonl
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -147,3 +147,4 @@
{"at": "2026-10-26T00:47:34Z", "task": "AF-NODE-02", "kind": "note", "text": "MR-13 fixed (original ledger26 item3): Design#33 merged as 6358f94ebf32f4e2968d591413d86f5e2939decc before ArcForges#55 merged as 448e28dc247fb4234607cc4e811f3723f53e9b31; reviewed final head fcd99c085bb8b5bba44e4be4cb43d97fc147979e, tree 5517871c08c62ec72e659a37ddf014cadc51caa3, base 54e0809. Design complete native Windows3/25s, Linux3/46s; source complete native Windows13/1135s and Linux16/1355s, all results and exact-head statuses read with fresh main/head/tree. Retained backups never carry sealed task input: sanitize new copies before publish and older copies before open, reuse tombstones, compact/check integrity, fail closed on corrupt/locked/unreadable copies; records/names/schema/live retention/key remain. Restored sealed inputs require new submission. Real SQLite/GCM before 5 failures/0 skips each native host (test exit2), final fixed regressions5/5 both; full affected store project Windows93-pass/2-platform-skips of95 and Linux92-pass/3-platform-skips of95; Host and Bench AOT zero warnings/errors both. Initial CS0111 duplicate-constructor compile-only failure removed, existing constructor reused; no weakened assertion. External key backend is isolated under policy, never real OS-store stress/credential/orphan-file operation; no new or deployed database schema migration, enum/RPC/package change, node-key deletion/rotation or later feature. Exact-head direct self-review by OpenAI Codex alone, no other model/agent. MR-16 blocked and MR-14 remains next separate page/WAL audit. See maintenance/2026-10-26-mr13-validation.md."}
{"at": "2026-10-26T02:12:43Z", "task": "AF-NODE-02", "kind": "note", "text": "MR-14 fixed: Design#34 merge 3df9db7946e9af6fd1e43f97216adfc44fd55507 before ArcForges#56 merge 5fa2ad5fe77bef791b96db11129b3851fe55c940, reviewed head 0d261bbe1d48f3c1e9848c15f32a26132cf030da. Existing secure_delete plus manual post-commit/replay WAL TRUNCATE and periodic zero-new-row retry; busy/failed manual checkpoint returns existing INTERNAL/internal, effect HAPPENED, no rollback/dedup rewrite. Real before 6 failures per native host, after all 6 own-platform regressions pass; full Store Windows95/97 (2 platform skips), Linux94/97 (3), affected AOT zero warnings/errors. Final complete Windows13/1148s and Linux16/1364s passed at exact head; Design Windows3/15s Linux3/34s. Evidence maintenance/2026-10-26-mr14-validation.md. Direct work/review by OpenAI Codex alone. No new schema migration, restricted credential operation or task activation. MR-16 remains blocked; MR-07 and MR-11 deferred/unresolved. Owner's 2026-10-26 instruction: finish MR-14 then stop for context adjustment to 872000; MR-15 remains next and unstarted. Current handoff maintenance/2026-10-26-handoff.md; 35 disposed,67 open,2 deferred,1 blocked of105. Design#35 merge 2d9995e78c4aa0f6327371ab0b5639857718da5c resolved the common idempotency-summary review finding before source merge; native full Windows3/26s Linux3/45s passed, no source/profile input change."}
{"at": "2026-10-26T02:27:22Z", "task": "AF-NODE-02", "kind": "note", "text": "MR-15 audited, unresolved/deferred AR-22, not fixed or validated: ArcForges#12 original Windows/WSL-concurrent CrashConsistencyTests(420) all20 unavailable has no retained unsealing reason; historical Windows selected real Credential Manager, whereas 6978b0c moved writer/reopen to file-backed isolated keys. Read-only Windows audit1940 text logs/11920238bytes across88 run directories/86 summaries (70win,16legacy host-unspecified;75success,11other) found only6 explicit isolated case passes, no original failure/cause; no read errors or size skips. Audit ba16cbe71de88665228e807460632a52c1915591f5d42fe36d28e6cd96f8c0c4. Existing native source profiles and isolated Store passes do not prove the old real-store path. Evidence maintenance/2026-10-26-mr15-observation.md, owning AF-NODE-02 AR-22 closure requires actual nonsecret causal evidence, no new OS-store operation/stress/cleanup authorized. MR-16 remains blocked; no runtime/Design patch, assertion relaxation, real-store test, migration or task-state activation. Owner resumed on2026-10-26 and requests remaining counts after each item:35 disposed,66open,3deferred,1blocked of105;70 unresolved total. Next MR-17, skip blocked MR-16. Direct OpenAI Codex work/review only."}
{"at": "2026-10-26T03:23:31Z", "task": "AF-EXEC-01", "kind": "note", "text": "MR-17 fixed: Design#36 merge9da4127003a90842ddafeb03a6f07e7c5877f34c before ArcForges#57 merge852d23d8b4daca78cef509cb7a6925b18795b6b1, reviewed head92235c31a30d7434a3abe78c09083c3bd507bf9e/tree8013bd2effc7df38b2c14000b073f2a5f643ccd0. Holder retains job_memory from own durable peak above actual enabled notification threshold after released-memory/query sequence; nonzero-exit condition and original hard fallback remain, not proof of allocation refusal. Initial4/4 and5/5 diagnostic batches retained; discriminating before9ec4939 two recovery reason failures plus3 controls, no skips,13.413s. Final author Windows new5/5 JIT+5/5 AOT holder, TaskControl15/15,Job8/8,Repo72/72,HostAOT0warnings/errors; LinuxRepo72/72,pureformat14/14,HostAOT0warnings/errors. Full final pushed-head Windows13/1163s and Linux16/1364s passed, all results/statuses read; Design Windows3/12s Linux3/45s. Evidence maintenance/2026-10-26-mr17-validation.md. Direct OpenAI Codex work/review alone; no new schema migration, real credential operation, altered assertions/steps, old-head pass or later task activation. MR-16 blocked;MR-07/11/15 deferred. Counts36 disposed,65open,3deferred,1blocked of105;69 unresolved total. Report counts and continue MR-18 under owner's resumed authorization."}
2 changes: 1 addition & 1 deletion maintenance/2026-10-24-repair-list.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -59,7 +59,7 @@ managed tasks, no option that disables protection to enable elevation. The stop

| ID | Issue | Ledger | Repos | Evidence | Proposed disposition | Status |
|---|---|---|---|---|---|---|
| MR-17 | Windows: an attempt that crossed `mem_ceiling` while the node was down is classified EXITED, not MEMORY_LIMIT (the exit record's violation query reads the job state at exit, empty once memory is released). | 39 (2); ArcForges#12 | Design, ArcForges | ArcForges#12 open question (3 workstation runs) | fix: the holder records the job's peak against the notification limit; Design 04 §7 states the node-gap rule | open |
| MR-17 | Windows: an attempt that crossed `mem_ceiling` while the node was down is classified EXITED, not MEMORY_LIMIT (the exit record's violation query reads the job state at exit, empty once memory is released). | 39 (2); ArcForges#12 | Design, ArcForges | ArcForges#12 open question (3 workstation runs) | fix: the holder records the job's peak against the notification limit; Design 04 §7 states the node-gap rule | fixed: Design#36 (merge9da4127; native full Windows3/12s,Linux3/45s) before ArcForges#57 (merge852d23d, reviewed head92235c3; native full Windows13/1163s,Linux16/1364s); durable peak compared with actual enabled notification threshold preserves job_memory after post-release queries; nonzero-exit guard/hard fallback retained, no allocation-refusal guarantee; real before2 failures/3 controls, after JIT5/5 and AOT holder5/5; [validation](2026-10-26-mr17-validation.md); ledger AF-EXEC-01 note |
| MR-18 | MEMORY_LIMIT is recorded for a non-zero exit after the commit notification at `mem_ceiling` with no allocation refused, and when a workload's own nested job reports a limit. | 55 (1) | Design, ArcForges | *(confirm in the classification code)* | fix: classification follows the specified evidence; Design states it | open |
| MR-19 | `windows_status` labels every code at or above 0xC0000000 as NTSTATUS, including customer-bit codes (0xE0434352, 0xE0000008). | 55 (2), 63 | ArcForges, Design | `ExecutionSupervisor.cs` 1994 | fix: customer-bit codes are application codes (not NTSTATUS); Design and node agree | open |
| MR-20 | OOM_KILLED says "killed at its memory limit" for four causes (its own MemoryMax, the slice's, a machine OOM, systemd-oomd). | 56 (1) | Design, ArcForges | *(confirm)* | fix: classify from `memory.events.local` of the attempt and the slice and systemd-oomd's attribute; detail names the cause | open |
Expand Down
34 changes: 23 additions & 11 deletions maintenance/2026-10-26-handoff.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3,14 +3,14 @@
## Latest owner instruction

On 2026-10-26 the owner resumed the authorized maintenance list after adjusting context and asked for the remaining
counts after every item. Continue serially from MR-15, completing each item's review, required native checks, merge,
counts after every item. Continue serially in maintenance priority order, completing each item's review, required native checks, merge,
maintenance list and CLI ledger record before reporting the new counts and automatically continuing.
This supersedes the temporary finish-MR-14-and-stop instruction. Work alone; do not start AF-REC-01 or later features.

This supersedes the resumption instruction and stale operational state in the [2026-10-25 handoff](2026-10-25-handoff.md).
Scope/order and current dispositions are in the [repair list](2026-10-24-repair-list.md); read the append-only ledger too.

## Completed MR-14 and current maintenance position
## Completed MR-14 and MR-17; current maintenance position

- MR-14: [Design#34](https://github.com/ArcForges/ArcForges-Design/pull/34), merge 3df9db7946e9af6fd1e43f97216adfc44fd55507,
precedes [ArcForges#56](https://github.com/ArcForges/ArcForges/pull/56), merge 5fa2ad5fe77bef791b96db11129b3851fe55c940.
Expand All @@ -22,17 +22,29 @@ Scope/order and current dispositions are in the [repair list](2026-10-24-repair-
tombstones, live key and retention remain. Busy readers can defer physical erasure, and manual success is withheld then.
- MR-13 is already complete: [source#55](https://github.com/ArcForges/ArcForges/pull/55) and [Plan#66](https://github.com/ArcForges/Plan/pull/66).
MR-101, MR-04 and MR-102 are also merged; never resume their historical diagnostics as unfinished work.
- This Plan record updates the list, appends the AF-NODE-02 note only through the Plan CLI, and links the new handoff from
the old one. It needs its own fresh native full profiles and unchanged freshly fetched Design context before merge.
- The MR-14 Plan record updated the list, appended its AF-NODE-02 note through the Plan CLI, and linked this handoff from
the old one after its own fresh native full profiles passed.

- Additional [Design#35](https://github.com/ArcForges/ArcForges-Design/pull/35), merge 2d9995e78c4aa0f6327371ab0b5639857718da5c, aligned AD-0004 and the common replay summary before source merge; both native full legs passed (Windows 3/26 s, Linux 3/45 s). It is the Design context used by this Plan record.
- Additional [Design#35](https://github.com/ArcForges/ArcForges-Design/pull/35), merge 2d9995e78c4aa0f6327371ab0b5639857718da5c, aligned AD-0004 and the common replay summary before source merge; both native full legs passed (Windows 3/26 s, Linux 3/45 s). That was the Design context used by the MR-14 record; current verification fetches Design main afresh.

## Latest completed repair: MR-17

[Design#36](https://github.com/ArcForges/ArcForges-Design/pull/36) merge9da4127003a90842ddafeb03a6f07e7c5877f34c
precedes [source#57](https://github.com/ArcForges/ArcForges/pull/57) merge852d23d8b4daca78cef509cb7a6925b18795b6b1.
Reviewed head92235c31a30d7434a3abe78c09083c3bd507bf9e/tree8013bd2effc7df38b2c14000b073f2a5f643ccd0;
native full Windows13/1163s and Linux16/1364s passed at that head.
[Evidence](2026-10-26-mr17-validation.md) distinguishes the initial4/4 and5/5 nondiscriminating batches from the final
before2-fail/3-control and after JIT/AOT-holder5/5 cases. Current job notification threshold, strict peak crossing and
nonzero-exit condition; no hard-allocation-refusal proof, new schema/key operation or later feature.
The owning AF-EXEC-01 completion note is appended only through Plan CLI after source merge.
MR-18 correlated/nested-limit classification/detail remains next; MR-16 and AR-20/21/22 remain unchanged.

## Remaining and boundaries

105 list items: 35 disposed, 66 open, 3 deferred, 1 blocked: 70 unresolved total.
105 list items: 36 disposed, 65 open, 3 deferred, 1 blocked: 69 unresolved total.
MR-15 has been audited and recorded as unresolved/deferred AR-22, never repaired or passed; see
[the observation](2026-10-26-mr15-observation.md). Current isolated-key passes cannot validate its historical real-store cause.
MR-17 is next eligible; MR-16 keeps its existing block. Check actual logs/processes/results before retrying a timed-out launch.
MR-18 is next eligible; MR-16 keeps its existing block. Check actual logs/processes/results before retrying a timed-out launch.

MR-16 stays blocked: no real Windows Credential Manager cleanup/stress or orphan credential-file operations; preserve
the recovery proposal. MR-07/AR-20, MR-11/AR-21 and MR-15/AR-22 stay unresolved/deferred. Other acceptance-register limitations,
Expand All @@ -46,9 +58,9 @@ Refresh GitHub PRs/statuses, git refs, all applicable AGENTS/governance and Desi
Recorded SHAs and PIDs are historical evidence, never live state. Preserve all worktrees, user changes and ignored caches.
Primary checkouts intentionally remain stale; use fresh origin/main and task worktrees.

Windows source worktree: ArcForges/.worktree/mr-14 (clean detached final head).
Linux source worktree: ArcForges/.worktree/mr-14 (task/mr-14, clean final head).
Plan reusable Windows worktree: Plan/.worktree/maintenance-resume-2026-10-25, branch plan/maintenance-mr15-observation.
Formal logs remain under each repository's .git/arcforges-verify/logs; mr14-* wrapper logs are in the hosts' dev/logs or logs.
Windows source worktree: ArcForges/.worktree/mr-17 (clean detached final head).
Linux source worktree: ArcForges/.worktree/mr-17 (task/mr-17, clean final head).
Plan reusable Windows worktree: Plan/.worktree/maintenance-resume-2026-10-25, branch plan/maintenance-mr17-memory-gap.
Formal logs remain under each repository's .git/arcforges-verify/logs; mr14-*/mr17-* wrapper logs are in the hosts' dev/logs or logs.
Never terminate the Linux user systemd manager, use pkill -f or daemon-reload. Keep formal hosts free of competing tests.
Work alone under the owner's instruction; use truthful OpenAI Codex attribution and direct review.
Loading