[Japabu]

Memory track S0-a, its no-copy-under-a-lock half, built as the orchestrator ruled (2026-11-04, option c): every user copy goes through the process's own mapping, and a fault in it is served as the process's own fault would be, or ends it as BadAddress. No user copy runs under a kernel lock, and one that would is refused by name. Small pipe and message transfers bounce through the caller's stack in one hold. The rest of S0-a (chunk fault-in before any lock, the lost-page outcomes, its H model) is built as part of 7b, per the same ruling, recorded in both tracks. Rebased onto main after #1152: the kernel has no sleeping lock left, so the spin-lock gate covers every kernel lock, and 7b's two prerequisites are both met.

Design (approved; note: #1150 (comment))

The copy (arch::uaccess, user_ptr):

• x86-64: stac; rep movsb; clac (rep stosb to clear). A string instruction that faults leaves RCX, RSI and RDI at its progress (SDM Vol. 2B, MOVS).
• AArch64: unprivileged LDTR/STTR, checked at EL0's permissions, so no PAN toggle is needed. They move 32 bytes a step, four 8-byte user accesses beside paired kernel-side LDP/STP, then 8, then single bytes. The count left is lowered only after each step's accesses.
• The window: checked against the user half's top, start + len with overflow refused, and nothing else. Copies answer Result<(), Faulted>, which becomes BadAddress.

The fault. A Ring 0 or EL1 fault at one of those accesses, on a user address, goes to process::handle_page_fault, at the syscall's depth.

• Served, the access resumes where it stopped. Not served, or a protection fault, it resumes at the routine's fixup, which answers the bytes not copied. Any other kernel fault is fatal, as before, and a copy fault seals no blackbox.
• It composes with 7b's parking faults: the copy holds no lock, and the service runs at baseline depth.
• State across a trap (point 2): a fault or interrupt mid-copy pushes RFLAGS, AC included, into the thread's own trap frame, and its iretq restores it; the clac every entry runs (main's, kernel entry: no kernel register reaches Ring 3, AC and PAN bind at every entry #1154) means the handler runs with AC clear. No switch can happen mid-copy today: the depth is at least 1, the Ring 0 timer only flags, and the fault state blocks preemption during the service. The served-fault test is what proves AC survives the trap and its clac: without iretq restoring it, the resumed rep movsb would fault on a present page and the test would fail. A fault whose saved AC is clear is a kernel bug and panics. AArch64's state is the frame's SPSR/ELR and the copy's registers.

SMAP and PAN are main's (#1154), taken as they are: SMAP and PAN required by the control-register declarations, clac at every entry, and virt_pan_refuses_el1_a_user_page. This PR adds only the routines that open SMAP's door (stac to clac, around one string instruction) and the AArch64 copies, which use unprivileged accesses and so need no PAN toggle. AC is set only between a routine's stac and clac.

Spectre v1 (point 1). Each routine replaces a user address at or past the top with 0 without a branch (cmovae; csel plus csdb), so a mispredicted range check loads no kernel byte. issues/user-pointer-checks-have-no-spectre-v1-fence.md records what is built and what is left: the gadget measurement on each proving machine.

Device memory (point 3). A copy through a user mapping of device memory makes only the accesses the process could make through that same mapping with the same instruction (x86: one rep movsb/stosb, which the SDM lets resume after a fault with its registers preserved; AArch64: unprivileged loads and stores at EL0's permissions, where an unaligned one to Device memory faults and ends the copy). That is nothing the holder could not do itself, so it is allowed and stated at the routines.

Locks (point 4). A copy may be called holding no kernel lock. The gate stays: a copy above its syscall's preemption depth panics, naming the depth. The callers this PR restructured to copy with every lock let go:

• Objects: cloned out of the process lock for read/write/fstat and the rest (syscall::handles::object). A device claim is described under it and copied out after (DeviceRead).
• Laid out, then copied: query_modules, sysinfo, and the counters. Main's later copy sites (namespace names, the device set, last words) answer a fault as BadAddress; a device-set read that faults is not marked seen.
• Pipes and connection directions: they bounce or claim, below.

Paged reads. A holder's paged read copies a chunk of frames one frame at a time. A chunk of more than one frame first faults its destination pages in, in one hold of the space's lock (UserBytesMut::fault_in), so fresh memory takes no exception per page. A part-way copy fault is answered only after the chunk's gone and written checks.

User leaves (x86-64). User page-table leaves are made accessed, and dirty where writable, from creation (Prot::leaf_bits; SDM Vol. 3A §4.8, AMD APM Vol. 2 §5.4.2). Otherwise the walker writes those flags on a fresh page's first access, which a kernel copy into new memory paid for every page. Nothing reads either flag, and AArch64's leaves already carry AF.

Speed (point 5). Measured against main on KVM and on both metal benches (Checks).

Pipes and messages

• One turn protocol (kernel::pipe::Bytes::turn, frame_bytes::turn/wake). A copy records itself as waiting every time it finds the claim held, so no wake is lost. An abandoned write announces its room.
• A pipe read or write of at most 4 KiB bounces through the stack, in the hold that looks its handle up (PipeRef::put_now/take_now). A larger one claims its bytes and copies them straight between the frames and the user.
• Every message bounces (MessageBounce, at most MAX_MESSAGE_LEN). A send is queued whole as its handles move. A receive pops the message and installs its handles in one hold. Only a send whose bytes want new frames claims its room.
• The SDK's lock (toyos::sync::lock_word) spins briefly before it sleeps. The descriptor table's process-wide lock is recorded: issues/every-descriptor-call-takes-one-process-wide-lock-twice.md.

Semantics a reviewer should know

• A bad buffer: a buffer unmapped, or never mapped, answers BadAddress at the copy, with partial progress. Before, it was refused only when its window was made, and a page taken away mid-call read as zeros.
• After a take: a pipe read or message receive that faults copying out has already taken its bytes, which are lost. A receive closes the handles it installed at numbers the reader was never told.
• Device grants: they write their output window before opening anything, so a window the caller cannot write leaves no grant it was not told of.
• Late writes: a pipe write already past its handle lookup when a sibling closes the last writer can land after the reader's EOF. The window is one claim wait and one copy.
• Non-blocking calls: a non-blocking pipe or message call may wait for another copy's claim to end, bounded by one copy.

Records

• Closes issues/a-user-copy-demand-pages-under-whatever-its-caller-holds.md.
• issues/a-pipes-lock-spans-a-user-copy-and-its-first-allocation.md keeps what is left: the hold time unread on a bench, and the queue's lists growing under its lock.
• New: issues/every-descriptor-call-takes-one-process-wide-lock-twice.md.
• New: issues/a-large-spawn-on-the-amd-bench-is-a-seventh-slower-than-mains.md.
• New: issues/osbench-s-malloc-churn-on-the-t14-moves-a-fifth-with-the-allocator-s-code-placement.md. The SDK lock's growth moved std's allocator 32 bytes modulo 64; an arm that only moves it back reads main's figures.
• The 7b ruling, that its sleeping-lock prerequisite is met by Consolidate storage around native bcachefs and userland filesystems #1152's deletion of the sleeping lock, and the bounced read's copy-out after its take, are recorded in the process-memory track and the swap track's S0-a row.
• issues/user-pointer-checks-have-no-spectre-v1-fence.md records the clamp as built and the gadget measurement as left.
• issues/toyos-runs-on-arm64.md records AArch64 copy throughput as unmeasured, until an AArch64 bench measures it.

Checks

The kernel code was measured at bdec5ab36. Commits after it change comments and records only. The branch is rebased onto main after #1182-#1187: the only conflict was sys_getcwd, which #1182 deleted along with my rewrite of it, plus mechanical context. Host CI was re-run on the rebased head. Evidence is on ToyOSOrg/evidence branch pr-1150, rounds r0-r19. The hot-row verdict is r19; both benches' last full round is r15.

• Host: cargo run -- --ci host at bdec5ab36: rc=0, 108 green.

  • The kernel's host tests include the turn and bounce models and an_abandoned_write_pin_grows_nothing_and_holds_nothing.
  • Their controls (N4-N8) are red on exactly their tests.

• Guest rows, on the guest runner:

  • At bdec5ab36, 11 x86 rows that exercise user paging, 11 of 11 (run): a_user_copy_faults_in_mid_copy_or_ends_as_bad_address, a_spawn_from_an_image_frees_it_when_the_child_ends, nvme_disk_boots, console_image_boots, std_threads_count_the_cpus_and_give_back_ended_threads, stream_holders, session_lifecycle, a_launch_gives_each_unsent_stdio_slot_a_null, a_link_ends_under_a_held_stream_and_comes_back, a_client_killed_mid_connect_gives_its_place_back, the_uart_carries_the_log_again_once_the_console_carrier_goes.
  • AArch64 (virt_user_copy_faults, virt_readonly_copyout, virt_unmap_touch, virt_jobs_at_el2, virt_pan_refuses_el1_a_user_page): green at 5c600a4ae and, for the first two, at 378cea3a7. No AArch64 code changed after that.

• Metal round 8, bdec5ab36 against main with the per-syscall probe; rules set before any boot, AMD judged by pair:

  • T14: every hot row moved better: pipe 64-4096 +22% to +40%, ipc.reqrep.8 -15%, ipc.reqrep.4096 -13%, ipc.reqrep_handle.8 -20%.
  • AMD:
    • pipe.throughput.4096 better (+24%), pipe.throughput.64 better (+0.8%, +4.1%), pipe.throughput.1024 unchanged.
    • pipe.throughput.256 was worse in both pairs (-1.0%, -0.8%), a REGRESSION by that round's rule. Round 10 settled it under a stricter rule set first: passed, with a mean of +1.18% and 2 of 6 pairs worse.
    • The IPC hot rows are unchanged: pair 2's head ran AMD's slow mode, and pair 1 reads -10% to -16%.
  • proc.spawn_wait.large: T14 8.02/8.10 ms against main's 8.34/8.42 (better). AMD 12.79/12.75 against 11.00/11.53 (+13%), recorded.
  • AMD copy probe: 760 ns per page against 1871 before the leaf change (main 685). The accessed/dirty mechanism is confirmed by the rule's 800 ns threshold.
  • mem.anon.touch: T14 -38%; KVM -40%.
  • Testcases: 19 of 19 on both benches, including the served-file and pager members. No gate panic and no kernel panic.
  • Other rows past 5% worse: each is bimodal or clock-bound in every arm, base included, across rounds 5-8: time.sleep_late.1ms, mem.realloc_grow.1m_256m, mem.memcpy.262144, proc.posix_spawn_wait.std, wake.*, and the T14's thread.spawn_join, which follows its clock (3791 against 4189 MHz).
  • T14 malloc churn: -16% to -20%, code placement (round 7, recorded).

• Metal round 10, AMD only: the hot-row verdict (r19). It ran six alternating base/head pairs of main's whole benchcase, under a rule set before any boot (r18). A hot row regresses if head is worse than its paired base in at least 5 of 6 pairs (one-sided p ≈ 0.11), or if the mean pair change is below -1%. Every hot row passes:

  Hot row	pair changes (negative = head worse)	worse	mean
  pipe.throughput.64	+4.1 +1.6 +2.5 +3.3 +1.6 -1.6	1/6	+1.90%
  pipe.throughput.256	+3.8 -0.4 +0.0 +1.7 +3.1 -1.0	2/6	+1.18%
  pipe.throughput.1024	+1.8 -1.0 -0.9 +3.1 +0.8 -1.9	3/6	+0.31%
  pipe.throughput.4096	+32.2 +23.0 +23.4 +25.6 +24.1 +17.7	0/6	+24.33%
  ipc.reqrep.8	+9.3 +9.8 +12.2 +10.5 +38.2 -15.4	1/6	+10.76%
  ipc.reqrep.4096	+8.7 +10.5 +12.8 +10.8 +47.9 -19.5	1/6	+11.88%
  ipc.reqrep_handle.8	+15.6 +18.3 +21.5 +17.7 +52.7 -12.2	1/6	+18.93%

  The IPC outliers are AMD's slow mode landing on one boot of a pair, and they count as measured. Rows past -5% that are not hot: mem.anon.map and pingpong.futex.cross_core are bimodal in both arms in every round; proc.spawn_wait.large is -9.4%, recorded.

• Metal round 9, AMD only, diagnostic (r17):

  • Round 8 flagged pipe.throughput.256 (-1.0%, -0.8%); round 9's plain pairs read +0.4% and +3.5%.
  • A pipe-path probe put head's extra time per 256-byte write-read pair mostly in the bounce copies (69 ns in, 69 ns out), which run outside the locks, beside the other side's work.
  • The bounce stays: it is the approved design (coordinator's ruling).

• Earlier rounds:

  • Round 5 found the spawn's per-page copy faults.
  • Round 6 passed every hot row on both benches before the leaf change.
  • Round 7 located AMD's cost in the copy through a fresh leaf, and the malloc loss in placement.

Risks

• Small pipe transfers cost more per call: a 256-byte write or read on AMD takes about 70 ns more for its bounce copy, run outside the locks. Round 10 found no throughput loss on any hot row.
• AMD large spawn (+9% to +13% across rounds 8 and 10): recorded; translating each page through the user mapping is a hypothesis.
• T14 malloc churn: a benchmark's sensitivity to code placement, recorded; the SDK spin stays.
• AArch64 copy throughput: unmeasured until an AArch64 bench exists.

🤖 Generated with Claude Code

[Japabu]

Review r1 of #1150 at 42e2716c3 (base 058bea2d3)

Architecture

The design fits the stage. A user copy now runs outside every spin lock, and one gate in user_ptr refuses any copy made above trap depth. A pipe or connection direction claims its bytes under its own lock, so a copy needs no new lock and no kernel thread. A receive claims, copies, then takes, so a refusal still takes nothing. I checked the pure Bytes claim model against concurrent reads, rotation, an unsupplied write, relisting and the surplus fix, and it holds. The defects below are in the kernel-side wait protocol around the claims, which has no host model.

Size: the kernel grows by +1355/−524 lines. About 215 of the added lines are pure tests, so production grows by about 615 net, mostly in frame_bytes (+211) and message (+187). The TURN protocol is the same code twice, in pipe.rs and message.rs, and both copies carry the same bug (finding 1).

Gate reachability: verified by inspection. SyscallContext is built only in dispatch. I checked every caller of copy_in, copy_out, user_vec, user_str, read_at/write_at, *_pieces, *_raw, fill_zero and translate_user, changed or not:

• random, counters, log, paged, loader, proc, vm, device, machine, fs and ipc;
• console lines, file ops, futex.

None runs under a sync::Lock, and sync::Lock always raises the preemption count. FileObject::with and ConsoleObject::write call Parkable::at_entry, which asserts the baseline depth, so a mis-nested sleeping lock also fails loudly. G1 shows that the gate fires.

Required

1. Lost wakeup in the TURN wait. Sites: kernel/src/pipe.rs:189,218,256-261 and kernel/src/message.rs:301,330,353-358. The waiting flag is set only once, before await_turn. wait_until then loops on its predicate internally without setting the flag again. Interleaving:

   1. Thread A holds a claim. B sets waiting and parks.
   2. A ends its claim. It takes waiting (now false) and posts TURN.
   3. Before B re-checks, A (looping) or a third thread C claims again.
   4. B sees the claim held and parks again. waiting is still false.
   5. The second claim ends and posts nothing.

   B now sleeps until some unrelated post() on that watch. That means the next write for a pipe reader, the next read for a pipe writer, the next send for a receiver, or the next receive for a sender. If none comes, B hangs. Examples:

   • a worker pool receiving on one connection;
   • two threads writing to one stdout pipe while the reader waits for the parked thread's output;
   • a non-blocking op, which the PR says is "bounded by one copy" but is then unbounded.

   Fix: set waiting inside the predicate, under the lock, whenever it finds the claim held. Do it once, in one shared helper for pipes and queues. Inspection then suffices as evidence.

2. An abandoned write claim gives its room back without waking anyone. Sites: kernel/src/pipe.rs:243 (NoMemory) and kernel/src/message.rs:198. The message path is reached by a commit refused because a sibling closed a handle being transferred, or by NoMemory. While a claim is held, space() excludes the claimed room. So a room-waiter (has_space, fits) or a writable poll that checks during the claim finds no room and stays parked. When the claim is abandoned, nobody posts write_watch. Example:

   1. The queue is full and sender S2 is parked waiting for room.
   2. The receiver takes a message and posts, but S1 claims the freed bytes first.
   3. S2 re-checks, finds no room, and parks again.
   4. A sibling of S1 closes one of the handles S1 is sending, so S1's commit check fails.
   5. The room comes back with no post.

   S2 stays parked until another receive, and the receiver may be waiting for S2. Before this PR no claim could hold room, so this state is new. Fix: post write_watch when a write claim with room is abandoned.

3. The records overclaim S0-a. issues/process-memory-in-4-kib-pages-still-copies-each-programs-text.md:115 says "the swap track's S0-a, met", and the PR body says the same. The S0-a row in anonymous-memory-swaps-to-an-encrypted-partition.md:322 still lists unbuilt work:

   • chunk fault-in before any lock, with pins after a re-check;
   • a lost anonymous page ends the caller, and a lost file-mapping page answers EIO;
   • the H model.

   Its exit issue (the pipes one) also stays open. Say instead that S0-a's no-copy-under-a-spin-lock half is met, and that 7b needs only that half, if that is the ruling.

4. Misplaced test comment. At kernel/pure/message.rs:196, the new test was inserted between a_message_without_handles_asks_no_room and its doc comment. the_front_is_answered_and_left_queued now opens with "The table's room is asked only for a message that carries handles", which is false for it, and the old test has lost its doc. Move the line back.

5. The header's invariant is false on one path. The header at kernel/src/frame_bytes.rs:7-8 says the lock is never held across the frame allocator. But at kernel/src/frame_bytes.rs:174, when a concurrent read made the longer list unnecessary, the closure drops the new list's Charge and VecDeque under the object's lock. Charge::drop calls pmm::uncharge, which takes the allocator's lock, and the PR body claims the same invariant. Fix: hand the unused pair out of the hold as old is handed out.

Optional

• At kernel/src/message.rs:343, rt_boost_pending is taken at claim time. If the take is refused NoRoom, the loan is lost. The old code took it only on success.
• pread/pwrite (kernel/src/object/ops.rs:462,470) hold the position SleepLock across the copy, but neither uses the position (pwrite sets only mtime). Copying file_id out first serialises less, and it shrinks the 7b deadlock this PR records.
• kernel/src/object/device.rs:201-213: a console's partial line is now flushed after released_by gives the log back to the UART, and later than before. Confirm that this order is intended.
• kernel/src/syscall/dispatch.rs:123-125: the fsync comment says "fsync parks between flush attempts". ops::fsync never parks, and object() now documents the rule, so delete the comment.
• A read or write keeps its cloned object across parks, but a pipe end's holder is released at zero handles. So a writer blocked after its last handle closed can land bytes after the reader has already seen EOF. Mention this in the PR's semantics list.
• When a reader shares a page, the writer must copy it (Page::own), and that copy can fail with Exhausted. This is a new way a write can fail. Either record it in issues/ or copy under the lock into a bounce buffer, as the write path already does.
• sysinfo_laid repeats HEADER_SIZE/ENTRY_SIZE, and the header-only path now allocates.

Evidence

• Host --ci host at the head: rc=0.
• Guest evidence was reused from 0d7a6ecfc. The only differences from the head are the pure surplus fix, its test and two issue sightings, so the reuse stands.
• N1–N3 are meaningful controls for the Bytes model.
• There is no test or model of the TURN/waiting protocol, which is where findings 1 and 2 are. A host model of claim end against a waiter is not required if the fix makes the invariant evident by inspection.
• Metal is staged, not run, which is acceptable for a draft. The orchestrator owns that run.
• Performance: the extra lock round trips follow from the claim design. I found none avoidable.

SEND BACK

[Japabu]

Design note: user copies through the user mapping, with a fault fixup (option c)

The copy. One primitive per direction: copy_from_user(dst, user, len) and copy_to_user(user, src, len). Each answers the bytes not copied (0 on success), and the Rust layer turns a remainder into BadAddress. A window is checked only against the user half's bound (toyos_userbound), with no lock: SMAP and PAN do not keep the kernel from following a kernel address, so the bound check is what does. The process's current tables are live during a syscall, so the copy uses the user's own virtual address.

• x86_64: a global_asm routine stac; rep movsb; clac. A fault inside rep movsb leaves RCX, RSI and RDI at the progress made (SDM Vol. 2B, MOVS/REP: string instructions restart). An exception table names the one faulting instruction and its fixup label, which runs clac and returns RCX. Where SMAP is absent, stac/clac are skipped behind the declaration's flag (they are #UD there).
• AArch64: a routine that accesses user memory with LDTR/STTR (8 bytes a step, a byte tail). Those are checked at EL0's permissions, so PAN never needs toggling for a copy. The same exception table names their addresses.

The fault. A Ring 0 or EL1 fault whose PC is in the table and whose address is in the user half goes to process::handle_page_fault, the user-mode path. That path is served for a not-present page now and may park in 7b, which composes because no lock is held. Served, the access retries. Not served, or a protection fault, sets the PC to the fixup. Any other kernel fault is fatal, as today.

SMAP and PAN elsewhere. x86: AC is set only between stac and clac in the routine. Every IDT entry from Ring 0 runs clac when SMAP is declared (iret restores the copy's AC), and syscall entry clears AC through FMASK, as it does now. AArch64: PAN joins the control_regs declaration as optional (ID_AA64MMFR1_EL1.PAN). Where the CPU has it, SCTLR_EL1.SPAN = 0, so every entry to EL1 sets PSTATE.PAN; the boot sets it, and check asserts both. Everything but the copy routine then faults on a user address.

Locks. A copy may be called holding no kernel lock: user_ptr's gate (preempt depth above the trap's refuses by name) stays. The fault it may take runs the user fault path, which takes the space's lock; in 7b it may park. No caller lock is held for either to re-enter or park under.

Behaviour that changes.

• A buffer unmapped, or never mapped, answers BadAddress at the copy, with partial progress. Before, it was refused only when its window was made, under the space's lock, and a page taken away mid-call read as zeros.
• A user aperture (device memory) is copied through rather than refused. On AArch64, an unaligned access to Device memory faults to BadAddress.

Deleted.

• user_ptr::View::pieces: the chunked translate, pin and unpin under the space's lock, the zero page and the Arc clone per copy. Also View::window's inventory check (Space::allows), Space::reach with its copyable filter, and pmm::pin_range/unpin_range if copies were their only users.
• frame_bytes's user-side Gather walk: a claim copies each run between the frame's direct-map address and the user address.
• translate_user stays for the futex word, which is not a copy.

Measured after it lands: KVM first, then the staged metal A/B (benchcase without the DATA files group in both arms), against the hot-path bar.

[Japabu]

Review of #1150 at e23d11219 (base a0e8f1333): user copies through the user mapping (option c)

Architecture

The copy design follows the approved note (comment 6634600818). arch::uaccess owns the instructions and the exception table on each architecture. user_ptr is now a thin generic layer: a range check, the gate and Faulted. The fault path hands a copy's fault to process::handle_page_fault. The kernel genuinely has to own this; no userland component could. Each piece I traced has the right owner. The deletions are real: View::pieces, the per-copy pin, the zero page, the Arc<Space> clone and Space::allows. Space::reach and copyable stay for translate_user's futex word, as the note says. pmm::pin_range stays because object/paged.rs still pins with it.

Size: non-test lines are +2746/−904 over 12 commits. The bulk is the earlier rounds' pipe and message bounce/claim work, which r1 reviewed. The option-c part is about 330 lines of arch asm and Rust, plus a smaller user_ptr.

Verified by inspection:

• FMASK: it includes AC (syscall.rs:36). syscall::init writes it on the BSP (boot.rs:113) and on every AP (smp.rs:259).
• clac on every gate: every IDT gate runs it, including the claim stubs through device_irq_entry.
• x86 fixup: rep movsb/rep stosb are F3 A4/F3 AA, so rip + 2 lands on clac.
• AArch64 count: x2 is lowered only after each step's accesses. In copy_to_user it over-counts by at most one step, and the remainder is discarded by whole() anyway.
• Range check (condition 1): in_user_half refuses start + len past the top and refuses overflow (checked_add). The cmovae, and the csel plus csdb, mask the pointer without a branch.
• Copy construction: only dispatch builds a SyscallContext. The gate covers every copy and translate_user.
• Error paths: a receive that faults closes the handles it installed. The pager's write_abandon leaves the size alone, and a host test covers it.
• Call sites sampled: ipc send/recv/badge/namespace_build, io read/write and the bounces, fs readdir/getcwd/readlink, the device grants, paged notices/read/write/supply, random, counters, the log reader, and dispatch env/affinity/thread name.

Required

1. The head is on a stale base, and main already landed half of this PR. Main's 938c4a7cb (kernel entry: no kernel register reaches Ring 3, AC and PAN bind at every entry #1154) already has these:

   • clac after cld on every gate, with syscall_entry left to FMASK;
   • SMEP and SMAP required;
   • PAN required on AArch64: SPAN clear, PAN set by the entry's ERET, checked by check, with virt_pan_refuses_el1_a_user_page.

   This PR re-implements that work differently: PAN is optional (aarch64/boot.rs:127-133, aarch64/control_regs.rs:31-38). The issue it edits is now issues/user-pointer-checks-have-no-spectre-v1-fence.md. kernel/src/sleeplock.rs and the kernel console object are gone (kernel: no kernel thread but the idle loop; the console UART is a userland driver's (K4, K6) #1175). git merge-tree against origin/main shows 15 conflicts.

   Consequence: e23d11219 cannot land, and the PR body's "SMAP and PAN elsewhere" describes main's work as this PR's. The rebase must take main's entry, CR4 and PAN code wholesale and drop this PR's copies of them. The design note's "PAN optional" is superseded by what landed. The file object's position lock also needs its post-Consolidate storage around native bcachefs and userland filesystems #1152 form, because main's FileObject still copies under a spin Lock, which this PR's gate would panic on.

   Evidence to redo on the rebased head:

   • host CI;
   • the 9 x86 machine rows;
   • the AArch64 rows, adding virt_pan_refuses_el1_a_user_page, since the copies now rely on main's PAN.

   ab5: it is staged as a0e8f1333 + probe against e23d11219 + probe. The landing bar is "no loss against main", and main has since changed the IPC path's scheduler (sched: a steal probe is a bit in its victim's word, which a thief can take back #1170). The testcases boot also gates a revision that will not land. Staging ab5 on the rebased head against current main avoids running it twice.

2. Condition 5, AArch64 copy throughput, has no evidence. The KVM A/B and both metal benches are x86. AArch64 now copies at 8 bytes per unprivileged access, where before it used the direct-map memcpy. arm-1 is correctness only.

   • Cheapest sufficient check: an interleaved main/head run under HVF on the development Mac, while it is quiet, of the pipe.throughput.* and ipc.reqrep.* rows (or one large pipe transfer).
   • Label it as HVF on Apple cores: a relative copy cost, not a metal verdict.

3. kernel/src/arch/x86_64/uaccess.rs:18-20: the device-memory contract (condition 3) overclaims.

   • "In order" is false for a WriteCombining mapping, which CachePolicy::WriteCombining provides. Fast-string rep movsb may perform its stores out of order (Intel SDM Vol. 1 §7.3.9.3; Vol. 3A, "Fast-String Operation and Out-of-Order Stores"), and WC is weakly ordered in any case.
   • "At most once per attempt" has no citation for a fast-string restart after a fault.
   • Fix: state only what holds: the accesses are those the process could make through that mapping itself with the same instruction. Or cite each claim per memory type. The same sentence is in the PR body.

4. kernel/src/arch/x86_64/uaccess.rs:6-8 (and the PR body): the hardware citation is misattributed. The quoted "fault ... the registers ... are updated" is not, as far as I can find, the text of the MOVS/STOS entries. The guarantee the fixup rests on is in Vol. 2B's REP-prefix entry: "A repeating string operation can be suspended by an exception or interrupt. When this happens, the state of the registers is preserved to allow the string operation to be resumed …". Cite that entry and its words.

5. kernel/src/syscall/ipc.rs:623: the comment is false. "Installed in this call and named to nobody: a close cannot miss." Handle numbers are predictable, so a sibling thread can close h between the install and the fault. The close then fails, and its error is ignored. Worse, the sibling may already have reopened that number, and the close then shuts the sibling's own new handle. The harm stays within the misbehaving process, but the comment asserts the opposite.

   • Fix (either): close only an entry that is still the installed one, or reword the comment to the race the code accepts.

6. lib/userbound/src/span.rs:89-90: the comment is stale. "A value across a page boundary is copied in pieces, one per page" stopped being true when View::pieces was deleted. Delete the sentence.

7. issues/every-descriptor-call-takes-one-process-wide-lock-twice.md:27-28: delete the last sentence. "For comparison, Linux's fdget … (my recollection, not checked against its source)" is an unverified recollection of another implementation's internals, written in the first person in a repository record. The exit stands without it.

Suggestions

• Fail loudly on a lost AC (idt/exceptions.rs:527). A present-page fault at a copy instruction whose saved RFLAGS.AC is clear can only be a kernel bug, the exact class condition 2 is about. Today it becomes a silent BadAddress. Asserting frame.rflags & AC != 0 there costs nothing.
• Interrupt state. Both fault paths re-enable interrupts unconditionally, but IrqGuard raises no preempt depth, so the gate (user_ptr.rs:62) would let a copy run with interrupts masked. Asserting in gate that interrupts are open closes that gap cheaply.
• Black box. trap_dispatch (idt/mod.rs:352-354) skips the black-box record for any page fault whose rip is a copy instruction. That includes a fault on the kernel-side address, which is fatal. Checking cr2 against the user half there, as page_fault_handler already does, keeps a real kernel fault on the record.
• Device grants. After the zero-fill, a fault in the final write_at (device.rs:263,307,361) can still come from a sibling's munmap. The call then answers BadAddress with the grant in place. Undo it, as sys_inbox_setup does, or narrow the "must not leave a grant" comments to what they ensure.
• Condition 2's wording. Today no switch can happen mid-copy: the depth is at least 1, the Ring 0 timer only flags, and fault_state blocks preempt::enable during the service. The served-fault test is what proves AC survives the trap and its clac; without iretq restoring it, the resumed rep movsb would fault on a present page and the test would fail. Say this in the body.
  • Per-CPU fault_state stays PageFault across handle_page_fault. So "where a fault 7b parks may park" in both handlers is an inference that 7b must still make true. Consider dropping the phrase.
• arm-1.log records no revision. Name it in the log or in the README.

Evidence

• Host: CI at e23d11219, rc=0, 109 steps.
• x86 guest rows: 9/9 at 4856a502c on the remote KVM host. The new row fails correctly at 323a1f513 for an unrelated reason (the window check). Reusing these results for e23d11219 is sound: it changes only the AArch64 registration.
• AArch64: 4/4 under HVF, including virt_user_copy_faults.
• KVM A/B: two interleaved boots per arm. Head beats main on every hot row except reqrep_handle.8, which is inside main's range.
• All of this predates the required rebase (finding 1). The test is focused and hermetic, and it exercises a served fault in both directions, past a bounce, and four refusals. It is a sound negative check of the fixup and of AC across a trap.

SEND BACK

[Japabu]

Review of #1150 at e43c7cf, after the rebase onto e11b6a7

Scope: what the rebase changed. I compared each file's diff against a0e8f13 (old base) to 070c17e with its diff against e11b6a7 to HEAD, read the new diff against main in full where it changed, and audited every user-copy site on main. The approved design and earlier settled findings are not reopened.

Architecture

The rebase fits main's structure. SMAP, PAN and the entry clac are now main's (#1154). The PR adds only the stac/clac window and the copy routines. It no longer has its own clac in common_entry. The file, console and sleeping-lock parts are gone with the code #1152 deleted. Nothing left refers to with_object_ref, ByteSource, UserBytes::sub, AddressSpace::allows or the closed issue. Ungating x86 interrupts_enabled is needed for the gate's assert and matches AArch64, which never had the gate. Net change against main: +2799/−820 lines, mostly the turn and bounce models, frame_bytes and the uaccess routines, as in earlier rounds.

Implementer's claims, checked against the code:

• Gate covers every lock: holds. sync::Lock raises the preemption count, which the depth check catches. IrqLock, and the serial BackendLock under its IrqGuard, mask interrupts, which the interrupts_enabled assert catches. No sleeping lock remains.
• Copy sites on main: none runs under a lock. I checked every read_at/write_at/read_raw/write_raw/fill_zero/copy_in/copy_out/user_vec/user_str/translate_user site in kernel/src. Each runs outside its lock's scope. sys_getcwd was the only one on main that copied inside with_process_data, and it is now fixed. The namespace names, last words and the device set answer a fault with BadAddress. A faulting device-set read returns before set.saw. The last-words length is bounded at dispatch, before copy[..text.len()].
• supply_pages hand merge: correct. I traced it by inspection:
  • Frames at k >= filled go back through the same give_back as refused frames.
  • Promised frames drawn after the fault are still allocated, then given back, so the promise accounting stays balanced.
  • Held-aside frames are zeroed when drawn, so a partly filled frame given back leaks no bytes to the next supply.
  • BadAddress applies only when !stopped, so it never overrides a refusal or a dirty Present stop. refusal(installed, …) keeps the partial count.
• Evidence matches. Host CI log r9/host-ci-e5c600a4ae.log ends rc=0. Guest run 41968136861 ran on 5c600a4ae and succeeded. In the KVM runs, base 51ed020 = e11b6a7 + the probe and head f6dc708 = 5c600a4 + the probe; all four runs succeeded. e43c7cfea differs from 5c600a4ae only in two comment lines in the uaccess files, so reusing these runs is valid.

Findings

BLOCKER kernel/src/object/paged.rs:507-509: the hand merge returns early on a fault before main's holder-atomicity check.

• Code: on a fault mid-chunk, read returns done as soon as the pins are released. Main (Consolidate storage around native bcachefs and userland filesystems #1152) added a check after each chunk's copy: under the lock, waits && !Self::unwritten(object, &mut since) answers WouldBlock, and gone answers Io. The early return skips that check.
• Contract broken: the module header says "A holder's read is atomic against its pager's writes … one that meets a write is answered WouldBlock".
• Consequence: a holder read that faults part-way through a chunk returns done as valid bytes. Those bytes include the unchecked part of the current chunk, which may be torn by a concurrent pager write or copied as the pager ended. The success path answers both cases with an error.
• Evidence: compare the fault return with the success path directly below it (lines 510-521). Before the rebase this site had only the gone check, also skipped. The written check arrived with main's Consolidate storage around native bcachefs and userland filesystems #1152, and the merge put the fault return before it.
• Fix: on a fault, take the lock and run the same gone/written test before answering done, or BadAddress when done == 0. Inspection is sufficient evidence for this. No new guest row is needed.

SUGGESTION issues/process-memory-in-4-kib-pages-still-copies-each-programs-text.md:121: a line added by the rebase runs past the file's wrap width ("so the spin-lock gate is that gate and both are met. The rest of S0-a's row …"). Rewrap it.

For the orchestrator (PR body): "Objects: cloned out of the process lock for read/write/fstat/fsync" names fsync, but main has no fsync syscall after #1152. The handoff confirms the FSYNC arm was dropped. Change it to "read/write/fstat". The rest of the body describes the final change accurately. The open risks (the metal hot-path bar, the unexplained +2.5%/+2.3% on KVM, AArch64 throughput) are stated as open.

SEND BACK

[Japabu]

Review of 199228a (fix for the r-previous blocker)

Scope: only the fix e43c7cfea..199228a81, read in a detached worktree at 199228a and compared with origin/main's PagerObject::read.

Assessment. The fix moves the early faulted return in kernel/src/object/paged.rs read to after the post-copy gone/written re-check. On a fault, the chunk now goes through main's order: unpin, re-check under the state lock, then answer. That resolves the blocker.

What I checked:

• Locks. The user copy (write_raw / fill_zero) still runs with no state lock held. The guard from the step block is dropped before the copy, and the re-check takes the lock briefly and drops it before any return. No new lock is held across the copy or any return.
• Frames. The unpin loop is unchanged and still covers pieces[..count], including pieces after the faulting one. pin_chunk pinned all of them, so a fault leaks no pin and does not unpin twice.
• Holder atomicity. A faulted chunk whose pager ended answers Io. A faulted holder chunk that met a write answers WouldBlock. Both are the same answers a fully copied chunk gets, so bytes copied before the fault are reported only under the same conditions. For the owner path, waits == false makes written false, so only the gone check applies, as on success.
• Accounting. done only advances after a piece copies successfully, so a fault mid-piece is not counted. BadAddress is returned only when done == 0, meaning nothing was copied across all chunks. Otherwise the call returns done.

Findings

No blockers.

SUGGESTION kernel/src/object/paged.rs:519-520: The comment "bytes copied before the fault count only as every copied chunk's do" is hard to parse. A shorter version would be clearer, for example "A fault answers only after the chunk's checks, as a whole copy does." Deleting the comment is also acceptable.

SUGGESTION issues/process-memory-in-4-kib-pages-still-copies-each-programs-text.md:123: The rewrap left one line well over the file's width ("outcomes, its H model) ... of at most"). This is cosmetic.

LAND

[Japabu]

Review of 199228a81..378cea3a7 (focused: UserBytesMut::fault_in)

Architecture. This fits the S0-a row ("every user copy faults … its chunk in before any kernel lock"). The new code doesn't change who owns anything. It adds a batched form of the fault path the copy already takes, Space::fault under the space lock. The user window, at and its gate, and the refusal semantics are unchanged. It adds 27 lines of production code and no tests. I found no extra mechanism, and none is needed.

Correctness, checked against the code:

• Lock order and the gate: the paged state lock is scoped to the step block, so it is released before Step::Copy runs. UserBytesMut::fault_in goes through View::at, which runs gate(). That asserts the syscall's preemption depth and that interrupts are enabled, so fault_in is held to the same rule as a copy. process::fault_in holds the space lock only within its own body and releases it before the first write_raw. The only other things held are the pin_chunk frame pins, which the copy's own faults already ran under.
• Populating ahead: fault → populate checks inventory.find (unmapped → stop) and prot.allows(Write) (read-only or Prot::None guard → stop). View::window already confines the window to the user half. The loop stops at the first refusal, and that page is the one the copy would fault on, so the copy still finds the refusal and answers BadAddress or a partial count as before. Private and copy ranges draw alloc_promised frames, committed when the range was mapped, and still pass through usage.populated. A page populated and then left unwritten, which happens only when the copy stops early because of a race or a gone/written return, therefore takes nothing beyond its commitment. Object pages are mapped exactly as the process's own fault would map them. The hold is bounded by CHUNK (16) pages plus one fault-around run.
• Holder atomicity (199228a): unaffected. The gone and written re-check still follows the copy, and faulted is still answered after it.
• Races: an unmap or protect between fault_in and the copy is serialized by the space lock. The copy then faults through handle_page_fault, which refuses, and the result is Faulted → BadAddress or a partial count. That is the same outcome as before. Nothing is cached from fault_in: it returns ().
• fault_in does not increment usage().faulted. That counter counts serviced fault exceptions, and none is taken here, so this is consistent.

Evidence. I checked r11/host-ci-378cea3a7.log: rc=0, 108 steps green. Guest run 41984456896 has headSha 378cea3 and concluded success. It covers the x86 and arm copy-fault rows and the image spawn, which runs this path. KVM r11 shows one copy fault per spawn instead of 8193, and spawn_wait.large at 8.37–8.42 ms against base 7.63–7.79 ms. The remaining +9% is labelled as an inference that metal round 6 will measure. Inspection is enough to establish the refusal semantics, and I don't ask for a new test.

Findings

BLOCKER kernel/src/object/paged.rs:492 — "One frame's copy faults at most once anyway." is false. A piece of up to 4 KiB is written at out + done, and the user buffer's page alignment is arbitrary, so a single frame's copy can span two destination pages and fault twice. This false sentence is the stated reason for the count > 1 condition. Delete it. Either keep the condition with no justification, or say plainly that a single frame does not merit the extra hold.

SUGGESTION kernel/src/user_ptr.rs:269-273 — the doc comment repeats the motivation in paged.rs ("a caller about to copy … takes no fault for each"). The contract is enough: it populates writable pages under [off, off+len) in one hold and stops at the first refusal; a page left out is the copy's to fault on; the caller bounds len.

SEND BACK. The only blocker is the one-line false comment. When it is fixed, compiling is enough, and no evidence needs to be rerun.

[Japabu]

Orchestrator verification of the r12 fix (378cea3..015a99b), which changes no code:

• The diff touches only comments in two files: kernel/src/object/paged.rs (+2/-1) and kernel/src/user_ptr.rs (+3/-5). No code line changed.
• The required finding from kernel: user copies go through the process's own mapping, and none runs under a lock (S0-a's lock half) #1150 (comment) is resolved. The false sentence "One frame's copy faults at most once anyway" is gone. The comment now gives the real reason for count > 1: a one-frame read would pay the lock hold to save at most two faults.
• The optional suggestion was taken: the fault_in doc now states only its contract.
• Host CI at 015a99b is green (108/108). Metal round 6 runs on 378cea3's images, which are code-identical to this head.

[Japabu]

Focused review: 015a99b52..bdec5ab36 (mainly bdec5ab36)

Architectural assessment

The change is two flags in Prot::leaf_bits (kernel/src/arch/x86_64/paging.rs:63-70), plus 7 lines of comment. It adds 2 lines of code; the two issue records add 65 lines. It is in the right place. Every user leaf on x86-64 is written by Leaf::encode. That covers fault, populate and fault-around (LeafTable::set/fill), object mapping (space.rs:789-864), protect (space.rs:484, through edit), move_leaves, and discard_owned/take_owned. I found no other writer of a user-half leaf. stage_writable_executable and kernel_leaf cover only the kernel half and are unchanged. So "Dirty only where writable" holds by construction. Dirty is a function of Prot::ReadWrite, and Prot::ReadWrite decodes from PAGE_WRITE alone (decode, :696).

1. Correctness on both vendors

• No COW exists. kernel/src/mm has no copy-on-write and ToyOS has no fork. A read-only leaf is either a never-writable image segment or relro (image.rs:83-84), or a range the caller narrowed.
• Protection changes. Narrowing goes through edit. When the protection is unchanged, the leaf is not rewritten (space.rs:485), so hardware-set bits stay. When it changes, the leaf is re-encoded: RW→R drops D, and R→RW sets it. Neither direction can lose a write. Every rewrite of a present leaf is followed by the caller's shootdown, as before. The stale TLB entry that keeps W=1, D=1 until then is the same window that existed before.
• No W=0, D=1 leaf is ever produced. That combination is what CET reads as a shadow-stack page (SDM Vol. 1 ch. 17 / Vol. 3A §4.6). CET is not enabled today, but the invariant that matters for it holds structurally.
• Software may set the bits. Setting A, and D on a writable leaf, from software is architecturally permitted. The processor only ever sets the bits with locked updates, and never clears them, on both vendors. A preset bit just suppresses the walker's assist. UC/WC device leaves gain A/D harmlessly.
• AArch64 is consistent. user_leaf already presets AF. No leaf sets DBM, and nothing enables TCR_EL1.HA/HD. So on AArch64 a writable leaf has no hardware dirty state at all, and "writable means dirty" is already the model there. x86-64 now matches it.

2. Architectural tension with the swap track

The swap record does give the kernel "Accessed and dirty harvesting" (issues/anonymous-memory-swaps-to-an-encrypted-partition.md:55). P2's mem_scan and P4's "scan signals" rely on Accessed. Presetting is compatible with that plan:

• Harvesting is read-and-clear. After a harvest clears A or D (with its shootdown), the walker tracks normally. The preset affects only a leaf's first epoch, and it errs toward young/dirty. It can cost a spurious writeback or keep a page one more round, but it never drops a write.
• The planned design does not use hardware D for correctness. Anonymous re-dirtying during cleaning uses the software CLEANING write-protect (record lines 164-169). Swap-in frees the block, so no "clean anonymous page with a valid swap copy" exists to protect. Droppable copied pages are chosen by "never writable", not by D (P2). File pages are the file servers' (paged.rs: "Nothing here is mapped").
• The real future hazard predates this change. encode discards any hardware-set A/D whenever a leaf is rewritten. Before this change, a writable re-encode would silently clear a hardware D. Now it re-presets D. For a future harvester, the change therefore moves from the losing direction to the safe one.
• The swap record needs no edit now. The stage that builds harvesting must decide whether D is preset and whether encode preserves hardware bits. The site comment flags that. Recording it in P2's row would be harmless but is not required.

3. Comment accuracy and scope

• The comment is accurate in substance. "a clean-page tracker would have to stop setting DIRTY" overstates a little: a harvester that clears D works with the preset and only loses "never written since creation". As a warning at the site it is acceptable.
• It is long, and it duplicates the constant's note (see the suggestion below).
• Scope: it belongs in kernel: user copies go through the process's own mapping, and none runs under a lock (S0-a's lock half) #1150. It is the fix for a regression kernel: user copies go through the process's own mapping, and none runs under a lock (S0-a's lock half) #1150 introduced (proc.spawn_wait.large on AMD through the user-mapping copy), it is 2 lines, and landing it separately would land the regression first. A separate PR would add nothing.

Evidence

• What the commit message claims: the A/D update is only a hypothesis until round 8 confirms it on AMD metal. The KVM result (spawn_wait.large 6.95/6.90 against main's 7.63/7.61) is consistent with it, but it does not establish metal timing.
• What the code needs: its correctness rests on the specification and on inspection. No new test is warranted, since nothing reads the bits.
• Checks run at head: host CI is green, and guest run 42002954748 passed. These cover the remaining risk of a mis-encoded leaf, for example a broken decode round-trip.

Findings

BLOCKER

• issues/a-large-spawns-image-read-costs-about-85-ns-a-page-more-than-mains.md:16-26: this record is stale at head.
  • What is wrong: it gives the KVM gap (8.37-8.42 ms against 7.63-7.79) and the "TLB fill through the 4 KiB mapping" hypothesis as the current state. At bdec5ab36, KVM measures head faster than main, and the branch now carries a different, competing explanation: the walker's A/D assist on a fresh leaf.
  • Required fix: once round 8 reports, update the record in this branch. If the T14 is within main's range, close it. Otherwise record the A/D change's measured effect and revise the hypothesis. If the AMD large-spawn regression (+96%, 1871 against 685 ns/page) survives round 8, it needs a record too, because today only the PR body carries it.
  • Why it matters: "Update issue records directly affected by the change." The record is a 3-line edit and needs no new measurement beyond round 8.

SUGGESTION

• kernel/src/arch/x86_64/paging.rs:32-33, 57-62: make the comments shorter and remove the duplication.
  • The constant's doc is now half stale. "Set by hardware on access or write" no longer holds for user leaves, which software presets. "masked out when comparing entries" applies only to map_2m.
  • The new leaf_bits paragraph repeats "nothing here reads either bit".
  • Possible wording: one line on the constants, e.g. "Preset on user leaves ([Prot::leaf_bits]), set by hardware otherwise; nothing reads either". On leaf_bits, two lines: "Accessed, and dirty where writable, preset (SDM Vol. 3A §4.8; APM Vol. 2 §5.4.2) so the walker never updates a fresh leaf. A clean-page tracker must stop presetting dirty."
  • The AArch64 aside can go; it belongs to AArch64's file, which already says it.

Verdict

The code is correct on both vendors and compatible with the swap plan. The one blocker is the stale record, which round 8's result settles.

SEND BACK

[Japabu]

Orchestrator verification of the changes after the review of bdec5ab (#1150 (comment)). Neither commit changes code:

• c875c3f: only the two A/D comments in kernel/src/arch/x86_64/paging.rs (+4/-8), which was the optional suggestion. No code line changed.
• cf3e5bb: records only. It resolves the review's blocker using round 8: the T14 large spawn is now better than main (8.02/8.10 against 8.34/8.42 ms), so the stale T14 record is deleted. AMD's remaining +13% gets its own record, issues/a-large-spawn-on-the-amd-bench-is-a-seventh-slower-than-mains.md.
• The round 8 metal (r15) ran on bdec5ab's code.

#1150 is still not landable. Under rule 1, AMD pipe.throughput.256 is worse in both round 8 pairs (-1.0% / -0.8%). The owner's no-loss bar for hot paths applies.

[Japabu]

Orchestrator verification of the rebase onto main (cf3e5bb → 61e0bf0), which adds no logic:

• git range-diff e11b6a784..cf3e5bbc0 origin/main..61e0bf05f: commits 2–10 are identical (=). Commit 1 differs only by dropping its sys_getcwd rewrite, because Step 9's file half: the kernel keeps no working directory; a spawn states it in the child's PWD #1182 deleted sys_getcwd on main, plus context lines next to its own hunks from Step 9's file half: the kernel keeps no working directory; a spawn states it in the child's PWD #1182 and A process's tids stop below Tid::MAX, and its next thread start is refused #1184.
• Host CI on 61e0bf0 is green (108/108) after a full rebuild.
• Main added no new kernel user-copy sites since the measured base, and none of the paths measured in metal rounds 5–10 changed, so the review verdicts and the metal evidence (round 10: no hot-path loss on AMD; the T14 better on every hot row) apply to this head.

Landing.