[Japabu]

Problem

Two ways a claim's holder could harm the kernel, both in pcidev's ownership of a function:

1. Grants spent device addresses for good. Each SYS_DEVICE_DMA_ALLOC
   grant went at a fresh address of the function's domain (Domains::map,
   over iova::Window::reserve, which hands each address out once per boot).
   A holder that claims, allocates and ends again and again (a supervisor
   restarting a service does exactly this) spent up to 32 MiB a claim, plus
   the remapping tables under it, and none of it came back. The slot then
   refused every grant for the rest of the boot. If physical memory ran out
   first, Tables::alloc panicked.
2. A late write from a released function halted the machine. release
   clears Function::held at its end, before a reset has quieted the
   function. A transfer already in flight that faults after that point was
   read as a fault from a function no process drives (owner=kernel), and
   iommu::fault::conclude halted the machine.

Change

One window per function. Every device address a claim is given now lies
in one window reserved with its function's domain: MAX_GRANT_TOTAL
(32 MiB) for grants, then LEND_WINDOW (256 MiB) for lends, below 4 GiB.
The lend window already worked this way.

• kernel::claimspace (new, pure) places both. A grant goes first fit in
  the 32 MiB grant part, at the lowest room no live grant touches. The window
  replaces the per-claim byte sum: a claim is refused where no run of the
  grant's length is free, which can happen below 32 MiB once grants are
  taken back out of order (see Not done).
• A grant may be placed over residue (what a function no reset quiets may
  still be aimed at) because it is fresh pages, and the residue it covers is
  carved off. A lend never goes over residue.
• A reset function's retired grants are unmapped before its next attach, so
  every claim reuses the same addresses.
• With a fixed window, a function's remapping tables are bounded: no grant
  allocates a table outside the window's span. The tables still come from
  Tables::alloc's expect, which is now reachable only through the
  kernel's own superframes running out, not through a holder's churn.
• Deleted: Domains::map, map and map_at in every backend (VT-d,
  AMD-Vi, SMMUv3), x86's dispatch of both, OwnSpace::map and
  IommuError::AddressesExhausted.
• OwnSpace::map_at and OwnSpace::map_again place a grant and write the
  same iommu: domainN maps … at …[ again] record as before.
  domains_map_clear_of_the_host_bridges reads that record.

A released function's fault is its last holder's. Function::claimed is
set by claim and never cleared. note_fault hands such a function's fault
to its last holder: bus mastering cleared, the line written as
owner=process, and the machine goes on. A function only this kernel
drives, or one never claimed, still halts the machine. The issue's exit
proposed clearing ownership only once the reset is quiet. That needs a
quiet event the kernel does not have (a reset's quiet is a deadline read at
the next claim), and a function no reset quiets is never quiet. "Once
claimed, never the kernel's" answers both cases with one flag.

No driver states an address width (round 2, by the orchestrator's
ruling). Every claim address is in its window, below 4 GiB, and every PCI
function issues those whole, so SYS_DEVICE_DMA_LIMIT decided nothing.

• Deleted in the kernel: the syscall, pcidev::dma_limit and its
  "stated first" refusals.
• Deleted from the SDK and pci-claim: toyos_abi::syscall::device_dma_limit
  (number 131), PciDev::dma_limit, toyos_pci_claim::limit, and
  toyos_pci_claim::id, whose only user was hda's width.
• Deleted from the drivers: the calls in nvme, usbd, i219 and the virtio
  bring-up, plus usbd's Reset::address_bits. The usbd_spare and
  i219_msi guest jobs lose theirs too.
• kernel::iova::Window::new now clips every domain to 4 GiB, which is the
  rule Limit used to enforce. Its control iova-ignores-limit becomes
  iova-past-four-gib.
• hda's address_bits, and its refusal of AMD's 1002:1637, go. That
  function drops address bits 40 and up, and no address it is given now has
  one, so it is driven like any other function.

Closes:

• issues/a-claim-spends-device-addresses-its-slot-never-gets-back.md
• issues/a-late-write-from-a-released-function-halts-the-machine.md
• issues/a-narrow-functions-device-addresses-are-spent-once-per-boot.md
  (every claim address is now below 4 GiB and reused)
• issues/a-domains-addresses-are-never-given-back.md (no domain hands out
  an address after it is made)
• issues/the-hdmi-audio-function-1002-1637-is-driven-at-no-address-width.md
  (no width is needed: its exit's boot is the AMD testcases boot staged
  below, whose log should show 03:00.1's codec walked with no DMA FAULT)

Checks

Head 83ddb42 (round 3: review r1's text fixes and deletions; --ci host exit 0, log). Round 3 evidence: b69b43a8c/pr-1188/r3. Round 2 evidence: ce6f9f175/pr-1188/r2.
Round 1: 190bc99a4/pr-1188/r1.

• cargo run -- --ci host at f6dc7d0: exit 0 (log).
  This includes the iova-past-four-gib control reddening its named tests,
  and kernel clippy on x86_64 and aarch64. The run before it was red because
  Gcap::wide had been removed with hda's width
  (log), and it is restored.
• cargo run -- --build-only, exit 0 (log), and
  --build-only --arch aarch64, exit 0 (log).
• The kernel::claimspace tests and their two controls are round 1's, at
  a0d5cd8. The file has changed since only in a doc comment, so those results still hold.
  • a_function_no_reset_quiets_serves_every_holder_whatever_the_last_one_laid_out
    runs 2000 claims with random grant layouts and lends, each ending in a
    release that leaves everything as residue. Every grant is served inside
    the grant part, no lend lands on residue, and the residue stays inside
    the window and disjoint.
  • Controls: grants that avoid residue (the old rule) red that test, and a
    carve that removes nothing reds it and
    carving_keeps_what_a_grant_does_not_cover; both exit 101
    (controls).
• Guest rows usbd_drives_the_spare, nvme_disk_boots and virt_claim_lpi
  at f6dc7d0 (run 42023736614): success. They cover a VT-d claim with
  grants and a fault on a held claim, an nvme claim's grants and lends, and
  an SMMUv3 claim.
• Metal at f6dc7d0, T14 testcases boot (judges):
  • Passed: claim_reuses_its_remapping_entry,
    domains_map_clear_of_the_host_bridges ("4 mapping(s), none reaching a
    root-bridge window") and usbd_drives_the_type_c_controller.
  • Failed: i219_messages_on_its_msi_vector, at its last step only, "no
    frame arrived on a message under traffic". Every forced cause was
    delivered on the claim's MSI, and 65 frames were sent through the new
    window's grants. The MAC's own pre-DMA receive counters saw 0 frames on
    the wire, and the boot has no DMA FAULT
    (excerpt). This is the known independent failure recorded in
    issues/i219-msi-passes-only-when-the-network-sends-it-a-frame.md (main showed it on 2026-10-21 and 10-24); the MAC counts frames before any DMA, so placement cannot cause a zero count.
  • AMD testcases boot (judge):
    amdvi_translates_the_boot_controller passes. 03:00.1 [1002:1637] is
    handed over in domain3, which ends at 0x100000000. Its grant is at
    0x200000. hda walks codec0 (1002:aa01), ends with "plays nothing", and
    the function is released with a D3hot reset. The log has 0 DMA FAULT
    lines (excerpt). That meets the closed
    1002:1637 record's exit.

Not done, and why

• No test makes the late write. A function with Bus Master Enable clear
  issues no request (PCI 3.0 §6.2.2), and issues/toyos-runs-on-arm64.md
  records that no QEMU function writes once mastering is off except
  iommu-testdev, which no claim can drive. No bench function can be made to
  write between release and quiet either. The change is a flag and two
  branches in note_fault, so this rests on reading the code.
• First fit can refuse a grant below 32 MiB. Take back 2 MiB grants out
  of order and the free runs are 2 MiB apart, so a 4 MiB grant is refused
  while the claim holds 16 MiB. Before, the byte sum alone decided. Current
  drivers grant once at start and never take one back except through
  dma_undo's failure path, so none meets it.
• A lend taken back before a non-reset release is not residue
  (issues/a-lend-taken-back-before-a-release-is-not-residue.md, security,
  filed from the review's sighting). Main has the same gap, and this branch
  neither causes nor fixes it: the fix changes placement.
• Lend-window residue still accumulates for a function no reset
  quiets, because lends avoid residue and nothing carves it there.
  issues/a-restarted-display-holder-lends-around-its-predecessors-scanouts.md
  already tracks this, and this change does not alter it. One small
  capability is lost: before, a grant of exactly a lent range's size could
  reuse that lend-window residue. Grants now stay in the grant part.

🤖 Generated with Claude Code

[Japabu]

Review of #1188 at f6dc7d0

Architecture

This is the right ownership. Each function's record already kept one domain for the whole boot. Now that domain reserves one fixed window, and pcidev places everything a claim is given inside it with a small pure placer (kernel::claimspace). Each backend loses its fresh-address-per-grant path. The domain's table set is bounded by the window. The address-width syscall becomes unnecessary because nothing is handed out at or above 4 GiB. That deletes a mechanism, not just a bug. Net production code goes down. claimspace adds about 100 lines of code and about 100 lines of tests. iova, pcidev, the ABI, the SDK, pci-claim and four drivers lose more than that. Overall the diff is +436/−683.

I checked these points and found them sound:

• Isolation. Every claim address comes from the function's own domain. Domains::place asserts the address is inside what create reserved. Grant pages are fresh and belong to the claim. A grant placed over residue only takes a stale transfer from the same function. A lend never goes over residue. I found no path from one claim to another domain's addresses or another function's memory.
• Churn and the free list. carve splits a range correctly. Live grants and residue stay disjoint: a residual grant's span goes back to bound.residue on undo, and it is removed from grants when that happens. tear_down merges them without overlap. The 2000-claim test asserts this, and the two controls red it.
• Below 4 GiB on all three backends. VT-d, AMD-Vi and SMMUv3 all build their domains through Domains::prepare → iova::Window::new, which now clips at 4 GiB. RMRR identity ranges go through reserved_space, not through the window, so the clip does not touch them.
• Teardown and reuse. No domain is ever destroyed. A domain is reused across claims and across hot-plug arrivals on the same record. finish_retired unmaps a reset function's parked grants before the next attach. retire clears both parked grants and residue for a function that left. Retired and residue stay mutually exclusive.
• The remaining Tables::alloc panic. The table set is bounded, and the bound is real. A 288 MiB window of 2 MiB leaves needs a root, one PDPT-level table and at most two PD-level tables, and none is ever freed. alloc_page → Frames::alloc_run may use the kept superframes, which processes cannot take. So holder churn cannot reach the panic, and what is left is a kernel-reserve invariant. That does not literally meet the closed record's "refusal, not a panic" clause, and the PR body says why it is no longer reachable from userland. I accept that.
• Late-write attribution. This is safe. note_fault still clears BME before deciding anything. A released, non-reset function's grants are already unmapped, and a reset one's are still mapped until it is quiet. Either way, a device that ignores BME meets only that function's own domain. Before, a fault in that state halted the machine. Attributing it to the last holder is the correct containment under the trust model. Setting claimed after the held CAS has no harmful interleaving. A function can still be refused as KernelDriven before the flag is set, so a kernel-driven function's faults still halt.
• Deletion completeness. No dma_limit, DMA_LIMIT, iova::Limit, AddressesExhausted, address_bits, pci_claim::limit or iova-ignores-limit is left in the tree, the issues or the rust/ fork's library. The ABI, the SDK, the drivers and the guest jobs change together in this one landing.

Required

1. kernel/src/pcidev/mod.rs:1821-1822, sdk/abi/src/syscall.rs:2692-2693, kernel/src/pcidev/mod.rs:1856. The grant-bound contract is false.
   • The comment says "no room there is also more than a claim may hold". The ABI says ResourceExhausted comes "once the claim holds all it may".
   • Placement is first-fit inside a fixed 32 MiB part, so a claim can be refused well below MAX_GRANT_TOTAL. Example: 16 grants of 2 MiB, undo every other one. The claim holds 16 MiB, every free run is 2 MiB, and a 4 MiB grant is refused.
   • Before this change, the sum alone decided. This is a real, small loss of capability. It is acceptable for current drivers, which grant once at start, but the contract must say what is true.
   • State the bound as "no run of the grant's length free in the claim's 32 MiB grant part". Fix dma_undo's doc, which still says "counted against MAX_GRANT_TOTAL". Add the fragmentation to the PR's "Not done" list. The body's "this is also the per-claim bound, and the separate sum is gone" overstates it; that is the orchestrator's correction to make.
2. issues/a-restarted-display-holder-lends-around-its-predecessors-scanouts.md:13 and issues/a-lent-scanout-is-one-contiguous-run.md:20 cite Bound::lend_room, which this branch deletes. These records are directly affected and must stay accurate. Point them at kernel::claimspace::Window::lend_room. In the second record, the first-fit placement it describes now covers the grant part too; one clause is enough.

Landing conditions (evidence, no code change)

• Guest run 42023736614 (usbd_drives_the_spare, nvme_disk_boots, virt_claim_lpi) was still in_progress at f6dc7d0 when I checked. These rows are the only execution of the new place/map_at/map_again paths and of the 288 MiB create on VT-d and SMMUv3.
• The deletion of issues/the-hdmi-audio-function-1002-1637-is-driven-at-no-address-width.md, and hda now driving 1002:1637, depend on the AMD testcases boot. That boot has to show 03:00.1's codec walked with no DMA FAULT. The 4 GiB reasoning is sound by inspection, since the function keeps bits 0 to 39. But the record's exit is that observation, and this is new hardware behaviour. Do not land the record's deletion before that log is in evidence.

Suggestions

• kernel/pure/iova.rs. Window::reserve now has exactly one caller and runs once per domain (domains.rs:58). The hand-out-once cursor, the multi-call model test and the_amd_benchs_layout's drain all exercise behaviour nothing uses. The module could become "the lowest allowed run of room below 4 GiB" plus handed_out, which would cut both code and tests.
• system/hda/driver/src/controller.rs:374,543. Gcap::wide was restored only for two checks against device addresses at or above 4 GiB. The ABI now promises the kernel never gives one, so both checks are always false. Deleting them, and wide with them, finishes the width removal.
• system/usbd/xhci/src/tests.rs, a_controller_without_ac64_is_driven. Nothing reads AC64 any more, so this test cannot red. Delete it, and the stub's ac64 field if nothing else uses it.
• kernel/pure/claimspace.rs:220-226. The test's doc comment is one run-on sentence restating the assertions. Two short lines would do. In note_fault, the second paragraph repeats the claimed field's doc.
• Late-write test. The PR's "an smmu-selftest arm … could hold it later" is either intent, so record it in issues/toyos-runs-on-arm64.md, or drop it. Inspection of note_fault is enough for this change.

Sighting (pre-existing, not introduced here)

dma_unmap does not add the lend address it frees to bound.residue, and tear_down makes residue only from current grants. Sequence: claim A unmaps a lend at X, A is released without a reset, then claim B lends a region at X. A stale transfer from A's time can land in B's lent region, which another process may read. That breaks the module header's guarantee for lends. main behaves the same way. It is worth recording; it does not need to be fixed here.

SEND BACK

[Japabu]

Review r2 of #1188 at 83ddb42 (changes since f6dc7d0)

Architecture

The architecture is unchanged since r1, which accepted it. This round only fixes text and deletes code: 10 files, +59/−73. Across the whole PR the diff is now +478/−739.

r1's required findings

1. The grant bound is now described correctly. All three sites match the code.
   • pcidev::dma_alloc's comment says first fit in the grant part, refused where no run of span is free.
   • device_dma_alloc's doc says ResourceExhausted where no run of that length is free in the claim's 32 MiB.
   • dma_undo's doc now says "holding room in its grant part".
   • The comment's "taken back out of order" is the only way to get fragmentation. dma_undo is the only path that takes a grant back (syscall/device.rs:315). Residue does not block a grant, because a grant may be placed over residue.
   • The fragmentation example is in the PR body's "Not done".
2. The stale citations are fixed. Both records now point to kernel::claimspace::Window::lend_room. No Bound::lend_room is left in the tree.

The deletions

All of them remove dead code only:

• hda Gcap::wide and Beyond32Bits. Every address hda programs comes from the claim. CORB, RIRB and BDL come from mem.device_addr, and the stream ring is lent. The claim's window lies below 4 GiB (iova::Window::new clips there, as r1 verified), so neither check could fire.
  • Reset still carries gcap, which Stream::open uses for output_streams and input_streams.
  • Link::open keeps &Reset as an ordering witness, named _brought_up and documented as one. That is a reasonable way to keep the type-level precondition.
• xHCI ac64. Nothing in the driver reads AC64. The stub now reports AC64 set as a constant. The test had no way to fail.
• The note_fault paragraph and the claimspace test doc. Both are trimmed as suggested. The new two-line test doc is accurate: live holds lends as well as grants, and tear_down makes residue of both at a non-reset release.
• The smmu-selftest line. The body now cites issues/toyos-runs-on-arm64.md, which already records the owed smmu-selftest arm on main (lines 393–398). No new promise is made.

The new record issues/a-lend-taken-back-before-a-release-is-not-residue.md

• The frontmatter is valid: open, defect, [security], dated.
• security is the right flag. The fault is a stale DMA write into memory another process may still read, with no second bug needed.
• The scenario matches the code:
  • dma_unmap (pcidev/mod.rs:1931) records nothing.
  • tear_down's non-reset arm makes residue only from grants plus bound.residue.
  • lend_room avoids only residue and live grants.
  • The guarantee it breaks is the module header's "across claims, only once the function was reset and is quiet, or in a grant's fresh pages".
• The record has an owner and an exit. The exit names a host test that must red without the fix.
• The exit keeps the lend reusable within the same claim, which is the right detail. Without it, a holder that lends and takes back repeatedly would exhaust its own window.

Evidence

• --ci host at 83ddb42, exit 0. This is enough for a round that changes text and removes code and tests.
  • The f6dc7d0 image builds, guest run 42023736614 and both metal boots still stand. The deletions only remove checks that could not fire, so they change no behaviour on any target.
  • The claimspace controls from r1 stand too. The file changed only in a doc comment.
• AMD. 03:00.1 [1002:1637] is placed in a domain ending at 0x100000000, with its grant at 0x200000. Its codec walks, it is released by D3hot, and the log has 0 DMA FAULT lines. That meets the deleted 1002:1637 record's exit, which was r1's landing condition.
• The T14 i219 red is independent of this PR.
  • The failure matches the existing record issues/i219-msi-passes-only-when-the-network-sends-it-a-frame.md exactly. Main shows the same signature on 2026-10-21 and 2026-10-24: the link comes up, 65 frames are sent and written back, and the MAC's pre-DMA counters read "saw 0 frame(s), took 0 good, missed 0".
  • This boot repeats that signature: link up after 2.6 s, 65 frames sent and written back, the MAC saw 0, and no DMA FAULT.
  • The MAC counts a frame before any descriptor or DMA is involved. So placing receive buffers in the new window cannot cause a zero count. A placement defect would show frames counted by the MAC with no write-back, or a unit fault.
  • The transmit side shows the new window's grants work on this function.
  • One correction: the governing record is that issue file, not A claimed function interrupts at most once per read of its record #1193. A claimed function interrupts at most once per read of its record #1193's sighting is a different one, the ring that never fills.

Findings

No blockers.

SUGGESTION

• issues/a-lend-taken-back-before-a-release-is-not-residue.md:32: "on the small-kernel track" names no file. issues/README.md asks for the file, here issues/every-driver-is-still-in-the-kernel.md.
• issues/a-lend-taken-back-before-a-release-is-not-residue.md:21-22: "any function with no FLR and No_Soft_Reset set" is narrower than the real condition. A function with no PM capability, or one that is not in D0, is also released unreset (Declined::Absent and Declined::NotInD0). "Any function no reset mechanism covers" would be accurate.

For the orchestrator (PR body and evidence, not the branch)

• The i219 paragraph should cite issues/i219-msi-passes-only-when-the-network-sends-it-a-frame.md rather than leaving "a silent link partner on the bench" as its own inference.
• The "Checks" bullet says the claimspace file "has not changed since" a0d5cd8. It has, in a doc comment only, so the results still hold, but the sentence should say so.
• The r3 evidence README says the machine model is masked, but metal-judge-t14.log prints the laptop's model and BIOS string. That is not an identifying shape under build/sourcegate.rs, but the README's statement is false.

LAND