Skip to content

Stop virtio devices from blocking guest memory layout changes - #519

Draft
Lencerf wants to merge 1 commit into
google:mainfrom
Lencerf:fix/virtio/mem-deadlock
Draft

Lencerf wants to merge 1 commit into
google:mainfrom
Lencerf:fix/virtio/mem-deadlock

Conversation

@Lencerf

@Lencerf Lencerf commented Oct 5, 2026 •

Copy link
Copy Markdown
Collaborator

A virtio worker holds RamBus::lock_layout() for as long as its device runs,
while any change of the guest RAM layout takes the write lock. A change while a
device runs therefore blocks until that device resets. For example, a vCPU
hangs when the guest enables the DAX window BAR of a virtio-fs device while a
virtio-blk device serves the root filesystem, and so does hot-plugging a device
with such a BAR.

Turn RamBus into an RCU cell of Ram snapshots:

  • A reader calls RamBus::load() to get an Arc<Ram> snapshot, and holds the
    read lock only to clone the Arc.
  • A writer calls RamBus::update(), which copies the layout, changes the copy
    and publishes it. A failed update publishes nothing, and Memory changes all
    RAM ranges of a region in one update.

A snapshot holds the ArcMemPages it maps, so the host mappings stay valid as
long as anyone uses the snapshot, even after they are removed from the bus.

A running worker keeps using the snapshot taken when its device was activated,
so it does not see later layout changes yet. Follow-up changes let virtqueues
own their snapshot and move to new layouts between events, wake idle workers
after an update, and let the vhost-user backend accept memory changes while the
device runs.

The commit passes cargo +nightly fmt --check, cargo clippy --all-targets -D warnings on x86_64-unknown-linux-gnu/musl, aarch64-unknown-linux-gnu and
aarch64-apple-darwin, and cargo test --workspace locally.

@coveralls

coveralls commented Oct 5, 2026 •

Copy link
Copy Markdown

Coverage Report for CI Build 42779402185

Coverage increased (+0.003%) to 41.758%

Details

  • Coverage increased (+0.003%) from the base build.
  • Patch coverage: 17 uncovered changes across 3 files (41 of 58 lines covered, 70.69%).
  • No coverage regressions found.

Uncovered Changes

File Changed Covered %
alioth/src/mem/mem.rs 34 20 58.82%
alioth/src/virtio/vu/backend.rs 2 0 0.0%
alioth/src/cpu/cpu_amd64/sev.rs 1 0 0.0%
Total (5 files) 58 41 70.69%

Coverage Regressions

No coverage regressions found.


Coverage Stats

Coverage Status
Relevant Lines: 16429
Covered Lines: 6644
Line Coverage: 40.44%
Relevant Branches: 1740
Covered Branches: 943
Branch Coverage: 54.2%
Branches in Coverage %: Yes
Coverage Strength: 901.05 hits per line

💛 - Coveralls

@Lencerf
Lencerf force-pushed the fix/virtio/mem-deadlock branch from 559998d to 5d94d21 Compare October 5, 2026 07:53
@Lencerf
Lencerf force-pushed the fix/virtio/mem-deadlock branch from 5d94d21 to 37ac48e Compare October 5, 2026 08:02
@Lencerf
Lencerf marked this pull request as draft October 5, 2026 08:20
A virtio worker holds RamBus::lock_layout() for as long as its device
runs, while any change of the guest RAM layout takes the write lock. A
change while a device runs therefore blocks until that device resets.
For example, a vCPU hangs when the guest enables the DAX window BAR of
a virtio-fs device while a virtio-blk device serves the root
filesystem, and so does hot-plugging a device with such a BAR.

Borrowing the read-copy-update idea of Linux RCU,

- a reader calls RamBus::load() to get an Arc<Ram> snapshot, and holds
  the read lock only to clone the Arc;
- a writer calls RamBus::update(), which copies the layout, changes the
  copy and publishes it. A failed update publishes nothing, and Memory
  changes all RAM ranges of a region in one update.

A snapshot holds the ArcMemPages it maps, so the host mappings stay
valid as long as anyone uses the snapshot, even after they are removed
from the bus.

A running worker keeps using the snapshot taken when its device was
activated, so it does not see later layout changes yet.

Signed-off-by: Changyuan Lyu <changyuanl@google.com>
@Lencerf
Lencerf force-pushed the fix/virtio/mem-deadlock branch from 37ac48e to 030b1b4 Compare November 10, 2026 08:01
@Lencerf Lencerf changed the title Fix memory layout deadlock caused by virtio devices Stop virtio devices from blocking guest memory layout changes Nov 10, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants