Skip to content

sysupgrade: hand PID 1 off for every rootfs write, not only on UBI - #2648

Merged
widgetii merged 1 commit into
masterfrom
sysupgrade-nor-handoff
Oct 21, 2026
Merged

widgetii merged 1 commit into
masterfrom
sysupgrade-nor-handoff

Conversation

@widgetii

Copy link
Copy Markdown
Member

Problem

#2647. sysupgrade --rootfs on a low-memory camera kills PID 1 in the middle of the erase and leaves the rootfs half-written. A lab XM510 (mem=18M) did this in 3 of 4 runs and then boot-looped. Without a UART it would be bricked.

SQUASHFS error: Unable to read page, block 60, size cedc        (x68)
init/1: potentially unexpected fatal signal 7.
Kernel panic - not syncing: Attempted to kill init!
...
Kernel panic - not syncing: VFS: Unable to mount root fs on unknown-block(31,3)

Cause. On NOR the squashfs rootfs can be rewritten while mounted, so the PID 1 hand-off was used only on UBI, where it is required. busybox init therefore keeps running from the old root through the whole erase. free_resources has already dropped the page cache. When init next runs (to reap one of the old root's processes dying of SIGBUS), it faults its own text in from flash being wiped, takes SIGBUS, and the kernel panics.

The free-RAM floor (check_flash_ram) doesn't cover this, for two reasons:

  • what it counts as reclaimable is exactly those pages;
  • on kernels without MemAvailable (before 3.14, including XM510, XM530 and hi3516cv100) it doesn't run at all.

Change

want_handoff() makes any run that rewrites the rootfs hand PID 1 to the RAM root (busybox init ::restart:/sbin/init), as UBI layouts already do. Before the first write, init execs into RAM, every process on the old root is killed, and the old root is released (umount -l /mnt). Nothing that can fault in from the erased partition is left running.

Unlike need_handoff, this is a preference, not a requirement. The run flashes exactly as before, from a plain RAM root or in place, in any of these cases:

  • the image's inittab has no ::restart: entry;
  • no RAM root can be built (--no_ramfs, no pivot_root, staging fails);
  • the image is only reachable behind the old root, such as an SD card.

Kernel-only writes are unchanged and are flashed in place.

Trade-off: a hand-off ends SSH and the WebUI session, so a WebUI upgrade loses majestic's live progress bar. Progress goes to the HTTP log on port 80 (curl -sN http://<camera>/) and to the serial console, as on UBI cameras today.

Hardware tested on

  • XM510 + JX-H62 (xm510_lite, kernel 3.0.101, 15.6 MB MemTotal).
  • XM550 + IMX335 (xm530_lite).

Both run firmware master plus OpenIPC/linux#130 and majestic#1098. This branch's sysupgrade, comment-stripped as the build ships it, was placed on the camera:

  • XM510: in the overlay, removed afterwards;
  • XM550: run from /tmp.

Evidence

Before (XM510, sysupgrade -z -f --rootfs=/tmp/rootfs.squashfs.xm510), 3 of 4 runs:

init/1: potentially unexpected fatal signal 7.
Kernel panic - not syncing: Attempted to kill init!

Each time the board was recovered through U-Boot TFTP, with a CRC32 read-back.

After (XM510, same command, three runs in a row, UART captured):

run 1: booted after 67 s; seen: Handing PID 1 over, Flashing from RAM, RootFS updated; SQUASHFS errors on UART: 0
run 2: booted after 66 s; seen: Handing PID 1 over, Flashing from RAM, RootFS updated; SQUASHFS errors on UART: 0
run 3: booted after 70 s; seen: Handing PID 1 over, Flashing from RAM, RootFS updated; SQUASHFS errors on UART: 0
rootfs partition (image-sized prefix) md5 after each run: b5d974d1 = image b5d974d1

XM550, kernel + rootfs:

Handing PID 1 over to the RAM root.
Follow it from another machine (curl -sN, or a browser):
  http://10.216.128.90/
after: kernel #2 SMP Wed Oct 21 00:08:45, rootfs md5 80fcf3c6 = image 80fcf3c6, majestic re-armed the watchdog at boot

Tests:

  • test_sysupgrade.sh: 323 checks pass, 5 of them new:

    • NOR rootfs write hands PID 1 off before anything is written;
    • without ::restart: it still pivots into a plain RAM root;
    • an image outside /tmp isn't handed off;
    • kernel-only stays in place;
    • stage 2 on NOR releases the old root before the kernel and rootfs writes.

    The first of these fails on master.

  • STRICT=1 test_shell_parse.sh and test_strip_shell_comments.sh: clean.

Scope

  • No kernel patches under general/package/all-patches/linux/ (those go to OpenIPC/linux)
  • No files specific to a single retail camera model (those go to OpenIPC/builder)
  • No probing or bring-up tooling (that goes to OpenIPC/ipctool)
  • Nothing under general/overlay/ or in a shared load_<vendor> script hardcodes a value specific to my board
  • Package sources come from an OpenIPC repository, and any version bump keeps at least the specificity of the pin it replaces (a new package should pin a full 40-character SHA)
  • No LD_PRELOAD, and no binaries that cannot be rebuilt from source
  • New code is selected by a defconfig, so CI actually builds it

On NOR a squashfs rootfs can be rewritten while mounted, so the PID 1
hand-off was kept for UBI, where it is the only way. But busybox init then
runs from the old root through the whole erase, and free_resources has
dropped the page cache: when init next runs -- to reap one of the old
root's processes dying of SIGBUS -- it faults its own text in from flash
being wiped, takes SIGBUS itself, and "Attempted to kill init!" reboots the
camera with its rootfs half written. A lab XM510 (18 MB) did that in three
runs of four (#2647); the free-RAM floor did not catch it, since what it
counts as reclaimable is exactly those pages, and on a kernel without
MemAvailable it does not run at all.

want_handoff() makes any rootfs write hand PID 1 to the RAM root where it
can: the image's inittab has ::restart:/sbin/init, a RAM root can be built,
and the images are not behind the old root. Unlike need_handoff it is a
preference: without any of those the run flashes as it did before.
@qodo-free-for-open-source-projects

Copy link
Copy Markdown

PR Summary by Qodo

Hand PID 1 to RAM before eligible NOR rootfs writes

🐞 Bug fix 🧪 Tests 🕐 20-40 Minutes

Grey Divider

AI Description

• Prefer a PID 1 hand-off before NOR rootfs writes to prevent init faulting during erase.
• Preserve existing flashing behavior when hand-off is unavailable; leave kernel-only writes in
 place.
• Test NOR hand-off, fallback paths, and release of the old root before flashing.
Diagram

graph TD
  A{"Live flash write?"} -->|yes| B{"UBI handoff required?"} -->|no| C{"Handoff viable?"} -->|yes| D["PID 1 in RAM"] --> E["Release old root"] --> F["Flash images"]
  B -->|yes| D
  C -->|no| G["Plain RAM or in place"] --> F
  A -->|no| F
Loading
High-Level Assessment

The following are alternative approaches to this PR:

1. Require hand-off for NOR rootfs writes
  • ➕ Would prevent a NOR rootfs erase while PID 1 still runs from the old root.
  • ➖ Would refuse upgrades on images without init restart support or a usable RAM root, and when images cannot survive old-root release.

Recommendation: The optional hand-off is a compatibility-conscious fix: it removes the observed hazard when staging is possible without blocking previously supported upgrades. Reviewers should explicitly accept that fallback NOR writes retain the original PID 1 risk.

Files changed (2) +99 / -0

Bug fix (1) +33 / -0
sysupgradePrefer PID 1 hand-off for rootfs writes +33/-0

Prefer PID 1 hand-off for rootfs writes

• Adds an eligibility check for optional hand-off when rewriting the rootfs and invokes the existing RAM-root hand-off path. Mandatory UBI behavior remains separate; unsuccessful optional staging falls back to the existing flash path.

general/overlay/usr/sbin/sysupgrade

Tests (1) +66 / -0
test_sysupgrade.shCover NOR hand-off and fallback behavior +66/-0

Cover NOR hand-off and fallback behavior

• Adds five checks for hand-off before writing, plain-RAM fallback without an init restart entry or reachable image, kernel-only in-place flashing, and old-root release before stage-two writes.

.github/scripts/test_sysupgrade.sh

@qodo-free-for-open-source-projects

Copy link
Copy Markdown

Code Review by Qodo

🐞 Bugs (0) 📘 Rule violations (0) 📎 Requirement gaps (0)

Grey Divider

Great, no issues found!

Qodo reviewed your code and found no material issues that require review

Grey Divider

Tip of the day
💡 Did you know, you can start a comment with 'qodo' or '@qodo' to chat about any finding

More tips ↗ | Customize Qodo ↗ | Qodo docs ↗

Grey Divider

Qodo Logo

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant