Skip to content

sysupgrade: carry the settings on the split NOR layout after the hand-off - #2677

Merged
widgetii merged 2 commits into
masterfrom
sysupgrade/split-settings-handoff
Oct 24, 2026
Merged

widgetii merged 2 commits into
masterfrom
sysupgrade/split-settings-handoff

Conversation

@widgetii

Copy link
Copy Markdown
Member

Every split NOR upgrade since #2648 (2026-10-21) has reset the camera's settings. This was reported on a gk7205v300 upgrading to nightly-20261022 and nightly-20261023.

Cause

  1. sysupgrade: hand PID 1 off for every rootfs write, not only on UBI #2648 hands PID 1 to the RAM root for every NOR rootfs write.
  2. In the second phase, release_old_root runs umount -l /mnt, which removes the old root and the jffs2 overlay with it.
  3. do_update_firmware_split then finds no rootfs_data in /proc/mounts. It treats the camera as having no settings, stages an empty blob, and still prints "Settings staged".
  4. On the first boot, /init handles the empty blob as -n and erases the overlay.

Fix

  • First phase: records that the overlay was mounted (split_overlay_mounted, exported into the RAM phase).
  • RAM phase: when that mount is gone, mounts rootfs_data read-only and copies the settings from it, as rewrite_ubifs already does on UBI.
  • Mount fails: the run stops before anything is written, so the camera reboots unchanged rather than on defaults.
  • Unchanged: -n, and an overlay that was never mounted (tmpfs), still stage an empty blob.

Cameras fetch sysupgrade from master when they upgrade (self_update), so this fixes upgrades to any build once it is merged. Settings already lost cannot be recovered: the first boot erased them.

Verification

  • test_sysupgrade.sh: all 336 checks pass. New checks cover the split layout's handed-off RAM phase:

    • the settings are carried;
    • an overlay that won't mount stops the run with nothing written;
    • -n stays empty;
    • the first phase records the mount.

    Run against master's sysupgrade, the first check reproduces the field result: blob length 000000000000.

  • QEMU (qemu-hisilicon hi3516ev300, split layout): a boot script in the jffs2 overlay writes a marker and runs sysupgrade -z --archive with the hi3516ev300 split package. Both runs hand PID 1 off.

    • master: Clearing settings after an upgrade..., and the overlay's script is gone after the reboot.
    • this branch: Restoring settings carried over by an upgrade..., and the script runs again from the carried overlay.

Not run on hardware.

…-off

Since #2648 every NOR rootfs write hands PID 1 to the RAM root, and the
second phase's release_old_root lazily unmounts /mnt -- the old root and
the jffs2 overlay under it. do_update_firmware_split then looked for
rootfs_data in /proc/mounts, found nothing, took the camera for one
without settings and staged an empty blob, still printing "Settings
staged". The first boot honours an empty blob by erasing the overlay:
every split NOR upgrade since 2026-10-21 came back on defaults
(reported on a gk7205v300, nightly-20261022 and -20261023).

The first phase now records that the overlay was mounted
(split_overlay_mounted, exported into the RAM phase). Finding it gone,
the RAM phase mounts rootfs_data read-only and copies the settings from
that, as rewrite_ubifs does on UBI. Nothing holds the partition by then.
If that mount fails, the run stops before the write: a reboot with
nothing written, not an upgrade to defaults. -n and an overlay that was
never mounted (a tmpfs one) still stage an empty blob.

Tests: test_sysupgrade.sh (336 checks pass) gains the handed-off RAM
phase on the split layout: the settings are carried, an unmountable
overlay refuses before writing, -n stays empty, and the first phase
records the mount. Without this change the first of those reproduces
the field result exactly (blob length 000000000000).

QEMU (qemu-hisilicon hi3516ev300, split layout, OpenIPC U-Boot): a boot
script in the jffs2 overlay writes a marker and runs sysupgrade -z
--archive with the hi3516ev300 split package; both runs hand PID 1 off.
- master's sysupgrade: "Clearing settings after an upgrade...", and the
  script (part of the overlay) is gone after the reboot;
- this one: "Restoring settings carried over by an upgrade...", and the
  script runs again from the carried overlay.
Not run on hardware.
@qodo-free-for-open-source-projects

Copy link
Copy Markdown

PR Summary by Qodo

Preserve split NOR settings after the RAM handoff

🐞 Bug fix 🧪 Tests 🕐 20-40 Minutes

Grey Divider

AI Description

• Preserve mounted-overlay state across the RAM handoff so split NOR upgrades retain settings.
• Refuse to flash if the detached overlay cannot be read; preserve -n behavior.
• Test settings preservation, read failures, -n, and handoff state.
Diagram

graph TD
A["Preflight check"] --> B["Export overlay flag"] --> C["RAM handoff"] --> D["Detach old root"] --> E{"Carry settings?"}
E -->|"yes"| F["Read overlay"] --> G["Stage and flash"]
E -->|"no or -n"| G
F -->|"read fails"| H["Abort before write"]
Loading
High-Level Assessment

The following are alternative approaches to this PR:

1. Archive settings before the handoff
  • ➕ Avoids remounting JFFS2 after detaching the old root.
  • ➖ Keeps the settings archive in RAM throughout the handoff and requires carrying the staged artifact into the flashing phase.

Recommendation: Keep the proposed approach: carry only the overlay-presence flag across the handoff, then read settings immediately before flashing. It fits the existing RAM-phase staging flow and makes an unreadable overlay a pre-write failure.

Files changed (2) +93 / -4

Bug fix (1) +32 / -4
sysupgradeCarry split NOR overlay state into RAM and recover settings +32/-4

Carry split NOR overlay state into RAM and recover settings

• Records and exports whether rootfs_data was mounted before the handoff. If that mount has disappeared, the RAM phase mounts it read-only to stage settings and refuses to flash if it cannot read them; '-n' still stages an empty blob. Bumps the script version to 1.0.77.

general/overlay/usr/sbin/sysupgrade

Tests (1) +61 / -0
test_sysupgrade.shCover split NOR settings after the handoff +61/-0

Cover split NOR settings after the handoff

• Adds a read-only JFFS2 mount stub and split-layout RAM-phase checks for preserved settings, mount failure before writing, and unchanged '-n' behavior. Also verifies that the first phase exports the overlay-mounted flag.

.github/scripts/test_sysupgrade.sh

@qodo-free-for-open-source-projects

qodo-free-for-open-source-projects Bot commented Oct 24, 2026 •

Copy link
Copy Markdown

Code Review by Qodo

🐞 Bugs (0) 📘 Rule violations (1) 📎 Requirement gaps (0) 🎨 UX issues (0) 🔗 Cross-repo conflicts (0) 📜 Skill insights (0)

Grey Divider


Action required

1. Camera changes lack real-camera proof 📘 Rule violation ≡ Correctness
Description
do_update_firmware_split changes the split NOR upgrade path to mount and copy the overlay, but the
PR reports no real-camera run for that behavior. Because this code changes what a camera does during
a split NOR upgrade, the missing board output leaves the shipped behavior unverified even though the
shell tests and QEMU checks pass.
Code

general/overlay/usr/sbin/sysupgrade[R965-969]

+	if [ "1" != "$clear_overlay" ] && [ -z "$mp" ] && [ "1" = "$split_overlay_mounted" ]; then
+		mp=/tmp/overlay.old
+		mkdir -p "$mp" &&
+			mount -t jffs2 -o ro "$(get_device rootfs_data | sed 's|/dev/mtd|/dev/mtdblock|')" "$mp" 2>&3 ||
+			die "Could not read the settings to carry them over. Nothing was written; -n upgrades without them."
Evidence
Compliance ID 3 requires real-camera evidence for changes that can alter firmware behavior and
explicitly treats stating that hardware testing was not performed as a failure. The changed upgrade
logic mounts rootfs_data and copies settings during the handed-off split NOR upgrade, directly
changing camera behavior.

Rule 3: Hardware evidence is present and honest
general/overlay/usr/sbin/sysupgrade[965-969]

Agent prompt
The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

## Issue description
The PR changes camera firmware behavior during split NOR upgrades but explicitly states that it was not tested on hardware. The compliance checklist requires symptom-focused before/after evidence from an affected real camera for firmware behavior changes.
## Fix Focus Areas
- general/overlay/usr/sbin/sysupgrade[965-969]
## Recommended Fix
Run the split NOR upgrade on an affected real camera, recording the board model and before/after evidence that settings survive the handed-off RAM phase and that an unreadable overlay refuses the write. Add those results to the PR verification section, including the observed output or logs.

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools


2. A stuck overlay stays open during flashing ✓ Resolved
Description
do_update_firmware_split ignores failure from busybox umount "$mp" after archiving the remounted
settings. If that unmount fails, the function still overwrites the split firmware and settings while
the old rootfs_data filesystem remains mounted.
Code

general/overlay/usr/sbin/sysupgrade[976]

+			busybox umount "$mp" 2>&3
Evidence
The new cleanup performs an unchecked unmount, and the same function subsequently calls
nor_write_at for both the image and settings blob. The existing handoff path treats inability to
release the old root as fatal before writing.

general/overlay/usr/sbin/sysupgrade[965-978]
general/overlay/usr/sbin/sysupgrade[989-1000]
general/overlay/usr/sbin/sysupgrade[2467-2480]

Agent prompt
The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

## Issue description
The temporary JFFS2 mount can remain active if unmount fails, but the upgrade continues to overwrite its underlying flash.
## Fix Focus Areas
- general/overlay/usr/sbin/sysupgrade[975-978]
- general/overlay/usr/sbin/sysupgrade[989-1000]
## Recommended Fix
Check the temporary mount's unmount result and call `die` before marking flash dirty or starting either write if it cannot be released. Add a test that injects an unmount failure and verifies that nothing is written.

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools



Remediation recommended

3. Tests pass without recovering settings ✓ Resolved
Description
The new JFFS2 mount stub creates etc/marker in the target directory instead of obtaining settings
from the simulated rootfs_data partition. When the handed-off test checks the staged blob for that
marker, it can pass even if the upgrade mounts the wrong device or never recovers the camera's
original settings.
Code

.github/scripts/test_sysupgrade.sh[R397-398]

+        mkdir -p "$target/etc"; echo settings > "$target/etc/marker"
+        exit 0 ;;
Evidence
split_env places the original marker in the sandbox overlay, while the new stub independently
creates a marker at the mount target. The handed-off assertion checks only for the marker supplied
by that stub.

.github/scripts/test_sysupgrade.sh[3337-3352]
.github/scripts/test_sysupgrade.sh[391-398]
.github/scripts/test_sysupgrade.sh[3398-3413]

Agent prompt
The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

## Issue description
The handed-off settings test validates a marker fabricated by its mount stub, not settings recovered from the simulated partition.
## Fix Focus Areas
- .github/scripts/test_sysupgrade.sh[391-398]
- .github/scripts/test_sysupgrade.sh[3398-3413]
## Recommended Fix
Seed distinguishable settings in the simulated rootfs_data device, have the JFFS2 mount stub validate the requested device and expose those seeded settings, and assert that the original contents appear in the staged blob.

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools


Grey Divider

Tip of the day
💡 Did you know, you can choose which labels appear on a finding, and whether they show icons or text

More tips ↗ | Customize Qodo ↗ | Qodo docs ↗

Grey Divider

Qodo Logo

Comment thread general/overlay/usr/sbin/sysupgrade Resolved
Comment thread general/overlay/usr/sbin/sysupgrade Resolved
Comment thread .github/scripts/test_sysupgrade.sh Resolved
…et go

The temporary read-only mount of rootfs_data has to be released before
the flash under it is rewritten; if busybox umount fails, die before
the write instead of going on.

The jffs2 mount stub now serves files from a directory named for the
device it is asked to mount, so the handed-off test checks that the
settings come from rootfs_data (mtdblock5) rather than from the stub.
A new test injects the unmount failure: refused, nothing written.
test_sysupgrade.sh: 337 checks pass.
@widgetii
widgetii merged commit 063b452 into master Oct 24, 2026
143 checks passed
@widgetii
widgetii deleted the sysupgrade/split-settings-handoff branch October 24, 2026 20:16
widgetii added a commit to OpenIPC/wiki that referenced this pull request Oct 27, 2026
…s last rounds (#631)

* nor-split, sysupgrade, install pages: catch up with the split layout's last rounds

- nor-split-layout: the 3.0 and 3.4 kernels without a device tree (the
  Hi3516CV100 family, Hi3520DV200, GK7102/GK7102S) find the layout from
  the images and log the offsets; the fixed-slot package note covers
  every SoC that ships both packages; the settings sysupgrade lost on
  split upgrades from 2026-10-21 to 10-24 (OpenIPC/firmware#2677) and
  the split upgrade on 3.x kernels without sysfs partition offsets
  (#2683); a "Recovery from U-Boot" section: fwupd takes
  firmware.bin.<soc>, else the fixed-slot pair, and a split kernel that
  fails its CRC starts recovery by itself.
- sysupgrade: points a camera that came back on defaults then to that note.
- install-goke: the split layout does not need the OpenIPC U-Boot; any
  U-Boot that loads the kernel from 0x50000 boots it, as nor-split-layout
  already says.
- install-novatek: OpenIPC U-Boot builds for NT98562 and NT98566 are
  published since 2026-10-27, QEMU only so far.

* nor-split: recovery needs a split-mode U-Boot, a pair to fall back on, and erases the settings

Review of #631: the recovery section names the U-Boot builds that have
it (from OpenIPC/u-boot#77 on the Hi3516CV100 family and Hi3520DV200,
#87 on GK7102/GK7102S; earlier ones install only the fixed-slot pair),
offers the fixed-slot fallback only where the SoC ships that package
(the split-only ones need firmware.bin on the server), and warns that
fwupd, by hand or after a failed kernel CRC, erases the settings.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant