Skip to content

load_novatek: refuse to load HDAL over the kernel's own memory - #2657

Merged
widgetii merged 2 commits into
masterfrom
novatek-mem-guard
Oct 22, 2026
Merged

widgetii merged 2 commits into
masterfrom
novatek-mem-guard

Conversation

@widgetii

Copy link
Copy Markdown
Member

Problem

On NT9856x boards the vendor U-Boot lays memory out from the osmem environment variable: the DT memory node it passes ends at osmem and the HDAL video pool starts there. The kernel takes its RAM from mem= in bootargs alone, and the bootcmd these boards were given re-runs the saved bootargs, so the two values drift apart once one is changed without the other (#2107).

When mem= is larger than osmem, the kernel owns the start of the HDAL pool. The first HDAL module to initialise its heap writes over kernel memory, oopses in nvtmpp_heap_init, and the camera panics, then reboots and panics again on every boot.

load_novatek now compares mem= with the DT memory node before loading anything. On a mismatch it logs why, skips the HDAL modules, and writes the size U-Boot actually used back into bootargs, so the next boot is consistent.

The opposite drift, osmem larger than mem=, hangs the kernel before the console, so nothing in userspace can catch it. That case is documented in the wiki instead (follow-up PR there).

This isn't a toolchain problem. The 2022 gcc-7.5 kernel, a local gcc-13.3 build, and the 2026-10-21 nightly all boot at mem=54M when osmem matches.

Hardware tested on

NT98566 + SC501AI, 128 MB, XM vendor U-Boot 2019.04, 2026-10-21 nightly. I tested the new load_novatek placed in the overlay; it was removed afterwards. Serial console, observed directly.

Evidence

Before: stock load_novatek, saved mem=54M with osmem=32M:

@fdt_fixup_memory_banks(), ddrStart: 0x0, ddrAll: 0x8000000, hdalStart: 0x2000000, hdalSize: 0x6000000, size[0]: 0x2000000.
Kernel command line: earlyprintk console=ttyS0,115200 mem=54M panic=20 nprofile_irq_duration=on root=/dev/mtdblock3 rootfstype=squashfs init=/init ...
Unable to handle kernel NULL pointer dereference at virtual address 00000000
Internal error: Oops: 805 [#1] PREEMPT ARM
Modules linked in: kflow_common(O+) kdrv_videoprocess(O) kdrv_videocapture(O) kdrv_gfx2d(O) kdrv_comm(O) kdrv_builtin(O) nvtmem(PO) kwrap(O)
PC is at mmioset+0x30/0xa8
[<804adc90>] (mmioset) from [<7f1faff4>] (nvtmpp_heap_init+0xcc/0x284 [kflow_common])
Kernel panic - not syncing: Fatal exception
Rebooting in 20 seconds..

After: this load_novatek, same environment:

@fdt_fixup_memory_banks(), ddrStart: 0x0, ddrAll: 0x8000000, hdalStart: 0x2000000, hdalSize: 0x6000000, size[0]: 0x2000000.
Kernel command line: earlyprintk console=ttyS0,115200 mem=54M panic=20 nprofile_irq_duration=on root=/dev/mtdblock3 rootfstype=squashfs init=/init ...
novatek: Kernel booted with mem=54M, past the 32M U-Boot laid out from osmem; HDAL would overwrite kernel memory, not loading it
novatek: bootargs set to mem=32M; reboot to load HDAL
Starting majestic: OK
nt98566-sc501ai login: 

The next reboot comes up with mem=32M, all 27 HDAL modules loaded, and the encoder at 25 fps (VIDEOENC USER WORK: PULL 25 REL 25). In the normal state (mem=32M, osmem=32M) the check reads dt_mem=32 booted_mem=32 and returns without doing anything.

Local make BOARD=nt98566_lite:

- uImage: [1902KB/2048KB]
- rootfs.squashfs: [4804KB/5120KB]

STRICT=1 test_shell_parse.sh, STRICT=1 test_strip_shell_comments.sh and ci-matrix.py --self-test pass. The selector builds nt98562_lite and nt98566_lite.

Scope

  • No kernel patches under general/package/all-patches/linux/ (those go to OpenIPC/linux)
  • No files specific to a single retail camera model (those go to OpenIPC/builder)
  • No probing or bring-up tooling (that goes to OpenIPC/ipctool)
  • Nothing under general/overlay/ or in a shared load_<vendor> script hardcodes a value specific to my board
  • Package sources come from an OpenIPC repository, and any version bump keeps at least the specificity of the pin it replaces (a new package should pin a full 40-character SHA)
  • No LD_PRELOAD, and no binaries that cannot be rebuilt from source
  • New code is selected by a defconfig, so CI actually builds it

The vendor U-Boot lays memory out from osmem (DT memory node up to it, HDAL
pool from it), while the kernel takes its RAM from mem= alone. The bootcmd the
wiki gives these boards re-runs the saved bootargs, so the two drift apart as
soon as one is changed without the other. Booted with mem= past the node, the
kernel owns the start of the HDAL pool and nvtmpp_heap_init in kflow_common
oopses (mmioset, NULL dereference) and panics the camera, every boot.

Compare mem= with the DT memory node first. When it runs past it, say so, skip
the HDAL modules, and write the size U-Boot used back into bootargs so the next
boot is consistent. Proven on an NT98566 + SC501AI: saved mem=54M with
osmem=32M boots without the panic and heals on the next reboot.
@qodo-free-for-open-source-projects

Copy link
Copy Markdown

PR Summary by Qodo

Guard Novatek HDAL loading against overlapping kernel memory

🐞 Bug fix 🕐 10-20 Minutes

Grey Divider

AI Description

• Compare kernel mem= with the device-tree memory limit before loading HDAL modules.
• On unsafe overlap, skip HDAL and report why to avoid a kernel panic.
• Correct matching saved bootargs so HDAL can load after the next reboot.
Diagram

graph TD
  DT["DT memory reg"] --> Check["Compare RAM sizes"] --> Unsafe{"mem exceeds DT?"}
  Cmd["Kernel cmdline"] --> Check
  Unsafe -->|No| HDAL["Load HDAL modules"]
  Unsafe -->|Yes| Guard["Log and attempt repair"] --> Skip["Skip HDAL"]
  Guard -.-> Env["U-Boot bootargs"]
Loading
High-Level Assessment

The following are alternative approaches to this PR:

1. Synchronize memory settings in U-Boot
  • ➕ Could prevent both mismatch directions before the kernel starts.
  • ➖ Depends on vendor bootloader configuration across boards and cannot protect already-booted systems.

Recommendation: Keep the userspace guard as a fail-safe for the overlap it can detect. Bootloader-level synchronization would address the earlier, opposite failure too, but is a separate board-dependent change; this script cannot recover once the kernel hangs before userspace.

Files changed (1) +29 / -0

Bug fix (1) +29 / -0
load_novatekBlock HDAL loading when kernel RAM overlaps its pool +29/-0

Block HDAL loading when kernel RAM overlaps its pool

• Adds a pre-load check comparing 'mem=' in '/proc/cmdline' with the device-tree memory size. If kernel RAM extends beyond that size, it logs the risk, conditionally rewrites the saved 'bootargs' to match, and exits before loading HDAL modules.

general/package/novatek-osdrv-nt9856x/files/script/load_novatek

@qodo-free-for-open-source-projects

qodo-free-for-open-source-projects Bot commented Oct 22, 2026 •

Copy link
Copy Markdown

Code Review by Qodo

🐞 Bugs (0) 📘 Rule violations (0) 📎 Requirement gaps (0) 🎨 UX issues (0) 🔗 Cross-repo conflicts (0) 📜 Skill insights (0)

Grey Divider


Action required

1. Cameras can panic with memory last ✓ Resolved
Description
check_mem uses a space-only awk record separator and requires the mem= value to end immediately
after M, so it misses a final command-line token that retains /proc/cmdline's newline. If that
token sets memory beyond the device-tree limit, the empty result makes the guard return and
insert_ko still loads the HDAL modules.
Code

general/package/novatek-osdrv-nt9856x/files/script/load_novatek[R167-168]

+	booted_mem=$(awk -F '=' '$1=="mem" && $2 ~ /^[0-9]+M$/ {print $2+0; exit}' RS=" " /proc/cmdline)
+	[ -n "$booted_mem" ] && [ "$booted_mem" -gt "$dt_mem" ] || return 0
Evidence
The new awk command separates records only on spaces and anchors its value match at the record end.
With mem=NNM last, the newline in /proc/cmdline remains in that record; the next line treats the
missing parsed value as permission to continue to module insertion.

general/package/novatek-osdrv-nt9856x/files/script/load_novatek[163-168]
general/package/novatek-osdrv-nt9856x/files/script/load_novatek[179-185]

Agent prompt
The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

## Issue description
The memory guard misses `mem=NNM` when it is the final `/proc/cmdline` token because the token retains a newline and fails the anchored value match.
## Fix Focus Areas
- general/package/novatek-osdrv-nt9856x/files/script/load_novatek[167-168]
## Recommended Fix
Parse the command line as whitespace-separated fields, including the final newline, and apply the strict `mem=` validation to each complete field before comparing memory sizes.

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools


Grey Divider

Tip of the day
💡 Did you know, you can ask Qodo to dismiss a finding you disagree with, with your reason on record

More tips ↗ | Customize Qodo ↗ | Qodo docs ↗

Grey Divider

Qodo Logo

Comment thread general/package/novatek-osdrv-nt9856x/files/script/load_novatek Outdated
A space-separated awk record kept /proc/cmdline's trailing newline on the final argument, so a mem= placed last never matched and the guard let HDAL load over the kernel. Split the command line in the shell instead.
@widgetii
widgetii merged commit 92415b6 into master Oct 22, 2026
44 of 48 checks passed
@widgetii
widgetii deleted the novatek-mem-guard branch October 22, 2026 13:07
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant