[bsek]

aarch64 had no KrnIsSuper() and used the generic one, which always
returns FALSE. Enable() in an interrupt handler therefore unmasked IRQ
and FIQ, and so did Signal(), which does its own Disable()/Enable() on
SMP. A handler could nest onto itself or onto a spinlock it held.

• KrnIsSuper() now reads SPSel: tasks run at EL1t, handlers and
  SuperState() at EL1h.
• SuperState() returns NULL when already at EL1h. SC_USERSTATE sets the
  IRQ/FIQ mask from the Disable() nesting instead of always unmasking.
• Every supervisor check in exec now applies at EL1h, as on m68k and
  x86: Permit() does not reschedule there, AllocMem() takes its
  supervisor path, and CheckSemaphore() alerts on semaphore calls from
  a handler or inside SuperState().
• New test SMP-IntCtx: a VBlank server checks that KrnIsSuper() is set,
  that it is entered masked, and that Disable()/Enable() and Signal()
  leave the mask alone. The task then checks nested SuperState() and
  the mask after UserState(). Skipped where SuperState() has no
  supervisor mode to enter. Expected to fail on arm-native, whose
  SuperState() has the same nesting bug.
• SMP-Stress: the softint worker waits for NT_INTERRUPT before the next
  Cause(), a second softint Causes itself while running, and main
  clears its Ctrl-D/Ctrl-F worker signals so it no longer aborts the
  calling script.

Tested on Pi 3B+ (4 cores) and QEMU raspi3b. Not run on Pi 4/5.

[Kalamatee]

1. aarch64's own SuperState() doesn't use the new KrnIsSuper(). The generic rom/exec/superstate.c:62 returns NULL when already in supervisor mode, which is what makes nesting safe. The aarch64 override in arch/aarch64-native/exec/superstate.c issues SC_SUPERSTATE unconditionally. The SVC still works from EL1h, so a nested SuperState(), or one called from an interrupt server, returns a non-NULL SP. Its UserState() then drops the CPU to EL1t in the middle of the outer supervisor section, or in the middle of the handler on the interrupted task's stack. The fix is one line, if (KrnIsSuper()) return NULL;.

2. SC_USERSTATE now unmasks FIQ unconditionally. That fixes the case the commit names (Disable()/Enable() inside SuperState()), but SC_SUPERSTATE never touches F. A task that calls SuperState() while inside its own Disable() comes back from UserState() unmasked even though IDNESTCOUNT says disabled. This was already true for IRQ; it now covers FIQ too, and FIQ is the IPI line on aarch64 SMP. kernel_ipi.c:106 explains why IPIs must not be drained under Disable(). The kernel already reads IDNESTCOUNT_GET, so the reply proposes setting the mask from it, which handles both cases.

3. More behaviour changes than the PR describes. Every supervisor check in exec now applies at EL1h:

   • Permit() no longer reschedules there.
   • AllocMem() takes its supervisor path.
   • CheckSemaphore() Alerts and refuses semaphore calls made from a handler or inside SuperState().

   That matches m68k and x86 rather than arm-native, which deliberately returns FALSE in SYSTEM mode, and it's the right reading. But any aarch64 driver that takes a semaphore in those contexts will start alerting, so it belongs in the commit message.

4. SMP-IntCtx can corrupt the shell's stack. The interrupt server chain is walked without a lock (intservers.c:36), soRemIntServer() doesn't wait for a pass al With server and ic in main()'s stack frame, a late pass writes into the shell's stack and runs code that is being unloaded. A Delay(1) after RemIntServer() closes the window.

5. SMP-Stress fix is correct but drops coverage. Waiting for NT_INTERRUPT is the right fix, checked against cause.c's states: the old wait let a Cause() land ot is coalesced and the count comes up short.But the test now only ever Causes an idle softint, so the re-Cause-while-running path (RUNNING → PENDING) is no longer tested.

Minor: the arm-native 32-bit Pi could get thCPSR.

[bsek]

Thanks, all five were right. Updated:

1. SuperState() returns NULL when KrnIsSuper() is set.
2. SC_USERSTATE sets the IRQ/FIQ mask from IDNESTCOUNT instead of
   always unmasking.
3. The commit message now lists the exec behaviour changes at EL1h
   (Permit(), AllocMem(), CheckSemaphore()).
4. SMP-IntCtx waits a tick after RemIntServer().
5. SMP-Stress has a second softint that Causes itself twice while
   running and must run exactly once per request, so RUNNING -> PENDING
   and the coalesced Cause() are both covered.

SMP-IntCtx also checks 1 and 2 from the task now: a nested SuperState()
must return NULL, and UserState() must come back masked under Disable()
and unmasked otherwise. Both checks fail on the previous revision and
pass on this one.

[Kalamatee]

CheckSuperState() fails wherever SuperState() cannot enter supervisor mode

The SMP tests are built for every target (#MM- test : test-smp), and the new task-side check
assumes SuperState() always gets there. It does not everywhere:

• Hosted, the generic rom/exec/superstate.c returns NULL outside
  AROS_FLAVOUR_STANDALONE, and the all-unix KrnIsSuper() is the signal handler depth, 0 in a
  task.
• arch/riscv64-opensbi/exec/superstate.c returns NULL unconditionally, everything already
  running in S-mode, and its KrnIsSuper() is the trap depth.

Both report bit 1 ("not supervisor") and SMP-IntCtx fails there with nothing wrong. Called from
a task, a NULL SuperState() can only mean there is no supervisor mode to enter, so that is the
case to skip, the same way the mask checks are skipped where there is no reader:

    ssp = SuperState();
    if (!ssp)
        return -1;      /* no supervisor mode to enter */

    super = CheckSuperState();
    if (super < 0)
    {
        bug("[smpintctx] SuperState() has no supervisor mode here, skipped\n");
        super = 0;
    }
    else
        bug("[smpintctx] SuperState(): not supervisor %d, ...", ...);

arm-native will fail it for real

arch/arm-native/exec/superstate.c issues SC_SUPERSTATE unconditionally, as aarch64's did, so
a nested SuperState() there also returns a non-NULL SP and its UserState() drops back to user
mode inside the outer section. It cannot take the one-line fix used here: arm-native's
KrnIsSuper() deliberately returns FALSE in SYSTEM mode, which is where SuperState() puts it.
Bits 1 and 2 will come up on the 32-bit Pi, and that is a real bug, but whether it is fixed by
counting SYSTEM mode in KrnIsSuper() (taking on the same exec behaviour changes this PR lists
for aarch64) or by having SuperState() test CPSR itself is a decision for arm-native. Until one
of those lands the failure there is expected, which is worth saying in the commit message.

Minor

• The PR description still says SC_USERSTATE "clears the FIQ mask as well as IRQ" and does not
  mention the SuperState() change; the commit messages are current, the description is not.

[bsek]

Thanks, fixed:

• CheckSuperState() returns -1 when SuperState() gives NULL from a
  task, and the test prints "skipped" instead of failing.
• The SMP-IntCtx commit message now says the check is expected to fail
  on arm-native until its SuperState() is fixed. I have left that fix
  out of this PR.
• PR description updated.