Make configured memory-limit exhaustion optionally terminal

[yuhaouno]

Summary

Fixes #1773

• JS_SetMemoryLimitTermination() opts in (off by default) so a rejection from the engine's own JS_SetMemoryLimit check is a runtime-scoped terminal condition.
• The condition is recorded at that check, not from the exception object. It stays set when the out-of-memory Error cannot be allocated and the pending exception is null.
• JavaScript catch does not resume, and JS_ExecutePendingJob() returns failure instead of running a rejection continuation. JS_GetTerminationStatus() reports JS_TERMINATION_MEMORY_LIMIT without parsing exception text.
• Arbitrary allocator NULL, ordinary exceptions, and the previous catchable memory-limit behavior are unchanged. After termination the runtime is not reusable and should be freed.

Test plan

• api-test in Debug and ASan+UBSan, including catchable exhaustion, terminal sync (including 1-byte headroom / null), promise reaction, async resumption, promise constructor, thenable job, ordinary Error/RangeError/TypeError/syntax/rejection, custom-allocator NULL, and a fresh runtime
• lre-test in Debug and ASan+UBSan
• JavaScript suite (run-test262 -c tests.conf): 0/117 errors, 9 excluded, in Debug and ASan+UBSan
• ctest, cxxtest, and jscheck (Apple Clang needs -Wno-unused-command-line-argument for jscheck)
• JS_ABORT_ON_LEAKS via api-test; macOS leaks --atExit reported 0 leaks
• Full Test262 not run; the default path is unchanged and the new behavior is opt-in

Made with Cursor

[saghul]

What's the use case here? Can't the host app check the error and decide whether it allows the app to continue or stop it?

[josefguenther]

Use case: a host running untrusted scripts under a memory limit. The script can catch the out-of-memory error itself (try { … } catch {} around the allocation) and keep going, so the host never gets to decide. Making it terminal leaves that decision with the host, the same way an interrupt can't be caught.