feat(threads): run an isolated application in a dedicated worker thread

[kriszyp]

An application declared with isolated: true now runs in one dedicated HTTP worker that loads no other application, exposes an application-addressed secure UDS mirror, and can be restarted without rotating unrelated workers. Admission, placement, restart, expiration, and cleanup paths fail closed so an unsupported, over-budget, malformed, or exhausted application is reported instead of silently appearing on the shared listener.

For the human reviewer

1. UDS-only ingress. Dedicated workers expose only application-named secure UDS mirrors. The metadata includes the application identity and optional hosts; a host is intentionally not required because application-ID routing and direct socket consumers are supported. Joining public listeners would change the proxy and security boundary.

2. Asynchronous readiness. Pool readiness still marks the node started while dedicated applications boot, so one application cannot delay the node. A deploy can therefore finish before the dedicated UDS appears; the integration test waits on the socket itself. Making a deploy success mean “dedicated route is serving” requires a product decision about whether one app may stall a pool restart.

3. Shared-store TTL ownership. Schema and runtime TTL declared by an isolated application run where that application code runs, even for a shared store already maintained by a pool owner. This guarantees eventual cleanup when only the isolated app declares the schema, but can duplicate version-guarded scans. Central ownership needs IPC for application-defined functions/configuration. The same locality means a shared sourcedFrom cache has no dedicated low-disk reclamation owner.

4. Node-local admission. The origin rejects unreachable or over-budget isolation, while replicated peers persist desired config and may refuse locally. Concurrent deployments of different applications can both observe the final free slot; reconciliation admits one and reports the other failed. Atomic admission requires a main-thread reservation spanning configuration, installation, replication, and rollback.

5. Configuration semantics. Omitting isolated preserves an existing true value, and isolation intent is persisted before package preparation so a failed install leaves desired topology for recovery. Reads, admission, persistence, and activation for one application are now serialized by the component preparation lock. Changing failed-deploy rollback or sticky isolation is a separate API decision.

6. Restart and drop policy. Restart scope uses absent/* for all workers, empty string for the pool, and an application name for one dedicated worker; scopeFallback accepts only the pool sentinel. A drop queries live topology under the component lock and returns 503 before mutation if topology is unavailable. The core managed-worker path is covered; whether an external replication executor can invoke this without a topology channel needs validation in the enterprise wrapper.

7. Package boundary. Ordinary package entries are applications and obey isolation placement. Trusted built-ins still load everywhere, including a forced package replacement using a protected name. If forced protected-name deployment must support isolation, the loader needs a distinct built-in-versus-application contract.

8. Lifecycle fencing. The slot registry is the authoritative lease: removing a slot prevents every old generation from auto-restarting, shutdown is idempotent, and dedicated UDS replacements do not pre-start over a live fixed socket path. A start that fails or never reports ready now awaits slot.shutdown() before withdrawing the lease, so the application is never registered-as-absent while its worker is still draining; the earlier ordering let a reconcile start a replacement over the live mirror and let a drop's removeBranchesForApplication() run past a worker still holding the stores. Unlinking application sockets on every worker exit was rejected because an outgoing rolling-restart worker can share the path with its replacement and would delete the replacement's live endpoint. Cleanup therefore runs on deliberate retirement or final restart exhaustion, guarded by slot identity.

9. Restart-required is one bit, so only a restart that covers every worker clears it. restartWorkers() cleared the node's restart-required flag on every call, which was correct while every restart replaced every worker. Restart scopes broke that in two ways: an application-scoped restart leaves the whole pool on its old code, and a pool-scoped restart leaves an already-running dedicated worker on its old modules. Either one clearing the bit reports restartRequired: false while a deployed component is loaded nowhere. It is now cleared for all workers always, and for the pool only while no dedicated worker is running — which is every node that uses no isolated application, leaving the historic behavior unchanged there. The rejected alternative, clearing only on the all-workers sentinel, would stop an ordinary shared deploy_component with restart: true from ever clearing the bit (that path passes the pool scope), leaving the flag permanently on for every node. The one remaining conservative case: a dedicated worker the reconcile is about to stop is still counted, because the check reads topology before the reconcile runs. A precise answer needs a per-application signal rather than one process-wide bit.

10. Where to look hardest. The lease ordering above has no direct unit coverage: watchDedicatedStart and isolatedSlots are module-private, and asserting the ordering would mean exporting both purely for a test. The integration suite exercises the happy path and the drop paths, not a hung start racing a reconcile. The two coverage threads that were open on the last head are now closed by unit tests: the dedicated worker's uWS entry rule is named as shouldStartUwsListenerHere() and pinned across all four quadrants, and the preserved-eviction gate has a differential case. One honest limit on the latter: it reaches scheduleCleanup() through the ownsStoreMaintenance() leg, not the ttlConfiguredByApplication && isDedicatedWorker() leg a real dedicated worker takes, so it pins the scheduling gate rather than the ownership path. The per-thread rulings for every review finding on this PR are now published as a single comment rather than as review threads: the author's pending review (pullrequestreview-5184029425, 19 comments) anchors on outdated commits, so submitting it would open 19 new threads on lines that have since moved.

Implementation

• Configuration validates isolated as a boolean and threads.maxIsolated as a bounded count. Malformed values fail placement before any application import.
• The main thread owns a serialized slot registry, admits only installed and reachable applications, preserves incumbents when capacity is lowered, adjusts heap shares, reports lifecycle failures, and removes stale UDS and metadata on retirement or restart exhaustion.
• Pool workers skip isolated applications; a dedicated worker loads only its named application and presents application-relative worker identity 0/1. Scheduler, data-loader, cache-source, audit, reclamation, branch, and expiration ownership follow the defining application or store.
• Restart messages preserve historical all-worker behavior while supporting pool-only and application-only scopes. Dedicated UDS workers use shutdown-before-replacement; shared HTTP workers retain overlap where the platform supports it.
• Deploy and drop derive restart scope from live topology, serialize config/topology decisions with preparation, preserve root config and package linkage for file-only drops, and keep isolated applications off the public port through reloads.
• Isolated GraphQL and programmatic schema declarations tag their owning application. Live table redeclarations refresh derived @expiresAt state, and the isolated schema integration fixture proves physical eviction from the shared store.

Verification

• npm run build — passed on the final head.

• Prettier, oxlint, and git diff --check over every changed source/test file — passed. Repository-wide npm run lint still reports 15 warnings in files unchanged from origin/main.

• The final focused Mocha set covering activation, serialized preparation, TTL, placement, topology timeouts, UDS binding, safe-mode preloads, and deploy operations — 115 passing, 1 platform-dependent pending.

• npm run test:integration -- integrationTests/components/isolated-application.test.ts — 7 passing: placement, capacity refusal, public-port exclusion, UDS routing, shared drop, file-only isolated drop, and full isolated drop. The same 7 pass with HARPER_UWS_UDS=1, including the application-named uWS mirror.

• The schema-TTL integration assertion failed before the TTL ownership fix (rawTtlRecords remained 1) and passed after it. The stale-config capacity assertion likewise failed on the rebased pre-fix head because the second real isolated worker was absent.

• npm run test:unit:main reached 5,515 passing and 197 pending; 32 failures were baseline/environment failures from ancestor-checkout dependency resolution plus existing credential/path-length cases. The Windows/API baseline issues and the six current API assertions reproduce on main.

• Four full independent reviews and exact remediation deltas ran across Claude, Gemini, Cursor Grok/Composer, and Harper domain adjudication. A required round-9 framing recheck compared global serialization, universal generations, config checks, and slot-owned leases; Gemini certified the slot-lease approach. On the latest whole-branch pass Gemini completed, Claude hit its weekly usage limit, and the Harper domain pass timed out; the one reported major was falsified by the unconditional restart !== true early return. Two subsequent Claude/Gemini deltas accepted the TTL-preservation fix and its eviction-only hardening with no surviving new finding. Human-Review-Need remains 4 for the policy choices above.

• Review-adjudication pass over all 14 threads at da5de6772: 12 rulings verified against source, 2 (codex/cb1kenobi on the failed dedicated start) found not to hold and fixed here. unitTests/server/threads/isolatedApplications.test.js, threadInfoTimeout.test.js, expiresAtAttribute.test.js, requestRestart.test.js — 38 passing; integrationTests/components/isolated-application.test.ts — 7 passing; npm run build, Prettier and oxlint clean on the changed files.

• npm run test:unit:resources — 2486 passing, 33 pending, 3 failing. The 3 are crud.test.js "publishes and subscribes" (2 == 1), and they reproduce on this branch's merge base bba808e74 with every PR source file swapped out, so they are inherited from main, not from this change. The same file passes at 6d725818c (Sep 8), which bounds the regression to 6d725818c..bba808e74.

• Round 19 (delta, Claude graded + Gemini, 8e74b1b5e6a8): no new finding survived. Gemini's blocker — a late slot.ready rejection crashing the process — is wrong on JS semantics: Promise.race subscribes a rejection reaction to every input at call time, so the losing input's late rejection is handled, not unhandled. Verified by running the exact shape (race loses to a timeout, the input rejects 50 ms later, process.on('unhandledRejection') never fires, exit 0). The graded leg re-raised its major on drop_component with restart: false for the third round; refuted again by exact count — components/operations.js has exactly one req.restart !== true guard (line 1418) with an unconditional return, and exactly one manageThreads.restartWorkers(...) call (line 1441) after it, so restartScope is unreachable on that path and branched only affects the message text and post-restart branch removal. Both legs' comment-narration nit is declined: it targets comments already on the branch, not this delta, and the graded leg itself exempts the delta's comments as rationale rather than narration.

• Round 20 (delta, Claude graded + Gemini, 1d79483c1177): no new finding. Both legs traced the predicate extraction as behaviour-preserving and the eviction case as a true differential. The graded leg re-raised its major on drop_component with restart: false for the fourth time, this round on a new premise — that a branched application bypasses the !req.restart early return. It does not: components/operations.js has exactly one req.restart !== true guard (line 1418) whose return (line 1422) is unconditional, the only restartWorkers(...) in that function is at line 1441 after it, and branched only appends BRANCH_STORAGE_RETAINED to the message — text that says the branch storage is kept, the opposite of the claimed lock-clearing bypass. Both legs' comment-narration nit is declined again: it names lines already on the branch, and the graded leg itself exempts the delta's comments as rationale.

• Test-coverage follow-up at 1d79483c1: unitTests/resources/expiresAtAttribute.test.js 13 passing, unitTests/server/threads/isolatedApplications.test.js 19 passing; npm run test:unit:resources 2487 passing / 33 pending / 3 failing (the same 3 crud.test.js "publishes and subscribes" cases inherited from main); unitTests/server/**/*test.*js 931 passing / 0 failing; integrationTests/components/isolated-application.test.ts 7/7 with HARPER_UWS_UDS=1. Both new tests were verified differentially — removing || evictionMs fails the eviction case.

• Review-adjudication pass over all 17 threads at 1d79483c1 → 74fb011cc: every thread on the PR was already resolved, so each ruling was re-verified against source rather than trusted. Fifteen hold as written; one (cb1kenobi's schema-TTL finding, first ruled no-change) had already been overturned and fixed in-PR; cb1kenobi's dynamicThreads counter-read was re-checked independently and is correct (poolSize = threadCount at server/threads/socketRouter.ts:98 is a sibling of the if/else closing at :97, origin/main carries the same fall-through, and neither bin/run.ts:225 nor bin/lite.ts:2 passes the flag). The per-thread rulings are in this comment.

• One defect nobody had raised turned up beside them and is fixed here: restartWorkers()'s unconditional resetRestartNeeded() under the new restart scopes (item 9 above). Verified differentially — removing the gate fails the new case's first assertion.

• At 74fb011cc: npm run build passed; Prettier and git diff --check clean on both changed files; unitTests/server/**/*test*.js 932 passing / 0 failing; unitTests/components/**/*test*.js 1641 passing / 2 pending / 1 failing (gitCredentials.test.js credential-scoping, a known local-environment failure unrelated to this diff); the focused isolation/TTL/restart set 41 passing.

• Rounds 21 and 22 (delta, Claude graded + Gemini). Round 21 raised one finding against the new gate's comment rather than its behaviour — the comment claimed the flag only ever stands for pool-loaded code, which the isolated case contradicts. The comment was rewritten to state the trade instead. Round 22 on 74fb011cc: no findings.

• @cb1kenobi then raised the symmetric case on the pushed head — a pool-scoped restart also leaves a running dedicated worker on its old modules — and it holds. 6868c4eba replaces the gate with the covers-every-worker rule in item 9 rather than the suggested all-workers-only rule; the thread has the four-cell comparison. Verified differentially: removing the workers.some(...) clause fails the running-dedicated-worker assertion.

• Rounds 23–25. Round 23 flagged the test's topology stand-in as a possible monitor-tick flake; round 24 self-corrected that finding (manageThreads.js:1328 backfills recentELU in the same tick, before the listener reads it), so the property was kept for self-containment and only its comment was corrected. Round 25 on b1d5cbc5c ran a full whole-branch review: Gemini and Cursor Grok both completed with no surviving finding; the Claude graded leg failed on its review profile (exit 1) and the domain adjudication timed out, so that round's coverage is the two outside legs rather than three.

• unitTests/server/**/*test*.js — 932 passing / 0 failing at each of 74fb011cc, 6868c4eba and b1d5cbc5c.

• CI on b1d5cbc5c: 45 pass, 3 skipping, 1 failure — every required check is green (validate, runLinter, Format Check, Unit Test v24, all six Integration Tests shards), plus the non-required v22/v26/Windows/Bun/uWS matrices, Build Harper, Docker smoke and the Next.js downstream suite. The one failure is the non-required review / review job, which ran 25 minutes and then exited on SDK execution error: Claude Code returned an error result: Prompt is too long — a capacity failure of the review agent's own transcript on a 43-file diff at --effort max / --max-turns 96, not a finding about this code. No comment was posted because the run never reached one.
  unitTests/components/**/*test*.js — 1641 passing / 2 pending / 1 failing (gitCredentials.test.js credential-scoping, a known local-environment failure unrelated to this diff).

Complexity: complicated

Changed files

• Core lifecycle/config: DESIGN.md, bin/restart.ts, components/Application.ts, components/Scope.ts, components/componentLoader.ts, components/operations.js, components/operationsValidation.js, components/requestRestart.ts, config-root.schema.json, utility/hdbTerms.ts, validation/configValidator.ts.
• Runtime/storage: resources/Table.ts, resources/auditStore.ts, resources/dataLoader.ts, resources/databases.ts, resources/graphql.ts, resources/scheduler/scheduler.ts, security/jsLoader.ts, server/http.ts, server/storageReclamation.ts, server/threads/isolatedApplications.ts, server/threads/manageThreads.js, server/threads/socketRouter.ts, server/threads/threadServer.js.
• Integration: integrationTests/components/isolated-application.test.ts, isolated-app/config.yaml, isolated-app/resources.js, isolated-app/schema.graphql, isolated-app/drop-me.txt, isolated-app-two/config.yaml, isolated-app-two/resources.js, shared-sibling/config.yaml, shared-sibling/resources.js.
• Unit coverage: unitTests/components/componentLoader.test.js, unitTests/components/prepareApplicationSerialization.test.js, unitTests/components/requestRestart.test.js, unitTests/resources/dataLoader.test.js, unitTests/resources/expiresAtAttribute.test.js, unitTests/server/fastifyRoutes/operations.test.js, unitTests/server/storageReclamation.test.js, unitTests/server/threads/fixtures/exitImmediately.cjs, unitTests/server/threads/isolatedApplications.test.js, unitTests/server/threads/threadInfoTimeout.test.js.

_{Review-Coverage: authored=codex; ran=gemini,cursor-grok; blocked=claude(exit-1),domain(timeout); declined=cursor-composer; rounds=25; full=4 @ b1d5cbc}

_{Human-Review-Need: 4 @ b1d5cbc}

[gemini-code-assist]

Code Review

This pull request implements isolated applications, allowing configured applications to run in dedicated worker threads that are reachable only via Unix Domain Socket (UDS) mirrors. The changes span worker management, socket routing, and deployment validation. The review feedback identifies a style guide violation regarding the node: prefix for the path import, and a bug in server/threads/socketRouter.ts where heapShareCount is calculated inconsistently inside a loop during worker reconciliation.

[cb1kenobi]

Isolated apps that keep tables on a shared store never run schema TTL or expiresAt eviction, because ownership is only granted for branch paths and the runtime-TTL exception skips the load path. A dedicated worker that fails to start frees its slot without waiting for shutdown, so a follow-up deploy or drop can start a replacement or delete branch files while the old worker still holds the store. Set the existing dedicated-worker TTL exception on the fromSchema and expiresAt paths, and await slot.shutdown() in the start-failure catch.

—
Reviewed 39d28d6

[claude]

Reviewed; no blockers found.

[cb1kenobi]

The two latest commits stop an @expiresAt-only declaration from clearing loaded TTL and re-arm cleanup when that preserved config still has a scan or eviction. Earlier blockers on dedicated-worker TTL ownership, start-failure shutdown, and isolated file-drop restart scope are already addressed at this head. No new confirmed defect remains on the changed lines.

—
Reviewed da5de67

[kriszyp]

Review-adjudication pass at da5de6772 → 8e74b1b5e

I re-verified all 14 review threads against the source rather than trusting the resolutions. Twelve rulings hold. Two did not, and one defect nobody had raised turned up next to them.

Reopened and fixed: "Failed dedicated start does not await shutdown"

@cb1kenobi's finding (r3994233740) and the equivalent Codex finding were marked fixed at f712781. They were not. watchDedicatedStart() added the identity guard but kept the deletion before the await:

const withdrewLease = isolatedSlots.get(application) === slot;
if (withdrewLease) isolatedSlots.delete(application);
await slot.shutdown();

For the whole drain window the application therefore looked unclaimed — the state the finding described. After the 60 s readiness backstop fires on a worker that is alive but never reported ready, slot.shutdown() has real work to do (threadTerminationTimeout * 2, extendable to the drain ceiling), and during it:

• reconcileIsolatedWorkersNow() finds no slot and starts a replacement. listenOnDomainSocket() unlinks an existing path before binding (server/threads/threadServer.js:300), so the replacement takes the live application mirror out from under the draining worker and both run the application's singletons and hold its branch stores.
• dropComponent relies on reconcile's await slot.shutdown() to know the worker is gone before removeBranchesForApplication(). With no slot registered that await never covers this worker — the branch-storage race the finding named.

Nothing else serializes the two: reconcileIsolatedWorkers() chains reconciles against each other, but watchDedicatedStart is void-called and is not in that chain.

bd78e9d16 awaits the shutdown first, then withdraws the lease under the same identity guard, yields to a drop that already withdrew it, and keeps the existing replacement re-check before cleanupApplicationSockets.

New: the @expiresAt sweep outlives its attribute

8e74b1b5e. updatedAttributes() now refreshes expiresAtProperty on every live redeclaration (resources/Table.ts:5894), so a redeclaration that drops the @expiresAt directive clears it while recordExpirationInterval stays armed. The next tick dereferenced expiresAtProperty.name — once a minute, for the life of the worker. The sweep now returns early instead; a re-added directive resumes on the same timer, so the "exactly one interval" invariant still holds.

⚠️ The per-thread rulings on this PR are invisible

Every reply on this PR — all the "Fixed in <sha>" rulings, plus the five original Codex findings that were never posted any other way — is sitting in an unsubmitted pending review (pullrequestreview-5184029425, 19 comments). GitHub shows a pending review only to its author, so from every other account these threads read as resolved with no explanation, and the Codex findings do not appear at all.

It is also now actively blocking the conversation: GitHub allows one pending review per user per PR, so POST .../comments/{id}/replies returns 422 user_id can only have one pending review per pull request. That is why this is a PR comment rather than a reply on the thread above, and it is almost certainly how the 19 comments accumulated in the first place.

@kriszyp — submitting that review with event: COMMENT publishes the record and unblocks replies. I have deliberately not submitted it for you; it is your draft and the action cannot be undone.

Everything else

The other twelve rulings verify against source, including the reversal on schema TTL ownership (the first "no change" ruling there was wrong and was correctly overturned by bcb6aa13). Round 19 of the pre-push review (Claude graded + Gemini) produced no surviving finding: Gemini's blocker on a late slot.ready rejection is wrong on JS semantics — Promise.race subscribes a rejection reaction to every input at call time, so the losing input's late rejection is handled — and the graded leg's major on drop_component with restart: false is refuted by exact count, one req.restart !== true guard with an unconditional return at components/operations.js:1418 ahead of the only restartWorkers call at :1441. Full ledger in the PR description.

Verified: npm run build, 38 focused unit tests, integrationTests/components/isolated-application.test.ts 7/7, Prettier and oxlint clean. npm run test:unit:resources is 2486 passing with 3 crud.test.js pub/sub failures that reproduce on this branch's merge base with every PR source file swapped out — inherited from main, bounded to 6d725818c..bba808e74, not this change.

_{— Claude Opus 5}

[cb1kenobi]

With dynamic threads enabled, startHTTPThreads takes a branch that never assigns poolSize, so the isolated-worker reconcile is skipped entirely. An isolated: true application then gets no dedicated worker and is also skipped by every pool thread, so it loads nowhere and nothing is logged or reported as a failed component. Assign poolSize in that branch so the reconcile runs, or report the refusal there so the application fails loudly instead of disappearing. Everything else in the diff checked out, including the eight earlier review threads, which are addressed at this head.

—
Reviewed 8e74b1b

[kriszyp]

Ruling on the dynamicThreads finding — no change, premise does not hold

Thanks for the fast turnaround on 8e74b1b5e. I traced this one and it does not reproduce; here is the evidence so you can check my work.

poolSize is assigned on the dynamicThreads path. The assignment is outside the branch, not inside the else:

server/threads/socketRouter.ts
 73    if (dynamicThreads) {
 78      const slot = startHTTPWorker(0, 1);
 80      poolSlots.push(slot);
 81    } else {
 83      if (threadCount === 0) { … return Promise.resolve([]); }
 92      await loadRootComponents(); …
 97    }
 98    poolSize = threadCount;                                    ← both branches reach this
 99    nextIsolatedIndex = Math.max(nextIsolatedIndex, threadCount);
100    const isolated = admittedIsolatedApplications([...isolatedSlots.keys()]);
109    for (const application of isolated) { … startHTTPWorker(…, application, …) }

So with dynamicThreads the reconcile is not skipped, admission runs, and dedicated workers start. isolatedApplicationsUnreachableReason() does not refuse there either: it gates on isMainThread && getWorkerIndex() === 0, and that path never calls setMainIsWorker(true), so getWorkerIndex() is undefined — same as the ordinary multi-threaded path.

The one branch that does return before line 98 is threadCount === 0, and it already fails loudly. Line 90 returns early, so poolSize stays 0 and reconcileIsolatedWorkersNow() bails — but the application does not disappear silently, because the refusal is reported from the loader rather than from the router:

components/componentLoader.ts:184-191
  if (isIsolatedApplication(appName) && isMainThread && getWorkerIndex() === 0) {
    componentLifecycle.failed(appName,
      new Error(`Application '${appName}' is isolated, which needs a worker thread of its own, but threads.count is 0`),
      `Component '${appName}' failed to load`);
    return false;
  }

placedOnThisThread() runs before any of the application's modules are imported, at all four load sites (:302, :351, :383, :887), so a threads.count: 0 instance reports the isolated application as a failed component and loads it nowhere — which is the fail-closed behaviour the finding asks for.

Worth noting for anyone reading later: dynamicThreads has no caller today (see the comment at :74), and the pre-existing quirk that its branch starts one worker and then the threadCount loop starts more is unchanged from origin/main — not something this PR introduces.

Declining the change. Happy to revisit if you can point at a path where line 98 is skipped with dynamicThreads set.

_{— Claude Opus 5}

[cb1kenobi]

No new confirmed blockers on the changed lines at this head. Earlier TTL ownership, start-failure lease, file-drop scope, and expiresAt-wipe threads are already addressed. The dynamic-threads poolSize comment is already on that path, so it is not repeated.

—
Reviewed 8e74b1b

[cb1kenobi]

Traced the isolation lifecycle paths at this head: restart-scope encoding, the failed-start lease ordering, TTL ownership and the expiresAt interval, drop scoping, and dedicated-worker ingress all hold up. The prior blocking threads are genuinely fixed rather than papered over, and the one comment already on this head is refuted by the actual placement of the poolSize assignment. No new confirmed blocking defect on changed lines.

—
Reviewed 8e74b1b

[cb1kenobi]

No confirmed blocking findings remain at this commit. The latest predicate extraction preserves behavior, and the new eviction test covers the intended scheduling gate.

—
Reviewed 1d79483

[kriszyp]

Re-adjudication of all 17 review threads at 74fb011cc

Every thread on this PR was already marked resolved. Per the review protocol, a resolution is a claim like any other, so this pass re-verified each ruling against source at the current head rather than trusting the resolution. Fifteen of seventeen hold as written. One was overturned earlier in the PR's own history and is now genuinely fixed. One defect nobody raised turned up next to them and is fixed here.

The new defect: a scoped restart was clearing the node's restart-required flag

restartWorkers() has always opened with an unconditional resetRestartNeeded(). That was right while every restart replaced every worker. This PR added restart scopes, and the reset did not follow:

await restartWorkers('http', undefined, undefined, onProgress, restartScope); // restartScope = 'my-isolated-app'

restarts exactly one dedicated worker and leaves every pool worker on the old code — yet it cleared the flag. Reachable sequence:

1. deploy_component a shared component with restart: false. It is new, so requestRestartAfterDeploy sets the bit (operations.js:900). get_status reports restartRequired: true, and a super_user route miss gets the actionable 404 (server/REST.ts:188).
2. deploy_component an already-isolated application with restart: true. wasIsolated && nowIsolated scopes the restart to that application's own worker.
3. The bit is cleared. The shared component is still loaded nowhere, and now nothing says so.

That is the one outcome the flag exists to prevent — a silent 404 with no signal. The gate now clears it only for the pool sentinel or all workers. Verified differentially: with the gate removed, the new case in unitTests/server/threads/isolatedApplications.test.js fails on the first assertion.

The trade this makes, deliberately. One process-wide bit cannot say which application is pending. A restart: false deploy of an already-isolated application also sets it, and a later app-scoped restart of that same application now brings it live without clearing it — so restartRequired stays true until some pool restart happens. A needless pool restart is recoverable and self-clearing; an un-live component with no signal is not. The comment at the gate states this rather than claiming the flag only ever means pool-loaded code.

Rulings verified against source

Thread	Finding	Prior ruling	Verification at 74fb011cc
gemini	path import lacks the node: prefix	fixed	holds — socketRouter.ts:22-23, databases.ts:6
gemini	heapShareCount computed inside the start loop	fixed	holds — one denominator at socketRouter.ts:239, applied to pool and incumbent isolated slots (:240-241) before any start
codex	isolation flip must reach restart-required detection	fixed	holds — operations.js:899-905 → requestRestartAfterDeploy, which returns early only on wasIsolated === isIsolated; both flips unit-covered
codex	worker-local isolatedSlots is not an authoritative admission source	fixed	holds — operations.js:616 queries the main thread's registry under a 5 s bound and unions it with present-and-unrefused names; unavailable topology is a 503, not a pass
codex	presence filtered after admission spent the budget	fixed	holds — admittedIsolatedApplications() resolves presentIsolatedApplicationNames() first (socketRouter.ts:143) and filters the running set through it
codex + cb1kenobi	failed start frees the slot without awaiting shutdown	fixed, reopened once, re-fixed	holds — socketRouter.ts:200 awaits slot.shutdown() before the identity-guarded delete at :202 and the socket cleanup after it
codex	repeated startHTTPThreads() double-starts dedicated workers	fixed	holds — seeds from isolatedSlots.keys() (:100), monotonic index via Math.max (:99), has() skip (:110)
cb1kenobi	schema TTL never runs on an isolated app's shared store	no-change, then fixed	the no-change ruling did not hold (already overturned in-PR); the fix does — Table.ts:7496 and :7638 both carry the ttlConfiguredByApplication && isDedicatedWorker() leg
cb1kenobi	isolated file drop restarts the pool	fixed	holds — operations.js:1381-1391 derives scope from live topology under the preparation lock; the !file guards now cover only config/symlink/package.json/branch removal
cb1kenobi + claude	@expiresAt-only declaration wipes the loaded TTL	fixed + hardened	holds — preserveLoadedConfiguration (Table.ts:1579) skips the replace, and scheduleCleanup() is reached only when something was loaded to clean up, so no default daily timer is manufactured
claude	|| evictionMs disjunct unexercised	covered	holds, and the reasoning checks out — both evictionMs (:1591) and expirationScanScheduled (:1598) are assigned inside !preserveLoadedConfiguration, so only a table whose eviction came from persisted metadata reaches the disjunct; that is exactly the shape the new case builds
claude	no coverage for the dedicated-worker uWS entry rule	fixed	holds — shouldStartUwsListenerHere() pinned across all four quadrants
claude	coverage parity for runRecordExpirationEviction	coverage ruling	holds — accepted as a non-blocking coverage judgment; the unit pins exactly one 60 s interval on initial and live declarations

The one counter-read, re-checked independently

@cb1kenobi's dynamicThreads finding — isolated applications load nowhere on that path — was declined with a counter-read rather than a fix. That refutation is correct, and here is the check rather than the assertion:

• poolSize = threadCount; is at server/threads/socketRouter.ts:98, a sibling statement after the if (dynamicThreads) { … } else { … } construct closes at :97. It runs on both branches, so the reconcile is not skipped and poolSlots.push(slot) is not dead.
• The "start one worker in the if-branch, then fall through to the loop that starts threadCount more" shape is on origin/main too (startHTTPThreads there, same structure), so this PR neither introduces nor worsens it.
• No caller passes the flag: bin/run.ts:225 and bin/lite.ts:2 both call startHTTPThreads with a single argument, matching the No caller currently passes dynamicThreads comment already on the branch.

Verification

• npm run build — passed. Prettier and git diff --check clean on both changed files.
• unitTests/server/**/*test*.js — 932 passing, 0 failing.
• unitTests/components/**/*test*.js — 1641 passing, 2 pending, 1 failing: gitCredentials.test.js "grants the credential environment to the spawn that clones, and to no other", a known local-environment failure unrelated to this diff.
• unitTests/server/threads/isolatedApplications.test.js, expiresAtAttribute.test.js, requestRestart.test.js, threadInfoTimeout.test.js — 40 passing before the change, 41 after.
• Independent pre-push review, two delta rounds on the new commits (Claude graded + Gemini). Round 21 raised one finding against the gate's comment, not its behaviour; the comment was rewritten and round 22 on 74fb011cc returned no findings.

— Claude Opus 5

[cb1kenobi]

A pool-only restart still clears the process-wide restart-required flag, so a pending isolated deploy can report restartRequired false while its dedicated worker keeps old code. The new gate treats the pool sentinel as if it replaced every worker, but that scope skips dedicated workers that already have a slot. Clear the flag only when the restart scope is all workers (*), and update the unit case that currently expects a pool-only restart to clear the bit.

—
Reviewed 74fb011

[cb1kenobi]

Traced the isolation lifecycle at this head: restart-scope encoding and arity fallback, the failed-start lease ordering, dedicated-worker ingress, TTL ownership and eviction arming, and deploy/drop restart scoping all hold up. Prior blocking threads are genuinely fixed rather than worked around, and the new code fails closed on unreachable, over-budget, malformed, and topology-unavailable cases. No new confirmed blocking defect on changed lines; the one open reviewer thread on manageThreads.js:679 is not repeated here.

—
Reviewed 74fb011

[cb1kenobi]

No new confirmed blocking defects were found on the changed lines. Previously reported lifecycle, TTL, drop-scope, and restart-state issues are addressed at this commit.

—
Reviewed b1d5cbc

[kriszyp]

Independent adjudication pass at b1d5cbc5c — no open feedback, no warranted change

All 18 review threads on this PR are resolved, and the three most recent reviewer passes (8e74b1b5, 1d79483c, b1d5cbc5) report no confirmed blocking defect. The published ledger (comment) covers 17 threads at 74fb011cc. This pass re-verified, against source at the current head, the rulings that ledger does not cover, plus the two "no-change" rulings most exposed to being wrong.

Rulings re-verified — all hold

Finding	Ruling	Evidence at b1d5cbc5c
dynamicThreads skips the isolated reconcile (socketRouter.ts:80)	no change — premise does not hold	poolSize = threadCount is socketRouter.ts:98, a sibling of the if/else that closes at :97, so the dynamicThreads branch (:73–:80) falls through to it and to the reconcile at :100. origin/main carries the identical fall-through into its own for (let i = 0; i < threadCount; i++) loop, so the shape is pre-existing, not introduced here.
A pool-only restart clears restart-required while a dedicated worker keeps old code (manageThreads.js:679)	confirmed, fixed at 6868c4eba with the covers-every-worker rule rather than the suggested '*'-only rule	coversEveryWorker at manageThreads.js:675 is application === '*' || (application === undefined && !workers.some((w) => w.application)). The '*'-only alternative is genuinely worse: components/operations.js:858 computes restartScope = wasIsolated && nowIsolated ? application.name : undefined, so an everyday shared deploy_component with restart: true passes the pool scope and would never clear the bit on any node.
drop_component with restart: false reaches restartWorkers / clears a lock (raised four rounds by the graded leg)	refuted	components/operations.js has exactly one req.restart !== true guard (:1418) whose return at :1421 is unconditional, and exactly one manageThreads.restartWorkers(...) in dropComponent at :1441, after it. branched only appends BRANCH_STORAGE_RETAINED to the message.
Failed dedicated start frees its slot before the worker exits	fixed	socketRouter.ts:200 awaits slot.shutdown() before the identity-guarded delete at :201–:202, and :205 re-checks isolatedSlots.has(application) after the dynamic import() so a replacement's live mirror is never unlinked.
Topology stand-in may flake on a monitor tick (b1d5cbc5c)	self-corrected in-round; property kept for self-containment	The stand-in is wasShutdown: true, so the restart loop's :737 filter skips it; it stands in only for what the :675 gate reads.

One property of the restart-flag gate worth stating plainly

On a node with a running dedicated worker, deploy_component/drop_component with restart: true never clears restartRequired — those paths pass the pool scope, and the gate requires no dedicated worker. Only an all-workers restart clears it there: restart_service with no scope (decodeRestartScope returns '*' for an absent scope, manageThreads.js:1000), the watcher-driven restart at componentLoader.ts:1202, and manageThreads.js:1952.

That is the intended direction of the trade in item 9 — a stale restartRequired: true that the next restart_service clears, rather than a false reported while a deployed component is loaded nowhere. Recording it because the current item-9 wording names only the narrower "a dedicated worker the reconcile is about to stop is still counted" case; the operator-visible consequence is the broader one.

Not a finding about this code

The non-required review / review check on this head failed after 25 minutes with Prompt is too long — the review agent's own transcript capacity on a 43-file diff at --effort max, with no comment posted. Every required check is green.

No code change was warranted, so nothing was pushed and no thread state was altered.

_{🤖 Adjudicated with Claude Opus 5}

[cb1kenobi]

Prior blocking threads on TTL ownership, failed-start lease ordering, file-drop restart scope, expiresAt config wipe, and the restart-required flag are addressed at this commit. I re-traced admission, placement, UDS binding, slot shutdown, drop/deploy restart scope, and store-maintenance ownership against the current diff and existing discussion. No new confirmed blocking defect remains on the changed lines.

—
Reviewed b1d5cbc