Skip to content

runtime: a panic during the prelude still leaks the acceptance entry, the Thread Lock, and a burst slot #139

Closed

Description

@ThomasK33

runtime: three more places where a panic during the prelude leaves state behind.

#138 fixed the admission slot and queue turn record leak from #135. Its independent verifier found three more leaks on the same trigger: a panic while the prelude runs. All three behave the same on main before #138, so #138 did not cause them. None has a committed test yet.

What happens

  1. Acceptance entry. The prelude opens an acceptance entry for the event ID (runtime.go#L800). After a panic it never calls finishEventAcceptance (runtime.go#L1599). A redelivery of the same event ID then waits until its own request ends, and its handler never runs. The verifier saw the redelivery hang until the client gave up after 1.7 s. The fix(runtime): free the admission slot when the deferred prelude panics #138 owner saw the same in both dispatch modes.
  2. Thread Lock. The prelude takes the Thread Lock (runtime.go#L849) and marks the event (runtime.go#L812). If MarkEvent panics while the prelude holds the lock, nothing releases it, so the lock stays held until ThreadLockTTL. The verifier saw the next delivery for that thread wait and not run within 2 s.
  3. Burst member. A burst member logs chat burst member joined (burst.go#L98) after the handoff and before the burst runner starts. If the app's slog handler panics there, that member never releases its admission slot, later members for the thread never run, and Shutdown times out.

Decision first

Decide the contract: does the runtime promise to survive a panic from State, an Observer, or the app's slog handler during the prelude? #134 and #138 already release the turn record and the admission slot on that path. If the answer is yes, release every resource the prelude takes on the panic path too, with a test per case, and without a recover that hides the panic from net/http. If the answer is no, document that such a panic is fatal for the Chat.

Owner and trigger

The coder/chat maintainer triages this. Decide the contract with the next change to the prelude, or before the queue strategy is called production-grade, whichever comes first.


Generated with xum • Model: anthropic:claude-opus-5-5 • Thinking: xhigh

Activity

  1. added
    needs-triageMaintainer needs to evaluate this issue
    bugSomething isn't working
    ready-for-agentFully specified, ready for an AFK agent
    and removed
    needs-triageMaintainer needs to evaluate this issue
    on Oct 29, 2026
  2. ThomasK33 commented on Oct 29, 2026

    @ThomasK33
    MemberAuthor

    Triage: confirmed bug, ready-for-agent. I decided the contract.

    Decision: yes. A panic from State or from the app's slog handler during the prelude must not leak anything that the prelude took. The runtime releases every resource it took, then lets the panic continue to net/http, with no recover that hides it. This extends what #134 and #138 already did for the turn record and the admission slot. A panic in an Observer is already covered: ADR 0010 says every Observer call is wrapped, so a panic there cannot fail an event.

    Reasons:

    1. fix(runtime): harden the queue turn record against a double release and a State panic #134 and fix(runtime): free the admission slot when the deferred prelude panics #138 already chose this rule for two resources. A partial rule is harder to reason about than a full one.
    2. net/http recovers handler panics, so the process keeps running. Each leak then blocks real work: a redelivery that never runs, a thread that waits for ThreadLockTTL, or a burst slot that Shutdown waits on.

    Work items, each with its own failing test first:

    1. Acceptance entry: call finishEventAcceptance on the panic path, then show that a redelivery of the same event ID runs.
    2. Thread Lock: release it if MarkEvent panics while the prelude holds it, then show that the next delivery for the thread runs at once, not after ThreadLockTTL.
    3. Burst member: release the admission slot if the slog handler panics after the handoff, then show that Shutdown returns promptly.

    Record the rule in ADR 0015 as one decision-only sentence. Put it in the same PR, so the contract and the code land together.


    Generated with xum • Model: anthropic:claude-opus-5-5 • Thinking: medium

  3. added a commit that references this issue on Oct 30, 2026
    f141502
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't workingready-for-agentFully specified, ready for an AFK agent

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions