[an1310]

What does the PR do?

When a model instance fails to launch after its stub process has started, for example after missed health updates during initialization, StubLauncher::Launch() waited in a blocking waitpid for a stub that still considers itself initialized and never exits. The model load hung forever, which also blocks server startup, and the original error was never logged.

The cleanup now waits up to stub-timeout-seconds for the stub to exit, then kills it and logs a warning. A stub that reports its own initialization error still exits on its own.

Checklist

• I have read the Contribution guidelines and signed the Contributor License
  Agreement
• PR title reflects the change and is of format <commit_type>: <Title>
• Changes are described in the pull request.
• Related issues are referenced.
• Populated github labels field
• Added test plan and verified test passes.
• Verified that the PR passes existing CI.
• I ran pre-commit locally (pre-commit install, pre-commit run --all)
• Verified copyright is correct on all changed files.
• Added succinct git squash message before merging ref.
• All template sections are filled out.
• Optional: Additional screenshots for behavior/output changes with before/after.

Commit Type:

Check the conventional commit type
box here and add the label to the github PR.

• build
• ci
• docs
• feat
• fix
• perf
• refactor
• revert
• style
• test

Related PRs:

• test: Cover a Python backend stub that stops responding during initialization server#9061

Where should the reviewer start?

src/stub_launcher.cc: the ScopedDefer in the non-Windows StubLauncher::Launch().

Test plan:

• Added the init_unresponsive lifecycle test in test: Cover a Python backend stub that stops responding during initialization server#9061. The model stops its own stub during initialize() until the server gives up on it.
• Without this change, the server never exits. With it, the load fails with Stub process 'init_unresponsive_0_0' is not healthy., the server exits, and no stub process or shared memory is left.
• init_error and add_sub behave as before.
• Run on aarch64 with nvcr.io/nvidia/tritonserver:26.04-py3 and a CPU-only build of this branch.

Caveats:

• Linux only. The Windows StubLauncher::Launch() is unchanged.
• A stub the server gave up on is killed after the full stub-timeout-seconds.

Background

On Triton 26.04 (aarch64), Python model loads hung after a server restart and the server never started serving. gdb showed each load thread in StubLauncher::Launch() → ~ScopedDefer → WaitForStubProcess() → waitpid, with the stub processes alive and idle.

Related Issues: (use one of the action keywords Closes / Fixes / Resolves / Relates to)

• None

[greptile-apps]

Confidence Score: 5/5

[Medium impact] The change appears safe to merge.

Summary

StubLauncher::Launch() now waits up to stub-timeout-seconds for a failed Linux stub to exit, then warns and kills it.

• Failed model launches kill stubs that outlast the configured timeout.

Diagram

%%{init: {'theme': 'neutral'}}%%
flowchart TD
  A[Linux launch fails] --> B[Send existing acknowledgment and release parent resources]
  B --> C[Wait up to configured timeout]
  C --> D{Stub exited?}
  D -->|Yes| E[Return launch error]
  D -->|No| F[Log warning]
  F --> G[Kill and reap stub]
  G --> E

Loading

_{Reviews (1) · Last reviewed commit: "fix: Kill a Python backend stub that out..." · Reviewed by Greptile}