Skip to content

fix(plugin): converge plugin toggles and reload on lost CLI replies, reap in-flight clients - #39

Merged
chhoumann merged 3 commits into
mainfrom
fix/lost-cli-reply-plugin-reload
Oct 9, 2026
Merged

chhoumann merged 3 commits into
mainfrom
fix/lost-cli-reply-plugin-reload

Conversation

@chhoumann

Copy link
Copy Markdown
Owner

Why

A lost Obsidian CLI reply could leave the plugin under test disabled for the rest of a run. That is the failure in chhoumann/quickadd#2217: 22 files failed with Timed out waiting for plugin "quickadd" to be ready, and enabledPlugins was empty afterwards. A CLI client still waiting for a reply also outlived its Vitest worker, which produced the 20+ orphaned obsidian ... __obsidian-e2e:dispatch processes. The fix reads the plugin's real state instead of trusting replies, and it kills in-flight clients when the worker stops.

What changed

  • plugin.enable() / plugin.disable() read the enabled flag back after each send. An unanswered send is resent once, unless dispatch reports it still running. A served reply whose flag did not change throws. plugin.reload() enables a disabled plugin instead of accepting Obsidian's exit-0 not enabled error.
  • createPluginHarness no longer re-disables a plugin it found disabled. That snapshot is left over from an earlier failure, and re-applying it cascaded across files. Each restore re-enables the plugin even when the restore fails, and the afterEach/afterAll budget is now 90 s.
  • The transport kills live clients on SIGTERM and then re-raises the signal. Vitest stops forks workers with SIGTERM, and the default action runs no timers.

Scope

Tradeoffs

  • reload() reads the flag before it sends, which costs one extra CLI call per reload. The alternative parses Obsidian's error text to detect the same state.
  • I kept this as a patch release so ^0.12 consumers get it without a dependency bump. The new throws happen only where 0.12.1 left a silently broken state.

Blast Radius

The change touches every enable/disable/reload call and every createPluginHarness suite. Toggles now cost one extra read. enable/disable/reload can throw where they used to resolve over a broken state. createPluginTest keeps its documented restore-prior-state behavior.

Verification

  • Real quickadd suite (98 files) starting with quickadd disabled, on a Linux Obsidian 1.13.7 instance: 0.12.1 fails 18 files (15 plugin-ready timeouts) and ends with quickadd disabled. This branch has 0 plugin-ready timeouts, 1 failure that also fails on 0.12.1, ends with quickadd enabled, and leaves no client running.
  • Live fixture rig through a reply-swallowing socket proxy. A 40 s reply blackout at plugin:disable gives 65/65 passing, where main fails the next test. A test that ends with a dispatch in flight: its client closes about 1 s after connect, where on main it was still alive 69 s after vitest exited.
  • Regression tests land first and fail on main. pnpm exec vp check is clean, and pnpm exec vp test passes 386 tests.

Regression tests that fail on main:
- plugin.reload() accepts Obsidian's exit-0 'not enabled' reply and never
  re-enables a plugin an earlier failure left disabled.
- plugin.disable()/enable() fail on a lost reply even when Obsidian applied
  the toggle, and never resend a toggle whose request was lost.
- createPluginHarness re-disables a plugin it found disabled at teardown,
  and a failed data restore skips the re-enable.
- A CLI client in flight when Vitest stops the worker outlives it.
…reap in-flight clients

A lost or failed Obsidian CLI reply could leave the plugin under test
disabled for the rest of a run, and a CLI client still waiting for a reply
outlived the Vitest worker that spawned it.

- plugin.enable()/disable() read Obsidian's enabled flag back instead of
  trusting the reply. An unanswered send is resent once unless dispatch
  reports it still running; a served reply whose flag did not change throws
  ObsidianCommandError. Toggles always send with allowNonZeroExit: false so a
  resolved exec means a served reply.
- plugin.reload() reads the flag first: a disabled plugin is enabled (Obsidian
  answers plugin:reload on a disabled plugin with an exit-0 "not enabled"
  error and does nothing). Any other served Error: reply throws. The caller's
  exec options reach every call reload makes.
- createPluginHarness no longer re-disables a plugin it found disabled (that
  snapshot is the product of an earlier failure and cascaded across files).
  The per-test restore always re-enables, reporting both errors when restore
  and re-enable fail, and afterEach/afterAll use teardownTimeoutMs (default
  30 s -> 90 s).
- The transport tracks live clients and kills them on SIGTERM (Vitest stops
  forks workers with SIGTERM, whose default action runs no timers), then
  re-raises when it is the only listener. A signal-killed client reports
  128 + signal instead of resolving as exit 0.
- The dist-binding test gets a 60 s timeout: it rebuilds dist after any
  fixture edit, which takes longer than the default 5 s under load.
…e tests

Spawning a child `vp test` takes longer than 2 s under load (load average
40-60 on the 12-core dev host), so both cross-process lock tests failed on
main. The wait only bounds a failure; a healthy child still signals at once.
@chhoumann

chhoumann commented Oct 9, 2026 •

Copy link
Copy Markdown
Owner Author

Investigation notes: where replies are dropped

Rig: Linux, Obsidian 1.13.7 under Xvfb, an isolated instance (--user-data-dir, plus private HOME and XDG_RUNTIME_DIR; the Linux CLI socket lives in XDG_RUNTIME_DIR), and the native obsidian-cli. A logging unix-socket proxy between the client and Obsidian records each request, the reply timing, and which side closed first. Load average was 40 to 68 on 12 cores.

Socket and client: nothing dropped.

Run CLI connections Replies written by Obsidian p99
quickadd suite, window visible 15,033 15,033 183 ms
quickadd suite, window hidden 15,535 15,535 240 ms
quickadd suite, this branch 15,951 15,951 219 ms

In the last run, one dispatch attempt's client hit its 2.5 s budget 150 ms before Obsidian replied, and a registry poll recovered the reply. CPU load alone did not reproduce a lost reply.

Obsidian's command handler can stall forever. The main process awaits webContents.executeJavaScript("window.handleCli(argv)") with no timeout (read from the shipped obsidian.asar). With the window hidden, a 10 ms setTimeout took 223 ms. An eval that awaited requestAnimationFrame never replied, and its client was still waiting after more than 2 minutes. The runner launches macOS instances with open -g, so an occluded window is a plausible trigger there. That part is unverified.

Why one lost reply cascaded. I read these from Obsidian's app.js:

  • plugin:disable clears enabledPlugins and saves the config synchronously, so the toggle has taken effect even when its reply is lost.
  • plugin:reload on a disabled plugin throws Plugin "<id>" is not enabled., and the CLI delivers that throw with exit code 0.

After that, context.plugin()'s wasEnabled snapshot re-disabled the plugin at every harness file's cleanup. I reproduced the cascade by starting a 25-file fixture suite with the plugin disabled. All 5 reload()-based files failed with Timed out waiting for plugin "fixture" to be ready, and the run ended with community-plugins.json = [].

Orphans. vite-plus-test's ProcessWorker.stop sends SIGTERM, then SIGKILL after a timeout. With no SIGTERM listener, the worker dies without running timers or exit handlers. In the reproduction, a client with a 2.5 s budget was reparented to systemd --user and was still alive 69 s after vitest exited.

quickadd

No consumer config change is needed. The patch reaches quickadd through ^0.12.1, and quickadd's hookTimeout: 30_000 no longer bounds the harness restore. One quickadd test, choice-builder-pages "says once that a choice deleted elsewhere...", failed in my runs with 0.12.1 and with this branch alike. That points to state in my test vault, not to this change.

Review findings not acted on

  • GPT-6 Astra, accepted tradeoff. On the default path, setEnabled resends after ambiguous-delivery. If the original enable's onload were still running, the resend would mark it enabled early. I kept the resend. A veto here would only move the same fresh plugin:enable to the harness's next step (the finally re-enable, the next restore, or the next file's reload()), and it would turn a converging run into a failed test. The case needs a full 30 s reply blackout overlapping an onload longer than 30 s; the slowest reply across about 46,000 calls on the rig was 2.6 s. Closing it fully would need dispatch receipts reused across calls, which this PR deliberately leaves out.
  • Claude Fable. Three findings:
    • reload() could save a roundtrip by branching on the reload reply instead of reading the flag first. I kept the structured read.
    • The flag read inside setEnabled uses allowNonZeroExit: true, so a failed read is reported against the toggle.
    • No test covers exec-option threading into reload()'s enable branch.
  • Unverified, released code, not this diff. captureOnTestFailure splices the shared afterEach hook array while Vitest may iterate it under sequence.hooks: "list".

@chhoumann

Copy link
Copy Markdown
Owner Author

PASS

Head: 80f7650161c1f4558a363f520045800382f7d9ee
Base: 226a66358a6db5a270fa66f1a2c508963b0357ef

Independent live verification used Linux Obsidian 1.13.7 on display :200, the native CLI, and the supplied socket proxy. Both frozen installs and package builds succeeded. Identical consumer suites imported each worktree's built dist.

Check Base Head
Cascade, suite starts disabled 5 bespoke files failed, 20 harness files passed; 60 tests passed, 5 skipped; ended with no plugin enabled. 25 files and 65 tests passed; fixture remained enabled.
40 s reply blackout on plugin:disable 6 files failed, 19 passed; 2 tests failed, 58 passed, 5 skipped. Hook timed out at 30 s and the next test found the plugin unloaded; ended with no plugin enabled. 25 files and 65 tests passed; fixture remained enabled.
Worker exits with a swallowed reply in flight Test passed, but 1 CLI client remained 10 s after Vitest exited. Test passed; client closed 1,003 ms after connect; 0 clients remained 10 s after Vitest exited.
HEAD check and unit tests Not run. vp check clean. 38 files and 386 tests passed; 3 files and 3 tests skipped.

The proxy confirmed six swallowed CLI connections during each 40 s blackout and one during each orphan check. Shutdown used the rig PID files. No matching rig Obsidian process or obsidian-cli vault=vault client remains. Tracked files in both worktrees were unchanged.

@chhoumann
chhoumann merged commit b8189ae into main Oct 9, 2026
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant