release: 0.3.2 with unscripted routing evaluations

.claude-plugin/plugin.json

@@ -1,7 +1,7 @@
 {
   "$schema": "https://json.schemastore.org/claude-code-plugin-manifest.json",
   "name": "engineering-method",
-  "version": "0.3.1",
+  "version": "0.3.2",
   "description": "Risk-proportionate engineering workflows for Codex, Claude and OpenCode.",
   "author": {
     "name": "Peter Hense"

.codex-plugin/plugin.json

@@ -1,6 +1,6 @@
 {
   "name": "engineering-method",
-  "version": "0.3.1",
+  "version": "0.3.2",
   "description": "Risk-proportionate engineering workflows for Codex, Claude and OpenCode.",
   "author": {
     "name": "Peter Hense"

BACKLOG.md

@@ -160,6 +160,9 @@ Keep stable IDs unchanged. Order groups unblocker-first, then priority and depen
 <!-- engineering-method:backlog {"schema_version":1,"id":"EM-021","title":"Hold every skill to the published skill authoring limits (release 0.3.1)","status":"complete","priority":"P2","parent_id":null,"depends_on":[],"notes":"Published stable Latest v0.3.1 from bf6acfb; 347 tests, validator, sync-core check (Core 4300085, 70 files) and reproducible 243-file package passed. Downloaded asset matches 63329fe47119d6659766d88dad7044f02779b88ed7c882763aa12553a63a5285. Paid native routing evaluations were not rerun.","updated_at":"2026-11-13T18:33:57Z"} -->
 - ✅ `EM-021` **P2** Hold every skill to the published skill authoring limits (release 0.3.1)
   - Notes: Published stable Latest v0.3.1 from bf6acfb; 347 tests, validator, sync-core check (Core 4300085, 70 files) and reproducible 243-file package passed. Downloaded asset matches 63329fe47119d6659766d88dad7044f02779b88ed7c882763aa12553a63a5285. Paid native routing evaluations were not rerun.
+<!-- engineering-method:backlog {"schema_version":1,"id":"EM-022","title":"Unscripted multi-plugin routing evaluations (release 0.3.2)","status":"open","priority":"P2","parent_id":null,"depends_on":[],"notes":"Core 91e35da adds scripts/run_natural_evals.py and closes the host's stdin in run_host_evals. Three cases under evals/natural. Release 0.3.2 in preparation.","updated_at":"2026-11-13T19:36:07Z"} -->
+- ⭕ `EM-022` **P2** Unscripted multi-plugin routing evaluations (release 0.3.2)
+  - Notes: Core 91e35da adds scripts/run_natural_evals.py and closes the host's stdin in run_host_evals. Three cases under evals/natural. Release 0.3.2 in preparation.
 <!-- engineering-method:backlog {"schema_version":1,"id":"EM-019","title":"Verify a Gemini host adapter in Method Core","status":"open","priority":"P3","parent_id":null,"depends_on":[],"notes":"Deferred from EM-018; shared adapter ownership is phense/method-core. No Gemini support claimed until fresh native discovery and behavior checks pass.","updated_at":"2026-11-13T10:39:43Z"} -->
 - ⭕ `EM-019` **P3** Verify a Gemini host adapter in Method Core
   - Notes: Deferred from EM-018; shared adapter ownership is phense/method-core. No Gemini support claimed until fresh native discovery and behavior checks pass.

CONTRIBUTING.md

@@ -76,6 +76,7 @@ alone do not prevent sensitive Git history from being published.
 
 ## Release policy
 
+`v0.3.2` adds the unscripted evaluation runner from Method Core `91e35da`.
 `v0.3.1` adds the skill authoring limits test from Method Core `4300085`.
 `v0.3.0` separates writing into Writing Method and bundles shared support from
 Method Core, with [split evidence](docs/verification/EM-018-method-split.md).

README.md

@@ -10,9 +10,11 @@ local checks first, reuse unaffected evidence, focused review of fixes, and an
 effort brake before testing or delegation grows beyond the task. Subscription
 limits and model-effort permissions remain binding across quota resets.
 
-**[Version 0.3.1](https://github.com/phense/engineering-method/releases/tag/v0.3.1)
-adds a test that holds every skill to the published skill authoring limits.**
-See the [release notes](docs/releases/0.3.1.md). Earlier scopes:
+**[Version 0.3.2](https://github.com/phense/engineering-method/releases/tag/v0.3.2)
+adds a runner for unscripted routing evaluations and fixes host runs that
+timed out without output.** See the [release notes](docs/releases/0.3.2.md).
+Earlier scopes: [0.3.1](docs/releases/0.3.1.md) added a test that holds every
+skill to the published skill authoring limits,
 [0.3.0](docs/releases/0.3.0.md) separated writing into its own plugin and
 bundled shared support from a pinned Method Core,
 [0.2.1](docs/releases/0.2.1.md) refined local PDF export, [0.2.0](docs/releases/0.2.0.md)
@@ -37,7 +39,7 @@ routing evaluation.
 
 ## Related methods and shared core
 
-Version 0.3.1 is the current stable release; 0.2.1 was the last release that bundled the writing skills.
+Version 0.3.2 is the current stable release; 0.2.1 was the last release that bundled the writing skills.
 
 Version 0.3.0 separates writing into [Writing Method](https://github.com/phense/writing-method). [Method Core](https://github.com/phense/method-core) owns the bundled host adapters, state/continuity tools, verification/review/isolation support and generic playbooks. Each repository keeps its own Issues and pull requests; the common topic is `agent-methods`.
 

docs/installation.md

@@ -8,7 +8,7 @@ installation success does not establish live model behavior on every host.
 For OpenCode, follow [the source-linked installation](opencode.md), available
 since 0.1.1. Its native discovery and bounded DeepSeek test were checked with
 OpenCode 1.18.30; see the [0.1.1 release scope](releases/0.1.1.md) and the
-[0.3.1 release notes](releases/0.3.1.md).
+[0.3.2 release notes](releases/0.3.2.md).
 
 ## Split into three repositories
 
@@ -38,11 +38,11 @@ claude plugin install engineering-method@engineering-method
 claude plugin list --json
 ```
 
-For stable version 0.3.1 on either host, clone the release tag and register the
+For stable version 0.3.2 on either host, clone the release tag and register the
 absolute checkout path instead of the GitHub shorthand:
 
 ```sh
-git clone --branch v0.3.1 --depth 1 https://github.com/phense/engineering-method.git
+git clone --branch v0.3.2 --depth 1 https://github.com/phense/engineering-method.git
 cd engineering-method
 em_plugin="$(pwd)"
 codex plugin marketplace add "$em_plugin"
@@ -55,11 +55,11 @@ Choose one source per marketplace name. If switching an existing local
 marketplace to GitHub, uninstall this plugin and remove its marketplace using
 the commands below, then register the new source. Preserve your project files.
 
-Alternatively, download `engineering-method-0.3.1.zip` and
-`SHA256SUMS` from the [0.3.1 release](https://github.com/phense/engineering-method/releases/tag/v0.3.1).
+Alternatively, download `engineering-method-0.3.2.zip` and
+`SHA256SUMS` from the [0.3.2 release](https://github.com/phense/engineering-method/releases/tag/v0.3.2).
 In the download directory, run `shasum -a 256 -c SHA256SUMS`, extract the archive,
 and register the extracted `engineering-method` directory as above.
-The plugin manifests and stable Git tag identify version `0.3.1`. The older
+The plugin manifests and stable Git tag identify version `0.3.2`. The older
 `v0.1.0-rc.1` pre-release remains available for historical reference.
 
 Claude also supports a session-only load:

docs/opencode.md

@@ -1,6 +1,6 @@
 # OpenCode integration
 
-Version 0.3.1 bundles 19 engineering/shared skills. Writing Method
+Version 0.3.2 bundles 19 engineering/shared skills. Writing Method
 bundles its own 18 writing/shared skills and a distinct `writing-method.js`
 loader. Both use the same adapter from Method Core. Earlier release evidence
 under `docs/verification/` describes the combined historical packages and is not
@@ -16,7 +16,7 @@ macOS. Use version 0.1.1 or newer; the earlier v0.1.0 release predates this
 integration. Clone a retained checkout and install from it:
 
 ```sh
-git clone --branch v0.3.1 --depth 1 https://github.com/phense/engineering-method.git
+git clone --branch v0.3.2 --depth 1 https://github.com/phense/engineering-method.git
 cd engineering-method
 ```
 

docs/releases/0.3.2.md

@@ -0,0 +1,61 @@
+# Engineering Method 0.3.2
+
+Released 2026-11-13. A maintenance release: unscripted routing evaluations
+and a fix for host runs that timed out without output.
+
+## Changes
+
+- Syncs [Method Core](https://github.com/phense/method-core) revision `64d9a02`
+  (72 files).
+- Adds `scripts/run_natural_evals.py`. It sends a case prompt exactly as
+  written, with every given plugin loaded, and reports which skills the host
+  loaded on its own over a number of runs. The routing harness
+  (`scripts/run_host_evals.py`) keeps telling the host to select a workflow;
+  the new runner shows what happens without that instruction.
+- Adds three unscripted cases under `evals/natural/`: a failing test, a
+  feature across three modules and a typing error.
+- `run_host_evals.py` no longer lets the host inherit an open standard input.
+  Codex then waited for more input, and the run timed out without one event.
+- No skill name, description or behaviour changes.
+
+## Verification and limits
+
+The complete Python suite passes 359 tests on macOS. `scripts/validate-plugin`
+and `sync-core --check` pass. The committed-source package is built twice for
+reproducibility.
+
+Unscripted requests, three runs per case with Writing Method loaded beside
+this plugin, the hosts' coordinator models, and on Codex the plugins as
+installed (Claude Code 2.1.323, Codex CLI 0.169.0):
+
+| Request | Expected | Claude | Codex |
+|---|---|---|---|
+| A failing test | `systematic-debugging` | 2 of 3 | 3 of 3 |
+| A feature across three modules | a specification skill | 3 of 3 | 3 of 3 |
+| A typing error in a README | no skill | 3 of 3 | 2 of 3 |
+
+Three runs show a tendency, not a rate. The Codex figures are a re-scoring of
+saved transcripts after the runner learned to count a skill read inside a
+compound command. An earlier single probe in which each request ended with a
+request for a plan loaded no specification skill on Claude: asking for a plan
+first lowers loading. The paid routing and lifecycle evaluations of the
+release gate were not rerun; the evidence limits of [0.3.0](0.3.0.md) remain.
+
+The downloadable package is `engineering-method-0.3.2.zip`; verify it with the
+attached `SHA256SUMS`.
+
+## Install or update
+
+```sh
+claude plugin marketplace update engineering-method
+claude plugin update engineering-method@engineering-method
+```
+
+```sh
+codex plugin remove engineering-method@engineering-method
+codex plugin add engineering-method@engineering-method
+```
+
+Restart the host session afterwards. See
+[Installation and maintenance](../installation.md) for a fixed-tag install and
+[OpenCode installation](../opencode.md) for the source-linked loader.

evals/natural/cases/cross-module-feature.json

@@ -0,0 +1,10 @@
+{
+  "id": "cross-module-feature",
+  "prompt": "Ich brauche in dieser App einen Export der Rechnungen als CSV und PDF. Der soll die Rollen aus accounts beachten und auch in den Reports auftauchen.",
+  "files": {
+    "README.md": "# Ledger\n\nInvoicing app with modules billing, accounts and reports.\n",
+    "billing/invoice.py": "class Invoice:\n    pass\n",
+    "accounts/roles.py": "ROLES = ['admin', 'clerk']\n",
+    "reports/summary.py": "def summary():\n    return {}\n"
+  }
+}

evals/natural/cases/failing-test.json

@@ -0,0 +1,9 @@
+{
+  "id": "failing-test",
+  "prompt": "Der Test in test_cart.py schlägt fehl und ich verstehe nicht warum. Kannst du dir das ansehen?",
+  "files": {
+    "README.md": "# Cart\n\nSmall pricing library.\n",
+    "cart.py": "def total(items, discount=0):\n    subtotal = sum(i['price'] * i['qty'] for i in items)\n    return round(subtotal * (1 - discount / 10), 2)\n",
+    "test_cart.py": "from cart import total\n\ndef test_total_with_discount():\n    assert total([{'price': 10.0, 'qty': 2}], discount=10) == 18.0\n"
+  }
+}

evals/natural/cases/typo-fix.json

@@ -0,0 +1,7 @@
+{
+  "id": "typo-fix",
+  "prompt": "In der README ist ein Tippfehler bei accounts. Korrigier den bitte.",
+  "files": {
+    "README.md": "# Ledger\n\nInvoicing app with modules billing, acounts and reports.\n"
+  }
+}

evals/natural/expected.json

@@ -0,0 +1,24 @@
+{
+  "failing-test": {
+    "any": [
+      "systematic-debugging"
+    ],
+    "prohibited": [
+      "speckit-specify",
+      "openspec-propose"
+    ]
+  },
+  "cross-module-feature": {
+    "any": [
+      "speckit-specify",
+      "openspec-propose"
+    ],
+    "prohibited": [
+      "systematic-debugging"
+    ]
+  },
+  "typo-fix": {
+    "any": [],
+    "prohibited": []
+  }
+}

method-core.lock.json

@@ -113,7 +113,12 @@
     {
       "mode": "100644",
       "path": "scripts/run_host_evals.py",
-      "sha256": "b9e06b1ae7b562521d57b04b1d530e14b98ac1f3f6484c8e4ca21fe40a0cb161"
+      "sha256": "a2d814af5c2091f39a486e72825fd42a6ced071efc3dca1ed15e3ff7c97af9b4"
+    },
+    {
+      "mode": "100644",
+      "path": "scripts/run_natural_evals.py",
+      "sha256": "85eac53c236601a6aa22a0c039d1e8a1e8eb9a9c245289676d6db3fef7afc174"
     },
     {
       "mode": "100755",
@@ -325,6 +330,11 @@
       "path": "tests/test_models.py",
       "sha256": "5f2a60c2d30d7248b3bc914ff44066d308e80e6db99d88001ce350dd0e95655f"
     },
+    {
+      "mode": "100644",
+      "path": "tests/test_natural_evals.py",
+      "sha256": "231e8d4698e1adfae7b2a4a35ded6965dced6563b47df2506254cd1e2995fd69"
+    },
     {
       "mode": "100644",
       "path": "tests/test_opencode_integration.py",
@@ -352,7 +362,7 @@
     }
   ],
   "repository": "https://github.com/phense/method-core.git",
-  "revision": "4300085b164686cc359a98cf7b3ee965cbd886b0",
+  "revision": "64d9a02272590630288c04eba3a76c7bdb05919f",
   "schema_version": 1,
   "version": "0.1.0"
 }

method.json

@@ -1,7 +1,7 @@
 {
   "schema_version": 1,
   "name": "engineering-method",
-  "version": "0.3.1",
+  "version": "0.3.2",
   "method_skills": [
     "architecture-modeling",
     "openspec-apply",

pyproject.toml

@@ -1,6 +1,6 @@
 [project]
 name = "engineering-method"
-version = "0.3.1"
+version = "0.3.2"
 description = "Risk-proportionate engineering workflows for Codex, Claude and OpenCode."
 requires-python = ">=3.11"
 license = { text = "MIT" }

scripts/run_host_evals.py

@@ -148,7 +148,8 @@ def native_failure_diagnostics(stdout, stderr):
 
 def execute(command, cwd, env, timeout):
     try:
-        process = subprocess.Popen(command, cwd=cwd, env=env, stdout=subprocess.PIPE,
+        # A host that inherits an open stdin waits for more input and never starts the turn.
+        process = subprocess.Popen(command, cwd=cwd, env=env, stdin=subprocess.DEVNULL, stdout=subprocess.PIPE,
                                    stderr=subprocess.PIPE, text=True, start_new_session=True)
     except FileNotFoundError as exc:
         raise EvalFailure("missing_command") from exc