From f43b4353772d6c528407d5b206ffdd5e1339cb60 Mon Sep 17 00:00:00 2001 From: Ed Harrod Date: Sun, 25 Oct 2026 18:26:36 +0000 Subject: [PATCH 1/3] feat(droid): add --no-model-invocation integration option Droid skills are always written with `disable-model-invocation: false`. `specify init --integration droid --integration-options="--no-model-invocation"` now writes `true` for the core command skills (and preset overrides of them) so Droid does not invoke them automatically; they stay available as `/speckit-`. The default is unchanged (#2654). Mirrors the Claude option from #4919: setup() records the option, and post_process_skill_content() applies it only to `speckit-` skills in CORE_COMMAND_NAMES, so extension skills stay model-invocable for mandatory hooks. The option persists in .specify/integration.json and is reapplied by `specify integration upgrade droid`. Commands that rewrite skills without calling setup() (preset add/remove/update, integration use/switch) keep the default; the docs say so and point at `upgrade droid --force`. Assisted-by: Claude Code (model: claude-sonnet-5-5, autonomous) Co-Authored-By: Claude Sonnet 5.5 Claude-Session: https://claude.ai/code/session_01J3taepmtmjGTxTn4W1xb7q --- docs/integrations/droid.md | 36 ++- .../integrations/droid/__init__.py | 80 +++++- tests/integrations/test_integration_droid.py | 248 ++++++++++++++++++ 3 files changed, 360 insertions(+), 4 deletions(-) diff --git a/docs/integrations/droid.md b/docs/integrations/droid.md index 59461ed698..8e0e124609 100644 --- a/docs/integrations/droid.md +++ b/docs/integrations/droid.md @@ -23,7 +23,39 @@ Skills are installed as `.factory/skills/speckit-/SKILL.md`. Invoke them in the agent's chat, for example `/speckit-specify` or `/speckit-bug-assess` for the installed bug extension. -Generated skills are both user- and model-invocable. +Generated skills are both user- and model-invocable by default. + +## User-Invocable-Only Skills + +To stop Droid invoking the core Spec Kit command skills automatically: + +```bash +specify init my-project --integration droid --integration-options="--no-model-invocation" +``` + +This writes `disable-model-invocation: true` into the core command skills. +Droid runs them only when the user invokes `/speckit-`. Factory's +[skills documentation](https://docs.factory.com/cli/configuration/skills) does +not say whether a skill with this flag keeps its description out of Droid's +context. + +- Extension skills stay model-invocable, because mandatory extension hooks rely + on the model invoking them. If an extension's mandatory hook runs a core + command (its `command:` is a core `speckit.`), leave this option off: + the model cannot run that hook while the core skill is user-invocable only. +- The option is stored with the integration's other options and reapplied by + `specify integration upgrade droid`. Passing `--integration-options` to + `upgrade` replaces all stored options; `--integration-options=""` clears + them. Re-running `specify init` likewise stores only the options passed to it. +- Core command skills that a preset overrides, or stops overriding, are reverted + to the default by commands that re-register preset skills without reinstalling + Droid: commands that install, remove or update presets (including `specify + bundle` commands that carry presets), `specify integration use droid`, and + `specify integration switch droid` when Droid is installed but not the + default. To reapply the option, run `specify integration upgrade droid + --force` while Droid is the default integration (`--force` is needed because + those commands modify installed skill files, and it overwrites local edits to + Spec Kit-managed files). ## Workflow Dispatch @@ -32,4 +64,4 @@ Workflow steps use `droid exec `, forwarding `--model` and requesting permission-bypass flag. Operators can explicitly configure permission flags through `SPECKIT_INTEGRATION_DROID_EXTRA_ARGS`; those arguments follow Spec Kit's model/output flags and may override them under duplicate-flag parsing. -There are no integration-specific setup options. +The only integration-specific setup option is `--no-model-invocation`. diff --git a/src/specify_cli/integrations/droid/__init__.py b/src/specify_cli/integrations/droid/__init__.py index fb165f5309..dc789b4acd 100644 --- a/src/specify_cli/integrations/droid/__init__.py +++ b/src/specify_cli/integrations/droid/__init__.py @@ -14,7 +14,8 @@ from collections.abc import Mapping, Sequence from typing import Any -from ..base import SkillsIntegration +from ..base import IntegrationOption, SkillsIntegration +from ..manifest import IntegrationManifest class DroidIntegration(SkillsIntegration): @@ -36,6 +37,48 @@ class DroidIntegration(SkillsIntegration): } multi_install_safe = True + # Set by ``setup()`` from ``--no-model-invocation``; applies to core + # command skills only, since mandatory extension hooks rely on the model + # invoking the hook skill. Integrations are process-wide singletons, so + # preset overrides re-registered later in the same run see the same value. + # Commands that rewrite skills without calling ``setup()`` (e.g. + # ``preset add``, ``integration use``) get the default. + _disable_model_invocation: bool = False + + @classmethod + def options(cls) -> list[IntegrationOption]: + opts = super().options() + opts.append( + IntegrationOption( + "--no-model-invocation", + is_flag=True, + default=False, + help=( + "Install core command skills as user-invocable only " + "(disable-model-invocation: true)" + ), + ) + ) + return opts + + def setup( + self, + project_root: Path, + manifest: IntegrationManifest, + parsed_options: dict[str, Any] | None = None, + **opts: Any, + ) -> list[Path]: + self._disable_model_invocation = ( + (parsed_options or {}).get("no_model_invocation") is True + ) + try: + return super().setup( + project_root, manifest, parsed_options=parsed_options, **opts + ) + except BaseException: + self._disable_model_invocation = False + raise + @staticmethod def _inject_frontmatter_flag(content: str, key: str, value: str = "true") -> str: """Insert ``key: value`` before the closing ``---`` if not already present. @@ -75,6 +118,30 @@ def _inject_frontmatter_flag(content: str, key: str, value: str = "true") -> str out.append(line) return "".join(out) + @staticmethod + def _skill_name_from_content(content: str) -> str | None: + """Return the ``name:`` field of a skill's first frontmatter block.""" + dash_count = 0 + for line in content.splitlines(): + stripped = line.rstrip("\r\n") + if stripped == "---": + dash_count += 1 + if dash_count == 2: + break + continue + if dash_count == 1 and stripped.startswith("name:"): + name = stripped[len("name:"):].strip().strip('"').strip("'") + return name or None + return None + + @staticmethod + def _is_core_command_skill(name: str | None) -> bool: + from specify_cli.extensions import CORE_COMMAND_NAMES # lazy: avoids circular import + + return bool(name) and name.startswith("speckit-") and ( + name.removeprefix("speckit-") in CORE_COMMAND_NAMES + ) + def post_process_skill_content(self, content: str) -> str: """Inject Droid-specific skill frontmatter flags. @@ -82,10 +149,19 @@ def post_process_skill_content(self, content: str) -> str: hyphenated ``/speckit-`` invocations, not dotted ``/speckit.``) and the Droid-specific ``user-invocable`` / ``disable-model-invocation`` frontmatter flags so skills are both user- and Droid-invocable. + + ``disable-model-invocation`` is ``true`` for core command skills when + the last ``setup()`` in this process was given + ``--no-model-invocation``, else ``false``. """ updated = super().post_process_skill_content(content) updated = self._inject_frontmatter_flag(updated, "user-invocable") - updated = self._inject_frontmatter_flag(updated, "disable-model-invocation", "false") + disable = self._disable_model_invocation and self._is_core_command_skill( + self._skill_name_from_content(updated) + ) + updated = self._inject_frontmatter_flag( + updated, "disable-model-invocation", "true" if disable else "false" + ) return updated def build_exec_args( diff --git a/tests/integrations/test_integration_droid.py b/tests/integrations/test_integration_droid.py index 09848af930..434256894d 100644 --- a/tests/integrations/test_integration_droid.py +++ b/tests/integrations/test_integration_droid.py @@ -1,15 +1,38 @@ """Tests for DroidIntegration (Factory Droid CLI).""" +import json +import os +from pathlib import Path +from unittest.mock import patch from urllib.parse import urlparse import pytest +import yaml from specify_cli.integrations import get_integration +from specify_cli.integrations.base import SkillsIntegration from specify_cli.integrations.droid import DroidIntegration from specify_cli.integrations.manifest import IntegrationManifest from .test_integration_base_skills import SkillsIntegrationTests +REPO_ROOT = Path(__file__).resolve().parents[2] + + +@pytest.fixture(autouse=True) +def _reset_droid_model_invocation(): + """``setup()`` stores ``--no-model-invocation`` on the process-wide Droid + singleton; keep it from leaking between tests.""" + droid = get_integration("droid") + droid._disable_model_invocation = False + yield + droid._disable_model_invocation = False + + +def _skill_frontmatter(skill_file: Path) -> dict: + content = skill_file.read_text(encoding="utf-8") + return yaml.safe_load(content.split("---", 2)[1]) + class TestDroidIntegration(SkillsIntegrationTests): KEY = "droid" @@ -274,3 +297,228 @@ def test_build_command_invocation_uses_hyphenated_skill_name(self): "/speckit-plan feature-x" ) assert i.build_command_invocation("plan") == "/speckit-plan" + + +class TestDroidNoModelInvocationOption: + """``--integration-options="--no-model-invocation"`` opts out of model invocation (#2654).""" + + def test_options_include_no_model_invocation_flag(self): + opts = get_integration("droid").options() + matching = [o for o in opts if o.name == "--no-model-invocation"] + assert len(matching) == 1 + assert matching[0].is_flag is True + # Opt-in: skills stay model-invocable unless the flag is passed. + assert matching[0].default is False + + def test_setup_with_option_sets_disable_model_invocation_true(self, tmp_path): + i = get_integration("droid") + m = IntegrationManifest("droid", tmp_path) + created = i.setup( + tmp_path, m, parsed_options={"no_model_invocation": True}, script_type="sh" + ) + skill_files = [f for f in created if f.name == "SKILL.md"] + assert len(skill_files) > 0 + for f in skill_files: + fm = _skill_frontmatter(f) + assert fm["disable-model-invocation"] is True, f.parent.name + assert fm["user-invocable"] is True, f.parent.name + assert f.read_text(encoding="utf-8").count("disable-model-invocation:") == 1 + + def test_post_process_follows_option_from_setup(self, tmp_path): + """Preset overrides of core commands re-registered in the same run as + ``setup()`` (init, upgrade) get the same value.""" + i = get_integration("droid") + i.setup( + tmp_path, + IntegrationManifest("droid", tmp_path), + parsed_options={"no_model_invocation": True}, + script_type="sh", + ) + result = i.post_process_skill_content("---\nname: speckit-plan\n---\n\nBody\n") + assert "disable-model-invocation: true" in result + assert "disable-model-invocation: false" not in result + + def test_extension_skills_stay_model_invocable(self, tmp_path): + """Mandatory extension hooks rely on the model invoking the hook skill, + so the option never applies to extension commands.""" + i = get_integration("droid") + i.setup( + tmp_path, + IntegrationManifest("droid", tmp_path), + parsed_options={"no_model_invocation": True}, + script_type="sh", + ) + result = i.post_process_skill_content( + "---\nname: speckit-git-feature\n---\n\nBody\n" + ) + assert "disable-model-invocation: false" in result + assert "disable-model-invocation: true" not in result + + def test_option_requires_speckit_prefixed_core_name(self, tmp_path): + """A skill merely named like a core stem is not a core command skill.""" + i = get_integration("droid") + i.setup( + tmp_path, + IntegrationManifest("droid", tmp_path), + parsed_options={"no_model_invocation": True}, + script_type="sh", + ) + result = i.post_process_skill_content("---\nname: tasks\n---\n\nBody\n") + assert "disable-model-invocation: false" in result + + def test_failed_setup_does_not_leave_option_set(self, tmp_path): + i = get_integration("droid") + with patch.object(SkillsIntegration, "setup", side_effect=RuntimeError("boom")): + with pytest.raises(RuntimeError): + i.setup( + tmp_path, + IntegrationManifest("droid", tmp_path), + parsed_options={"no_model_invocation": True}, + script_type="sh", + ) + result = i.post_process_skill_content("---\nname: speckit-plan\n---\n\nBody\n") + assert "disable-model-invocation: false" in result + + def test_setup_without_option_resets_to_false(self, tmp_path): + i = get_integration("droid") + first = tmp_path / "first" + first.mkdir() + i.setup( + first, + IntegrationManifest("droid", first), + parsed_options={"no_model_invocation": True}, + script_type="sh", + ) + second = tmp_path / "second" + second.mkdir() + created = i.setup(second, IntegrationManifest("droid", second), script_type="sh") + skill_files = [f for f in created if f.name == "SKILL.md"] + assert len(skill_files) > 0 + for f in skill_files: + assert _skill_frontmatter(f)["disable-model-invocation"] is False, f.parent.name + result = i.post_process_skill_content("---\nname: speckit-plan\n---\n\nBody\n") + assert "disable-model-invocation: false" in result + + def test_cli_rejects_value_for_flag(self, tmp_path): + from typer.testing import CliRunner + from specify_cli import app + + project = tmp_path / "droid-bad-option" + project.mkdir() + old_cwd = os.getcwd() + try: + os.chdir(project) + result = CliRunner().invoke( + app, + [ + "init", + "--here", + "--integration", + "droid", + "--integration-options", + "--no-model-invocation=yes", + "--script", + "sh", + "--ignore-agent-tools", + ], + catch_exceptions=False, + ) + finally: + os.chdir(old_cwd) + + assert result.exit_code != 0 + assert "does not accept a value" in result.output + assert not (project / ".factory" / "skills").exists() + + def test_option_persists_across_upgrade_and_can_be_cleared(self, tmp_path): + from tests.specify_cli.integrations._helpers import _init_project, _run_in_project + + project = _init_project(tmp_path, "droid", "--no-model-invocation") + skills_dir = project / ".factory" / "skills" + skill_files = sorted(skills_dir.glob("*/SKILL.md")) + assert len(skill_files) > 0 + for f in skill_files: + assert _skill_frontmatter(f)["disable-model-invocation"] is True, f.parent.name + + state = json.loads((project / ".specify" / "integration.json").read_text(encoding="utf-8")) + settings = state["integration_settings"]["droid"] + assert settings["raw_options"] == "--no-model-invocation" + assert settings["parsed_options"] == {"no_model_invocation": True} + + # A later ``specify integration upgrade`` is a new process: drop the + # in-memory value so only the persisted options can supply it. + get_integration("droid")._disable_model_invocation = False + result = _run_in_project(project, ["integration", "upgrade", "droid", "--force"]) + assert result.exit_code == 0, result.output + for f in skill_files: + assert _skill_frontmatter(f)["disable-model-invocation"] is True, f.parent.name + + # Passing replacement options (here: none) clears it. + result = _run_in_project( + project, + ["integration", "upgrade", "droid", "--force", "--integration-options", ""], + ) + assert result.exit_code == 0, result.output + for f in skill_files: + assert _skill_frontmatter(f)["disable-model-invocation"] is False, f.parent.name + state = json.loads((project / ".specify" / "integration.json").read_text(encoding="utf-8")) + assert "parsed_options" not in state["integration_settings"]["droid"] + + def test_upgrade_reapplies_option_to_preset_overrides(self, tmp_path): + """``preset add`` runs without ``setup()`` (documented limitation); + ``integration upgrade droid --force`` reapplies the option to the + core-command skills a preset overrides.""" + from tests.specify_cli.integrations._helpers import _init_project, _run_in_project + + project = _init_project(tmp_path, "droid", "--no-model-invocation") + skill_file = project / ".factory" / "skills" / "speckit-plan" / "SKILL.md" + assert _skill_frontmatter(skill_file)["disable-model-invocation"] is True + + get_integration("droid")._disable_model_invocation = False + result = _run_in_project( + project, ["preset", "add", "--dev", str(REPO_ROOT / "presets" / "lean")] + ) + assert result.exit_code == 0, result.output + fm = _skill_frontmatter(skill_file) + assert fm["metadata"]["source"] == "preset:lean" + # Documented in docs/integrations/droid.md; update the docs if + # preset commands start honouring the option. + assert fm["disable-model-invocation"] is False + + # The docs say ``--force`` is needed here: preset add modified + # installed skill files, so a plain upgrade refuses. + result = _run_in_project(project, ["integration", "upgrade", "droid"]) + assert result.exit_code != 0 + assert "modified since installation" in result.output + assert _skill_frontmatter(skill_file)["disable-model-invocation"] is False + + get_integration("droid")._disable_model_invocation = False + result = _run_in_project(project, ["integration", "upgrade", "droid", "--force"]) + assert result.exit_code == 0, result.output + fm = _skill_frontmatter(skill_file) + assert fm["metadata"]["source"] == "preset:lean" + assert fm["disable-model-invocation"] is True + + def test_extension_hook_skills_stay_model_invocable_after_upgrade(self, tmp_path): + """``integration upgrade`` re-registers extension skills in the same + process as ``setup()``; they must keep ``false`` so mandatory hooks + (e.g. ``speckit.git.feature``) can still be invoked by the model.""" + from tests.specify_cli.integrations._helpers import _init_project, _run_in_project + + project = _init_project(tmp_path, "droid", "--no-model-invocation") + result = _run_in_project( + project, ["extension", "add", "--dev", str(REPO_ROOT / "extensions" / "git")] + ) + assert result.exit_code == 0, result.output + get_integration("droid")._disable_model_invocation = False + result = _run_in_project(project, ["integration", "upgrade", "droid", "--force"]) + assert result.exit_code == 0, result.output + + skills_dir = project / ".factory" / "skills" + git_skills = sorted(skills_dir.glob("speckit-git-*/SKILL.md")) + assert git_skills, "git extension skills were not generated" + for f in git_skills: + assert _skill_frontmatter(f)["disable-model-invocation"] is False, f.parent.name + assert _skill_frontmatter(skills_dir / "speckit-specify" / "SKILL.md")[ + "disable-model-invocation" + ] is True From 90170ded07b199e8285b138a31ca8efeb63f8772 Mon Sep 17 00:00:00 2001 From: Ed Harrod Date: Wed, 28 Oct 2026 10:31:14 +0000 Subject: [PATCH 2/3] docs(droid): note skills are model-invocable by default The post_process_skill_content() docstring said skills are always both user- and Droid-invocable, which is no longer true once --no-model-invocation is set. Assisted-by: Claude Code (model: claude-opus-5-5, autonomous) Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_015w5WirGUspLsYEpCkaHWxV --- src/specify_cli/integrations/droid/__init__.py | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/src/specify_cli/integrations/droid/__init__.py b/src/specify_cli/integrations/droid/__init__.py index dc789b4acd..3bf9f39bd9 100644 --- a/src/specify_cli/integrations/droid/__init__.py +++ b/src/specify_cli/integrations/droid/__init__.py @@ -148,7 +148,8 @@ def post_process_skill_content(self, content: str) -> str: Applies the shared hook-command normalization note (skills agents use hyphenated ``/speckit-`` invocations, not dotted ``/speckit.``) and the Droid-specific ``user-invocable`` / ``disable-model-invocation`` - frontmatter flags so skills are both user- and Droid-invocable. + frontmatter flags so skills are both user- and Droid-invocable by + default. ``disable-model-invocation`` is ``true`` for core command skills when the last ``setup()`` in this process was given From fc261308bdc0c7b610d254c296cd08275f9e78a9 Mon Sep 17 00:00:00 2001 From: Ed Harrod Date: Fri, 30 Oct 2026 14:38:45 +0000 Subject: [PATCH 3/3] docs(droid): record observed --no-model-invocation behaviour Validated against Droid 1.0.0: flagged core skills drop out of the model's skill listing, /speckit- and workflow dispatch still run them, and the model can still invoke them via its Skill tool when a prompt names one. Assisted-by: Claude Code (model: claude-opus-5-5, autonomous) --- docs/integrations/droid.md | 9 ++++++--- 1 file changed, 6 insertions(+), 3 deletions(-) diff --git a/docs/integrations/droid.md b/docs/integrations/droid.md index 8e0e124609..fd3b795e1e 100644 --- a/docs/integrations/droid.md +++ b/docs/integrations/droid.md @@ -34,10 +34,13 @@ specify init my-project --integration droid --integration-options="--no-model-in ``` This writes `disable-model-invocation: true` into the core command skills. -Droid runs them only when the user invokes `/speckit-`. Factory's +In Droid 1.0.0 the model no longer lists them among its available skills, so it +does not run them unprompted, while `/speckit-` and workflow dispatch +still run them. It is not a hard block: a prompt that names a skill (for +example "use the speckit-specify skill") still lets the model invoke it through +its `Skill` tool. Factory's [skills documentation](https://docs.factory.com/cli/configuration/skills) does -not say whether a skill with this flag keeps its description out of Droid's -context. +not describe this behaviour. - Extension skills stay model-invocable, because mandatory extension hooks rely on the model invoking them. If an extension's mandatory hook runs a core