Skip to content

fix(presets): recompose surviving skills when a lower preset layer is removed - #5013

Open
yxshee wants to merge 11 commits into
github:mainfrom
yxshee:fix/preset-remove-stale-composed-skill
Open

yxshee wants to merge 11 commits into
github:mainfrom
yxshee:fix/preset-remove-stale-composed-skill

Conversation

@yxshee

@yxshee yxshee commented Oct 17, 2026

Copy link
Copy Markdown
Contributor

Description

Removing a preset that is the base, or a middle layer, of another preset's prepend/append/wrap composition leaves the removed preset's content in the active agent's SKILL.md. That affects Claude, Codex, Copilot with --skills, and other skills-based integrations. specify preset resolve reports the new chain, and command-file agents such as Gemini are recomposed onto it. But the skill keeps the removed preset's instructions, so /speckit-specify runs content the user just removed.

Reproduction (alpha replaces speckit.specify with ALPHA BODY; beta appends BETA APPENDED):

$ specify init p --integration claude --ignore-agent-tools --non-interactive && cd p
$ specify preset add --dev ../alpha --priority 5
$ specify preset add --dev ../beta --priority 2
$ specify preset remove alpha
✓ Preset 'alpha' removed successfully
$ grep -c "ALPHA BODY" .claude/skills/speckit-specify/SKILL.md
1          # main; 0 with this change
$ specify preset resolve speckit.specify
    (top layer from: beta v1.0.0)
    1. [base] core (bundled) → ...
    2. [append] beta v1.0.0 → ...

The same steps with --integration gemini drop ALPHA BODY from .gemini/commands/speckit.specify.toml on main, so only skills output is stale.

Root cause, in presets/_manager_skills.py:

The fix:

  • _reconcile_skills() now records commands whose winning layer composes (strategy other than replace).
  • After a removal, it also re-renders those skills in the active directory, but only when the SKILL.md still carries the winning preset's metadata.source: preset:<id> marker. A new helper, _skills_owned_by_preset(), does that check.
  • A hand-edited skill is left alone, matching the existing protection for non-owned skills during removal.
  • The install and rescaffold paths are unchanged.

Testing

  • Tested locally with uv run specify --help
  • Ran existing tests with uv sync && uv run pytest
  • Tested with a sample project (if applicable): the reproduction above with claude (on main and this branch) and gemini (on main)

New tests in tests/specify_cli/presets/test_manager_skills.py:

  • test_remove_base_preset_recomposes_surviving_composed_skill: fails on main with assert "Base preset body" not in content
  • test_remove_middle_preset_drops_its_layer_from_composed_skill: fails on main with assert "Middle body" not in content
  • test_remove_base_preset_preserves_hand_edited_composed_skill: the negative case, guarding the hand-edit protection; passes before and after

With the change the file gives 70 passed.

  • Preset, integration, bundle and extension suites: 6353 passed, 9 skipped.
  • Full suite (macOS, Python 3.13): 12332 passed, 376 skipped. 11 failures in the tests/test_*_python_parity.py files occur identically on unmodified main on this machine under full-suite load; those files pass when run on their own (257 passed, 112 skipped).
  • One more local failure, test_resolver.py::TestResolveCore::test_resolve_core_does_not_return_preset_files, was an artifact of a worktree directory literally named presets: the test asserts "presets" not in path.parts. It fails identically on main in that directory and passes elsewhere.
  • uvx ruff@0.15.0 check src tests: clean

AI Disclosure

  • I did not use AI assistance for this contribution
  • I did use AI assistance (fill in the disclosure below)

AI disclosure: Claude Code (model: Claude Opus 5.5, max reasoning effort, autonomous agent mode) opened this PR on behalf of @yxshee. A Claude Code subagent (same model, autonomous) found the bug and wrote the fix and tests. The lead agent independently re-ran the new tests against main and the branch, reproduced the behavior with the real CLI (Claude and Gemini), confirmed the #3459 origin with git log -S, reviewed the diff, and wrote this description. No human reviewed the change line by line before it was opened.

🤖 Generated with Claude Code

… removed

Removing a preset that is the base (or a middle layer) of another preset's
prepend/append/wrap composition left the removed preset's content in the
active agent's SKILL.md (Claude, Codex, Copilot with --skills, ...), while
command-file agents such as Gemini were recomposed onto the new stack.

The composed skill is owned by the surviving preset at the top of the
chain, so _unregister_skills() restores nothing for the removed preset,
and _reconcile_skills() only revisited the active skills directory when
that restore had touched it. The recomposed .composed/<cmd>.md that
_reconcile_composed_commands() writes was therefore never rendered into
the skill, and `specify preset resolve` disagreed with what the agent ran.

After a removal, also reconcile active-directory skills for commands whose
winning layer composes, as long as the skill still carries the winning
preset's source marker. A hand-edited skill is left alone, matching the
existing protection for non-owned skills during removal.

Assisted-by: Claude Code (model: Claude Opus 5.5, autonomous)
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@yxshee
yxshee requested a review from mnriem as a code owner October 17, 2026 19:15
@mnriem mnriem added the triage-must-have Verdict: high-value, important work for Spec Kit — do first label Oct 18, 2026
@mnriem
mnriem requested a max review from Copilot October 19, 2026 13:03

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

Removal can overwrite a user-owned legacy skill despite the ownership filter.

1 open issue
What changed in this PR

Updates Spec Kit preset removal so surviving composed skills reflect the remaining preset layers.

Changes:

  • Re-renders active composed skills that retain the winning preset’s ownership marker.
  • Adds regression coverage for base/middle removal and preservation of non-owned skills.

Validation used an isolated ownership check; project tests were not run.

AI disclosure: GitHub Copilot (model: unknown, autonomous) generated this review for the requesting reviewer.

File Description
tests/​specify_cli/​presets/​test_manager_skills.py Adds composition-removal and preservation tests.
src/​specify_cli/​presets/​_manager_skills.py Adds ownership-based reconciliation for surviving composed skills.

Review effort: Max

When both the modern (speckit-specify) and legacy (speckit.specify)
skill directories exist, the active-directory recomposition added for
lower-layer removal selected the command because the modern skill kept
the winning preset's marker, and _register_skills() then rewrote both
directories, overwriting a user-owned legacy skill.

Give _register_skills() an optional only_skill_names filter and pass the
directory's managed names on removal, so only names that are still
preset-owned are written. Install and rescaffold calls pass None and
keep writing every existing directory, as before.

Assisted-by: Claude Code (model: Claude Opus 5.5, autonomous)
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@yxshee

yxshee commented Oct 19, 2026

Copy link
Copy Markdown
Contributor Author

Review round 1, posted on behalf of @yxshee.

Addressed the Copilot finding that removal recomposition could overwrite a user-owned legacy skill (commit 8c0328f):

  • _register_skills() gains an optional only_skill_names filter; removal passes the directory's managed names for the active directory. Install and rescaffold behavior is unchanged.
  • New regression test_remove_base_preset_preserves_user_owned_legacy_skill: fails on the previous head (1239cbe), passes now.
  • tests/specify_cli/presets, tests/integrations, tests/specify_cli/integrations, tests/specify_cli/extensions, tests/specify_cli/bundles, tests/test_presets.py, tests/test_extension_skills.py: 6096 passed, 9 skipped. uvx ruff@0.15.0 check src tests: clean.

AI disclosure: written and pushed by Claude Code (model: Claude Opus 5.5, max reasoning effort, autonomous agent mode); the code, test and this comment are AI-generated.

🤖 Addressed by Claude Code

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

Removing the sole composition base still re-renders cached instructions from the removed preset.

1 open issue
1 resolved since last review

Review effort: Max

@mnriem

mnriem commented Oct 19, 2026

Copy link
Copy Markdown
Collaborator

Please address Copilot feedback

@mnriem mnriem added the author-awaiting Waiting on author response label Oct 19, 2026
When the removed preset was the only replace base of a composing
command, with no core or extension fallback, resolution returns None.
Command-file agents unregister that command, but the survivor's stale
.composed/<cmd>.md stayed behind, and the removal recomposition rendered
it back into the skill, so the removed preset's instructions stayed
active.

Track composing winners whose stack no longer resolves, delete their
stale composed cache, and on removal delete the active skill only while
it still carries the winning preset's source marker. Hand-edited skills
are kept, matching the existing ownership protection.

Assisted-by: Claude Code (model: Claude Opus 5.5, autonomous)
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@yxshee

yxshee commented Oct 19, 2026

Copy link
Copy Markdown
Contributor Author

Review round 2, posted on behalf of @yxshee.

Addressed the Copilot finding about removing a command's only base with no core or extension fallback (commit 57bdfc9):

  • The stale .composed/<cmd>.md of a composing winner whose stack no longer resolves is deleted, so it can't be rendered back into a skill.
  • On removal, the active skill for that command is deleted only while it still carries the winner's source marker; hand-edited skills are kept.
  • New tests: test_remove_sole_base_preset_retires_stale_composed_skill (fails on the previous head, passes now) and test_remove_sole_base_preset_keeps_hand_edited_skill.
  • Preset, integration, extension and bundle suites: 6098 passed, 9 skipped. uvx ruff@0.15.0 check src tests: clean.

AI disclosure: posted on behalf of @yxshee by Claude Code (model: Claude Opus 5.5, max reasoning effort, autonomous agent mode); the fix, tests and this reply are AI-generated.

🤖 Addressed by Claude Code

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

Cache cleanup can delete external files, and removed preset instructions can remain runnable.

3 open issues
1 previously missed issue

In code that hasn't changed since last review

Medium severity Handle aliases during preset retirement and recomposition

src/​specify_cli/​presets/​_manager_skills.py:465

This loop checks only the primary command's modern and legacy names, not its manifest aliases. On Codex, an appending preset can declare speckit.alt as an alias of speckit.custom; removing its only base retires speckit-custom but leaves speckit-alt/SKILL.md runnable with the removed instructions. agents.py:934–1015 renders aliases separately, using <id>:<file> source markers, so the preset:<id> check also misses their ownership. Include the winning template's aliases and their actual ownership markers in recomposition and retirement, preserve user-owned files, and add a native-skill alias regression.

AI disclosure: GitHub Copilot (model: unknown, autonomous mode) generated this feedback for the requesting reviewer.

1 resolved since last review

Review effort: Max

Round-3 review of the no-base removal fix:

- Delete a cache only when no path component below the project root is a
  symlink, so a symlinked .composed directory can't redirect the
  deletion to a file outside the project.
- Drop every surviving preset's .composed/<cmd>.md for an uncomposable
  command, not only the winner's: a middle preset's stale cache could be
  rendered back into the skill by a later rescaffold.
- Delete through the journal-aware unlink helper, as the cache writes
  already do, so a failing integration lifecycle can restore the cache.

Assisted-by: Claude Code (model: Claude Opus 5.5, autonomous)
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@mnriem

mnriem commented Oct 19, 2026

Copy link
Copy Markdown
Collaborator

Please address Copilot feedback

@yxshee

yxshee commented Oct 19, 2026

Copy link
Copy Markdown
Contributor Author

Review round 3, posted on behalf of @yxshee.

Addressed the three Copilot findings on stale composed-cache deletion (commit c184e1a):

  • Deletion skips any cache reached through a symlinked path component below the project root.
  • All surviving presets' caches for an uncomposable command are dropped, so a rescaffold can't render the removed base back.
  • Deletion uses the journal-aware unlink helper, like the cache writes.
  • Three new tests, each failing on the previous head. Preset, integration, extension and bundle suites: 6101 passed, 9 skipped. uvx ruff@0.15.0 check src tests: clean.

AI disclosure: posted on behalf of @yxshee by Claude Code (model: Claude Opus 5.5, max reasoning effort, autonomous agent mode); the fix, tests and this reply are AI-generated.

🤖 Addressed by Claude Code

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 Needs a closer look

Destructive cleanup spans multiple lifecycle paths and still has unresolved safety and correctness issues.

1 open issue
3 previously missed issues

In code that hasn't changed since last review

Medium severity Invalidate stale caches for disabled presets

src/​specify_cli/​presets/​_manager_skills.py:208

presets_by_priority excludes disabled presets (_registry.py:227), so their stale caches survive this cleanup. With a replace base beneath two append presets, disable the middle preset and then remove the sole base: the middle cache still contains the removed instructions. Re-enabling it and rescaffolding can render that cache before reconciliation deletes it; rescaffold reconciliation does not retire the newly written skill. Invalidate caches for all installed presets, including disabled ones, and extend the three-layer regression to cover disabling and re-enabling the middle layer.

AI disclosure: GitHub Copilot (model: unknown, autonomous mode) generated and verified this feedback for the requesting reviewer.

Medium severity Preserve output when surviving layers cannot be read

src/​specify_cli/​presets/​_manager_skills.py:211

resolve_content() also returns None when a surviving layer cannot be read or decoded (_resolver.py:780–783,865–871). Treating every None as a missing base deletes the last valid cache and owned skill even though a replace base still exists. A dependency-free execution with a non-UTF-8 base confirmed both deletions. Only retire output when the stack has no replace layer; preserve existing output on read failures, as test_unregister_skills_in_dir_unreadable_core_template_skips already requires for restoration. Add a composed-stack regression for this case.

AI disclosure: GitHub Copilot (model: unknown, autonomous mode) generated and verified this feedback for the requesting reviewer.

Medium severity Journal skill-directory removals during force replacement

src/​specify_cli/​presets/​_manager_skills.py:434

This rmtree() bypasses the lifecycle file-change journal. Force replacement calls remove() from _manager.py:408; when that runs under an active journal, a later failure cannot restore this deleted skill, although cache deletion is observed. design/integration.md, “Storage, loading, and lifecycle,” requires preset replacements and generated artifacts to participate in the journal. Wrap this removal like the other skill deletions at lines 1377–1381 and 1644–1648, and extend the observer regression to check the skill-directory removal too.

AI disclosure: GitHub Copilot (model: unknown, autonomous mode) generated and verified this feedback for the requesting reviewer.

3 resolved since last review

Review effort: Max

…lures

Round-4 review of the no-base removal fix:

- Skip registry keys (and command names) that are not a single plain
  path segment, so a malformed key such as "../../../outside" cannot
  steer cache cleanup outside the project.
- Invalidate stale caches for every installed preset, disabled ones
  included: re-enabling a disabled composer and rescaffolding could
  otherwise render its stale cache back.
- Retire output only when the stack has no replace layer left.
  resolve_content() also returns None when a surviving layer can't be
  read; that keeps the existing cache and skill instead of deleting them.
- Journal the stale skill-directory removal like the other skill
  deletions, so a failing lifecycle can restore it.

Assisted-by: Claude Code (model: Claude Opus 5.5, autonomous)
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@mnriem

mnriem commented Oct 19, 2026

Copy link
Copy Markdown
Collaborator

Please address Copilot feedback

@yxshee

yxshee commented Oct 19, 2026

Copy link
Copy Markdown
Contributor Author

Review round 4, posted on behalf of @yxshee.

Addressed the new finding and the three previously missed ones (commit b9a2c35):

  • Traversing registry keys: cleanup accepts only plain single-segment preset ids and command names.
  • Disabled presets: stale caches are invalidated for every installed preset, so re-enabling and rescaffolding can't render the removed base back.
  • Read failures: output is retired only when no replace layer remains; an unreadable surviving base keeps the existing cache and skill.
  • Journal: the stale skill-directory removal is wrapped in before_file_change/after_file_change like the other skill deletions.
  • Regressions: test_stale_cache_cleanup_ignores_traversing_registry_key, test_remove_sole_base_drops_disabled_preset_cache, test_unreadable_surviving_base_keeps_composed_output, and the journal test now also checks the skill directory; each fails on the previous head.
  • Preset, integration, extension and bundle suites: 6104 passed, 9 skipped. uvx ruff@0.15.0 check src tests: clean.

AI disclosure: posted on behalf of @yxshee by Claude Code (model: Claude Opus 5.5, max reasoning effort, autonomous agent mode); the fix, tests and this reply are AI-generated.

🤖 Addressed by Claude Code

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 Needs a closer look

File-deleting reconciliation spans multiple lifecycle paths, with unresolved safety and ownership gaps requiring changes and human review.

1 open issue
3 previously missed issues

In code that hasn't changed since last review

Medium severity Limit historical-directory reconciliation to managed skill names

src/​specify_cli/​presets/​_manager_skills.py:395

The new name filter protects only the active directory. The historical-directory branch below still calls _register_skills() without only_skill_names, so it writes every existing modern/legacy sibling. If removal restored only speckit-specify in an inactive agent's directory and preserved a user-owned speckit.specify beside it, reconciliation overwrites that preserved file with the surviving preset's content. Pass managed_names to the targeted call too, and add a regression with mixed ownership in a non-active agent directory.

AI disclosure: GitHub Copilot (model: unknown, autonomous) generated this feedback for the requesting reviewer.

Medium severity Retire stale skills during activation and rescaffolding

src/​specify_cli/​presets/​_manager_skills.py:453

This retirement only runs during removal, but activation and rescaffolding call _reconcile_skills() without extra_skills_dirs. After removing a custom command's only base while Codex is active, a previously written Claude skill can retain the removed body when Claude is reactivated: registration has no base/cache, and this deletion is skipped. The same problem occurs when the sole composer was disabled during removal; re-enabling it lets registration render its stale cache before reconciliation deletes only the cache. Move ownership-checked retirement outside the removal-only guard, and add regressions for both flows.

AI disclosure: GitHub Copilot (model: unknown, autonomous) generated this feedback for the requesting reviewer.

Medium severity Handle native-skill aliases during recomposition and retirement

src/​specify_cli/​presets/​_manager_skills.py:520

Native-skill aliases still retain removed instructions. CommandRegistrar.register_commands() renders a separate skill for each manifest alias (agents.py:934–1015), but this ownership scan and _register_skills() consider only the primary command's modern/legacy names. For an appending preset with alias speckit.shortcut, removing its base refreshes the primary skill but leaves speckit-shortcut/SKILL.md stale. Include aliases in recomposition and retirement, and recognize their registrar-generated source markers (<preset-id>:<source-file>), which differ from the primary skill's preset:<id> marker. Add a native-skill alias regression.

AI disclosure: GitHub Copilot (model: unknown, autonomous) generated this feedback for the requesting reviewer.

1 resolved since last review

Review effort: Max

@mnriem

mnriem commented Oct 19, 2026

Copy link
Copy Markdown
Collaborator

Please address Copilot feedback

@yxshee

yxshee commented Oct 19, 2026

Copy link
Copy Markdown
Contributor Author

Review round 7, posted on behalf of @yxshee. No code changes in this round; here is why.

Pre-existing, outside this PR's change:

  1. Ownership across all surviving composers (r4318233397): skill ownership has always been tracked per winning preset. A lower surviving composer's skill in another agent's directory, or a lower composer's raw fragment registered on rescaffold, was never reconciled before this PR either. Fixing it means discovering ownership across every composer independently of which preset supplies the winning content, which reworks how reconciliation selects owners.
  2. Legacy flat-list registered_skills for survivors (r4318233493): remove() infers legacy provenance only for the removed preset, and that predates this PR. Inferring it for survivors before the scan is a separate migration change.

Introduced by the alias handling added in round 6 (3738814):
3. Alias not restored when the base is re-added (r4318233545), and
4. Alias ownership marker rewritten by a later native-agent rescaffold (r4318233595).

Before round 6, alias skills were never retired or refreshed on removal; they just kept stale content. Handling the full alias lifecycle (restore on re-add, a consistent marker across registrar and preset passes) is a design question of its own. I'd suggest removing the round-6 alias handling from this PR, so it keeps its original scope (primary skills) without these regressions, and moving alias lifecycle into the tracking issue below.

Proposal: I open a tracking issue listing all four with Copilot's reproduction scenarios (different-historical-owner, first-post-upgrade legacy provenance, remove → re-add base with an alias, recompose → rescaffold → remove composer). On this PR I'll either drop the round-6 alias handling or keep it as is, whichever you prefer.

AI disclosure: written by Claude Code (model: Claude Opus 5.5, max reasoning effort, autonomous agent mode) on behalf of @yxshee; this comment is AI-generated.

🤖 Addressed by Claude Code

@mnriem

mnriem commented Oct 20, 2026

Copy link
Copy Markdown
Collaborator

Thanks for working through the review feedback. The implementation has grown considerably, and I think we should step back and simplify it around the tracking and reconciliation mechanisms already in place.

Existing registration records should identify the affected destinations, ownership checks should determine which files can be changed, and the resolver should determine the replacement content. After removing a preset, those managed destinations should be refreshed from the remaining stack—or removed if nothing resolves—while preserving non-owned files.

The original gap appears to be that reconciliation skips affected skills owned by a surviving preset. Could you rework the fix around that distinction, rather than continuing to add removal-specific cases? Please retain the reproduction and useful regression coverage, and ensure any regressions introduced by this PR are resolved before merging rather than deferred to follow-up issues

…records

Rework the fix as requested in review: reconcile the destinations the
registry already tracks instead of adding removal-specific cases.

- On removal, _unregister_skills() only restores what the removed preset
  owned. Also collect the affected commands' skill destinations from the
  surviving presets' registered_skills (inferring per-agent provenance
  for legacy flat lists), keep those whose SKILL.md still carries that
  preset's marker, and pass them through the existing extra_skills_dirs
  reconciliation, so active and historical directories are refreshed
  from the remaining stack.
- When a composing winner's stack has no replace base left, nothing
  resolves: remove its preset-owned output (journaled) instead of
  rendering it. Ownership accepts both preset:<id> and the registrar's
  <id>:<file> marker; anything else is preserved.
- _register_skills() lets the resolver decide even when a .composed cache
  exists, so a stale cache is never rendered. This replaces the cache
  deletion and its path guards.
- Keep the only_skill_names filter so a user-owned modern/legacy sibling
  is not overwritten.
- Drop the alias handling added in an earlier round, which introduced the
  alias regressions; aliases behave as on main.

Assisted-by: Claude Code (model: Claude Opus 5.5, autonomous)
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@yxshee

yxshee commented Oct 20, 2026

Copy link
Copy Markdown
Contributor Author

Rework, posted on behalf of @yxshee (commit a9a2000). Thanks for the direction: the fix is now built on the existing tracking and reconciliation instead of removal-specific cases.

How it works now

  • Destinations come from registration records. On removal, _unregister_skills() still restores what the removed preset owned. In addition, the affected commands' skill destinations are read from the surviving presets' registered_skills, per agent, including inactive agents. Legacy flat lists go through the existing _infer_legacy_skill_provenance(). These feed the existing extra_skills_dirs reconciliation. This was the original gap: skills owned by a surviving preset were skipped.
  • Ownership decides what can change. A destination is refreshed only while its SKILL.md still carries that survivor's marker, so hand-edited files and modern/legacy siblings are left alone (the only_skill_names filter is kept).
  • The resolver decides the content. The existing per-directory pass re-renders the winner. When a composing winner's stack has no replace base left, nothing resolves and its preset-owned output is removed, journaled. Ownership accepts preset:<id> and the registrar's <id>:<file> marker. _register_skills() now asks the resolver even when a .composed cache exists, so a stale cache is never rendered. That replaces the cache-deletion code and its path guards, which are removed.

Regressions introduced by this PR: the two alias issues came from alias handling added in an earlier round. That handling is removed, so aliases behave exactly as on main.

Size: _manager_skills.py is +138/−3 against the merge base, down from +244/−11.

Coverage

  • The reproduction and earlier regressions are kept: base, middle and sole-base removal; hand-edited and legacy-sibling preservation in active and historical directories; unreadable surviving base; journaled removal.
  • The three-layer and disabled-composer tests are tightened to require no SKILL.md after a rescaffold.
  • Added tests: a skill owned by a lower survivor in a historical directory (with and without a fallback base), and a survivor with a legacy flat-list record.
  • 14 of the PR's tests fail on the merge base and pass now. Preset, integration, extension and bundle suites: 6109 passed, 9 skipped. uvx ruff@0.15.0 check src tests: clean. Merges cleanly with current main.

AI disclosure: written and pushed by Claude Code (model: Claude Opus 5.5, max reasoning effort, autonomous agent mode) on behalf of @yxshee; the code, tests and this comment are AI-generated.

🤖 Addressed by Claude Code

@yxshee

yxshee commented Nov 1, 2026

Copy link
Copy Markdown
Contributor Author

@mnriem a9a2000 is the simplified rework you asked for on Oct 20, so I think this is waiting on review now rather than on me. I checked it against today's main (7370e59): it still merges without conflicts, tests/test_presets.py, tests/integrations and tests/specify_cli/presets pass on the merged tree (4,017 passed, 9 skipped), and uvx ruff@0.15.0 check src tests is clean. No new commits since a9a2000.

Posted on behalf of @yxshee. AI disclosure: Claude Code (model: Claude Opus 5.5, autonomous) ran the merge and test check and drafted this comment. No code changes in this round.


Generated by Claude Code

@mnriem mnriem removed the author-awaiting Waiting on author response label Nov 2, 2026
@mnriem
mnriem requested an auto review from Copilot November 2, 2026 15:10

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

Removing the only replace base can leave native-skill aliases running removed instructions because cleanup overlooks their destinations and ownership markers.

3 open issues
2 resolved since last review

Review effort: Balanced (auto)

@mnriem

mnriem commented Nov 2, 2026

Copy link
Copy Markdown
Collaborator

Please address Copilot feedback. If not applicable, please explain why. Note we should stay within the limited scope, so please assess it against that too when making changes. Also please address security audit CI/CD issue

@mnriem mnriem added the author-awaiting Waiting on author response label Nov 2, 2026
For native skill agents (e.g. Claude) a preset's command aliases are
written by command registration and recorded in registered_commands, so
the skill reconciliation pass never saw them. When a layer under a
composing preset changed, the alias kept the old composition: removing
the only base deleted the primary skill but left the alias running the
removed base's text, and a core fallback recomposed the primary but not
the alias until the next rescaffold.

After skill reconciliation, read each surviving preset's
registered_commands for native skill agents, map aliases to their
primary through the preset manifest, and for aliases whose SKILL.md
still carries the preset's `<id>:` marker: re-render from the current
stack, or remove (journaled) when nothing is left to compose onto. On
install/rescaffold passes a recorded alias missing from the active
agent is rendered again, so an alias retired with its base comes back
when a new base makes the command composable.

Assisted-by: Claude Code (model: Claude Opus 5.5, autonomous)
@yxshee

yxshee commented Nov 2, 2026

Copy link
Copy Markdown
Contributor Author

cf44847 handles the native-skill aliases from the Nov 2 review.

For Claude-style agents, a preset's command aliases are written by command registration and recorded in registered_commands, so the skill pass never refreshed them. After skill reconciliation, the new _reconcile_preset_aliases reads each surviving preset's registered_commands for native skill agents and maps aliases to their primary through the preset manifest. It only acts on an alias whose SKILL.md still carries that preset's <id>: marker:

  • No base left: the alias is removed (journaled), the same as its primary skill, and a rescaffold doesn't bring it back.
  • Base removed with a fallback below: the alias is recomposed on removal and matches what a rescaffold renders.
  • New base installed after the alias was retired: install and rescaffold passes render the recorded alias again for the active agent, so it comes back with its primary.
  • Aliases stay in registered_commands only, so removing the composer after a rescaffold still cleans them up through the existing command cleanup.

Four new tests in test_manager_skills.py. Three fail on a9a2000; the fourth checks that a hand-edited alias is left alone. Full suite on cf44847: 12,356 passed, 380 skipped. The 2 failures (test_list_skips_unreadable_file, test_registry_unreadable_file_resets) also fail on main here because the container runs as root. uvx ruff@0.15.0 check src tests is clean, and the branch merges cleanly with main at 39cf3a7 (presets + integrations suites pass on the merged tree).

Posted on behalf of @yxshee. AI disclosure: Claude Code (model: Claude Opus 5.5, autonomous) reproduced the finding, wrote the fix and tests in cf44847 (commit carries an Assisted-by: trailer), ran the checks above, and drafted this comment.


Generated by Claude Code

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

Alias reconciliation can delete aliases when a surviving layer is unreadable, modify inactive agents during installation, and use the wrong agent syntax for shared-directory aliases.

3 open issues
3 resolved since last review

Review effort: Balanced (auto)

@mnriem

mnriem commented Nov 2, 2026

Copy link
Copy Markdown
Collaborator

Please address Copilot feedback

Follow-up to the Nov 2 review of the alias reconciliation:

- Outside a removal pass, render and restore aliases only in the active
  agent's directory (single-active rule, github#2948). Retiring an alias that
  is no longer composable is still allowed wherever it is recorded.
- Group recorded agents by resolved skills directory and render each
  shared directory once with the active agent's renderer, as
  _unregister_skills does, so codex/agy sharing .agents/skills keep the
  active agent's command-reference syntax.
- Retire an alias only when no replace layer remains. When a replace
  base is still present but resolve_content() fails (e.g. an
  undecodable layer), keep the alias, matching the primary-skill path.

Assisted-by: Claude Code (model: Claude Opus 5.5, autonomous)
@yxshee

yxshee commented Nov 3, 2026

Copy link
Copy Markdown
Contributor Author

960f902 addresses the three findings from the cf44847 review.

  • Install-time refreshes: outside a removal pass, aliases are rendered and restored only in the active agent's directory, so installing a new base while Codex is active no longer rewrites Claude's alias. Retiring an alias that can no longer be composed is still allowed wherever it is recorded.
  • Shared directories: recorded agents are grouped by resolved skills directory and each directory is rendered once, with the active agent's renderer when it shares the directory, the same grouping _unregister_skills() uses. Codex and agy sharing .agents/skills now get Codex's $speckit-plan references while Codex is active.
  • Resolution failures: an alias is retired only when no replace layer remains. If a replace base is still there but resolve_content() returns None (for example an undecodable layer), the alias is kept, matching the primary-skill path.

Three new tests in test_manager_skills.py cover these cases, and all three fail on cf44847. Full suite on 960f902: 12,359 passed, 380 skipped. The 2 failures (test_list_skips_unreadable_file, test_registry_unreadable_file_resets) also fail on main here because the container runs as root. uvx ruff@0.15.0 check src tests is clean, and the branch merges cleanly with main at 848541a. On the security audit: the Dependency audit check that failed on a9a2000 passed on cf44847, along with every other check.

Posted on behalf of @yxshee. AI disclosure: Claude Code (model: Claude Opus 5.5, autonomous) wrote the fix and tests in 960f902 (commit carries an Assisted-by: trailer), ran the checks above, and drafted this comment.


Generated by Claude Code

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

Removing a shadowed preset can overwrite an inactive Claude integration with previously unregistered content, violating the active-only requirement and requiring a targeted fix with regression coverage.

1 open issue
3 resolved since last review

Review effort: Balanced (auto)

@mnriem

mnriem commented Nov 3, 2026

Copy link
Copy Markdown
Collaborator

Please address Copilot feedback

Removing a lower preset refreshed every skill a survivor still owned by
rendering the global winner into it. When a higher preset was installed
later under another agent, that wrote its content and ownership marker
into an integration it never registered in (github#2948, active-only).

Each survivor-owned destination is now handled by its owner's layer:

- a replace owner never depended on the removed layer, so the skill is
  left untouched;
- a composing owner that is also the winner is rendered as before;
- a composing owner below the winner is rendered from its own layer and
  the layers under it (resolve_content() takes an optional layers slice,
  _register_skills() takes the resolved text), and retired when nothing
  is left under it to compose onto.

Assisted-by: Claude Code (model: Claude Opus 5.5, autonomous)
@yxshee

yxshee commented Nov 3, 2026 •

Copy link
Copy Markdown
Contributor Author

a0defa5 addresses the shadowed-preset finding (r4457842206).

The removal refresh rendered the global winner into every directory a survivor still owned, so top (installed under Codex only) was written into Claude's skill. The same leak also happened in the existing ..._owned_by_lower_survivor_in_historical_dir test, where middle appends: Claude's skill ended up as core + middle + top with a preset:top-preset marker. That test didn't check for it.

Now each survivor-owned destination is refreshed from its own owner:

  • If the owner's layer is replace, its output never depended on the removed preset, so the skill is left untouched (Copilot's scenario).
  • If the owner composes and is also the winner, nothing changes from before.
  • If the owner composes but a higher preset wins, the skill is rendered from the owner's layer and the layers below it. resolve_content() takes an optional layers= slice for this, and _register_skills() takes the resolved text. When nothing is left under the owner to compose onto, the owned skill is retired, as before.

Scope stays the removal pass. Install-time and active-agent paths are unchanged.

Tests:

  • New test_remove_shadowed_base_leaves_replace_survivor_in_historical_dir (Copilot's steps). Claude's skill must be byte-for-byte unchanged.
  • test_remove_base_reconciles_skill_owned_by_lower_survivor_in_historical_dir[core-fallback] now also asserts that Top body is absent and that the marker is still preset:middle-preset.
  • Both fail on 960f902 and pass with this commit. Full suite: 12,365 passed, 376 skipped. One local failure, test_resolve_core_does_not_return_preset_files, is caused by my worktree folder being named presets: it fails on 960f902 too. ruff@0.15.0 check src tests: clean.

AI disclosure: written with Claude Code (Claude Opus 5.5, autonomous), as in the earlier rounds.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

An unreadable surviving layer can incorrectly trigger deletion of a historical skill even when its replace base still exists, and this case needs regression coverage.

1 open issue
1 resolved since last review

Review effort: Balanced (auto)

Comment on lines +496 to +498
if composed is None:
# Nothing left under the owner to compose onto.
self._remove_owned_skills(skills_dir, names)
@mnriem

mnriem commented Nov 3, 2026

Copy link
Copy Markdown
Collaborator

Please address Copilot feedback

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

author-awaiting Waiting on author response triage-must-have Verdict: high-value, important work for Spec Kit — do first

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants