[Feature]: Replace incremental scaffold reconciliation with full regeneration and snapshot-based recovery #5058

Problem Statement

Extension, preset, workflow, workflow-step, and bundle changes currently rely on multiple incremental cleanup and reconciliation paths. Handling additions, removals, updates, precedence changes, and previously active integrations creates substantial lifecycle complexity.

Replace these paths with one predictable process:

Check → snapshot → uninstall → regenerate → commit or restore.

Proposed Solution

Use one shared regeneration operation for applicable component additions, removals, updates, enablement changes, and priority changes.

Two distinct manifest concepts

The recovery ZIP contains both the newly computed snapshot manifest and the original hash manifest unchanged.

These cannot be conflated: with --force, a locally modified file’s actual snapshot hash differs from its expected scaffold hash. Recovery must restore the modified bytes and the original baseline so that the file remains recognized as modified.

1. Validate and check

Determine the desired installed state and validate that the inputs needed for regeneration are available.

Check every file in the existing hash manifest. Report modified, missing, or unexpectedly replaced paths and abort without mutation unless --force permits proceeding.

--force authorizes discarding scaffold customizations during regeneration. It does not bypass path safety, trust requirements, or ownership boundaries.

2. Create the recovery ZIP

Before any destructive work, create and verify a local ZIP containing:

• All existing hash-tracked scaffold files, preserving their actual current contents—including modifications accepted with --force.
• The original hash manifest, unchanged.
• Native settings files that the operation will modify.
• Project/component registries, configuration, and source files that the operation will replace or remove and that are needed for recovery.
• The snapshot manifest, computed from the exact bytes archived.

The snapshot manifest also records affected paths that did not previously exist, allowing recovery to restore absence rather than leave newly introduced files behind.

Create the snapshot whenever regeneration proceeds, with or without --force. If snapshot creation or verification fails, do not begin teardown.

Keep snapshots local, access-restricted, and outside version control; native settings can contain sensitive configuration.

3. Uninstall and remove existing scaffolding

Remove every file in the existing scaffold hash manifest. When --force is supplied, this includes locally modified files.

Keep the ownership boundary simple:

• Scaffold inventory: only wholly owned, disposable Spec Kit files; no partial/shared ownership categories.
• .specify/events.py: wholly owned by Spec Kit, tracked once at project level, removed and regenerated if required.
• Native settings: outside the scaffold hash inventory. Integration/events uninstall removes only Spec Kit’s contributions, preserving unrelated settings and subsequent user edits.

Normal uninstall reverses Spec Kit’s native settings edits; it does not overwrite a settings file with an old installation-time copy.

4. Apply the change and regenerate

Apply the requested component change and regenerate scaffolding for every installed integration, not just the active one, using existing precedence rules.

Build a new, in-progress hash manifest outside the recovery ZIP:

• Persist each generated destination before writing, so failed or partial writes remain discoverable.
• Record the resulting hash after a successful write.
• Do not replace the original authoritative hash manifest until the operation succeeds.

Preserve unchanged versions, provenance, priorities, enablement, integration options, configuration, authored artifacts, and workflow execution state. Do not unnecessarily redownload or reinstall unchanged source packages.

5. Commit or restore

On success: finalize the new hash manifest as the authoritative scaffold inventory.

On failure:

1. Clean up generated output from the failed attempt using its in-progress hash manifest.
2. Restore files, settings, and metadata from the ZIP using the snapshot manifest.
3. Restore the original hash manifest unchanged.

Unlike ordinary uninstall, failure recovery restores the immediate pre-operation settings snapshot.

Report recovery failures explicitly and retain the recovery artifacts. Document manual restoration, including warnings before overwriting subsequent user changes.

Alternatives Considered

• Continue maintaining separate incremental reconciliation paths.
• Regenerate only the active integration and defer other installed integrations.
• Reinstall every source package unnecessarily.
• Rely on Git alone, which does not reliably cover ignored files or uncommitted local state.

Component

Specify CLI (initialization, commands)

AI Agent (if applicable)

All agents

Use Cases

• Removing an override consistently restores the surviving contribution.
• Updates and enablement changes leave no stale generated commands or skills.
• Forced regeneration remains recoverable without losing accepted local modifications permanently.
• Teams commit the upstream-documented none baseline while developers select their chosen integration locally.

Acceptance Criteria

• Relevant component changes use one shared regeneration operation.
• Non-forced conflicts abort before mutation and report affected paths.
• Scaffold hash-manifest entries represent wholly owned files, including project-owned events.py.
• The verified recovery ZIP contains a snapshot manifest, the unchanged original hash manifest, and all required recovery contents.
• Forced teardown removes modified tracked files only after successfully snapshotting them.
• Native settings remain outside the scaffold inventory; uninstall removes only Spec Kit’s contributions.
• Regeneration covers every installed integration, matches a fresh scaffold of the desired state, and is stable when repeated.
• The in-progress hash manifest records destinations before writes; the authoritative manifest changes only after success.
• Recovery restores files, original baseline hashes, metadata, and originally absent paths consistently.
• Positive and negative coverage includes snapshot failure, partial writes, regeneration failure, and restoration failure.
• Documentation explains --force, snapshot storage, and recovery.
• Performance measurements include snapshot overhead and prioritize one to three integrations.

Additional Context

Exploratory feasibility benchmark: macOS ARM64, Python 3.13.16, seven warm-local samples per case.

Extension/preset changes plus full regeneration across three integrations took 0.68–0.90 seconds median.

The prototype matched fresh generated output, produced stable repeated builds, and passed modified/deleted-output refusal and forced-restoration checks.

These measurements exclude ZIP snapshot creation/restoration, CLI startup, and downloads. They used synthetic extension/preset packages on commit 6ad20824, not current upstream. Workflow, step, and bundle inputs were preserved, but their lifecycle mutations, native-event interactions, and failure rollback were not benchmarked.

The evidence supports pursuing the design, not claiming production readiness.

AI Disclosure

GitHub Copilot, powered by GPT-6.1 Sol, assisted with repository investigation, the benchmark prototype, analysis, and this issue draft under human supervision. Reasoning-effort settings were not recorded.