Skip to content

docs: record Phase 13 UAT and close Phase 13 - #61

Open
sellke wants to merge 5 commits into
mainfrom
docs/phase-13-uat
Open

sellke wants to merge 5 commits into
mainfrom
docs/phase-13-uat

Conversation

@sellke

@sellke sellke commented Oct 22, 2026 •

Copy link
Copy Markdown
Owner

Summary

This PR records the human UAT for Phase 13 (Subtract and Repair) and closes the phase. 173 of 176 scenarios pass. One failure is tracked as an issue, and two scenarios can't be run without OpenClaw. The one regression UAT found is fixed in #62, so merge #62 first.

Changes

  • UAT plans (5 files): each scenario has its Status and evidence notes.
    • Setup blocks cd into their temp folder, and gc refuses any path outside it.
    • Stale steps are fixed in ci S18/S19, harness S7, names S17, and parity S8.
Plan Pass Fail Open
ci-and-update-integrity 43 0 0
harness-cuts 32 0 0
claude-code-parity 31 0 0
platform-neutral-names 38 1 (S9) 2 (S12, S13: OpenClaw)
shipped-command-hygiene 29 0 0
  • names S9 (Fail): headless claude -p claimed the adapter "doesn't define canvas-preview". Installed projects get no adapters/ folder, which the existing adapter-mappings issue tracks.
  • hygiene S29: run 1 locked a contract it never displayed. Run 2, on fix(create-spec): show the contract before the lock prompt #62, printed the contract before the lock prompt, and the package passed every check. It's marked Pass under the reworded discovery rule.
  • Product docs: /verify-spec --product found the docs lagging reality, and /plan-product --reconcile closed Phase 13:
    • Phase 13 is marked released in v0.43.0 and closed, with a condensed-history row.
    • CI success criterion 2 is marked met.
    • mission.md and roadmap.md are updated, and mission-lite.md and context.md are regenerated.
    • The check's report is .writ/product/verification-2026-10-22.md.
  • Issues filed:
    • bug: create-spec can lock an unseen contract (fixed in fix(create-spec): show the contract before the lock prompt #62)
    • improvement: UAT plan helpers can write to the repo
    • improvement: update-writ kept-file wording
    • improvement: prototype says nothing about an extra comparison page
    • improvement: create-spec discovery skips Plan Mode on Claude Code
  • Lesson: parallel UAT agents must not share scratch files. This follows the stray commit a8aa8d8, which was reset before any push.

Spec Reference

  • .writ/specs/2026-10-20-{ci-and-update-integrity,harness-cuts,claude-code-parity,platform-neutral-names,shipped-command-hygiene}/uat-plan.md
  • .writ/product/roadmap.md (Phase 13)

Test Results

Docs-only change. bash scripts/eval.sh passes on this branch (last run .writ/state/eval-20261022-175242.md, not committed).

Drift Report

No spec drift. Only the UAT records and product docs changed.

Review Notes

No pre-landing review performed.

🤖 Generated with Claude Code

sellke and others added 2 commits October 22, 2026 11:16
172 of 176 scenarios pass, 2 fail, and 2 are not runnable here.

- ci-and-update-integrity: 43/43 pass, including S8, the first live red CI run
- harness-cuts: 32/32 pass
- claude-code-parity: 31/31 pass
- platform-neutral-names: 38 pass, 1 fail, 2 open
  - fail: S9 (headless claude -p claimed the adapter does not define canvas-preview)
  - open: S12 and S13, which need OpenClaw
- shipped-command-hygiene: 28 pass, 1 fail
  - fail: S29 (/create-spec asked to lock a contract it never showed)

The plans also get setup-block guards: cd into the temp folder, and gc refuses any
path outside it. This follows the stray commit a8aa8d8 on 2026-10-21, which was reset
before any push. Stale steps in ci S18/S19, harness S7, names S17, and parity S8 are fixed.

Adds the lesson on parallel agents sharing scratch files.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- bug: /create-spec can ask to lock a contract it never showed (hygiene S29)
- /create-uat-plan setup helpers can commit into the project repo
- /update-writ kept-file wording does not match update.sh (ci S28)
- /prototype says nothing about an extra comparison page (names S8, S10)

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@sellke sellke added the documentation Improvements or additions to documentation label Oct 22, 2026
sellke and others added 2 commits October 22, 2026 12:55
Hygiene S29 was re-run on the fix branch (PR #62). The contract now prints
before the lock prompt, and the package passes every check. The maintainer
marked it Pass under the reworded discovery rule: independent bounded
choices may share one prompt. Final UAT: 173 of 176 pass, 1 fail (names S9),
2 not runnable without OpenClaw.

Neither S29 run entered Plan Mode, which ADR-001 asks for during discovery.
Filed as an improvement.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
/verify-spec --product found that both mission.md and roadmap.md still
called Phase 13 unreleased and pending UAT. The /plan-product --reconcile
pass that followed:

- marks Phase 13 closed 2026-10-22, released in v0.43.0
- adds its condensed-history row
- marks success criterion 2 met from the first CI runs (39678517441 green,
  39795856168 red as designed)
- records UAT as 173 of 176: names S9 tracked as an issue, S12/S13 need
  OpenClaw, S29's regression fixed in PR #62

No new ADR. mission-lite.md and context.md regenerated. Merge PR #62 first.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@sellke sellke changed the title docs(uat): record Phase 13 UAT results docs: record Phase 13 UAT and close Phase 13 Oct 22, 2026
PR #62 does every item in the issue's Expected Outcome: the contract is
printed before the lock prompt, the prompt has four options, and hygiene
S29 passes on a re-run. The maintainer chose not to write a spec for
work that is already done. The Plan Mode gap stays open as its own issue.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

documentation Improvements or additions to documentation

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant