Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion install.sh
Original file line number Diff line number Diff line change
Expand Up @@ -145,7 +145,7 @@ copy_file() {
fi
}

for script in super-board-skills-ready.py super-board-calibration.py super-board-verification-ready.py super-board-github-read.py super-board-collect-state.py super-board-run.sh super-board-gh-guard.sh super-board-card.sh super-board-status.py super-board-wave-plan.sh super-board-codex-lane.sh super-board-codex-wave.sh super-board-deps.sh super-board-preflight.sh super-board-merge-gate.sh super-board-merge-policy.py super-board-approval.py super-board-env-check.sh super-board-agents-md.py super-board-settings.py super-board-setup.py super-board-usage.sh super-board-throttle.sh super-board-host.sh super-board-pr-body.sh super-review-file-refactor.sh super-qa-file-bug.sh super-board-stop.sh; do
for script in super-board-project-verifier.py super-board-skills-ready.py super-board-calibration.py super-board-verification-ready.py super-board-github-read.py super-board-collect-state.py super-board-run.sh super-board-gh-guard.sh super-board-card.sh super-board-status.py super-board-wave-plan.sh super-board-codex-lane.sh super-board-codex-wave.sh super-board-deps.sh super-board-preflight.sh super-board-merge-gate.sh super-board-merge-policy.py super-board-approval.py super-board-env-check.sh super-board-agents-md.py super-board-settings.py super-board-setup.py super-board-usage.sh super-board-throttle.sh super-board-host.sh super-board-pr-body.sh super-review-file-refactor.sh super-qa-file-bug.sh super-board-stop.sh; do
if [ -f "$REPO_ROOT/scripts/$script" ]; then
copy_file "$REPO_ROOT/scripts/$script" "$TARGET/.claude/bin/$script"
chmod +x "$TARGET/.claude/bin/$script"
Expand Down
222 changes: 222 additions & 0 deletions scripts/super-board-project-verifier.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,222 @@
#!/usr/bin/env python3
"""Read-only project verifier map inspection and affected-feature selection.

Exit 0: structurally valid (no scenarios executed); 9: missing/stale setup;
64: malformed input. Uses project files only, without git, shell or host access.
"""
import argparse
import fnmatch
from functools import lru_cache
import glob
import hashlib
import json
import ntpath
from pathlib import Path
import sys


class InputError(Exception):
def __init__(self, message, code=64):
super().__init__(message)
self.code = code


def relative(value, label, pattern=False):
if not isinstance(value, str) or not value.strip():
raise InputError(f'{label}: expected a nonempty repo-relative path')
if ('\x00' in value or '\\' in value or ntpath.splitdrive(value)[0]
or value.startswith('/') or any(part in ('', '.', '..') for part in value.split('/'))):
raise InputError(f'{label}: absolute, traversal or noncanonical path')
if not pattern and glob.has_magic(value):
raise InputError(f'{label}: patterns are allowed only in owners')
return value


def contained(root, path, label):
try:
resolved = path.resolve()
resolved.relative_to(root)
except (ValueError, OSError, RuntimeError):
raise InputError(f'{label}: path escapes the project or has a broken symlink', 9)
return resolved


def read_input(root, name, label):
relative(name, label)
path = contained(root, root / name, label)
try:
if not path.is_file() or not path.stat().st_mode & 0o444:
raise OSError('not a readable file')
data = path.read_bytes()
if not data.strip():
raise OSError('empty file')
except OSError:
raise InputError(f'{label}: missing, empty or unreadable file: {name}', 9)
return data, dict(path=path.relative_to(root).as_posix(), sha256=hashlib.sha256(data).hexdigest())


def text(value, label):
if not isinstance(value, str) or not value.strip():
raise InputError(f'{label}: expected nonempty text')


def behavior(value, label):
if isinstance(value, list):
if not value:
raise InputError(f'{label}: document behavior or explicit absence')
for item in value:
text(item, label)
else:
text(value, label)


def owner_matches(root, pattern, label):
"""Expand only within the project; validate intermediate matches before descent."""
components, visited = pattern.split('/'), set()

def walk(path, index):
resolved = contained(root, path, label)
if index == len(components):
if path.is_file():
yield path.relative_to(root).as_posix()
return
key = (resolved, index)
if key in visited or not path.is_dir():
return
visited.add(key)
component = components[index]
if component == '**':
yield from walk(path, index + 1)
for child in sorted(path.iterdir()):
if component == '**':
contained(root, child, label)
if child.is_dir():
yield from walk(child, index)
elif index + 1 == len(components) and child.is_file():
yield child.relative_to(root).as_posix()
elif fnmatch.fnmatchcase(child.name, component):
# A matching directory symlink is checked even if the remaining
# pattern would find no files, so another owner cannot hide it.
yield from walk(child, index + 1)

yield from walk(root, 0)


def load_map(root, name):
raw, map_input = read_input(root, name, 'map')
try:
data = json.loads(raw)
except (ValueError, UnicodeError):
raise InputError('map: expected valid UTF-8 JSON')
if not isinstance(data, dict) or type(data.get('version')) is not int or data['version'] != 1:
raise InputError('map: expected object with version 1')
text(data.get('app'), 'app')
_, skill_input = read_input(root, data.get('skill'), 'skill')
features = data.get('features')
if not isinstance(features, list) or not features:
raise InputError('features: expected a nonempty array')
ids, scenarios, owners = [], [], {}
for feature in features:
if not isinstance(feature, dict):
raise InputError('feature: expected object')
name = feature.get('id')
text(name, 'feature id')
if name in ids:
raise InputError(f'duplicate feature id: {name}')
ids.append(name)
for key in ['description', 'route', 'witness']:
text(feature.get(key), f'{name}.{key}')
for key in ['expected', 'side_effects']:
behavior(feature.get(key), f'{name}.{key}')
_, scenario = read_input(root, feature.get('scenario'), f'{name}.scenario')
if scenario not in scenarios:
scenarios.append(scenario)
patterns = feature.get('owners')
if not isinstance(patterns, list) or not patterns:
raise InputError(f'{name}.owners: expected nonempty path/glob array')
matches = set()
for pattern in patterns:
relative(pattern, f'{name}.owners', pattern=True)
matches.update(owner_matches(root, pattern, f'{name}.owners'))
if not matches:
raise InputError(f'{name}.owners: no existing file match', 9)
owners[name] = (patterns, matches)
return dict(valid=True, app=data['app'], features=ids,
inputs=dict(map=map_input, skill=skill_input, scenarios=scenarios)), owners


def matches_owner(path, pattern):
"""Root-anchored glob matching, including deleted paths and ** zero directories."""
parts, patterns = path.split('/'), pattern.split('/')

@lru_cache(None)
def match(i, j):
if j == len(patterns):
return i == len(parts)
if patterns[j] == '**':
return match(i, j + 1) or (i < len(parts) and match(i + 1, j))
return i < len(parts) and fnmatch.fnmatchcase(parts[i], patterns[j]) and match(i + 1, j + 1)

return match(0, 0)


class Parser(argparse.ArgumentParser):
def error(self, message):
raise InputError(message)


def main():
parser = Parser(description=__doc__)
parser.add_argument('command', choices=['inspect', 'select'])
parser.add_argument('--root', required=True)
parser.add_argument('--map', required=True)
parser.add_argument('--changed', help='JSON changed-path array file, or - for stdin')
parser.add_argument('--feature', action='append', default=[])
try:
args = parser.parse_args()
try:
root = Path(args.root).resolve()
except (OSError, RuntimeError):
raise InputError('root: project path is inaccessible or has a symlink loop', 9)
if not root.is_dir():
raise InputError('root: project directory is missing', 9)
result, owners = load_map(root, args.map)
if args.command == 'inspect' and (args.changed is not None or args.feature):
raise InputError('inspect does not accept --changed or --feature')
if args.command == 'select':
if args.changed is None:
raise InputError('select requires --changed (use [] for explicit-only selection)')
try:
raw = sys.stdin.read() if args.changed == '-' else read_input(root, args.changed, 'changed')[0]
changed = json.loads(raw)
except (ValueError, UnicodeError):
raise InputError('changed: expected a JSON path array')
if not isinstance(changed, list):
raise InputError('changed: expected a JSON path array')
for path in changed:
relative(path, 'changed path')
contained(root, root / path, 'changed path')
unknown = set(args.feature) - set(result['features'])
if unknown:
raise InputError('unknown feature id: ' + ', '.join(sorted(unknown)))
selected, covered = set(args.feature), set()
for feature, (patterns, matches) in owners.items():
hits = {path for path in changed if path in matches or
any(matches_owner(path, pattern) for pattern in patterns)}
if hits:
selected.add(feature)
covered.update(hits)
result['selected'] = [name for name in result['features'] if name in selected]
result['uncovered_paths'] = list(dict.fromkeys(path for path in changed if path not in covered))
print(json.dumps(result, indent=2))
return 0
except InputError as error:
print(json.dumps(dict(valid=False, error=str(error))))
return error.code
except OSError:
print(json.dumps(dict(valid=False, error='unreadable project input')))
return 9


if __name__ == '__main__':
sys.exit(main())
2 changes: 1 addition & 1 deletion scripts/super-board-setup.py
Original file line number Diff line number Diff line change
Expand Up @@ -71,7 +71,7 @@
}
SKILLS = ["super-board", "super-build", "super-qa", "super-review", "super-collect", "visual", "git-sync", "ui-refine-loop"]
OLD_SKILL_DIRS = ["super-refine", "cleanup-wt", "arch-loop"]
BIN = ["super-board-skills-ready.py", "super-board-host.sh", "super-board-codex-wave.sh", "super-board-codex-lane.sh", "super-board-calibration.py", "super-board-verification-ready.py", "super-board-github-read.py", "super-board-collect-state.py", "super-board-run.sh", "super-board-gh-guard.sh", "super-board-card.sh", "super-board-status.py", "super-board-wave-plan.sh",
BIN = ["super-board-project-verifier.py", "super-board-skills-ready.py", "super-board-host.sh", "super-board-codex-wave.sh", "super-board-codex-lane.sh", "super-board-calibration.py", "super-board-verification-ready.py", "super-board-github-read.py", "super-board-collect-state.py", "super-board-run.sh", "super-board-gh-guard.sh", "super-board-card.sh", "super-board-status.py", "super-board-wave-plan.sh",
"super-board-deps.sh", "super-board-preflight.sh", "super-board-merge-gate.sh",
"super-board-merge-policy.py", "super-board-approval.py", "super-board-env-check.sh", "super-board-agents-md.py",
"super-board-settings.py", "super-board-setup.py", "super-board-usage.sh", "super-board-pr-body.sh",
Expand Down
9 changes: 9 additions & 0 deletions skills/super-board/references/onboard.md
Original file line number Diff line number Diff line change
Expand Up @@ -140,6 +140,10 @@ Write it atomically (temp file + rename) after every answer. A halt never loses
├─ verification commands: preserve valid `verify_commands`; otherwise inspect the owning
│ project's documented checks and stage those commands. Unresolved checks are a setup
│ halt. Never save placeholders or invent a universal runner.
├─ project verifier: detect `.agents/skills/verify-*/SKILL.md` and feature maps.
│ For creation or upkeep read `../../super-qa/references/project-verifier.md`;
│ stage real project checks only when source names accepted features. Optional
│ verifier gaps never replace or block the required verification commands.
└─ machine time zone for config `timezone`: $TZ, else readlink /etc/localtime | sed
's#.*zoneinfo/##', else timedatectl show -p Timezone --value, else "UTC". Shown at step 8.

Expand Down Expand Up @@ -347,6 +351,9 @@ Write it atomically (temp file + rename) after every answer. A halt never loses
│ Board Ledgerly Roadmap #4 (+QA) Branch staging (new)
│ Rules safe defaults Perms 7 lines
│ AGENTS rules moved, CLAUDE.md→@ Sources sentry, github, prs, arch, linear
├─ Existing or staged project verifier: list skill/map/runner paths and state
│ `starter, not run` or the actual selected-feature proof. Preserve human-owned
│ critical paths and flows; missing source leaves this optional helper absent.
├─ AskUserQuestion "Write everything?" (header "Save") [Write everything (Recommended) /
│ Change one thing → pick the step, walk only it, back here / Cancel — nothing written].
├─ Write, in order (each atomic; stop and report on the first failure — answers are kept):
Expand All @@ -361,6 +368,8 @@ Write it atomically (temp file + rename) after every answer. A halt never loses
│ │ super-board-settings.py hooks .claude/settings.json <pack>/hooks/settings-protect-main.json
│ ├─ AGENTS.md / CLAUDE.md: super-board-agents-md.py backup, then write --src <staged>
│ │ --dest AGENTS.md, pointer --tail <staged tail> --force, block, check
│ ├─ staged project verifier skill/map/scenarios/runner, if source-backed; then
│ │ inspect the saved map and resolve the local skill for the actual host.
│ └─ docs/agents/issue-tracker.md; docs/super-board/PROJECT.md (a sub-agent drafts it from
│ manifests + README + tree, no question; listed in the table as "PROJECT.md drafted")
└─ Delete onboard-staged/; keep the answers file (re-runs use it).
Expand Down
3 changes: 3 additions & 0 deletions skills/super-board/references/run.md
Original file line number Diff line number Diff line change
Expand Up @@ -335,6 +335,9 @@ column), never built unchecked. `qa` cards are not pre-flighted: nothing is buil
2. `target.url` set → health-check it first. Unhealthy → Block.
3. Read issue + PR + Builder's handoff comment.
4. Build issue-scoped test plan: one observable test per AC.
4a. Existing project verifier → follow `../../super-qa/references/project-verifier.md`
for affected feature selection, actual host readiness and new run-owned proof.
Record uncovered/stale optional coverage alongside ordinary issue AC checks.
4b. **Test-gap check** (super-qa → "Test-gap check (after build)"): map every AC to unit / component / e2e tests, hunt edge cases, write the High gaps red-first. A High gap that needs app code changed → Fail (step 7) with the gap list. Medium/Low go in the handoff under `Test gaps (not written)`.
5. Run the tests. Capture evidence to `docs/super-board/runs/issue-<N>-qa-v<N>/`. For UI/visual ACs, capture screenshots at the standard viewports (1920×1080 desktop, 1024×768 tablet, 375×667 mobile). Commit the screenshots to the issue branch BEFORE writing the comment (the markdown image URLs depend on the files being present on the branch).
6. **Pass** → commit test files + screenshots to same branch + push → tick the `ac` block with proof lines, write `visual` for UI, append a `history` row, rewrite `status` → `[qa] [report] ✅` PR comment with results + evidence path **+ inline screenshot embeds** (see "Screenshot embed format" below) → issue comment with the SAME inline screenshot embeds → move card QA → Review. Clean up worktree.
Expand Down
9 changes: 9 additions & 0 deletions skills/super-qa/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -636,12 +636,21 @@ See `.claude/skills/super-board/references/run.md` → Tester (first pass — re
1. Pull latest of base; checkout `issue-<N>-<slug>` into worktree (a `qa` card: create it from the base).
2. Read issue + PR + Builder's handoff.
3. Build issue-scoped test plan: ONE observable test per AC.
3a. Existing project verifier → follow [project-verifier.md](references/project-verifier.md)
for affected selection, host readiness, real assertions and fresh proof.
3b. Run the **Test-gap check** (above) against the Builder's diff; its High gaps join the plan.
4. Run tests. Capture evidence to `docs/super-board/runs/issue-<N>-qa-v<N>/`. **For any UI-affecting issue, capture at least one screenshot per AC** (Playwright `page.screenshot` or `browse --screenshot`). Save with descriptive names: `ac1-<short-desc>.png`, not `screenshot1.png`.
5. **Commit the evidence directory** to the issue branch alongside test files (`git add docs/super-board/runs/issue-<N>-qa-v<N>/ && git commit && git push`). This is non-optional — without it, the inline image markdown in the issue comment won't render on GitHub.
6. **Pass** → tick the PR's `ac` block with proof lines, write `visual` for UI, append a `history` row, rewrite `status` (`super-board-pr-body.sh`) → `[qa] [report] ✅` PR comment with results + evidence path → **issue comment with screenshot evidence** (see "Issue-comment evidence format" below) → move QA → Review. Clean up worktree.
7. **Fail** → leave failing ACs unchecked with the reason, append a `history` row, rewrite `status` → `[qa] [report] ❌` PR comment with per-AC expected/actual + repro file:line + evidence path + "what fixed should look like" → issue comment with the failure screenshots → increment rebuild counter → move QA → Ready (label `loop:rebuild-N`). Clean up worktree.

### Optional project feature verifier

For an existing `.agents/skills/verify-<app>/` map, verifier creation during Onboard,
or source/selector upkeep, read [project-verifier.md](references/project-verifier.md).
Run relevant mapped features alongside issue AC checks; state missing/stale coverage
in the existing handoff. A map inspection or starter template proves no test pass.

### Optional runtime performance and memory checks

For runtime evidence requested by the issue, or a configured
Expand Down
5 changes: 5 additions & 0 deletions skills/super-qa/references/iteration-preamble.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,11 @@ Resolve this lane for the actual host, load the report's files/fallbacks, halt
on missing requirements and recheck each conditional helper before use.


For existing project verifier coverage relevant to this iteration, follow
[project-verifier.md](project-verifier.md): select mapped features, resolve the
local skill for the actual host, run real checks and preserve proof. State
uncovered/stale coverage in the report; continue ordinary issue checks.

Run **one iteration** of the loop:
1. **Regression** — re-run every spec in `e2e/paths/`. Fix any reds via TDD.
2. **Explore** — pop the top `[ ]` item from `docs/super-qa/queue.md`,
Expand Down
Loading
Loading