Build out the template: scripts, tests, Makefile, CI, flake, skills

[williamdemeo]

Builds the empty stub into the working template.

What's here

• Vendored _utils (pipeline_types, file_ops, command_runner) with all five utils fixes from Follow-up to #289 — known issues in the gh_project tooling ualib/agda-algebras#293, each pinned by a regression test that fails against the unfixed copies (verified: 8 failures/errors before, green after). This repo is now the upstream of the shared functional core.
• scripts/_gh_project_lib.py — shared model + parsing + pure planners + a snapshot-based gh client. Populate now fetches live state once per run (#293 item 6, the O(n²) idempotency check) and plans against it.
• Three CLIs: gh_project_populate.py (file → GitHub, crash-safe (#N) write-back after each creation, label near-collision reconciliation), gh_project_update.py (GitHub → file; the renamed render, diff(1) exit codes, assignees rendered), gh_project_lint.py (structural gate, no network).
• Offline test suite (86 tests): unit tests for every pure function plus integration tests that run the real scripts against a recorded fake gh on PATH — one-snapshot-per-run, rerun-after-crash-without-duplicates, collision-skip, and update round-trip are all pinned.
• Makefile as the primary UX (help default; populate-dry / populate / update / update-check / lint / test).
• Worked example docs/GITHUB_PROJECT.md (2 milestones, 4 issues) that lints clean and demonstrates every construct once.
• CI via setup-python (3.11 + 3.12) — lint, tests, shellcheck, ruff. Deliberately not Nix-bound.
• Opt-in freshness workflow: report-only by default; the repo variable PROJECT_PLAN_AUTO_UPDATE=pr|commit arms automatic updates. All logic in a locally-testable shell script (all three advisory paths exercised).
• flake.nix devShell (python3, gh, gnumake), strictly optional.
• Committed Claude config: CLAUDE.md (the structure/state contract) and two skills (project-setup, roadmap-upkeep).
• README: quickstart, direction table, freshness story, one-liner re-vendor upgrade pinned to scripts/VERSION.

Follow-up (separate PRs, William-gated)

Adoption PRs re-vendoring scripts/ into agda-algebras, williamdemeo.github.io, and agda-native-air; close ualib/agda-algebras#293 from the aa PR.

AI-assisted development: Claude Fable 5 (Anthropic)

[williamdemeo]

Quality-gate evidence (all four gates met)

#293 fixes — all six items fixed, each with a regression test; the suite was run against the pristine (unfixed) vendored copies first: 8 failures/errors, then 86/86 green after the fixes. Item 6 (O(n²) idempotency) is pinned by the fake-gh transcript asserting exactly one list call per entity kind per run.

Live E2E (throwaway williamdemeo/gh-project-template-e2e, fresh clone of this branch, plain path): make lint → make populate-dry → make populate (4 issues, (#N) write-back into the headings) → make update → make update-check (current) → closed #2 on GitHub → make update (heading flips to (#2, closed)) → re-ran make populate → 0 created, 4 already existed, 0 duplicates. Bonus: GitHub's default documentation/good first issue labels on the fresh repo exercised the never-overwrite path live (reported, not clobbered).

README commands — every command executed as written from a fresh clone via BOTH paths (plain: system python3/gh/make; Nix: nix develop → lint + 86/86 tests). The two commands that structurally require a merged main are post-merge steps: the gh repo create --template quickstart line, and the re-vendor one-liner (needs the v0.1.0 tag — please tag after merging).

Scheduled workflow — tested twice via workflow_dispatch on the throwaway (which also ran this branch's ci.yml for real: green on push). Report mode: success. Then armed PROJECT_PLAN_AUTO_UPDATE=pr, closed another issue to create genuine drift, re-dispatched: the run detected the drift and opened a real PR whose diff is exactly the one-line state flip inside the generated region — proving GH_TOKEN auth (NO_ENV_PREFIX), drift detection, and the armed path end-to-end in Actions.

One fix fell out of the E2E: GNU make maps any failing recipe to exit 2, so make update-check cannot relay the script's diff(1) exit 1; README + skill now document the make target as pass/fail-with-explanation and point at the direct script invocation when the 0/1/2 distinction matters (the freshness workflow already calls the script directly, so it was unaffected).

Cleanup (needs the delete_repo scope, which the current token lacks):

gh auth refresh -h github.com -s delete_repo
gh repo delete williamdemeo/gh-project-template-e2e --yes

[Copilot]

Pull request overview

Builds the repository into a functional GitHub roadmap template and addresses the six issues tracked in agda-algebras#293.

Changes:

• Adds roadmap parsing, linting, population, synchronization, and CI automation.
• Vendors corrected functional utilities with an 86-test offline suite.
• Adds documentation, example plans, Nix support, and workflow skills.

Reviewed changes

Copilot reviewed 27 out of 28 changed files in this pull request and generated 7 comments.

Show a summary per file

File	Description
scripts/VERSION	Sets the tooling version.
scripts/tests/test_pipeline_types.py	Tests Result regressions.
scripts/tests/test_integration.py	Tests end-to-end roadmap workflows.
scripts/tests/test_gh_project_update.py	Tests update rendering.
scripts/tests/test_gh_project_lint.py	Tests structural linting.
scripts/tests/test_gh_project_lib.py	Tests parsing and planning.
scripts/tests/test_file_ops.py	Tests filesystem utilities.
scripts/tests/test_command_runner.py	Tests command execution regressions.
scripts/tests/fake_gh/gh	Provides an offline GitHub CLI fake.
scripts/gh_project_update.py	Synchronizes GitHub state into Markdown.
scripts/gh_project_populate.py	Creates GitHub roadmap entities.
scripts/gh_project_lint.py	Validates roadmap structure.
scripts/ci/project-plan-update.sh	Implements freshness automation.
scripts/_utils/pipeline_types.py	Provides functional result types.
scripts/_utils/file_ops.py	Provides filesystem wrappers.
scripts/_utils/command_runner.py	Provides subprocess execution.
scripts/_utils/__init__.py	Exports utility APIs.
scripts/_gh_project_lib.py	Implements the shared roadmap core.
README.md	Documents setup and operation.
Makefile	Adds primary user commands.
flake.nix	Defines an optional development shell.
flake.lock	Pins Nix dependencies.
docs/GITHUB_PROJECT.md	Supplies the worked roadmap example.
CLAUDE.md	Defines repository workflow conventions.
.github/workflows/project-plan-update.yml	Adds scheduled freshness checks.
.github/workflows/ci.yml	Adds tests and static checks.
.claude/skills/roadmap-upkeep/SKILL.md	Documents roadmap maintenance.
.claude/skills/project-setup/SKILL.md	Documents project bootstrap.

---

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

[williamdemeo]

Confirmed — this was the sharpest finding of the review. The fallback was dead code: _parse_issues_json dropped prefix-less issues before plan_issues ever saw them, and the unit test had simulated the wrong scenario (a changed ID, which still parses) so it passed anyway. Fixed in 9ad5f20: the parser now keeps unparseable-title issues with id="" (and filters PRs, which the REST endpoint includes); populate matches them by recorded number; update's group_by_milestone excludes them from rendering. Covered by a corrected unit test, a by_id-guard test (blank ids never match each other), and a new integration test that strips a live title's prefix and re-runs populate with zero duplicates.

[williamdemeo]

Agreed — continuing past a failed write-back violated the persist-before-next-mutation contract stated in the same docstring. Fixed in 9ad5f20: either failure mode (heading not found, or the write itself failing) increments failed, prints which issue IS live on GitHub, and aborts before any further remote mutation; the rerun is safe via the snapshot match. New integration test: a chmod-444 plan file aborts after the first creation (1 issue live, exit 1), and the rerun completes to exactly 4 with no duplicates.

[williamdemeo]

Agreed — ms_title_map keys on the plan number, so a duplicate makes issue→milestone attachment ambiguous (last definition wins, silently). Escalated from warning to error in 9ad5f20; populate's lint gate now refuses to run. The lint test was flipped to expect the error.

[williamdemeo]

Agreed, and the failure mode is nasty: a typo'd heading inside a generated region isn't just skipped — the next make update erases it when the region is rebuilt from GitHub. Fixed in 9ad5f20: lint scans every ### Issue-prefixed line and errors on any that ISSUE_HEADING_RE doesn't match, with the line number and an explanation of the erasure hazard. Test: ### Issue M1-X: ... is now a lint error.

[williamdemeo]

Agreed — the old count only saw bullets that already opened a backtick, so - documentation (0e8a16) — ... was invisible; in the worst case (all entries backtick-less) the explicit section parsed as empty and lint skipped the reference check entirely. Fixed in 9ad5f20: every bullet line in the ## Labels section must now match the full - \name` (RRGGBB) — description` shape, each failure reported with its own line number. Test added.

[williamdemeo]

Agreed — a silently truncated snapshot would defeat exactly the guarantees it feeds (idempotency for populate, completeness for update). Fixed in 9ad5f20: list_issues moves to gh api repos/{repo}/issues --paginate (with a pull_request-key filter, since the REST endpoint includes PRs), and list_labels — which had the same defect at --limit 200 — moves to repos/{repo}/labels --paginate. The fake-gh transcript was updated to the REST invocations, and both paths were verified against live GitHub, including PR filtering.

[williamdemeo]

Agreed — and the incompleteness was permanent, not just transient: populate never revisits existing issues, so a milestone-less creation would never be healed. Fixed in 9ad5f20: when the issue's milestone is declared in the plan but unavailable on GitHub, the issue is skipped and counted as a failure (exit 1) with a create-milestones-first hint. Plans that declare no milestones at all still populate milestone-free, which keeps the labels+issues-only use case working. Integration test: --issues-only on a fresh repo exits 1 and creates nothing.

[Copilot]

Pull request overview

Copilot reviewed 27 out of 28 changed files in this pull request and generated 1 comment.

Suppressed comments (10)

scripts/gh_project_update.py:132

• The renderer emits **Labels**:, while _parse_issues and the documented plan format recognize only **Labels:**. After make update, reparsing treats this metadata as body text and yields an empty label tuple; a new issue copied from the rendered format would be created without labels. Render the canonical **Labels:** form and update the corresponding assertion.

        f"**Labels**: {labels}\n"

scripts/tests/test_gh_project_update.py:32

• This assertion enshrines the non-canonical **Labels**: spelling even though the plan parser accepts **Labels:**. Update it with the renderer and add a round-trip assertion that reparsed issues retain their labels.

        self.assertIn("**Labels**: `milestone-1-core`\n", out)

scripts/gh_project_populate.py:382

• Dry-run exits based only on label collisions, but live-only prerequisite checks happen later in execute_issues. For example, --dry-run --issues-only on a repository with no milestones reports all issues as creatable and exits 0, while the real run skips every issue and exits 1. Move milestone/label availability into the pure issue plan so dry-run reports and returns exactly what execution will do.

    if args.dry_run:
        print("Dry run — nothing was created.")
        return 1 if collisions else 0

scripts/_gh_project_lib.py:705

• This guard disables milestone-reference validation when a ## Milestones section exists but none of its headings parse. A one-milestone plan using - instead of —, for example, produces an empty tuple, passes lint, and then creates all issues without a milestone. Guard on the section's presence so malformed headings cannot bypass the consistency checks.

    if plan.milestones:

scripts/_utils/pipeline_types.py:262

• Path.suffix returns only .md for Module.lagda.md, so this property returns false for a file type explicitly listed as supported. Match against the filename (or suffixes) so literate Markdown Agda sources are recognized.

        return self.source_path.suffix in {'.agda', '.lagda', '.lagda.md'}

scripts/_gh_project_lib.py:178

• Treating every level-2 heading as the end of an issue silently truncates valid Markdown bodies. An issue containing ## Acceptance criteria is created with everything from that heading onward omitted. Bound bodies by the next issue heading or generated-region end instead of generic Markdown headings.

BODY_END_RE = re.compile(r"^(?:## (?!#)|<!--\s*END GENERATED)", re.MULTILINE)

scripts/gh_project_populate.py:287

• These stage-selection flags are not mutually exclusive, and any pair makes all three do_* values false, so the command fetches a snapshot, reports success, and performs no work. Define them in an argparse mutually exclusive group so contradictory invocations fail immediately.

    parser.add_argument("--milestones-only", action="store_true")
    parser.add_argument("--labels-only", action="store_true")
    parser.add_argument("--issues-only", action="store_true")

scripts/gh_project_populate.py:438

• This created count is wrong when write-back aborts the loop. If the first of four issues is created but cannot be persisted, issue_failed is 1 and the summary claims three issues were created even though the remaining three were not attempted. Have execute_issues return an explicit created/attempted count and report that value.

            f"DONE: {len(issue_plan.to_create) - issue_failed} issues created, "
            f"{len(issue_plan.existing)} already existed, {issue_failed} failed"

scripts/gh_project_update.py:128

• Live issue bodies are inserted without neutralizing the file's control markers. If a GitHub body contains <!-- END GENERATED: milestone-N -->, the next parse can close the region inside that body and treat the remaining generated content as manual prose, causing stale/duplicated content to survive updates. Escape delimiter sequences in dynamic titles/bodies or use framing that cannot occur in issue content.

    body = issue.body.strip() if issue.body else "_(no description on GitHub)_"

scripts/_gh_project_lib.py:456

• A stray END marker is checked only when there is no later BEGIN. Thus <!-- END GENERATED: stray --> before any valid region is silently absorbed into a manual segment, despite lint claiming to reject stray markers. At each position, compare the next BEGIN and END and error when END occurs first.

        begin = BEGIN_RE.search(content, pos)
        if begin is None:
            # A stray END with no matching BEGIN would otherwise vanish
            # silently into a manual segment.
            stray = END_RE.search(content, pos)

[williamdemeo]

Confirmed — a real bug I introduced with the REST switch, and the ancestors' paginated milestones call carried the same latent defect. Fixed in 634ca0e, with one deviation from the suggested remedy: instead of --slurp (which requires gh ≥ 2.47 and would silently raise the template's documented "any gh" floor — the machine this was built on runs 2.45), _parse_json now decodes the concatenated output document-by-document with json.JSONDecoder.raw_decode and concatenates the lists. Same behavior for every gh version, single-document output unaffected, and all three list calls (labels, milestones, issues) go through the one decoder. Pinned by six new unit tests (multi-document, newline-separated, single, empty, non-list, trailing garbage) and live-verified against a real repository.

[williamdemeo]

Response to the ten suppressed comments (review of 634ca0e)

Copilot's confidence filter was too conservative: all ten suppressed findings were legitimate, and all ten are fixed in 634ca0e. The test suite grew 95 → 111. Item by item, in the review's order:

1. **Labels**: / **Labels:** render/parse asymmetry — real, inherited from the ancestor renderer. Update now emits the canonical **Labels:** (and **Assignees:**), populate strips the Assignees metadata line from bodies, and a round-trip test pins that a rendered block reparses with its labels intact. One consequence needed handling: now that rendered labels lines parse, the label-reference lint is scoped to issues without a recorded (#N) — an existing issue's line is rendered state and may legitimately carry labels added on GitHub.
2. Test enshrining the misspelling — updated together with (1), plus the new round-trip assertion the comment asked for.
3. Dry-run not modeling live-availability blocks — real divergence. A shared issue_blocker predicate now computes label/milestone availability from mode-appropriate prospective sets; the dry run prints ! blocked: lines and exits 1 exactly when the real run would. Pinned by --dry-run --issues-only vs live comparison tests.
4. if plan.milestones: bypass — real. The consistency checks now guard on the section's presence, and numbered-but-malformed headings (### Milestone 1 - Core, hyphen for em-dash) are their own lint error. One refinement over the suggestion: only numbered ### Milestone N headings are scanned — prose headings like ### Milestone dependencies appear in the real exemplar plans and stay legal (caught by running the stricter first draft against the shipped example).
5. is_agda_file vs .lagda.md — real, though dead code for the gh-project scripts themselves. Fixed anyway (filename match, not Path.suffix) because this repository is now the upstream of the vendored _utils; the three downstream projects whose doc pipelines do use it inherit the fix on re-vendor.
6. BODY_END_RE truncating ## bodies — the sharpest catch of the group: my earlier trailing-prose fix had regressed the exemplar format, where issue bodies really do contain ## Description / ## Tasks / ## Acceptance criteria sections. Bodies are now bounded by the next issue heading, generated-region markers, and the two structural headers the ancestors special-cased (## Milestone N, ## Summary) — not generic headings. The shared fixture now includes a ##-section body.
7. Non-exclusive --*-only flags — real silent no-op; now an argparse mutually exclusive group (exit 2 with a clear message), with a test.
8. Wrong DONE count after a write-back abort — real. execute_issues returns explicit created/failed counts; the summary reports created, already-existed, failed-or-skipped, and not-attempted honestly.
9. Region markers inside live GitHub bodies — real corruption vector (populate strips markers before pushing, but bodies edited directly on GitHub can contain anything). Rendered bodies now defang the comment-opening form (<!-- END GENERATED: → <!-- END GENERATED (escaped):), deterministically, so --check stays stable; pinned by a hostile-body test.
10. Stray END absorbed when a later BEGIN exists — real gap in the stray-marker check; parse_file now compares the next BEGIN and END at each position and rejects an END-first ordering, with a test.

Everything re-verified live against the E2E throwaway: canonical rendering round-trips (--check stable), pagination decoding works against the real API, and the updated file re-parses cleanly through populate's dry run.