From 2c97a4e9dea5cf26071026c0119ccc97bf493e78 Mon Sep 17 00:00:00 2001 From: William DeMeo Date: Wed, 19 Aug 2026 00:26:54 -0600 Subject: [PATCH 1/2] docs: man-page command reference in docs/HowToRun.md; short form in README MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit PRs #8 and #10 shipped flags the docs never caught up with (--keep-line-breaks, --sync-bodies, --force). docs/HowToRun.md is now the complete option-by-option account, man-page shaped per tool (SYNOPSIS / DESCRIPTION / OPTIONS / EXIT STATUS / EXAMPLES): populate with all fourteen options including the --sync-bodies classification table (in-sync / reflow / divergent) and the update-artifact in-sync rules; update with its diff(1) exit codes and the make-collapses-to-2 caveat; lint. It also shows the interchangeable invocation forms per engine channel (make / installed CLI / nix run / plain python), and — the requested nicety — how to use _utils.text_unwrap directly on any markdown file, both as a heredoc one-liner and as a tiny PATH script; both examples were executed verbatim before landing. The README stays lean: a short-form flag list per tool plus the link, keeping the landing page clean while every option is still findable from it. AI-assisted: Claude Fable 5 (Anthropic) --- README.md | 22 ++++ docs/HowToRun.md | 277 +++++++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 299 insertions(+) create mode 100644 docs/HowToRun.md diff --git a/README.md b/README.md index f355169..85f8faa 100644 --- a/README.md +++ b/README.md @@ -123,6 +123,28 @@ that.) `make help` lists every target. +## Command reference (short form) + +Full man-page-style documentation — every option, exit codes, the +`--sync-bodies` classification rules, and how to unwrap arbitrary +markdown files — lives in [docs/HowToRun.md](docs/HowToRun.md). +The flags at a glance: + +**populate** (file → GitHub) — `--repo`, `--dry-run`, +`--milestones-only` / `--labels-only` / `--issues-only` / +`--sync-bodies` (mutually exclusive stages; sync pushes existing +issues' bodies and milestone descriptions from the file, refusing on +content divergence), `--force` (with sync: last writer wins), +`--skip-labels`, `--keep-line-breaks` (default strips hard line breaks +from prose so GitHub soft-wraps it), `--start-from ID`, +`--delay SECONDS`, `--yes`, `--env-prefix` / `--no-env-prefix`. + +**update** (GitHub → file) — `--repo`, `--check` (diff(1) exit codes: +0 current / 1 stale / 2 failed), `--no-env-prefix`. + +**lint** (no network) — no options; exit 0 clean / 1 errors / +2 unreadable. + ## Staying fresh automatically (opt-in) The repository ships a scheduled workflow diff --git a/docs/HowToRun.md b/docs/HowToRun.md new file mode 100644 index 0000000..84a17f6 --- /dev/null +++ b/docs/HowToRun.md @@ -0,0 +1,277 @@ + + +# Running the engine — command reference + +Man-page-style reference for the three tools that operate on +`docs/GITHUB_PROJECT.md`. The quickstart lives in the +[README](../README.md); this page is the complete option-by-option +account. + +Every tool can be invoked through any engine +[channel](../README.md#the-engine-getting-and-upgrading-it); the forms +are interchangeable: + +```sh +make populate # Makefile target (primary UX) +gh-project-populate docs/GITHUB_PROJECT.md # installed CLI (Nix dev shell) +nix run .#populate -- docs/GITHUB_PROJECT.md # flake app +python3 scripts/gh_project_populate.py docs/GITHUB_PROJECT.md # plain python +``` + +The Makefile targets pass `PLAN` (default `docs/GITHUB_PROJECT.md`), +`REPO`, and `NO_ENV_PREFIX` through; any other option is a direct- +invocation affair. + +--- + +## gh-project-populate + +### SYNOPSIS + + gh-project-populate MARKDOWN [--repo OWNER/NAME] [--dry-run] + [--milestones-only | --labels-only | --issues-only | + --sync-bodies] [--force] [--skip-labels] + [--keep-line-breaks] [--start-from ID] + [--delay SECONDS] [--yes] + [--env-prefix | --no-env-prefix] + +### DESCRIPTION + +The file → GitHub direction. Creates the plan's labels, milestones, +and issues on GitHub, writing each new issue's number back into the +plan file (as a `(#N)` heading suffix) immediately after creation, so +an interrupted run is always rerun-safe. Idempotent: one snapshot of +live GitHub state is fetched per run and everything that already +exists is skipped — labels by exact name, milestones by exact title, +issues by their `[MN-k]` title prefix or recorded `(#N)` number. + +By default, hard line breaks in prose are stripped before pushing +(see `--keep-line-breaks`), so GitHub soft-wraps issue bodies and +milestone descriptions. + +`--sync-bodies` switches the tool into its one deliberate +GitHub-mutating maintenance mode: pushing bodies of *already existing* +items (see below). + +### OPTIONS + +- `MARKDOWN` + Path to the project plan markdown file (positional, required). + +- `--repo OWNER/NAME` + Target repository. Overrides the plan file's + `**Repository**:` header; one of the two must be present. + +- `--dry-run` + Print exactly the plan a real run would execute — including + live-availability blocks and, under `--sync-bodies`, the per-item + sync classification — and mutate nothing. Reads still happen, so + the preview reflects real GitHub state. The exit code mirrors what + the real run would return. + +- `--milestones-only` / `--labels-only` / `--issues-only` + Run a single creation stage. Mutually exclusive with each other and + with `--sync-bodies`. `--issues-only` requires the plan's milestones + to already exist on GitHub: issues whose declared milestone is + missing are skipped and counted as failures rather than created + milestone-less (populate never revisits existing issues, so that + incompleteness would be permanent). + +- `--sync-bodies` + Create nothing; push existing issues' bodies and milestone + descriptions from the plan file to GitHub — the per-issue inverse of + `update`. Each target is classified, live runs re-validating against + a fresh read immediately before mutating: + + | verdict | meaning | action | + | --- | --- | --- | + | `in-sync` | identical after CRLF/trailing-whitespace normalization; or the file holds update's rendering artifacts for this body (the escaped form of marker-like text, or the empty-body placeholder) | skip | + | `reflow` | identical after unwrapping both sides — only line wrapping differs, so pushing loses neither words nor GitHub-side formatting | push | + | `divergent` | content differs, or GitHub's body carries hard-break or region-marker text a push would destroy; no base version exists to tell which side moved | refuse (exit 1) unless `--force` | + + Refusals are per item: one divergent body does not block the + reflow-safe rest. + +- `--force` + With `--sync-bodies` only: push divergent items too — last writer + wins, including any escape artifacts the file may carry. Rejected + in every other mode. + +- `--skip-labels` + Skip the label-creation stage in a full run. + +- `--keep-line-breaks` + Push bodies and descriptions exactly as authored. By default the + input is unwrapped in memory before parsing (the plan file on disk + is never rewritten by this), so authored ~72-column prose reaches + GitHub as single-line paragraphs that the issue page soft-wraps. + The run output states which mode applied. + +- `--start-from ID` + Consider only issues with ID ≥ `ID` (e.g. `M1-3`), by + (milestone, ordinal, suffix) order. Composes with `--issues-only` + for resuming an interrupted bootstrap, and with `--sync-bodies`. + +- `--delay SECONDS` + Pause between mutating API calls (default: 1.0) to stay clear of + rate limits. Reads are not delayed. + +- `--yes` + Skip the interactive confirmation (for scripts and CI). + +- `--env-prefix` / `--no-env-prefix` + By default every `gh` call is prefixed with + `env -u GH_TOKEN -u GITHUB_TOKEN`, preventing those variables from + shadowing a keychain-stored token. Pass `--no-env-prefix` wherever + authentication comes *through* those variables — GitHub Actions + included: stripping them there leaves `gh` with no credentials. + +### EXIT STATUS + + 0 everything requested was created / synced or already existed + 1 some items failed, were refused (label collisions, divergent + bodies), or were skipped + 2 the run could not proceed (unreadable file, lint errors, no + repository, snapshot failure, invalid flag combination) + +### EXAMPLES + +```sh +# Preview a fresh bootstrap: +gh-project-populate docs/GITHUB_PROJECT.md --dry-run + +# Bootstrap, keeping authored line breaks: +gh-project-populate docs/GITHUB_PROJECT.md --keep-line-breaks + +# Resume creation from M2-1 after an interruption: +gh-project-populate docs/GITHUB_PROJECT.md --issues-only --start-from M2-1 + +# Preview what a body sync would do, then do it: +gh-project-populate docs/GITHUB_PROJECT.md --sync-bodies --dry-run +gh-project-populate docs/GITHUB_PROJECT.md --sync-bodies + +# Make the file win over independently edited GitHub bodies: +gh-project-populate docs/GITHUB_PROJECT.md --sync-bodies --force +``` + +--- + +## gh-project-update + +### SYNOPSIS + + gh-project-update MARKDOWN [--repo OWNER/NAME] [--check] + [--no-env-prefix] + +### DESCRIPTION + +The GitHub → file direction. Rewrites only the regions between +`` / `` +markers from live GitHub state — `milestone-N` regions from issues +carrying the `milestone-N-*` label, the `unplanned` region from +organically-filed issues (no `[MN-k]` prefix) grouped by GitHub +milestone. Hand-written prose outside the markers is preserved +byte-for-byte. + +Note: the first run after a fresh populate is a *normalization* (its +rendering drops `**Milestone:**` lines, reorders labels to GitHub's +order, trims trailing `---`); commit it once, then treat `--check` +failures as drift. + +### OPTIONS + +- `MARKDOWN` + Path to the plan file (positional, required). + +- `--repo OWNER/NAME` + Target repository; overrides the `**Repository**:` header. + +- `--check` + Verify the file already matches the rendered output; write nothing. + +- `--no-env-prefix` + As in populate. + +### EXIT STATUS + +Follows diff(1), so callers can tell "stale" from "the check never ran": + + 0 the file is current (or, without --check, was written) + 1 --check only: the file differs from live GitHub state + 2 the run failed — authentication, API error, unreadable file, + bad markers + +(`make update-check` cannot relay code 1 — GNU make maps any recipe +failure to its own exit 2; call the script directly when the +distinction matters.) + +--- + +## gh-project-lint + +### SYNOPSIS + + gh-project-lint MARKDOWN + +### DESCRIPTION + +Structural validation of the plan file — no network, no `gh`. Checks +marker balance and uniqueness, heading well-formedness, the +`## Labels` section grammar (including case/whitespace collisions), +milestone-reference consistency, and issue-ID uniqueness. Populate +runs the same checks and refuses to mutate anything when they fail. +Cheap enough for every PR: this is the repository's no-network drift +gate. + +### EXIT STATUS + + 0 no errors (warnings may have been printed) + 1 at least one error + 2 the file could not be read + +--- + +## Stripping hard line breaks from any markdown file + +The unwrap logic populate uses lives in the engine's `_utils` package +(`scripts/_utils/text_unwrap.py`) as a pure function — structure-aware +(headings, tables, fenced/indented code, blockquotes, list items, +region markers, and the plan grammar's `**Labels:**`-style metadata +all survive byte-for-byte), sentence-spacing-preserving, and +idempotent. To use it directly on a file, from an engine checkout: + +```sh +python3 - docs/SOMEFILE.md <<'EOF' +import sys +sys.path.insert(0, "scripts") +from pathlib import Path +from _utils.text_unwrap import unwrap + +path = Path(sys.argv[1]) +path.write_text(unwrap(path.read_text(encoding="utf-8")), encoding="utf-8") +EOF +``` + +Or as a tiny reusable script on your PATH (adjust the checkout path): + +```python +#!/usr/bin/env python3 +"""md-unwrap FILE [FILE ...] — remove hard line breaks from markdown prose.""" +import sys +from pathlib import Path + +sys.path.insert(0, str(Path.home() / "git/github-project/scripts")) +from _utils.text_unwrap import unwrap + +for name in sys.argv[1:]: + path = Path(name) + text = path.read_text(encoding="utf-8") + result = unwrap(text) + path.write_text(result, encoding="utf-8") + print(("unwrapped: " if result != text else "unchanged: ") + name) +``` + +For plan files specifically, prefer letting populate do it (the +default), and run `gh-project-lint` afterwards if you unwrap by hand — +the transform is verified to leave plan parsing identical, and lint is +the cheap proof on your own file. From ed5f2e12b84564f25b45f088f25ab857d6f42f99 Mon Sep 17 00:00:00 2001 From: William DeMeo Date: Wed, 19 Aug 2026 01:11:02 -0600 Subject: [PATCH 2/2] docs: three accuracy fixes from Copilot's review of the reference MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit All three findings were legitimate documentation inaccuracies: - --issues-only requires referenced LABELS to exist, not only milestones — the mode creates neither, and issue_blocker counts an issue with an unavailable label as a failure. Fixed in HowToRun and in populate's own module docstring, which carried the same milestone-only claim since PR #1. - Exit status 1's unqualified 'were skipped' invited misreading: --start-from filtering and already-existing items exit 0; only blocked issues (unavailable label/milestone), creation errors, collisions, and sync refusals reach exit 1. The wording now says exactly that, and exit 0 explicitly covers deliberate filtering. - 'List items survive byte-for-byte' overclaimed: list STRUCTURE is preserved (markers, one item per line, empty items untouched) while wrapped prose INSIDE an item is deliberately reflowed — half the feature. The unwrap section now draws that line. AI-assisted: Claude Fable 5 (Anthropic) --- docs/HowToRun.md | 27 ++++++++++++++++----------- scripts/gh_project_populate.py | 3 ++- 2 files changed, 18 insertions(+), 12 deletions(-) diff --git a/docs/HowToRun.md b/docs/HowToRun.md index 84a17f6..17a54e8 100644 --- a/docs/HowToRun.md +++ b/docs/HowToRun.md @@ -72,10 +72,11 @@ items (see below). - `--milestones-only` / `--labels-only` / `--issues-only` Run a single creation stage. Mutually exclusive with each other and with `--sync-bodies`. `--issues-only` requires the plan's milestones - to already exist on GitHub: issues whose declared milestone is - missing are skipped and counted as failures rather than created - milestone-less (populate never revisits existing issues, so that - incompleteness would be permanent). + AND every label its issues reference to already exist on GitHub — + this mode creates neither: an issue whose declared milestone or any + referenced label is unavailable is not created and counts as a + failure, rather than being created incomplete (populate never + revisits existing issues, so the incompleteness would be permanent). - `--sync-bodies` Create nothing; push existing issues' bodies and milestone @@ -128,9 +129,11 @@ items (see below). ### EXIT STATUS - 0 everything requested was created / synced or already existed - 1 some items failed, were refused (label collisions, divergent - bodies), or were skipped + 0 everything requested was created / synced, already existed, or + was deliberately filtered (--start-from) + 1 some items failed or were refused: creation errors, issues + blocked by an unavailable label or milestone, label + near-collisions, divergent bodies under --sync-bodies 2 the run could not proceed (unreadable file, lint errors, no repository, snapshot failure, invalid flag combination) @@ -234,11 +237,13 @@ gate. ## Stripping hard line breaks from any markdown file The unwrap logic populate uses lives in the engine's `_utils` package -(`scripts/_utils/text_unwrap.py`) as a pure function — structure-aware -(headings, tables, fenced/indented code, blockquotes, list items, +(`scripts/_utils/text_unwrap.py`) as a pure function — +structure-aware (headings, tables, fenced/indented code, blockquotes, region markers, and the plan grammar's `**Labels:**`-style metadata -all survive byte-for-byte), sentence-spacing-preserving, and -idempotent. To use it directly on a file, from an engine checkout: +all survive byte-for-byte; list STRUCTURE is preserved — markers, one +item per line, empty items untouched — while wrapped prose INSIDE an +item is reflowed onto its marker line, exactly like any paragraph), +sentence-spacing-preserving, and idempotent. To use it directly on a file, from an engine checkout: ```sh python3 - docs/SOMEFILE.md <<'EOF' diff --git a/scripts/gh_project_populate.py b/scripts/gh_project_populate.py index 9baab32..9adcbc2 100755 --- a/scripts/gh_project_populate.py +++ b/scripts/gh_project_populate.py @@ -38,7 +38,8 @@ --dry-run Print the plan without mutating anything --milestones-only Only create milestones --labels-only Only create labels - --issues-only Only create issues (milestones must already exist) + --issues-only Only create issues (their milestones and every + referenced label must already exist) --skip-labels Skip label creation --start-from ID Create only issues with ID >= this (e.g. M1-3) --sync-bodies Create nothing; push existing issues' bodies and