Skip to content

feat(cli): --help that never starts a run, and unknown flags are refused - #34

Merged
kevinccbsg merged 3 commits into
mainfrom
feat/help-flag
Sep 24, 2026
Merged

kevinccbsg merged 3 commits into
mainfrom
feat/help-flag

Conversation

@kevinccbsg

Copy link
Copy Markdown
Member

What you get

$ npx twd-cli run --help        # every run option, exit 0, no browser
$ npx twd-cli merge --help
$ npx twd-cli --help            # the commands; also `help`, `-h`, bare `twd-cli`

$ npx twd-cli run --tests "Login"
twd-cli run: unknown option --tests

Did you mean --test?
Run `twd-cli run --help` to see every option.
$ echo $?
1

Today run --help runs the entire suite. The bin dispatches on argv[2] alone and parseRunArgs drops any token it does not know, so the run proceeds with nothing said. An agent hit exactly this, waited out the whole suite, and ended up reading bin/twd-cli.js from node_modules to find --changed-since.

How it is built

Three layers, each one there to make the one above it trustworthy:

  1. Help is decided in the bin, before any dynamic import. src/index.js is imported dynamically so merge never loads puppeteer; the help path returns at that same point, so it reads no config, touches no git and launches no browser. --help / -h win wherever they appear after the command.
  2. One usage block per command (src/usage.js). run --help now lists --update-snapshots and --ci, which appeared in neither the old block nor the README. RUN_FLAGS / MERGE_FLAGS are exported from the parser and the usage test derives the expected help from them, so a flag added without a help line fails the suite.
  3. Unknown --flags throw instead of being dropped. This is what makes --help reliable rather than cosmetic: a future flag the bin forgets to route fails loudly instead of running the suite. The suggestion is a prefix match, else the nearest known flag within two edits. Tight on purpose: --out passed to run must not come back as "Did you mean --ci?".

Behaviour changes to glance at

  • twd-cli bogus prints usage on stderr now (was stdout) and still exits 1.
  • A --flag neither parser knows used to be ignored; it now fails the command before anything runs. Both composite actions only pass known flags, and I found no other caller in the workspace passing an unknown one.

tests/cli.test.js spawns the real bin. Every case returns before the dynamic import, so no browser is involved, and a regression to the old behaviour fails on exit code alone since there is no dev server.

`run --help` used to run the entire suite. bin/twd-cli.js dispatched on
argv[2] alone, parseRunArgs dropped the token it did not know, and the
run proceeded: minutes of wall clock and a browser nobody asked for, with
nothing printed to say the flag was not understood. An agent hit exactly
this and ended up reading the bin from node_modules to find
--changed-since.

Help is now decided in the bin before either parser runs and before the
dynamic import of src/index.js, so it loads no puppeteer, reads no config
and touches no git. `--help`/`-h` win wherever they appear after the
command, and a bare `help` (optionally `help <command>`) does the same.

The one usage block is split per command in src/usage.js. `run --help`
now lists --update-snapshots and --ci, which appeared in neither the old
block nor the README, and both value forms are stated once in the shared
header. RUN_FLAGS / MERGE_FLAGS are exported from the parser so the usage
test derives the expected set from the code rather than a copy of it.

An unknown command moves to stderr with exit 1; it used to print usage on
stdout with exit 1, which a script cannot tell from a run's output.

tests/cli.test.js spawns the real bin. Every case returns before the
dynamic import, so no browser is involved, and a regression to the old
behaviour fails on exit code alone since there is no dev server.
parseRunArgs and parseMergeArgs ignored any --token no branch claimed.
That is what let `run --help` run the suite, and it is still what a typo
does today: `--tests foo` ran every test with a filter the caller
believed they had set, and nothing said otherwise.

Both parsers now collect the `--`-prefixed strays and throw before
anything runs:

  twd-cli run: unknown option --tests

  Did you mean --test?
  Run `twd-cli run --help` to see every option.

The bin already prints a parser error to stderr and exits 1, so no
routing changed. The suggestion is a prefix match first (--output → --out,
--test-filter → --test) and otherwise the nearest known flag within two
edits, which is tight on purpose: --out passed to run is a real mistake
and must not come back as "Did you mean --ci?".

This is what makes --help reliable rather than cosmetic. A future flag
the bin forgets to route fails here instead of running the suite.
Positionals keep their treatment: merge takes the first one as <dir>, run
ignores them, and only `--` strays are refused.
README gets a short Getting help section next to Basic Usage: the three
help invocations and what a rejected typo looks like. CLAUDE.md records
where help is resolved and why that position matters, what RUN_FLAGS /
MERGE_FLAGS are for, the suggestion budget, and that tests/cli.test.js is
the one file allowed to spawn a real process.
@github-actions

Copy link
Copy Markdown

TWD Contract Validation

Spec Passed Failed Warnings Mode
./contracts/users-3.0.json 2 3 1 warn
./contracts/posts-3.1.json 2 2 0 warn
./contracts/products-3.0.json 13 23 2 warn
./contracts/events-3.1.json 6 13 0 warn

23 passed · 41 failed · 3 warnings · 1 skipped

Failed validations

./contracts/users-3.0.json

  • GET /users/{userId} (200) — mock getUserNoAddress — in "Contract Validation - Mismatches > should fail: missing nested address field"
    • response.address: missing required property "address"
  • GET /users/{userId} (200) — mock getUserBadAddress — in "Contract Validation - Mismatches > should fail: nested address missing required city"
    • response.address.city: missing required property "city"
    • response.address.country: missing required property "country"
  • GET /users/{userId} (200) — mock getUserBadRole — in "Contract Validation - Mismatches > should fail: oneOf role with invalid variant"
    • response.role: oneOf best match (branch 2 of 2) failed: must be one of: "viewer"

./contracts/posts-3.1.json

  • GET /posts/{postId} (200) — mock getPostNoAuthor — in "Contract Validation - Mismatches > should fail: post missing nested author object"
    • response.author: missing required property "author"
  • GET /posts/{postId} (200) — mock getPostBadMeta — in "Contract Validation - Mismatches > should fail: post oneOf metadata matches neither variant"
    • response.metadata: oneOf best match (branch 1 of 2) failed: missing required property "category", unexpected property "duration", must be one of: "article"

./contracts/products-3.0.json

  • GET /products (200) — mock getProductEmptyName — in "Contract Validation - Products Mismatches (OpenAPI 3.0 — error mode) > should fail: empty name violates minLength"
    • response[0].name: must NOT have fewer than 1 characters
  • GET /products (200) — mock getProductBadSku — in "Contract Validation - Products Mismatches (OpenAPI 3.0 — error mode) > should fail: invalid SKU pattern"
    • response[0].sku: must match pattern "^[A-Z]{2,4}-\d{4,8}$"
  • GET /products (200) — mock getProductBadUuid — in "Contract Validation - Products Mismatches (OpenAPI 3.0 — error mode) > should fail: invalid uuid format for id"
    • response[0].id: must match format "uuid"
  • GET /products (200) — mock getProductBadDateTime — in "Contract Validation - Products Mismatches (OpenAPI 3.0 — error mode) > should fail: invalid date-time format"
    • response[0].createdAt: must match format "date-time"
  • GET /products (200) — mock getProductBadDate — in "Contract Validation - Products Mismatches (OpenAPI 3.0 — error mode) > should fail: invalid date format"
    • response[0].releaseDate: must match format "date"
  • GET /products (200) — mock getProductBadEmail — in "Contract Validation - Products Mismatches (OpenAPI 3.0 — error mode) > should fail: invalid email format"
    • response[0].contactEmail: must match format "email"
  • GET /products (200) — mock getProductBadUri — in "Contract Validation - Products Mismatches (OpenAPI 3.0 — error mode) > should fail: invalid uri format"
    • response[0].website: must match format "uri"
  • GET /products (200) — mock getProductBadIp — in "Contract Validation - Products Mismatches (OpenAPI 3.0 — error mode) > should fail: invalid ipv4 format"
    • response[0].serverIp: must match format "ipv4"
  • GET /products (200) — mock getProductBadIpV6 — in "Contract Validation - Products Mismatches (OpenAPI 3.0 — error mode) > should fail: invalid ipv6 format"
    • response[0].serverIpV6: must match format "ipv6"
  • GET /products (200) — mock getProductZeroPrice — in "Contract Validation - Products Mismatches (OpenAPI 3.0 — error mode) > should fail: price of 0 violates exclusiveMinimum"
    • response[0].price: must be > 0
  • GET /products (200) — mock getProductNegQty — in "Contract Validation - Products Mismatches (OpenAPI 3.0 — error mode) > should fail: negative quantity violates minimum"
    • response[0].quantity: must be >= 0
  • GET /products (200) — mock getProductOverQty — in "Contract Validation - Products Mismatches (OpenAPI 3.0 — error mode) > should fail: quantity exceeds maximum"
    • response[0].quantity: must be <= 999999
  • GET /products (200) — mock getProductBadWeight — in "Contract Validation - Products Mismatches (OpenAPI 3.0 — error mode) > should fail: weight not multipleOf 0.01"
    • response[0].weight: must be multiple of 0.01
  • GET /products (200) — mock getProductBadRating — in "Contract Validation - Products Mismatches (OpenAPI 3.0 — error mode) > should fail: rating above maximum (5)"
    • response[0].rating: must be <= 5
  • GET /products (200) — mock getProductBadCurrency — in "Contract Validation - Products Mismatches (OpenAPI 3.0 — error mode) > should fail: invalid enum value for currency"
    • response[0].currency: must be one of: "USD", "EUR", "GBP", "JPY"
  • GET /products (200) — mock getProductBadCategory — in "Contract Validation - Products Mismatches (OpenAPI 3.0 — error mode) > should fail: invalid enum value for category"
    • response[0].category: must be one of: "electronics", "clothing", "food", "books", "toys"
  • GET /products (200) — mock getProductBadBool — in "Contract Validation - Products Mismatches (OpenAPI 3.0 — error mode) > should fail: string value for boolean inStock"
    • response[0].inStock: expected boolean, got string
  • GET /products (200) — mock getProductDupTags — in "Contract Validation - Products Mismatches (OpenAPI 3.0 — error mode) > should fail: duplicate tags violates uniqueItems"
    • response[0].tags: must NOT have duplicate items (items ## 1 and 0 are identical)
  • GET /products (200) — mock getProductTooManyTags — in "Contract Validation - Products Mismatches (OpenAPI 3.0 — error mode) > should fail: tags exceeds maxItems (10)"
    • response[0].tags: must NOT have more than 10 items
  • GET /products (200) — mock getProductBadMeta — in "Contract Validation - Products Mismatches (OpenAPI 3.0 — error mode) > should fail: non-string value in metadata additionalProperties"
    • response[0].metadata.count: expected string, got number
  • GET /settings (200) — mock getSettingsBadExtra — in "Contract Validation - Products Mismatches (OpenAPI 3.0 — error mode) > should fail: extra property on Settings (additionalProperties: false)"
    • response.extraField: unexpected property "extraField"
  • GET /settings (200) — mock getSettingsBadLang — in "Contract Validation - Products Mismatches (OpenAPI 3.0 — error mode) > should fail: invalid language pattern in Settings"
    • response.language: must match pattern "^[a-z]{2}(-[A-Z]{2})?$"
  • GET /products (200) — mock getProductBadNullable — in "Contract Validation - Products Mismatches (OpenAPI 3.0 — error mode) > should fail: wrong type for nullable description (number instead of string|null)"
    • response[0].description: expected string,null, got number

./contracts/events-3.1.json

  • GET /events (200) — mock getEventsEmpty — in "Contract Validation - Events Mismatches (OpenAPI 3.1 — error mode) > should fail: empty events array violates minItems (1)"
    • response: must NOT have fewer than 1 items
  • GET /events (200) — mock getEventShortName — in "Contract Validation - Events Mismatches (OpenAPI 3.1 — error mode) > should fail: event name too short (minLength: 3)"
    • response[0].name: must NOT have fewer than 3 characters
  • GET /events (200) — mock getEventBadDate — in "Contract Validation - Events Mismatches (OpenAPI 3.1 — error mode) > should fail: invalid date-time format for startDate"
    • response[0].startDate: must match format "date-time"
  • GET /events (200) — mock getEventFloatId — in "Contract Validation - Events Mismatches (OpenAPI 3.1 — error mode) > should fail: float value for integer id"
    • response[0].id: expected integer, got number
    • response[0].id: must match format "int64"
  • GET /events (200) — mock getEventBadBool — in "Contract Validation - Events Mismatches (OpenAPI 3.1 — error mode) > should fail: number value for boolean active"
    • response[0].active: expected boolean, got number
  • GET /events (200) — mock getEventBadStatus — in "Contract Validation - Events Mismatches (OpenAPI 3.1 — error mode) > should fail: invalid enum value for status"
    • response[0].status: must be one of: "draft", "published", "archived"
  • GET /events (200) — mock getEventScoreMax — in "Contract Validation - Events Mismatches (OpenAPI 3.1 — error mode) > should fail: score at exclusiveMaximum boundary (100)"
    • response[0].score: must be < 100
  • GET /events (200) — mock getEventLowPriority — in "Contract Validation - Events Mismatches (OpenAPI 3.1 — error mode) > should fail: priority below minimum (1)"
    • response[0].priority: must be >= 1
  • GET /events (200) — mock getEventHighPriority — in "Contract Validation - Events Mismatches (OpenAPI 3.1 — error mode) > should fail: priority above maximum (5)"
    • response[0].priority: must be <= 5
  • GET /events (200) — mock getEventDupAttendees — in "Contract Validation - Events Mismatches (OpenAPI 3.1 — error mode) > should fail: duplicate attendees violates uniqueItems"
    • response[0].attendees: must NOT have duplicate items (items ## 1 and 0 are identical)
  • GET /events (200) — mock getEventNoAttendees — in "Contract Validation - Events Mismatches (OpenAPI 3.1 — error mode) > should fail: empty attendees array violates minItems (1)"
    • response[0].attendees: must NOT have fewer than 1 items
  • GET /events (200) — mock getEventBadAttendee — in "Contract Validation - Events Mismatches (OpenAPI 3.1 — error mode) > should fail: invalid email format in attendees"
    • response[0].attendees[0]: must match format "email"
  • GET /events/{eventId} (200) — mock getEventBadNullable — in "Contract Validation - Events Mismatches (OpenAPI 3.1 — error mode) > should fail: wrong type for nullable description (number instead of string|null)"
    • response.description: expected string,null, got number

View full report →

@kevinccbsg
kevinccbsg merged commit c95f7aa into main Sep 24, 2026
6 checks passed
@kevinccbsg kevinccbsg mentioned this pull request Sep 24, 2026
kevinccbsg added a commit that referenced this pull request Sep 24, 2026
--help that never starts a run, per-command usage, and the refusal of
unknown flags (#34).

The record action's cli-version default moves to 1.9.0 in the same
commit. Pinning the action to a SHA only fixes how the CLI is invoked, so
leaving the default on 1.8.1 would ship an action that still runs the CLI
without any of this.

package-lock.json carries the two version fields and nothing else: no
install was run, so the wasm32-wasi optional packages and their @emnapi
deps are untouched and npm ci on Linux is unaffected.
@kevinccbsg
kevinccbsg deleted the feat/help-flag branch September 30, 2026 14:14
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant