feat(review): fail review --branch on an invalid workflow by SaulMoro · Pull Request #1266 · Tencent/teamai-cli

SaulMoro · 2026-10-21T16:45:55Z

Summary

teamai review --branch checked skills, rules, and agents, so a branch with a broken workflow still exited 0. The failure showed up on the next pull, which keeps the last valid copy. This check reuses syncWorkflowDefinitions, the validator pull already runs, and turns each invalid diagnostic into a fail finding. A valid workflow tree still prints no findings. Branch materialization (materializeRef) preserves symlinks (git mode 120000) and covers workflows, prompts, and scorers (prepared for #1085).

Type of Change

Bug fix (non-breaking change that fixes an issue)
New feature (non-breaking change that adds functionality)
Breaking change (fix or feature causing existing behavior to change)
Documentation only
Refactor / internal cleanup

Test Plan

npx tsc --noEmit passes
npm run lint passes
npx vitest run passes
Added/updated tests for the change

npx vitest run src/__tests__/review-branch.test.ts passes 17 tests, including bad step needs, missing prompt file, missing child workflow, and materializeRef extracting workflows, prompts, scorers, and preserving symlinks from a git revision.

After npm run build, the real CLI ran against a fixture team repo in a sandbox home. teamai review --branch feature/deploy exited 1 and printed:

[review] branch feature/deploy: 1 fail, 0 warn, 0 info
  ✖ [workflows] workflows/deploy.yaml:6: field "steps[0].needs": step "release" needs "biuld", which is not a step in this workflow

--json printed one object with ok: false and no warning lines in front of it. teamai review --branch feature/valid exited 0 and printed No findings. The sandbox home had no workflow cache afterwards.

Head 026042e: the same teamai review --branch feature/deploy run, with workflows/deploy.yaml copied verbatim from the guide, exited 1 and printed the guide's line exactly:

[review] branch feature/deploy: 1 fail, 0 warn, 0 info
  ✖ [workflows] workflows/deploy.yaml:6: field "steps[0].needs": step "release" needs "biuld", which is not a step in this workflow

workflow-docs-examples.test.ts now accepts an invalid example only when the same page prints its exact ✖ [workflows] diagnostic, and every such line must match a real diagnostic. Changing the shown "biuld" to "bild" fails it (1 of 21). review-branch.test.ts keeps a symlink to target.md (trailing space) verbatim. npx tsc --noEmit and npm run lint pass. The full npx vitest run suite was not run on this head.

Related Issues

Closes #1248. Scorer and benchmark validation in review --branch is now in the scope of #1085 and #1180, which add those definitions.

Notes for Reviewers

Push still warns and continues. Scorer validation joins here with #1085 (scorers is already in READ_ROOTS in materializeRef), and benchmark validation with #1180. A branch with no workflows/ directory gets no workflow finding.

materializeRef creates real symlinks for git mode 120000 blobs, ensuring syncWorkflowDefinitions inspects symlink prompt files and symlink workflow definitions with the exact same behavior as pull.

The call is dry-run and uses an empty temporary data home, so the verdict does not read or write this machine's workflow cache. A parent whose child has no runnable version on the branch fails here. Pull can still keep that parent when a cached child exists.

review --branch will gate a merge on workflow files. Lock the invalid file, the missing prompt, a parent whose child cannot run, and the materialization of workflows, prompts, scorers and symlinks before that check exists.

A kept cache can hide a parent whose child is invalid on the branch, so the gate checks an empty data home. The validator's warnings would also land in front of --json. Materialization preserves symlinks and covers scorers for #1085.

A workflow branch is gated with the same rules pull already runs, and the sample has to be the log that command prints.

github-actions · 2026-10-21T16:46:29Z

Code Erosion Report

Informational — never blocks the merge. scb-check==0.2.0 SlopCodeBench metrics; method & caveats in docs/dev/ci-code-erosion.md.

Metric	Value	Human band	Agent band	Reading
Verbosity*	0.063	0.15	0.33	human
Erosion	0.711	0.31	0.68	agent
Cognitive erosion	0.878	–	–	–

Scanned 393 files / 88281 SLOC · high-CC functions 682/7525

* On TypeScript, verbosity is partial and the bands are Python-calibrated.

Rule hits (TS verbosity layer)

Standalone ast-grep, separate from the number above.

Rule	Hits
`unnecessary-else-after-return`	53
`empty-catch-block`	5

58 distinct lines flagged across src/.

github-actions · 2026-10-21T16:54:31Z

No confident findings in the specified PR diff.

The PR description satisfies the testing requirement: it records a build followed by real-CLI checks for invalid and valid branches, JSON output, and cache isolation. The full suite is not required by the review rules.
The earlier Code Erosion Report is informational, not an actionable finding to re-raise or mark resolved.

Review was read-only; no PR code was run, built, or installed.

github-actions · 2026-10-21T17:02:41Z

No confident findings in the specified PR diff.

The PR description satisfies the testing requirement: it records a build followed by real-CLI checks for invalid and valid workflows, JSON output, and cache isolation. A full test suite or provider × agent matrix is not required.
The supplied earlier review raised no actionable findings, so none need marking resolved.

Review was read-only; no PR code was run, built, or installed.

github-actions · 2026-10-21T17:07:58Z

No confident findings in the specified PR diff.

The PR description satisfies the testing requirement: it records a build followed by real-CLI checks for invalid and valid workflows, JSON output, and cache isolation. A full suite or provider × agent matrix is not required.
Earlier reviews raised no actionable findings, so none need marking resolved.

Review was read-only; no PR code was run, built, or installed.

github-actions · 2026-10-21T17:25:49Z

[P3 nit] Preserve symlink targets verbatim — src/review-branch.ts:513: .trim() changes valid targets containing leading or trailing whitespace. A committed link to target.md becomes a link to target.md, potentially dangling or pointing to a different file. Remove .trim() to preserve the Git blob exactly.

The PR description satisfies the testing requirement: it records a build followed by real-CLI checks for invalid and valid workflows, JSON output, and cache isolation.

Earlier reviews raised no actionable findings to mark resolved. Review was read-only; no PR code was run, built, or installed.

…shown output

A link to `target.md ` was trimmed to `target.md`, so review could check a different file than pull reads.

github-actions · 2026-10-21T17:40:52Z

No confident new findings in the specified PR diff.

Resolved: the earlier symlink-whitespace finding. src/review-branch.ts:513 preserves the target without .trim(), and the added test checks a trailing-space target.
The PR description satisfies the testing requirement: it records a build followed by real-CLI checks for invalid and valid workflows, JSON output, and cache isolation, plus verification at head 026042e4. A full suite or provider × agent matrix is not required.

Review was read-only; no PR code was run, built, or installed.

SaulMoro added 3 commits October 21, 2026 18:39

test(review): expect an invalid workflow to fail review

5871223

review --branch will gate a merge on workflow files. Lock the invalid file, the missing prompt, a parent whose child cannot run, and the materialization of workflows, prompts, scorers and symlinks before that check exists.

docs(workflow): show the review --branch CI gate

f718d17

A workflow branch is gated with the same rules pull already runs, and the sample has to be the log that command prints.

SaulMoro force-pushed the fix/resolve-issue-1248 branch from ce1ae47 to c8d02cc Compare October 21, 2026 16:47

SaulMoro assigned jeff-r2026 Oct 21, 2026

SaulMoro mentioned this pull request Oct 21, 2026

[feat] Validate workflows, prompts and scorers in review --branch #1248

Closed

SaulMoro force-pushed the fix/resolve-issue-1248 branch from c8d02cc to 0c1d55b Compare October 21, 2026 17:01

SaulMoro force-pushed the fix/resolve-issue-1248 branch from 0c1d55b to 70a3d54 Compare October 21, 2026 17:06

SaulMoro force-pushed the fix/resolve-issue-1248 branch from 70a3d54 to f718d17 Compare October 21, 2026 17:23

SaulMoro added 3 commits October 21, 2026 19:33

test(workflow): let a guide show a workflow that fails review

0bb2ddb

test(workflow): check the guide's failing review example against its …

3830adf

…shown output

fix(review): keep a symlink's target exactly as git stores it

026042e

A link to `target.md ` was trimmed to `target.md`, so review could check a different file than pull reads.

SaulMoro mentioned this pull request Oct 22, 2026

[feat] Add workflow scorers and recorded quality labels #1085

Open

jeff-r2026 merged commit 5ca50c0 into main Oct 22, 2026
25 of 26 checks passed

Navigation Menu

Repository navigation

feat(review): fail review --branch on an invalid workflow - #1266

feat(review): fail review --branch on an invalid workflow#1266
jeff-r2026 merged 6 commits into
mainfrom
fix/resolve-issue-1248

SaulMoro commented Oct 21, 2026 •

edited

Loading

Uh oh!

github-actions Bot commented Oct 21, 2026 •

edited

Loading

Uh oh!

github-actions Bot commented Oct 21, 2026

Uh oh!

github-actions Bot commented Oct 21, 2026

Uh oh!

github-actions Bot commented Oct 21, 2026

Uh oh!

github-actions Bot commented Oct 21, 2026

Uh oh!

github-actions Bot commented Oct 21, 2026

Uh oh!

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Conversation

SaulMoro commented Oct 21, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Type of Change

Test Plan

Related Issues

Notes for Reviewers

Uh oh!

github-actions Bot commented Oct 21, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Code Erosion Report

Rule hits (TS verbosity layer)

Uh oh!

github-actions Bot commented Oct 21, 2026

Uh oh!

github-actions Bot commented Oct 21, 2026

Uh oh!

github-actions Bot commented Oct 21, 2026

Uh oh!

github-actions Bot commented Oct 21, 2026

Uh oh!

github-actions Bot commented Oct 21, 2026

Uh oh!

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

SaulMoro commented Oct 21, 2026 •

edited

Loading

github-actions Bot commented Oct 21, 2026 •

edited

Loading