From 682d53ac9152660a986206176d164c68cf2ab943 Mon Sep 17 00:00:00 2001 From: mrizzi Date: Thu, 27 Aug 2026 19:00:37 +0200 Subject: [PATCH 001/175] docs(verify-pr): record TC-5805 environment + resolution findings Confirm fullsend v0.37.0 toolchain, stock claude-runtime image binaries (all present -> TC-5806 no-op), and root-level harness resolution of the in-place sdlc-workflow plugin. Note v0.37.0 constraints for TC-5807: --fullsend-dir must be the repo root for relative children to resolve, and verify-pr is a valid harness role but not a valid config.yaml role (enum: fullsend/triage/coder/ review/fix/retro/prioritize/e2e). Implements TC-5805 Assisted-by: Claude Code Co-Authored-By: Claude Opus 4.8 --- docs/plans/notes/task1-findings.md | 112 +++++++++++++++++++++++++++++ 1 file changed, 112 insertions(+) create mode 100644 docs/plans/notes/task1-findings.md diff --git a/docs/plans/notes/task1-findings.md b/docs/plans/notes/task1-findings.md new file mode 100644 index 000000000..a4aed67e5 --- /dev/null +++ b/docs/plans/notes/task1-findings.md @@ -0,0 +1,112 @@ +# Task 1 findings — Environment + delivery/resolution confirmation + +**Jira:** [TC-5805](https://redhat.atlassian.net/browse/TC-5805) (Epic A: Delivery foundation, parent TC-5800) +**Date:** 2026-08-27 +**fullsend CLI:** v0.37.0 (built from `github.com/fullsend-ai/fullsend` tag `v0.37.0`) + +## IMAGE digest + +``` +IMAGE=ghcr.io/fullsend-ai/fullsend-code@sha256:9743bc7b6e451e0bcea25ae4a67e0c040c296f1fee04c08988ae80c53fafcfe6 +``` + +Source: `image:` in `/Users/mrizzi/git/cloned/agents/harness/review.yaml`. + +## 1. Toolchain — fullsend v0.37.0 + +Built with the version ldflag (plain `go build` stamps `version dev`): + +```bash +go build -C /Users/mrizzi/git/cloned/fullsend \ + -ldflags "-X github.com/fullsend-ai/fullsend/internal/cli.version=v0.37.0" \ + -o /Users/mrizzi/.local/bin/fullsend ./cmd/fullsend +fullsend --version # -> fullsend version v0.37.0 +``` + +**Confirmed:** `fullsend --version` reports `v0.37.0`. + +## 2. Stock image binaries + +`podman run --rm --entrypoint "" "$IMAGE" bash -lc 'command -v ...'`: + +| binary | path | +|---------|-----------------------------| +| claude | /usr/local/bin/claude | +| node | /usr/bin/node | +| python3 | /sandbox/.venv/bin/python3 | +| gh | /usr/bin/gh | +| git | /usr/bin/git | +| jq | /usr/bin/jq | +| curl | /usr/bin/curl | + +**Confirmed:** all required binaries present, none MISSING. `claude`/`node`/`python3`/`gh`/`git` +all resolve (python3 = `/sandbox/.venv/bin/python3`, required for ADR-0090 python sandbox hooks). + +➡️ **TC-5806 (sandbox image extension) is a NO-OP** — no required binary is missing. + +## 3. Root-level harness resolution + +Validated against a throwaway probe (`role: verify-pr`, `plugins: [plugins/sdlc-workflow]`, +`image: $IMAGE`) run from the repo root; probe never committed (working tree clean afterwards). + +**Confirmed:** a root-level harness bases its relative children at the repo root, so +`plugins/sdlc-workflow` resolves in place and the `sdlc-workflow:verify-pr` skill is present +with its sibling `shared/` intact: + +- `plugin.json` name = `sdlc-workflow`; `skills/` and `shared/` are siblings at the plugin root. +- `plugins/sdlc-workflow/skills/verify-pr/SKILL.md` exists; the skill's `../../shared/*.md` + links resolve to `plugins/sdlc-workflow/shared/` (comment-footnote.md, jira-rest-fallback.md, …). +- Skill id therefore = `sdlc-workflow:verify-pr`. + +Evidence chain (resolution-only, no dispatch): + +- `fullsend agent add harness/probe-verify-pr.yaml --name probe-verify-pr --fullsend-dir .` + → `✓ Added agent "probe-verify-pr"` (the root-level harness path resolves at the repo root). +- `fullsend lock probe-verify-pr --fullsend-dir . --offline --max-depth 0` + → loads + validates the harness and runs `ResolveRelativeTo(absFullsendDir)` with **no** + "resolves outside fullsend directory" error → `plugins/sdlc-workflow` resolves cleanly + inside the repo root; "no remote dependencies" confirms it is treated as a local, in-place path. +- Source confirmation (fullsend v0.37.0): + - `internal/cli/run.go:464` and `internal/cli/lock.go:248` call + `Harness.ResolveRelativeTo(absFullsendDir)` — relative local paths resolve against the + **fullsend-dir**, which must be the repo root. + - `internal/cli/run.go:670` calls `Harness.ValidateFilesExist()`, which `os.Stat`s every + plugin path (`internal/harness/harness.go:798`). + +### ⚠️ Deviations from the task's literal commands (important for TC-5807) + +These are real constraints the delivery model must account for; the task's example commands +do **not** work verbatim on fullsend v0.37.0: + +1. **`--fullsend-dir` must be the repo root**, not a scratch `/tmp` dir. fullsend resolves the + harness source path *and* the harness's relative children (`plugins/…`) against + `absFullsendDir` = the `--fullsend-dir` value (`ResolveRelativeTo`, `validateLocalPath`). + With `--fullsend-dir /tmp/probe-fs` the probe fails + (`local path does not exist: /tmp/probe-fs/harness/probe-verify-pr.yaml`) and + `plugins/sdlc-workflow` would resolve under `/tmp`, not the repo. The probe here used + `--fullsend-dir .` at the repo root and a scratch `config.yaml`/`harness/` (both removed). + +2. **`verify-pr` is NOT a valid `config.yaml` role.** `config.yaml` `roles:` are validated + against a fixed enum — `fullsend, triage, coder, review, fix, retro, prioritize, e2e` + (`internal/config/config.go:196`); `verify-pr` is rejected + (`invalid role "verify-pr": must be one of …`). The **harness** `role:` field is only + regex-validated (`internal/harness/harness.go:474`), so `role: verify-pr` is accepted in the + harness YAML. The scratch `config.yaml` therefore declared a valid enum role (`review`) while + the harness kept `role: verify-pr`, and `agent add` succeeded. TC-5807 must reconcile this: + the installation `config.yaml` cannot list `verify-pr` as a role in v0.37.0. + +3. **`fullsend agent add` does not stage plugins.** It only validates the harness source path + exists and records the source string in `config.yaml` (`internal/cli/agent.go:174-205`). + Plugin resolution/staging and existence checks happen at `fullsend run` time. The + `fullsend run … --offline --max-depth 0` "secondary gate" was **not** executed here because + `fullsend run` sets up the sandbox and dispatches a real agent (not resolution-only); + resolution was confirmed via `agent add` + `lock` + source review instead. + +## Acceptance + +- [x] `fullsend --version` reports v0.37.0; IMAGE digest recorded. +- [x] Required binaries present (none MISSING) → TC-5806 no-op. +- [x] Throwaway probe: `agent add` resolves the in-place plugin at the repo root; + `sdlc-workflow:verify-pr` present with `skills/verify-pr/SKILL.md` + sibling `shared/`; + probe removed (working tree clean). +- [x] Findings committed (trailers required). From 7f2bc8c1c006d5eb11d614cbf6706b0282e579c0 Mon Sep 17 00:00:00 2001 From: mrizzi Date: Mon, 31 Aug 2026 13:30:35 +0200 Subject: [PATCH 002/175] feat(verify-pr): add standalone root-level fullsend harness Author harness/verify-pr.yaml at the repo root so its relative children resolve against the repo root and plugins/sdlc-workflow is delivered in place as a whole plugin. Declares every field explicitly with no base composition; omits security: (Go defaults supply it) and any forge block (GitHub is tier-1, Vertex-only provider). Split-trust env: Jira/GitHub tokens on the runner only, read-only context (JIRA_ISSUE_ID, JIRA_BASE_URL) in the sandbox. Also adds the sandbox agent prompt (agents/verify-pr.md) and the Vertex env file (env/gcp-vertex.env). Referenced children (providers, policy, profile, schema, pre/post scripts) are authored by later epic tasks; this task validates the harness syntactically only. Implements TC-5807 Assisted-by: Claude Code Co-Authored-By: Claude Opus 4.8 --- harness/verify-pr.yaml | 95 +++++++++++++++++++++++ plugins/sdlc-workflow/agents/verify-pr.md | 46 +++++++++++ plugins/sdlc-workflow/env/gcp-vertex.env | 5 ++ 3 files changed, 146 insertions(+) create mode 100644 harness/verify-pr.yaml create mode 100644 plugins/sdlc-workflow/agents/verify-pr.md create mode 100644 plugins/sdlc-workflow/env/gcp-vertex.env diff --git a/harness/verify-pr.yaml b/harness/verify-pr.yaml new file mode 100644 index 000000000..8cead2388 --- /dev/null +++ b/harness/verify-pr.yaml @@ -0,0 +1,95 @@ +# verify-pr — standalone root-level fullsend harness (no base composition). +# +# Runs /sdlc-workflow:verify-pr as a fullsend BYOA agent that checks a PR +# against its Jira task's acceptance criteria. Placed at the REPO ROOT so +# fullsend (invoked with --fullsend-dir = repo root) resolves this harness's +# relative children against the repo root: `plugins/sdlc-workflow` is delivered +# in place as a whole plugin (fullsend fabricates the marketplace cache) and its +# sibling `shared/` resources resolve intact. Confirmed against fullsend v0.37.0 +# — see docs/plans/notes/task1-findings.md (TC-5805). +# +# No base composition and no explicit `security:` block: the Go defaults already +# supply security (enabled: true, fail_mode: closed) plus the ADR-0090 sandbox +# hooks. Composing agents/review.yaml would only leak its review-agent semantics. +# +# Split-trust I/O: the Jira and GitHub tokens live ONLY on the runner (pre_script +# prefetch + post_script writes) and are NEVER exposed to the sandbox. The +# sandbox receives read-only context only (JIRA_ISSUE_ID, JIRA_BASE_URL). +# +# Several referenced children are authored by later tasks in this epic and do not +# exist yet — providers/vertex-ai.yaml (TC-5808), policies/verify-pr.yaml +# (TC-5809), profiles/fullsend-vertex-ai.yaml + schemas/verify-pr-result.schema.json +# + scripts/pre-verify-pr.sh (TC-5810), scripts/post-verify-pr.sh (TC-5811). This +# task validates the harness syntactically only; full fullsend resolution is +# deferred to TC-5810/TC-5811. + +role: verify-pr +agent: plugins/sdlc-workflow/agents/verify-pr.md +model: opus +# Reasoning effort pinned to the fleet's implicit default: Claude Code runs at +# high effort on effort-capable models, and the fullsend pi runtime passes +# --thinking high when effort is unset. Revisit if `model:` moves to a +# generation with a different default effort. +effort: high +image: ghcr.io/fullsend-ai/fullsend-code@sha256:9743bc7b6e451e0bcea25ae4a67e0c040c296f1fee04c08988ae80c53fafcfe6 +readonly_repo: true + +# In-place whole-plugin delivery — resolved at the repo root. +plugins: + - plugins/sdlc-workflow + +policy: plugins/sdlc-workflow/policies/verify-pr.yaml + +# Vertex only — GitHub is tier-1 and needs no provider. +providers: + - plugins/sdlc-workflow/providers/vertex-ai.yaml + +openshell: + profiles: + - plugins/sdlc-workflow/profiles/fullsend-vertex-ai.yaml + +host_files: + - src: plugins/sdlc-workflow/env/gcp-vertex.env + dest: /sandbox/workspace/.env.d/gcp-vertex.env + expand: true + - src: ${GOOGLE_APPLICATION_CREDENTIALS} + dest: /tmp/.gcp-credentials.json + - src: ${GCP_OIDC_TOKEN_FILE} + dest: /sandbox/workspace/.gcp-oidc-token + optional: true + # pre_script prefetch output, mounted read-only into the sandbox (optional — + # absent until pre-verify-pr.sh runs, authored in TC-5810). + - src: /tmp/fullsend-pre-output/verify-pr-input.json + dest: /sandbox/workspace/.pre-script/verify-pr-input.json + optional: true + +pre_script: plugins/sdlc-workflow/scripts/pre-verify-pr.sh +post_script: plugins/sdlc-workflow/scripts/post-verify-pr.sh + +validation_loop: + # script: added in TC-5810 alongside the schema's producer/validator. + schema: plugins/sdlc-workflow/schemas/verify-pr-result.schema.json + max_iterations: 2 + +env: + # Tokens (JIRA_API_TOKEN, GH_TOKEN) live on the runner ONLY — the pre_script + # prefetch and post_script writes need them; they are never placed in + # env.sandbox. + runner: + JIRA_ISSUE_ID: "${JIRA_ISSUE_ID}" + JIRA_SERVER_URL: "${JIRA_SERVER_URL}" + JIRA_EMAIL: "${JIRA_EMAIL}" + JIRA_API_TOKEN: "${JIRA_API_TOKEN}" + JIRA_PROJECT_KEY: "${JIRA_PROJECT_KEY}" + GH_TOKEN: "${GH_TOKEN}" + FULLSEND_OUTPUT_SCHEMA: "${FULLSEND_DIR}/plugins/sdlc-workflow/schemas/verify-pr-result.schema.json" + FULLSEND_OUTPUT_FILE: "agent-result.json" + # Read-only context only — no tokens, and no atlassian/github egress from the + # sandbox (network policy enforced in TC-5809). JIRA_ISSUE_ID is required by + # the agent prompt; JIRA_BASE_URL is the base for display links only and is + # sourced from the canonical JIRA_SERVER_URL host var (no token). + sandbox: + JIRA_ISSUE_ID: "${JIRA_ISSUE_ID}" + JIRA_BASE_URL: "${JIRA_SERVER_URL}" + +timeout_minutes: 30 diff --git a/plugins/sdlc-workflow/agents/verify-pr.md b/plugins/sdlc-workflow/agents/verify-pr.md new file mode 100644 index 000000000..7c14d3e3c --- /dev/null +++ b/plugins/sdlc-workflow/agents/verify-pr.md @@ -0,0 +1,46 @@ +--- +name: verify-pr +description: >- + Verify a PR against its Jira task acceptance criteria using the + sdlc-workflow verify-pr skill inside an OpenShell sandbox. +model: opus +--- + +# Verify PR Agent + +You are a PR verification agent running inside an OpenShell sandbox. The +sdlc-workflow plugin is delivered in place, so the `/sdlc-workflow:verify-pr` +skill and its shared resources are available to you directly. + +The sandbox is read-only and has no Jira or GitHub write access: your Jira +context is pre-fetched onto disk and any Jira/GitHub side effects are performed +by the runner after you finish. Produce the structured output and nothing else. + +## Startup procedure + +1. Read the `JIRA_ISSUE_ID` environment variable: + ```bash + echo $JIRA_ISSUE_ID + ``` + +2. Invoke the verify-pr skill with that issue ID. The skill is available as + `/sdlc-workflow:verify-pr`. Example: + ``` + /sdlc-workflow:verify-pr TC-4715 + ``` + +3. The skill handles everything: reading the pre-fetched Jira task, identifying + the PR, dispatching sub-agents for analysis, and producing the output. + +4. After the skill completes, verify the output file exists: + ```bash + ls -la $FULLSEND_OUTPUT_DIR/agent-result.json + ``` + +## Constraints + +- Do not modify code. This agent only verifies. +- Do not push branches or create PRs. +- Do not call Jira write APIs directly — the skill writes structured JSON output. +- Do not post GitHub comments directly — the post_script handles this. +- Follow the skill's output — do not improvise verification steps. diff --git a/plugins/sdlc-workflow/env/gcp-vertex.env b/plugins/sdlc-workflow/env/gcp-vertex.env new file mode 100644 index 000000000..6eedfa648 --- /dev/null +++ b/plugins/sdlc-workflow/env/gcp-vertex.env @@ -0,0 +1,5 @@ +export CLAUDE_CODE_USE_VERTEX=1 +export ANTHROPIC_VERTEX_PROJECT_ID=${ANTHROPIC_VERTEX_PROJECT_ID} +export CLOUD_ML_REGION=${CLOUD_ML_REGION} +export GOOGLE_APPLICATION_CREDENTIALS=/tmp/.gcp-credentials.json +export GOOGLE_CLOUD_PROJECT=${GOOGLE_CLOUD_PROJECT} From 118ca193e7041b2d8597a3ca1bd25109a8051d17 Mon Sep 17 00:00:00 2001 From: mrizzi Date: Mon, 31 Aug 2026 16:28:23 +0200 Subject: [PATCH 003/175] feat(verify-pr): add Vertex AI provider and profile for fullsend harness Vertex is the sole in-sandbox provider (tier 4). Jira and GitHub are tier-1 (prefetched host-side) and need no in-sandbox provider, so only the Vertex pair is declared locally for the standalone harness. Copied verbatim from the stock agents Vertex pair: provider type fullsend-vertex-ai matches profile id fullsend-vertex-ai (endpoint *.googleapis.com:443), which the root harness/verify-pr.yaml references via providers:/openshell.profiles:. Implements TC-5808 Assisted-by: Claude Code Co-Authored-By: Claude Opus 4.8 --- .../profiles/fullsend-vertex-ai.yaml | 15 +++++++++++++++ plugins/sdlc-workflow/providers/vertex-ai.yaml | 5 +++++ 2 files changed, 20 insertions(+) create mode 100644 plugins/sdlc-workflow/profiles/fullsend-vertex-ai.yaml create mode 100644 plugins/sdlc-workflow/providers/vertex-ai.yaml diff --git a/plugins/sdlc-workflow/profiles/fullsend-vertex-ai.yaml b/plugins/sdlc-workflow/profiles/fullsend-vertex-ai.yaml new file mode 100644 index 000000000..153bdf14d --- /dev/null +++ b/plugins/sdlc-workflow/profiles/fullsend-vertex-ai.yaml @@ -0,0 +1,15 @@ +--- +id: fullsend-vertex-ai +display_name: Fullsend Vertex AI +description: Google Cloud APIs for Vertex AI inference +category: inference +endpoints: + - host: "*.googleapis.com" + port: 443 + protocol: rest + access: read-write + enforcement: enforce +binaries: + - "**/claude" + - "**/pi" + - "**/node" diff --git a/plugins/sdlc-workflow/providers/vertex-ai.yaml b/plugins/sdlc-workflow/providers/vertex-ai.yaml new file mode 100644 index 000000000..50ba5f207 --- /dev/null +++ b/plugins/sdlc-workflow/providers/vertex-ai.yaml @@ -0,0 +1,5 @@ +--- +name: vertex-ai +type: fullsend-vertex-ai +credentials: + _NOOP_VERTEX_AI: "" From 2cb0b758dfb04dc97fe08f9d9431dcb82a419280 Mon Sep 17 00:00:00 2001 From: mrizzi Date: Mon, 31 Aug 2026 17:22:58 +0200 Subject: [PATCH 004/175] feat(verify-pr): add read-only sandbox policy with no tier-1 egress Add plugins/sdlc-workflow/policies/verify-pr.yaml for the verify-pr fullsend harness (referenced by harness/verify-pr.yaml). Filesystem is read-only (include_workdir: false); egress is reduced to Anthropic + Vertex AI (*.googleapis.com) inference and telemetry only. No *.atlassian.net (Jira) and no api.github.com (GitHub) egress: those are tier-1 with runner-only tokens under the split-trust I/O model (prefetch/post-script run host-side). curl and gh are excluded from the binary allowlist to block raw HTTP with injected tokens. Live fullsend egress verification is deferred to TC-5810/TC-5811, which author the harness's remaining siblings (pre/post scripts, result schema). Implements TC-5809 Assisted-by: Claude Code Co-Authored-By: Claude Opus 4.8 --- plugins/sdlc-workflow/policies/verify-pr.yaml | 69 +++++++++++++++++++ 1 file changed, 69 insertions(+) create mode 100644 plugins/sdlc-workflow/policies/verify-pr.yaml diff --git a/plugins/sdlc-workflow/policies/verify-pr.yaml b/plugins/sdlc-workflow/policies/verify-pr.yaml new file mode 100644 index 000000000..ce18a449b --- /dev/null +++ b/plugins/sdlc-workflow/policies/verify-pr.yaml @@ -0,0 +1,69 @@ +version: 1 + +# Sandbox policy for the verify-pr fullsend harness (read-only agent). +# +# The agent checks a PR against its Jira task's acceptance criteria. It needs +# ONLY inference egress: Anthropic hosts + Vertex AI (`*.googleapis.com`). +# Format confirmed against fullsend v0.37.0 upstream policies +# (/Users/mrizzi/git/cloned/agents/policies/*): per-entry `name:` is still +# required, hosts support `*` wildcards, and binaries use `**/…` globs. +# +# Split-trust I/O — deliberately NO tier-1 egress from the sandbox: +# * NO `*.atlassian.net` (Jira): the Jira token lives only on the runner; +# issue context is prefetched host-side (pre_script, TC-5810) and mounted +# read-only, and all Jira writes run host-side (post_script, TC-5811). +# * NO `api.github.com`/`github.com` (GitHub): the GitHub token lives only on +# the runner; PR context is prefetched host-side and writes run host-side. +# A `gh` read reaching api.github.com from the sandbox is a leak to fix in +# the skill/prefetch — NOT a reason to add a github egress rule here. +# +# curl and gh are excluded from the binary allowlist to prevent raw HTTP +# egress with any injected token; only claude/node may open the network. +# +# include_workdir: false — the fullsend dir is not mounted into the sandbox; +# the target repo is delivered read-only under /sandbox (readonly_repo: true). + +filesystem_policy: + include_workdir: false + read_only: [/var/log, /usr, /lib, /lib64, /proc, /dev/urandom, /etc, /opt] + read_write: [/sandbox, /tmp, /dev/null] +landlock: + compatibility: best_effort +process: + run_as_user: sandbox + run_as_group: sandbox + +network_policies: + claude_code: + name: claude-code + endpoints: + # Anthropic inference + Vertex AI (googleapis) — the only egress allowed. + - host: "api.anthropic.com" + port: 443 + protocol: rest + enforcement: enforce + access: read-write + - host: "*.googleapis.com" + port: 443 + protocol: rest + enforcement: enforce + access: read-write + - host: "platform.claude.com" + port: 443 + protocol: rest + enforcement: enforce + access: read-write + # Telemetry / error reporting (POST) — kept minimal. + - host: "statsig.anthropic.com" + port: 443 + protocol: rest + enforcement: enforce + access: read-write + - host: "sentry.io" + port: 443 + protocol: rest + enforcement: enforce + access: read-write + binaries: + - path: "**/claude" + - path: "**/node" From 828294798c65ba6c668b940e6a4ac047d37f15fe Mon Sep 17 00:00:00 2001 From: mrizzi Date: Mon, 31 Aug 2026 18:37:29 +0200 Subject: [PATCH 005/175] fix(verify-pr): add **/pi to sandbox policy binary allowlist MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The verify-pr fullsend policy granted Vertex AI (*.googleapis.com) egress but omitted the pi runtime from the binary allowlist. Under fullsend's dual host-AND-binary match rule, the pi runtime — which brokers Vertex inference — would be blocked. Add **/pi alongside **/claude and **/node to match the composed fullsend-vertex-ai profile and every upstream vertex policy. Implements TC-5880 Assisted-by: Claude Code Co-Authored-By: Claude Opus 4.8 --- plugins/sdlc-workflow/policies/verify-pr.yaml | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/plugins/sdlc-workflow/policies/verify-pr.yaml b/plugins/sdlc-workflow/policies/verify-pr.yaml index ce18a449b..8d4108c94 100644 --- a/plugins/sdlc-workflow/policies/verify-pr.yaml +++ b/plugins/sdlc-workflow/policies/verify-pr.yaml @@ -18,7 +18,9 @@ version: 1 # the skill/prefetch — NOT a reason to add a github egress rule here. # # curl and gh are excluded from the binary allowlist to prevent raw HTTP -# egress with any injected token; only claude/node may open the network. +# egress with any injected token; only the fullsend runtimes (claude/pi/node) +# may open the network. `**/pi` is the runtime that brokers Vertex AI +# (`*.googleapis.com`) inference, so it must be present alongside claude/node. # # include_workdir: false — the fullsend dir is not mounted into the sandbox; # the target repo is delivered read-only under /sandbox (readonly_repo: true). @@ -66,4 +68,5 @@ network_policies: access: read-write binaries: - path: "**/claude" + - path: "**/pi" - path: "**/node" From 4cfc42d2db4e9e29d10b040cb84a6102cb8e5c61 Mon Sep 17 00:00:00 2001 From: mrizzi Date: Mon, 31 Aug 2026 19:39:19 +0200 Subject: [PATCH 006/175] =?UTF-8?q?feat(verify-pr):=20split-trust=20I/O=20?= =?UTF-8?q?=E2=80=94=20schemas,=20prefetch,=20native=20validation=5Floop?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Add the split-trust I/O contract for the verify-pr fullsend harness: the pre_script prefetches everything on the trusted runner (where the Jira and GitHub tokens live) and the sandbox reads only JSON — no api.github.com or atlassian egress. - schemas/verify-pr-input.schema.json — tracker-agnostic pre-script input, extended with a `github` tier-1 read bundle (diff, stat, reviews, review_comments, issue_comments, commits, headRefName, commit_sha). - schemas/verify-pr-result.schema.json — agent result contract (ported). - scripts/pre-verify-pr.sh — validates env, fetches the Jira issue, resolves the linked PR, prefetches every GitHub read the skill performs, and writes verify-pr-input.json for host_files to mount. - scripts/pre_verify_pr.py — PR-URL extraction + input transform, extended with build_github_bundle and a --github-dir transform mode. - scripts/{validate-output-schema.sh,strip_extra_properties.py} — kept per the Step 5 native-validator decision (below); wired via validation_loop.script. - scripts/test_pre_verify_pr.py — unit tests incl. the github bundle. - harness/verify-pr.yaml — set validation_loop.script. Deviations from the literal task steps, dictated by the verified fullsend v0.37.0 contract (same class as TC-5805): - Skip signal (step 4) uses the pre-script output protocol v1 — line-based `skipped=true` / `reason=...` appended to $FULLSEND_PRESCRIPT_OUTPUT (guarded, exit 0) — not a JSON `{"skipped":true}` document. That env var is the skip-signal file, not the input path. - GitHub bundle is embedded in verify-pr-input.json (single file) rather than separate in-sandbox file paths, keeping the harness change scoped to validation_loop; input JSON stays at /tmp/fullsend-pre-output (host_files src), with PRE_DIR overridable for host-side testing. - No host-side PR-head checkout (step 3): the prefetched PR diff is sufficient for verification, so the sandbox needs no writable checkout. - Native validator decision (step 5): additionalProperties:false rejects benign agent extras, so validate-output-schema.sh + strip_extra_properties.py are KEPT and validation_loop.script is set (fullsend also requires a script for validation_loop — schema alone is insufficient). Validated host-side: both schemas are valid JSON; 15 unit tests pass; the validator strips benign extras and passes, and fails a bad enum (exit 1); the skip path exits 0 with the correct signal; the happy path (stub gh + jira) produces a verify-pr-input.json that validates against the input schema with the full github bundle embedded. Implements TC-5810 Co-Authored-By: Claude Opus 4.8 Assisted-by: Claude Code --- harness/verify-pr.yaml | 7 +- .../schemas/verify-pr-input.schema.json | 107 ++++++++ .../schemas/verify-pr-result.schema.json | 134 ++++++++++ .../sdlc-workflow/scripts/pre-verify-pr.sh | 136 ++++++++++ .../sdlc-workflow/scripts/pre_verify_pr.py | 156 ++++++++++++ .../scripts/strip_extra_properties.py | 101 ++++++++ .../scripts/test_pre_verify_pr.py | 240 ++++++++++++++++++ .../scripts/validate-output-schema.sh | 85 +++++++ 8 files changed, 965 insertions(+), 1 deletion(-) create mode 100644 plugins/sdlc-workflow/schemas/verify-pr-input.schema.json create mode 100644 plugins/sdlc-workflow/schemas/verify-pr-result.schema.json create mode 100755 plugins/sdlc-workflow/scripts/pre-verify-pr.sh create mode 100644 plugins/sdlc-workflow/scripts/pre_verify_pr.py create mode 100644 plugins/sdlc-workflow/scripts/strip_extra_properties.py create mode 100644 plugins/sdlc-workflow/scripts/test_pre_verify_pr.py create mode 100755 plugins/sdlc-workflow/scripts/validate-output-schema.sh diff --git a/harness/verify-pr.yaml b/harness/verify-pr.yaml index 8cead2388..8367df650 100644 --- a/harness/verify-pr.yaml +++ b/harness/verify-pr.yaml @@ -67,7 +67,12 @@ pre_script: plugins/sdlc-workflow/scripts/pre-verify-pr.sh post_script: plugins/sdlc-workflow/scripts/post-verify-pr.sh validation_loop: - # script: added in TC-5810 alongside the schema's producer/validator. + # Custom validator: the result schema uses additionalProperties:false + # throughout to document the contract, so strip_extra_properties.py drops + # benign agent-added metadata before jsonschema validates (fullsend's + # native schema-only check would reject it). fullsend requires a script for + # validation_loop regardless — schema alone is not sufficient. + script: plugins/sdlc-workflow/scripts/validate-output-schema.sh schema: plugins/sdlc-workflow/schemas/verify-pr-result.schema.json max_iterations: 2 diff --git a/plugins/sdlc-workflow/schemas/verify-pr-input.schema.json b/plugins/sdlc-workflow/schemas/verify-pr-input.schema.json new file mode 100644 index 000000000..f40d09e06 --- /dev/null +++ b/plugins/sdlc-workflow/schemas/verify-pr-input.schema.json @@ -0,0 +1,107 @@ +{ + "$schema": "https://json-schema.org/draft/2020-12/schema", + "$id": "verify-pr-input.schema.json", + "title": "Verify PR Pre-Script Input", + "description": "Pre-fetched task data written by the pre_script and mounted into the sandbox. The skill reads this instead of calling the issue tracker or GitHub APIs directly.", + "type": "object", + "required": ["task_id", "task", "pr_url", "github"], + "additionalProperties": false, + "properties": { + "task_id": { + "type": "string", + "minLength": 1, + "description": "Issue tracker task ID (e.g., TC-4741, #42)" + }, + "task": { + "type": "object", + "required": ["summary", "description", "status", "labels"], + "properties": { + "summary": { "type": "string", "minLength": 1 }, + "description": { + "type": "object", + "description": "Task description in the issue tracker's native format (e.g., ADF for Jira)" + }, + "status": { "type": "string" }, + "labels": { + "type": "array", + "items": { "type": "string" } + }, + "issue_links": { + "type": "array", + "items": { + "type": "object", + "properties": { + "type": { "type": "string" }, + "direction": { "type": "string", "enum": ["inward", "outward"] }, + "key": { "type": "string" } + } + }, + "description": "Related issues (parent, blocks, relates)" + }, + "custom_fields": { + "type": "object", + "description": "Tracker-specific custom field values" + } + } + }, + "pr_url": { + "type": "string", + "description": "PR URL extracted from the task, or empty if not linked" + }, + "github": { + "type": "object", + "description": "GitHub tier-1 read bundle, prefetched on the trusted runner (where GH_TOKEN lives) so the sandbox needs no api.github.com egress. Every read the verify-pr skill performs against the PR is captured here.", + "required": ["pr_repo", "pr_number", "headRefName", "commit_sha", "diff", "stat", "reviews", "review_comments", "issue_comments", "commits"], + "additionalProperties": false, + "properties": { + "pr_repo": { + "type": "string", + "pattern": "^[a-zA-Z0-9._-]+/[a-zA-Z0-9._-]+$", + "description": "owner/repo parsed from pr_url" + }, + "pr_number": { "type": "integer", "minimum": 1 }, + "headRefName": { + "type": "string", + "description": "PR head branch name (gh pr view --json headRefName)" + }, + "commit_sha": { + "type": "string", + "pattern": "^[0-9a-f]{7,40}$", + "description": "OID of the PR head (last) commit (gh pr view --json commits --jq '.commits[-1].oid')" + }, + "diff": { + "type": "string", + "description": "Unified PR diff (gh pr diff)" + }, + "stat": { + "type": "string", + "description": "PR diffstat (gh pr diff --stat)" + }, + "reviews": { + "type": "array", + "description": "PR reviews (gh api repos//pulls//reviews)" + }, + "review_comments": { + "type": "array", + "description": "PR review (inline code) comments (gh api repos//pulls//comments)" + }, + "issue_comments": { + "type": "array", + "description": "PR conversation comments (gh api repos//issues//comments)" + }, + "commits": { + "type": "array", + "description": "PR commits (gh pr view --json commits)" + } + } + }, + "source": { + "type": "object", + "description": "Raw issue tracker response for fields not captured above", + "properties": { + "tracker": { "type": "string", "description": "Issue tracker type (e.g., jira, github)" }, + "raw": { "type": "object", "description": "Full raw API response" } + } + } + } +} diff --git a/plugins/sdlc-workflow/schemas/verify-pr-result.schema.json b/plugins/sdlc-workflow/schemas/verify-pr-result.schema.json new file mode 100644 index 000000000..aeaf9c365 --- /dev/null +++ b/plugins/sdlc-workflow/schemas/verify-pr-result.schema.json @@ -0,0 +1,134 @@ +{ + "$schema": "https://json-schema.org/draft/2020-12/schema", + "$id": "verify-pr-result.schema.json", + "title": "Verify PR Agent Result", + "description": "Structured output from the verify-pr agent. Validated by fullsend's validation_loop before the post_script runs.", + "type": "object", + "additionalProperties": false, + "required": ["report", "actions"], + "properties": { + "report": { "$ref": "#/$defs/report" }, + "actions": { + "type": "array", + "items": { "$ref": "#/$defs/action" } + } + }, + "$defs": { + "report": { + "type": "object", + "required": ["jira_issue_id", "pr_repo", "pr_number", "commit_sha", "overall", "table_md", "report_md", "report_adf", "plugin_version"], + "additionalProperties": false, + "properties": { + "jira_issue_id": { "type": "string", "pattern": "^[A-Z]+-[0-9]+$" }, + "pr_repo": { "type": "string", "pattern": "^[a-zA-Z0-9._-]+/[a-zA-Z0-9._-]+$" }, + "pr_number": { "type": "integer", "minimum": 1 }, + "commit_sha": { "type": "string", "pattern": "^[0-9a-f]{7,40}$" }, + "overall": { "type": "string", "enum": ["PASS", "WARN", "FAIL"] }, + "table_md": { "type": "string", "minLength": 1 }, + "report_md": { "type": "string", "minLength": 1 }, + "report_adf": { "type": "object" }, + "plugin_version": { "type": "string", "pattern": "^[0-9]+\\.[0-9]+\\.[0-9]+$" } + } + }, + "action": { + "type": "object", + "required": ["type"], + "properties": { + "type": { "type": "string", "enum": ["create_subtask", "create_link", "post_pr_reply", "post_pr_comment", "create_root_cause_task", "post_comment", "post_report"] } + }, + "allOf": [ + { + "if": { "properties": { "type": { "const": "create_subtask" } } }, + "then": { + "required": ["type", "ref", "parent", "summary", "labels", "description_adf"], + "properties": { + "type": {}, + "ref": { "type": "string", "pattern": "^[a-z0-9-]+$" }, + "parent": { "type": "string", "pattern": "^[A-Z]+-[0-9]+$" }, + "summary": { "type": "string", "minLength": 1, "maxLength": 255 }, + "labels": { "type": "array", "items": { "type": "string" } }, + "description_adf": { "type": "object" } + }, + "additionalProperties": false + } + }, + { + "if": { "properties": { "type": { "const": "create_link" } } }, + "then": { + "required": ["type", "link_type", "inward", "outward"], + "properties": { + "type": {}, + "link_type": { "type": "string", "enum": ["Blocks", "Related"] }, + "inward": { "type": "string", "minLength": 1 }, + "outward": { "type": "string", "minLength": 1 } + }, + "additionalProperties": false + } + }, + { + "if": { "properties": { "type": { "const": "post_pr_reply" } } }, + "then": { + "required": ["type", "repo", "pr_number", "comment_id", "body"], + "properties": { + "type": {}, + "repo": { "type": "string" }, + "pr_number": { "type": "integer", "minimum": 1 }, + "comment_id": { "type": "integer", "minimum": 1 }, + "body": { "type": "string", "minLength": 1 } + }, + "additionalProperties": false + } + }, + { + "if": { "properties": { "type": { "const": "post_pr_comment" } } }, + "then": { + "required": ["type", "repo", "pr_number", "body"], + "properties": { + "type": {}, + "repo": { "type": "string" }, + "pr_number": { "type": "integer", "minimum": 1 }, + "body": { "type": "string", "minLength": 1 } + }, + "additionalProperties": false + } + }, + { + "if": { "properties": { "type": { "const": "create_root_cause_task" } } }, + "then": { + "required": ["type", "ref", "summary", "labels", "description_adf"], + "properties": { + "type": {}, + "ref": { "type": "string", "pattern": "^[a-z0-9-]+$" }, + "summary": { "type": "string", "minLength": 1, "maxLength": 255 }, + "labels": { "type": "array", "items": { "type": "string" } }, + "description_adf": { "type": "object" } + }, + "additionalProperties": false + } + }, + { + "if": { "properties": { "type": { "const": "post_comment" } } }, + "then": { + "required": ["type", "issue", "body_adf"], + "properties": { + "type": {}, + "issue": { "type": "string", "minLength": 1 }, + "body_adf": { "type": "object" } + }, + "additionalProperties": false + } + }, + { + "if": { "properties": { "type": { "const": "post_report" } } }, + "then": { + "required": ["type"], + "properties": { + "type": {} + }, + "additionalProperties": false + } + } + ] + } + } +} diff --git a/plugins/sdlc-workflow/scripts/pre-verify-pr.sh b/plugins/sdlc-workflow/scripts/pre-verify-pr.sh new file mode 100755 index 000000000..0842c28a1 --- /dev/null +++ b/plugins/sdlc-workflow/scripts/pre-verify-pr.sh @@ -0,0 +1,136 @@ +#!/usr/bin/env bash +# pre-verify-pr.sh — Validate inputs and pre-fetch Jira + GitHub data. +# +# Runs on the fullsend runner BEFORE the sandbox is created, where the Jira +# and GitHub tokens live. The sandbox never sees a token — it reads only the +# JSON this script produces. +# +# 1. Validates required env vars and JIRA_ISSUE_ID format +# 2. Fetches the full Jira issue and extracts the linked PR URL +# 3. If no PR URL: emits an ADR-0072 skip signal and exits 0 (nothing to verify) +# 4. If a PR URL: prefetches the GitHub tier-1 read bundle (diff, stat, +# reviews, comments, commits, head ref + commit SHA) so the sandbox needs +# no api.github.com egress +# 5. Writes the tracker-agnostic verify-pr-input.json that host_files mounts +# into the sandbox +# +# Required env vars: +# JIRA_ISSUE_ID — Jira issue key (e.g., TC-4741) +# JIRA_SERVER_URL — Jira instance URL +# JIRA_EMAIL — Jira user email +# JIRA_API_TOKEN — Jira API token +# GH_TOKEN — GitHub token (only needed once a PR URL is resolved) +# +# Optional env vars: +# PRE_DIR — output directory (default: /tmp/fullsend-pre-output). +# The harness host_files src is the default path. +# FULLSEND_PRESCRIPT_OUTPUT — key=value skip-signal file created by fullsend run +# (pre-script output protocol v1). Guarded — older +# CLIs leave it unset. + +set -euo pipefail + +# 1. Validate required env vars are set +: "${JIRA_ISSUE_ID:?JIRA_ISSUE_ID is required}" +: "${JIRA_SERVER_URL:?JIRA_SERVER_URL is required}" +: "${JIRA_EMAIL:?JIRA_EMAIL is required}" +: "${JIRA_API_TOKEN:?JIRA_API_TOKEN is required}" + +# 2. Validate JIRA_ISSUE_ID format +# https://confluence.atlassian.com/adminjiraserver/changing-the-project-key-format-938847081.html +if [[ ! "${JIRA_ISSUE_ID}" =~ ^[A-Z][A-Z0-9_]+-[0-9]+$ ]]; then + echo "ERROR: JIRA_ISSUE_ID '${JIRA_ISSUE_ID}' does not match expected format (e.g., TC-4741)" + exit 1 +fi + +echo "Issue: ${JIRA_ISSUE_ID}" + +SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)" +PRE_OUTPUT_DIR="${PRE_DIR:-/tmp/fullsend-pre-output}" +mkdir -p "${PRE_OUTPUT_DIR}" + +# request_skip REASON — emit an ADR-0072 skip signal and exit cleanly. +# The pre-script output protocol (v1) is line-based key=value; the guard on +# FULLSEND_PRESCRIPT_OUTPUT matches the scaffold's GITHUB_OUTPUT guard, so a +# missing variable (older CLI) fails open to a normal run rather than erroring. +request_skip() { + local reason="$1" + echo "SKIP: ${reason}" + if [[ -n "${FULLSEND_PRESCRIPT_OUTPUT:-}" ]]; then + { + echo "skipped=true" + echo "reason=${reason}" + } >> "${FULLSEND_PRESCRIPT_OUTPUT}" + fi + exit 0 +} + +# 3. Fetch full issue details (validates existence + pre-fetches for sandbox) +ISSUE_JSON=$(python3 "${SCRIPT_DIR}/jira-client.py" get_issue "${JIRA_ISSUE_ID}" --fields "*all" 2>"/tmp/fullsend-pre-jira-stderr.txt") || { + JIRA_STDERR=$(cat /tmp/fullsend-pre-jira-stderr.txt 2>/dev/null || echo "") + if echo "${JIRA_STDERR}" | grep -qi "401\|unauthorized"; then + echo "ERROR: Jira authentication failed — check JIRA_EMAIL and JIRA_API_TOKEN" + elif echo "${JIRA_STDERR}" | grep -qi "403\|forbidden"; then + echo "ERROR: Jira permission denied — check that the API token has access to ${JIRA_ISSUE_ID}" + elif echo "${JIRA_STDERR}" | grep -qi "404\|not found"; then + echo "ERROR: Jira issue ${JIRA_ISSUE_ID} not found" + else + echo "ERROR: Failed to fetch Jira issue ${JIRA_ISSUE_ID}" + echo "${JIRA_STDERR}" + fi + rm -f /tmp/fullsend-pre-jira-stderr.txt + exit 1 +} +rm -f /tmp/fullsend-pre-jira-stderr.txt + +echo "Jira issue verified: ${JIRA_ISSUE_ID}" + +# 4. Extract PR URL from custom field (best-effort) +PR_URL=$(printf '%s\n' "${ISSUE_JSON}" | python3 "${SCRIPT_DIR}/pre_verify_pr.py" extract-pr-url 2>/dev/null || echo "") + +# 5. No PR linked — nothing to verify. Signal a skip (ADR-0072) and stop before +# the sandbox is created. +if [[ -z "${PR_URL}" ]]; then + request_skip "no PR URL on the Jira issue" +fi + +echo "PR linked: ${PR_URL}" + +# 6. Parse owner/repo/number from the PR URL. +if [[ ! "${PR_URL}" =~ ^https://github\.com/([^/]+/[^/]+)/pull/([0-9]+) ]]; then + echo "ERROR: PR URL '${PR_URL}' is not a github.com pull request URL" + exit 1 +fi +PR_REPO="${BASH_REMATCH[1]}" +PR_NUM="${BASH_REMATCH[2]}" +echo "PR: ${PR_REPO}#${PR_NUM}" + +# 7. GitHub tier-1 prefetch — runs on the trusted runner where GH_TOKEN lives. +# Every read the verify-pr skill performs against the PR is captured here so +# the sandbox needs no api.github.com egress. +: "${GH_TOKEN:?GH_TOKEN is required to prefetch PR ${PR_REPO}#${PR_NUM}}" + +HEAD_REF=$(gh pr view "${PR_NUM}" -R "${PR_REPO}" --json headRefName --jq .headRefName) +COMMIT_SHA=$(gh pr view "${PR_NUM}" -R "${PR_REPO}" --json commits --jq '.commits[-1].oid') + +gh pr diff "${PR_NUM}" -R "${PR_REPO}" > "${PRE_OUTPUT_DIR}/pr.diff" +gh pr diff "${PR_NUM}" -R "${PR_REPO}" --stat > "${PRE_OUTPUT_DIR}/pr.stat" +gh api "repos/${PR_REPO}/pulls/${PR_NUM}/reviews" > "${PRE_OUTPUT_DIR}/reviews.json" +gh api "repos/${PR_REPO}/pulls/${PR_NUM}/comments" > "${PRE_OUTPUT_DIR}/review-comments.json" +gh api "repos/${PR_REPO}/issues/${PR_NUM}/comments" > "${PRE_OUTPUT_DIR}/issue-comments.json" +gh pr view "${PR_NUM}" -R "${PR_REPO}" --json commits --jq .commits > "${PRE_OUTPUT_DIR}/commits.json" + +echo "GitHub read bundle prefetched to ${PRE_OUTPUT_DIR}" + +# 8. Write pre-fetched data for sandbox consumption (tracker-agnostic format, +# with the GitHub bundle embedded under `github`). +printf '%s\n' "${ISSUE_JSON}" | python3 "${SCRIPT_DIR}/pre_verify_pr.py" transform \ + "${JIRA_ISSUE_ID}" "${PR_URL}" \ + --github-dir "${PRE_OUTPUT_DIR}" \ + --pr-repo "${PR_REPO}" \ + --pr-number "${PR_NUM}" \ + --head-ref "${HEAD_REF}" \ + --commit-sha "${COMMIT_SHA}" > "${PRE_OUTPUT_DIR}/verify-pr-input.json" + +echo "Pre-fetched data written to ${PRE_OUTPUT_DIR}/verify-pr-input.json" +echo "Input validation passed" diff --git a/plugins/sdlc-workflow/scripts/pre_verify_pr.py b/plugins/sdlc-workflow/scripts/pre_verify_pr.py new file mode 100644 index 000000000..431ee3174 --- /dev/null +++ b/plugins/sdlc-workflow/scripts/pre_verify_pr.py @@ -0,0 +1,156 @@ +#!/usr/bin/env python3 +"""Pre-verify-pr data extraction functions. + +Extracts PR URL from Jira custom fields, assembles the GitHub tier-1 read +bundle prefetched on the runner, and transforms Jira issue JSON into the +tracker-agnostic input schema used by the sandbox. + +CLI usage (called by pre-verify-pr.sh): + echo "$ISSUE_JSON" | python3 pre_verify_pr.py extract-pr-url + echo "$ISSUE_JSON" | python3 pre_verify_pr.py transform TASK_ID PR_URL \\ + [--github-dir DIR --pr-repo REPO --pr-number N \\ + --head-ref REF --commit-sha SHA] + +When the --github-* options are supplied, `transform` reads the raw GitHub +reads from DIR (pr.diff, pr.stat, reviews.json, review-comments.json, +issue-comments.json, commits.json) and embeds them under a `github` key. +""" + +import argparse +import json +import sys + + +def extract_pr_url(issue): + """Extract PR URL from Jira custom field (ADF or string). + + Returns empty string if the field is missing or has no URL. + """ + field = issue.get("fields", {}).get("customfield_10875") + if not field: + return "" + if isinstance(field, str): + return field + if isinstance(field, dict): + for block in field.get("content", []): + for inline in block.get("content", []): + if inline.get("type") == "inlineCard": + return inline.get("attrs", {}).get("url", "") + return "" + + +def build_github_bundle(pr_repo, pr_number, head_ref, commit_sha, + diff, stat, reviews, review_comments, + issue_comments, commits): + """Assemble the GitHub tier-1 read bundle embedded in the input. + + diff/stat are raw text; the four *_comments/reviews/commits arguments + are already-parsed JSON (lists). Keys mirror the reads the verify-pr + skill performs so the sandbox needs no api.github.com egress. + """ + return { + "pr_repo": pr_repo, + "pr_number": int(pr_number), + "headRefName": head_ref, + "commit_sha": commit_sha, + "diff": diff, + "stat": stat, + "reviews": reviews, + "review_comments": review_comments, + "issue_comments": issue_comments, + "commits": commits, + } + + +def transform_to_input(issue, task_id, pr_url, github=None): + """Transform Jira issue JSON to tracker-agnostic input schema.""" + fields = issue.get("fields", {}) + result = { + "task_id": task_id, + "task": { + "summary": fields.get("summary", ""), + "description": fields.get("description", {}), + "status": (fields.get("status") or {}).get("name", ""), + "labels": fields.get("labels", []), + "issue_links": [ + { + "type": (link.get("type") or {}).get("name", ""), + "direction": "inward" if "inwardIssue" in link else "outward", + "key": ( + link.get("inwardIssue") or link.get("outwardIssue") or {} + ).get("key", ""), + } + for link in fields.get("issuelinks", []) + ], + "custom_fields": { + k: v + for k, v in fields.items() + if k.startswith("customfield_") + }, + }, + "pr_url": pr_url, + "source": { + "tracker": "jira", + "raw": issue, + }, + } + if github is not None: + result["github"] = github + return result + + +def _read_text(path): + with open(path) as f: + return f.read() + + +def _read_json(path): + with open(path) as f: + return json.load(f) + + +def _github_from_dir(args): + """Assemble the github bundle from raw read files written by the shell.""" + d = args.github_dir.rstrip("/") + return build_github_bundle( + pr_repo=args.pr_repo, + pr_number=args.pr_number, + head_ref=args.head_ref, + commit_sha=args.commit_sha, + diff=_read_text(f"{d}/pr.diff"), + stat=_read_text(f"{d}/pr.stat"), + reviews=_read_json(f"{d}/reviews.json"), + review_comments=_read_json(f"{d}/review-comments.json"), + issue_comments=_read_json(f"{d}/issue-comments.json"), + commits=_read_json(f"{d}/commits.json"), + ) + + +def main(argv): + parser = argparse.ArgumentParser(prog="pre_verify_pr.py") + sub = parser.add_subparsers(dest="command", required=True) + + sub.add_parser("extract-pr-url") + + t = sub.add_parser("transform") + t.add_argument("task_id") + t.add_argument("pr_url") + t.add_argument("--github-dir") + t.add_argument("--pr-repo") + t.add_argument("--pr-number", type=int) + t.add_argument("--head-ref") + t.add_argument("--commit-sha") + + args = parser.parse_args(argv) + issue = json.load(sys.stdin) + + if args.command == "extract-pr-url": + print(extract_pr_url(issue)) + elif args.command == "transform": + github = _github_from_dir(args) if args.github_dir else None + result = transform_to_input(issue, args.task_id, args.pr_url, github) + json.dump(result, sys.stdout, indent=2) + + +if __name__ == "__main__": + main(sys.argv[1:]) diff --git a/plugins/sdlc-workflow/scripts/strip_extra_properties.py b/plugins/sdlc-workflow/scripts/strip_extra_properties.py new file mode 100644 index 000000000..1a361a416 --- /dev/null +++ b/plugins/sdlc-workflow/scripts/strip_extra_properties.py @@ -0,0 +1,101 @@ +#!/usr/bin/env python3 +"""Strip additional properties from JSON based on a JSON Schema. + +Recursively walks the schema tree and removes properties not declared +in `properties` or `allOf/if/then/properties` at every node where +`additionalProperties: false`. Works with any schema structure +including discriminated unions (allOf with if/then). + +CLI usage (called by validate-output-schema.sh): + python3 strip_extra_properties.py + +Strips the JSON file in-place and exits 0. The caller validates after. +""" + +import json +import sys + + +def _resolve_ref(ref, root): + node = root + for part in ref.lstrip("#/").split("/"): + node = node[part] + return node + + +def _deref(schema, root): + if "$ref" in schema: + return _resolve_ref(schema["$ref"], root) + return schema + + +def _matching_then(instance, branches): + """Find the allOf branch whose `if` matches the instance.""" + for branch in branches: + if_clause = branch.get("if", {}) + if_props = if_clause.get("properties", {}) + match = all( + instance.get(k) == v.get("const") + for k, v in if_props.items() + if "const" in v + ) + if match and "then" in branch: + return branch["then"] + return None + + +def strip(instance, schema, root): + """Recursively strip properties not allowed by the schema.""" + schema = _deref(schema, root) + + if not isinstance(instance, dict) or schema.get("type") not in ("object", None): + return instance + + then = _matching_then(instance, schema.get("allOf", [])) + has_strict = schema.get("additionalProperties") is False + then_strict = then is not None and then.get("additionalProperties") is False + + if has_strict or then_strict: + allowed = set(schema.get("properties", {}).keys()) + if then: + allowed |= set(then.get("properties", {}).keys()) + removed = [k for k in instance if k not in allowed] + if removed: + print(f" stripped: {removed}") + instance = {k: v for k, v in instance.items() if k in allowed} + + for key, prop_schema in schema.get("properties", {}).items(): + if key not in instance: + continue + prop_schema = _deref(prop_schema, root) + if isinstance(instance[key], dict): + instance[key] = strip(instance[key], prop_schema, root) + elif isinstance(instance[key], list): + items_schema = prop_schema.get("items", {}) + items_schema = _deref(items_schema, root) + instance[key] = [ + strip(item, items_schema, root) if isinstance(item, dict) else item + for item in instance[key] + ] + + return instance + + +def main(): + if len(sys.argv) < 3: + print("Usage: strip_extra_properties.py ", file=sys.stderr) + sys.exit(1) + + with open(sys.argv[1]) as f: + instance = json.load(f) + with open(sys.argv[2]) as f: + schema = json.load(f) + + instance = strip(instance, schema, schema) + + with open(sys.argv[1], "w") as f: + json.dump(instance, f, indent=2) + + +if __name__ == "__main__": + main() diff --git a/plugins/sdlc-workflow/scripts/test_pre_verify_pr.py b/plugins/sdlc-workflow/scripts/test_pre_verify_pr.py new file mode 100644 index 000000000..14f877bfe --- /dev/null +++ b/plugins/sdlc-workflow/scripts/test_pre_verify_pr.py @@ -0,0 +1,240 @@ +#!/usr/bin/env python3 +"""Tests for pre_verify_pr.py — PR URL extraction, GitHub bundle, transform.""" + +import json +import os +import subprocess +import sys +import tempfile + +script_dir = os.path.dirname(os.path.abspath(__file__)) +sys.path.insert(0, script_dir) +import pre_verify_pr + + +# --- extract_pr_url --- + +def test_extract_pr_url_adf_inline_card(): + issue = {"fields": {"customfield_10875": { + "type": "doc", "version": 1, + "content": [{"type": "paragraph", "content": [ + {"type": "inlineCard", "attrs": {"url": "https://github.com/org/repo/pull/42"}} + ]}] + }}} + result = pre_verify_pr.extract_pr_url(issue) + assert result == "https://github.com/org/repo/pull/42", f"Got: {result}" + + +def test_extract_pr_url_plain_string(): + issue = {"fields": {"customfield_10875": "https://github.com/org/repo/pull/7"}} + result = pre_verify_pr.extract_pr_url(issue) + assert result == "https://github.com/org/repo/pull/7", f"Got: {result}" + + +def test_extract_pr_url_missing_field(): + issue = {"fields": {}} + result = pre_verify_pr.extract_pr_url(issue) + assert result == "", f"Expected empty string, got: {result}" + + +def test_extract_pr_url_null_field(): + issue = {"fields": {"customfield_10875": None}} + result = pre_verify_pr.extract_pr_url(issue) + assert result == "", f"Expected empty string, got: {result}" + + +def test_extract_pr_url_adf_no_inline_card(): + issue = {"fields": {"customfield_10875": { + "type": "doc", "version": 1, + "content": [{"type": "paragraph", "content": [ + {"type": "text", "text": "no link here"} + ]}] + }}} + result = pre_verify_pr.extract_pr_url(issue) + assert result == "", f"Expected empty string, got: {result}" + + +# --- build_github_bundle --- + +def test_build_github_bundle(): + bundle = pre_verify_pr.build_github_bundle( + pr_repo="org/repo", pr_number="42", head_ref="feat/x", + commit_sha="abc1234", diff="diff --git a b", stat=" 1 file changed", + reviews=[{"id": 1}], review_comments=[{"id": 2}], + issue_comments=[{"id": 3}], commits=[{"oid": "abc1234"}], + ) + assert bundle["pr_repo"] == "org/repo" + assert bundle["pr_number"] == 42 # coerced to int + assert bundle["headRefName"] == "feat/x" + assert bundle["commit_sha"] == "abc1234" + assert bundle["diff"] == "diff --git a b" + assert bundle["stat"] == " 1 file changed" + assert bundle["reviews"] == [{"id": 1}] + assert bundle["review_comments"] == [{"id": 2}] + assert bundle["issue_comments"] == [{"id": 3}] + assert bundle["commits"] == [{"oid": "abc1234"}] + + +# --- transform_to_input --- + +def test_transform_basic(): + issue = {"fields": { + "summary": "Add feature X", + "description": {"type": "doc", "content": []}, + "status": {"name": "In Progress"}, + "labels": ["backend", "api"], + "issuelinks": [], + }} + result = pre_verify_pr.transform_to_input(issue, "TC-100", "https://github.com/o/r/pull/1") + assert result["task_id"] == "TC-100" + assert result["task"]["summary"] == "Add feature X" + assert result["task"]["status"] == "In Progress" + assert result["task"]["labels"] == ["backend", "api"] + assert result["task"]["issue_links"] == [] + assert result["pr_url"] == "https://github.com/o/r/pull/1" + assert result["source"]["tracker"] == "jira" + assert result["source"]["raw"] is issue + + +def test_transform_without_github_omits_key(): + issue = {"fields": {"summary": "S", "status": {"name": "Open"}, "labels": [], "issuelinks": []}} + result = pre_verify_pr.transform_to_input(issue, "TC-1", "") + assert "github" not in result + + +def test_transform_with_github(): + issue = {"fields": {"summary": "S", "status": {"name": "Open"}, "labels": [], "issuelinks": []}} + github = pre_verify_pr.build_github_bundle( + "o/r", 5, "b", "deadbee", "d", "s", [], [], [], [], + ) + result = pre_verify_pr.transform_to_input(issue, "TC-1", "https://github.com/o/r/pull/5", github) + assert result["github"]["pr_repo"] == "o/r" + assert result["github"]["pr_number"] == 5 + assert result["github"]["commit_sha"] == "deadbee" + + +def test_transform_issue_links(): + issue = {"fields": { + "summary": "S", "description": {}, "status": {"name": "Open"}, + "labels": [], "issuelinks": [ + {"type": {"name": "Blocks"}, "outwardIssue": {"key": "TC-200"}}, + {"type": {"name": "Related"}, "inwardIssue": {"key": "TC-300"}}, + ], + }} + links = pre_verify_pr.transform_to_input(issue, "TC-100", "")["task"]["issue_links"] + assert len(links) == 2 + assert links[0] == {"type": "Blocks", "direction": "outward", "key": "TC-200"} + assert links[1] == {"type": "Related", "direction": "inward", "key": "TC-300"} + + +def test_transform_custom_fields(): + issue = {"fields": { + "summary": "S", "description": {}, "status": None, "labels": [], + "issuelinks": [], + "customfield_10875": "https://github.com/o/r/pull/5", + "customfield_99999": {"value": "something"}, + "priority": {"name": "High"}, + }} + cf = pre_verify_pr.transform_to_input(issue, "TC-1", "")["task"]["custom_fields"] + assert "customfield_10875" in cf + assert "customfield_99999" in cf + assert "priority" not in cf + + +def test_transform_empty_fields(): + issue = {"fields": {}} + result = pre_verify_pr.transform_to_input(issue, "TC-1", "") + assert result["task"]["summary"] == "" + assert result["task"]["status"] == "" + assert result["task"]["labels"] == [] + assert result["task"]["issue_links"] == [] + + +def test_transform_null_status(): + issue = {"fields": {"summary": "S", "status": None, "labels": [], "issuelinks": []}} + result = pre_verify_pr.transform_to_input(issue, "TC-1", "") + assert result["task"]["status"] == "" + + +def test_transform_large_payload(): + """Regression test: large payloads must work via stdin, not argv.""" + issue = {"fields": { + "summary": "Large issue", + "description": "x" * 500_000, + "status": {"name": "Open"}, + "labels": [], + "issuelinks": [], + }} + payload = json.dumps(issue) + assert len(payload) > 500_000 + + result = subprocess.run( + [sys.executable, os.path.join(script_dir, "pre_verify_pr.py"), + "transform", "TC-BIG", "https://example.com/pr/1"], + input=payload, capture_output=True, text=True, + ) + assert result.returncode == 0, f"Exit {result.returncode}: {result.stderr}" + output = json.loads(result.stdout) + assert output["task_id"] == "TC-BIG" + assert output["task"]["summary"] == "Large issue" + assert "github" not in output + + +def test_cli_transform_github_dir(): + """CLI transform reads the raw GitHub files and embeds the bundle.""" + issue = {"fields": {"summary": "S", "status": {"name": "Open"}, "labels": [], "issuelinks": []}} + with tempfile.TemporaryDirectory() as d: + with open(os.path.join(d, "pr.diff"), "w") as f: + f.write("diff --git a b\n") + with open(os.path.join(d, "pr.stat"), "w") as f: + f.write(" 1 file changed\n") + for name, payload in [ + ("reviews.json", [{"id": 1, "state": "APPROVED"}]), + ("review-comments.json", [{"id": 2}]), + ("issue-comments.json", [{"id": 3}]), + ("commits.json", [{"oid": "abc1234def"}]), + ]: + with open(os.path.join(d, name), "w") as f: + json.dump(payload, f) + + result = subprocess.run( + [sys.executable, os.path.join(script_dir, "pre_verify_pr.py"), + "transform", "TC-9", "https://github.com/o/r/pull/9", + "--github-dir", d, "--pr-repo", "o/r", "--pr-number", "9", + "--head-ref", "feat/x", "--commit-sha", "abc1234def"], + input=json.dumps(issue), capture_output=True, text=True, + ) + assert result.returncode == 0, f"Exit {result.returncode}: {result.stderr}" + output = json.loads(result.stdout) + gh = output["github"] + assert gh["pr_repo"] == "o/r" + assert gh["pr_number"] == 9 + assert gh["headRefName"] == "feat/x" + assert gh["commit_sha"] == "abc1234def" + assert gh["diff"] == "diff --git a b\n" + assert gh["stat"] == " 1 file changed\n" + assert gh["reviews"] == [{"id": 1, "state": "APPROVED"}] + assert gh["review_comments"] == [{"id": 2}] + assert gh["issue_comments"] == [{"id": 3}] + assert gh["commits"] == [{"oid": "abc1234def"}] + + +# --- runner --- + +if __name__ == "__main__": + tests = [v for k, v in sorted(globals().items()) if k.startswith("test_")] + failed = [] + for t in tests: + try: + t() + print(f" ✓ {t.__name__}") + except AssertionError as e: + print(f" ✗ {t.__name__}: {e}") + failed.append(t.__name__) + print(f"{'=' * 60}") + if failed: + print(f"FAILED: {len(failed)}/{len(tests)} test(s) failed") + sys.exit(1) + else: + print(f"SUCCESS: All {len(tests)} tests passed") + sys.exit(0) diff --git a/plugins/sdlc-workflow/scripts/validate-output-schema.sh b/plugins/sdlc-workflow/scripts/validate-output-schema.sh new file mode 100755 index 000000000..edef3c972 --- /dev/null +++ b/plugins/sdlc-workflow/scripts/validate-output-schema.sh @@ -0,0 +1,85 @@ +#!/usr/bin/env bash +# validate-output-schema.sh — Validate agent output against a JSON Schema. +# +# Generic script used by the harness validation_loop (ADR 0022). +# Works for any agent — the schema path is configured in the harness. +# +# Required env vars: +# FULLSEND_OUTPUT_SCHEMA — path to the JSON Schema file +# +# Optional env vars: +# FULLSEND_OUTPUT_FILE — filename to validate (default: agent-result.json) +# +# The script looks for the output file in the iteration output directory. +# The working directory is the iteration dir (set by run.go). + +set -euo pipefail + +: "${FULLSEND_OUTPUT_SCHEMA:?FULLSEND_OUTPUT_SCHEMA must be set}" + +SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)" + +# Find the output JSON file in this iteration's output directory. +OUTPUT_DIR="output" +if [[ ! -d "${OUTPUT_DIR}" ]]; then + echo "FAIL: output directory not found" + exit 1 +fi + +_output_file="${FULLSEND_OUTPUT_FILE:-agent-result.json}" +_output_file="$(basename "${_output_file}")" +RESULT_FILE="${OUTPUT_DIR}/${_output_file}" +if [[ ! -f "${RESULT_FILE}" ]]; then + # Agents sometimes write "result.json" instead of "agent-result.json". + # Accept the common variant rather than burning a full retry iteration. + _fallback="${OUTPUT_DIR}/result.json" + if [[ "${_output_file}" == "agent-result.json" && -f "${_fallback}" ]]; then + echo "WARN: expected ${RESULT_FILE} but found ${_fallback} — using fallback" + RESULT_FILE="${_fallback}" + else + echo "FAIL: ${RESULT_FILE} not found" + exit 1 + fi +fi +echo "Validating: ${RESULT_FILE} against ${FULLSEND_OUTPUT_SCHEMA}" + +# Validate JSON is parseable. +if ! python3 -m json.tool "${RESULT_FILE}" > /dev/null 2>&1; then + echo "FAIL: ${RESULT_FILE} is not valid JSON" + exit 1 +fi + +# Validate against schema using Python's jsonschema. +# jsonschema is required — fail hard if not installed. +if ! python3 -c "import jsonschema" 2>/dev/null; then + echo "FAIL: python3 jsonschema package is not installed (required by ADR 0022)" + exit 1 +fi + +# Strip extra properties before validation. The schema stays strict +# (additionalProperties: false) to document the contract, but the +# stripping makes it forgiving for benign metadata the agent adds. +python3 "${SCRIPT_DIR}/strip_extra_properties.py" "${RESULT_FILE}" "${FULLSEND_OUTPUT_SCHEMA}" + +if ! python3 -c " +import json, sys +from jsonschema import validate, ValidationError + +with open(sys.argv[1]) as f: + instance = json.load(f) +with open(sys.argv[2]) as f: + schema = json.load(f) +try: + validate(instance=instance, schema=schema) + print('PASS: output validated against schema') +except ValidationError as e: + print(f'FAIL: schema validation error: {e.message}') + if e.path: + print(f' at: {\".\".join(str(p) for p in e.path)}') + if 'properties' in e.schema: + allowed = ', '.join(sorted(e.schema['properties'].keys())) + print(f' allowed properties: {allowed}') + sys.exit(1) +" "${RESULT_FILE}" "${FULLSEND_OUTPUT_SCHEMA}"; then + exit 1 +fi From 859cbaa7be3fb2a224c54d175a48f8246781702a Mon Sep 17 00:00:00 2001 From: mrizzi Date: Tue, 1 Sep 2026 09:38:44 +0200 Subject: [PATCH 007/175] fix(verify-pr): derive PR diffstat with git apply --stat gh pr diff has no --stat flag, so the prefetch aborted under set -euo pipefail before writing verify-pr-input.json. Derive the per-file stat from the already-fetched patch with git apply --stat, guarding the empty-diff case so the script still exits 0. Add regression tests exercising the real stat command and guarding against reintroducing the unsupported gh flag. Implements TC-5884 Co-Authored-By: Claude Opus 4.8 Assisted-by: Claude Code --- .../sdlc-workflow/scripts/pre-verify-pr.sh | 9 ++- .../scripts/test_pre_verify_pr.py | 58 +++++++++++++++++++ 2 files changed, 66 insertions(+), 1 deletion(-) diff --git a/plugins/sdlc-workflow/scripts/pre-verify-pr.sh b/plugins/sdlc-workflow/scripts/pre-verify-pr.sh index 0842c28a1..c058a914d 100755 --- a/plugins/sdlc-workflow/scripts/pre-verify-pr.sh +++ b/plugins/sdlc-workflow/scripts/pre-verify-pr.sh @@ -114,7 +114,14 @@ HEAD_REF=$(gh pr view "${PR_NUM}" -R "${PR_REPO}" --json headRefName --jq .headR COMMIT_SHA=$(gh pr view "${PR_NUM}" -R "${PR_REPO}" --json commits --jq '.commits[-1].oid') gh pr diff "${PR_NUM}" -R "${PR_REPO}" > "${PRE_OUTPUT_DIR}/pr.diff" -gh pr diff "${PR_NUM}" -R "${PR_REPO}" --stat > "${PRE_OUTPUT_DIR}/pr.stat" +# `gh pr diff` has no --stat flag; derive the per-file diffstat from the patch we +# just fetched using a supported git command. Guard the empty-diff case: `git +# apply --stat` errors on an empty patch, which would abort under set -euo pipefail. +if [[ -s "${PRE_OUTPUT_DIR}/pr.diff" ]]; then + git apply --stat "${PRE_OUTPUT_DIR}/pr.diff" > "${PRE_OUTPUT_DIR}/pr.stat" +else + : > "${PRE_OUTPUT_DIR}/pr.stat" +fi gh api "repos/${PR_REPO}/pulls/${PR_NUM}/reviews" > "${PRE_OUTPUT_DIR}/reviews.json" gh api "repos/${PR_REPO}/pulls/${PR_NUM}/comments" > "${PRE_OUTPUT_DIR}/review-comments.json" gh api "repos/${PR_REPO}/issues/${PR_NUM}/comments" > "${PRE_OUTPUT_DIR}/issue-comments.json" diff --git a/plugins/sdlc-workflow/scripts/test_pre_verify_pr.py b/plugins/sdlc-workflow/scripts/test_pre_verify_pr.py index 14f877bfe..5402045d9 100644 --- a/plugins/sdlc-workflow/scripts/test_pre_verify_pr.py +++ b/plugins/sdlc-workflow/scripts/test_pre_verify_pr.py @@ -219,6 +219,64 @@ def test_cli_transform_github_dir(): assert gh["commits"] == [{"oid": "abc1234def"}] +# --- stat production (pre-verify-pr.sh) --- + +pre_verify_sh = os.path.join(script_dir, "pre-verify-pr.sh") + + +def test_stat_produced_by_git_apply_stat(): + """The stat mechanism (git apply --stat) yields a diffstat matching the + downstream github.stat contract: a per-file line plus a summary line. + + Exercises the real command pre-verify-pr.sh runs, not a gh stub that + silently accepts the unsupported --stat flag. + """ + # Given a unified diff like the one gh pr diff writes to pr.diff + patch = ( + "diff --git a/foo.txt b/foo.txt\n" + "index 1111111..2222222 100644\n" + "--- a/foo.txt\n" + "+++ b/foo.txt\n" + "@@ -1,3 +1,3 @@\n" + " line1\n" + "-line2\n" + "+CHANGED\n" + " line3\n" + ) + with tempfile.TemporaryDirectory() as d: + diff_path = os.path.join(d, "pr.diff") + with open(diff_path, "w") as f: + f.write(patch) + + # When producing the stat with the exact command pre-verify-pr.sh uses + result = subprocess.run( + ["git", "apply", "--stat", diff_path], + capture_output=True, text=True, + ) + + # Then it succeeds and emits a git diffstat downstream can consume + assert result.returncode == 0, f"Exit {result.returncode}: {result.stderr}" + assert "foo.txt" in result.stdout, f"Got: {result.stdout!r}" + assert "1 file changed" in result.stdout, f"Got: {result.stdout!r}" + + +def test_pre_verify_sh_uses_supported_stat_command(): + """Regression guard: the prefetch derives the stat with git apply --stat and + never passes the unsupported --stat flag to gh pr diff (Sourcery id 3896899424). + """ + # Given the current pre-verify-pr.sh source + with open(pre_verify_sh) as f: + script = f.read() + + # Then no gh pr diff invocation uses --stat, and git apply --stat is present + for line in script.splitlines(): + if line.lstrip().startswith("#"): + continue # skip comments (which may mention the removed flag) + if "gh pr diff" in line: + assert "--stat" not in line, f"unsupported gh flag reintroduced: {line!r}" + assert "git apply --stat" in script, "expected git apply --stat stat mechanism" + + # --- runner --- if __name__ == "__main__": From fd05082a154af434a624d1090dfa6b3094b330cf Mon Sep 17 00:00:00 2001 From: mrizzi Date: Tue, 1 Sep 2026 09:53:46 +0200 Subject: [PATCH 008/175] fix(verify-pr): paginate gh api review/comment fetches in prefetch MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The GitHub tier-1 prefetch fetched PR reviews, review comments, and issue comments without --paginate, so GitHub REST returned only the first ~30 items and the bundle silently truncated on active PRs — degrading the very review-comment analysis it feeds the verify-pr sub-agents. Add --paginate --slurp and merge pages with `jq 'add'` into the flat arrays that build_github_bundle expects. (--slurp is incompatible with gh's built-in --jq, so a standalone jq performs the merge.) Also make the sibling git-apply-stat test deterministic by running it from a non-repo cwd: git apply --stat is CWD-sensitive and reports "0 files changed" when launched from inside a repo subdirectory. Implements TC-5885 Assisted-by: Claude Code Co-Authored-By: Claude Opus 4.8 --- .../sdlc-workflow/scripts/pre-verify-pr.sh | 12 ++- .../scripts/test_pre_verify_pr.py | 75 +++++++++++++++++-- 2 files changed, 79 insertions(+), 8 deletions(-) diff --git a/plugins/sdlc-workflow/scripts/pre-verify-pr.sh b/plugins/sdlc-workflow/scripts/pre-verify-pr.sh index c058a914d..27425a38d 100755 --- a/plugins/sdlc-workflow/scripts/pre-verify-pr.sh +++ b/plugins/sdlc-workflow/scripts/pre-verify-pr.sh @@ -122,9 +122,15 @@ if [[ -s "${PRE_OUTPUT_DIR}/pr.diff" ]]; then else : > "${PRE_OUTPUT_DIR}/pr.stat" fi -gh api "repos/${PR_REPO}/pulls/${PR_NUM}/reviews" > "${PRE_OUTPUT_DIR}/reviews.json" -gh api "repos/${PR_REPO}/pulls/${PR_NUM}/comments" > "${PRE_OUTPUT_DIR}/review-comments.json" -gh api "repos/${PR_REPO}/issues/${PR_NUM}/comments" > "${PRE_OUTPUT_DIR}/issue-comments.json" +# GitHub REST returns ~30 items per page; without --paginate the reviews and +# comments are silently truncated on any active PR. --slurp aggregates the +# per-page arrays into an array-of-pages, which `jq 'add'` concatenates back +# into the single flat array that pre_verify_pr.py's build_github_bundle +# expects. (--slurp cannot be combined with gh's built-in --jq, so the merge +# uses a standalone jq.) pipefail makes a failed gh or jq abort the script. +gh api --paginate --slurp "repos/${PR_REPO}/pulls/${PR_NUM}/reviews" | jq 'add' > "${PRE_OUTPUT_DIR}/reviews.json" +gh api --paginate --slurp "repos/${PR_REPO}/pulls/${PR_NUM}/comments" | jq 'add' > "${PRE_OUTPUT_DIR}/review-comments.json" +gh api --paginate --slurp "repos/${PR_REPO}/issues/${PR_NUM}/comments" | jq 'add' > "${PRE_OUTPUT_DIR}/issue-comments.json" gh pr view "${PR_NUM}" -R "${PR_REPO}" --json commits --jq .commits > "${PRE_OUTPUT_DIR}/commits.json" echo "GitHub read bundle prefetched to ${PRE_OUTPUT_DIR}" diff --git a/plugins/sdlc-workflow/scripts/test_pre_verify_pr.py b/plugins/sdlc-workflow/scripts/test_pre_verify_pr.py index 5402045d9..54fbc2374 100644 --- a/plugins/sdlc-workflow/scripts/test_pre_verify_pr.py +++ b/plugins/sdlc-workflow/scripts/test_pre_verify_pr.py @@ -244,14 +244,17 @@ def test_stat_produced_by_git_apply_stat(): " line3\n" ) with tempfile.TemporaryDirectory() as d: - diff_path = os.path.join(d, "pr.diff") - with open(diff_path, "w") as f: + with open(os.path.join(d, "pr.diff"), "w") as f: f.write(patch) - # When producing the stat with the exact command pre-verify-pr.sh uses + # When producing the stat with the exact command pre-verify-pr.sh uses. + # Run it from the (non-repo) temp dir: `git apply --stat` is CWD-sensitive + # — inside a repo subdirectory it scopes the patch to that subtree and + # reports "0 files changed", so cwd=d keeps this a pure textual diffstat + # independent of where the test runner is launched. result = subprocess.run( - ["git", "apply", "--stat", diff_path], - capture_output=True, text=True, + ["git", "apply", "--stat", "pr.diff"], + cwd=d, capture_output=True, text=True, ) # Then it succeeds and emits a git diffstat downstream can consume @@ -277,6 +280,68 @@ def test_pre_verify_sh_uses_supported_stat_command(): assert "git apply --stat" in script, "expected git apply --stat stat mechanism" +# --- paginated fetch aggregation (pre-verify-pr.sh) --- + +def test_paginated_pages_aggregate_into_flat_array(): + """Multi-page fetches merge into one complete array, not truncated at page 1. + + Exercises the exact merge pre-verify-pr.sh runs on the `gh api --paginate + --slurp` output — a per-page array-of-arrays piped through `jq 'add'` — and + asserts every page's items survive in order with their object shape intact. + """ + # Given the array-of-pages that `gh api --paginate --slurp` emits: three + # pages, so page-2 and page-3 items only appear if pagination is honored. + slurped_pages = [ + [{"id": 1}, {"id": 2}], + [{"id": 3}, {"id": 4}], + [{"id": 5}], + ] + + # When merged with the same standalone `jq 'add'` the script pipes through + result = subprocess.run( + ["jq", "add"], + input=json.dumps(slurped_pages), capture_output=True, text=True, + ) + + # Then the pages flatten into one array carrying items beyond the first page + assert result.returncode == 0, f"Exit {result.returncode}: {result.stderr}" + merged = json.loads(result.stdout) + assert merged == [{"id": 1}, {"id": 2}, {"id": 3}, {"id": 4}, {"id": 5}], \ + f"expected flat concatenation, got: {merged!r}" + assert [o["id"] for o in merged] == [1, 2, 3, 4, 5] # order preserved + assert {"id": 5} in merged # last page (beyond first) not truncated + + +def test_pre_verify_sh_paginates_review_comment_fetches(): + """Regression guard: all three gh api review/comment fetches request every + page and merge with `jq 'add'` (Sourcery id 3896899429), keeping the stored + value a flat array for pre_verify_pr.py. + """ + # Given the current pre-verify-pr.sh source + with open(pre_verify_sh) as f: + script = f.read() + + # Then each of the three paginated endpoints is fetched with --paginate + # --slurp and merged via jq add on a non-comment line. + endpoints = [ + "/pulls/${PR_NUM}/reviews", + "/pulls/${PR_NUM}/comments", + "/issues/${PR_NUM}/comments", + ] + for endpoint in endpoints: + matches = [ + line for line in script.splitlines() + if endpoint in line and not line.lstrip().startswith("#") + ] + assert matches, f"no fetch line found for {endpoint}" + for line in matches: + if "gh api" not in line: + continue + assert "--paginate" in line, f"missing --paginate: {line!r}" + assert "--slurp" in line, f"missing --slurp: {line!r}" + assert "jq 'add'" in line, f"missing jq 'add' merge: {line!r}" + + # --- runner --- if __name__ == "__main__": From d668e20cc0cd58e644fdc4c5722eb70b0712c56a Mon Sep 17 00:00:00 2001 From: mrizzi Date: Tue, 1 Sep 2026 10:01:03 +0200 Subject: [PATCH 009/175] fix(verify-pr): coerce null Jira description to {} in prefetch transform MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit transform_to_input set task.description via fields.get("description", {}), whose default applies only to an ABSENT key. When Jira returns an explicit null (an issue with no description), .get returns None, so task.description became null — violating verify-pr-input.schema.json (which requires an object) and producing a schema-invalid verify-pr-input.json. Use `fields.get("description") or {}` to also coerce null to {}, matching the existing `(fields.get("status") or {})` idiom in the same transform. Add regression tests: a direct coercion assertion and a full-input (task + github bundle) validation against verify-pr-input.schema.json that fails on the pre-fix null description. Implements TC-5886 Assisted-by: Claude Code Co-Authored-By: Claude Opus 4.8 --- .../sdlc-workflow/scripts/pre_verify_pr.py | 5 +- .../scripts/test_pre_verify_pr.py | 51 +++++++++++++++++++ 2 files changed, 55 insertions(+), 1 deletion(-) diff --git a/plugins/sdlc-workflow/scripts/pre_verify_pr.py b/plugins/sdlc-workflow/scripts/pre_verify_pr.py index 431ee3174..0174a2ba6 100644 --- a/plugins/sdlc-workflow/scripts/pre_verify_pr.py +++ b/plugins/sdlc-workflow/scripts/pre_verify_pr.py @@ -69,7 +69,10 @@ def transform_to_input(issue, task_id, pr_url, github=None): "task_id": task_id, "task": { "summary": fields.get("summary", ""), - "description": fields.get("description", {}), + # Jira may return an explicit null description; `.get(key, {})` only + # defaults on an ABSENT key, so `or {}` also coerces null → {} to + # keep task.description an object per verify-pr-input.schema.json. + "description": fields.get("description") or {}, "status": (fields.get("status") or {}).get("name", ""), "labels": fields.get("labels", []), "issue_links": [ diff --git a/plugins/sdlc-workflow/scripts/test_pre_verify_pr.py b/plugins/sdlc-workflow/scripts/test_pre_verify_pr.py index 54fbc2374..6a34d64c6 100644 --- a/plugins/sdlc-workflow/scripts/test_pre_verify_pr.py +++ b/plugins/sdlc-workflow/scripts/test_pre_verify_pr.py @@ -156,6 +156,57 @@ def test_transform_null_status(): assert result["task"]["status"] == "" +def test_transform_null_description_coerced_to_object(): + """An explicit null description becomes {} so task.description stays an object. + + Regression for Sourcery id 3896899434: `fields.get("description", {})` only + defaults on an absent key, so an explicit JSON null (an issue with no + description) yielded task.description = null, violating the input schema. + """ + # Given a Jira issue whose description field is an explicit null + issue = {"fields": {"summary": "S", "description": None, "status": {"name": "Open"}, + "labels": [], "issuelinks": []}} + + # When transforming it to the tracker-agnostic input + result = pre_verify_pr.transform_to_input(issue, "TC-1", "") + + # Then the null is coerced to an empty object, not left as null + assert result["task"]["description"] == {}, \ + f"expected {{}}, got: {result['task']['description']!r}" + assert isinstance(result["task"]["description"], dict) + + +def test_null_description_input_validates_against_schema(): + """A produced input with a null-source description validates against the schema. + + Drives the full transform (task + github bundle) for an issue with a null + description and asserts the result satisfies verify-pr-input.schema.json — + the acceptance criterion for TC-5886. Without the null coercion the instance + would carry task.description = null and fail (description must be an object). + """ + from jsonschema import validate + + # Given an issue with a null description and the prefetched github bundle a + # real run embeds (the schema requires `github`, so a bare task won't do) + issue = {"fields": {"summary": "S", "description": None, "status": {"name": "Open"}, + "labels": [], "issuelinks": []}} + github = pre_verify_pr.build_github_bundle( + "o/r", 5, "feat/x", "deadbee", "diff", "stat", [], [], [], [], + ) + + # When producing the input and loading the input schema + result = pre_verify_pr.transform_to_input( + issue, "TC-1", "https://github.com/o/r/pull/5", github) + schema_path = os.path.join( + script_dir, "..", "schemas", "verify-pr-input.schema.json") + with open(schema_path) as f: + schema = json.load(f) + + # Then it validates cleanly (validate raises ValidationError on failure) + assert result["task"]["description"] == {} + validate(instance=result, schema=schema) + + def test_transform_large_payload(): """Regression test: large payloads must work via stdin, not argv.""" issue = {"fields": { From 8838ba031c1c592a69f915f362c1c3de0554d4ff Mon Sep 17 00:00:00 2001 From: mrizzi Date: Tue, 1 Sep 2026 10:56:53 +0200 Subject: [PATCH 010/175] fix(verify-pr): end-anchor pre-verify-pr.sh PR-URL regex MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The github.com PR-URL match at pre-verify-pr.sh:100 was start- but not end-anchored, so a malformed value like `.../pull/42abc` or `.../pull/42/invalid` matched and BASH_REMATCH truncated the pull number to 42 — making the pre-script prefetch and embed a different PR than the Jira custom field identifies. Append `/?$` so the full URL must match: a trailing non-numeric character or extra path segment is now rejected with the existing "not a github.com pull request URL" error, while a canonical URL with or without a trailing slash still parses to owner/repo and number. Add regression tests that run the script's actual regex (extracted from source) through bash's [[ =~ ]], asserting well-formed URLs parse to the correct owner/repo and number and malformed ones are rejected rather than truncated. Implements TC-5891 Assisted-by: Claude Code Co-Authored-By: Claude Opus 4.8 --- .../sdlc-workflow/scripts/pre-verify-pr.sh | 7 +- .../scripts/test_pre_verify_pr.py | 99 +++++++++++++++++++ 2 files changed, 104 insertions(+), 2 deletions(-) diff --git a/plugins/sdlc-workflow/scripts/pre-verify-pr.sh b/plugins/sdlc-workflow/scripts/pre-verify-pr.sh index 27425a38d..fe1f3369b 100755 --- a/plugins/sdlc-workflow/scripts/pre-verify-pr.sh +++ b/plugins/sdlc-workflow/scripts/pre-verify-pr.sh @@ -96,8 +96,11 @@ fi echo "PR linked: ${PR_URL}" -# 6. Parse owner/repo/number from the PR URL. -if [[ ! "${PR_URL}" =~ ^https://github\.com/([^/]+/[^/]+)/pull/([0-9]+) ]]; then +# 6. Parse owner/repo/number from the PR URL. End-anchor the pattern (allowing +# only an optional trailing slash) so a malformed value like `.../pull/42abc` +# or `.../pull/42/extra` is rejected outright instead of silently truncating +# the pull number to 42 and prefetching the wrong PR. +if [[ ! "${PR_URL}" =~ ^https://github\.com/([^/]+/[^/]+)/pull/([0-9]+)/?$ ]]; then echo "ERROR: PR URL '${PR_URL}' is not a github.com pull request URL" exit 1 fi diff --git a/plugins/sdlc-workflow/scripts/test_pre_verify_pr.py b/plugins/sdlc-workflow/scripts/test_pre_verify_pr.py index 6a34d64c6..fb24485da 100644 --- a/plugins/sdlc-workflow/scripts/test_pre_verify_pr.py +++ b/plugins/sdlc-workflow/scripts/test_pre_verify_pr.py @@ -3,6 +3,7 @@ import json import os +import re import subprocess import sys import tempfile @@ -393,6 +394,104 @@ def test_pre_verify_sh_paginates_review_comment_fetches(): assert "jq 'add'" in line, f"missing jq 'add' merge: {line!r}" +# --- PR-URL parsing regex (pre-verify-pr.sh) --- + +def _github_pr_url_regex(): + """Extract the github.com PR-URL match regex from pre-verify-pr.sh. + + The tests below run the *actual* regex the script ships (not a re-typed + copy), so an accidental loss of the end anchor is caught behaviorally. + """ + with open(pre_verify_sh) as f: + for line in f: + if "=~" in line and "github" in line and "/pull/" in line: + m = re.search(r"=~\s+(\S.*?)\s+\]\]", line) + if m: + return m.group(1) + raise AssertionError("github.com PR-URL regex not found in pre-verify-pr.sh") + + +def _match_pr_url(url): + """Run pre-verify-pr.sh's exact `[[ =~ ]]` test against url. + + Returns (matched, repo, number) using the same BASH_REMATCH groups the + script consumes downstream, so a truncating match surfaces as a wrong + `number` rather than a silent pass. + """ + regex = _github_pr_url_regex() + snippet = ( + 'r="$1"; u="$2"\n' + 'if [[ "$u" =~ $r ]]; then\n' + ' printf "MATCH\\t%s\\t%s" "${BASH_REMATCH[1]}" "${BASH_REMATCH[2]}"\n' + 'else\n' + ' printf "NOMATCH"\n' + 'fi\n' + ) + result = subprocess.run( + ["bash", "-c", snippet, "bash", regex, url], + capture_output=True, text=True, + ) + assert result.returncode == 0, f"bash error: {result.stderr}" + parts = result.stdout.split("\t") + if parts[0] == "MATCH": + return True, parts[1], parts[2] + return False, None, None + + +def test_pr_url_wellformed_accepted(): + """A canonical github.com PR URL parses to its owner/repo and PR number.""" + # Given a well-formed PR URL like a Jira inlineCard stores + # When matched by the script's regex + matched, repo, number = _match_pr_url("https://github.com/org/repo/pull/42") + + # Then it matches and captures the exact repo and number + assert matched, "well-formed URL should match" + assert repo == "org/repo", f"Got repo: {repo!r}" + assert number == "42", f"Got number: {number!r}" + + +def test_pr_url_trailing_slash_accepted(): + """A well-formed PR URL with a trailing slash still parses correctly.""" + # Given a PR URL with a trailing slash + # When matched by the script's regex + matched, repo, number = _match_pr_url("https://github.com/org/repo/pull/42/") + + # Then it matches with the same repo and number (the slash is tolerated) + assert matched, "trailing-slash URL should match" + assert repo == "org/repo", f"Got repo: {repo!r}" + assert number == "42", f"Got number: {number!r}" + + +def test_pr_url_nonnumeric_suffix_rejected(): + """A pull number with a trailing non-numeric suffix is rejected, not truncated. + + Regression for Sourcery id 3902059934: the un-anchored regex accepted + `.../pull/42abc` and truncated the number to 42, so the pre-script fetched + the wrong PR. The end-anchored regex must reject it outright. + """ + # Given a malformed URL with a non-numeric suffix on the pull number + # When matched by the script's regex + matched, _repo, number = _match_pr_url("https://github.com/org/repo/pull/42abc") + + # Then it does not match (rather than truncating to 42) + assert not matched, f"expected rejection, but matched with number={number!r}" + + +def test_pr_url_extra_path_segment_rejected(): + """An extra path segment after the pull number is rejected, not truncated. + + Regression for Sourcery id 3902059934: `.../pull/42/invalid` previously + matched and truncated to PR 42. The end anchor must reject it. + """ + # Given a malformed URL with an extra path segment after the number + # When matched by the script's regex + matched, _repo, number = _match_pr_url( + "https://github.com/org/repo/pull/42/invalid") + + # Then it does not match (rather than truncating to 42) + assert not matched, f"expected rejection, but matched with number={number!r}" + + # --- runner --- if __name__ == "__main__": From f002c1abebf74423d989e24c7acfb6cd7d6229fa Mon Sep 17 00:00:00 2001 From: mrizzi Date: Tue, 1 Sep 2026 12:46:02 +0200 Subject: [PATCH 011/175] fix(verify-pr): derive COMMIT_SHA from headRefOid in pre-verify-pr.sh MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit gh's `pr view` commits connection is bounded, so `.commits[-1].oid` returns the last commit of a truncated page rather than the PR head on large PRs — a silently-wrong commit_sha that still satisfies the result schema's hex pattern. Read the head ref tip OID directly (`--json headRefOid`), which is correct regardless of commit count. Also document that the bundled commits list uses the same bounded connection (best-effort context; head SHA is authoritative). Adds a behavioral regression test that runs the shipped COMMIT_SHA command against a gh stub whose headRefOid and commits[-1].oid disagree, plus a source guard, both of which fail on the old logic and pass on the new. Implements TC-5924 Assisted-by: Claude Code Co-Authored-By: Claude Opus 4.8 --- .../sdlc-workflow/scripts/pre-verify-pr.sh | 12 ++- .../scripts/test_pre_verify_pr.py | 79 +++++++++++++++++++ 2 files changed, 90 insertions(+), 1 deletion(-) diff --git a/plugins/sdlc-workflow/scripts/pre-verify-pr.sh b/plugins/sdlc-workflow/scripts/pre-verify-pr.sh index fe1f3369b..14bfa414e 100755 --- a/plugins/sdlc-workflow/scripts/pre-verify-pr.sh +++ b/plugins/sdlc-workflow/scripts/pre-verify-pr.sh @@ -114,7 +114,11 @@ echo "PR: ${PR_REPO}#${PR_NUM}" : "${GH_TOKEN:?GH_TOKEN is required to prefetch PR ${PR_REPO}#${PR_NUM}}" HEAD_REF=$(gh pr view "${PR_NUM}" -R "${PR_REPO}" --json headRefName --jq .headRefName) -COMMIT_SHA=$(gh pr view "${PR_NUM}" -R "${PR_REPO}" --json commits --jq '.commits[-1].oid') +# Derive the head SHA from the ref tip OID, not from `.commits[-1].oid`: gh's +# `pr view` commits connection is bounded, so on a PR with more commits than the +# cap `.commits[-1]` is the last commit of a truncated page rather than the head +# — a silently-wrong SHA that still satisfies the result schema's hex pattern. +COMMIT_SHA=$(gh pr view "${PR_NUM}" -R "${PR_REPO}" --json headRefOid --jq .headRefOid) gh pr diff "${PR_NUM}" -R "${PR_REPO}" > "${PRE_OUTPUT_DIR}/pr.diff" # `gh pr diff` has no --stat flag; derive the per-file diffstat from the patch we @@ -134,6 +138,12 @@ fi gh api --paginate --slurp "repos/${PR_REPO}/pulls/${PR_NUM}/reviews" | jq 'add' > "${PRE_OUTPUT_DIR}/reviews.json" gh api --paginate --slurp "repos/${PR_REPO}/pulls/${PR_NUM}/comments" | jq 'add' > "${PRE_OUTPUT_DIR}/review-comments.json" gh api --paginate --slurp "repos/${PR_REPO}/issues/${PR_NUM}/comments" | jq 'add' > "${PRE_OUTPUT_DIR}/issue-comments.json" +# The bundled commits list uses gh's bounded `pr view` commits connection, so it +# may be truncated on a very large PR. The authoritative head SHA is taken from +# headRefOid above; this list is best-effort context for commit-traceability. If +# a consumer ever treats it as authoritative, switch to a paginated +# `gh api --paginate .../pulls/${PR_NUM}/commits` (which returns a different, +# `sha`-shaped object needing reshaping to match the `oid` contract). gh pr view "${PR_NUM}" -R "${PR_REPO}" --json commits --jq .commits > "${PRE_OUTPUT_DIR}/commits.json" echo "GitHub read bundle prefetched to ${PRE_OUTPUT_DIR}" diff --git a/plugins/sdlc-workflow/scripts/test_pre_verify_pr.py b/plugins/sdlc-workflow/scripts/test_pre_verify_pr.py index fb24485da..ab5f76842 100644 --- a/plugins/sdlc-workflow/scripts/test_pre_verify_pr.py +++ b/plugins/sdlc-workflow/scripts/test_pre_verify_pr.py @@ -492,6 +492,85 @@ def test_pr_url_extra_path_segment_rejected(): assert not matched, f"expected rejection, but matched with number={number!r}" +# --- COMMIT_SHA derivation (pre-verify-pr.sh) --- + +def _commit_sha_command(): + """Extract the COMMIT_SHA assignment command from pre-verify-pr.sh. + + The behavioral test below runs the *actual* command the script ships (not a + re-typed copy), so a regression back to the bounded commits connection is + caught by execution, not just by source inspection. + """ + with open(pre_verify_sh) as f: + for line in f: + if line.lstrip().startswith("COMMIT_SHA="): + return line.strip() + raise AssertionError("COMMIT_SHA assignment not found in pre-verify-pr.sh") + + +def test_commit_sha_derived_from_head_ref_oid(): + """The prefetched COMMIT_SHA is the PR head ref tip OID, not the last commit + of gh's bounded commits connection. + + Runs the exact COMMIT_SHA command pre-verify-pr.sh ships against a gh stub + whose headRefOid and commits[-1].oid disagree (simulating a large PR whose + commits connection is truncated below the head). Regression for Sourcery id + 3902563589: the old `.commits[-1].oid` read returns the truncated oid. + """ + # Given a gh stub where the head ref OID and the (truncated) commits + # connection's last oid disagree + head_oid = "a" * 40 + truncated_oid = "b" * 40 + with tempfile.TemporaryDirectory() as d: + gh_stub = os.path.join(d, "gh") + with open(gh_stub, "w") as f: + f.write( + "#!/usr/bin/env bash\n" + "for arg in \"$@\"; do\n" + f' if [[ "$arg" == headRefOid ]]; then echo {head_oid}; exit 0; fi\n' + f' if [[ "$arg" == commits ]]; then echo {truncated_oid}; exit 0; fi\n' + "done\n" + "echo UNEXPECTED >&2; exit 1\n" + ) + os.chmod(gh_stub, 0o755) + + # When running the exact COMMIT_SHA assignment the script ships, with the + # stub gh ahead on PATH and PR_NUM/PR_REPO supplied + command = _commit_sha_command() + env = {**os.environ, "PATH": d + os.pathsep + os.environ["PATH"], + "PR_NUM": "275", "PR_REPO": "o/r"} + result = subprocess.run( + ["bash", "-c", command + "\nprintf '%s' \"$COMMIT_SHA\""], + capture_output=True, text=True, env=env, + ) + + # Then the emitted commit SHA is the head ref OID, not the truncated last commit + assert result.returncode == 0, f"Exit {result.returncode}: {result.stderr}" + assert result.stdout == head_oid, f"Got: {result.stdout!r} (stderr: {result.stderr!r})" + assert result.stdout != truncated_oid + + +def test_pre_verify_sh_derives_commit_sha_from_head_ref_oid(): + """Regression guard: COMMIT_SHA reads headRefOid and never the truncatable + commits connection (.commits[-1].oid) (Sourcery id 3902563589). + """ + # Given the current pre-verify-pr.sh source + with open(pre_verify_sh) as f: + script = f.read() + + # Then every COMMIT_SHA assignment reads headRefOid and none falls back to + # gh's bounded commits connection + commit_sha_lines = [ + line for line in script.splitlines() + if line.lstrip().startswith("COMMIT_SHA=") + ] + assert commit_sha_lines, "no COMMIT_SHA assignment found in pre-verify-pr.sh" + for line in commit_sha_lines: + assert "headRefOid" in line, f"COMMIT_SHA not derived from headRefOid: {line!r}" + assert ".commits[-1]" not in line, \ + f"COMMIT_SHA reintroduced the bounded commits read: {line!r}" + + # --- runner --- if __name__ == "__main__": From f3bb34b00e5932aae4db063263a041f867c1472b Mon Sep 17 00:00:00 2001 From: mrizzi Date: Tue, 1 Sep 2026 14:26:52 +0200 Subject: [PATCH 012/175] docs(verify-pr): sync github-bundle schema descriptions to shipped derivations MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Two field descriptions in the `github` read bundle of verify-pr-input.schema.json cited commands that no longer produce their values (doc-string drift only; the field values are correct): - `commit_sha` cited `.commits[-1].oid`, but pre-verify-pr.sh derives it from `gh pr view --json headRefOid` (TC-5924 — the pr-view commits connection is bounded, so `.commits[-1]` can be a truncated-page tip). - `stat` cited `gh pr diff --stat`, an unsupported flag; it is derived from `git apply --stat` on the already-fetched pr.diff (TC-5884). Documentation-only; the schema remains valid JSON. Implements TC-5929 Assisted-by: Claude Code Co-Authored-By: Claude Opus 4.8 --- plugins/sdlc-workflow/schemas/verify-pr-input.schema.json | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/plugins/sdlc-workflow/schemas/verify-pr-input.schema.json b/plugins/sdlc-workflow/schemas/verify-pr-input.schema.json index f40d09e06..d7f853829 100644 --- a/plugins/sdlc-workflow/schemas/verify-pr-input.schema.json +++ b/plugins/sdlc-workflow/schemas/verify-pr-input.schema.json @@ -67,7 +67,7 @@ "commit_sha": { "type": "string", "pattern": "^[0-9a-f]{7,40}$", - "description": "OID of the PR head (last) commit (gh pr view --json commits --jq '.commits[-1].oid')" + "description": "OID of the PR head ref tip commit (gh pr view --json headRefOid --jq .headRefOid)" }, "diff": { "type": "string", @@ -75,7 +75,7 @@ }, "stat": { "type": "string", - "description": "PR diffstat (gh pr diff --stat)" + "description": "PR diffstat (git apply --stat on the fetched pr.diff; gh pr diff has no --stat flag)" }, "reviews": { "type": "array", From 752c43bbcd2e657ff12fc6e7fb0bbaf641afca1f Mon Sep 17 00:00:00 2001 From: mrizzi Date: Tue, 1 Sep 2026 14:56:50 +0200 Subject: [PATCH 013/175] feat(verify-pr): add host-side write path with native Jira comments MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Port the verify-pr write executor to run host-side after the sandbox is destroyed. Jira comments and the verification report are posted through the native `fullsend issues post-comment` sticky-comment CLI (idempotent via a stable marker), while the irreducible custom Jira writes with no native primitive yet — sub-tasks, links, and root-cause tasks — call jira-client.py directly. GitHub writes stay host-side on `gh`. - post-verify-pr.sh: locate the latest agent-result.json, validate, delegate - execute-actions.py: resolve {{ref.key}}/{{ref.url}} placeholders, route post_comment/post_report Jira side through the native CLI, keep gh for GitHub, keep jira-client.py for sub-tasks/links/root-cause - jira-client.py: create_issue accepts a pre-rendered ADF description and an optional parent key (reconciled additively; existing callers unaffected) - test_execute_actions.py: ref-resolution tests plus native-CLI argv/env, stdin body, and post_comment/post_report routing tests Implements TC-5811 Assisted-by: Claude Code Co-Authored-By: Claude Opus 4.8 --- .../sdlc-workflow/scripts/execute-actions.py | 296 ++++++++++++++++++ plugins/sdlc-workflow/scripts/jira-client.py | 26 +- .../sdlc-workflow/scripts/post-verify-pr.sh | 45 +++ .../scripts/test_execute_actions.py | 265 ++++++++++++++++ 4 files changed, 627 insertions(+), 5 deletions(-) create mode 100755 plugins/sdlc-workflow/scripts/execute-actions.py create mode 100755 plugins/sdlc-workflow/scripts/post-verify-pr.sh create mode 100644 plugins/sdlc-workflow/scripts/test_execute_actions.py diff --git a/plugins/sdlc-workflow/scripts/execute-actions.py b/plugins/sdlc-workflow/scripts/execute-actions.py new file mode 100755 index 000000000..3a320e5e5 --- /dev/null +++ b/plugins/sdlc-workflow/scripts/execute-actions.py @@ -0,0 +1,296 @@ +#!/usr/bin/env python3 +"""Execute verify-pr structured output actions. + +Reads agent-result.json, processes actions sequentially, resolves +{{ref.key}} and {{ref.url}} placeholders as entities are created. + +Jira comments (post_comment / post_report) are posted through the native +`fullsend issues post-comment` sticky-comment CLI. The irreducible Jira +writes that have no native primitive yet — sub-tasks, links, and root-cause +tasks — call jira-client.py functions directly (imported as a module). +GitHub operations use the gh CLI. Runs on the fullsend runner (trusted +side), not inside the sandbox. + +Not idempotent for entity creation: if an action fails mid-execution, +previously created Jira sub-tasks are not rolled back. Manual cleanup may be +needed after partial failures. Comment posting is idempotent — the sticky +marker updates an existing comment instead of duplicating it. + +Usage: + execute-actions.py + +Required env vars: + JIRA_SERVER_URL, JIRA_EMAIL, JIRA_API_TOKEN — Jira credentials + GH_TOKEN — GitHub token + JIRA_PROJECT_KEY — Jira project key (for root-cause task creation) +""" + +import importlib.util +import json +import os +import re +import subprocess +import sys +from typing import Any + +_script_dir = os.path.dirname(os.path.abspath(__file__)) +_spec = importlib.util.spec_from_file_location( + "jira_client", os.path.join(_script_dir, "jira-client.py") +) +_jira_mod = importlib.util.module_from_spec(_spec) +_spec.loader.exec_module(_jira_mod) + +REF_PATTERN = re.compile(r"\{\{([a-z0-9-]+)\.(key|url)\}\}") + +# Stable marker so the native sticky-comment CLI updates the same Jira +# comment across re-runs instead of posting duplicates. +STICKY_COMMENT_MARKER = "" + + +def resolve_refs(text: str, registry: dict[str, dict[str, str]]) -> str: + """Replace {{ref.key}} and {{ref.url}} placeholders with resolved values.""" + def replacer(match): + ref_name, field = match.group(1), match.group(2) + if ref_name not in registry: + raise KeyError(f"Unknown ref: {ref_name}") + return registry[ref_name][field] + return REF_PATTERN.sub(replacer, text) + + +def resolve_refs_in_obj(obj: Any, registry: dict[str, dict[str, str]]) -> Any: + """Recursively resolve refs in a JSON-like object (dicts, lists, strings).""" + if isinstance(obj, str): + return resolve_refs(obj, registry) + if isinstance(obj, list): + return [resolve_refs_in_obj(item, registry) for item in obj] + if isinstance(obj, dict): + return {k: resolve_refs_in_obj(v, registry) for k, v in obj.items()} + return obj + + +def build_issue_url(key: str) -> str: + """Build Jira issue browse URL from key.""" + server = os.environ.get("JIRA_SERVER_URL", "").rstrip("/") + if not server: + print("JIRA_SERVER_URL is required for issue URL construction", file=sys.stderr) + sys.exit(1) + return f"{server}/browse/{key}" + + +def _jira_native_env() -> dict[str, str]: + """Build the environment for the native fullsend Jira CLI. + + Maps this script's JIRA_SERVER_URL / JIRA_EMAIL / JIRA_API_TOKEN onto the + JIRA_BASE_URL / JIRA_USER_EMAIL / JIRA_TOKEN names the fullsend CLI expects. + Exits with a clear message if any credential is missing. + """ + try: + return { + **os.environ, + "JIRA_BASE_URL": os.environ["JIRA_SERVER_URL"], + "JIRA_USER_EMAIL": os.environ["JIRA_EMAIL"], + "JIRA_TOKEN": os.environ["JIRA_API_TOKEN"], + } + except KeyError as e: + print(f"Missing required env var for native Jira comment: {e.args[0]}", file=sys.stderr) + sys.exit(1) + + +def post_jira_comment_native(issue_key: str, body_md: str) -> None: + """Post a Jira comment via the native fullsend sticky-comment CLI. + + The body is passed as markdown on stdin (``--result -``). A stable marker + makes the post idempotent: re-runs update the existing comment rather than + creating duplicates. + """ + project, _, number = issue_key.rpartition("-") + if not project or not number: + print(f"Invalid Jira issue key: {issue_key}", file=sys.stderr) + sys.exit(1) + + result = subprocess.run( + ["fullsend", "issues", "post-comment", "--tracker", "jira", + "--project", project, "--number", number, + "--marker", STICKY_COMMENT_MARKER, "--result", "-"], + input=body_md, text=True, capture_output=True, + env=_jira_native_env(), + ) + if result.returncode != 0: + print(f"fullsend post-comment failed for {issue_key}: {result.stderr}", file=sys.stderr) + sys.exit(1) + + +def _create_and_register(action: dict, registry: dict, *, + project_key: str, issue_type: str, + parent: str | None = None, label: str = "issue") -> None: + ref = action["ref"] + summary = resolve_refs(action["summary"], registry) + labels = action["labels"] + description_adf = resolve_refs_in_obj(action["description_adf"], registry) + + try: + result = _jira_mod.create_issue( + project_key=project_key, + summary=summary, + issue_type=issue_type, + parent=parent, + description_adf=description_adf, + labels=labels, + ) + except SystemExit: + print(f" Failed to create {label}: {summary}", file=sys.stderr) + sys.exit(1) + + key = result.get("key", "") + url = build_issue_url(key) + registry[ref] = {"key": key, "url": url} + print(f" Created {label}: {key} (ref: {ref})") + + +def execute_create_subtask(action: dict, registry: dict) -> None: + parent = resolve_refs(action["parent"], registry) + _create_and_register( + action, registry, + project_key=parent.split("-")[0], + issue_type="Sub-task", + parent=parent, + label="sub-task", + ) + + +def execute_create_link(action: dict, registry: dict) -> None: + link_type = action["link_type"] + inward = resolve_refs(action["inward"], registry) + outward = resolve_refs(action["outward"], registry) + + try: + _jira_mod.create_link( + inward_issue=inward, + outward_issue=outward, + link_type=link_type, + ) + except SystemExit: + print(f" Failed to create link: {inward} {link_type} {outward}", file=sys.stderr) + sys.exit(1) + + print(f" Created link: {inward} {link_type} {outward}") + + +def execute_post_pr_reply(action: dict, registry: dict) -> None: + repo = action["repo"] + pr_number = action["pr_number"] + comment_id = action["comment_id"] + body = resolve_refs(action["body"], registry) + + result = subprocess.run( + ["gh", "api", + f"repos/{repo}/pulls/{pr_number}/comments/{comment_id}/replies", + "-f", f"body={body}"], + capture_output=True, text=True, + ) + if result.returncode != 0: + print(f"gh api failed: {result.stderr}", file=sys.stderr) + sys.exit(1) + print(f" Posted PR reply on comment {comment_id}") + + +def execute_post_pr_comment(action: dict, registry: dict) -> None: + repo = action["repo"] + pr_number = action["pr_number"] + body = resolve_refs(action["body"], registry) + + result = subprocess.run( + ["gh", "api", + f"repos/{repo}/issues/{pr_number}/comments", + "-f", f"body={body}"], + capture_output=True, text=True, + ) + if result.returncode != 0: + print(f"gh api failed: {result.stderr}", file=sys.stderr) + sys.exit(1) + print(f" Posted PR comment on #{pr_number}") + + +def execute_create_root_cause_task(action: dict, registry: dict) -> None: + project_key = os.environ.get("JIRA_PROJECT_KEY", "") + if not project_key: + print("JIRA_PROJECT_KEY is required for root-cause task creation", file=sys.stderr) + sys.exit(1) + + _create_and_register( + action, registry, + project_key=project_key, + issue_type="Task", + label="root-cause task", + ) + + +def execute_post_comment(action: dict, registry: dict) -> None: + issue = resolve_refs(action["issue"], registry) + body_md = resolve_refs(action["body_md"], registry) + + post_jira_comment_native(issue, body_md) + print(f" Posted comment on {issue}") + + +def execute_post_report(action: dict, registry: dict, report: dict) -> None: + repo = report["pr_repo"] + pr_number = report["pr_number"] + jira_issue_id = report["jira_issue_id"] + report_md = resolve_refs(report["report_md"], registry) + + result = subprocess.run( + ["gh", "pr", "comment", str(pr_number), "--body", report_md, "-R", repo], + capture_output=True, text=True, + ) + if result.returncode != 0: + print(f"Failed to post GitHub PR comment: {result.stderr}", file=sys.stderr) + sys.exit(1) + print(f" Posted report to PR #{pr_number}") + + post_jira_comment_native(jira_issue_id, report_md) + print(f" Posted report to Jira {jira_issue_id}") + + +EXECUTORS = { + "create_subtask": execute_create_subtask, + "create_link": execute_create_link, + "post_pr_reply": execute_post_pr_reply, + "post_pr_comment": execute_post_pr_comment, + "create_root_cause_task": execute_create_root_cause_task, + "post_comment": execute_post_comment, +} + + +def main(): + if len(sys.argv) != 2: + print(f"Usage: {sys.argv[0]} ", file=sys.stderr) + sys.exit(1) + + result_path = sys.argv[1] + with open(result_path) as f: + data = json.load(f) + + report = data["report"] + actions = data["actions"] + registry: dict[str, dict[str, str]] = {} + + print(f"Executing {len(actions)} actions for {report['jira_issue_id']}...") + + for i, action in enumerate(actions): + action_type = action["type"] + print(f"[{i + 1}/{len(actions)}] {action_type}") + + if action_type == "post_report": + execute_post_report(action, registry, report) + elif action_type in EXECUTORS: + EXECUTORS[action_type](action, registry) + else: + print(f" Unknown action type: {action_type}", file=sys.stderr) + sys.exit(1) + + print(f"Done. {len(actions)} actions executed successfully.") + + +if __name__ == "__main__": + main() diff --git a/plugins/sdlc-workflow/scripts/jira-client.py b/plugins/sdlc-workflow/scripts/jira-client.py index 5f5feec71..6d3566c65 100755 --- a/plugins/sdlc-workflow/scripts/jira-client.py +++ b/plugins/sdlc-workflow/scripts/jira-client.py @@ -443,15 +443,17 @@ def get_issue(issue_key: str, fields: str = "*all") -> Dict[str, Any]: def create_issue( project_key: str, summary: str, - description_md: str, - issue_type: str, + description_md: Optional[str] = None, + issue_type: str = "", labels: Optional[List[str]] = None, assignee_id: Optional[str] = None, priority: Optional[str] = None, fix_versions: Optional[List[str]] = None, - custom_fields: Optional[Dict[str, Any]] = None + custom_fields: Optional[Dict[str, Any]] = None, + description_adf: Optional[Dict[str, Any]] = None, + parent: Optional[str] = None ) -> Dict[str, Any]: - """Create JIRA issue with markdown description. + """Create JIRA issue with a markdown or pre-rendered ADF description. Args: project_key: Project key (e.g., TC) @@ -463,19 +465,33 @@ def create_issue( priority: Optional priority name (e.g., "Major") fix_versions: Optional list of fixVersion names custom_fields: Optional custom field values (field_id: value) + description_adf: Pre-rendered ADF description (takes precedence over description_md) + parent: Optional parent issue key (creates a sub-task under it) Returns: Created issue object with key and ID """ + # Prefer a pre-rendered ADF description; fall back to converting markdown, + # then to an empty document when neither is supplied. + if description_adf is not None: + description = description_adf + elif description_md is not None: + description = markdown_to_adf(description_md) + else: + description = {"type": "doc", "version": 1, "content": []} + data = { "fields": { "project": {"key": project_key}, "summary": summary, - "description": sanitize_adf(markdown_to_adf(description_md)), + "description": sanitize_adf(description), "issuetype": {"id": issue_type} if issue_type.isdigit() else {"name": issue_type}, } } + if parent: + data["fields"]["parent"] = {"key": parent} + if labels: data["fields"]["labels"] = labels diff --git a/plugins/sdlc-workflow/scripts/post-verify-pr.sh b/plugins/sdlc-workflow/scripts/post-verify-pr.sh new file mode 100755 index 000000000..7045eb0f0 --- /dev/null +++ b/plugins/sdlc-workflow/scripts/post-verify-pr.sh @@ -0,0 +1,45 @@ +#!/usr/bin/env bash +# post-verify-pr.sh — Execute verify-pr structured output actions. +# +# Runs on the fullsend runner AFTER the sandbox is destroyed. +# Working directory is the fullsend run output directory. +# +# Required env vars: +# JIRA_SERVER_URL — Jira instance URL +# JIRA_EMAIL — Jira user email +# JIRA_API_TOKEN — Jira API token +# JIRA_PROJECT_KEY — Jira project key (for root-cause task creation) +# GH_TOKEN — GitHub token +# +# The agent writes its output to output/agent-result.json (relative to +# the iteration directory). This script finds the most recent iteration's +# output and delegates to execute-actions.py. + +set -euo pipefail + +RESULT_FILE="" +for dir in iteration-*/output; do + if [[ -f "${dir}/agent-result.json" ]]; then + RESULT_FILE="${dir}/agent-result.json" + fi +done + +if [[ -z "${RESULT_FILE}" ]]; then + echo "ERROR: agent-result.json not found in any iteration output directory" + exit 1 +fi + +echo "Reading verify-pr result from: ${RESULT_FILE}" + +if ! jq empty "${RESULT_FILE}" 2>/dev/null; then + echo "ERROR: ${RESULT_FILE} is not valid JSON" + exit 1 +fi + +OVERALL=$(jq -r '.report.overall' "${RESULT_FILE}") +ACTION_COUNT=$(jq '.actions | length' "${RESULT_FILE}") +echo "Overall: ${OVERALL}" +echo "Actions: ${ACTION_COUNT}" + +SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)" +python3 "${SCRIPT_DIR}/execute-actions.py" "${RESULT_FILE}" diff --git a/plugins/sdlc-workflow/scripts/test_execute_actions.py b/plugins/sdlc-workflow/scripts/test_execute_actions.py new file mode 100644 index 000000000..03067451a --- /dev/null +++ b/plugins/sdlc-workflow/scripts/test_execute_actions.py @@ -0,0 +1,265 @@ +#!/usr/bin/env python3 +"""Tests for execute-actions.py ref resolution and native Jira comment posting.""" + +import sys +import os +import json +import importlib.util + +script_dir = os.path.dirname(os.path.abspath(__file__)) +spec = importlib.util.spec_from_file_location( + "execute_actions", + os.path.join(script_dir, "execute-actions.py"), +) +execute_actions = importlib.util.module_from_spec(spec) +spec.loader.exec_module(execute_actions) + +resolve_refs = execute_actions.resolve_refs + + +def test_resolve_refs_replaces_key(): + """A single {{ref.key}}/{{ref.url}} pair resolves to registered values.""" + registry = {"subtask-1": {"key": "TC-100", "url": "https://jira.example.com/browse/TC-100"}} + text = "Sub-task [{{subtask-1.key}}]({{subtask-1.url}}) created." + result = resolve_refs(text, registry) + assert result == "Sub-task [TC-100](https://jira.example.com/browse/TC-100) created.", f"Got: {result}" + + +def test_resolve_refs_no_placeholders(): + """Text without placeholders is returned unchanged.""" + registry = {} + text = "No placeholders here." + result = resolve_refs(text, registry) + assert result == "No placeholders here." + + +def test_resolve_refs_unknown_ref_raises(): + """An unregistered ref raises KeyError.""" + registry = {} + text = "{{unknown-ref.key}}" + try: + resolve_refs(text, registry) + assert False, "Should have raised KeyError" + except KeyError: + pass + + +def test_resolve_refs_in_adf(): + """resolve_refs_in_obj resolves placeholders nested inside an ADF doc.""" + registry = {"rc-1": {"key": "TC-200", "url": "https://jira.example.com/browse/TC-200"}} + adf = { + "type": "doc", + "content": [ + {"type": "text", "text": "Task {{rc-1.key}} created"} + ] + } + result = execute_actions.resolve_refs_in_obj(adf, registry) + assert result["content"][0]["text"] == "Task TC-200 created" + + +def test_resolve_refs_multiple_different_refs(): + """Multiple distinct refs in one string each resolve independently.""" + registry = { + "subtask-1": {"key": "TC-100", "url": "https://jira.example.com/browse/TC-100"}, + "rc-1": {"key": "TC-200", "url": "https://jira.example.com/browse/TC-200"}, + } + text = "Sub-task {{subtask-1.key}} and root-cause {{rc-1.key}} ({{rc-1.url}})." + result = resolve_refs(text, registry) + assert result == "Sub-task TC-100 and root-cause TC-200 (https://jira.example.com/browse/TC-200).", f"Got: {result}" + + +def test_resolve_refs_repeated_placeholder(): + """A placeholder repeated in one string resolves at every occurrence.""" + registry = {"subtask-1": {"key": "TC-100", "url": "https://jira.example.com/browse/TC-100"}} + text = "{{subtask-1.key}} depends on {{subtask-1.key}}." + result = resolve_refs(text, registry) + assert result == "TC-100 depends on TC-100.", f"Got: {result}" + + +def test_resolve_refs_mixed_key_url_same_ref(): + """The .key and .url fields of one ref resolve to their respective values.""" + registry = {"subtask-1": {"key": "TC-100", "url": "https://jira.example.com/browse/TC-100"}} + text = "See {{subtask-1.key}} at {{subtask-1.url}}; {{subtask-1.key}} must be done first." + result = resolve_refs(text, registry) + assert result == "See TC-100 at https://jira.example.com/browse/TC-100; TC-100 must be done first.", f"Got: {result}" + + +class _FakeCompleted: + """Stand-in for subprocess.CompletedProcess.""" + + def __init__(self, returncode=0, stderr=""): + self.returncode = returncode + self.stderr = stderr + + +class _RunRecorder: + """Captures the argv/input/env of a single subprocess.run call.""" + + def __init__(self, returncode=0, stderr=""): + self.returncode = returncode + self.stderr = stderr + self.cmd = None + self.input = None + self.env = None + + def __call__(self, cmd, input=None, text=None, capture_output=None, env=None): + self.cmd = cmd + self.input = input + self.env = env + return _FakeCompleted(self.returncode, self.stderr) + + +_JIRA_ENV = { + "JIRA_SERVER_URL": "https://jira.example.com", + "JIRA_EMAIL": "bot@example.com", + "JIRA_API_TOKEN": "s3cr3t", +} + + +def _with_jira_env_and_recorder(recorder): + """Install a fake subprocess.run + Jira env; return a restore callback.""" + saved_run = execute_actions.subprocess.run + saved_env = {k: os.environ.get(k) for k in _JIRA_ENV} + execute_actions.subprocess.run = recorder + os.environ.update(_JIRA_ENV) + + def restore(): + execute_actions.subprocess.run = saved_run + for k, v in saved_env.items(): + if v is None: + os.environ.pop(k, None) + else: + os.environ[k] = v + + return restore + + +def test_post_jira_comment_native_builds_argv(): + """post_jira_comment_native calls the native CLI with marker, project, and number.""" + recorder = _RunRecorder() + restore = _with_jira_env_and_recorder(recorder) + try: + execute_actions.post_jira_comment_native("TC-321", "hello **world**") + finally: + restore() + + assert recorder.cmd[:4] == ["fullsend", "issues", "post-comment", "--tracker"], f"Got: {recorder.cmd}" + assert "jira" in recorder.cmd + assert "--project" in recorder.cmd and recorder.cmd[recorder.cmd.index("--project") + 1] == "TC" + assert "--number" in recorder.cmd and recorder.cmd[recorder.cmd.index("--number") + 1] == "321" + assert "--marker" in recorder.cmd + assert recorder.cmd[recorder.cmd.index("--marker") + 1] == execute_actions.STICKY_COMMENT_MARKER + assert "--result" in recorder.cmd and recorder.cmd[recorder.cmd.index("--result") + 1] == "-" + assert recorder.input == "hello **world**" + + +def test_post_jira_comment_native_maps_env(): + """The native CLI receives JIRA_BASE_URL/JIRA_USER_EMAIL/JIRA_TOKEN mapped from this script's vars.""" + recorder = _RunRecorder() + restore = _with_jira_env_and_recorder(recorder) + try: + execute_actions.post_jira_comment_native("TC-1", "body") + finally: + restore() + + assert recorder.env["JIRA_BASE_URL"] == "https://jira.example.com" + assert recorder.env["JIRA_USER_EMAIL"] == "bot@example.com" + assert recorder.env["JIRA_TOKEN"] == "s3cr3t" + + +def test_post_jira_comment_native_nonzero_exits(): + """A non-zero CLI exit aborts with sys.exit(1).""" + recorder = _RunRecorder(returncode=1, stderr="boom") + restore = _with_jira_env_and_recorder(recorder) + try: + execute_actions.post_jira_comment_native("TC-1", "body") + assert False, "Should have exited" + except SystemExit as e: + assert e.code == 1 + finally: + restore() + + +def test_post_jira_comment_native_invalid_key_exits(): + """A malformed issue key (no hyphen) aborts before invoking the CLI.""" + recorder = _RunRecorder() + restore = _with_jira_env_and_recorder(recorder) + try: + execute_actions.post_jira_comment_native("TC123", "body") + assert False, "Should have exited" + except SystemExit as e: + assert e.code == 1 + finally: + restore() + assert recorder.cmd is None, "CLI should not run for an invalid key" + + +def test_execute_post_comment_routes_to_native(): + """execute_post_comment resolves refs then posts body_md to the action's issue.""" + recorder = _RunRecorder() + restore = _with_jira_env_and_recorder(recorder) + registry = {"sub-1": {"key": "TC-500", "url": "https://jira.example.com/browse/TC-500"}} + try: + execute_actions.execute_post_comment( + {"type": "post_comment", "issue": "{{sub-1.key}}", "body_md": "See {{sub-1.url}}"}, + registry, + ) + finally: + restore() + + assert recorder.cmd[recorder.cmd.index("--number") + 1] == "500" + assert recorder.input == "See https://jira.example.com/browse/TC-500" + + +def test_execute_post_report_posts_github_then_jira(): + """execute_post_report posts the report to GitHub via gh, then to Jira via the native CLI.""" + calls = [] + + def fake_run(cmd, input=None, text=None, capture_output=None, env=None): + calls.append({"cmd": cmd, "input": input, "env": env}) + return _FakeCompleted(0, "") + + saved_run = execute_actions.subprocess.run + saved_env = {k: os.environ.get(k) for k in _JIRA_ENV} + execute_actions.subprocess.run = fake_run + os.environ.update(_JIRA_ENV) + try: + report = { + "pr_repo": "acme/widget", + "pr_number": 42, + "jira_issue_id": "TC-777", + "report_md": "## Verify report\nAll good.", + } + execute_actions.execute_post_report({"type": "post_report"}, {}, report) + finally: + execute_actions.subprocess.run = saved_run + for k, v in saved_env.items(): + if v is None: + os.environ.pop(k, None) + else: + os.environ[k] = v + + assert len(calls) == 2, f"Expected gh + native calls, got {len(calls)}" + gh_call, jira_call = calls + assert gh_call["cmd"][:3] == ["gh", "pr", "comment"] + assert "acme/widget" in gh_call["cmd"] + assert jira_call["cmd"][:3] == ["fullsend", "issues", "post-comment"] + assert jira_call["cmd"][jira_call["cmd"].index("--number") + 1] == "777" + assert jira_call["input"] == "## Verify report\nAll good." + + +if __name__ == "__main__": + test_resolve_refs_replaces_key() + test_resolve_refs_no_placeholders() + test_resolve_refs_unknown_ref_raises() + test_resolve_refs_in_adf() + test_resolve_refs_multiple_different_refs() + test_resolve_refs_repeated_placeholder() + test_resolve_refs_mixed_key_url_same_ref() + test_post_jira_comment_native_builds_argv() + test_post_jira_comment_native_maps_env() + test_post_jira_comment_native_nonzero_exits() + test_post_jira_comment_native_invalid_key_exits() + test_execute_post_comment_routes_to_native() + test_execute_post_report_posts_github_then_jira() + print("All tests passed.") From 1695632caab4593deb0c89b1a18cc25be0cbdf73 Mon Sep 17 00:00:00 2001 From: mrizzi Date: Tue, 1 Sep 2026 15:39:28 +0200 Subject: [PATCH 014/175] fix(verify-pr): render post_comment body_adf to markdown for native Jira CLI execute_post_comment read action["body_md"], but the result schema requires body_adf (an ADF object) and strip_extra_properties.py removes any extra key, so every schema-valid post_comment raised KeyError before posting. Add adf_to_markdown to render the ADF body back to the markdown the native fullsend CLI consumes on stdin, and update the masking test to feed a schema-valid body_adf action. TC-5930 Assisted-by: Claude Code Co-Authored-By: Claude Opus 4.8 --- .../sdlc-workflow/scripts/execute-actions.py | 85 ++++++++++++++++++- .../scripts/test_execute_actions.py | 60 ++++++++++++- 2 files changed, 141 insertions(+), 4 deletions(-) diff --git a/plugins/sdlc-workflow/scripts/execute-actions.py b/plugins/sdlc-workflow/scripts/execute-actions.py index 3a320e5e5..b5b06757a 100755 --- a/plugins/sdlc-workflow/scripts/execute-actions.py +++ b/plugins/sdlc-workflow/scripts/execute-actions.py @@ -68,6 +68,88 @@ def resolve_refs_in_obj(obj: Any, registry: dict[str, dict[str, str]]) -> Any: return obj +def _render_adf_inline(nodes: list) -> str: + """Render a list of ADF inline nodes to markdown.""" + parts = [] + for node in nodes: + node_type = node.get("type") + if node_type == "text": + text = node.get("text", "") + marks = node.get("marks", []) + mark_types = {m.get("type") for m in marks} + if "code" in mark_types: + text = f"`{text}`" + if "strong" in mark_types: + text = f"**{text}**" + if "em" in mark_types: + text = f"*{text}*" + link = next((m for m in marks if m.get("type") == "link"), None) + if link: + href = link.get("attrs", {}).get("href", "") + text = f"[{text}]({href})" + parts.append(text) + elif node_type == "hardBreak": + parts.append("\n") + else: + # Unknown inline node — recurse into any nested content. + parts.append(_render_adf_inline(node.get("content", []))) + return "".join(parts) + + +def _render_adf_list(node: dict, *, ordered: bool) -> str: + """Render an ADF bulletList/orderedList to markdown.""" + lines = [] + for index, item in enumerate(node.get("content", []), start=1): + marker = f"{index}." if ordered else "-" + item_md = _render_adf_blocks(item.get("content", [])) + for line_number, line in enumerate(item_md.split("\n")): + prefix = f"{marker} " if line_number == 0 else " " + lines.append(f"{prefix}{line}") + return "\n".join(lines) + + +def _render_adf_block(node: dict) -> str: + """Render a single ADF block node to markdown.""" + node_type = node.get("type") + if node_type == "paragraph": + return _render_adf_inline(node.get("content", [])) + if node_type == "heading": + level = node.get("attrs", {}).get("level", 1) + return f"{'#' * level} {_render_adf_inline(node.get('content', []))}" + if node_type == "rule": + return "---" + if node_type == "codeBlock": + language = node.get("attrs", {}).get("language", "") or "" + code = "".join(child.get("text", "") for child in node.get("content", [])) + return f"```{language}\n{code}\n```" + if node_type == "bulletList": + return _render_adf_list(node, ordered=False) + if node_type == "orderedList": + return _render_adf_list(node, ordered=True) + # Unknown block — recurse into nested content. + return _render_adf_blocks(node.get("content", [])) + + +def _render_adf_blocks(nodes: list) -> str: + """Render a list of ADF block nodes to markdown, separated by blank lines.""" + blocks = [_render_adf_block(node) for node in nodes] + return "\n\n".join(block for block in blocks if block) + + +def adf_to_markdown(doc: dict) -> str: + """Render an ADF document to markdown (inverse of jira-client's markdown_to_adf). + + The native ``fullsend issues post-comment`` CLI consumes markdown on stdin, + but the result schema carries ``post_comment`` bodies as ADF (``body_adf``). + This renders the ADF back to markdown covering the node set markdown_to_adf + produces: headings, paragraphs, bullet/ordered lists, code blocks, rules, and + the strong/em/code/link inline marks. + """ + if not isinstance(doc, dict): + raise TypeError("adf_to_markdown expects an ADF document object") + return _render_adf_blocks(doc.get("content", [])) + + def build_issue_url(key: str) -> str: """Build Jira issue browse URL from key.""" server = os.environ.get("JIRA_SERVER_URL", "").rstrip("/") @@ -227,7 +309,8 @@ def execute_create_root_cause_task(action: dict, registry: dict) -> None: def execute_post_comment(action: dict, registry: dict) -> None: issue = resolve_refs(action["issue"], registry) - body_md = resolve_refs(action["body_md"], registry) + body_adf = resolve_refs_in_obj(action["body_adf"], registry) + body_md = adf_to_markdown(body_adf) post_jira_comment_native(issue, body_md) print(f" Posted comment on {issue}") diff --git a/plugins/sdlc-workflow/scripts/test_execute_actions.py b/plugins/sdlc-workflow/scripts/test_execute_actions.py index 03067451a..7f7c82f4d 100644 --- a/plugins/sdlc-workflow/scripts/test_execute_actions.py +++ b/plugins/sdlc-workflow/scripts/test_execute_actions.py @@ -194,21 +194,74 @@ def test_post_jira_comment_native_invalid_key_exits(): assert recorder.cmd is None, "CLI should not run for an invalid key" +def test_adf_to_markdown_renders_blocks_and_marks(): + """adf_to_markdown renders headings, lists, code blocks, rules, and inline marks.""" + doc = { + "type": "doc", + "version": 1, + "content": [ + {"type": "heading", "attrs": {"level": 2}, + "content": [{"type": "text", "text": "Title"}]}, + {"type": "paragraph", "content": [ + {"type": "text", "text": "See "}, + {"type": "text", "text": "TC-1", "marks": [{"type": "strong"}]}, + {"type": "text", "text": " and "}, + {"type": "text", "text": "run", "marks": [{"type": "code"}]}, + {"type": "text", "text": " at "}, + {"type": "text", "text": "here", + "marks": [{"type": "link", "attrs": {"href": "https://x.example/y"}}]}, + ]}, + {"type": "bulletList", "content": [ + {"type": "listItem", "content": [ + {"type": "paragraph", "content": [{"type": "text", "text": "first"}]}]}, + {"type": "listItem", "content": [ + {"type": "paragraph", "content": [{"type": "text", "text": "second"}]}]}, + ]}, + {"type": "rule"}, + {"type": "codeBlock", "attrs": {"language": "python"}, + "content": [{"type": "text", "text": "x = 1"}]}, + ], + } + result = execute_actions.adf_to_markdown(doc) + expected = ( + "## Title\n\n" + "See **TC-1** and `run` at [here](https://x.example/y)\n\n" + "- first\n- second\n\n" + "---\n\n" + "```python\nx = 1\n```" + ) + assert result == expected, f"Got: {result!r}" + + def test_execute_post_comment_routes_to_native(): - """execute_post_comment resolves refs then posts body_md to the action's issue.""" + """execute_post_comment resolves refs in body_adf, renders to markdown, and posts it.""" recorder = _RunRecorder() restore = _with_jira_env_and_recorder(recorder) registry = {"sub-1": {"key": "TC-500", "url": "https://jira.example.com/browse/TC-500"}} + body_adf = { + "type": "doc", + "version": 1, + "content": [ + {"type": "paragraph", "content": [ + {"type": "text", "text": "See "}, + {"type": "text", "text": "{{sub-1.key}}", "marks": [{"type": "strong"}]}, + {"type": "text", "text": " at "}, + {"type": "text", "text": "link", + "marks": [{"type": "link", "attrs": {"href": "{{sub-1.url}}"}}]}, + ]}, + ], + } try: execute_actions.execute_post_comment( - {"type": "post_comment", "issue": "{{sub-1.key}}", "body_md": "See {{sub-1.url}}"}, + {"type": "post_comment", "issue": "{{sub-1.key}}", "body_adf": body_adf}, registry, ) finally: restore() assert recorder.cmd[recorder.cmd.index("--number") + 1] == "500" - assert recorder.input == "See https://jira.example.com/browse/TC-500" + assert recorder.input == "See **TC-500** at [link](https://jira.example.com/browse/TC-500)", \ + f"Got: {recorder.input!r}" def test_execute_post_report_posts_github_then_jira(): @@ -260,6 +313,7 @@ def fake_run(cmd, input=None, text=None, capture_output=None, env=None): test_post_jira_comment_native_maps_env() test_post_jira_comment_native_nonzero_exits() test_post_jira_comment_native_invalid_key_exits() + test_adf_to_markdown_renders_blocks_and_marks() test_execute_post_comment_routes_to_native() test_execute_post_report_posts_github_then_jira() print("All tests passed.") From 3ca8ba989e41dd1616ffa74ea2bd6133f1f232ea Mon Sep 17 00:00:00 2001 From: mrizzi Date: Tue, 1 Sep 2026 15:53:26 +0200 Subject: [PATCH 015/175] fix(verify-pr): accept result.json fallback in post-verify-pr.sh MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit post-verify-pr.sh only located agent-result.json, but its sibling validate-output-schema.sh accepts result.json as a fallback. A result that passed validation via the fallback name was then rejected by the write path. Prefer agent-result.json per iteration output dir, and fall back to result.json when absent — matching the validator's precedence. Implements TC-5931 Assisted-by: Claude Code Co-Authored-By: Claude Opus 4.8 --- plugins/sdlc-workflow/scripts/post-verify-pr.sh | 7 ++++++- 1 file changed, 6 insertions(+), 1 deletion(-) diff --git a/plugins/sdlc-workflow/scripts/post-verify-pr.sh b/plugins/sdlc-workflow/scripts/post-verify-pr.sh index 7045eb0f0..f154ef13a 100755 --- a/plugins/sdlc-workflow/scripts/post-verify-pr.sh +++ b/plugins/sdlc-workflow/scripts/post-verify-pr.sh @@ -19,13 +19,18 @@ set -euo pipefail RESULT_FILE="" for dir in iteration-*/output; do + # Prefer agent-result.json; fall back to result.json when it is absent, + # matching the precedence in validate-output-schema.sh (agents sometimes + # write "result.json" instead of "agent-result.json"). if [[ -f "${dir}/agent-result.json" ]]; then RESULT_FILE="${dir}/agent-result.json" + elif [[ -f "${dir}/result.json" ]]; then + RESULT_FILE="${dir}/result.json" fi done if [[ -z "${RESULT_FILE}" ]]; then - echo "ERROR: agent-result.json not found in any iteration output directory" + echo "ERROR: no agent-result.json or result.json found in any iteration output directory" exit 1 fi From 32e857797682c5c4537a927935567a7bfaa9ff7e Mon Sep 17 00:00:00 2001 From: mrizzi Date: Tue, 1 Sep 2026 15:58:48 +0200 Subject: [PATCH 016/175] fix(verify-pr): select highest-numbered iteration output numerically post-verify-pr.sh selected the "most recent" iteration by keeping the last match in shell glob order, which is lexicographic. Once there are >= 10 iterations, iteration-9 sorts after iteration-20 and a stale iteration is chosen, so the write path would execute the wrong (older) result. Iterate the iteration-*/output directories in ascending numeric order via `sort -V` and keep the highest-numbered one that has a result file. Preserves the `set -euo pipefail` behavior and the result.json fallback (TC-5931); a `[[ -d ]]` guard skips the literal glob pattern when no iteration dir exists. TC-5932 Assisted-by: Claude Code Co-Authored-By: Claude Opus 4.8 --- plugins/sdlc-workflow/scripts/post-verify-pr.sh | 11 +++++++++-- 1 file changed, 9 insertions(+), 2 deletions(-) diff --git a/plugins/sdlc-workflow/scripts/post-verify-pr.sh b/plugins/sdlc-workflow/scripts/post-verify-pr.sh index f154ef13a..c6ae68abd 100755 --- a/plugins/sdlc-workflow/scripts/post-verify-pr.sh +++ b/plugins/sdlc-workflow/scripts/post-verify-pr.sh @@ -18,7 +18,14 @@ set -euo pipefail RESULT_FILE="" -for dir in iteration-*/output; do +# Iterate iteration directories in ascending numeric order so the +# highest-numbered iteration that has a result file wins. Plain glob order is +# lexicographic (iteration-9 sorts after iteration-20), which would select a +# stale iteration once there are >= 10 iterations. `sort -V` orders the +# embedded iteration numbers numerically; the `[[ -d ]]` guard skips the +# literal glob pattern when no iteration directory exists. +while IFS= read -r dir; do + [[ -d "${dir}" ]] || continue # Prefer agent-result.json; fall back to result.json when it is absent, # matching the precedence in validate-output-schema.sh (agents sometimes # write "result.json" instead of "agent-result.json"). @@ -27,7 +34,7 @@ for dir in iteration-*/output; do elif [[ -f "${dir}/result.json" ]]; then RESULT_FILE="${dir}/result.json" fi -done +done < <(printf '%s\n' iteration-*/output | sort -V) if [[ -z "${RESULT_FILE}" ]]; then echo "ERROR: no agent-result.json or result.json found in any iteration output directory" From 99da584de431d264cb96a8442205e7857672d38a Mon Sep 17 00:00:00 2001 From: mrizzi Date: Tue, 1 Sep 2026 16:47:37 +0200 Subject: [PATCH 017/175] fix(verify-pr): render ADF taskList/taskItem in adf_to_markdown The ADF block renderer had no taskList/taskItem case, so a schema-valid post_comment task list fell through to the unknown-block fallback and was flattened to plain paragraph text, losing the checklist markers. Add _render_adf_task_list rendering DONE/TODO items as markdown checkboxes, handling both paragraph-wrapped and inline taskItem content. The comment route calls adf_to_markdown directly (not sanitize_adf), so the renderer is the correct fix site. Implements TC-5936 Assisted-by: Claude Code Co-Authored-By: Claude Opus 4.8 --- .../sdlc-workflow/scripts/execute-actions.py | 36 ++++++++++++++++++- .../scripts/test_execute_actions.py | 19 ++++++++++ 2 files changed, 54 insertions(+), 1 deletion(-) diff --git a/plugins/sdlc-workflow/scripts/execute-actions.py b/plugins/sdlc-workflow/scripts/execute-actions.py index b5b06757a..91aa571f9 100755 --- a/plugins/sdlc-workflow/scripts/execute-actions.py +++ b/plugins/sdlc-workflow/scripts/execute-actions.py @@ -68,6 +68,11 @@ def resolve_refs_in_obj(obj: Any, registry: dict[str, dict[str, str]]) -> Any: return obj +# ADF inline node types that may appear directly inside a taskItem (per the ADF +# spec) rather than being wrapped in a paragraph block. +_INLINE_NODE_TYPES = {"text", "hardBreak", "mention", "emoji", "inlineCard", "date", "status"} + + def _render_adf_inline(nodes: list) -> str: """Render a list of ADF inline nodes to markdown.""" parts = [] @@ -108,6 +113,31 @@ def _render_adf_list(node: dict, *, ordered: bool) -> str: return "\n".join(lines) +def _render_adf_task_list(node: dict) -> str: + """Render an ADF taskList to a markdown checklist. + + Each taskItem carries a ``state`` attr of ``DONE`` or ``TODO``, rendered as + ``- [x]`` / ``- [ ]``. taskItem content may be inline nodes (ADF spec) or + paragraph blocks (as ``sanitize_adf`` and some producers treat them); both + are handled so no variant falls through to the flattening unknown-block path. + """ + lines = [] + for item in node.get("content", []): + if item.get("type") != "taskItem": + continue + state = item.get("attrs", {}).get("state", "TODO") + marker = "- [x]" if state == "DONE" else "- [ ]" + content = item.get("content", []) + if content and all(child.get("type") in _INLINE_NODE_TYPES for child in content): + item_md = _render_adf_inline(content) + else: + item_md = _render_adf_blocks(content) + for line_number, line in enumerate(item_md.split("\n")): + prefix = f"{marker} " if line_number == 0 else " " + lines.append(f"{prefix}{line}") + return "\n".join(lines) + + def _render_adf_block(node: dict) -> str: """Render a single ADF block node to markdown.""" node_type = node.get("type") @@ -126,6 +156,8 @@ def _render_adf_block(node: dict) -> str: return _render_adf_list(node, ordered=False) if node_type == "orderedList": return _render_adf_list(node, ordered=True) + if node_type == "taskList": + return _render_adf_task_list(node) # Unknown block — recurse into nested content. return _render_adf_blocks(node.get("content", [])) @@ -143,7 +175,9 @@ def adf_to_markdown(doc: dict) -> str: but the result schema carries ``post_comment`` bodies as ADF (``body_adf``). This renders the ADF back to markdown covering the node set markdown_to_adf produces: headings, paragraphs, bullet/ordered lists, code blocks, rules, and - the strong/em/code/link inline marks. + the strong/em/code/link inline marks. taskList/taskItem nodes (which agents may + emit directly in ``body_adf``) render as markdown checklists (``- [ ]`` / + ``- [x]``). """ if not isinstance(doc, dict): raise TypeError("adf_to_markdown expects an ADF document object") diff --git a/plugins/sdlc-workflow/scripts/test_execute_actions.py b/plugins/sdlc-workflow/scripts/test_execute_actions.py index 7f7c82f4d..b85544329 100644 --- a/plugins/sdlc-workflow/scripts/test_execute_actions.py +++ b/plugins/sdlc-workflow/scripts/test_execute_actions.py @@ -233,6 +233,24 @@ def test_adf_to_markdown_renders_blocks_and_marks(): assert result == expected, f"Got: {result!r}" +def test_adf_to_markdown_renders_task_list(): + """adf_to_markdown renders taskList DONE/TODO items as - [x] / - [ ] markers, for both paragraph-wrapped and inline taskItem content.""" + doc = { + "type": "doc", + "version": 1, + "content": [ + {"type": "taskList", "content": [ + {"type": "taskItem", "attrs": {"state": "DONE"}, "content": [ + {"type": "paragraph", "content": [{"type": "text", "text": "done item"}]}]}, + {"type": "taskItem", "attrs": {"state": "TODO"}, "content": [ + {"type": "text", "text": "todo item"}]}, + ]}, + ], + } + result = execute_actions.adf_to_markdown(doc) + assert result == "- [x] done item\n- [ ] todo item", f"Got: {result!r}" + + def test_execute_post_comment_routes_to_native(): """execute_post_comment resolves refs in body_adf, renders to markdown, and posts it.""" recorder = _RunRecorder() @@ -314,6 +332,7 @@ def fake_run(cmd, input=None, text=None, capture_output=None, env=None): test_post_jira_comment_native_nonzero_exits() test_post_jira_comment_native_invalid_key_exits() test_adf_to_markdown_renders_blocks_and_marks() + test_adf_to_markdown_renders_task_list() test_execute_post_comment_routes_to_native() test_execute_post_report_posts_github_then_jira() print("All tests passed.") From a75e12aca7aee32dca179cc846c6746dce614ad0 Mon Sep 17 00:00:00 2001 From: mrizzi Date: Tue, 1 Sep 2026 17:00:13 +0200 Subject: [PATCH 018/175] fix(verify-pr): make execute_post_report GitHub comment idempotent on retry execute_post_report posted the verification report to GitHub via a plain gh pr comment (no sticky marker) before the idempotent Jira post, so a retry after a failed Jira post double-posted the GitHub report comment. Embed a commit-scoped marker in the GitHub report body and, before posting, list the PR comments and PATCH-update an existing same-commit report comment instead of creating a duplicate. Dedup is scoped to the commit SHA so a later commit still gets a fresh comment, preserving the per-run verification history. The Jira side is unchanged and receives the clean body. Updated the module docstring so its idempotency claim matches actual behaviour. Implements TC-5937 Assisted-by: Claude Code Co-Authored-By: Claude Opus 4.8 --- .../sdlc-workflow/scripts/execute-actions.py | 81 ++++++++++++++++--- .../scripts/test_execute_actions.py | 68 ++++++++++++++-- 2 files changed, 134 insertions(+), 15 deletions(-) diff --git a/plugins/sdlc-workflow/scripts/execute-actions.py b/plugins/sdlc-workflow/scripts/execute-actions.py index 91aa571f9..32b12847b 100755 --- a/plugins/sdlc-workflow/scripts/execute-actions.py +++ b/plugins/sdlc-workflow/scripts/execute-actions.py @@ -13,8 +13,11 @@ Not idempotent for entity creation: if an action fails mid-execution, previously created Jira sub-tasks are not rolled back. Manual cleanup may be -needed after partial failures. Comment posting is idempotent — the sticky -marker updates an existing comment instead of duplicating it. +needed after partial failures. Comment posting is idempotent: the Jira sticky +marker updates the existing comment instead of duplicating it, and the GitHub +report comment carries a commit-scoped marker so a retry after a partial +failure updates the same commit's report comment rather than posting a +duplicate. Usage: execute-actions.py @@ -46,6 +49,14 @@ # comment across re-runs instead of posting duplicates. STICKY_COMMENT_MARKER = "" +# GitHub has no native sticky-comment mechanism, so the verify-pr report comment +# embeds this marker (an invisible HTML comment) in its body. The commit SHA is +# appended per post, scoping dedup to a single verification run/commit: a retry +# for the same commit updates the existing comment instead of duplicating it, +# while a later commit gets a fresh comment — preserving the per-run +# verification history that verify-pr SKILL.md Step 9 posts. +GITHUB_REPORT_MARKER_PREFIX = "" + github_body = f"{report_md}\n\n{marker}" + + existing_id = _find_report_comment_id(repo, pr_number, marker) + if existing_id is not None: + result = subprocess.run( + ["gh", "api", f"repos/{repo}/issues/comments/{existing_id}", + "-X", "PATCH", "-f", f"body={github_body}"], + capture_output=True, text=True, + ) + if result.returncode != 0: + print(f"Failed to update GitHub PR comment: {result.stderr}", file=sys.stderr) + sys.exit(1) + print(f" Updated existing report comment on PR #{pr_number}") + else: + result = subprocess.run( + ["gh", "pr", "comment", str(pr_number), "--body", github_body, "-R", repo], + capture_output=True, text=True, + ) + if result.returncode != 0: + print(f"Failed to post GitHub PR comment: {result.stderr}", file=sys.stderr) + sys.exit(1) + print(f" Posted report to PR #{pr_number}") post_jira_comment_native(jira_issue_id, report_md) print(f" Posted report to Jira {jira_issue_id}") diff --git a/plugins/sdlc-workflow/scripts/test_execute_actions.py b/plugins/sdlc-workflow/scripts/test_execute_actions.py index b85544329..51bbcf6c4 100644 --- a/plugins/sdlc-workflow/scripts/test_execute_actions.py +++ b/plugins/sdlc-workflow/scripts/test_execute_actions.py @@ -87,9 +87,10 @@ def test_resolve_refs_mixed_key_url_same_ref(): class _FakeCompleted: """Stand-in for subprocess.CompletedProcess.""" - def __init__(self, returncode=0, stderr=""): + def __init__(self, returncode=0, stderr="", stdout=""): self.returncode = returncode self.stderr = stderr + self.stdout = stdout class _RunRecorder: @@ -283,12 +284,13 @@ def test_execute_post_comment_routes_to_native(): def test_execute_post_report_posts_github_then_jira(): - """execute_post_report posts the report to GitHub via gh, then to Jira via the native CLI.""" + """execute_post_report lists PR comments, creates a marked GitHub comment when none exists, then posts to Jira.""" calls = [] def fake_run(cmd, input=None, text=None, capture_output=None, env=None): calls.append({"cmd": cmd, "input": input, "env": env}) - return _FakeCompleted(0, "") + # No existing report comment on the PR yet. + return _FakeCompleted(0, "", "[]") saved_run = execute_actions.subprocess.run saved_env = {k: os.environ.get(k) for k in _JIRA_ENV} @@ -299,6 +301,7 @@ def fake_run(cmd, input=None, text=None, capture_output=None, env=None): "pr_repo": "acme/widget", "pr_number": 42, "jira_issue_id": "TC-777", + "commit_sha": "946556e", "report_md": "## Verify report\nAll good.", } execute_actions.execute_post_report({"type": "post_report"}, {}, report) @@ -310,15 +313,69 @@ def fake_run(cmd, input=None, text=None, capture_output=None, env=None): else: os.environ[k] = v - assert len(calls) == 2, f"Expected gh + native calls, got {len(calls)}" - gh_call, jira_call = calls + assert len(calls) == 3, f"Expected list + create + native calls, got {len(calls)}" + list_call, gh_call, jira_call = calls + # Existing comments are listed first to check for a prior report. + assert list_call["cmd"][:2] == ["gh", "api"] + assert list_call["cmd"][2] == "repos/acme/widget/issues/42/comments" + # No existing comment → a new PR comment is created, carrying the commit marker. assert gh_call["cmd"][:3] == ["gh", "pr", "comment"] assert "acme/widget" in gh_call["cmd"] + gh_body = gh_call["cmd"][gh_call["cmd"].index("--body") + 1] + assert gh_body.startswith("## Verify report\nAll good.") + assert "" in gh_body + # Jira side is unchanged: sticky CLI, clean body without the GitHub marker. assert jira_call["cmd"][:3] == ["fullsend", "issues", "post-comment"] assert jira_call["cmd"][jira_call["cmd"].index("--number") + 1] == "777" assert jira_call["input"] == "## Verify report\nAll good." +def test_execute_post_report_updates_existing_github_comment_on_retry(): + """A retry for the same commit PATCH-updates the existing GitHub report comment instead of creating a duplicate.""" + # Given a prior report comment for this commit already exists on the PR + calls = [] + existing = [{"id": 555, + "body": "old report\n\n"}] + + def fake_run(cmd, input=None, text=None, capture_output=None, env=None): + calls.append({"cmd": cmd, "input": input, "env": env}) + if cmd[:2] == ["gh", "api"] and cmd[2].endswith("/comments"): + return _FakeCompleted(0, "", json.dumps(existing)) + return _FakeCompleted(0, "") + + saved_run = execute_actions.subprocess.run + saved_env = {k: os.environ.get(k) for k in _JIRA_ENV} + execute_actions.subprocess.run = fake_run + os.environ.update(_JIRA_ENV) + try: + report = { + "pr_repo": "acme/widget", + "pr_number": 42, + "jira_issue_id": "TC-777", + "commit_sha": "946556e", + "report_md": "## Verify report\nAll good.", + } + # When posting the report again (e.g. after a prior Jira failure) + execute_actions.execute_post_report({"type": "post_report"}, {}, report) + finally: + execute_actions.subprocess.run = saved_run + for k, v in saved_env.items(): + if v is None: + os.environ.pop(k, None) + else: + os.environ[k] = v + + # Then no new GitHub comment is created; the existing one is PATCH-updated + assert not any(c["cmd"][:3] == ["gh", "pr", "comment"] for c in calls), \ + "retry must not create a new GitHub comment" + patch_calls = [c for c in calls if "PATCH" in c["cmd"]] + assert len(patch_calls) == 1, f"Expected one PATCH update, got {len(patch_calls)}" + assert patch_calls[0]["cmd"][2] == "repos/acme/widget/issues/comments/555" + # And the Jira report is still posted + assert any(c["cmd"][:3] == ["fullsend", "issues", "post-comment"] for c in calls), \ + "Jira report must still be posted on retry" + + if __name__ == "__main__": test_resolve_refs_replaces_key() test_resolve_refs_no_placeholders() @@ -335,4 +392,5 @@ def fake_run(cmd, input=None, text=None, capture_output=None, env=None): test_adf_to_markdown_renders_task_list() test_execute_post_comment_routes_to_native() test_execute_post_report_posts_github_then_jira() + test_execute_post_report_updates_existing_github_comment_on_retry() print("All tests passed.") From 239c0dfafb1c5e0299a6e6305c6959dddbb00e9c Mon Sep 17 00:00:00 2001 From: mrizzi Date: Tue, 1 Sep 2026 18:09:46 +0200 Subject: [PATCH 019/175] fix(verify-pr): parse paginated gh api output in _find_report_comment_id MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit _find_report_comment_id listed a PR's issue comments with `gh api ... --paginate` and parsed the result with a single json.loads. Without `--slurp`, `gh --paginate` concatenates one JSON array per page (`[...][...]`), which is not valid combined JSON once the PR has more than one page of comments (>30). json.loads then raised JSONDecodeError, which the code swallowed by returning None; execute_post_report read that as "no existing report comment" and created a duplicate GitHub report comment instead of PATCH-updating the existing one, silently defeating the retry idempotency TC-5937 introduced. Add `--slurp` so gh emits a single array-of-pages and flatten the pages into one comment list, so a same-commit report comment is found even when it lands on a later page. Stop swallowing JSONDecodeError: with `--slurp` a parse error is a real failure and now surfaces via sys.exit(1) rather than being misread as "no existing comment". Added a multi-page test (report comment on a non-first page → PATCH-update, not duplicate) and a parse-failure test (exits instead of returning None). Implements TC-5948 Assisted-by: Claude Code Co-Authored-By: Claude Opus 4.8 --- .../sdlc-workflow/scripts/execute-actions.py | 20 ++++- .../scripts/test_execute_actions.py | 86 ++++++++++++++++++- 2 files changed, 98 insertions(+), 8 deletions(-) diff --git a/plugins/sdlc-workflow/scripts/execute-actions.py b/plugins/sdlc-workflow/scripts/execute-actions.py index 32b12847b..8d64d0d29 100755 --- a/plugins/sdlc-workflow/scripts/execute-actions.py +++ b/plugins/sdlc-workflow/scripts/execute-actions.py @@ -367,18 +367,30 @@ def _find_report_comment_id(repo: str, pr_number: int, marker: str) -> int | Non Lists the PR's issue-level comments and matches on the commit-scoped marker so a retry updates the same commit's report comment instead of duplicating it. Returns ``None`` when no marked comment exists yet. + + ``--slurp`` is required alongside ``--paginate``: without it ``gh`` emits one + JSON array per page concatenated (``[...][...]``), which is not valid combined + JSON once the PR has more than one page of comments (>30) and would fail to + parse. ``--slurp`` wraps the per-page arrays in a single outer array, so the + output is valid JSON regardless of page count; the pages are then flattened + into one comment list. A parse failure is a real error (surfaced via + ``sys.exit``), never silently treated as "no existing comment" — doing so + would defeat retry idempotency by creating a duplicate report comment. """ result = subprocess.run( - ["gh", "api", f"repos/{repo}/issues/{pr_number}/comments", "--paginate"], + ["gh", "api", f"repos/{repo}/issues/{pr_number}/comments", + "--paginate", "--slurp"], capture_output=True, text=True, ) if result.returncode != 0: print(f"Failed to list PR comments: {result.stderr}", file=sys.stderr) sys.exit(1) try: - comments = json.loads(result.stdout or "[]") - except json.JSONDecodeError: - return None + pages = json.loads(result.stdout or "[]") + except json.JSONDecodeError as e: + print(f"Failed to parse PR comments JSON: {e}", file=sys.stderr) + sys.exit(1) + comments = [comment for page in pages for comment in page] for comment in comments: if marker in (comment.get("body") or ""): return comment.get("id") diff --git a/plugins/sdlc-workflow/scripts/test_execute_actions.py b/plugins/sdlc-workflow/scripts/test_execute_actions.py index 51bbcf6c4..450236e3a 100644 --- a/plugins/sdlc-workflow/scripts/test_execute_actions.py +++ b/plugins/sdlc-workflow/scripts/test_execute_actions.py @@ -332,15 +332,17 @@ def fake_run(cmd, input=None, text=None, capture_output=None, env=None): def test_execute_post_report_updates_existing_github_comment_on_retry(): """A retry for the same commit PATCH-updates the existing GitHub report comment instead of creating a duplicate.""" - # Given a prior report comment for this commit already exists on the PR + # Given a prior report comment for this commit already exists on the PR. + # `gh api --paginate --slurp` wraps each page's comment array in one outer + # array, so the listing is a single-page array-of-pages here. calls = [] - existing = [{"id": 555, - "body": "old report\n\n"}] + existing_page = [{"id": 555, + "body": "old report\n\n"}] def fake_run(cmd, input=None, text=None, capture_output=None, env=None): calls.append({"cmd": cmd, "input": input, "env": env}) if cmd[:2] == ["gh", "api"] and cmd[2].endswith("/comments"): - return _FakeCompleted(0, "", json.dumps(existing)) + return _FakeCompleted(0, "", json.dumps([existing_page])) return _FakeCompleted(0, "") saved_run = execute_actions.subprocess.run @@ -376,6 +378,80 @@ def fake_run(cmd, input=None, text=None, capture_output=None, env=None): "Jira report must still be posted on retry" +def test_execute_post_report_updates_comment_on_later_page(): + """When the listing spans multiple pages, an existing report comment on a + non-first page is still found and PATCH-updated (no duplicate created).""" + # Given a slurped, multi-page listing (array-of-pages) where the marked report + # comment lives on the SECOND page — the exact case a single json.loads on + # concatenated per-page arrays could not parse. + calls = [] + page_one = [ + {"id": 101, "body": "just a normal review comment"}, + {"id": 102, "body": "another unrelated comment"}, + ] + page_two = [ + {"id": 103, "body": "chatter"}, + {"id": 555, + "body": "old report\n\n"}, + ] + + def fake_run(cmd, input=None, text=None, capture_output=None, env=None): + calls.append({"cmd": cmd, "input": input, "env": env}) + if cmd[:2] == ["gh", "api"] and cmd[2].endswith("/comments"): + # --paginate --slurp wraps each page's array in one outer array. + return _FakeCompleted(0, "", json.dumps([page_one, page_two])) + return _FakeCompleted(0, "") + + saved_run = execute_actions.subprocess.run + saved_env = {k: os.environ.get(k) for k in _JIRA_ENV} + execute_actions.subprocess.run = fake_run + os.environ.update(_JIRA_ENV) + try: + report = { + "pr_repo": "acme/widget", + "pr_number": 42, + "jira_issue_id": "TC-777", + "commit_sha": "946556e", + "report_md": "## Verify report\nAll good.", + } + # When posting the report again for the same commit + execute_actions.execute_post_report({"type": "post_report"}, {}, report) + finally: + execute_actions.subprocess.run = saved_run + for k, v in saved_env.items(): + if v is None: + os.environ.pop(k, None) + else: + os.environ[k] = v + + # Then the comment on the second page is PATCH-updated, not duplicated. + assert not any(c["cmd"][:3] == ["gh", "pr", "comment"] for c in calls), \ + "must not create a new GitHub comment when the report exists on a later page" + patch_calls = [c for c in calls if "PATCH" in c["cmd"]] + assert len(patch_calls) == 1, f"Expected one PATCH update, got {len(patch_calls)}" + assert patch_calls[0]["cmd"][2] == "repos/acme/widget/issues/comments/555" + + +def test_find_report_comment_id_exits_on_unparseable_json(): + """A JSON parse failure aborts with sys.exit(1) instead of silently returning + None (which would let a retry create a duplicate report comment).""" + # Given `gh` returns malformed JSON (e.g. concatenated per-page arrays, the + # pre-fix --paginate-without-slurp shape that is not valid combined JSON) + def fake_run(cmd, input=None, text=None, capture_output=None, env=None): + return _FakeCompleted(0, "", "[{\"id\": 1}][{\"id\": 2}]") + + saved_run = execute_actions.subprocess.run + execute_actions.subprocess.run = fake_run + try: + # When the id lookup runs, it must fail loudly rather than swallow the error + execute_actions._find_report_comment_id("acme/widget", 42, "marker") + assert False, "Should have exited on unparseable JSON" + except SystemExit as e: + assert e.code == 1 + finally: + execute_actions.subprocess.run = saved_run + + if __name__ == "__main__": test_resolve_refs_replaces_key() test_resolve_refs_no_placeholders() @@ -393,4 +469,6 @@ def fake_run(cmd, input=None, text=None, capture_output=None, env=None): test_execute_post_comment_routes_to_native() test_execute_post_report_posts_github_then_jira() test_execute_post_report_updates_existing_github_comment_on_retry() + test_execute_post_report_updates_comment_on_later_page() + test_find_report_comment_id_exits_on_unparseable_json() print("All tests passed.") From 1213fb1bfd4f28f33b6f87422c8c39c0d4f3e27b Mon Sep 17 00:00:00 2001 From: mrizzi Date: Tue, 1 Sep 2026 18:20:41 +0200 Subject: [PATCH 020/175] fix(verify-pr): escape markdown-active chars in _render_adf_inline literal text MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit _render_adf_inline emitted a text node's literal value verbatim into the markdown handed to the native `fullsend issues post-comment` CLI. Because that CLI re-parses the markdown, literal markdown-active sequences inside ADF text (`*`, `_`, `[`, `]`, backticks, and backslash) were reinterpreted as formatting — e.g. a literal `*note*` rendered as emphasized text and `[x](y)` as a link. Add _escape_markdown, which backslash-escapes those characters (backslash first so its escapes are not re-escaped), and apply it to a text node's plain value before the mark wrapping the renderer intentionally adds, so the mark syntax is not double-escaped. Inline code is exempt: its content is literal to the CLI and escaping would corrupt inline-code semantics. Link hrefs are also left untouched. Added tests: literal `* _ [ ]` and backtick are escaped; strong/link marks, inline-code content, and a link href with an underscore are unaffected. Implements TC-5949 Assisted-by: Claude Code Co-Authored-By: Claude Opus 4.8 --- .../sdlc-workflow/scripts/execute-actions.py | 25 ++++++++++ .../scripts/test_execute_actions.py | 49 +++++++++++++++++++ 2 files changed, 74 insertions(+) diff --git a/plugins/sdlc-workflow/scripts/execute-actions.py b/plugins/sdlc-workflow/scripts/execute-actions.py index 8d64d0d29..856c98c1e 100755 --- a/plugins/sdlc-workflow/scripts/execute-actions.py +++ b/plugins/sdlc-workflow/scripts/execute-actions.py @@ -83,6 +83,26 @@ def resolve_refs_in_obj(obj: Any, registry: dict[str, dict[str, str]]) -> Any: # spec) rather than being wrapped in a paragraph block. _INLINE_NODE_TYPES = {"text", "hardBreak", "mention", "emoji", "inlineCard", "date", "status"} +# Markdown-active characters that the native fullsend CLI reinterprets when it +# re-parses the emitted markdown. Literal occurrences in ADF text nodes must be +# backslash-escaped so they render verbatim (e.g. a literal ``*note*`` stays +# ``*note*`` instead of becoming emphasized). Backslash is listed first so the +# escapes inserted for the other characters are not themselves re-escaped. +_MARKDOWN_ESCAPE_CHARS = ("\\", "`", "*", "_", "[", "]") + + +def _escape_markdown(text: str) -> str: + """Backslash-escape markdown-active characters in literal text. + + Applied to a text node's plain value before the renderer wraps it in the + mark syntax it intentionally adds (``**``, backticks, ``[]()``), so literal + text round-trips faithfully without double-escaping the marks. Not applied to + inline-code content, which the CLI treats literally. + """ + for ch in _MARKDOWN_ESCAPE_CHARS: + text = text.replace(ch, f"\\{ch}") + return text + def _render_adf_inline(nodes: list) -> str: """Render a list of ADF inline nodes to markdown.""" @@ -93,6 +113,11 @@ def _render_adf_inline(nodes: list) -> str: text = node.get("text", "") marks = node.get("marks", []) mark_types = {m.get("type") for m in marks} + # Escape markdown-active characters in the literal text before mark + # wrapping. Inline code is exempt: its content is literal to the CLI + # and escaping would corrupt inline-code semantics. + if "code" not in mark_types: + text = _escape_markdown(text) if "code" in mark_types: text = f"`{text}`" if "strong" in mark_types: diff --git a/plugins/sdlc-workflow/scripts/test_execute_actions.py b/plugins/sdlc-workflow/scripts/test_execute_actions.py index 450236e3a..ce145db18 100644 --- a/plugins/sdlc-workflow/scripts/test_execute_actions.py +++ b/plugins/sdlc-workflow/scripts/test_execute_actions.py @@ -252,6 +252,53 @@ def test_adf_to_markdown_renders_task_list(): assert result == "- [x] done item\n- [ ] todo item", f"Got: {result!r}" +def test_adf_to_markdown_escapes_markdown_active_chars_in_literal_text(): + """Literal markdown-active characters in a plain text node are backslash-escaped + so the native CLI renders them verbatim instead of reinterpreting them as + formatting.""" + # Given a paragraph whose literal text contains * _ [ ] and a backtick + doc = { + "type": "doc", + "version": 1, + "content": [ + {"type": "paragraph", "content": [ + {"type": "text", "text": "a*b_c[d]e`f"}, + ]}, + ], + } + # When rendering the ADF to markdown + result = execute_actions.adf_to_markdown(doc) + # Then each active character is escaped with a leading backslash + assert result == "a\\*b\\_c\\[d\\]e\\`f", f"Got: {result!r}" + + +def test_adf_to_markdown_does_not_double_escape_marks_or_code(): + """Intentional marks (strong/link) and inline code render correctly: the mark + syntax the renderer adds is not escaped, inline-code content stays literal, and + link hrefs are not escaped.""" + # Given marked text, an inline-code span containing an asterisk, and a link + # whose href contains an underscore + doc = { + "type": "doc", + "version": 1, + "content": [ + {"type": "paragraph", "content": [ + {"type": "text", "text": "bold", "marks": [{"type": "strong"}]}, + {"type": "text", "text": " and "}, + {"type": "text", "text": "a*b", "marks": [{"type": "code"}]}, + {"type": "text", "text": " see "}, + {"type": "text", "text": "here", + "marks": [{"type": "link", "attrs": {"href": "https://x.example/a_b"}}]}, + ]}, + ], + } + # When rendering the ADF to markdown + result = execute_actions.adf_to_markdown(doc) + # Then the ** stays, the code asterisk stays literal, and the href underscore + # is preserved (none are escaped) + assert result == "**bold** and `a*b` see [here](https://x.example/a_b)", f"Got: {result!r}" + + def test_execute_post_comment_routes_to_native(): """execute_post_comment resolves refs in body_adf, renders to markdown, and posts it.""" recorder = _RunRecorder() @@ -466,6 +513,8 @@ def fake_run(cmd, input=None, text=None, capture_output=None, env=None): test_post_jira_comment_native_invalid_key_exits() test_adf_to_markdown_renders_blocks_and_marks() test_adf_to_markdown_renders_task_list() + test_adf_to_markdown_escapes_markdown_active_chars_in_literal_text() + test_adf_to_markdown_does_not_double_escape_marks_or_code() test_execute_post_comment_routes_to_native() test_execute_post_report_posts_github_then_jira() test_execute_post_report_updates_existing_github_comment_on_retry() From 7b01dc5a59d18907cfd30f29a0162bbf665ddcb8 Mon Sep 17 00:00:00 2001 From: mrizzi Date: Tue, 1 Sep 2026 18:30:53 +0200 Subject: [PATCH 021/175] fix(verify-pr): render non-text inline ADF nodes in _render_adf_inline _render_adf_inline previously handled only text and hardBreak nodes; every other inline node fell into the else branch and recursed into a nonexistent content array, so mention/emoji/inlineCard/date/status inline nodes (whose value lives in attrs, not content) rendered as empty strings and were silently dropped -- contradicting the module's own _INLINE_NODE_TYPES set. Render each from its attrs: mention/emoji/status from attrs.text (escaped as literal text; emoji falls back to attrs.shortName), inlineCard from attrs.url (bare, unescaped), and date from attrs.timestamp via a new _render_adf_date helper (epoch millis to UTC YYYY-MM-DD). The final else is kept for genuinely unknown container-like inline nodes. Works in both paragraph and taskItem inline contexts. Implements TC-5950 Assisted-by: Claude Code Co-Authored-By: Claude Opus 4.8 --- .../sdlc-workflow/scripts/execute-actions.py | 44 +++++++++++++- .../scripts/test_execute_actions.py | 59 +++++++++++++++++++ 2 files changed, 102 insertions(+), 1 deletion(-) diff --git a/plugins/sdlc-workflow/scripts/execute-actions.py b/plugins/sdlc-workflow/scripts/execute-actions.py index 856c98c1e..908d26a68 100755 --- a/plugins/sdlc-workflow/scripts/execute-actions.py +++ b/plugins/sdlc-workflow/scripts/execute-actions.py @@ -28,6 +28,7 @@ JIRA_PROJECT_KEY — Jira project key (for root-cause task creation) """ +import datetime import importlib.util import json import os @@ -104,8 +105,36 @@ def _escape_markdown(text: str) -> str: return text +def _render_adf_date(timestamp: Any) -> str: + """Render an ADF ``date`` node's timestamp as a readable ``YYYY-MM-DD`` date. + + ADF ``date`` nodes store ``attrs.timestamp`` as a string of milliseconds + since the Unix epoch. It is formatted as a UTC calendar date. If the value is + missing or not an integer, its literal form is returned so the node still + contributes its value instead of being dropped. + """ + try: + ms = int(timestamp) + except (TypeError, ValueError): + return str(timestamp) if timestamp else "" + return datetime.datetime.fromtimestamp( + ms / 1000, tz=datetime.timezone.utc + ).strftime("%Y-%m-%d") + + def _render_adf_inline(nodes: list) -> str: - """Render a list of ADF inline nodes to markdown.""" + """Render a list of ADF inline nodes to markdown. + + Handles every inline leaf type in ``_INLINE_NODE_TYPES``: ``text`` and + ``hardBreak``, plus the leaf nodes that carry their displayable value in + ``attrs`` rather than a ``content`` array — ``mention``/``emoji``/``status`` + (``attrs.text``), ``inlineCard`` (``attrs.url``) and ``date`` + (``attrs.timestamp``). These are rendered explicitly so they are not routed + to the final ``else`` (which recurses into ``content``) and silently dropped + as empty strings; that keeps the renderer in sync with ``_INLINE_NODE_TYPES``. + The ``else`` remains for genuinely unknown container-like inline nodes so they + still degrade gracefully by rendering any nested content. + """ parts = [] for node in nodes: node_type = node.get("type") @@ -131,6 +160,19 @@ def _render_adf_inline(nodes: list) -> str: parts.append(text) elif node_type == "hardBreak": parts.append("\n") + elif node_type in ("mention", "status"): + # Displayable text lives in attrs.text; escape it as literal text. + parts.append(_escape_markdown(node.get("attrs", {}).get("text", ""))) + elif node_type == "emoji": + # Prefer the unicode/text fallback; custom emoji may only have a + # shortName. Escaped as literal text. + attrs = node.get("attrs", {}) + parts.append(_escape_markdown(attrs.get("text") or attrs.get("shortName", ""))) + elif node_type == "inlineCard": + # Render the card's URL as a bare link target; URLs are not escaped. + parts.append(node.get("attrs", {}).get("url", "")) + elif node_type == "date": + parts.append(_render_adf_date(node.get("attrs", {}).get("timestamp", ""))) else: # Unknown inline node — recurse into any nested content. parts.append(_render_adf_inline(node.get("content", []))) diff --git a/plugins/sdlc-workflow/scripts/test_execute_actions.py b/plugins/sdlc-workflow/scripts/test_execute_actions.py index ce145db18..5f4eec2bb 100644 --- a/plugins/sdlc-workflow/scripts/test_execute_actions.py +++ b/plugins/sdlc-workflow/scripts/test_execute_actions.py @@ -299,6 +299,63 @@ def test_adf_to_markdown_does_not_double_escape_marks_or_code(): assert result == "**bold** and `a*b` see [here](https://x.example/a_b)", f"Got: {result!r}" +def test_adf_to_markdown_renders_non_text_inline_nodes(): + """Each non-text inline leaf node (mention/emoji/inlineCard/date/status) + renders its attrs-sourced displayable value instead of being dropped to an + empty string.""" + # Given a paragraph containing one of each non-text inline leaf type, with + # an underscore in the inlineCard URL (URLs must not be escaped) and an + # epoch-millisecond date timestamp for 2021-01-01 UTC + doc = { + "type": "doc", + "version": 1, + "content": [ + {"type": "paragraph", "content": [ + {"type": "mention", "attrs": {"id": "abc", "text": "@Marco Rizzi"}}, + {"type": "text", "text": " "}, + {"type": "emoji", "attrs": {"shortName": ":smile:", "text": "😄"}}, + {"type": "text", "text": " "}, + {"type": "inlineCard", "attrs": {"url": "https://example.com/a_b"}}, + {"type": "text", "text": " "}, + {"type": "date", "attrs": {"timestamp": "1609459200000"}}, + {"type": "text", "text": " "}, + {"type": "status", "attrs": {"text": "In Progress", "color": "yellow"}}, + ]}, + ], + } + # When rendering the ADF to markdown + result = execute_actions.adf_to_markdown(doc) + # Then every node contributes its attrs value (URL underscore preserved, date + # formatted as YYYY-MM-DD) and nothing is silently dropped + assert result == "@Marco Rizzi 😄 https://example.com/a_b 2021-01-01 In Progress", \ + f"Got: {result!r}" + + +def test_adf_to_markdown_renders_inline_nodes_in_task_item(): + """A taskItem whose inline content mixes text with a mention and an + inlineCard renders all of them — the inline nodes are not dropped in the + taskItem context.""" + # Given a TODO taskItem with inline mention and inlineCard nodes + doc = { + "type": "doc", + "version": 1, + "content": [ + {"type": "taskList", "content": [ + {"type": "taskItem", "attrs": {"state": "TODO"}, "content": [ + {"type": "text", "text": "ping "}, + {"type": "mention", "attrs": {"text": "@dev"}}, + {"type": "text", "text": " re "}, + {"type": "inlineCard", "attrs": {"url": "https://example.com/pr/1"}}, + ]}, + ]}, + ], + } + # When rendering the ADF to markdown + result = execute_actions.adf_to_markdown(doc) + # Then the checklist item retains the mention and inlineCard values + assert result == "- [ ] ping @dev re https://example.com/pr/1", f"Got: {result!r}" + + def test_execute_post_comment_routes_to_native(): """execute_post_comment resolves refs in body_adf, renders to markdown, and posts it.""" recorder = _RunRecorder() @@ -515,6 +572,8 @@ def fake_run(cmd, input=None, text=None, capture_output=None, env=None): test_adf_to_markdown_renders_task_list() test_adf_to_markdown_escapes_markdown_active_chars_in_literal_text() test_adf_to_markdown_does_not_double_escape_marks_or_code() + test_adf_to_markdown_renders_non_text_inline_nodes() + test_adf_to_markdown_renders_inline_nodes_in_task_item() test_execute_post_comment_routes_to_native() test_execute_post_report_posts_github_then_jira() test_execute_post_report_updates_existing_github_comment_on_retry() From fcacee5e0a4728f33d4f9bb57bfaa6504ed3e231 Mon Sep 17 00:00:00 2001 From: mrizzi Date: Tue, 1 Sep 2026 19:24:53 +0200 Subject: [PATCH 022/175] fix(verify-pr): constrain post_comment "issue" to Jira-key format in result schema MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The post_comment action schema accepted any non-empty string for "issue", but the executor post_jira_comment_native requires a hyphenated Jira key and sys.exit(1)s otherwise — a schema-valid {"issue":"12345"} (numeric ID, URL, or other non-key identifier) passed validation and then failed at execution, aborting the whole post_script run. Constrain the schema "issue" field to the Jira-key pattern ^[A-Z]+-[0-9]+$ (matching jira_issue_id/parent in the same schema) so non-key identifiers are rejected at the producer boundary. The executor rpartition fail-fast guard is kept as defence in depth. Add schema-validation tests asserting the pattern accepts a valid key and rejects non-key issues (numeric ID, URL, lowercase, missing hyphen/number/project). Implements TC-5953 Assisted-by: Claude Code Co-Authored-By: Claude Opus 4.8 --- .../schemas/verify-pr-result.schema.json | 2 +- .../scripts/test_execute_actions.py | 56 +++++++++++++++++++ 2 files changed, 57 insertions(+), 1 deletion(-) diff --git a/plugins/sdlc-workflow/schemas/verify-pr-result.schema.json b/plugins/sdlc-workflow/schemas/verify-pr-result.schema.json index aeaf9c365..7b346b11e 100644 --- a/plugins/sdlc-workflow/schemas/verify-pr-result.schema.json +++ b/plugins/sdlc-workflow/schemas/verify-pr-result.schema.json @@ -112,7 +112,7 @@ "required": ["type", "issue", "body_adf"], "properties": { "type": {}, - "issue": { "type": "string", "minLength": 1 }, + "issue": { "type": "string", "pattern": "^[A-Z]+-[0-9]+$" }, "body_adf": { "type": "object" } }, "additionalProperties": false diff --git a/plugins/sdlc-workflow/scripts/test_execute_actions.py b/plugins/sdlc-workflow/scripts/test_execute_actions.py index 5f4eec2bb..e12a90d34 100644 --- a/plugins/sdlc-workflow/scripts/test_execute_actions.py +++ b/plugins/sdlc-workflow/scripts/test_execute_actions.py @@ -6,6 +6,8 @@ import json import importlib.util +from jsonschema import validate, ValidationError + script_dir = os.path.dirname(os.path.abspath(__file__)) spec = importlib.util.spec_from_file_location( "execute_actions", @@ -16,6 +18,24 @@ resolve_refs = execute_actions.resolve_refs +# The result schema the fullsend validation_loop enforces before the post_script +# runs. Loaded once so the schema-validation tests below assert against the real +# shipped constraints rather than a reimplementation. +_RESULT_SCHEMA_PATH = os.path.join( + script_dir, "..", "schemas", "verify-pr-result.schema.json" +) +with open(_RESULT_SCHEMA_PATH) as _schema_f: + _RESULT_SCHEMA = json.load(_schema_f) + +# Validate a single action instance against the schema's action definition. The +# action def carries no external $refs, so wrapping it with the document's $defs +# and dialect lets `validate` exercise the post_comment if/then branch directly. +_ACTION_SCHEMA = { + "$schema": _RESULT_SCHEMA["$schema"], + "$defs": _RESULT_SCHEMA["$defs"], + "$ref": "#/$defs/action", +} + def test_resolve_refs_replaces_key(): """A single {{ref.key}}/{{ref.url}} pair resolves to registered values.""" @@ -195,6 +215,40 @@ def test_post_jira_comment_native_invalid_key_exits(): assert recorder.cmd is None, "CLI should not run for an invalid key" +def test_schema_post_comment_accepts_valid_jira_key(): + """The post_comment schema accepts a well-formed hyphenated Jira key so a + legitimate action still validates and routes to the native CLI.""" + # Given a post_comment action whose issue is a valid Jira key + action = {"type": "post_comment", "issue": "TC-5811", "body_adf": {}} + # When validating it against the result schema's action definition + # Then validation passes (validate raises ValidationError on failure) + validate(instance=action, schema=_ACTION_SCHEMA) + + +def test_schema_post_comment_rejects_non_key_issue(): + """A schema-valid-string-but-non-key issue (numeric ID, URL, lowercase, or + missing hyphen) is rejected at validation, so it can never pass the producer + boundary only to hit the executor's rpartition guard and sys.exit(1).""" + # Given post_comment actions whose issue is not a hyphenated Jira key + non_keys = [ + "12345", # numeric Jira ID + "https://jira.example.com/browse/TC-5811", # URL + "tc-5811", # lowercase project + "TC5811", # missing hyphen + "TC-", # missing number + "-5811", # missing project + ] + for issue in non_keys: + action = {"type": "post_comment", "issue": issue, "body_adf": {}} + # When validating each against the schema + # Then validation fails before the action can reach the executor + try: + validate(instance=action, schema=_ACTION_SCHEMA) + assert False, f"non-key issue should be rejected: {issue!r}" + except ValidationError: + pass + + def test_adf_to_markdown_renders_blocks_and_marks(): """adf_to_markdown renders headings, lists, code blocks, rules, and inline marks.""" doc = { @@ -568,6 +622,8 @@ def fake_run(cmd, input=None, text=None, capture_output=None, env=None): test_post_jira_comment_native_maps_env() test_post_jira_comment_native_nonzero_exits() test_post_jira_comment_native_invalid_key_exits() + test_schema_post_comment_accepts_valid_jira_key() + test_schema_post_comment_rejects_non_key_issue() test_adf_to_markdown_renders_blocks_and_marks() test_adf_to_markdown_renders_task_list() test_adf_to_markdown_escapes_markdown_active_chars_in_literal_text() From dac5676c4ad92a712cd9b2e2b02ddba4723f5e24 Mon Sep 17 00:00:00 2001 From: mrizzi Date: Tue, 1 Sep 2026 19:33:10 +0200 Subject: [PATCH 023/175] docs(verify-pr): document same-commit report retry-update in Step 9 MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Step 9 'Post to GitHub PR' claimed every verification run creates a new PR comment and never overwrites previous reports, and showed a plain `gh pr comment` create. That contradicted the shipped executor behavior (TC-5937): execute_post_report embeds a commit-scoped marker and, via _find_report_comment_id, PATCH-updates an existing same-commit report comment instead of duplicating it — a new comment is created only for a later commit. Rewrite Step 9 to match the code: describe the commit-scoped marker, the find-then-PATCH-or-create path (gh api --paginate --slurp + PATCH, else gh pr comment), and reframe the history claim as per-commit (a later commit gets a new comment) rather than per-run. Keep the (commit ) header convention. Add an eval assertion (evals/verify-pr/evals.json case 3) covering the commit-scoped report header / per-commit history. Documentation-only alignment; executor behavior is unchanged. Implements TC-5954 Assisted-by: Claude Code Co-Authored-By: Claude Opus 4.8 --- evals/verify-pr/evals.json | 1 + .../sdlc-workflow/skills/verify-pr/SKILL.md | 34 ++++++++++++++++--- 2 files changed, 30 insertions(+), 5 deletions(-) diff --git a/evals/verify-pr/evals.json b/evals/verify-pr/evals.json index 94fa96088..85367e887 100644 --- a/evals/verify-pr/evals.json +++ b/evals/verify-pr/evals.json @@ -57,6 +57,7 @@ "The skill reads and processes all PR review comments before generating findings (constraint 1.10)", "The report does NOT auto-merge the PR (constraint 1.13)", "The report contains a Test Change Classification row with ADDITIVE — only new test files were added (tests/api/sbom_delete.rs is a new file)", + "The verification report header for TC-9103 carries a commit-scoped identifier — `## Verification Report for TC-9103 (commit )`, or `(commit unknown)` when the eval sandbox provides no HEAD SHA — reflecting Step 9's per-commit report convention: each report is scoped to the commit it verifies, so a re-run on the same commit refreshes that commit's existing report comment while a later commit gets a new one (per-commit history, not a new comment on every run)", "Convention upgrade eligibility is evaluated for review comment 30002 (index suggestion) — the review classification output (review-30002.md) or the report's Style/Conventions analysis explains whether the suggestion matches a documented or demonstrated project convention", "Review comment 30002 (index suggestion) does NOT result in a sub-task — the suggestion classification is correct (suggestive language, no directive) and no project convention in the fixture data backs an upgrade from suggestion to code change request", "Eval Quality is N/A because no eval result reviews exist in the PR — the 3-criteria detection (author github-actions[bot], marker ## Eval Results, footer sdlc-workflow/run-evals) found no matches, so Eval Quality does not affect the Test Quality combination", diff --git a/plugins/sdlc-workflow/skills/verify-pr/SKILL.md b/plugins/sdlc-workflow/skills/verify-pr/SKILL.md index 3069fad70..f42c90c04 100644 --- a/plugins/sdlc-workflow/skills/verify-pr/SKILL.md +++ b/plugins/sdlc-workflow/skills/verify-pr/SKILL.md @@ -968,11 +968,22 @@ Store the full SHA and its short form (first 7 characters) for use in the report ### Post to GitHub PR -Each verification run creates a **new** PR comment (never overwrites previous reports). -This provides a verification history over time, with each report clearly referencing -the commit SHA it verified. +The report comment is **scoped to the commit it verifies**. Posting is idempotent +per commit: a re-run (or a retry after a partial failure) on the **same commit** +updates the existing report comment in place, while a **later commit** gets a fresh +comment. This preserves a **per-commit** verification history — one report comment +per commit, refreshed on re-runs — rather than a new comment on every run. -Update the report header from Step 8 to include the commit SHA: +To make this work, the report body embeds an invisible commit-scoped marker (the +mechanism GitHub lacks a native sticky-comment for): + +``` + +``` + +Update the report header from Step 8 to include the commit SHA — the +`(commit )` makes which commit each comment verifies legible in the PR +timeline: ``` ## Verification Report for (commit ) @@ -988,8 +999,21 @@ Append a markdown footnote at the end of the report body, separated by a horizon Read the plugin version from `plugins/sdlc-workflow/.claude-plugin/plugin.json` and substitute `{version}` before posting. +Build the comment body as the report (with the commit-scoped header and footnote) +followed by the commit marker, then post it via a find-then-update-or-create path +(as implemented by `_find_report_comment_id` + `execute_post_report` in +`scripts/execute-actions.py`): + ``` -gh pr comment --body "" -R +# Find an existing report comment for this commit, matched by the marker above. +# --slurp is required with --paginate so multi-page (>30) comment output is valid JSON. +gh api repos//issues//comments --paginate --slurp + +# If a comment carrying this commit's marker exists → update it in place: +gh api repos//issues/comments/ -X PATCH -f body="" + +# Otherwise (first report for this commit) → create a new comment: +gh pr comment --body "" -R ``` ### Post to Jira From 1bd5f4ac91cab089e9ed79d07e99708bf9049748 Mon Sep 17 00:00:00 2001 From: mrizzi Date: Wed, 2 Sep 2026 09:12:46 +0200 Subject: [PATCH 024/175] fix(verify-pr): accept {{ref.key}} placeholder in post_comment.issue schema TC-5953 tightened the post_comment 'issue' field to the Jira-key pattern '^[A-Z]+-[0-9]+$', but execute_post_comment resolves the field through resolve_refs, which supports the {{.key}} placeholder so a comment can target an issue created by an earlier action. Since fullsend's validation_loop validates the result schema before post_script runs, a placeholder issue was rejected before the reference could be resolved. Widen the pattern to '^([A-Z]+-[0-9]+|\\{\\{[a-z0-9-]+\\.key\\}\\})$' so it accepts either a literal Jira key or a {{.key}} placeholder, scoped to .key (not .url) since the field must resolve to a key. Bare numeric IDs, URLs, and other non-key/non-ref identifiers stay rejected (TC-5953's intent). Add a schema-validation test that a {{.key}} post_comment action passes jsonschema.validate, and extend the negative test with .url placeholders, uppercase refs, and unanchored/brace-less forms. Implements TC-5959 Assisted-by: Claude Code Co-Authored-By: Claude Opus 4.8 --- .../schemas/verify-pr-result.schema.json | 2 +- .../scripts/test_execute_actions.py | 28 ++++++++++++++++--- 2 files changed, 25 insertions(+), 5 deletions(-) diff --git a/plugins/sdlc-workflow/schemas/verify-pr-result.schema.json b/plugins/sdlc-workflow/schemas/verify-pr-result.schema.json index 7b346b11e..3c63813cb 100644 --- a/plugins/sdlc-workflow/schemas/verify-pr-result.schema.json +++ b/plugins/sdlc-workflow/schemas/verify-pr-result.schema.json @@ -112,7 +112,7 @@ "required": ["type", "issue", "body_adf"], "properties": { "type": {}, - "issue": { "type": "string", "pattern": "^[A-Z]+-[0-9]+$" }, + "issue": { "type": "string", "pattern": "^([A-Z]+-[0-9]+|\\{\\{[a-z0-9-]+\\.key\\}\\})$" }, "body_adf": { "type": "object" } }, "additionalProperties": false diff --git a/plugins/sdlc-workflow/scripts/test_execute_actions.py b/plugins/sdlc-workflow/scripts/test_execute_actions.py index e12a90d34..0b4159b87 100644 --- a/plugins/sdlc-workflow/scripts/test_execute_actions.py +++ b/plugins/sdlc-workflow/scripts/test_execute_actions.py @@ -225,11 +225,25 @@ def test_schema_post_comment_accepts_valid_jira_key(): validate(instance=action, schema=_ACTION_SCHEMA) +def test_schema_post_comment_accepts_ref_key_placeholder(): + """The post_comment schema accepts a {{.key}} placeholder so a comment + targeting an issue created by an earlier action (which execute_post_comment + resolves via resolve_refs) survives fullsend's validation_loop instead of + being rejected before the reference can be resolved.""" + # Given a post_comment action whose issue is a {{.key}} placeholder + action = {"type": "post_comment", "issue": "{{sub-1.key}}", "body_adf": {}} + # When validating it against the result schema's action definition + # Then validation passes (validate raises ValidationError on failure) + validate(instance=action, schema=_ACTION_SCHEMA) + + def test_schema_post_comment_rejects_non_key_issue(): - """A schema-valid-string-but-non-key issue (numeric ID, URL, lowercase, or - missing hyphen) is rejected at validation, so it can never pass the producer - boundary only to hit the executor's rpartition guard and sys.exit(1).""" - # Given post_comment actions whose issue is not a hyphenated Jira key + """A schema-valid-string-but-non-key issue (numeric ID, URL, lowercase, + missing hyphen, or a non-.key placeholder) is rejected at validation, so it + can never pass the producer boundary only to hit the executor's rpartition + guard and sys.exit(1).""" + # Given post_comment actions whose issue is neither a hyphenated Jira key + # nor a {{.key}} placeholder non_keys = [ "12345", # numeric Jira ID "https://jira.example.com/browse/TC-5811", # URL @@ -237,6 +251,11 @@ def test_schema_post_comment_rejects_non_key_issue(): "TC5811", # missing hyphen "TC-", # missing number "-5811", # missing project + "{{sub-1.url}}", # .url placeholder (not a key) + "{{SUB.key}}", # uppercase ref name + "{{sub-1.status}}", # unsupported placeholder attr + "sub-1.key", # missing braces + "prefix {{sub-1.key}}", # placeholder not anchored ] for issue in non_keys: action = {"type": "post_comment", "issue": issue, "body_adf": {}} @@ -623,6 +642,7 @@ def fake_run(cmd, input=None, text=None, capture_output=None, env=None): test_post_jira_comment_native_nonzero_exits() test_post_jira_comment_native_invalid_key_exits() test_schema_post_comment_accepts_valid_jira_key() + test_schema_post_comment_accepts_ref_key_placeholder() test_schema_post_comment_rejects_non_key_issue() test_adf_to_markdown_renders_blocks_and_marks() test_adf_to_markdown_renders_task_list() From c8c59a36e6ef155bdb9a35588c61ad0dc20d9c08 Mon Sep 17 00:00:00 2001 From: mrizzi Date: Wed, 2 Sep 2026 10:38:32 +0200 Subject: [PATCH 025/175] fix(verify-pr): guard _render_adf_date against out-of-range timestamps datetime.fromtimestamp can raise OverflowError/OSError (or ValueError) for an out-of-range epoch-ms value. The call sat outside the guarded try, so a single malformed ADF date node in a post_comment/post_report body aborted the entire execute-actions.py post_script and posted nothing. Move the fromtimestamp call inside the try and catch OverflowError/OSError alongside the int() parse errors, falling back to the literal timestamp string so the node degrades gracefully. Implements TC-5965 Assisted-by: Claude Code Co-Authored-By: Claude Opus 4.8 --- plugins/sdlc-workflow/scripts/execute-actions.py | 16 ++++++++++------ .../scripts/test_execute_actions.py | 16 ++++++++++++++++ 2 files changed, 26 insertions(+), 6 deletions(-) diff --git a/plugins/sdlc-workflow/scripts/execute-actions.py b/plugins/sdlc-workflow/scripts/execute-actions.py index 908d26a68..7cec164e0 100755 --- a/plugins/sdlc-workflow/scripts/execute-actions.py +++ b/plugins/sdlc-workflow/scripts/execute-actions.py @@ -110,16 +110,20 @@ def _render_adf_date(timestamp: Any) -> str: ADF ``date`` nodes store ``attrs.timestamp`` as a string of milliseconds since the Unix epoch. It is formatted as a UTC calendar date. If the value is - missing or not an integer, its literal form is returned so the node still - contributes its value instead of being dropped. + missing, not an integer, or outside the representable date range, its literal + form is returned so the node still contributes its value instead of aborting + the run. ``datetime.fromtimestamp`` can raise ``OverflowError`` or ``OSError`` + (platform-dependent) for out-of-range epoch values, so both are caught here + alongside the ``int()`` parse errors — an uncaught exception would otherwise + abort the entire post_script and post nothing. """ try: ms = int(timestamp) - except (TypeError, ValueError): + return datetime.datetime.fromtimestamp( + ms / 1000, tz=datetime.timezone.utc + ).strftime("%Y-%m-%d") + except (TypeError, ValueError, OverflowError, OSError): return str(timestamp) if timestamp else "" - return datetime.datetime.fromtimestamp( - ms / 1000, tz=datetime.timezone.utc - ).strftime("%Y-%m-%d") def _render_adf_inline(nodes: list) -> str: diff --git a/plugins/sdlc-workflow/scripts/test_execute_actions.py b/plugins/sdlc-workflow/scripts/test_execute_actions.py index 0b4159b87..e823a8fd5 100644 --- a/plugins/sdlc-workflow/scripts/test_execute_actions.py +++ b/plugins/sdlc-workflow/scripts/test_execute_actions.py @@ -429,6 +429,21 @@ def test_adf_to_markdown_renders_inline_nodes_in_task_item(): assert result == "- [ ] ping @dev re https://example.com/pr/1", f"Got: {result!r}" +def test_render_adf_date_falls_back_on_out_of_range_timestamp(): + """An integer-parseable but out-of-range epoch-ms timestamp renders as its + literal string instead of raising, so a single malformed date node cannot + abort the whole post_script. datetime.fromtimestamp raises OverflowError/OSError + (or ValueError) for out-of-range values, and that call now sits inside the + guarded try; before the fix it was outside and any such exception propagated.""" + # Given an integer-parseable epoch-millisecond value far outside the + # representable datetime range + out_of_range = "99999999999999999" + # When rendering it as an ADF date node + result = execute_actions._render_adf_date(out_of_range) + # Then the literal value is returned and no exception propagates + assert result == out_of_range, f"Got: {result!r}" + + def test_execute_post_comment_routes_to_native(): """execute_post_comment resolves refs in body_adf, renders to markdown, and posts it.""" recorder = _RunRecorder() @@ -650,6 +665,7 @@ def fake_run(cmd, input=None, text=None, capture_output=None, env=None): test_adf_to_markdown_does_not_double_escape_marks_or_code() test_adf_to_markdown_renders_non_text_inline_nodes() test_adf_to_markdown_renders_inline_nodes_in_task_item() + test_render_adf_date_falls_back_on_out_of_range_timestamp() test_execute_post_comment_routes_to_native() test_execute_post_report_posts_github_then_jira() test_execute_post_report_updates_existing_github_comment_on_retry() From 2302f644a5963fbb57b76c0396b8805f7771bca8 Mon Sep 17 00:00:00 2001 From: mrizzi Date: Wed, 2 Sep 2026 10:45:28 +0200 Subject: [PATCH 026/175] fix(verify-pr): fail fast on empty issue_type in create_issue create_issue declared issue_type: str = "" (reconciled under TC-5811 for the new description_adf/parent params), so an omitted value serialized to {"name": ""} and surfaced as an opaque Jira 400 instead of failing fast at the call site. Restore the prior fail-fast contract: reject an empty (or whitespace-only) issue_type with a clear stderr message and sys.exit(1) before any HTTP request, matching the module's existing error style. Added test_create_issue_fails_fast_on_empty_issue_type (asserts SystemExit and no request issued); jira-client suite 19 -> 20 passing. Implements TC-5966 Assisted-by: Claude Code Co-Authored-By: Claude Opus 4.8 --- plugins/sdlc-workflow/scripts/jira-client.py | 14 +++++++++ .../sdlc-workflow/scripts/test_jira_client.py | 31 +++++++++++++++++++ 2 files changed, 45 insertions(+) diff --git a/plugins/sdlc-workflow/scripts/jira-client.py b/plugins/sdlc-workflow/scripts/jira-client.py index 6d3566c65..0086db8e7 100755 --- a/plugins/sdlc-workflow/scripts/jira-client.py +++ b/plugins/sdlc-workflow/scripts/jira-client.py @@ -471,6 +471,20 @@ def create_issue( Returns: Created issue object with key and ID """ + # Fail fast on a missing issue type: an empty (or whitespace-only) value would + # otherwise serialize to {"name": ""} below and surface as an opaque Jira 400 + # instead of a clear programming error at the call site. + if not issue_type or not issue_type.strip(): + print( + "❌ Missing issue type: create_issue requires a non-empty issue_type", + file=sys.stderr, + ) + print( + "Pass an issue type ID (e.g., 10001) or name (e.g., Task, Sub-task).", + file=sys.stderr, + ) + sys.exit(1) + # Prefer a pre-rendered ADF description; fall back to converting markdown, # then to an empty document when neither is supplied. if description_adf is not None: diff --git a/plugins/sdlc-workflow/scripts/test_jira_client.py b/plugins/sdlc-workflow/scripts/test_jira_client.py index 2b957f589..e37431cda 100644 --- a/plugins/sdlc-workflow/scripts/test_jira_client.py +++ b/plugins/sdlc-workflow/scripts/test_jira_client.py @@ -556,6 +556,36 @@ def fake_make_request(method, endpoint, data=None): print("✓ create_issue fix_versions filters empty names test passed") +def test_create_issue_fails_fast_on_empty_issue_type(): + """Verifies create_issue with an empty issue_type fails fast without issuing a request.""" + called = {"made": False} + original_make_request = jira_client.make_request + + def fake_make_request(method, endpoint, data=None): + called["made"] = True + return {"key": "TEST-1", "id": "1"} + + jira_client.make_request = fake_make_request + try: + # When creating an issue with an empty issue_type + exit_code = None + try: + create_issue("TC", "Test", "desc", "") + raised = False + except SystemExit as e: + raised = True + exit_code = e.code + + # Then it fails fast (SystemExit 1) and issues no HTTP request + assert raised, "Expected create_issue to fail fast (SystemExit) on empty issue_type" + assert exit_code == 1, f"Expected exit code 1, got {exit_code}" + assert not called["made"], "Expected no HTTP request to be issued" + finally: + jira_client.make_request = original_make_request + + print("✓ create_issue fails fast on empty issue_type test passed") + + def run_all_tests(): """Run all tests and report results.""" tests = [ @@ -578,6 +608,7 @@ def run_all_tests(): test_create_issue_omits_fix_versions_when_none, test_create_issue_all_optional_fields_together, test_create_issue_fix_versions_filters_empty_names, + test_create_issue_fails_fast_on_empty_issue_type, ] failed = [] From b7c347049073d20d821bc3cfbd2a3849bc7133a6 Mon Sep 17 00:00:00 2001 From: mrizzi Date: Wed, 2 Sep 2026 10:53:08 +0200 Subject: [PATCH 027/175] fix(verify-pr): scope Jira sticky comment marker per purpose post_comment and post_report both posted via post_jira_comment_native with the single shared STICKY_COMMENT_MARKER, which the native fullsend CLI treats as the sticky-comment identity. Two comments targeting the same Jira issue in one run would share that marker, so the second post would overwrite the first and silently lose a comment. Thread a marker parameter through post_jira_comment_native (default preserves the report path's historical marker) and give post_comment a distinct POST_COMMENT_STICKY_MARKER. Each path keeps its own stable marker, so per-path re-run idempotency is preserved while cross-purpose clobbering is prevented. GitHub report dedup (GITHUB_REPORT_MARKER_PREFIX) is untouched. Added test_post_comment_and_report_use_distinct_sticky_markers; execute_actions suite 26 -> 27 passing. Implements TC-5967 Assisted-by: Claude Code Co-Authored-By: Claude Opus 4.8 --- .../sdlc-workflow/scripts/execute-actions.py | 26 ++++++--- .../scripts/test_execute_actions.py | 54 +++++++++++++++++++ 2 files changed, 73 insertions(+), 7 deletions(-) diff --git a/plugins/sdlc-workflow/scripts/execute-actions.py b/plugins/sdlc-workflow/scripts/execute-actions.py index 7cec164e0..59d7c5c33 100755 --- a/plugins/sdlc-workflow/scripts/execute-actions.py +++ b/plugins/sdlc-workflow/scripts/execute-actions.py @@ -47,9 +47,17 @@ REF_PATTERN = re.compile(r"\{\{([a-z0-9-]+)\.(key|url)\}\}") # Stable marker so the native sticky-comment CLI updates the same Jira -# comment across re-runs instead of posting duplicates. +# comment across re-runs instead of posting duplicates. Used by post_report +# (the verification report on the main task). STICKY_COMMENT_MARKER = "" +# Distinct sticky marker for standalone analysis comments (post_comment). The +# native CLI treats the marker as the sticky-comment identity, so a post_comment +# and a post_report targeting the SAME Jira issue in one run must not share one +# marker — otherwise the second post would overwrite the first. Each path keeps +# its own stable marker, so per-path re-run idempotency is preserved. +POST_COMMENT_STICKY_MARKER = "" + # GitHub has no native sticky-comment mechanism, so the verify-pr report comment # embeds this marker (an invisible HTML comment) in its body. The commit SHA is # appended per post, scoping dedup to a single verification run/commit: a retry @@ -294,12 +302,16 @@ def _jira_native_env() -> dict[str, str]: sys.exit(1) -def post_jira_comment_native(issue_key: str, body_md: str) -> None: +def post_jira_comment_native( + issue_key: str, body_md: str, marker: str = STICKY_COMMENT_MARKER +) -> None: """Post a Jira comment via the native fullsend sticky-comment CLI. - The body is passed as markdown on stdin (``--result -``). A stable marker - makes the post idempotent: re-runs update the existing comment rather than - creating duplicates. + The body is passed as markdown on stdin (``--result -``). The ``marker`` is + the sticky-comment identity: re-runs with the same marker update the existing + comment rather than creating duplicates. Callers pass a per-purpose marker so + two comments on the same issue (report vs analysis) do not clobber each other; + the default preserves the report path's historical marker. """ project, _, number = issue_key.rpartition("-") if not project or not number: @@ -309,7 +321,7 @@ def post_jira_comment_native(issue_key: str, body_md: str) -> None: result = subprocess.run( ["fullsend", "issues", "post-comment", "--tracker", "jira", "--project", project, "--number", number, - "--marker", STICKY_COMMENT_MARKER, "--result", "-"], + "--marker", marker, "--result", "-"], input=body_md, text=True, capture_output=True, env=_jira_native_env(), ) @@ -428,7 +440,7 @@ def execute_post_comment(action: dict, registry: dict) -> None: body_adf = resolve_refs_in_obj(action["body_adf"], registry) body_md = adf_to_markdown(body_adf) - post_jira_comment_native(issue, body_md) + post_jira_comment_native(issue, body_md, marker=POST_COMMENT_STICKY_MARKER) print(f" Posted comment on {issue}") diff --git a/plugins/sdlc-workflow/scripts/test_execute_actions.py b/plugins/sdlc-workflow/scripts/test_execute_actions.py index e823a8fd5..2baf2ce17 100644 --- a/plugins/sdlc-workflow/scripts/test_execute_actions.py +++ b/plugins/sdlc-workflow/scripts/test_execute_actions.py @@ -475,6 +475,59 @@ def test_execute_post_comment_routes_to_native(): f"Got: {recorder.input!r}" +def test_post_comment_and_report_use_distinct_sticky_markers(): + """A post_comment and a post_report pass DIFFERENT --marker values to the + native CLI, so two sticky comments on the same Jira issue never share one + marker identity and clobber each other.""" + # Given the comment path + comment_recorder = _RunRecorder() + restore = _with_jira_env_and_recorder(comment_recorder) + try: + execute_actions.execute_post_comment( + {"type": "post_comment", "issue": "TC-777", + "body_adf": {"type": "doc", "version": 1, "content": []}}, + {}, + ) + finally: + restore() + comment_marker = comment_recorder.cmd[comment_recorder.cmd.index("--marker") + 1] + + # And the report path (no existing GitHub comment → lists then posts) + report_calls = [] + + def fake_run(cmd, input=None, text=None, capture_output=None, env=None): + report_calls.append(cmd) + return _FakeCompleted(0, "", "[]") + + saved_run = execute_actions.subprocess.run + saved_env = {k: os.environ.get(k) for k in _JIRA_ENV} + execute_actions.subprocess.run = fake_run + os.environ.update(_JIRA_ENV) + try: + report = { + "pr_repo": "acme/widget", + "pr_number": 42, + "jira_issue_id": "TC-777", + "commit_sha": "946556e", + "report_md": "## Verify report", + } + execute_actions.execute_post_report({"type": "post_report"}, {}, report) + finally: + execute_actions.subprocess.run = saved_run + for k, v in saved_env.items(): + if v is None: + os.environ.pop(k, None) + else: + os.environ[k] = v + jira_cmd = next(c for c in report_calls if c[:3] == ["fullsend", "issues", "post-comment"]) + report_marker = jira_cmd[jira_cmd.index("--marker") + 1] + + # Then the two markers differ, and each matches its dedicated constant + assert comment_marker == execute_actions.POST_COMMENT_STICKY_MARKER + assert report_marker == execute_actions.STICKY_COMMENT_MARKER + assert comment_marker != report_marker, "post_comment and post_report must use distinct markers" + + def test_execute_post_report_posts_github_then_jira(): """execute_post_report lists PR comments, creates a marked GitHub comment when none exists, then posts to Jira.""" calls = [] @@ -667,6 +720,7 @@ def fake_run(cmd, input=None, text=None, capture_output=None, env=None): test_adf_to_markdown_renders_inline_nodes_in_task_item() test_render_adf_date_falls_back_on_out_of_range_timestamp() test_execute_post_comment_routes_to_native() + test_post_comment_and_report_use_distinct_sticky_markers() test_execute_post_report_posts_github_then_jira() test_execute_post_report_updates_existing_github_comment_on_retry() test_execute_post_report_updates_comment_on_later_page() From 9f5b335d942e5b6a4bc284de52c0ed1a3cbabe73 Mon Sep 17 00:00:00 2001 From: mrizzi Date: Wed, 2 Sep 2026 10:54:35 +0200 Subject: [PATCH 028/175] fix(verify-pr): normalize commit_sha in GitHub report dedup marker execute_post_report embedded the raw commit_sha in the GitHub dedup marker and _find_report_comment_id matched it by substring. The result schema permits commit_sha to be 7-40 hex chars, so a run recording a full SHA and a retry recording an abbreviation (or vice-versa) for the same commit produced different marker strings; the substring lookup missed and the retry posted a duplicate report comment, defeating the TC-5937 idempotency guarantee. Normalize the SHA to the schema-minimum 7 chars in the marker (and thus the lookup, which reuses the same marker). Git abbreviations are always prefixes of the full SHA, so first-7 is the only fixed length that unifies a full SHA with any valid abbreviation of the same commit; a longer length (e.g. 12) would leave a 7-char abbreviation unchanged and still mismatch a full SHA. Added test_execute_post_report_dedup_marker_invariant_to_sha_length (full-SHA create then abbreviated-SHA retry -> single PATCH-updated comment); execute_actions suite 27 -> 28 passing. Implements TC-5968 Assisted-by: Claude Code Co-Authored-By: Claude Opus 4.8 --- .../sdlc-workflow/scripts/execute-actions.py | 36 +++++++++--- .../scripts/test_execute_actions.py | 57 +++++++++++++++++++ 2 files changed, 85 insertions(+), 8 deletions(-) diff --git a/plugins/sdlc-workflow/scripts/execute-actions.py b/plugins/sdlc-workflow/scripts/execute-actions.py index 59d7c5c33..768f44a8a 100755 --- a/plugins/sdlc-workflow/scripts/execute-actions.py +++ b/plugins/sdlc-workflow/scripts/execute-actions.py @@ -59,13 +59,32 @@ POST_COMMENT_STICKY_MARKER = "" # GitHub has no native sticky-comment mechanism, so the verify-pr report comment -# embeds this marker (an invisible HTML comment) in its body. The commit SHA is -# appended per post, scoping dedup to a single verification run/commit: a retry -# for the same commit updates the existing comment instead of duplicating it, -# while a later commit gets a fresh comment — preserving the per-run -# verification history that verify-pr SKILL.md Step 9 posts. +# embeds this marker (an invisible HTML comment) in its body. A length-normalized +# commit SHA is appended per post, scoping dedup to a single verification +# run/commit: a retry for the same commit updates the existing comment instead of +# duplicating it, while a later commit gets a fresh comment — preserving the +# per-run verification history that verify-pr SKILL.md Step 9 posts. GITHUB_REPORT_MARKER_PREFIX = "" + marker = f"{GITHUB_REPORT_MARKER_PREFIX}{_normalize_commit_sha(commit_sha)} -->" github_body = f"{report_md}\n\n{marker}" existing_id = _find_report_comment_id(repo, pr_number, marker) diff --git a/plugins/sdlc-workflow/scripts/test_execute_actions.py b/plugins/sdlc-workflow/scripts/test_execute_actions.py index 2baf2ce17..02a2a96d0 100644 --- a/plugins/sdlc-workflow/scripts/test_execute_actions.py +++ b/plugins/sdlc-workflow/scripts/test_execute_actions.py @@ -677,6 +677,62 @@ def fake_run(cmd, input=None, text=None, capture_output=None, env=None): assert patch_calls[0]["cmd"][2] == "repos/acme/widget/issues/comments/555" +def test_execute_post_report_dedup_marker_invariant_to_sha_length(): + """A full-length SHA on one run and an abbreviated SHA for the same commit on + a retry resolve to the SAME report comment (PATCH-update, not duplicate), + because the dedup marker normalizes the SHA to a canonical length.""" + full_sha = "946556e" + "a" * 33 # 40 hex chars + short_sha = "946556e" # 7-char abbreviation of the same commit + + def _run_report(commit_sha, existing_comments): + """Run execute_post_report; return (calls, created_body_or_None).""" + calls = [] + + def fake_run(cmd, input=None, text=None, capture_output=None, env=None): + calls.append({"cmd": cmd, "input": input}) + if cmd[:2] == ["gh", "api"] and cmd[2].endswith("/comments"): + return _FakeCompleted(0, "", json.dumps([existing_comments])) + return _FakeCompleted(0, "") + + saved_run = execute_actions.subprocess.run + saved_env = {k: os.environ.get(k) for k in _JIRA_ENV} + execute_actions.subprocess.run = fake_run + os.environ.update(_JIRA_ENV) + try: + report = { + "pr_repo": "acme/widget", + "pr_number": 42, + "jira_issue_id": "TC-777", + "commit_sha": commit_sha, + "report_md": "## Verify report", + } + execute_actions.execute_post_report({"type": "post_report"}, {}, report) + finally: + execute_actions.subprocess.run = saved_run + for k, v in saved_env.items(): + if v is None: + os.environ.pop(k, None) + else: + os.environ[k] = v + return calls + + # First run with the FULL SHA and no existing comment → a comment is created; + # capture the marker it embedded. + first_calls = _run_report(full_sha, []) + create_call = next(c for c in first_calls if c["cmd"][:3] == ["gh", "pr", "comment"]) + created_body = create_call["cmd"][create_call["cmd"].index("--body") + 1] + + # Retry with the ABBREVIATED SHA; the PR already has the comment created above. + retry_calls = _run_report(short_sha, [{"id": 555, "body": created_body}]) + + # Then the retry PATCH-updates the existing comment instead of duplicating it. + assert not any(c["cmd"][:3] == ["gh", "pr", "comment"] for c in retry_calls), \ + "abbreviated-SHA retry must not create a duplicate report comment" + patch_calls = [c for c in retry_calls if "PATCH" in c["cmd"]] + assert len(patch_calls) == 1, f"Expected one PATCH update, got {len(patch_calls)}" + assert patch_calls[0]["cmd"][2] == "repos/acme/widget/issues/comments/555" + + def test_find_report_comment_id_exits_on_unparseable_json(): """A JSON parse failure aborts with sys.exit(1) instead of silently returning None (which would let a retry create a duplicate report comment).""" @@ -724,5 +780,6 @@ def fake_run(cmd, input=None, text=None, capture_output=None, env=None): test_execute_post_report_posts_github_then_jira() test_execute_post_report_updates_existing_github_comment_on_retry() test_execute_post_report_updates_comment_on_later_page() + test_execute_post_report_dedup_marker_invariant_to_sha_length() test_find_report_comment_id_exits_on_unparseable_json() print("All tests passed.") From 1a25fe395a8f5ef41b1e188ac42b3d1d296ed3a0 Mon Sep 17 00:00:00 2001 From: mrizzi Date: Wed, 2 Sep 2026 10:58:27 +0200 Subject: [PATCH 029/175] fix(verify-pr): resolve report_md refs independent of action order execute_post_report calls resolve_refs on report_md, which raises an uncaught KeyError for any {{ref.key}} placeholder whose entity has not yet been registered. The registry is populated by create_subtask / create_root_cause_task as main()'s sequential loop runs, so a post_report ordered before an action it references would abort the whole run with a bare KeyError. verify-pr emits post_report last today, but that ordering was undocumented and unenforced. Defer every post_report until after the actions loop in main() so the registry is fully populated before any report_md is resolved. post_report is still a recognized action; only its execution moves to the end. Observed behavior is unchanged for the always-last emission order. Added test_post_report_resolves_ref_created_by_later_action, which drives main() with a temp result JSON that orders post_report before the create_subtask its report_md interpolates and asserts the resolved key reaches the GitHub report body; execute_actions suite 28 -> 29 passing. Implements TC-5969 Assisted-by: Claude Code Co-Authored-By: Claude Opus 4.8 --- .../sdlc-workflow/scripts/execute-actions.py | 15 +++- .../scripts/test_execute_actions.py | 71 +++++++++++++++++++ 2 files changed, 85 insertions(+), 1 deletion(-) diff --git a/plugins/sdlc-workflow/scripts/execute-actions.py b/plugins/sdlc-workflow/scripts/execute-actions.py index 768f44a8a..cf852fa15 100755 --- a/plugins/sdlc-workflow/scripts/execute-actions.py +++ b/plugins/sdlc-workflow/scripts/execute-actions.py @@ -570,18 +570,31 @@ def main(): print(f"Executing {len(actions)} actions for {report['jira_issue_id']}...") + # post_report resolves {{ref.key}} placeholders in report_md against the + # registry, which the entity-creating actions (create_subtask, + # create_root_cause_task) populate as they run. Defer every post_report until + # after the actions loop so the registry is fully populated first — otherwise + # a post_report ordered before an action it references would raise an + # uncaught KeyError from resolve_refs and abort the run. verify-pr already + # emits post_report last, so this only removes an undocumented, + # order-dependent trap; it does not change the observed behavior. + deferred_reports: list[dict] = [] + for i, action in enumerate(actions): action_type = action["type"] print(f"[{i + 1}/{len(actions)}] {action_type}") if action_type == "post_report": - execute_post_report(action, registry, report) + deferred_reports.append(action) elif action_type in EXECUTORS: EXECUTORS[action_type](action, registry) else: print(f" Unknown action type: {action_type}", file=sys.stderr) sys.exit(1) + for action in deferred_reports: + execute_post_report(action, registry, report) + print(f"Done. {len(actions)} actions executed successfully.") diff --git a/plugins/sdlc-workflow/scripts/test_execute_actions.py b/plugins/sdlc-workflow/scripts/test_execute_actions.py index 02a2a96d0..41a213782 100644 --- a/plugins/sdlc-workflow/scripts/test_execute_actions.py +++ b/plugins/sdlc-workflow/scripts/test_execute_actions.py @@ -4,6 +4,7 @@ import sys import os import json +import tempfile import importlib.util from jsonschema import validate, ValidationError @@ -733,6 +734,75 @@ def fake_run(cmd, input=None, text=None, capture_output=None, env=None): assert patch_calls[0]["cmd"][2] == "repos/acme/widget/issues/comments/555" +def test_post_report_resolves_ref_created_by_later_action(): + """report_md refs resolve regardless of action ordering: a post_report ordered + BEFORE the create_subtask it references still resolves, because main() defers + every post_report until after the actions loop populates the registry.""" + calls = [] + + def fake_run(cmd, input=None, text=None, capture_output=None, env=None): + calls.append({"cmd": cmd, "input": input}) + if cmd[:2] == ["gh", "api"] and cmd[2].endswith("/comments"): + return _FakeCompleted(0, "", "[]") # no existing report comment + return _FakeCompleted(0, "") + + def fake_create_issue(**kwargs): + return {"key": "TC-999"} + + # Actions deliberately order post_report FIRST, then the create_subtask whose + # ref its report_md interpolates — the pre-fix inline order would KeyError. + data = { + "report": { + "pr_repo": "acme/widget", + "pr_number": 42, + "jira_issue_id": "TC-777", + "commit_sha": "946556e", + "report_md": "Filed sub-task {{sub-1.key}}.", + }, + "actions": [ + {"type": "post_report"}, + { + "type": "create_subtask", + "ref": "sub-1", + "parent": "TC-100", + "summary": "A sub-task", + "labels": ["review-feedback"], + "description_adf": {"type": "doc", "version": 1, "content": []}, + }, + ], + } + + saved_run = execute_actions.subprocess.run + saved_create = execute_actions._jira_mod.create_issue + saved_argv = sys.argv + saved_env = {k: os.environ.get(k) for k in _JIRA_ENV} + execute_actions.subprocess.run = fake_run + execute_actions._jira_mod.create_issue = fake_create_issue + os.environ.update(_JIRA_ENV) + fd, path = tempfile.mkstemp(suffix=".json") + try: + with os.fdopen(fd, "w") as f: + json.dump(data, f) + sys.argv = ["execute-actions.py", path] + execute_actions.main() + finally: + execute_actions.subprocess.run = saved_run + execute_actions._jira_mod.create_issue = saved_create + sys.argv = saved_argv + os.remove(path) + for k, v in saved_env.items(): + if v is None: + os.environ.pop(k, None) + else: + os.environ[k] = v + + # The GitHub report body carries the resolved key, not the raw placeholder. + create_call = next(c for c in calls if c["cmd"][:3] == ["gh", "pr", "comment"]) + gh_body = create_call["cmd"][create_call["cmd"].index("--body") + 1] + assert "Filed sub-task TC-999." in gh_body, f"ref not resolved: {gh_body}" + assert "{{sub-1.key}}" not in gh_body, "placeholder leaked into report body" + + def test_find_report_comment_id_exits_on_unparseable_json(): """A JSON parse failure aborts with sys.exit(1) instead of silently returning None (which would let a retry create a duplicate report comment).""" @@ -781,5 +851,6 @@ def fake_run(cmd, input=None, text=None, capture_output=None, env=None): test_execute_post_report_updates_existing_github_comment_on_retry() test_execute_post_report_updates_comment_on_later_page() test_execute_post_report_dedup_marker_invariant_to_sha_length() + test_post_report_resolves_ref_created_by_later_action() test_find_report_comment_id_exits_on_unparseable_json() print("All tests passed.") From 931466a4103173d25af15dc6843244f914cc56a5 Mon Sep 17 00:00:00 2001 From: mrizzi Date: Wed, 2 Sep 2026 11:00:07 +0200 Subject: [PATCH 030/175] fix(verify-pr): render ADF table/panel/blockquote/media blocks MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit _render_adf_block handled paragraph/heading/rule/codeBlock/bullet+ordered list/taskList; every other block hit the unknown-block fallback that recurses into content and flattens structure. So table cells were concatenated with no grid, blockquote/panel lost their framing, and media/mediaSingle/mediaGroup (which carry no text content) were dropped entirely. A schema-valid post_comment.body_adf can carry any of these, so a comment silently lost content — the same silent-degradation class already closed for taskList (TC-5936) and inline leaf nodes (TC-5950). Add explicit renderers: blockquote/panel -> "> "-prefixed lines (panelType surfaced as a bold label), table -> a GitHub-flavored markdown table (first tableRow is the header, pipes in cells escaped), and media/mediaSingle/ mediaGroup -> an image link or a non-empty [alt] placeholder, never dropped. The recursing fallback is kept for genuinely unknown container nodes, and the adf_to_markdown docstring's node-set contract now lists the covered nodes. Added rendering tests for table, blockquote+panel, and media, plus one asserting an unknown block still degrades via the fallback; execute_actions suite 29 -> 33 passing. Implements TC-5970 Assisted-by: Claude Code Co-Authored-By: Claude Opus 4.8 --- .../sdlc-workflow/scripts/execute-actions.py | 85 ++++++++++++++++- .../scripts/test_execute_actions.py | 93 +++++++++++++++++++ 2 files changed, 177 insertions(+), 1 deletion(-) diff --git a/plugins/sdlc-workflow/scripts/execute-actions.py b/plugins/sdlc-workflow/scripts/execute-actions.py index cf852fa15..3d2d4e1df 100755 --- a/plugins/sdlc-workflow/scripts/execute-actions.py +++ b/plugins/sdlc-workflow/scripts/execute-actions.py @@ -247,6 +247,77 @@ def _render_adf_task_list(node: dict) -> str: return "\n".join(lines) +def _render_adf_blockquote(node: dict) -> str: + """Render an ADF ``blockquote``/``panel`` as markdown ``> ``-prefixed lines. + + The framing children are rendered as ordinary blocks and every resulting line + (blank lines included, so the quote stays contiguous) gets the ``> `` prefix. + A ``panel``'s ``attrs.panelType`` (e.g. ``info``/``warning``) is emitted as a + leading bold label so the panel's kind is not lost in the markdown, which has + no native panel construct. + """ + inner = _render_adf_blocks(node.get("content", [])) + lines = inner.split("\n") if inner else [""] + panel_type = node.get("attrs", {}).get("panelType") + if panel_type: + lines = [f"**{panel_type}**", ""] + lines + return "\n".join(f"> {line}".rstrip() for line in lines) + + +def _render_adf_table_cell(cell: dict) -> str: + """Render one ADF ``tableCell``/``tableHeader`` to a single GFM table cell. + + Cell content is block-level (usually a paragraph), but a GFM cell must be one + line, so inter-block newlines are collapsed to spaces and any literal pipe is + escaped so it does not break the column structure. + """ + text = _render_adf_blocks(cell.get("content", [])) + return text.replace("\n", " ").replace("|", "\\|").strip() + + +def _render_adf_table(node: dict) -> str: + """Render an ADF ``table`` to a GitHub-flavored markdown table. + + The first ``tableRow`` is treated as the header row (followed by the ``---`` + separator); remaining rows are body rows. The separator is sized to the header + column count so the table is well-formed even when body rows are ragged. + """ + rows = [row for row in node.get("content", []) if row.get("type") == "tableRow"] + if not rows: + return "" + rendered = [ + [_render_adf_table_cell(cell) for cell in row.get("content", [])] + for row in rows + ] + header = rendered[0] + lines = [ + "| " + " | ".join(header) + " |", + "| " + " | ".join(["---"] * len(header)) + " |", + ] + for body_row in rendered[1:]: + lines.append("| " + " | ".join(body_row) + " |") + return "\n".join(lines) + + +def _render_adf_media(node: dict) -> str: + """Render an ADF ``media``/``mediaSingle``/``mediaGroup`` node. + + A ``mediaSingle``/``mediaGroup`` wraps one or more ``media`` children; each is + rendered as a markdown image ``![alt](url)`` when a URL attr is present, or a + non-empty ``[alt]`` placeholder otherwise — never dropped silently. Attachment + media (``type: file`` with only an ``id``) has no dereferenceable URL here, so + it falls back to the alt/id placeholder. + """ + if node.get("type") in ("mediaSingle", "mediaGroup"): + parts = [_render_adf_media(child) for child in node.get("content", []) + if child.get("type") == "media"] + return "\n".join(part for part in parts if part) + attrs = node.get("attrs", {}) + url = attrs.get("url", "") + alt = attrs.get("alt") or attrs.get("id") or "media" + return f"![{alt}]({url})" if url else f"[{alt}]" + + def _render_adf_block(node: dict) -> str: """Render a single ADF block node to markdown.""" node_type = node.get("type") @@ -267,6 +338,12 @@ def _render_adf_block(node: dict) -> str: return _render_adf_list(node, ordered=True) if node_type == "taskList": return _render_adf_task_list(node) + if node_type in ("blockquote", "panel"): + return _render_adf_blockquote(node) + if node_type == "table": + return _render_adf_table(node) + if node_type in ("media", "mediaSingle", "mediaGroup"): + return _render_adf_media(node) # Unknown block — recurse into nested content. return _render_adf_blocks(node.get("content", [])) @@ -286,7 +363,13 @@ def adf_to_markdown(doc: dict) -> str: produces: headings, paragraphs, bullet/ordered lists, code blocks, rules, and the strong/em/code/link inline marks. taskList/taskItem nodes (which agents may emit directly in ``body_adf``) render as markdown checklists (``- [ ]`` / - ``- [x]``). + ``- [x]``). Additional block nodes an agent may emit in ``body_adf`` are also + rendered rather than flattened: ``blockquote``/``panel`` as ``> `` quotes, + ``table`` as a GitHub-flavored markdown table, and ``media``/``mediaSingle``/ + ``mediaGroup`` as an image link or ``[alt]`` placeholder. Non-text inline leaf + nodes (``mention``/``emoji``/``status``/``inlineCard``/``date``) are rendered + from their ``attrs``. Genuinely unknown container nodes still degrade + gracefully by recursing into their nested content. """ if not isinstance(doc, dict): raise TypeError("adf_to_markdown expects an ADF document object") diff --git a/plugins/sdlc-workflow/scripts/test_execute_actions.py b/plugins/sdlc-workflow/scripts/test_execute_actions.py index 41a213782..00ea2adcf 100644 --- a/plugins/sdlc-workflow/scripts/test_execute_actions.py +++ b/plugins/sdlc-workflow/scripts/test_execute_actions.py @@ -430,6 +430,95 @@ def test_adf_to_markdown_renders_inline_nodes_in_task_item(): assert result == "- [ ] ping @dev re https://example.com/pr/1", f"Got: {result!r}" +def test_adf_to_markdown_renders_table(): + """A table renders as a GFM table: first tableRow is the header (with a --- + separator), tableCell/tableHeader content is rendered, and literal pipes in a + cell are escaped so they do not break the column grid.""" + doc = { + "type": "doc", + "version": 1, + "content": [ + {"type": "table", "content": [ + {"type": "tableRow", "content": [ + {"type": "tableHeader", "content": [ + {"type": "paragraph", "content": [{"type": "text", "text": "Name"}]}]}, + {"type": "tableHeader", "content": [ + {"type": "paragraph", "content": [{"type": "text", "text": "Note"}]}]}, + ]}, + {"type": "tableRow", "content": [ + {"type": "tableCell", "content": [ + {"type": "paragraph", "content": [{"type": "text", "text": "a"}]}]}, + {"type": "tableCell", "content": [ + {"type": "paragraph", "content": [{"type": "text", "text": "b|c"}]}]}, + ]}, + ]}, + ], + } + result = execute_actions.adf_to_markdown(doc) + assert result == ( + "| Name | Note |\n" + "| --- | --- |\n" + "| a | b\\|c |" + ), f"Got: {result!r}" + + +def test_adf_to_markdown_renders_blockquote_and_panel(): + """blockquote renders as > -prefixed lines; a panel renders as a quote with a + bold panelType label so its kind is preserved.""" + doc = { + "type": "doc", + "version": 1, + "content": [ + {"type": "blockquote", "content": [ + {"type": "paragraph", "content": [{"type": "text", "text": "quoted"}]}]}, + {"type": "panel", "attrs": {"panelType": "info"}, "content": [ + {"type": "paragraph", "content": [{"type": "text", "text": "heads up"}]}]}, + ], + } + result = execute_actions.adf_to_markdown(doc) + assert result == ( + "> quoted\n" + "\n" + "> **info**\n" + ">\n" + "> heads up" + ), f"Got: {result!r}" + + +def test_adf_to_markdown_renders_media_instead_of_dropping(): + """media/mediaSingle render a non-empty image link (or [alt] placeholder when + no URL is present) rather than being silently dropped — media nodes have no + text content, so the pre-fix flattening fallback emitted nothing.""" + doc = { + "type": "doc", + "version": 1, + "content": [ + {"type": "mediaSingle", "content": [ + {"type": "media", "attrs": {"url": "https://ex.com/i.png", "alt": "chart"}}]}, + {"type": "mediaSingle", "content": [ + {"type": "media", "attrs": {"type": "file", "id": "abc-123"}}]}, + ], + } + result = execute_actions.adf_to_markdown(doc) + assert result == "![chart](https://ex.com/i.png)\n\n[abc-123]", f"Got: {result!r}" + + +def test_adf_to_markdown_unknown_block_still_flattens(): + """A genuinely unknown container block still degrades gracefully by recursing + into its nested content (the preserved fallback), so the fix does not regress + forward-compat handling of block nodes it does not explicitly cover.""" + doc = { + "type": "doc", + "version": 1, + "content": [ + {"type": "someFutureBlock", "content": [ + {"type": "paragraph", "content": [{"type": "text", "text": "still here"}]}]}, + ], + } + result = execute_actions.adf_to_markdown(doc) + assert result == "still here", f"Got: {result!r}" + + def test_render_adf_date_falls_back_on_out_of_range_timestamp(): """An integer-parseable but out-of-range epoch-ms timestamp renders as its literal string instead of raising, so a single malformed date node cannot @@ -844,6 +933,10 @@ def fake_run(cmd, input=None, text=None, capture_output=None, env=None): test_adf_to_markdown_does_not_double_escape_marks_or_code() test_adf_to_markdown_renders_non_text_inline_nodes() test_adf_to_markdown_renders_inline_nodes_in_task_item() + test_adf_to_markdown_renders_table() + test_adf_to_markdown_renders_blockquote_and_panel() + test_adf_to_markdown_renders_media_instead_of_dropping() + test_adf_to_markdown_unknown_block_still_flattens() test_render_adf_date_falls_back_on_out_of_range_timestamp() test_execute_post_comment_routes_to_native() test_post_comment_and_report_use_distinct_sticky_markers() From b399dd9923e92920620088b6880d853f24cc7cbc Mon Sep 17 00:00:00 2001 From: mrizzi Date: Wed, 2 Sep 2026 11:51:48 +0200 Subject: [PATCH 031/175] fix(verify-pr): escape line-leading markdown block markers in ADF text MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit _escape_markdown only neutralized inline-active characters (` * _ [ ]); line-leading block markers were left intact. Because the native fullsend CLI re-parses the emitted markdown, a paragraph whose literal text starts with a heading (# ), bullet (- / + ), blockquote (>) or ordered-list (N. / N)) marker was reinterpreted as that block instead of rendering verbatim — a schema-valid post_comment body_adf could silently change structure. Same class as the inline-escaping gap closed for TC-5949. Add _escape_line_leading_markers (regex-driven, per line so post-hardBreak lines are covered) and apply it to fully-rendered paragraph content — the only place literal text sits at a true line start. Each alternative matches only where the construct actually forms (heading/list/ordered require a trailing space or EOL), so non-markers like "-5" or "1.5" are untouched; ordered lists escape the . / ) separator since a backslash before a digit is not a valid escape. Intentional heading prefixes and list bullets are emitted by other code paths and never flow through the escaper. * bullets need no handling — _escape_markdown already escapes * everywhere. Added tests for each marker, for a marker after a hardBreak, and for midline / non-marker text (and a real heading) staying unescaped; execute_actions suite 33 -> 36 passing (90 -> 93 overall). Implements TC-5971 Assisted-by: Claude Code Co-Authored-By: Claude Opus 4.8 --- .../sdlc-workflow/scripts/execute-actions.py | 54 +++++++++++- .../scripts/test_execute_actions.py | 85 +++++++++++++++++++ 2 files changed, 138 insertions(+), 1 deletion(-) diff --git a/plugins/sdlc-workflow/scripts/execute-actions.py b/plugins/sdlc-workflow/scripts/execute-actions.py index 3d2d4e1df..13425ba2f 100755 --- a/plugins/sdlc-workflow/scripts/execute-actions.py +++ b/plugins/sdlc-workflow/scripts/execute-actions.py @@ -126,12 +126,64 @@ def _escape_markdown(text: str) -> str: mark syntax it intentionally adds (``**``, backticks, ``[]()``), so literal text round-trips faithfully without double-escaping the marks. Not applied to inline-code content, which the CLI treats literally. + + Only *inline* markers are handled here; line-leading block markers (headings, + lists, blockquotes) are neutralized separately by + ``_escape_line_leading_markers`` because they are position-sensitive. """ for ch in _MARKDOWN_ESCAPE_CHARS: text = text.replace(ch, f"\\{ch}") return text +# Line-leading Markdown block markers the native CLI reinterprets when a rendered +# line *starts* with one: ATX headings (``#``..``######`` + space/EOL), blockquotes +# (``>``), bullet lists (``-``/``+`` + space/EOL) and ordered lists (``N.``/``N)`` + +# space/EOL). ``*`` bullets need no entry: ``_escape_markdown`` already escapes ``*`` +# everywhere. Each alternative only matches where the construct actually forms, so +# non-markers like ``-5`` or ``1.5`` are left untouched. +_LINE_LEADING_MARKER_RE = re.compile( + r"""^(?P[ \t]*) + (?: + (?P\#{1,6})(?=\s|$) + | (?P>) + | (?P[-+])(?=\s|$) + | (?P\d+)(?P[.)])(?=\s|$) + ) + """, + re.VERBOSE, +) + + +def _escape_line_leading_markers(text: str) -> str: + """Backslash-escape a Markdown block marker at the start of each line. + + Applied to fully-rendered *paragraph* content (the only place literal text + sits at a true line start, and where a leading ``#``/``-``/``>``/``N.`` would + otherwise be re-parsed by the native CLI as a heading/list/blockquote). Runs + per line so markers after a hardBreak are covered too. Intentional block + markers the renderer itself emits (heading prefixes, list bullets) are added + by other code paths and never flow through here, so they are not affected. + + A backslash before ASCII punctuation renders as that punctuation, so an + escape is visually invisible; for ordered lists the ``.``/``)`` separator is + escaped (``1\\.``) since a backslash before the digit is not a valid escape. + """ + def _escape_line(line: str) -> str: + match = _LINE_LEADING_MARKER_RE.match(line) + if not match: + return line + rest = line[match.end():] + if match.group("ordsep") is not None: + marker = f"{match.group('ordnum')}\\{match.group('ordsep')}" + else: + token = match.group("heading") or match.group("quote") or match.group("bullet") + marker = f"\\{token}" + return f"{match.group('indent')}{marker}{rest}" + + return "\n".join(_escape_line(line) for line in text.split("\n")) + + def _render_adf_date(timestamp: Any) -> str: """Render an ADF ``date`` node's timestamp as a readable ``YYYY-MM-DD`` date. @@ -322,7 +374,7 @@ def _render_adf_block(node: dict) -> str: """Render a single ADF block node to markdown.""" node_type = node.get("type") if node_type == "paragraph": - return _render_adf_inline(node.get("content", [])) + return _escape_line_leading_markers(_render_adf_inline(node.get("content", []))) if node_type == "heading": level = node.get("attrs", {}).get("level", 1) return f"{'#' * level} {_render_adf_inline(node.get('content', []))}" diff --git a/plugins/sdlc-workflow/scripts/test_execute_actions.py b/plugins/sdlc-workflow/scripts/test_execute_actions.py index 00ea2adcf..461feefe1 100644 --- a/plugins/sdlc-workflow/scripts/test_execute_actions.py +++ b/plugins/sdlc-workflow/scripts/test_execute_actions.py @@ -373,6 +373,88 @@ def test_adf_to_markdown_does_not_double_escape_marks_or_code(): assert result == "**bold** and `a*b` see [here](https://x.example/a_b)", f"Got: {result!r}" +def test_adf_to_markdown_escapes_line_leading_block_markers(): + """A paragraph whose literal text begins with a Markdown block marker + (heading/bullet/blockquote/ordered-list) has that marker backslash-escaped so + the native CLI renders it verbatim instead of re-parsing it as a block.""" + # Given paragraphs each starting with a different line-leading block marker + doc = { + "type": "doc", + "version": 1, + "content": [ + {"type": "paragraph", "content": [{"type": "text", "text": "# not a heading"}]}, + {"type": "paragraph", "content": [{"type": "text", "text": "### also not"}]}, + {"type": "paragraph", "content": [{"type": "text", "text": "- not a bullet"}]}, + {"type": "paragraph", "content": [{"type": "text", "text": "+ not a bullet"}]}, + {"type": "paragraph", "content": [{"type": "text", "text": "> not a quote"}]}, + {"type": "paragraph", "content": [{"type": "text", "text": "1. not ordered"}]}, + {"type": "paragraph", "content": [{"type": "text", "text": "2) not ordered"}]}, + ], + } + # When rendering the ADF to markdown + result = execute_actions.adf_to_markdown(doc) + # Then the leading marker of each line is escaped (heading/bullet/quote escape + # the first char; ordered lists escape the . / ) separator) + assert result == ( + "\\# not a heading\n\n" + "\\### also not\n\n" + "\\- not a bullet\n\n" + "\\+ not a bullet\n\n" + "\\> not a quote\n\n" + "1\\. not ordered\n\n" + "2\\) not ordered" + ), f"Got: {result!r}" + + +def test_adf_to_markdown_escapes_line_leading_marker_after_hardbreak(): + """A block marker that starts a line *after* a hardBreak inside a paragraph is + escaped too, since it is at a real line start once rendered.""" + # Given a paragraph with a hardBreak followed by text starting with "# " + doc = { + "type": "doc", + "version": 1, + "content": [ + {"type": "paragraph", "content": [ + {"type": "text", "text": "see:"}, + {"type": "hardBreak"}, + {"type": "text", "text": "# heading"}, + ]}, + ], + } + # When rendering the ADF to markdown + result = execute_actions.adf_to_markdown(doc) + # Then only the post-hardBreak line-leading marker is escaped + assert result == "see:\n\\# heading", f"Got: {result!r}" + + +def test_adf_to_markdown_does_not_escape_midline_or_non_marker_text(): + """Escaping is line-position-sensitive: a marker char mid-line, or a + marker-like prefix that does not actually form a block (no trailing space, a + heading start intentionally emitted by the heading renderer), is left alone.""" + # Given a heading node, a paragraph with a mid-line '#', and paragraphs whose + # leading chars do not form a block construct ("-5", "1.5" have no space) + doc = { + "type": "doc", + "version": 1, + "content": [ + {"type": "heading", "attrs": {"level": 2}, + "content": [{"type": "text", "text": "Real Heading"}]}, + {"type": "paragraph", "content": [{"type": "text", "text": "not # a heading"}]}, + {"type": "paragraph", "content": [{"type": "text", "text": "-5 degrees"}]}, + {"type": "paragraph", "content": [{"type": "text", "text": "1.5 times"}]}, + ], + } + # When rendering the ADF to markdown + result = execute_actions.adf_to_markdown(doc) + # Then the real heading keeps its intentional prefix and nothing else is escaped + assert result == ( + "## Real Heading\n\n" + "not # a heading\n\n" + "-5 degrees\n\n" + "1.5 times" + ), f"Got: {result!r}" + + def test_adf_to_markdown_renders_non_text_inline_nodes(): """Each non-text inline leaf node (mention/emoji/inlineCard/date/status) renders its attrs-sourced displayable value instead of being dropped to an @@ -931,6 +1013,9 @@ def fake_run(cmd, input=None, text=None, capture_output=None, env=None): test_adf_to_markdown_renders_task_list() test_adf_to_markdown_escapes_markdown_active_chars_in_literal_text() test_adf_to_markdown_does_not_double_escape_marks_or_code() + test_adf_to_markdown_escapes_line_leading_block_markers() + test_adf_to_markdown_escapes_line_leading_marker_after_hardbreak() + test_adf_to_markdown_does_not_escape_midline_or_non_marker_text() test_adf_to_markdown_renders_non_text_inline_nodes() test_adf_to_markdown_renders_inline_nodes_in_task_item() test_adf_to_markdown_renders_table() From 7ce05af32723f3a29c40983634e695b53f595b29 Mon Sep 17 00:00:00 2001 From: mrizzi Date: Wed, 2 Sep 2026 11:56:50 +0200 Subject: [PATCH 032/175] fix(verify-pr): canonicalize commit SHA up to full form for dedup marker _normalize_commit_sha truncated the commit SHA DOWN to 7 chars so a full SHA and an abbreviation of the same commit produced one stable dedup marker. But truncation also makes two DISTINCT commits that share a 7-hex prefix collide, so a later commit's report could PATCH-overwrite an earlier commit's GitHub report comment (astronomically rare, but a correctness hole introduced with the 7-char normalization). Resolve UP to the full 40-char SHA via `git rev-parse --verify --quiet ^{commit}` instead. The write path runs host-side in the checked-out repo, so the commit is normally resolvable; a full SHA and any abbreviation of it resolve to the same object (same-commit invariance kept) while distinct commits get distinct 40-char markers (collision removed). If git is unavailable or the value cannot be resolved (git missing -> OSError, object absent/ambiguous -> non-zero exit or non-40-hex output), it falls back to the prior fixed-length prefix so dedup keeps working rather than being disabled. Marker construction and lookup share the same helper, so both sides stay consistent. Adapted the invariance test to route git rev-parse (both forms -> one full SHA), added a distinct-commits-same-7-hex-prefix test proving no collision, and a fallback unit test for the git-unavailable path; execute_actions suite 36 -> 38 (93 -> 95 overall). Implements TC-5972 Assisted-by: Claude Code Co-Authored-By: Claude Opus 4.8 --- .../sdlc-workflow/scripts/execute-actions.py | 57 +++++-- .../scripts/test_execute_actions.py | 151 +++++++++++++----- 2 files changed, 152 insertions(+), 56 deletions(-) diff --git a/plugins/sdlc-workflow/scripts/execute-actions.py b/plugins/sdlc-workflow/scripts/execute-actions.py index 13425ba2f..37b38666c 100755 --- a/plugins/sdlc-workflow/scripts/execute-actions.py +++ b/plugins/sdlc-workflow/scripts/execute-actions.py @@ -59,30 +59,53 @@ POST_COMMENT_STICKY_MARKER = "" # GitHub has no native sticky-comment mechanism, so the verify-pr report comment -# embeds this marker (an invisible HTML comment) in its body. A length-normalized -# commit SHA is appended per post, scoping dedup to a single verification -# run/commit: a retry for the same commit updates the existing comment instead of -# duplicating it, while a later commit gets a fresh comment — preserving the -# per-run verification history that verify-pr SKILL.md Step 9 posts. +# embeds this marker (an invisible HTML comment) in its body. A canonical +# (full-length) commit SHA is appended per post, scoping dedup to a single +# verification run/commit: a retry for the same commit updates the existing +# comment instead of duplicating it, while a later commit gets a fresh comment — +# preserving the per-run verification history that verify-pr SKILL.md Step 9 posts. GITHUB_REPORT_MARKER_PREFIX = "" in created_body, \ + f"marker should carry the canonical full SHA: {created_body!r}" # Retry with the ABBREVIATED SHA; the PR already has the comment created above. - retry_calls = _run_report(short_sha, [{"id": 555, "body": created_body}]) + retry_calls = _run_post_report_with_sha_resolution( + short_sha, [{"id": 555, "body": created_body}], resolve) # Then the retry PATCH-updates the existing comment instead of duplicating it. assert not any(c["cmd"][:3] == ["gh", "pr", "comment"] for c in retry_calls), \ @@ -905,6 +918,64 @@ def fake_run(cmd, input=None, text=None, capture_output=None, env=None): assert patch_calls[0]["cmd"][2] == "repos/acme/widget/issues/comments/555" +def test_execute_post_report_distinct_commits_same_prefix_do_not_collide(): + """Two distinct commits sharing a 7-hex prefix get DISTINCT dedup markers + (each resolved up to its full 40-char SHA), so the second commit's report is + created as a new comment instead of PATCH-overwriting the first commit's.""" + full_a = "946556e" + "a" * 33 # commit A + full_b = "946556e" + "b" * 33 # commit B — same first 7 hex chars, distinct object + resolve = {full_a: full_a, full_b: full_b} + + # Commit A posts first with no existing comment → a comment carrying A's marker. + a_calls = _run_post_report_with_sha_resolution(full_a, [], resolve) + a_create = next(c for c in a_calls if c["cmd"][:3] == ["gh", "pr", "comment"]) + a_body = a_create["cmd"][a_create["cmd"].index("--body") + 1] + assert f"commit:{full_a} -->" in a_body + + # Commit B posts while A's comment already exists on the PR. + b_calls = _run_post_report_with_sha_resolution( + full_b, [{"id": 555, "body": a_body}], resolve) + + # Then B does NOT PATCH A's comment (no collision); it creates its own. + assert not any("PATCH" in c["cmd"] for c in b_calls), \ + "distinct commit must not overwrite another commit's report comment" + b_create = next(c for c in b_calls if c["cmd"][:3] == ["gh", "pr", "comment"]) + b_body = b_create["cmd"][b_create["cmd"].index("--body") + 1] + assert f"commit:{full_b} -->" in b_body + + +def test_normalize_commit_sha_falls_back_to_prefix_when_unresolvable(): + """When git cannot resolve the SHA (git missing or object absent), + _normalize_commit_sha falls back to the fixed-length prefix so dedup keeps + working instead of being disabled.""" + long_sha = "946556e" + "f" * 33 + + # git rev-parse reports failure (non-zero, empty stdout) → fallback to prefix. + def failing_run(cmd, input=None, text=None, capture_output=None, env=None): + return _FakeCompleted(1, "fatal: Needed a single revision", "") + + saved_run = execute_actions.subprocess.run + execute_actions.subprocess.run = failing_run + try: + result = execute_actions._normalize_commit_sha(long_sha) + finally: + execute_actions.subprocess.run = saved_run + assert result == long_sha[:execute_actions.COMMIT_SHA_MARKER_LENGTH], \ + f"Got: {result!r}" + + # git binary missing (OSError) → same fallback. + def raising_run(cmd, input=None, text=None, capture_output=None, env=None): + raise FileNotFoundError("git") + + execute_actions.subprocess.run = raising_run + try: + result = execute_actions._normalize_commit_sha(long_sha) + finally: + execute_actions.subprocess.run = saved_run + assert result == long_sha[:execute_actions.COMMIT_SHA_MARKER_LENGTH], \ + f"Got: {result!r}" + + def test_post_report_resolves_ref_created_by_later_action(): """report_md refs resolve regardless of action ordering: a post_report ordered BEFORE the create_subtask it references still resolves, because main() defers @@ -1029,6 +1100,8 @@ def fake_run(cmd, input=None, text=None, capture_output=None, env=None): test_execute_post_report_updates_existing_github_comment_on_retry() test_execute_post_report_updates_comment_on_later_page() test_execute_post_report_dedup_marker_invariant_to_sha_length() + test_execute_post_report_distinct_commits_same_prefix_do_not_collide() + test_normalize_commit_sha_falls_back_to_prefix_when_unresolvable() test_post_report_resolves_ref_created_by_later_action() test_find_report_comment_id_exits_on_unparseable_json() print("All tests passed.") From 2197cc763e1c7e6b5cfd80cc6538614ad19b4aa0 Mon Sep 17 00:00:00 2001 From: mrizzi Date: Wed, 2 Sep 2026 13:47:20 +0200 Subject: [PATCH 033/175] docs(conventions): document Python test suite and enforce pytest execution CONVENTIONS.md described the repo as documentation-only ("no runtime code", "no unit test framework"), which no longer holds: the workflow now ships executable Python under plugins/sdlc-workflow/scripts/ with a pytest suite (95 tests). The stale wording let contributors and agents skip test execution. Update Language and Framework, File Organization, and Error Handling to acknowledge the Python scripts; rewrite Testing Conventions to document the pytest suite as a mandatory pre-commit/pre-merge gate; and add `python3 -m pytest plugins/sdlc-workflow/scripts/ -q` to the CI Checks pre-push commands plus a dedicated Python Unit Tests subsection. Note the suite is not yet wired into GitHub Actions, so local runs are the current enforcement point. Implements TC-5973 Assisted-by: Claude Code Co-Authored-By: Claude Opus 4.8 --- CONVENTIONS.md | 47 ++++++++++++++++++++++++++++++++++++++++++----- 1 file changed, 42 insertions(+), 5 deletions(-) diff --git a/CONVENTIONS.md b/CONVENTIONS.md index 36c88b393..829d8b6ce 100644 --- a/CONVENTIONS.md +++ b/CONVENTIONS.md @@ -9,7 +9,7 @@ - **Primary format**: Markdown documentation - **Configuration**: YAML (`.serena/project.yml`) and JSON (plugin manifests) - **Plugin system**: Claude Code plugin format -- **No source code**: This is a documentation-heavy repository — skills are defined in Markdown (`SKILL.md` files) rather than traditional programming languages +- **Documentation-first, with Python tooling**: Skills are defined in Markdown (`SKILL.md` files), but the repository also ships executable Python and shell helpers under `plugins/sdlc-workflow/scripts/` that back the workflow. Changes to those scripts require running their automated test suite (see **Testing Conventions**) ## Code Style @@ -34,7 +34,7 @@ - **`plugins/sdlc-workflow/`** — main plugin directory - **`skills//`** — individual skill directories, each containing a `SKILL.md` file - **`shared/`** — shared resources like `task-description-template.md` - - **`scripts/`** — utility scripts (if any) + - **`scripts/`** — executable Python and shell helpers (e.g., `execute-actions.py`, `jira-client.py`, `pre-verify-pr.sh`) with a `pytest` unit-test suite (`test_*.py`) alongside them - **`.claude-plugin/`** — plugin manifest (`plugin.json`) - **`.claude-plugin/`** — marketplace manifest at root level (`marketplace.json`) - **`.serena/`** — Serena configuration files @@ -46,7 +46,11 @@ ## Error Handling -Not applicable — this is a documentation repository with no runtime code. +The Markdown skills have no runtime error handling, but the Python scripts under +`plugins/sdlc-workflow/scripts/` do. They must fail fast and loud: validate inputs, +exit non-zero (`sys.exit(1)`) with a message on `stderr` on error, and never swallow +an exception into a silent fallback. Cover both the success and the failure path with +tests (see **Testing Conventions**). ## Testing Conventions @@ -56,7 +60,17 @@ Not applicable — this is a documentation repository with no runtime code. 3. Run `/agents` to verify no plugin agents are missing 4. Edit a `SKILL.md`, then `/reload-plugins` to verify changes are picked up - **CI validation**: Uses `claude plugin validate` on all plugin directories under `plugins/` -- **No automated tests**: Skills are validated through CI and manual testing; no unit test framework +- **Automated unit tests (mandatory)**: The Python scripts under + `plugins/sdlc-workflow/scripts/` have a `pytest` suite in sibling `test_*.py` files + (e.g., `test_execute_actions.py`, `test_jira_client.py`, `test_pre_verify_pr.py`). + Any change to a script under `scripts/` **must** add or update the matching test and + keep the whole suite green. Run it before every commit and before opening or updating + a PR: + ```bash + python3 -m pytest plugins/sdlc-workflow/scripts/ -q + ``` + A passing run is a precondition for merge, not an optional step — do not commit a + script change without running it. - **Fixture documentation**: Eval and test fixture files must include a leading comment header in the file's native comment syntax (e.g., `` for Markdown/HTML, `// ...` for JSON with comments, `# ...` for YAML) explaining that the content is deliberate test material. Use the canonical prefixes below so tooling (linters, scanners, grep filters) can reliably identify annotated fixtures. Two categories require annotation: - **Adversarial fixtures** — files containing intentionally adversarial, malicious-looking, or unusual content (e.g., injection vectors, malformed input, security-sensitive patterns). Use the prefix `ADVERSARIAL TEST FIXTURE — ` (e.g., ``). - **Synthetic data fixtures** — files representing synthetic or mock entities (e.g., fake repository structures, mock Jira issues, fabricated API responses). Use the prefix `SYNTHETIC TEST DATA — ` (e.g., ``). @@ -70,10 +84,17 @@ Not applicable — this is a documentation repository with no runtime code. ## CI Checks -All CI checks must pass before merging. Run locally before pushing: +All checks must pass before merging. Run locally before pushing: ```bash +# 1. Skill instruction lint uvx skillsaw + +# 2. Plugin manifest validation +claude plugin validate plugins/sdlc-workflow + +# 3. Python unit tests — required whenever anything under scripts/ changes +python3 -m pytest plugins/sdlc-workflow/scripts/ -q ``` ### Skill Lint (Skillsaw) @@ -90,6 +111,22 @@ claude plugin validate plugins/sdlc-workflow Validates plugin manifests under `plugins/`. CI workflow: `.github/workflows/validate-plugins.yml`. +### Python Unit Tests + +The executable scripts under `plugins/sdlc-workflow/scripts/` are covered by a `pytest` +suite in sibling `test_*.py` files. Run the full suite and keep it green whenever you +touch anything under `scripts/`: + +```bash +python3 -m pytest plugins/sdlc-workflow/scripts/ -q +``` + +This suite is **required before every commit that changes a script and before merge**. +It is not yet wired into a GitHub Actions workflow, so it will not block the PR +automatically — running it locally (and confirming a green run in the PR) is the +current enforcement point. Adding a CI workflow that runs it is a recommended +follow-up. + ## Commit Messages - **Format**: Conventional Commits — `type(scope): description` From 6fc19c1182ab573fd903b6213ccf997dc36b535d Mon Sep 17 00:00:00 2001 From: mrizzi Date: Wed, 2 Sep 2026 13:54:34 +0200 Subject: [PATCH 034/175] ci(scripts): run pytest suite on push and PR to main The scripts under plugins/sdlc-workflow/scripts/ had a pytest suite but no CI job ran it, so test execution relied on contributor discipline. Add a Python Tests workflow that installs pytest + jsonschema and runs `python3 -m pytest plugins/sdlc-workflow/scripts/ -q` on pushes and pull requests targeting main, matching the trigger convention of the existing skillsaw and validate-plugins workflows. Update CONVENTIONS.md to reference the workflow as the enforcement point instead of calling it a follow-up. Implements TC-5973 Assisted-by: Claude Code Co-Authored-By: Claude Opus 4.8 --- .github/workflows/python-tests.yml | 26 ++++++++++++++++++++++++++ CONVENTIONS.md | 8 +++----- 2 files changed, 29 insertions(+), 5 deletions(-) create mode 100644 .github/workflows/python-tests.yml diff --git a/.github/workflows/python-tests.yml b/.github/workflows/python-tests.yml new file mode 100644 index 000000000..863ba5b3c --- /dev/null +++ b/.github/workflows/python-tests.yml @@ -0,0 +1,26 @@ +name: Python Tests + +on: + push: + branches: [main] + pull_request: + branches: [main] + +jobs: + pytest: + name: Script Unit Tests + runs-on: ubuntu-latest + steps: + - name: Checkout repository + uses: actions/checkout@v7 + + - name: Set up Python + uses: actions/setup-python@v5 + with: + python-version: '3.12' + + - name: Install test dependencies + run: pip install pytest jsonschema + + - name: Run pytest + run: python3 -m pytest plugins/sdlc-workflow/scripts/ -q diff --git a/CONVENTIONS.md b/CONVENTIONS.md index 829d8b6ce..3a04f21e0 100644 --- a/CONVENTIONS.md +++ b/CONVENTIONS.md @@ -121,11 +121,9 @@ touch anything under `scripts/`: python3 -m pytest plugins/sdlc-workflow/scripts/ -q ``` -This suite is **required before every commit that changes a script and before merge**. -It is not yet wired into a GitHub Actions workflow, so it will not block the PR -automatically — running it locally (and confirming a green run in the PR) is the -current enforcement point. Adding a CI workflow that runs it is a recommended -follow-up. +This suite is **required before every commit that changes a script and before merge**, +and is enforced in CI by `.github/workflows/python-tests.yml` (runs on pushes and pull +requests targeting `main`). Run it locally before pushing so failures surface before CI. ## Commit Messages From 1aeb30dfd24bb968cfa293545b69fc7a6f51ff78 Mon Sep 17 00:00:00 2001 From: mrizzi Date: Wed, 2 Sep 2026 13:59:26 +0200 Subject: [PATCH 035/175] ci(scripts): temporarily run pytest on the feature branch Add TC-5787-verify-pr-fullsend-v037 to the push/pull_request branch filters of python-tests.yml so the suite runs as a check on feature-branch PRs (e.g. #285) before the branch merges to main. Marked TEMPORARY in the workflow; removal is tracked by TC-5816 (merge feature branch to main). Implements TC-5973 Assisted-by: Claude Code Co-Authored-By: Claude Opus 4.8 --- .github/workflows/python-tests.yml | 10 ++++++++-- 1 file changed, 8 insertions(+), 2 deletions(-) diff --git a/.github/workflows/python-tests.yml b/.github/workflows/python-tests.yml index 863ba5b3c..68bb82dd5 100644 --- a/.github/workflows/python-tests.yml +++ b/.github/workflows/python-tests.yml @@ -2,9 +2,15 @@ name: Python Tests on: push: - branches: [main] + # TEMPORARY: the TC-5787-verify-pr-fullsend-v037 entry runs the suite on the + # feature branch before it merges to main. Remove it (leave only `main`) once + # the feature branch is merged — tracked by TC-5816. + branches: [main, TC-5787-verify-pr-fullsend-v037] pull_request: - branches: [main] + # TEMPORARY: the TC-5787-verify-pr-fullsend-v037 entry makes this workflow run + # as a required check on PRs targeting the feature branch (e.g. PR #285). + # Remove it (leave only `main`) once the feature branch merges to main — TC-5816. + branches: [main, TC-5787-verify-pr-fullsend-v037] jobs: pytest: From 3283e42b4f71d27a173b383a0a3647491caa1f83 Mon Sep 17 00:00:00 2001 From: mrizzi Date: Wed, 2 Sep 2026 14:21:42 +0200 Subject: [PATCH 036/175] ci(scripts): run pytest on a Python 3.11-3.14 matrix Replace the single 3.12 pin in python-tests.yml with a version matrix (3.11, 3.12, 3.13, 3.14, fail-fast disabled). The scripts declare no minimum Python and are run locally by contributors on varying versions, so the suite is validated across the range they are likely to use, up to the latest stable (3.14). The suite uses only standard-library features plus jsonschema, so no code change is required. Implements TC-5973 Assisted-by: Claude Code Co-Authored-By: Claude Opus 4.8 --- .github/workflows/python-tests.yml | 9 ++++++++- 1 file changed, 8 insertions(+), 1 deletion(-) diff --git a/.github/workflows/python-tests.yml b/.github/workflows/python-tests.yml index 68bb82dd5..78dc1e1af 100644 --- a/.github/workflows/python-tests.yml +++ b/.github/workflows/python-tests.yml @@ -16,6 +16,13 @@ jobs: pytest: name: Script Unit Tests runs-on: ubuntu-latest + strategy: + # Run every version so one failure doesn't hide others. + fail-fast: false + matrix: + # No declared minimum Python; test the range contributors are likely + # to run locally, up to the latest stable (3.14, Oct 2025). + python-version: ['3.11', '3.12', '3.13', '3.14'] steps: - name: Checkout repository uses: actions/checkout@v7 @@ -23,7 +30,7 @@ jobs: - name: Set up Python uses: actions/setup-python@v5 with: - python-version: '3.12' + python-version: ${{ matrix.python-version }} - name: Install test dependencies run: pip install pytest jsonschema From b8592cfd0be43375ca598a9c7ab0c9c5a86fee46 Mon Sep 17 00:00:00 2001 From: mrizzi Date: Wed, 2 Sep 2026 15:22:20 +0200 Subject: [PATCH 037/175] docs(conventions): correct script test-coverage and dependency claims CONVENTIONS.md overstated the scripts layer's test coverage and retained a stale "documentation only" dependency claim after PR #285 acknowledged the executable tooling. - File Organization, Testing Conventions, and the Python Unit Tests subsection now name the covered core modules (execute-actions.py, jira-client.py, pre_verify_pr.py) and state that strip_extra_properties.py and the shell helpers (pre-verify-pr.sh, post-verify-pr.sh, validate-output-schema.sh) are not yet unit-tested. - Dependencies no longer says the repo "contains only documentation and configuration files"; it now records the Python 3 runtime requirement, validate-output-schema.sh's runtime jsonschema dependency, and the test-time pytest/jsonschema requirements. Implements TC-5974 Assisted-by: Claude Code Co-Authored-By: Claude Opus 4.8 --- CONVENTIONS.md | 26 ++++++++++++++++---------- 1 file changed, 16 insertions(+), 10 deletions(-) diff --git a/CONVENTIONS.md b/CONVENTIONS.md index 3a04f21e0..c9d9a3270 100644 --- a/CONVENTIONS.md +++ b/CONVENTIONS.md @@ -34,7 +34,7 @@ - **`plugins/sdlc-workflow/`** — main plugin directory - **`skills//`** — individual skill directories, each containing a `SKILL.md` file - **`shared/`** — shared resources like `task-description-template.md` - - **`scripts/`** — executable Python and shell helpers (e.g., `execute-actions.py`, `jira-client.py`, `pre-verify-pr.sh`) with a `pytest` unit-test suite (`test_*.py`) alongside them + - **`scripts/`** — executable Python and shell helpers (e.g., `execute-actions.py`, `jira-client.py`, `pre-verify-pr.sh`); the core Python modules have a `pytest` unit-test suite (`test_*.py`) alongside them, while `strip_extra_properties.py` and the shell helpers are not yet unit-tested - **`.claude-plugin/`** — plugin manifest (`plugin.json`) - **`.claude-plugin/`** — marketplace manifest at root level (`marketplace.json`) - **`.serena/`** — Serena configuration files @@ -60,11 +60,14 @@ tests (see **Testing Conventions**). 3. Run `/agents` to verify no plugin agents are missing 4. Edit a `SKILL.md`, then `/reload-plugins` to verify changes are picked up - **CI validation**: Uses `claude plugin validate` on all plugin directories under `plugins/` -- **Automated unit tests (mandatory)**: The Python scripts under - `plugins/sdlc-workflow/scripts/` have a `pytest` suite in sibling `test_*.py` files - (e.g., `test_execute_actions.py`, `test_jira_client.py`, `test_pre_verify_pr.py`). - Any change to a script under `scripts/` **must** add or update the matching test and - keep the whole suite green. Run it before every commit and before opening or updating +- **Automated unit tests (mandatory)**: The core Python modules under + `plugins/sdlc-workflow/scripts/` (`execute-actions.py`, `jira-client.py`, + `pre_verify_pr.py`) have a `pytest` suite in sibling `test_*.py` files + (`test_execute_actions.py`, `test_jira_client.py`, `test_jira_client_cli.py`, + `test_pre_verify_pr.py`). `strip_extra_properties.py` and the shell helpers + (`pre-verify-pr.sh`, `post-verify-pr.sh`, `validate-output-schema.sh`) are not yet + unit-tested. Any change to a script under `scripts/` **must** add or update the + matching test and keep the whole suite green. Run it before every commit and before opening or updating a PR: ```bash python3 -m pytest plugins/sdlc-workflow/scripts/ -q @@ -113,9 +116,12 @@ Validates plugin manifests under `plugins/`. CI workflow: `.github/workflows/val ### Python Unit Tests -The executable scripts under `plugins/sdlc-workflow/scripts/` are covered by a `pytest` -suite in sibling `test_*.py` files. Run the full suite and keep it green whenever you -touch anything under `scripts/`: +The core Python modules under `plugins/sdlc-workflow/scripts/` (`execute-actions.py`, +`jira-client.py`, `pre_verify_pr.py`) are covered by a `pytest` suite in sibling +`test_*.py` files; `strip_extra_properties.py` and the shell helpers +(`pre-verify-pr.sh`, `post-verify-pr.sh`, `validate-output-schema.sh`) are not yet +covered. Run the full suite and keep it green whenever you touch anything under +`scripts/`: ```bash python3 -m pytest plugins/sdlc-workflow/scripts/ -q @@ -178,7 +184,7 @@ requests targeting `main`). Run it locally before pushing so failures surface be ## Dependencies -- **No external dependencies** — this repository contains only documentation and configuration files +- **No external dependencies for the plugins themselves** — the Claude Code plugins are Markdown, YAML, and JSON. The repository also ships executable Python and shell helpers under `plugins/sdlc-workflow/scripts/`: these require Python 3 (`validate-output-schema.sh` additionally requires the `jsonschema` package at runtime), and the test suite requires `pytest` (plus `jsonschema`) - **Runtime dependency**: Claude Code CLI (users must have Claude Code installed to use the plugins) - **Plugin system**: Uses Claude Code's plugin marketplace and validation system (`claude plugin validate`) - **Version synchronization**: The plugin version must be kept in sync between: From 2c8defa6c577fe65a4682bea5d1dcc19888a8457 Mon Sep 17 00:00:00 2001 From: mrizzi Date: Wed, 2 Sep 2026 16:18:21 +0200 Subject: [PATCH 038/175] feat(verify-pr): re-sync FULLSEND_OUTPUT_DIR sandbox dual-mode onto SKILL.md Re-apply the sandbox dual-mode adaptation onto the current verify-pr SKILL.md, gated on the FULLSEND_OUTPUT_DIR env var so interactive (marketplace) behavior is unchanged when it is unset. When set (fullsend sandbox): skip Jira/GitHub API calls, read pre-fetched task and PR data from the mounted verify-pr-input.json, accumulate every write as an action in { "report": {}, "actions": [] }, and write the result to $FULLSEND_OUTPUT_DIR/agent-result.json. Adds Step 0.6 (Sandbox Mode Detection), Step 0.7 (Load Pre-Fetched Data), inline sandbox-mode notes on each read/write step, and a Step 9 Sandbox Mode Output block. Every emitted action type matches the verify-pr-result.schema.json enum (create_subtask, create_link, post_pr_reply, post_pr_comment, create_root_cause_task, post_comment, post_report); link_type values use the schema's Blocks/Related enum. Implements TC-5812 Assisted-by: Claude Code Co-Authored-By: Claude Opus 4.8 --- .../sdlc-workflow/skills/verify-pr/SKILL.md | 228 ++++++++++++++++++ 1 file changed, 228 insertions(+) diff --git a/plugins/sdlc-workflow/skills/verify-pr/SKILL.md b/plugins/sdlc-workflow/skills/verify-pr/SKILL.md index f42c90c04..9493007ab 100644 --- a/plugins/sdlc-workflow/skills/verify-pr/SKILL.md +++ b/plugins/sdlc-workflow/skills/verify-pr/SKILL.md @@ -58,6 +58,75 @@ Before attempting any JIRA operations throughout this skill, determine the acces Refer to `shared/jira-rest-fallback.md` for complete implementation details. +## Step 0.6 – Sandbox Mode Detection + +Check whether the `FULLSEND_OUTPUT_DIR` environment variable is set: + +```bash +echo ${FULLSEND_OUTPUT_DIR:-not-set} +``` + +If **set**, this skill is running inside a fullsend sandbox (no Jira or GitHub +tokens, no `gh` CLI, no network egress). Switch to **sandbox mode**: + +- Do NOT call Jira write APIs (`create_issue`, `add_comment`, `create_issue_link`, + `transition_issue`) directly. +- Do NOT call GitHub read APIs (`gh pr view`/`gh pr diff`/`gh api`) — every PR read + is pre-fetched into `verify-pr-input.json` and the PR head is already checked out + (GitHub is tier-1; there is no `gh`/`GH_TOKEN` in the sandbox). +- Do NOT post GitHub PR comments or replies directly. +- Instead, accumulate every write operation as an action in an in-memory JSON + structure, and at the end of execution write the complete result to + `$FULLSEND_OUTPUT_DIR/agent-result.json` (see Step 9's **Sandbox Mode Output**). + +If **not set**, execute in **interactive mode** — the current behavior, calling Jira +and GitHub APIs directly. Marketplace users are unaffected. + +Throughout the remaining steps, when you encounter a write operation: +- **Interactive mode:** execute it directly (existing behavior). +- **Sandbox mode:** append it to the actions array instead — each step below gives the + matching action shape. + +Initialize the accumulator as an in-memory JSON structure: + +```json +{ + "report": {}, + "actions": [] +} +``` + +The `report` object and every action conform to +`plugins/sdlc-workflow/schemas/verify-pr-result.schema.json`; the runner's +`post_script` executes the accumulated actions after the sandbox exits. + +## Step 0.7 – Load Pre-Fetched Data (sandbox mode only) + +**Skip this step in interactive mode.** + +In sandbox mode the `pre_script` fetches all task and PR data on the trusted runner +(where the tokens live) and mounts it read-only into the sandbox. Read it: + +```bash +cat /sandbox/workspace/.pre-script/verify-pr-input.json | python3 -m json.tool > /dev/null 2>&1 && echo "Pre-fetched data available" || echo "ERROR: Pre-fetched data missing or invalid" +``` + +If the file exists and contains valid JSON (shape defined by +`plugins/sdlc-workflow/schemas/verify-pr-input.schema.json`): + +- Read `task` (`summary`, `description` in the tracker's native ADF, `status`, + `labels`, `issue_links`, `custom_fields`) and `pr_url`. **Skip Step 1 (Fetch and + Parse Jira Task) and Step 2 (Identify PR)** — parse the task sections from + `task.description` and take the PR URL from `pr_url`. +- Read the `github` bundle (`pr_repo`, `pr_number`, `headRefName`, `commit_sha`, + `diff`, `stat`, `reviews`, `review_comments`, `issue_comments`, `commits`) and use + the already-checked-out PR-head tree. **Skip every GitHub read step** — Step 3 + (checkout), Step 4a (review/comment fetches), Step 5a (diff/stat/commits), and + Step 9's HEAD-SHA retrieval — using the bundle fields instead. + +If the file is missing or invalid, log a warning and fall back to Steps 1–3 and the +direct GitHub reads (interactive behavior). + ## Inputs The user will provide a Jira issue ID for a task that has an associated PR. @@ -140,6 +209,10 @@ or not configured, ask the user to provide the PR URL. Extract the PR number and repository from the URL for use in subsequent `gh` commands. +**Sandbox mode:** skip the custom-field read — take the PR URL from `pr_url` and the +`owner/repo` + PR number from `github.pr_repo` / `github.pr_number` in the pre-fetched +data (Step 0.7). + ## Step 3 – Checkout PR Branch Ensure the PR branch is checked out locally so that subsequent steps inspect the correct code. @@ -169,6 +242,10 @@ This step supports two use cases: - **Author self-verification** — the contributor already has the PR branch checked out; no checkout needed. - **Reviewer/CI audit** — another person or CI job runs `/verify-pr` from an arbitrary branch; the PR branch must be checked out first. +**Sandbox mode:** skip this step entirely — the runner has already checked out the PR +head tree (`github.headRefName` at `github.commit_sha`) and there is no `gh` CLI in +the sandbox. Inspect the working tree directly. + ## Step 4 – Classify Review Feedback Read PR review feedback and classify each comment thread for downstream processing @@ -183,6 +260,10 @@ gh api repos//pulls//reviews gh api repos//pulls//comments ``` +**Sandbox mode:** do not call `gh api` — use `github.reviews`, `github.review_comments`, +and (for the enumeration below) `github.issue_comments` from the pre-fetched data +(Step 0.7). + Group inline comments into threads using the `in_reply_to_id` field. Each top-level comment (no `in_reply_to_id`) starts a thread; replies are grouped under their parent. @@ -312,6 +393,10 @@ constructs dispatch envelopes following the structure defined in ### Step 5a – Gather Dispatch Inputs +**Sandbox mode:** do not run the `gh pr diff`/`gh pr view` commands below — the full +diff, diffstat, and commits are pre-fetched as `github.diff`, `github.stat`, and +`github.commits` (Step 0.7). Use those values as the corresponding dispatch inputs. + Collect all inputs needed for sub-agent dispatch envelopes: 1. **PR diff (full)** — for Security, Correctness, and Style/Conventions sub-agents: @@ -502,6 +587,35 @@ After creating each sub-task, create a "Blocks" issue link from the sub-task to jira.create_issue_link(type="Blocks", inwardIssue=, outwardIssue=) +**Sandbox mode:** Instead of calling `jira.create_issue` and `jira.create_issue_link`, +append a `create_subtask` action and a `create_link` action. Use this pattern for **all** +sub-task categories below (review feedback, CI failure, eval failure): + +```json +{ + "type": "create_subtask", + "ref": "subtask-N", + "parent": "", + "summary": "", + "labels": ["ai-generated-jira", ""], + "description_adf": +} +``` + +```json +{ + "type": "create_link", + "link_type": "Blocks", + "inward": "{{subtask-N.key}}", + "outward": "" +} +``` + +Use incrementing refs (`subtask-1`, `subtask-2`, …). The `{{subtask-N.key}}` (and, in +replies, `{{subtask-N.url}}`) placeholders are resolved by the `post_script` once the +sub-task is created. Set `` to `review-feedback` for review-feedback and +CI-failure sub-tasks, or `eval-failure` for eval-failure sub-tasks. + #### CI failure sub-tasks Process `create-sub-task` actions from the Correctness sub-agent. For each action: @@ -529,6 +643,9 @@ Process `create-sub-task` actions from the Correctness sub-agent. For each actio 3. **Create issue link:** jira.create_issue_link(type="Blocks", inwardIssue=, outwardIssue=) +**Sandbox mode:** use the `create_subtask` + `create_link` pattern from the review +feedback sandbox-mode block above, with labels `["ai-generated-jira", "review-feedback"]`. + #### Eval failure sub-tasks Process eval assertion failures from the Style/Conventions sub-agent's Check 5 @@ -573,6 +690,9 @@ and sub-task creation below. 4. **Create issue link:** jira.create_issue_link(type="Blocks", inwardIssue=, outwardIssue=) +**Sandbox mode:** use the `create_subtask` + `create_link` pattern from the review +feedback sandbox-mode block above, with labels `["ai-generated-jira", "eval-failure"]`. + ### Step 6e – Reply to Review Comments Reply to **every** classified review comment thread with the classification label and @@ -639,6 +759,31 @@ When a review body contains multiple classified suggestions (sub-identifiers), p a single standalone comment that lists all classifications together rather than one comment per sub-identifier. +**Sandbox mode:** Instead of calling `gh api`, append one action per reply. + +For **inline comment threads** (they have a `comment_id`), use `post_pr_reply`: + +```json +{ + "type": "post_pr_reply", + "repo": "", + "pr_number": , + "comment_id": , + "body": "" +} +``` + +For **review body items** (no `comment_id`), use `post_pr_comment`: + +```json +{ + "type": "post_pr_comment", + "repo": "", + "pr_number": , + "body": "" +} +``` + ### Step 6f – Idempotency Guarantees Idempotency is enforced **within** Step 4a's mandatory enumeration, not as a @@ -864,6 +1009,29 @@ Link the root-cause task to the parent task: jira.create_issue_link(type="Relates", inwardIssue=, outwardIssue=) +**Sandbox mode:** Instead of calling `jira.create_issue` and `jira.create_issue_link`, +append a `create_root_cause_task` action and a `create_link` action (the schema's +`link_type` enum uses `Related`, the equivalent of the interactive `Relates`): + +```json +{ + "type": "create_root_cause_task", + "ref": "rc-N", + "summary": "Root-cause: ", + "labels": ["ai-generated-jira", "root-cause"], + "description_adf": +} +``` + +```json +{ + "type": "create_link", + "link_type": "Related", + "inward": "{{rc-N.key}}", + "outward": "" +} +``` + #### Action 2 – Post the root-cause analysis as a comment After creating the task, post a Jira comment on the newly created root-cause task @@ -881,6 +1049,17 @@ The comment must include: Include the **Comment Footnote** at the end of the comment (see the Comment Footnote section at the top of this skill for the required ADF format). +**Sandbox mode:** Instead of calling `jira.add_comment`, append a `post_comment` action +targeting the not-yet-created root-cause task by its ref: + +```json +{ + "type": "post_comment", + "issue": "{{rc-N.key}}", + "body_adf": +} +``` + ### Step 7c – Idempotency Check Before creating a root-cause task (Step 7b), check for existing root-cause tasks @@ -966,6 +1145,9 @@ gh pr view --json commits --jq '.commits[-1].oid' -R Store the full SHA and its short form (first 7 characters) for use in the report. +**Sandbox mode:** do not call `gh pr view` — use `github.commit_sha` from the +pre-fetched data (Step 0.7) as the full SHA. + ### Post to GitHub PR The report comment is **scoped to the commit it verifies**. Posting is idempotent @@ -1027,6 +1209,52 @@ Include the Comment Footnote at the end of the Jira comment. **Important:** This skill does NOT merge the PR and does NOT transition the Jira issue. The report is informational — a human reviewer decides whether to merge. +### Sandbox Mode Output + +**Sandbox mode only:** Instead of posting to GitHub and Jira directly, populate the +`report` object and append a single `post_report` action. The runner's `post_script` +posts the GitHub PR comment (from `report_md`, applying the commit-scoped +find-then-update-or-create path above) and the Jira comment (from `report_adf`) after +the sandbox exits. + +Populate `report` (all fields required by +`plugins/sdlc-workflow/schemas/verify-pr-result.schema.json`): + +```json +{ + "jira_issue_id": "", + "pr_repo": "", + "pr_number": , + "commit_sha": "", + "overall": "PASS|WARN|FAIL", + "table_md": "", + "report_md": "", + "report_adf": , + "plugin_version": "" +} +``` + +Append the final action: + +```json +{ "type": "post_report" } +``` + +Write the complete accumulated JSON (the `report` object plus every action collected +across Steps 6–9) to the output file: + +```bash +cat > $FULLSEND_OUTPUT_DIR/agent-result.json << 'RESULT_EOF' + +RESULT_EOF +``` + +Verify it is valid JSON: + +```bash +python3 -m json.tool $FULLSEND_OUTPUT_DIR/agent-result.json > /dev/null && echo "Output validated" +``` + ## Important Rules - This skill does **NOT** modify code. It only verifies, creates sub-tasks, and reports. From 8df7aa285d3564938f158054a3c0c4d79146ddb1 Mon Sep 17 00:00:00 2001 From: mrizzi Date: Wed, 2 Sep 2026 18:28:21 +0200 Subject: [PATCH 039/175] fix(verify-pr): validate prefetched sandbox input against schema Step 0.7 treated verify-pr-input.json as usable if it merely parsed as JSON. A syntactically valid but structurally incomplete prefetch (missing the github bundle or task fields) passed the check and then failed deep inside a later step. Replace the json.tool syntax check with jsonschema validation against verify-pr-input.schema.json so a structurally invalid-but-parseable prefetch is treated as invalid, mirroring the existing validate-output-schema.sh pattern. Implements TC-5980 Assisted-by: Claude Code Co-Authored-By: Claude Opus 4.8 --- .../sdlc-workflow/skills/verify-pr/SKILL.md | 31 +++++++++++++++++-- 1 file changed, 28 insertions(+), 3 deletions(-) diff --git a/plugins/sdlc-workflow/skills/verify-pr/SKILL.md b/plugins/sdlc-workflow/skills/verify-pr/SKILL.md index 9493007ab..4f6f356bd 100644 --- a/plugins/sdlc-workflow/skills/verify-pr/SKILL.md +++ b/plugins/sdlc-workflow/skills/verify-pr/SKILL.md @@ -107,12 +107,37 @@ The `report` object and every action conform to In sandbox mode the `pre_script` fetches all task and PR data on the trusted runner (where the tokens live) and mounts it read-only into the sandbox. Read it: +Validate it against `plugins/sdlc-workflow/schemas/verify-pr-input.schema.json` +before using it — a syntactically valid but structurally wrong or incomplete +prefetch (e.g., missing the `github` bundle or `task` fields) must be treated as +invalid rather than passing and failing deep inside a later step: + ```bash -cat /sandbox/workspace/.pre-script/verify-pr-input.json | python3 -m json.tool > /dev/null 2>&1 && echo "Pre-fetched data available" || echo "ERROR: Pre-fetched data missing or invalid" +python3 - << 'PYEOF' +import json, sys +from jsonschema import validate, ValidationError + +INPUT = "/sandbox/workspace/.pre-script/verify-pr-input.json" +SCHEMA = "plugins/sdlc-workflow/schemas/verify-pr-input.schema.json" +try: + with open(INPUT) as f: + instance = json.load(f) + with open(SCHEMA) as f: + schema = json.load(f) + validate(instance=instance, schema=schema) + print("Pre-fetched data available") +except FileNotFoundError: + print("ERROR: Pre-fetched data missing"); sys.exit(1) +except json.JSONDecodeError as e: + print(f"ERROR: Pre-fetched data is not valid JSON: {e}"); sys.exit(1) +except ValidationError as e: + path = ".".join(str(p) for p in e.path) or "" + print(f"ERROR: Pre-fetched data failed schema validation at {path}: {e.message}") + sys.exit(1) +PYEOF ``` -If the file exists and contains valid JSON (shape defined by -`plugins/sdlc-workflow/schemas/verify-pr-input.schema.json`): +If the file validates against the schema: - Read `task` (`summary`, `description` in the tracker's native ADF, `status`, `labels`, `issue_links`, `custom_fields`) and `pr_url`. **Skip Step 1 (Fetch and From 683b957831a1db39b269cbb887347880e44c7d68 Mon Sep 17 00:00:00 2001 From: mrizzi Date: Wed, 2 Sep 2026 18:30:22 +0200 Subject: [PATCH 040/175] fix(verify-pr): fail fast on bad sandbox prefetch instead of credentialed fallback Step 0.7 documented that a missing/invalid prefetch falls back to Steps 1-3 and direct Jira/GitHub reads. That "interactive" fallback cannot work in sandbox mode (FULLSEND_OUTPUT_DIR set): the sandbox has no tokens, no gh CLI, and no network egress, so credentialed calls fail obscurely or hang. Split the failure handling: in sandbox mode write a structured error to $FULLSEND_OUTPUT_DIR/agent-result.json and stop (the runner's output validation surfaces it as a hard failure), while interactive mode keeps the existing warn-and-fall-back behavior unchanged. Implements TC-5981 Assisted-by: Claude Code Co-Authored-By: Claude Opus 4.8 --- .../sdlc-workflow/skills/verify-pr/SKILL.md | 26 +++++++++++++++++-- 1 file changed, 24 insertions(+), 2 deletions(-) diff --git a/plugins/sdlc-workflow/skills/verify-pr/SKILL.md b/plugins/sdlc-workflow/skills/verify-pr/SKILL.md index 4f6f356bd..120bcab83 100644 --- a/plugins/sdlc-workflow/skills/verify-pr/SKILL.md +++ b/plugins/sdlc-workflow/skills/verify-pr/SKILL.md @@ -149,8 +149,30 @@ If the file validates against the schema: (checkout), Step 4a (review/comment fetches), Step 5a (diff/stat/commits), and Step 9's HEAD-SHA retrieval — using the bundle fields instead. -If the file is missing or invalid, log a warning and fall back to Steps 1–3 and the -direct GitHub reads (interactive behavior). +If the validation above fails (file missing, not valid JSON, or not conforming to the +schema), the handling depends on the mode: + +- **Sandbox mode (`FULLSEND_OUTPUT_DIR` is set):** do **not** fall back to Steps 1–3 or + the direct Jira/GitHub reads. The sandbox has no tokens, no `gh` CLI, and no network + egress, so a credentialed fallback cannot succeed — it would fail obscurely or hang. + Fail fast and loud instead: write a structured failure result and stop the skill + without performing any further steps or write operations. + +```bash +cat > "$FULLSEND_OUTPUT_DIR/agent-result.json" << 'RESULT_EOF' +{ + "error": "verify-pr aborted: pre-fetched input (/sandbox/workspace/.pre-script/verify-pr-input.json) is missing, unparseable, or does not conform to verify-pr-input.schema.json. The pre_script must produce a valid input bundle before the sandbox runs; no credentialed fallback is possible inside the sandbox." +} +RESULT_EOF +``` + + This result intentionally omits the `report` and `actions` keys required by + `verify-pr-result.schema.json`, so the runner's output validation rejects it and + surfaces the `error` message as a hard failure rather than silently attempting the + interactive path. **Stop execution here — do not run any subsequent step.** + +- **Interactive mode (`FULLSEND_OUTPUT_DIR` is unset):** log a warning and fall back to + Steps 1–3 and the direct GitHub reads (existing interactive behavior, unchanged). ## Inputs From d0ce26e414049ba7a3589dbb7fe824ad9b14210d Mon Sep 17 00:00:00 2001 From: mrizzi Date: Wed, 2 Sep 2026 18:35:40 +0200 Subject: [PATCH 041/175] feat(verify-pr): prefetch related-issue metadata for sandbox idempotency Steps 6d/6f/7c dedupe against the task's existing sub-tasks and linked root-cause issues via Jira reads, but the sandbox has no Jira token, so those idempotency checks had no data source and risked creating duplicate sub-tasks/root-cause tasks on re-run. Prefetch the read data on the trusted runner (split-trust): pre-verify-pr.sh enumerates the task's related keys and fetches each (summary, labels, description, issuetype, comments), and pre_verify_pr.py embeds them under a new `idempotency` bundle in verify-pr-input.json. SKILL.md Steps 0.7/6d/6f/7c now read `idempotency.related_issues` in sandbox mode instead of calling Jira; the input schema documents the new (optional) bundle. No token or egress is added to the sandbox. Implements TC-5982 Assisted-by: Claude Code Co-Authored-By: Claude Opus 4.8 --- .../schemas/verify-pr-input.schema.json | 26 +++ .../sdlc-workflow/scripts/pre-verify-pr.sh | 23 ++- .../sdlc-workflow/scripts/pre_verify_pr.py | 88 ++++++++- .../scripts/test_pre_verify_pr.py | 174 ++++++++++++++++++ .../sdlc-workflow/skills/verify-pr/SKILL.md | 20 +- 5 files changed, 324 insertions(+), 7 deletions(-) diff --git a/plugins/sdlc-workflow/schemas/verify-pr-input.schema.json b/plugins/sdlc-workflow/schemas/verify-pr-input.schema.json index d7f853829..33cb26965 100644 --- a/plugins/sdlc-workflow/schemas/verify-pr-input.schema.json +++ b/plugins/sdlc-workflow/schemas/verify-pr-input.schema.json @@ -95,6 +95,32 @@ } } }, + "idempotency": { + "type": "object", + "description": "Related-issue metadata prefetched on the trusted runner so the sandbox can run the Steps 6d/6f/7c idempotency checks (duplicate sub-task and root-cause detection) without a Jira token or egress. Optional: absent or empty when the task has no sub-tasks or linked issues yet.", + "additionalProperties": false, + "properties": { + "related_issues": { + "type": "array", + "description": "The task's existing sub-tasks and linked issues (e.g., root-cause tasks), each with the fields the dedup checks inspect.", + "items": { + "type": "object", + "properties": { + "key": { "type": "string", "description": "Issue key (e.g., TC-4742)" }, + "summary": { "type": "string" }, + "labels": { "type": "array", "items": { "type": "string" } }, + "description": { "type": "object", "description": "Issue description in the tracker's native format (e.g., ADF)" }, + "issuetype": { "type": "string", "description": "Issue type name (e.g., Sub-task, Task)" }, + "comments": { + "type": "array", + "description": "Comment bodies in the tracker's native format; searched by Step 7c for the root-cause marker.", + "items": { "type": "object" } + } + } + } + } + } + }, "source": { "type": "object", "description": "Raw issue tracker response for fields not captured above", diff --git a/plugins/sdlc-workflow/scripts/pre-verify-pr.sh b/plugins/sdlc-workflow/scripts/pre-verify-pr.sh index 14bfa414e..f7dca5241 100755 --- a/plugins/sdlc-workflow/scripts/pre-verify-pr.sh +++ b/plugins/sdlc-workflow/scripts/pre-verify-pr.sh @@ -148,15 +148,34 @@ gh pr view "${PR_NUM}" -R "${PR_REPO}" --json commits --jq .commits > "${PRE_OUT echo "GitHub read bundle prefetched to ${PRE_OUTPUT_DIR}" +# 7b. Idempotency prefetch — the sandbox has no Jira token, but Steps 6d/6f/7c +# dedupe against the task's existing sub-tasks and linked (e.g., root-cause) +# issues. Fetch each related issue here on the trusted runner (summary, +# labels, description, issuetype, and comments) so the sandbox can run those +# checks tokenlessly. No `|| true`: a related issue the token created should +# be readable, so a fetch failure is a real error surfaced under set -e. +REL_DIR="${PRE_OUTPUT_DIR}/related-issues" +rm -rf "${REL_DIR}" +mkdir -p "${REL_DIR}" +RELATED_KEYS=$(printf '%s\n' "${ISSUE_JSON}" | python3 "${SCRIPT_DIR}/pre_verify_pr.py" related-keys) +for key in ${RELATED_KEYS}; do + python3 "${SCRIPT_DIR}/jira-client.py" get_issue "${key}" \ + --fields "summary,labels,description,issuetype,comment" \ + > "${REL_DIR}/${key}.json" +done +echo "Idempotency read bundle prefetched to ${REL_DIR}" + # 8. Write pre-fetched data for sandbox consumption (tracker-agnostic format, -# with the GitHub bundle embedded under `github`). +# with the GitHub bundle embedded under `github` and the idempotency +# related-issue metadata under `idempotency`). printf '%s\n' "${ISSUE_JSON}" | python3 "${SCRIPT_DIR}/pre_verify_pr.py" transform \ "${JIRA_ISSUE_ID}" "${PR_URL}" \ --github-dir "${PRE_OUTPUT_DIR}" \ --pr-repo "${PR_REPO}" \ --pr-number "${PR_NUM}" \ --head-ref "${HEAD_REF}" \ - --commit-sha "${COMMIT_SHA}" > "${PRE_OUTPUT_DIR}/verify-pr-input.json" + --commit-sha "${COMMIT_SHA}" \ + --idempotency-dir "${REL_DIR}" > "${PRE_OUTPUT_DIR}/verify-pr-input.json" echo "Pre-fetched data written to ${PRE_OUTPUT_DIR}/verify-pr-input.json" echo "Input validation passed" diff --git a/plugins/sdlc-workflow/scripts/pre_verify_pr.py b/plugins/sdlc-workflow/scripts/pre_verify_pr.py index 0174a2ba6..037ed0153 100644 --- a/plugins/sdlc-workflow/scripts/pre_verify_pr.py +++ b/plugins/sdlc-workflow/scripts/pre_verify_pr.py @@ -7,17 +7,26 @@ CLI usage (called by pre-verify-pr.sh): echo "$ISSUE_JSON" | python3 pre_verify_pr.py extract-pr-url + echo "$ISSUE_JSON" | python3 pre_verify_pr.py related-keys echo "$ISSUE_JSON" | python3 pre_verify_pr.py transform TASK_ID PR_URL \\ [--github-dir DIR --pr-repo REPO --pr-number N \\ - --head-ref REF --commit-sha SHA] + --head-ref REF --commit-sha SHA --idempotency-dir DIR] When the --github-* options are supplied, `transform` reads the raw GitHub reads from DIR (pr.diff, pr.stat, reviews.json, review-comments.json, issue-comments.json, commits.json) and embeds them under a `github` key. + +`related-keys` prints the task's sub-task and linked-issue keys (one per line) +so the shell can prefetch each on the runner. When --idempotency-dir is given, +`transform` reads those prefetched issue JSONs and embeds their summary/labels/ +description/comments under an `idempotency` key, giving the sandbox a tokenless +data source for the Steps 6d/6f/7c idempotency checks. """ import argparse +import glob import json +import os import sys @@ -62,7 +71,58 @@ def build_github_bundle(pr_repo, pr_number, head_ref, commit_sha, } -def transform_to_input(issue, task_id, pr_url, github=None): +def related_keys(issue): + """Keys of the task's sub-tasks and linked issues (idempotency targets). + + Steps 6d/6f/7c dedupe against the parent task's existing sub-tasks and its + linked (e.g., root-cause) issues. The pre_script fetches each of these keys + on the trusted runner so the sandbox can run those idempotency checks + without a Jira token. Returns a sorted, de-duplicated list. + """ + fields = issue.get("fields", {}) + keys = set() + for sub in fields.get("subtasks") or []: + key = sub.get("key") + if key: + keys.add(key) + for link in fields.get("issuelinks") or []: + related = link.get("inwardIssue") or link.get("outwardIssue") or {} + key = related.get("key") + if key: + keys.add(key) + return sorted(keys) + + +def build_idempotency_bundle(related_issue_jsons): + """Bundle related-issue metadata for the sandbox idempotency checks. + + Each item is a full issue JSON the runner fetched with fields summary, + labels, description, issuetype, and comment. The sandbox reads this instead + of calling Jira to dedupe sub-tasks (Steps 6d/6f) and root-cause tasks + (Step 7c). Descriptions and comment bodies are kept in the tracker's native + format (ADF for Jira) — the sandbox agent inspects them directly. + """ + related = [] + for ri in related_issue_jsons: + fields = ri.get("fields", {}) + comments = [ + c.get("body") or {} + for c in (fields.get("comment") or {}).get("comments", []) + ] + related.append({ + "key": ri.get("key", ""), + "summary": fields.get("summary", ""), + "labels": fields.get("labels", []), + # Coerce an explicit null description to {} so the value stays an + # object per verify-pr-input.schema.json (see transform_to_input). + "description": fields.get("description") or {}, + "issuetype": (fields.get("issuetype") or {}).get("name", ""), + "comments": comments, + }) + return {"related_issues": related} + + +def transform_to_input(issue, task_id, pr_url, github=None, idempotency=None): """Transform Jira issue JSON to tracker-agnostic input schema.""" fields = issue.get("fields", {}) result = { @@ -99,6 +159,8 @@ def transform_to_input(issue, task_id, pr_url, github=None): } if github is not None: result["github"] = github + if idempotency is not None: + result["idempotency"] = idempotency return result @@ -129,11 +191,22 @@ def _github_from_dir(args): ) +def _idempotency_from_dir(path): + """Read every related-issue JSON the shell wrote and build the bundle. + + Globs ``/*.json`` (one file per related key, written by + pre-verify-pr.sh). An empty directory yields an empty related_issues list. + """ + files = sorted(glob.glob(os.path.join(path, "*.json"))) + return build_idempotency_bundle([_read_json(f) for f in files]) + + def main(argv): parser = argparse.ArgumentParser(prog="pre_verify_pr.py") sub = parser.add_subparsers(dest="command", required=True) sub.add_parser("extract-pr-url") + sub.add_parser("related-keys") t = sub.add_parser("transform") t.add_argument("task_id") @@ -143,15 +216,24 @@ def main(argv): t.add_argument("--pr-number", type=int) t.add_argument("--head-ref") t.add_argument("--commit-sha") + t.add_argument("--idempotency-dir") args = parser.parse_args(argv) issue = json.load(sys.stdin) if args.command == "extract-pr-url": print(extract_pr_url(issue)) + elif args.command == "related-keys": + for key in related_keys(issue): + print(key) elif args.command == "transform": github = _github_from_dir(args) if args.github_dir else None - result = transform_to_input(issue, args.task_id, args.pr_url, github) + idempotency = ( + _idempotency_from_dir(args.idempotency_dir) + if args.idempotency_dir else None + ) + result = transform_to_input( + issue, args.task_id, args.pr_url, github, idempotency) json.dump(result, sys.stdout, indent=2) diff --git a/plugins/sdlc-workflow/scripts/test_pre_verify_pr.py b/plugins/sdlc-workflow/scripts/test_pre_verify_pr.py index ab5f76842..fbb3675d0 100644 --- a/plugins/sdlc-workflow/scripts/test_pre_verify_pr.py +++ b/plugins/sdlc-workflow/scripts/test_pre_verify_pr.py @@ -271,6 +271,155 @@ def test_cli_transform_github_dir(): assert gh["commits"] == [{"oid": "abc1234def"}] +# --- idempotency prefetch (related_keys, build_idempotency_bundle, transform) --- + +def test_related_keys_from_subtasks_and_links(): + """related_keys collects sub-task keys and linked-issue keys, deduped/sorted.""" + issue = {"fields": { + "subtasks": [{"key": "TC-201"}, {"key": "TC-202"}], + "issuelinks": [ + {"type": {"name": "Blocks"}, "inwardIssue": {"key": "TC-202"}}, + {"type": {"name": "Related"}, "outwardIssue": {"key": "TC-300"}}, + ], + }} + # TC-202 appears as both a sub-task and a link — deduped; result is sorted + assert pre_verify_pr.related_keys(issue) == ["TC-201", "TC-202", "TC-300"] + + +def test_related_keys_empty_when_no_relations(): + issue = {"fields": {"summary": "S"}} + assert pre_verify_pr.related_keys(issue) == [] + + +def test_build_idempotency_bundle_extracts_fields_and_comments(): + """The bundle carries the fields the dedup checks inspect, incl. comment bodies.""" + ri = { + "key": "TC-500", + "fields": { + "summary": "Fix eval-3 assertion failures", + "labels": ["ai-generated-jira", "eval-failure"], + "description": {"type": "doc", "content": []}, + "issuetype": {"name": "Sub-task"}, + "comment": {"comments": [ + {"body": {"type": "doc", "content": [{"type": "text"}]}}, + {"body": {"type": "doc"}}, + ]}, + }, + } + bundle = pre_verify_pr.build_idempotency_bundle([ri]) + entry = bundle["related_issues"][0] + assert entry["key"] == "TC-500" + assert entry["summary"] == "Fix eval-3 assertion failures" + assert entry["labels"] == ["ai-generated-jira", "eval-failure"] + assert entry["description"] == {"type": "doc", "content": []} + assert entry["issuetype"] == "Sub-task" + assert len(entry["comments"]) == 2 + assert entry["comments"][0] == {"type": "doc", "content": [{"type": "text"}]} + + +def test_build_idempotency_bundle_handles_missing_fields(): + """A related issue lacking comments/description yields empty defaults, not errors.""" + bundle = pre_verify_pr.build_idempotency_bundle([{"key": "TC-9", "fields": {}}]) + entry = bundle["related_issues"][0] + assert entry["summary"] == "" + assert entry["labels"] == [] + assert entry["description"] == {} # null/absent coerced to object + assert entry["issuetype"] == "" + assert entry["comments"] == [] + + +def test_build_idempotency_bundle_empty(): + assert pre_verify_pr.build_idempotency_bundle([]) == {"related_issues": []} + + +def test_transform_with_idempotency_attaches_key(): + issue = {"fields": {"summary": "S", "status": {"name": "Open"}, "labels": [], "issuelinks": []}} + idem = pre_verify_pr.build_idempotency_bundle([{"key": "TC-1", "fields": {}}]) + result = pre_verify_pr.transform_to_input(issue, "TC-1", "", None, idem) + assert result["idempotency"]["related_issues"][0]["key"] == "TC-1" + + +def test_transform_without_idempotency_omits_key(): + issue = {"fields": {"summary": "S", "status": {"name": "Open"}, "labels": [], "issuelinks": []}} + result = pre_verify_pr.transform_to_input(issue, "TC-1", "") + assert "idempotency" not in result + + +def test_cli_transform_idempotency_dir(): + """CLI transform globs the related-issue JSONs and embeds the idempotency bundle.""" + issue = {"fields": {"summary": "S", "status": {"name": "Open"}, "labels": [], "issuelinks": []}} + with tempfile.TemporaryDirectory() as d: + with open(os.path.join(d, "TC-501.json"), "w") as f: + json.dump({"key": "TC-501", "fields": { + "summary": "Existing sub-task", "labels": ["review-feedback"], + "description": {"type": "doc"}, "issuetype": {"name": "Sub-task"}, + "comment": {"comments": [{"body": {"type": "doc"}}]}, + }}, f) + result = subprocess.run( + [sys.executable, os.path.join(script_dir, "pre_verify_pr.py"), + "transform", "TC-1", "https://github.com/o/r/pull/1", + "--idempotency-dir", d], + input=json.dumps(issue), capture_output=True, text=True, + ) + assert result.returncode == 0, f"Exit {result.returncode}: {result.stderr}" + output = json.loads(result.stdout) + related = output["idempotency"]["related_issues"] + assert len(related) == 1 + assert related[0]["key"] == "TC-501" + assert related[0]["labels"] == ["review-feedback"] + assert related[0]["comments"] == [{"type": "doc"}] + + +def test_cli_transform_empty_idempotency_dir(): + """An empty related-issues dir yields an empty related_issues list, not an error.""" + issue = {"fields": {"summary": "S", "status": {"name": "Open"}, "labels": [], "issuelinks": []}} + with tempfile.TemporaryDirectory() as d: + result = subprocess.run( + [sys.executable, os.path.join(script_dir, "pre_verify_pr.py"), + "transform", "TC-1", "https://github.com/o/r/pull/1", + "--idempotency-dir", d], + input=json.dumps(issue), capture_output=True, text=True, + ) + assert result.returncode == 0, f"Exit {result.returncode}: {result.stderr}" + assert json.loads(result.stdout)["idempotency"] == {"related_issues": []} + + +def test_cli_related_keys(): + """The related-keys subcommand prints sub-task and linked-issue keys.""" + issue = {"fields": { + "subtasks": [{"key": "TC-201"}], + "issuelinks": [{"type": {"name": "Related"}, "outwardIssue": {"key": "TC-300"}}], + }} + result = subprocess.run( + [sys.executable, os.path.join(script_dir, "pre_verify_pr.py"), "related-keys"], + input=json.dumps(issue), capture_output=True, text=True, + ) + assert result.returncode == 0, f"Exit {result.returncode}: {result.stderr}" + assert result.stdout.split() == ["TC-201", "TC-300"] + + +def test_idempotency_input_validates_against_schema(): + """A produced input carrying the idempotency bundle validates against the schema.""" + from jsonschema import validate + + issue = {"fields": {"summary": "S", "description": {}, "status": {"name": "Open"}, + "labels": [], "issuelinks": []}} + github = pre_verify_pr.build_github_bundle( + "o/r", 5, "feat/x", "deadbee", "diff", "stat", [], [], [], []) + idem = pre_verify_pr.build_idempotency_bundle([{"key": "TC-9", "fields": { + "summary": "Existing", "labels": ["review-feedback"], + "description": {"type": "doc"}, "issuetype": {"name": "Sub-task"}, + "comment": {"comments": [{"body": {"type": "doc"}}]}, + }}]) + result = pre_verify_pr.transform_to_input( + issue, "TC-1", "https://github.com/o/r/pull/5", github, idem) + schema_path = os.path.join( + script_dir, "..", "schemas", "verify-pr-input.schema.json") + with open(schema_path) as f: + schema = json.load(f) + validate(instance=result, schema=schema) # raises on failure + + # --- stat production (pre-verify-pr.sh) --- pre_verify_sh = os.path.join(script_dir, "pre-verify-pr.sh") @@ -332,6 +481,31 @@ def test_pre_verify_sh_uses_supported_stat_command(): assert "git apply --stat" in script, "expected git apply --stat stat mechanism" +# --- idempotency prefetch (pre-verify-pr.sh) --- + +def test_pre_verify_sh_prefetches_related_issues(): + """Regression guard: pre-verify-pr.sh fetches related-issue metadata for the + sandbox idempotency checks (TC-5982) — it lists related keys, fetches each + issue including its comments, and passes --idempotency-dir to transform. + """ + with open(pre_verify_sh) as f: + script = f.read() + + # It enumerates the task's related keys via pre_verify_pr.py related-keys + assert "related-keys" in script, "expected related-keys enumeration" + # It fetches each related issue including the comment field (for Step 7c) + non_comment = [ + line for line in script.splitlines() + if 'get_issue "${key}"' in line and not line.lstrip().startswith("#") + ] + assert non_comment, "expected a per-key get_issue fetch" + # The fields list feeding that fetch must include comment, description, labels + assert "summary,labels,description,issuetype,comment" in script, \ + "related-issue fetch must request comment/description/labels" + # And it wires the prefetched dir into transform + assert "--idempotency-dir" in script, "transform must receive --idempotency-dir" + + # --- paginated fetch aggregation (pre-verify-pr.sh) --- def test_paginated_pages_aggregate_into_flat_array(): diff --git a/plugins/sdlc-workflow/skills/verify-pr/SKILL.md b/plugins/sdlc-workflow/skills/verify-pr/SKILL.md index 120bcab83..a5de34924 100644 --- a/plugins/sdlc-workflow/skills/verify-pr/SKILL.md +++ b/plugins/sdlc-workflow/skills/verify-pr/SKILL.md @@ -148,6 +148,11 @@ If the file validates against the schema: the already-checked-out PR-head tree. **Skip every GitHub read step** — Step 3 (checkout), Step 4a (review/comment fetches), Step 5a (diff/stat/commits), and Step 9's HEAD-SHA retrieval — using the bundle fields instead. +- Read the `idempotency.related_issues` array (each entry has `key`, `summary`, + `labels`, `description`, `issuetype`, `comments`) — the task's existing sub-tasks + and linked issues, prefetched on the runner. **Use it for every idempotency read** + (Steps 6d, 6f, 7c) instead of calling Jira; the sandbox has no token. The array is + empty when the task has no sub-tasks or linked issues yet. If the validation above fails (file missing, not valid JSON, or not conforming to the schema), the handling depends on the mode: @@ -670,7 +675,9 @@ Process `create-sub-task` actions from the Correctness sub-agent. For each actio 1. **Idempotency check:** check the parent task's existing sub-tasks (issue links) for sub-tasks with labels `["ai-generated-jira", "review-feedback"]` whose descriptions reference the same CI check name or failure. If a matching sub-task - already exists, skip creation for that failure. + already exists, skip creation for that failure. **Sandbox mode:** read the + candidate sub-tasks from `idempotency.related_issues` (Step 0.7) — match on + `labels` and `description` — instead of a live Jira read. 2. **Create sub-task:** create a Jira sub-task using the action's Title, Relevant files, and Root cause fields: @@ -717,7 +724,9 @@ and sub-task creation below. 2. **Idempotency check:** Check the parent task's existing sub-tasks (issue links) for sub-tasks with labels `["ai-generated-jira", "eval-failure"]` whose summaries reference the same eval ID. If a matching sub-task already exists, skip creation - for that eval. + for that eval. **Sandbox mode:** read the candidate sub-tasks from + `idempotency.related_issues` (Step 0.7) — match on `labels` and `summary` — + instead of a live Jira read. 3. **Create sub-task:** For each failing eval, create a Jira sub-task: @@ -857,6 +866,8 @@ check the parent task's issue links for existing sub-tasks whose descriptions reference the same review comment or review body. If a matching sub-task already exists, skip creation. This guards against edge cases where a classification reply was not posted (e.g., due to a network error) but the sub-task was created. +**Sandbox mode:** read these existing sub-tasks from `idempotency.related_issues` +(Step 0.7) — inspect each entry's `description` — instead of a live Jira read. Do **not** interpret this step as a top-level decision that can skip item enumeration. The enumeration in Step 4a is always mandatory; this step only @@ -1115,6 +1126,11 @@ comments and search for a comment containing "Root-cause analysis from ". If a root-cause task already exists for the same phase and the same defect, skip creation. +**Sandbox mode:** do not fetch comments from Jira. Read the linked tasks from +`idempotency.related_issues` (Step 0.7), filter to entries whose `labels` include +`root-cause`, and search each entry's `comments` for the "Root-cause analysis from +" marker. + Record the Root-Cause Investigation result: - **N/A** — no sub-tasks were created in Step 6d (nothing to investigate) - **DONE** — investigation completed and root-cause tasks created From bb028cb80d71cc04f482907693c2b31da1f283a9 Mon Sep 17 00:00:00 2001 From: mrizzi Date: Wed, 2 Sep 2026 18:41:16 +0200 Subject: [PATCH 042/175] fix(verify-pr): post Jira report from report_adf, not report_md execute_post_report posted the GitHub-only report_md (with its embedded commit marker) to Jira and never read report_adf, contradicting SKILL.md Step 9's documented contract. Render the Jira comment body from report_adf via adf_to_markdown so the tracker-native report (and its Comment Footnote) is used; GitHub keeps report_md plus the commit-scoped marker. Implements TC-5983 Assisted-by: Claude Code Co-Authored-By: Claude Opus 4.8 --- .../sdlc-workflow/scripts/execute-actions.py | 10 +++-- .../scripts/test_execute_actions.py | 39 ++++++++++++++++++- 2 files changed, 44 insertions(+), 5 deletions(-) diff --git a/plugins/sdlc-workflow/scripts/execute-actions.py b/plugins/sdlc-workflow/scripts/execute-actions.py index 37b38666c..0ab0176c5 100755 --- a/plugins/sdlc-workflow/scripts/execute-actions.py +++ b/plugins/sdlc-workflow/scripts/execute-actions.py @@ -666,14 +666,18 @@ def execute_post_report(action: dict, registry: dict, report: dict) -> None: (``gh api ... -X PATCH``) instead of creating a duplicate, so a retry after a partial failure is idempotent while a new commit still gets a fresh comment. The Jira side is - already idempotent via its sticky marker; only ``report_md`` (without the - GitHub marker) is sent there. + already idempotent via its sticky marker; it receives the report rendered + from ``report_adf`` (the tracker-native body) via ``adf_to_markdown``, which + the native CLI converts back to ADF — the GitHub-only ``report_md`` (and its + embedded marker) is never sent to Jira. """ repo = report["pr_repo"] pr_number = report["pr_number"] jira_issue_id = report["jira_issue_id"] commit_sha = report["commit_sha"] report_md = resolve_refs(report["report_md"], registry) + report_adf = resolve_refs_in_obj(report["report_adf"], registry) + jira_body_md = adf_to_markdown(report_adf) marker = f"{GITHUB_REPORT_MARKER_PREFIX}{_normalize_commit_sha(commit_sha)} -->" github_body = f"{report_md}\n\n{marker}" @@ -699,7 +703,7 @@ def execute_post_report(action: dict, registry: dict, report: dict) -> None: sys.exit(1) print(f" Posted report to PR #{pr_number}") - post_jira_comment_native(jira_issue_id, report_md) + post_jira_comment_native(jira_issue_id, jira_body_md) print(f" Posted report to Jira {jira_issue_id}") diff --git a/plugins/sdlc-workflow/scripts/test_execute_actions.py b/plugins/sdlc-workflow/scripts/test_execute_actions.py index 8463a727f..7c118c726 100644 --- a/plugins/sdlc-workflow/scripts/test_execute_actions.py +++ b/plugins/sdlc-workflow/scripts/test_execute_actions.py @@ -616,6 +616,20 @@ def test_render_adf_date_falls_back_on_out_of_range_timestamp(): assert result == out_of_range, f"Got: {result!r}" +# A report's tracker-native body. execute_post_report renders this (not the +# GitHub-only report_md) to markdown for Jira via adf_to_markdown, so the fixtures +# below give it text that renders to a string distinct from report_md — proving +# Jira receives the ADF-derived body while GitHub keeps report_md + its marker. +_REPORT_ADF = { + "type": "doc", + "version": 1, + "content": [ + {"type": "paragraph", "content": [{"type": "text", "text": "Jira native body."}]} + ], +} +_REPORT_ADF_MD = "Jira native body." + + def test_execute_post_comment_routes_to_native(): """execute_post_comment resolves refs in body_adf, renders to markdown, and posts it.""" recorder = _RunRecorder() @@ -682,6 +696,7 @@ def fake_run(cmd, input=None, text=None, capture_output=None, env=None): "jira_issue_id": "TC-777", "commit_sha": "946556e", "report_md": "## Verify report", + "report_adf": _REPORT_ADF, } execute_actions.execute_post_report({"type": "post_report"}, {}, report) finally: @@ -720,6 +735,7 @@ def fake_run(cmd, input=None, text=None, capture_output=None, env=None): "jira_issue_id": "TC-777", "commit_sha": "946556e", "report_md": "## Verify report\nAll good.", + "report_adf": _REPORT_ADF, } execute_actions.execute_post_report({"type": "post_report"}, {}, report) finally: @@ -743,10 +759,14 @@ def fake_run(cmd, input=None, text=None, capture_output=None, env=None): gh_body = gh_call["cmd"][gh_call["cmd"].index("--body") + 1] assert gh_body.startswith("## Verify report\nAll good.") assert "" in gh_body - # Jira side is unchanged: sticky CLI, clean body without the GitHub marker. + # Jira side: sticky CLI, and the body is rendered from report_adf (the + # tracker-native content) via adf_to_markdown — NOT the GitHub-only report_md, + # which carries the commit marker Jira must never receive. assert jira_call["cmd"][:3] == ["fullsend", "issues", "post-comment"] assert jira_call["cmd"][jira_call["cmd"].index("--number") + 1] == "777" - assert jira_call["input"] == "## Verify report\nAll good." + assert jira_call["input"] == _REPORT_ADF_MD + assert jira_call["input"] != report["report_md"], "Jira must not receive report_md" + assert "sdlc-workflow:verify-pr report commit:" not in jira_call["input"] def test_execute_post_report_updates_existing_github_comment_on_retry(): @@ -775,6 +795,7 @@ def fake_run(cmd, input=None, text=None, capture_output=None, env=None): "jira_issue_id": "TC-777", "commit_sha": "946556e", "report_md": "## Verify report\nAll good.", + "report_adf": _REPORT_ADF, } # When posting the report again (e.g. after a prior Jira failure) execute_actions.execute_post_report({"type": "post_report"}, {}, report) @@ -832,6 +853,7 @@ def fake_run(cmd, input=None, text=None, capture_output=None, env=None): "jira_issue_id": "TC-777", "commit_sha": "946556e", "report_md": "## Verify report\nAll good.", + "report_adf": _REPORT_ADF, } # When posting the report again for the same commit execute_actions.execute_post_report({"type": "post_report"}, {}, report) @@ -877,6 +899,7 @@ def fake_run(cmd, input=None, text=None, capture_output=None, env=None): "jira_issue_id": "TC-777", "commit_sha": commit_sha, "report_md": "## Verify report", + "report_adf": _REPORT_ADF, } execute_actions.execute_post_report({"type": "post_report"}, {}, report) finally: @@ -1000,6 +1023,14 @@ def fake_create_issue(**kwargs): "jira_issue_id": "TC-777", "commit_sha": "946556e", "report_md": "Filed sub-task {{sub-1.key}}.", + "report_adf": { + "type": "doc", + "version": 1, + "content": [ + {"type": "paragraph", + "content": [{"type": "text", "text": "Filed sub-task {{sub-1.key}}."}]} + ], + }, }, "actions": [ {"type": "post_report"}, @@ -1043,6 +1074,10 @@ def fake_create_issue(**kwargs): gh_body = create_call["cmd"][create_call["cmd"].index("--body") + 1] assert "Filed sub-task TC-999." in gh_body, f"ref not resolved: {gh_body}" assert "{{sub-1.key}}" not in gh_body, "placeholder leaked into report body" + # The Jira body is rendered from report_adf, and its refs resolve too. + jira_call = next(c for c in calls if c["cmd"][:3] == ["fullsend", "issues", "post-comment"]) + assert jira_call["input"] == "Filed sub-task TC-999.", f"ref not resolved in ADF: {jira_call['input']}" + assert "{{sub-1.key}}" not in jira_call["input"], "placeholder leaked into Jira body" def test_find_report_comment_id_exits_on_unparseable_json(): From e526abd3ae4624691e4c4d591f4478371f363fc4 Mon Sep 17 00:00:00 2001 From: mrizzi Date: Thu, 3 Sep 2026 14:19:38 +0200 Subject: [PATCH 043/175] fix(verify-pr): require idempotency bundle in sandbox prefetch schema MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The TC-5982 fix added an optional top-level `idempotency` object to verify-pr-input.schema.json, but the sandbox SKILL.md steps read `idempotency.related_issues` unconditionally for the Steps 6d/6f/7c dedup checks. A prefetch that passed Step 0.7 validation while omitting the bundle left those tokenless checks with no data source, silently skipping dedup and creating duplicate sub-tasks / root-cause tasks on reruns. Option 1 (require it): the producer always runs on the trusted runner with a Jira token, so it can always emit the bundle (empty related_issues when the task has no relations). Make the schema require `idempotency` and its `related_issues`, so Step 0.7 rejects a prefetch lacking it before any consuming step runs — consistent with the TC-5980/TC-5981 fail-fast contract. transform_to_input now always emits the bundle; SKILL.md states the invariant. Implements TC-6026 Co-Authored-By: Claude Opus 4.8 Assisted-by: Claude Code --- .../schemas/verify-pr-input.schema.json | 5 ++- .../sdlc-workflow/scripts/pre_verify_pr.py | 14 ++++-- .../scripts/test_pre_verify_pr.py | 43 ++++++++++++++++++- .../sdlc-workflow/skills/verify-pr/SKILL.md | 8 +++- 4 files changed, 61 insertions(+), 9 deletions(-) diff --git a/plugins/sdlc-workflow/schemas/verify-pr-input.schema.json b/plugins/sdlc-workflow/schemas/verify-pr-input.schema.json index 33cb26965..ee093c8f8 100644 --- a/plugins/sdlc-workflow/schemas/verify-pr-input.schema.json +++ b/plugins/sdlc-workflow/schemas/verify-pr-input.schema.json @@ -4,7 +4,7 @@ "title": "Verify PR Pre-Script Input", "description": "Pre-fetched task data written by the pre_script and mounted into the sandbox. The skill reads this instead of calling the issue tracker or GitHub APIs directly.", "type": "object", - "required": ["task_id", "task", "pr_url", "github"], + "required": ["task_id", "task", "pr_url", "github", "idempotency"], "additionalProperties": false, "properties": { "task_id": { @@ -97,8 +97,9 @@ }, "idempotency": { "type": "object", - "description": "Related-issue metadata prefetched on the trusted runner so the sandbox can run the Steps 6d/6f/7c idempotency checks (duplicate sub-task and root-cause detection) without a Jira token or egress. Optional: absent or empty when the task has no sub-tasks or linked issues yet.", + "description": "Related-issue metadata prefetched on the trusted runner so the sandbox can run the Steps 6d/6f/7c idempotency checks (duplicate sub-task and root-cause detection) without a Jira token or egress. Always present in a valid prefetch: the producer runs on the trusted runner with a Jira token, so it can always emit the bundle — with an empty related_issues array when the task has no sub-tasks or linked issues yet. Requiring it (rather than leaving it optional) closes the gap that let a schema-valid prefetch omit the bundle and silently skip dedup in the tokenless sandbox.", "additionalProperties": false, + "required": ["related_issues"], "properties": { "related_issues": { "type": "array", diff --git a/plugins/sdlc-workflow/scripts/pre_verify_pr.py b/plugins/sdlc-workflow/scripts/pre_verify_pr.py index 037ed0153..b9aa39582 100644 --- a/plugins/sdlc-workflow/scripts/pre_verify_pr.py +++ b/plugins/sdlc-workflow/scripts/pre_verify_pr.py @@ -123,7 +123,14 @@ def build_idempotency_bundle(related_issue_jsons): def transform_to_input(issue, task_id, pr_url, github=None, idempotency=None): - """Transform Jira issue JSON to tracker-agnostic input schema.""" + """Transform Jira issue JSON to tracker-agnostic input schema. + + The ``idempotency`` bundle is always emitted (defaulting to an empty + ``related_issues`` list when none is supplied) so the tokenless sandbox + always has a data source for the Steps 6d/6f/7c dedup checks. This matches + verify-pr-input.schema.json, which requires ``idempotency.related_issues``: + a schema-valid prefetch can never omit the bundle and silently skip dedup. + """ fields = issue.get("fields", {}) result = { "task_id": task_id, @@ -159,8 +166,9 @@ def transform_to_input(issue, task_id, pr_url, github=None, idempotency=None): } if github is not None: result["github"] = github - if idempotency is not None: - result["idempotency"] = idempotency + result["idempotency"] = ( + idempotency if idempotency is not None else {"related_issues": []} + ) return result diff --git a/plugins/sdlc-workflow/scripts/test_pre_verify_pr.py b/plugins/sdlc-workflow/scripts/test_pre_verify_pr.py index fbb3675d0..9af5aef70 100644 --- a/plugins/sdlc-workflow/scripts/test_pre_verify_pr.py +++ b/plugins/sdlc-workflow/scripts/test_pre_verify_pr.py @@ -339,10 +339,16 @@ def test_transform_with_idempotency_attaches_key(): assert result["idempotency"]["related_issues"][0]["key"] == "TC-1" -def test_transform_without_idempotency_omits_key(): +def test_transform_without_idempotency_defaults_to_empty(): + """With no idempotency supplied, transform still emits an empty bundle. + + Option 1 of TC-6026: the key is always present so the tokenless sandbox + always has a data source for the Steps 6d/6f/7c dedup checks — a missing + argument degrades to "no known duplicates", never an omitted key. + """ issue = {"fields": {"summary": "S", "status": {"name": "Open"}, "labels": [], "issuelinks": []}} result = pre_verify_pr.transform_to_input(issue, "TC-1", "") - assert "idempotency" not in result + assert result["idempotency"] == {"related_issues": []} def test_cli_transform_idempotency_dir(): @@ -420,6 +426,39 @@ def test_idempotency_input_validates_against_schema(): validate(instance=result, schema=schema) # raises on failure +def test_prefetch_omitting_idempotency_fails_schema_validation(): + """A prefetch lacking the idempotency bundle is rejected by the schema. + + Option 1 of TC-6026: idempotency is a top-level required key, so Step 0.7 + validation rejects a bundle that omits it *before* any consuming step runs — + the fail-fast contract from TC-5980/TC-5981. This closes the gap that let a + schema-valid prefetch skip the tokenless dedup checks silently. + """ + from jsonschema import validate + from jsonschema.exceptions import ValidationError + + # Given an otherwise-valid input (task + github) with no idempotency key + issue = {"fields": {"summary": "S", "description": {}, "status": {"name": "Open"}, + "labels": [], "issuelinks": []}} + github = pre_verify_pr.build_github_bundle( + "o/r", 5, "feat/x", "deadbee", "diff", "stat", [], [], [], []) + instance = pre_verify_pr.transform_to_input( + issue, "TC-1", "https://github.com/o/r/pull/5", github) + del instance["idempotency"] # simulate an older/hand-supplied bundle + + schema_path = os.path.join( + script_dir, "..", "schemas", "verify-pr-input.schema.json") + with open(schema_path) as f: + schema = json.load(f) + + # When validating it against the input schema, Then it is rejected + try: + validate(instance=instance, schema=schema) + assert False, "schema accepted a prefetch missing idempotency" + except ValidationError as e: + assert "idempotency" in str(e), f"unexpected error: {e}" + + # --- stat production (pre-verify-pr.sh) --- pre_verify_sh = os.path.join(script_dir, "pre-verify-pr.sh") diff --git a/plugins/sdlc-workflow/skills/verify-pr/SKILL.md b/plugins/sdlc-workflow/skills/verify-pr/SKILL.md index a5de34924..5b7bb2501 100644 --- a/plugins/sdlc-workflow/skills/verify-pr/SKILL.md +++ b/plugins/sdlc-workflow/skills/verify-pr/SKILL.md @@ -151,8 +151,12 @@ If the file validates against the schema: - Read the `idempotency.related_issues` array (each entry has `key`, `summary`, `labels`, `description`, `issuetype`, `comments`) — the task's existing sub-tasks and linked issues, prefetched on the runner. **Use it for every idempotency read** - (Steps 6d, 6f, 7c) instead of calling Jira; the sandbox has no token. The array is - empty when the task has no sub-tasks or linked issues yet. + (Steps 6d, 6f, 7c) instead of calling Jira; the sandbox has no token. The schema + **requires** `idempotency.related_issues`, so a prefetch that passes the Step 0.7 + validation above always carries this array — it is an empty list (never absent) + when the task has no sub-tasks or linked issues yet. Treat a present-but-empty + array as "no known duplicates"; you never need to fall back to a Jira read for + the dedup checks. If the validation above fails (file missing, not valid JSON, or not conforming to the schema), the handling depends on the mode: From e82740223e4e3988ad0f471c1080fb0fb24ce14b Mon Sep 17 00:00:00 2001 From: mrizzi Date: Thu, 3 Sep 2026 16:08:44 +0200 Subject: [PATCH 044/175] fix(verify-pr): require per-item fields in idempotency.related_issues MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The idempotency.related_issues array is required (TC-6026) but its items declared properties without a required list, so a prefetched related-issue entry missing key/summary/labels/description/issuetype/comments passed Step 0.7 schema validation. The tokenless-sandbox dedup checks (Steps 6d/6f/7c) then silently treated the incomplete entry as non-matching, letting duplicate sub-tasks and root-cause tasks be created on reruns — the item-level analogue of the array-level gap TC-6026 closed. Add a per-item required list of the six fields the dedup checks consume. build_idempotency_bundle already emits all six unconditionally, so the producer stays schema-valid. Add a test asserting an incomplete entry raises ValidationError. TC-6032 Assisted-by: Claude Code Co-Authored-By: Claude Opus 4.8 --- .../schemas/verify-pr-input.schema.json | 1 + .../scripts/test_pre_verify_pr.py | 37 +++++++++++++++++++ 2 files changed, 38 insertions(+) diff --git a/plugins/sdlc-workflow/schemas/verify-pr-input.schema.json b/plugins/sdlc-workflow/schemas/verify-pr-input.schema.json index ee093c8f8..b53431b84 100644 --- a/plugins/sdlc-workflow/schemas/verify-pr-input.schema.json +++ b/plugins/sdlc-workflow/schemas/verify-pr-input.schema.json @@ -106,6 +106,7 @@ "description": "The task's existing sub-tasks and linked issues (e.g., root-cause tasks), each with the fields the dedup checks inspect.", "items": { "type": "object", + "required": ["key", "summary", "labels", "description", "issuetype", "comments"], "properties": { "key": { "type": "string", "description": "Issue key (e.g., TC-4742)" }, "summary": { "type": "string" }, diff --git a/plugins/sdlc-workflow/scripts/test_pre_verify_pr.py b/plugins/sdlc-workflow/scripts/test_pre_verify_pr.py index 9af5aef70..e11ca5873 100644 --- a/plugins/sdlc-workflow/scripts/test_pre_verify_pr.py +++ b/plugins/sdlc-workflow/scripts/test_pre_verify_pr.py @@ -459,6 +459,43 @@ def test_prefetch_omitting_idempotency_fails_schema_validation(): assert "idempotency" in str(e), f"unexpected error: {e}" +def test_incomplete_related_issue_fails_schema_validation(): + """A related-issue entry missing a per-item field is rejected by the schema. + + TC-6032: idempotency.related_issues.items requires the fields the dedup + checks consume (Steps 6d/6f/7c). Without a per-item `required` list, a + structurally incomplete entry passed Step 0.7 and dedup silently treated it + as non-matching — the item-level analogue of the TC-6026 array-level gap. + """ + from jsonschema import validate + from jsonschema.exceptions import ValidationError + + # Given an otherwise-valid input whose sole related issue omits `comments` + issue = {"fields": {"summary": "S", "description": {}, "status": {"name": "Open"}, + "labels": [], "issuelinks": []}} + github = pre_verify_pr.build_github_bundle( + "o/r", 5, "feat/x", "deadbee", "diff", "stat", [], [], [], []) + instance = pre_verify_pr.transform_to_input( + issue, "TC-1", "https://github.com/o/r/pull/5", github) + instance["idempotency"] = {"related_issues": [{ + "key": "TC-9", "summary": "Existing", "labels": [], + "description": {}, "issuetype": "Sub-task", + # `comments` intentionally omitted + }]} + + schema_path = os.path.join( + script_dir, "..", "schemas", "verify-pr-input.schema.json") + with open(schema_path) as f: + schema = json.load(f) + + # When validating it against the input schema, Then it is rejected + try: + validate(instance=instance, schema=schema) + assert False, "schema accepted a related issue missing a required field" + except ValidationError as e: + assert "comments" in str(e), f"unexpected error: {e}" + + # --- stat production (pre-verify-pr.sh) --- pre_verify_sh = os.path.join(script_dir, "pre-verify-pr.sh") From e696d00be2f1df81d8dd254e10a9d9b771fd7ddc Mon Sep 17 00:00:00 2001 From: mrizzi Date: Thu, 3 Sep 2026 16:08:52 +0200 Subject: [PATCH 045/175] fix(verify-pr): fail fast on exported-but-empty FULLSEND_OUTPUT_DIR MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Step 0.6 detected sandbox mode with `echo ${FULLSEND_OUTPUT_DIR:-not-set}`. The `:-` operator treats an exported-but-empty value the same as unset, so a runner exporting the gate with an empty value would select the credentialed interactive path despite the sandbox gate being present — the opposite of the intended tokenless behavior. Use presence detection (`${FULLSEND_OUTPUT_DIR+x}`) and reject an empty value with a non-zero exit. Sandbox mode is selected whenever the variable is present and non-empty; interactive mode only when it is genuinely unset. TC-6033 Assisted-by: Claude Code Co-Authored-By: Claude Opus 4.8 --- .../sdlc-workflow/skills/verify-pr/SKILL.md | 28 +++++++++++++++---- 1 file changed, 22 insertions(+), 6 deletions(-) diff --git a/plugins/sdlc-workflow/skills/verify-pr/SKILL.md b/plugins/sdlc-workflow/skills/verify-pr/SKILL.md index 5b7bb2501..3429a1e26 100644 --- a/plugins/sdlc-workflow/skills/verify-pr/SKILL.md +++ b/plugins/sdlc-workflow/skills/verify-pr/SKILL.md @@ -60,14 +60,29 @@ Refer to `shared/jira-rest-fallback.md` for complete implementation details. ## Step 0.6 – Sandbox Mode Detection -Check whether the `FULLSEND_OUTPUT_DIR` environment variable is set: +Detect whether the `FULLSEND_OUTPUT_DIR` environment variable is **present** in the +environment (exported at all), independently of whether it holds a value. Use +presence detection (`${VAR+x}`), not a default-value expansion (`${VAR:-...}`): the +`:-` operator treats an exported-but-empty value the same as unset, which would send +a runner that exported the gate with an empty value down the credentialed +interactive path — the opposite of the intended tokenless behavior. An +exported-but-empty value is a misconfiguration and must fail fast, not silently fall +back: ```bash -echo ${FULLSEND_OUTPUT_DIR:-not-set} +if [ "${FULLSEND_OUTPUT_DIR+x}" = x ]; then + if [ -z "$FULLSEND_OUTPUT_DIR" ]; then + echo "ERROR: FULLSEND_OUTPUT_DIR is set but empty" >&2 + exit 1 + fi + echo "sandbox mode: $FULLSEND_OUTPUT_DIR" +else + echo "interactive mode" +fi ``` -If **set**, this skill is running inside a fullsend sandbox (no Jira or GitHub -tokens, no `gh` CLI, no network egress). Switch to **sandbox mode**: +If **present** (and non-empty), this skill is running inside a fullsend sandbox (no +Jira or GitHub tokens, no `gh` CLI, no network egress). Switch to **sandbox mode**: - Do NOT call Jira write APIs (`create_issue`, `add_comment`, `create_issue_link`, `transition_issue`) directly. @@ -79,8 +94,9 @@ tokens, no `gh` CLI, no network egress). Switch to **sandbox mode**: structure, and at the end of execution write the complete result to `$FULLSEND_OUTPUT_DIR/agent-result.json` (see Step 9's **Sandbox Mode Output**). -If **not set**, execute in **interactive mode** — the current behavior, calling Jira -and GitHub APIs directly. Marketplace users are unaffected. +If **genuinely unset** (absent from the environment), execute in **interactive +mode** — the current behavior, calling Jira and GitHub APIs directly. Marketplace +users are unaffected. Throughout the remaining steps, when you encounter a write operation: - **Interactive mode:** execute it directly (existing behavior). From 34e6362f8177dd4694140fa4e9f40ff24bb96032 Mon Sep 17 00:00:00 2001 From: mrizzi Date: Fri, 4 Sep 2026 15:18:40 +0200 Subject: [PATCH 046/175] feat(verify-pr): make URL-pinnable base harness + root plugin.json marker MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Two changes so the root harness resolves cleanly when consumed via a pinned raw URL (config.yaml / a child's `base:`), enabling byte-identical local+CI runs off one pin: 1. Strip the runner-local absolute host_files (GCP credential/OIDC mounts and the pre_script prefetch output) from harness/verify-pr.yaml. fullsend v0.37.0 rejects an absolute host_files.src inherited from a URL-sourced harness (ADR-0038); only the repo-relative gcp-vertex.env mount remains. The absolute mounts move to a local composing child (.fullsend/harness/verify-pr.yaml) whose `base:` pins this file by raw URL. 2. Add plugins/sdlc-workflow/plugin.json as an inert marker. fullsend's URL plugin resolver (internal/harness/compose.go fetchBasePluginDir) requires a root-level plugin.json before it sparse-checks-out the whole plugin dir; it only checks existence and never parses it. The marker carries no metadata (nothing to keep in sync, no drift); the canonical Claude Code manifest stays at .claude-plugin/plugin.json (read by Claude Code, local-path resolution, and `claude plugin validate`). A symlink was ruled out — fullsend gitfetch rejects symlinks ("gitfetch: symlinks are not supported"). The harness stays credential-source-agnostic (same env.runner/env.sandbox), so BYO-local and mint+WIF-CI differ only in runtime env, not harness content. TC-5813 Assisted-by: Claude Code Co-Authored-By: Claude Opus 4.8 --- harness/verify-pr.yaml | 18 ++++++++---------- plugins/sdlc-workflow/plugin.json | 3 +++ 2 files changed, 11 insertions(+), 10 deletions(-) create mode 100644 plugins/sdlc-workflow/plugin.json diff --git a/harness/verify-pr.yaml b/harness/verify-pr.yaml index 8367df650..e68b6ea85 100644 --- a/harness/verify-pr.yaml +++ b/harness/verify-pr.yaml @@ -48,20 +48,18 @@ openshell: profiles: - plugins/sdlc-workflow/profiles/fullsend-vertex-ai.yaml +# URL-pinned base: only repo-relative host_files may live here. When this harness +# is consumed via a pinned raw URL (config.yaml / a child's `base:`), fullsend +# fetches these entries from the base URL and rewrites their paths (ADR-0038). +# Runner-local absolute mounts — the GCP credential/OIDC files and the pre_script +# prefetch output — CANNOT be URL-sourced (fullsend v0.37.0 rejects an absolute +# host_files.src that is inherited from a URL-sourced harness). They are declared +# instead in the local composing child (.fullsend/harness/verify-pr.yaml), whose +# host_files are concatenated with these (dedup by dest, child wins). host_files: - src: plugins/sdlc-workflow/env/gcp-vertex.env dest: /sandbox/workspace/.env.d/gcp-vertex.env expand: true - - src: ${GOOGLE_APPLICATION_CREDENTIALS} - dest: /tmp/.gcp-credentials.json - - src: ${GCP_OIDC_TOKEN_FILE} - dest: /sandbox/workspace/.gcp-oidc-token - optional: true - # pre_script prefetch output, mounted read-only into the sandbox (optional — - # absent until pre-verify-pr.sh runs, authored in TC-5810). - - src: /tmp/fullsend-pre-output/verify-pr-input.json - dest: /sandbox/workspace/.pre-script/verify-pr-input.json - optional: true pre_script: plugins/sdlc-workflow/scripts/pre-verify-pr.sh post_script: plugins/sdlc-workflow/scripts/post-verify-pr.sh diff --git a/plugins/sdlc-workflow/plugin.json b/plugins/sdlc-workflow/plugin.json new file mode 100644 index 000000000..8292622b4 --- /dev/null +++ b/plugins/sdlc-workflow/plugin.json @@ -0,0 +1,3 @@ +{ + "_comment": "TEMPORARY workaround for fullsend bug — tracked upstream at https://github.com/fullsend-ai/fullsend/issues/7008. fullsend's URL plugin resolver (internal/harness/compose.go fetchBasePlugin/fetchBasePluginDir) requires a plugin.json at the plugin-directory ROOT before it sparse-checks-out the plugin over a pinned raw URL. fullsend only checks that this file EXISTS — it never parses it (and its runtime derives the plugin name from the directory basename, not this file). A root-level plugin.json is INVALID per the Claude Code plugin spec, which mandates .claude-plugin/plugin.json; that canonical manifest (name/description/version/author) is what Claude Code, local-path resolution, `claude plugin validate`, and the marketplace read — edit THAT one, not this. This marker intentionally carries no metadata, so there is nothing to keep in sync and no drift. A symlink is not an option: fullsend's gitfetch rejects symlinks. Remove this file once #7008 ships a fix that accepts .claude-plugin/plugin.json." +} From 888a1781dd81f68bc6f4daecdcccef5d91b9cef6 Mon Sep 17 00:00:00 2001 From: mrizzi Date: Mon, 7 Sep 2026 16:20:11 +0200 Subject: [PATCH 047/175] feat(verify-pr): post GitHub report + Jira comments via native fullsend CLI Replace the gh-CLI GitHub report-comment path in execute-actions.py with native `fullsend issues post-comment`, unifying report + Jira comments + standalone analysis comments onto one idempotent code path. Irreducible Jira writes (sub-tasks, links, root-cause) still use jira-client.py; remaining GitHub PR ops (review-comment reply, per-item comment) still use gh. The native CLI prepends the --marker and edits the marked comment in place on reruns (collapsing prior content into
), giving per-commit sticky idempotency without manual comment-ID lookup. The report marker is commit-scoped (GITHUB_REPORT_MARKER_PREFIX + canonical 40-char SHA via `git rev-parse`), so a new commit posts a fresh comment while a re-verify of the same commit edits. execute_post_report strips the leading marker line the agent embeds in report_md before handing the body to the CLI, avoiding a doubled marker. Jira receives adf_to_markdown(report_adf), never report_md. _find_report_comment_id and the gh-api pagination/PATCH path are deleted. Rewrite the affected unit tests for the native flow (36 pass). TC-5813 Assisted-by: Claude Code Co-Authored-By: Claude Opus 4.8 --- .../sdlc-workflow/scripts/execute-actions.py | 143 +++++----- .../scripts/test_execute_actions.py | 251 ++++++------------ 2 files changed, 145 insertions(+), 249 deletions(-) diff --git a/plugins/sdlc-workflow/scripts/execute-actions.py b/plugins/sdlc-workflow/scripts/execute-actions.py index 0ab0176c5..31e87c320 100755 --- a/plugins/sdlc-workflow/scripts/execute-actions.py +++ b/plugins/sdlc-workflow/scripts/execute-actions.py @@ -4,12 +4,15 @@ Reads agent-result.json, processes actions sequentially, resolves {{ref.key}} and {{ref.url}} placeholders as entities are created. -Jira comments (post_comment / post_report) are posted through the native -`fullsend issues post-comment` sticky-comment CLI. The irreducible Jira -writes that have no native primitive yet — sub-tasks, links, and root-cause -tasks — call jira-client.py functions directly (imported as a module). -GitHub operations use the gh CLI. Runs on the fullsend runner (trusted -side), not inside the sandbox. +Comments — the verify-pr report on both the GitHub PR and the Jira issue, plus +standalone Jira analysis comments — are posted through the native +`fullsend issues post-comment` sticky-comment CLI (`--tracker github|jira`). +The irreducible Jira writes that have no native primitive yet — sub-tasks, +links, and root-cause tasks — call jira-client.py functions directly (imported +as a module). The remaining GitHub PR operations that have no native fullsend +command — replying to a review-comment thread and posting a per-item PR comment +— still use the gh CLI. Runs on the fullsend runner (trusted side), not inside +the sandbox. Not idempotent for entity creation: if an action fails mid-execution, previously created Jira sub-tasks are not rolled back. Manual cleanup may be @@ -58,12 +61,14 @@ # its own stable marker, so per-path re-run idempotency is preserved. POST_COMMENT_STICKY_MARKER = "" -# GitHub has no native sticky-comment mechanism, so the verify-pr report comment -# embeds this marker (an invisible HTML comment) in its body. A canonical -# (full-length) commit SHA is appended per post, scoping dedup to a single -# verification run/commit: a retry for the same commit updates the existing -# comment instead of duplicating it, while a later commit gets a fresh comment — -# preserving the per-run verification history that verify-pr SKILL.md Step 9 posts. +# The verify-pr report comment is posted via the native fullsend sticky-comment +# CLI (`fullsend issues post-comment --tracker github`), which prepends this +# marker as a hidden HTML comment and, on re-runs, finds the marked comment and +# edits it in place. A canonical (full-length) commit SHA is appended per post, +# scoping the sticky identity to a single verification run/commit: a retry for +# the same commit updates that comment instead of duplicating it, while a later +# commit gets a fresh comment — preserving the per-run verification history that +# verify-pr SKILL.md Step 9 posts. GITHUB_REPORT_MARKER_PREFIX = "" - github_body = f"{report_md}\n\n{marker}" - existing_id = _find_report_comment_id(repo, pr_number, marker) - if existing_id is not None: - result = subprocess.run( - ["gh", "api", f"repos/{repo}/issues/comments/{existing_id}", - "-X", "PATCH", "-f", f"body={github_body}"], - capture_output=True, text=True, - ) - if result.returncode != 0: - print(f"Failed to update GitHub PR comment: {result.stderr}", file=sys.stderr) - sys.exit(1) - print(f" Updated existing report comment on PR #{pr_number}") - else: - result = subprocess.run( - ["gh", "pr", "comment", str(pr_number), "--body", github_body, "-R", repo], - capture_output=True, text=True, - ) - if result.returncode != 0: - print(f"Failed to post GitHub PR comment: {result.stderr}", file=sys.stderr) - sys.exit(1) - print(f" Posted report to PR #{pr_number}") + # The CLI prepends the marker; drop the agent's embedded leading marker line + # (any commit-scoped variant, matched by prefix) so the comment does not open + # with two duplicate marker lines. + if report_md.startswith(GITHUB_REPORT_MARKER_PREFIX): + report_md = report_md.split("\n", 1)[1] if "\n" in report_md else "" + report_md = report_md.lstrip("\n") + + post_github_comment_native(repo, pr_number, report_md, marker) + print(f" Posted report to PR #{pr_number}") post_jira_comment_native(jira_issue_id, jira_body_md) print(f" Posted report to Jira {jira_issue_id}") diff --git a/plugins/sdlc-workflow/scripts/test_execute_actions.py b/plugins/sdlc-workflow/scripts/test_execute_actions.py index 7c118c726..855789b5e 100644 --- a/plugins/sdlc-workflow/scripts/test_execute_actions.py +++ b/plugins/sdlc-workflow/scripts/test_execute_actions.py @@ -678,12 +678,12 @@ def test_post_comment_and_report_use_distinct_sticky_markers(): restore() comment_marker = comment_recorder.cmd[comment_recorder.cmd.index("--marker") + 1] - # And the report path (no existing GitHub comment → lists then posts) + # And the report path (native GitHub comment + native Jira comment) report_calls = [] def fake_run(cmd, input=None, text=None, capture_output=None, env=None): report_calls.append(cmd) - return _FakeCompleted(0, "", "[]") + return _FakeCompleted(0, "") saved_run = execute_actions.subprocess.run saved_env = {k: os.environ.get(k) for k in _JIRA_ENV} @@ -706,7 +706,9 @@ def fake_run(cmd, input=None, text=None, capture_output=None, env=None): os.environ.pop(k, None) else: os.environ[k] = v - jira_cmd = next(c for c in report_calls if c[:3] == ["fullsend", "issues", "post-comment"]) + jira_cmd = next(c for c in report_calls + if c[:3] == ["fullsend", "issues", "post-comment"] + and c[c.index("--tracker") + 1] == "jira") report_marker = jira_cmd[jira_cmd.index("--marker") + 1] # Then the two markers differ, and each matches its dedicated constant @@ -716,13 +718,13 @@ def fake_run(cmd, input=None, text=None, capture_output=None, env=None): def test_execute_post_report_posts_github_then_jira(): - """execute_post_report lists PR comments, creates a marked GitHub comment when none exists, then posts to Jira.""" + """execute_post_report posts the report to the GitHub PR then to Jira, both via + the native `fullsend issues post-comment` sticky CLI (GitHub first).""" calls = [] def fake_run(cmd, input=None, text=None, capture_output=None, env=None): calls.append({"cmd": cmd, "input": input, "env": env}) - # No existing report comment on the PR yet. - return _FakeCompleted(0, "", "[]") + return _FakeCompleted(0, "") saved_run = execute_actions.subprocess.run saved_env = {k: os.environ.get(k) for k in _JIRA_ENV} @@ -746,100 +748,41 @@ def fake_run(cmd, input=None, text=None, capture_output=None, env=None): else: os.environ[k] = v - # SHA canonicalization (git rev-parse) + list + create + native. - assert len(calls) == 4, f"Expected rev-parse + list + create + native calls, got {len(calls)}" - list_call = next(c for c in calls if c["cmd"][:2] == ["gh", "api"]) - gh_call = next(c for c in calls if c["cmd"][:3] == ["gh", "pr", "comment"]) - jira_call = next(c for c in calls if c["cmd"][:3] == ["fullsend", "issues", "post-comment"]) - # Existing comments are listed to check for a prior report. - assert list_call["cmd"][2] == "repos/acme/widget/issues/42/comments" - # No existing comment → a new PR comment is created, carrying the commit marker. - assert gh_call["cmd"][:3] == ["gh", "pr", "comment"] - assert "acme/widget" in gh_call["cmd"] - gh_body = gh_call["cmd"][gh_call["cmd"].index("--body") + 1] - assert gh_body.startswith("## Verify report\nAll good.") - assert "" in gh_body + # SHA canonicalization (git rev-parse) + native GitHub + native Jira. + assert len(calls) == 3, f"Expected rev-parse + github + jira calls, got {len(calls)}" + native = [c for c in calls if c["cmd"][:3] == ["fullsend", "issues", "post-comment"]] + gh_call = next(c for c in native if c["cmd"][c["cmd"].index("--tracker") + 1] == "github") + jira_call = next(c for c in native if c["cmd"][c["cmd"].index("--tracker") + 1] == "jira") + # GitHub side: native sticky CLI targets the PR by number, the marker carries + # the commit SHA, and the body (stdin) is report_md with NO embedded marker + # (the CLI prepends it). + assert gh_call["cmd"][gh_call["cmd"].index("--project") + 1] == "acme/widget" + assert gh_call["cmd"][gh_call["cmd"].index("--number") + 1] == "42" + assert gh_call["cmd"][gh_call["cmd"].index("--marker") + 1] == \ + "" + assert gh_call["input"] == "## Verify report\nAll good." + assert "sdlc-workflow:verify-pr report commit:" not in gh_call["input"], \ + "marker must not be embedded in the GitHub body; the CLI prepends it" + # GitHub is posted before Jira. + assert native[0] is gh_call, "GitHub report must be posted before Jira" # Jira side: sticky CLI, and the body is rendered from report_adf (the - # tracker-native content) via adf_to_markdown — NOT the GitHub-only report_md, + # tracker-native content) via adf_to_markdown, NOT the GitHub-only report_md, # which carries the commit marker Jira must never receive. - assert jira_call["cmd"][:3] == ["fullsend", "issues", "post-comment"] assert jira_call["cmd"][jira_call["cmd"].index("--number") + 1] == "777" assert jira_call["input"] == _REPORT_ADF_MD assert jira_call["input"] != report["report_md"], "Jira must not receive report_md" assert "sdlc-workflow:verify-pr report commit:" not in jira_call["input"] -def test_execute_post_report_updates_existing_github_comment_on_retry(): - """A retry for the same commit PATCH-updates the existing GitHub report comment instead of creating a duplicate.""" - # Given a prior report comment for this commit already exists on the PR. - # `gh api --paginate --slurp` wraps each page's comment array in one outer - # array, so the listing is a single-page array-of-pages here. - calls = [] - existing_page = [{"id": 555, - "body": "old report\n\n"}] - - def fake_run(cmd, input=None, text=None, capture_output=None, env=None): - calls.append({"cmd": cmd, "input": input, "env": env}) - if cmd[:2] == ["gh", "api"] and cmd[2].endswith("/comments"): - return _FakeCompleted(0, "", json.dumps([existing_page])) - return _FakeCompleted(0, "") - - saved_run = execute_actions.subprocess.run - saved_env = {k: os.environ.get(k) for k in _JIRA_ENV} - execute_actions.subprocess.run = fake_run - os.environ.update(_JIRA_ENV) - try: - report = { - "pr_repo": "acme/widget", - "pr_number": 42, - "jira_issue_id": "TC-777", - "commit_sha": "946556e", - "report_md": "## Verify report\nAll good.", - "report_adf": _REPORT_ADF, - } - # When posting the report again (e.g. after a prior Jira failure) - execute_actions.execute_post_report({"type": "post_report"}, {}, report) - finally: - execute_actions.subprocess.run = saved_run - for k, v in saved_env.items(): - if v is None: - os.environ.pop(k, None) - else: - os.environ[k] = v - - # Then no new GitHub comment is created; the existing one is PATCH-updated - assert not any(c["cmd"][:3] == ["gh", "pr", "comment"] for c in calls), \ - "retry must not create a new GitHub comment" - patch_calls = [c for c in calls if "PATCH" in c["cmd"]] - assert len(patch_calls) == 1, f"Expected one PATCH update, got {len(patch_calls)}" - assert patch_calls[0]["cmd"][2] == "repos/acme/widget/issues/comments/555" - # And the Jira report is still posted - assert any(c["cmd"][:3] == ["fullsend", "issues", "post-comment"] for c in calls), \ - "Jira report must still be posted on retry" - - -def test_execute_post_report_updates_comment_on_later_page(): - """When the listing spans multiple pages, an existing report comment on a - non-first page is still found and PATCH-updated (no duplicate created).""" - # Given a slurped, multi-page listing (array-of-pages) where the marked report - # comment lives on the SECOND page — the exact case a single json.loads on - # concatenated per-page arrays could not parse. +def test_execute_post_report_strips_embedded_leading_marker_from_github_body(): + """When the agent's report_md already begins with the commit-scoped marker + line, execute_post_report strips it before calling the native CLI (which + prepends the marker), so the GitHub comment never opens with two duplicate + marker lines.""" calls = [] - page_one = [ - {"id": 101, "body": "just a normal review comment"}, - {"id": 102, "body": "another unrelated comment"}, - ] - page_two = [ - {"id": 103, "body": "chatter"}, - {"id": 555, - "body": "old report\n\n"}, - ] def fake_run(cmd, input=None, text=None, capture_output=None, env=None): - calls.append({"cmd": cmd, "input": input, "env": env}) - if cmd[:2] == ["gh", "api"] and cmd[2].endswith("/comments"): - # --paginate --slurp wraps each page's array in one outer array. - return _FakeCompleted(0, "", json.dumps([page_one, page_two])) + calls.append({"cmd": cmd, "input": input}) return _FakeCompleted(0, "") saved_run = execute_actions.subprocess.run @@ -847,15 +790,15 @@ def fake_run(cmd, input=None, text=None, capture_output=None, env=None): execute_actions.subprocess.run = fake_run os.environ.update(_JIRA_ENV) try: + marker_line = "" report = { "pr_repo": "acme/widget", "pr_number": 42, "jira_issue_id": "TC-777", "commit_sha": "946556e", - "report_md": "## Verify report\nAll good.", + "report_md": f"{marker_line}\n## Verify report\nAll good.", "report_adf": _REPORT_ADF, } - # When posting the report again for the same commit execute_actions.execute_post_report({"type": "post_report"}, {}, report) finally: execute_actions.subprocess.run = saved_run @@ -865,15 +808,16 @@ def fake_run(cmd, input=None, text=None, capture_output=None, env=None): else: os.environ[k] = v - # Then the comment on the second page is PATCH-updated, not duplicated. - assert not any(c["cmd"][:3] == ["gh", "pr", "comment"] for c in calls), \ - "must not create a new GitHub comment when the report exists on a later page" - patch_calls = [c for c in calls if "PATCH" in c["cmd"]] - assert len(patch_calls) == 1, f"Expected one PATCH update, got {len(patch_calls)}" - assert patch_calls[0]["cmd"][2] == "repos/acme/widget/issues/comments/555" + gh_call = next(c for c in calls + if c["cmd"][:3] == ["fullsend", "issues", "post-comment"] + and c["cmd"][c["cmd"].index("--tracker") + 1] == "github") + # The leading marker line is stripped; the body starts with the report heading + # and carries no embedded marker (the CLI prepends the single marker copy). + assert gh_call["input"] == "## Verify report\nAll good.", f"Got: {gh_call['input']!r}" + assert "sdlc-workflow:verify-pr report commit:" not in gh_call["input"] -def _run_post_report_with_sha_resolution(commit_sha, existing_comments, resolve): +def _run_post_report_with_sha_resolution(commit_sha, resolve): """Run execute_post_report with a fake ``git rev-parse`` that maps each input ref to a canonical full SHA via ``resolve``. Returns the recorded calls.""" calls = [] @@ -884,8 +828,6 @@ def fake_run(cmd, input=None, text=None, capture_output=None, env=None): # cmd[-1] is "^{commit}"; strip the peel suffix to look up. ref = cmd[-1].split("^", 1)[0] return _FakeCompleted(0, "", resolve.get(ref, "")) - if cmd[:2] == ["gh", "api"] and cmd[2].endswith("/comments"): - return _FakeCompleted(0, "", json.dumps([existing_comments])) return _FakeCompleted(0, "") saved_run = execute_actions.subprocess.run @@ -912,59 +854,51 @@ def fake_run(cmd, input=None, text=None, capture_output=None, env=None): return calls +def _github_marker_from_calls(calls): + """Return the --marker passed to the native GitHub post-comment call.""" + gh = next(c for c in calls + if c["cmd"][:3] == ["fullsend", "issues", "post-comment"] + and c["cmd"][c["cmd"].index("--tracker") + 1] == "github") + return gh["cmd"][gh["cmd"].index("--marker") + 1] + + def test_execute_post_report_dedup_marker_invariant_to_sha_length(): - """A full-length SHA on one run and an abbreviated SHA for the same commit on - a retry resolve to the SAME report comment (PATCH-update, not duplicate), - because git rev-parse canonicalizes both forms to the same full SHA.""" + """A full-length SHA and an abbreviated SHA for the SAME commit produce the + SAME --marker on the native GitHub call, because git rev-parse canonicalizes + both forms to the same full SHA. The native CLI then deduplicates on that + shared marker, so a retry edits one comment instead of duplicating it.""" full_sha = "946556e" + "a" * 33 # 40 hex chars short_sha = "946556e" # 7-char abbreviation of the same commit # git rev-parse resolves either form of this one commit to its full SHA. resolve = {full_sha: full_sha, short_sha: full_sha} - # First run with the FULL SHA and no existing comment → a comment is created; - # capture the marker it embedded. - first_calls = _run_post_report_with_sha_resolution(full_sha, [], resolve) - create_call = next(c for c in first_calls if c["cmd"][:3] == ["gh", "pr", "comment"]) - created_body = create_call["cmd"][create_call["cmd"].index("--body") + 1] - assert f"commit:{full_sha} -->" in created_body, \ - f"marker should carry the canonical full SHA: {created_body!r}" - - # Retry with the ABBREVIATED SHA; the PR already has the comment created above. - retry_calls = _run_post_report_with_sha_resolution( - short_sha, [{"id": 555, "body": created_body}], resolve) + full_marker = _github_marker_from_calls( + _run_post_report_with_sha_resolution(full_sha, resolve)) + short_marker = _github_marker_from_calls( + _run_post_report_with_sha_resolution(short_sha, resolve)) - # Then the retry PATCH-updates the existing comment instead of duplicating it. - assert not any(c["cmd"][:3] == ["gh", "pr", "comment"] for c in retry_calls), \ - "abbreviated-SHA retry must not create a duplicate report comment" - patch_calls = [c for c in retry_calls if "PATCH" in c["cmd"]] - assert len(patch_calls) == 1, f"Expected one PATCH update, got {len(patch_calls)}" - assert patch_calls[0]["cmd"][2] == "repos/acme/widget/issues/comments/555" + assert full_marker == f"{execute_actions.GITHUB_REPORT_MARKER_PREFIX}{full_sha} -->", \ + f"marker should carry the canonical full SHA: {full_marker!r}" + assert full_marker == short_marker, \ + "full and abbreviated SHA of one commit must yield the same sticky marker" def test_execute_post_report_distinct_commits_same_prefix_do_not_collide(): - """Two distinct commits sharing a 7-hex prefix get DISTINCT dedup markers - (each resolved up to its full 40-char SHA), so the second commit's report is - created as a new comment instead of PATCH-overwriting the first commit's.""" + """Two distinct commits sharing a 7-hex prefix get DISTINCT --marker values + (each resolved up to its full 40-char SHA), so the native CLI keeps them as + separate sticky comments instead of one overwriting the other.""" full_a = "946556e" + "a" * 33 # commit A - full_b = "946556e" + "b" * 33 # commit B — same first 7 hex chars, distinct object + full_b = "946556e" + "b" * 33 # commit B, same first 7 hex chars, distinct object resolve = {full_a: full_a, full_b: full_b} - # Commit A posts first with no existing comment → a comment carrying A's marker. - a_calls = _run_post_report_with_sha_resolution(full_a, [], resolve) - a_create = next(c for c in a_calls if c["cmd"][:3] == ["gh", "pr", "comment"]) - a_body = a_create["cmd"][a_create["cmd"].index("--body") + 1] - assert f"commit:{full_a} -->" in a_body + marker_a = _github_marker_from_calls( + _run_post_report_with_sha_resolution(full_a, resolve)) + marker_b = _github_marker_from_calls( + _run_post_report_with_sha_resolution(full_b, resolve)) - # Commit B posts while A's comment already exists on the PR. - b_calls = _run_post_report_with_sha_resolution( - full_b, [{"id": 555, "body": a_body}], resolve) - - # Then B does NOT PATCH A's comment (no collision); it creates its own. - assert not any("PATCH" in c["cmd"] for c in b_calls), \ - "distinct commit must not overwrite another commit's report comment" - b_create = next(c for c in b_calls if c["cmd"][:3] == ["gh", "pr", "comment"]) - b_body = b_create["cmd"][b_create["cmd"].index("--body") + 1] - assert f"commit:{full_b} -->" in b_body + assert marker_a == f"{execute_actions.GITHUB_REPORT_MARKER_PREFIX}{full_a} -->" + assert marker_b == f"{execute_actions.GITHUB_REPORT_MARKER_PREFIX}{full_b} -->" + assert marker_a != marker_b, "distinct commits must get distinct sticky markers" def test_normalize_commit_sha_falls_back_to_prefix_when_unresolvable(): @@ -1007,8 +941,6 @@ def test_post_report_resolves_ref_created_by_later_action(): def fake_run(cmd, input=None, text=None, capture_output=None, env=None): calls.append({"cmd": cmd, "input": input}) - if cmd[:2] == ["gh", "api"] and cmd[2].endswith("/comments"): - return _FakeCompleted(0, "", "[]") # no existing report comment return _FakeCompleted(0, "") def fake_create_issue(**kwargs): @@ -1069,37 +1001,18 @@ def fake_create_issue(**kwargs): else: os.environ[k] = v - # The GitHub report body carries the resolved key, not the raw placeholder. - create_call = next(c for c in calls if c["cmd"][:3] == ["gh", "pr", "comment"]) - gh_body = create_call["cmd"][create_call["cmd"].index("--body") + 1] - assert "Filed sub-task TC-999." in gh_body, f"ref not resolved: {gh_body}" - assert "{{sub-1.key}}" not in gh_body, "placeholder leaked into report body" + # The GitHub report body (native call stdin) carries the resolved key, not the + # raw placeholder. + native = [c for c in calls if c["cmd"][:3] == ["fullsend", "issues", "post-comment"]] + gh_call = next(c for c in native if c["cmd"][c["cmd"].index("--tracker") + 1] == "github") + assert "Filed sub-task TC-999." in gh_call["input"], f"ref not resolved: {gh_call['input']}" + assert "{{sub-1.key}}" not in gh_call["input"], "placeholder leaked into report body" # The Jira body is rendered from report_adf, and its refs resolve too. - jira_call = next(c for c in calls if c["cmd"][:3] == ["fullsend", "issues", "post-comment"]) + jira_call = next(c for c in native if c["cmd"][c["cmd"].index("--tracker") + 1] == "jira") assert jira_call["input"] == "Filed sub-task TC-999.", f"ref not resolved in ADF: {jira_call['input']}" assert "{{sub-1.key}}" not in jira_call["input"], "placeholder leaked into Jira body" -def test_find_report_comment_id_exits_on_unparseable_json(): - """A JSON parse failure aborts with sys.exit(1) instead of silently returning - None (which would let a retry create a duplicate report comment).""" - # Given `gh` returns malformed JSON (e.g. concatenated per-page arrays, the - # pre-fix --paginate-without-slurp shape that is not valid combined JSON) - def fake_run(cmd, input=None, text=None, capture_output=None, env=None): - return _FakeCompleted(0, "", "[{\"id\": 1}][{\"id\": 2}]") - - saved_run = execute_actions.subprocess.run - execute_actions.subprocess.run = fake_run - try: - # When the id lookup runs, it must fail loudly rather than swallow the error - execute_actions._find_report_comment_id("acme/widget", 42, "marker") - assert False, "Should have exited on unparseable JSON" - except SystemExit as e: - assert e.code == 1 - finally: - execute_actions.subprocess.run = saved_run - - if __name__ == "__main__": test_resolve_refs_replaces_key() test_resolve_refs_no_placeholders() @@ -1132,11 +1045,9 @@ def fake_run(cmd, input=None, text=None, capture_output=None, env=None): test_execute_post_comment_routes_to_native() test_post_comment_and_report_use_distinct_sticky_markers() test_execute_post_report_posts_github_then_jira() - test_execute_post_report_updates_existing_github_comment_on_retry() - test_execute_post_report_updates_comment_on_later_page() + test_execute_post_report_strips_embedded_leading_marker_from_github_body() test_execute_post_report_dedup_marker_invariant_to_sha_length() test_execute_post_report_distinct_commits_same_prefix_do_not_collide() test_normalize_commit_sha_falls_back_to_prefix_when_unresolvable() test_post_report_resolves_ref_created_by_later_action() - test_find_report_comment_id_exits_on_unparseable_json() print("All tests passed.") From 8fdaa49f12ba6a293fe882499861ca9b9d24106e Mon Sep 17 00:00:00 2001 From: mrizzi Date: Mon, 7 Sep 2026 16:21:59 +0200 Subject: [PATCH 048/175] chore(verify-pr): pin fullsend base to 888a1781 + re-lock; add .fullsend MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Register the local composing child harness and freeze the dependency lock at the code commit (888a1781) so a full pinned run resolves the migrated execute-actions.py. The child's base: URL advances 34e6362f → 888a1781; the base file is byte-identical (only scripts changed), so its #sha256 stays addc8686. lock.yaml re-resolved from 888a1781 (11 deps). Track only config.yaml, harness/verify-pr.yaml, and lock.yaml. Add .fullsend/.gitignore excluding the regenerated .fullsend-cache/ (fetched from the pinned URLs on every run) and the secret-bearing .local-run/ scaffolding. TC-5813 Assisted-by: Claude Code Co-Authored-By: Claude Opus 4.8 --- .fullsend/.gitignore | 6 + .fullsend/config.yaml | 24 ++++ .fullsend/harness/verify-pr.yaml | 43 ++++++ .fullsend/lock.yaml | 240 +++++++++++++++++++++++++++++++ 4 files changed, 313 insertions(+) create mode 100644 .fullsend/.gitignore create mode 100644 .fullsend/config.yaml create mode 100644 .fullsend/harness/verify-pr.yaml create mode 100644 .fullsend/lock.yaml diff --git a/.fullsend/.gitignore b/.fullsend/.gitignore new file mode 100644 index 000000000..302039c0d --- /dev/null +++ b/.fullsend/.gitignore @@ -0,0 +1,6 @@ +# fullsend-managed local resource cache — regenerated from the pinned URLs in +# lock.yaml on every run/lock; never committed. +.fullsend-cache/ + +# Local-only run scaffolding (env files with live secrets, launchers). +.local-run/ diff --git a/.fullsend/config.yaml b/.fullsend/config.yaml new file mode 100644 index 000000000..f0d73e8b3 --- /dev/null +++ b/.fullsend/config.yaml @@ -0,0 +1,24 @@ +# fullsend per-repo configuration +# https://github.com/fullsend-ai/fullsend +# +# This file configures fullsend for per-repo installation mode. +# See ADR 0033 for details. +# +# The "runtime" key selects which agent runtime runs the agents, claude +# (default when unset) or pi. For one run, the 'fullsend run --runtime' +# flag wins, then FULLSEND_RUNTIME, then this file. See docs/runtimes.md. +version: "1" +kill_switch: false +# The registered source is the LOCAL composing child harness (base + runner-local +# host_files); its `base:` pins the repo content by raw URL. See +# .fullsend/harness/verify-pr.yaml. The base URL's host prefix must remain listed +# in allowed_remote_resources below (base URLs are not inherited — they are +# validated against this allowlist). +agents: + - name: verify-pr + source: harness/verify-pr.yaml +allowed_remote_resources: + # Exactly one prefix: this harness's base URL is self-hosted on + # RHEcosystemAppEng and its children resolve relative to that base (no + # fullsend-ai remote overlay in the native v0.37.0 pinned-URL model). + - https://raw.githubusercontent.com/RHEcosystemAppEng/sdlc-plugins/ diff --git a/.fullsend/harness/verify-pr.yaml b/.fullsend/harness/verify-pr.yaml new file mode 100644 index 000000000..2a2dfe2e8 --- /dev/null +++ b/.fullsend/harness/verify-pr.yaml @@ -0,0 +1,43 @@ +# Local composing child for the verify-pr harness. +# +# fullsend v0.37.0 rejects an absolute host_files.src that is inherited from a +# URL-sourced harness, so the runner-local credential and pre_script-output +# mounts cannot live in the URL-pinned base (harness/verify-pr.yaml). This local +# child pins that base by raw URL (so the pinned bytes are byte-identical across +# local and CI) and adds those absolute mounts directly. host_files are +# concatenated base+child, deduplicated by dest with the child winning (ADR-0045; +# harness-fields.md). Absolute host_files declared directly in this local child +# are NOT URL-sourced, so they are accepted. +# +# ONE child serves BOTH environments — the mounts are env-expanded and the OIDC +# token is optional: +# GOOGLE_APPLICATION_CREDENTIALS local: service-account key JSON +# CI: WIF external_account config +# GCP_OIDC_TOKEN_FILE local: unset → optional mount skipped +# CI: runner-refreshed OIDC token file +# Only the runtime environment differs — fullsend's sanctioned "same harness, +# different runtime env" model (ADR-0055; running-agents-locally.md). +# +# PIN: the base cannot be a cross-boundary local path — fullsend rejects a +# `base:` that escapes the .fullsend workspace root, and a URL base is required +# anyway so the base's relative children resolve as pinned raw URLs. The base is +# pinned to a commit SHA (not a branch) plus the base file's sha256; both local +# and CI resolve these exact bytes. lock.yaml freezes every child SHA256. +# The base URL host prefix must stay listed in config.yaml allowed_remote_resources +# (base URLs are validated against that allowlist, not inherited). +# Re-pin after any base edit: push harness/verify-pr.yaml, set the SHA to the new +# commit + the sha256 to `shasum -a 256 harness/verify-pr.yaml`, `fullsend lock`. +base: https://raw.githubusercontent.com/RHEcosystemAppEng/sdlc-plugins/888a1781dd81f68bc6f4daecdcccef5d91b9cef6/harness/verify-pr.yaml#sha256=addc868659a4391b18fe09cb2a0523d134a9242e35d644a6db36bcce8277cb16 + +host_files: + - src: ${GOOGLE_APPLICATION_CREDENTIALS} + dest: /tmp/.gcp-credentials.json + - src: ${GCP_OIDC_TOKEN_FILE} + dest: /sandbox/workspace/.gcp-oidc-token + optional: true + # pre_script prefetch output — pre-verify-pr.sh writes /tmp/fullsend-pre-output + # on the runner; mounted read-only into the sandbox (optional — absent until + # the pre_script runs, and skipped on an ADR-0072 skip). + - src: /tmp/fullsend-pre-output/verify-pr-input.json + dest: /sandbox/workspace/.pre-script/verify-pr-input.json + optional: true diff --git a/.fullsend/lock.yaml b/.fullsend/lock.yaml new file mode 100644 index 000000000..389e098e4 --- /dev/null +++ b/.fullsend/lock.yaml @@ -0,0 +1,240 @@ +# Generated by fullsend lock — DO NOT EDIT +version: 1 +generated_at: 2026-09-04T13:39:01.038924Z +harnesses: + verify-pr: + source: harness/verify-pr.yaml + sha256: 7de5dd2b5dacc019fcc42c4292844142708b849d42e3e75c6cbcfe6a2f9f92f5 + resolved_at: 2026-09-07T14:21:22.78544Z + dependencies: + - field: base + url: https://raw.githubusercontent.com/RHEcosystemAppEng/sdlc-plugins/888a1781dd81f68bc6f4daecdcccef5d91b9cef6/harness/verify-pr.yaml + sha256: addc868659a4391b18fe09cb2a0523d134a9242e35d644a6db36bcce8277cb16 + type: file + fetched_at: 2026-09-04T13:20:03.650092Z + - field: pre_script + url: https://raw.githubusercontent.com/RHEcosystemAppEng/sdlc-plugins/888a1781dd81f68bc6f4daecdcccef5d91b9cef6/plugins/sdlc-workflow/scripts/pre-verify-pr.sh + sha256: 085379c03f7436e1920dd07d9e50056ee4f1430c591b524645b271899ce093d7 + type: directory + files: + - path: execute-actions.py + sha256: fafbed0c81ed2754366cae424cdd9063b3e01b815c8c94fa80612dd406878b9b + - path: jira-client.py + sha256: f6bf79d2c98a2ae020775e68a538f612953c0464f09bd750c988f0d20fd4b159 + - path: post-verify-pr.sh + sha256: 0cdd85d68e1c94546964747badcb1837602aae0e47a168903f39e3760d1be687 + - path: pre-verify-pr.sh + sha256: 041bd777d3a8915d0818aa6d24ed188443aba4c199f5ecdd543d8a86677a8c5b + - path: pre_verify_pr.py + sha256: 2ee6ab01b7e277a526ca5fb4f0110a5bf0845ce6562bce56a6c6d0e4edaa93fd + - path: strip_extra_properties.py + sha256: 9099200014a0a4a9da8795a3346f963efc34c53f4603d340551c46b8d135d122 + - path: test_execute_actions.py + sha256: 014402bd1c773ad9a2a4ab930be95af3230a2b37349b7ba59ed230d76164d978 + - path: test_jira_client.py + sha256: 2685494b0e725bdf281f61052eb5af7c2cf771613fa3ec6692228377d1bc8518 + - path: test_jira_client_cli.py + sha256: d36b790c1a310111a0d2f3e81415f9576e80acc5f0bc7d0ff8084ffad001285f + - path: test_pre_verify_pr.py + sha256: 7f6b2704a8e620669c275712844124e2f06d857fa1f09fbd689589b5313f09cf + - path: validate-output-schema.sh + sha256: 56b964145da0b62f438dbc1885780d86e2d589c5340adb93c00e3c8777979cbe + fetched_at: 2026-09-07T14:21:12.125687Z + - field: post_script + url: https://raw.githubusercontent.com/RHEcosystemAppEng/sdlc-plugins/888a1781dd81f68bc6f4daecdcccef5d91b9cef6/plugins/sdlc-workflow/scripts/post-verify-pr.sh + sha256: 085379c03f7436e1920dd07d9e50056ee4f1430c591b524645b271899ce093d7 + type: directory + files: + - path: execute-actions.py + sha256: fafbed0c81ed2754366cae424cdd9063b3e01b815c8c94fa80612dd406878b9b + - path: jira-client.py + sha256: f6bf79d2c98a2ae020775e68a538f612953c0464f09bd750c988f0d20fd4b159 + - path: post-verify-pr.sh + sha256: 0cdd85d68e1c94546964747badcb1837602aae0e47a168903f39e3760d1be687 + - path: pre-verify-pr.sh + sha256: 041bd777d3a8915d0818aa6d24ed188443aba4c199f5ecdd543d8a86677a8c5b + - path: pre_verify_pr.py + sha256: 2ee6ab01b7e277a526ca5fb4f0110a5bf0845ce6562bce56a6c6d0e4edaa93fd + - path: strip_extra_properties.py + sha256: 9099200014a0a4a9da8795a3346f963efc34c53f4603d340551c46b8d135d122 + - path: test_execute_actions.py + sha256: 014402bd1c773ad9a2a4ab930be95af3230a2b37349b7ba59ed230d76164d978 + - path: test_jira_client.py + sha256: 2685494b0e725bdf281f61052eb5af7c2cf771613fa3ec6692228377d1bc8518 + - path: test_jira_client_cli.py + sha256: d36b790c1a310111a0d2f3e81415f9576e80acc5f0bc7d0ff8084ffad001285f + - path: test_pre_verify_pr.py + sha256: 7f6b2704a8e620669c275712844124e2f06d857fa1f09fbd689589b5313f09cf + - path: validate-output-schema.sh + sha256: 56b964145da0b62f438dbc1885780d86e2d589c5340adb93c00e3c8777979cbe + fetched_at: 2026-09-07T14:21:12.125687Z + - field: validation_loop.script + url: https://raw.githubusercontent.com/RHEcosystemAppEng/sdlc-plugins/888a1781dd81f68bc6f4daecdcccef5d91b9cef6/plugins/sdlc-workflow/scripts/validate-output-schema.sh + sha256: 085379c03f7436e1920dd07d9e50056ee4f1430c591b524645b271899ce093d7 + type: directory + files: + - path: execute-actions.py + sha256: fafbed0c81ed2754366cae424cdd9063b3e01b815c8c94fa80612dd406878b9b + - path: jira-client.py + sha256: f6bf79d2c98a2ae020775e68a538f612953c0464f09bd750c988f0d20fd4b159 + - path: post-verify-pr.sh + sha256: 0cdd85d68e1c94546964747badcb1837602aae0e47a168903f39e3760d1be687 + - path: pre-verify-pr.sh + sha256: 041bd777d3a8915d0818aa6d24ed188443aba4c199f5ecdd543d8a86677a8c5b + - path: pre_verify_pr.py + sha256: 2ee6ab01b7e277a526ca5fb4f0110a5bf0845ce6562bce56a6c6d0e4edaa93fd + - path: strip_extra_properties.py + sha256: 9099200014a0a4a9da8795a3346f963efc34c53f4603d340551c46b8d135d122 + - path: test_execute_actions.py + sha256: 014402bd1c773ad9a2a4ab930be95af3230a2b37349b7ba59ed230d76164d978 + - path: test_jira_client.py + sha256: 2685494b0e725bdf281f61052eb5af7c2cf771613fa3ec6692228377d1bc8518 + - path: test_jira_client_cli.py + sha256: d36b790c1a310111a0d2f3e81415f9576e80acc5f0bc7d0ff8084ffad001285f + - path: test_pre_verify_pr.py + sha256: 7f6b2704a8e620669c275712844124e2f06d857fa1f09fbd689589b5313f09cf + - path: validate-output-schema.sh + sha256: 56b964145da0b62f438dbc1885780d86e2d589c5340adb93c00e3c8777979cbe + fetched_at: 2026-09-07T14:21:12.125687Z + - field: validation_loop.schema + url: https://raw.githubusercontent.com/RHEcosystemAppEng/sdlc-plugins/888a1781dd81f68bc6f4daecdcccef5d91b9cef6/plugins/sdlc-workflow/schemas/verify-pr-result.schema.json + sha256: 62c3e5ce0c47e73823b5cab9a577be7b721d8fb31389bc8602dc82ee05526ad6 + type: resource + fetched_at: 2026-09-07T14:21:17.494543Z + - field: agent + url: https://raw.githubusercontent.com/RHEcosystemAppEng/sdlc-plugins/888a1781dd81f68bc6f4daecdcccef5d91b9cef6/plugins/sdlc-workflow/agents/verify-pr.md + sha256: 568b2ab7e32300c84038d500d95f4534ff6c4b20e52eebf2ed5a6c0ae706786a + type: resource + fetched_at: 2026-09-07T14:21:17.75376Z + - field: policy + url: https://raw.githubusercontent.com/RHEcosystemAppEng/sdlc-plugins/888a1781dd81f68bc6f4daecdcccef5d91b9cef6/plugins/sdlc-workflow/policies/verify-pr.yaml + sha256: 62d1374be253b0b225921bd8e22e679781076ecdb3a9ea22d5e6a4805753c5a7 + type: resource + fetched_at: 2026-09-07T14:21:18.012637Z + - field: host_files[0].src + url: https://raw.githubusercontent.com/RHEcosystemAppEng/sdlc-plugins/888a1781dd81f68bc6f4daecdcccef5d91b9cef6/plugins/sdlc-workflow/env/gcp-vertex.env + sha256: 10b2ba695b1d4e65e0964a233b75d6a06853f1eaefc260c56b9eedd989ae7d41 + type: resource + fetched_at: 2026-09-07T14:21:18.26852Z + - field: openshell.profiles[0] + url: https://raw.githubusercontent.com/RHEcosystemAppEng/sdlc-plugins/888a1781dd81f68bc6f4daecdcccef5d91b9cef6/plugins/sdlc-workflow/profiles/fullsend-vertex-ai.yaml + sha256: 76535a148387b1281be2bfb23b6724e4133327189b38c8ce1de2c6dddb8d8341 + type: resource + fetched_at: 2026-09-07T14:21:18.556217Z + - field: providers[0] + url: https://raw.githubusercontent.com/RHEcosystemAppEng/sdlc-plugins/888a1781dd81f68bc6f4daecdcccef5d91b9cef6/plugins/sdlc-workflow/providers/vertex-ai.yaml + sha256: ae5ebe527e5d7b0fa6994346fde3f6ba11b632a3ba78ece96591b5ed85083ec1 + type: resource + fetched_at: 2026-09-07T14:21:18.880884Z + - field: plugins[0] + url: https://raw.githubusercontent.com/RHEcosystemAppEng/sdlc-plugins/888a1781dd81f68bc6f4daecdcccef5d91b9cef6/plugins/sdlc-workflow/plugin.json + sha256: 83c90724d8b581e7f173e3734e461b3cd515ddede27e40c2032d84f319f9e10f + type: directory + files: + - path: .claude-plugin/plugin.json + sha256: a10c3191ab7957bd496b0ebbbb212844c48d3a95c1556097d2466d5770cf1c20 + - path: agents/verify-pr.md + sha256: 568b2ab7e32300c84038d500d95f4534ff6c4b20e52eebf2ed5a6c0ae706786a + - path: env/gcp-vertex.env + sha256: 10b2ba695b1d4e65e0964a233b75d6a06853f1eaefc260c56b9eedd989ae7d41 + - path: plugin.json + sha256: 727186c0e7e8d0d6b241ff61cb57a979a2d9c7c0c684a4d1c12940534e9f2b18 + - path: policies/verify-pr.yaml + sha256: 62d1374be253b0b225921bd8e22e679781076ecdb3a9ea22d5e6a4805753c5a7 + - path: profiles/fullsend-vertex-ai.yaml + sha256: 76535a148387b1281be2bfb23b6724e4133327189b38c8ce1de2c6dddb8d8341 + - path: providers/vertex-ai.yaml + sha256: ae5ebe527e5d7b0fa6994346fde3f6ba11b632a3ba78ece96591b5ed85083ec1 + - path: schemas/verify-pr-input.schema.json + sha256: 30a46d16831087f65b6168db8f94965c9ff1516ac2b56d9a5d7d63f7ba0b3809 + - path: schemas/verify-pr-result.schema.json + sha256: 62c3e5ce0c47e73823b5cab9a577be7b721d8fb31389bc8602dc82ee05526ad6 + - path: scripts/execute-actions.py + sha256: fafbed0c81ed2754366cae424cdd9063b3e01b815c8c94fa80612dd406878b9b + - path: scripts/jira-client.py + sha256: f6bf79d2c98a2ae020775e68a538f612953c0464f09bd750c988f0d20fd4b159 + - path: scripts/post-verify-pr.sh + sha256: 0cdd85d68e1c94546964747badcb1837602aae0e47a168903f39e3760d1be687 + - path: scripts/pre-verify-pr.sh + sha256: 041bd777d3a8915d0818aa6d24ed188443aba4c199f5ecdd543d8a86677a8c5b + - path: scripts/pre_verify_pr.py + sha256: 2ee6ab01b7e277a526ca5fb4f0110a5bf0845ce6562bce56a6c6d0e4edaa93fd + - path: scripts/strip_extra_properties.py + sha256: 9099200014a0a4a9da8795a3346f963efc34c53f4603d340551c46b8d135d122 + - path: scripts/test_execute_actions.py + sha256: 014402bd1c773ad9a2a4ab930be95af3230a2b37349b7ba59ed230d76164d978 + - path: scripts/test_jira_client.py + sha256: 2685494b0e725bdf281f61052eb5af7c2cf771613fa3ec6692228377d1bc8518 + - path: scripts/test_jira_client_cli.py + sha256: d36b790c1a310111a0d2f3e81415f9576e80acc5f0bc7d0ff8084ffad001285f + - path: scripts/test_pre_verify_pr.py + sha256: 7f6b2704a8e620669c275712844124e2f06d857fa1f09fbd689589b5313f09cf + - path: scripts/validate-output-schema.sh + sha256: 56b964145da0b62f438dbc1885780d86e2d589c5340adb93c00e3c8777979cbe + - path: shared/comment-footnote.md + sha256: bd78a3a34db53aaec53149c2d5557d1c55d10c6f4cd1622d906652cf9b72ca5c + - path: shared/convention-applicability-rules.md + sha256: 1ccabd6491148fdfc9911fcd2f233c9092d4dd54bc82cd67f52512e3e35d1823 + - path: shared/description-digest-protocol.md + sha256: 06cf26d8677d2e9866f7c72d8fb8037cca13df8a7ffcdbf3cd333775283fb7ea + - path: shared/eval-coverage-propagation.md + sha256: d04efdb625e6b1382b2a4b8682bf41accfd99f286c40856238e0735c1f4a00f0 + - path: shared/jira-access-strategy.md + sha256: c00aebec96792156cdb2a80a5eb1431cbf9b1d35ba7b54d5fbca37227a2b6303 + - path: shared/jira-api-token-guide.md + sha256: f8fefd4e12ffca9127e692de99b18aa8e3b007cb5f5e9fb2c7b16336fd84518a + - path: shared/jira-rest-fallback.md + sha256: fe39a0eab5febfb77c7c5112c0ec1c3f1ab5ec581d9f3c40b4cd968228d23cb5 + - path: shared/task-description-template.md + sha256: 8f0d8fbddb8b8f662db3c1c5c47e9224aa5c0d504fd615e935a960ead6660a7b + - path: skills/define-feature/SKILL.md + sha256: c314f3f549c3ef835e03c38ed8ce2212422c77a876d9b786dd4e08806f5d5e4e + - path: skills/implement-task/SKILL.md + sha256: 0276a52292d68908e56ab213746c93a931bede189d3b6e7a914f6a2024f6f4c1 + - path: skills/plan-feature/SKILL.md + sha256: a8603de28ac9ff7d163b3ef5e11c44ab3f190d735e7ec6fd57604b6b9766a9ce + - path: skills/report-bug/SKILL.md + sha256: cc841f523aa9b2620f01426c32a84ee29710c732157356743a87e8ff60823cc4 + - path: skills/run-evals/SKILL.md + sha256: 221c45e692d4becfd839bc99ce6d2c4a3fabc9a68dd5479c40de28b2870ec5c9 + - path: skills/run-evals/scripts/aggregate_benchmark.py + sha256: b89b6471d6abad6684863afcdc8634c2097986afb3ff64f80f179cd71c3dd325 + - path: skills/run-evals/scripts/render_summary.py + sha256: 32da827a4db496a20d80148583b1f4906f595bcc5b77393a5e39dfdab30db1a5 + - path: skills/run-evals/scripts/test_render_summary.py + sha256: 34a2333e5bee8b0f07c23e64a9e7ab096cf16e4a027cacf91b86c4b39e584cf7 + - path: skills/setup/SKILL.md + sha256: a9d4a55fc35123607c445ea4ad2609e8306443f2032d360fa55b54827871bde6 + - path: skills/setup/constraints.template.md + sha256: 6a0c75f5e390eea5c42daffd8517b19ac58d41d6600d46a067bc31d3056b52b0 + - path: skills/setup/conventions.template.md + sha256: 264e53e4b2c2cad10d69293e009ca3fd7be3437de0422c81f6a9e4430f018361 + - path: skills/setup/project-config.template.md + sha256: d2b607d8d226bbe303b00c9e6e875b115f8f138e8e4afa29c38f37fce7b03dbd + - path: skills/setup/security-config.template.md + sha256: 2422a28db45539c16cd6bfc45ab9d390dc41c816784065ad84c82cc8b5aea120 + - path: skills/triage-bug/SKILL.md + sha256: e3c5f28d1c7bafdcf706d8be89110cef28217ef878a754fb89c571c5731242d4 + - path: skills/triage-security/SKILL.md + sha256: 040ba49ffcedb2c0301389ff4ca1572f45c9b9c0d33f7a8f463b0b1ad472d459 + - path: skills/triage-security/jira-triage-operations.md + sha256: 665aef8f2529ee44be4fdee57783e6d589867f71793e9b7d280ce8f3973a9dcb + - path: skills/triage-security/remediation-templates.md + sha256: 0b45136eb63e18a0bb81eebcb1dd033a27a69134117949eedf9f8e43d0d76010 + - path: skills/triage-security/version-impact-analysis.md + sha256: a18376094360d435e96436c9fd1bd6a04831123d903e44cf6cf57e600285e3c9 + - path: skills/verify-pr/SKILL.md + sha256: b57b1ac91b1996239429518dd418277a2f8e6f29aef08275a37f4df95990f825 + - path: skills/verify-pr/correctness.md + sha256: 44a4c095038d41434b5e658dfd209970a77c1c0417bc2c00548ce1f85abdbade + - path: skills/verify-pr/dispatch-template.md + sha256: 5d1c691c7b24f2975664ea002e5a0d3be2eb6127e901c3be8bf4d98121f04c48 + - path: skills/verify-pr/finding-template.md + sha256: b3a7c52d60d1ffc30f5d80219b2fc9db6c28eabeca83bcc0ffa1e424b3a4d0c2 + - path: skills/verify-pr/intent-alignment.md + sha256: 1490378c0510b9645e3a3f550b7b0383803959585a658eb898084efc6505d808 + - path: skills/verify-pr/security.md + sha256: 128877cf7e156666dbb403805a0112059cc8cc3255e148c6794da29d48b8f5cf + - path: skills/verify-pr/style-conventions.md + sha256: 5ab920c596615c0d23354cb0e90fe5ebf5f08635ffa3602e8e2d2abd74028a19 + fetched_at: 2026-09-07T14:21:22.780456Z From 198fbb7cefbfa9c1bf7be2e4201acd0d4c0e6583 Mon Sep 17 00:00:00 2001 From: mrizzi Date: Mon, 7 Sep 2026 17:48:20 +0200 Subject: [PATCH 049/175] fix(verify-pr): compute Commit Traceability deterministically on runner The verify-pr Commit Traceability check (Intent Alignment Check 3) produced a false FAIL under fullsend: the tokenless sandbox agent re-derived traceability with `git log --oneline main..pr | grep`, which emits subject lines only and misses a Jira ID that lives in the commit body or a trailer. Close the parity gap so the skill in fullsend matches Claude Code on main: 1. Runner (pre_verify_pr.py): annotate each github.commit with a `references_task_id` boolean computed over the FULL headline + body, with word boundaries so TC-5812 does not match TC-58120 / TC-5982. The sandbox reads this fact instead of re-deriving it. 2. SKILL.md: forbid substituting a live `git log` read for pre-fetched values; require full messageHeadline + messageBody (no truncation) in sandbox mode. 3. intent-alignment.md + dispatch-template.md: Check 3 uses references_task_id as the authoritative source; never a subject-only git command; full body scan only as fallback when the flag is absent. Adds 6 tests for commit_references_task and the transform annotation. Assisted-by: Claude Code Co-Authored-By: Claude Opus 4.8 --- .../sdlc-workflow/scripts/pre_verify_pr.py | 31 +++++++++++ .../scripts/test_pre_verify_pr.py | 53 ++++++++++++++++++- .../sdlc-workflow/skills/verify-pr/SKILL.md | 12 ++++- .../skills/verify-pr/dispatch-template.md | 4 +- .../skills/verify-pr/intent-alignment.md | 29 ++++++---- 5 files changed, 117 insertions(+), 12 deletions(-) diff --git a/plugins/sdlc-workflow/scripts/pre_verify_pr.py b/plugins/sdlc-workflow/scripts/pre_verify_pr.py index b9aa39582..0ee534fdc 100644 --- a/plugins/sdlc-workflow/scripts/pre_verify_pr.py +++ b/plugins/sdlc-workflow/scripts/pre_verify_pr.py @@ -27,9 +27,30 @@ import glob import json import os +import re import sys +def commit_references_task(commit, task_id): + """True if the commit references ``task_id`` in its headline OR body. + + The Jira task ID conventionally sits in a trailer or body line (e.g. + ``Implements TC-5812``), not only the subject, so BOTH ``messageHeadline`` + and ``messageBody`` are scanned. Word boundaries keep ``TC-5812`` from + matching ``TC-58120`` or another ID like ``TC-5982``. This computes the + Commit Traceability fact deterministically on the runner so the tokenless + sandbox agent never has to re-derive it from ``git log`` (which, with + ``--oneline``/``%s``, would see subjects only and miss the trailer). + """ + if not task_id: + return False + text = "{}\n{}".format( + commit.get("messageHeadline", "") or "", + commit.get("messageBody", "") or "", + ) + return re.search(r"\b{}\b".format(re.escape(task_id)), text) is not None + + def extract_pr_url(issue): """Extract PR URL from Jira custom field (ADF or string). @@ -165,6 +186,16 @@ def transform_to_input(issue, task_id, pr_url, github=None, idempotency=None): }, } if github is not None: + # Annotate each commit with the deterministic Commit Traceability fact + # (Check 3). commits.items is unconstrained in the input schema, so the + # extra key is schema-valid; the sandbox agent reads references_task_id + # instead of running its own subjects-only git log. + commits = github.get("commits") + if isinstance(commits, list): + for commit in commits: + if isinstance(commit, dict): + commit["references_task_id"] = commit_references_task( + commit, task_id) result["github"] = github result["idempotency"] = ( idempotency if idempotency is not None else {"related_issues": []} diff --git a/plugins/sdlc-workflow/scripts/test_pre_verify_pr.py b/plugins/sdlc-workflow/scripts/test_pre_verify_pr.py index e11ca5873..6557e90bc 100644 --- a/plugins/sdlc-workflow/scripts/test_pre_verify_pr.py +++ b/plugins/sdlc-workflow/scripts/test_pre_verify_pr.py @@ -114,6 +114,56 @@ def test_transform_with_github(): assert result["github"]["commit_sha"] == "deadbee" +# --- commit_references_task (Commit Traceability determinism) --- + +def test_commit_references_task_in_body_trailer(): + # The canonical failure mode: the ID lives only in the body trailer, far + # past where a subject-only or truncated read would look. + commit = { + "messageHeadline": "feat(verify-pr): re-sync sandbox dual-mode onto SKILL.md", + "messageBody": "Long body paragraph ...\n\nImplements TC-5812\n\nAssisted-by: x", + } + assert pre_verify_pr.commit_references_task(commit, "TC-5812") is True + + +def test_commit_references_task_in_headline(): + commit = {"messageHeadline": "TC-5812: fix scope", "messageBody": ""} + assert pre_verify_pr.commit_references_task(commit, "TC-5812") is True + + +def test_commit_references_task_absent(): + commit = {"messageHeadline": "fix: thing", "messageBody": "no id here"} + assert pre_verify_pr.commit_references_task(commit, "TC-5812") is False + + +def test_commit_references_task_word_boundary(): + # A superstring ID must not match, nor a different task in the same family. + assert pre_verify_pr.commit_references_task( + {"messageHeadline": "x", "messageBody": "see TC-58120"}, "TC-5812") is False + assert pre_verify_pr.commit_references_task( + {"messageHeadline": "x", "messageBody": "the TC-5982 fix"}, "TC-5812") is False + + +def test_commit_references_task_missing_body_key(): + assert pre_verify_pr.commit_references_task( + {"messageHeadline": "TC-5812: x"}, "TC-5812") is True + + +def test_transform_annotates_commit_references_task_id(): + issue = {"fields": {"summary": "S", "status": {"name": "Open"}, "labels": [], "issuelinks": []}} + commits = [ + {"oid": "aaa", "messageHeadline": "feat: x", "messageBody": "Implements TC-5812"}, + {"oid": "bbb", "messageHeadline": "fix: y", "messageBody": "TC-6033 unrelated"}, + ] + github = pre_verify_pr.build_github_bundle( + "o/r", 5, "b", "aaa", "d", "s", [], [], [], commits, + ) + result = pre_verify_pr.transform_to_input(issue, "TC-5812", "", github) + annotated = result["github"]["commits"] + assert annotated[0]["references_task_id"] is True + assert annotated[1]["references_task_id"] is False + + def test_transform_issue_links(): issue = {"fields": { "summary": "S", "description": {}, "status": {"name": "Open"}, @@ -268,7 +318,8 @@ def test_cli_transform_github_dir(): assert gh["reviews"] == [{"id": 1, "state": "APPROVED"}] assert gh["review_comments"] == [{"id": 2}] assert gh["issue_comments"] == [{"id": 3}] - assert gh["commits"] == [{"oid": "abc1234def"}] + # transform annotates each commit with the deterministic traceability fact. + assert gh["commits"] == [{"oid": "abc1234def", "references_task_id": False}] # --- idempotency prefetch (related_keys, build_idempotency_bundle, transform) --- diff --git a/plugins/sdlc-workflow/skills/verify-pr/SKILL.md b/plugins/sdlc-workflow/skills/verify-pr/SKILL.md index 3429a1e26..aab309b7e 100644 --- a/plugins/sdlc-workflow/skills/verify-pr/SKILL.md +++ b/plugins/sdlc-workflow/skills/verify-pr/SKILL.md @@ -468,6 +468,10 @@ constructs dispatch envelopes following the structure defined in **Sandbox mode:** do not run the `gh pr diff`/`gh pr view` commands below — the full diff, diffstat, and commits are pre-fetched as `github.diff`, `github.stat`, and `github.commits` (Step 0.7). Use those values as the corresponding dispatch inputs. +The sandbox has no `gh` CLI; do NOT substitute a live repository read (e.g. +`git log`) for any pre-fetched value — for commit traceability in particular, +`git log --oneline`/`--format=%s` emit subject lines only and will miss a Jira +ID in a commit body/trailer, producing a false FAIL. Collect all inputs needed for sub-agent dispatch envelopes: @@ -482,10 +486,16 @@ Collect all inputs needed for sub-agent dispatch envelopes: gh pr diff --stat -R ``` -3. **PR commits** — for Intent Alignment sub-agent: +3. **PR commits** — for Intent Alignment sub-agent. Pass each commit's `oid`, its + **full** `messageHeadline` and `messageBody` (do NOT truncate the body — a Jira + trailer such as `Implements PROJ-231` typically sits at the very end), and, in + sandbox mode, the runner-computed `references_task_id` boolean. In interactive + mode fetch with: ``` gh pr view --json commits --jq '.commits[] | {oid: .oid, messageHeadline: .messageHeadline, messageBody: .messageBody}' -R ``` + In sandbox mode use `github.commits` as-is (each item already carries + `references_task_id`); never re-slice or subject-only-summarize the bodies. 4. **Task specification sections** — extracted from the Jira task description parsed in Step 1: diff --git a/plugins/sdlc-workflow/skills/verify-pr/dispatch-template.md b/plugins/sdlc-workflow/skills/verify-pr/dispatch-template.md index 17fe153fc..092bb8819 100644 --- a/plugins/sdlc-workflow/skills/verify-pr/dispatch-template.md +++ b/plugins/sdlc-workflow/skills/verify-pr/dispatch-template.md @@ -40,7 +40,9 @@ structure; the Agent-Specific Inputs section varies per agent. - `### PR Diff Summary` — file list with per-file line counts (additions/deletions), not full diff content - `### Task Specification` — Repository, Files to Modify, Files to Create sections from Jira task description - `### Jira Task ID` — the task key for commit traceability checking -- `### PR Commits` — commit list with hashes and messages +- `### PR Commits` — commit list; per commit the hash, the **full** headline and + body (never truncated or subject-only), and a `references_task_id` boolean + (runner-computed against the Jira Task ID over the full message) ### Security diff --git a/plugins/sdlc-workflow/skills/verify-pr/intent-alignment.md b/plugins/sdlc-workflow/skills/verify-pr/intent-alignment.md index 0cc2e27e2..c3c8a10f7 100644 --- a/plugins/sdlc-workflow/skills/verify-pr/intent-alignment.md +++ b/plugins/sdlc-workflow/skills/verify-pr/intent-alignment.md @@ -18,7 +18,9 @@ The orchestrator provides these sections in the Agent-Specific Inputs block - **Task Specification** — Repository, Files to Modify, Files to Create sections from the Jira task description - **Jira Task ID** — the task key (e.g., `PROJ-231`) for commit traceability checking -- **PR Commits** — commit list with hashes and messages +- **PR Commits** — commit list with, per commit, the hash, the **full** + headline and body (never subject-only), and a `references_task_id` boolean + that the runner computed against the Jira Task ID over the full message The dispatch envelope also includes **Context** (Jira Task, PR URL, Branch, Base Branch) and **Classified Review Comments** (all classified comments with IDs, @@ -83,20 +85,29 @@ Assess whether the total change size is proportionate to the task scope. Verify that commit messages reference the Jira task ID. -1. From the PR Commits input, extract all commit messages (headline + body). - -2. For each commit, check whether the message contains the Jira Task ID - (from the Jira Task ID input). The reference may appear in the headline, - body, or trailer (e.g., `Implements PROJ-231`, `PROJ-231: fix scope`, - `--trailer="Implements: PROJ-231"`). +1. Use ONLY the **PR Commits** input from the dispatch envelope. It is the + authoritative source: each commit carries the full headline + body and a + runner-computed `references_task_id` boolean. **Do NOT re-derive traceability + from the repository.** In particular, never run `git log --oneline`, + `git log --format=%s`, or any subject-only command — those emit only the + subject line and will miss a Jira ID that lives in the body or a trailer + (e.g. `Implements PROJ-231`), producing a false FAIL. The reference may + appear in the headline, body, or trailer (`Implements PROJ-231`, + `PROJ-231: fix scope`, `--trailer="Implements: PROJ-231"`). + +2. For each commit, take its `references_task_id` value. (If — and only if — + that field is absent for a commit, fall back to scanning that commit's full + headline AND body from the PR Commits input for the Jira Task ID; never fall + back to a subject-only git command.) 3. Determine verdict: - - **PASS** — every commit message references the Jira task ID + - **PASS** — every commit references the Jira task ID - **WARN** — some commits reference the task ID but others do not - **FAIL** — no commit references the Jira task ID 4. Evidence: list each commit hash and headline, marking which ones reference - the task ID and which do not. + the task ID (per `references_task_id`) and which do not. Do not assert a + commit's body lacks the ID unless the full body was inspected. ## Output Format From 3ff588f006a12d1afc40bc57e67ecc59df28662c Mon Sep 17 00:00:00 2001 From: mrizzi Date: Mon, 7 Sep 2026 17:49:42 +0200 Subject: [PATCH 050/175] chore(verify-pr): re-pin .fullsend base to 198fbb7c + re-lock Advance the URL-pinned base commit to 198fbb7c (the deterministic Commit Traceability fix) so a pinned `fullsend run verify-pr` resolves the corrected pre_verify_pr.py and skill files. Base file bytes unchanged (sha256 addc8686); lock.yaml re-frozen over the new commit's 11 deps. Assisted-by: Claude Code Co-Authored-By: Claude Opus 4.8 --- .fullsend/harness/verify-pr.yaml | 2 +- .fullsend/lock.yaml | 76 ++++++++++++++++---------------- 2 files changed, 39 insertions(+), 39 deletions(-) diff --git a/.fullsend/harness/verify-pr.yaml b/.fullsend/harness/verify-pr.yaml index 2a2dfe2e8..470e0a8af 100644 --- a/.fullsend/harness/verify-pr.yaml +++ b/.fullsend/harness/verify-pr.yaml @@ -27,7 +27,7 @@ # (base URLs are validated against that allowlist, not inherited). # Re-pin after any base edit: push harness/verify-pr.yaml, set the SHA to the new # commit + the sha256 to `shasum -a 256 harness/verify-pr.yaml`, `fullsend lock`. -base: https://raw.githubusercontent.com/RHEcosystemAppEng/sdlc-plugins/888a1781dd81f68bc6f4daecdcccef5d91b9cef6/harness/verify-pr.yaml#sha256=addc868659a4391b18fe09cb2a0523d134a9242e35d644a6db36bcce8277cb16 +base: https://raw.githubusercontent.com/RHEcosystemAppEng/sdlc-plugins/198fbb7cefbfa9c1bf7be2e4201acd0d4c0e6583/harness/verify-pr.yaml#sha256=addc868659a4391b18fe09cb2a0523d134a9242e35d644a6db36bcce8277cb16 host_files: - src: ${GOOGLE_APPLICATION_CREDENTIALS} diff --git a/.fullsend/lock.yaml b/.fullsend/lock.yaml index 389e098e4..070410f64 100644 --- a/.fullsend/lock.yaml +++ b/.fullsend/lock.yaml @@ -4,17 +4,17 @@ generated_at: 2026-09-04T13:39:01.038924Z harnesses: verify-pr: source: harness/verify-pr.yaml - sha256: 7de5dd2b5dacc019fcc42c4292844142708b849d42e3e75c6cbcfe6a2f9f92f5 - resolved_at: 2026-09-07T14:21:22.78544Z + sha256: b91cccc07e80a2dd64cde9904eee69e7806adaf4cad0c64758395a0baa204b41 + resolved_at: 2026-09-07T15:49:32.445338Z dependencies: - field: base - url: https://raw.githubusercontent.com/RHEcosystemAppEng/sdlc-plugins/888a1781dd81f68bc6f4daecdcccef5d91b9cef6/harness/verify-pr.yaml + url: https://raw.githubusercontent.com/RHEcosystemAppEng/sdlc-plugins/198fbb7cefbfa9c1bf7be2e4201acd0d4c0e6583/harness/verify-pr.yaml sha256: addc868659a4391b18fe09cb2a0523d134a9242e35d644a6db36bcce8277cb16 type: file fetched_at: 2026-09-04T13:20:03.650092Z - field: pre_script - url: https://raw.githubusercontent.com/RHEcosystemAppEng/sdlc-plugins/888a1781dd81f68bc6f4daecdcccef5d91b9cef6/plugins/sdlc-workflow/scripts/pre-verify-pr.sh - sha256: 085379c03f7436e1920dd07d9e50056ee4f1430c591b524645b271899ce093d7 + url: https://raw.githubusercontent.com/RHEcosystemAppEng/sdlc-plugins/198fbb7cefbfa9c1bf7be2e4201acd0d4c0e6583/plugins/sdlc-workflow/scripts/pre-verify-pr.sh + sha256: 7e6144024f052b513bf50eb9c71199587bbaa8d20499f0be26c563a0da447b8c type: directory files: - path: execute-actions.py @@ -26,7 +26,7 @@ harnesses: - path: pre-verify-pr.sh sha256: 041bd777d3a8915d0818aa6d24ed188443aba4c199f5ecdd543d8a86677a8c5b - path: pre_verify_pr.py - sha256: 2ee6ab01b7e277a526ca5fb4f0110a5bf0845ce6562bce56a6c6d0e4edaa93fd + sha256: 4d9124447aee93ca4e623058d1a0842b39d8fc6f458e569195ae2c73a10dc5cb - path: strip_extra_properties.py sha256: 9099200014a0a4a9da8795a3346f963efc34c53f4603d340551c46b8d135d122 - path: test_execute_actions.py @@ -36,13 +36,13 @@ harnesses: - path: test_jira_client_cli.py sha256: d36b790c1a310111a0d2f3e81415f9576e80acc5f0bc7d0ff8084ffad001285f - path: test_pre_verify_pr.py - sha256: 7f6b2704a8e620669c275712844124e2f06d857fa1f09fbd689589b5313f09cf + sha256: d244bf91dfce478b041f8a0651b3241aa191734b5d5ad91899fafd2fbcc54825 - path: validate-output-schema.sh sha256: 56b964145da0b62f438dbc1885780d86e2d589c5340adb93c00e3c8777979cbe - fetched_at: 2026-09-07T14:21:12.125687Z + fetched_at: 2026-09-07T15:49:23.749776Z - field: post_script - url: https://raw.githubusercontent.com/RHEcosystemAppEng/sdlc-plugins/888a1781dd81f68bc6f4daecdcccef5d91b9cef6/plugins/sdlc-workflow/scripts/post-verify-pr.sh - sha256: 085379c03f7436e1920dd07d9e50056ee4f1430c591b524645b271899ce093d7 + url: https://raw.githubusercontent.com/RHEcosystemAppEng/sdlc-plugins/198fbb7cefbfa9c1bf7be2e4201acd0d4c0e6583/plugins/sdlc-workflow/scripts/post-verify-pr.sh + sha256: 7e6144024f052b513bf50eb9c71199587bbaa8d20499f0be26c563a0da447b8c type: directory files: - path: execute-actions.py @@ -54,7 +54,7 @@ harnesses: - path: pre-verify-pr.sh sha256: 041bd777d3a8915d0818aa6d24ed188443aba4c199f5ecdd543d8a86677a8c5b - path: pre_verify_pr.py - sha256: 2ee6ab01b7e277a526ca5fb4f0110a5bf0845ce6562bce56a6c6d0e4edaa93fd + sha256: 4d9124447aee93ca4e623058d1a0842b39d8fc6f458e569195ae2c73a10dc5cb - path: strip_extra_properties.py sha256: 9099200014a0a4a9da8795a3346f963efc34c53f4603d340551c46b8d135d122 - path: test_execute_actions.py @@ -64,13 +64,13 @@ harnesses: - path: test_jira_client_cli.py sha256: d36b790c1a310111a0d2f3e81415f9576e80acc5f0bc7d0ff8084ffad001285f - path: test_pre_verify_pr.py - sha256: 7f6b2704a8e620669c275712844124e2f06d857fa1f09fbd689589b5313f09cf + sha256: d244bf91dfce478b041f8a0651b3241aa191734b5d5ad91899fafd2fbcc54825 - path: validate-output-schema.sh sha256: 56b964145da0b62f438dbc1885780d86e2d589c5340adb93c00e3c8777979cbe - fetched_at: 2026-09-07T14:21:12.125687Z + fetched_at: 2026-09-07T15:49:23.749776Z - field: validation_loop.script - url: https://raw.githubusercontent.com/RHEcosystemAppEng/sdlc-plugins/888a1781dd81f68bc6f4daecdcccef5d91b9cef6/plugins/sdlc-workflow/scripts/validate-output-schema.sh - sha256: 085379c03f7436e1920dd07d9e50056ee4f1430c591b524645b271899ce093d7 + url: https://raw.githubusercontent.com/RHEcosystemAppEng/sdlc-plugins/198fbb7cefbfa9c1bf7be2e4201acd0d4c0e6583/plugins/sdlc-workflow/scripts/validate-output-schema.sh + sha256: 7e6144024f052b513bf50eb9c71199587bbaa8d20499f0be26c563a0da447b8c type: directory files: - path: execute-actions.py @@ -82,7 +82,7 @@ harnesses: - path: pre-verify-pr.sh sha256: 041bd777d3a8915d0818aa6d24ed188443aba4c199f5ecdd543d8a86677a8c5b - path: pre_verify_pr.py - sha256: 2ee6ab01b7e277a526ca5fb4f0110a5bf0845ce6562bce56a6c6d0e4edaa93fd + sha256: 4d9124447aee93ca4e623058d1a0842b39d8fc6f458e569195ae2c73a10dc5cb - path: strip_extra_properties.py sha256: 9099200014a0a4a9da8795a3346f963efc34c53f4603d340551c46b8d135d122 - path: test_execute_actions.py @@ -92,43 +92,43 @@ harnesses: - path: test_jira_client_cli.py sha256: d36b790c1a310111a0d2f3e81415f9576e80acc5f0bc7d0ff8084ffad001285f - path: test_pre_verify_pr.py - sha256: 7f6b2704a8e620669c275712844124e2f06d857fa1f09fbd689589b5313f09cf + sha256: d244bf91dfce478b041f8a0651b3241aa191734b5d5ad91899fafd2fbcc54825 - path: validate-output-schema.sh sha256: 56b964145da0b62f438dbc1885780d86e2d589c5340adb93c00e3c8777979cbe - fetched_at: 2026-09-07T14:21:12.125687Z + fetched_at: 2026-09-07T15:49:23.749776Z - field: validation_loop.schema - url: https://raw.githubusercontent.com/RHEcosystemAppEng/sdlc-plugins/888a1781dd81f68bc6f4daecdcccef5d91b9cef6/plugins/sdlc-workflow/schemas/verify-pr-result.schema.json + url: https://raw.githubusercontent.com/RHEcosystemAppEng/sdlc-plugins/198fbb7cefbfa9c1bf7be2e4201acd0d4c0e6583/plugins/sdlc-workflow/schemas/verify-pr-result.schema.json sha256: 62c3e5ce0c47e73823b5cab9a577be7b721d8fb31389bc8602dc82ee05526ad6 type: resource - fetched_at: 2026-09-07T14:21:17.494543Z + fetched_at: 2026-09-07T15:49:27.977532Z - field: agent - url: https://raw.githubusercontent.com/RHEcosystemAppEng/sdlc-plugins/888a1781dd81f68bc6f4daecdcccef5d91b9cef6/plugins/sdlc-workflow/agents/verify-pr.md + url: https://raw.githubusercontent.com/RHEcosystemAppEng/sdlc-plugins/198fbb7cefbfa9c1bf7be2e4201acd0d4c0e6583/plugins/sdlc-workflow/agents/verify-pr.md sha256: 568b2ab7e32300c84038d500d95f4534ff6c4b20e52eebf2ed5a6c0ae706786a type: resource - fetched_at: 2026-09-07T14:21:17.75376Z + fetched_at: 2026-09-07T15:49:28.283001Z - field: policy - url: https://raw.githubusercontent.com/RHEcosystemAppEng/sdlc-plugins/888a1781dd81f68bc6f4daecdcccef5d91b9cef6/plugins/sdlc-workflow/policies/verify-pr.yaml + url: https://raw.githubusercontent.com/RHEcosystemAppEng/sdlc-plugins/198fbb7cefbfa9c1bf7be2e4201acd0d4c0e6583/plugins/sdlc-workflow/policies/verify-pr.yaml sha256: 62d1374be253b0b225921bd8e22e679781076ecdb3a9ea22d5e6a4805753c5a7 type: resource - fetched_at: 2026-09-07T14:21:18.012637Z + fetched_at: 2026-09-07T15:49:28.616909Z - field: host_files[0].src - url: https://raw.githubusercontent.com/RHEcosystemAppEng/sdlc-plugins/888a1781dd81f68bc6f4daecdcccef5d91b9cef6/plugins/sdlc-workflow/env/gcp-vertex.env + url: https://raw.githubusercontent.com/RHEcosystemAppEng/sdlc-plugins/198fbb7cefbfa9c1bf7be2e4201acd0d4c0e6583/plugins/sdlc-workflow/env/gcp-vertex.env sha256: 10b2ba695b1d4e65e0964a233b75d6a06853f1eaefc260c56b9eedd989ae7d41 type: resource - fetched_at: 2026-09-07T14:21:18.26852Z + fetched_at: 2026-09-07T15:49:29.17099Z - field: openshell.profiles[0] - url: https://raw.githubusercontent.com/RHEcosystemAppEng/sdlc-plugins/888a1781dd81f68bc6f4daecdcccef5d91b9cef6/plugins/sdlc-workflow/profiles/fullsend-vertex-ai.yaml + url: https://raw.githubusercontent.com/RHEcosystemAppEng/sdlc-plugins/198fbb7cefbfa9c1bf7be2e4201acd0d4c0e6583/plugins/sdlc-workflow/profiles/fullsend-vertex-ai.yaml sha256: 76535a148387b1281be2bfb23b6724e4133327189b38c8ce1de2c6dddb8d8341 type: resource - fetched_at: 2026-09-07T14:21:18.556217Z + fetched_at: 2026-09-07T15:49:29.416034Z - field: providers[0] - url: https://raw.githubusercontent.com/RHEcosystemAppEng/sdlc-plugins/888a1781dd81f68bc6f4daecdcccef5d91b9cef6/plugins/sdlc-workflow/providers/vertex-ai.yaml + url: https://raw.githubusercontent.com/RHEcosystemAppEng/sdlc-plugins/198fbb7cefbfa9c1bf7be2e4201acd0d4c0e6583/plugins/sdlc-workflow/providers/vertex-ai.yaml sha256: ae5ebe527e5d7b0fa6994346fde3f6ba11b632a3ba78ece96591b5ed85083ec1 type: resource - fetched_at: 2026-09-07T14:21:18.880884Z + fetched_at: 2026-09-07T15:49:29.677281Z - field: plugins[0] - url: https://raw.githubusercontent.com/RHEcosystemAppEng/sdlc-plugins/888a1781dd81f68bc6f4daecdcccef5d91b9cef6/plugins/sdlc-workflow/plugin.json - sha256: 83c90724d8b581e7f173e3734e461b3cd515ddede27e40c2032d84f319f9e10f + url: https://raw.githubusercontent.com/RHEcosystemAppEng/sdlc-plugins/198fbb7cefbfa9c1bf7be2e4201acd0d4c0e6583/plugins/sdlc-workflow/plugin.json + sha256: bc42886e3d430db57820d99c6afee5c9aadd6045b68f4636dcb5ba05a3d7819c type: directory files: - path: .claude-plugin/plugin.json @@ -158,7 +158,7 @@ harnesses: - path: scripts/pre-verify-pr.sh sha256: 041bd777d3a8915d0818aa6d24ed188443aba4c199f5ecdd543d8a86677a8c5b - path: scripts/pre_verify_pr.py - sha256: 2ee6ab01b7e277a526ca5fb4f0110a5bf0845ce6562bce56a6c6d0e4edaa93fd + sha256: 4d9124447aee93ca4e623058d1a0842b39d8fc6f458e569195ae2c73a10dc5cb - path: scripts/strip_extra_properties.py sha256: 9099200014a0a4a9da8795a3346f963efc34c53f4603d340551c46b8d135d122 - path: scripts/test_execute_actions.py @@ -168,7 +168,7 @@ harnesses: - path: scripts/test_jira_client_cli.py sha256: d36b790c1a310111a0d2f3e81415f9576e80acc5f0bc7d0ff8084ffad001285f - path: scripts/test_pre_verify_pr.py - sha256: 7f6b2704a8e620669c275712844124e2f06d857fa1f09fbd689589b5313f09cf + sha256: d244bf91dfce478b041f8a0651b3241aa191734b5d5ad91899fafd2fbcc54825 - path: scripts/validate-output-schema.sh sha256: 56b964145da0b62f438dbc1885780d86e2d589c5340adb93c00e3c8777979cbe - path: shared/comment-footnote.md @@ -224,17 +224,17 @@ harnesses: - path: skills/triage-security/version-impact-analysis.md sha256: a18376094360d435e96436c9fd1bd6a04831123d903e44cf6cf57e600285e3c9 - path: skills/verify-pr/SKILL.md - sha256: b57b1ac91b1996239429518dd418277a2f8e6f29aef08275a37f4df95990f825 + sha256: bc409c8120e6b1cdd58140b1fb523dc80bcd51fde77d93f1264c8791b919adc1 - path: skills/verify-pr/correctness.md sha256: 44a4c095038d41434b5e658dfd209970a77c1c0417bc2c00548ce1f85abdbade - path: skills/verify-pr/dispatch-template.md - sha256: 5d1c691c7b24f2975664ea002e5a0d3be2eb6127e901c3be8bf4d98121f04c48 + sha256: 666acc69b83f81b3dbcbf72ffe45cd1a678101440354c7585c17e48296f6e84e - path: skills/verify-pr/finding-template.md sha256: b3a7c52d60d1ffc30f5d80219b2fc9db6c28eabeca83bcc0ffa1e424b3a4d0c2 - path: skills/verify-pr/intent-alignment.md - sha256: 1490378c0510b9645e3a3f550b7b0383803959585a658eb898084efc6505d808 + sha256: 8a1ffc88f4072df77fc50a12bb6443be6e6884ac5c51b4501dac0e0e18c37f01 - path: skills/verify-pr/security.md sha256: 128877cf7e156666dbb403805a0112059cc8cc3255e148c6794da29d48b8f5cf - path: skills/verify-pr/style-conventions.md sha256: 5ab920c596615c0d23354cb0e90fe5ebf5f08635ffa3602e8e2d2abd74028a19 - fetched_at: 2026-09-07T14:21:22.780456Z + fetched_at: 2026-09-07T15:49:32.441189Z From e5c825358fab4ff7114bfadbd6fe07ffa1cb244f Mon Sep 17 00:00:00 2001 From: mrizzi Date: Mon, 7 Sep 2026 19:09:51 +0200 Subject: [PATCH 051/175] fix(verify-pr): anchor own bundled files to ${CLAUDE_PLUGIN_ROOT} The verify-pr skill read its own sub-skill files, dispatch/finding templates, JSON schemas, and plugin manifest via repo-relative paths (plugins/sdlc-workflow/...). The current working directory is not always the plugin's repository: when verify-pr reviews a PR against sdlc-plugins itself, CWD is the target checkout, so those repo-relative paths resolved into the branch under review and let the PR shadow the pinned, stable skill actually running (e.g. an old intent-alignment.md, causing a fall back to subject-only `git log` for Commit Traceability). Anchor every reference this skill reads to ${CLAUDE_PLUGIN_ROOT}, the env var Claude Code sets to the delivered plugin's install directory. It resolves regardless of CWD in both interactive (Claude Code) and sandbox (fullsend) modes, so the stable skill always reads its own bundled files. Inside the quoted schema-validation heredoc the shell does not expand the variable, so it is read via os.environ["CLAUDE_PLUGIN_ROOT"] instead. PR-diff filter patterns (run-evals paths) stay repo-relative: they match files inside the PR under review, not files this skill reads. Assisted-by: Claude Code Co-Authored-By: Claude Opus 4.8 --- .../sdlc-workflow/skills/verify-pr/SKILL.md | 57 +++++++++++++------ 1 file changed, 41 insertions(+), 16 deletions(-) diff --git a/plugins/sdlc-workflow/skills/verify-pr/SKILL.md b/plugins/sdlc-workflow/skills/verify-pr/SKILL.md index aab309b7e..a58da9f19 100644 --- a/plugins/sdlc-workflow/skills/verify-pr/SKILL.md +++ b/plugins/sdlc-workflow/skills/verify-pr/SKILL.md @@ -9,6 +9,30 @@ argument-hint: "[jira-issue-id]" You are an AI verification assistant that orchestrates PR verification through parallel domain sub-agents. You verify a pull request against its Jira task's acceptance criteria and deterministic guardrails. You classify PR review feedback, dispatch domain sub-agents for parallel analysis, aggregate their findings, create tracked Jira sub-tasks for required code fixes, and investigate root causes of implementation mistakes across the full workflow chain. You post findings to both GitHub and Jira, but you do **NOT** modify code and do **NOT** auto-merge. +## Resolving this skill's own files + +This skill reads several of its own bundled files: sub-skill instruction files, +dispatch/finding templates, JSON schemas, and the plugin manifest. **Always resolve +these from `${CLAUDE_PLUGIN_ROOT}`** — the environment variable Claude Code sets to +this plugin's installation directory — never from a repo-relative path like +`plugins/sdlc-workflow/...`. + +This matters because the current working directory is **not** always the plugin's +repository. When verify-pr reviews a PR against the `sdlc-plugins` repo itself, the +CWD is the target checkout (whatever branch is under review), so a repo-relative path +would read that branch's copy of these files and let the PR under review **shadow** +the pinned, stable skill actually running. `${CLAUDE_PLUGIN_ROOT}` always points at +the delivered plugin, so the stable skill reads its own bundled files in every mode — +interactive (Claude Code) and sandbox (fullsend). + +`${CLAUDE_PLUGIN_ROOT}` expands in this skill's body text and in Bash commands. Inside +a quoted heredoc (`<< 'PYEOF'`), the shell does **not** expand it — read it with +`os.environ["CLAUDE_PLUGIN_ROOT"]` in Python instead. + +Note: path patterns used to **filter the PR diff** (e.g., detecting changes under +`plugins/sdlc-workflow/skills/run-evals/`) stay repo-relative — those describe files +inside the PR being reviewed, not files this skill reads. + ## Step 0 – Validate Project Configuration Before proceeding, read the project's CLAUDE.md and verify that the following sections exist under `# Project Configuration`: @@ -113,7 +137,7 @@ Initialize the accumulator as an in-memory JSON structure: ``` The `report` object and every action conform to -`plugins/sdlc-workflow/schemas/verify-pr-result.schema.json`; the runner's +`${CLAUDE_PLUGIN_ROOT}/schemas/verify-pr-result.schema.json`; the runner's `post_script` executes the accumulated actions after the sandbox exits. ## Step 0.7 – Load Pre-Fetched Data (sandbox mode only) @@ -123,18 +147,19 @@ The `report` object and every action conform to In sandbox mode the `pre_script` fetches all task and PR data on the trusted runner (where the tokens live) and mounts it read-only into the sandbox. Read it: -Validate it against `plugins/sdlc-workflow/schemas/verify-pr-input.schema.json` +Validate it against `${CLAUDE_PLUGIN_ROOT}/schemas/verify-pr-input.schema.json` before using it — a syntactically valid but structurally wrong or incomplete prefetch (e.g., missing the `github` bundle or `task` fields) must be treated as invalid rather than passing and failing deep inside a later step: ```bash python3 - << 'PYEOF' -import json, sys +import json, os, sys from jsonschema import validate, ValidationError INPUT = "/sandbox/workspace/.pre-script/verify-pr-input.json" -SCHEMA = "plugins/sdlc-workflow/schemas/verify-pr-input.schema.json" +# ${CLAUDE_PLUGIN_ROOT} is not expanded inside a quoted heredoc — read it from the env. +SCHEMA = os.path.join(os.environ["CLAUDE_PLUGIN_ROOT"], "schemas/verify-pr-input.schema.json") try: with open(INPUT) as f: instance = json.load(f) @@ -213,7 +238,7 @@ Every comment posted to Jira by this skill MUST end with the following footnote, separated from the main content by a horizontal rule. Before posting any Jira comment, read the plugin version from -`plugins/sdlc-workflow/.claude-plugin/plugin.json` and extract the `version` field. +`${CLAUDE_PLUGIN_ROOT}/.claude-plugin/plugin.json` and extract the `version` field. Use this value as `{version}` in the footer below. Use ADF `contentFormat` to ensure the rule and text render correctly: @@ -461,7 +486,7 @@ change requests before sub-task creation. Dispatch four domain sub-agents in parallel for comprehensive PR analysis. Each sub-agent performs focused checks and returns structured findings. The orchestrator constructs dispatch envelopes following the structure defined in -`plugins/sdlc-workflow/skills/verify-pr/dispatch-template.md`. +`${CLAUDE_PLUGIN_ROOT}/skills/verify-pr/dispatch-template.md`. ### Step 5a – Gather Dispatch Inputs @@ -528,13 +553,13 @@ Collect all inputs needed for sub-agent dispatch envelopes: Read the following files to construct dispatch prompts: -1. **Dispatch template:** `plugins/sdlc-workflow/skills/verify-pr/dispatch-template.md` -2. **Finding template:** `plugins/sdlc-workflow/skills/verify-pr/finding-template.md` +1. **Dispatch template:** `${CLAUDE_PLUGIN_ROOT}/skills/verify-pr/dispatch-template.md` +2. **Finding template:** `${CLAUDE_PLUGIN_ROOT}/skills/verify-pr/finding-template.md` 3. **Sub-agent skill files:** - - `plugins/sdlc-workflow/skills/verify-pr/intent-alignment.md` - - `plugins/sdlc-workflow/skills/verify-pr/security.md` - - `plugins/sdlc-workflow/skills/verify-pr/correctness.md` - - `plugins/sdlc-workflow/skills/verify-pr/style-conventions.md` + - `${CLAUDE_PLUGIN_ROOT}/skills/verify-pr/intent-alignment.md` + - `${CLAUDE_PLUGIN_ROOT}/skills/verify-pr/security.md` + - `${CLAUDE_PLUGIN_ROOT}/skills/verify-pr/correctness.md` + - `${CLAUDE_PLUGIN_ROOT}/skills/verify-pr/style-conventions.md` ### Step 5c – Construct and Dispatch @@ -576,7 +601,7 @@ execute side effects (sub-task creation, PR comment replies). ### Step 6a – Collect Sub-Agent Results Parse the structured findings returned by each sub-agent using the format defined -in `plugins/sdlc-workflow/skills/verify-pr/finding-template.md`: +in `${CLAUDE_PLUGIN_ROOT}/skills/verify-pr/finding-template.md`: 1. **Extract verdicts** from each sub-agent's Verdicts table. Map sub-agent check names to report rows: @@ -1271,7 +1296,7 @@ Append a markdown footnote at the end of the report body, separated by a horizon *This comment was AI-generated by [sdlc-workflow/verify-pr](https://github.com/RHEcosystemAppEng/sdlc-plugins) v{version}.* ``` -Read the plugin version from `plugins/sdlc-workflow/.claude-plugin/plugin.json` and +Read the plugin version from `${CLAUDE_PLUGIN_ROOT}/.claude-plugin/plugin.json` and substitute `{version}` before posting. Build the comment body as the report (with the commit-scoped header and footnote) @@ -1311,7 +1336,7 @@ find-then-update-or-create path above) and the Jira comment (from `report_adf`) the sandbox exits. Populate `report` (all fields required by -`plugins/sdlc-workflow/schemas/verify-pr-result.schema.json`): +`${CLAUDE_PLUGIN_ROOT}/schemas/verify-pr-result.schema.json`): ```json { @@ -1323,7 +1348,7 @@ Populate `report` (all fields required by "table_md": "", "report_md": "", "report_adf": , - "plugin_version": "" + "plugin_version": "" } ``` From 56a1c9e977cc27c4cf51b00bb51a9fab3ccdef91 Mon Sep 17 00:00:00 2001 From: mrizzi Date: Mon, 7 Sep 2026 19:26:36 +0200 Subject: [PATCH 052/175] chore(verify-pr): re-pin .fullsend base URL to e5c82535, re-lock MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Advance the URL-pinned base commit segment to e5c825358fab (the ${CLAUDE_PLUGIN_ROOT} shadowing fix) and regenerate lock.yaml (11 deps). Base file harness/verify-pr.yaml is byte-identical, so its sha256 (addc8686…) is unchanged; only the commit segment advances. Proven end-to-end: pinned verify-pr run against an sdlc-plugins target checked out on `main` (old subject-only intent-alignment.md) still used its own delivered skill files via ${CLAUDE_PLUGIN_ROOT} — Commit Traceability WARN ("1/8 commits references TC-5812" via references_task_id, not the git-log fallback), Overall WARN, no FAIL. Shadowing eliminated. Assisted-by: Claude Code Co-Authored-By: Claude Opus 4.8 --- .fullsend/harness/verify-pr.yaml | 2 +- .fullsend/lock.yaml | 50 ++++++++++++++++---------------- 2 files changed, 26 insertions(+), 26 deletions(-) diff --git a/.fullsend/harness/verify-pr.yaml b/.fullsend/harness/verify-pr.yaml index 470e0a8af..ffb353b35 100644 --- a/.fullsend/harness/verify-pr.yaml +++ b/.fullsend/harness/verify-pr.yaml @@ -27,7 +27,7 @@ # (base URLs are validated against that allowlist, not inherited). # Re-pin after any base edit: push harness/verify-pr.yaml, set the SHA to the new # commit + the sha256 to `shasum -a 256 harness/verify-pr.yaml`, `fullsend lock`. -base: https://raw.githubusercontent.com/RHEcosystemAppEng/sdlc-plugins/198fbb7cefbfa9c1bf7be2e4201acd0d4c0e6583/harness/verify-pr.yaml#sha256=addc868659a4391b18fe09cb2a0523d134a9242e35d644a6db36bcce8277cb16 +base: https://raw.githubusercontent.com/RHEcosystemAppEng/sdlc-plugins/e5c825358fab4ff7114bfadbd6fe07ffa1cb244f/harness/verify-pr.yaml#sha256=addc868659a4391b18fe09cb2a0523d134a9242e35d644a6db36bcce8277cb16 host_files: - src: ${GOOGLE_APPLICATION_CREDENTIALS} diff --git a/.fullsend/lock.yaml b/.fullsend/lock.yaml index 070410f64..468e65112 100644 --- a/.fullsend/lock.yaml +++ b/.fullsend/lock.yaml @@ -4,16 +4,16 @@ generated_at: 2026-09-04T13:39:01.038924Z harnesses: verify-pr: source: harness/verify-pr.yaml - sha256: b91cccc07e80a2dd64cde9904eee69e7806adaf4cad0c64758395a0baa204b41 - resolved_at: 2026-09-07T15:49:32.445338Z + sha256: 6257c910d87735834ea40947a2307479099adeaf13f123b39a3e6795b8a47b7a + resolved_at: 2026-09-07T17:10:50.749797Z dependencies: - field: base - url: https://raw.githubusercontent.com/RHEcosystemAppEng/sdlc-plugins/198fbb7cefbfa9c1bf7be2e4201acd0d4c0e6583/harness/verify-pr.yaml + url: https://raw.githubusercontent.com/RHEcosystemAppEng/sdlc-plugins/e5c825358fab4ff7114bfadbd6fe07ffa1cb244f/harness/verify-pr.yaml sha256: addc868659a4391b18fe09cb2a0523d134a9242e35d644a6db36bcce8277cb16 type: file fetched_at: 2026-09-04T13:20:03.650092Z - field: pre_script - url: https://raw.githubusercontent.com/RHEcosystemAppEng/sdlc-plugins/198fbb7cefbfa9c1bf7be2e4201acd0d4c0e6583/plugins/sdlc-workflow/scripts/pre-verify-pr.sh + url: https://raw.githubusercontent.com/RHEcosystemAppEng/sdlc-plugins/e5c825358fab4ff7114bfadbd6fe07ffa1cb244f/plugins/sdlc-workflow/scripts/pre-verify-pr.sh sha256: 7e6144024f052b513bf50eb9c71199587bbaa8d20499f0be26c563a0da447b8c type: directory files: @@ -39,9 +39,9 @@ harnesses: sha256: d244bf91dfce478b041f8a0651b3241aa191734b5d5ad91899fafd2fbcc54825 - path: validate-output-schema.sh sha256: 56b964145da0b62f438dbc1885780d86e2d589c5340adb93c00e3c8777979cbe - fetched_at: 2026-09-07T15:49:23.749776Z + fetched_at: 2026-09-07T17:10:46.703717Z - field: post_script - url: https://raw.githubusercontent.com/RHEcosystemAppEng/sdlc-plugins/198fbb7cefbfa9c1bf7be2e4201acd0d4c0e6583/plugins/sdlc-workflow/scripts/post-verify-pr.sh + url: https://raw.githubusercontent.com/RHEcosystemAppEng/sdlc-plugins/e5c825358fab4ff7114bfadbd6fe07ffa1cb244f/plugins/sdlc-workflow/scripts/post-verify-pr.sh sha256: 7e6144024f052b513bf50eb9c71199587bbaa8d20499f0be26c563a0da447b8c type: directory files: @@ -67,9 +67,9 @@ harnesses: sha256: d244bf91dfce478b041f8a0651b3241aa191734b5d5ad91899fafd2fbcc54825 - path: validate-output-schema.sh sha256: 56b964145da0b62f438dbc1885780d86e2d589c5340adb93c00e3c8777979cbe - fetched_at: 2026-09-07T15:49:23.749776Z + fetched_at: 2026-09-07T17:10:46.695617Z - field: validation_loop.script - url: https://raw.githubusercontent.com/RHEcosystemAppEng/sdlc-plugins/198fbb7cefbfa9c1bf7be2e4201acd0d4c0e6583/plugins/sdlc-workflow/scripts/validate-output-schema.sh + url: https://raw.githubusercontent.com/RHEcosystemAppEng/sdlc-plugins/e5c825358fab4ff7114bfadbd6fe07ffa1cb244f/plugins/sdlc-workflow/scripts/validate-output-schema.sh sha256: 7e6144024f052b513bf50eb9c71199587bbaa8d20499f0be26c563a0da447b8c type: directory files: @@ -95,40 +95,40 @@ harnesses: sha256: d244bf91dfce478b041f8a0651b3241aa191734b5d5ad91899fafd2fbcc54825 - path: validate-output-schema.sh sha256: 56b964145da0b62f438dbc1885780d86e2d589c5340adb93c00e3c8777979cbe - fetched_at: 2026-09-07T15:49:23.749776Z + fetched_at: 2026-09-07T17:10:46.695617Z - field: validation_loop.schema - url: https://raw.githubusercontent.com/RHEcosystemAppEng/sdlc-plugins/198fbb7cefbfa9c1bf7be2e4201acd0d4c0e6583/plugins/sdlc-workflow/schemas/verify-pr-result.schema.json + url: https://raw.githubusercontent.com/RHEcosystemAppEng/sdlc-plugins/e5c825358fab4ff7114bfadbd6fe07ffa1cb244f/plugins/sdlc-workflow/schemas/verify-pr-result.schema.json sha256: 62c3e5ce0c47e73823b5cab9a577be7b721d8fb31389bc8602dc82ee05526ad6 type: resource - fetched_at: 2026-09-07T15:49:27.977532Z + fetched_at: 2026-09-07T17:10:47.026662Z - field: agent - url: https://raw.githubusercontent.com/RHEcosystemAppEng/sdlc-plugins/198fbb7cefbfa9c1bf7be2e4201acd0d4c0e6583/plugins/sdlc-workflow/agents/verify-pr.md + url: https://raw.githubusercontent.com/RHEcosystemAppEng/sdlc-plugins/e5c825358fab4ff7114bfadbd6fe07ffa1cb244f/plugins/sdlc-workflow/agents/verify-pr.md sha256: 568b2ab7e32300c84038d500d95f4534ff6c4b20e52eebf2ed5a6c0ae706786a type: resource - fetched_at: 2026-09-07T15:49:28.283001Z + fetched_at: 2026-09-07T17:10:47.303981Z - field: policy - url: https://raw.githubusercontent.com/RHEcosystemAppEng/sdlc-plugins/198fbb7cefbfa9c1bf7be2e4201acd0d4c0e6583/plugins/sdlc-workflow/policies/verify-pr.yaml + url: https://raw.githubusercontent.com/RHEcosystemAppEng/sdlc-plugins/e5c825358fab4ff7114bfadbd6fe07ffa1cb244f/plugins/sdlc-workflow/policies/verify-pr.yaml sha256: 62d1374be253b0b225921bd8e22e679781076ecdb3a9ea22d5e6a4805753c5a7 type: resource - fetched_at: 2026-09-07T15:49:28.616909Z + fetched_at: 2026-09-07T17:10:47.556767Z - field: host_files[0].src - url: https://raw.githubusercontent.com/RHEcosystemAppEng/sdlc-plugins/198fbb7cefbfa9c1bf7be2e4201acd0d4c0e6583/plugins/sdlc-workflow/env/gcp-vertex.env + url: https://raw.githubusercontent.com/RHEcosystemAppEng/sdlc-plugins/e5c825358fab4ff7114bfadbd6fe07ffa1cb244f/plugins/sdlc-workflow/env/gcp-vertex.env sha256: 10b2ba695b1d4e65e0964a233b75d6a06853f1eaefc260c56b9eedd989ae7d41 type: resource - fetched_at: 2026-09-07T15:49:29.17099Z + fetched_at: 2026-09-07T17:10:47.80023Z - field: openshell.profiles[0] - url: https://raw.githubusercontent.com/RHEcosystemAppEng/sdlc-plugins/198fbb7cefbfa9c1bf7be2e4201acd0d4c0e6583/plugins/sdlc-workflow/profiles/fullsend-vertex-ai.yaml + url: https://raw.githubusercontent.com/RHEcosystemAppEng/sdlc-plugins/e5c825358fab4ff7114bfadbd6fe07ffa1cb244f/plugins/sdlc-workflow/profiles/fullsend-vertex-ai.yaml sha256: 76535a148387b1281be2bfb23b6724e4133327189b38c8ce1de2c6dddb8d8341 type: resource - fetched_at: 2026-09-07T15:49:29.416034Z + fetched_at: 2026-09-07T17:10:48.047183Z - field: providers[0] - url: https://raw.githubusercontent.com/RHEcosystemAppEng/sdlc-plugins/198fbb7cefbfa9c1bf7be2e4201acd0d4c0e6583/plugins/sdlc-workflow/providers/vertex-ai.yaml + url: https://raw.githubusercontent.com/RHEcosystemAppEng/sdlc-plugins/e5c825358fab4ff7114bfadbd6fe07ffa1cb244f/plugins/sdlc-workflow/providers/vertex-ai.yaml sha256: ae5ebe527e5d7b0fa6994346fde3f6ba11b632a3ba78ece96591b5ed85083ec1 type: resource - fetched_at: 2026-09-07T15:49:29.677281Z + fetched_at: 2026-09-07T17:10:48.312935Z - field: plugins[0] - url: https://raw.githubusercontent.com/RHEcosystemAppEng/sdlc-plugins/198fbb7cefbfa9c1bf7be2e4201acd0d4c0e6583/plugins/sdlc-workflow/plugin.json - sha256: bc42886e3d430db57820d99c6afee5c9aadd6045b68f4636dcb5ba05a3d7819c + url: https://raw.githubusercontent.com/RHEcosystemAppEng/sdlc-plugins/e5c825358fab4ff7114bfadbd6fe07ffa1cb244f/plugins/sdlc-workflow/plugin.json + sha256: def66f4c52585908a51a91b005d5932ccf79658ca84edac9483bb1da68444314 type: directory files: - path: .claude-plugin/plugin.json @@ -224,7 +224,7 @@ harnesses: - path: skills/triage-security/version-impact-analysis.md sha256: a18376094360d435e96436c9fd1bd6a04831123d903e44cf6cf57e600285e3c9 - path: skills/verify-pr/SKILL.md - sha256: bc409c8120e6b1cdd58140b1fb523dc80bcd51fde77d93f1264c8791b919adc1 + sha256: 401beec545bf964606405bfcd6bd932f0f8c3a1d3221b9f46f31f41595346f14 - path: skills/verify-pr/correctness.md sha256: 44a4c095038d41434b5e658dfd209970a77c1c0417bc2c00548ce1f85abdbade - path: skills/verify-pr/dispatch-template.md @@ -237,4 +237,4 @@ harnesses: sha256: 128877cf7e156666dbb403805a0112059cc8cc3255e148c6794da29d48b8f5cf - path: skills/verify-pr/style-conventions.md sha256: 5ab920c596615c0d23354cb0e90fe5ebf5f08635ffa3602e8e2d2abd74028a19 - fetched_at: 2026-09-07T15:49:32.441189Z + fetched_at: 2026-09-07T17:10:50.747725Z From b75e96485327249b7dd9177bd3b13a2602311256 Mon Sep 17 00:00:00 2001 From: mrizzi Date: Mon, 7 Sep 2026 19:47:46 +0200 Subject: [PATCH 053/175] fix(verify-pr): use ${CLAUDE_PLUGIN_ROOT} as a body token, not an env var The schema-validation heredoc read os.environ["CLAUDE_PLUGIN_ROOT"], which raises KeyError: ${CLAUDE_PLUGIN_ROOT} is NOT an OS environment variable. It is a textual token Claude Code substitutes into a plugin skill's markdown body (and allowed-tools Bash rules) before the skill runs. The prose/Read references already relied on this correctly; only the heredoc was wrong. Write the literal token in the heredoc so Claude Code replaces it with the delivered plugin's absolute path before Python runs; drop the unused os import. Update the "Resolving this skill's own files" note to state the token is body-substituted (not exported to the shell/subprocess). Assisted-by: Claude Code Co-Authored-By: Claude Opus 4.8 --- .../sdlc-workflow/skills/verify-pr/SKILL.md | 18 ++++++++++++------ 1 file changed, 12 insertions(+), 6 deletions(-) diff --git a/plugins/sdlc-workflow/skills/verify-pr/SKILL.md b/plugins/sdlc-workflow/skills/verify-pr/SKILL.md index a58da9f19..4dfeeeede 100644 --- a/plugins/sdlc-workflow/skills/verify-pr/SKILL.md +++ b/plugins/sdlc-workflow/skills/verify-pr/SKILL.md @@ -25,9 +25,14 @@ the pinned, stable skill actually running. `${CLAUDE_PLUGIN_ROOT}` always points the delivered plugin, so the stable skill reads its own bundled files in every mode — interactive (Claude Code) and sandbox (fullsend). -`${CLAUDE_PLUGIN_ROOT}` expands in this skill's body text and in Bash commands. Inside -a quoted heredoc (`<< 'PYEOF'`), the shell does **not** expand it — read it with -`os.environ["CLAUDE_PLUGIN_ROOT"]` in Python instead. +`${CLAUDE_PLUGIN_ROOT}` is a **textual token** that Claude Code substitutes into this +skill's markdown body (this whole file, including fenced code blocks) before the skill +runs — it is **not** an OS environment variable. Write the literal token wherever you +need the path: prose, Read/Glob paths, and inside a `<< 'PYEOF'` heredoc alike. Do +**not** read it at runtime via `os.environ["CLAUDE_PLUGIN_ROOT"]` or `$CLAUDE_PLUGIN_ROOT` +in a shell — it is not exported to the shell or to subprocesses, so those resolve to +empty / `KeyError`. Substitution happens before execution, so a literal token even +inside a single-quoted heredoc is already the absolute path by the time Python runs. Note: path patterns used to **filter the PR diff** (e.g., detecting changes under `plugins/sdlc-workflow/skills/run-evals/`) stay repo-relative — those describe files @@ -154,12 +159,13 @@ invalid rather than passing and failing deep inside a later step: ```bash python3 - << 'PYEOF' -import json, os, sys +import json, sys from jsonschema import validate, ValidationError INPUT = "/sandbox/workspace/.pre-script/verify-pr-input.json" -# ${CLAUDE_PLUGIN_ROOT} is not expanded inside a quoted heredoc — read it from the env. -SCHEMA = os.path.join(os.environ["CLAUDE_PLUGIN_ROOT"], "schemas/verify-pr-input.schema.json") +# ${CLAUDE_PLUGIN_ROOT} below is a Claude Code body-substitution token: it is already +# the delivered plugin's absolute path by the time this command runs (NOT a shell/env var). +SCHEMA = "${CLAUDE_PLUGIN_ROOT}/schemas/verify-pr-input.schema.json" try: with open(INPUT) as f: instance = json.load(f) From efa39e062881da2d5338509e55a8ba0cd6d68208 Mon Sep 17 00:00:00 2001 From: mrizzi Date: Mon, 7 Sep 2026 19:48:40 +0200 Subject: [PATCH 054/175] chore(verify-pr): re-pin .fullsend base URL to b75e9648, re-lock MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Advance the URL-pinned base commit segment to b75e964853 (the ${CLAUDE_PLUGIN_ROOT} body-token correction). Base file harness/verify-pr.yaml is byte-identical; sha256 (addc8686…) unchanged. Assisted-by: Claude Code Co-Authored-By: Claude Opus 4.8 --- .fullsend/harness/verify-pr.yaml | 2 +- .fullsend/lock.yaml | 50 ++++++++++++++++---------------- 2 files changed, 26 insertions(+), 26 deletions(-) diff --git a/.fullsend/harness/verify-pr.yaml b/.fullsend/harness/verify-pr.yaml index ffb353b35..3bddc991a 100644 --- a/.fullsend/harness/verify-pr.yaml +++ b/.fullsend/harness/verify-pr.yaml @@ -27,7 +27,7 @@ # (base URLs are validated against that allowlist, not inherited). # Re-pin after any base edit: push harness/verify-pr.yaml, set the SHA to the new # commit + the sha256 to `shasum -a 256 harness/verify-pr.yaml`, `fullsend lock`. -base: https://raw.githubusercontent.com/RHEcosystemAppEng/sdlc-plugins/e5c825358fab4ff7114bfadbd6fe07ffa1cb244f/harness/verify-pr.yaml#sha256=addc868659a4391b18fe09cb2a0523d134a9242e35d644a6db36bcce8277cb16 +base: https://raw.githubusercontent.com/RHEcosystemAppEng/sdlc-plugins/b75e96485327249b7dd9177bd3b13a2602311256/harness/verify-pr.yaml#sha256=addc868659a4391b18fe09cb2a0523d134a9242e35d644a6db36bcce8277cb16 host_files: - src: ${GOOGLE_APPLICATION_CREDENTIALS} diff --git a/.fullsend/lock.yaml b/.fullsend/lock.yaml index 468e65112..815f4c3a0 100644 --- a/.fullsend/lock.yaml +++ b/.fullsend/lock.yaml @@ -4,16 +4,16 @@ generated_at: 2026-09-04T13:39:01.038924Z harnesses: verify-pr: source: harness/verify-pr.yaml - sha256: 6257c910d87735834ea40947a2307479099adeaf13f123b39a3e6795b8a47b7a - resolved_at: 2026-09-07T17:10:50.749797Z + sha256: 7bf1b9fc1427e26f12a12a277e4e6102aac8b9d15dff455e374680bfee8b618f + resolved_at: 2026-09-07T17:48:22.319781Z dependencies: - field: base - url: https://raw.githubusercontent.com/RHEcosystemAppEng/sdlc-plugins/e5c825358fab4ff7114bfadbd6fe07ffa1cb244f/harness/verify-pr.yaml + url: https://raw.githubusercontent.com/RHEcosystemAppEng/sdlc-plugins/b75e96485327249b7dd9177bd3b13a2602311256/harness/verify-pr.yaml sha256: addc868659a4391b18fe09cb2a0523d134a9242e35d644a6db36bcce8277cb16 type: file fetched_at: 2026-09-04T13:20:03.650092Z - field: pre_script - url: https://raw.githubusercontent.com/RHEcosystemAppEng/sdlc-plugins/e5c825358fab4ff7114bfadbd6fe07ffa1cb244f/plugins/sdlc-workflow/scripts/pre-verify-pr.sh + url: https://raw.githubusercontent.com/RHEcosystemAppEng/sdlc-plugins/b75e96485327249b7dd9177bd3b13a2602311256/plugins/sdlc-workflow/scripts/pre-verify-pr.sh sha256: 7e6144024f052b513bf50eb9c71199587bbaa8d20499f0be26c563a0da447b8c type: directory files: @@ -39,9 +39,9 @@ harnesses: sha256: d244bf91dfce478b041f8a0651b3241aa191734b5d5ad91899fafd2fbcc54825 - path: validate-output-schema.sh sha256: 56b964145da0b62f438dbc1885780d86e2d589c5340adb93c00e3c8777979cbe - fetched_at: 2026-09-07T17:10:46.703717Z + fetched_at: 2026-09-07T17:48:14.235569Z - field: post_script - url: https://raw.githubusercontent.com/RHEcosystemAppEng/sdlc-plugins/e5c825358fab4ff7114bfadbd6fe07ffa1cb244f/plugins/sdlc-workflow/scripts/post-verify-pr.sh + url: https://raw.githubusercontent.com/RHEcosystemAppEng/sdlc-plugins/b75e96485327249b7dd9177bd3b13a2602311256/plugins/sdlc-workflow/scripts/post-verify-pr.sh sha256: 7e6144024f052b513bf50eb9c71199587bbaa8d20499f0be26c563a0da447b8c type: directory files: @@ -67,9 +67,9 @@ harnesses: sha256: d244bf91dfce478b041f8a0651b3241aa191734b5d5ad91899fafd2fbcc54825 - path: validate-output-schema.sh sha256: 56b964145da0b62f438dbc1885780d86e2d589c5340adb93c00e3c8777979cbe - fetched_at: 2026-09-07T17:10:46.695617Z + fetched_at: 2026-09-07T17:48:14.2256Z - field: validation_loop.script - url: https://raw.githubusercontent.com/RHEcosystemAppEng/sdlc-plugins/e5c825358fab4ff7114bfadbd6fe07ffa1cb244f/plugins/sdlc-workflow/scripts/validate-output-schema.sh + url: https://raw.githubusercontent.com/RHEcosystemAppEng/sdlc-plugins/b75e96485327249b7dd9177bd3b13a2602311256/plugins/sdlc-workflow/scripts/validate-output-schema.sh sha256: 7e6144024f052b513bf50eb9c71199587bbaa8d20499f0be26c563a0da447b8c type: directory files: @@ -95,40 +95,40 @@ harnesses: sha256: d244bf91dfce478b041f8a0651b3241aa191734b5d5ad91899fafd2fbcc54825 - path: validate-output-schema.sh sha256: 56b964145da0b62f438dbc1885780d86e2d589c5340adb93c00e3c8777979cbe - fetched_at: 2026-09-07T17:10:46.695617Z + fetched_at: 2026-09-07T17:48:14.2256Z - field: validation_loop.schema - url: https://raw.githubusercontent.com/RHEcosystemAppEng/sdlc-plugins/e5c825358fab4ff7114bfadbd6fe07ffa1cb244f/plugins/sdlc-workflow/schemas/verify-pr-result.schema.json + url: https://raw.githubusercontent.com/RHEcosystemAppEng/sdlc-plugins/b75e96485327249b7dd9177bd3b13a2602311256/plugins/sdlc-workflow/schemas/verify-pr-result.schema.json sha256: 62c3e5ce0c47e73823b5cab9a577be7b721d8fb31389bc8602dc82ee05526ad6 type: resource - fetched_at: 2026-09-07T17:10:47.026662Z + fetched_at: 2026-09-07T17:48:17.663364Z - field: agent - url: https://raw.githubusercontent.com/RHEcosystemAppEng/sdlc-plugins/e5c825358fab4ff7114bfadbd6fe07ffa1cb244f/plugins/sdlc-workflow/agents/verify-pr.md + url: https://raw.githubusercontent.com/RHEcosystemAppEng/sdlc-plugins/b75e96485327249b7dd9177bd3b13a2602311256/plugins/sdlc-workflow/agents/verify-pr.md sha256: 568b2ab7e32300c84038d500d95f4534ff6c4b20e52eebf2ed5a6c0ae706786a type: resource - fetched_at: 2026-09-07T17:10:47.303981Z + fetched_at: 2026-09-07T17:48:17.914212Z - field: policy - url: https://raw.githubusercontent.com/RHEcosystemAppEng/sdlc-plugins/e5c825358fab4ff7114bfadbd6fe07ffa1cb244f/plugins/sdlc-workflow/policies/verify-pr.yaml + url: https://raw.githubusercontent.com/RHEcosystemAppEng/sdlc-plugins/b75e96485327249b7dd9177bd3b13a2602311256/plugins/sdlc-workflow/policies/verify-pr.yaml sha256: 62d1374be253b0b225921bd8e22e679781076ecdb3a9ea22d5e6a4805753c5a7 type: resource - fetched_at: 2026-09-07T17:10:47.556767Z + fetched_at: 2026-09-07T17:48:18.604634Z - field: host_files[0].src - url: https://raw.githubusercontent.com/RHEcosystemAppEng/sdlc-plugins/e5c825358fab4ff7114bfadbd6fe07ffa1cb244f/plugins/sdlc-workflow/env/gcp-vertex.env + url: https://raw.githubusercontent.com/RHEcosystemAppEng/sdlc-plugins/b75e96485327249b7dd9177bd3b13a2602311256/plugins/sdlc-workflow/env/gcp-vertex.env sha256: 10b2ba695b1d4e65e0964a233b75d6a06853f1eaefc260c56b9eedd989ae7d41 type: resource - fetched_at: 2026-09-07T17:10:47.80023Z + fetched_at: 2026-09-07T17:48:19.123935Z - field: openshell.profiles[0] - url: https://raw.githubusercontent.com/RHEcosystemAppEng/sdlc-plugins/e5c825358fab4ff7114bfadbd6fe07ffa1cb244f/plugins/sdlc-workflow/profiles/fullsend-vertex-ai.yaml + url: https://raw.githubusercontent.com/RHEcosystemAppEng/sdlc-plugins/b75e96485327249b7dd9177bd3b13a2602311256/plugins/sdlc-workflow/profiles/fullsend-vertex-ai.yaml sha256: 76535a148387b1281be2bfb23b6724e4133327189b38c8ce1de2c6dddb8d8341 type: resource - fetched_at: 2026-09-07T17:10:48.047183Z + fetched_at: 2026-09-07T17:48:19.378441Z - field: providers[0] - url: https://raw.githubusercontent.com/RHEcosystemAppEng/sdlc-plugins/e5c825358fab4ff7114bfadbd6fe07ffa1cb244f/plugins/sdlc-workflow/providers/vertex-ai.yaml + url: https://raw.githubusercontent.com/RHEcosystemAppEng/sdlc-plugins/b75e96485327249b7dd9177bd3b13a2602311256/plugins/sdlc-workflow/providers/vertex-ai.yaml sha256: ae5ebe527e5d7b0fa6994346fde3f6ba11b632a3ba78ece96591b5ed85083ec1 type: resource - fetched_at: 2026-09-07T17:10:48.312935Z + fetched_at: 2026-09-07T17:48:19.641898Z - field: plugins[0] - url: https://raw.githubusercontent.com/RHEcosystemAppEng/sdlc-plugins/e5c825358fab4ff7114bfadbd6fe07ffa1cb244f/plugins/sdlc-workflow/plugin.json - sha256: def66f4c52585908a51a91b005d5932ccf79658ca84edac9483bb1da68444314 + url: https://raw.githubusercontent.com/RHEcosystemAppEng/sdlc-plugins/b75e96485327249b7dd9177bd3b13a2602311256/plugins/sdlc-workflow/plugin.json + sha256: 76a33a65cb226a855305c08ab8a323c2b7ca279c5b56072e30a1d4548d2347bd type: directory files: - path: .claude-plugin/plugin.json @@ -224,7 +224,7 @@ harnesses: - path: skills/triage-security/version-impact-analysis.md sha256: a18376094360d435e96436c9fd1bd6a04831123d903e44cf6cf57e600285e3c9 - path: skills/verify-pr/SKILL.md - sha256: 401beec545bf964606405bfcd6bd932f0f8c3a1d3221b9f46f31f41595346f14 + sha256: e36f2aa73f8247fd9711ab738618dd959bedc67fe377d71b737bc970c36046ed - path: skills/verify-pr/correctness.md sha256: 44a4c095038d41434b5e658dfd209970a77c1c0417bc2c00548ce1f85abdbade - path: skills/verify-pr/dispatch-template.md @@ -237,4 +237,4 @@ harnesses: sha256: 128877cf7e156666dbb403805a0112059cc8cc3255e148c6794da29d48b8f5cf - path: skills/verify-pr/style-conventions.md sha256: 5ab920c596615c0d23354cb0e90fe5ebf5f08635ffa3602e8e2d2abd74028a19 - fetched_at: 2026-09-07T17:10:50.747725Z + fetched_at: 2026-09-07T17:48:22.316597Z From 1d59ab571e4f7af1bdd6e4a852f73fcfd2d6f785 Mon Sep 17 00:00:00 2001 From: mrizzi Date: Tue, 8 Sep 2026 11:24:14 +0200 Subject: [PATCH 055/175] fix(verify-pr): make PR-URL extraction format-agnostic extract_pr_url only recognized a plain string or an ADF inlineCard, so a Git Pull Request field populated as an ADF text node with a link mark, or as plain ADF text containing a bare URL, produced a false "no PR URL" skip. Walk the ADF depth-first (document order) collecting a URL from inlineCard attrs.url, a text node's link-mark href, or a bare URL in text; for a plain string, extract an embedded URL when present. +4 tests, 50 pass. Assisted-by: Claude Code Co-Authored-By: Claude Opus 4.8 --- .../sdlc-workflow/scripts/pre_verify_pr.py | 52 ++++++++++++++++--- .../scripts/test_pre_verify_pr.py | 34 ++++++++++++ 2 files changed, 79 insertions(+), 7 deletions(-) diff --git a/plugins/sdlc-workflow/scripts/pre_verify_pr.py b/plugins/sdlc-workflow/scripts/pre_verify_pr.py index 0ee534fdc..afe468b04 100644 --- a/plugins/sdlc-workflow/scripts/pre_verify_pr.py +++ b/plugins/sdlc-workflow/scripts/pre_verify_pr.py @@ -51,21 +51,59 @@ def commit_references_task(commit, task_id): return re.search(r"\b{}\b".format(re.escape(task_id)), text) is not None +_URL_RE = re.compile(r"https?://\S+") + + +def _first_url_in_adf(node): + """Return the first URL found in an ADF node, depth-first in document order. + + Covers every shape Jira uses to populate a URL/textarea custom field: + an ``inlineCard`` (smart link), a ``text`` node carrying a ``link`` mark, + or plain text that merely contains a bare URL. Returns "" when none is found. + """ + if isinstance(node, dict): + if node.get("type") == "inlineCard": + url = node.get("attrs", {}).get("url", "") + if url: + return url + if node.get("type") == "text": + for mark in node.get("marks", []): + if mark.get("type") == "link": + href = mark.get("attrs", {}).get("href", "") + if href: + return href + match = _URL_RE.search(node.get("text", "") or "") + if match: + return match.group(0).rstrip(".,);]") + for child in node.get("content", []): + url = _first_url_in_adf(child) + if url: + return url + elif isinstance(node, list): + for child in node: + url = _first_url_in_adf(child) + if url: + return url + return "" + + def extract_pr_url(issue): - """Extract PR URL from Jira custom field (ADF or string). + """Extract the PR URL from the Jira Git Pull Request custom field. - Returns empty string if the field is missing or has no URL. + The field is format-agnostic: it may be a plain string, an ADF + ``inlineCard`` smart link, an ADF ``text`` node with a ``link`` mark, or + plain ADF text holding a bare URL. All are handled so a linked PR is never + missed because of how the field happened to be populated. Returns "" when + the field is absent or contains no URL. """ field = issue.get("fields", {}).get("customfield_10875") if not field: return "" if isinstance(field, str): - return field + match = _URL_RE.search(field) + return match.group(0).rstrip(".,);]") if match else field.strip() if isinstance(field, dict): - for block in field.get("content", []): - for inline in block.get("content", []): - if inline.get("type") == "inlineCard": - return inline.get("attrs", {}).get("url", "") + return _first_url_in_adf(field) return "" diff --git a/plugins/sdlc-workflow/scripts/test_pre_verify_pr.py b/plugins/sdlc-workflow/scripts/test_pre_verify_pr.py index 6557e90bc..56e65d1ed 100644 --- a/plugins/sdlc-workflow/scripts/test_pre_verify_pr.py +++ b/plugins/sdlc-workflow/scripts/test_pre_verify_pr.py @@ -55,6 +55,40 @@ def test_extract_pr_url_adf_no_inline_card(): assert result == "", f"Expected empty string, got: {result}" +def test_extract_pr_url_adf_text_link_mark(): + """ADF text node carrying a link mark (how a manually-typed link is stored).""" + issue = {"fields": {"customfield_10875": { + "type": "doc", "version": 1, + "content": [{"type": "paragraph", "content": [ + {"type": "text", "text": "PR", "marks": [ + {"type": "link", "attrs": { + "href": "https://github.com/org/repo/pull/13"}} + ]} + ]}] + }}} + result = pre_verify_pr.extract_pr_url(issue) + assert result == "https://github.com/org/repo/pull/13", f"Got: {result}" + + +def test_extract_pr_url_adf_plain_text_url(): + """ADF plain text that merely contains a bare URL (no mark, no card).""" + issue = {"fields": {"customfield_10875": { + "type": "doc", "version": 1, + "content": [{"type": "paragraph", "content": [ + {"type": "text", "text": "see https://github.com/org/repo/pull/99 for details"} + ]}] + }}} + result = pre_verify_pr.extract_pr_url(issue) + assert result == "https://github.com/org/repo/pull/99", f"Got: {result}" + + +def test_extract_pr_url_string_with_surrounding_text(): + """Plain-string field whose value embeds a URL among other text.""" + issue = {"fields": {"customfield_10875": "PR: https://github.com/org/repo/pull/5."}} + result = pre_verify_pr.extract_pr_url(issue) + assert result == "https://github.com/org/repo/pull/5", f"Got: {result}" + + # --- build_github_bundle --- def test_build_github_bundle(): From ff5fb29d9eff1d43982b2342ece46ad221b04ddb Mon Sep 17 00:00:00 2001 From: mrizzi Date: Tue, 8 Sep 2026 11:25:55 +0200 Subject: [PATCH 056/175] chore(verify-pr): re-pin .fullsend base URL to 1d59ab57, re-lock Advance the base URL commit segment to 1d59ab57 (format-agnostic PR-URL extractor) and regenerate lock.yaml so a pinned/offline run resolves the updated pre_verify_pr.py. Base file sha256 (addc8686) unchanged. Assisted-by: Claude Code Co-Authored-By: Claude Opus 4.8 --- .fullsend/harness/verify-pr.yaml | 2 +- .fullsend/lock.yaml | 70 ++++++++++++++++---------------- 2 files changed, 36 insertions(+), 36 deletions(-) diff --git a/.fullsend/harness/verify-pr.yaml b/.fullsend/harness/verify-pr.yaml index 3bddc991a..183f06809 100644 --- a/.fullsend/harness/verify-pr.yaml +++ b/.fullsend/harness/verify-pr.yaml @@ -27,7 +27,7 @@ # (base URLs are validated against that allowlist, not inherited). # Re-pin after any base edit: push harness/verify-pr.yaml, set the SHA to the new # commit + the sha256 to `shasum -a 256 harness/verify-pr.yaml`, `fullsend lock`. -base: https://raw.githubusercontent.com/RHEcosystemAppEng/sdlc-plugins/b75e96485327249b7dd9177bd3b13a2602311256/harness/verify-pr.yaml#sha256=addc868659a4391b18fe09cb2a0523d134a9242e35d644a6db36bcce8277cb16 +base: https://raw.githubusercontent.com/RHEcosystemAppEng/sdlc-plugins/1d59ab571e4f7af1bdd6e4a852f73fcfd2d6f785/harness/verify-pr.yaml#sha256=addc868659a4391b18fe09cb2a0523d134a9242e35d644a6db36bcce8277cb16 host_files: - src: ${GOOGLE_APPLICATION_CREDENTIALS} diff --git a/.fullsend/lock.yaml b/.fullsend/lock.yaml index 815f4c3a0..7caa37cf2 100644 --- a/.fullsend/lock.yaml +++ b/.fullsend/lock.yaml @@ -4,17 +4,17 @@ generated_at: 2026-09-04T13:39:01.038924Z harnesses: verify-pr: source: harness/verify-pr.yaml - sha256: 7bf1b9fc1427e26f12a12a277e4e6102aac8b9d15dff455e374680bfee8b618f - resolved_at: 2026-09-07T17:48:22.319781Z + sha256: 22ec75f79966ac2164b9d2358446cc03397674fb97ac22bcd85cc3621a37bec9 + resolved_at: 2026-09-08T09:25:37.21332Z dependencies: - field: base - url: https://raw.githubusercontent.com/RHEcosystemAppEng/sdlc-plugins/b75e96485327249b7dd9177bd3b13a2602311256/harness/verify-pr.yaml + url: https://raw.githubusercontent.com/RHEcosystemAppEng/sdlc-plugins/1d59ab571e4f7af1bdd6e4a852f73fcfd2d6f785/harness/verify-pr.yaml sha256: addc868659a4391b18fe09cb2a0523d134a9242e35d644a6db36bcce8277cb16 type: file fetched_at: 2026-09-04T13:20:03.650092Z - field: pre_script - url: https://raw.githubusercontent.com/RHEcosystemAppEng/sdlc-plugins/b75e96485327249b7dd9177bd3b13a2602311256/plugins/sdlc-workflow/scripts/pre-verify-pr.sh - sha256: 7e6144024f052b513bf50eb9c71199587bbaa8d20499f0be26c563a0da447b8c + url: https://raw.githubusercontent.com/RHEcosystemAppEng/sdlc-plugins/1d59ab571e4f7af1bdd6e4a852f73fcfd2d6f785/plugins/sdlc-workflow/scripts/pre-verify-pr.sh + sha256: 3bbe5a044203a7884586aca8b124ac86e97570dcddc521a4e13354102e74cb16 type: directory files: - path: execute-actions.py @@ -26,7 +26,7 @@ harnesses: - path: pre-verify-pr.sh sha256: 041bd777d3a8915d0818aa6d24ed188443aba4c199f5ecdd543d8a86677a8c5b - path: pre_verify_pr.py - sha256: 4d9124447aee93ca4e623058d1a0842b39d8fc6f458e569195ae2c73a10dc5cb + sha256: e65e94373bfa992b184b1687a77ee8d925a98744e871e5a428521779a47ed681 - path: strip_extra_properties.py sha256: 9099200014a0a4a9da8795a3346f963efc34c53f4603d340551c46b8d135d122 - path: test_execute_actions.py @@ -36,13 +36,13 @@ harnesses: - path: test_jira_client_cli.py sha256: d36b790c1a310111a0d2f3e81415f9576e80acc5f0bc7d0ff8084ffad001285f - path: test_pre_verify_pr.py - sha256: d244bf91dfce478b041f8a0651b3241aa191734b5d5ad91899fafd2fbcc54825 + sha256: c7ae30979613fa3402de4450c9bb4faf5f011decac79b75a36ba2ef51e4d21aa - path: validate-output-schema.sh sha256: 56b964145da0b62f438dbc1885780d86e2d589c5340adb93c00e3c8777979cbe - fetched_at: 2026-09-07T17:48:14.235569Z + fetched_at: 2026-09-08T09:25:32.480449Z - field: post_script - url: https://raw.githubusercontent.com/RHEcosystemAppEng/sdlc-plugins/b75e96485327249b7dd9177bd3b13a2602311256/plugins/sdlc-workflow/scripts/post-verify-pr.sh - sha256: 7e6144024f052b513bf50eb9c71199587bbaa8d20499f0be26c563a0da447b8c + url: https://raw.githubusercontent.com/RHEcosystemAppEng/sdlc-plugins/1d59ab571e4f7af1bdd6e4a852f73fcfd2d6f785/plugins/sdlc-workflow/scripts/post-verify-pr.sh + sha256: 3bbe5a044203a7884586aca8b124ac86e97570dcddc521a4e13354102e74cb16 type: directory files: - path: execute-actions.py @@ -54,7 +54,7 @@ harnesses: - path: pre-verify-pr.sh sha256: 041bd777d3a8915d0818aa6d24ed188443aba4c199f5ecdd543d8a86677a8c5b - path: pre_verify_pr.py - sha256: 4d9124447aee93ca4e623058d1a0842b39d8fc6f458e569195ae2c73a10dc5cb + sha256: e65e94373bfa992b184b1687a77ee8d925a98744e871e5a428521779a47ed681 - path: strip_extra_properties.py sha256: 9099200014a0a4a9da8795a3346f963efc34c53f4603d340551c46b8d135d122 - path: test_execute_actions.py @@ -64,13 +64,13 @@ harnesses: - path: test_jira_client_cli.py sha256: d36b790c1a310111a0d2f3e81415f9576e80acc5f0bc7d0ff8084ffad001285f - path: test_pre_verify_pr.py - sha256: d244bf91dfce478b041f8a0651b3241aa191734b5d5ad91899fafd2fbcc54825 + sha256: c7ae30979613fa3402de4450c9bb4faf5f011decac79b75a36ba2ef51e4d21aa - path: validate-output-schema.sh sha256: 56b964145da0b62f438dbc1885780d86e2d589c5340adb93c00e3c8777979cbe - fetched_at: 2026-09-07T17:48:14.2256Z + fetched_at: 2026-09-08T09:25:32.471929Z - field: validation_loop.script - url: https://raw.githubusercontent.com/RHEcosystemAppEng/sdlc-plugins/b75e96485327249b7dd9177bd3b13a2602311256/plugins/sdlc-workflow/scripts/validate-output-schema.sh - sha256: 7e6144024f052b513bf50eb9c71199587bbaa8d20499f0be26c563a0da447b8c + url: https://raw.githubusercontent.com/RHEcosystemAppEng/sdlc-plugins/1d59ab571e4f7af1bdd6e4a852f73fcfd2d6f785/plugins/sdlc-workflow/scripts/validate-output-schema.sh + sha256: 3bbe5a044203a7884586aca8b124ac86e97570dcddc521a4e13354102e74cb16 type: directory files: - path: execute-actions.py @@ -82,7 +82,7 @@ harnesses: - path: pre-verify-pr.sh sha256: 041bd777d3a8915d0818aa6d24ed188443aba4c199f5ecdd543d8a86677a8c5b - path: pre_verify_pr.py - sha256: 4d9124447aee93ca4e623058d1a0842b39d8fc6f458e569195ae2c73a10dc5cb + sha256: e65e94373bfa992b184b1687a77ee8d925a98744e871e5a428521779a47ed681 - path: strip_extra_properties.py sha256: 9099200014a0a4a9da8795a3346f963efc34c53f4603d340551c46b8d135d122 - path: test_execute_actions.py @@ -92,43 +92,43 @@ harnesses: - path: test_jira_client_cli.py sha256: d36b790c1a310111a0d2f3e81415f9576e80acc5f0bc7d0ff8084ffad001285f - path: test_pre_verify_pr.py - sha256: d244bf91dfce478b041f8a0651b3241aa191734b5d5ad91899fafd2fbcc54825 + sha256: c7ae30979613fa3402de4450c9bb4faf5f011decac79b75a36ba2ef51e4d21aa - path: validate-output-schema.sh sha256: 56b964145da0b62f438dbc1885780d86e2d589c5340adb93c00e3c8777979cbe - fetched_at: 2026-09-07T17:48:14.2256Z + fetched_at: 2026-09-08T09:25:32.471929Z - field: validation_loop.schema - url: https://raw.githubusercontent.com/RHEcosystemAppEng/sdlc-plugins/b75e96485327249b7dd9177bd3b13a2602311256/plugins/sdlc-workflow/schemas/verify-pr-result.schema.json + url: https://raw.githubusercontent.com/RHEcosystemAppEng/sdlc-plugins/1d59ab571e4f7af1bdd6e4a852f73fcfd2d6f785/plugins/sdlc-workflow/schemas/verify-pr-result.schema.json sha256: 62c3e5ce0c47e73823b5cab9a577be7b721d8fb31389bc8602dc82ee05526ad6 type: resource - fetched_at: 2026-09-07T17:48:17.663364Z + fetched_at: 2026-09-08T09:25:32.813669Z - field: agent - url: https://raw.githubusercontent.com/RHEcosystemAppEng/sdlc-plugins/b75e96485327249b7dd9177bd3b13a2602311256/plugins/sdlc-workflow/agents/verify-pr.md + url: https://raw.githubusercontent.com/RHEcosystemAppEng/sdlc-plugins/1d59ab571e4f7af1bdd6e4a852f73fcfd2d6f785/plugins/sdlc-workflow/agents/verify-pr.md sha256: 568b2ab7e32300c84038d500d95f4534ff6c4b20e52eebf2ed5a6c0ae706786a type: resource - fetched_at: 2026-09-07T17:48:17.914212Z + fetched_at: 2026-09-08T09:25:33.113808Z - field: policy - url: https://raw.githubusercontent.com/RHEcosystemAppEng/sdlc-plugins/b75e96485327249b7dd9177bd3b13a2602311256/plugins/sdlc-workflow/policies/verify-pr.yaml + url: https://raw.githubusercontent.com/RHEcosystemAppEng/sdlc-plugins/1d59ab571e4f7af1bdd6e4a852f73fcfd2d6f785/plugins/sdlc-workflow/policies/verify-pr.yaml sha256: 62d1374be253b0b225921bd8e22e679781076ecdb3a9ea22d5e6a4805753c5a7 type: resource - fetched_at: 2026-09-07T17:48:18.604634Z + fetched_at: 2026-09-08T09:25:33.609926Z - field: host_files[0].src - url: https://raw.githubusercontent.com/RHEcosystemAppEng/sdlc-plugins/b75e96485327249b7dd9177bd3b13a2602311256/plugins/sdlc-workflow/env/gcp-vertex.env + url: https://raw.githubusercontent.com/RHEcosystemAppEng/sdlc-plugins/1d59ab571e4f7af1bdd6e4a852f73fcfd2d6f785/plugins/sdlc-workflow/env/gcp-vertex.env sha256: 10b2ba695b1d4e65e0964a233b75d6a06853f1eaefc260c56b9eedd989ae7d41 type: resource - fetched_at: 2026-09-07T17:48:19.123935Z + fetched_at: 2026-09-08T09:25:33.881539Z - field: openshell.profiles[0] - url: https://raw.githubusercontent.com/RHEcosystemAppEng/sdlc-plugins/b75e96485327249b7dd9177bd3b13a2602311256/plugins/sdlc-workflow/profiles/fullsend-vertex-ai.yaml + url: https://raw.githubusercontent.com/RHEcosystemAppEng/sdlc-plugins/1d59ab571e4f7af1bdd6e4a852f73fcfd2d6f785/plugins/sdlc-workflow/profiles/fullsend-vertex-ai.yaml sha256: 76535a148387b1281be2bfb23b6724e4133327189b38c8ce1de2c6dddb8d8341 type: resource - fetched_at: 2026-09-07T17:48:19.378441Z + fetched_at: 2026-09-08T09:25:34.240717Z - field: providers[0] - url: https://raw.githubusercontent.com/RHEcosystemAppEng/sdlc-plugins/b75e96485327249b7dd9177bd3b13a2602311256/plugins/sdlc-workflow/providers/vertex-ai.yaml + url: https://raw.githubusercontent.com/RHEcosystemAppEng/sdlc-plugins/1d59ab571e4f7af1bdd6e4a852f73fcfd2d6f785/plugins/sdlc-workflow/providers/vertex-ai.yaml sha256: ae5ebe527e5d7b0fa6994346fde3f6ba11b632a3ba78ece96591b5ed85083ec1 type: resource - fetched_at: 2026-09-07T17:48:19.641898Z + fetched_at: 2026-09-08T09:25:34.499549Z - field: plugins[0] - url: https://raw.githubusercontent.com/RHEcosystemAppEng/sdlc-plugins/b75e96485327249b7dd9177bd3b13a2602311256/plugins/sdlc-workflow/plugin.json - sha256: 76a33a65cb226a855305c08ab8a323c2b7ca279c5b56072e30a1d4548d2347bd + url: https://raw.githubusercontent.com/RHEcosystemAppEng/sdlc-plugins/1d59ab571e4f7af1bdd6e4a852f73fcfd2d6f785/plugins/sdlc-workflow/plugin.json + sha256: 69f3e7a90f932ad213c09177a0eb760af2e665d72b972416a828f7bfb712da27 type: directory files: - path: .claude-plugin/plugin.json @@ -158,7 +158,7 @@ harnesses: - path: scripts/pre-verify-pr.sh sha256: 041bd777d3a8915d0818aa6d24ed188443aba4c199f5ecdd543d8a86677a8c5b - path: scripts/pre_verify_pr.py - sha256: 4d9124447aee93ca4e623058d1a0842b39d8fc6f458e569195ae2c73a10dc5cb + sha256: e65e94373bfa992b184b1687a77ee8d925a98744e871e5a428521779a47ed681 - path: scripts/strip_extra_properties.py sha256: 9099200014a0a4a9da8795a3346f963efc34c53f4603d340551c46b8d135d122 - path: scripts/test_execute_actions.py @@ -168,7 +168,7 @@ harnesses: - path: scripts/test_jira_client_cli.py sha256: d36b790c1a310111a0d2f3e81415f9576e80acc5f0bc7d0ff8084ffad001285f - path: scripts/test_pre_verify_pr.py - sha256: d244bf91dfce478b041f8a0651b3241aa191734b5d5ad91899fafd2fbcc54825 + sha256: c7ae30979613fa3402de4450c9bb4faf5f011decac79b75a36ba2ef51e4d21aa - path: scripts/validate-output-schema.sh sha256: 56b964145da0b62f438dbc1885780d86e2d589c5340adb93c00e3c8777979cbe - path: shared/comment-footnote.md @@ -237,4 +237,4 @@ harnesses: sha256: 128877cf7e156666dbb403805a0112059cc8cc3255e148c6794da29d48b8f5cf - path: skills/verify-pr/style-conventions.md sha256: 5ab920c596615c0d23354cb0e90fe5ebf5f08635ffa3602e8e2d2abd74028a19 - fetched_at: 2026-09-07T17:48:22.316597Z + fetched_at: 2026-09-08T09:25:37.209769Z From 87f7e9b59a77ae443fe1a52298a77cf1d606cb65 Mon Sep 17 00:00:00 2001 From: mrizzi Date: Tue, 8 Sep 2026 11:54:06 +0200 Subject: [PATCH 057/175] fix(verify-pr): hyphenate Jira post_comment sticky marker MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit fullsend issues post-comment rejects a Jira --marker containing \*_`[]& because Jira's markdown round-trip escapes those on read-back, breaking sticky-comment re-detection on later runs. POST_COMMENT_STICKY_MARKER carried "post_comment" — the underscore aborted the post-script mid-run (post_comment action to a root-cause task). Rename the suffix to the hyphenated "post-comment"; add a regression guard asserting no Jira-bound marker contains a forbidden char. 37 pass. Assisted-by: Claude Code Co-Authored-By: Claude Opus 4.8 --- plugins/sdlc-workflow/scripts/execute-actions.py | 5 ++++- .../sdlc-workflow/scripts/test_execute_actions.py | 12 ++++++++++++ 2 files changed, 16 insertions(+), 1 deletion(-) diff --git a/plugins/sdlc-workflow/scripts/execute-actions.py b/plugins/sdlc-workflow/scripts/execute-actions.py index 31e87c320..fd6533ee2 100755 --- a/plugins/sdlc-workflow/scripts/execute-actions.py +++ b/plugins/sdlc-workflow/scripts/execute-actions.py @@ -59,7 +59,10 @@ # and a post_report targeting the SAME Jira issue in one run must not share one # marker — otherwise the second post would overwrite the first. Each path keeps # its own stable marker, so per-path re-run idempotency is preserved. -POST_COMMENT_STICKY_MARKER = "" +# The suffix is hyphenated (not "post_comment"): fullsend rejects a Jira +# --marker containing \*_`[]& because Jira's markdown round-trip escapes those +# characters on read-back, which would break marker re-detection on later runs. +POST_COMMENT_STICKY_MARKER = "" # The verify-pr report comment is posted via the native fullsend sticky-comment # CLI (`fullsend issues post-comment --tracker github`), which prepends this diff --git a/plugins/sdlc-workflow/scripts/test_execute_actions.py b/plugins/sdlc-workflow/scripts/test_execute_actions.py index 855789b5e..ba8480965 100644 --- a/plugins/sdlc-workflow/scripts/test_execute_actions.py +++ b/plugins/sdlc-workflow/scripts/test_execute_actions.py @@ -717,6 +717,18 @@ def fake_run(cmd, input=None, text=None, capture_output=None, env=None): assert comment_marker != report_marker, "post_comment and post_report must use distinct markers" +def test_jira_bound_markers_have_no_forbidden_chars(): + """Markers posted to Jira must avoid characters Jira's markdown round-trip + escapes (\\*_`[]&) — fullsend rejects such a --marker because the escaping + would break sticky-comment re-detection on later runs. Guards against a + regression like the underscore in the original "post_comment" marker.""" + forbidden = set("\\*_`[]&") + for marker in (execute_actions.STICKY_COMMENT_MARKER, + execute_actions.POST_COMMENT_STICKY_MARKER): + offending = forbidden & set(marker) + assert not offending, f"{marker!r} contains forbidden char(s) {offending}" + + def test_execute_post_report_posts_github_then_jira(): """execute_post_report posts the report to the GitHub PR then to Jira, both via the native `fullsend issues post-comment` sticky CLI (GitHub first).""" From f6eb1025496e76b1e4536f3df577a990f359ed53 Mon Sep 17 00:00:00 2001 From: mrizzi Date: Tue, 8 Sep 2026 11:54:58 +0200 Subject: [PATCH 058/175] chore(verify-pr): re-pin .fullsend base URL to 87f7e9b5, re-lock Advance the base URL commit segment to 87f7e9b5 (hyphenated Jira post-comment sticky marker) and regenerate lock.yaml so a pinned/offline run resolves the fixed execute-actions.py. Base file sha256 (addc8686) unchanged. Assisted-by: Claude Code Co-Authored-By: Claude Opus 4.8 --- .fullsend/harness/verify-pr.yaml | 2 +- .fullsend/lock.yaml | 70 ++++++++++++++++---------------- 2 files changed, 36 insertions(+), 36 deletions(-) diff --git a/.fullsend/harness/verify-pr.yaml b/.fullsend/harness/verify-pr.yaml index 183f06809..a95df27b7 100644 --- a/.fullsend/harness/verify-pr.yaml +++ b/.fullsend/harness/verify-pr.yaml @@ -27,7 +27,7 @@ # (base URLs are validated against that allowlist, not inherited). # Re-pin after any base edit: push harness/verify-pr.yaml, set the SHA to the new # commit + the sha256 to `shasum -a 256 harness/verify-pr.yaml`, `fullsend lock`. -base: https://raw.githubusercontent.com/RHEcosystemAppEng/sdlc-plugins/1d59ab571e4f7af1bdd6e4a852f73fcfd2d6f785/harness/verify-pr.yaml#sha256=addc868659a4391b18fe09cb2a0523d134a9242e35d644a6db36bcce8277cb16 +base: https://raw.githubusercontent.com/RHEcosystemAppEng/sdlc-plugins/87f7e9b59a77ae443fe1a52298a77cf1d606cb65/harness/verify-pr.yaml#sha256=addc868659a4391b18fe09cb2a0523d134a9242e35d644a6db36bcce8277cb16 host_files: - src: ${GOOGLE_APPLICATION_CREDENTIALS} diff --git a/.fullsend/lock.yaml b/.fullsend/lock.yaml index 7caa37cf2..0bc7e2b89 100644 --- a/.fullsend/lock.yaml +++ b/.fullsend/lock.yaml @@ -4,21 +4,21 @@ generated_at: 2026-09-04T13:39:01.038924Z harnesses: verify-pr: source: harness/verify-pr.yaml - sha256: 22ec75f79966ac2164b9d2358446cc03397674fb97ac22bcd85cc3621a37bec9 - resolved_at: 2026-09-08T09:25:37.21332Z + sha256: 53d0d2dcafa279ae45c7c5050b70319067f0825047fa48a554fb0db0af5c20e1 + resolved_at: 2026-09-08T09:54:34.35545Z dependencies: - field: base - url: https://raw.githubusercontent.com/RHEcosystemAppEng/sdlc-plugins/1d59ab571e4f7af1bdd6e4a852f73fcfd2d6f785/harness/verify-pr.yaml + url: https://raw.githubusercontent.com/RHEcosystemAppEng/sdlc-plugins/87f7e9b59a77ae443fe1a52298a77cf1d606cb65/harness/verify-pr.yaml sha256: addc868659a4391b18fe09cb2a0523d134a9242e35d644a6db36bcce8277cb16 type: file fetched_at: 2026-09-04T13:20:03.650092Z - field: pre_script - url: https://raw.githubusercontent.com/RHEcosystemAppEng/sdlc-plugins/1d59ab571e4f7af1bdd6e4a852f73fcfd2d6f785/plugins/sdlc-workflow/scripts/pre-verify-pr.sh - sha256: 3bbe5a044203a7884586aca8b124ac86e97570dcddc521a4e13354102e74cb16 + url: https://raw.githubusercontent.com/RHEcosystemAppEng/sdlc-plugins/87f7e9b59a77ae443fe1a52298a77cf1d606cb65/plugins/sdlc-workflow/scripts/pre-verify-pr.sh + sha256: 4f5d1bfea092559de8f44469a32cd636949d01c5a029f0782c94187057d11f33 type: directory files: - path: execute-actions.py - sha256: fafbed0c81ed2754366cae424cdd9063b3e01b815c8c94fa80612dd406878b9b + sha256: 6df3bc2598a891868ac3a02f3265b7036c9d4b6d1fdb6394ca619fe89946c520 - path: jira-client.py sha256: f6bf79d2c98a2ae020775e68a538f612953c0464f09bd750c988f0d20fd4b159 - path: post-verify-pr.sh @@ -30,7 +30,7 @@ harnesses: - path: strip_extra_properties.py sha256: 9099200014a0a4a9da8795a3346f963efc34c53f4603d340551c46b8d135d122 - path: test_execute_actions.py - sha256: 014402bd1c773ad9a2a4ab930be95af3230a2b37349b7ba59ed230d76164d978 + sha256: 4170eea78768a516097d212a48b4b459ffb47370c0cc224bc157113d581ab0d4 - path: test_jira_client.py sha256: 2685494b0e725bdf281f61052eb5af7c2cf771613fa3ec6692228377d1bc8518 - path: test_jira_client_cli.py @@ -39,14 +39,14 @@ harnesses: sha256: c7ae30979613fa3402de4450c9bb4faf5f011decac79b75a36ba2ef51e4d21aa - path: validate-output-schema.sh sha256: 56b964145da0b62f438dbc1885780d86e2d589c5340adb93c00e3c8777979cbe - fetched_at: 2026-09-08T09:25:32.480449Z + fetched_at: 2026-09-08T09:54:29.231053Z - field: post_script - url: https://raw.githubusercontent.com/RHEcosystemAppEng/sdlc-plugins/1d59ab571e4f7af1bdd6e4a852f73fcfd2d6f785/plugins/sdlc-workflow/scripts/post-verify-pr.sh - sha256: 3bbe5a044203a7884586aca8b124ac86e97570dcddc521a4e13354102e74cb16 + url: https://raw.githubusercontent.com/RHEcosystemAppEng/sdlc-plugins/87f7e9b59a77ae443fe1a52298a77cf1d606cb65/plugins/sdlc-workflow/scripts/post-verify-pr.sh + sha256: 4f5d1bfea092559de8f44469a32cd636949d01c5a029f0782c94187057d11f33 type: directory files: - path: execute-actions.py - sha256: fafbed0c81ed2754366cae424cdd9063b3e01b815c8c94fa80612dd406878b9b + sha256: 6df3bc2598a891868ac3a02f3265b7036c9d4b6d1fdb6394ca619fe89946c520 - path: jira-client.py sha256: f6bf79d2c98a2ae020775e68a538f612953c0464f09bd750c988f0d20fd4b159 - path: post-verify-pr.sh @@ -58,7 +58,7 @@ harnesses: - path: strip_extra_properties.py sha256: 9099200014a0a4a9da8795a3346f963efc34c53f4603d340551c46b8d135d122 - path: test_execute_actions.py - sha256: 014402bd1c773ad9a2a4ab930be95af3230a2b37349b7ba59ed230d76164d978 + sha256: 4170eea78768a516097d212a48b4b459ffb47370c0cc224bc157113d581ab0d4 - path: test_jira_client.py sha256: 2685494b0e725bdf281f61052eb5af7c2cf771613fa3ec6692228377d1bc8518 - path: test_jira_client_cli.py @@ -67,14 +67,14 @@ harnesses: sha256: c7ae30979613fa3402de4450c9bb4faf5f011decac79b75a36ba2ef51e4d21aa - path: validate-output-schema.sh sha256: 56b964145da0b62f438dbc1885780d86e2d589c5340adb93c00e3c8777979cbe - fetched_at: 2026-09-08T09:25:32.471929Z + fetched_at: 2026-09-08T09:54:29.222533Z - field: validation_loop.script - url: https://raw.githubusercontent.com/RHEcosystemAppEng/sdlc-plugins/1d59ab571e4f7af1bdd6e4a852f73fcfd2d6f785/plugins/sdlc-workflow/scripts/validate-output-schema.sh - sha256: 3bbe5a044203a7884586aca8b124ac86e97570dcddc521a4e13354102e74cb16 + url: https://raw.githubusercontent.com/RHEcosystemAppEng/sdlc-plugins/87f7e9b59a77ae443fe1a52298a77cf1d606cb65/plugins/sdlc-workflow/scripts/validate-output-schema.sh + sha256: 4f5d1bfea092559de8f44469a32cd636949d01c5a029f0782c94187057d11f33 type: directory files: - path: execute-actions.py - sha256: fafbed0c81ed2754366cae424cdd9063b3e01b815c8c94fa80612dd406878b9b + sha256: 6df3bc2598a891868ac3a02f3265b7036c9d4b6d1fdb6394ca619fe89946c520 - path: jira-client.py sha256: f6bf79d2c98a2ae020775e68a538f612953c0464f09bd750c988f0d20fd4b159 - path: post-verify-pr.sh @@ -86,7 +86,7 @@ harnesses: - path: strip_extra_properties.py sha256: 9099200014a0a4a9da8795a3346f963efc34c53f4603d340551c46b8d135d122 - path: test_execute_actions.py - sha256: 014402bd1c773ad9a2a4ab930be95af3230a2b37349b7ba59ed230d76164d978 + sha256: 4170eea78768a516097d212a48b4b459ffb47370c0cc224bc157113d581ab0d4 - path: test_jira_client.py sha256: 2685494b0e725bdf281f61052eb5af7c2cf771613fa3ec6692228377d1bc8518 - path: test_jira_client_cli.py @@ -95,40 +95,40 @@ harnesses: sha256: c7ae30979613fa3402de4450c9bb4faf5f011decac79b75a36ba2ef51e4d21aa - path: validate-output-schema.sh sha256: 56b964145da0b62f438dbc1885780d86e2d589c5340adb93c00e3c8777979cbe - fetched_at: 2026-09-08T09:25:32.471929Z + fetched_at: 2026-09-08T09:54:29.222533Z - field: validation_loop.schema - url: https://raw.githubusercontent.com/RHEcosystemAppEng/sdlc-plugins/1d59ab571e4f7af1bdd6e4a852f73fcfd2d6f785/plugins/sdlc-workflow/schemas/verify-pr-result.schema.json + url: https://raw.githubusercontent.com/RHEcosystemAppEng/sdlc-plugins/87f7e9b59a77ae443fe1a52298a77cf1d606cb65/plugins/sdlc-workflow/schemas/verify-pr-result.schema.json sha256: 62c3e5ce0c47e73823b5cab9a577be7b721d8fb31389bc8602dc82ee05526ad6 type: resource - fetched_at: 2026-09-08T09:25:32.813669Z + fetched_at: 2026-09-08T09:54:29.533021Z - field: agent - url: https://raw.githubusercontent.com/RHEcosystemAppEng/sdlc-plugins/1d59ab571e4f7af1bdd6e4a852f73fcfd2d6f785/plugins/sdlc-workflow/agents/verify-pr.md + url: https://raw.githubusercontent.com/RHEcosystemAppEng/sdlc-plugins/87f7e9b59a77ae443fe1a52298a77cf1d606cb65/plugins/sdlc-workflow/agents/verify-pr.md sha256: 568b2ab7e32300c84038d500d95f4534ff6c4b20e52eebf2ed5a6c0ae706786a type: resource - fetched_at: 2026-09-08T09:25:33.113808Z + fetched_at: 2026-09-08T09:54:29.791935Z - field: policy - url: https://raw.githubusercontent.com/RHEcosystemAppEng/sdlc-plugins/1d59ab571e4f7af1bdd6e4a852f73fcfd2d6f785/plugins/sdlc-workflow/policies/verify-pr.yaml + url: https://raw.githubusercontent.com/RHEcosystemAppEng/sdlc-plugins/87f7e9b59a77ae443fe1a52298a77cf1d606cb65/plugins/sdlc-workflow/policies/verify-pr.yaml sha256: 62d1374be253b0b225921bd8e22e679781076ecdb3a9ea22d5e6a4805753c5a7 type: resource - fetched_at: 2026-09-08T09:25:33.609926Z + fetched_at: 2026-09-08T09:54:30.121781Z - field: host_files[0].src - url: https://raw.githubusercontent.com/RHEcosystemAppEng/sdlc-plugins/1d59ab571e4f7af1bdd6e4a852f73fcfd2d6f785/plugins/sdlc-workflow/env/gcp-vertex.env + url: https://raw.githubusercontent.com/RHEcosystemAppEng/sdlc-plugins/87f7e9b59a77ae443fe1a52298a77cf1d606cb65/plugins/sdlc-workflow/env/gcp-vertex.env sha256: 10b2ba695b1d4e65e0964a233b75d6a06853f1eaefc260c56b9eedd989ae7d41 type: resource - fetched_at: 2026-09-08T09:25:33.881539Z + fetched_at: 2026-09-08T09:54:30.569749Z - field: openshell.profiles[0] - url: https://raw.githubusercontent.com/RHEcosystemAppEng/sdlc-plugins/1d59ab571e4f7af1bdd6e4a852f73fcfd2d6f785/plugins/sdlc-workflow/profiles/fullsend-vertex-ai.yaml + url: https://raw.githubusercontent.com/RHEcosystemAppEng/sdlc-plugins/87f7e9b59a77ae443fe1a52298a77cf1d606cb65/plugins/sdlc-workflow/profiles/fullsend-vertex-ai.yaml sha256: 76535a148387b1281be2bfb23b6724e4133327189b38c8ce1de2c6dddb8d8341 type: resource - fetched_at: 2026-09-08T09:25:34.240717Z + fetched_at: 2026-09-08T09:54:30.893861Z - field: providers[0] - url: https://raw.githubusercontent.com/RHEcosystemAppEng/sdlc-plugins/1d59ab571e4f7af1bdd6e4a852f73fcfd2d6f785/plugins/sdlc-workflow/providers/vertex-ai.yaml + url: https://raw.githubusercontent.com/RHEcosystemAppEng/sdlc-plugins/87f7e9b59a77ae443fe1a52298a77cf1d606cb65/plugins/sdlc-workflow/providers/vertex-ai.yaml sha256: ae5ebe527e5d7b0fa6994346fde3f6ba11b632a3ba78ece96591b5ed85083ec1 type: resource - fetched_at: 2026-09-08T09:25:34.499549Z + fetched_at: 2026-09-08T09:54:31.394996Z - field: plugins[0] - url: https://raw.githubusercontent.com/RHEcosystemAppEng/sdlc-plugins/1d59ab571e4f7af1bdd6e4a852f73fcfd2d6f785/plugins/sdlc-workflow/plugin.json - sha256: 69f3e7a90f932ad213c09177a0eb760af2e665d72b972416a828f7bfb712da27 + url: https://raw.githubusercontent.com/RHEcosystemAppEng/sdlc-plugins/87f7e9b59a77ae443fe1a52298a77cf1d606cb65/plugins/sdlc-workflow/plugin.json + sha256: 68c39a34d12a767023e6d16d8f4c0ec774d3edae2bb8ed52307390ddbb9b75fc type: directory files: - path: .claude-plugin/plugin.json @@ -150,7 +150,7 @@ harnesses: - path: schemas/verify-pr-result.schema.json sha256: 62c3e5ce0c47e73823b5cab9a577be7b721d8fb31389bc8602dc82ee05526ad6 - path: scripts/execute-actions.py - sha256: fafbed0c81ed2754366cae424cdd9063b3e01b815c8c94fa80612dd406878b9b + sha256: 6df3bc2598a891868ac3a02f3265b7036c9d4b6d1fdb6394ca619fe89946c520 - path: scripts/jira-client.py sha256: f6bf79d2c98a2ae020775e68a538f612953c0464f09bd750c988f0d20fd4b159 - path: scripts/post-verify-pr.sh @@ -162,7 +162,7 @@ harnesses: - path: scripts/strip_extra_properties.py sha256: 9099200014a0a4a9da8795a3346f963efc34c53f4603d340551c46b8d135d122 - path: scripts/test_execute_actions.py - sha256: 014402bd1c773ad9a2a4ab930be95af3230a2b37349b7ba59ed230d76164d978 + sha256: 4170eea78768a516097d212a48b4b459ffb47370c0cc224bc157113d581ab0d4 - path: scripts/test_jira_client.py sha256: 2685494b0e725bdf281f61052eb5af7c2cf771613fa3ec6692228377d1bc8518 - path: scripts/test_jira_client_cli.py @@ -237,4 +237,4 @@ harnesses: sha256: 128877cf7e156666dbb403805a0112059cc8cc3255e148c6794da29d48b8f5cf - path: skills/verify-pr/style-conventions.md sha256: 5ab920c596615c0d23354cb0e90fe5ebf5f08635ffa3602e8e2d2abd74028a19 - fetched_at: 2026-09-08T09:25:37.209769Z + fetched_at: 2026-09-08T09:54:34.348138Z From 430a29c86bfbc601370a32251a555c9d17af6bcb Mon Sep 17 00:00:00 2001 From: mrizzi Date: Tue, 8 Sep 2026 12:02:20 +0200 Subject: [PATCH 059/175] docs(verify-pr): migrate Step 9 report-posting to native fullsend CLI MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Step 9's "Post to GitHub PR" and "Sandbox Mode Output" sections still described the removed gh-CLI find-then-update flow (`gh api --paginate --slurp`, `gh api -X PATCH`, `gh pr comment`, `_find_report_comment_id`) and instructed the agent to embed the commit marker in the report body. The runtime migrated to the native `fullsend issues post-comment --tracker github` sticky CLI, which prepends the commit-scoped marker via `--marker` and edits the marked comment in place on re-runs — so a marker embedded in `report_md` produces a duplicate marker line (the sourcery-ai finding on PR #291). Rewrite Step 9 to document the native CLI, state that the agent must NOT embed the marker in the report body (the runner supplies it via `--marker`), and point the Sandbox Mode Output at the native path. Add eval assertions (evals 1, 3) covering the no-embedded-marker contract. Implements TC-6102 Assisted-by: Claude Code Co-Authored-By: Claude Opus 4.8 --- evals/verify-pr/evals.json | 4 +- .../sdlc-workflow/skills/verify-pr/SKILL.md | 46 +++++++++++-------- 2 files changed, 30 insertions(+), 20 deletions(-) diff --git a/evals/verify-pr/evals.json b/evals/verify-pr/evals.json index 85367e887..afb561f3f 100644 --- a/evals/verify-pr/evals.json +++ b/evals/verify-pr/evals.json @@ -18,7 +18,8 @@ "The report includes detailed findings with specific evidence for each domain — file-by-file scope comparison (Intent Alignment), line-level pattern scanning results (Security), per-criterion code-level verification (Correctness), and test quality assessment (Style/Conventions) — not just pass/fail verdicts in the summary table", "Eval Quality is N/A because no eval result reviews exist in the PR — no reviews match the eval result detection criteria, so Eval Quality does not affect the Test Quality combination", "No eval failure sub-tasks are created because Eval Quality is N/A — eval failure sub-tasks are only created when Eval Quality is WARN (at least one eval assertion failed)", - "Review Feedback and Root-Cause Investigation verdicts are determined by the orchestrator independently from domain analysis — Review Feedback is N/A because no review comments exist, Root-Cause Investigation is N/A because no sub-tasks were created" + "Review Feedback and Root-Cause Investigation verdicts are determined by the orchestrator independently from domain analysis — Review Feedback is N/A because no review comments exist, Root-Cause Investigation is N/A because no sub-tasks were created", + "The report body contains NO sticky-comment marker line — it does not include an HTML comment of the form ``; the report body is only the header, table, summary, and footnote, because the runner supplies the commit-scoped marker separately via the native `fullsend issues post-comment --tracker github --marker` CLI (Step 9), so the agent must not embed the marker in the report body" ] }, { @@ -58,6 +59,7 @@ "The report does NOT auto-merge the PR (constraint 1.13)", "The report contains a Test Change Classification row with ADDITIVE — only new test files were added (tests/api/sbom_delete.rs is a new file)", "The verification report header for TC-9103 carries a commit-scoped identifier — `## Verification Report for TC-9103 (commit )`, or `(commit unknown)` when the eval sandbox provides no HEAD SHA — reflecting Step 9's per-commit report convention: each report is scoped to the commit it verifies, so a re-run on the same commit refreshes that commit's existing report comment while a later commit gets a new one (per-commit history, not a new comment on every run)", + "The report body carries the commit-scoped identity in its header only and contains NO embedded marker line — it does not include an HTML comment of the form ``; per Step 9 the runner posts the report via the native `fullsend issues post-comment --tracker github` sticky CLI and supplies the commit-scoped marker with `--marker` (the CLI prepends it and edits the marked comment in place on re-runs), so the agent must not write the marker into the report body", "Convention upgrade eligibility is evaluated for review comment 30002 (index suggestion) — the review classification output (review-30002.md) or the report's Style/Conventions analysis explains whether the suggestion matches a documented or demonstrated project convention", "Review comment 30002 (index suggestion) does NOT result in a sub-task — the suggestion classification is correct (suggestive language, no directive) and no project convention in the fixture data backs an upgrade from suggestion to code change request", "Eval Quality is N/A because no eval result reviews exist in the PR — the 3-criteria detection (author github-actions[bot], marker ## Eval Results, footer sdlc-workflow/run-evals) found no matches, so Eval Quality does not affect the Test Quality combination", diff --git a/plugins/sdlc-workflow/skills/verify-pr/SKILL.md b/plugins/sdlc-workflow/skills/verify-pr/SKILL.md index 4dfeeeede..1f2320b02 100644 --- a/plugins/sdlc-workflow/skills/verify-pr/SKILL.md +++ b/plugins/sdlc-workflow/skills/verify-pr/SKILL.md @@ -1280,13 +1280,18 @@ updates the existing report comment in place, while a **later commit** gets a fr comment. This preserves a **per-commit** verification history — one report comment per commit, refreshed on re-runs — rather than a new comment on every run. -To make this work, the report body embeds an invisible commit-scoped marker (the -mechanism GitHub lacks a native sticky-comment for): +Idempotency uses a commit-scoped marker — an invisible HTML comment (GitHub has +no native sticky-comment) whose identity is the verified commit: ``` ``` +The marker is supplied to the native sticky CLI via `--marker`, which prepends it +to the comment. **Do NOT embed this marker in the report body** — the agent writes +only the report text, and the runner supplies the marker separately. Embedding it +would leave a duplicate marker line the CLI does not strip. + Update the report header from Step 8 to include the commit SHA — the `(commit )` makes which commit each comment verifies legible in the PR timeline: @@ -1305,23 +1310,24 @@ Append a markdown footnote at the end of the report body, separated by a horizon Read the plugin version from `${CLAUDE_PLUGIN_ROOT}/.claude-plugin/plugin.json` and substitute `{version}` before posting. -Build the comment body as the report (with the commit-scoped header and footnote) -followed by the commit marker, then post it via a find-then-update-or-create path -(as implemented by `_find_report_comment_id` + `execute_post_report` in +The comment body is the report — commit-scoped header + table + summary + footnote, +with **no** marker line — posted via the native fullsend sticky-comment CLI (as +implemented by `post_github_comment_native` ← `execute_post_report` in `scripts/execute-actions.py`): ``` -# Find an existing report comment for this commit, matched by the marker above. -# --slurp is required with --paginate so multi-page (>30) comment output is valid JSON. -gh api repos//issues//comments --paginate --slurp - -# If a comment carrying this commit's marker exists → update it in place: -gh api repos//issues/comments/ -X PATCH -f body="" - -# Otherwise (first report for this commit) → create a new comment: -gh pr comment --body "" -R +fullsend issues post-comment --tracker github \ + --project --number \ + --marker "" \ + --result - # report body (no marker) on stdin ``` +The CLI prepends the `--marker` as a hidden HTML comment and, on re-runs, finds the +comment carrying this commit's marker and edits it in place (collapsing the prior +body into a `
` block) — so a retry on the same commit never duplicates, +while a later commit gets a fresh comment. GitHub treats a PR as an issue for +comments, so `--tracker github --number ` targets the PR. + ### Post to Jira Add a comment to the Jira task with the verification report: @@ -1336,10 +1342,12 @@ The report is informational — a human reviewer decides whether to merge. ### Sandbox Mode Output **Sandbox mode only:** Instead of posting to GitHub and Jira directly, populate the -`report` object and append a single `post_report` action. The runner's `post_script` -posts the GitHub PR comment (from `report_md`, applying the commit-scoped -find-then-update-or-create path above) and the Jira comment (from `report_adf`) after -the sandbox exits. +`report` object and append a single `post_report` action. After the sandbox exits, +the runner's `post_script` posts the GitHub PR comment (from `report_md`) and the +Jira comment (from `report_adf`) via the native `fullsend issues post-comment` +sticky CLI — supplying the commit-scoped marker with `--marker` (GitHub) so a re-run +updates the same per-commit comment in place. `report_md` must contain **no** marker +line; the runner supplies the marker. Populate `report` (all fields required by `${CLAUDE_PLUGIN_ROOT}/schemas/verify-pr-result.schema.json`): @@ -1352,7 +1360,7 @@ Populate `report` (all fields required by "commit_sha": "", "overall": "PASS|WARN|FAIL", "table_md": "", - "report_md": "", + "report_md": "", "report_adf": , "plugin_version": "" } From 7e6d4619fe9df239d8ca6d7a5006b1357491c663 Mon Sep 17 00:00:00 2001 From: mrizzi Date: Tue, 8 Sep 2026 15:11:40 +0200 Subject: [PATCH 060/175] chore(verify-pr): re-pin .fullsend base URL to 430a29c8, re-lock The Step 9 SKILL.md report-posting migration (430a29c8) edited pinned plugin content without advancing the .fullsend pin, so fullsend kept serving the stale pre-fix SKILL.md (locked at 87f7e9b5). Re-pin the base URL to 430a29c8 and re-lock so the plugins[0] dir dep resolves the new skills/verify-pr/SKILL.md (sha256 94a6c189). Base file harness/verify-pr.yaml is unchanged (sha256 addc8686); only the commit segment advances. Fixes the code-review finding on PR #291 (comment 5585442940). Assisted-by: Claude Code Co-Authored-By: Claude Opus 4.8 --- .fullsend/harness/verify-pr.yaml | 2 +- .fullsend/lock.yaml | 52 ++++++++++++++++---------------- 2 files changed, 27 insertions(+), 27 deletions(-) diff --git a/.fullsend/harness/verify-pr.yaml b/.fullsend/harness/verify-pr.yaml index a95df27b7..750b04414 100644 --- a/.fullsend/harness/verify-pr.yaml +++ b/.fullsend/harness/verify-pr.yaml @@ -27,7 +27,7 @@ # (base URLs are validated against that allowlist, not inherited). # Re-pin after any base edit: push harness/verify-pr.yaml, set the SHA to the new # commit + the sha256 to `shasum -a 256 harness/verify-pr.yaml`, `fullsend lock`. -base: https://raw.githubusercontent.com/RHEcosystemAppEng/sdlc-plugins/87f7e9b59a77ae443fe1a52298a77cf1d606cb65/harness/verify-pr.yaml#sha256=addc868659a4391b18fe09cb2a0523d134a9242e35d644a6db36bcce8277cb16 +base: https://raw.githubusercontent.com/RHEcosystemAppEng/sdlc-plugins/430a29c86bfbc601370a32251a555c9d17af6bcb/harness/verify-pr.yaml#sha256=addc868659a4391b18fe09cb2a0523d134a9242e35d644a6db36bcce8277cb16 host_files: - src: ${GOOGLE_APPLICATION_CREDENTIALS} diff --git a/.fullsend/lock.yaml b/.fullsend/lock.yaml index 0bc7e2b89..a69aafcd6 100644 --- a/.fullsend/lock.yaml +++ b/.fullsend/lock.yaml @@ -4,16 +4,16 @@ generated_at: 2026-09-04T13:39:01.038924Z harnesses: verify-pr: source: harness/verify-pr.yaml - sha256: 53d0d2dcafa279ae45c7c5050b70319067f0825047fa48a554fb0db0af5c20e1 - resolved_at: 2026-09-08T09:54:34.35545Z + sha256: 1d5889ef6f71dae2523db790bd848af6a8353ad739499a4ae583b264bae1c8d2 + resolved_at: 2026-09-08T13:11:05.215335Z dependencies: - field: base - url: https://raw.githubusercontent.com/RHEcosystemAppEng/sdlc-plugins/87f7e9b59a77ae443fe1a52298a77cf1d606cb65/harness/verify-pr.yaml + url: https://raw.githubusercontent.com/RHEcosystemAppEng/sdlc-plugins/430a29c86bfbc601370a32251a555c9d17af6bcb/harness/verify-pr.yaml sha256: addc868659a4391b18fe09cb2a0523d134a9242e35d644a6db36bcce8277cb16 type: file - fetched_at: 2026-09-04T13:20:03.650092Z + fetched_at: 2026-09-08T13:10:57.161392Z - field: pre_script - url: https://raw.githubusercontent.com/RHEcosystemAppEng/sdlc-plugins/87f7e9b59a77ae443fe1a52298a77cf1d606cb65/plugins/sdlc-workflow/scripts/pre-verify-pr.sh + url: https://raw.githubusercontent.com/RHEcosystemAppEng/sdlc-plugins/430a29c86bfbc601370a32251a555c9d17af6bcb/plugins/sdlc-workflow/scripts/pre-verify-pr.sh sha256: 4f5d1bfea092559de8f44469a32cd636949d01c5a029f0782c94187057d11f33 type: directory files: @@ -39,9 +39,9 @@ harnesses: sha256: c7ae30979613fa3402de4450c9bb4faf5f011decac79b75a36ba2ef51e4d21aa - path: validate-output-schema.sh sha256: 56b964145da0b62f438dbc1885780d86e2d589c5340adb93c00e3c8777979cbe - fetched_at: 2026-09-08T09:54:29.231053Z + fetched_at: 2026-09-08T13:11:00.144348Z - field: post_script - url: https://raw.githubusercontent.com/RHEcosystemAppEng/sdlc-plugins/87f7e9b59a77ae443fe1a52298a77cf1d606cb65/plugins/sdlc-workflow/scripts/post-verify-pr.sh + url: https://raw.githubusercontent.com/RHEcosystemAppEng/sdlc-plugins/430a29c86bfbc601370a32251a555c9d17af6bcb/plugins/sdlc-workflow/scripts/post-verify-pr.sh sha256: 4f5d1bfea092559de8f44469a32cd636949d01c5a029f0782c94187057d11f33 type: directory files: @@ -67,9 +67,9 @@ harnesses: sha256: c7ae30979613fa3402de4450c9bb4faf5f011decac79b75a36ba2ef51e4d21aa - path: validate-output-schema.sh sha256: 56b964145da0b62f438dbc1885780d86e2d589c5340adb93c00e3c8777979cbe - fetched_at: 2026-09-08T09:54:29.222533Z + fetched_at: 2026-09-08T13:11:00.138536Z - field: validation_loop.script - url: https://raw.githubusercontent.com/RHEcosystemAppEng/sdlc-plugins/87f7e9b59a77ae443fe1a52298a77cf1d606cb65/plugins/sdlc-workflow/scripts/validate-output-schema.sh + url: https://raw.githubusercontent.com/RHEcosystemAppEng/sdlc-plugins/430a29c86bfbc601370a32251a555c9d17af6bcb/plugins/sdlc-workflow/scripts/validate-output-schema.sh sha256: 4f5d1bfea092559de8f44469a32cd636949d01c5a029f0782c94187057d11f33 type: directory files: @@ -95,40 +95,40 @@ harnesses: sha256: c7ae30979613fa3402de4450c9bb4faf5f011decac79b75a36ba2ef51e4d21aa - path: validate-output-schema.sh sha256: 56b964145da0b62f438dbc1885780d86e2d589c5340adb93c00e3c8777979cbe - fetched_at: 2026-09-08T09:54:29.222533Z + fetched_at: 2026-09-08T13:11:00.138536Z - field: validation_loop.schema - url: https://raw.githubusercontent.com/RHEcosystemAppEng/sdlc-plugins/87f7e9b59a77ae443fe1a52298a77cf1d606cb65/plugins/sdlc-workflow/schemas/verify-pr-result.schema.json + url: https://raw.githubusercontent.com/RHEcosystemAppEng/sdlc-plugins/430a29c86bfbc601370a32251a555c9d17af6bcb/plugins/sdlc-workflow/schemas/verify-pr-result.schema.json sha256: 62c3e5ce0c47e73823b5cab9a577be7b721d8fb31389bc8602dc82ee05526ad6 type: resource - fetched_at: 2026-09-08T09:54:29.533021Z + fetched_at: 2026-09-08T13:11:00.630765Z - field: agent - url: https://raw.githubusercontent.com/RHEcosystemAppEng/sdlc-plugins/87f7e9b59a77ae443fe1a52298a77cf1d606cb65/plugins/sdlc-workflow/agents/verify-pr.md + url: https://raw.githubusercontent.com/RHEcosystemAppEng/sdlc-plugins/430a29c86bfbc601370a32251a555c9d17af6bcb/plugins/sdlc-workflow/agents/verify-pr.md sha256: 568b2ab7e32300c84038d500d95f4534ff6c4b20e52eebf2ed5a6c0ae706786a type: resource - fetched_at: 2026-09-08T09:54:29.791935Z + fetched_at: 2026-09-08T13:11:00.883248Z - field: policy - url: https://raw.githubusercontent.com/RHEcosystemAppEng/sdlc-plugins/87f7e9b59a77ae443fe1a52298a77cf1d606cb65/plugins/sdlc-workflow/policies/verify-pr.yaml + url: https://raw.githubusercontent.com/RHEcosystemAppEng/sdlc-plugins/430a29c86bfbc601370a32251a555c9d17af6bcb/plugins/sdlc-workflow/policies/verify-pr.yaml sha256: 62d1374be253b0b225921bd8e22e679781076ecdb3a9ea22d5e6a4805753c5a7 type: resource - fetched_at: 2026-09-08T09:54:30.121781Z + fetched_at: 2026-09-08T13:11:01.150243Z - field: host_files[0].src - url: https://raw.githubusercontent.com/RHEcosystemAppEng/sdlc-plugins/87f7e9b59a77ae443fe1a52298a77cf1d606cb65/plugins/sdlc-workflow/env/gcp-vertex.env + url: https://raw.githubusercontent.com/RHEcosystemAppEng/sdlc-plugins/430a29c86bfbc601370a32251a555c9d17af6bcb/plugins/sdlc-workflow/env/gcp-vertex.env sha256: 10b2ba695b1d4e65e0964a233b75d6a06853f1eaefc260c56b9eedd989ae7d41 type: resource - fetched_at: 2026-09-08T09:54:30.569749Z + fetched_at: 2026-09-08T13:11:01.447018Z - field: openshell.profiles[0] - url: https://raw.githubusercontent.com/RHEcosystemAppEng/sdlc-plugins/87f7e9b59a77ae443fe1a52298a77cf1d606cb65/plugins/sdlc-workflow/profiles/fullsend-vertex-ai.yaml + url: https://raw.githubusercontent.com/RHEcosystemAppEng/sdlc-plugins/430a29c86bfbc601370a32251a555c9d17af6bcb/plugins/sdlc-workflow/profiles/fullsend-vertex-ai.yaml sha256: 76535a148387b1281be2bfb23b6724e4133327189b38c8ce1de2c6dddb8d8341 type: resource - fetched_at: 2026-09-08T09:54:30.893861Z + fetched_at: 2026-09-08T13:11:01.712662Z - field: providers[0] - url: https://raw.githubusercontent.com/RHEcosystemAppEng/sdlc-plugins/87f7e9b59a77ae443fe1a52298a77cf1d606cb65/plugins/sdlc-workflow/providers/vertex-ai.yaml + url: https://raw.githubusercontent.com/RHEcosystemAppEng/sdlc-plugins/430a29c86bfbc601370a32251a555c9d17af6bcb/plugins/sdlc-workflow/providers/vertex-ai.yaml sha256: ae5ebe527e5d7b0fa6994346fde3f6ba11b632a3ba78ece96591b5ed85083ec1 type: resource - fetched_at: 2026-09-08T09:54:31.394996Z + fetched_at: 2026-09-08T13:11:02.357982Z - field: plugins[0] - url: https://raw.githubusercontent.com/RHEcosystemAppEng/sdlc-plugins/87f7e9b59a77ae443fe1a52298a77cf1d606cb65/plugins/sdlc-workflow/plugin.json - sha256: 68c39a34d12a767023e6d16d8f4c0ec774d3edae2bb8ed52307390ddbb9b75fc + url: https://raw.githubusercontent.com/RHEcosystemAppEng/sdlc-plugins/430a29c86bfbc601370a32251a555c9d17af6bcb/plugins/sdlc-workflow/plugin.json + sha256: 5234e62de3143c5b549f90111888f4d34d290daf7f37c61f40646e269a9daa33 type: directory files: - path: .claude-plugin/plugin.json @@ -224,7 +224,7 @@ harnesses: - path: skills/triage-security/version-impact-analysis.md sha256: a18376094360d435e96436c9fd1bd6a04831123d903e44cf6cf57e600285e3c9 - path: skills/verify-pr/SKILL.md - sha256: e36f2aa73f8247fd9711ab738618dd959bedc67fe377d71b737bc970c36046ed + sha256: 94a6c1898b31832007d929d131fdebb5b542bcb382a428f12a9b7b947ee2708e - path: skills/verify-pr/correctness.md sha256: 44a4c095038d41434b5e658dfd209970a77c1c0417bc2c00548ce1f85abdbade - path: skills/verify-pr/dispatch-template.md @@ -237,4 +237,4 @@ harnesses: sha256: 128877cf7e156666dbb403805a0112059cc8cc3255e148c6794da29d48b8f5cf - path: skills/verify-pr/style-conventions.md sha256: 5ab920c596615c0d23354cb0e90fe5ebf5f08635ffa3602e8e2d2abd74028a19 - fetched_at: 2026-09-08T09:54:34.348138Z + fetched_at: 2026-09-08T13:11:05.210442Z From 1712e2d7f35bbbba7fe2358092f292e95d89b55d Mon Sep 17 00:00:00 2001 From: mrizzi Date: Tue, 8 Sep 2026 16:53:34 +0200 Subject: [PATCH 061/175] docs(verify-pr): rewrite fullsend.md for native v0.37.0 path Rewrite the fullsend integration guide for the v0.37.0 native path: standalone root-level harness, stock digest-pinned fullsend-code image, single pinned-URL registration with in-place plugin (no duplication), and native fullsend CLI for tracker I/O. - Credential tiers: Jira tier 1, GitHub tier 1 (both prefetched host-side), Vertex tier 4 (only in-sandbox credential); sandbox egress is *.googleapis.com only. - Release/update loop for the self-hosted .fullsend/ model: edit -> commit -> push -> re-pin base SHA/hash -> fullsend lock -> commit .fullsend/. - Documents merge-commit-only (no squash/rebase), re-pin/re-lock-after-edit for pinned plugin files, the fullsend-run-path-only scope note, and the stale-cache troubleshooting step. - Adopter path: fullsend agent add , run, agent update; main is the release channel. - Corrects the release procedure: fullsend agent update does not apply to this repo's local-source registration (verified: errors 'local path - nothing to update'); re-pin is done by editing the base: SHA/hash and re-locking. Removes the resolved #2113 marketplace-baking rationale, custom Dockerfile/bootstrap sections, and the invalid Jira/github-ro provider rows. Implements TC-5814 Assisted-by: Claude Code Co-Authored-By: Claude Opus 4.8 --- fullsend.md | 266 ++++++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 266 insertions(+) create mode 100644 fullsend.md diff --git a/fullsend.md b/fullsend.md new file mode 100644 index 000000000..7b8e5f051 --- /dev/null +++ b/fullsend.md @@ -0,0 +1,266 @@ +# Running sdlc-workflow skills with fullsend + +Run sdlc-workflow skills inside secure sandboxes via +[fullsend](https://github.com/fullsend-ai/fullsend). The agent runs in an +isolated container with least-privilege network and filesystem policies, while +all issue-tracker and GitHub writes happen on the trusted runner — never inside +the sandbox. + +This guide documents the **native v0.37.0 path**: a standalone root-level +harness, the stock digest-pinned `fullsend-code` image, a single pinned-URL +registration, and the plugin referenced in place with zero duplication. The only +skill wired up today is `verify-pr`. + +## How it works + +The setup has three moving parts, all in this repo: + +- **`harness/verify-pr.yaml`** (repo root) — a **standalone harness** with no + base composition. Placed at the repo root so that, when fullsend is invoked + with `--fullsend-dir` = repo root, its relative children resolve against the + repo root: `plugins/sdlc-workflow` is delivered **in place as a whole plugin** + and its sibling `shared/` resources resolve intact. It pins the stock + `fullsend-code` image by digest and declares the policy, provider, profile, + env mount, pre/post scripts, and validation loop. +- **`.fullsend/config.yaml`** — registers a single agent, `verify-pr`, whose + source is the **local composing child** `.fullsend/harness/verify-pr.yaml`. +- **`.fullsend/harness/verify-pr.yaml`** — the local composing child. It pins + the root harness by **raw URL at a commit SHA plus a `#sha256=`** on its + `base:` line, and adds the runner-local *absolute* mounts (GCP credential, + OIDC token, pre-script output) that fullsend v0.37.0 refuses to inherit from a + URL-sourced base. `host_files` are concatenated base + child (dedup by dest, + child wins). + +Because the base is pinned to one commit SHA, **every relative child of the +harness** — the pre/post scripts, schemas, agent prompt, policy, provider, +profile, and the whole `plugins/sdlc-workflow` directory — resolves at that same +SHA. The single `#sha256=` on the `base:` line verifies the base file itself; +`.fullsend/lock.yaml` then freezes the content hash of every transitively +resolved child. One pinned URL, one integrity anchor, everything else covered +transitively. + +There is **no custom image and no marketplace baking**. fullsend fabricates the +Claude Code marketplace cache from the in-place `plugins:` entry at runtime, so +the Claude Code marketplace structure is unchanged and the same plugin files +serve interactive Claude Code users and fullsend runs with zero duplication. + +### Split-trust I/O + +The Jira and GitHub tokens live **only on the runner**. The pre_script prefetches +everything the agent needs (the Jira issue and a GitHub read bundle) and the +post_script performs every write after the sandbox is destroyed. The sandbox +itself receives read-only context only — `JIRA_ISSUE_ID` and `JIRA_BASE_URL` +(display links) — plus the prefetched read bundle mounted read-only. The **only** +credential that ever enters the sandbox is the Vertex AI service-account key, +because model inference runs in-sandbox (see below). + +## Credential delivery and tiers + +Credentials use the highest isolation tier possible. Two of the three services +never place a credential in the sandbox at all. + +| Service | Tier | Credential in sandbox? | How it is delivered | +|---|---|---|---| +| **Jira** | 1 | No | The pre_script prefetches the issue on the runner; the post_script posts comments with `fullsend issues post-comment --tracker jira`. The token stays in runner env only. | +| **GitHub** | 1 | No | The pre_script prefetches the read bundle (`gh pr diff` / `gh pr view` → diff, diffstat, commits) and checks out the PR head on the runner; the post_script writes via `fullsend issues post-comment --tracker github` and `gh` (PR reviews/replies via `gh api`). The token stays in runner env only. | +| **Vertex AI** | 4 (fullsend-mandated) | **Yes** | The in-sandbox runtime does the model inference and reads `GOOGLE_APPLICATION_CREDENTIALS` from a file (`/tmp/.gcp-credentials.json`). Vertex auth requires local JWT signing, so tier 4 (file on the sandbox filesystem) is unavoidable. This is the one credential set in the sandbox. | + +Because Vertex is the only in-sandbox credential, the sandbox's **only network +egress is `*.googleapis.com`** (declared in +`plugins/sdlc-workflow/profiles/fullsend-vertex-ai.yaml`). There is no +atlassian, github, or anthropic egress from the sandbox — those are all handled +on the runner. + +> GitHub is a **tier-1** service — it needs no OpenShell provider. There is no +> Jira provider and no `github-ro` provider. The only provider is +> `plugins/sdlc-workflow/providers/vertex-ai.yaml`, which selects the Vertex +> egress profile (its credential still arrives via the host-file mount, not a +> proxied placeholder). + +## Prerequisites + +- [fullsend](https://github.com/fullsend-ai/fullsend) v0.37.0 CLI installed. +- An OpenShell gateway running — fullsend uses OpenShell as its sandbox runtime. +- GCP credentials for Vertex AI — a service-account key JSON (local) or a WIF + external-account config (CI). Referenced by `GOOGLE_APPLICATION_CREDENTIALS`. +- A Jira API token and a GitHub token — used by the pre/post scripts on the + runner only. +- The Python `jsonschema` package on the runner — the `validation_loop` + validates agent output against the JSON schema before the post_script runs + (`pip install jsonschema`). + +## Running verify-pr + +The command is **identical locally and in CI** — same harness, same registration, +only the runtime environment differs (a local SA key vs. a CI WIF config; a CI +run additionally supplies an OIDC token file): + +```bash +fullsend run verify-pr \ + --fullsend-dir .fullsend \ + --target-repo /tmp/my-repo-clone \ + --env-file secrets.env \ + --env-file <(echo "JIRA_ISSUE_ID=TC-1234") +``` + +`--target-repo` must be a **disposable clone**, not your working directory — +fullsend deletes and re-creates it after each run. + +A minimal `secrets.env` (never commit it): + +```bash +# Vertex AI (tier 4 — the only credential that enters the sandbox) +ANTHROPIC_VERTEX_PROJECT_ID=my-project +GOOGLE_CLOUD_PROJECT=my-project +CLOUD_ML_REGION=global +GOOGLE_APPLICATION_CREDENTIALS=/path/to/sa-key.json +# Jira (tier 1 — runner only) +JIRA_SERVER_URL=https://myorg.atlassian.net +JIRA_EMAIL=me@example.com +JIRA_API_TOKEN=my-jira-token +JIRA_PROJECT_KEY=TC +# GitHub (tier 1 — runner only) +GH_TOKEN=my-github-token +``` + +## Releasing an update (this repo) + +The `.fullsend/` registration pins content by commit SHA, so **editing a file is +not enough** — you must re-pin and re-lock in the same change. The full loop: + +1. **Edit** the harness (`harness/verify-pr.yaml`) and/or any pinned plugin file + under `plugins/sdlc-workflow/` (see the re-pin rule below). +2. **Commit and push** to `RHEcosystemAppEng`. `main` is the release channel; + during feature work, push to the feature branch and merge to `main`. +3. **Re-pin** the `base:` line in `.fullsend/harness/verify-pr.yaml`: set the + commit SHA to the new commit and the `#sha256=` to the base file's hash: + ```bash + shasum -a 256 harness/verify-pr.yaml + ``` +4. **Re-lock** so every transitive child is re-resolved at the new SHA: + ```bash + fullsend lock verify-pr --fullsend-dir .fullsend + ``` + `fullsend lock --all --fullsend-dir .fullsend` locks every harness at once — + equivalent here since `verify-pr` is the only one (TC-5813 used `--all`). +5. **Commit `.fullsend/`** (the updated `lock.yaml` and re-pinned child) in the + same PR as the content edit. + +> **`fullsend agent update` does not apply to this repo.** The registered +> `verify-pr` agent is a *local path* (`source: harness/verify-pr.yaml`), so +> `fullsend agent update verify-pr` fails with *"agent 'verify-pr' is a local +> path — nothing to update"*. `agent update` re-pins **URL** agents only — it is +> for adopters (below), not for the self-hosted `.fullsend/` model. Here you +> re-pin by editing the `base:` SHA/hash and re-locking. + +### Constraints and gotchas + +- **Merge with a merge commit — never squash or rebase.** The pinned raw URL in + `.fullsend/` (and the adopter `fullsend agent add …/blob/main/…` URL) resolves + to a specific commit SHA. A squash or rebase merge rewrites or drops that SHA, + so the pin — and any adopter fetch — 404s. This applies at **every** merge hop + up to `main`. +- **Re-pin + re-lock is mandatory after editing ANY pinned plugin file** — + `SKILL.md`, scripts, schemas, sub-skill templates, `plugin.json`, and so on. + Each such file is locked as a *member* of the `plugins[0]` directory dependency + in `.fullsend/lock.yaml`. An edit that is not paired with a re-pin/re-lock is + served **stale** at runtime: the pinned pre-edit content runs. Pair every + content edit with edit → commit → push → re-pin `base:` → `fullsend lock …` → + commit `.fullsend/` in the same PR. +- **Scope — what re-pinning does *not* affect.** The re-pin/re-lock loop governs + only the `fullsend run` path (local + CI). Interactive / marketplace installs + of the skill ignore `.fullsend/lock.yaml` entirely and use the plugin's + working-tree copy — so a missed re-pin never affects interactive users, only + fullsend runs. +- **Troubleshooting `cache integrity check failed`** at harness/lock load: caused + by a stale `.fullsend/.fullsend-cache` (and the sibling root `./.fullsend-cache`). + Fix: + ```bash + rm -rf .fullsend/.fullsend-cache .fullsend-cache + ``` + then re-run online so the pinned content re-fetches. + +## Adopting verify-pr (external fullsend users) + +Other repos do not need to clone sdlc-plugins or copy any files. Register the +harness by URL — no SHA needed, fullsend resolves `main` HEAD and pins it for +you: + +```bash +# 1. Register (one command, SHA-free — fullsend pins main HEAD) +fullsend agent add https://github.com/RHEcosystemAppEng/sdlc-plugins/blob/main/harness/verify-pr.yaml + +# 2. Lock and run +fullsend lock verify-pr --fullsend-dir .fullsend +fullsend run verify-pr --fullsend-dir .fullsend --target-repo /tmp/clone --env-file secrets.env + +# 3. Later, pull in a new release +fullsend agent update verify-pr --fullsend-dir .fullsend +``` + +`main` is the **release channel** — the URL above tracks it. There is no separate +`release` branch yet because `fullsend agent update` does not track a branch: +it re-pins to a commit SHA you resolve at update time, so a dedicated release +branch would add a maintenance hop without buying reproducibility that the SHA +pin does not already provide. The **merge-commit-only** rule above applies to +adopters too — a squash/rebase on any hop to `main` breaks the pinned URL the +adopter fetched. + +## File inventory + +All plugin paths are relative to `plugins/sdlc-workflow/`. + +| File | Purpose | +|---|---| +| `harness/verify-pr.yaml` (repo root) | Standalone harness — stock digest-pinned image, in-place plugin, policy, provider, profile, env mount, pre/post scripts, validation loop, split-trust env. | +| `.fullsend/config.yaml` | Registers the `verify-pr` agent (local source) and the single allowed remote-resource prefix. | +| `.fullsend/harness/verify-pr.yaml` | Local composing child — pins the root harness by raw URL (`base:` + `#sha256=`) and adds the runner-local absolute mounts. | +| `.fullsend/lock.yaml` | Generated by `fullsend lock` — freezes every transitive dependency URL + SHA256. Do not edit by hand. | +| `agents/verify-pr.md` | Agent prompt (YAML frontmatter). fullsend launches Claude Code with this as the system prompt; it reads `JIRA_ISSUE_ID` and invokes the skill. | +| `policies/verify-pr.yaml` | Sandbox network/filesystem policy. | +| `profiles/fullsend-vertex-ai.yaml` | OpenShell egress profile — `*.googleapis.com:443` only. | +| `providers/vertex-ai.yaml` | Selects the Vertex egress profile (no proxied credential). | +| `env/gcp-vertex.env` | Vertex env template, expanded from the secrets file (`expand: true`); points `GOOGLE_APPLICATION_CREDENTIALS` at `/tmp/.gcp-credentials.json`. | +| `schemas/verify-pr-result.schema.json` | JSON Schema for the agent's structured output; enforced by `validation_loop`. | +| `scripts/pre-verify-pr.sh` | Pre_script — validates inputs, prefetches the Jira issue and the GitHub read bundle, checks out the PR head. Delegates to `pre_verify_pr.py`. | +| `scripts/post-verify-pr.sh` | Post_script — finds `agent-result.json` and delegates to `execute-actions.py`. Runs on the trusted runner after the sandbox is destroyed. | +| `scripts/execute-actions.py` | Action executor — posts Jira/GitHub sticky comments via `fullsend issues post-comment`, and PR reviews/replies via `gh api`. | +| `scripts/validate-output-schema.sh` + `strip_extra_properties.py` | Strips benign agent-added metadata, then validates against the schema. | + +## Design decisions + +### Why a standalone root-level harness + +Placing the harness at the repo root lets fullsend resolve its relative children +against the repo root, so `plugins/sdlc-workflow` is delivered in place as a +whole plugin and its `shared/` resources resolve intact. No base composition +keeps the harness self-contained — the Go defaults already supply security +(`enabled: true`, `fail_mode: closed`) plus the sandbox hooks, so no explicit +`security:` block is needed. + +### Why the plugin is referenced in place (no duplication) + +fullsend fabricates the Claude Code marketplace cache from the `plugins:` entry +at runtime. The same plugin files back both interactive Claude Code installs and +fullsend runs, so there is no custom image, no Dockerfile, no bootstrap script, +and no second copy of the skills to keep in sync. + +### Why Jira reads/writes use native fullsend CLI (not MCP) in the sandbox + +MCP servers are not available inside the sandbox. All tracker and GitHub I/O is +handled on the runner: the pre_script prefetches with `gh` and the Jira REST +client, and the post_script posts sticky comments with +`fullsend issues post-comment`, which creates a marker-tagged comment on the +first run and edits it in place on re-runs (no comment flooding). + +### Why the stock digest-pinned image + +The harness pins `ghcr.io/fullsend-ai/fullsend-code` by `@sha256:` digest. The +stock image already carries Claude Code, `git`, the `gh` CLI, Python, and the +fullsend security tooling, so there is nothing to add — the sandbox *policy*, not +the image, is the enforcement layer. + +## Known issues + +- **fullsend deletes the target repo directory** after each run — always pass a + disposable clone as `--target-repo`, never your working directory. From 4ed6d79a4b26d0494d602dff07230d3ca604473d Mon Sep 17 00:00:00 2001 From: mrizzi Date: Tue, 8 Sep 2026 18:39:23 +0200 Subject: [PATCH 062/175] docs(verify-pr): correct pre_script PR-head checkout claim in fullsend.md The credential-delivery table and file inventory claimed the pre_script checks out the PR head on the runner. pre-verify-pr.sh only records the head ref name and head commit SHA in the read bundle; it performs no checkout. Correct both spots and document that the PR-head working tree is the caller-supplied --target-repo clone (gh pr checkout locally, CI checkout step), which the sandbox then inspects directly. Implements TC-6108 Assisted-by: Claude Code Co-Authored-By: Claude Opus 4.8 --- fullsend.md | 11 ++++++++--- 1 file changed, 8 insertions(+), 3 deletions(-) diff --git a/fullsend.md b/fullsend.md index 7b8e5f051..30cf71a0f 100644 --- a/fullsend.md +++ b/fullsend.md @@ -62,7 +62,7 @@ never place a credential in the sandbox at all. | Service | Tier | Credential in sandbox? | How it is delivered | |---|---|---|---| | **Jira** | 1 | No | The pre_script prefetches the issue on the runner; the post_script posts comments with `fullsend issues post-comment --tracker jira`. The token stays in runner env only. | -| **GitHub** | 1 | No | The pre_script prefetches the read bundle (`gh pr diff` / `gh pr view` → diff, diffstat, commits) and checks out the PR head on the runner; the post_script writes via `fullsend issues post-comment --tracker github` and `gh` (PR reviews/replies via `gh api`). The token stays in runner env only. | +| **GitHub** | 1 | No | The pre_script prefetches the read bundle (`gh pr diff` / `gh pr view` → diff, diffstat, reviews, comments, commits) and records the PR head ref name and head commit SHA in it; the post_script writes via `fullsend issues post-comment --tracker github` and `gh` (PR reviews/replies via `gh api`). The token stays in runner env only. The PR-head working tree is the `--target-repo` clone, checked out at the head SHA before the run (see below) — the pre_script does not check out. | | **Vertex AI** | 4 (fullsend-mandated) | **Yes** | The in-sandbox runtime does the model inference and reads `GOOGLE_APPLICATION_CREDENTIALS` from a file (`/tmp/.gcp-credentials.json`). Vertex auth requires local JWT signing, so tier 4 (file on the sandbox filesystem) is unavoidable. This is the one credential set in the sandbox. | Because Vertex is the only in-sandbox credential, the sandbox's **only network @@ -104,7 +104,12 @@ fullsend run verify-pr \ ``` `--target-repo` must be a **disposable clone**, not your working directory — -fullsend deletes and re-creates it after each run. +fullsend deletes and re-creates it after each run. It must already be **checked +out at the PR head** (`github.commit_sha` from the read bundle): neither the +pre_script nor the harness performs the checkout, so the caller establishes the +PR-head working tree. Locally, `gh pr checkout ` in the clone before +the run; in CI, the checkout step fetches the PR head. The sandbox then inspects +that tree directly (there is no `gh` CLI in the sandbox). A minimal `secrets.env` (never commit it): @@ -222,7 +227,7 @@ All plugin paths are relative to `plugins/sdlc-workflow/`. | `providers/vertex-ai.yaml` | Selects the Vertex egress profile (no proxied credential). | | `env/gcp-vertex.env` | Vertex env template, expanded from the secrets file (`expand: true`); points `GOOGLE_APPLICATION_CREDENTIALS` at `/tmp/.gcp-credentials.json`. | | `schemas/verify-pr-result.schema.json` | JSON Schema for the agent's structured output; enforced by `validation_loop`. | -| `scripts/pre-verify-pr.sh` | Pre_script — validates inputs, prefetches the Jira issue and the GitHub read bundle, checks out the PR head. Delegates to `pre_verify_pr.py`. | +| `scripts/pre-verify-pr.sh` | Pre_script — validates inputs, prefetches the Jira issue and the GitHub read bundle, and records the PR head ref name + head commit SHA in the bundle (it does **not** check out — the PR-head working tree comes from `--target-repo`). Delegates to `pre_verify_pr.py`. | | `scripts/post-verify-pr.sh` | Post_script — finds `agent-result.json` and delegates to `execute-actions.py`. Runs on the trusted runner after the sandbox is destroyed. | | `scripts/execute-actions.py` | Action executor — posts Jira/GitHub sticky comments via `fullsend issues post-comment`, and PR reviews/replies via `gh api`. | | `scripts/validate-output-schema.sh` + `strip_extra_properties.py` | Strips benign agent-added metadata, then validates against the schema. | From c67f6911d66f4b5624d4a28b79339a0900a71024 Mon Sep 17 00:00:00 2001 From: mrizzi Date: Wed, 9 Sep 2026 10:39:31 +0200 Subject: [PATCH 063/175] fix(verify-pr): stop post_script writing __pycache__ into fullsend cache tree MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit fullsend fetches the pre/post script directory into its content-addressed cache tree and executes the scripts in place. execute-actions.py loads its sibling jira-client.py via importlib exec_module, and SourceFileLoader writes __pycache__/jira_client.*.pyc next to the source — into the cache tree. That mutates the tree, so fullsend's next-run integrity walk recomputes a different hash and fails with 'cache integrity check failed'. Disable bytecode writes at two layers (defense in depth): - execute-actions.py sets sys.dont_write_bytecode = True before exec_module. - harness/verify-pr.yaml adds PYTHONDONTWRITEBYTECODE: "1" to env.runner so every runner-side Python invocation from the cache tree is inert. Verified: running execute-actions.py as a script (as the post_script does) triggers the exec_module vector and writes no __pycache__/*.pyc. Implements TC-6112 Co-Authored-By: Claude Opus 4.8 Assisted-by: Claude Code --- harness/verify-pr.yaml | 6 ++++++ plugins/sdlc-workflow/scripts/execute-actions.py | 7 +++++++ 2 files changed, 13 insertions(+) diff --git a/harness/verify-pr.yaml b/harness/verify-pr.yaml index e68b6ea85..d9f94d016 100644 --- a/harness/verify-pr.yaml +++ b/harness/verify-pr.yaml @@ -79,6 +79,12 @@ env: # prefetch and post_script writes need them; they are never placed in # env.sandbox. runner: + # Disable Python bytecode writes for every runner-side Python invocation. + # The pre/post scripts execute in place inside fullsend's content-addressed + # cache tree; a stray __pycache__/*.pyc written there mutates the tree and + # trips fullsend's next-run integrity check. Defense in depth alongside + # execute-actions.py's own sys.dont_write_bytecode (TC-6112). + PYTHONDONTWRITEBYTECODE: "1" JIRA_ISSUE_ID: "${JIRA_ISSUE_ID}" JIRA_SERVER_URL: "${JIRA_SERVER_URL}" JIRA_EMAIL: "${JIRA_EMAIL}" diff --git a/plugins/sdlc-workflow/scripts/execute-actions.py b/plugins/sdlc-workflow/scripts/execute-actions.py index fd6533ee2..d963416d9 100755 --- a/plugins/sdlc-workflow/scripts/execute-actions.py +++ b/plugins/sdlc-workflow/scripts/execute-actions.py @@ -45,6 +45,13 @@ "jira_client", os.path.join(_script_dir, "jira-client.py") ) _jira_mod = importlib.util.module_from_spec(_spec) +# fullsend executes this script in place inside its content-addressed cache tree. +# SourceFileLoader.exec_module writes __pycache__/*.pyc next to the source, which +# mutates that tree and breaks fullsend's next-run integrity check ("cache +# integrity check failed"). Disable bytecode writes before loading the sibling +# module so nothing is written into the cache tree (TC-6112). The harness also +# sets PYTHONDONTWRITEBYTECODE for defense in depth. +sys.dont_write_bytecode = True _spec.loader.exec_module(_jira_mod) REF_PATTERN = re.compile(r"\{\{([a-z0-9-]+)\.(key|url)\}\}") From dca34de8ecedf3b2b38308bbbdfc84bdc76666af Mon Sep 17 00:00:00 2001 From: mrizzi Date: Wed, 9 Sep 2026 10:40:58 +0200 Subject: [PATCH 064/175] chore(verify-pr): re-pin .fullsend base URL to c67f6911, re-lock Re-pin the local composing child's base: to the content commit that adds the bytecode-write guards, and re-lock so every transitive child (including the edited execute-actions.py) resolves at the new SHA. Mandatory pairing after editing a pinned plugin file. Implements TC-6112 Co-Authored-By: Claude Opus 4.8 Assisted-by: Claude Code --- .fullsend/harness/verify-pr.yaml | 2 +- .fullsend/lock.yaml | 66 ++++++++++++++++---------------- 2 files changed, 34 insertions(+), 34 deletions(-) diff --git a/.fullsend/harness/verify-pr.yaml b/.fullsend/harness/verify-pr.yaml index 750b04414..62ee61c27 100644 --- a/.fullsend/harness/verify-pr.yaml +++ b/.fullsend/harness/verify-pr.yaml @@ -27,7 +27,7 @@ # (base URLs are validated against that allowlist, not inherited). # Re-pin after any base edit: push harness/verify-pr.yaml, set the SHA to the new # commit + the sha256 to `shasum -a 256 harness/verify-pr.yaml`, `fullsend lock`. -base: https://raw.githubusercontent.com/RHEcosystemAppEng/sdlc-plugins/430a29c86bfbc601370a32251a555c9d17af6bcb/harness/verify-pr.yaml#sha256=addc868659a4391b18fe09cb2a0523d134a9242e35d644a6db36bcce8277cb16 +base: https://raw.githubusercontent.com/RHEcosystemAppEng/sdlc-plugins/c67f6911d66f4b5624d4a28b79339a0900a71024/harness/verify-pr.yaml#sha256=d8ebedc8978b1ea2df938110eaef20c669319786c4db2e45faf6c9040e3ebb95 host_files: - src: ${GOOGLE_APPLICATION_CREDENTIALS} diff --git a/.fullsend/lock.yaml b/.fullsend/lock.yaml index a69aafcd6..e13e490e0 100644 --- a/.fullsend/lock.yaml +++ b/.fullsend/lock.yaml @@ -4,21 +4,21 @@ generated_at: 2026-09-04T13:39:01.038924Z harnesses: verify-pr: source: harness/verify-pr.yaml - sha256: 1d5889ef6f71dae2523db790bd848af6a8353ad739499a4ae583b264bae1c8d2 - resolved_at: 2026-09-08T13:11:05.215335Z + sha256: 272f21acb6e5c73ea5bdfe301ba39b8f5ae0122da8a354f33fe42f293c3318c6 + resolved_at: 2026-09-09T08:40:35.251104Z dependencies: - field: base - url: https://raw.githubusercontent.com/RHEcosystemAppEng/sdlc-plugins/430a29c86bfbc601370a32251a555c9d17af6bcb/harness/verify-pr.yaml - sha256: addc868659a4391b18fe09cb2a0523d134a9242e35d644a6db36bcce8277cb16 + url: https://raw.githubusercontent.com/RHEcosystemAppEng/sdlc-plugins/c67f6911d66f4b5624d4a28b79339a0900a71024/harness/verify-pr.yaml + sha256: d8ebedc8978b1ea2df938110eaef20c669319786c4db2e45faf6c9040e3ebb95 type: file - fetched_at: 2026-09-08T13:10:57.161392Z + fetched_at: 2026-09-09T08:40:27.050053Z - field: pre_script - url: https://raw.githubusercontent.com/RHEcosystemAppEng/sdlc-plugins/430a29c86bfbc601370a32251a555c9d17af6bcb/plugins/sdlc-workflow/scripts/pre-verify-pr.sh - sha256: 4f5d1bfea092559de8f44469a32cd636949d01c5a029f0782c94187057d11f33 + url: https://raw.githubusercontent.com/RHEcosystemAppEng/sdlc-plugins/c67f6911d66f4b5624d4a28b79339a0900a71024/plugins/sdlc-workflow/scripts/pre-verify-pr.sh + sha256: 6ff7f8857ce1bf53da7e82d1d756491bd3cc9aa0c4260c9b03a0aea588e3f051 type: directory files: - path: execute-actions.py - sha256: 6df3bc2598a891868ac3a02f3265b7036c9d4b6d1fdb6394ca619fe89946c520 + sha256: 595220e3cac4426acaecfda17e9c7b73532cd9bf59a453c4093dc8e9a57a0936 - path: jira-client.py sha256: f6bf79d2c98a2ae020775e68a538f612953c0464f09bd750c988f0d20fd4b159 - path: post-verify-pr.sh @@ -39,14 +39,14 @@ harnesses: sha256: c7ae30979613fa3402de4450c9bb4faf5f011decac79b75a36ba2ef51e4d21aa - path: validate-output-schema.sh sha256: 56b964145da0b62f438dbc1885780d86e2d589c5340adb93c00e3c8777979cbe - fetched_at: 2026-09-08T13:11:00.144348Z + fetched_at: 2026-09-09T08:40:29.851598Z - field: post_script - url: https://raw.githubusercontent.com/RHEcosystemAppEng/sdlc-plugins/430a29c86bfbc601370a32251a555c9d17af6bcb/plugins/sdlc-workflow/scripts/post-verify-pr.sh - sha256: 4f5d1bfea092559de8f44469a32cd636949d01c5a029f0782c94187057d11f33 + url: https://raw.githubusercontent.com/RHEcosystemAppEng/sdlc-plugins/c67f6911d66f4b5624d4a28b79339a0900a71024/plugins/sdlc-workflow/scripts/post-verify-pr.sh + sha256: 6ff7f8857ce1bf53da7e82d1d756491bd3cc9aa0c4260c9b03a0aea588e3f051 type: directory files: - path: execute-actions.py - sha256: 6df3bc2598a891868ac3a02f3265b7036c9d4b6d1fdb6394ca619fe89946c520 + sha256: 595220e3cac4426acaecfda17e9c7b73532cd9bf59a453c4093dc8e9a57a0936 - path: jira-client.py sha256: f6bf79d2c98a2ae020775e68a538f612953c0464f09bd750c988f0d20fd4b159 - path: post-verify-pr.sh @@ -67,14 +67,14 @@ harnesses: sha256: c7ae30979613fa3402de4450c9bb4faf5f011decac79b75a36ba2ef51e4d21aa - path: validate-output-schema.sh sha256: 56b964145da0b62f438dbc1885780d86e2d589c5340adb93c00e3c8777979cbe - fetched_at: 2026-09-08T13:11:00.138536Z + fetched_at: 2026-09-09T08:40:29.845646Z - field: validation_loop.script - url: https://raw.githubusercontent.com/RHEcosystemAppEng/sdlc-plugins/430a29c86bfbc601370a32251a555c9d17af6bcb/plugins/sdlc-workflow/scripts/validate-output-schema.sh - sha256: 4f5d1bfea092559de8f44469a32cd636949d01c5a029f0782c94187057d11f33 + url: https://raw.githubusercontent.com/RHEcosystemAppEng/sdlc-plugins/c67f6911d66f4b5624d4a28b79339a0900a71024/plugins/sdlc-workflow/scripts/validate-output-schema.sh + sha256: 6ff7f8857ce1bf53da7e82d1d756491bd3cc9aa0c4260c9b03a0aea588e3f051 type: directory files: - path: execute-actions.py - sha256: 6df3bc2598a891868ac3a02f3265b7036c9d4b6d1fdb6394ca619fe89946c520 + sha256: 595220e3cac4426acaecfda17e9c7b73532cd9bf59a453c4093dc8e9a57a0936 - path: jira-client.py sha256: f6bf79d2c98a2ae020775e68a538f612953c0464f09bd750c988f0d20fd4b159 - path: post-verify-pr.sh @@ -95,40 +95,40 @@ harnesses: sha256: c7ae30979613fa3402de4450c9bb4faf5f011decac79b75a36ba2ef51e4d21aa - path: validate-output-schema.sh sha256: 56b964145da0b62f438dbc1885780d86e2d589c5340adb93c00e3c8777979cbe - fetched_at: 2026-09-08T13:11:00.138536Z + fetched_at: 2026-09-09T08:40:29.845646Z - field: validation_loop.schema - url: https://raw.githubusercontent.com/RHEcosystemAppEng/sdlc-plugins/430a29c86bfbc601370a32251a555c9d17af6bcb/plugins/sdlc-workflow/schemas/verify-pr-result.schema.json + url: https://raw.githubusercontent.com/RHEcosystemAppEng/sdlc-plugins/c67f6911d66f4b5624d4a28b79339a0900a71024/plugins/sdlc-workflow/schemas/verify-pr-result.schema.json sha256: 62c3e5ce0c47e73823b5cab9a577be7b721d8fb31389bc8602dc82ee05526ad6 type: resource - fetched_at: 2026-09-08T13:11:00.630765Z + fetched_at: 2026-09-09T08:40:30.10887Z - field: agent - url: https://raw.githubusercontent.com/RHEcosystemAppEng/sdlc-plugins/430a29c86bfbc601370a32251a555c9d17af6bcb/plugins/sdlc-workflow/agents/verify-pr.md + url: https://raw.githubusercontent.com/RHEcosystemAppEng/sdlc-plugins/c67f6911d66f4b5624d4a28b79339a0900a71024/plugins/sdlc-workflow/agents/verify-pr.md sha256: 568b2ab7e32300c84038d500d95f4534ff6c4b20e52eebf2ed5a6c0ae706786a type: resource - fetched_at: 2026-09-08T13:11:00.883248Z + fetched_at: 2026-09-09T08:40:30.353891Z - field: policy - url: https://raw.githubusercontent.com/RHEcosystemAppEng/sdlc-plugins/430a29c86bfbc601370a32251a555c9d17af6bcb/plugins/sdlc-workflow/policies/verify-pr.yaml + url: https://raw.githubusercontent.com/RHEcosystemAppEng/sdlc-plugins/c67f6911d66f4b5624d4a28b79339a0900a71024/plugins/sdlc-workflow/policies/verify-pr.yaml sha256: 62d1374be253b0b225921bd8e22e679781076ecdb3a9ea22d5e6a4805753c5a7 type: resource - fetched_at: 2026-09-08T13:11:01.150243Z + fetched_at: 2026-09-09T08:40:30.598631Z - field: host_files[0].src - url: https://raw.githubusercontent.com/RHEcosystemAppEng/sdlc-plugins/430a29c86bfbc601370a32251a555c9d17af6bcb/plugins/sdlc-workflow/env/gcp-vertex.env + url: https://raw.githubusercontent.com/RHEcosystemAppEng/sdlc-plugins/c67f6911d66f4b5624d4a28b79339a0900a71024/plugins/sdlc-workflow/env/gcp-vertex.env sha256: 10b2ba695b1d4e65e0964a233b75d6a06853f1eaefc260c56b9eedd989ae7d41 type: resource - fetched_at: 2026-09-08T13:11:01.447018Z + fetched_at: 2026-09-09T08:40:31.177742Z - field: openshell.profiles[0] - url: https://raw.githubusercontent.com/RHEcosystemAppEng/sdlc-plugins/430a29c86bfbc601370a32251a555c9d17af6bcb/plugins/sdlc-workflow/profiles/fullsend-vertex-ai.yaml + url: https://raw.githubusercontent.com/RHEcosystemAppEng/sdlc-plugins/c67f6911d66f4b5624d4a28b79339a0900a71024/plugins/sdlc-workflow/profiles/fullsend-vertex-ai.yaml sha256: 76535a148387b1281be2bfb23b6724e4133327189b38c8ce1de2c6dddb8d8341 type: resource - fetched_at: 2026-09-08T13:11:01.712662Z + fetched_at: 2026-09-09T08:40:32.276039Z - field: providers[0] - url: https://raw.githubusercontent.com/RHEcosystemAppEng/sdlc-plugins/430a29c86bfbc601370a32251a555c9d17af6bcb/plugins/sdlc-workflow/providers/vertex-ai.yaml + url: https://raw.githubusercontent.com/RHEcosystemAppEng/sdlc-plugins/c67f6911d66f4b5624d4a28b79339a0900a71024/plugins/sdlc-workflow/providers/vertex-ai.yaml sha256: ae5ebe527e5d7b0fa6994346fde3f6ba11b632a3ba78ece96591b5ed85083ec1 type: resource - fetched_at: 2026-09-08T13:11:02.357982Z + fetched_at: 2026-09-09T08:40:32.590801Z - field: plugins[0] - url: https://raw.githubusercontent.com/RHEcosystemAppEng/sdlc-plugins/430a29c86bfbc601370a32251a555c9d17af6bcb/plugins/sdlc-workflow/plugin.json - sha256: 5234e62de3143c5b549f90111888f4d34d290daf7f37c61f40646e269a9daa33 + url: https://raw.githubusercontent.com/RHEcosystemAppEng/sdlc-plugins/c67f6911d66f4b5624d4a28b79339a0900a71024/plugins/sdlc-workflow/plugin.json + sha256: f9c78da59d809dd1f784b9d552d552f4e0395495852e179854b8203c2e2ee46f type: directory files: - path: .claude-plugin/plugin.json @@ -150,7 +150,7 @@ harnesses: - path: schemas/verify-pr-result.schema.json sha256: 62c3e5ce0c47e73823b5cab9a577be7b721d8fb31389bc8602dc82ee05526ad6 - path: scripts/execute-actions.py - sha256: 6df3bc2598a891868ac3a02f3265b7036c9d4b6d1fdb6394ca619fe89946c520 + sha256: 595220e3cac4426acaecfda17e9c7b73532cd9bf59a453c4093dc8e9a57a0936 - path: scripts/jira-client.py sha256: f6bf79d2c98a2ae020775e68a538f612953c0464f09bd750c988f0d20fd4b159 - path: scripts/post-verify-pr.sh @@ -237,4 +237,4 @@ harnesses: sha256: 128877cf7e156666dbb403805a0112059cc8cc3255e148c6794da29d48b8f5cf - path: skills/verify-pr/style-conventions.md sha256: 5ab920c596615c0d23354cb0e90fe5ebf5f08635ffa3602e8e2d2abd74028a19 - fetched_at: 2026-09-08T13:11:05.210442Z + fetched_at: 2026-09-09T08:40:35.249442Z From 6572360e3e6cae77a364ed25324e54f67bb77549 Mon Sep 17 00:00:00 2001 From: mrizzi Date: Wed, 9 Sep 2026 11:26:58 +0200 Subject: [PATCH 065/175] fix(verify-pr): condense SKILL.md under Skillsaw 16k context budget MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The verify-pr SKILL.md context-budget estimate was 16,172 tokens, 172 over Skillsaw's 16,000-token error limit. Condense to 15,419 tokens (581 headroom) via prose compression and structural deduplication only — no semantic or behavioral changes: - Move the inline ADF comment-footnote block to a reference to shared/comment-footnote.md (skill name verify-pr), keeping the ${CLAUDE_PLUGIN_ROOT} version-path override so the footnote is byte-identical. - Collapse the four duplicated Step 6e review-body reply templates into one referential form; fold illustrative examples into inline hints. - Compress the 'Resolving this skill's own files', Step 0.7, Step 3, Step 4a/4b, Step 6f, and Step 7 prose; every directive preserved. skillsaw: 0 errors. claude plugin validate: passed. Assisted-by: Claude Code Co-Authored-By: Claude Opus 4.8 --- .../sdlc-workflow/skills/verify-pr/SKILL.md | 242 ++++++------------ 1 file changed, 74 insertions(+), 168 deletions(-) diff --git a/plugins/sdlc-workflow/skills/verify-pr/SKILL.md b/plugins/sdlc-workflow/skills/verify-pr/SKILL.md index 1f2320b02..baf5ab41f 100644 --- a/plugins/sdlc-workflow/skills/verify-pr/SKILL.md +++ b/plugins/sdlc-workflow/skills/verify-pr/SKILL.md @@ -11,30 +11,25 @@ You are an AI verification assistant that orchestrates PR verification through p ## Resolving this skill's own files -This skill reads several of its own bundled files: sub-skill instruction files, -dispatch/finding templates, JSON schemas, and the plugin manifest. **Always resolve -these from `${CLAUDE_PLUGIN_ROOT}`** — the environment variable Claude Code sets to -this plugin's installation directory — never from a repo-relative path like -`plugins/sdlc-workflow/...`. - -This matters because the current working directory is **not** always the plugin's -repository. When verify-pr reviews a PR against the `sdlc-plugins` repo itself, the -CWD is the target checkout (whatever branch is under review), so a repo-relative path -would read that branch's copy of these files and let the PR under review **shadow** -the pinned, stable skill actually running. `${CLAUDE_PLUGIN_ROOT}` always points at -the delivered plugin, so the stable skill reads its own bundled files in every mode — +This skill reads several of its own bundled files (sub-skill instruction files, +dispatch/finding templates, JSON schemas, the plugin manifest). **Always resolve these +from `${CLAUDE_PLUGIN_ROOT}`** — never a repo-relative path like `plugins/sdlc-workflow/...`. +The CWD is **not** always the plugin's repository: when verify-pr reviews a PR against +`sdlc-plugins` itself, the CWD is the target checkout, so a repo-relative path would read +that branch's copy and let the PR under review **shadow** the pinned, stable skill actually +running. `${CLAUDE_PLUGIN_ROOT}` always points at the delivered plugin, in every mode — interactive (Claude Code) and sandbox (fullsend). -`${CLAUDE_PLUGIN_ROOT}` is a **textual token** that Claude Code substitutes into this -skill's markdown body (this whole file, including fenced code blocks) before the skill -runs — it is **not** an OS environment variable. Write the literal token wherever you -need the path: prose, Read/Glob paths, and inside a `<< 'PYEOF'` heredoc alike. Do -**not** read it at runtime via `os.environ["CLAUDE_PLUGIN_ROOT"]` or `$CLAUDE_PLUGIN_ROOT` -in a shell — it is not exported to the shell or to subprocesses, so those resolve to -empty / `KeyError`. Substitution happens before execution, so a literal token even -inside a single-quoted heredoc is already the absolute path by the time Python runs. +`${CLAUDE_PLUGIN_ROOT}` is a **textual token** Claude Code substitutes into this skill's +markdown body (this whole file, including fenced code blocks) before the skill runs — it +is **not** an OS environment variable. Write the literal token wherever you need the path: +prose, Read/Glob paths, and inside a `<< 'PYEOF'` heredoc alike — substitution happens +before execution, so even inside a single-quoted heredoc it is already the absolute path by +the time Python runs. Do **not** read it at runtime via `os.environ["CLAUDE_PLUGIN_ROOT"]` +or `$CLAUDE_PLUGIN_ROOT` in a shell — it is not exported to the shell or subprocesses, so +those resolve to empty / `KeyError`. -Note: path patterns used to **filter the PR diff** (e.g., detecting changes under +Note: path patterns used to **filter the PR diff** (e.g. detecting changes under `plugins/sdlc-workflow/skills/run-evals/`) stay repo-relative — those describe files inside the PR being reviewed, not files this skill reads. @@ -150,12 +145,10 @@ The `report` object and every action conform to **Skip this step in interactive mode.** In sandbox mode the `pre_script` fetches all task and PR data on the trusted runner -(where the tokens live) and mounts it read-only into the sandbox. Read it: - -Validate it against `${CLAUDE_PLUGIN_ROOT}/schemas/verify-pr-input.schema.json` -before using it — a syntactically valid but structurally wrong or incomplete -prefetch (e.g., missing the `github` bundle or `task` fields) must be treated as -invalid rather than passing and failing deep inside a later step: +(where the tokens live) and mounts it read-only into the sandbox. Validate it against +`${CLAUDE_PLUGIN_ROOT}/schemas/verify-pr-input.schema.json` before using it — a +valid-JSON but structurally wrong or incomplete prefetch (e.g. missing the `github` +bundle or `task` fields) must be treated as invalid, not fail deep inside a later step: ```bash python3 - << 'PYEOF' @@ -199,20 +192,18 @@ If the file validates against the schema: `labels`, `description`, `issuetype`, `comments`) — the task's existing sub-tasks and linked issues, prefetched on the runner. **Use it for every idempotency read** (Steps 6d, 6f, 7c) instead of calling Jira; the sandbox has no token. The schema - **requires** `idempotency.related_issues`, so a prefetch that passes the Step 0.7 - validation above always carries this array — it is an empty list (never absent) - when the task has no sub-tasks or linked issues yet. Treat a present-but-empty - array as "no known duplicates"; you never need to fall back to a Jira read for - the dedup checks. + **requires** this array, so a prefetch that passes validation always carries it — an + empty list (never absent) means the task has no sub-tasks/linked issues yet. Treat + present-but-empty as "no known duplicates"; never fall back to a Jira read for dedup. If the validation above fails (file missing, not valid JSON, or not conforming to the schema), the handling depends on the mode: - **Sandbox mode (`FULLSEND_OUTPUT_DIR` is set):** do **not** fall back to Steps 1–3 or - the direct Jira/GitHub reads. The sandbox has no tokens, no `gh` CLI, and no network - egress, so a credentialed fallback cannot succeed — it would fail obscurely or hang. - Fail fast and loud instead: write a structured failure result and stop the skill - without performing any further steps or write operations. + direct Jira/GitHub reads — the sandbox has no tokens, no `gh` CLI, and no network + egress, so a credentialed fallback cannot succeed (it would fail obscurely or hang). + Fail fast and loud: write a structured failure result and stop the skill without any + further step or write operation. ```bash cat > "$FULLSEND_OUTPUT_DIR/agent-result.json" << 'RESULT_EOF' @@ -222,9 +213,9 @@ cat > "$FULLSEND_OUTPUT_DIR/agent-result.json" << 'RESULT_EOF' RESULT_EOF ``` - This result intentionally omits the `report` and `actions` keys required by + This result intentionally omits the `report`/`actions` keys required by `verify-pr-result.schema.json`, so the runner's output validation rejects it and - surfaces the `error` message as a hard failure rather than silently attempting the + surfaces the `error` as a hard failure rather than silently attempting the interactive path. **Stop execution here — do not run any subsequent step.** - **Interactive mode (`FULLSEND_OUTPUT_DIR` is unset):** log a warning and fall back to @@ -240,47 +231,12 @@ Example: ## Comment Footnote -Every comment posted to Jira by this skill MUST end with the following footnote, -separated from the main content by a horizontal rule. - -Before posting any Jira comment, read the plugin version from -`${CLAUDE_PLUGIN_ROOT}/.claude-plugin/plugin.json` and extract the `version` field. -Use this value as `{version}` in the footer below. - -Use ADF `contentFormat` to ensure the rule and text render correctly: - -```json -{ - "type": "rule" -}, -{ - "type": "paragraph", - "content": [ - { - "type": "text", - "text": "This comment was AI-generated by " - }, - { - "type": "text", - "text": "sdlc-workflow/verify-pr", - "marks": [ - { - "type": "link", - "attrs": { - "href": "https://github.com/RHEcosystemAppEng/sdlc-plugins" - } - } - ] - }, - { - "type": "text", - "text": " v{version}." - } - ] -} -``` - -Append these two nodes at the end of the ADF document's `content` array. +Every comment posted to Jira by this skill MUST end with the footnote defined in +`shared/comment-footnote.md`, using skill name `verify-pr`. **Override** that doc's +version-path instruction: read the plugin version from +`${CLAUDE_PLUGIN_ROOT}/.claude-plugin/plugin.json` — never the repo-relative path (see +"Resolving this skill's own files"). Append the two ADF nodes (rule + paragraph) at the +end of the comment document's `content` array. ## Step 1 – Fetch and Parse Jira Task @@ -333,18 +289,14 @@ gh pr view --json headRefName -R git branch --show-current ``` -3. If the branches match, proceed without action — the correct code is already available locally (e.g. the author running self-verification after `/implement-task`). +3. If the branches match, proceed — the correct code is already local (author self-verification after `/implement-task`; no checkout needed). -4. If they differ, check out the PR branch: +4. If they differ, check out the PR branch (reviewer/CI audit from an arbitrary branch): ``` gh pr checkout -R ``` -This step supports two use cases: -- **Author self-verification** — the contributor already has the PR branch checked out; no checkout needed. -- **Reviewer/CI audit** — another person or CI job runs `/verify-pr` from an arbitrary branch; the PR branch must be checked out first. - **Sandbox mode:** skip this step entirely — the runner has already checked out the PR head tree (`github.headRefName` at `github.commit_sha`) and there is no `gh` CLI in the sandbox. Inspect the working tree directly. @@ -425,12 +377,10 @@ Log both lists so the run output shows the full enumeration result, making it auditable that all items were considered. > **Why this matters:** Without mandatory enumeration, a re-run can check only for -> existing classification replies and conclude "nothing to do" — completely missing -> new items that arrived after the previous run. This caused a real failure where a -> bot review comment posted after `/implement-task` pushed a fix commit was missed -> by the subsequent `/verify-pr` re-run. The same gap allowed review body -> suggestions (e.g., from sourcery-ai) to be silently skipped because only inline -> comments were enumerated. +> existing classification replies, conclude "nothing to do", and miss new items that +> arrived after the previous run — a real failure mode (a bot review comment posted +> after an `/implement-task` fix commit was missed on re-run; review-body suggestions +> from sourcery-ai were skipped because only inline comments were enumerated). ### Step 4a.1 – Detect Eval Result Reviews @@ -460,9 +410,9 @@ inform classification decisions. with the absolute path from the Registry (e.g., `/CONVENTIONS.md`). If present, read its contents. This provides explicit, documented project conventions. -2. **Codebase convention cache:** This step does not perform exhaustive codebase analysis - yet — that happens in the Style/Conventions sub-agent. The goal here is only to load - CONVENTIONS.md once for reuse across comment classification and sub-agent dispatch. +2. **Scope:** load CONVENTIONS.md once here for reuse across comment classification and + sub-agent dispatch; exhaustive codebase analysis happens later in the + Style/Conventions sub-agent. If `CONVENTIONS.md` does not exist, proceed normally — the Style/Conventions sub-agent will check for implicit conventions demonstrated by codebase usage patterns. @@ -822,54 +772,34 @@ gh api repos//pulls//comments//replies -f bod ``` **For suggestions upgraded to code change requests via convention check (Step 6b):** - -Include the convention evidence in the reply so the upgrade reasoning is transparent: +Include the convention evidence so the upgrade reasoning is transparent (e.g. "…this +matches project convention: 17 migrations use Index::create for FK columns; CONVENTIONS.md +§Indexes documents this pattern. Sub-task […] created…"): ``` gh api repos//pulls//comments//replies -f body="[sdlc-workflow/verify-pr] Classified as **code change request** (upgraded from suggestion) — this matches project convention: . Sub-task []() created to address this feedback." ``` -Example: `"[sdlc-workflow/verify-pr] Classified as **code change request** (upgraded from suggestion) — this matches project convention: 17 migrations use Index::create for FK columns; CONVENTIONS.md §Indexes documents this pattern. Sub-task [PROJ-456](https://redhat.atlassian.net/browse/PROJ-456) created to address this feedback."` - -**For all other classifications (suggestion, question, nit):** - -Reply with a brief explanation of the classification and why no sub-task was created: +**For all other classifications (suggestion, question, nit):** reply with a brief +explanation of the classification and why no sub-task was created (e.g. suggestion → "not +documented in CONVENTIONS.md and no established codebase pattern"; question → "asks for +clarification; no code change needed"; nit → "minor style, does not affect correctness"): ``` gh api repos//pulls//comments//replies -f body="[sdlc-workflow/verify-pr] Classified as **** — . No sub-task created." ``` -Example replies: -- `"[sdlc-workflow/verify-pr] Classified as **suggestion** — this proposes an alternative approach that is not documented in CONVENTIONS.md and has no established codebase pattern. No sub-task created."` -- `"[sdlc-workflow/verify-pr] Classified as **question** — this asks for clarification; no code change needed. No sub-task created."` -- `"[sdlc-workflow/verify-pr] Classified as **nit** — minor style feedback that does not affect correctness. No sub-task created."` - #### Review body items Review body items (identified by `review-body-*` synthetic IDs) lack a `comment_id` -and cannot receive threaded replies. Instead, post a **standalone PR comment** using -the issues API: - -``` -gh api repos//issues//comments -f body="[sdlc-workflow/verify-pr] Re: @ review — Classified as **** — . " -``` - -**For code change requests that resulted in a sub-task:** - -``` -gh api repos//issues//comments -f body="[sdlc-workflow/verify-pr] Re: @ review — Classified as **code change request** — sub-task []() created to address this feedback." -``` - -**For suggestions upgraded via convention check (Step 6b):** - -``` -gh api repos//issues//comments -f body="[sdlc-workflow/verify-pr] Re: @ review — Classified as **code change request** (upgraded from suggestion) — this matches project convention: . Sub-task []() created to address this feedback." -``` - -**For all other classifications:** +and cannot receive threaded replies. Post a **standalone PR comment** via the issues +API instead — use the **same classification body as the matching inline case above**, +but prefixed with `Re: @ review — ` right after the `[sdlc-workflow/verify-pr]` +tag (code change request → sub-task link; upgraded suggestion → convention evidence + +sub-task link; suggestion/question/nit → reasoning + "No sub-task created."): ``` -gh api repos//issues//comments -f body="[sdlc-workflow/verify-pr] Re: @ review — Classified as **** — . No sub-task created." +gh api repos//issues//comments -f body="[sdlc-workflow/verify-pr] Re: @ review — Classified as **** — " ``` When a review body contains multiple classified suggestions (sub-identifiers), post @@ -903,18 +833,14 @@ For **review body items** (no `comment_id`), use `post_pr_comment`: ### Step 6f – Idempotency Guarantees -Idempotency is enforced **within** Step 4a's mandatory enumeration, not as a -separate gate before Steps 6d/6e. The enumeration in Step 4a partitions all -classifiable items (inline comment threads and review body items) into unclassified -and already-classified lists — only unclassified items proceed to Steps 4b–4c for -classification and then to Steps 6c–6e for side effects. This design ensures that: - -- Every classifiable item is always discovered (enumeration cannot be bypassed). -- Already-processed items are filtered out per-item (no duplicate replies or - sub-tasks). -- New items arriving between runs (e.g., from a bot re-analyzing code after a - fix commit, or a reviewer adding a new review) are always detected because the - enumeration is unconditional. +Idempotency is enforced **within** Step 4a's mandatory enumeration, not as a separate +gate before Steps 6d/6e: the enumeration partitions all classifiable items (inline +comment threads and review body items) into unclassified and already-classified lists — +only unclassified items proceed to classification (4b–4c) and side effects (6c–6e). +Every item is always discovered (enumeration is unconditional and cannot be bypassed), +already-processed items are filtered per-item (no duplicate replies/sub-tasks), and new +items arriving between runs (a bot re-analyzing after a fix commit, a new review) are +always detected. Do **not** treat this as a top-level gate that can skip enumeration. **Idempotency detection per item type:** - **Inline comment threads:** a reply containing `"[sdlc-workflow/verify-pr] Classified as"` @@ -930,10 +856,6 @@ was not posted (e.g., due to a network error) but the sub-task was created. **Sandbox mode:** read these existing sub-tasks from `idempotency.related_issues` (Step 0.7) — inspect each entry's `description` — instead of a live Jira read. -Do **not** interpret this step as a top-level decision that can skip item -enumeration. The enumeration in Step 4a is always mandatory; this step only -documents the idempotency mechanisms embedded within that enumeration. - ### Step 6g – Record Result Record the Review Feedback check result: @@ -975,22 +897,10 @@ The sub-agent receives these inputs: 3. **Review comments** — the code change requests that triggered sub-tasks in Step 6d 4. **Relevant code** — the files on the PR branch related to each flagged defect 5. **Project CONVENTIONS.md** — if it exists in the repository root -6. **Aggregated domain findings** — findings from all four domain sub-agents, - organized by source with clear attribution: - - ``` - ### From Intent Alignment - - - ### From Security - - - ### From Correctness - - - ### From Style/Conventions -