Skip to content

perf(release-verify-rc): classify RAT, JVM, Nexus and reproducibility results as code - #1619

Open
potiuk wants to merge 1 commit into
perf/release-mgmt-1-rc-cutfrom
perf/release-mgmt-2-verify-rc
Open

potiuk wants to merge 1 commit into
perf/release-mgmt-1-rc-cutfrom
perf/release-mgmt-2-verify-rc

Conversation

@potiuk

@potiuk potiuk commented Oct 13, 2026

Copy link
Copy Markdown
Member

Summary

Second PR of the release-management stack; it builds on #1604.

  • What changes: release-verify-rc ran the release-verify checks, then had the agent work out their arguments from release-build.md and classify what came back, by hand. tools/release-management verify-rc now does it, reading saved files only.
  • New commands: args (the exact release-verify invocations), rat (Step 4), source-tree (Step 7 REVIEW resolution), jvm (6b gates and findings), nexus (6c gates, staging-repo id check, every classification rule over saved probe output), repro (Step 9) and report (Step 10 report and --post-to comment).
  • release-config: load --skill verify-rc now exposes the required/optional artefact marker, binary-exclude globs, RAT paths, validators and JVM keys. Existing fields are unchanged.
  • Size: SKILL.md goes from 9,936 to 6,280 tokens and from 684 to 466 lines. The 36 failure-mode rows move into per-step files, and the JVM, Nexus and reproducibility siblings keep only their policy plus the command to run.

Security notes

  • The staging-repo id is checked before any URL is built. It is spliced into curl URLs, so it must be ^orgapache[a-z0-9]+-[0-9]+$ or snapshots. The recipes stay GETs only.
  • The trusted-hardware attestation fails closed. It needs automated signing, Step 9 PASS, the source identical, every declared convenience artefact positively identical (an artefact that was skipped or never rebuilt also withholds it), and --trusted-hardware.
  • Rendered recipes quote every configured value with the shared helpers from perf(release-rc-cut): render the RC command blocks as code, one file per outcome #1604.
  • Literals that release-promote reads are unchanged: "Reproducibility validated on trusted hardware", binaries.differs, and the Step 9 numbering. A test pins the reminder and attestation literals against both skills.

Behaviour I checked against the normative text

  • Nexus checksums: companions need a checksum sibling, but the main jar's own checksum is not checked. This matches tools/asf-nexus/staging-verification.md and the classification rules; only the one-line summary in nexus-staging.md reads looser.
  • jvm_companion_location unset: this behaves as before (not staged), so an ABSENT jar is not a FAIL.
  • A known divergence matches by substring. "Match" was never defined more strictly.

Type of change

  • Skill change — eval fixtures updated below
  • Python package

Test plan

  • prek run --all-files passes on the stack tip.
  • tools/release-management: pytest / ruff / mypy, including a real-load integration test. tools/release-config has 4 new load tests.
  • Evals go from 33 cases in 9 suites to 17 in 8; the 17 kept cases pass 17/17. The step-9 suite and the scripted 6b and 6c cases are now pytest cases named after each former case.
  • The baseline on main was 32/33. step-6c case-3 failed there, and that case's fixture now carries the real nexus output, including the existence-200 line and .sha1 siblings its "otherwise complete" scenario describes.

Was generative AI tooling used to co-author this PR?
  • Yes — Claude Code (Opus 5). I reviewed the diff and the test results; commits carry Generated-by:.

🤖 Generated with Claude Code

https://claude.ai/code/session_01GTPsZ5aEE5bUVYqp47Dpvv

@potiuk potiuk added family:tools tools/* capability:triage Sweep + classify + propose disposition family:release-management release-* skills substrate:release Tool substrate: release-artefact helpers (reproducible archive build, lint, comparison) labels Oct 13, 2026
@potiuk
potiuk added this pull request to stack #1620 October 13, 2026 14:14
… results as code

release-verify-rc ran the release-verify checks and then had the agent
turn the configuration into their arguments and classify what came
back, by hand, from ~10k tokens of prose. tools/release-management now
does it, reading saved files only (no network, no gpg, nothing run):

- `args` emits the exact release-verify invocations for Steps 1-3, 5-8
  and `all` from release-build.md: --expect / --expect-optional per
  artefact, the digest set (never md5), --prohibit / --accept,
  validators, the manifest and the unpacked directory.
- `rat` (Step 4): exit code and report → PASS / FAIL with the
  unapproved files, or SKIP + WARN when no RAT config exists.
- `source-tree` (Step 7): REVIEW resolved from validator exit codes.
- `jvm` (Step 6b): the gate, jvm_digest_set, --podling from a root
  DISCLAIMER, ABSENT → FAIL only under `staged`, findings and the
  per-companion `gpg --verify` line.
- `nexus` (Step 6c): the gates, the staging-repo id shape
  (`orgapache<project>-NNNN` or `snapshots`, checked before any URL is
  built, because the id is spliced into curl URLs), the read-only
  recipe, and every classification rule over the saved probe output.
- `repro` (Step 9): mandatory / skip, the RM-key vs ci-automated source
  table, SWHIDs compared without qualifiers, per-mode binary rules,
  `binaries.differs`.
- `report` (Step 10): the report and --post-to comment from
  `release-verify verdict`, with the voter-obligation reminder and the
  trusted-hardware attestation only when every condition holds.

The trusted-hardware attestation fails closed: it needs automated
signing, Step 9 PASS, the source identical and every declared
convenience artefact positively `identical` (one that was skipped or
never rebuilt withholds it too), and --trusted-hardware. Every rendered
recipe quotes its configured values with the shared shell helpers.

release-config's `load --skill verify-rc` now exposes the
required/optional artefact marker, binary-exclude globs, RAT paths,
validators and JVM keys the arguments need (existing fields unchanged).

SKILL.md becomes a router (9,936 → 6,280 tokens, 684 → 466 lines): the
36 failure-mode rows move into per-step files, and the JVM, Nexus and
reproducibility siblings keep their policy plus the command to run.
Step numbering ("Step 9"), the "Reproducibility validated on trusted
hardware" literal and `binaries.differs`, which release-promote reads,
are unchanged, and a test pins the reminder and attestation literals.

Evals: 33 cases in 9 suites → 17 in 8. The step-9 suite and the
scripted 6b / 6c cases become pytest cases named after each former
case; the judgement cases stay, their fixtures now carrying the real
tool output.

Generated-by: Claude Opus 5
Claude-Session: https://claude.ai/code/session_01GTPsZ5aEE5bUVYqp47Dpvv
@potiuk
potiuk force-pushed the perf/release-mgmt-2-verify-rc branch from a7d855d to 1fa7801 Compare October 13, 2026 15:23
from dataclasses import dataclass, field
from typing import Any

from release_management.shell import dq as dq # re-exported for the step modules
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

capability:triage Sweep + classify + propose disposition family:release-management release-* skills family:tools tools/* substrate:release Tool substrate: release-artefact helpers (reproducible archive build, lint, comparison)

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants