perf(release-prepare): plan, prep, post and the signing setup as code

docs/labels-and-capabilities.md

@@ -372,7 +372,7 @@ or a contract-free mix of substrates (e.g. `tools/spec-inventory` is
 | [`tools/vendor-neutrality-score`](../tools/vendor-neutrality-score/) | `substrate:framework-dev` + `substrate:analytics` | Deterministic vendor-neutrality score — reads each contract tool's `**Kind:**` / `**Vendor:**` metadata and scores per-contract + per-skill neutrality (`analytics`); backs the score block in [`docs/vendor-neutrality.md`](vendor-neutrality.md) |
 | [`tools/spec-validator`](../tools/spec-validator/) | `substrate:framework-dev` | Spec-frontmatter and body-section validator — counterpart to `skill-and-tool-validator` for `tools/spec-loop/specs/` |
 | [`tools/release-config`](../tools/release-config/) | `substrate:release` | `release-config` — loads the release-management config (`<project-config>` layered lookup, defaults, derived values) and runs each `release-*` skill's deterministic Step 0 pre-flight checks as JSON; used by every `release-*` skill's Step 0 and the Step 1 metadata of `rc-cut`, `vote-draft`, `announce-draft`, `promote` and `verify-rc` |
-| [`tools/release-management`](../tools/release-management/) | `substrate:release` | `release-management` — renders what the `release-*` skills hand the release manager (paste-ready command blocks, drafts, planning-issue comments) from `release-config load`, refusing a `dist/release/` staging target or an MD5 / SHA-1 digest in code; runs nothing. Used by `rc-cut` Steps 0 and 2–4, `verify-rc` Steps 1–10, `vote-draft` Steps 0, 2 and 3 (verify-rc gate, `[VOTE]` body, planning-issue comment), `vote-tally` Steps 0–3 (pre-classification, the tally, the `[RESULT] [VOTE]` body), `promote` Steps 0 and 2, and `announce-draft` Steps 0, 2 and 3 |
+| [`tools/release-management`](../tools/release-management/) | `substrate:release` | `release-management` — renders what the `release-*` skills hand the release manager (paste-ready command blocks, drafts, planning-issue comments) from `release-config load`, refusing a `dist/release/` staging target or an MD5 / SHA-1 digest in code; runs nothing. Used by `rc-cut` Steps 0 and 2–4, `verify-rc` Steps 1–10, `vote-draft` Steps 0, 2 and 3 (verify-rc gate, `[VOTE]` body, planning-issue comment), `vote-tally` Steps 0–3 (pre-classification, the tally, the `[RESULT] [VOTE]` body), `promote` Steps 0 and 2, `announce-draft` Steps 0, 2 and 3, and `prepare` Steps 0, 1, 2, 14 and A (train coverage, the planning issue, the changelog and its coverage, the 2e archive review, the prep and post-release PRs, the automated-signing gate and workflow) |
 | [`tools/reproducible-archive`](../tools/reproducible-archive/) | `substrate:release` | `repro-archive` — build, lint and compare reproducible source archives from a git ref (`git archive` + `.gitattributes` `export-ignore`, every reproducible-builds.org archive rule applied); used by `release-rc-cut` Step 2/2b and `release-verify-rc` Step 9 |
 | [`tools/maven-artifact-verify`](../tools/maven-artifact-verify/) | `substrate:release` | `maven-artifact-verify` — verify locally staged JVM RC artefacts: POM ALv2 licence / developers / scm (with parent-POM inheritance resolution), podling incubation disclaimer in `<description>`, companion `-sources.jar` / `-javadoc.jar` presence with their own signatures and checksums; used by `release-verify-rc` Step 6b |
 | [`tools/release-verify`](../tools/release-verify/) | `substrate:release` | `release-verify` — deterministic, read-only checks of a staged RC (inventory, signatures against the project KEYS only, checksums, NOTICE/LICENSE, prohibited binaries, dangling symlinks, version strings) and the verdict roll-up, each as JSON; used by `release-verify-rc` Steps 1–3, 5–8 and 10 |

plugins/magpie-release-management/skills/prepare/SKILL.md

@@ -21,9 +21,9 @@ when_to_use: |
   (default), `prep`, `post`, `automated-signing`.
 argument-hint: "[prep | post] <version> [--review-archive] | automated-signing"
 capability: capability:resolve
-surface_hash: sha256:43e928f52996ee1b
+surface_hash: sha256:9380580d6421d8b4
 license: Apache-2.0
-measured_tokens: 5592
+measured_tokens: 5095
 ---
 
 <!-- SPDX-License-Identifier: Apache-2.0
@@ -178,30 +178,8 @@ Local modifications go in the override file; framework changes go via PR to `apa
 
 ## Prerequisites
 
-- **`<project-config>/release-trains.md` readable** — identifies the release train, release branch, and release manager for `<version>`.
-- **`<project-config>/release-management-config.md` readable** —
-  provides `release_branch_base`, `version_manifest_files`, `category_x_dependencies`, and `release_planning_issue_template`.
-- **`<upstream>` access** — read access to the upstream repo to list merged PRs via `gh pr list` since the previous release tag.
-
-For Step 2 (`prep`):
-- **Planning issue open and labelled `release-planning`** — confirms Step 1 completed.
-  The skill can also accept `--planning-issue <url>`.
-
-For Step 14 (`post`):
-- **Planning issue labelled `announced`** — confirms Steps 10–11 completed. Accepted via `--planning-issue <url>`.
-
-For Step 2's source-archive review (`prep`, Step 2e) — optional:
-- **`<project-config>/release-build.md § Source archive`** — `source_archive_method` (default `git-archive`) and `export_ignore_reviewed`.
-  Absent file or key = the review has not happened yet, which is exactly when the sub-step runs.
-- **A local clone of `<upstream>`** at the release branch tip (the resolved `user.md` clone path) —
-  the review lists what `git archive` would ship from *that* tree.
-
-For Step A (`automated-signing`, 🪶 ASF-specific):
-- **The project's organization offers it.** The organization manifest key `release_process.automated_signing`,
-  resolved `project.md` → organization manifest → framework default (not offered), is set;
-  of the shipped organizations only [`organizations/ASF/organization.md`](../../../../organizations/ASF/organization.md) sets it.
-  The sub-command is not offered otherwise.
-- **`release-build.md § Reproducibility checks`** — `reproducibility_source: on` and `reproducibility_binaries: byte-identical` (or no binaries).
+Config files, upstream access, the planning-issue labels each sub-command needs, the local clone for the 2e review, and the 🪶 ASF-only Step A conditions:
+[`prerequisites.md`](prerequisites.md).
 
 ---
 
@@ -227,31 +205,44 @@ passing the arguments as the RM typed them:
 ```bash
 uv run --project <framework>/tools/release-config release-config preflight \
   --skill prepare [prep|post|automated-signing] [<version>] \
-  [--release-branch <branch>] [--previous-tag <tag>]
+  [--release-branch <branch>] [--previous-tag <tag>] > <scratch>/prepare-preflight.json
 ```
 
 It covers the sub-command, the version format (see *Inputs*), the automated-signing gate, the required config keys and `release-trains.md`,
 and prints `{"ok", "blockers", "warnings", "values"}`.
 Each `blockers` entry is a hard blocker; surface it as written.
 `automated-signing` is 🪶 ASF-specific: when the tool blocks it (the organization does not offer it), do not describe the flow further.
 Surface `warnings` and carry on.
-Copy `sub_command`, `version`, `release_branch_base` and `previous_tag` from `values`;
-fill `previous_tag` yourself when it is detectable at pre-flight.
+The verdict below copies `sub_command`, `version`, `release_branch_base` and `previous_tag` from `values`;
+when `previous_tag` is detectable at pre-flight, pass it as `--previous-tag`.
 
 Then check what the tool cannot see:
 
-1. **Train record exists for `<version>`.** One of `values.release_lines` (the release lines `release-trains.md` lists) covers `<version>`.
+1. **Train record exists for `<version>`.** One of `values.release_lines` (the release lines `release-trains.md` lists) covers `<version>`;
+   the verdict below decides it from the train patterns (`2.x`) and versions those lines name.
 2. **For Step 2 (`prep`):** Planning issue found and labelled `release-planning`.
    Either `--planning-issue <url>` was passed or the skill finds a `release-planning` issue on `<upstream>` matching `<version>` in its title.
 3. **For Step 14 (`post`):** Planning issue found and labelled `announced`.
 4. **`<upstream>` access.** `gh pr list --repo <upstream>` succeeds.
 5. **Drift check** — the generated pre-flight block reports snapshot drift.
 6. **Override consultation** — see *Adopter overrides* above.
 
+Fold the facts into the verdict with the [`release-management`](../../../../tools/release-management/README.md#prepare) tool, passing each flag only when its fact holds:
+
+```bash
+uv run --project <framework>/tools/release-management release-management prepare verdict \
+  --preflight <scratch>/prepare-preflight.json [--planning-issue <url> --planning-label <label> ...] \
+  [--gh-access] [--previous-tag <tag>] [--train-confirmed]
+```
+
+The verdict is fail-closed: without `--gh-access` or (for `prep` / `post`) a labelled `--planning-issue` it blocks.
+It keeps the tool's `blockers` as written and adds one per failed check; surface each as written.
+When no release line names a train or version at all, or the lines name only other trains, it blocks until you read `values.release_lines` and pass `--train-confirmed` for the one that is the train record.
+Load each file its `read` list names ([`automated-signing-ineligible.md`](automated-signing-ineligible.md)).
 If any check fails (and is not overridden), stop and surface what is missing
 with the exact config key name that is missing or the exact condition that blocks progress.
 
-Return ONLY valid JSON with this structure:
+Return ONLY valid JSON with this structure (the verdict's fields, without `train_covered` and `read`):
 
 ```json
 {
@@ -373,29 +364,4 @@ The AI-driven part ends with a hand-back artefact containing:
 
 ## References
 
-- [`docs/release-management/process.md`](../../../../docs/release-management/process.md) —
-  Steps 1, 2, and 14 context.
-- [`docs/release-management/spec.md`](../../../../docs/release-management/spec.md) —
-  `release-prepare` per-skill specification.
-- [`docs/release-management/reproducibility.md`](../../../../docs/release-management/reproducibility.md) —
-  the source-archive review (2e) buckets and rationale, and the 🪶
-  ASF-specific automated-signing setup (Step A).
-- [`<project-config>/release-build.md`](../../../magpie-setup/templates/release-build.md) —
-  `§ Source archive` (`source_archive_method`, `export_ignore_reviewed`)
-  and `§ Reproducibility checks`.
-- [`projects/_template/workflows/release-candidate.yml`](../../../magpie-setup/templates/workflows/release-candidate.yml) —
-  the workflow template Step A renders.
-- [`tools/reproducible-archive`](../../../../tools/reproducible-archive/README.md) —
-  `repro-archive build` / `compare` used for the before/after listing.
-- [`<project-config>/release-management-config.md`](../../../magpie-setup/templates/release-management-config.md) —
-  adopter keys this skill reads (`release_branch_base`,
-  `version_manifest_files`, `category_x_dependencies`,
-  `release_planning_issue_template`).
-- [`<project-config>/release-trains.md`](../../../magpie-setup/templates/release-trains.md) —
-  release train identity and RM roster.
-- `release-keys-sync` (proposed) — downstream Step 3.
-- `release-rc-cut` (proposed) — downstream Steps 4–5.
-- [ASF release policy](https://www.apache.org/legal/release-policy.html), canonical.
-- [ASF licensing-howto](https://www.apache.org/legal/resolved.html) —
-  Category-A/B/X dependency rules; Category-X is a hard stop for this
-  skill.
+The process and spec sections, the config templates, the workflow template and the tools this skill uses: [`references.md`](references.md).

plugins/magpie-release-management/skills/prepare/archive-review.md

@@ -0,0 +1,74 @@
+<!-- SPDX-License-Identifier: Apache-2.0
+     https://www.apache.org/licenses/LICENSE-2.0 -->
+
+# Step 2e — the source-archive contents review
+
+Loaded from [`prep.md`](prep.md) § 2e when `prepare archive-mode` printed `mode: "full"` or `"drift"`.
+
+With `source_archive_method: git-archive` (the default in `release-build.md § Source archive`)
+the source artefact is an export of the tagged tree that honours `.gitattributes` `export-ignore`.
+The attributes are read from the tree being archived, so they have to be committed **before** the RC tag —
+which is why this review lands in the prep PR and why `release-rc-cut` blocks while it is outstanding.
+Full rationale and the classification buckets:
+[`docs/release-management/reproducibility.md` § The first-release `.gitattributes` review](../../../../docs/release-management/reproducibility.md#the-first-release-gitattributes-review).
+
+**When the full review runs:** `export_ignore_reviewed` is unset in `release-build.md`, or `--review-archive` was passed,
+or `source_archive_method` is `git-archive` and the file has no `§ Source archive` at all (`mode: "full"`).
+**Otherwise** run only the drift check (`mode: "drift"`, below).
+
+This is an **education step**: the operator ends up knowing why every top-level path ships or does not.
+Do not guess; show, classify, explain, and ask.
+
+1. **List what would ship today** from the local clone at the release branch tip, and what is tracked:
+   run the `commands` `archive-mode` printed there (read-only `git` listings into `$TMPDIR`).
+   With no local clone (`user.md` names none), stop 2e and ask the RM to set the clone path or clone and rerun `prep`.
+
+2. **Classify every top-level entry** into one bucket and say which:
+
+   ```bash
+   prepare archive-review --tracked "$TMPDIR/tracked.txt" --gitattributes <clone>/.gitattributes \
+     --symlinks "$TMPDIR/symlinks.txt" --load <scratch>/prepare-load.json [--voter-input <path> ...]
+   ```
+
+   `entries` is the bucket table: *ship* (source, docs, build descriptors, lock files, `README*`),
+   *ship, never excludable* (`LICENSE`, `NOTICE`, `DISCLAIMER`, `licenses/`), *ship, input to voter checks* (RAT excludes, in-tree validators),
+   *project's call* (`.asf.yaml`, `doap_*.rdf`, `.gitignore`), the *exclude* buckets (VCS metadata, CI / bot config, editor / IDE,
+   lint config not needed to build, release-tooling scratch), and the *agent-view dirs* (`.claude/`, `.agents/`, `.kiro/`, `.cursor/`).
+   `.github/` and the agent-view dirs are classified one level down (`inside`):
+   part of a directory may ship (issue templates a shipped skill links to) while the rest is excluded,
+   and an agent-view relay dir (its symlinks chain through another symlink) is excluded while a single-hop canonical view stays when shipped files link into it.
+   Every `needs_rm` entry — *project's call*, large assets, anything `unclassified` — is the RM's decision: explain the trade-off and ask.
+
+3. **Check references before proposing any exclusion**: run the printed `refs_command` (one `git grep -l -F` per proposed entry over tracked files)
+   and rerun `archive-review` with `--refs "$TMPDIR/refs.txt"`.
+   `refs_unchecked` lists every proposed path the refs file does not cover (one added later with `--exclude`, say);
+   while it is non-empty, `gitattributes_block` stays empty and `refs_command` checks exactly those paths, appending to the same refs file: run it and rerun with that file.
+   A path that a shipped file links to must not be excluded, or `release-verify-rc` Step 7 fails the RC on a dangling reference:
+   `refused` names the referrers; offer the alternative (keep it, or repoint the reference).
+   A reference from a file that is itself excluded does not count, unless that file's own exclusion is refused.
+   `stripped_symlink_targets` is every committed symlink whose target would be stripped,
+   and `symlink_chains` every shipped symlink that points at another symlink (safe extractors reject chains); flag each.
+
+4. **Propose the entries** — `gitattributes_block` holds them root-anchored (`/.pre-commit-config.yaml`) for root-only files,
+   directory form (`.idea/`) for directories, one rationale comment per entry (`--reason <path>=<why>` replaces it) —
+   and show the before/after listing diff with `before_after_commands` (`removed` = exactly what the review strips).
+   Walk the RM through each proposed entry; each one is confirmed or dropped individually,
+   and the final render takes the RM's answers as `--exclude <path>` (added) and `--keep <path>` (dropped).
+   Never exclude `LICENSE`, `NOTICE`, `DISCLAIMER`, a build descriptor, the RAT excludes, or a referenced path, even if asked — say why and keep it
+   (`refused` already holds them, and any glob pattern: name each path instead).
+   `violations` lists every entry the committed `.gitattributes` already has that strips one of those files (`matches` names them):
+   the review is not clean while one remains.
+   Show each to the RM, remove the line from `.gitattributes` in the prep PR,
+   and record it as `--archive-review proposed` with `--archive-entry "<entry>=removed: <why>"`.
+
+5. **Record the decision.** `.gitattributes` joins the prep PR file set (2f),
+   and the prep PR sets `export_ignore_reviewed: <version>` in `release-build.md § Source archive` so the full review does not repeat:
+   pass `--archive-review proposed` and one `--archive-entry "<entry>=<reason>"` per confirmed entry to `prep-pr`.
+   If the RM confirms an existing `.gitattributes` unchanged, still set the marker (`archive_review: "confirmed-existing"`).
+
+**Drift check (later releases).** Run the `commands` `archive-mode` printed
+(`git diff --name-only <previous-tag> HEAD` into `$TMPDIR/changed.txt`, plus `tracked.txt` and `symlinks.txt`),
+then `archive-review` with `--changed "$TMPDIR/changed.txt"` and the same inputs.
+A `violations` entry blocks the skip in the drift check too.
+Top-level entries added since the last reviewed tag that fall in an *exclude* bucket are surfaced as candidates with the same confirm-each flow;
+nothing new → `archive_review: "skipped"` with the note *"no new top-level paths since `<previous-tag>`"*.

plugins/magpie-release-management/skills/prepare/automated-signing-drafts.md

@@ -0,0 +1,23 @@
+<!-- SPDX-License-Identifier: Apache-2.0
+     https://www.apache.org/licenses/LICENSE-2.0 -->
+
+# Step A2–A3 — the Infra ticket and the Security Team notification (🪶 ASF-specific)
+
+Loaded from [`automated-signing.md`](automated-signing.md) once A1 is `eligible`.
+Both are drafts: the skill never files the ticket, never sends the mail, and never adds key material anywhere.
+
+## A2 — Draft the Infra Jira ticket
+
+Draft (never file) an `INFRA` ticket titled *"CI release signing key for Apache <PROJECT>"* that:
+- requests the key per the policy (4096-bit RSA, signing-only, private half held by infra-root, public block to `KEYS`, encrypted revocation certificate to the project's private repo);
+- names the workflow (`ci_release_workflow`) and the staging target (ATR via `apache/tooling-actions/upload-to-atr`, pinned by commit SHA);
+- **highlights the trusted-hardware validation step** — `release-verify-rc` Step 9 with `--trusted-hardware`,
+  `repro-archive compare --require-identical` for every artefact, recorded on the planning issue, gating `release-promote`;
+- references the background ticket in `release_process.automated_signing.key_request_background`.
+
+## A3 — Draft the Security Team notification
+
+Draft (never send — [spec § Boundary 3](../../../../docs/release-management/spec.md#boundary-3-agent-never-sends-mail-to-dev-users-announce))
+a mail to `release_process.automated_signing.approval_body` (`security@apache.org`) from the RM,
+pointing at the ticket, the workflow PR and the validation step, asking for the approval the policy requires before the workflow is used.
+Plain text, real links, per the repository's email rules.