Skip to content

fix(setup): detect the security list on adopt instead of assuming one - #1623

Open
shahar1 wants to merge 2 commits into
apache:mainfrom
shahar1:setup-detect-security-list
Open

shahar1 wants to merge 2 commits into
apache:mainfrom
shahar1:setup-detect-security-list

Conversation

@shahar1

@shahar1 shahar1 commented Oct 13, 2026

Copy link
Copy Markdown
Collaborator

Summary

  • setup adopt (via install.md Step 4b) filled in security_list as security@<project>.apache.org for every ASF project. Not every project has that list. Apache Mahout doesn't, so adopting Magpie there wrote a project.md that sends security reports to an address that doesn't exist.
  • Adopt now checks for the list before asking. It looks in the repo's own docs (SECURITY*, CONTRIBUTING*, README*, docs/) and on https://www.apache.org/security/projects.html.
  • Adopt then always asks the maintainer where security reports should go. security@<project> is the recommended option only when the check found it. Otherwise private@<project> is recommended. The maintainer can also enter a custom address.

Type of change

  • Skill change (.claude/skills/<name>/) — eval fixtures updated below
  • Tool / bridge contract (tools/<system>/*.md)
  • Python package (tools/*/ with pyproject.toml)
  • Groovy reference impl
  • Cross-cutting (RFC, AGENTS.md, sandbox, privacy-LLM)
  • Documentation (docs/, README.md, CONTRIBUTING.md)
  • Project template (projects/_template/)
  • CI / dev loop (prek, workflows, validators)
  • Other:

Test plan

  • prek run passes on every changed file (not --all-files)
  • For Python packages touched: uv run pytest / ruff check / mypy passes
  • For Groovy bridges touched: command-line invocation tested end-to-end
  • For skill changes: eval suite passes for the affected skill. The new suite passes 3/3 with skill-eval --cli "claude -p" --exact tools/skill-evals/evals/setup/adopt-security-list/fixtures/.
  • For skill behaviour changes: a new or updated eval fixture is included in this PR. setup/adopt-security-list has three cases: a dedicated list is found (recommend security@), no dedicated list (recommend private@), and a non-ASF project (adopt doesn't ask).
  • Other:

RFC-AI-0004 compliance

  • HITL — the maintainer always chooses the destination; nothing is assumed.
  • Sandbox — one read-only fetch of a public ASF page.
  • Vendor neutrality — the new behaviour is limited to organization: ASF.
  • Conversational + correctable — agentic-override path documented if behaviour is adopter-tunable
  • Write-access discipline — no autonomous outbound messages; drafts only, sent on confirmation
  • Privacy LLM — private content does not reach a non-approved LLM; redactor invoked where needed

Linked issues

Found while adopting Magpie in apache/mahout#1519.

Notes for reviewers (optional)

  • The rule is defined once, in adopt.md Step 4b. install.md Step 4b no longer has a default and points to it.
  • On the ASF page, projects without their own list appear as security@apache.org, so a project shown that way counts as having no dedicated list. I left security@apache.org out of the options: the template treats it as the org-level forwarding address, not a per-project list. A maintainer who wants it can still enter it as the custom address.

🤖 Generated with Claude Code

Not every ASF project has a dedicated security@<project>.apache.org
list (Apache Mahout, for one), yet Step 4b defaulted security_list to
it, so adopt could commit a project.md that routes reports to a list
that does not exist.

Adopt now looks for the list in the repo's docs and on
https://www.apache.org/security/projects.html, then asks where security
reports go: security@ is recommended only when found, private@
otherwise.

Generated-by: Claude Code (Opus 5.5)
Generated-by: Claude Code (Opus 5.5)
@shahar1
shahar1 requested a review from potiuk October 13, 2026 14:59

@potiuk potiuk left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good catch. Detecting the list instead of inventing security@<project>.apache.org is clearly right. But recommending private@ as security_list has a privacy side effect that needs handling before this lands.

private@ as security_list skips the private-list LLM gate (adopt.md:343)

security-issue-import treats <security-list> as its inbox. It runs privacy-llm-check without --reads-private-list, because the security list holds only reports, under the third-party-PII redaction contract (plugins/magpie-security/skills/issue-import/SKILL.md, Step 0 item 4). AGENTS.md keeps PMC-private content under a stricter rule:

<private-list> content never reaches a non-approved LLM.

With security_list = private@<project>.apache.org, the import sweep reads the entire PMC list (votes, nominations, conduct threads) as candidate reports, under the weaker gate. Either the option needs to carry the private-list gate with it, for example by having the security skills pass --reads-private-list whenever security_list == private_list, or the recommendation for a project without a dedicated list should be to ask Infra for one, with private@ offered only as an explicitly flagged fallback. Details inline.

Smaller observations

  • The "evidence is unclear" branch has no eval case. A fixture where SECURITY.md names security@foo.apache.org but the ASF page shows the project under security@apache.org would pin which source wins (inline).
  • install.md:1292 (Step 9) still lists security_list among the values "auto-sourced in Step 4b". It no longer is, so on that path it falls through to the generic "ask for absent values" prompt rather than the detect-and-recommend rule. Dropping it from that list and pointing at the adopt.md rule there too would close the gap.

CI is still running.


This review was drafted by an AI-assisted tool and
confirmed by an Apache Magpie maintainer. The findings
below are observations, not blockers; an Apache Magpie
maintainer — a real person — will take the next look at the
PR. If you think a finding is mis-applied, please reply on
the PR and a maintainer will weigh in.

More on how Apache Magpie handles maintainer review:
Contributing guide.

Then one structured question, with these options:

- `security@<project>.apache.org` — recommend it only when step 1 or 2 found it, and say where.
- `private@<project>.apache.org` — the PMC list; recommend it when the dedicated list was not found or the evidence is unclear.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choosing private@ here makes it the import skill's inbox, and security-issue-import runs privacy-llm-check without --reads-private-list. Every PMC-private thread would then be read as a candidate report under the security-list contract. Either make the security skills run the private-list gate when security_list == private_list, or recommend requesting a dedicated security@ list from Infra and keep private@ as an explicitly flagged fallback that names this consequence.

1. **The repo's own docs** — grep `SECURITY*`, `CONTRIBUTING*`, `README*` and `docs/` for `security@<project>.apache.org`.
2. **The ASF's public list** — fetch <https://www.apache.org/security/projects.html>.
A project with a dedicated list appears there with `security@<project>.apache.org`;
one without appears with `security@apache.org`, or not at all.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What wins when the two sources disagree? For example, SECURITY.md names security@foo.apache.org but the ASF page lists Foo under security@apache.org. The prose says recommend private@ when the evidence is unclear, but none of the three fixtures exercises that branch. A fourth case would pin it.

@github-actions github-actions Bot added family:setup setup-* skills family:tools tools/* family:docs Docs, MISSION.md, READMEs capability:platform Framework / agent substrate skills (install, verify, doctor, override, status, setup bootstrap) substrate:framework-dev Tool substrate: build / validate / eval the framework itself labels Oct 14, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

capability:platform Framework / agent substrate skills (install, verify, doctor, override, status, setup bootstrap) family:docs Docs, MISSION.md, READMEs family:setup setup-* skills family:tools tools/* substrate:framework-dev Tool substrate: build / validate / eval the framework itself

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants