audit: record the adversarial mutation-test scan of 5c165a8 - #46
Merged
Merged
Conversation
Campaign bookkeeping, split out of #37 so each concern stands on its own PR. One record of the whole-repo adversarial mutation-test scan at `5c165a880ac105d6c038cee096500f76849a2190`: 59 behaviours probed, 28 tests before and 38 after, and the four issues it filed (#33, #34, #35, #36). This is a scan log, not coverage and not a behaviour change. It closes no issue and is tied to none: it exists so a later scan can tell what was already probed, at which commit, with which tool version, rather than re-deriving it. `audit/**/` is already covered by REUSE.toml, so the file needs no SPDX header of its own. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
|
Warning Review limit reachedNext included review available in 50 minutes. View limit detailsLimit details: You’ve used the included review currently available. You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository. Review configuration: ⚙️ Run configurationConfiguration used: Organization UI Review profile: ASSERTIVE Plan: Advanced Run ID: 📒 Files selected for processing (1)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
This was referenced Sep 20, 2026
`rainix-static mutation-ledger` requires every SHA a record names to be an
ancestor of HEAD, and red-flagged this record on a branch off `main`:
ERROR: audit/mutation-test-scans.json[0].testsAfterCommit:
70d4e18 is not an ancestor of HEAD
That is correct and the record was wrong, not the gate. `70d4e18` is a commit on
the branch behind #37; it was an ancestor only while the scan record and the
tests shared one branch. Now that the scan record stands alone -- one PR per
issue -- no commit on this history carries those tests, so the field has no
truthful value here.
The field is optional by design: the gate validates `testsAfterCommit` when
present and never requires it, precisely so that a run whose test tree cannot be
named is not pushed into inventing a SHA. Its module doc is explicit that "the
only way to go green would be to invent a SHA, which is the fabricated evidence
this gate exists to catch."
So it is dropped rather than repointed. Everything the gate does require is
unchanged and valid: `commit` 5c165a8 is an ancestor of this branch, the
timestamp is well-formed, `tool` matches, and `scope`/`summary` are intact.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Campaign bookkeeping, split out of #37 so each concern stands on its own PR.
Branched off
main, depends on nothing, and is independently mergeable.What
One file:
audit/mutation-test-scans.json, a record of the whole-repoadversarial mutation-test scan at
5c165a880ac105d6c038cee096500f76849a2190— 59 behaviours probed, 28 testsbefore and 38 after, and the four issues the scan filed (#33, #34, #35, #36).
Why
So a later scan can tell what was already probed, at which commit, with which
tool and skill version, instead of re-deriving it. Without the record the next
run has no way to distinguish "this behaviour was probed and killed" from "this
behaviour was never looked at."
Scope
This closes no issue and is tied to none. It is a scan log: no source
change, no test change, no behaviour change, nothing executable. It is being
sent on its own precisely because it is not coverage and should not ride along
with tests that are.
audit/**/is already covered byREUSE.toml, so the file needs no SPDX headerof its own and the
legaljob is unaffected.QA
executable code. There is no behaviour for a test to discriminate, so a
discriminating test here would be meaningless rather than merely absent.
suite. Mutating any byte of a scan record changes no program behaviour and no
assertion, so every mutant would be equivalent by construction.
adversarial-mutation-test(skill 0.35.0) run over5c165a880ac105d6c038cee096500f76849a2190that produced it — the commit underscan, the tests-before/after counts, and the issue numbers it filed, which are
checkable against Frozen-record check cannot see a release dropped from the declaration when two releases share bytecode #33, The candidate source anchor can be made self-comparing, and the suite stays green #34, Currency check cannot detect a generator that has stopped emitting a file #35 and .gas-snapshot is stale and no CI job regenerates or checks it #36 as actually filed on this repo.
output and the ruling is one PR per issue, so it is sent alone rather than
attached to unrelated coverage. The two concerns it used to ship with in test: cover the deploy declaration's metadata #37
are now test: cover the deploy declaration's metadata #37 itself (the declaration-metadata tests) and test: give the open-salt entry points an oracle independent of the library #45 (the open-salt
oracle; the issue it closes is The open-salt entry points have no test oracle independent of the library under test #43).
The
testsAfterCommitfield, and why it is not hereThe first push of this branch went red on
rainix-sol / static, and thefailure was real rather than incidental:
rainix-static mutation-ledgerrequires every SHA a record names to be anancestor of
HEAD, because "what changed since the last mutation run" isotherwise computed against a tree this history does not contain.
70d4e18is acommit on the branch behind #37. It was an ancestor only while the scan record
and the tests shared one branch — which is exactly the bundling this split
undoes.
The field has been dropped, not repointed. It is optional by design: the gate
validates it when present and never requires it, and its module doc says why —
"the only way to go green would be to invent a SHA, which is the fabricated
evidence this gate exists to catch." Repointing it at a commit on this branch
would be that invention.
Everything the gate requires is present and valid:
commit5c165a8is anancestor of this branch (verified with
git merge-base --is-ancestor), thetimestamp is well-formed
YYYY-MM-DDTHH:MM:SSZ,toolmatches the expectedadversarial-mutation-test, andscope/summaryare intact.summary.testsAfter: 38is still accurate for the run: 28 before, plus the 5declaration tests on #37 and the 5 open-salt tests on #45.
QA
executable code. There is no behaviour for a test to discriminate, so a
discriminating test here would be meaningless rather than merely absent. The
file's own correctness is gated by
rainix-static mutation-ledgerin CI,which this PR is now green on.
test suite. Mutating any byte of a scan record changes no program behaviour
and no assertion, so every mutant would be equivalent by construction.
adversarial-mutation-test(skill 0.35.0) run over5c165a880ac105d6c038cee096500f76849a2190that produced it — the commit underscan, the tests-before/after counts, and the issue numbers it filed, which are
checkable against Frozen-record check cannot see a release dropped from the declaration when two releases share bytecode #33, The candidate source anchor can be made self-comparing, and the suite stays green #34, Currency check cannot detect a generator that has stopped emitting a file #35 and .gas-snapshot is stale and no CI job regenerates or checks it #36 as actually filed on this repo. The
ledger's shape is checked against
rainix-static mutation-ledger, which isindependent of the record.
output and the ruling is one PR per issue, so it is sent alone rather than
attached to unrelated coverage. The two concerns it used to ship with in test: cover the deploy declaration's metadata #37
are now test: cover the deploy declaration's metadata #37 itself (the declaration-metadata tests) and test: give the open-salt entry points an oracle independent of the library #45 (the open-salt
oracle; the issue it closes is The open-salt entry points have no test oracle independent of the library under test #43).
Checks
The diff contains no Solidity, so nothing here can move the build or the suite.
Verified anyway, locally in the pinned rainix
sol-shellon this branch offcurrent
main: 28 non-chain tests pass, 0 failed —main's count exactly,unchanged, as a file with no executable content should leave it.
forge fmt --checkpasses andgit status --porcelainis empty afterforge build, so thegit-cleancurrency job is unaffected.audit/**/isalready covered by
REUSE.toml, solegalis too.🤖 Generated with Claude Code