Skip to content

Currency check cannot detect a generator that has stopped emitting a file #35

Description

@thedavidmeister

What

The Git is clean job (rainix-copy-artifacts) is the only thing checking that
this repo's committed generated sources are current. Its method is "re-run the
generator, then git diff --exit-code", which cannot detect a generator that has
stopped generating a file. The check reports green, and the generated file goes
on being whatever was last committed.

Verified repro

At 5c165a880ac105d6c038cee096500f76849a2190, with a control.

Control — regeneration normally rewrites the aggregate. Append a marker to
src/lib/LibReleasedSuites.sol, then run the generator:

$ printf '\n// CORRUPTION MARKER\n' >> src/lib/LibReleasedSuites.sol
$ forge script ./script/Build.sol
Script ran successfully.
$ grep -c 'CORRUPTION MARKER' src/lib/LibReleasedSuites.sol
0                     # rewritten, marker gone
$ git status --porcelain
                      # clean

Mutant — remove the one call that emits it, in script/Build.sol:

-        LibRainDeploySnapshot.writeReleasedSuitesAggregate(vm, LibRainDeploySnapshot.LIB_DIR, snapshotContractNames());
+        // aggregate emitter removed

then the same marker and the same generator run:

$ forge script ./script/Build.sol
Script ran successfully.
$ grep -c 'CORRUPTION MARKER' src/lib/LibReleasedSuites.sol
1                     # NOT rewritten — the generator no longer emits this file

The script still exits 0. In a real PR the committed
src/lib/LibReleasedSuites.sol is already correct, so nothing is rewritten,
nothing differs, git diff --exit-code passes and the job is green — with the
aggregate generator dead.

Why it matters here

LibReleasedSuites.releasedSuites() is the whole released side of the
declaration: it is what CloneFactoryDeploySuites.releasedSuites() returns, and
therefore what the chain group and the frozen-record check are handed.

Its CONTENT does not change when a release is cut — it concatenates the
per-contract libs, and this repo has exactly one contract — so a dead emitter is
inert and invisible for as long as that stays true. It stops being inert the
moment a SECOND deployed contract is added: the aggregate would need a new entry,
would not get one, and the new contract's releases would be silently absent from
releasedSuites() with the currency check green throughout. That is the state
the file's own NatSpec says must be impossible:

a release missing from the declaration is a release every check quietly stops
asking about.

Scope

This is a property of the regenerate-and-diff pattern rather than of anything
written in this repo, so it applies to every repo using rainix-copy-artifacts
with committed generated sources. Filed here because this is where it was found
and where the consequence above is concrete.

Possible directions, for triage rather than as a recommendation: have the
generator write to a clean temporary tree and diff that against the committed one
(so a file never written shows as missing), or have script/Build.sol assert it
emitted every file its own declaration names, or delete the generated files
before regenerating in CI so absence is visible.


Found by adversarial mutation testing (skill 0.35.0).

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    adversarialFound by the adversarial half of a mutation-test scanauditFinding from an audit or mutation-test scan

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions