Skip to content

test(conformance): report feedback failure rules - #1552

Draft
overtrue wants to merge 2 commits into
mainfrom
p1/1549-signing-feedback
Draft

overtrue wants to merge 2 commits into
mainfrom
p1/1549-signing-feedback

Conversation

@overtrue

@overtrue overtrue commented Oct 29, 2026 •

Copy link
Copy Markdown
Collaborator

Summary

When a feedback case fails, the assertion currently reports only the exit status, which does not identify the failing rule. Report the selected case ID, verdict, phase and deny rule names without emitting diagnostic messages or pointers. This is diagnostic instrumentation for #1549; it does not reproduce or fix the intermittent failure, and does not close the issue.

Verification

$ cargo fmt --all --check
real 2.44
$ CARGO_BUILD_JOBS=2 CARGO_INCREMENTAL=0 cargo clippy --workspace --all-targets -- -D warnings
Finished dev profile; real 119.60
$ CARGO_BUILD_JOBS=2 CARGO_INCREMENTAL=0 cargo test --workspace
exit 0; real 619.23
$ CARGO_BUILD_JOBS=2 CARGO_INCREMENTAL=0 cargo xtask verify --crate rustfs-gateway-conformance
build compiled 50 crate(s) in 25.22s outside the budget
passed in 23.51s; real 49.32

All commands used an executor-owned target. Workspace wall time includes a cold build and is not a CI gate measurement. The original signing feedback case passed in isolation. A temporary case status mismatch made the real feedback assertion fail with expect/status, and was restored. Each of the three non-disclosure assertions and the exact summary assertion was separately mutated and observed failing. The earlier full library run overlapped an assertion-order edit and is not counted as a gate. Separate earlier candidate workspace failures in #1424 remain unexplained.

Role Verdicts

  • simplicity-adversary: attacked target-data disclosure, failing-rule provenance and actual assertion wiring — no break found.

IDs come from the local corpus and rule names from static producers; their String types do not enforce that invariant for future producers.

Breaking Change

  • BREAKING — this PR touches a protected file or changes a public contract.

Checklist

  • The four-command gate passed (fmt, clippy, test, cargo xtask verify)
  • New or changed public API has rustdoc
  • Negative test cases outnumber positive ones
  • Every new assertion was mutated — the implementation was broken on purpose and the assertion went red. The PR description names which ones
  • No Protected File touched; if one was, the PR description contains BREAKING and a migration path
  • No unsafe introduced
  • No inventory / linkme introduced
  • No #[non_exhaustive] added to a DTO
  • No s3s code copied; behavioural evidence is URL + self-written summary
  • No credentials, signatures, or expected-signature values reachable from logs or error bodies
  • No plan document, analysis report, or working note committed
  • Commits follow Conventional Commits; PR title ≤72 characters
  • A Handoff comment has been appended to the issue

@overtrue

Copy link
Copy Markdown
Collaborator Author

Role Verdicts

  • simplicity-adversary: attacked target-data disclosure, deny-only selection, actual feedback assertion wiring and rule provenance. No break found in the current runner paths: case IDs are loaded from local corpus metadata or file stems; production rule names derive from literals, static helper arguments, enum variants or static schema keywords. The diagnostic message and pointer are not rendered. The String types do not enforce these provenance assumptions for future producers.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant