Skip to content

fix(pr-management-stats): keep the parity tests from publishing a gist - #1603

Open
MakerFlyFly wants to merge 1 commit into
apache:mainfrom
MakerFlyFly:fix/pr-stats-tests-no-publish-upstream
Open

MakerFlyFly wants to merge 1 commit into
apache:mainfrom
MakerFlyFly:fix/pr-stats-tests-no-publish-upstream

Conversation

@MakerFlyFly

Copy link
Copy Markdown
Contributor

Summary

  • The JSON parity tests ran dashboard.main() with gist publishing enabled, allowing an authenticated developer to overwrite or create a real gist while both tests passed. Pass --no-publish to dashboard only.
  • Guard gist_scope_available, publish_gist, read_stats_gist_id, and store_stats_gist_id with pytest.fail; ordinary except Exception handling cannot swallow a reached guard, and protection is independent of call order.
  • Preserve existing JSON assertions and production behavior. One test file changes, with 15 added lines.

Type of change

  • Python package (tools/*/ with pyproject.toml)

Test plan

Fresh validation on upstream baseline dadb315caa8b881fc1d2195bef03223dac25915a, Linux/CPython 3.12.12 and pytest 9.1.1.

  • Recording gh replacement, before the fix: the parity tests pass but PATCH an existing gist twice; without session state, they create, PATCH, and record a fake ID. The replacement never invokes real gh.
  • After the fix, both state scenarios pass with zero gh calls and no session-state changes.
  • Removing --no-publish, including with an ordinary exception wrapper around the scope guard, makes both tests fail with expected Failed outcomes.
  • Moving each of the four gist entries ahead of opt-out also fails inside except Exception, with zero underlying sentinel calls and zero gh calls. Probe runs preserve source and state.
  • Complete package pytest: 625 passed, 15 subtests passed; JUnit has 640 entries, zero failures, errors, or skips.
  • Package ruff, format check, mypy and standalone zizmor pass.
  • prek run --all-files passes, including lychee, workspace checks, skill-script tests and spec validation.
  • Normal commit hook passes. Magpie pre-first-pr-check reports ready with zero blocking findings and zero advisory items; the PR template has been checked.

RFC-AI-0004 compliance

  • Write-access discipline — parity tests do not publish using the developer's credentials; all four known gist entries reject unexpected test access.

Linked issues

Closes #1602

Notes for reviewers

The reference parser rejects --no-publish, so only dashboard argv receives it.
Each entry is guarded separately to protect against future changes in call order.
Publication-wiring tests, a stats-wide gh guard, removing the tracked legacy state, and explicit UTF-8 for production HTML reads/writes are independent follow-ups.
Linux UTF-8 validation does not imply Windows GBK compatibility; the latter remains a known issue.

Suggested labels: bug, family:pr-management, capability:stats.


Was generative AI tooling used to co-author this PR?
  • Yes — Codex (GPT-6). The contributor completed manual code review and approved the exact patch before upstream submission. The commit carries Generated-by:; the approved source is unchanged.

Why: the JSON parity tests call dashboard.main() with publishing enabled,
and their stubs cover only data fetching. An authenticated gh can PATCH
a real dashboard gist or create a new secret gist while both tests pass.

Pass --no-publish to the dashboard invocation only, since reference.main()
does not accept it. Guard gist authentication, publication and session-state
reads/writes with pytest.fail, so ordinary exception handlers cannot swallow
a reached guard and the checks do not depend on main's current call order.

Generated-by: Codex (GPT-6)
@github-actions github-actions Bot added family:tools tools/* substrate:analytics Tool substrate: read-only metrics / dashboards / renderers labels Oct 12, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

family:tools tools/* substrate:analytics Tool substrate: read-only metrics / dashboards / renderers

Projects

None yet

Development

Successfully merging this pull request may close these issues.

PR stats parity tests can publish or overwrite real gists

1 participant