Skip to content

fix: pin the Agg matplotlib backend for the test suite (#280) - #287

Merged
RN0311 merged 1 commit into
mainfrom
fix/280-pin-matplotlib-backend
Sep 16, 2026
Merged

RN0311 merged 1 commit into
mainfrom
fix/280-pin-matplotlib-backend

Conversation

@RN0311

@RN0311 RN0311 commented Sep 16, 2026

Copy link
Copy Markdown
Collaborator

Summary

Fixes #280. The repository pinned no matplotlib backend anywhere (matplotlib.use/MPLBACKEND appear nowhere in the tree), so the plot tests opened whatever display connection the host's default backend wanted. On the reporter's Windows host that resolved to tkagg, and one plot test per full run failed with _tkinter.TclError — a different one each time, each passing alone. Their own third run showed MPLBACKEND=Agg makes the suite deterministic.

tests/conftest.py now pins the headless backend at collection time:

  • matplotlib.use("Agg", force=True) for the test process itself (force so it holds even if a plugin imported matplotlib first),
  • os.environ.setdefault("MPLBACKEND", "Agg") so subprocesses the tests spawn inherit it,
  • guarded by ImportError so a core-only environment without the [ml] extra still collects.

No product code changes — the backend choice for real CLI runs is untouched; this pins only the test suite.

Testing

  • New tests/unit/test_matplotlib_backend.py asserts the suite runs on Agg and exports MPLBACKEND.
  • All plot-touching suites pass: test_audio_quality_plots.py, test_demographic_analyzer.py, test_report_generator.py + the new file — 53 passed.
  • Smoke: forcing the reporter's configuration with MPLBACKEND=tkagg on this headless host, the suite still resolves to Agg and all plot tests pass. (The pre-fix failure itself is host-specific — this Linux box's Tk happens to be usable — but the issue's three-run table already established the mechanism.)
  • ruff check and ruff format --check clean.

Nothing in the repository set a backend, so the plot tests used whatever
the host resolved to. On a host with an interactive default (tkagg on
Windows) one plot test per full run failed on the display connection —
a different one each time — while each passed alone. conftest.py now
pins Agg with matplotlib.use(force=True) for the process and exports
MPLBACKEND for spawned subprocesses, so a full run no longer depends on
whether the host can open a display. A small regression test asserts the
suite runs headless.
@RN0311
RN0311 requested a review from AndyBoWu as a code owner September 16, 2026 06:34
@github-actions github-actions Bot added track: toolkit Core psdn-sonar eval toolkit code and tests track: qa Evaluation correctness and regression testing labels Sep 16, 2026
@RN0311
RN0311 merged commit 6c8ad9b into main Sep 16, 2026
17 checks passed
@RN0311
RN0311 deleted the fix/280-pin-matplotlib-backend branch September 16, 2026 07:08
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

track: qa Evaluation correctness and regression testing track: toolkit Core psdn-sonar eval toolkit code and tests

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Bug] The test suite pins no matplotlib backend, so a plot test fails per full run on a host with an interactive default and a different one each time

1 participant