Repository navigation
Conversation
Under pytest -n auto these tests compared real wall-clock time with tight bounds and failed intermittently; one also failed every time its file ran on its own. test_permission_denied_budget: the branch-parity test compared the slowest real sample from each branch against a 15 ms tolerance, and the overrun test required enforce() to return in under 5 ms. Both now run the budget module on a virtual clock: perf_counter reads it, and sleep and the fake registry advance it exactly. Each branch ends at exactly the 50 ms budget, and a registry slower than the budget is not padded. test_task_options: three "timeout after dispatch" tests used a real 10-100 ms deadline that also covered pre-dispatch work (cold schema loads, validation, the MCP adapter path). When that work was slow the deadline expired before dispatch, leaving no recovery metadata. They now use a generous deadline and expire it through the run_with_timeout seam exactly when dispatch is signalled, raising what the real timer raises. The client's deadline state, dispatch marking and recovery stay real; other tests in the file still exercise the real timer. This also removes the partial nullable-schema warm-up workaround. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
There was a problem hiding this comment.
Ladon verdict: Approve
Approve — test-only PR making timing-sensitive tests deterministic.
Checked:
- Two test files changed only (
tests/test_permission_denied_budget.py,tests/test_task_options.py); no src/, public surface, wire-shape, forward-compat, type-layering, generated code, or credential surface touched. - Reviewer confirmed the
_VirtualClockfaithfully models_permission_denied_budget(module-level time/asyncio seams; budget built before registry read at handler.py:774), so theapprox(50.0)branch-parity assertions still catch a disabled budget. _deadline_expires_onreplaces only therun_with_timeouttimer seam; deadline state,mark_task_dispatched, andrecovery=state.mutation_recoveryassembly stay real — a dropped recovery still fails.- Removed
nullable_request_pathswarm-up and import fully excised; no test disabled/weakened (CI gates stay armed).
No critical/high/medium findings. high_risk false, gated_paths false, no author team gate. Falls through to row 9.
|
Closing: these tests don't fail in CI. I checked the last 40 CI runs on The failures are real only locally: under |
Fixes the parallel-run test failures noted in #1860. Test-only change; no source changes.
What was failing
Under
pytest -n autothese tests compared real wall-clock time against tight bounds:test_branch_parity_absorbs_registry_variancePERMISSION_DENIEDbranch against a 15 ms tolerance. One scheduling stall exceeds it.test_enforce_does_not_sleep_when_already_overrunenforce()to return in under 5 ms of real time.test_real_mcp_dispatch_reports_the_exact_pinned_wire_keytest_new_mutations_timeout_with_recovery_metadata[sync_principal]test_mutation_timeout_carries_secret_safe_recoveryFix
Make the timing deterministic, without widening any tolerance.
adcp.decisioning._permission_denied_budgeton a virtual clock:perf_counterreads it, andsleepand the fake registry advance it exactly. The parity test now asserts that both branches end at exactly the 50 ms budget, and adds a case showing a registry slower than the budget is not padded further. With the budget disabled, the test fails.adcp.client.run_with_timeoutseam exactly when the test's dispatch signal fires, raising_TaskDeadlineExpiredErroras the real timer does. The client's deadline state, dispatch marking and recovery assembly all stay real, and other tests in the file still exercise the real timer. Ifrecoveryis dropped from the timeout error, 5 tests fail. The partial warm-up workaround is removed.Verification
tests/test_task_options.py: 23/23 passed serially and on 3 of 3 runs under-n auto. Before the fix, thesync_principalcase failed on 4 of 4 runs.-n autowith all 48 cores pegged by busy loops: 53/53 passed on 3 of 3 runs.-n auto: 21,708 passed, 0 failed.make lintandmake typecheck-allpass.🤖 Generated with Claude Code