Skip to content

Stop the schedule fixture probe from applying a published PHP write twice - #674

Merged
rmcdaniel merged 2 commits into
mainfrom
rust-325-sched-diagnostics
Oct 23, 2026
Merged

rmcdaniel merged 2 commits into
mainfrom
rust-325-sched-diagnostics

Conversation

@rmcdaniel

@rmcdaniel rmcdaniel commented Oct 22, 2026 •

Copy link
Copy Markdown
Member

Main's Shared Server fixtures fail schedule-policies (or schedule-catalog) intermittently, at a different step each time:

Run Difference
main 29fb77e3 (job 119962357942) step 47, the second POST schedules/quota/trigger, returned 404 instead of 200
#667 b2999cac9 the first policy-skip trigger returned skipped instead of triggered
#667, earlier head both DELETE schedules/policy-skip steps returned 503

Cause. These differences come from the published PHP reference, not native. When native differs, the step log first shows {"outcome":"pass","target":"php"} and …"embedded"}; none of these failures has those lines.

A local reproduction of the CI step shows the mechanism:

  • PHP on SQLite can report database is locked for a schedule write that had already committed. The started run's task is queued after the trigger's transaction.
  • The probe sent that write again, so it applied twice:
    • a retried first trigger read as skipped, because its own first run was still running;
    • a retried quota trigger used up the second action, so the next trigger found the schedule deleted.

Change.

  1. Safe write retries. The probe sends a write again only when its schedule reads exactly as it did before the write, which shows the write rolled back. Otherwise it records the failure, and the recorded lock replays the whole sequence in a new namespace, now up to five times. Reads are still retried. Native never reports this lock, so it never retries or replays.
  2. Readable failures. A status difference now names each differing step with its response, and an outcome difference shows the trigger responses with their reasons.

Evidence, local runs of the CI step:

Probe Result
Main's probe PHP failed schedule-catalog after three replays
Replay-only (no write retries) PHP failed both fixtures after five replays
Replay-only, maintenance loop at 1 CPU Worse; contention is not CPU starvation
This head PHP and embedded pass. PHP's schedule-policies took five safe retries and one replay; schedule-catalog passed first time. Native's only difference is the known frozen signals-before-worker; it matched both schedule fixtures on attempt 1 with no retries.

The schedule contract tests pass (20/20 and 18/18), including the new message and attempt-bound cases.

Main's Shared Server fixtures fail schedule-policies on native SQLite
intermittently, with a different step each time: a 503 on DELETE, a
first skip trigger reported as skipped, and a 404 on the second quota
trigger. The contract printed only positions and outcomes, so the cause
was not in the log. A status difference now names each differing step
with its response, and an outcome difference shows the trigger
responses with their reasons.
The schedule-policies and schedule-catalog differences on main come
from the published PHP reference, not native: no PHP pass line precedes
them. PHP on SQLite can report "database is locked" for a write that
already committed (the started run's task is queued after the trigger's
transaction), and the probe sent the write again. A retried trigger then
read as skipped (its first run was running), and a retried quota trigger
used the second action, so the next trigger found the schedule deleted.

A write is now sent again only when its schedule reads exactly as it did
before the write. Otherwise the failure is recorded, and the recorded
lock replays the whole sequence in a new namespace, now up to five times.
Reads are still retried. Native never reports this lock.

In a local run of the CI step on these sources, published PHP passed
both schedule fixtures (five safe retries and one replay for
schedule-policies), and native matched them on the first attempt.
@rmcdaniel rmcdaniel changed the title Report the responses behind a schedule-policies difference Stop the schedule fixture probe from applying a published PHP write twice Oct 22, 2026
@rmcdaniel

Copy link
Copy Markdown
Member Author

Riding merge train #675 with #667, #669, #670 and #674 on main 17dc12cd, so they share one CI run. This PR closes with a link once the train merges.

@sonarqubecloud

Copy link
Copy Markdown

@rmcdaniel
rmcdaniel merged commit 544c66a into main Oct 23, 2026
21 of 27 checks passed
@rmcdaniel
rmcdaniel deleted the rust-325-sched-diagnostics branch October 23, 2026 13:31
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant