Repository navigation
fix(refresh): run the mirror after the source's weekly run, not an hour before it - #4
Merged
Merged
Conversation
…ur before it `refresh.yml` pulled https://smmscout.com/api/panels.json at Monday 03:00 UTC while the source repo (RelayStack/smmscout, weekly-maintenance.yml) is scheduled for Monday 04:00 UTC and publishes its new payload ~15 min into the run. GitHub fires scheduled runs hours late on this account (source starts observed 2026-10-12T11:29Z and 2026-10-05T11:01Z), so the mirror always pulled the build the source was about to supersede: on 2026-10-13 the dataset carried data_built_at 2026-10-12T01:51:34Z while live was 2026-10-12T11:36:05Z, and 59 of 129 published scores differed. In a week with no out-of-band dispatch the mirror is a full cycle (~7 days) behind the live API. - schedule moves to `0 20 * * 1`: ~8 h past the latest observed source start, with margin for the schedule delay. The comment block in the workflow records why the ordering matters so the slot cannot be moved back by accident. - new step "Report which build was mirrored" prints the mirrored `data_built_at` against the committed copy on every run, and emits a `::warning::` when the pull is not newer than what is already committed — the silent-staleness case that hid this bug for weeks (no diff, green run, nothing to notice). - README refreshing section updated (Mondays 03:00 -> 20:00 UTC, with the reason). The pull URL and the dataset shape are unchanged: the job still consumes the live Function-derived payload (with `operational`), and the python writer is untouched (`indent=1`, same CSV header, same derived files). Verified locally: `node scripts/validate.mjs`, `node scripts/stats.mjs` + `git diff --exit-code -- data/stats.json` and `node scripts/verify-consistency.mjs --report` all pass; the new step was rehearsed against the real live payload (notice) and the committed one (warning). Refs kanban t_23be34f6.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The mirror's declared schedule runs one hour before the source it mirrors, so the
public dataset always carried the build the source was about to supersede.
Measured 2026-10-13 (before this PR, cache-busted live vs raw mirror):
Cause:
refresh.ymlwascron: '0 3 * * 1'; the sourceRelayStack/smmscout/.github/workflows/weekly-maintenance.ymliscron: '0 4 * * 1'and publishes ~15 min into its run. Scheduled runs on this account start hours late
(source starts: 2026-10-12T11:29Z, 2026-10-05T11:01Z for a 04:00 schedule), so the
03:00 mirror always pulled the previous build; with no out-of-band dispatch it ends
up a full cycle (~7 days) behind.
What changes
cron: '0 20 * * 1'— ~8 h past the latest observed source start, with marginfor GitHub's schedule delay. A comment block in the workflow records the ordering
constraint and the measurement that set it.
data_built_atagainst the committed copy and emits a::warning::when the pullis not newer than what is already committed. That was the silent failure mode:
an early run produced no diff, committed nothing and reported green.
Not changed (per the card)
Pull URL stays
https://smmscout.com/api/panels.json(Function-derivedoperational);the dataset writer is untouched (
indent=1, same CSV header, same derived files).Verification in this PR
node scripts/validate.mjs→ OK 129 panels ·node scripts/stats.mjs+git diff --exit-code -- data/stats.json→ clean ·node scripts/verify-consistency.mjs --report→ OK (the three
verify.ymlsteps).::notice::mirrored data_built_at 2026-10-12T11:36:05.417Z (committed copy was 2026-10-12T01:51:34.109Z)) and againstthe committed payload (
::warning title=Stale mirror pull::…), exit 0 both times.verifyrun on this PR: see checks below.Closes kanban t_23be34f6. Acceptance for "one Monday after" (mirror
data_built_at== live, 0 score diffs) lands on the first Monday 20:00 run; a manual
workflow_dispatchright after this merges pulls the current live build immediately and is the same check.