Repository navigation
Description
What changes
#926 (issue #830) stops the nightly report from judging jobs that merely happen
to run beside it. runtime-proof.yml declares the four jobs whose status decides
the verdict:
judged-jobs: |
incus
incus-ovn
stacks (incus-ovn)
stacks (incus-ovn, last release)That is the right fix for #830. It also means streak and verdict leave the
verdict entirely. Whatever they do, the night reads green, no scheduled-red
issue opens, and an open one is closed by --apply.
Why that is not fine as it stands
These jobs break, and they break quietly. runtime-proof.yml:1034 records it in
its own comment:
This job counted the streak from the Actions API and needed nothing from the
repository, so it checked nothing out, and then #795 added a step that runs
tools/ci/streak-report.sh. Six nights ofNo such file or directory,
exit 127.
That specific breakage is fixed, by a checkout step held by
TestEveryJobThatRunsARepositoryScriptChecksItOut. The point is the duration:
six nights, in a job whose status was still part of the verdict. With the judged
list, the same class of failure has no path to anyone at all.
So the state after #926 is two jobs with no reporting whatsoever. Before, they
reported by accident, which was the defect of #830. Removing the accident is
right; replacing it with nothing is the untreated half.
What closes this
One of these, decided and written down, not left implicit:
- A second verdict for the auxiliary jobs. They do not gate the night's
runtime proof, but a red one opens or updates an issue of its own, distinct
fromscheduled-redso the two signals stay separable. - Or an explicit declaration that they are unreported, in
docs/limits.md,
naming what is therefore not watched and what a reader should check by hand.
Plus, in either case, the closure check that #926's review asks for: run jobs,
minus judged, minus a declared ignore list, must be empty. Today a job can
leave the verdict by being forgotten. It should only be able to leave it by being
named. That is what turns this from "two jobs we know about" into a property that
survives the next job somebody adds.
What this is not
Not a request to re-judge streak and verdict in the night's verdict. That is
#830, and #926 is right to take them out.
Where it came from
Review of #926. Raised there and moved here rather than left as a paragraph in a
pull request thread, which is not a tracked debt. It is not the contributor's to
fix in that pull request.
Activity