Repository navigation
fix(faults): a receipt line says when it is still in flight (#925) - #929
Conversation
GET /_feint/faults reserves a line when a rule fires and completes it when the handler ends. In between it published `status: 0, abandoned: false` with nothing saying the firing was still running. A reader was not blind to it: `abandoned` is `!answered || status == 0` at the call site, so a finished line is always `status != 0, abandoned: false` or `status == 0, abandoned: true`, and that pair belonged to a line in flight alone. What was missing was a way to SAY it. The invariant was undocumented, it holds only as long as `abandoned` keeps that formula, and feint's own reader did not apply it: `feint fault list` printed `answered 0` for a request nobody had answered. The window is milliseconds for a delay and minutes for a `hang`, which is exactly when a suite polls, since its client is still blocked. How it was found, and what it says about the gate. The test that reads the receipt right after its client gives up failed 4 times in 10 runs on main at 51612e6, an hour after the lot that wrote it merged, and 3 times out of 3 under `-count=8`. The CI of that pull request was green on the same commit: one draw of a coin that comes up tails four times in ten. A verdict on an intermittent phenomenon needs repetition, not a single run. What changed: - `in_flight` on the receipt line, set at reserve and cleared in recordFired beside the three values the effect produced. It is omitempty, so a finished line is byte-identical for a version 3 reader. FaultsSchemaVersion 4. - The frozen population holds a connection open across the capture. Without it the fixture would freeze version 4 without the only field that distinguishes it, and faults.json already carries the comment explaining why that matters: a gate whose population cannot contain the thing it judges is not a gate. - `feint fault list` and a run's receipt read the field, through one shared describeFiredLine so the two renderings cannot drift. A run's receipt is written to disk and cannot be re-polled, so runReceiptSchemaVersion is 3. - Four falsify mutations, two of them written by a falsifier against the first version of this lot: clearing the marker by operation name instead of by slot, and clearing it before the epoch guard. Both compiled, both lied, and the whole package stayed green under each, because every test here drove a single client. The frozen fixture is reindented by the generator, which the committed file had drifted from. Entries 1 to 3 decode identically; only version 4 is new. Assisted-by: Claude Code (claude-opus-5)
|
Closing the run this pull request declared as not played. The six falsify specs
45 mutations, each compiled and each reddening its named test, baseline green before and after restoration in every run.
The full Logs: |
Summary
A fault receipt line was written in two steps and only the second one was
visible.
reserveappends the line when the rule fires;recordFiredfills instatus, transport and abandoned when the handler finishes. Between the two the
line existed and said nothing, and nothing in the JSON said so.
That gap is not theoretical: it is a race the test suite was already losing.
The measurement that found it
On unmodified
origin/mainat51612e6, in this repository:go test -race ./internal/core/emulator -run TestADroppedConnectionIsClaimedOnlyWhenItHappened -count=1, ten times-count=8The CI of #922 was green on that same commit, which is how it shipped. The
author of #926, working on an unrelated area, hit it independently and
disclosed it in his pull request body as a pre-existing flake. It is the same
defect.
The reason: the test polled the receipt and read
abandonedon a line that wasreserved and not yet completed. A zero value for "not yet" is indistinguishable
from a zero value for "no".
What changed
FiredFault.InFlight(internal/core/emulator/faults.go).reservesetsit true.
recordFiredclears it last, after the epoch guard and after thethree values the effect produced, so a reader never sees a half-written line
claim to be finished.
omitemptykeeps a finished line byte-identical to whatversion 3 produced.
What a version 3 reader was missing is worth stating precisely, because the
first draft of this said something false. A v3 reader was not blind: it
could infer "in flight" from the two-field invariant (
status == 0andabandoned == falsecannot both hold on a finished line, becauseconformance.gohandsrecordFired!answered || status == 0forabandoned). But that invariant was undocumented, unenforced, and feint's ownCLI did not apply it. An invariant nobody can read is not an interface.
Note also that a line in flight does not hold all-zero fields:
Transportis named at the hijack by
markTransport, so a hang carries its transport foras long as the connection is held. That is the whole reason
markTransportexists, and the new CLI rendering depends on it.
One renderer instead of three (
internal/cli/fault.go).describeFiredLinenow produces every receipt sentence, and
run_faults.gocalls it. A line stillin flight reads
the connection is held open right now, no answer yetfor ahang,
still in flight, no answer yetotherwise. Theemulatorimport leftrun_faults.go, which is the structural proof that only one renderer remains.Schema versions.
FaultsSchemaVersion3 → 4.runReceiptSchemaVersion2 → 3, because a run receipt is written to disk and cannot be re-polled.
The frozen fixture gained entry 4. The population now holds a raw
connection open across the capture loop, through
waitForAHeldLine, becausewithout an in-flight line observed the field tree is unchanged and
frozen:updateproduces nothing. That is the trap this change walked intofirst.
The whole
testdata/frozen/faults.jsonfile is reindented from 1 space to 2 bythe generator: the committed file had drifted from what the generator emits.
Entries 1 to 3 decode identically.
conformance.jsonis still 1-space and isuntouched here.
The one that would have shipped broken
Changing the sentence broke
tools/conformance/faults.sh:379, which grepsinstance/v1/API.ListServers call 1: answered 403against a real client.mise run prepushdoes not play that suite, so nothing local said a word. Itwas caught by the contract-guardian pass and is fixed, with a comment naming
why that grep exists.
README.md:751andREADME.fr.md:589carry the samesentence by hand, outside any generated block, so
docs:checkcould not seethem either; both updated.
Type of change
Runs
mise run prepushgo test -race ./internal/core/emulator ./internal/cli, 6 repeatsmise run falsify -- tools/falsify/specs/a-receipt-line-says-it-is-in-flight.jsonmise run conformance:leg -- fieldsMode:
--vm off. Nothing here touches a machine runtime, so no leg underFEINT_VMwas played, and none is owed.The
fieldsleg is the rowmise run testplannames for a response-shapechange, and it is the only leg that drives every client against one emulator.
It is also the only run that could have caught the
faults.shbreakage.Runs not played, and what that leaves unproven
mise run conformancepass.fieldscarries every client; whatthe full pass adds is the other legs' own gates, none of which judge the
sentence this change edits.
mise run testplannames, because they mutatefiles this diff touches.
mise run falsify:lintis green, so every mutation'sfindstring still applies: that is the cheap half. That each of them stillbites has not been replayed. This will be done before merge and the result
posted here.
Falsification
tools/falsify/specs/a-receipt-line-says-it-is-in-flight.json, four mutations,each biting
TestAReceiptLineSaysWhenItIsStillInFlight:InFlight: true && falseat reserve, so the line is never marked;InFlight = trueinrecordFiredinstead offalse, so it is never cleared;wrong line when one rule has several firings;
campaign's line.
Mutations 3 and 4 are held by two further tests written for exactly those
shapes,
TestSettlingOneFiringLeavesTheOtherInFlightandTestAStaleFiringDoesNotSettleTheNewCampaignsLine.Checklist
Always
mise run checkpassesinternal/corestill knows no providermise run testplanwas run, and the runs not played are named above withwhat they leave unproven
When a model wrote a substantive part of this
Assisted-by:trailer names the tool and the model versionmise run conformance:leg -- fieldswas run, not assumed. The full passwas not, and that is written above
in_flightis this emulator's own debugging surface,not a provider shape. No provider response changed
When a route is added or changed
N/A. No route added or changed, no upstream operation touched.
When behaviour a client can observe changes
docs/limits.mdnamesin_flightin what the receipt serveswere updated by hand, which is the gap that let them go stale
When
.github/workflows/is touchedN/A.
When a machine runtime is involved
N/A.
--vm offthroughout.Known gap, carried by its own issue
faultsDocument.SchemaVersion(internal/cli/fault.go:143) is decoded and readnowhere, so
feint fault listagainst an emulator at version 3 renders everyline as finished and an in-flight line reads
answered 0. That is the lie thispull request removes, reintroduced by a version skew.
#925 neither created nor worsened it, and it is not fixed here. It is #928,
with its closing criteria, rather than a sentence in a pull request body.
Related issues
Closes #925.
🤖 Generated with Claude Code