Repository navigation
feat(gateway): a not_ready 503 says the lease's place in line - #327
Merged
Merged
Conversation
A non-streaming request whose start timed out got a bare 503 with Retry-After 30, even while its lease stood in line for the card. The gateway now reads the lease's place in line from berthd when the start times out, and the 503 carries X-Berth-Queue-Position and, when the broker has measured the starts ahead, X-Berth-Start-Estimate: N; lower-bound. Retry-After is the larger of the estimate and 30 s. The broker's estimate counts only the measured starts ahead in the queue and not the holder's time left, so the header marks it as a lower bound instead of passing it off as a promise.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Milestone 3 of the Berth Next plan: make waits honest.
Problem
When a gateway start timed out (
not_ready), a non-streaming client got a 503 withRetry-After: 30and nothingelse, even while its lease stood in line for the card. The client could not tell "the engine is broken" apart from
"you are second in line behind a training".
Change
When the start times out, the gateway reads berthd's status once and finds the engine's waiting lease on its card.
If one is in line, the 503 now carries:
X-Berth-Queue-Position: N, where 1 is next in line.X-Berth-Start-Estimate: N; lower-bound, the broker'sestimate_start_srounded up. It is missing when nothingahead was measured.
Retry-After, the larger of the estimate and 30 s.The error message adds "its lease is number N in line for the card".
The estimate is a lower bound because the broker counts only the measured starts ahead in the queue and never the
holder's time left (
internal/broker/queue.go). Behind a pinned training it is far too low. The header says sorather than presenting it as a promise. A Retry-After that adds the holder's time needs the hold lengths that the
plan's decision log will record.
Streaming requests already got their headers when the SSE preamble went out, so they keep the same error as an SSE
event and get no new headers.
Proof
TestNotReadyCarriesTheQueueruns a real test berthd with a pinned hold on the card, so the gateway's spawnedberth runtruly waits. Without the change it fails withqueue "", Retry-After "30"and a barenot_readymessage.
TestQueuedStartHeadersserves a fake status with another engine ahead in line. It checks the rounding (44.2 sgives 45), the 30 s floor, a missing estimate, and that only the engine's own lease counts.
proof.shpasses in full: build, gofmt, vet,go test ./..., the Windows build and vet, and golangci-lint 2.14.0.docs/gateway.mddescribes the headers under the failure rules.