Skip to content

fix: report streaming upstream failures to the request log - #375

Merged
Priyanshu-u07 merged 1 commit into
mainfrom
fix/streaming-failure-status
Oct 2, 2026
Merged

Priyanshu-u07 merged 1 commit into
mainfrom
fix/streaming-failure-status

Conversation

@Priyanshu-u07

Copy link
Copy Markdown
Collaborator

stream_upstream cannot raise past its consumer without ending the stream, so it reports upstream failures as data frames. The except clauses in logging_generator_wrapper therefore never fire, and the only detector was
a substring check for Upstream Error — text that appears in a logger.error line, not in any frame the generator emits.

It never matched. Circuit breaker trips, size-limit aborts and provider 500s were all recorded as status_code=200 with no error message, in the inference log and in the Prometheus counters alike, since RequestLogger.log passes the same value to metrics.observe_request.

The failure now travels out of band in a sink dict, the way usage_trackeralready does three lines above. No message text is matched anywhere, so rewording one cannot break it again.

Two things worth a look

The client sees nothing new. json.dumps produces bytes identical to the hand-written frames, so this changes what is logged, not what is sent. There is a test pinning that.

Statuses are no longer flattened to one code. A circuit breaker trip logs 503, a rejected upstream URL 500, and genuine upstream failures 502. That is a judgement call rather than a correctness one, and worth disagreeing with if you would rather a gateway reported 502 for anything upstream.

Closes #373

Signed-off-by: Priyanshu-u07 <connect.priyanshu8271@gmail.com>
@Priyanshu-u07
Priyanshu-u07 merged commit 5dabd0e into main Oct 2, 2026
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Streaming upstream failures are logged as successes

1 participant