Skip to content

Stop nesting async generators in response streams - #1281

Open
Kludex wants to merge 3 commits into
mainfrom
stop-nesting-response-stream-generators
Open

Kludex wants to merge 3 commits into
mainfrom
stop-nesting-response-stream-generators

Conversation

@Kludex

@Kludex Kludex commented Oct 20, 2026 •

Copy link
Copy Markdown
Member

Closes #1280.

When the event loop shuts down with streamed responses still open, loop.shutdown_asyncgens() closes every async generator concurrently. If one generator closes another inside aclosing, and the inner one awaits during cleanup (closing the response or connection), asyncio and the outer generator both call aclose() on it, and one of them gets RuntimeError: aclose(): asynchronous generator is already running. This has been there since #137.

This removes the nesting instead of the cleanup:

  • PoolByteStream.__aiter__ returns the inner stream iterator directly. It was a pure pass-through generator.
  • Response.aiter_raw() and Response.aiter_bytes() return one shared generator (identity decoder vs content decoder) instead of aiter_bytes() wrapping aiter_raw().

Connections are still closed at shutdown. A narrower fix that skipped cleanup on GeneratorExit silenced the errors but leaked sockets, so I dropped it.

aiter_raw and aiter_bytes are now plain methods returning an async generator, so inspect.isasyncgenfunction() returns False for them. Calling and iterating them works the same.

AI Disclaimer

This PR was developed with the assistance of either Claude or Codex. I've reviewed and verified the changes.

View guided diff

@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Oct 20, 2026 •

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-10-20T15:50:18.027571Z 0554774 PR opened
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

Comment thread src/httpx2/httpx2/_models.py Resolved
@codspeed

codspeed Bot commented Oct 20, 2026 •

Copy link
Copy Markdown

Merging this PR will not alter performance

✅ 17 untouched benchmarks
⏩ 7 skipped benchmarks1


Comparing stop-nesting-response-stream-generators (61b863f) with main (6dd59b9)

Open in CodSpeed

Footnotes

  1. 7 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports. ↩

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 0554774ef3

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread src/httpx2/httpx2/_models.py Resolved
Comment thread src/httpx2/httpx2/_models.py Resolved
@fancyboi999

Copy link
Copy Markdown

We see the same aclose(): asynchronous generator is already running in production during normal operation, not only at shutdown. It happens when the garbage collector finalizes a nested stream iterator chain in one batch. asyncio's finalizer hook then starts one aclose() task per generator, just as shutdown_asyncgens() does. Example production log (Python 3.13, httpx2 2.13.1):

Task exception was never retrieved
future: <Task finished coro=<<async_generator_athrow without __name__>()> exception=RuntimeError('aclose(): asynchronous generator is already running')>

Minimal repro of the GC path: open streams, read the first chunk, then drop them while they are still unreachable from a reference cycle.

import asyncio, gc, httpx2

async def handle(reader, writer):
    while (await reader.readline()) not in (b"\r\n", b""):
        pass
    writer.write(b"HTTP/1.1 200 OK\r\ntransfer-encoding: chunked\r\n\r\n5\r\nhello\r\n")
    await writer.drain()
    await reader.read()

async def main():
    server = await asyncio.start_server(handle, "127.0.0.1", 0)
    url = f"http://127.0.0.1:{server.sockets[0].getsockname()[1]}/"
    client = httpx2.AsyncClient()
    for _ in range(5):
        response = await client.send(client.build_request("GET", url), stream=True)
        chunks = response.aiter_bytes()
        await anext(chunks)
        holder = [chunks, response]
        holder.append(holder)
        del chunks, response, holder
    gc.collect()
    for _ in range(20):
        await asyncio.sleep(0.05)
    await client.aclose()
    server.close()

asyncio.run(main())

Results on Python 3.13.16:

httpx2 already running errors
2.13.1 5 / 5
2.16.0 5 / 5
this PR (61b863f) 0

The shutdown repro from #1280 also goes from 5 errors to 0 with this PR. So this PR fixes the GC-finalizer path too, not only shutdown. Thanks!

One observation, in case it is useful: with this PR, after a consumer stops iterating aiter_bytes() early, HTTP11ConnectionByteStream.__aiter__ is still finalized by the GC, even when the response is closed. On its own it no longer races.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

RuntimeError "aclose(): asynchronous generator is already running" from nested stream generators at event loop shutdown

2 participants