Both API scorers run their requests with a loop kept on the instance: _init_event_loop does asyncio.get_event_loop() (falling back to new_event_loop()) and _compute_similarities calls self._event_loop.run_until_complete(aiometer.run_all(...)) — LLMDescriptionScorer llm_encoder.py:282-291, :247-252; TypeSafeDescriptionScorer typesafe.py:259-270, :331-340. Two consequences:
1. predict() fails inside a running event loop (both scorers)
In a notebook, a FastAPI/uvicorn handler, or any async def caller, get_event_loop() returns the running loop and run_until_complete raises RuntimeError: This event loop is already running. A pipeline whose scoring node is one of these modules cannot be served from an async server without the max_concurrent=None (sequential, sync-client) escape hatch.
2. LLMDescriptionScorer leaks an unclosed event loop per instance on Python 3.14
On 3.14 asyncio.get_event_loop() raises RuntimeError when no loop is set, so the fallback new_event_loop() runs — without set_event_loop — for every instance; each HPO trial constructs a new module, and clear_cache delattrs the loop without closing it (:274-280). Reproduced with the _init_event_loop body verbatim on CPython 3.14.3:
$ python3.14 -W always repro.py # three "trials"
ResourceWarning: unclosed event loop <_UnixSelectorEventLoop running=False closed=False debug=False> (×3)
distinct loops: 3 closed: [False, False, False]
One selector fd per trial, never released until GC. On ≤3.13 the main-thread get_event_loop() creates and sets one loop, so it is shared and this does not show. TypeSafeDescriptionScorer calls set_event_loop and reuses the loop, so it only has problem 1.
Proposed
One helper in a shared base (#357) instead of a loop on the instance: no running loop → run the batch on a fresh loop that is closed afterwards (asyncio.run semantics); running loop → run it on a worker thread with its own loop (or expose an apredict coroutine for async callers). That also removes the Dumper.dump(..., exclude=[asyncio.BaseEventLoop, dict]) special-casing in both dump methods.
Deferred from #350.
Both API scorers run their requests with a loop kept on the instance:
_init_event_loopdoesasyncio.get_event_loop()(falling back tonew_event_loop()) and_compute_similaritiescallsself._event_loop.run_until_complete(aiometer.run_all(...))—LLMDescriptionScorerllm_encoder.py:282-291,:247-252;TypeSafeDescriptionScorertypesafe.py:259-270,:331-340. Two consequences:1.
predict()fails inside a running event loop (both scorers)In a notebook, a FastAPI/uvicorn handler, or any
async defcaller,get_event_loop()returns the running loop andrun_until_completeraisesRuntimeError: This event loop is already running. A pipeline whose scoring node is one of these modules cannot be served from an async server without themax_concurrent=None(sequential, sync-client) escape hatch.2.
LLMDescriptionScorerleaks an unclosed event loop per instance on Python 3.14On 3.14
asyncio.get_event_loop()raisesRuntimeErrorwhen no loop is set, so the fallbacknew_event_loop()runs — withoutset_event_loop— for every instance; each HPO trial constructs a new module, andclear_cachedelattrs the loop without closing it (:274-280). Reproduced with the_init_event_loopbody verbatim on CPython 3.14.3:One selector fd per trial, never released until GC. On ≤3.13 the main-thread
get_event_loop()creates and sets one loop, so it is shared and this does not show.TypeSafeDescriptionScorercallsset_event_loopand reuses the loop, so it only has problem 1.Proposed
One helper in a shared base (#357) instead of a loop on the instance: no running loop → run the batch on a fresh loop that is closed afterwards (
asyncio.runsemantics); running loop → run it on a worker thread with its own loop (or expose anapredictcoroutine for async callers). That also removes theDumper.dump(..., exclude=[asyncio.BaseEventLoop, dict])special-casing in bothdumpmethods.Deferred from #350.