Skip to content

fix: make LLMDescriptionScorer work inside a running event loop - #362

Open
kayaal34 wants to merge 1 commit into
deeppavlov:devfrom
kayaal34:fix/llm-scorer-event-loop
Open

kayaal34 wants to merge 1 commit into
deeppavlov:devfrom
kayaal34:fix/llm-scorer-event-loop

Conversation

@kayaal34

Copy link
Copy Markdown

Closes #353 (the LLMDescriptionScorer part. TypeSafeDescriptionScorer isn't on dev yet).

Problem

  • _init_event_loop took the loop from asyncio.get_event_loop(). When predict() is called from async code (a notebook, a FastAPI handler), that returns the running loop, and run_until_complete raises RuntimeError: This event loop is already running.
  • On Python 3.14 get_event_loop() raises when no loop is set, so every instance fell back to new_event_loop(), and clear_cache removed it with delattr without closing it: one leaked loop per HPO trial.

Fix

  • Each scorer owns a private loop (asyncio.new_event_loop()) and never touches the caller's loop. I kept one loop per instance on purpose rather than calling asyncio.run per batch: the generator's AsyncOpenAI client pools connections bound to the loop they were opened on, so running every batch on the same loop keeps that pool valid.
  • _run_async(coro): with no running loop in the current thread, it runs the batch directly on the scorer's loop. Inside a running loop, it runs the batch on a one-off worker thread and blocks until it finishes. predict() stays synchronous.
  • The loop is closed on refit (_init_event_loop) and in clear_cache.

Tests

  • test_description_scorer_llm_predict_inside_running_loop: predict() called from a coroutine returns the same result as the sync call, and the scorer still works from sync code afterwards.
  • test_description_scorer_llm_closes_its_event_loop: refit and clear_cache close the loop.
  • Both new tests fail on dev and pass with this change. tests/modules and tests/pipeline filtered to description/LLM: 24 passed. ruff and mypy on the changed files pass.

This change touches the same lines as #361, so whichever merges second will need a one-line conflict resolved. I can rebase if needed.

The scorer ran its async batch with `run_until_complete` on a loop taken from
`asyncio.get_event_loop()`. Called from async code (notebook, async web
handler) that returned the running loop and raised "This event loop is already
running". On Python 3.14 `get_event_loop()` raises when no loop is set, so
every instance created a new loop that `clear_cache` never closed, leaking one
loop per HPO trial.

Each scorer now owns a private loop, which the generator's async client stays
bound to. Batches run on it directly from sync code, or on a worker thread when
the caller is already inside a running loop. The loop is closed on refit and in
`clear_cache`.

Closes deeppavlov#353
Copilot AI lite review requested due to automatic review settings September 19, 2026 12:45

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🟡 Changes recommended

Unresolved event-loop thread-safety, concurrency, and lifecycle concerns remain.

Get a fresh assessment by requesting another Copilot review.

Review effort: Lite
Findings: 1 High severity

Open (1)
What changed in this PR

Updates LLMDescriptionScorer to support synchronous prediction from running asyncio loops and manage private event-loop cleanup.

Changes:

  • Adds worker-thread execution for running-loop callers.
  • Closes loops during refit and cache clearing.
  • Adds regression tests for async prediction and loop closure.
File Summary
tests/​modules/​scoring/​test_description_llm.py Tests running-loop prediction and loop closure.
src/​autointent/​modules/​scoring/​_description/​llm_encoder.py Implements private loop management and worker-thread execution.

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment on lines +304 to +308
return self._event_loop.run_until_complete(coro)
# Called from async code (notebook, async web handler): this thread's loop is busy,
# so drive the scorer's loop from a worker thread and block until it finishes.
with ThreadPoolExecutor(max_workers=1) as executor:
return executor.submit(self._event_loop.run_until_complete, coro).result()
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

API scorers: predict() fails inside a running event loop; LLMDescriptionScorer leaks an unclosed loop per instance on Python 3.14

2 participants