fix(live): stop a live run from writing session state onto the RunConfig - #6774
Closed
LHMQ878 wants to merge 1 commit into
Closed
fix(live): stop a live run from writing session state onto the RunConfig#6774LHMQ878 wants to merge 1 commit into
LHMQ878 wants to merge 1 commit into
Conversation
The basic request processor forwards `RunConfig.session_resumption` and `RunConfig.history_config` to `LiveConnectConfig` by reference, and the connect loop in `BaseLlmFlow.run_live` then writes onto whatever object it finds there: the newest server-issued resumption handle on every reconnect, `transparent` on the Vertex AI backend, and `initial_history_in_client_content` when it replays history. All of that lands on the caller's own RunConfig, so a caller that reuses it for the next run resumes the session that just ended instead of starting a new one. Forward a copy of both, which is what `http_options` already does for the same reason. Agent transfer is unaffected: it clears the handle on a deep-copied run config, and the child's request assembly copies from there.
Collaborator
|
This is already handled by a pending commit. Thanks. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Link to Issue or Description of Change
Related: #6765 is unrelated; this one has no issue, so the problem and solution are described below. The code path is the one added in eac32c3 (
feat(live): use RunConfig.session_resumption.handle when opening a live session).Problem:
_build_basic_requestforwards two caller-owned objects to the live connect config by reference:The connect loop in
BaseLlmFlow.run_livethen writes onto whatever object it finds there:live_connect_config.session_resumption.handle = invocation_context.live_session_resumption_handle— the newest server-issued handle, on every reconnect;session_resumption.transparent = True— the Vertex AI backend default;history_config.initial_history_in_client_content = True— when it replays history.All three land on the caller's own
RunConfig. So a caller that reuses itsRunConfig— the natural thing to do when the point ofRunConfig.session_resumption.handleis to resume across runs — starts the next run resuming the session that just ended, with a handle it never set. Concretely, after one run that reconnected once:transparent=Trueis the same kind of leak in the other direction: it is set only for the Vertex AI backend, and the Gemini API backend "explicitly rejects it" (per the comment at that site), so it sticks to a config that is later used against a backend that refuses it.Solution:
Forward a copy of each instead of the caller's object. This is exactly what
http_optionsalready does a few lines above, for the same reason — see_merge_run_config_http_options: "The RunConfig's options are copied in rather than aliased, so request assembly cannot write back into the RunConfig."model_copy()(shallow) is enough: both objects hold only scalars, and the flow only sets top-level fields on them.Nothing reads these back off the
RunConfigduring a run — the seed added in eac32c3 deliberately readsllm_request.live_connect_config.session_resumption, which is now the copy the reconnect path writes, so the seed and the writes stay on the same object. Agent transfer is unaffected: it clears the handle on adeep=Truecopy of the run config, and the child's own request assembly copies from there.Testing Plan
Unit Tests:
Three tests, each failing on
main:test_basic_processor.py::test_run_config_live_session_objects_are_not_aliased— assembly level: after the processor runs, writing the handle /transparent/initial_history_in_client_contenton the request must not show up on theRunConfigobjects.test_base_llm_flow.py::test_run_live_does_not_write_the_session_handle_onto_the_run_config— end to end: caller supplieshandle='caller_handle', the server issues'server_handle', the connection drops, and the reconnect uses'server_handle'whilerun_config.session_resumptionstaysSessionResumptionConfig(handle='caller_handle'). Runs with the Vertex AI backend so it covers thetransparentwrite too.test_base_llm_flow.py::test_run_live_does_not_write_initial_history_onto_the_run_config— end to end: the connect request declaresinitial_history_in_client_content=Truewhilerun_config.history_configstaysHistoryConfig().On
main:With the change:
pre-commit run --files <changed files>passes (isort, pyink, ruff, addlicense, codespell, ADK compliance checks).Manual End-to-End (E2E) Tests:
Not run against a live model — reproducing the leak needs a server-issued
session_resumption_updatefollowed by a dropped socket, which is what the second test simulates. The observable symptom is theRunConfigprinted above:handle='server_handle', transparent=Trueon an object the caller created withhandle='caller_handle'.Checklist