Repository navigation
test(serve): cover user images across three HTTP APIs - #5046
Open
silencelamb wants to merge 5 commits into
Open
silencelamb wants to merge 5 commits into
silencelamb wants to merge 5 commits into
Conversation
Add one shared HTTP/SSE image matrix for Chat, Messages, and Responses, exercising PNG/JPEG/WEBP, data and HTTP URLs, image-only input, interleaved multiple images, and multi-turn history with real image decoding. Keep protocol-specific Responses validation tests separate. Cover corrupt-image rejection and text-only model rejection for all three protocols. Generation and remote HTTP transport are simulated; no GPU service is started.
This was referenced Oct 20, 2026
4 tasks done
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Motivation
Equivalent user-image requests should preserve image contents, text order, and conversation history across
/v1/chat/completions,/v1/messages, and/v1/responses. Use a small shared set of scenarios to check these invariants across all three entry points.Modification
Add
tests/test_lmdeploy/serve/test_user_images.pywith three representative PNG scenarios:Each scenario runs through all three APIs in both non-streaming and SSE modes, also checking successful completion and session cleanup: 18 HTTP cases.
Six additional checks use protocol conversion and shared preprocessing directly to ensure corrupt image data is rejected and text-only models reject images, once per protocol.
The suite uses production routing, protocol conversion, media loading, and image decoding. Generation and remote HTTP transport are simulated; no model weights, GPU service, or external network access are required.
Test scope
This file owns representative cross-endpoint invariants, not a Cartesian product of image formats, sources, and conversation shapes. PNG/JPEG/WEBP decoder coverage, SDK compatibility, image parameters, and detailed protocol validation remain in the existing endpoint suites, including
openai/responses/test_images.py.A few successful image flows intentionally overlap with the focused Responses regression tests so that #5043 remains independently validated. Tool-result images and their three-endpoint regressions belong to #5044.
BC-breaking (Optional)
None. The incremental change adds one test file without changing production code or dependencies.
Validation
70689117: 510 passed (python -m pytest tests/test_lmdeploy/serve -q), including all 24 shared image checks.git diff --checkpassed.Dependency and review scope
This PR depends on #5043, which enables ordinary user images in Responses. It does not depend on #5044.
Until #5043 merges, the upstream
maindiff also includes that prerequisite. The incremental test-only comparison adds onlytests/test_lmdeploy/serve/test_user_images.py. This branch should be rebased onto upstream after #5043 merges.Checklist