Repository navigation
Return @search.nextPageParameters when a search leaves out top - #100
Open
danielgreen wants to merge 3 commits into
Open
danielgreen wants to merge 3 commits into
danielgreen wants to merge 3 commits into
Conversation
A POST search without top returned the first 50 results and no way to get the rest. Azure returns the same 50, plus @search.nextPageParameters and @odata.nextLink when more results match. The Azure Search SDKs follow these to read every result, so a client that enumerated a search saw only 50. SearchRequest.Top is now null when the request leaves it out, so the searcher can tell that apart from top: 50. The continuation is the same request with skip moved on and still no top, so each page gets a continuation of its own. A request that sets top gets no continuation, as in Azure. Not covered: GET searches, where Azure returns only a nextLink to another GET, and top above 1000, which still returns 400.
A search without top that started within 50 of skip 100,000, over more matches than that, returned @search.nextPageParameters with a skip above 100,000. Sending it back was refused with 400, so an SDK reading the results would throw part way through. Now no next page is offered once its skip would pass the limit, and the client stops after the last page it can request. The limit is SearchRequest.MaxSkip, which the controller's check uses too.
Check 10 uploads 120 more documents and searches with no Size, then follows each page through the SDK. It passes when every document is read: 123 in pages of 50, 50 and 23. Deleting the index becomes check 11.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #103
What was wrong
A POST search without
topreturned the first 50 results and no way to get the rest. Azure returns the same 50 plus@search.nextPageParametersand@odata.nextLinkwhen more results match. The Azure Search SDKs follow these to read every result, so a client that enumerated a search saw only 50.What changed
SearchRequest.Topis now null when the request leaves it out, so the searcher can tell that apart fromtop: 50.SearchRequest.DefaultTopis 50.topwas left out and more results match, the searcher reports the nextskip. The controller returns the same request withskipmoved on and still notop, so each page gets a continuation of its own, plus@odata.nextLinktodocs/search.post.search.topgets no continuation, as in Azure.skipwould pass 100,000 (SearchRequest.MaxSkip), since the emulator refuses thatskipwith 400. The client stops after the last page it can request.Not covered: GET searches, where Azure returns only a nextLink to another GET, and
topabove 1000, which still returns 400.Tests
SearchPagingTests(new, unit, 8 tests): the first page and its next skip; following each continuation over 120 documents gives pages of 50, 50 and 20 with every document once; no continuation when the matches fit on one page or whentopis set; and, over 100,100 documents, a continuation at the skip limit but none past it.SearchPagingIntegrationTests(new, 3 tests, against the container through the .NET SDK): enumerating withoutSizereads all 120 in pages of 50, 50 and 20;Size = 50reads 50; the REST response hasskip: 50, notop, the original search and filter, and the nextLink.main.azure-search-documents11.5.2: a search with notopyields all 120 results.All 981 unit tests pass.
dotnet testfrom the repo root currently stops before any test runs, onmaintoo: restoringAzureSearchEmulator.Aspirefails with NU1902, a moderate advisory for MessagePack 2.5.302 (GHSA-qhrr-8q5h-9q3h) that the project treats as an error. I ran the unit and integration test projects directly.