Repository navigation
fix(dynamodb): return LastEvaluatedKey when Limit ends the page - #72
Open
kylecannon wants to merge 1 commit into
Open
kylecannon wants to merge 1 commit into
kylecannon wants to merge 1 commit into
Conversation
DynamoDB returns a LastEvaluatedKey whenever Limit stops a Query or Scan, even if nothing is left, and the next page just comes back empty. rustack only returned it when more items followed, so code that pages with Limit stopped a page early compared to AWS. LocalStack behaves the same way as this change. Query, Scan and GSI queries now share one paging helper for it. Returning a key on the last page means the follow-up request always carries a start key, which turned up three places that didn't honour it: - queries on tables without a sort key returned the same item forever - `sk = :v` queries ignored the start key and looped the same way - a scan whose start item had been deleted started over from the top All three now move past the start key.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
DynamoDB returns a LastEvaluatedKey whenever Limit stops a Query or Scan, even if nothing is left, and the next page just comes back empty. rustack only returned it when more items followed, so code that pages with Limit stopped a page early compared to AWS. LocalStack behaves the same way as this change. Query, Scan and GSI queries now share one paging helper (
take_pageinstorage.rs) for it.Returning a key on the last page means the follow-up request always carries a start key, which turned up three places that didn't honour it:
provider.rsnow maps the missing sort key to the sentinel the storage layer uses)sk = :vqueries ignored the start key and looped the same way (collect_with_conditioninstorage.rs)scannow resumes by position in scan order)All three now move past the start key.
Test plan
rustack-dynamodb-corefor the key on a Limit-sized last page (storage and GSI), paging a hash-only table and ansk = :vquery withLimit: 1, and resuming a scan after the start item is deleted.cargo test -p rustack-dynamodb-corepasses.