Repository navigation
Every list route in the API — /pools, /pairs, /candles and /prices/history — now pages with the same cursor and limit parameters and the same opaque cursor shape that /screener already used, backed by keyset pagination so rows inserted while a client pages can never shift a page boundary, with all defaults and maxima documented per route in openapi.yaml and the behaviour locked down by tests including one that inserts a row mid-page. - #429
Open
DevScoopee wants to merge 2 commits into
Open
DevScoopee wants to merge 2 commits into
DevScoopee wants to merge 2 commits into
Conversation
|
@DevScoopee Great news! 🎉 Based on an automated assessment of this PR, the linked Wave issue(s) no longer count against your application limits. You can now already apply to more issues while waiting for a review of this PR. Keep up the great work! 🚀 |
…story — now pages with the same cursor and limit parameters and the same opaque cursor shape that /screener already used, backed by keyset pagination so rows inserted while a client pages can never shift a page boundary, with all defaults and maxima documented per route in openapi.yaml and the behaviour locked down by tests including one that inserts a row mid-page. Problem. /pools, /pairs, /candles and /prices/history returned everything they had in one response. That is tolerable today, but each venue added — Phoenix, Blend and DeFindex are queued — grows those responses without bound, and the first casualty will be a mobile client timing out on /pools. Offset pagination would not have been a fix either, because a row inserted while a client pages shifts every subsequent page and produces duplicates and skipped rows. What changed. The cursor-and-limit convention already proven on /screener has been extracted into a shared helper (src/pagination.ts, providing parseLimit, encodeCursor/decodeCursor and slicePage) and applied to every list route rather than reinvented per route. Every route now accepts the same two parameters: an opaque cursor, which is a base64 JSON [sortValue, id] tuple passed through unchanged from the previous page's nextCursor, and a limit, which is clamped into [1, max] and rejected with a 400 if it is not a valid integer. The cursor shape is identical everywhere, so a client can write a single paging loop for the whole API. Pagination is keyset-based, not offset-based: each route filters strictly past the cursor's sort value, so a row inserted mid-page cannot shift a page boundary. Handlers fetch limit + 1 rows to compute hasMore, so no COUNT(*) over the whole table is needed. Specifically, /pools keysets on pool_id with a default limit of 50 and a maximum of 200, /pairs on pairKey with the same 50/200, /candles on the bucket start time with 500/5000, and /prices/history on the snapshot timestamp with 1000/10000, reusing MAX_HISTORY_POINTS as the ceiling. When /prices/history is called with format=csv, the byte-exact CSV body stays untouched and the pagination envelope travels in the X-Next-Cursor and X-Has-More response headers instead. Closes Divinity-Labs#263
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem. /pools, /pairs, /candles and /prices/history returned everything they had in one response. That is tolerable today, but each venue added — Phoenix, Blend and DeFindex are queued — grows those responses without bound, and the first casualty will be a mobile client timing out on /pools. Offset pagination would not have been a fix either, because a row inserted while a client pages shifts every subsequent page and produces duplicates and skipped rows. What changed. The cursor-and-limit convention already proven on /screener has been extracted into a shared helper (src/pagination.ts, providing parseLimit, encodeCursor/decodeCursor and slicePage) and applied to every list route rather than reinvented per route. Every route now accepts the same two parameters: an opaque cursor, which is a base64 JSON [sortValue, id] tuple passed through unchanged from the previous page's nextCursor, and a limit, which is clamped into [1, max] and rejected with a 400 if it is not a valid integer. The cursor shape is identical everywhere, so a client can write a single paging loop for the whole API. Pagination is keyset-based, not offset-based: each route filters strictly past the cursor's sort value, so a row inserted mid-page cannot shift a page boundary. Handlers fetch limit + 1 rows to compute hasMore, so no COUNT(*) over the whole table is needed. Specifically, /pools keysets on pool_id with a default limit of 50 and a maximum of 200, /pairs on pairKey with the same 50/200, /candles on the bucket start time with 500/5000, and /prices/history on the snapshot timestamp with 1000/10000, reusing MAX_HISTORY_POINTS as the ceiling. When /prices/history is called with format=csv, the byte-exact CSV body stays untouched and the pagination envelope travels in the X-Next-Cursor and X-Has-More response headers instead.
Closes #263