Skip to content

Every list route in the API — /pools, /pairs, /candles and /prices/history — now pages with the same cursor and limit parameters and the same opaque cursor shape that /screener already used, backed by keyset pagination so rows inserted while a client pages can never shift a page boundary, with all defaults and maxima documented per route in openapi.yaml and the behaviour locked down by tests including one that inserts a row mid-page. - #429

Open
DevScoopee wants to merge 2 commits into
Divinity-Labs:mainfrom
DevScoopee:Only-/screener-can-be-paged

Conversation

@DevScoopee

Copy link
Copy Markdown

Problem. /pools, /pairs, /candles and /prices/history returned everything they had in one response. That is tolerable today, but each venue added — Phoenix, Blend and DeFindex are queued — grows those responses without bound, and the first casualty will be a mobile client timing out on /pools. Offset pagination would not have been a fix either, because a row inserted while a client pages shifts every subsequent page and produces duplicates and skipped rows. What changed. The cursor-and-limit convention already proven on /screener has been extracted into a shared helper (src/pagination.ts, providing parseLimit, encodeCursor/decodeCursor and slicePage) and applied to every list route rather than reinvented per route. Every route now accepts the same two parameters: an opaque cursor, which is a base64 JSON [sortValue, id] tuple passed through unchanged from the previous page's nextCursor, and a limit, which is clamped into [1, max] and rejected with a 400 if it is not a valid integer. The cursor shape is identical everywhere, so a client can write a single paging loop for the whole API. Pagination is keyset-based, not offset-based: each route filters strictly past the cursor's sort value, so a row inserted mid-page cannot shift a page boundary. Handlers fetch limit + 1 rows to compute hasMore, so no COUNT(*) over the whole table is needed. Specifically, /pools keysets on pool_id with a default limit of 50 and a maximum of 200, /pairs on pairKey with the same 50/200, /candles on the bucket start time with 500/5000, and /prices/history on the snapshot timestamp with 1000/10000, reusing MAX_HISTORY_POINTS as the ceiling. When /prices/history is called with format=csv, the byte-exact CSV body stays untouched and the pagination envelope travels in the X-Next-Cursor and X-Has-More response headers instead.

Closes #263

@drips-wave

drips-wave Bot commented Oct 17, 2026

Copy link
Copy Markdown

@DevScoopee Great news! 🎉 Based on an automated assessment of this PR, the linked Wave issue(s) no longer count against your application limits.

You can now already apply to more issues while waiting for a review of this PR. Keep up the great work! 🚀

Learn more about application limits

DevScoopee and others added 2 commits October 17, 2026 08:24
…story — now pages with the same cursor and limit parameters and the same opaque cursor shape that /screener already used, backed by keyset pagination so rows inserted while a client pages can never shift a page boundary, with all defaults and maxima documented per route in openapi.yaml and the behaviour locked down by tests including one that inserts a row mid-page.

Problem. /pools, /pairs, /candles and /prices/history returned everything they had in one response. That is tolerable today, but each venue added — Phoenix, Blend and DeFindex are queued — grows those responses without bound, and the first casualty will be a mobile client timing out on /pools. Offset pagination would not have been a fix either, because a row inserted while a client pages shifts every subsequent page and produces duplicates and skipped rows.
What changed. The cursor-and-limit convention already proven on /screener has been extracted into a shared helper (src/pagination.ts, providing parseLimit, encodeCursor/decodeCursor and slicePage) and applied to every list route rather than reinvented per route. Every route now accepts the same two parameters: an opaque cursor, which is a base64 JSON [sortValue, id] tuple passed through unchanged from the previous page's nextCursor, and a limit, which is clamped into [1, max] and rejected with a 400 if it is not a valid integer. The cursor shape is identical everywhere, so a client can write a single paging loop for the whole API. Pagination is keyset-based, not offset-based: each route filters strictly past the cursor's sort value, so a row inserted mid-page cannot shift a page boundary. Handlers fetch limit + 1 rows to compute hasMore, so no COUNT(*) over the whole table is needed. Specifically, /pools keysets on pool_id with a default limit of 50 and a maximum of 200, /pairs on pairKey with the same 50/200, /candles on the bucket start time with 500/5000, and /prices/history on the snapshot timestamp with 1000/10000, reusing MAX_HISTORY_POINTS as the ceiling. When /prices/history is called with format=csv, the byte-exact CSV body stays untouched and the pagination envelope travels in the X-Next-Cursor and X-Has-More response headers instead.

Closes Divinity-Labs#263
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Only /screener can be paged

1 participant