Skip to content

fix(atomicmarket): read the stored template mint of an unlinked listing and cap the collection names so the effective mint filters hold on a large collection - #235

Merged
robrigo merged 1 commit into
mainfrom
fix/eng-1632-effective-mint-stored-range
Oct 6, 2026
Merged

robrigo merged 1 commit into
mainfrom
fix/eng-1632-effective-mint-stored-range

Conversation

@robrigo

@robrigo robrigo commented Oct 6, 2026

Copy link
Copy Markdown
Contributor

The effective mint filter and sort of #234 need collection_name, but their cost still grew with the count of listings in the collection, and on a large collection the queries passed the statement timeout. This PR makes a listing with no linked asset read the mint range its row already stores, and it caps the count of collection names for these parameters. #234 is not in a release, so no released behavior changes. Tracking: ENG-1632. Lane S.

Cause. The merged form read the atomicassets_assets heap row of every listing asset in scope, two times when the request also sorts. Measured with read-only EXPLAIN (ANALYZE, BUFFERS) on a replica of a database with 2.9 million listed sales, for bounds 1 to 10 and state listed: a collection of 135,065 listed sales passed a 30 s statement timeout for the /v2/sales list and for its count on a cold cache, and a collection of 53,220 listed sales took 27.2 s for the count, with 68,601 pages read from disk. /v1/sales for the largest collection passed 30 s also on a warm cache, where the same request takes 4.4 s with the template_mint filter.

Stored range. Each listing source gains a scalar test that no asset of the listing has a link row with an original mint. When the test is true and the listing has a stored range, the filter compares the stored listing.template_mint range with the bounds, and the sort reads its lower bound. That column is the value the template_mint filter reads: the lowest and highest template mint of the assets of the listing, or empty. A linked listing, and a listing whose stored range is not written yet, keep the aggregate over the asset and link rows from #234.

CASE WHEN listing.template_mint IS NOT NULL AND (SELECT COUNT(*) FROM atomicassets_original_mints om
            WHERE om.contract = listing.assets_contract AND om.asset_id = ANY(listing.asset_ids)
              AND om.original_mint IS NOT NULL) = 0
     THEN listing.template_mint <> 'empty'
          AND int8range(lower(listing.template_mint)::bigint, upper(listing.template_mint)::bigint) <@ int8range($min, $max, '[]')
     ELSE (/* aggregate of #234 */) <@ int8range($min, $max, '[]') END

The link test is a scalar count because the planner turns an EXISTS into a join, the form that #234 measured at 17 s to 26 s. The empty test comes first because an empty range has NULL bounds, which int8range reads as unbounded. The stored range is canonical with an exclusive upper bound, so it is rebuilt as [lower, upper) and compared in int8, and a bound above the int4 range does not overflow. The v1 sources run the same test through their listing asset table.

Behavior. The match rule, the parameters, and the 400 responses of #234 stay. On the four listing tables the filler writes template_mint in a batch after the listing is irreversible, so a new listing has no stored range for a time. Such a listing falls back to the aggregate, and it is found at once as in #234. On the replica 678 of the 135,065 listed sales of the largest collection had no stored range, and no auction or buyoffer had none, so the fallback is rare. The sales filter table of /v2/sales derives the range on each refresh and is never NULL. After an asset gets its mint, that range is stale until the refresh runs, as for the template_mint filter.

Cap. The collection_name requirement tested only that the parameter was present. /v2/sales leaves its indexed filter path for more than 50 names, which is the unscoped cost the requirement exists to stop. A request with min_effective_mint, max_effective_mint, or sort=effective_mint and more than 50 collection names now returns 400. The test is on the raw list length with the constant of the handler threshold, so 51 copies of one name also return 400. The handler tests its list after the whitelist and blacklist filters, so the cap is never weaker than the threshold. A request without these parameters keeps its behavior for any count of names.

Plans of the generated SQL on the same replica, bounds 1 to 10, state listed, sort=effective_mint:

Request #234 This PR
/v2/sales, collection of 36,740 listed sales, not read before not run count 0.7 s, list 1.2 s, then 0.9 s
/v2/sales, collection of 135,065 listed sales passes 30 s cold, list 2.3 s and count 1.8 s warm count 1.4 s, list 1.5 s to 1.6 s
/v2/sales, collection of 14,343 listed sales with linked assets 8.9 s cold, 0.7 s warm count 3.8 s on its first run, list 1.1 s, then 0.8 s
/v1/sales, collection of 135,065 listed sales passes 30 s cold and warm 27 s to over 30 s on a cold cache, 3.0 s warm, count 2.7 s
/v1/sales, collection of 14,343 listed sales with linked assets 8.5 s and 15 s cold, 1.0 s warm 9.0 s first run, 1.4 s warm, count 1.1 s

The generated queries return the row counts of #234 on the two collections compared, 862 and 34,426, on /v2/sales and on /v1/sales. Known limit: /v1/sales on the largest collection can still pass the statement timeout on a cold cache, because that handler reads the index of the listing asset table one time per listing for the link test (111,627 pages read in the 27 s run). Warm, it is faster than the same request without a mint filter, which took 5.0 s. A cheaper link test for the v1 handlers needs the collection on the link table, which is a schema change and is not in this PR.

Tests. Integration on a fresh PostgreSQL 16 database: 836 passing, 0 failing (779 on the base). Unit: 596 passing, 37 pending (596 on the base). pnpm check-types and pnpm lint exit 0. The 64 tests of #234 keep their assertions. The new shared cases cover an unlinked listing matched by its stored range, a linked listing matched by the aggregate, a stored range set apart from the asset mints, a bound above the int4 range, each bound alone, the sort with its count over linked and unlinked listings, and the cap: 50 names, 51 names, 51 copies of one name, 51 names with the sort only, and 51 names without a mint parameter. They also cover a listing whose stored range is not written, linked and unlinked, with exact ids and counts. On /v2/sales and /v1/sales/templates a test sets the range of the filter table apart from the asset mints, so it fails if the query goes back to the aggregate. The fixtures of the four listing tables now set the stored range as the filler does, because they left it NULL.

Reviews. One cold code review: 3 findings, 0 blocking, 3 applied (test gaps), and one design point applied, the fallback for a listing with no stored range. One cold security review of the final diff: 0 blocking, 1 informational note that a listing with a linked asset and a stored range now pays the link test before the same aggregate. It left one open decision, a further limit for the v1 handlers, where a request over large collections can still reach the statement timeout as it could in #234.

Not verified. No run of the image on a deployed chain. The plans come from hand-run SQL that the handlers generate, not from HTTP requests.

…ng and cap the collection names so the effective mint filters hold on a large collection

The effective mint filter and sort read the asset row of every listing
asset in scope. The cost grew with the listings of the collection, and
on a collection of 135,000 listed sales the list and count queries of
/v2/sales passed a 30 s statement timeout on a cold cache.

A listing with no linked asset now reads the template mint range that
the listing row stores, the value the template_mint filter reads. Only
a listing with a linked asset keeps the aggregate over its asset and
link rows. The link test is a scalar count on the link table, because
the planner turns an EXISTS into a join. The filler writes the stored
range in a batch after the listing is irreversible, so a listing with
no stored range yet also keeps the aggregate and is found at once.

The collection_name requirement tested only that the parameter was
present. A request with more than 50 names leaves the indexed filter
path of /v2/sales, so the effective mint parameters now return 400 for
more than 50 names.

The public parameters and the match rule do not change.

Signed-off-by: Rob Konsdorf <rob@facings.io>

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🟡 Changes recommended

The stored-range conversion mishandles the maximum int4 mint by turning its finite range into an unbounded int8 range.

Review effort: Balanced
Findings: 1 High severity

Open (1)
What changed in this PR

Optimizes effective-mint filtering by using stored listing ranges and limits expensive collection scopes.

Changes:

  • Uses stored mint ranges for unlinked listings.
  • Caps effective-mint requests at 50 collections.
  • Expands integration coverage.
File Description
CHANGELOG.md Documents behavior changes.
src/​api/​namespaces/​atomicmarket/​utils.ts Implements range optimization and cap.
src/​api/​namespaces/​atomicmarket/​openapi.ts Documents the collection limit.
src/​api/​namespaces/​atomicmarket/​handlers/​sales2.ts Shares the collection threshold.
src/​api/​namespaces/​atomicmarket/​effective-mint-suite.ts Adds shared regression cases.
src/​api/​namespaces/​atomicmarket/​handlers/​auctions.integration.test.ts Updates auction fixtures.
src/​api/​namespaces/​atomicmarket/​handlers/​buyoffers.integration.test.ts Updates buyoffer fixtures.
src/​api/​namespaces/​atomicmarket/​handlers/​template-buyoffers.integration.test.ts Updates template-buyoffer fixtures.
src/​api/​namespaces/​atomicmarket/​handlers/​sales.get-sales.integration.test.ts Updates v1 sales fixtures.
src/​api/​namespaces/​atomicmarket/​handlers/​sales2.integration.test.ts Tests stored sales-filter ranges.
src/​api/​namespaces/​atomicmarket/​handlers/​sales2.get-sales-templates.integration.test.ts Tests template-sales behavior and cap.

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment thread src/api/namespaces/atomicmarket/utils.ts
@robrigo
robrigo merged commit 498c7ea into main Oct 6, 2026
8 checks passed
@robrigo robrigo mentioned this pull request Oct 6, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants