Repository navigation
Conversation
…tion pytest-benchmark cases for reader open, tile and point on a generated 4x720x1440 zarr store. The store is wrapped in zarr's `LatencyStore` (0 and 30 ms per `get`) so window-vs-slice reads and batched-vs-serial round trips show up as time, and a counting wrapper records `store_gets` per operation in extra_info as an exact, branch-comparable number. Skipped unless --benchmark-only/--benchmark-enable is given, so the normal suite is unaffected. The fixture already carries 2-D mask and flag variables so a masking feature can add its cases without changing the store. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
zarr only derives a copy of the store through `_with_store` when opening a writable store in mode "r". Wrapping a read-only LocalStore means the counting instance is used as-is, so the shared counter cell, the `_with_store` override and `reset()` can go; the count is a plain attribute. `_record` now asserts the count is non-zero so a future zarr change that reintroduces a copy fails loudly rather than reporting 0. zarr's LoggingStore was considered instead: it gives identical counts but calls inspect.stack() per store call (~0.7 ms each), which roughly doubled the 0 ms latency timings. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Relocking in the development sandbox wrote two things unrelated to this branch into uv.lock: - an [options] block with exclude-newer-span = "P1W" (the sandbox's package-install cooldown), which does not belong upstream; - four dependency markers removed (pycparser, h11, truststore, ptyprocess) by a different uv version; relocking an untouched main with that uv produces the same lines. Restore main's lines for both; the branch's own dependency change and the titiler-multidim 0.9.0 -> 0.9.1 self-version (main's lock is stale after the 0.9.1 release) are kept. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
maxrjones
marked this pull request as draft
October 2, 2026 19:17
Member
Author
|
I'm going to convert this to a draft. It's been useful for me testing out PRs, but I'm sure there's a better way to implement these benchmarks. So I'd like to take more time on this one before asking for a review. |
The existing cases drive XarrayReader directly, so they cannot see work the request-scoped mosaic backend adds on top (its compatibility validation opened every source a second time before fix/single-open-per-request). Add the single-URL backend tile and point paths so store_gets and latency for a real /tiles or /point request are measurable per branch. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Pin what each request type costs on main, so a performance PR reports its change as the diff of these numbers: - repository and dataset opens (/tiles and /point open twice today); - icechunk storage requests, metadata and chunks, by serving the local fixture over a Range-capable HTTP server and opening it through icechunk.http_storage, the same unit the performance reports use; - boto3.Session constructions, with AWS credentials in the environment as on Lambda (10 per tile, 3 per point, 2 per info). A session-scoped 17,228-step Icechunk fixture (TEMPO's time axis length, coarse chunks) shows the time coordinate fetched three times per open, which icechunk's chunk cache would remove. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
/info with and without sel, /info?show_times=true, /tiles and /point through TestClient on the long-time-axis fixture, so the per-time-step costs in rio-tiler's info() (about 1.8 s today) and the full request path are measurable per branch with pytest-benchmark compare. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Review findings on the counting helper: - the request kind was matched anywhere in the URL path, so a checkout or basetemp directory named chunks/, repo/, refs/ ... made every request count as that kind. Classify by the icechunk key under the store root instead, with a test that serves the fixture from a chunks/repo/ path. - icechunk issues chunk and manifest reads in parallel and Counter += is not atomic, so guard the increment with a lock. - docstrings: the long fixture's only stored chunk object is the time coordinate (~18 KB; the zero data chunks are inlined), so tile/point benchmarks on it do not fetch data chunks; rasterio needs both AWS variables; pytest-benchmark refuses --benchmark-only under xdist. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
main now opens each dataset once per request (#171), so /tiles, /point and /info?show_times=true pin 1 repository and 1 dataset open and 9 (or 8) metadata requests instead of 2 and 17 (16). Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
The counting server becomes a `storage_requests` yield fixture that calls `shutdown()` and `server_close()` after the test, so listening sockets and serve_forever threads no longer accumulate over a session. A test checks the port is closed afterwards. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
maxrjones
changed the title
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This PR adds some very minimal benchmarking. I created this in order to evaluate #151 relative to a dask-free alternative implementations.
Testing
PR checks
run-cdk-checkslabel to this PR.deploy-devlabel. It smoke-tests tiles from the native MUR, virtual MUR, and virtual NLDAS Icechunk stores after deployment.