Load an album's index off the event loop

[lstein]

The cost

Embeddings.indexes and open_cached_embeddings are a full np.load of the index whenever the three-entry lru_cache misses — any request touching a fourth album, and the first request after the index is rewritten. The load unpickles the per-image metadata object array, copies four arrays, and sorts one.

Measured on this branch: 0.34s for a 50,000-image index (159 MB) with modest synthetic metadata and a warm page cache. A real library with InvokeAI generation metadata and a cold cache is several times that.

Six async def endpoints did that inside the coroutine, which stops every other request for the duration — the slideshow, thumbnail fetches, and the indexing-progress polling the user is staring at while they wait:

file	endpoint	call
umap.py	/umap_data/	umap_embeddings + open_cached_embeddings
search.py	/image_info/	.indexes
search.py	/get_metadata/	.indexes
search.py	/lookup_image_indices/, /get_image_by_name/	.indexes
invoke.py	/recall_parameters/, /use_ref_image/	via _load_raw_metadata / _load_image_path
index.py	/index_metadata/	open_cached_embeddings

What changed

Two async accessors on Embeddings — load_indexes() and load_cached_embeddings() — that do the work in a thread, plus asyncio.to_thread at the two invoke.py call sites (the helpers there are plain sync utilities used from three places, so hopping at the call site beats making them async).

The blocking property keeps a docstring saying it is blocking and pointing at the async form.

The race this exposes, fixed here

Moving reads into worker threads makes one existing interleaving reachable, so it is fixed rather than left behind.

remove_images_from_embeddings clears the index cache, writes, then re-primes it "to verify the write". A reader that missed the cache before the clear can finish its load at any point after it and store the pre-delete snapshot — and then the re-prime is a cache hit that verifies nothing, so every later request goes on serving an image that is no longer in the index.

Clearing again immediately before the re-prime makes it a real load. This is what update_image_path already does a few hundred lines below, for exactly this reason.

Deliberately not included

The delete endpoints still run their load-and-rewrite on the loop. That one cannot simply be threaded: the event loop is currently what serializes concurrent deletes, and moving it without a per-index lock would let two deletes race and lose one. Worth doing, but it needs the lock, and that is a different change.

Tests

tests/backend/test_event_loop_blocking.py — 5 tests, all failing on master:

• four endpoints asserted to load the index only on a worker thread, using asyncio.get_running_loop() (which succeeds only on the loop thread) rather than thread names, and asserting a load happened so they cannot pass vacuously;
• the delete race, driven deterministically by priming the cache from the still-unmodified file during the write — precisely what a reader that loaded just before the rename leaves behind.

696 backend tests pass; ruff clean.

🤖 Generated with Claude Code