Skip to content

feat(client): prefetch dataclass instances in one wave - #14

Open
ahinoamp wants to merge 3 commits into
Kitware:mainfrom
ahinoamp:feat/prefetch-wave
Open

ahinoamp wants to merge 3 commits into
Kitware:mainfrom
ahinoamp:feat/prefetch-wave

Conversation

@ahinoamp

@ahinoamp ahinoamp commented Oct 6, 2026

Copy link
Copy Markdown
Contributor

Problem

A Provider renders its slot only once its own instance is available, so providers that nest hydrate strictly one after another: the second one is not even created until the first one's state.get has come back, the third waits for the second, and so on. A page built from a handful of nested providers therefore pays a dependency chain as long as the nesting, rather than making its requests in parallel. On a page with around twenty of them the chain, not the cost of any one request, is what the user waits for.

What this adds

Three commits, each standalone:

  1. fix(client) — link awaited only when dataStates[id] was missing. That entry is created as soon as the state reply lands, before the class definition is cached and before the nested refs are built, so a component linking during that window gets an instance whose fields are still undefined. Serial hydration made the window hard to reach; anything that starts several instances together makes it ordinary. fetchState deduplicates by id, so awaiting it unconditionally is a wait, not a second request.
  2. perf(client) — typeDefinitions[id] is only filled when the reply lands, so every instance of a class hydrating at the same time finds it missing and asks again: N instances of one class cost N definition requests. The in-flight promise is now shared, the same way pendingFetch shares a state request.
  3. feat(client) — DataclassManager.prefetch(ids) starts every id hydrating at once and reports which ones are ready, plus the trame-dataclass-prefetch component and the dataclass.Prefetch widget around it.

prefetch is a thin reuse of fetchState: each id is still completed over its whole graph (children included), every request underneath is still deduplicated, and a provider that mounts later finds the work in flight rather than starting it.

with dataclass.Prefetch(instances=[model._id for model in models]):
    with models[0].provide_as("settings"), models[1].provide_as("scene"):
        html.Div("{{ settings.title }} / {{ scene.name }}")

Why it is not a barrier

The slot renders immediately and always. Opening it only once every id is ready would also suppress partial renders — but each provider already does that for itself, and a page-wide gate would buy nothing more at the price of a new way for the page to stay blank. Warming is strictly additive: if the component did nothing at all, every provider would fetch exactly as it does today. That is also why removing the wrapper from a working page is safe.

Why it is not batched

prefetch sends the same requests it would have sent anyway, just without waiting between them — no new RPC, no server change, no coalescing. A batch endpoint would cut the request count, but one slow or blocked instance would then hold back every unrelated instance travelling with it, and a partial failure would have to be unpicked per id. Promise.allSettled is the other half of that: one missing or refused instance must not take the rest of the wave down.

Measured

Generic demo, examples/getting_started/prefetch.py — 20 providers, one class, nested; run it with --serial for the same page without the wrapper. Chromium, median of five loads, time from navigation to the last provider's text:

latency model without Prefetch with Prefetch
40 ms round trip (TCP relay in front of the server) 1.31 s 0.49 s
40 ms per RPC, server answering one at a time 0.96 s 0.94 s, and 1.73 s without commit 2

The second row is what motivated the definition de-duplication: without it a wave asks for the shared class definition once per instance (41 requests instead of 21), which costs nothing on the wire when they overlap but is strictly worse against a server that serializes. With it the request count is unchanged from serial hydration — the wave only removes the waiting.

Tests

vue-components/tests/prefetch.cjs (7 cases, new npm test script — node --test, no new dependency): it runs the real core.js in a vm against a fake wslink session with real Vue refs, and mounts the real component through a host-less createRenderer, so there is no DOM and nothing is stubbed out but the transport. It covers: every id requested in one wave; an already-started or repeated id not fetched twice; children of a container hydrated too; one definition request per class per wave; a component linked mid-wave waiting for complete refs; the slot rendering while everything is still in flight; and a replaced instances list warming the new ids.

tests/test_widgets_prefetch.py (5 cases) covers the Python widget: a list of ids becoming a bound, single-quoted array literal (a bare list is read as trame's (expression, default) shorthand and silently renders an undefined prop; a json.dumps-style double-quoted literal closes the HTML attribute at its first inner quote), a tuple staying a binding, an id a quote could escape being refused, and client_type="react" being refused.

python -m pip install -e . pytest pytest-asyncio trame
python -m pytest -q tests                 # 19 passed, 1 xfailed (the xfail is pre-existing)
cd vue-components && npm ci && npm test   # 7 passed

pre-commit run passes on the changed files. Pointing the client suite at a core.js without commit 1 (DATACLASS_CORE=…) fails exactly the mid-wave linking case, which is what that commit is for.

Relation to #13

This does not depend on #13 and is based on main. The two touch core.js in different places — #13 rewrites the publication path and fetchState's promise construction, this adds prefetch, shares fetchDefinition's promise and changes link — and merging the two branches conflicts only on the adjacent constructor lines where each adds its own fields.

One honest overlap: on main, a failed state.get leaves fetchState pending forever rather than rejecting, so prefetch's per-id ready: false can only be observed once #13 (or something like it) makes that promise reject. allSettled is correct either way and costs nothing, so it is here rather than waiting; the failure path simply isn't reachable yet, and the tests don't claim it is. Both branches also add the same "test" script to vue-components/package.json.

Deliberately left out

  • No new RPC, no batching, no server-side change.
  • No retry, backoff or priority ordering, and no cancellation of a wave whose component unmounted: the ids are already in flight and a later provider will want them. The component only drops the report of a retired wave, so an unmount or a replaced list cannot log a failure against the live page.
  • No React component: the React client has its own manager, and the widget refuses client_type="react" rather than rendering a tag nothing implements.
  • No change to getGUITemplate, which tests allocation the way link used to, but only needs the definition.

Happy to drop any of the three commits, or split them into separate PRs, if you would rather take them apart.

🤖 Generated with Claude Code

ahinoamp and others added 3 commits October 7, 2026 01:02
…d one

`link` tested `dataStates[id]` to decide whether to wait. That entry is created
as soon as the state reply lands - before the class definition is cached and
before the nested refs are built - so a component that links during that window
is handed an instance whose fields are still undefined and renders them.

Serial hydration made the window hard to reach: each provider's slot waits for
its own instance, so the next one was not created until the previous one was
complete. Anything that starts several instances at once makes it ordinary.

`fetchState` already deduplicates by id, so awaiting it unconditionally is a
wait on work that is already under way rather than a second request.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
`typeDefinitions[id]` is only filled when the reply lands, so every instance of
a class that is hydrating at the same time finds it missing and asks for it
again: N instances of one class cost N definition requests. Nothing noticed
while instances hydrated one after another, because the first reply was cached
before the second instance started.

The request is now deduplicated the way `fetchState` deduplicates state
requests, by sharing the pending promise. A rejection removes its own entry -
conditionally, so a failure cannot evict a request that replaced it - which
leaves the class fetchable again.

Measured in Chromium on a 20-provider page whose instances share one class,
started together, with 40 ms added to every RPC on a server that answers one
at a time: 41 requests and 1.73 s to the last provider rendered, against 21
requests and 0.94 s with this change.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
A `Provider` only renders its slot once its own instance is available, so
providers that nest hydrate strictly one after another: the second is not
created until the first one's state has landed. A page with many of them pays
that chain end to end rather than making its requests in parallel.

`DataclassManager.prefetch(ids)` starts all of them at once and reports which
ones are ready. It is a thin reuse of `fetchState`, so each id is still
completed over its whole graph and every request underneath is still
deduplicated; nothing is batched, because one blocked instance holding
unrelated instances back is worse than the requests it would save.

`dataclass.Prefetch(instances=[...])` is the widget around it. It renders its
slot immediately and always: each provider already withholds its own subtree
until its own data is there, so a page-wide gate would buy nothing more at the
price of a new way for the page to stay blank. Removing the wrapper changes
when the data arrives and nothing else.

Measured in Chromium on a 20-provider page behind a proxy adding 40 ms of
round-trip latency (examples/getting_started/prefetch.py, with and without
`--serial`), median of five loads: 1.31 s to the last provider rendered
without the wrapper, 0.49 s with it.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
alesgenova added a commit that referenced this pull request Oct 16, 2026
`fetchState` meant "this instance, plus its children unless someone else was
already loading them": it skipped any child whose `dataStates` entry existed,
which is what kept cycles from deadlocking, and that entry exists before the
child's definition and refs. Every caller then built its own wait on top of it,
and `link`, `getGUITemplate`, publications and nested hydration each handed out
half-built instances in some interleaving.

It is split in two:

- `fetchInstance(id)` fetches one instance: its state, its class definition
  and its own refs. The instances it points at are started, never awaited, so
  it never waits on another instance and no cycle can deadlock it.
  `dataStates[id]` is now only set once the refs are built.
- `fetchInstanceRecursive(id)` waits for everything reachable from `id`, one
  level of the graph at a time, from outside any fetch. Instances it has
  completed are remembered and not walked again.

`link` waits for the whole graph (superseding f4e4323 from #14) and no longer
links a component that moved on to another instance meanwhile. Publications
wait for the graph of every instance they reference. `getGUITemplate` waits
for the instance's definition. `dataToVue` only existed to refresh components
linked to a half-built instance, so it is gone.

A failed fetch used to stay cached in `pendingFetch`, so every later
publication for that id re-awaited the same rejection and was dropped until
the page reloaded. The failed attempt is now forgotten and the next request
starts over.

Children are no longer fetched one after another, so every state and
definition request now goes through `callServer`, which keeps at most
`maxConcurrentCalls` (16) in flight. Class definitions skip the queue, since
one unblocks every instance of its class.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
alesgenova added a commit that referenced this pull request Oct 16, 2026
`fetchState` meant "this instance, plus its children unless someone else was
already loading them": it skipped any child whose `dataStates` entry existed,
which is what kept cycles from deadlocking, and that entry exists before the
child's definition and refs. Every caller then built its own wait on top of it,
and `link`, `getGUITemplate`, publications and nested hydration each handed out
half-built instances in some interleaving.

It is split in two:

- `fetchInstance(id)` fetches one instance: its state, its class definition
  and its own refs. The instances it points at are started, never awaited, so
  it never waits on another instance and no cycle can deadlock it.
  `dataStates[id]` is now only set once the refs are built.
- `fetchInstanceRecursive(id)` waits for everything reachable from `id`, one
  level of the graph at a time, from outside any fetch. Instances it has
  completed are remembered and not walked again.

`link` waits for the whole graph (superseding f4e4323 from #14) and no longer
links a component that moved on to another instance meanwhile. Publications
wait for the graph of every instance they reference. `getGUITemplate` waits
for the instance's definition. `dataToVue` only existed to refresh components
linked to a half-built instance, so it is gone.

A failed fetch used to stay cached in `pendingFetch`, so every later
publication for that id re-awaited the same rejection and was dropped until
the page reloaded. The failed attempt is now forgotten and the next request
starts over.

Children are no longer fetched one after another, so every state and
definition request now goes through `callServer`, which keeps at most
`maxConcurrentCalls` (16) in flight. Class definitions skip the queue, since
one unblocks every instance of its class.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant