Skip to content

Let isolate types split their template builders across TUs - #8138

Open
danlapid wants to merge 2 commits into
mainfrom
dlapid/split-isolate-templates
Open

danlapid wants to merge 2 commits into
mainfrom
dlapid/split-isolate-templates

Conversation

@danlapid

@danlapid danlapid commented Nov 1, 2026 •

Copy link
Copy Markdown
Collaborator

Most of an isolate type's code is its resource types' template builders. These are the code that ResourceWrapper::getTemplate() instantiates to bind each type's members to JavaScript. All of it compiles, and is optimized at link time, in whichever translation unit uses the isolate type. For an application with hundreds of types, that one TU becomes both the longest compile and the longest ThinLTO backend job in the link.

This adds JSG_DECLARE_SPLIT_ISOLATE_TYPE. It is the same as JSG_DECLARE_ISOLATE_TYPE, except that the builders are compiled where the application registers them:

  • getTemplate() calls the builder through a per-type function pointer. registerTemplate() sets that pointer, so only the TU that registers a type instantiates its builder.
  • The application defines Type_TypeWrapper::registerTemplates(), and the TypeWrapper constructor calls it once per process. It calls one function in each of N TUs; the ith TU calls registerTemplateShard<i, N>(), which registers every Nth resource type of the declaration. Types added to the declaration later are covered with no further changes.
  • One of those TUs also calls registerContextTemplate<Global>(). It registers the context templates of Global and of its JSG superclasses, which the context template inherits.
  • Building the template of a type that was never registered throws.

Isolates declared with JSG_DECLARE_ISOLATE_TYPE are unchanged; the new paths are behind if constexpr. The mechanism is documented in docs/jsg.md, and split-isolate-test.c++ covers it, including an inherited context global and an unregistered type.

Results in edgeworker. Its isolate's builders are split over 8 TUs. This is a release build with ThinLTO and CFI, on bazelflare:

before after
slowest compile 183s 57s
slowest ThinLTO backend job 288s 60s
link peak RSS 7.0 GiB 6.3 GiB

Downstream, edgeworker is moving to Bazel's indexed (distributed) ThinLTO, where each TU's backend is a separate action. There, the slowest TU's compile and backend job are on the critical path of the link however many executors are available, which is what this split shortens.

workerd. The second commit applies the same split to workerd-api.c++'s JsgWorkerdIsolate, over 8 TUs. In a release build on bazelflare, its slowest compile drops from 109s to 40s; the shards take 17–31s.

The workerd-api target keeps its no-clang-tidy tag, and with it the new files.

Windows keeps the unsplit layout. Every shard parses all of the isolate's headers again, so the split trades total work for latency: the nine TUs take about 12–14 GB and 230s of CPU together, against 3.8 GB and 109s before. The Windows runner compiles 16 actions at once on one machine, and clang-cl ran out of memory there ("out of memory", "the paging file is too small"). So Windows builds leave out the shards and declare the isolate with JSG_DECLARE_ISOLATE_TYPE, as before.

🤖 Generated with Claude Code

danlapid and others added 2 commits November 2, 2026 00:10
Most of an isolate type's code is its resource types' template
builders: getTemplate() instantiates, for every resource type, the
code that binds its members to JavaScript. All of it is compiled,
and optimized at link time, wherever the isolate type is used,
which for an application with hundreds of types makes that one
translation unit the longest compile and the longest ThinLTO
backend job in the link.

JSG_DECLARE_SPLIT_ISOLATE_TYPE declares an isolate type whose
builders are instead compiled where the application registers
them. getTemplate() calls a builder through a per-type function
pointer that registerTemplate() sets, so only the registering
translation unit instantiates the builder. The application defines
registerTemplates(), which the TypeWrapper calls once per process.
It calls registerTemplateShard<i, N>() from N translation units,
the ith of which registers every Nth resource type of the
declaration, so types added later are covered without further
changes. One of them also calls registerContextTemplate<Global>(),
which registers the context templates of Global and of its JSG
superclasses. Building an unregistered type's template throws.

In edgeworker, splitting its isolate's builders over eight
translation units cut the longest compile from 183s to 57s and the
longest ThinLTO backend job from 288s to 60s.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
workerd-api.c++ instantiated the template builder of every one of
JsgWorkerdIsolate's resource types, which made it the slowest
compile in the workerd build: 109s in a release build.

Declare JsgWorkerdIsolate with JSG_DECLARE_SPLIT_ISOLATE_TYPE in
workerd-api-isolate.h, and register its builders from eight
workerd-api-templates-<n>.c++ translation units, the nth of which
registers every eighth type, starting with the nth. Shard 0 also
registers ServiceWorkerGlobalScope's context template. The slowest
of the nine translation units, workerd-api.c++ itself, now compiles
in 40s.

Each shard parses all of the isolate's headers again, so the split
trades total work for latency: together the nine translation units
take about 12-14 GB and 230s of CPU, against 3.8 GB and 109s
before. On the Windows CI runner, which compiles 16 actions at once
on one machine, that ran clang-cl out of memory. Windows builds
therefore leave out the shards and define
WORKERD_API_UNSPLIT_TEMPLATES, which declares JsgWorkerdIsolate
with JSG_DECLARE_ISOLATE_TYPE as before. Both declarations use the
JSG_WORKERD_ISOLATE_TYPES list.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@danlapid
danlapid force-pushed the dlapid/split-isolate-templates branch from 963afa7 to e7f9027 Compare November 2, 2026 05:10
@danlapid
danlapid marked this pull request as ready for review November 2, 2026 05:14
@danlapid
danlapid requested review from a team as code owners November 2, 2026 05:14

@clanky-the-clanker clanky-the-clanker Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💬 Approved with comments — No correctness, security or performance problems found in the split-isolate mechanism or the workerd-api shards. The only comment is an optional suggestion to point to it from the JSG AGENTS.md guidance.

Findings: 1 suggestion

Not opened by the correctness review: 6 changed files it was meant to read
  • src/workerd/server/workerd-api-templates-2.c++
  • src/workerd/server/workerd-api-templates-3.c++
  • src/workerd/server/workerd-api-templates-4.c++
  • src/workerd/server/workerd-api-templates-5.c++
  • src/workerd/server/workerd-api-templates-6.c++
  • src/workerd/server/workerd-api-templates-7.c++

The decision is advisory: this review is a comment and never approves, blocks or requests changes on the pull request.


📖 AGENTS.md: 💬 1 suggestion

Optional pointer to the split-isolate registration convention

Recommendation: 💬 Optional Nits

  • [SUGGESTION] Consider adding a brief pointer in src/workerd/jsg/AGENTS.md to the split-isolate registration requirements documented in docs/jsg.md (registerTemplateShard / registerContextTemplate).
View full review

JSG_DECLARE_SPLIT_ISOLATE_TYPE (src/workerd/jsg/type-wrapper.h) adds an isolate declaration pattern that has to be paired with shard and context-template registration. It is documented in docs/jsg.md. A short pointer in src/workerd/jsg/AGENTS.md would help someone extending a split isolate find the registration requirement, for example when adding a new context global type.


✅ All other checks passed

🔒 Security: ✅ LGTM

This is a build-layout refactor, and it does not touch any security boundary.


📋 Code Quality: ✅ LGTM

No issues found. Points checked:

  • Context-template inheritance: registerContextTemplate<Global>() walks jsgSuper until it reaches Object. That covers every context template that the inherited getTemplate<true> path builds.
  • Registration slots: The per-type function pointers are static members of ResourceWrapper<TypeWrapper, T>, so each isolate type has its own slots. The two isolates in split-isolate-test.c++ do not interfere with each other.
  • Registration timing: The static const bool registered in the TypeWrapper constructor runs registerTemplates() once per process. Function-local static initialization is thread-safe, and it happens before any template is built.
  • Context types: For JsgWorkerdIsolate, ServiceWorkerGlobalScope is the only context global type, and shard 0 registers it.
  • Unsplit path: The unsplit path (JSG_DECLARE_ISOLATE_TYPE, and the Windows WORKERD_API_UNSPLIT_TEMPLATES build) still calls the builder directly. The builder body is the previous inline code, moved unchanged.
  • Linkage: JsgWorkerdIsolate moves out of the anonymous namespace into workerd::server. No other definition of that name exists, so there is no ODR conflict.

⚡ Performance: ✅ LGTM

No runtime performance issues. The extra indirect call through the per-type function pointer happens only when a template is built, and each template is built once and then cached.


📝 Documentation: ✅ LGTM

The new section in docs/jsg.md matches how the code behaves.


Reviewed commit: e7f9027d

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant