Skip to content

perf: start check sooner on each hook edit (compile cache, kept detection, the CLI npx started) - #263

Merged
dgesteves merged 9 commits into
mainfrom
fix/os2-56-hook-speed
Oct 30, 2026
Merged

dgesteves merged 9 commits into
mainfrom
fix/os2-56-hook-speed

Conversation

@dgesteves

@dgesteves dgesteves commented Oct 30, 2026 •

Copy link
Copy Markdown
Owner

What was wrong

The Claude Code hook starts a new check process on every edit, and each one did three things all over again. In a project without onsystem installed, npx started npm first: the hook took 1.15 to 1.67 s per edit through npx, against 0.51 to 0.70 s when it ran the CLI directly (measurements below). Then the CLI and TypeScript were compiled: onsystem --version alone took 0.29 s, against 0.05 s for bare Node (node -e 0). Then zero config followed the design-system package's barrels to its components, parsing every file they lead to. In a CPU profile of check on one warm file, detectProject took 274 ms of 805 ms on dub's apps/web and 115 ms of 525 ms on midday's dashboard. Almost all of that was fromPackage → exportedFiles → ts.createSourceFile. src/bin.ts never turned on Node's compile cache.

What changed

  • Compile cache (src/bin.ts). The bin turns on module.enableCompileCache() (Node.js 22.1 and later; skipped on 20 and when NODE_DISABLE_COMPILE_CACHE is set) before it loads the CLI, which it now imports dynamically for that reason. The cache goes in onsystem-<uid> in the system temp folder, the folder the model cache already uses for projects without node_modules, and only while that folder is the user's alone (below). The bin creates compile-cache in it itself, with mode 0700 (set again after mkdir, which applies the umask: Node.js would create it 0777 under umask 000), and turns the cache on only while that folder passes the same rule. Node's default (<tmp>/node-compile-cache) is shared on a multi-user Linux host. The built dist/cli.js imports only the small own-folder chunk before the cache is on.
  • Detection kept on disk (src/detect.ts). A design system that zero config finds in one dependency's exports is kept beside the cached model (node_modules/.cache/onsystem/detection.json, or onsystem-<uid>/<id>.detection.json). The next run reads it back while all of these hold: every dependency it looked at is still found at the same folder (the same PackageFinder lookups run again); no file or folder in those folders changed size or mtime (found by walking them, without node_modules and .git); the app's package.json and components.json are unchanged; and the same build of onsystem runs. So an agent editing the app gets the kept detection, and an edit inside the design-system package (a new component in the barrel, a renamed export) makes it detect again. The brief proposed keying on the mtimes of package.json, tsconfig and the lockfile. That would miss a barrel that gains an export, so I went with the folder walk instead. It costs a few ms: 504 files in dub's packages/ui take about 3 to 8 ms to walk and stat.
    • It is kept only in the case it was measured on: one design system, found through a dependency's exports. When detection also read components.json, the root's own exports, the app's code (packages imported by path, ranking several design systems) or a package's tsconfig, nothing is kept and it runs as before. For the shadcn demo, detection takes 24 ms, so there is little to save there.
    • It is not kept when any file or folder it read changed in the last 2 s. A file written again within the same tick of its file system's clock keeps its mtime, and at the same size the change would go unseen; a file removed while detection runs changes its folder's mtime. inspect --explain (a trace) always detects again.
    • --no-cache neither reads nor keeps it (LoadConfigOptions.cache, passed by the CLI with its other config options and carried on to workspace projects and the server's config reloads), as scripts/corpus.mjs and scripts/perf.mjs expect.
    • Its key holds a stamp of the running build: the size and mtime of every file in the module's folder (dist, or src when run from source), so a detection kept by another build, a local edit to the detection code included, is not read.
    • Folders with more than 5,000 files are not watched. That is now kept too, so they are not walked on every run, until the folders themselves change.
  • The CLI npx started (plugins/onsystem/hooks/check-ui.mjs, src/bin.ts). When the hook runs npx, it passes ONSYSTEM_REMEMBER_CLI, a file in that same user-only folder, npx-cli-<key>, one per release, npx and project folder (npx runs a project's own install of the release when it has one, which belongs to that project alone). The CLI writes its own path there. On later edits in that project the hook runs that script with Node directly, but only if the file is a file, not a link, that only the user can write to, and the script's package.json says onsystem at the plugin's exact version with that script as its bin. Otherwise (another release, the npm cache cleaned, another project), it runs npx as before. When a run of the remembered script gives no results (a crash, output that is not JSON, a script that is gone, as when part of npm's _npx folder was removed), the hook removes the npx-cli-<key> file and runs npx once in that same edit, with ONSYSTEM_REMEMBER_CLI, so npx can install it again and the CLI it starts writes its path anew. Not after the time limit, which npx would not get. A broken config fails both ways, so while it is broken an edit runs the remembered script and then npx before the one-time notice. The CLI writes the file only when it is an npx-cli-<key> file directly in its own folder, with mode 0600, written beside it and renamed into place, so a link there is replaced rather than written through. I let the CLI report its own path instead of having the hook work out npm's _npx/<hash> layout or read npm's config, so it does not depend on where npm keeps its cache.
  • One rule for onsystem-<uid> (src/util/own-folder.ts). The bin, detection and the model cache share one helper, and the hook, which runs without the package, keeps a copy of the same rule. The folder is created with mode 0700 and used only when it is a folder, not a link; on Linux and macOS it must belong to the user and nobody else may write to it. One that others can write to is now skipped rather than closed and used, for the model cache too, since a file put in it before would stay. On Windows, which has no owner check here, it is used only inside the user's home folder (links resolved), where Windows keeps each user's temp folder. When the model cache cannot use it, the warning says to set TMPDIR (TEMP on Windows) to a folder of your own, inside your home folder where the Windows rule applies.
  • Docs. SECURITY.md lists what is kept in that folder (the model, the detection, the compile cache and the remembered CLI path) and these rules; docs/plugin.md and docs/how-it-works.md say the hook runs npx on the first edit in a project, then the CLI npx installed, and docs/plugin.md and SECURITY.md say it goes back to npx in the same edit when that CLI gives no results.

The PRIMITIVES copy in the hook is unchanged. The one change in src/cli.ts is the cache: values.cache line in the options it passes to loadConfig.

Measured

On an Apple M1 Max, Node.js 24.18.0. These are medians of 8 interleaved runs of main's build against this branch's first build, with the 1-minute load between 8 and 11, on the corpus checkouts (no node_modules) after a first run:

Run main this branch
onsystem --version 287 ms 204 ms
check on midday's transaction-details.tsx 505 ms 330 ms
check on dub's ui/token-avatar.tsx 767 ms 434 ms
The hook on that midday file, with the CLI run directly (ONSYSTEM_BIN) 549 ms 375 ms

For comparison, the same hook edit through npx with the published onsystem@0.11.1 (isolated npm cache, after a first run that filled it) took 1.15 to 1.67 s, median 1.58 s over 6 runs, with the load rising to 14. Once this ships, every edit after the first in a project runs the CLI npx installed the way the last row does. The review fixes did not change these paths enough to re-measure: the build stamp walks the 14 files of dist.

How it was tested

  • test/hook-speed.test.ts (24 tests):
    • The kept detection is read back while nothing changes and survives edits to the app's own files, and a trace detects again.
    • It is detected again when the package's barrel gains a component, when a file is rewritten at the same size, when the barrel is cut down, when the app's package.json drops the dependency, when a components.json appears, and when a design-system dependency that was missing joins the workspace.
    • Nothing is kept for just-written files, for a folder that just changed (a file removed), or for a package imported by path.
    • loadConfig({ cache: false }) and inspect --no-cache detect again, and leave the kept file as it was.
    • The key holds the stamp of src/, a detection kept under another stamp is not read, and the stamp changes with any file of the build.
    • A package with more than 5,000 files is remembered as such and not walked again until its folder changes.
    • It is kept in node_modules/.cache when the app has one.
    • It is neither read from nor kept in a temp folder that others can write to, and that folder is left as it is.
    • With the platform taken for Windows, the folder is trusted only inside the home folder, and never through a link.
    • The hook runs npx once, then the CLI it started, and goes back to npx when that is another release or gone; it remembers a project's own copy that npx ran for that project only; it does not run what a link, or a file others can write to, names. The CLI writes its path only to an npx-cli-<key> file in its own folder, with mode 0600, and replaces a link there. With the hook's platform taken for Windows, it remembers the CLI only when the temp folder is inside the home folder. These tests run src/bin.ts (transpiled with TypeScript, with a stand-in for the CLI) under a stand-in npx.
    • When the remembered script crashes (part of npm's copy removed) the hook removes its file and runs npx in the same edit, with no notice, and the next edit runs the new copy directly; when npx fails too, each edit runs npx once and shows the notice; when the remembered script runs out of time, npx is not started and the file is kept.
    • The compile cache lands in onsystem-<uid>/compile-cache; neither it nor the remembered path is used when that folder is writable by others. Under umask 000, onsystem-<uid> and compile-cache are both 0700, and a compile-cache that is a link, or that others can write to, is skipped and left as it is.
    • The cache warning's advice, with the platform taken for Windows and without.
  • test/cache-folder.test.ts: the model cache is neither read from nor written to a temp folder others can write to, with a warning that says what to do.
  • pnpm install --frozen-lockfile --prefer-offline, pnpm format:check, pnpm lint, pnpm typecheck, pnpm test (1483 passed, 3 skipped), pnpm build and pnpm smoke all pass, and so do pnpm --filter site generate and pnpm --filter site test (295 passed).
  • By hand, node dist/cli.js --version under umask 000 with TMPDIR pointing at an empty folder: onsystem-<uid> and compile-cache were drwx------; the versioned folder Node.js creates inside compile-cache was drwxrwxrwx, reachable only through the 0700 folder above it.
  • By hand, with the built CLI on a small monorepo whose kept detection was edited to list no components: check app read it (0 errors), check app --no-cache and inspect --no-cache detected again (1 error), and touching one file in dist made the next run detect again.
  • Earlier on this branch, on midday and dub, the full check . --format json output was byte-identical with the kept detection and with a fresh one. pnpm corpus runs with --no-cache, so it now neither reads nor keeps detections; I did not run it again for the review fixes.

Follow-ups

  • Perf table. I left the table in docs/how-it-works.md (Performance and footprint) as it is. Its npx row measures the published package from npm, so it can only show this change after the release. At the release-phase re-measure, run node scripts/perf.mjs --write. Its npx row will then measure edits after the first, which run the CLI npx installed, so relabel that row. The sentence above the table already says npx runs on the first edit in a project only.
  • Resolver setup. checkSource (resolver setup, about 48 ms of a midday check) is the next per-process cost. It lives in src/lint/, which OS2-59 owns, so it is not part of this PR.

Roadmap item: OS2-56.

The Claude Code hook starts a new `check` process on every edit. Each run
compiled the CLI and TypeScript again, followed the design-system package's
barrels again (parsing every file they lead to), and, without a project
install, started npm through npx first.

- src/bin.ts turns on Node's compile cache (22.1+) in the user's own temp
  folder, onsystem-<uid>, before it loads the CLI.
- src/detect.ts keeps a detection found in one dependency's exports beside
  the cached model, and reads it back while the dependency is found at the
  same folder and no file in that folder, nor the app's package.json or
  components.json, has changed size or mtime. Files changed in the last two
  seconds are not trusted to show a change, so nothing is kept then.
- The hook passes ONSYSTEM_REMEMBER_CLI to npx runs; the CLI writes its own
  path there, and later edits run that CLI with Node directly while it is
  still the plugin's release.

OS2-56
…e user's temp folder

- --no-cache reaches zero-config detection (LoadConfigOptions.cache): it neither reads nor
  keeps a detection. The CLI passes it with its other config options; workspace projects and
  the server's config reloads carry it on.
- The kept detection's key holds a stamp of the running build (size and mtime of each file in
  the module's folder), so a detection kept by another build is not read.
- The stamp also takes the mtime of each folder it walks, so a folder that just changed is not
  kept, and folders with too many files to watch are remembered rather than walked on each run.
- src/util/own-folder.ts: one rule for onsystem-<uid>, shared by the bin, detection and the
  model cache, and copied in the hook. A folder, not a link; on Linux and macOS the user's and
  not writable by others (refused, no longer closed and used); on Windows, inside the home folder.
- The hook remembers the CLI npx started per project folder, and reads that file only when it
  is a file only the user can write to. The CLI writes it only as an npx-cli-<key> file directly
  in its own folder, mode 0600, replacing a link rather than writing through it.
- SECURITY.md and docs/plugin.md list what is kept there and when it is used.
…ls; compile cache folder for the user alone

- The hook forgets the CLI it remembered when a run of it gives no results (a crash, no JSON,
  a script that is gone) and runs npx once in that same edit, which writes the path again. Not
  after the time limit, which npx would not get.
- bin.ts creates compile-cache itself (mode 0700, set again after mkdir, which applies the
  umask) and turns the compile cache on only while that folder passes the same rule as its
  parent: not a link, the user's, and no one else can write to it.
- The model cache's warning says to set TMPDIR (TEMP on Windows), and inside the home folder
  where that rule applies.
- docs/how-it-works.md: npx runs on the first edit in a project; later edits run the CLI npx
  installed with Node.js directly. The perf table is left for the release re-measure.
@dgesteves
dgesteves merged commit 5e67a83 into main Oct 30, 2026
19 checks passed
@dgesteves
dgesteves deleted the fix/os2-56-hook-speed branch October 30, 2026 20:19
@dgesteves-release dgesteves-release Bot mentioned this pull request Oct 30, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant