Repository navigation
perf: start check sooner on each hook edit (compile cache, kept detection, the CLI npx started) - #263
Merged
Merged
Conversation
The Claude Code hook starts a new `check` process on every edit. Each run compiled the CLI and TypeScript again, followed the design-system package's barrels again (parsing every file they lead to), and, without a project install, started npm through npx first. - src/bin.ts turns on Node's compile cache (22.1+) in the user's own temp folder, onsystem-<uid>, before it loads the CLI. - src/detect.ts keeps a detection found in one dependency's exports beside the cached model, and reads it back while the dependency is found at the same folder and no file in that folder, nor the app's package.json or components.json, has changed size or mtime. Files changed in the last two seconds are not trusted to show a change, so nothing is kept then. - The hook passes ONSYSTEM_REMEMBER_CLI to npx runs; the CLI writes its own path there, and later edits run that CLI with Node directly while it is still the plugin's release. OS2-56
…os.tmpdir() reads on Windows
…e user's temp folder - --no-cache reaches zero-config detection (LoadConfigOptions.cache): it neither reads nor keeps a detection. The CLI passes it with its other config options; workspace projects and the server's config reloads carry it on. - The kept detection's key holds a stamp of the running build (size and mtime of each file in the module's folder), so a detection kept by another build is not read. - The stamp also takes the mtime of each folder it walks, so a folder that just changed is not kept, and folders with too many files to watch are remembered rather than walked on each run. - src/util/own-folder.ts: one rule for onsystem-<uid>, shared by the bin, detection and the model cache, and copied in the hook. A folder, not a link; on Linux and macOS the user's and not writable by others (refused, no longer closed and used); on Windows, inside the home folder. - The hook remembers the CLI npx started per project folder, and reads that file only when it is a file only the user can write to. The CLI writes it only as an npx-cli-<key> file directly in its own folder, mode 0600, replacing a link rather than writing through it. - SECURITY.md and docs/plugin.md list what is kept there and when it is used.
…can keep its mtime within one tick
…ls; compile cache folder for the user alone - The hook forgets the CLI it remembered when a run of it gives no results (a crash, no JSON, a script that is gone) and runs npx once in that same edit, which writes the path again. Not after the time limit, which npx would not get. - bin.ts creates compile-cache itself (mode 0700, set again after mkdir, which applies the umask) and turns the compile cache on only while that folder passes the same rule as its parent: not a link, the user's, and no one else can write to it. - The model cache's warning says to set TMPDIR (TEMP on Windows), and inside the home folder where that rule applies. - docs/how-it-works.md: npx runs on the first edit in a project; later edits run the CLI npx installed with Node.js directly. The perf table is left for the release re-measure.
Merged
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What was wrong
The Claude Code hook starts a new
checkprocess on every edit, and each one did three things all over again. In a project without onsystem installed, npx started npm first: the hook took 1.15 to 1.67 s per edit through npx, against 0.51 to 0.70 s when it ran the CLI directly (measurements below). Then the CLI and TypeScript were compiled:onsystem --versionalone took 0.29 s, against 0.05 s for bare Node (node -e 0). Then zero config followed the design-system package's barrels to its components, parsing every file they lead to. In a CPU profile ofcheckon one warm file,detectProjecttook 274 ms of 805 ms on dub'sapps/weband 115 ms of 525 ms on midday's dashboard. Almost all of that wasfromPackage→exportedFiles→ts.createSourceFile.src/bin.tsnever turned on Node's compile cache.What changed
src/bin.ts). The bin turns onmodule.enableCompileCache()(Node.js 22.1 and later; skipped on 20 and whenNODE_DISABLE_COMPILE_CACHEis set) before it loads the CLI, which it now imports dynamically for that reason. The cache goes inonsystem-<uid>in the system temp folder, the folder the model cache already uses for projects withoutnode_modules, and only while that folder is the user's alone (below). The bin createscompile-cachein it itself, with mode 0700 (set again aftermkdir, which applies the umask: Node.js would create it 0777 under umask 000), and turns the cache on only while that folder passes the same rule. Node's default (<tmp>/node-compile-cache) is shared on a multi-user Linux host. The builtdist/cli.jsimports only the smallown-folderchunk before the cache is on.src/detect.ts). A design system that zero config finds in one dependency'sexportsis kept beside the cached model (node_modules/.cache/onsystem/detection.json, oronsystem-<uid>/<id>.detection.json). The next run reads it back while all of these hold: every dependency it looked at is still found at the same folder (the samePackageFinderlookups run again); no file or folder in those folders changed size or mtime (found by walking them, withoutnode_modulesand.git); the app'spackage.jsonandcomponents.jsonare unchanged; and the same build of onsystem runs. So an agent editing the app gets the kept detection, and an edit inside the design-system package (a new component in the barrel, a renamed export) makes it detect again. The brief proposed keying on the mtimes ofpackage.json, tsconfig and the lockfile. That would miss a barrel that gains an export, so I went with the folder walk instead. It costs a few ms: 504 files in dub'spackages/uitake about 3 to 8 ms to walk and stat.exports. When detection also readcomponents.json, the root's own exports, the app's code (packages imported by path, ranking several design systems) or a package's tsconfig, nothing is kept and it runs as before. For the shadcn demo, detection takes 24 ms, so there is little to save there.inspect --explain(a trace) always detects again.--no-cacheneither reads nor keeps it (LoadConfigOptions.cache, passed by the CLI with its other config options and carried on to workspace projects and the server's config reloads), asscripts/corpus.mjsandscripts/perf.mjsexpect.dist, orsrcwhen run from source), so a detection kept by another build, a local edit to the detection code included, is not read.plugins/onsystem/hooks/check-ui.mjs,src/bin.ts). When the hook runs npx, it passesONSYSTEM_REMEMBER_CLI, a file in that same user-only folder,npx-cli-<key>, one per release, npx and project folder (npx runs a project's own install of the release when it has one, which belongs to that project alone). The CLI writes its own path there. On later edits in that project the hook runs that script with Node directly, but only if the file is a file, not a link, that only the user can write to, and the script'spackage.jsonsaysonsystemat the plugin's exact version with that script as itsbin. Otherwise (another release, the npm cache cleaned, another project), it runs npx as before. When a run of the remembered script gives no results (a crash, output that is not JSON, a script that is gone, as when part of npm's_npxfolder was removed), the hook removes thenpx-cli-<key>file and runs npx once in that same edit, withONSYSTEM_REMEMBER_CLI, so npx can install it again and the CLI it starts writes its path anew. Not after the time limit, which npx would not get. A broken config fails both ways, so while it is broken an edit runs the remembered script and then npx before the one-time notice. The CLI writes the file only when it is annpx-cli-<key>file directly in its own folder, with mode 0600, written beside it and renamed into place, so a link there is replaced rather than written through. I let the CLI report its own path instead of having the hook work out npm's_npx/<hash>layout or read npm's config, so it does not depend on where npm keeps its cache.onsystem-<uid>(src/util/own-folder.ts). The bin, detection and the model cache share one helper, and the hook, which runs without the package, keeps a copy of the same rule. The folder is created with mode 0700 and used only when it is a folder, not a link; on Linux and macOS it must belong to the user and nobody else may write to it. One that others can write to is now skipped rather than closed and used, for the model cache too, since a file put in it before would stay. On Windows, which has no owner check here, it is used only inside the user's home folder (links resolved), where Windows keeps each user's temp folder. When the model cache cannot use it, the warning says to setTMPDIR(TEMPon Windows) to a folder of your own, inside your home folder where the Windows rule applies.SECURITY.mdlists what is kept in that folder (the model, the detection, the compile cache and the remembered CLI path) and these rules;docs/plugin.mdanddocs/how-it-works.mdsay the hook runs npx on the first edit in a project, then the CLI npx installed, anddocs/plugin.mdandSECURITY.mdsay it goes back to npx in the same edit when that CLI gives no results.The
PRIMITIVEScopy in the hook is unchanged. The one change insrc/cli.tsis thecache: values.cacheline in the options it passes toloadConfig.Measured
On an Apple M1 Max, Node.js 24.18.0. These are medians of 8 interleaved runs of
main's build against this branch's first build, with the 1-minute load between 8 and 11, on the corpus checkouts (nonode_modules) after a first run:onsystem --versioncheckon midday'stransaction-details.tsxcheckon dub'sui/token-avatar.tsxONSYSTEM_BIN)For comparison, the same hook edit through npx with the published
onsystem@0.11.1(isolated npm cache, after a first run that filled it) took 1.15 to 1.67 s, median 1.58 s over 6 runs, with the load rising to 14. Once this ships, every edit after the first in a project runs the CLI npx installed the way the last row does. The review fixes did not change these paths enough to re-measure: the build stamp walks the 14 files ofdist.How it was tested
test/hook-speed.test.ts(24 tests):package.jsondrops the dependency, when acomponents.jsonappears, and when a design-system dependency that was missing joins the workspace.loadConfig({ cache: false })andinspect --no-cachedetect again, and leave the kept file as it was.src/, a detection kept under another stamp is not read, and the stamp changes with any file of the build.node_modules/.cachewhen the app has one.npx-cli-<key>file in its own folder, with mode 0600, and replaces a link there. With the hook's platform taken for Windows, it remembers the CLI only when the temp folder is inside the home folder. These tests runsrc/bin.ts(transpiled with TypeScript, with a stand-in for the CLI) under a stand-in npx.onsystem-<uid>/compile-cache; neither it nor the remembered path is used when that folder is writable by others. Under umask 000,onsystem-<uid>andcompile-cacheare both 0700, and acompile-cachethat is a link, or that others can write to, is skipped and left as it is.test/cache-folder.test.ts: the model cache is neither read from nor written to a temp folder others can write to, with a warning that says what to do.pnpm install --frozen-lockfile --prefer-offline,pnpm format:check,pnpm lint,pnpm typecheck,pnpm test(1483 passed, 3 skipped),pnpm buildandpnpm smokeall pass, and so dopnpm --filter site generateandpnpm --filter site test(295 passed).node dist/cli.js --versionunder umask 000 withTMPDIRpointing at an empty folder:onsystem-<uid>andcompile-cacheweredrwx------; the versioned folder Node.js creates insidecompile-cachewasdrwxrwxrwx, reachable only through the 0700 folder above it.check appread it (0 errors),check app --no-cacheandinspect --no-cachedetected again (1 error), and touching one file indistmade the next run detect again.check . --format jsonoutput was byte-identical with the kept detection and with a fresh one.pnpm corpusruns with--no-cache, so it now neither reads nor keeps detections; I did not run it again for the review fixes.Follow-ups
docs/how-it-works.md(Performance and footprint) as it is. Its npx row measures the published package from npm, so it can only show this change after the release. At the release-phase re-measure, runnode scripts/perf.mjs --write. Its npx row will then measure edits after the first, which run the CLI npx installed, so relabel that row. The sentence above the table already says npx runs on the first edit in a project only.checkSource(resolver setup, about 48 ms of a midday check) is the next per-process cost. It lives insrc/lint/, which OS2-59 owns, so it is not part of this PR.Roadmap item: OS2-56.