perf(cachekit): an encrypted L1 hit on Node.js awaits no promise around the synchronous decrypt (LAB-12918)

CONTRIBUTING.md

@@ -147,7 +147,7 @@ Six harnesses measure the client.
 
 **Call shape (runs in `pnpm test`).** `packages/cachekit/test/transport/call-shape.test.ts` runs the CacheKit Cloud backend against a local TLS fake of CacheKit Cloud. It asserts the exact requests each op sends (a wrap miss is `GET` then `PUT`, and with `writeBehind` the caller waits for the `GET` only; a locked miss is `GET`, lock, `PUT`, unlock) and the connections 100 ops use. Each extra request is a round trip, and each extra connection is a TCP and TLS handshake. So if you change one of these numbers, change the expectation in the same PR and say why. The fake mints a throwaway certificate with the `openssl` CLI, and it needs Node 22.19+ or 24.5+ for `tls.setDefaultCACertificates`. Where either is missing, the tests fail with a message that names the requirement.
 
-**Instructions per op (`pnpm --filter @cachekit-io/cachekit bench:ir`).** This counts the main-thread instructions of each hot path under callgrind: key generation through the NAPI hash (`keygen`, `keygen-1kb`) and the wasm hash (`keygen-wasm`, `keygen-wasm-1kb`), at 45 B and 1 KB of arguments, the serializer, the NAPI and wasm envelopes (1 KB, plus 40 B under the core's 64 B literals-only cutover and 64 KiB), both encrypted paths (1 KB, plus 64 KiB of incompressible bytes), and the L1 hit through `get()` (`get-l1-hit`) and through `wrap()`, once with an object among the arguments and once with the single integer argument the README benchmark passes, and the encrypted `wrap()` L1 hit at 100 B, 10 KB and 1 MB. Unlike wall-clock timing, instruction counts repeat on a busy machine. The suite pins V8's flags and counts only the measured ops. It also lowers V8's TurboFan threshold, so no optimizing compile lands in any workload's window. For all but the two 1 MB workloads, the window counts the code V8 runs in steady state. Those run so few ops that the `set()` and `wrap()` code around each 1 MB encode or decode stays in V8's interpreter for the whole run, which costs too little to show beside the encode or decode. A workload that tiers up later sets its own `warmup`; `bench/ir-workload.mjs` says how to check one with `--trace-opt`. The 100,000-entry L1 at-capacity workload collects garbage just before its window, so V8 does not finish collecting the entries its setup made inside it: that had been two thirds of its count, and it moved with whatever the build allocated at load. The 1,000- and 10,000-entry ones do not collect: at 10,000 a collection made the count move 2.4% with heap layout. On Node 22 and 24, every path repeats within 0.01%. On Node 26, paths through the native cores repeat within about 0.3%, but `napi-encrypted` and the encrypted `wrap()` hits at 100 B and 1 MB sometimes have a run up to 1.7% off the rest, which makes them read inconclusive. Gate on Node 22 or 24. It needs Linux and valgrind. It imports `dist/` and the wasm build, so run `pnpm build` and `pnpm --filter @cachekit-io/cachekit-core-wasm build:wasm` first; the second needs the `wasm32-unknown-unknown` target, `wasm-bindgen` and `wasm-opt`, and without it pass `--only` to skip the seven wasm workloads (`wasm-envelope`, `wasm-envelope-40b`, `wasm-envelope-64kb`, `wasm-encrypted`, `wasm-encrypted-64kb`, `keygen-wasm`, `keygen-wasm-1kb`). To A/B a change, run it on both builds on the same machine:
+**Instructions per op (`pnpm --filter @cachekit-io/cachekit bench:ir`).** This counts the main-thread instructions of each hot path under callgrind: key generation through the NAPI hash (`keygen`, `keygen-1kb`) and the wasm hash (`keygen-wasm`, `keygen-wasm-1kb`), at 45 B and 1 KB of arguments, the serializer, the NAPI and wasm envelopes (1 KB, plus 40 B under the core's 64 B literals-only cutover and 64 KiB), both encrypted paths (1 KB, plus 64 KiB of incompressible bytes), and the L1 hit through `get()` (`get-l1-hit`) and through `wrap()`, once with an object among the arguments and once with the single integer argument the README benchmark passes, the encrypted `wrap()` L1 hit at 100 B, 10 KB and 1 MB, and the encrypted `get()` L1 hit at 100 B (`secure-get-l1-hit`). Unlike wall-clock timing, instruction counts repeat on a busy machine. The suite pins V8's flags and counts only the measured ops. It also lowers V8's TurboFan threshold, so no optimizing compile lands in any workload's window. For all but the two 1 MB workloads, the window counts the code V8 runs in steady state. Those run so few ops that the `set()` and `wrap()` code around each 1 MB encode or decode stays in V8's interpreter for the whole run, which costs too little to show beside the encode or decode. A workload that tiers up later sets its own `warmup`; `bench/ir-workload.mjs` says how to check one with `--trace-opt`. The 100,000-entry L1 at-capacity workload collects garbage just before its window, so V8 does not finish collecting the entries its setup made inside it: that had been two thirds of its count, and it moved with whatever the build allocated at load. The 1,000- and 10,000-entry ones do not collect: at 10,000 a collection made the count move 2.4% with heap layout. On Node 22 and 24, every path repeats within 0.01%. On Node 26, paths through the native cores repeat within about 0.3%, but `napi-encrypted` and the encrypted `wrap()` hits at 100 B and 1 MB sometimes have a run up to 1.7% off the rest, which makes them read inconclusive. Gate on Node 22 or 24. It needs Linux and valgrind. It imports `dist/` and the wasm build, so run `pnpm build` and `pnpm --filter @cachekit-io/cachekit-core-wasm build:wasm` first; the second needs the `wasm32-unknown-unknown` target, `wasm-bindgen` and `wasm-opt`, and without it pass `--only` to skip the seven wasm workloads (`wasm-envelope`, `wasm-envelope-40b`, `wasm-envelope-64kb`, `wasm-encrypted`, `wasm-encrypted-64kb`, `keygen-wasm`, `keygen-wasm-1kb`). To A/B a change, run it on both builds on the same machine:
 
 ```bash
 git checkout <your-branch> -- packages/cachekit/bench                 # in the base checkout: your bench/, as CI does

packages/cachekit/README.md

@@ -1189,7 +1189,7 @@ What one call costs in the client, on an in-memory backend, so no figure include
 
 - **`wrap()` L1 hit**: the hit returns the stored object as it is, so the value's size barely matters. A wrapped function remembers the key of each call whose arguments are all strings, numbers, booleans, `null` or `undefined`, and the L1 entry that call was last served from. A repeat call is served from that entry on one check, with no key generation and no lookup by key, while L1 still holds the entry and it is short of its stale-while-revalidate point (the top of that point's ±10% jitter band); once L1 evicts or expires it, or it reaches that band, the call looks its key up in L1 again. `delete()`, `invalidate()` and an invalidation event empty the memo too, so the next call generates its key first, and after `close()` every call is rejected. It remembers a call only if it has at most 8 arguments, no `NaN`, and strings whose `length` adds up to 256 or less. Every other call generates its key, which adds about the cost of the key generation row: an object, array, `Date`, `Map`, `Set` or binary argument, a call past those limits, a call with `skipL1` or under `ttl: 0`, every `interop` call, and every call on a cache that encrypts or has L1 off, or on Cloudflare Workers.
 - **Key memo**: a wrapped function remembers at most 1,024 keys and starts again when full: about 160 KiB when each call passes one number, and at most about 1 MiB. It holds copies of the arguments, so a cache that encrypts keeps no memo, and `delete()`, `invalidate()`, `close()` and, with `invalidation` configured, an invalidation event from another instance empty every memo of the cache. A timer also empties a memo one second after its first key, or one `ttl` after it if that is sooner, even if the function is not called again, so it is set to drop an argument no later than L1 stops serving a value stored with that `ttl`. The timer can run late, for example when the event loop is busy or the process is suspended, and the memo keeps its arguments until it runs. A call that comes one `ttl` or more after the first key, before the timer has run, empties the memo first. The call that stores the first key sets that timer, which keeps the call's async context, `AsyncLocalStorage` stores included, until it runs: one second or less after it was set, unless it runs late. So a function called again and again with the same arguments generates their key about once a second, or once per `ttl` if that is shorter. Wrap a function once, at module scope: a function wrapped inside each request keeps that request's async context and its memo until the memo's timer runs, one second later or one `ttl` later if that is sooner. With a `ttl` of a second or more, at 1,000 requests a second with a 16 KiB store, that is about 16 MB. A wrap under `ttl: 0`, which never expires, keeps no memo, and nor does a Worker, where a timer can be cancelled when its request ends.
-- **Encrypted L1 hit**: L1 holds the same ciphertext as L2, so every hit also generates the key, decrypts, verifies the AAD and decodes the value. That cost grows with the value. A read from L2 also walks the document to check its nesting depth, before the decode and again after it. So does the first hit on an entry, which then keeps its own copy of the ciphertext in L1; later hits on that copy skip the walks (an `interop` entry keeps the check after the decode).
+- **Encrypted L1 hit**: L1 holds the same ciphertext as L2, so every hit also generates the key, decrypts, verifies the AAD and decodes the value. That cost grows with the value. A read from L2 also walks the document to check its nesting depth, before the decode and again after it. So does the first hit on an entry, which then keeps its own copy of the ciphertext in L1; later hits on that copy skip the walks (an `interop` entry keeps the check after the decode). On Node.js the decrypt is a synchronous call into the native core, so once the cache has derived its keys, at its first encrypt or decrypt, a hit through `get()`, `exists()` or `wrap()` awaits no promise. On Cloudflare Workers, WebCrypto decrypts asynchronously, so the hit waits for it.
 - **L2 hit and `set`**: L1 is off, so each call reads or writes the backend and decodes or encodes the value. A real backend adds its round trip, which for Redis or CacheKit Cloud is a network call.
 
 To print this table for your own machine, run `pnpm install` and `pnpm build`, then `node packages/cachekit/bench/readme.mjs`, from a clone of the repository, or run the **Bench** workflow from its Actions tab. Across 11 runs on this CPU model, the 100 B hit took 290-360 ns, 320 ns at the median, and every other cell agreed within 10%. The encrypted L1 hit row is the median of four later runs on this CPU model, which read 9.0-9.7 µs and 119-125 µs. Hosted runners also come with other CPU models, on which the 100 B hit took from 160 ns (an EPYC 9V45) to 330 ns (an Intel Xeon Platinum 8370C or 8573C) in the same batch of runs, and other machines give other figures.

packages/cachekit/bench/ir-workload.mjs

@@ -465,6 +465,28 @@ export const WORKLOADS = {
       };
     },
   },
+  // get() served from an encrypted L1, which holds ciphertext: the lookup,
+  // then decrypt, unpack and decode of 100 B of msgpack, with no key
+  // generation, unlike secure-wrap-l1-hit-100b. Setup checks the get is served
+  // by L1. 4,000 warm-up ops, as for get-l1-hit and secure-wrap-l1-hit-100b.
+  'secure-get-l1-hit': {
+    n: 20_000,
+    warmup: 4_000,
+    setup: async () => {
+      const { createCache } = await load('index.js');
+      const backend = new MemoryBackend();
+      const cache = createCache({ backend, encryption: { masterKey: MASTER_KEY }, metrics: false });
+      let reads = 0;
+      const read = backend.get.bind(backend);
+      backend.get = (key) => (reads++, read(key));
+      await cache.set(KEY, rowsOf(3), { ttl: 3600 });
+      sink += (await cache.get(KEY)).rows.length;
+      if (reads !== 0) throw new Error('the get read L2: L1 did not keep the entry');
+      return async () => {
+        sink += (await cache.get(KEY)).rows.length;
+      };
+    },
+  },
   // wrap() with an object among the arguments, which the key memo does not
   // keep: keygen + L1 lookup, where a plaintext L1 holds the decoded value
   'wrap-l1-hit': {