[bartlomieju]

Ports MemoryCache.read() to Rust JSG, behind a new RUST_MEMORY_CACHE autogate. This is the last PR of a stack: #7963 → #8030 → #8034 → this one.

Note

This branch includes a cherry-pick of #8064 (Drop Rust JSG promise reaction data once a reaction runs), which the port depends on. When #8064 lands, I'll rebase and that commit will drop out. Please review it in #8064.

Commits to review

1. Add Local::to_detail_string() to Rust JSG. Wraps v8::Value::ToDetailString(). It describes a value the way C++ kj::str() of a v8::Local does ("TypeError: message", "[object Object]", and so on) without running any JavaScript. The Rust MemoryCache uses it for a failed fallback's fallback_error tag.
2. Port MemoryCache.read() to Rust, behind an autogate.
   • src/workerd/api/memory-cache-api holds the Rust MemoryCache. It sits on top of the cache in src/rust/memory-cache and uses the io-context crate's await_io(), add_object() and trace spans.
     • lib.rs: read(), key validation, and cache hits.
     • fallback.rs: a read that waits as the leader (until it runs its fallback) or as a waiter (until the read it waits for finishes).
     • trace.rs: the same spans and tags that memory-cache.c++ records.
   • MemoryCacheProvider::bindForRust() gives the Rust implementation the same shared namespaces the C++ one uses.
   • workerd-api.c++ creates the Rust MemoryCache for memory cache bindings when the autogate is enabled.

Behaviour

The Rust implementation is meant to be indistinguishable from the C++ one, including its trace spans. A few details:

• Cache errors become internal errors, as in C++.
• The fallback gets the key decoded from the UTF-8 bytes the cache holds. A lone surrogate therefore reaches the fallback as U+FFFD for each of its three bytes, the same as in C++.
• A fallback whose request ends is abandoned with that request, so another read runs its own fallback. It is not left waiting until the garbage collector drops the promise.
• The read resumes inside a Durable Object's blockConcurrencyWhile() critical section, rather than waiting on the input gate.

Testing

• Span parity: a new test in memory-cache-test.c++ runs the same scenario against both implementations and checks they record the same spans.

  • The scenario covers misses and hits, coalesced fallbacks, a failed leader followed by a waiter running its own fallback, sync and async throws, a NaN expiration, a rejection with a non-Error, a value that is too large, an expiration in the past, and eviction.
  • SpanRecorder now records spans in the order they open, not the order they end. When each implementation releases its spans is an implementation detail.

• New JS tests in memory-cache-test.js:

  • a fallback abandoned when its request ends;
  • the async context a fallback runs in;
  • the key a fallback gets for a lone surrogate;
  • a NaN expiration;
  • a fallback inside blockConcurrencyWhile().

  The @all-autogates variant runs all the JS tests against the Rust implementation.

Known gap

Rust JSG doesn't throw DOMExceptions yet (#7951). For a value that can't be serialized, Rust throws a plain Error where C++ throws a DataCloneError, so the fallback_error tags differ. The parity scenario leaves that case out, with a TODO(#7951).

[codecov-commenter]

Codecov Report

❌ Patch coverage is 88.69565% with 65 lines in your changes missing coverage. Please review.
✅ Project coverage is 68.98%. Comparing base (29b484a) to head (f73a613).
⚠️ Report is 14 commits behind head on main.

Files with missing lines	Patch %	Lines
src/workerd/api/memory-cache-api/fallback.rs	85.96%	32 Missing ⚠️
src/workerd/api/memory-cache-test.c++	83.09%	3 Missing and 9 partials ⚠️
src/rust/jsg/v8.rs	46.15%	7 Missing ⚠️
src/workerd/api/memory-cache-api/lib.rs	94.48%	7 Missing ⚠️
src/rust/jsg/ffi.c++	50.00%	3 Missing and 1 partial ⚠️
src/workerd/api/memory-cache-api/trace.rs	96.62%	3 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #8066      +/-   ##
==========================================
+ Coverage   68.84%   68.98%   +0.14%     
==========================================
  Files         734      742       +8     
  Lines      192834   194499    +1665     
  Branches    30009    30095      +86     
==========================================
+ Hits       132751   134173    +1422     
- Misses      42441    42667     +226     
- Partials    17642    17659      +17     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

[guybedford]

AI review suggestions:

• Argument errors reject instead of throwing (lib.rs:62). C++ JSG throws synchronously when an argument has the wrong type, so CACHE.read(123) and CACHE.read('k', 5) throw in C++ but return a rejected promise in Rust. Only "Key too large." and later failures should reject.
• delete() records no memory_cache_delete span. C++ records one with key and delete_completed. The parity scenario never calls delete, so the span parity test doesn't catch it.
• A thenable or non-object fallback result behaves differently (fallback.rs:177, fallback.rs:315). If the fallback returns { then(r) { r({ value: 'th' }) } }, C++ resolves undefined (it treats the object itself as the result) and Rust resolves 'th'. If it returns 42, C++ rejects with "Callback returned incorrect type; expected 'Promise'" and Rust with "The fallback must produce an object with a value property."
• Errors thrown while serializing keep their original identity in Rust (fallback.rs:292). If a getter on the value throws a MyErr, C++ rejects with a new plain Error and Rust rejects with the original MyErr. A thrown string becomes an Error in C++ but stays a string in Rust, and the fallback_error tag changes with it. Rust's behaviour is arguably better, but it's a divergence to either accept explicitly or match.
• A JS-level parity test with these cases, run in both the @ and @all-autogates variants, would back up the parity claim for what JS sees and not only for the spans.

[bartlomieju]

Rebased onto #8210 (Throw DOMExceptions from Rust JSG), so this branch now carries its two commits until it lands. With it, the Rust MemoryCache rejects an unserializable value with a DataCloneError DOMException, and the parity test covers that case.

Differences to fix before merging

I compared C++ and Rust on about 40 observable behaviours of env.CACHE, at compat dates 2000-01-01 and 2999-12-31, with and without memory_cache_delete. These still differ:

	Case	C++	Rust	Dates
A	The fallback produces something other than an object	sync 42: Callback returned incorrect type; expected 'Promise' async 42: Incorrect type for Promise: the Promise did not resolve to 'CacheValueProduceResult'. undefined/null: Cannot initialize N7workerd3api23CacheValueProduceResultE with required members from an undefined or null value.	always The fallback must produce an object with a value property.	all
B	The fallback returns a thenable that also has a value, e.g. { then(r) { r({ value: 'from then' }) }, value: 'own' }	resolves 'own' without unwrap_custom_thenables (before 2024-04-01), 'from then' with it	always 'from then'	before 2024-04-01
C	Serializing the value throws from a getter (throw new Custom())	rejects with a copy: a plain Error, not instanceof Custom	rejects with the original object	all
D	Settle timing of a hit, a miss and Key too large	after 2 microtasks	after 1	all

The leader and waiter paths cross an event-loop turn, so their microtask counts still need measuring.

Coverage to add before merging

• Rust at the newest compat date. @all-autogates runs only at 2000-01-01, so apart from the capture_async_api_throws service, no test runs Rust with flags enabled by date.
• Rust under GC stress. memory-cache-test@gc-stress has no autogates, so it runs only C++.
• memory_cache_delete off. Nothing checks that delete is absent.
• A fallback whose promise settles in another request (handle_cross_request_promise_resolution). Not tested against either implementation.

The plan for the first three is a second wd_test whose config enables workerd-autogate-rust-memory-cache, so @, @all-compat-flags and @gc-stress all run the JS tests against Rust. It gets deleted with the autogate.

Open question

Whatever Rust copies from C++ for A and C becomes its permanent behaviour once the C++ implementation is removed, including the mangled type name in A's message. Should we copy these exactly, or accept them as differences?