Skip to content

Serve token counts from a Rust worker - #4

Draft
guuzaa wants to merge 6 commits into
mainfrom
cursor/rust-worker-5bb1
Draft

guuzaa wants to merge 6 commits into
mainfrom
cursor/rust-worker-5bb1

Conversation

@guuzaa

@guuzaa guuzaa commented Oct 11, 2026

Copy link
Copy Markdown
Owner

The TypeScript worker and the tokenizer-wasm crate are now one Rust worker on wasm32-unknown-unknown. POST /v1/token/count still returns the recorded status, content type, Retry-After, and JSON body. Rust is pinned to 1.94.0.

Measurements

Workerd, seven warm runs after one discard. Handler time is performance.now() around the fetch call (about 1 ms resolution, not pure CPU). The first request after boot is listed separately.

TypeScript Rust
Ready 565 ms 626 ms
Cold, process start to first small response 581 ms 656 ms
Cold small, wall / handler 16.2 ms / 10 ms 29.8 ms / 23 ms
Warm small hello, wall median (min) 2.86 ms (2.62) 5.17 ms (4.48)
Warm small, handler median (min) 2 ms 4 ms (3)
Warm 1024 sevens, wall / handler 3.06 ms / 2 ms 5.24 ms / 4 ms
Warm 8 KiB digits, wall median (min) 8.19 ms (7.71) 9.75 ms (8.24)
Warm 8 KiB digits, handler median (min) 7 ms (7) 9 ms (7)
Tokens, small / mid / cap 5 / 346 / 2704 5 / 346 / 2704
Wasm memory at 8 KiB 57.69 MiB 69.82 MiB
Wrangler upload / gzip 8809.64 / 2695.12 KiB 8855.27 / 2772.42 KiB

The first 8 KiB request after boot measured 24 ms in the handler, then 9 ms. Gzip stays under 3 MB. Memory stays under 128 MiB. Local workerd did not enforce a CPU-time cutoff, so these figures are isolate wall time.

Behavior

All 299 golden responses match. The deepseek_v41 reasoning-effort text is unchanged because callers see it. A non-numeric Content-Length is still a platform 500 from workerd, before the worker runs.

crypto.getRandomValues throws during Wasm startup. The vocab is built in that startup budget, so its map seeds come from Math.random, which this runtime allows there and which differs per isolate. If both sources fail, startup aborts. The old fixed 0xA5 seed is gone. Token counts do not depend on the seed.

Toolchain

rust-toolchain.toml is channel 1.94.0 with wasm32-unknown-unknown. Every Cargo.toml sets rust-version = "1.94". CI installs 1.94.0.

No selected crate's latest release declares a rust-version above 1.94. worker 0.8.7, worker-build 0.8.7, and wasm-bindgen 0.2.129 are current (rust-version 1.91, 1.91, and 1.81). tokenizers stays on 0.23.2 rather than 1.0.0-rc.2. That rc publishes no rust-version; it is a different encoder, and the fixtures are locked to 0.23.2 encode_fast.

Open in Web Open in Cursor 

cursoragent and others added 6 commits October 11, 2026 07:11
Record status, content-type, retry-after, and JSON bodies for the
current workerd worker so the Rust port can match them request for
request.

Co-authored-by: Jowell Young <guza@duck.com>
Port the V4.1 chat encoder and replace the TypeScript fetch handler
with a wasm32 worker pinned to Rust 1.94.0.

Co-authored-by: Jowell Young <guza@duck.com>
Run fmt, clippy, the encoder tests, the wasm build, and the golden replay.

Co-authored-by: Jowell Young <guza@duck.com>
Drop the old sources, the tokenizer-wasm crate, vitest, and the bench
now that the Rust worker serves the same requests.

Co-authored-by: Jowell Young <guza@duck.com>
Update the development steps and the measurements that changed with the wasm worker.

Co-authored-by: Jowell Young <guza@duck.com>
Keep the rewritten README and describe the Rust worker in its development section.

Co-authored-by: Jowell Young <guza@duck.com>

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants