MTS at Inferact, working on vLLM.
I work on cold starts, checkpoint/restore, and reusing work across model launches. I care about making it practical for more people to run their own models. Based in San Jose.
I maintain the cold-start roadmap. Selected merged work:
- Precompiled Python bytecode: move Python compilation into the image build so fresh containers can reuse that work. Write-up.
- Initialized-engine snapshots: checkpoint and restore prepared engines instead of repeating startup work. Experimental, same-host, single-GPU support on Linux x86-64.
- Zstd container images: use zstd for CI images and publish a separate zstd Docker Hub variant while retaining the existing gzip images.
- Pluggable sleep-mode backends, compile-cache invalidation fixes, and memory-profile reuse across boots.
A Rust/CUDA project for snapshotting and forking live LLM sessions, with vLLM and SGLang integrations. It can also inspect and diff saved sessions without a GPU.
This is the project that led me into upstream vLLM.
Re-feeding Is Not Replaying · sole-author preprint.
Replaying a transcript can change which tokens a credit-estimation method identifies as important. I measured that against exact KV-state resume and a replica noise floor. Batch-invariant kernels eliminated the discrepancy in the tested configurations.
- Recall: a Rust/Tauri League of Legends overlay that recommends your next purchase from live game state.
- ProjectGorgon: speculative decoding with custom Triton/CUDA kernels. Where I learned GPU programming.
- sentinel: a Go log-streaming engine with LSM storage, a write-ahead log, and replication.
- Madison Bus ETA: arrival predictions with XGBoost and conformal uncertainty intervals. Retired; source available.
B.S. Data Science, computer science minor, UW–Madison.