Skip to content
#

ssd-offloading

Here are 9 public repositories matching this topic...

⚡ Zero-Stall MoE Inference via Lookahead Prediction & Async DMA Prefetching. Optimized for SSD I/O with Hybrid MLA+Sliding Window Attention.

  • Updated Nov 13, 2026
  • Python

Run DeepSeek-V4.1-Flash (551B MoE, IQ2_XXS) locally on Apple Silicon with 128K context, even though it's bigger than RAM. Patched llama.cpp Metal runtime streams experts from SSD with a GPU expert cache. Tested on M5 Pro 64 GB: 252 tok/s prefill, 16.3 tok/s decode.

  • Updated Oct 26, 2026
  • JavaScript

From-scratch Metal inference engine that runs DeepSeek-V4.1-Flash (233 GB GGUF) with 200K context on a 64 GB Mac — 7x faster prefill, 12x faster decode than llama.cpp. OpenAI-compatible server + web UI.

  • Updated Nov 3, 2026
  • C++

Add this topic to your repo

To associate your repository with the ssd-offloading topic, visit your repo's landing page and select "manage topics."

Learn more