dflash

LLM speculative inference server for consumer & heterogeneous hardware

Model compression toolkit engineered for enhanced usability, comprehensiveness, and efficiency.

Fully uncensored, capability-enhanced abliteration of Qwen3.6-27B. NVFP4 + z-lab DFlash speculative decoding (n=12) on the unified ghcr.io/aeon-7/aeon-vllm-ultimate:latest container, tuned for long-context draft acceptance on DGX Spark. 6 HF variants (BF16/NVFP4/MTP/MTP-XS), docker-compose, and QuickStart.

A unified, torch-native training framework for MTP and block-parallel speculative decoding.

DFlash vLLM for DGX Spark — Plug & Play Block-Diffusion Speculative Decoding

vLLM Qwen 3.6-27B (AWQ-INT4) + DFlash speculative decoding on AMD Strix Halo (gfx1151 iGPU, 128 GB UMA, ROCm 7.13). 24.8 t/s single-stream, vision, tool calling, 256K context, OpenAI-compatible, Docker. Matches DGX Spark FP8+DFlash+MTP at a third of the cost. No CUDA.

llama.cpp fork optimized for NVIDIA DGX Spark / GB10 (Blackwell, SM 12.1) — TurboQuant weights + KV, NVFP4, DFlash MTP

Local AI workstation — discover, run, chat, benchmark, and generate images from open-weight models. DFlash/DDTree speculative decoding, TurboQuant & TriAttention cache compression strategies, MLX + llama.cpp + vLLM + MTPLX backends.

Qwen3.6 27B × DFlash — 30-35 tok/s on NVIDIA DGX Spark (GB10) - LLama.Cpp

Experimental DDTree-on-vLLM research track for Qwen3.6 AEON Ultimate on DGX Spark / GB10.

Run Qwen 3.6-27B AWQ-INT4 models with DFlash speculative decoding on AMD Strix Halo hardware using vLLM for high-throughput inference.

Open-source Volkswagen/Audi DQ250 MQB DSG bench flash and boot mode tool. Dump and flash TC1766 PFlash, ASW, CAL and EEPROM/DFlash over SocketCAN using Raspberry Pi or CAN adapters.

poolside Laguna-S-2.1 INT4 + DFlash speculative decoding on 4x RTX 3090: 200K context, 282 tok/s peak decode, gate-proven with a 190K-token prompt. Full levers menu + failure catalog.

Qwen3.8-27B (Unsloth UD-Q4_K_XL GGUF) served at 37.4 tok/s decode over a 226,048-token context on a single NVIDIA L4 24 GB: llama.cpp with DFlash 2 block-diffusion speculative decoding, a one-line CUDA kernel-routing patch worth 16%, one-command GCP provisioning, quality verification, and a measured account of where the ceiling is.

Run large LLMs locally on AMD Ryzen AI Max+ 395 (Strix Halo, gfx1151) with ROCmFP4 4-bit quantization. Measured benchmarks, build + serving recipes, and 55 ready-to-run GGUF models.

Some benchmarks on running speculative decoding with Qwen-3.6-35B-A3B with EAGLE3, DFlash and MTP.

Measured Muse Glimmer 30B recipe for NVIDIA DGX Spark, with llama.cpp, DFlash parity checks, vision, tools, and GB10 benchmarks.

Reproducible efficient-inference stack for Qwen3.5-4B (AdaptFM Efficient Qwen Competition): GPTQ W4A16 g128, untied W8 lm_head, DFlash speculative decoding, and per-step vLLM latency optimizations. 7.745× average latency speedup with all quality gates passing.

CLI for building and testing DFlash-style speculative decoding draft models.

GGUF-native DFlash speculative decoding runtime for local models