Repository navigation

cuda: GGML_CUDA_FA_PREFILL_F16=N, f16 prefill for quantized K/V from pool memory (decode stays in place) - #330

Open

professorpalmer wants to merge 1 commit into

PrismML-Eng:prismfrom

professorpalmer:claude/fa-prefill-f16-pool

Open

cuda: GGML_CUDA_FA_PREFILL_F16=N, f16 prefill for quantized K/V from pool memory (decode stays in place)#330
professorpalmer wants to merge 1 commit into
PrismML-Eng:prismfrom
professorpalmer:claude/fa-prefill-f16-pool

Commits on Oct 7, 2026
- cuda: GGML_CUDA_FA_PREFILL_F16=N converts a quantized K/V cache of up to N cells to f16 in pool memory for prefill-sized FA batches
  professorpalmer
  committed
  066e3e3