Eval bug: kvarn6 aborts on Qwen3.8 Flash-Next (qwen4exp) during context initialization

[ryan4yin]

Name and Version

version: 0.4.6 (build 0, commit unknown-dirty)
built with GNU 13.3.0 for Linux x86_64

Image ghcr.io/anbeeld/beellama.cpp:server-cuda13-v0.4.6, org.opencontainers.image.revision=78af8326522d94fb5fc24b60cfd6f26e29f12490. The same assertion also reproduces on server-cuda13-v0.4.5 (f45c68ba8316dc1386808b7dd6614875c0f8507b) and on server-cuda13-preview-v0.4.5 (fab030c20c1f626c374ecff9c6b4f1822fd621a6).

Operating systems

Linux

GGML backends

CUDA

Hardware

Intel i5-13600KF + NVIDIA RTX 4090 (24GB)

Models

• Qwen3.8-Flash-Next UD-IQ3_XXS (HuggingFace repo unsloth/Qwen3.8-Flash-Next-GGUF)
• Qwen3.8-Flash-Next-Uncensored IQ3_XXS (HuggingFace repo orcarouter/Qwen3.8-Flash-Next-Uncensored-GGUF)

Both report general.architecture = qwen4exp with a QSA indexer (attention.indexer.head_count = 4, attention.indexer.top_k = 2048).

Problem description & steps to reproduce

Starting either model with KVarN KV aborts during context initialization (graph reservation), before the warmup and before the server reports ready. Exit code 139. No request is needed to trigger it.

llama-server \
  --model /models/unsloth/Qwen3.8-Flash-Next-GGUF/UD-IQ3_XXS/Qwen3.8-Flash-Next-UD-IQ3_XXS-00001-of-00003.gguf \
  --cache-type-k kvarn6 --cache-type-v kvarn6 \
  --flash-attn on --cont-batching

(/models/... is the path inside the CUDA container image; substitute your local path.)

Expected: the server loads the model and serves requests with kvarn6 KV.

Additional observations:

• Not a GGUF problem: the same file loads fine with q6_0 / q8_0.
• Other models with kvarn6 (e.g. Qwen3.8-27B) are unaffected.
• Not specific to the automatic fit step: the same assertion also reproduces with --fit off (then it fails in the real context init instead of the fit probe).

First Bad Commit

Not determined. The same assertion also reproduces on server-cuda13-preview-v0.4.5 (fab030c20c1f626c374ecff9c6b4f1822fd621a6), so it is not a regression introduced between that preview build and v0.4.5/v0.4.6.

Relevant log output

Logs

/__w/beellama.cpp/beellama.cpp/src/models/qwen4exp.cpp:527: GGML_ASSERT(mctx_idx->get_n_kv() == inp->mctx->get_attn()->get_n_kv() && "the indexer cache must track the attention cache cell for cell") failed
/app/libggml-base.so.0(+0x1e68b)[0x7b727544568b]
/app/libggml-base.so.0(ggml_print_backtrace+0x21c)[0x7b7275445b0c]
/app/libggml-base.so.0(ggml_abort+0x15b)[0x7b7275445ceb]
/app/libllama.so.0(_ZN20llama_model_qwen4exp5graphC2ERK11llama_modelRK16llm_graph_params+0x6f9)[0x7b72758ec009]
/app/libllama.so.0(_ZNK20llama_model_qwen4exp16build_arch_graphERK16llm_graph_params+0x3b)[0x7b72758ec05b]
/app/libllama.so.0(_ZNK11llama_model11build_graphERK16llm_graph_params+0x2f)[0x7b72757986cf]
/app/libllama.so.0(_ZN13llama_context13graph_reserveEjjjPK22llama_memory_context_ibPm+0x453)[0x7b727564e143]
/app/libllama.so.0(+0x13973a)[0x7b727564e73a]
/app/libllama.so.0(_ZN13llama_context17resolve_fused_opsEPK22llama_memory_context_ij+0x17b)[0x7b727564ea6b]
/app/libllama.so.0(_ZN13llama_context13sched_reserveEv+0x5bd)[0x7b727564f05d]
/app/libllama.so.0(_ZN13llama_contextC1ERK11llama_model20llama_context_params+0x1871)[0x7b7275657f41]
/app/libllama.so.0(llama_init_from_model+0x1d3)[0x7b727565a0e3]
/app/libllama-common.so.0(+0x237546)[0x7b7275db5546]
/app/libllama-common.so.0(_Z17common_fit_paramsPKcP18llama_model_paramsP20llama_context_paramsPfP32llama_model_tensor_buft_overridePmjPK22common_fit_extra_model14ggml_log_level+0x436)[0x7b7275dbb0d6]
/app/libllama-common.so.0(_ZN18common_init_resultC1ER13common_paramsb+0x365)[0x7b7275d88095]
/app/libllama-common.so.0(_Z23common_init_from_paramsR13common_paramsb+0x53)[0x7b7275d89523]
/app/libllama-server-impl.so(_ZN19server_context_impl10load_modelER13common_params+0x5f6)[0x7b727665ffa6]
/app/libllama-server-impl.so(_Z12llama_serverR13common_paramsiPPc+0x37f7)[0x7b72765b5db7]
/app/libllama-server-impl.so(_Z12llama_serveriPPc+0xd6)[0x7b72765b7256]
/lib/x86_64-linux-gnu/libc.so.6(+0x2a1ca)[0x7b72760441ca]
/lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0x8b)[0x7b727604428b]
/app/llama-server(+0x12a5)[0x612f23f602a5]

[Anbeeld]

Just pushed a fix to v0.4.7 branch, please try it.

[ryan4yin]

Confirmed fixed in preview-v0.4.7. Thanks for the quick turnaround!

Verified with ghcr.io/anbeeld/beellama.cpp:server-cuda13-preview-v0.4.7:

• Qwen3.8-Flash-Next UD-IQ3_XXS with --cache-type-k kvarn6 --cache-type-v kvarn6 --ctx-size 120000 --fit on --fit-target 512 now loads and serves requests. No exit 139, the qwen4exp indexer-cache assertion is gone.
• 32K-token prompt benchmark: prefill 673.6 / 743.7 tok/s, decode 33.42 / 35.31 tok/s across two seeds, on par with the q6_0 workaround on v0.4.6.

Closing as resolved.