cuda: keep the GDN state gather when its ids memory is reused

[bri-prism]

Overview

The CUDA GDN gather fusion (ggml_cuda_try_gdn_gather_skip, added in #221) skips the recurrent-state GET_ROWS and lets the GDN kernel read the cache row through the ids tensor. ggml-alloc does not know about this deferred read: for the allocator the skipped GET_ROWS is the last reader of ids, so it can give the ids memory to a node that runs before the GDN op. The kernel then reads activation data as a slot index and crashes with an illegal memory access was encountered.

This PR keeps the fusion, but falls back to the normal GET_ROWS for a layer when:

• any node from the GET_ROWS up to and including the GDN output overlaps the ids memory, or
• the GDN op runs on a concurrent stream, where node order is not execution order.

Other layers keep the fused path.

Additional information

Seen on an A100 with a hybrid GDN vision-language model: a prompt with two images crashed llama-mtmd-cli and killed llama-server on the first request. With a debug build, the overlap was the last GDN layer's alpha projection, which the allocator placed on the scheduler's device copy of inp_s_copy. The faulting index decoded to the float bit pattern of a gate value. Other models with different tensor sizes did not hit the overlap. That was layout luck, so the check does not depend on the model.

Validation (A100-SXM4-80GB, CUDA 12.4):

• The crashing case now completes in Q2_0, PQ2_0 and PTQ1_0. A second model passes with 1, 2 and 3 images in all three formats.
• llama-server handles three consecutive two-image requests (HTTP 200) and stays up. Before this change the process died on the first request.
• compute-sanitizer --tool memcheck: 0 errors. Before: invalid global read in gated_delta_net_cuda.
• Greedy output is byte-identical to GGML_CUDA_GDN_GATHER_FUSION=0 (300 tokens).
• Text-only llama-bench pp512 / tg128 on two models, 5 reps, arms interleaved twice: all differences are within one standard deviation.

Not tested: other GPUs, release binaries, and the SYCL fused GDN path.

Requirements

• I have read and agree with the contributing guidelines
• AI usage disclosure: YES. Claude found the cause, wrote the change and ran the validation; I reviewed it.

[Copilot]

Copilot review overview

🟡 Changes recommended

The overlap check ignores writable backend allocation space beyond a tensor's logical size and lacks regression coverage.

Review effort: Balanced
Findings: 2

Open (2)

• Add regression test for GET_ROWS gather-skip fallback · New
• Use backend allocation size instead of logical tensor size · New

What changed in this PR

Prevents CUDA GDN gather fusion from reading reused ids memory.

Changes:

• Detects overlapping tensor memory before GDN execution.
• Disables fusion when GDN runs concurrently.

File	Description
ggml/​src/​ggml-cuda/​ggml-cuda.cu	Adds safety checks for deferred GDN gather reads.

---

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.