Skip to content

ggml : possible fix for the radix TOP_K thread timing bug - #31084

Draft
sanmai wants to merge 1 commit into
ggml-org:masterfrom
sanmai:ggml/top-k-deterministic-ties
Draft

sanmai wants to merge 1 commit into
ggml-org:masterfrom
sanmai:ggml/top-k-deterministic-ties

Conversation

@sanmai

@sanmai sanmai commented Oct 24, 2026 •

Copy link
Copy Markdown
Contributor

Overview

thread timing causes a non-deterministic KLD (from 0.0004 to 0.011 between runs)

caught in the QSA indexer only for MoE fusions

Additional information

TBD, probably will file a bug - other backends has it too

Requirements

@github-actions github-actions Bot added testing Everything test related Vulkan Issues specific to the Vulkan backend ggml changes relating to the ggml tensor library for machine learning CUDA Related to the CUDA backend labels Oct 24, 2026
@ServeurpersoCom

Copy link
Copy Markdown
Contributor

This one is worth taking out of draft. I used it to test #31009 on the real GLM-5.3-Flash with CUDA: without it, two runs of the same master build don't give the same numbers, so you can't tell whether a change matches master. With it, the runs became identical (7 out of 8, the first one was off by a hair), and that's how I could show #31009 matches master exactly. I only tested the CUDA part, not Vulkan. Could you mark it ready so it gets review?

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

CUDA Related to the CUDA backend ggml changes relating to the ggml tensor library for machine learning testing Everything test related Vulkan Issues specific to the Vulkan backend

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants