[alanshurafa]

DRAFT: Alan rewrites this in his own words before posting (CONTRIBUTING.md).

Summary

On computers with two graphics chips, Handy now uses the fast one it says it's using instead of sometimes slipping onto the slow one.

ggml-vulkan picks its device list once, in ggml_vk_instance_init(), and stores each device as an index into vkEnumeratePhysicalDevices() (vk_instance.device_indices). The backend registry caches each device's name, type and PCI id from that list. The VkDevice itself is created later, in ggml_vk_get_device(), which enumerates again and takes physical_devices[device_indices[idx]]. If the loader returns the GPUs in a different order the second time, VulkanN gets created on a different GPU than the one it was registered and logged as. On Windows the order can change while the process is running, which is what cjpais/Handy#2358 reports: Handy said "RTX 5090" but ran Whisper on the Intel iGPU, about 40x slower, until restart.

This change saves the vk::PhysicalDevice handles from the init enumeration (vk_instance.physical_devices, parallel to device_indices) and uses them everywhere the code used to re-enumerate and index: device creation, the init-time info print, description, memory, device type and PCI id. Physical device handles belong to the instance and stay valid for its lifetime, so this keeps every path on the same GPU. When the order doesn't change, the selected devices are exactly the same as before.

Scope

• ggml/src/ggml-vulkan/ggml-vulkan-types.h, ggml/src/ggml-vulkan/ggml-vulkan.cpp (vendored ggml): the fix.
• patches/ggml/0004-vulkan-keep-physical-device-handles.patch: the same change as a downstream patch so scripts/sync-ggml.sh keeps it.
• ggml/UPSTREAM: lists the new patch.

The patch is numbered 0004 because that's the next free number on main. handy-computer#246 also adds a 0004; whichever lands second needs renumbering to 0005.

Related

Fixes cjpais/Handy#2358 (thanks to the reporter for the analysis, which held up line by line).

ggml-org/llama.cpp master and ggml-org/ggml master still have the same re-enumerate-and-index code in ggml_vk_get_device(), ggml_backend_vk_get_device_memory(), ggml_backend_vk_get_device_type() and ggml_backend_vk_get_device_pci_id(). I didn't find an open or merged PR there for this. The patch doesn't apply cleanly to ggml master (context drift in ggml_vk_instance_init()), so sending it upstream would need a small rebase.

AI Assistance

Claude Code wrote the change, the test layer and this draft; a person reviews it.

Validation

Windows 11, MSVC 2022 Build Tools, Vulkan SDK 1.4.363.0, laptop with an Intel UHD iGPU and an RTX 3080 Laptop GPU. ggml lists Vulkan0 = Intel UHD and Vulkan1 = RTX 3080, the same layout as in the issue. Built with -DTRANSCRIBE_VULKAN=ON (Release, Ninja). Model moonshine-tiny-Q8_0.gguf, audio samples/jfk.wav, transcribe-cli --backend vulkan --device N.

I couldn't make the real loader reorder GPUs on demand, so I wrote a small throwaway Vulkan layer (not part of this PR). It passes vkEnumeratePhysicalDevices through, rotates the returned order by one from the Nth call on, and logs the device each vkCreateDevice targets. The "before" binary is a main build with no ggml changes.

Reorder injected after registration, just before the device is created:

build	--device	registered as	vkCreateDevice went to	encode
before	1	RTX 3080	Intel UHD	2492 ms
before	0	Intel UHD	RTX 3080	48 ms
after	1	RTX 3080	RTX 3080	52 ms
after	0	Intel UHD	Intel UHD	142 ms

With the fix, ggml enumerates physical devices once per process (the layer sees one enumeration instead of eleven), so a later reorder can't reach it.

No reorder (layer logging only, and once without the layer): both builds create Intel UHD for --device 0 and the RTX 3080 for --device 1, transcripts are identical. GGML_VK_VISIBLE_DEVICES=1 still gives Vulkan0 = RTX 3080 and creates it there.

ctest --test-dir build-vk: 47 passed, 1 skipped (transcribe_cli_diarize_smoke, needs a model).

The patch file applies forward to the upstream/main ggml files and in reverse to the committed tree (git apply --check [-R] --directory=ggml, as sync-ggml.sh does). I didn't run the full sync script.

Not tested: the real Windows reorder from the issue (monitor switch or GPU wake on a desktop with one monitor per GPU), Linux, macOS/MoltenVK, and a system where a GPU is removed while the process runs (this change doesn't try to handle that).

🤖 Generated with Claude Code

[chatgpt-codex-connector]

Codex Review Summary: ✅ Completed

This comment shows the latest Codex review activity on this pull request.

Review	Status	Commit	Review trigger
📝 Code Review	✅ Completed 2026-11-07T04:21:46.140924Z	d8c43a6	PR opened

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.