[ZX-ModelCloud]

Summary

PyTorch exposes AMD GPUs through its cuda namespace on HIP builds. Treating that spelling as the NVIDIA hardware family allowed unsupported kernels through validation and hid ROCm support in other kernels.

• Preserve DEVICE.ROCM in GPTQModel family-valued configuration and kernel selection. Normalize runtime aliases centrally; resolve enums to Torch devices only at execution boundaries, preserving explicit ordinals and current-device behavior. Family-wide thread-pool locks cover every device.
• Audit all 57 registered quant-linears, the embedding kernel, and three specialized loader kernels. Add a complete regression inventory for their ROCm contracts.
• Declare bitsandbytes ROCm support for its existing 4/8-bit paths. Validate AllSpark's RDNA4 restriction against selected devices, allow a supported target when another current GPU is unsupported, and warm up in the target context.
• Give EXL3 and FP4 specialized wrappers explicit device declarations. Reject native EXL3 inference on HIP before extension loading, with an EXL3_TORCH alternative.
• Stop NVIDIA capability probes for AMD requests and make NVIDIA-specific tests deterministic on HIP hosts.
• Add an explicit accelerator-family policy to AGENTS.md and .agents/skills/gptqmodel-device-families/SKILL.md, covering self-declared kernel support, Torch runtime aliases, mixed-vendor hosts, ordinals, and per-target architecture validation.

Validation

• ROCm R9700/gfx1201, Torch 2.16.0.dev20261025+rocm10.1: audit/selection/gating/AllSpark/bitsandbytes run 615 passed, 54 skipped.
• Follow-up contract and AllSpark run on two AMD GPUs after the mixed-architecture fixes: 424 passed.
• NVIDIA, Torch 2.15.0+cu132: audit/gating/bitsandbytes/EXL3 import/FP4 CPU run 577 passed, 42 skipped. Final selector/contract rerun after the explicit-AMD capability guard: 508 passed, 29 skipped.
• New bitsandbytes tests cover NF4/FP4/INT8, FP16/BF16/FP32, vendor and supported Triton paths, using an independent FP64 oracle at rtol=atol=2e-3. Maximum observed absolute error: 6.14e-4; repeated forwards retain no additional weight storage. FP32 uses the vendor path.
• Earlier two-NVIDIA-GPU placement/checkpoint recovery checks: 265 passed, 31 skipped, 1 xpassed.
• Ruff, diff whitespace checks, and skill validation pass.

The full hardware declaration matrix is tested with simulated HIP; live numerical coverage here is on RDNA4, not every declared AMD architecture.

[Qubitium]

Updated this PR in 1631545d0 to keep ROCm explicit end-to-end per review feedback.

The previous revision added torch_compatible_device() and stored DEVICE.CUDA on a ROCm host, which turns ROCm handling back into CUDA checks. PyTorch already exposes ROCm through the cuda namespace, so for GPTQModel kernel loading we keep DEVICE.ROCM (cuda + HIP) and resolve it to a concrete torch.device only at torch boundaries.

Changes in 1631545d0:

• Removed torch_compatible_device(); auto_select_device() and the loader now retain DEVICE.ROCM on ROCm. normalize_device() still maps cuda/cuda:N/ordinals/torch.device/DEVICE.CUDA to DEVICE.ROCM on HIP.
• Added as_torch_device() (uses DEVICE.to_torch_device()) and applied it at module-move / p2p / structure / prewarm / rotation / model-definition boundaries.
• DeviceThreadPool now accepts DEVICE enums and maps them to the runtime family (DEVICE.ROCM -> cuda), fixing Unknown device for pool: DEVICE.ROCM in stage_inputs_capture without converting the stored device.
• Kernel _device_family() normalizes both torch.device("cuda:N") and DEVICE.CUDA to DEVICE.ROCM on HIP, so CUDA-only kernels (TritonV2, W4A-FP8, W4A-NVFP4) cannot pass the family gate on ROCm.
• Tests assert DEVICE.ROCM is preserved and resolved at the torch boundary; added thread-pool coverage for read_lock(DEVICE.ROCM).

Verification:

• ruff check gptqmodel tests passes; git diff --check clean.
• tests/kernels/test_selection.py, tests/test_threadx.py, tests/test_threadx_cpu.py: 160 passed, 31 skipped, 1 xpassed.
• tests/test_checkpoint_devices.py (CUDA_VISIBLE_DEVICES=0,1): 27 passed.
• Broad run: 209 passed, 32 skipped, 1 xpassed, 1 failed. The only failure, tests/test_rocm_backend_gating.py::test_cuda_only_backends_reject_rocm_with_actionable_errors (Swordfish runtime unavailable.), is pre-existing on base 3ebdc6e8f on an NVIDIA host because the test monkeypatches torch.version.hip while modules use the import-time IS_ROCM constant; it is not a regression from this change.

The PR description's torch_compatible_device() / auto_select_device notes are superseded by this commit.

[Qubitium]

Pushed review fixes in c87b7f9f5.

• Preserve the current accelerator at PyTorch boundaries. as_torch_device() previously called DEVICE.to_torch_device(), which defaults to ordinal 0. This sent CUDA/ROCm family-enum moves to GPU 0 even inside a GPU 1 context. The helper now converts through the enum's runtime type, leaving family selectors indexless. Explicit ordinals remain intact, and DEVICE.to_torch_device() retains its existing behavior.
• Close the string-selector validation gap. Direct kernel validation now normalizes string selectors too. Previously, TritonV2Linear.validate_device("cuda") could accept a CUDA-only kernel on ROCm because the string bypassed family normalization.
• Correct ROCm smoke-test device discovery. The test helper no longer treats HIP's HAS_CUDA as NVIDIA availability or probes NVIDIA compute capabilities on ROCm.

Added regression coverage for these cases, including live two-GPU tensor/module moves, explicit-ordinal overrides, and preserved values/outputs. Three regression cases failed before the fixes and pass afterward. DEVICE.ROCM remains explicit in configuration and kernel selection.

Validation:

• Targeted selector, device-error, thread-pool, P2P, prewarm, materialization, placement, checkpoint, and packed-weight tests: 265 passed, 31 skipped, 1 xpassed.
• Live two-GPU Llama and Qwen3 MoE checkpoint recovery passed across all tested interruption modes.
• Final deterministic placement tests rerun separately: 2 passed.
• ruff check gptqmodel tests and git diff --check passed.

Live tests used NVIDIA GPUs with Torch 2.15.0+cu132; ROCm dispatch was tested with simulated HIP flags. This revision has not been run end-to-end on AMD hardware. The PR description has also been updated to reflect the current implementation and validation.

[Qubitium]

Completed the ROCm kernel-contract audit and pushed 01ff2789e (code/tests) and b0a54303d (agent policy/skill).

Scope: all 57 registered quant-linears, TorchQuantEmbeddings, and the native EXL3, portable EXL3, and source-FP4 wrappers. tests/test_rocm_kernel_contracts.py contains the explicit inventory and fails when a new registered kernel has not been audited.

Fixes:

• BitsAndBytesLinear now declares DEVICE.ROCM; its existing NF4/FP4/INT8 implementations were being excluded by the family gate.
• GPTQ/AWQ AllSpark validate every selected AMD device against gfx1200/gfx1201. They reject unsupported targets, accept supported targets even when the current GPU is unsupported, and warm up/repack in the target's context.
• Native EXL3 declares NVIDIA-only support and rejects explicit HIP inference requests before importing its extension. The portable EXL3 wrapper declares DEVICE.ALL; the source-FP4 wrapper declares CPU/NVIDIA, consistent with its existing dtype gate.
• AMD requests no longer trigger NVIDIA compute-capability probes. Several NVIDIA-specific tests now simulate the NVIDIA family consistently on HIP hosts.

The remaining contracts stay unchanged: portable Torch kernels admit ROCm through DEVICE.ALL; FP8, AWQ GEMM/GEMV Triton, GGUF Triton and ParoQuant Triton explicitly admit ROCm. Marlin/Machete/Swordfish/BitBLAS, native AWQ, QQQ/VPTQ CUDA, W4A, ExllamaV2 and GGUF CPP CUDA remain excluded. TritonV2 remains NVIDIA-only because its ROCm path is unvalidated; using Triton alone is not sufficient evidence to widen support. CPU/XPU-specific and MLX implementations remain excluded from ROCm placement.

Policy: AGENTS.md now requires the device-family skill. It distinguishes static NVIDIA/AMD hardware declarations from Torch's shared cuda namespace, keeps AMD explicit in GPTQModel configuration, preserves ordinals, and explains active-runtime selection on mixed-vendor hosts.

Validation:

• R9700/gfx1201 + Torch 2.16.0.dev20261025+rocm10.1: 615 passed, 54 skipped across contract, selection, gating, AllSpark and bitsandbytes suites.
• Final AllSpark/contract follow-up on two AMD GPUs: 424 passed.
• NVIDIA + Torch 2.15.0+cu132: 577 passed, 42 skipped; final contract/selection rerun: 508 passed, 29 skipped.
• New bitsandbytes oracle/storage cases: 13 passed, 2 skipped (FP32 correctly uses vendor rather than tuned Triton). Shape M=64, K=512, N=128; NF4/FP4 with nested scales and INT8; FP16/BF16/FP32; independent FP64 decode/matmul, rtol=atol=2e-3. Maximum absolute error 0.000613637.
• Retained bitsandbytes module/QuantState tensor storage: 35,326–35,582 bytes for 4-bit and 66,304–66,560 bytes for INT8; zero additional retained bytes after repeated forwards. Peak extra Torch allocations, including outputs/transient tensors, were 40,960 bytes for the tuned 4-bit Triton path, 84,041,728–84,189,184 bytes for the vendor 4-bit path, and 82,432–115,200 bytes for INT8. These are allocator-visible peaks, not isolated dequantization scratch or external vendor workspace measurements.
• Ruff, diff checks, and skill validation pass.

Live numerical evidence is RDNA4-specific. The full inventory's family gates and mixed-architecture refusal cases are simulated; this audit does not claim execution validation on every AMD family.

[Qubitium]

Resolved the overlap with main's EXL3 sign-buffer dtype fix and pushed merge commit ca1145ab9. The native EXL3 path now rejects unsupported device families before extension import while preserving upstream FP16 sign-vector restoration.

Post-merge validation: 505 passed, 49 skipped on ROCm (contracts/selection/EXL3 gating/imports), 401 passed on CUDA (including upstream sign-buffer regressions), and 14 passed on ROCm for portable EXL3 FP16/BF16 forward/oracle and lazy materialization/shard-layout checks. Ruff and diff checks pass. The explicit-device-family agent policy and skill are included in this head.

[ZX-ModelCloud]

c4489283211 — Already fixed on current head e29295152: _const is imported once as model_const, and DEVICE references use model_const.DEVICE. The existing thread reports 380 focused tests passing; Ruff passed again in this sweep. Local pytest was unavailable because this shell lacks torch.

[ZX-ModelCloud]

c4489306623 — Confirmed fixed at HEAD e292951: tests/test_rocm_kernel_contracts.py imports gptqmodel.models._const as model_const, retains that module import for monkeypatching, and qualifies all enum references as model_const.DEVICE. No code change or push needed. I could not rerun pytest/Ruff in this shell because neither tool is installed.