feat(cambricon): add CNCL backend for local multi-device collectives

[baominghelly]

Summary

Add a Cambricon CNCL backend for local, single-process multi-device collective communication. This enables the existing infinicclCommInitAll, infinicclCommDestroy, infinicclAllReduce, and infinicclAllGather APIs to operate on Cambricon MLU devices through CNCL.

This is a stacked PR based on #59 because the CNCL AllGather provider reuses the shared CCL AllGather implementation introduced there.

Changes

• CNCL backend integration
  • Add the WITH_CNCL build option, dependency discovery, bridge registration, and libcncl linkage.
  • Register CNCL for Cambricon devices in backend selection and priority mapping.
• CNCL provider
  • Add CNCL API and data-type/reduction-operation mappings.
  • Implement CommInitAll, CommDestroy, AllReduce, and AllGather through the shared CCL provider abstraction.
• Communicator initialization
  • Update CommInitAll dispatch to return one communicator handle for every requested local device.
  • Preserve the existing OpenMPI initialization path while adapting it to the multi-handle interface.

Platform and Backend Affected

Platform

• CPU
• NVIDIA GPU
• Iluvatar GPU
• MetaX GPU
• Moore Threads GPU
• Cambricon MLU
• HYGON DCU

Backend

• OpenMPI
• MPICH
• NCCL
• MCCL
• CNCL

OpenMPI is marked because its CommInitAll implementation is adapted to the corrected multi-handle interface; its communication path remains unchanged.

Performance Impact

• No performance impact
• Performance improved
• Performance regression possible

Cambricon local multi-device collectives use CNCL directly instead of relying on a host-staged communication path. This PR does not make a quantified performance claim.

Known Issues & Future Work

• CNCL initialization currently supports the local, single-process multi-device path through infinicclCommInitAll.
• CNCL GetUniqueId and CommInitRank are not implemented, so cross-process and multi-node CNCL initialization are not supported by this PR.
• This PR is stacked on feat(nccl): add all-gather communication #59 and should be rebased onto master after its prerequisite PRs merge.

Test Results

Test environment: one Cambricon node/container with four MLUs and CNCL 1.27.2.

• InfiniCCL Release build with Cambricon and CNCL passed.
• Existing CTest cases passed:
  • bridge_dependency_contract
  • operation_backend_selection
• Two-device CommInitAll + AllReduce + AllGather validation passed.
• Four-device AllReduce validation passed: rank inputs 1, 2, 3, 4 produced 10 on every device.
• InfiniLM Qwen3-0.6B inference on MLU with tensor parallelism 4 passed.
• InfiniLM Qwen3-32B inference on MLU with tensor parallelism 4 passed.
• CNCL reported topology and RDMA fallback warnings but successfully formed two MLU-Link rings and completed the collectives.

The CNCL implementation was validated before the history-only rebase onto the updated #59 branch. The CNCL commit replayed without conflicts; this Draft PR has not been rebuilt after that rebase.

Test Involved Platform

• CPU
• NVIDIA GPU
• Iluvatar GPU
• MetaX GPU
• Moore Threads GPU
• Cambricon MLU
• HYGON DCU

Test Involved Backend

• OpenMPI
• MPICH
• NCCL
• MCCL
• CNCL

---

Checklist

Title, Branch, and Commits

• PR title follows Conventional Commits.
• Branch name follows the repository branch naming convention.
• Each commit message follows Conventional Commits.
• Relative to the stacked base, this PR is a single squashable commit.
• No stray merge commits from master are present.
• No fixup!, squash!, or wip commits remain.

Scope and Design

• Changes are minimal and limited to CNCL support plus the required CommInitAll interface correction.
• No dead code, debug output, or unowned TODOs were introduced.
• No unrelated formatting churn was introduced.
• The existing public C API signatures remain unchanged.

General Code Hygiene

• Comments are limited to non-obvious intent.
• Modified and added files end with trailing newlines.
• git diff --check passes.
• Comments and error messages are in English and follow repository conventions.

C++ Specific

• Code follows the repository C++ style.
• Applicable formatting was checked on the validated implementation.
• No exceptions are introduced.
• Error handling follows repository conventions.
• N/A: No constructor initializer-list changes are included.

Python Specific

N/A: This PR does not modify Python files.

Testing

• Applicable local multi-device CNCL operations were built and tested successfully on Cambricon hardware.

Build, CI, and Tooling

• CNCL backend auto-detection and build registration are included.
• git diff --check passes; hosted CI will run on this Draft PR.

Documentation

• README backend documentation can be added following maintainer feedback.
• N/A: This change is additive and has no user-visible breaking change.

Security and Safety

• No secrets, internal URLs, customer data, or personal hardware identifiers are included.
• No third-party source code is introduced.
• Communicator inputs and CNCL type mappings are validated before dispatch.