A distributed hardware-software co-design fabric for MoE models (DeepSeek-V3, Mixtral) that eradicates NCCL All-to-All communication stalls via distributed RoCEv2 RDMA virtual address MUX and JAX/XLA SPMD sharding.
hpc zero-copy distributed-computing cuda pytorch high-performance-computing moe rdma low-level computer-architecture bare-metal compiler-optimization jax xla mixture-of-experts mixtral deepseek-v3 rocev2 accelerator-architecture
-
Updated
Aug 9, 2026 - Python