[ZX-ModelCloud]

Summary

Add EXPERTS_DEFUSE registry entries for afmoe and bailing_hybrid, so their fused routed-expert tensors are split into per-expert gate_proj / up_proj / down_proj nn.Linear modules and driven by the existing linear_loop implementation.

Without these entries both families stayed packed as 3-D parameters (gate_up_proj / down_proj), so quantizers could not see or quantize any routed expert.

Changes

• defuser/model_registry.py
  • afmoe → transformers.models.afmoe.modeling_afmoe.AfmoeExperts, forward_impl="linear_loop"
  • bailing_hybrid → transformers.models.bailing_hybrid.modeling_bailing_hybrid.BailingHybridExperts, forward_impl="linear_loop"
• README.md: add bailing_hybrid to the "Standard routed expert tensors" table
• pyproject.toml: version 0.0.28 → 0.0.29

Both expert classes store gate_up_proj as (num_experts, 2 * intermediate, hidden) and down_proj as (num_experts, hidden, intermediate), which is exactly the layout linear_loop / SplitFusedExpertGateUpProj expects, so no new forward implementation was required.

Tests

New tests in tests/test_convert_model.py:

• test_afmoe_registry_and_defusion / test_afmoe_split_forward_matches_fused_math
• test_bailing_hybrid_registry_and_defusion / test_bailing_hybrid_split_forward_matches_fused_math

Each asserts: registry entry shape, fused parameter shapes before conversion, that conversion removes the 3-D parameters, that numbered containers 0..N-1 exist with per-expert nn.Linear projections, per-expert weight equality against the original slices, and forward equivalence against the fused math (atol=1e-5).

Also adds bailing_hybrid to META_MODEL_CASES in tests/test_meta_model_defusion.py.

pytest tests/test_convert_model.py tests/test_meta_model_defusion.py tests/test_moe_experts_interface.py
→ 13 failed, 95 passed

The 13 failures are pre-existing on main (transformers-version / environment issues) and are identical with and without this branch — no regressions introduced. Focused runs:

pytest -k "afmoe or bailing_hybrid"  → 6 passed

The bailing_hybrid forward-equivalence test exercises the experts module only, not the full decoder: its delta-attention path calls causal_conv1d, which requires a CUDA device.

Note for reviewers

This is the prerequisite for the GPTQModel PR that fixes MoE expert quantization coverage. GPTQModel's requirements.txt needs defuser>=0.0.29, so this release must land first.

[Qubitium]

Reviewed a954de2da93f74bb1842b2075f3012f6b79560b8 against base 0f0b5899b3e96a779f54b0cfe1e32600c1634a1a.

No actionable correctness findings in this diff. Both upstream expert classes have the projection layout, activation, expert-count attribute, and routing call signature expected by the existing linear_loop path. The declarative entries also install the instance-level forward required by these undecorated expert classes. The added tests cover conversion structure, weight slices, expert-output equivalence, AFMoE logits, and meta-device conversion.

Validation on CPU with Transformers 5.20.2 and PyTorch 2.16.0.dev20261021+cpu:

• Focused AFMoE/BailingHybrid tests: 6 passed.
• pytest tests/test_convert_model.py tests/test_meta_model_defusion.py tests/test_moe_experts_interface.py -q --tb=short: 95 passed, 12 failed, 1 skipped on the PR.
• The same three files on the base commit: 89 passed, 12 failed, 1 skipped. The failing test IDs are identical, so this run introduces no additional failures.

Validation limit: no CUDA execution or full BailingHybrid delta-attention decoder forward was exercised; its numerical check covers the routed experts. The broader suite is not green in this environment, although its failures also reproduce on the base commit.