dLLM Notes

Paper notes · arXiv 2026

TEAM: Temporal–Spatial Consistency Guided Expert Activation for MoE Diffusion Language Model Acceleration

Linye Wei, Zixiang Luo, Pingzhi Tang, Meng Li

Background: M09, Mixture-of-Experts dLLMs. In an MoE dLLM every token of the 32-token block routes to its own 8 experts at every step, so SDAR-30B-A3B loads over 50 of 128 experts per layer to accept about 3 tokens. TEAM splits the block’s tokens into three groups and treats each differently:

Result on SDAR-30B-A3B: experts per accepted token fall from 17.7 to 6.9, below the nominal 8, for a 1.94× average speedup at about the same accuracy.

The PDF's license does not allow reposting it here; page and line numbers refer to the version linked above.