Paper notes · arXiv 2026
TEAM: Temporal–Spatial Consistency Guided Expert Activation for MoE Diffusion Language Model Acceleration
Background: M09, Mixture-of-Experts dLLMs. In an MoE dLLM every token of the 32-token block routes to its own 8 experts at every step, so SDAR-30B-A3B loads over 50 of 128 experts per layer to accept about 3 tokens. TEAM splits the block’s tokens into three groups and treats each differently:
- Decoded tokens: cache them one step after they are decoded, so they stop routing (DCD).
- Cold masks (low confidence, far from decoded text): may only use experts already loaded for others (LAC).
- Hot masks (high confidence or next to decoded text): speculate, so more tokens are accepted per step (SEH).
Result on SDAR-30B-A3B: experts per accepted token fall from 17.7 to 6.9, below the nominal 8, for a 1.94× average speedup at about the same accuracy.
The PDF's license does not allow reposting it here; page and line numbers refer to the version linked above.