dLLM Notes

Speculative Drafting · S03

Diffusion drafters: DFlash 1 and 2

A whole block in one pass. Then make the block hold together.

4 min read

S01 left a trade-off: a drafter must be cheap and accurate, and an AR drafter pays one pass per guess. A diffusion drafter guesses the whole block in one pass. It brings the problem of every parallel decoder with it (M02, P05): positions guessed together do not see each other.

contextLast weektarget would write

One round for the same context, in toy cost units (one target pass = 1). EAGLE-3 guesses one token per pass with a 1-layer drafter. DFlash 1 fills 8 masks in one pass of a 5-layer drafter, each position on its own. DFlash 2 runs the same pass and picks a path through the candidates. SDAR is a block-diffusion LM writing alone: several steps of the full model, and no check. The thin bar under a draft is its confidence; the bar under each lane is time.

DFlash: one step, one pass

DFlash (notes) keeps the target as it is and trains a 5-layer block-diffusion drafter next to it:

  • One step. The block after the target’s last token is all masks, filled in one denoising step. No iterative refinement, unlike a dLLM such as SDAR.
  • It reads the target. Hidden states from 5 target layers are fused and added as extra keys and values in every drafter layer. Without them the drafter manages 2–3×; with them, about 5×.
  • Depth is affordable. EAGLE-3 pays one pass per guess, so it uses a single layer. DFlash pays once per block, so it can use five, and its drafts are accepted more.

The cleanest evidence for the cost argument is Qwen3.5’s built-in MTP heads, which draft one token after another. They are accepted about as often as DFlash (6.5 against 7.1 tokens per round) and give half the speedup (1.5× against 3.0×).

The headline “up to 2.5× faster than EAGLE-3” comes from batch-1 runs in HuggingFace Transformers. With both drafters trained on the same data and served by SGLang, DFlash’s lead at one request is about 1.3× (2.4× against 1.9×).

What one step costs. With one step, the guesses inside the block are independent given the context. Two things follow:

  1. Neighbours can disagree. Each position takes its own most likely token. If position 6 is “New” or “San” and position 7 is “York” or “Francisco”, each can be the top pick on its own and still give “New Francisco”. The target rejects there, and everything after it is lost.
  2. The end of the block is weak. The farther a position is from the last known token, the less it can be guessed alone. Acceptance falls along the block (suffix decay). DFlash weights the loss toward early positions, which helps training but does not change the shape.

DFlash 2: keep the pass, fix the block

DFlash 2 (Z Lab and Inco AI, August 2026) has no paper; the source is the release post. It keeps the one-pass drafter and adds two small parts.

A path selector, for disagreeing neighbours. Keep the top 16 candidates at each position and score every adjacent pair (a,b)(a, b):

St(a,b)=Ut(b)+⟨A(a)⊙H(ht), B(b)⟩S_t(a, b) = U_t(b) + \langle A(a) \odot H(h_t),\ B(b) \rangle

Ut(b)U_t(b) is DFlash’s own logit for bb. The second term asks whether bb fits after aa: AA and BB are small 256-d token embeddings, gated by the drafter’s state hth_t. All scores come from the one pass, with no extra pass of the backbone or the LM head. A greedy walk over them, left to right, picks the path. It is a learned bigram on top of the drafter: cheap, and enough to turn “New Francisco” into “New York”.

A two-tap convolution, for the end of the block. Around each attention and feed-forward sublayer:

Conv(x)t=kt,0⊙xt+kt,1⊙xt−1\text{Conv}(x)_t = k_{t,0} \odot x_t + k_{t,1} \odot x_{t-1}

with coefficients computed from the input. Each position mixes in its left neighbour’s state. It stays inside the block and keeps no state across rounds. On a 5-layer drafter it adds 3% parameters.

Together they add about 1% to a round and about one accepted token. Matched training, Qwen3.5-4B, mean tokens per round:

MTPDFlashDSparkDFlash 2
4.544.925.495.97

Caveats:

  • These are the authors’ runs, in a post, not a paper.
  • On Qwen3.8-27B in SGLang (one H200, GSM8K), DFlash 2 gives 3.43× at one request and 1.45× at 32. The fall with load is the same as DFlash’s (S07).
  • One independent llama.cpp test (a 2-bit Muse Glimmer 30B on an RTX A4000) finds DFlash 2 a little ahead of DFlash 1 on code, and both slower than no speculation on short Japanese prompts.

Against a dLLM that writes alone

SDAR (P03, M07) is the same idea without a target: an AR model converted to block diffusion, decoding a block in several steps and committing only confident positions per step. It avoids “New Francisco” by waiting, and “York” is committed a step after “New”. But every step is a pass of the full model, and nothing checks the result, so the text is only as good as the dLLM.

A diffusion drafter takes the opposite position: one step, a small model, and a target that rejects whatever does not fit. It can afford to be wrong, as long as it is cheap.

References

  1. Chen, Liang, Liu. DFlash: Block Diffusion for Flash Speculative Decoding. ICML 2026. arXiv
  2. Inco AI. DFlash 2: Keep Drafting Parallel. 2026. Post
  3. Li et al. EAGLE-3: Scaling up Inference Acceleration of Large Language Models via Training-Time Test. 2025. arXiv
  4. Cheng et al. SDAR: A Synergistic Diffusion-AutoRegression Paradigm for Scalable Sequence Generation. 2025.