<?xml version="1.0" encoding="UTF-8"?><rss version="2.0"><channel><title>dLLM Notes</title><description>Learning notes on diffusion language models: from the masked-diffusion objective to sampling, scaling, and fast inference.</description><link>https://buffett0323.github.io/</link><item><title>Starting from an AR checkpoint</title><link>https://buffett0323.github.io/learning-dllms/posts/ar-to-diffusion/</link><guid isPermaLink="true">https://buffett0323.github.io/learning-dllms/posts/ar-to-diffusion/</guid><description>How a pretrained autoregressive model is turned into a diffusion LLM: what has to change, the two routes (full attention and block-causal attention), and the models that took them, from DiffuLLaMA and Dream to SDAR, Fast-dLLM v2 and LLaDA 2.0.</description><pubDate>Sat, 31 Oct 2026 00:00:00 GMT</pubDate></item><item><title>Batching denoising steps</title><link>https://buffett0323.github.io/learning-dllms/posts/batching/</link><guid isPermaLink="true">https://buffett0323.github.io/learning-dllms/posts/batching/</guid><description>With a KV cache, a dLLM alternates between two kinds of step, Refresh and Reuse, with very different costs. What each one is for, and how to batch many requests whose steps do not line up.</description><pubDate>Sun, 01 Nov 2026 00:00:00 GMT</pubDate></item><item><title>An exact cache again</title><link>https://buffett0323.github.io/learning-dllms/posts/block-cache/</link><guid isPermaLink="true">https://buffett0323.github.io/learning-dllms/posts/block-cache/</guid><description>The caching side of block diffusion: why finished blocks give an exact KV cache, the extra pass that writes it, the approximate cache that returns inside the current block, what a step costs against AR and full attention, and how D2F decodes several blocks over the same cache.</description><pubDate>Sat, 14 Nov 2026 00:00:00 GMT</pubDate></item><item><title>Block diffusion</title><link>https://buffett0323.github.io/learning-dllms/posts/block-diffusion/</link><guid isPermaLink="true">https://buffett0323.github.io/learning-dllms/posts/block-diffusion/</guid><description>What block diffusion (BD3-LM) changes from a full-attention dLLM: the block-causal mask, the block-wise likelihood, training on [x_t, x_0] in one pass, and what it buys and costs at inference: an exact KV cache, variable length, and blocks that run in sequence.</description><pubDate>Mon, 09 Nov 2026 00:00:00 GMT</pubDate></item><item><title>Continuous diffusion on embeddings</title><link>https://buffett0323.github.io/learning-dllms/posts/continuous-diffusion/</link><guid isPermaLink="true">https://buffett0323.github.io/learning-dllms/posts/continuous-diffusion/</guid><description>The other route M02 set aside: map tokens to low-dimensional embeddings, add Gaussian noise there, and denoise with image-diffusion samplers. How it works, how it differs from masking, and what Nemotron-Labs-Sigma shows at 3B and 8B against a matched masked model.</description><pubDate>Sat, 21 Nov 2026 00:00:00 GMT</pubDate></item><item><title>Diffusion drafters: DFlash 1 and 2</title><link>https://buffett0323.github.io/learning-dllms/posts/diffusion-drafters/</link><guid isPermaLink="true">https://buffett0323.github.io/learning-dllms/posts/diffusion-drafters/</guid><description>DFlash drafts a block for an AR target with a small one-step block-diffusion model that reads the target&apos;s hidden features. DFlash 2 keeps the one pass and fixes two weaknesses of drafting in parallel: neighbours chosen independently, and guesses that get worse toward the end of the block. Compared with EAGLE-3 and with SDAR, a block-diffusion model that writes on its own.</description><pubDate>Sat, 21 Nov 2026 00:00:00 GMT</pubDate></item><item><title>Diffusion on discrete tokens</title><link>https://buffett0323.github.io/learning-dllms/posts/discrete-diffusion/</link><guid isPermaLink="true">https://buffett0323.github.io/learning-dllms/posts/discrete-diffusion/</guid><description>How diffusion carries over from images to text: corrupting token IDs with a transition matrix, and why the masking (absorbing) version wins.</description><pubDate>Sat, 24 Oct 2026 00:00:00 GMT</pubDate></item><item><title>Caching without causality</title><link>https://buffett0323.github.io/learning-dllms/posts/kv-cache/</link><guid isPermaLink="true">https://buffett0323.github.io/learning-dllms/posts/kv-cache/</guid><description>Why the autoregressive KV cache does not carry over to diffusion LLMs, and how the main papers get one back: Fast-dLLM, dKV-Cache, dLLM-Cache, Elastic-Cache, block diffusion and Fast-dLLM v2.</description><pubDate>Sun, 25 Oct 2026 00:00:00 GMT</pubDate></item><item><title>Masked diffusion models</title><link>https://buffett0323.github.io/learning-dllms/posts/masked-diffusion/</link><guid isPermaLink="true">https://buffett0323.github.io/learning-dllms/posts/masked-diffusion/</guid><description>The masked diffusion model end to end: the forward process, what the reverse process needs, and the training objective, step by step, for pre-training and for chat fine-tuning.</description><pubDate>Sat, 24 Oct 2026 00:00:00 GMT</pubDate></item><item><title>Mixture-of-Experts dLLMs</title><link>https://buffett0323.github.io/learning-dllms/posts/moe-dllm/</link><guid isPermaLink="true">https://buffett0323.github.io/learning-dllms/posts/moe-dllm/</guid><description>How Mixture-of-Experts layers work inside a diffusion LLM, where routing and expert selection happen, what MoE buys, and why block-wise parallel decoding erodes its sparsity.</description><pubDate>Sat, 31 Oct 2026 00:00:00 GMT</pubDate></item><item><title>LLaDA, Dream, and what came next</title><link>https://buffett0323.github.io/learning-dllms/posts/open-dllms/</link><guid isPermaLink="true">https://buffett0323.github.io/learning-dllms/posts/open-dllms/</guid><description>The open dLLMs most papers build on. LLaDA 8B, trained from scratch: architecture, training, scaling against AR, benchmarks, the reversal curse. Dream 7B, converted from Qwen2.5. And the LLaDA line after them: preference optimization in 1.5, block diffusion at 100B in 2.0, token editing in 2.1.</description><pubDate>Sun, 15 Nov 2026 00:00:00 GMT</pubDate></item><item><title>Parallel decoding</title><link>https://buffett0323.github.io/learning-dllms/posts/parallel-decoding/</link><guid isPermaLink="true">https://buffett0323.github.io/learning-dllms/posts/parallel-decoding/</guid><description>Confidence-aware parallel decoding from Fast-dLLM: commit every token the model is sure of, and why high confidence makes parallel decoding match sequential decoding.</description><pubDate>Sun, 01 Nov 2026 00:00:00 GMT</pubDate></item><item><title>Where the prompt goes</title><link>https://buffett0323.github.io/learning-dllms/posts/prompting/</link><guid isPermaLink="true">https://buffett0323.github.io/learning-dllms/posts/prompting/</guid><description>How a chat prompt enters a diffusion language model at inference: a clean prefix, a fixed canvas of masks, and no prefill. Compared pass by pass with an autoregressive model.</description><pubDate>Sat, 24 Oct 2026 00:00:00 GMT</pubDate></item><item><title>Sampling and remasking</title><link>https://buffett0323.github.io/learning-dllms/posts/sampling/</link><guid isPermaLink="true">https://buffett0323.github.io/learning-dllms/posts/sampling/</guid><description>How a masked diffusion model turns a canvas of masks into text: the sampling loop, which positions to unmask, how many per step, semi-autoregressive blocks, and remasking.</description><pubDate>Sat, 24 Oct 2026 00:00:00 GMT</pubDate></item><item><title>Keeping fewer tokens</title><link>https://buffett0323.github.io/learning-dllms/posts/sparse-kv/</link><guid isPermaLink="true">https://buffett0323.github.io/learning-dllms/posts/sparse-kv/</guid><description>Sparse KV caches for diffusion LLMs: why attention is sparse, how Sparse-dLLM picks the tokens to keep, why one shared set fails some heads, and how dLLM-Serve stores a per-head set without wasting memory.</description><pubDate>Sun, 08 Nov 2026 00:00:00 GMT</pubDate></item><item><title>Speculative decoding</title><link>https://buffett0323.github.io/learning-dllms/posts/spec-basics/</link><guid isPermaLink="true">https://buffett0323.github.io/learning-dllms/posts/spec-basics/</guid><description>How speculative decoding makes an AR model faster without changing its output: draft, verify, keep the matching prefix plus one token. The cost formula that every drafter in the S track trades on, and the kinds of drafter, AR, diffusion, and self-drafting.</description><pubDate>Sat, 21 Nov 2026 00:00:00 GMT</pubDate></item><item><title>Trimming the masked suffix</title><link>https://buffett0323.github.io/learning-dllms/posts/suffix-window/</link><guid isPermaLink="true">https://buffett0323.github.io/learning-dllms/posts/suffix-window/</guid><description>What the masked suffix of a full-attention dLLM actually does, and the four ways papers deal with it: recompute it, freeze it (Fast-dLLM DualCache, Elastic-Cache), drop most of it (DPad), or never create it (block diffusion). With the evidence, the costs, and how to read the speedups.</description><pubDate>Sat, 14 Nov 2026 00:00:00 GMT</pubDate></item><item><title>Why not left to right?</title><link>https://buffett0323.github.io/learning-dllms/posts/why-diffusion/</link><guid isPermaLink="true">https://buffett0323.github.io/learning-dllms/posts/why-diffusion/</guid><description>The case for diffusion language models: generating text by filling in blanks, many at once, instead of one token per forward pass.</description><pubDate>Sat, 26 Sep 2026 00:00:00 GMT</pubDate></item></channel></rss>