0:02You're probably familiar with how diffusion is used to generate images. Start with noise, then denoise it over steps.
0:09At each step, the picture gets clearer. Each pixel, turned from noise into signal.
0:16And crucially, every pixel is looked at simultaneously, at each step. The entire image considered in full context.
0:23Go over enough steps, and the noise becomes a crisp image. Exactly as the prompt described.
0:29Here's a question. Could the same approach, work for text generation?
0:33What if you could give a model a prompt, and have it denoise a sentence into existence? How would that work? And more importantly, what would it get us?
0:42To see why that would be different, start with the way text generation works today.
0:46Every large language model you've used, generates text like this. One token at a time. Left to right.
0:54Token three can't be chosen until token two exists.
0:57Token two waits for token one.
0:59Each word is conditioned only on what came before it, never on what comes after.
1:04No revision. No backtracking. Once a word is committed, it's committed. And every decision is made with only half the context. The left half.
1:13But borrowing diffusion from images creates one immediate problem.
1:17An image can be ten percent noisy.
1:19fifty percent noisy.
1:21ninety percent noisy. It's a continuous value, corrupt it by any amount you want.
1:25But how would we translate this to text? What's the equivalent of ten percent noise, or fifty percent, for a sentence?
1:32A word is discrete.
1:33"River" is either there, or it isn't. There's no thirty-percent river.
1:38So instead, we mask the tokens. Mask one of nine, that's eleven percent corrupted.
1:44Mask five of nine, fifty-five percent.
1:47Mask all nine, and the sentence is fully corrupted.
1:50Noise in pixels equals masking in tokens.
1:53Once masking gives us a version of noise for text, the rest starts to look more like diffusion again.
1:59Step one. The model scans all nine positions in a single forward pass. Every masked position gets a prediction, in parallel.
2:07Two come back with high confidence, "river" and "stone."
2:10Those get filled. Two done, seven to go.
2:13Step two. The model again scans all nine positions. It makes predictions for all seven remaining. Picks the highest-confidence ones.
2:21Three more tokens get filled.
2:24Step three. Finally, the model makes predictions for the remaining four positions. The sentence is complete.
2:30And something interesting starts to happen. The content words, the ones carrying the most meaning, emerged first.
2:37Then more tokens filled in around them.
2:39The function words filled in last.
2:41This is something an auto-regressive model doesn't do. It fills in the next token no matter how unsure it was about what comes next.
2:48That's the important shift.
2:50The "diffusion" model isn't racing from left to right. It's letting the sentence come into focus, one confident iteration at a time.
2:58And this isn't just a different way to generate text. It's a completely different workload for the hardware.
3:03Here's what autoregressive generation looks like to the hardware. Nine tokens, nine sequential forward passes.
3:10The GPU does a burst of work, then waits, then another burst, then waits.
3:15Most of the silicon sits idle.
3:17Diffusion. Three steps, but each step processes all nine positions in parallel.
3:23One forward pass covers every token at once. The GPU stays saturated.
3:28The hardware was designed for exactly this kind of parallel work.
3:32For a short sentence, the difference is modest. But as sequences get longer, hundreds of tokens, thousands, watch the gap.
3:40The sequential approach grows with every token. The parallel approach barely changes.
3:45Make it concrete. A five hundred twelve token response. Autoregressive, five hundred twelve sequential forward passes.
3:53Diffusion with sixty four denoising steps, sixty four forward passes, regardless of how long the output is.
3:60The sequence length drops out of the equation.
4:04So when more of the work can happen in parallel, what does that actually translate to?
4:09Same prompt. Same model size. One generates left to right, one generates everything at once.
4:14Autoregressive. One token. Then the next. Then the next.
4:19Diffusion. The whole response, crystallizes in a few passes.
4:22Look at the tokens per second.
4:24Autoregressive, around fifty tokens per second. Diffusion, over a thousand.
4:29Same answer. Same model size. The sequential bottleneck, removed.
4:32That's the difference. The hardware was always capable of this speed. Autoregression was the thing slowing it down.
4:39But speed alone doesn't decide the whole story. Diffusion language models haven't replaced autoregressive ones across the board. There are honest gaps.
4:48Open-ended reasoning.
4:49long-form coherence.
4:50creative writing.
4:52Autoregressive models still lead on tasks like these. The gap is narrowing with each new paper, but it's real.
4:60This architecture is promising, not dominant. Not yet.
5:04So the point isn't that diffusion replaces autoregressive models overnight. It doesn't.
5:09It's that it changes the framing entirely.
5:11Modern language models taught us to think of generation as next token prediction: given everything so far, what comes next?
5:18Diffusion changes the question: given the whole shape of the answer, what's still missing?
5:23From writing one token at a time, to shaping the whole answer into place.
5:27THAT, is a "real paradigm shift".

Diffusion Language Models Explained: The Shift to Parallel Generation

Created by Abhyuday DesaiUpdated 2 Oct, v2Runs 5 min 35 sec

0 views

About this video

A visual explainer of diffusion language models. Covers how image diffusion translates to text via token masking, confidence-based parallel decoding where the most certain words emerge first, why autoregressive generation leaves GPUs idle between sequential passes, and how a fixed number of denoising steps makes generation speed independent of sequence length. Closes with an honest look at where autoregressive models still lead.

What changed

v202 Oct 2026abhydesai5 min 35 sec

Re-rendered in the studio with chapters and a transcript.

v1 has no transcript to compare. 6 sec shorter.

0 views0 likes on this version

v113 Jul 2026abhydesai5 min 41 sec

First published cut.

0 views0 likes on this version

Watch v1