Chapter 1 — The role of reverse diffusion

In Fundamentals, we framed generation as motion on a time-varying landscape $p_t(x)$.

Forward diffusion then created noisy particle trajectories for training, starting from real images in the training set (see left plot). A cluster of such particles empirically recreates the probability density $p_t(x)$ (middle plot).

In Model training, we discussed how a model leverages these simulated paths to learn the score $\nabla_x \log p_t(x)$, which points uphill.

Reverse diffusion is the inference-time process that guides the ball in the opposite direction—from a valey to a peak or, equivalently, from a noisy frame to a well-structured image. It repeatedly invokes the trained model and gives us a recipe for how to follow its local slope predictions.

Why not gradient ascent?

"Follow the gradient" should sound familiar—it happens during training with gradient descent, when we navigate the loss landscape. So why not apply the same philosophy here, but follow the gradient ascent instead?

$$x_{t-1} = x_t + \eta \nabla_x \log p_t(x_t)$$

where $\eta$ is the step size.

The reason is: gradient ascent is an optimization algorithm. It assumes that there is a single best solution and aims to find it. But data generation is different. There is no single best picture of a golden retriever, so we don't always want to visit the exact same peak. What we need is a sampling algorithm that visits peaks proportional to their probability $p_0(x)$.

This is where diffusion comes in handy again.