Chapter 1 — The data landscape

Captain Obvious wants to let you know that images are made of pixels and each pixel can be seen as a separate dimension in a multidimensional space. But as human mortals, we'll have to settle for a 2D visualization, where the axes indicate the pixel intensities in a 2-pixel image.

The blue cloud represents a probability density. Bright regions are more likely to contain meaningful images (hover over $x_1$, $x_1^{\prime}$, $x_1^{\prime\prime}$), and dark regions are full of noise (hover over $x_1^{\prime\prime\prime}$).

We'll call the density $p_\text{data}$ because it reflects the true (but unknown) density of the data. The ultimate goal of generative modeling is to learn $p_\text{data}$ from a set of training images. That way, at inference time, we can generate new ones, unseen during training.

Older methods like Generative Adversarial Networks (GANs) attempted to learn $p_\text{data}$ directly, but their output quality suffered. The issue is — this density is extremely complex to learn in one go. It literally makes a judgment about every possible combination of pixels.