Like flow matching models, diffusion models seek to learn a generative process between a simple distribution and a target distribution . But where flow matching constructs a deterministic probability path between the two distributions, diffusion models construct a stochastic forward noising process that progressively corrupts samples from toward samples from . Learning how to reverse this process allows us to start from samples of and transform them into samples from .

The noising process can be written as a stochastic differential equation that defines the evolution of a sample from over time:

Here is the vector field that determines the drift of the trajectory over time. is an increment of the continuous-time Brownian motion, and this term determines the diffusion or “random scatter” of the trajectories over time. Let denote the marginal density of . We want to derive a reverse-time SDE that passes through these marginals in reverse order. To do this, we can start with the Fokker-Planck equation, which relates the change in the density to the drift and diffusion:

Here the terms are second-order spatial derivatives acting on the products . We can express this in a similar form to the continuity equation:

where the divergence of the matrix-valued term is taken row-wise, so that

We denote reverse time as , so that forward time is then . Define the reverse-time marginal density

In reverse time, the probability current changes sign, so we get

From Anderson’s result for diffusion processes, we know that the reverse-time process for a diffusion is also a diffusion with the same diffusion matrix. So let’s assume a reverse-time SDE with position , drift and diffusion matrix . Matching this to flow above we get

We can then match the reverse-time current to the negative of the forward current:

Solving for the reverse drift gives

Expanding the numerator yields

so that

In the common isotropic case, where the diffusion covariance is spatially constant and proportional to the identity so that

the correction term vanishes and the reverse-time drift simplifies to

Given that is fixed and chosen as part of the forward noising process, we need to learn in order to simulate the reverse-time SDE and draw samples from .

Training

The training target of diffusion models is the score function

Equivalently, in reverse-time notation this is . To learn it, we can use a mean-square penalty on the marginal score:

Since we do not have access to the true marginal score, we instead use a conditional score target. This yields the conditional score matching loss:

This conditional score matching loss expectation is an unbiased estimator of the true score function due to a similar argument to the conditional flow matching loss unbiasedness proof.

For Gaussian noising processes, the conditional score is available in closed form, which makes this objective tractable. In practice, many denoising diffusion models use equivalent parameterizations such as noise prediction or denoiser prediction rather than predicting the score directly.

Sampling

Once we’ve learned an estimator of the score function, we can draw a sample from the simple distribution and simulate the reverse-time SDE to obtain a sample from . In the isotropic case, which is used in most diffusion models, this reverse process is

Using a simple Euler-Maruyama solver, this becomes:

After the final step, is an approximate sample from .

In practice, numerical solvers or more advanced sampling methods like Langevin Dynamics are used for better results.