Scalar and Vector Fields

A scalar field is a function that maps each point in a space to a scalar . In the context of generative models, probability densities are modeled as scalar fields where each point in space is assigned a scalar density value.

A vector field is defined as a function mapping each point to a vector . Velocity fields for a flowing quantity (e.g. mass in fluids, charge) can be modeled as vector fields, where each point in space is assigned a velocity vector representing the magnitude and direction of the flow along the coordinate directions for . The components are scalar functions of .

These fields may depend on time in addition to , in which case they may be written as and .

Gradient, Divergence and the Laplacian

The derivative of a scalar field with respect to the coordinate system is the gradient of the field. The gradient is a vector field, since each direction has a corresponding component in the rate of change. The gradient describes how a scalar field changes in space.

For a vector field in , the divergence is a scalar field defined as

Loosely, divergence tells us whether a vector field “expands” or “compresses” at a point. If we consider an -dimensional region with boundary . Let denote the outward unit normal at a point . Then the total outward flux through the boundary is

The divergence of the vector field can also be written as the net outward flux per unit volume around

Taking an infinitesimal hypercube with side lengths and applying the integral above to it yields the previous component-wise definition of divergence.

For a continuously differentiable field and a region with piecewise-smooth surface , the divergence theorem relates the flux across and the total net outflow over as follows

Intuitively, for a smooth flow without any sudden jumps, the total outflow through the boundary should be equal to the sum of all outflows within the region. If we were to slice up into multiple smaller sub-volumes , the total flux across all resulting boundaries would remain the same even though the sub-volumes have new surfaces along the cuts. This is because each cut produces two surfaces and the outward flux across one is the negative of the outward flux (i.e. inflow) across the other, leaving only the flux at the outer boundary of . If , we end up with infinitesimal sub-volumes, and the sum of net outflows over the sub-volumes is exactly the integral of the divergence over .

The divergence of the gradient of a scalar field is defined as the Laplacian of the scalar field

The Continuity Equation

The continuity equation describes how the density of a conserved quantity evolves when particles of that quantity move along a time-varying vector field. It does this by looking at the local dynamics of the quantity at a point in space. Let be the density of the quantity at a given point in space at a given time . Since the quantity is conserved, the total amount over the whole domain remains constant.

Now let’s assume that particles of this quantity move according to the velocity field . We can then define the dynamics of the system as an ODE

We define the local flow rate of the quantity through space as its flux field :

Under these assumptions, the density satisfies the continuity equation:

To prove this, consider an arbitrary region . The change in the total quantity inside must be equal to the negative flux across . If this were not true, the region might “accumulate” or “lose” the quantity in such a way that the total is not conserved at a given time .

Let’s assume that is integrable over and that is continuously differentiable in , so that we can interchange differentiation and integration. Substituting the divergence theorem then yields

This states that the integral is zero for every region . This means that the function being integrated must be zero pointwise, otherwise we could choose to isolate the point where it is non-zero and violate the above equation. Therefore:

This is the continuity equation in strong form. Strong form here implies that it gives us a pointwise (i.e. in ) PDE. With some additional assumptions, we can also derive a weaker form of this equation. Let’s take a scalar test function such that:

  • is infinitely differentiable
  • has compact support i.e. it vanishes outside a bounded region of

This class of test functions is rich enough that the following is true

In the case of the continuity equation , so we get:

We can apply the multivariable integration-by-parts identity here, and compact support will remove the boundary term.

Let be large enough to contain the support of . Then along the boundary of so the boundary term on the right is 0.

Substituting that back into the integrated continuity equation we get:

This is the weak form of the continuity equation.