Variational Inference and Learning

Mean field and the direction of KL

Section 19.1 showed that inference can be viewed as maximizing over , and learning as maximizing it over . The core idea of variational learning is to maximize over a restricted family of distributions. The family should make the expectation easy to compute. A common choice is to assume that factorizes completely:

This is the mean field approach. Any graphical structure can be placed on instead, which the book calls structured variational inference. The factorial case is the simplest. The bound can be rewritten as a KL divergence, so maximizing over means minimizing in the reverse direction from the one used in maximum likelihood. The book chooses this direction for computational reasons: the bound takes its expectations under q, which we can design to be simple, while the other direction needs expectations under the true posterior.

The two directions treat errors differently. Minimizing punishes for putting mass where the posterior has little, so locks onto one peak. The other direction spreads over every peak. The demo fits a single Gaussian to a two-peaked posterior both ways and reports both divergences.

Binary sparse coding

The book applies mean field to a binary sparse coding model. Each hidden unit is on or off, with a bias that sets its prior. The visible vector is a linear combination of the active units, plus Gaussian noise with precision :

Maximum likelihood needs the derivative of with respect to a bias. The derivative then becomes a sum over every configuration of the hidden units. This expectation is taken under the true posterior, which has a complete graph over the hidden units (figure 19.2 of the book, which shows this graph), so no shortcut helps:

Equation 19.27 shows the derivative as an expectation under . That is the expectation that is too hard to compute, and variational inference replaces it with the factorial .

Fixed point updates

With the factorial of equation 19.28, the bound becomes a sum of simple terms, so it can be evaluated in closed form. The mean field parameters are probabilities, , and the book keeps them away from 0 and 1 so that logarithms stay finite:

Equation 19.36 is the bound written only with arithmetic, so it can replace the intractable log likelihood. Gradient ascent on would work, but it stores one vector per example. Fixed point equations are faster, because each unit is set by solving in turn:

The derivative of 19.36 with respect to one unit gives the expressions below. The first line is the derivative of the Bernoulli part, the second the derivative of the Gaussian part:

The update follows from setting 19.43 to zero, and the book calls it a recurrent network. The input sends a fixed message to each unit, and the units send messages to each other:

In the form of 19.45 each unit sees the residual rather than the raw input. Two units whose weight vectors point in similar directions inhibit each other. Set the sliders to (0.75, -2) and the second unit falls to about 0.09 while the first stays near 0.55. This is the mean field version of explaining away, and the factorial family cannot capture both modes of the true posterior. The demo uses two unit-norm atoms 70 degrees apart, unit noise precision and zero biases.

Binary sparse coding does not admit block updates, so the book uses a heuristic called damping. It solves for the individually optimal value of every unit, then moves all of them a small step in that direction. This is not guaranteed to increase the bound at each step, but it works well in practice for many models. The demo above updates one unit at a time, so it does not use damping.

Calculus of variations

When the latent variables are continuous, the family is a function, so maximizing over it needs calculus of variations. A functional takes a function as its argument, and its functional derivative is the rate of change at one point of that function. The book states the key identity:

As a worked example, consider the distribution of maximum entropy for a fixed mean and fixed variance . The entropy is defined as an expectation of the log density:

The entropy alone has no finite maximum, because it grows with the variance. The problem is made well posed by three constraints, handled with Lagrange multipliers: the density integrates to one, its mean is , and its variance is .

The functional derivative must vanish at every point, which gives an equation that determines the density:

The multipliers can be chosen so that the three constraints hold, and this gives a Gaussian. The book uses this result to justify the normal distribution when the true distribution is unknown: among all distributions with a given variance, the Gaussian imposes the least structure.

The demo compares three distributions with the same variance. The Gaussian always has the highest entropy. Its entropy is , measured in nats, and the other two are lower for every width.

Continuous latent variables

With continuous latent variables the optimal can be found from the model itself. If every other factor is held fixed, the best is the normalized exponential of the expected log joint:

Equation 19.56 is a fixed point equation, and it also gives the functional form of the optimum. In most applications the form can be read off and its parameters optimized with any method. Calculus of variations is only needed to design a new family.

The book's example has two latent variables and one visible variable, with and . The true posterior does not factorize, because the cross term in its exponent couples and :

Equation 19.56 applied to averages out . Only two moments of are needed, its mean and its second moment :

The result is a Gaussian, so with variational parameters that can be optimized freely. Nothing about the Gaussian form was assumed. Calculus of variations produced it. The demo below uses the same model to show what the factorial family misses.

Learning and inference interact

Approximate inference changes what the model learns. Variational learning increases the expectation over the parameters, which is equation 19.68:

For a fixed , this raises where is large and lowers it where is small. So the approximating assumptions become self fulfilling: a model trained with a unimodal tends to have a nearly unimodal true posterior.

The gap between the bound and can be small for the learned parameters and still hide a large loss. The gap only shows that the approximation is accurate at that . The real harm is measured against the best parameters , and the book notes that finding out is very hard: it needs a superior learning algorithm that can reach those parameters for comparison.

Check yourself

Questions that mix the whole module. Each one is scored, and the score feeds your mastery of this module.

Source: Ian Goodfellow, Yoshua Bengio and Aaron Courville, Deep Learning, MIT Press, 2016, Section 19.4 (pages 638–650), free to read at deeplearningbook.org. Where the book prints equation numbers, the numbers here follow the book. The order of ideas follows the book. The explanations and all demos were written for this page.