Noise-Contrastive Estimation

A normaliser that is a free parameter

Many models define a score for every state and leave the normalising constant unknown. The partition function is the sum of the scores, and it is hard to compute when there are many states. Noise-contrastive estimation avoids computing it by treating the constant as one more parameter to learn.

Noise-contrastive estimation (NCE) writes the model log-probability as the unnormalised score plus a constant , as in Equation 18.28. The constant stands in for minus the log partition function, and it is learned together with the parameters .

The toy has four states, labelled 1 to 4, with unnormalised scores 1, 2, 3 and 4. Their sum is . The data distribution puts probability 0.1, 0.2, 0.3 and 0.4 on the four states, which is exactly the normalised score, so a correct value of the constant exists. The demos use exact averages over this distribution, so no random draws are involved.

The answer to the prediction is goes up by one. Adding to every log-probability shifts the average by exactly . The readout shows that average, which equals 1.02 plus , using the identity below. The identity is added here and is not printed in the book.

Maximum likelihood cannot leave the constant free. Its average always rises with , so it would set the constant arbitrarily high. The book makes this point: the criterion would choose a large rather than one that makes the model a valid distribution.

The total mass of the model is , as the next line states. It equals one only at , which is . Challenge c1 asks you to find that point with the slider.

Turning the problem into a classifier

NCE builds a classifier that separates data from noise. Choose a noise distribution that is easy to evaluate and easy to sample from. Then add a binary switch variable , which says whether a point came from the model or from the noise. Equations 18.29 to 18.31 define the joint distribution.

The training set uses the same switch with the data in place of the model. The switch is one for a data point and zero for a noise point, each with probability one half, so the classifier sees both kinds of point. That is the setup for Equation 18.32, which Section 3 examines.

The demo below uses a noise distribution built from two parts, which is our construction and not the book's. The noise is a blend of the uniform distribution and the data, with weight on the data. Its formula is added here:

The readout gives the probability that the classifier assigns to the data for each state, which is the joint probability of Equation 18.33 when the model already equals the data. When is zero the noise is uniform, and the probability runs from about 0.29 on state 1 to about 0.62 on state 4. As rises, the noise looks more like the data and each probability moves towards one half. At = 1 the probability is exactly one half on every state.

The logistic form of the classifier

Equations 18.33 to 18.37 rewrite the classifier as a logistic function. Start from the probability that the point came from the model, in Equation 18.33. Divide the top and bottom by the model term, and the probability depends only on the ratio of the two densities.

Call the log ratio . The logistic function turns this log ratio into a probability, and it has the classic S shape. Equations 18.36 and 18.37 agree because minus the log of a ratio is the log of its inverse.

Drive the slider to its ends. At the model is about 55 times less likely than the noise, and the probability is 0.02. At the model is about 55 times more likely, and the probability is 0.98. The curve is steepest at , where the model and the noise agree.

Where the classifier objective peaks

Equation 18.32 says that the model parameters and the constant are chosen to maximise the classifier log-likelihood. Written as an average over the data and the noise, with the switch weights of one half from Equation 18.29, the objective becomes the population form below. Here is the log ratio of the unnormalised model to the noise, . The population form is added here and is not printed in the book.

The objective is concave in , because each term is the logarithm of a logistic function of a line in . Its slope is from the next line, and the peak is where the slope is zero.

At each state's term in the slope vanishes on its own, because the logistic function then equals . The peak is at the same place for every noise distribution. This argument is added here, and the demo checks it.

The peak is at for both noises, which is with . The largest value of the objective differs between the two noises: about -0.67 for uniform noise and about -0.59 for skewed noise. Only the position of the peak is shared.

Choosing the noise and its limits

The noise must be easy to evaluate and easy to sample from, and it must also be hard enough to matter. The book's point is that very simple noise yields samples that are too easy to reject, so the model improves little. The demo measures this with two Gaussian densities, an example added here and not taken from the book.

Take data from a Gaussian with mean 0 and noise from a Gaussian with mean , both with spread 1. The best classifier thresholds at and is correct with the probability given by the added formula below.

At the two densities coincide and the classifier is at chance, 0.50. At the best classifier is correct about 98 percent of the time, and the samples are so easy to separate that the model learns little from them. Challenge c5 asks for the separation at which accuracy first reaches 95 percent.

The book makes three further points about NCE in section 18.6. First, NCE becomes less efficient with many random variables, because the classifier can reject a noise sample on one unlikely variable and learning then slows. Second, like score matching and pseudolikelihood, it does not work if only a lower bound on the unnormalised model is available. Third, when the noise is a copy of the model made before each gradient step, the method becomes self-contrastive estimation, whose expected gradient equals that of maximum likelihood. The book closes the section with a closely related idea: a good generative model should produce samples that no classifier can distinguish from data. That idea yields generative adversarial networks (section 20.10.4).

• NCE writes the model log-probability as the unnormalised score plus a constant, and it learns that constant with the parameters (Equation 18.28).
• Maximum likelihood alone would push the constant up without limit, because the average log-likelihood of the data is a fixed term plus c.
• NCE turns the problem into a classifier between data and noise, with a switch variable and a logistic form for the probability (Equations 18.29 to 18.37).
• The population objective peaks at minus the log partition function for every noise distribution, because each term of its slope vanishes there.
• The noise must be hard enough to matter. A separation of about 3.3 gives 95 percent accuracy in the Gaussian example, and very simple noise gives little to learn.

Check yourself

Questions that mix the whole module. Each one is scored, and the score feeds your mastery of this module.

Source: Ian Goodfellow, Yoshua Bengio and Aaron Courville, Deep Learning, MIT Press, 2016, Section 18.6 (pages 620–622), free to read at deeplearningbook.org. Where the book prints equation numbers, the numbers here follow the book. The order of ideas follows the book. The explanations and all demos were written for this page.