Score Matching and Ratio Matching

The score and the vanishing normalizer

Score matching trains a model without computing or its derivatives. The trick starts with a definition. For a density, the score is the gradient of its logarithm with respect to the input, . It points in the direction in which the density grows fastest, and it is the same whatever the normalizer is, because the normalizer is a constant with respect to .

Write a model as an unnormalized score times a constant, . Taking the logarithm turns the constant into an additive term, and the derivative with respect to removes it. The score of the model does not see at all.

The demo uses a one-dimensional model with a Gaussian shape, centred at with spread . The unknown constant multiplies the whole unnormalized curve. The data are drawn from the standard normal distribution, whose score is .

Only the top chart changes when the scale moves. The score curve in the bottom chart stays where it was, so any loss built from scores cannot be affected by . A likelihood would change, because it contains . A score has no such term. The book states the reason briefly: the normalizer is not a function of , so its derivative with respect to is zero.

Matching two scores

Score matching compares the model score with the data score. The natural measure is the squared difference between the two vectors, averaged over the data. The book writes the loss for one point as

Half the squared length of the difference of two gradients. It is zero exactly where the model and the data have the same slope at . The average over the data gives the objective, and the best parameters are the minimizer:

The objective is an average over the training distribution . Equation 18.24 defines the best parameters as the minimizer. Equation 18.22 needs the data score, which is the difficulty the next section removes.

The model here is a normal distribution with centre and spread . Its score is , and the data score is . Squaring the difference and averaging over the data gives the objective in closed form. This closed form is our calculation for this example and is not printed in the book:

With the objective is , so moving the centre away from zero raises it. The demo shows the same trade-off in the score curves. Where the two curves separate, the loss is large. The spread matters too: a wrong spread bends the model score even when the centre is right.

An objective that needs no data score

Equation 18.23 contains the data score through , which requires the unknown . The book removes it with an equivalence: the objective can be replaced by an average of derivatives of the model's own log density. This is Equation 18.25, which is the same as up to a constant that does not depend on .

The sum runs over every coordinate of . Each term needs the second derivative of the model's log density, and the squared first derivative, but never the data density. The next demo checks the claim: the two objectives differ by one constant for every setting of the Gaussian.

Both curves have the same shape, and the gap between them stays at one half wherever you move the sliders. The gap comes from the data alone, so it is the same constant for every parameter, and minimizing either curve gives the same best parameters. The lowest point of both is at the true values, centre 0 and spread 1.

The book notes that score matching can be seen as a form of contrastive divergence. Section 4 returns to that point.

What score matching can and cannot train

Score matching has two requirements, and the book states both. The data must be continuous, because the score needs derivatives with respect to , so the method does not apply to discrete data. The latent variables of the model may be discrete. Second, the method needs the derivatives of itself, so it cannot be combined with a lower bound on , since a lower bound says nothing about derivatives. For the same reason, it cannot train models with complicated interactions between hidden units, such as sparse coding or deep Boltzmann machines. The book notes that score matching can pretrain the first hidden layer of a larger model, but it has not been used for the deeper layers, which usually contain discrete variables.

The book also observes that score matching is a version of contrastive divergence, using a Markov chain that makes small moves guided by the gradient rather than Gibbs sampling. It is equivalent to contrastive divergence with that chain when the size of the moves approaches zero. Lyu (2009) generalized score matching to discrete data, and Marlin et al. (2010) corrected an error in that derivation. They found that generalized score matching does not work in high-dimensional discrete spaces where the observed probability of many events is zero.

Score matching is therefore a tool for continuous densities with tractable derivatives. For binary visible data the next section gives the matching tool.

Ratio matching for binary data

For binary data, the derivative in Equation 18.25 does not exist. The book's answer is ratio matching (Hyvärinen, 2007b). It compares the probability of each data point with the probability of each of its neighbours. A neighbour is the same binary vector with bit flipped. Ratio matching minimizes the sum of squared terms of the form

Read the inner fraction as a ratio . It is large when the data point is much more likely than its neighbour, and small when the neighbour is more likely. The inner term is . It is close to 0 when is large, and it is close to 1 when is small. Squaring it makes the objective smallest when every neighbour is much less likely than the data point. The partition function cancels in the ratio, which is the same trick as in Section 1 of this page.

Each bar is one neighbour term. A term near 1 means the neighbour is more probable than the data point, which the objective dislikes. A term near 0 means the data point is much more probable than that neighbour. Two parameters control the picture. The bias acts on every bit, and the coupling links bits 1 and 2, so it changes the terms for flipping bit 1 or bit 2, and leaves the terms for bits 3 and 4 unchanged.

For the data point 1100, no bias alone can make all four terms small. Two bits are on and two are off, and the bias pushes the on bits and the off bits in opposite directions. The coupling breaks that symmetry, and that is why the challenge needs both sliders.

Cost and a random shortcut

Ratio matching needs one evaluation of the unnormalized score for each neighbour. With bits, that is evaluations of for every data point, roughly times the cost of stochastic maximum likelihood per update. The book reports that Marlin et al. (2010) found ratio matching outperforms stochastic maximum likelihood, pseudolikelihood and generalized score matching at denoising test set images.

The cost matters most for sparse, high-dimensional data such as word count vectors. The book describes the difficulty: such data are expensive to write in dense form, and an MCMC sampler produces sparse values only after the model has learned the sparsity. Dauphin and Bengio (2013) designed an unbiased stochastic approximation to ratio matching. It evaluates a randomly selected subset of the terms and does not need complete fantasy samples. The demo below shows the principle behind an unbiased subset, using our own simple choice of one random neighbour. It is not Dauphin and Bengio's method.

Each draw is unbiased for the sum, since every bit has probability one quarter and the reported value is scaled by four. The running average is noisy at first and settles as the draws accumulate. The spread of single draws is large when the four terms differ a lot, which is why a single draw is a poor guide. Dauphin and Bengio's approximation applies the same principle to the full objective, with a random subset of its terms.

• The score of a model does not contain the partition function, because a constant has zero derivative with respect to x.
• Score matching minimizes the squared mismatch between model and data scores (Equations 18.22 to 18.24). Equation 18.25 removes the data score, and the two objectives differ by a constant.
• Score matching applies to continuous visible data with tractable derivatives. It does not apply to discrete visible data, and the book names sparse coding and deep Boltzmann machines as cases it cannot train.
• Ratio matching (Equation 18.26) compares each binary data point with its n neighbours, at a cost of about n evaluations per update.
• A random subset of the terms gives an unbiased estimate of the sum, which is the principle behind the cheaper approximation for sparse data.

Check yourself

Questions that mix the whole module. Each one is scored, and the score feeds your mastery of this module.

Source: Ian Goodfellow, Yoshua Bengio and Aaron Courville, Deep Learning, MIT Press, 2016, Section 18.4 (pages 617–619), free to read at deeplearningbook.org. Where the book prints equation numbers, the numbers here follow the book. The order of ideas follows the book. The explanations and all demos were written for this page.