Section 1
Restricting attention
probability
conditional probability
, variables and values
scale factor
How to use this pageClick any colored symbol. Inside an equation, its meaning appears under the equation. In running text, a small note appears. Answer each prediction before the demo reveals the result, and move the sliders before you read the sentence under them.
How many days carry an umbrella? The answer depends on which days you count.
Predict first. Rain is on, so you look only at the rainy days. What share of them carry an umbrella? The demo uses a rate of 0.80 on rainy days and 0.15 on dry days.
Think of a spreadsheet filter. Filter the table down to the rainy rows, and the rows that remain are exactly the rows you started with. A conditional probability is a probability measured among the rows that a filter leaves. That choice of filter is conditioning. The weather does not change any day. It changes which days count.
Filtering a population
umbrella carriedno umbrella
The dots are 400 imagined days, one dot per day. Pick a tab to restrict attention to one weather. The printed fraction is the share of umbrella days among the days in view. If the rate on rainy days equals the rate on dry days, the weather tells you nothing, and the conditional probability equals the overall one.
Challenge +8 XPMake the umbrella rate on rainy days equal to the rate on dry days. The readout should say the weather no longer changes the answer.
The definition
Write the probability of given with the bar. The book’s Equation 3.5 gives it as a ratio:
Click any coloured symbol to see what it means.
The numerator is the joint probability: the chance that both things hold. The denominator is the chance of the known condition. The fraction asks: among the cases where , what share also has ?
Predict first. In the Linked model, the sunny row holds three joint probabilities, 0.30, 0.02 and 0.01, which total 0.33. What is the probability of walking given sunny?
The row is a slice of the joint table. Its entries add up to the row total, not to 1. That total is the scale. Divide the slice by it, and the slice stretches into a complete distribution. The result is the renormalized slice.
Slice, then divide
joint probability, the sliceafter rescaling
At rescaling 0 the bars show the raw slice. Slide to 1 and every bar is multiplied by the same . The shape never changes, only the scale, and the total becomes exactly 1. The Unrelated model gives the same conditional row for every weather. The Linked model gives a different row for each weather.
Challenge +8 XPChoose the Linked model and the rainy row, then rescale fully. The three bars must total exactly 1.
Going deeper: why the bars add to 1 (added here, not printed in the book)
Sum Equation 3.5 over every value of y. The denominator does not depend on y, so it comes out of the sum:
The numerator sums to the denominator, so the fraction is 1:
Every conditional distribution is therefore a proper distribution on its own. Section 3.6 uses the same identity in the product form P(y, x) = P(y | x) P(x).
When the condition never happens
Make rain a thousand times rarer. Does the umbrella share among rainy days move? Predict first, then slide the rain probability down and watch the rainy column.
Predict first. Rain becomes a thousand times rarer, but it stays possible. What happens to the probability of an umbrella given rain?
A condition that fades away
carries an umbrellano umbrella
The mosaic’s column widths are , the probability of each condition, and each column is split by the umbrella share inside it. The share among rainy days is 0.80, so the printed stays put while the column becomes a sliver. A conditional probability describes the inside of its column. Its width does not enter.
Predict first. Now make rain exactly 0. What does the umbrella share among rainy days read?
A rare condition is fine. An impossible one is not: with rain at exactly 0, the fraction reads 0 divided by 0.
Challenge +8 XPDrive the probability of rain below 0.001, while rain stays possible. The readout must still give a value for the umbrella share among rainy days.
This lesson is added here and is not printed in the book. A conditional distribution can be learned or checked only for conditions that can actually occur.
Section 4
The direction of the condition
This section is added here and is not in the book. Equation 3.5 is not symmetric. Swapping the roles of and divides by a different number, so in general the two conditionals are different numbers:
Predict first. A test is positive for 90% of people who have a condition and for 5% of people who do not. The condition affects 2% of people. If a test is positive, what is the chance that the person has the condition?
Two conditionals from one population
The people who lack the condition vastly outnumber those who have it, so even a small positive rate among them produces many positive tests. Each conditional uses its own denominator. The two numbers agree only in special cases.
Challenge +8 XPRaise the share with the condition until at least half of the positive tests come from people who have it.
Conditioning is not intervening
The book closes with a warning. That someone speaks German is evidence about them: it narrows the group you look at. Teaching someone German is an action, and the action does not change where that person comes from.
Predict first. The chance that a person comes from Germany, given that they speak German, is high. Now a random person is taught German. What happens to the chance that they come from Germany?
Seeing against doing
GermanyFranceBrazil
The world here is a toy with three countries. In Observe, only the filled dots stay in view, and nearly all of them are German. In Intervene, everyone is taught German, and the country shares stay exactly as they were.
Predict first. In Observe mode, the German share among speakers is about 0.95 at the default settings. After everyone is taught German, what is the German share among speakers?
The first is a conditional probability. The second answers an intervention query, which belongs to causal modeling. The book says it does not explore causal modeling.
Challenge +8 XPLower the German share to 0.10 or less. Then, in Observe mode, raise the chance that outsiders speak German until the share of Germans among speakers falls below one half.
What to take away
- A conditional probability is the joint probability divided by the probability of the condition (Equation 3.5).
- Geometrically it is a slice of the joint table, rescaled by so that it totals 1.
- It needs , however small. An impossible condition has no conditional probability.
- and differ in general (added here, not printed in the book).
- Conditioning on an observation is evidence. An intervention answers a different question.
Final quizCheck yourself
Questions that mix the whole module. Each one is scored, and the score feeds your mastery of this module.
Source: Ian Goodfellow, Yoshua Bengio and Aaron Courville, Deep Learning, MIT Press, 2016, Section 3.5 (pages 59), free to read at deeplearningbook.org. Where the book prints equation numbers, the numbers here follow the book. The order of ideas follows the book. The explanations and all demos were written for this page.