Joint, Marginal and Conditional Probability
A discrete joint probability table describes combinations of values. Add cells to find marginals, divide by a positive conditioning probability to find conditionals, and compare joint entries with marginal products to check independence.
What a learner can do afterwards
- Read a joint table and compute both marginals and a conditional from it
- Say what independence means as a statement about the joint distribution
- Compare a conditional probability with its reverse and explain when they can match
1 · Read
A joint table records combinations of two variables. Let X be 1 for bike delivery and 0 for van delivery. Let Y be 1 for on time and 0 for late. A fictional set has 20 parcels: bike/on time 6, bike/late 4, van/on time 8, van/late 2. Choose one parcel uniformly. Each joint probability is its cell count divided by 20. The four probabilities total 1.
A marginal combines every cell for one requested value. Add both bike counts: P(X=1)=(6+4)/20=0.50. Add both on-time counts: P(Y=1)=(6+8)/20=0.70. The other marginals are P(X=0)=0.50 and P(Y=0)=0.30. Each variable's marginal distribution totals 1. With probability entries, add them directly. With counts, divide the selected total by the whole group's count.
P(A given B) means the probability of A within B. Calculate P(A given B)=P(A and B)/P(B), provided P(B)>0. With counts, divide the overlap count by the B count. If B includes several rows or columns, include every cell in B before dividing. All conditional probabilities within that group total 1. When P(B)=0, this ratio does not define a conditional probability.
Among bike parcels, 6 of 10 are on time: P(Y=1 given X=1)=6/10=0.60. Among on-time parcels, 6 of 14 came by bike: P(X=1 given Y=1)=6/14, about 0.43. Reversing the condition keeps the overlap but changes the denominator. With both denominators positive, the reverse probabilities agree exactly when the overlap is zero or the denominators are equal.
Independence means every joint entry equals the product of its two marginals. Here 6/20=0.30 differs from 0.50*0.70=0.35, proving dependence. One mismatch is enough. One match is not generally enough, but in a complete 2-by-2 table, one match and all marginals force the other three matches by subtraction. For independent variables, conditioning on a value with positive probability leaves the other variable's distribution unchanged.
Recover missing entries by subtraction. All joint entries total 1, and entries in a row or column total its marginal. If a two-entry row totals 0.50 and one entry is 0.30, the other is 0.20. You can also recover an overlap: P(A and B)=P(A given B)*P(B), when P(B)>0. For bike parcels, multiply the on-time conditional 0.60 by the bike marginal 0.50 to get the joint probability 0.30.
An event can include several joint cells. Select all cells satisfying the event, then add their probabilities. The cells describe separate combinations, so they do not overlap. For X=Y, select pairs with equal values, such as (0,0) and (1,1). For X at least 2, select all rows whose X value is 2 or more. To condition on an event, divide its overlap with the requested event by its positive total probability.
Marginals alone do not determine a joint table. Suppose X has probabilities 0.20 and 0.80, and Y has probabilities 0.50 and 0.50. In row order, joint entries 0.10,0.10; 0.40,0.40 give independence. Entries 0.20,0; 0.30,0.50 have the same row and column totals but are dependent. A joint table describes only its recorded variables, not every detail of the original outcomes.
Add joint cells for marginals and events. Divide by a positive conditioning probability for conditionals. Independence requires the marginal product rule in every cell.
Lesson, questions and flashcards written for this dataset. Numerical scenarios use fictional data.
Practice questions and supplementary readings written for this dataset. Numerical scenarios use fictional data.
More to read
Choosing the denominator
A fictional box contains 12 books. Four are nonfiction: three hardbacks and one paperback. Eight are fiction: two hardbacks and six paperbacks. Choose one book uniformly from the box.
The four joint counts are 3, 1, 2 and 6. Divide each count by 12 to get the joint probabilities. A nonfiction hardback belongs to one cell, so its probability is 3/12=0.25.
To find the marginal probability of a hardback, include both fiction categories. There are three nonfiction hardbacks and two fiction hardbacks. The marginal is therefore 5/12.
For P(hardback given nonfiction), restrict the group to the four nonfiction books. Three are hardbacks, giving 3/4. For the reverse question, restrict to the five hardbacks. Three are nonfiction, giving 3/5.
The same overlap count appears in both conditional calculations. The conditioning group determines the denominator. These probabilities describe choices from this box; they do not establish proportions in other book collections.
When reverse conditionals agree
A fictional collection contains 80 accounts. Twenty use a password manager, 32 use two-factor authentication, and 16 use both. Choose one account uniformly.
Among accounts using two-factor authentication, 16 of 32 use a password manager. This conditional probability is 0.50. Among accounts using a password manager, 16 of 20 use two-factor authentication. The reverse conditional is 0.80.
Both numerators count the same 16 accounts. The questions use different conditioning groups, so their denominators differ. Reversing a conditional does not mean taking the reciprocal of its value.
If the overlap has positive probability, the two reverse conditionals agree exactly when their positive denominators are equal. If the overlap has probability zero, both conditionals are zero whenever both denominators are positive. Different denominators alone do not guarantee different answers.
The ratio formula requires a positive conditioning probability. When that probability is zero, division by zero does not define the conditional probability.
Summarizing a feature and a label
A fictional training set contains 50 labeled messages. Each row records whether the message contains a link and whether its label is spam. Grouping the rows gives four counts: link/spam 12, link/not spam 8, no link/spam 3, no link/not spam 27.
Choose one training message uniformly. Fifteen of the 50 are labeled spam, so P(spam)=0.30. Within the 20 messages containing a link, 12 are spam, so P(spam given link)=0.60. Within the 15 spam messages, 12 contain a link, so P(link given spam)=0.80.
This table summarizes two fields from the original rows. It leaves out details such as the sender and the rest of the message text. The difference between the conditional and marginal proportions shows an association in this set. It does not show that adding a link causes a message to be spam.
Training proportions also need not match those of future messages. Some classifiers estimate conditional probabilities, but a classifier score need not be a probability or come from a stored joint table.
2 · Watch
Take it off screen
Where it sits
18 questions wait behind this lesson, each with its answer explained. Every answer feeds the sky: stars light as they are learned, and dim when it is time to come back.