Joint Distributions, Covariance and Independence
Two variables have a joint distribution, from which marginals follow. Covariance measures how they move together, and independence forces zero covariance without the converse holding.
What a learner can do afterwards
- Find marginal distributions from a joint table or density
- Compute a covariance and interpret its sign
- Give dependent variables with zero covariance
1 · Read
A joint table holds both variables at once, and a marginal sums the other one out. Take P(X=0,Y=0)=0.4, P(X=0,Y=2)=0.1, P(X=1,Y=0)=0.2, P(X=1,Y=2)=0.3. Then P(X=1) adds the X=1 column: 0.2 + 0.3 = 0.5. And P(Y=2) adds the Y=2 row: 0.1 + 0.3 = 0.4. Test independence by factorising: P(X=0) times P(Y=0) is 0.5 times 0.6 = 0.3, but the joint cell holds 0.4, so these two are dependent.
Covariance averages (X minus mean) times (Y minus mean), with the shortcut Cov = E[XY] minus E[X]E[Y]. Here E[X] = 0.5 and E[Y] = 0.8. Only cell (1, 2) feeds E[XY]: 1 times 2 times 0.3 = 0.6. The means multiply to 0.5 times 0.8 = 0.4. So Cov = 0.6 minus 0.4 = 0.2. The positive sign says above average X tends to arrive with above average Y. That sign is direction only: it proves no cause and measures no strength.
Independence forces zero covariance, but zero never proves independence. Take X symmetric about zero with Y = X squared: deviations cancel by symmetry, so the covariance is zero, yet Y is fully set by X. Lee's claim that zero covariance certifies independence breaks on this one example. Independence is the strong demand that every joint chance factorises into its marginals.
Run the quick dependence check: compare one joint cell against the product of its marginals. Any mismatch certifies dependence. And never read a zero covariance as a clean bill of independence.
Marginals sum the joint table out, covariance signs the co-movement, and zero covariance never certifies independence.
2 · Watch
Take it off screen
Where it sits
Where this leads
Jobs that lean on this skill. Follow one to see everything it is built on.
8 questions wait behind this lesson, each with its answer explained. Every answer feeds the sky: stars light as they are learned, and dim when it is time to come back.