Conditional Expectation Given a Sigma-Algebra
For integrable X, define E[X|G] by G-measurability and matching integrals on G-events. For square-integrable X, interpret it as a least-squares prediction.
What a learner can do afterwards
- State the defining property and prove uniqueness up to a null set
- Recover conditional expectation from a finite partition and explain why a constant conditional mean need not imply independence
- Use the tower property for nested sigma-algebras and the L2 projection interpretation on worked examples
1 · Read
Work on a probability space (Omega,F,P). Let X be real with E[|X|]<infinity, and G be a sub-sigma-algebra of F. Conditional expectation Y=E[X|G] exists and is integrable and G-measurable. It satisfies E[1_A Y]=E[1_A X] for every A in G. Here 1_A is 1 on A and 0 elsewhere. The identity equates integrals over A, not normalized averages. A null set has probability 0; almost surely means outside a null set.
If G is generated by a finite measurable partition A_1,...,A_n, a G-measurable forecast is constant on each block. On a block A with P(A)>0, the defining identity gives its value c=E[1_A X]/P(A). For finite outcomes, add value times outcome probability within A, then divide by P(A). An unweighted average works when the outcomes within A are equally likely. On a probability-zero block, any finite constant works.
Suppose outcomes a,b,c have probabilities 0.1,0.4,0.5 and X-values 1,6,9. G distinguishes {a,b} from {c}. The first block has probability 0.5 and weighted total 0.1(1)+0.4(6)=2.5. Its forecast is 2.5/0.5=5. The singleton block's forecast is 9.
Suppose Y,Z satisfy the definition. D=Y-Z is integrable and G-measurable. For A={D>0} in G, subtract their identities: E[1_A D]=0. A nonnegative variable with mean 0 is 0 almost surely: otherwise some set where it is at least 1/n would have positive probability and force a positive mean. Thus P(D>0)=0. Swap Y and Z to get P(D<0)=0, so Y=Z almost surely.
Taking A=Omega gives E[E[X|G]]=E[X]. For partition-block forecasts c_i, this says E[X]=sum_i P(A_i)c_i. The nested tower property says that if H is a sub-sigma-algebra of G, then E[E[X|G]|H]=E[X|H] almost surely. The outer conditioning uses the coarser information. No independence is required.
Let X count heads in three independent fair coin flips. G records the first flip. After heads, the forecast is 1+0.5+0.5=2. After tails, it is 0+0.5+0.5=1. Each branch has probability 1/2, so the forecast's mean is (2+1)/2=1.5. This equals E[X]=0.5+0.5+0.5.
If integrable X is independent of G, E[X|G]=E[X] almost surely. The constant E[X] is integrable and G-measurable. For every A in G, independence gives E[1_A X]=P(A)E[X], which also equals E[1_A E[X]]. Thus the constant satisfies the definition, and uniqueness identifies it as the forecast.
For E[X^2]<infinity, use L2(F), the space of square-integrable real variables, identifying variables equal almost surely. Its inner product is E[UV]. The G-measurable elements form a closed subspace L2(G). Y=E[X|G] is the orthogonal projection of X onto L2(G). Its residual satisfies E[(X-Y)Z]=0 for every Z in L2(G). This projection description requires square integrability.
For W in L2(G), orthogonality gives E[(X-W)^2]=E[(X-Y)^2]+E[(Y-W)^2]. Thus Y uniquely minimizes mean squared error up to null sets. With trivial G, predictors are constants. If X is 0 or 8 with equal probability, Y=4. A constant c has error (c^2+(8-c)^2)/2=(c-4)^2+16, minimized at c=4 with value 16.
If integrable X is already G-measurable, X itself satisfies both definition requirements. Hence E[X|G]=X almost surely. With full information G=F, the forecast keeps the observed value. On a finite space with positive outcome probabilities, the full sigma-algebra distinguishes singleton blocks, each with its own X-value.
Matching overall means does not verify conditional expectation. Let a,b be equally likely, with X(a)=0 and X(b)=4, and let G contain all subsets. The constant Y=2 has E[Y]=E[X]=2. But for A={a}, E[1_A Y]=1 and E[1_A X]=0. It fails the defining identity on a known event. Check integrability, G-measurability and the identity on every G-event.
Nesting cannot be dropped from the tower identity. Let X be 0 or 2 with equal probability. Let G be trivial and let H reveal X. Then E[X|G]=1, so E[E[X|G]|H]=1. But E[X|H]=X, which is 0 or 2. Here H is not contained in G, and the identity fails.
A constant conditional mean does not prove independence. Let X take -2,0,2 with probabilities 1/4,1/2,1/4. Let G reveal whether X=0. Both information blocks have mean 0, so E[X|G]=0=E[X]. Yet on A={X=0}, the probability of X=0 becomes 1 instead of 1/2. The conditional distributions differ even though their means agree.
For integrable X, match integrals on every G-event. Nested conditioning obeys the tower property. For square-integrable X, the forecast is the projection onto L2(G).
The lesson, questions, worked solutions and flashcards were written for this dataset.
The reading notes, questions, worked solutions and flashcards were written for this dataset. "Conditional probability and independence" is published by Khan Academy. "Conditional probability with Bayes' Theorem" is published by Art of the Problem.
More to read
Computing a forecast on information blocks
A finite partition describes information that distinguishes blocks without distinguishing outcomes inside a block. A measurable forecast assigns a constant to each block.
For a positive-probability block A, the defining identity gives cP(A)=E[1_A X]. Thus c=E[1_A X]/P(A). This is a conditional mean, so the probabilities inside the block must be normalized.
Suppose two outcomes in A have probabilities 0.1 and 0.3, and values 2 and 6. Their weighted total is 0.1(2)+0.3(6)=2. The block probability is 0.4, so its forecast is 2/0.4=5. The unweighted average 4 ignores the unequal probabilities.
On a partition block of probability zero, division by its probability is unavailable. Any finite constant there preserves the defining integrals. This freedom does not extend to positive-probability blocks.
Checking candidates and versions
For integrable X, a candidate Y must be integrable, G-measurable and satisfy E[1_A Y]=E[1_A X] for every A in G. Equal overall means check only the event Omega. Another known event may expose a failed candidate.
Uniqueness uses such an event. If Y and Z both satisfy the definition, let D=Y-Z and A={D>0}. Measurability puts A in G. Subtracting their identities gives E[1_A D]=0. This nonnegative variable must vanish almost surely, so P(D>0)=0. Interchanging Y and Z gives P(D<0)=0. Hence Y=Z almost surely.
A null set may have many points; probability, rather than cardinality, controls this conclusion. Changing a version on a null set preserves its integrals, but the changed variable must still be integrable and G-measurable. Without completeness of G, an arbitrary null-set modification need not retain measurability.
Projection and repeated conditioning
For square-integrable real variables, L2(F) has inner product E[UV], with variables equal almost surely identified. Its G-measurable elements form the closed subspace L2(G).
If Y=E[X|G], the residual X-Y is orthogonal to every Z in L2(G). For any competing W in that subspace, expanding the square gives E[(X-W)^2]=E[(X-Y)^2]+E[(Y-W)^2]. The last term cannot be negative. Thus Y minimizes mean squared error, and equality requires W=Y almost surely. This projection interpretation needs E[X^2]<infinity; the integral definition needs only E[|X|]<infinity.
For integrable X and H contained in G, the tower property gives E[E[X|G]|H]=E[X|H] almost surely. Information lost in the inner forecast cannot in general be restored by conditioning on an unrelated sigma-algebra. Independence is not required for the nested identity.
Explore more
2 · Watch
Take it off screen
Where it sits
Where this leads
Jobs that lean on this skill. Follow one to see everything it is built on.
18 questions wait behind this lesson, each with its answer explained. Every answer feeds the sky: stars light as they are learned, and dim when it is time to come back.