Stationary Distributions and Long-Run Behaviour

Markov updates weight and add, as expected value does. Multiply each transition to a target by its current-state probability and add: a row distribution u becomes uP. Use finite chains with fixed P and one order for rows (current) and columns (next). A stationary row vector pi has nonnegative entries totaling 1 and satisfies pi P = pi. It is a left eigenvector for eigenvalue 1. Transposing gives P^T pi^T = pi^T, where T means transpose. Check all three conditions.

A current row distribution u becomes uP after one step. For destination j, add u_i × P_ij over current states i. Repeat for further steps. With order (0, 1) and rows (0.6, 0.4), (0.1, 0.9), starting at 0 gives (0.6, 0.4) after one step. After two steps, the state 1 probability is 0.6 × 0.4 + 0.4 × 0.9 = 0.60.

A chain is irreducible when every state can reach every other along a path of positive-probability transitions. The path may take several steps; a zero one-step entry does not prove reducibility. A state that stays at itself with probability 1 is absorbing. In a chain with other states, it cannot reach them, so the whole chain is reducible.

A state's period is the greatest common divisor of its possible positive return times. Period 1 means aperiodic. A self-loop makes time 1 possible, so that state has period 1. All states of an irreducible chain share a period; one self-loop therefore makes that chain aperiodic. An absorbing state has period 1. Without self-loops, inspect longer return paths: possible times 2 and 3 already give greatest common divisor 1.

The deterministic flip moves 0 to 1 and 1 to 0. It is irreducible, but returns occur at times 2, 4, 6 and so on, giving period 2. Its stationary distribution is (0.5, 0.5): exchanging equal weights leaves them unchanged. Starting at state 0 instead gives alternating distributions, so its matrix powers do not converge. Any stationary starting distribution stays fixed, even in a periodic chain.

Every finite chain has a stationary distribution. Irreducibility makes it unique. If the chain is also aperiodic, uP^n approaches pi for every starting distribution u; every row of P^n approaches pi. Irreducibility and aperiodicity together are sufficient, not necessary, for a common limit. Rows (1, 0), (0.5, 0.5) give state 1 weight b × 0.5^n from starting weight b, so every start approaches (1, 0).

Stationary distributions need not be unique. The identity transition matrix leaves every (a, 1 - a), with 0 <= a <= 1, unchanged. To turn a nonzero, nonnegative stationary row vector v into a probability distribution, divide every entry by the sum of its entries. If vP = v, this scaling preserves the equation and makes the total 1.

For two states, let r = P_01 and s = P_10. Stationarity gives pi_0 × r = pi_1 × s. Derive this by cancelling the stay-in-state-0 term from pi_0 = pi_0(1 - r) + pi_1s. If pi_0 is positive, an unknown r equals pi_1 × s divided by pi_0. Check that the result lies between 0 and 1, then check the completed matrix and stationary update.

In order (L, M, R), let L and R be absorbing and M's row be (r, t, s), with t < 1. From M, the limiting R probability h satisfies h = s + th. Solve: h = s/(1 - t).