Markov Chains and the Memoryless Assumption
The next state depends on the current state and on nothing before it. That single assumption turns a process into a matrix, and questions about the far future become questions about matrix powers.
What a learner can do afterwards
- Write the transition matrix for a described process
- Compute an n-step transition probability
- Give a process that is not Markov and say what extra memory would make it one
1 · Read
A Markov chain is a process where tomorrow depends only on today, never on earlier history. That single memoryless assumption turns the process into a matrix. Rows are the current state and columns are the next state, and each row sums to 1 because tomorrow always brings some state. For a Sunny and Cloudy model, the Sunny row might read 0.8 to Sunny then 0.2 to Cloudy, while the Cloudy row reads 0.4 then 0.6.
Start from Sunny with the rows above. The chance of Sunny two days out needs both paths: Sunny then Sunny gives 0.8 times 0.8, which is 0.64, and Sunny then Cloudy then Sunny gives 0.2 times 0.4, which is 0.08. Add the disjoint paths to get 0.72. Tree diagrams draw the same idea: multiply along each branch path, then add across paths that land together.
Matrix powers carry this further: the square of the transition matrix holds every two-step probability, the cube holds every three-step one. From Sunny, the chance of Cloudy two days out is 0.8 times 0.2 plus 0.2 times 0.6, which is 0.16 plus 0.12, giving 0.28. Every n-step question is the multiplication rule in disguise, chaining today into tomorrow and summing over middles.
A rule using the last two days is not Markov, since older history matters. The fix is to enlarge the state so it carries the memory: track pairs of consecutive days as the state. Given a pair like (Sunny, Cloudy), tomorrow depends only on the pair. When memory breaks the rule, a bigger state restores it.
One-step chances form a matrix, multi-step chances come from its powers, and memory needs a bigger state.
2 · Watch
Take it off screen
Where it sits
Learn first
Where this leads
Jobs that lean on this skill. Follow one to see everything it is built on.
8 questions wait behind this lesson, each with its answer explained. Every answer feeds the sky: stars light as they are learned, and dim when it is time to come back.