--- title: "Markov Chains and the Memoryless Assumption" description: "Given the current state, earlier states add no information about the next state. For finitely many states, transitions form a matrix. When transition probabilities stay fixed over time, matrix powers " canonical: https://lightmysky.com/learn/mathematics/markov-chains-and-the-memoryless-assumption-mt_2wDa_JxJ_o source: https://lightmysky.com/learn/mathematics/markov-chains-and-the-memoryless-assumption-mt_2wDa_JxJ_o.md retrieved: 2026-11-12 --- > **Agent view.** This is the Markdown twin of the page, for tools and assistants. > When to use this site, and the call that answers each job: https://lightmysky.com/agent-instructions.md > API description (OpenAPI 3.1): https://lightmysky.com/openapi.json · Authentication: https://lightmysky.com/auth.md > Pricing: https://lightmysky.com/pricing.md · Catalog: https://lightmysky.com/llms.txt · Full catalog: https://lightmysky.com/llms-full.txt > Every machine-readable file on this domain: https://lightmysky.com/.well-known/ai-catalog.json > Ask for Markdown with `Accept: text/markdown`, a `.md` address, or `?mode=agent`. # Markov Chains and the Memoryless Assumption Given the current state, earlier states add no information about the next state. For finitely many states, transitions form a matrix. When transition probabilities stay fixed over time, matrix powers give multi-step probabilities. Subject: Mathematics · Area: Probability · Ages 22 to 24 Page: https://lightmysky.com/learn/mathematics/markov-chains-and-the-memoryless-assumption-mt_2wDa_JxJ_o ## Ready when they can - Write a finite-state transition matrix using a stated row and column order - Compute an n-step transition probability, distinguishing fixed transitions from time-dependent transitions - Give a process whose next-state law depends on older history and explain how a sufficient finite-memory state makes it Markov ## Lesson: Tomorrow depends only on today You have conditioned forecasts on known information. A Markov chain says that, given the current state, earlier states add no information about the next-state probabilities. Equivalently, the conditional mean of each next-state indicator given the full past equals its conditional mean given the current state. Successive states can still be dependent. We use finitely many states here. Fix the same state order for rows and columns. In a transition matrix P, row i, column j gives the chance of moving from state i to state j. Entries are nonnegative. Each row sums to 1 because the next states are exhaustive and mutually exclusive. In order (Sunny, Cloudy), consider rows (0.7, 0.3) and (0.2, 0.8). From Cloudy, the next state is Sunny with probability 0.2. *(drawing: Read each row in the destination order Sunny, Cloudy.)* **Example.** Use these weather transitions at both steps and start in Sunny. To be Sunny two steps later, the middle state can be Sunny or Cloudy. The path Sunny, Sunny, Sunny has probability 0.7 × 0.7 = 0.49. The path Sunny, Cloudy, Sunny has probability 0.3 × 0.2 = 0.06. These paths are disjoint, so add them: 0.49 + 0.06 = 0.55. For a time-homogeneous chain, one matrix P applies at every step. P^n is the product of n copies of P. Its (i,j) entry gives the n-step chance from i to j. Compute P^2(i,j) by adding P(i,k) × P(k,j) over middle states k. For one specified path, multiply its transitions without adding alternatives. To update a state distribution, weight each transition to the target by its current-state probability and add. Repeat for further steps. A machine may use one transition table in daytime and another at night. This can be Markov if, at each time, earlier states add no information once the current state is known. If consecutive tables are P then Q, the two-step table is PQ, not generally P^2. Its (i,j) entry adds P(i,k) × Q(k,j) over middle states k. To disprove the Markov property, find two possible histories with the same current state but different next-state probabilities. If a rule depends only on the last two states, store the pair (a,b), with a older and b current. Its next pair is (b,c), where c is the new state. This pair process is Markov. Mentioning two days does not prove failure: the older day must change the conditional probabilities. For a reflecting random walk on consecutive integer states, specify the end rule. Here an end sends the walker to its only neighbour with probability 1; an interior state chooses either neighbour with probability 0.5. Some models allow a stay at an end, so 'reflecting' alone is not enough. For a two-step chance, list possible intermediate states and destinations. If no path reaches the target, its probability is 0. **Recap.** Given the present, older states add no next-state information. Fixed transition matrices give powers; changing matrices give ordered products. Store sufficient memory in the state. *The lesson, practice questions and flashcards were written for this dataset.* ## Practice 13 questions on this page, each with its working shown. ## Needs first - [Conditional Expectation Given a Sigma-Algebra](https://lightmysky.com/learn/mathematics/conditional-expectation-given-a-sigma-algebra-mt_i2sxZc-ER8) ## Optional connections - [Weak Solutions and Test Functions](https://lightmysky.com/learn/mathematics/weak-solutions-and-test-functions-mt_AKlC3Ol70p) - [Conditional Probability, Independence and Bayes' Theorem](https://lightmysky.com/learn/mathematics/conditional-probability-independence-and-bayes-theorem-mt_wIi_XRzwwK) - [Separation of Variables and the Heat Equation](https://lightmysky.com/learn/mathematics/separation-of-variables-and-the-heat-equation-mt_XnPDL0X1TL) ## Opens up - [Markov Decision Processes: States, Actions and Return](https://lightmysky.com/learn/computing/markov-decision-processes-states-actions-and-return-mt_bTCO27sgc6) - [Stationary Distributions and Long-Run Behaviour](https://lightmysky.com/learn/mathematics/stationary-distributions-and-long-run-behaviour-mt_MGfHvUGkeh)