ISEGORIA / MATH ENCYCLOPEDIA
Markov chains: transition, stationarity, and absorption
Iterate transition matrices, locate stationary distributions, and solve a gambler’s-ruin model.
Before you begin: Probability and linear algebra
Predict, manipulate, then check your reasoning against the example and question. Graphs illustrate the mathematics; they do not replace a proof.
1. Transition dynamics
Iterate a two-state transition matrix and watch probability mass move toward equilibrium.
Worked example. The row sums of P stay one, so state probabilities remain normalized.
Watch out. A transition matrix describes a model assumption, not a causal law for every data set.
What does one matrix multiplication represent?
One time step of conditional probability updates.
2. Stationary distribution
Vary the transition probabilities and compare starting distributions with the stationary solution.
Worked example. When both transition probabilities are positive, the chain converges to its stationary distribution.
Watch out. Reducibility or periodicity can prevent convergence from every initial distribution.
What does stationarity mean here?
Applying P leaves the distribution unchanged.
3. Gambler’s ruin
Each round wins one unit with probability α and loses one otherwise. Change α, the starting fortune and the goal, watch sample games end in ruin or success, and compare the share of wins with the exact probability hᵢ of reaching N before 0.
Worked example. For a fair walk, the probability of reaching N before zero is i/N.
Watch out. The finite-state model assumes independent fixed-probability steps.
What happens when the walk is fair?
The absorption probability changes linearly with the starting fortune.