Markov Convergence Theorem

A standard finite-state Markov convergence theorem states that an irreducible, aperiodic chain with fixed transition probabilities approaches a unique stationary distribution independently of its initial distribution. This is convergence of state probabilities, not necessarily convergence of a realized path to one state.

For a finite, time-homogeneous, irreducible, aperiodic Markov chain, the distribution of the state converges to a unique stationary distribution regardless of the initial distribution. Stationary means that applying the transition rule leaves that probability distribution unchanged. It does not mean an individual chain stops moving or reaches one inevitable state.

The conditions carry the conclusion. Reducible chains can preserve dependence on the starting class; periodic chains can oscillate in distribution; infinite-state chains need additional conditions. Even when the theorem applies, convergence can be slow. Compare transient probabilities over the decision's actual horizon before declaring that initial conditions or early actions do not matter.

When to use it

When a specified finite Markov chain meets the conditions and a long-run probability distribution is relevant to the question.

How it can help

Verify the conditions, compute the stationary distribution, and compare transient behavior over the relevant horizon. Changes to transition rules can alter the limit, while initial conditions can still strongly affect near-term outcomes.