A stationary distribution of a Markov chain is a probability distribution that remains unchanged in the Markov chain as time progresses. That is, if the initial position of the chain is given by this distribution, then the position of the chain at any future time has the same distribution.

Typically, it is represented as a row vector \(\pi\) whose entries are probabilities summing to \(1.\) If \(\textbf{P}\) is the transition matrix \(\textbf{P},\) for the chain, then \(\pi\) is a left eigenvector of \(\textbf{P},\) i.e. \[\pi = \pi \textbf{P}.\]

Unfortunately, such a stationary distribution may not always exist. For example, if a Markov chain is periodic, then a stationary distribution may not exist. In essence, a periodic Markov chain is one that can only visit certain states in fixed intervals, for example, always after an even number of steps. The chains for which such a distribution does exist are said to be ergodic.

Which states, if any, are periodic in the Markov chain pictured?

What is the stationary distribution of the Markov chain pictured below?

×

Problem Loading...

Note Loading...

Set Loading...