Markov chain with transient states is an absorbing one. An absorbing Markov chain is a Markov chain in which it is impossible to leave some states, and any state could (after some number of steps, with positive probability) reach such a state. It follows that all non-absorbing states in an absorbing Markov chain are transient.A common type of
An absorbing state is a state in a Markov chain such that . Note that it is not sufficient for a Markov chain to contain an absorbing state (or even several!) in order for it to be an absorbing Markov chain. It must also have all other states eventually reach an absorbing state with probability .
If the chain has transient states and absorbing states, the transition matrix for a time-homogeneous absorbing Markov chain may then be written as where is a matrix, is a matrix, is the zero matrix, and is the identity matrix.
A simple example of an absorbing Markov chain is the drunkard's walk of length . In the drunkard's walk, the drunkard is at one of intersections between their house and the pub. The drunkard wants to go home, but if they ever reach the pub (or the house), they will stay there forever. However, at each intersection along the way, there is a probability , typically , that the drunkard will become confused and, losing their sense of direction, return to the previous intersection.
Taking Pub to be the state and Home to be the state, the transition matrix is It follows that
Many calculations for Markov chains are made simple by that decomposition of the transition matrix. In particular, the fundamental matrix contains a lot of information. Note so
It follows from the linearity of expectation that, starting from state , the expected number of steps before entering an absorbing state is given by the entry of the column vector .
Furthermore, the probability of being absorbed by state after starting in state is given by the entry of the matrix ,
What is the probability the drunkard, starting at intersection or , makes it home in the above example?
Note Then, So the drunkard has probability of making it home from intersection and probability of making it home from intersection .