Markov Chains
Which states, if any, are periodic in the Markov chain pictured?
The only states of weather are sun and rain. If it's sunny today, there is an chance it will be sunny tomorrow, and if it's rainy today, there is a chance it's rainy tomorrow. In the long run, on what fraction of days will it be sunny?
What is the stationary distribution of the Markov chain pictured below?
In a particular (very large) deck of cards, the suit of each card is dependent upon the suit of the previous card. If the dependencies are given by the Markov chain pictured, determine the stationary distribution of suits using the order (Spades, Hearts, Diamonds, Clubs).
A player wins a game of tennis by being the first to win games more than the other player, provided they have won at least games. If the game is tied at to and the probabilities are given by the Markov chain pictured, determine the probability that player A wins.
