Practice Problems - Markov Chains

Understanding Markov Transitions - from 8.1

On any given night, Justin is either at Kilroy’s on Kirkwood, or at Brothers. If he was at KoK yesterday, there is a 30% chance he will go to KoK again today. If he was at Brother’s yesterday, there’s a 40% chance he will go to Brothers again today. If he was at Brothers yesterday, what’s the probability that he will be at KoK for the next 3 days?

Markov Transition Matrices - from 8.1

A Markov chain has two states. If the chain is in state 1 on a given observation, then it is three times as likely to be in state 1 as to be in state 2 on the next observation. If the chain is in state 2 on a given observation, then it is twice as likely to be in state 1 as to be in state 2 on the next observation. Create the transition matrix that represents this Markov chain.

Initial State Vectors - from 8.2

The initial state has no chance to be in state 2, it’s equally likely to be in state 1 as state 3, and it’s four times as likely to be in state 4 as state 1. Find the initial state vector.

Markov Matrices and Initial State Vectors - from 8.2

Bob take the bus or the subway to work. If he took the subway yesterday, there’s an 80% chance he’ll take the subway again. If he took the bus, there’s a 40% chance he’ll take the bus again. If he is four times as likely to take the subway as taking the bus today, what’s the probability that he takes the subway tomorrow?

Markov Matrices with Multiple Transitions - from 8.2

A Markov chain has the transition matrix P below. If the chain is in state 1 on the 2nd observation, what is the probability that it will be state 2 on the 4th observation?

transition matrix

Regular Markov Matrices - from 8.3

Which of the following transition matrices are regular?

transitions matrices

Long-run State Vector - from 8.3

Bob takes the bus or the subway to work. If he took the subway yesterday, there’s an 80% chance he’ll take the subway again. If he took the bus, there’s a 40% chance he’ll take the bus again. What’s the long run probability that he takes the subway?

Advanced Long Run State Vector - from 8.3

A Markov transition matrix is given as follow:

transition matrix

The long run stable vector is as follows:

stable vector

Solve for a.