Explanation Theorem used: Theorem 5: Let P be the transition matrix for an irreducible Markov chain with period d > 1 and q be the steady-state vector for Markov chain. Then, for any initial probability vector X 0 , lim n → ∞ 1 d ( P n + 1 + P n + 2 + ⋅ ⋅ ⋅ ⋅ ⋅ ⋅ ⋅ P n + d ) X 0 = q . Definition used: The steady-state vector: If P is a stochastic matrix, then a steady state vector (or equilibrium vector or invariant probability vector) for P is a probability vector q such that P q = q . Given: The Markov chain is { 1 , 2 , 3 } with the transition matrix P = [ 0 0 1 1 0 0 0 1 0 ] . Calculation: The period of the state 1 is given by GCD ( 3 , 6 , 9 , … ) . However, the state 1 reach to the state 1 in 3 step, 6 step and so on. That is, the period of the state 1 is 3. Similarly the period of the state 2 and 3 is given by GCD ( 3 , 6 , 9 , … ) . Thus, the period of the communication class { 1 , 2 , 3 } is 3. Note that, the period of the Markov chain { 1 , 2 , 3 } is d = 3 (which is d > 1 ). Use the definition state above and obtain the steady-state vector as follows. The given transition matrix is P = [ 0 0 1 1 0 0 0 1 0 ] . From the definition stated above, obtain the steady-state vector q by solve the system P q = q where q = [ q 1 q 2 q 3 ] . It is clear that the system P q = q is equivalent to the homogeneous system ( P − I ) q = 0 . Hence, it is enough to solve the system ( P − I ) q = 0 to obtain the steady-state vector q . The system ( P − I ) can be written as follows. [ 0 0 1 1 0 0 0 1 0 ] − [ 1 0 0 0 1 0 0 0 1 ] = [ − 1 0 1 1 − 1 0 0 1 − 1 ] Thus, the augmented matrix is, [ − 1 0 1 : 0 1 − 1 0 : 0 0 1 − 1 : 0 ] To obtain q , the row-echelon form of the augmented matrix is obtained as follows. [ − 1 0 1 : 0 1 − 1 0 : 0 0 1 − 1 : 0 ] ~ [ 1 0 − 1 : 0 0 1 − 1 : 0 0 0 0 : 0 ] . So the general solution of the steady-state vector is, q = q 3 [ 1 1 1 ] , q 3 is an arbitrary variable with any nonzero value. Since q must be a probability vector, hence the state vectors must sum to 1. The sum of the entries in the vector is 3. Thus, divide the entries of q by 3 with q 3 = 1 and obtain the steady-state vector as follows, q = [ 1 3 1 3 1 3 ] . Thus, the steady-state vector is q = [ 1 3 1 3 1 3 ] . For verification of the Theorem 5, obtain the higher power of the transition matrix P . Calculation of the higher power of the transition matrix as follows. Multiplied the matrix P with itself and obtain P 2 as follows. P 2 = P × P = [ 0 0 1 1 0 0 0 1 0 ] × [ 0 0 1 1 0 0 0 1 0 ] = [ 0 1 0 0 0 1 1 0 0 ] Thus, P 2 = [ 0 1 0 0 0 1 1 0 0 ]

Thomas' Calculus and Linear Algebra and Its Applications Package for the Georgia Institute of Technology, 1/e

5th Edition

ISBN: 9781323132098

Author: Thomas, Lay

Publisher: PEARSON C

See similar textbooks

Videos

Question

Chapter 10.4, Problem 27E

To determine

To verify: The statement of Theorem 5 for the Markov chain in Example 6.

Expert Solution & Answer

Want to see the full answer?

Check out a sample textbook solution

See solution

Chapter 10.4, Problem 26E

Chapter 10.4, Problem 28E

Students have asked these similar questions

Please find the transition matrix for this Markov process

Please do not give solution in image format thanku

Plz don't give solution in image format and give me solutions as soon as possible

Chapter 10 Solutions

Thomas' Calculus and Linear Algebra and Its Applications Package for the Georgia Institute of Technology, 1/e

Ch. 10.1 - Fill in the missing entries in the stochastic...Ch. 10.1 - Prob. 2PP Ch. 10.1 - In Exercises 1 and 2, determine whether P is a...Ch. 10.1 - In Exercises 1 and 2, determine whether P is a...Ch. 10.1 - Prob. 3E Ch. 10.1 - Prob. 4E Ch. 10.1 - In Exercises 5 and 6, the transition matrix P for...Ch. 10.1 - Prob. 6E Ch. 10.1 - In Exercises 7 and 8, the transition matrix P for...Ch. 10.1 - In Exercises 7 and 8, the transition matrix P for...

Knowledge Booster

$Algebra$

Learn more about

Need a deep-dive on the concept behind this application? Look no further. Learn more about this topic, algebra and related others by exploring similar questions and additional content below.

To verify: The statement of Theorem 5 for the Markov chain in Example 6.

Solution Summary: The author explains Theorem 5: Let P be the transition matrix for an irreducible Markov chain with period d>1.

Videos

To verify: The statement of Theorem 5 for the Markov chain in Example 6.

Want to see the full answer?

Chapter 10 Solutions