Explanation Definition used: The steady-state vector: If P is a stochastic matrix, then a steady state vector (or equilibrium vector or invariant probability vector) for P is a probability vector q such that P q = q . Given: There are 6 rooms for the mouse to go and each room has some doors to exit the rooms. Calculation: Start from room 1, there are 2 doors, so the walk can be made towards other rooms that are 2 and 4. Thus, the probability for the walk from room 1 toward rooms 2 and 4 is 1 2 each. From room 2, there are 3 doors, so the walk can be made towards other rooms that are 1, 3 and 5. Thus, the probability for the walk from room 2 toward rooms 1, 3 and 5 is 1 3 each. From room 3, there are 2 doors, so the walk can be made towards other rooms that are 2 and 6. Thus, the probability for the walk from room 3 toward rooms 2 and 6 is 1 2 each. From room 4, there only one door, so the walk can be made towards only one room that is 5. Thus, the probability for the walk from room 4 towards 5 is 1. From room 5, there are 3 doors, so the walk can be made towards other rooms that are 2, 4 and 6. Thus, the probability for the walk from room 5 towards 2, 4 and 6 is 1 3 each. For room 6, there is no opening to exit; hence the probability for mouse to stay in room 6 is 1. Based upon above computation, obtain the transition matrix for the simple random walk as follows. [ 0 1 3 0 0 0 0 1 2 0 1 2 0 1 3 0 0 1 3 0 0 0 0 1 2 0 0 0 1 3 0 0 1 3 0 1 0 0 0 0 1 2 0 1 3 1 ] From the definition stated above, it is observed that, to obtain the steady-state vector q , solve the system P q = q , where q = [ q 1 q 2 q 3 q 4 q 5 q 6 ] . It is clear that the system P q = q is equivalent to the homogeneous system ( P − I ) q = 0 . Hence, it is enough to solve the system ( P − I ) q = 0 to obtain the steady-state vector q . The system ( P − I ) can be written as, [ 0 1 3 0 0 0 0 1 2 0 1 2 0 1 3 0 0 1 3 0 0 0 0 1 2 0 0 0 1 3 0 0 1 3 0 1 0 0 0 0 1 2 0 1 3 1 ] − [ 1 0 0 0 0 0 0 1 0 0 0 0 0 0 1 0 0 0 0 0 0 1 0 0 0 0 0 0 1 0 0 0 0 0 0 1 ] = [ − 1 1 3 0 0 0 0 1 2 − 1

Thomas' Calculus and Linear Algebra and Its Applications Package for the Georgia Institute of Technology, 1/e

5th Edition

ISBN: 9781323132098

Author: Thomas, Lay

Publisher: PEARSON C

See similar textbooks

Question

Chapter 10.2, Problem 19E

To determine

To show: The matrix q=[000001] is the steady-state vector for the associated Markov chain and interpret this result in terms of the mouse`s travels through the maze.

Expert Solution & Answer

Want to see the full answer?

Check out a sample textbook solution

See solution

Chapter 10.2, Problem 18E

Chapter 10.2, Problem 20E

Students have asked these similar questions

Can someone please help me with this question. I am having so much trouble.

Construct the Markov diagram for this game and show that the associate transition matrix T can be written as the following. Show that λ = 1 is an eigenvalue of T. If can answer both (a)and(b), else answer (b).

A state vectorX for a three-state Markov chain is such that the system is as likely to be in state 3 as in state 2 and is three times as likely to be in state 1 as in 3. Find the state vector X.

Chapter 10 Solutions

Thomas' Calculus and Linear Algebra and Its Applications Package for the Georgia Institute of Technology, 1/e

Ch. 10.1 - Fill in the missing entries in the stochastic...Ch. 10.1 - Prob. 2PP Ch. 10.1 - In Exercises 1 and 2, determine whether P is a...Ch. 10.1 - In Exercises 1 and 2, determine whether P is a...Ch. 10.1 - Prob. 3E Ch. 10.1 - Prob. 4E Ch. 10.1 - In Exercises 5 and 6, the transition matrix P for...Ch. 10.1 - Prob. 6E Ch. 10.1 - In Exercises 7 and 8, the transition matrix P for...Ch. 10.1 - In Exercises 7 and 8, the transition matrix P for...

Knowledge Booster

The matrix q = [ 0 0 0 0 0 1 ] is the steady-state vector for the associated Markov chain and interpret this result in terms of the mouse`s travels through the maze.

Solution Summary: The author explains the steady-state vector for the associated Markov chain and interprets it in terms of the mouse's travels through the maze.

To show: The matrix q=[000001] is the steady-state vector for the associated Markov chain and interpret this result in terms of the mouse`s travels through the maze.

Want to see the full answer?

Chapter 10 Solutions