5. Consider a Markov chain (Xn) with the state space S = {0, 1, 2,...} of non-negative integersand the one-step transition probabilities given by p(i, i + 1) = p and p(i, 0) = 1 - p for i ≥ 0,where p € (0, 1).(a) Identify the closed irreducible sets.(b) Is the Markov chain aperiodic?(c) Is there a stationary distribution? Is it unique? Compute the stationary distribution ifit exists.(d) Find the transient states and recurrent states.(e) Compute E; [Ti] = E[Ti|Xo = i] for each i≥ 0, where T; is the (usual) first return time.

The considered Markov chain has the state space of non-negative integers and the -step transition…

Answered: 5. Consider a Markov chain (Xn) with…

MATLAB: An Introduction with Applications

6th Edition

ISBN:9781119256830

Author:Amos Gilat

Publisher:Amos Gilat

Chapter1: Starting With Matlab

Section: Chapter Questions

Problem 1P

See similar textbooks

Similar questions

Consider the Markov chain with state space S = {0, 1, 2,...} and transition probabili- ties Poo = 1 - P. P02 = P, and Pr,x+2= P₁ P2,2-1=1-p, for x = 1,2,... For which values of p is this a transient chain?
For the attached transition probability matrix for a Markov chain with {Xn ; n = 0, 1, 2,.........}: a) How many classes exist, and which two states are the absorption states? b) What is the limn->inf P{Xn = 3 | X0 = 3}? b) What is the limn->inf P{Xn = 1 | X0 = 3}?
• General notation for Markov chains: P(A) is the probability of the event A when the Markov chain starts in state x, Pμ(A) the probability when the initial state is random with distribution μ. Ty = min{n ≥ 1: X₂ = y} is the first time after 0 that the chain visits state y. Px,y = Px(Ty < ∞) . Ny is the number of visits to state y after time 0. 4. Let Sn, n20 be a random walk on Z with step distribution 1-p 2 P 1) = 1/², P(X₁ = 1) = 2/₁ 2' 2' P(X₂ = 0) P(X₂ = − 1) = for some 0 < p < 1, p ‡ / . We may denote q=1 - p. That is, the increments (X₂)₁ are i.i.d. and S₂ = X₁ + … + Xn for n ≥ 1 and S = 0. Compute E[S2+1 Sn] for n ≥ 1. (b) Show that Mn = (q/p) Sn defines a martingale (with respect to (Xk)k-1). (c) Does the limit limn→∞ Mn exist almost surely? If yes, give a justification. If your answer is no, explain why. (d) Let T be the first time that S is equal to either −3 or 3. Compute P(ST = 3). Hint: You may use, without proof, the fact P(T<∞) = 1 and that the Optional Stopping Theorem…
The sequence (Xn)n>o is a Markov chain with transition matrix 0 을 0 0 0 3 1 0 0 0 4 4 0 } 0 0 0 1 2 1 1 0 0 3 3 0 } 0 } 0 0 0 0 0 0 0 (a) Draw the transition diagram for (Xn). (b) Determine the communicating classes of the chain, stating whether each class is recurrent or transient. (c) Determine the period of each communicating class.
Consider a continuous-time Markov chain whose jump chain is a random walk with reflecting barriers 0 and m where po,1 = 1 and pm,m-1 =1 and pii-1 = Pii+1 = for 1
(6) Give an example of a Markov chain Xn on a countable state space S and a function g on S such that g(Xn) is not a Markov chain. Can you give any conditions on Xn and g which ensure that g(Xn) is a Markov chain?
Recall that the graph of a discrete-time, finite-state, homogeneous Markov chain on two states 1, 2 with transition matrix is CO (: :) 1 T = If instead a new Markov chain on three states {1, 2, 3} has transition matrix 10 1 0 (a) Draw the corresponding graph for this new Markov chain. (b) Is the Markov chain with transition matrix T irreducible?
1. For each of the transition matrices given below, list absorbing state(s), and determine the period of each of the states in each Markov chain. (a) P = (d) P = 1120 CLI2L 1434L3 0 ܚ . ܟ ܝ. | ܟ ܝܕ . ܚ 0 00 0 0 0 0 0 3I4LI4 LIABI+ 0 0 (b) P = 1 0 Lo (e) P = 0 1 1 0 0 1333 COLITOLIN 0 0 0 0 0 213 100 0 0 3 00 100 0 -0 WIN 1 0 0 0 0 0 0 00 1 0 0 4|012 TWO O 0 0 0 0 0 (c) P = (f) P = 0 1|2 1|3 OLT20IN +15 O N|TN|TO 120 HINO OLI3 O 0 LI5L4L 1113 2I5
2. Consider a Markov chain {Xn}n≥o having the following transition diagram: 1/2 1 2 5 1/4 3 4 1/2 1/4 6 7 1/2 For this chain, there are two recurrent classes R₁ = {6,7} and R₂ = {1,2,5}, and one transient class R3 = {3,4}. (a) Find the period of state 3. (b) Find f33 and f22. (c) Starting at state 3, find the probability that the chain is absorbed into R₁. (d) Starting at state 3, find the mean absorbation time, i.e., the expected number of steps that the chain is absorbed into R₁ or R₂. Note: there are missing transition probabilities for this chain, but no impact for your solution.
12. Let Z be an irreducible discrete-time Markov chain on a countably infinite state space S, having transition matrix H = (hij) satisfying hii = 0 for all states i, and with stationary distribution v. Construct a continuous-time process X on S for which Z is the imbedded chain, such that X has no stationary distribution.
c) Consider the transition Markov chain with three states S = (1, 2, 3) as shown below. %3D 1 Given that; P(X, = 1) = 3' fine P(X, = 1,X, = 2,X2 = 3) N/3 1/2 1/2 1/4 1/2 1/4
Consider the mouse in the maze shown to the right that includes "one-way" doors. Show that q (also given to the right), is a steady state vector for the associated Markov chain, and interpret this result in terms of the mouse's travels through the maze. To show that q is a steady-state vector for the Markov chain, first find the transition matrix. The transition matrix is P= 0. (Type integers or simplified fractions for any values in the matrix.) 4 2 3 5 6 q= 0

SEE MORE QUESTIONS

Recommended textbooks for you

MATLAB: An Introduction with Applications

Statistics

ISBN:

9781119256830

Author:

Amos Gilat

Publisher:

John Wiley & Sons Inc

Probability and Statistics for Engineering and th…

Statistics

ISBN:

9781305251809

Author:

Jay L. Devore

Publisher:

Cengage Learning

Statistics for The Behavioral Sciences (MindTap C…

Statistics

ISBN:

9781305504912

Author:

Frederick J Gravetter, Larry B. Wallnau

Publisher:

Cengage Learning

MATLAB: An Introduction with Applications

Statistics

ISBN:

9781119256830

Author:

Amos Gilat

Publisher:

John Wiley & Sons Inc

Probability and Statistics for Engineering and th…

Statistics

ISBN:

9781305251809

Author:

Jay L. Devore

Publisher:

Cengage Learning

Statistics for The Behavioral Sciences (MindTap C…

Statistics

ISBN:

9781305504912

Author:

Frederick J Gravetter, Larry B. Wallnau

Publisher:

Cengage Learning

Elementary Statistics: Picturing the World (7th E…

Statistics

ISBN:

9780134683416

Author:

Ron Larson, Betsy Farber

Publisher:

PEARSON

The Basic Practice of Statistics

Statistics

ISBN:

9781319042578

Author:

David S. Moore, William I. Notz, Michael A. Fligner

Publisher:

W. H. Freeman

Introduction to the Practice of Statistics

Statistics

ISBN:

9781319013387

Author:

David S. Moore, George P. McCabe, Bruce A. Craig

Publisher:

W. H. Freeman

SEE MORE TEXTBOOKS