Consider a Markov chain {Xn : n = 0, 1, . . .} on the nonnegative integers such that starting from state x, the chain goes to state x + 1 with probability 0.3, and goes to state 0 with probability 0.7. (a) Argue that this chain is irreducible. (b) Show that the chain is recurrent. (Hint: Consider the state 0.)

Consider a Markov chain {Xn : n = 0, 1, . . .} on the nonnegative integers such that starting from state x, the chain goes to state x + 1 with probability 0.3, and goes to state 0 with probability 0.7. (a) Argue that this chain is irreducible. (b) Show that the chain is recurrent. (Hint: Consider the state 0.)

MATLAB: An Introduction with Applications

6th Edition

ISBN:9781119256830

Author:Amos Gilat

Publisher:Amos Gilat

Chapter1: Starting With Matlab

Section: Chapter Questions

Problem 1P

See similar textbooks

Similar questions

• General notation for Markov chains: P(A) is the probability of the event A when the Markov chain starts in state x, Pμ(A) the probability when the initial state is random with distribution μ. Ty = min{n ≥ 1: X₂ = y} is the first time after 0 that the chain visits state y. Px,y = Px(Ty < ∞) . Ny is the number of visits to state y after time 0. 4. Let Sn, n20 be a random walk on Z with step distribution 1-p 2 P 1) = 1/², P(X₁ = 1) = 2/₁ 2' 2' P(X₂ = 0) P(X₂ = − 1) = for some 0 < p < 1, p ‡ / . We may denote q=1 - p. That is, the increments (X₂)₁ are i.i.d. and S₂ = X₁ + … + Xn for n ≥ 1 and S = 0. Compute E[S2+1 Sn] for n ≥ 1. (b) Show that Mn = (q/p) Sn defines a martingale (with respect to (Xk)k-1). (c) Does the limit limn→∞ Mn exist almost surely? If yes, give a justification. If your answer is no, explain why. (d) Let T be the first time that S is equal to either −3 or 3. Compute P(ST = 3). Hint: You may use, without proof, the fact P(T<∞) = 1 and that the Optional Stopping Theorem…
Let {Xn, n ≥ 0} be a Markov chain with three states 0, 1, 2 and has the transition probability matrix, P= [ 3/4 1/4 0 1/4 1/2 1/4 0 3/4 1/4 ] The initial distribution is, P {Xn; n = 0, 1, 2} = 1/3 a. What is P {X1 = 1| X0 = 2}?b. What is P {X2 = 2| X1 = 1}?c. What is P {X2 = 2, X1 = 1| X0 = 2}?d. What is P {X2 = 2, X1 = 1, X0 = 2}?e. What is P {X3 = 1, X2 = 2, X1 = 1, X0 = 2}?
Which statements are true? Select one or more: a. Markov’s inequality is only useful if I am interested in that X is larger than its expectation. b. Chebyshev’s inequality gives better bounds than Markov’s inequality. c. Markov’s inequality is easier to use. d. One can prove Chebyshev’s inequality using Markov’s inequality with (X−E(X))2.
(6) Give an example of a Markov chain Xn on a countable state space S and a function g on S such that g(Xn) is not a Markov chain. Can you give any conditions on Xn and g which ensure that g(Xn) is a Markov chain?
a)
9. |Show that if X,, X2,... is a Markov chain, then it is statistically determined by its |second-order probability masses: P{X, = Xị , X, = x}, for any i, j = 1, 2,..
7
Construct the HMM (Hidden Markov Model) for the following sequences
Determine the classes of the Markov chain and whether they are recurrent.
Let X be a Markov chain on S, and let I: S" {0, 1}. Show that the distribution of Xn, Xn+1,..., conditional on {I (X₁,..., Xn) = 1} n {Xn = i}, is identical to the distribution of Xn, Xn+1,... conditional on {Xn = i}.
A state vector X for a four-state Markov chain is such that the system is three times as likely to be in state 4 as in 3, is not in state 2, and is in state 1 with probability 0.2. Find the state vector X.
Let {Xn: n = 0, 1, 2, ...} be a Markov chain with the four states 1,2,3,4. If p44 = 1, what is state 4 called? %3D an absorbing state O a certain state a recurring state O a cyclic state • Previous Simpfun