2.) Recall that the simple random walk is just a Markov chain on the integers where we movefrom j to j+1 with probability, and from j to j-1 with probability: .Find1P(Xn+1 =j+1| Xn = j) =21P(Xn+1 =j-1 | Xn = j) = ²/P(Xn+1 = m | Xn = j) = 0 ifm #j+1 or j - 1.(a) P(Xn+1 = 2 | X, = 0) (i.e. the probability of going from 0 to 2 in one-step)(b) P(X+1 = 2 | X,,= 1) (i.e. the probability of going from 1 to 2 in one-step)(c) P(Xn+2= 2 | X, = 2) (i.e. the probability of going from 2 to 2 in two-steps)

Answered: Image /qna-images/answer/a04aec4c-362d-4b0a-b3f1-e327bcc442d3.jpg

Answered: ) Recall that the simple random walk is…

Advanced Engineering Mathematics

10th Edition

ISBN:9780470458365

Author:Erwin Kreyszig

Publisher:Erwin Kreyszig

Chapter2: Second-order Linear Odes

Section: Chapter Questions

Problem 1RQ

See similar textbooks

Similar questions

Let X Exponential (3) a) Find the Markov upper bound for P(X>10). b) Find the Chebyshev upper bound for P(X>10). State which of the two bounds found is better. c) Find the Chernoff upper bound for P(X>10). Clearly state the allowable range for t. d) Let Zn be a RV from a Gamma(n, 3) distribution for n=1,2,... . What does converge in probability to? Please state both the value and the reasoning by which Z converges in probability to it. n
Q3: The probability parameters of a homogeneous Markov chain are as follows: 0.8 0.8 0.8 C = 0.2 0.1 0.1 0.1 1. 0.1 If the chain is in the third state after 10 steps of the chain, evaluate the probability that it was also in the third state after 5 steps.
Consider three r.v.'s X1, X2, X3 with finite sample space X; E {1,..., K}. The sequence X¡, i = 1, 2, 3, is called a Markov chain if Pr (X3 = i| X2 = j, X1 = k) = Pr (X3 = i | X2 = j) = Pr (X2 = i| X1 = j) . That is, Pr (X3 = i| X2 = j) = Pi does not depend on X1, and it is the same as Pr (X2 = i| X1 = j). We call Pji the transition probabilities. Let P = [Pi] denote the (K × K) matrix of transition probabilties. Let T = (T1,..., TK)' be a probability vector (i.e., T; > 0 and Tk = 1) with Tk Pkj = T;Pjk %3D for any pair of states j and k. If q1 = T, show that q2 = q3 = r as well (T is called an equilibrium distribution"). By the law of total probability φΣ, Τ P- Σ, ΤP πι Σ, P= πι By Bayes' theorem q2i = 92i = Ti %3D q1 is a probability vector and P is a stochastic matrix = q2 = q3 = 91. By definition of conditional probability Pj Pr(X1=k,X2=j) Pr(X1=k) = Tj. %3D %3D none of these
Consider a sequece of iid random variables {§n, n= 0, 1, 2,...} with mass probabilities P(§n = 0) = 0.1, P(§n = 1) = 0.5, and P(§n = 2) = 0.4. Define a Markov chain (X₂)n≥o on the state space S = {0, 1, 2} using the rule Xn = |Xn−1 − Én |• Write the transition matrix for this Markov chain: P = =
Suppose I flip n independent biased coins such that the jth coin has probability j/n of being heads, for j = 1,...,n. Let Hn be a random variable equal to the total number of heads. (a) What is the expectation of H,? (b) What is the variance of Hn? (c) Use Markov's inequality to derive an upper bound on P(Hn 2 9n/10).
Consider a Markov chain in the set {1, 2, 3} with transition probabilities p12 = p23 = p31 = p, p13 = p32 = p21 = q = 1 − p, where 0 < p < 1. Determine whether the Markov chain is reversible.
Consider a (non)-symmetric random walk M = Xo +X1+X2+…+Xn where X;'s (i> 1) are independent and identical distributed, P(X; = 1) = p and P(X;=-1) =1–p and Xo = 0. Let Sn = eaS,-n Denote Fn= 0(X1,...,Xn). Find a such that (Sn)n20 is a (Fn)n20 - martingale.
• General notation for Markov chains: P(A) is the probability of the event A when the Markov chain starts in state x, Pμ(A) the probability when the initial state is random with distribution μ. Ty = min{n ≥ 1: X₂ = y} is the first time after 0 that the chain visits state y. Px,y = Px(Ty < ∞) . Ny is the number of visits to state y after time 0. 4. Let Sn, n20 be a random walk on Z with step distribution 1-p 2 P 1) = 1/², P(X₁ = 1) = 2/₁ 2' 2' P(X₂ = 0) P(X₂ = − 1) = for some 0 < p < 1, p ‡ / . We may denote q=1 - p. That is, the increments (X₂)₁ are i.i.d. and S₂ = X₁ + … + Xn for n ≥ 1 and S = 0. Compute E[S2+1 Sn] for n ≥ 1. (b) Show that Mn = (q/p) Sn defines a martingale (with respect to (Xk)k-1). (c) Does the limit limn→∞ Mn exist almost surely? If yes, give a justification. If your answer is no, explain why. (d) Let T be the first time that S is equal to either −3 or 3. Compute P(ST = 3). Hint: You may use, without proof, the fact P(T<∞) = 1 and that the Optional Stopping Theorem…
Suppose qo = [1/4, 3/4] is the initial state distribution for a Markov process with the following transition matrix: %3D [1/2 1/2] M = [3/4 1/4] (a) Find q1, 92, and q3. (b) Find the vector v that qo M" approaches. (c) Find the matrix that M" approaches.
Which statements are true? Select one or more: a. Markov’s inequality is only useful if I am interested in that X is larger than its expectation. b. Chebyshev’s inequality gives better bounds than Markov’s inequality. c. Markov’s inequality is easier to use. d. One can prove Chebyshev’s inequality using Markov’s inequality with (X−E(X))2.
(6) Give an example of a Markov chain Xn on a countable state space S and a function g on S such that g(Xn) is not a Markov chain. Can you give any conditions on Xn and g which ensure that g(Xn) is a Markov chain?
Let X be a Markov chain on S, and let I: S" {0, 1}. Show that the distribution of Xn, Xn+1,..., conditional on {I (X₁,..., Xn) = 1} n {Xn = i}, is identical to the distribution of Xn, Xn+1,... conditional on {Xn = i}.

SEE MORE QUESTIONS

Recommended textbooks for you

Advanced Engineering Mathematics

Advanced Math

ISBN:

9780470458365

Author:

Erwin Kreyszig

Publisher:

Wiley, John & Sons, Incorporated

Numerical Methods for Engineers

Advanced Math

ISBN:

9780073397924

Author:

Steven C. Chapra Dr., Raymond P. Canale

Publisher:

McGraw-Hill Education

Introductory Mathematics for Engineering Applicat…

Advanced Math

ISBN:

9781118141809

Author:

Nathan Klingbeil

Publisher:

WILEY

Advanced Engineering Mathematics

Advanced Math

ISBN:

9780470458365

Author:

Erwin Kreyszig

Publisher:

Wiley, John & Sons, Incorporated

Numerical Methods for Engineers

Advanced Math

ISBN:

9780073397924

Author:

Steven C. Chapra Dr., Raymond P. Canale

Publisher:

McGraw-Hill Education

Introductory Mathematics for Engineering Applicat…

Advanced Math

ISBN:

9781118141809

Author:

Nathan Klingbeil

Publisher:

WILEY

Mathematics For Machine Technology

Advanced Math

ISBN:

9781337798310

Author:

Peterson, John.

Publisher:

Cengage Learning,

Basic Technical Mathematics

Advanced Math

ISBN:

9780134437705

Author:

Washington

Publisher:

PEARSON

Topology

Advanced Math

ISBN:

9780134689517

Author:

Munkres, James R.

Publisher:

Pearson,

SEE MORE TEXTBOOKS