General notation for Markov chains: Pr(A) is the probability of the event A when the Markov chain starts in state x, Pμ(A) the probability when the initial state is random with distribution μ. Ty = min{n ≥ 1: Xn = y} is the first time after 0 that the chain visits state y. Pxy = P(Ty <∞). Ny is the number of visits to state y after time 0. 4. Let Sn, n20 be a random walk on Z with step distribution 1-p P(X; = 0) = ¹, P(X₁ = 1) = 2/₁ P(X;= -1) = ¹ 2 P for some 0 < p < 1, p‡½. We may denote q = 1 - p. That is, the increments (X₂)1 are i.i.d. and Sn = X₁ + ... + Xn for n ≥ 1 and So = 0. (a) Compute E[S2+1 Sn] for n ≥ 1. (b) Show that M₁ = (q/p) Sn defines a martingale (with respect to (X)-1).
General notation for Markov chains: Pr(A) is the probability of the event A when the Markov chain starts in state x, Pμ(A) the probability when the initial state is random with distribution μ. Ty = min{n ≥ 1: Xn = y} is the first time after 0 that the chain visits state y. Pxy = P(Ty <∞). Ny is the number of visits to state y after time 0. 4. Let Sn, n20 be a random walk on Z with step distribution 1-p P(X; = 0) = ¹, P(X₁ = 1) = 2/₁ P(X;= -1) = ¹ 2 P for some 0 < p < 1, p‡½. We may denote q = 1 - p. That is, the increments (X₂)1 are i.i.d. and Sn = X₁ + ... + Xn for n ≥ 1 and So = 0. (a) Compute E[S2+1 Sn] for n ≥ 1. (b) Show that M₁ = (q/p) Sn defines a martingale (with respect to (X)-1).
MATLAB: An Introduction with Applications
6th Edition
ISBN:9781119256830
Author:Amos Gilat
Publisher:Amos Gilat
Chapter1: Starting With Matlab
Section: Chapter Questions
Problem 1P
Related questions
Question
![• General notation for Markov chains: P(A) is the probability of the event A when the Markov
chain starts in state x, Pμ(A) the probability when the initial state is random with distribution μ.
Ty = min{n ≥ 1: X₂ = y} is the first time after 0 that the chain visits state y. Px,y = Px(Ty < ∞) . Ny
is the number of visits to state y after time 0.
4.
Let Sn, n20 be a random walk on Z with step distribution
1-p
2
P
1) = 1/², P(X₁ = 1) = 2/₁
2'
2'
P(X₂ = 0)
P(X₂ = − 1)
=
for some 0 < p < 1, p ‡ / . We may denote
q=1 - p.
That is, the increments (X₂)₁ are i.i.d. and S₂ = X₁ + … + Xn for n ≥ 1 and S = 0.
Compute E[S2+1 Sn] for n ≥ 1.
(b) Show that Mn = (q/p) Sn defines a martingale (with respect to (Xk)k-1).
(c) Does the limit limn→∞ Mn exist almost surely? If yes, give a justification. If your answer is no,
explain why.
(d) Let T be the first time that S is equal to either −3 or 3. Compute P(ST = 3).
Hint: You may use, without proof, the fact P(T<∞) = 1 and that the Optional Stopping Theorem
(OST) applies to the martingale (Mn) and the stopping time T.](/v2/_next/image?url=https%3A%2F%2Fcontent.bartleby.com%2Fqna-images%2Fquestion%2Fc0eba744-2ca5-4b9b-b356-631e195ec79e%2Fd53fd11c-37b9-4e31-9fb7-f9e4c729250b%2F7xw2689_processed.png&w=3840&q=75)
Transcribed Image Text:• General notation for Markov chains: P(A) is the probability of the event A when the Markov
chain starts in state x, Pμ(A) the probability when the initial state is random with distribution μ.
Ty = min{n ≥ 1: X₂ = y} is the first time after 0 that the chain visits state y. Px,y = Px(Ty < ∞) . Ny
is the number of visits to state y after time 0.
4.
Let Sn, n20 be a random walk on Z with step distribution
1-p
2
P
1) = 1/², P(X₁ = 1) = 2/₁
2'
2'
P(X₂ = 0)
P(X₂ = − 1)
=
for some 0 < p < 1, p ‡ / . We may denote
q=1 - p.
That is, the increments (X₂)₁ are i.i.d. and S₂ = X₁ + … + Xn for n ≥ 1 and S = 0.
Compute E[S2+1 Sn] for n ≥ 1.
(b) Show that Mn = (q/p) Sn defines a martingale (with respect to (Xk)k-1).
(c) Does the limit limn→∞ Mn exist almost surely? If yes, give a justification. If your answer is no,
explain why.
(d) Let T be the first time that S is equal to either −3 or 3. Compute P(ST = 3).
Hint: You may use, without proof, the fact P(T<∞) = 1 and that the Optional Stopping Theorem
(OST) applies to the martingale (Mn) and the stopping time T.
Expert Solution
This question has been solved!
Explore an expertly crafted, step-by-step solution for a thorough understanding of key concepts.
Step by step
Solved in 6 steps with 9 images
Recommended textbooks for you
MATLAB: An Introduction with Applications
Statistics
ISBN:
9781119256830
Author:
Amos Gilat
Publisher:
John Wiley & Sons Inc
Probability and Statistics for Engineering and th…
Statistics
ISBN:
9781305251809
Author:
Jay L. Devore
Publisher:
Cengage Learning
Statistics for The Behavioral Sciences (MindTap C…
Statistics
ISBN:
9781305504912
Author:
Frederick J Gravetter, Larry B. Wallnau
Publisher:
Cengage Learning
MATLAB: An Introduction with Applications
Statistics
ISBN:
9781119256830
Author:
Amos Gilat
Publisher:
John Wiley & Sons Inc
Probability and Statistics for Engineering and th…
Statistics
ISBN:
9781305251809
Author:
Jay L. Devore
Publisher:
Cengage Learning
Statistics for The Behavioral Sciences (MindTap C…
Statistics
ISBN:
9781305504912
Author:
Frederick J Gravetter, Larry B. Wallnau
Publisher:
Cengage Learning
Elementary Statistics: Picturing the World (7th E…
Statistics
ISBN:
9780134683416
Author:
Ron Larson, Betsy Farber
Publisher:
PEARSON
The Basic Practice of Statistics
Statistics
ISBN:
9781319042578
Author:
David S. Moore, William I. Notz, Michael A. Fligner
Publisher:
W. H. Freeman
Introduction to the Practice of Statistics
Statistics
ISBN:
9781319013387
Author:
David S. Moore, George P. McCabe, Bruce A. Craig
Publisher:
W. H. Freeman