Suppose that you start with an initial fortune of 20 dollars, and you bet one dollar each time. The probability of winning each hand is p. You quit if you reach your goal of L dollars or when you go broke. Let Q(xo) denote that probability that you eventually win. We need to characterize this probability. Next we give a more mathematical description of the problem. Now define your fortune at time n ≥ 1 by - Xn = Xn-1+ I{Y=1} = I{Yn=0} with the initial condition X = x0 ≥0. Let L = N be given, and define T inf{k : Xk = 0 or Xk = L}. Finally, let Q(x0) = P{XT = . L}. Show that X is a Markov chain and obtain its transition probability matrix. Is it irreducible? Use conditional expectations to prove that Q(x)=pQ(x+1) + (1 − p)Q(x − 1), - x = 1, L-1, (1) with Q(0) 0 and Q(L) = 1. = d) Solve the difference equation in (1) analytically. Let P(x0) P{XT = 0}. Determine P(x) directly and show that P(x) + Q(x) = 1. Hint: Use the same form of the difference equation and analytical solution as in (d) but use different initial conditions. f) Show that all intermediate states are transient and that both absorbing states are recurrent. Use (f) to show that the gambler's fortune converges almost surely to a binary random variable and characterize the limit.
Suppose that you start with an initial fortune of 20 dollars, and you bet one dollar each time. The probability of winning each hand is p. You quit if you reach your goal of L dollars or when you go broke. Let Q(xo) denote that probability that you eventually win. We need to characterize this probability. Next we give a more mathematical description of the problem. Now define your fortune at time n ≥ 1 by - Xn = Xn-1+ I{Y=1} = I{Yn=0} with the initial condition X = x0 ≥0. Let L = N be given, and define T inf{k : Xk = 0 or Xk = L}. Finally, let Q(x0) = P{XT = . L}. Show that X is a Markov chain and obtain its transition probability matrix. Is it irreducible? Use conditional expectations to prove that Q(x)=pQ(x+1) + (1 − p)Q(x − 1), - x = 1, L-1, (1) with Q(0) 0 and Q(L) = 1. = d) Solve the difference equation in (1) analytically. Let P(x0) P{XT = 0}. Determine P(x) directly and show that P(x) + Q(x) = 1. Hint: Use the same form of the difference equation and analytical solution as in (d) but use different initial conditions. f) Show that all intermediate states are transient and that both absorbing states are recurrent. Use (f) to show that the gambler's fortune converges almost surely to a binary random variable and characterize the limit.
A First Course in Probability (10th Edition)
10th Edition
ISBN:9780134753119
Author:Sheldon Ross
Publisher:Sheldon Ross
Chapter1: Combinatorial Analysis
Section: Chapter Questions
Problem 1.1P: a. How many different 7-place license plates are possible if the first 2 places are for letters and...
Related questions
Question
Transcribed Image Text:Suppose that you start with an initial fortune of 20 dollars, and you bet one dollar
each time. The probability of winning each hand is p. You quit if you reach your goal of L dollars
or when you go broke. Let Q(xo) denote that probability that you eventually win. We need to
characterize this probability. Next we give a more mathematical description of the problem.
Now define your fortune at time n ≥ 1 by
-
Xn = Xn-1+ I{Y=1} = I{Yn=0}
with the initial condition X = x0 ≥0. Let L = N be given, and define T inf{k : Xk = 0 or Xk =
L}. Finally, let Q(x0) = P{XT = .
L}.
Show that X is a Markov chain and obtain its transition probability matrix.
Is it irreducible?
Use conditional expectations to prove that
Q(x)=pQ(x+1) + (1 − p)Q(x − 1),
-
x = 1, L-1,
(1)
with Q(0) 0 and Q(L) = 1.
=
d) Solve the difference equation in (1) analytically.
Let P(x0) P{XT = 0}. Determine P(x) directly and show that P(x) + Q(x) = 1.
Hint: Use the same form of the difference equation and analytical solution as in (d) but use
different initial conditions.
f) Show that all intermediate states are transient and that both absorbing states are recurrent.
Use (f) to show that the gambler's fortune converges almost surely to a binary random
variable and characterize the limit.
Expert Solution
This question has been solved!
Explore an expertly crafted, step-by-step solution for a thorough understanding of key concepts.
This is a popular solution!
Trending now
This is a popular solution!
Step by step
Solved in 2 steps
Recommended textbooks for you
A First Course in Probability (10th Edition)
Probability
ISBN:
9780134753119
Author:
Sheldon Ross
Publisher:
PEARSON
A First Course in Probability (10th Edition)
Probability
ISBN:
9780134753119
Author:
Sheldon Ross
Publisher:
PEARSON