Question no. 24 A Markov chain with state space {-2,-1,0, 1, 2} has the following one- step transition probabilities: Pi,i+1=3/4 for i = -2,-1,0,1 Pi,i-1=1/4 for i = -1,0, 1,2 P-2,2 = 1/4 = 1-P2.-2 (a) Determine the classes of the Markov chain. For each class, establish whether it is recurrent or transient and whether it is periodic or aperiodic. (b) Do the limiting probabilities exist? If they do, compute them. (c) Is the chain time-reversible (see p. 147)? Justify.

Suppose that a Markov chain with 4 states and with transition matrix P is in state 4 on the fourth observation. Which of the following expressions represents the probability that it will be in state 1 on the sixth observation? (A) the (4, 6) entry of P (B) the (1,4) entry of P2 (C) the (4, 1) entry of P2 (D) the (4, 6) entry of P4 (E) the (1,4) entry of P6 (F) the (4, 1) entry of P6 (G) the (6,4) entry of P (H) the (6, 4) entry of P4 O A В C O D O E F O H

7

Consider a continuous time Markov chain with three states {0, 1, 2} and transitions rates as follows: q01 = 3, q12 = 5, q21 = 6, q10 = 4 and the remaining rates are zeros. Find the limiting probabilities for the chain.

Consider a Markov chain {Xn}n≥0 having the following transition diagram: For this chain, there are two recurrent classes R1 = {6, 7} and R2 = {1, 2, 5}, and one transient class R3 = {3, 4}. Find the period of state Find f33 and f22. Starting at state 3, find the probability that the chain is absorbed into R1. Starting at state 3, find the mean absorbation time, i.e., the expected number of steps that the chain is absorbed into R1 or R2.   Note: there are missing transition probabilities for this chain, but no impact for your solution.

Could you please help with b?

A Markov chain has the transition probability matrix 0.3 0.2 0.5 0.5 0.1 0.4 0.5 0.2 0.3 In the long run, what proportion of time is spent by the Markov chain in state 3?

Suppose Xn, n = 0, 1, ... is a two-state Markov chain with states Sx = {0, 1}, whose transition probability matrix is 0.1 0.9 0.95 0.05 Then, Zn = (Xn-1, Xn), n = = 1,2,. That is, Zn i j iff Xn-1 i and Xn Pz= = = Px = is a four-state Markov chain with states Sz = {00, 01, 10, 11}. = j. Determine its transition probability matrix.

Consider a continuous-time Markov chain with transition rate matrix 0 2 31 Q10 3 2 0/ What are the one-step transition probability matrix of the embedded chain? Enter your answer as a vector of the form ((a₁, b₁, ₁), (a₂, b₂, ₂), (as, bs, cs)) where each of a, b, c, is an integer of the form or a fraction of the form m/n; eg ((1, 1/2, 1/3), (1, 1/2, 1/3), (1, 1/2, 1/3)). Note that there are 8 brackets and 8 commas in total. All should be included. Do not include spaces.

Please do question 1b, 1c and 1d with full working out. I'm struggling to understand what to write