4512. Every Saturday night a man plays poker at his home with the same group of friends. If heprovides refreshments for the group (at an expected cost of $30) on any given Saturday night,the group will begin the following Saturday night in a good mood with probability and in a badmood with probability. However, if he fails to provide refreshments, the group will begin thefollowing Saturday night in a good mood with probability and in a bad mood with probability,regardless of their mood this Saturday. Furthermore, if the group begins the night in a bad moodand then he fails to provide refreshments, the group will gang up on him so that he incurs1expected poker losses of $70. Under other circumstances, he averages no gain or loss on hispoker play. The man wishes to find the policy regarding when to provide refreshments that willminimize his (long-run) expected average cost per week.(a) Formulate this problem as a Markov decision process by identifying the states and decisionsand then finding the Cik.(b) Identify all the (stationary deterministic) policies. For each one, find the transition matrixand write an expression for the (long-run) expected average cost per period in terms of theunknown steady-state probabilities (To,₁,TM).(c) Find these steady-state probabilities (To, ₁,,TM) for each policy. Then evaluate theexpression obtained in part (b) to find the optimal policy by exhaustive enumeration.

Answered: Image /qna-images/answer/02c5d15d-faf5-4466-b8bc-52fa0f2b0724.jpg

Answered: (a) Formulate this problem as a Markov…

A First Course in Probability (10th Edition)

10th Edition

ISBN:9780134753119

Author:Sheldon Ross

Publisher:Sheldon Ross

Chapter1: Combinatorial Analysis

Section: Chapter Questions

Problem 1.1P: a. How many different 7-place license plates are possible if the first 2 places are for letters and...

See similar textbooks

Similar questions

Central theme: Markov chainsA supermarket chain sells Alfa and Beta brand bread. Currently, Alfa has 50% of the chain's daily bread sales.of the chain's daily bread sales. To increase sales, Alfa launches a new advertising campaign. The bakerybelieves that the change in bread sales in the chain will be based on the transition matrix: A) In the long run, by what percentage can alpha expect to increase its current sales in the chain? Assumethat total daily sales of bread remain constant.
Solve bout the volume of securities sold by a brokerage firm, using the following data:
(6) Population Modeling with Discrete Dynamical Linear Systems. We now discuss populations with discrete breeding seasons, where reproduc- tion is limited to a particluar season of the year. For example, let's consider the number of female ruby-throated hummingbirds in a population with an annual breeding season of March to July. A female usually lays one clutch of two eggs; sometimes two clutches are laid. We will assume that the average life span of a female hummingbird is four years. We define the age of a bird at the END of a breeding season as follows: age zero (0) : any bird that is born during the current breeding season age one (1): any zero-year old bird which survives to the end of the next breeding season age two (2): any one-year old bird which survives to the end of the next breeding season age three (3): any two-year old bird which survives to the end of the next breeding season We make the following assumptions about the reproductive viability of female birds: age zero…
3. suppose that the manufacturer keeps a spare machine that only is used when the primary machine is being repaired. During a repair day, the spare machine has a probability of 0.1 of breaking down, in which case it is repaired the next day. Denote the state of the system by (x, y), where x and y, respectively, take on the values 1 or 0 depending upon whether the primary machine (x) and the spare machine (y) are operational (value of 1) or not operational (value of 0) at the end of the day. [Hint: Note that (0, 0) is not a possible state.] (a) Construct the (one-step) transition matrix for this Markov chain. (b) Find the expected recurrence time for the state (1, 0).
Which of the following can be accurately described by a “deterministic” model, that is, a model which does not require any concept of probability? (a) The position of a small particle in space (b) The velocity of an object dropped from the leaning tower of Pisa (c) The lifetime of a heavy smoker (d) The value of a stock which was purchased for $20 one month ago (e) The number of servers at a large data center which crash on a given day
(i) If volume is high this week, then next week it will be high with a probability of 0.6 and low with a probability of 0.4. (ii) If volume is low this week then it will be high next week with a probability of 0.1. Assume that state 1 is high volume and that state 2 is low volume (1) Find the transition matrix for this Markov process. (2) If the volume this week is high, what is the probability that the volume will be high two weeks from now? (3) What is the probability that volume will be high for three consecutive weeks?
Consider a set of three assets, one risk-free and two risky. The covariance matrix [0.36 0.07] for the risky assets is given by = 0.07 The risk-free rate is 0.03 0.07 0.29 and risk aversion is A = 4. Calculate the optimal portfolio for this individual.
Please show steps and solutions to solve problem
Three components are connected to form a system as shown in the accompanying diagram. Because the components in the 2-3 subsystem are connected in parallel, that subsystem will function if at least one of the two individual components functions. For the entire system to function, component 1 must function and so must the 2-3 subsystem. 2 The experiment consists of determining the condition of each component [S (success) for a functioning component and F (failure) for a nonfunctioning component]. (Enter your answers in set notation. Enter EMPTY or for the empty set.) (a) Which outcomes are contained in the event A that exactly two of the three components function? A = x (b) Which outcomes are contained in the event B that at least two of the components function? B = (c) Which outcomes are contained in the event C that the system functions? C = (d) List outcomes in C'. C' = List outcomes in A u C. AUC= List outcomes in An C. An C= List outcomes in Bu C. BUC= List outcomes in B n C.. BAC…
A market demand has 3 possible states namely GOOD, NORMAL and BAD for each period. At each period there are 2 possible decisions for the manager as do nothing/ make promotion. The transition matrix of states regarding for possible decisions are given as below Pof do nothing GOOD NORMAL BAD GOOD 0.4 0.2 04 0.3 0.2 0.5 NORMAL 0.5 BAD 0.1 0.4 Pof make promotion GOOD NORMAL BAD 0.4 0.4 0.5 GOOD 0.2 NORMAL 04 0.1 BAD 0.3 0.4 0.3 It is known that the income for each period when the state in GOOD, NORMAL and BAD are 10000, 7000, 2000. Cost for do nothing is 0, and make promotion is 3000. a) Given at period 0 the state is NORMAL, estimate expected beneft obtain two periods when the sequence of decisions is do nothing, do nothing b) Given at period 0 the state is NORMAL, estimate expected beneft obtain two periods when the sequence of decisions is make promotion, make promotion
Determine whether the statement below is true or false. Justify the answer. If (x) is a Markov chain, then X₁+1 must depend only on the transition matrix and xn- Choose the correct answer below. O A. The statement is false because x, depends on X₁+1 and the transition matrix. B. The statement is true because it is part of the definition of a Markov chain. C. The statement is false because X₁ +1 can also depend on X-1 D. The statement is false because X₁ + 1 can also depend on any previous entry in the chain.
3. (a) A Markov process with 2 states is used to model the weather in a certain town. State 1 corresponds to a suny day. State 2 corresponds to a rainy day. The transition matrix for this Markov process is [0.7 0.4] 0.3 0.6 %3D (i) If today is rainy, what is the probability that tomorrow will be sunny? (ii) Find the steady state probability vector. (iii) In the long run, how many days a week are sunny?

Recommended textbooks for you

A First Course in Probability (10th Edition)

Probability

ISBN:

9780134753119

Author:

Sheldon Ross

Publisher:

PEARSON

A First Course in Probability

Probability

ISBN:

9780321794772

Author:

Sheldon Ross

Publisher:

PEARSON

A First Course in Probability (10th Edition)

Probability

ISBN:

9780134753119

Author:

Sheldon Ross

Publisher:

PEARSON

A First Course in Probability

Probability

ISBN:

9780321794772

Author:

Sheldon Ross

Publisher:

PEARSON

SEE MORE TEXTBOOKS