Homework Help is Here – Start Your Trial Now!
Math
Probability
Question. Consider a single-server system where there can be at most two customers in the system (including the one being served). In each hour, a new customer enters to the system with probability % unless there are already 2 customers in the system. Assume that new arrival occurs at the end of each hour.

Question. Consider a single-server system where there can be at most two customers in the system (including the one being served). In each hour, a new customer enters to the system with probability % unless there are already 2 customers in the system. Assume that new arrival occurs at the end of each hour.

Algebra & Trigonometry with Analytic Geometry
Algebra & Trigonometry with Analytic Geometry
13th Edition
ISBN:9781133382119
Author:Swokowski
Publisher:Swokowski
Chapter10: Sequences, Series, And Probability
Section10.8: Probability
Problem 32E
See similar textbooks
icon
Related questions
Question
**Question**: Consider a single-server system where there can be at most two customers in the system (including the one being served). In each hour, a new customer enters the system with a probability of ⅓ unless there are already 2 customers in the system. Assume that new arrivals occur at the end of each hour. At the beginning of each hour, the server can decide a configuration if there is a customer in the system. If the configuration is fast, with probability 0.7, one customer is served and leaves the system in a given hour. On the other hand, if the configuration is slow, this probability decreases to 0.4. 80 TL revenue is obtained for each customer whose service is completed. The costs of slow and fast configurations are 10 and 15 TL per hour, respectively. The hourly discount rate is β=0.8. **Objective**: We would like to maximize the total expected discounted profit over an infinite horizon. **Questions**: a. Formulate the problem as an MDP (Markov Decision Process) model by defining states, decision sets, transition probabilities, and expected rewards clearly. b. Find the optimal policy using Policy Iteration where the initial policy is to use the fast configuration whenever there is at least one customer in the system. c. Develop an LP (Linear Programming) model for the MDP. Assume that you have solved the LP optimally. What is the optimal solution, which constraints in your model are binding and why? (Do not solve the LP model; you can determine the binding constraints by observing the optimal solution found in part b).
Transcribed Image Text:**Question**: Consider a single-server system where there can be at most two customers in the system (including the one being served). In each hour, a new customer enters the system with a probability of ⅓ unless there are already 2 customers in the system. Assume that new arrivals occur at the end of each hour. At the beginning of each hour, the server can decide a configuration if there is a customer in the system. If the configuration is fast, with probability 0.7, one customer is served and leaves the system in a given hour. On the other hand, if the configuration is slow, this probability decreases to 0.4. 80 TL revenue is obtained for each customer whose service is completed. The costs of slow and fast configurations are 10 and 15 TL per hour, respectively. The hourly discount rate is β=0.8. **Objective**: We would like to maximize the total expected discounted profit over an infinite horizon. **Questions**: a. Formulate the problem as an MDP (Markov Decision Process) model by defining states, decision sets, transition probabilities, and expected rewards clearly. b. Find the optimal policy using Policy Iteration where the initial policy is to use the fast configuration whenever there is at least one customer in the system. c. Develop an LP (Linear Programming) model for the MDP. Assume that you have solved the LP optimally. What is the optimal solution, which constraints in your model are binding and why? (Do not solve the LP model; you can determine the binding constraints by observing the optimal solution found in part b).
Expert Solution
trending now

Trending now

This is a popular solution!

steps

Step by step

Solved in 3 steps

SEE SOLUTIONCheck out a sample Q&A here
Blurred answer
Recommended textbooks for you
Algebra & Trigonometry with Analytic Geometry
Algebra & Trigonometry with Analytic Geometry
Algebra
ISBN:
9781133382119
Author:
Swokowski
Publisher:
Cengage
Calculus For The Life Sciences
Calculus For The Life Sciences
Calculus
ISBN:
9780321964038
Author:
GREENWELL, Raymond N., RITCHEY, Nathan P., Lial, Margaret L.
Publisher:
Pearson Addison Wesley,
Holt Mcdougal Larson Pre-algebra: Student Edition…
Holt Mcdougal Larson Pre-algebra: Student Edition…
Algebra
ISBN:
9780547587776
Author:
HOLT MCDOUGAL
Publisher:
HOLT MCDOUGAL
SEE MORE TEXTBOOKS