Equilibrium Customer Strategies in the Queue with Threshold Policy and Setup Times

We consider the equilibrium behavior of customers in the M/M/1 queue with N policy and setup times. The server is shut down once the system is empty and begins an exponential setup time to start service again when the number of customers in the system accumulates the threshold N (N ≥ 1). We consider the equilibrium threshold strategies for fully observable case and mixed strategies for fully unobservable case, respectively. We get various performance measures of the system and investigate some numerical examples of system size, social benefit, and expected cost function per unit time for the two different cases under equilibrium customer strategies.


Introduction
Due to wide applications for management in service system and modern electronic commerce, there exists an emerging trend to study the behavior of customers in queueing systems.In these models, the customers decide to join or balk in a decentralized way.Their decisions are affected by how much system information can be observed and how other customers make their decisions.This can be viewed as a game among the customers.Researches about the economic analysis of queueing models can go back at least to the pioneering work of Naor [1] who analyzed customers' optimal strategies in an observable Markovian queue with a concise reward-cost structure.Naor's model and results had already been extended by several authors; readers can refer to [2][3][4].
The concept of an  policy was first introduced by Yadin and Naor [5].The server is immediately turned on whenever  ( ≥ 1) or more customers are present in the system and turned off once there are no customers in the system.Once the server is shut down, the server may not operate until  customers are present in the model.Wang and Ke [6] studied the optimal operation of a single removable server in an //1 queueing system with either infinite waiting room or finite waiting room by a supplementary variable technique, where the "removable server" is merely an abbreviation for the system of turning on and turning off the server depending on the number of customers in the queue.Wang et al. [7] further consider an //1 queueing system with  policy, where the server is typically subject to breakdowns unpredictably.
Guo and Hassin [8] first consider customers' strategic behavior and social optimization in a single Markovian queue with  policy, in which the server is activated if there are  customers in the system and turned off once there are no customers in the system.They found an interesting conclusion which is that a customer can bring positive externality to others by joining, which means that a new arrival can either keep the server operational for a longer time or stir up an inactive server to come back to work sooner, which leads to promoting other customers' benefits.Guo and Li [9] further investigated customers' strategic behavior and social optimization in //1 queues with  policy under two partial information cases.
Queues with setup times have also been investigated by many scholars [6,[10][11][12].In queueing systems with setup times, once server is reactivated, a generally random time is required for setup before it can start serving customers.As for the research on the equilibrium customer behavior 2 Mathematical Problems in Engineering in Markovian queues with setup times was first presented by Burnetas and Economou [13].Subsequently, Economou and Kanta [14] further investigated the Markovian queue that alternates between on and off periods in observable cases.Chen and Zhou [15] studied three different scenarios of balking strategies in the Markovian queue with setup times and server breakdowns.
Frequent setups will inevitably lead to the operating cost being too high, so it is very crucial to determine when the server should start his service in many practical queueing systems.By utilizing  policy, the server does not turn on his service until the number of customers in the system accumulates a threshold .Selecting appropriate value  can avoid excessively frequent setups.It is more practical to analyze equilibrium strategies for the  policy system with setup times.For example, from an economic perspective, a driver may start driving a shuttle whenever a certain amount of seats is occupied.Another example is a queue-based power-saving scheme to alleviate the energy hole problem [16,17].Handling with sensor nodes in wireless sensor networks (WSNs) is actually crucial to understand the optimal operation of the threshold policy for power saving in a WSN.In the " policy" scheme, when the number of packets accumulates to , the sensor node turns on its transmitting function of the radio sever and starts transmission process for the waiting packets.The sensor node is closed as soon as all present packets are transmitted.The  policy holds effective for a power-saving mechanism since it reduces the setup power consumption required to switch between a busy mode and an idle mode of a radio server in sensor nodes.The proposed queuebased power-saving technique can be used for prolonging the lifetime of the WSN effectively and economically.To the best of our knowledge, no other scientific paper has dealt with equilibrium customer strategies in the Markovian queue with  policy and setup times.Customers make their decisions based on a nature reward-cost structure, which incorporates their desire for service as well as their unwillingness to wait.We discuss, respectively, the fully observable scenario where customers are informed not only about the server state but also about the exact number of customers in the queue and the fully unobservable scenario where an arriving customer does not observe the exact number of customers waiting for service and the state of server at all.
The structure of this paper is organized as follows.In Section 2, we describe the dynamics of the queueing system.In Section 3, we obtain the equilibrium threshold strategies for the fully observable case and stationary probabilities of the corresponding queue.In Section 4, we derive the stationary probabilities and equilibrium mixed strategies for the fully unobservable case.In Section 5, some numerical results are presented to illustrate the effect of two different information levels and  policy on the customers' strategies.Finally, Section 6 provides some conclusions.

Model Description
We assume that there is a single server queue with infinite capacity where customers arrive according to a Poisson  process with rate .The service times are supposed to be exponentially distributed with parameter .We further assume that 0 <  < .The server utilizes the  policy: turn the server on as soon as  ( ≥ 1) or more customers are present in the system and turn the server off whenever there are no customers in the system.After the server is shut down, the server cannot work until  customers are presented in the queue and a setup process begins for the server to be activated.The time needed to setup is exponentially distributed with rate .We further suppose that interarrival times, service times, and setup times are mutually independent.
In this paper, we suppose that every customer gets a reward of  units for completing his service.Moreover, we suppose that there exists a waiting cost of  units per time unit that is continuously accumulated from the time that the customer joins the system till the time he leaves the system after he finished service.

Equilibrium Strategies and Some Performance Measures for the Fully Observable System
We now commit to the fully observable scenario, which assumes that customers observe not only the server state, but also the exact number of customers in the system.We will study the equilibrium threshold strategies and stationary probabilities for the fully observable case as well as some interesting performance measures of this system.

Equilibrium Threshold Strategies and Stationary Probabilities for the Fully Observable Case.
In order to study the general threshold strategy adopted by all customers in the fully observable case, we will consider the mean overall sojourn time of the customer who is confronted with different server state upon arrival firstly.We start with the fully observable case where customers know the exact state of the system and a customer in state (, ) has mean sojourn time as follows.
(I) If a new customer finds the system in state (0, ), where 0 ≤  ≤  − 1, then his expected benefit is Let () =  − (( − ( + 1))/ + 1/ + ( + 1)/); we have   () = (1/ − 1/) > 0 (since we assume that 0 <  <  in Section 2).Obviously, () increases by increasing .That is to say, when the system is on  policy, the least expected benefit is the first customer in the system.For the sake of ensuring a customer finds the system in (0, 0) state and still joins the system, we must have which means that (II) If a new customer finds the system in state (1, ), where  ≥  (this phenomenon means the system is in setup phase), then his expected benefit is which means that if the server is currently in a setup state, the upper bound on the number of customers that an incoming customer is willing to stay behind is (III) If a new customer finds the system in state (2, ), then his expected benefit is which means that the upper bound on the number of service epochs that a customer is willing to wait is Proof.From (3), we immediately find that a customer who observes the server in vacation is motivated to join if and only if  ≤ ⌊1 + / − / − /⌋.From (5), we find that if an arriving customer observes the server in setup state he is motivated to join on the condition that the number of customers in the system is no more than   (1), which means that  ≤   (1).From ( 7), we find that if an arriving customer observes the server in service he is motivated to join if and only if  ≤   (2).If an arriving customer finds the number of customers in the system exceeds   (2), which means that his expected net benefit is negative in any case, then his rational decision is to balk the system.
We now turn our attention to the stationary distribution probabilities.Be aware that if all customers follow the threshold strategy in Theorem 2, then the system follows a Markov chain ((), ()) with state space of the system The transition diagram is depicted in Figure 2.

The System Performance Measures for the Fully Observable
Case.We now study an expected cost function per unit time for the (,   (1),   (2)) threshold strategy system.Our objectives are to analyze performance measures of the fully observable system and determine the optimum value of the important control threshold  by numerical computations.
(1) The Actual Arrival Rate.Because an arrival balks whenever he finds the system at state (1,   (1) + 1) or (2,   (2) + 1), then the system's effective arrival rate is given by (2) The Expected Number of Customers.Let  ob denote the expected number of customers in the (,   (1),   (2)) threshold strategy system.Then we obtain (3) The Probability of the Server in Different State.We further give the probability that the server in idle, setup, and busy steady state is as follows: ( = 0) =  ob (0, 0) ; (4) The Expected Busy Cycle.Let the expected length of the idle period be denoted by ( = 0).Applying the memoryless property of the Poisson process, the length of the idle period is the sum of  exponential random variables, each having a mean of 1/.Hence, the expected length of the idle period is presented by ( = 0) = /.The busy cycle for the fully observable case is denoted by (  ), which is the length of time from the beginning of completing all the customers in system to the beginning of the next completing of all the customers in system, which included vacation period, setup period, and busy period.From the relationship we can get the expected length of the busy cycle (  ) = 1/[ ob (0, 0)].
(5) The Social Benefit per Time.The social benefit per time unit as soon as all customers follow the equilibrium policy (,   (0),   (1)) equals (6) The Expected Cost Function.We next define the following: ℎ = holding cost per unit time for every customer present in the system; = cost incurred per busy cycle; 0 = cost per unit time for keeping the server off; 1 = setup cost per unit time for turning the server on;  2 = cost per unit time for keeping the server on and in operation;  3 = a fixed cost for every lost customer.By using the definitions of all costs listed above, the expected cost function per unit time per customer is given by +  2  ( = 2) (33)

Equilibrium Strategies and Some Performance Measures for the Fully Unobservable System
We turn our attention to the fully unobservable case, where the customers do not observe the state and system size at all when they join the system.We will study the equilibrium arrival rate and stationary probabilities for the fully unobservable case as well as some interesting performance measures of this system.

Equilibrium Mixed Strategies and Some Stationary Probabilities for the Fully Unobservable
Case.We will prove that there exists a mixed equilibrium strategy.An arriving customer joins the fully unobservable system with a certain probability ; then the effective arrival rate is  0 = .If all customers adopt the same mixed strategy, then the system follows a Markov chain which is similar to that described in Figure 1.The state space  un for the fully unobservable case is identical to the original state space .In addition to substituting  0 for , the state transfer diagram is exactly the same as Figure 1.
Proof.The balance equations are presented below: From (40), we easily get (34) in Theorem 5. From (41) and (42), we can get (35) in Theorem 5. From (39) and (43), we get  un (2, 1) and  un (2,2).By substituting them into (44), we immediately get expression (36) in Theorem 5. Next, we put  un (2,  − 1) and  un (2, ) into expression (45); we can get (37) in Theorem 5.By putting (34)-(37) into (46), we obtain the expression of  un (0, 0) at last.Theorem 6.In the fully unobservable //1 queue with  policy and setup times, where the customers enter the system with probability , when a customer enters the system in state  ( = 0, 1, 2), his expected sojourn time   is given below: and the average sojourn time of an arriving customer is Proof.Let  un () be the probability that a customer arrives during state , where  = 0, 1, 2. From Theorem 5, we can easily get the steady state probabilities of the server: Let ( | ) be the stationary probability that the queue length is  conditional on observing the servers status which is , where  = 0, 1, 2 and  = 0, 1, . ... The conditional probabilities can be written by Note that if the queue length is  and the server is busy, the expected sojourn time of the new arriving customer is denoted as ( | 2), which is the expected service time of the  + 1 customer in the system, that is, ( | 2) = ( + 1)/, which means the expected sojourn time upon seeing a busy server is The expected sojourn time upon seeing a setup server is The expected sojourn time upon seeing the server on  policy is Then the expected sojourn time   ( = 0, 1, 2) is given as follows: Then the average sojourn time of an arriving customer is Next, according to the value of , we divide  into two cases as follows.
Case 1.When  = 1, the system becomes the fully unobservable //1 queue with setup times and  < .The average sojourn time reduces to We consider a tagged customer at his arrival instant, if he decides to enter his expected net benefit which is ) , where  0 = .
Proposition 7. In the fully unobservable //1 queue with setup times and  < , there exists a unique mixed equilibrium strategy "enter with probability   ," where the vector   is given by ( This conclusion is consistent with Theorem 5 in [13].
Case 2. When  ≥ 2, we have which means that ( 0 ) is strictly convex in  0 and lim  0 → 0+ ( 0 ) = lim  0 →  − ( 0 ) = +∞, and then there is a unique minimum point λ ∈ (0, ), so that ( λ) is the minimum sojourn time and λ satisfies the following equation: It is clear that when  < ( λ), there is no positive equilibrium arrival rate; when  = ( λ), there exists one positive equilibrium arrival rate λ ∈ (0, ) iff λ < ; when  > ( λ), that is,  ∈ (( λ), +∞), the equation has two real roots  1 ∈ (0, λ) and  2 ∈ ( λ, ).We analyze the strictly convex ( 0 ) carefully and then find that the equilibrium with  1 is unstable, for, with any small increase of the arrival rate, the expected waiting time decreases and more customers will arrive in the system; this will further increase  1 .The equilibrium with  2 and  is stable; that is, if there is a small perturbation to them, the system will converge back to them.We denote the stable equilibrium arrival rate as  * ; then we have the following proposition.
According to Proposition 7 and case (c) of Proposition 8, we give concrete numerical forms as shown in Tables 1, 2, and 3.

The System Performance
Measures for the Fully Unobservable Case.Our objective is to give some interesting performance measures of the fully unobservable system and determine the optimum value of the control threshold  by numerical computations.(65) (3) The Expected Busy Cycle.Similar to the fully observable case, the expected length of the idle period is given by ( = 0) = / 0 and the expected length of the busy cycle (  ) = 1/[ 0  un (0, 0)].

Figure 1 :
Figure 1: Transition rate diagram of the original model.

( 1 )( 2 )
The Expected Number of Customers.Let  un denote the expected number of customers in the (, ) equilibrium strategy.Then we obtain  un =  ( − 1)  un (0, The Probability of the Server in Idle, Setup, and Busy Steady State as Follows.Consider  ( = 0) =  un (0,