Control of Dams Using P M λ , τ Policies When the Input Process Is a Nonnegative Lévy Process

We consider P λ,τ policy of a dam inwhich thewater input is an increasing Lévy process. The release rate of the water is changed from 0 toM and fromM to 0 M > 0 at the moments when the water level upcrosses level λ and downcrosses level τ τ < λ , respectively. We determine the potential of the dam content and compute the total discounted as well as the long-run average cost. We also find the stationary distribution of the dam content. Our results extend the results in the literature when the water input is assumed to be a Poisson process.


Introduction and Summary
Lam and Lou 1 consider the control of a finite dam where the water input is a Wiener process, using P M λ,τ policies. In these policies, the water release rate is assumed to be zero until the water reaches level λ > 0, as soon as this happens the water is released at rate M > 0 until the water content reaches level τ > 0, λ > τ. Abdel-Hameed and Nakhi 2 discuss the optimal control of a finite dam using P M λ,τ policies, using the total discounted as well as the long-run average costs. They consider the cases where the water input is a Wiener process and a geometric Brownian motion process. Lee and Ahn 3 consider the long-run average cost case when the water input is a compound Poisson process. Abdel-Hameed 4 treats the case where the water input is a compound Poisson process with a positive drift. He obtains the total discounted cost as well as the long-run average cost. Bae et al. 5 consider the P M λ,0 policy in assessing the workload of an M/G/1 queuing system. Bae et al. 6 consider the log-run average cost for P M λ,τ policy in a finite dam, when the input process is a compound Poisson process. In this paper, we consider the P M λ,τ policy for the more general case where 2 International Journal of Stochastic Analysis the water input is assumed to be an increasing Lévy process. At any time, the release rate can be increased from 0 to M with a starting cost K 1 M or decreased from M to zero with a closing cost K 2 M. Moreover, for each unit of output, a reward R is received. Furthermore, there is a penalty cost which accrues at a rate f, where f is a bounded measurable function on the state space of the content process. We will use the term "increasing" to mean "nondecreasing" throughout this paper.
In Section 2, we discuss the potentials of the processes of interest as well as the other results that are needed to compute the total discounted and long-run average costs. In Section 3, we obtain formulas for the cost functionals using the total discounted as well as the long-run average cost cases. In Section 4, we discuss the special cases where the water input is an increasing compound Poisson process as well as inverse Gaussian process.

Basic Results
The content process is best described by the bivariate process B Z, R , where Z {Z t , t ≥ 0} and R {R t , t ≥ 0} describe the dam content and the release rate, respectively. We define the following sequence of stopping times:

2.1
The process B has as its state space the pair of line segments Let I {I t , t ≥ 0} be an increasing Lévy process with drift a ≥ 0. For each t ≥ 0, we let I * t I t − Mt. From the definition of the P M λ,τ policy, it follows that, for each t ∈ 0, T 0 , Z t I t ,

2.3
Furthermore, I * T n I T n , n 0, 1, . . .. It follows that the content process Z is a delayed regenerative process with the regeneration points being the * T n , n 1, 2, . . . . The penalty cost rate function is defined as follows: where g : 0, λ → R and g * : τ, ∞ : → R are bounded measurable functions. x max y and x ∧ y x min y. Throughout, we define W λ inf{t ≥ 0 : I t ≥ λ} and W * τ inf{t ≥ 0 : I * t ≤ τ}. For any x < λ and y > τ, let C α g 0, x, λ and C α g * M, y, τ be the expected discounted penalty costs, during the intervals 0, W λ and 0, W * τ , respectively. Furthermore, let C g 0, x, λ and C g * M, y, τ be the expected nondiscounted penalty costs during the same intervals. It follows that

2.5
The functionals above, which we aim to evaluate, are basic ingredients in computing the total discounted and long-run average costs associated with the P M λ,τ policy as discussed in Section 3.
Let a ≥ 0 and ν be the drift term and the Lévy measure of input process I, respectively, then, for all t ≥ 0, x ≥ 0, and α ≥ 0 the Laplace transform of I t is of the form, The function φ α is known as the Lévy component and is given by where ν is a measure on 0, ∞ satisfying Increasing Lévy processes include increasing compound Poisson processes, inverse Gaussian processes, gamma processes, and stable processes. We assume that the expected value of I 1 is finite throughout this paper.
To evaluate the cost functionals and other parameters of the content process, we define the Lévy process killed at W λ as follows: From Theorem 3.3.12 of Blumenthal and Getoor 7 , it follows that the process X is a strong Markov process.
Remark 2.2. Throughout, we denote the α-potential of the process I by U α . Since the process I has stationary independent increments, it follows that U α x, dy U α 0, dy − x , for each x and y in the state space of the process I satisfying y ≥ x. We denote U α 0, dy by U α dy , throughout.
Since the process I is increasing and has stationary independent increments, it follows that The following lemma follows by taking g x 1 for all x ∈ 0, λ in 2.11 and 2.12 , respectively.

Lemma 2.3. For x ≤ λ one has
The following Lemma gives the Laplace transform of I W λ as well as the expected value of I W λ .
International Journal of Stochastic Analysis 5 Proof of a . For x < λ and α ≥ 0, since the process I has stationary independent increments, we have where the second equation follows from 8 of Alili and Kyprianou 9 , and the fourth equation follows from the definition of φ α and U 0 .
where the first equation follows since the process I is a Lévy process, the third equation follows from 2.15 , the fourth equation follows because φ 0 0, the fifth equation follows since φ 0 E 0 I 1 , and the last equation follows from 2.14 .
To derive C α g * M, y, τ , C g * M, y, τ , E y exp −αW * τ , and E y W * τ , we define International Journal of Stochastic Analysis Throughout, we assume that M ≥ a. Using Doob's optional sampling theorem, the following is easy to see.
where η α is the solution of the integral equation The following Lemma gives, among other things, a formula for computing E x W * τ and condition under which this expectation is finite.

2.22
Proof of a . From 2.20 , it follows that η α is an increasing function on R and lim α → ∞ η α ∞. Let f x η −1 x , using 2.21 it follows that f x Mx − φ x . Furthermore, η 0 is the largest root of f, and 0 is indeed a root of f and, since η α is an increasing function, f is an increasing function on the domain η 0 , ∞ . It follows that the only root of the function f above is zero if and only if f 0 > 0. Observe that where the interchange of the differentiation and integration in the second equation is permissible using the Lebesgue dominated convergence theorem, since for each Proof of b . To prove part b , first we observe that f x is an increasing function in its argument, and hence f x is a convex function in its argument. Since f x η −1 x , it follows that η α is a concave function.
Proof of c . If the proof of part c follows since, from 2.20 , W * τ < ∞ almost everywhere if and only if η 0 0, in this case, Remark 2.7. The equation given in part c of Lemma 2.6 is consistent with the well-known fact about the expected busy period of the M/G/1 queue.
Let * U α be the potential of the process X * . To find * U α , we first need to introduce the following definition. where ψ θ Mθ − φ θ .

2.25
We note that the function η is the right-hand inverse of the function ψ.
We now define the α-scale function, which plays a major role in the applications of spectrally positive negative Lévy processes. This function is closely connected to the twosided exit problem of such processes cf. Bertion 10 .
Definition 2.9. For α ≥ 0, the α-scale function of the process * I t W α : R → R is the unique function whose restriction to R is continuous and has Laplace transform and is defined to be identically zero on the interval −∞, 0 .
Letting α 0, we get the 0-scale function, which is referred to as the "scale function" in the literature. We denote this function by W instead of W 0 throughout. We note that For every x ∈ R , let F x x 0 ν y, ∞ dy/μ be the equilibrium distribution function corresponding to ν. Let ρ μ/N and assume that ρ < 1. It follows that where F k is the kth convolution of F. Furthermore, we note that for α, x ∈ R , where W k is the kth convolution of W.
We are now in a position to state and prove a lemma that characterizes * U α .
Proof. Define the process ∧ I to be equal to − * I; it follows that ∧ I is a spectrally negative Lévy process. For a, b ∈ R, we let

2.30
Supurn 11 proved that for b > 0 the α-potential of the process obtained by killing the process ∧ I at T b ∧ T − 0 is absolutely continuous with respect to the Lebesgue measure on 0, b , and its density is equal to It follows that, for a, b ∈ R, a < b, the α-potential of the process obtained by killing the process ∧ I at T b ∧ T − a is absolutely continuous with respect to the Lebesgue measure on a, b , and its density is equal to From Lemma 4 of Pistorious 12 , we have W α x O e η α x as x → ∞. Letting a → −∞ in the last density above, then the the α potential of the process obtained by killing the process ∧ I at T b is absolutely continuous with respect to the Lebesgue measure on −∞, b , and its density denoted by u α b x, y is as follows:

2.34
International Journal of Stochastic Analysis 9 Thus, *

2.35
It is seen that, for x ≥ τ, U 0 x, dy .

2.38
Proof of a . Let be the sigma algebra generated by W λ , I W λ , then we have

The Total Discounted, Long-Run Average Costs and the Stationary Distribution of the Dam Content
We now discuss the computations of the cost functionals using the total discounted cost as well as the long-run average cost criteria. Let W be the length of the first cycle, that is, W * T 1 − * T 0 , and let C α x be the expected cost during the interval 0, * T 0 , when Z 0 x. Since the content process Z is a delayed regenerative process with regeneration points * T 0 , * T 1 , . . ., using the delayed regeneration property, it follows that the total discounted cost associated with an P M λ,τ policy is given by where C α 1 is the total discounted cost during the interval 0, W . From the definitions of C α x , it follows that, for x > λ, To compute C α x for x ≤ λ, we let be the sigma algebra generated by W λ , I W λ and proceed as follows: International Journal of Stochastic Analysis where the second equation follows from the definition of the process Z, the third equation follows from the definition of C α g 0, τ, λ , the fourth equation follows from the definition of the content process Z and since, given , * T 0 − W λ W * τ almost everywhere, and the last equation follows from the definition of C α g * M, x, λ . We note that The following lemma shows how E τ exp −αW given in 3.1 can be computed and also gives a formula for computing the expected value of W, which we will need later on to compute the long-run average cost. Lemma 3.1. Let W be the length of the first cycle as defined above, then a

12 International Journal of Stochastic Analysis
Proof of a . We note that, given Z 0 τ, T * 0 W almost everywhere. Thus, for each α ≥ 0, where the second equation follows 2.37 upon substituting τ for x, the third equation follows from the definition of the U α and 2.21 .
Proof of (b). From 3.5 , it is evident that, starting at τ, W is finite almost everywhere if and only if η 0 0. From part a of Lemma 2.6, it follows that W is finite almost everywhere if and only if E 0 I 1 < M. From 2.14 and 3.5 , we have

3.8
The proof of b is complete, since as shown in the proof of part c of Lemma 2.6 Now, we turn our attention to computing the long-run average cost per a unit of time. Let M − E 0 I 1 M * and assume that M * > 0. From 3.1 , 3.3 , and 3.4 , it follows, by a Tauberian theorem, that the long-run average cost per unit of time, denoted by C λ, τ , is given by

3.10
where K K 1 K 2 , and the second equation follows from 3.6 and the first equation.
International Journal of Stochastic Analysis 13 Remark 3.2. Assume that both penalty functions g and g * are identically zero on their domains, and M * defined above is greater than zero. The following follows from 3.10 above: Letting R 0, K 0 and g x I τ,z x , x ∈ 0, λ and g * x I τ,z x , x ∈ τ, ∞ in 3.10 , we get the following proposition which generalizes the results obtained by Lee and Ahn 3 , where they assumed that the input process is a compound Poisson process and τ 0.
and, H z be the distribution function of the process Z, then, for z ∈ τ, ∞ , 3.12

Special Cases
In this section, we give the basic identities needed to compute the cost functionals when the input process is an inverse Gaussian process and a compound Poisson process, respectively.

4.1
It follows that the process I is an increasing Lévy process with state space R , Lévy measure and Lévy component φ α Furthermore, E 0 I 1 1/μ.

International Journal of Stochastic Analysis
Substituting this Lévy component above in 2.21 , it is seen that the solution of this equation is as follows we omit the proof : To find the α-potential of the process I, for each x ≥ 0 and β ≥ 0, we define f β x exp −βx , and it is easily seen that

4.5
Throughout we let ϕ Z · be as the standard normal density function and let erf and erf c be the well-known error and complimentary error functions, respectively. Inverting the above function with respect to β, we have where the last equation follows by integrating U α dy over the interval λ − x, ∞ . Inverting the right hand side of 4.8 with respect to α, it follows that, given I 0 x ≤ λ, the distribution function of W λ denoted by F W λ is given by Furthermore, for x ≤ λ, where the third equation follows from the second equation upon tedious calculations which we omit. We now turn our attention to computing the distribution function of I W λ denoted by F I W λ x . We first need the following identity which expresses the Lévy component φ α given in 4.3 in a form suitable for computing F I W λ . The proof of this identity follows from 4.3 after some simple algebraic manipulations which we omit:

4.11
For each β ∈ R , we write Case 2. Assume that I is an increasing compound Poison process with intensity u and F as the distribution function of the size of each jump. This model is treated in details in references 3, 4, 6 . Here, we give the basic entities involved when the drift terms a 0. For the proof of these entities and more in depth analysis of this case, the reader is referred to the abovementioned references. It is obvious that φ α u ∞ 0 1 − e −αx F dx , 4.14 E 0 I 1 uμ, where μ is the expected jump size of the compound Poisson process. Define, for any α ≥ 0 and y ≥ 0, F α y u/u α F y . For n ∈ N , we let F n α y be the nth convolution of F α y , where F 0 α y 1 for all y ≥ 0. For each y ≥ 0, we define R α y ∞ n 0 F n α y to be the renewal function corresponding to F α y . It follows that U α dy 1 u α R α dy .

4.16
Also, Inverting 4.17 with respect to α, the distribution function of I W λ , denoted by G, is given through