A FIRST PASSAGE PROBLEM AND ITS APPLICATIONS TO THE ANALYSIS OF A CLASS OF STOCHASTIC MODELS

A problem of the first passage of a cumulative random process with generally distributed discrete or continuous increments over a fixed level is considered in the article as an essential part of the analysis of a class of stochastic models (bulk queueing systems, inventory control and dam models). Using drect probability methods the authors find various characteristics of this problem: the magnitude of the first excess of the process over a fixed level, the shortage before the first excess, the levels of the first and prefirst excesses, the index of the first excess and others. The results obtained are llustrated by a number of numerical examples and then are applied to a bulk queueing system with a service delay discipline.


I. INTRODUCTION
In many controlled stochastic models encountered in applications (queues, inventories, dams), a control policy is employed in which a system is restricted in its capability to engage all or a part of its facilities until the total amount of accumulated "work" reaches or exceeds a certain con- trol level (or certain control levels).Some examples include systems with warm-up, orientation, hys- teresis service, and multilevel feedback control, bulk queueing systems with service delay discipline, queueing systems with server vacations, inventory control systems, and certain controlled dam mo- 84 LEV ABOLNIKOV and JEWGENI H. DSHALALOW A bulk queueing system acts similar to a service discipline in which the server can start a new service act only if, after service completion, it finds at least r (r > 1) units in the queue; otherwise, the server remains idle until the queue length reaches or exceeds level r.It is clear that a preliminary analysis of the first passage problem is necessary, and it is an essential part of any attempt to investigate the functioning of such a system.This fact is illustrated by the authors in [1], where a general control MX/GY/1 bulk queueing system with a service delay discipline of this kind is considered.
In the present article, the authors study a general first passage problem and its applications to the analysis of some stochastic models.Having originated from needs of reliability theory, the first passage problem is traditionally concerned with the distribution of the moment of the first passage (so-called "passage time") of a cumulative random process with single increments over a certain level.In this article, the authors, keeping in mind stochastic model applications, concentrate their attention on the other important aspect of the problem: the distribution of the value of the first excess of a cumulative process (with generally distributed increments) over a fixed level.This random variable is especially important in the analysis of queueing and inventory models with bulk input.In addition, the distribution of the shortage of the first excess, the levels of the first and pre- first excesses, and the moments of the first and pre-first excesses are found.The authors introduce various functionals of the above mentioned processes and manage to express them in terms of a cer- tain function called the "generator."Not only does this generator have a number of fine probabilistic qualities, but it turns out to be a polynomial which considerably facilitates the further analysis (for example, by using factorization methods).Some results obtained by a direct probability approach can be derived (as it was kindly suggested by Lajos Takcs to one of the authors) after some adjustments of more general results from Dynkin [4] and Takcs [5].However, for the sake of simplicity and for an illustration of the methods used in this article, the authors preferred to use in these cases their original proofs.The other results are new.A direct approach for finding a number of various characteristics of the first passage problem, which is developed in the article, is believed to be of independent methodological interest.
The results obtained in this paper are illustrated by a number of examples and, then, are applied to a bulk queueing system with a service delay discipline.

FORMULATION OF TIIE PROBLEM AND GENERAL RESULTS
All stochastic processes will be considered on a probability space {12,,P}.Let Z-n >_ o Sn rn (where a is the Dirac point mass) be a compound random counting measure, on (R %(R )) (where is the Sorel r-algebra) such that the counting measures, r = n >_ 0 ern +' + and S-OOn=oSn on (, %()), C_ R+, be delayed renewal processes, and such that the compound random measure Z be obtained from 7" by position independent marking.We will consider two different cases: 9 C_ N O and 9 R (equipped with the usual topology).Observe that, due to its analytic properties and an importance in applications to stochastic models (for example, embedded Markov chains in queueing theory), the discrete case is of main interest, and, consequent- ly, it will be discussed in greater detail than its continuous counterpart.Therefore, in this section we will study the "critical behavior" of a compound delayed renewal process, Z, determined by a delayed renewal process v = {r n t o + t 1 + + t n ;n > 0} on +, marked by a discrete-valued, delayed renewal process, S = {S n = X o + X 1 + + X n ;n >_ 0} on 9.As mentioned above, we assume that the processes 7" and S are independent.[In the case of their dependence we may arrive at more general results, but we have to indicate what kind of dependence is to be employed.This.will not be discussed in this paper.]Let X 0 = S O be distributed as PXo = ie giei and let X be distributed as PX1 = i aiei (both arbitrary atomic probabi- lity measures).The corresponding generating functions are denoted g(z)= E[zX] and a(z) = E[zX], analytic inside the open unit ball B(0,1) and continuous on its boundary 0B(0,1), with finite means y-E[X0] and -E[X].Without loss of generality we set t o -0.We also assume that inter-renewal times t n -r n-rn_, are described in terms of the common Laplace-Stieltjes transform V(O) E[e-Otn], with the finite mean = Z[tn] n = 1,2, For a fixed integer r > 1 we will be interested in the behavior of the process S and some related processes about the level r.
The following terminology is introduced and will be used throughout the paper.
(i) Denote v = inf{k >_ 0: S k > r} and call it the index of lhe first excess (above level r-1).
(ii) Call the random variable S v the level of the first excess (above r-1).
(iii) Call r/-Sr the magnitude of the first excess of S above level r-1.
(v) Denote the random variable r v as the first passage time when S exceeds level r-1.
(vi) Denote r r-Sy and call it the shortage before the first excess of S of level r.
We shall be interested primarily in the joint distributions of the first passage time and the random variables listed in 2.1 (i-v) in terms of the following functionals: 7r(0,z) = E[e r'z'], gr(0,z) = E[e or,zSU], g(r)(0,z) = E[e rUzS' ].
A very important property of Gr(z and Gr + (z) is that they are polynomials of (r-1)th degree.As mentioned in the introduction, this fact plays an important role in the analysis of stochastic proces- ses, specifically, it enables one to factor some functionals in polynomials and in known analytic functions.
The rationale behind the use of the term "generator of the first excess level" comes from the following major theorems and properties, which follow from corollary 2.3 and proposition 2.5.
2.6 Corollary.The "projective" generator Gr(z of the first excess has the following proper- ty: Gr(1 = ,.= E[u]. 2.7 Proposition.Gr" (0, z) can be obtained from the following formula: Then, by reasonings similar to those in the proof of theorem 2.2, we get The statement of the proposition follows.
2.8 Proposition.The functional ar + (0, z) can be obtained from the following formula: Proof.The proof of proposition 2.8 is an analog to that of proposition 2.7.
2.9 Theorem.The functional Or(O, z) of the first passage time and the first excess level can be expressed in terms of the generator of the first excess as r(O, z) = g(z) -[1 V(O)a(z)]ar(o z).
Proof.From The third sum in (2.9a)is obviously Gr(O,z less E[zSIur_(So)]-i=0Yiz', and the second sum in (2.9a) is G (0, z).The statement of the theorem follows from propositions 2.5 and 2.7.
2.10 Corollary.The mean value of the first excess level can be determined from the following formula: (2.10a) (Jr E[Su] -+ Proof.The validity of formula (2.10a) is due to corollary 2.6, theorem 2.9 and routine calculus.

F!
A First Passage ProbleJn and Its Applications 89 2.11 Remark.Observe that the "total magnitude" S v -S O of the first excess level has the mean value {}r-and, due to (2.10a), equals r, i.e., the product of the mean "batch" size and the mean value of the index.Thus, it seems as if we could proceed with Wald's formula to get the same result.However, it would be unjustified, since, in Sv, X1,...,X v are not independent of v (as trivial counterexamples show); consequently, a similar factorization of other functionals of S v S O is not possible.This tells us that Wald's equation apparently holds true for weaker sufficient conditions.
2.12 Theorem.The functional }(r)(, z) of the pre.first passage time and the pre-first excess level can be determined from the following .formula: where we define S~= 0 and r~= 0 on the set {S O > r}.
Proof.The functional 0(r)(0, z) can be decomposed as The expected value on the right-hand side of (2.12c) can be modified as follows.

CONTINUOUS-VALUED PROCESSES
In this section we obtain joint functionals of the first passage time and the first excess level above some positive real number s.We assume that Z = n >oSner is a compound random measure obtained from r by position independent marking, where S-]n >_ o esn is a counting measure on (R +, N(R + )) such that S is a delayed renewal process.We denote We evaluate {}s(0,tg) in terms of the Laplace transform by using methods similar to those in the previous section.And we let s=O e tSs(O,O)ds Re(t) > O.
In what follows we assume that g(z) z (i.e. S o -i a.s.).We then label the corresponding functionals of all discussed random variables with index "i." 4.2 Corollary.The joint functional 7!i)(O,z) of the first passage time and the index of the first excess level satisfies the following formula: Proof.From (2.2a) it directly follows that i<r i>r.
From formulas (4.2b-4.2c),we immediately obtain the mean value of the first excess index: (4.3) O, i>_r.
4.4 Corollary.The generating function i)(z) of the first excess level is determined by the following formula: (4.4b)By using change of variables in (4.4a) we can transform it into an equivalent expression Oi) Proof of corollary 4.4.Formula (4.4a) follows from (2.3a) by direct computations.Alterna- tively, formula (4.4a) or its equivalent (4.4b) can be derived from entirely different probability arguments that can be of independent interest.Our preliminary target is the generating function 3i)(z) of the magnitude r/--r/ i) of the first excess level with the above assumption that S O -i a.s.Since obviously r/!i)= r/!_) i, we can operate with r/! ) to determine !)(z).Then we shall return to the general case by restoring corres- ponding indices.Introduce the following notation: (4.4c) qns-P{Sn-s}, qno = O, s = .=0 q.s, 10 1, (4.4d) b,(z) E zmk, n,s 0,1 rn=l a n't" rn r-1 From direct probability arguments it follows that P{r/! ) k} s o lsar + k-s, and thus (4.4e) 50)(z) E r br_s_l(zll S 0 On the other hand, the series o n =obn(z)zn (with bn(z defined in (4.4d)) converges in the region LEV ABOLNIKOV and JEWGENI H. DSHALALOW (4.9b) where K(z) = fl(A-,a(z)),where fl(0), Re(O)> O, forms the Laplace-Stieltjes transform of the probability distribution function B, and !i) satisfies formula (4.4a) or (4.45).
It can be shown that A is reduced to a form of the homogeneous AR-matrix, which is a special case of a Am, n-matrix introduced and studied in [2].There the stochastic matrix A-(aij; i,j E = {0,1,...}) is called a homogeneous AR-malrix if it is of the form A = (aij" i,jE " aij = kj-i + r > R ,j > i-R aij = 0 > R, j < i-R), where ki is an atomic probabili- j=0 ty measure.
In [2] it was shown that the embedded queueing process Qn is ergodic if and only if cAb < R.Under this condition, the generating function P(z) of the invariant probability measure P of the operator A is determined by the following formula: 121 , ,} P(z) z R K(z) where //r,R) satisfies (4.9b).In addition, it was shown in [1] that the probabilities Po, "",PR form the unique solution of the following system of//linear equations: R dk I.