MARKOV CHAINS WITH QUASITOEPLITZ TRANSITION MATRIX : FIRST ZERO HITTING

This pnper continues the investigation of Markov Chains with a quasitoeplitz transition matrix. Generating functions of first zero hitting probabilities and mean times re found by the solution of special t’iemann boundary value problems on the unit circle. Duality is discussed. AMS Subject Classification: 60J, 60K25.

Here we continue the investigation started in [1] of Markov chains with a quasitoeplitz transition matrix, elements of which depend on the difference of indices except for the first column and the flint several rows.The primary focus of this paper is the problem of first zero hitting (FZH) in terms of generating functions (GF) of probabilities and mean times.
The practical importance of this problem is in its obvious connection vith the problems of first emptying and response time arising in applica- tions.There is also another reason for investigation: duality of this problem and state probabilities evaluation.
The FZH problem was only studied in that simplest particular case when the transition matrix was in fact toeplitz (except, of course, in the first column).In this case FZH is in fact a problem of the first entering nonpositive area for some random walk on the integers, so relevant GF's were found either in a purely probabilistic manner (see, for example, [2]), or by the solution of a Riemann boundary value problem (for example, Dukhovny [3] for a random walk that depends on a parameter).
In this paper we study a quasitoeplitz case arising in applications with feedback, so there is no structure of a random walk at some early states.
We transform the FZH problem to nonuniform Riemann boundary value problerns for the GF's of probabilities and mean times.As the sums of these do not converge, we deal in fact with boundary problems in a wide sense, that is in special Banach spaces (see [4]).
We also prove here that FZH mea.n times do not exceed some linear function of a starting state's number.
Having solved the boundary value problems, we obtain some finite sys- tems of linear equations and prove that they have only one solution.
In conclusion we discuss a duality between transient a.nd steady state probabilities from one side, and FZH probabilities and mean times' incre- ments, respectively, from the other, imposed by the method of bounda.ryvalue problems.
We retain the notation of [1] for Ai(z) =-jOO=oaijzJ, H(z) = ,_ hjz j, level n such that 'v'i > n aij =-hi_i, j > 0; aio = [i hi; operatores T + and Tthat cut out pa.rts wit;h nonnegative and negative exponents of Laurent power series, D that gets the ith coecieng of the series, space L of Lauren power series with absolutely sumnmble coefficients, unig circle r: Izl-x, its interior r + and exterior F-.
Let us introduce q/', i > O, k > O, the probability of rea.ching zero for the first time in k seps st.aring from i, and #i _,-_ kq, the mean number of steps in the FZH roue s:art:ing from i, which is known to converge if the chain is ergodic.We also denote q = -1, i > O.
Let x (0,1), Izl > 1, and denote q,(x We introduce the linear Banach space Loo of formal Laurent power series with bounded coefficients with a norm and a formal product of series (convolution) provided the absolute convergence of sums Z ai_jbj, Note that if a(z) L and b(z) Loo then their convolution obviously belongs to L.
Whatever i is, a total FZH probability ]a__ x q does not exceed 1, so Denote e(z)= Ei__l z-i e Leo.
Every FZH route from i > 0 is either a direct step to zero or consists of one step to some j > 0 and a FZH route fi'om j, so Multiplying both sides by z k+ obtain and sunanaing over k from 0 to c we () q,() = ,,,q(.)+ ( ) i>0 or, in vector form, (4) where 1 = (0,1, 1,...), and A differs from A only by elements of the first row and column, which are all zeros except a00 = 1.
Proceeding from (4) we noxv find the GF Q(x, z).Theorem 1.The GF Q(x, z), x (0,1), [z[ > 1, of the first zero hitting probabilities of the Markov chain { fk } with a quasitoeplitz transition matrix with n > 0 is defined by the following formulas: (5 and the unknowns qi(x), i = 1,2,..., n-I are the only solution of the system of linear equations (7) (n = 0 and n = 1 obviously generate the same FZH problems).
(8) Proof.According to the definitions made j=l Note that the convolutions Ai(z)Q(z, z) and H(z)Q(z, z) exist and be- long to L as A(z) and H(z) by definition belong to L. Therefore, the subset of equalities (3) where n-1 = i=1 = Ve transform (9) to the formal equality of power series (10) In (I0) the power series Q(x,z) and F+(x,z) correspond to functions analytic in I'and r'+, respectively; but they need not converge on r', so the equality (I0) is the formal equality of the power series, not their sums.Therefore (I0) must be treated as a Riemann boundary value problem in a wide sense.
Had (7) another solution, then (5) would generate another ,(x,z) sat- isfying (7) and (9).As R-(x, z)-' De.longs to L, (z,z) defined by (5) belongs to L. Therefore, the vector Q(x) of coefficients of its Lattrent ex- pansion, thag sagisfies (4:) because of (7) and (9), is a column wigh bounded elements.As matrix A is sgochast;ic, the norm of the operator A in the Banach space of columns with bounded elements does no exceed Ixl < 1, so (4) has the only solution in this space.Hence Q(x) and Q(x) mus be equal.'1"his completes the proof of Theorem 1.
In his section we estimate the order of growth for FZH mean imes depeding on the mmber of the starting state.This estimate makes it possible to fid #(z) by he same me hod used for Q(x, z), ensuring the existence of all necessary convoluios.Let s, be the set of states O, 1,..., m; d'(i,j), i > m, j < m, be the first-descen-to-s, route from i to j, and Dm(i,j) be the set of all such routes.We introduce functions N(d), the number of steps in the route d, and PR(d), the probability of this route.Obviously, if d is a composition of dx and d: then  Proof.Every FZH route from i is d(i, 0) and obviously is either d'(i, O) or a conposition of d'(i,j) and d(j, 0), j 1,2,..., m.Now (18) readily follows from (16) and (17).
Let us apply (18) to the chain {a} with n = 1 and denote here , instead of #i.For rn = I we have here (19) = i+1 "q-A1 Xi+l,1 Lemma a.The FZH mean times for the ergodic Markov chain {} with a quasitoeplitz transition matrix with n = 1 satisfy the inequality (20) , < + < + i > 0.
Proof.Note that for this chain , 0i_, _ i > m > 0 because all the transitions bove zero depend on the difference of states.
Hence A A so (19)leads to (20) i+1 i Now we are back to a.n arbitrary n > 0. Lemma 4. The FZH mean times for the ergodic Markov chain with quasitoeplitz transition matrix do not exceed some linear function of starting state's numbers.
Proof.As the transition matrix is quasitoeplitz, all the transitions above n-1 depend on differece of states, so Hence when m = n-1 (18) reduces to j=l Note that the sum of coefficients at p on the right side is less than the total probability of first descent to s_, i.e., less than 1; because of (20) Ai-n+x < (i-n + 1).Hence which proves the statement of Lemma 4. Let introduce pi-#i-a i > n i=n Corollary.(z) L. 5. FZH MEAN TIMES: THE GENERATING FUNCTION.
Theorem 2. The GF #(z), Izl > 1, of the FZH mean times for the ergodic Markov chain {} with a quasitoeplitz transition matrix with "natural properties" (see [1], pp.72-7a) is defined by the formulas n-1 R=I:(z) = exp (-T +/-In z e(z)[1 H(z)]) where the tmknown #i, i = 1,2,..., n-1, n > 0, are the only solution of the system of linear equations Proof.Every FZH or first-return-to-zero route is either one step to zero or consists of one step to some j > 0 and then a FZH route from j. Hence for all i #i = aij(#j + 1) + aio = aij#j 21-1.
j=l j=l For i > n-1 we can rewrite (26) as (27 According to the definition of {vi} we have (28) n-1 = ,,z + i=1 Therefore, multiplying both sides of (27) by z --i and summing over i from n to cx we obtain z(z)e(z) T-z-l(z)H(z) + e(z).
In conclusion we discuss the relationship of duality between transient and steady state probabilities from one side, and FZH probabilities and mean times increments, respectively, from the other.This relationship is imposed in a naturM way by the method of Riemann boundary value problems.