REMARKS ON A MONOTONE MARKOV CHAIN *

In applications, considerations on stochastic models often 
involve a Markov chain { ζ n } 0 ∞ with state space in R + , and a transition 
probability Q . For each x R + the support of Q ( x , . ) is [ 0 , x ] . 
This implies that ζ 0 ≥ ζ 1 ≥ … . Under certain regularity assumptions 
on Q we show that Q n ( x , B u ) → 1 as n → ∞ for all 0$" xmlns:mml="http://www.w3.org/1998/Math/MathML"> u > 0 and 
that 1 − Q n ( x , B u ) ≤ [ 1 − Q ( x , B u ) ] n where B u = [ 0 , u ) . Set τ 0 = max { k ; ζ k = ζ 0 } , τ n = max { k ; ζ k = ζ τ n − 1 + 1 } and write X n = ζ τ n − 1 + 1 , T n = τ n − τ n − 1 . We investigate some properties of the imbedded 
Markov chain { X n } 0 ∞ and of { T n } 0 ∞ . We determine all the marginal 
distributions of { T n } 0 ∞ and show that it is asymptotically stationary 
and that it possesses a monotonicity property. We also prove that 
under some mild regularity assumptions on β ( x ) = 1 − Q ( x , B x ) , 
 ∑ 1 n ( T i − a ) / b n → d Z ∼ N ( 0 , 1 ) .

Various applications often involve a Markov chain {n}0 with state space in R+, an initial distribution t and a transition probability Q.The t and Q determine completely R and uniquely a probability measure Pt on the measure space { +, c-B (R)} We shall write Px instead of Pit when bt is the Dirac measure e x (') on c-B (R+).
Corresponding to the probability measure Pit is the expectation operator Eit.
From (1.1) it follows that Q(x, ") is an atomic probability measure whose only atom is {x}.
Markov processes of this type arise in various applications.A particular problem which motivated consideration of {n}0 is related to modeling of the long term effect of close up surface soil erosion on crop production.It has been shown (Todorovic et al.,   1987 and Todorovic and Gani., 1987) that the sequence of annual yields {Xn} 0 of a crop, from a crop growing area subject to soil erosion, is specified as Here {Yn}0 is an i.i.d, sequence of positive r.v.s and {Ln} 0 a Markov process independent of {Yn} and such that L 0 > L > L 2 > Let 0 be the depth of top soil of the crop growing area at time 0; let 1 be its depth one year later, and so on.We clearly have that 0 > 1 > One can also show that L n q)(n ), where q0" R+-+[0, 1] is strictly increasing, continuous on [0, do) and such that q0(0) 0 and q0(u) for u >_ d 0.
Due to the fact that n q-l(Ln)' it seems reasonable to assume that {n} 0 represents a Markov process with transition probability Q satisfying regularity conditions A A 4. The atom {x} of Q(x,') reflects the fact that there may be no soil loss due to erosion in a particular year.On the other hand, the smaller depth of the top soil layer the larger is the probability of reaching a lower level y.This justifies the monotonicity assumption A 4. Stochastic monotonicity is a very common phenomenon.
Random walks, diffusions, birth and death and branching processes have this property.
Another problem where the process {n } 0 can be used is to model the decrease of a dam's capacity due to silting.Denote by T O the time the chain stays in its initial state.Let T be the duration of its stay in the next state, and so on.We investigate a number of properties of the sequence {Tn}0.We determine all its marginal distributions, and show that this process is asymptotically stationary and that it possesses a mixing property.We also show that under certain regularity assumptions on [3(') Here we discuss some basic features of the n-step transition probability Qn and of (2.1) which follow from assumptions A A 4. First of all, from A 3 we have As usual, we define the n-step transition probabilities by setting Ql(x, B) Q(x, B) and for n > 2 (2.3) Qn (x, B) I Q(s, B) Qn-1 (x, ds), where B B x c-B (R+).From this we clearly have: (2.4) Qn(x,{x}) ([3(x)) n.

(u,x]
This and (1.4) yield: from which we obtain the following recursion U which proves (2.5).
which gives some information concerning the rate of convergence Proposition 2.2 The function 3(x) is continuous at every x > 0. Proof: From (1.4) and (2.1) we have: Hence, (0 Letting Xl " x2 we obtain" 13(x-0)-13(x) 0 for all x > 0, which proves that (x) is left continuous.
Remark 2.1 From the last inequality and condition (1.4) we have Inequality (2.2) implies that the closer the chain is to the state {0}, the larger is the probability that the next step will lead to the same state.However, it seems reasonable to assume that regardless of how close the chain is to the zero state, the probability of landing somewhere in [0, x), given that it was in x, is a positive number.This gives rise to the following assumption: (2.11) 13(0+) 1 lim Q(x, Bx) p < 1. x--)0
The pn can be interpreted as a non-negative linear operator on the cone of non- negative Borel functions by defining: (3.4) (pn h) (x) f h(u) pn (x, du).
Remark 4.1 Both functions on the right-hand side of (4.8) and (4.9) are monotone.