On the Study of Transience and Recurrence of the Markov Chain Defined by Directed Weighted Circuits Associated with a Random Walk in Fixed Environment

1 ), r > 1, are called circuit chains. Following the context of the theory of Markov processes’ cycle-circuit representation, the present work arises as an attempt to investigate proper criterions regarding the properties of transience and recurrence of the corresponding Markov chain represented uniquely by directed cycles (especially by directed circuits) and weights of a random walk with jumps (having one elastic left barrier) in a fixed ergodic environment (Kalpazidou [1], Derriennic [5]). The paper is organized as follows. In Section 2, we give a brief account of certain concepts of cycle representation theory of Markov processes that we will need throughout the paper. In Section 3, we present some auxiliary results in order to make the presentation of the paper more comprehensible. In particular, in Section 3, a randomwalk with jumps (having one elastic left barrier) in a fixed ergodic environment is considered, and the unique representations by directed cycles (especially by directed circuits) and weights of the corresponding Markov chain are investigated. These representations will give us the possibility to study proper criterions regarding transience and recurrence of the abovementioned Markov chain, as it is described in Section 4. Throughout the paper, we will need the following notations:

Following the context of the theory of Markov processes' cycle-circuit representation, the present work arises as an attempt to investigate proper criterions regarding the properties of transience and recurrence of the corresponding Markov chain represented uniquely by directed cycles (especially by directed circuits) and weights of a random walk with jumps (having one elastic left barrier) in a fixed ergodic environment (Kalpazidou [1], Derriennic [5]).
The paper is organized as follows.In Section 2, we give a brief account of certain concepts of cycle representation theory of Markov processes that we will need throughout the paper.In Section 3, we present some auxiliary results in order to make the presentation of the paper more comprehensible.In particular, in Section 3, a random walk with jumps (having one elastic left barrier) in a fixed ergodic environment is considered, and the unique representations by directed cycles (especially by directed circuits) and weights of the corresponding Markov chain are investigated.These representations will give us the possibility to study proper criterions regarding transience and recurrence of the abovementioned Markov chain, as it is described in Section 4.
Throughout the paper, we will need the following notations:
The smallest integer  ≡ () ≥ 1 satisfying the equation ( + ) = (), for all  ∈ , is the period of .A directed circuit  such that () = 1 is called a loop.(In the present work we will use directed circuits with distinct point elements.) Let us also consider a directed circuit  (or a directed cycle ĉ) with period () > 1.Then we may define by  ()   (, ) = 1, if there exists an  ∈  such that  =  () ,  =  ( + ) , = 0, otherwise, the -step passage function associated with the directed circuit , for any ,  ∈ ,  ≥ 1. Furthermore we may define by   () = 1, if there exists an  ∈  such that  =  () , = 0, otherwise, the passage function associated with the directed circuit , for any  ∈ .The above definitions are due to MacQueen [2] and Kalpazidou [1].Given a denumerable set  and an infinite denumerable class  of overlapping directed circuits (or directed cycles) with distinct points (except for the terminals) in  such that all the points of  can be reached from one another following paths of circuit edges; that is, for each two distinct points  and  of  there exists a finite sequence  1 , . . .,   ,  ≥ 1, of circuits (or cycles) of  such that  lies on  1 and  lies on   , and any pair of consecutive circuits (  ,  +1 ) have at least one point in common.Generally we may assume that the class  contains, among its elements, circuits (or cycles) with period greater or equal to 2.
With each directed circuit (or directed cycle)  ∈  let us associate a strictly positive weight   which must be independent of the choice of the representative of , that is, it must satisfy the consistency condition  cot  =   ,  ∈ , where   is the translation of length  (that is,   () ≡  + ,  ∈ , for any fixed  ∈ ).
For a given class  of overlapping directed circuits (or cycles) and for a given sequence (  ) ∈ of weights we may define by the elements of a Markov transition matrix on , if and only if ∑ ∈   ⋅   () < ∞, for any  ∈ .This means that a given Markov transition matrix  = (  ), ,  ∈ , can be represented by directed circuits (or cycles) and weights if and only if there exists a class of overlapping directed circuits (or cycles)  and a sequence of positive weights (  ) ∈ such that the formula (5) holds.In this case, the Markov transition matrix  has a unique stationary distribution  which is a solution of  =  and is defined by It is known that the following classes of Markov chains may be represented uniquely by circuits (or cycles) and weights: (i) the recurrent Markov chains (Minping and Min [3]), (ii) the reversible Markov chains.
Furthermore we consider also the "adjoint" Markov chain (   ) ≥0 on ℵ whose elements of the corresponding Markov transition matrix are defined by such that    +    +    = 1, 0 <    ,    ,    ≤ 1, for every  ≥ 1, as it is shown in (Figure 2).
Consequently we have the following.
Proposition 1.The Markov chain (  ) ≥0 defined as above has a unique representation by directed cycles (especially by directed circuits) and weights.
Proof.Let us consider the set of directed circuits   = (,  + 1, ) and    = (, ), for every  ≥ 0, since only the transitions from  to  + 1,  to  − 1, and  to  are possible.There are three circuits through each point  ≥ 1,  −1 ,   , and    and two circuits through 0:  0 ,   0 .The problem we have to manage is the definition of the weights.We may symbolize by   the weight    of the circuit c  and by    the weight     of the circuit    , for any  ≥ 0. The sequences (  ) ≥0 , (   ) ≥0 must be a solution of Let us take by

As a consequence we may have
Figure 2 Given the sequences (  ) ≥0 and (  ) ≥0 , it is clear that the above sequences (  ) ≥1 and (  ) ≥1 exist and are unique.This means that the sequences (  ) ≥0 and (   ) ≥0 are defined uniquely, up to multiplicative constant factors, by (The unicity is understood up to the constant factors  0 ,   0 .)Proposition 2. The "adjoint" Markov chain (   ) ≥0 defined as above has a unique representation by directed cycles (especially by directed circuits) and weights.
Proof.Following an analogous way of that given in the proof of Proposition 1 the problem we have also to manage here is the definition of the weights.
For given sequences (   ) ≥0 , (   ) ≥0 it is obvious that (  ) ≥1 , (  ) ≥1 exist and are unique for those sequences, that is, the sequences (   ) ≥0 , (   ) ≥0 are defined uniquely, up to multiplicative constant factors, by (ii) The Markov chain (   ) ≥0 defined as above is positive recurrent if and only if In order to obtain recurrence and transience criterions for the Markov chains (  ) ≥0 and (   ) ≥0 we shall need the following proposition (Karlin and Taylor [6]).Proposition 4. Let us consider a Markov chain on ℵ which is irreducible.Then if there exists a strictly increasing function that is harmonic on the complement of a finite interval and that is bounded, then the chain is transient.In the case that there exists such a function which is unbounded then the chain is recurrent.
To this direction we may symbolize by    the weight     of the circuit    and by    the weight     of the circuit    , for every  ≥ 0. The sequences (   ) ≥0 and (   ) ≥0 must be solutions of By considering the sequences (  )  and (  )  where   =   −1 /   ,   =    /  −1 ,  ≥ 1, we may obtain that

Recurrence and Transience of the Markov Chains
(  ) ≥0 and (   ) ≥0We have that for the Markov chain (  ) ≥0 , there is a unique invariant measure up to a multiplicative constant factor  =  −1 +   +    ,  ≥ 1,  0 =  0 +   0 , while for the Markov chain (   ) ≥0 ,    =   −1 +   +   ,  ≥ 1 with   0 =   0 +  0 .In the case that an irreducible chain is recurrent there is only one invariant measure (finite or not), so we may obtain the following.(i) The Markov chain (  ) ≥0 defined as above is positive recurrent if and only if (The unicity is based to the constant factors   0 ,   0 .)4.