IJSA International Journal of Stochastic Analysis 2090-3340 2090-3332 Hindawi Publishing Corporation 961571 10.1155/2013/961571 961571 Research Article Some Limit Properties of the Harmonic Mean of Transition Probabilities for Markov Chains in Markovian Environments Indexed by Cayley's Trees Huang Huilin Hernandez Lerma Onesimo College of Mathematics and Information Science Wenzhou University Zhejiang 325035 China wzu.edu.cn 2013 5 12 2013 2013 07 08 2013 24 10 2013 31 10 2013 2013 Copyright © 2013 Huilin Huang. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

We prove some limit properties of the harmonic mean of a random transition probability for finite Markov chains indexed by a homogeneous tree in a nonhomogeneous Markovian environment with finite state space. In particular, we extend the method to study the tree-indexed processes in deterministic environments to the case of random enviroments.

1. Introduction

A tree  T  is a graph which is connected and doesn't contain any circuits. Given any two vertices  αβT, let  αβ¯  be the unique path connecting  α  and  β. Define the graph distance  d(α,β)  to be the number of edges contained in the path  αβ¯.

Let  T  be an infinite tree with root  0. The set of all vertices with distance  n  from the root is called the  nth generation of  T, which is denoted by  Ln. We denote by  T(n)  the union of the first  n  generations of  T. For each vertex  t, there is a unique path from  0  to  t and  |t|  for the number of edges on this path. We denote the first predecessor of  t  by 1t. The degree of a vertex is defined to be the number of neighbors of it. If every vertex of the tree has degree  d+1, we say it is Cayley’s tree, which is denoted by  TC,d. Thus, the root vertex has  d+1  neighbors in the first generation and every other vertex has  d  neighbors in the next generation. For any two vertices  s and t  of tree  T, write  st  if  s  is on the unique path from the root  0  to  t. We denote by  st  the farthest vertex from  0  satisfying  sts  and  stt. We use the notation  XA={Xt,tA}  and denote by  |A|  the number of vertices of  A.

In the following, we always let  T  denote the Cayley tree  TC,d.

A tree-indexed Markov chain is the particular case of a Markov random field on a tree. Kemeny et al.  and Spitzer  introduced two special finite tree-indexed Markov chains with finite transition matrix which is assumed to be positive and reversible to its stationary distribution, and these tree-indexed Markov chains ensure that the cylinder probabilities are independent of the direction we travel along a path. In this paper, we omit such assumption and adopt another version of the definition of tree-indexed Markov chains which is put forward by Benjamini and Peres . Yang and Ye extended it to the case of nonhomogeneous Markov chains indexed by infinite Cayley’s tree and we restate it here as follows.

Definition 1 (<italic>T</italic>-indexed nonhomogeneous Markov chains (see [<xref ref-type="bibr" rid="B10">4</xref>])).

Let  T  be an infinite Cayley tree, 𝒳  a finite state space, and  {Xt,tT}  a stochastic process defined on probability space  (Ω,F,P), which takes values in the finite set  𝒳. Let (1)p={p(i),i𝒳} be a distribution on  𝒳 and (2)Pt=(Pt(ji)),i,j𝒳, a transition probability matrix on  𝒳2. If, for any vertex  t, (3)P(Xt=jX1t=i,Xs=xsfor  ts1t)=P(Xt=jX1t=i)=Pt(ji),i,j𝒳,(4)P(X0=i)=p(i)i𝒳, then  {Xt,tT}  will be called 𝒳-valued nonhomogeneous Markov chains indexed by infinite Cayley’s tree with initial distribution (1) and transition probability matrices  {Pt,tT}.

The subject of tree-indexed processes has been deeply studied and made abundant achievements. Benjamini and Peres  have given the notion of the tree-indexed Markov chains and studied the recurrence and ray-recurrence for them. Berger and Ye  have studied the existence of entropy rate for some stationary random fields on a homogeneous tree. Ye and Berger [6, 7], by using Pemantle's result  and a combinatorial approach, have studied the Shannon-McMillan theorem with convergence in probability for a PPG-invariant and ergodic random field on a homogeneous tree. Yang and Liu  and Yang  have studied a strong law of large numbers for Markov chains fields on a homogeneous tree (a particular case of tree-indexed Markov chains and PPG-invariant random fields). Yang and Ye  have established the Shannon-McMillan theorem for nonhomogeneous Markov chains on a homogeneous tree. Huang and Yang  has studied the strong law of large numbers for finite homogeneous Markov chains indexed by a uniformly bounded infinite tree.

The previous results are all about tree-indexed processes in deterministic environments. Recently, we are interested in random fields indexed by trees in random environments. In the rest of this paper we formulate a model of Markov chain indexed by trees in random environment especially in Markovian environment and study some limit properties of the harmonic mean of random transition probability for finite Markov chains indexed by a homogeneous tree in a nonhomogeneous Markovian environment. We are also interested in the strong law of large numbers of Markov chains indexed by trees in random environments which we will prepare for in another paper.

Definition 2.

Let  T  be an infinite Cayley tree,  𝒳  and  Θ  two finite state spaces. Suppose that  {ξt,tT}  is a  Θ-valued random field indexed by  T, if, for any vertex  tT, (5)P(Xt=jX1t=i,Xs=xs,for  ts1t;ξr,rT)=Pξ1t(i,j),  a.s.,(6)P(X0=iξr,rT)=P(iξ0)i𝒳 for each  i,j𝒳, where Px=(Px(i,j))i,j𝒳, xΘ  is a family of stochastic matrices. Then we call  {Xt,tT}  a Markov chain indexed by tree  T  in a random environment  {ξt,tT}. The  ξts  are called the environmental process or control process indexed by tree T. Moreover, if  {ξt,tT}  is a T-indexed Markov chain with initial distribution  μ={μ(θ),θΘ}  and one-step transition probability matrices  Kt={Kt(α,β),α,βΘ,tT}, we call  {Xt,tT}  a Markov chain indexed by tree  T  in a nonhomogeneous Markovian environment.

Remark 3.

If every vertex has degree two, then our model of Markov chain indexed by homogeneous tree with Markovian environments is reduced to the model of Markov chain in Markovian environments which was introduced by Cogburn  in spirit.

Remark 4.

We also point out that our model is different from the tree-valued random walk in random environment (RWRE) that is studied by Pemantle and Peres  and Hu and Shi . For the model of RWRE on the trees that they studied, the random environment process  ω:=(ω(x,·),x,yT)  is a family of i.i.d nondegenerate random vectors and the process  X={Xn,nZ+}  is a nearest-neighbor walk satisfying some conditions. But in our model,our environmental process  ξ={ξt,tT}  can be any Markov chain indexed by trees. Given  ξ, the process  {Xt,tT}  is another Markov chain indexed by trees with the law  Pξ.

In this paper we assume that  {ξt,tT}  is a nonhomogeneous  T-indexed Markov chain on state space  Θ. The probability of going from  i  to  j  in one step in the  θth environment is denoted by  Pθ(i,j). We also suppose that the one-step transition probability of going from  α  to  β  for nonhomogeneous  T-indexed Markov chain  {ξt,tT}  is  Kt(α,β). In this case,  {ξt,Xt,tT}  is a Markov chain indexed by  T  with initial distribution  q=(q(θ,i))  and one-step transition on  Θ×𝒳 determined by (7)Pt(α,i;β,j)=Kt(α,β)Pα(i,j), where  q(θ,i)=P(ξ0=θ,X0=i). Then  {ξt,Xt,tT}  will be called the bichain indexed by tree  T. Obviously, we have (8)P(ξT(n)=αT(n),XT(n)=xT(n))=q(α0,x0)tT(n){0}Pt(α1t,x1t;αt,xt).

2. Main Results

For every finite  nN, let  {Xt,tT}  be a Markov chain indexed by an infinite Cayley tree  T  in Markovian environment  {ξt,tT}, which is defined as in Definition 2. Now we suppose that  gt(α,i,β,j)  are functions defined on  Θ×𝒳×Θ×𝒳. Let  λ  be a real number, L0={0},  n=σ(ξT(n),XT(n)); now we define a stochastic sequence as follows: (9)φn(λ,ω)=eλtT(n){0}gt(ξ1t,X1t,ξt,Xt)tT(n){0}E[eλgt(ξ1t,X1t,ξt,Xt)ξ1t,X1t]  . At first we come to prove the following fact.

Lemma 5.

{ φ n ( λ , ω ) , n , n 1 }    is a nonnegative martingale.

Proof of Lemma <xref ref-type="statement" rid="lem2.1">5</xref>.

Obviously, we have (10)P(ξLn=αLn,XLn=xLnξT(n-1)11=αT(n-1),XT(n-1)=xT(n-1))=P(ξT(n)=αT(n),XT(n)=xT(n))P(ξT(n-1)=αT(n-1),XT(n-1)=xT(n-1))=tLnP(ξt=αt,Xt=xtξ1t=α1t,X1t=x1t). Here, the second equation holds because of the fact that  {ξt,Xt,tT}  is a bichain indexed by tree  T  and (8) is being used. Furthermore, we have (11)E[eλtLngt(ξ1t,X1t,ξt,Xt)n-1]=αLn,xLneλtLngt(ξ1t,X1t,αt,xt)ggggg×P(ξLn=αLn,XLn=xLnξT(n-1),XT(n-1))=αLn,xLntLneλgt(ξ1t,X1t,αt,xt)ggggg×P(ξt=αt,Xt=xtξ1t,X1t)=tLn(αt,xt)Θ×𝒳eλgt(ξ1t,X1t,αt,xt)×P(ξt=αt,Xt=xtξ1t,X1t)=tLnE[eλgt(ξ1t,X1t,ξt,Xt)ξ1t,X1t]a.s. On the other hand, we also have (12)φn(λ,ω)=φn-1(λ,ω)×eλtLngt(ξ1t,X1t,ξt,Xt)tLnE[eλgt(ξ1t,X1t,ξt,Xt)ξ1t,X1t]. Combining (11) and (12), we get (13)E[φn(λ,ω)n-1]=φn-1(λ,ω)a.s. Thus, we complete the proof of Lemma 5.

Theorem 6.

Let  {Xt,tT}  be a Markov chain indexed by an infinite Cayley tree  T  in a nonhomogeneous Markovian environment  {ξt,tT}. Suppose that the initial distribution and the transition probability functions satisfy (14)q(α0,x0)>0,Pt(α,i;θ,j)>0,fffor  α0,α,θΘ;x0,i,j𝒳,at=min{Pt(α,i;θ,j),α,θΘ;i,j𝒳},tT{0}, if there exist two positive constants  c  and  m  such that (15)limsupn1|T(n)|tT(n){0}ec/at=m<. Denote  |Θ|=M, |𝒳|=N, and (16)Pt(ξ1t,X1t;ξt,Xt)=Pt(ξt,Xtξ1t,X1t); then we have (17)limn|T(n)|tT(n){0}Pt(ξ1t,X1t;ξt,Xt)-1=1MNa.s.

Proof.

By Lemma 5, we have known that  {φn(λ,ω),n,n1}  is a nonnegative martingale. According to Doob martingale convergence theorem, we have (18)limnφn(λ,ω)=φ(λ,ω)< a.s., so that (19)limnlnφn(λ,ω)|T(n)|=0a.s., which implies that (20)limsupnlnφn(λ,ω)|T(n)|0a.s. We arrive at (21)limsupn1|T(n)|×tT(n){0}{λgt(ξ1t,X1t,ξt,Xt)eλgt(ξ1t,X1t,ξt,Xt)hhhhhhhfk.h-ln[E[eλgt(ξ1t,X1t,ξt,Xt)ξ1t,X1t]]}0a.s. Combining (20) with the inequalities  lnxx-1(x>0)  and  0ex-1-x2-1x2e|x| and taking  gt(ξ1t,X1t,ξt,Xt)=Pt(ξ1t,X1t;ξt,Xt)-1, it follows that (22)limsupn1|T(n)|tT(n){0}[Pt(ξ1t,X1t;ξt,Xt)-1λ-λMN]limsupn1|T(n)|×tT(n){0}{ln[E[eλPt(ξ1t,X1t;ξt,Xt)-1ξ1t,X1t]]gggggggggg.[E[eλPt(ξ1t,X1t;ξt,Xt)-1ξ1t,X1t]]-λMN}limsupn1|T(n)|×tT(n){0}{E[eλPt(ξ1t,X1t;ξt,Xt)-1ξ1t,X1t]hhhhhhhhhhhh[eλPt(ξ1t,X1t;ξt,Xt)-1ξ1t,X1t]-1-λMN}limsupn1|T(n)|×tT(n){0}αΘi𝒳Pt(ξ1t,X1t;α,i)×[eλ/Pt(ξ1t,X1t;α,i)-1-λPt(ξ1t,X1t;α,i)]λ22limsupn1|T(n)|×tT(n){0}αΘi𝒳1Pt(ξ1t,X1t;α,i)e|λ|/Pt(ξ1t,X1t;α,i)λ2MN2limsupn1|T(n)|tT(n){0}1ate|λ|/ata.s. Note that the following elementary fact holds: (23)maxx>0{xγx}=-e-1lnγ,0<γ<1. Let  0<λ<c. It follows from (15), (22), and (23) that (24)limsupn1|T(n)|tT(n){0}[Pt(ξ1t,X1t;ξt,Xt)-1-MN]λMN2limsupn1|T(n)|tT(n){0}1ateλ/at=λMN2limsupn1|T(n)|tT(n){0}1at(eλ-c)1/atec/atλMN2e(c-λ)ma.s. Here, the second equation holds because by letting  λ0+  in inequality (24), we get (25)limsupn1|T(n)|×tT(n){0}[Pt(ξ1t,X1t;ξt,Xt)-1-MN]0a.s. If  -c<λ<0, similar to the analysis of inequality (24), by using (15), (22), and (23) again, we can arrive at (26)liminfn1|T(n)|tT(n){0}[Pt(ξ1t,X1t;ξt,Xt)-1-MN]λMN2limsupn1|T(n)|tT(n){0}1ate-λ/atλMN2limsupn1|T(n)|tT(n){0}1at(e-λec)1/atec/atλMN2e(λ+c)ma.s. Letting  λ0-  in inequality (26), we get (27)liminfn1|T(n)|×tT(n){0}  [Pt(ξ1t,X1t;ξt,Xt)-1-MN]0a.s. Combining (25) and (27), we obtain that our assertion (17) is true.

Corollary 7.

Let  {Xt,tT}  be a nonhomogeneous Markov chain indexed by an infinite Cayley tree  T. Suppose that the initial distribution and the transition probability functions satisfy (28)p(x0)>0,Pt(i,j)>0,at=min{Pt(i,j),i,j𝒳},tT{0}, if there exist two positive constants  c  and  m  such that (29)limsupn1|T(n)|tT(n){0}ec/at=m<. Denote  |𝒳|=N; then we have (30)limn|T(n)|tT(n){0}Pt(XtX1t)-1=1Na.s.

Proof.

If we take  Θ={θ}, that is,  |Θ|=1, then the model of Markov chain indexed by tree  T  in Markovian environment reduces to the formulation of a nonhomogeneous Markov chain indexed by tree  T. Then we arrive at our conclusion (30) directly from Theorem 6.

Corollary 8 (see [<xref ref-type="bibr" rid="B12">15</xref>]).

Let  {Xn,n0}  be a Markov chain in a nonhomogeneous Markovian environment  {ξn,n0}. Suppose that the initial distribution and the transition probability functions satisfy (31)q(α0,x0)>0,Pn(α,i;θ,j)>0,an=min{Pn(α,i;θ,j),α,θΘ;i,j𝒳},n1, if there exist two positive constants  c  and  m  such that (32)limsupn1nk=1nec/ak=m<. Denote  |Θ|=M, |𝒳|=N, and (33)Pk(ξk-1,Xk-1;ξk,Xk)=Pk(ξk,Xkξk-1,Xk-1); then we have (34)limnnk=1nPk(ξk-1,Xk-1;ξk,Xk)-1=1MNa.s.

Proof.

If every vertex of the tree  T  has degree  2, then the nonhomogeneous Markov chain indexed by tree  T  degenerates into the nonhomogeneous Markov chain on line; thus, this corollary can be obtained from Theorem 6 directly.

Acknowledgment

This work was supported by the National Natural Science Foundation of China no. 11201344. The author declares that there is no conflict of interests regarding the publication of this paper.

Kemeny J. G. Snell J. L. Knapp A. W. Denumerable Markov Chains 1976 2nd New York, NY, USA Springer xii+484 MR0407981 Spitzer F. Markov random fields on an infinite tree The Annals of Probability 1975 3 3 387 398 MR0378152 ZBL0313.60072 Benjamini I. Peres Y. Markov chains indexed by trees The Annals of Probability 1994 22 1 219 243 MR1258875 10.1214/aop/1176988857 ZBL0793.60080 Yang W. Ye Z. The asymptotic equipartition property for nonhomogeneous Markov chains indexed by a homogeneous tree IEEE Transactions on Information Theory 2007 53 9 3275 3280 10.1109/TIT.2007.903134 MR2417693 Berger T. Ye Z. X. Entropic aspects of random fields on trees IEEE Transactions on Information Theory 1990 36 5 1006 1018 10.1109/18.57200 MR1066588 ZBL0738.60100 Ye Z. Berger T. Ergodicity, regularity and asymptotic equipartition property of random fields on trees Journal of Combinatorics, Information & System Sciences 1996 21 2 157 184 MR1738533 ZBL0953.94008 Ye Z. Berger T. Information Measures for Discrete Random Fields 1998 Beijing, China Science Press iv+160 MR1672220 Pemantle R. Automorphism invariant measures on trees The Annals of Probability 1992 20 3 1549 1566 MR1175277 10.1214/aop/1176989706 ZBL0760.05055 Yang W. Liu W. Strong law of large numbers for Markov chains field on a Bethe tree Statistics & Probability Letters 2000 49 3 245 250 10.1016/S0167-7152(00)00053-5 MR1794741 ZBL0981.60064 Yang W. Some limit properties for Markov chains indexed by a homogeneous tree Statistics & Probability Letters 2003 65 3 241 250 10.1016/j.spl.2003.04.001 MR2018036 ZBL1068.60045 Huang H. Yang W. Strong law of large numbers for Markov chains indexed by an infinite tree with uniformly bounded degree Science in China A 2008 51 2 195 202 10.1007/s11425-008-0015-1 MR2395461 ZBL1141.60011 Cogburn R. Markov chains in random environments: the case of Markovian environments The Annals of Probability 1980 8 5 908 916 MR586775 10.1214/aop/1176994620 ZBL0444.60053 Pemantle R. Peres Y. Critical random walk in random environment on trees The Annals of Probability 1995 23 1 105 140 MR1330763 10.1214/aop/1176988379 ZBL0837.60066 Hu Y. Shi Z. Slow movement of random walk in random environment on a regular tree The Annals of Probability 2007 35 5 1978 1997 10.1214/009117906000001150 MR2349581 ZBL1124.60083 Li Y. Q. Wang H. S. Wang Z. Some strong limit theorems of geometric average of transition probabilities and their function for Markov chains in Markovian environments Acta Mathematica Scientia A 2011 31 2 508 517 MR2828040