A decomposable branching process in a Markovian environment

A population has two types of individuals, each occupying an island. One of those, where individuals of type 1 live, offers a variable environment. Type 2 individuals dwell on the other island, in a constant environment. Only one-way migration (1->2) is possible. We study the asymptotics of the survival probability in critical and subcritical cases.


Introduction
Multi-type branching process in random environment is a challenging topic with many motivations from population dynamics (see e.g. [9,18,22]). Very little is known in the general case and in this paper we consider a particular two-type branching process with two key restrictions: the process is decomposable and the final type individuals live in a constant environment.
The subject can be viewed as a stochastic model for the sizes of a geographically structured population occupying two islands. Time is assumed discrete, so that one unit of time represents a generation of individuals, some living on island 1 and others on island 2. Those on island 1 give birth under influence of a randomly changing environment. They may migrate to island 2 immediately after birth, with a probability again depending upon the current environmental state. Individuals on island 2 do not migrate and their reproduction law is not influenced by any changing environment. Our main concern is the survival probability of the whole population.
An alternative interpretation of the model under study might be a population (type 1) subject to a changing environment, say in the form of a predator population of stationary but variable size. Its individuals may mutate into a second type, no longer exposed to the environmental variation (the predators do not regard the mutants as prey). Our framework may be also suitable for modeling early carcinogenesis, a process in which mutant clones repeatedly arise and disappear before one of them becomes established [21,23]. See [7] for yet another possible application.
The model framework is that furnished by Bienaymé-Galton-Watson (BGW) processes with individuals living one unit of time and replaced by random numbers of offspring which are conditionally independent given the current state of the environment. We refer to such individuals as particles in order to emphasize the simplicity of their lives. Particles of type 1 and 2 are distinguished according to the island number they are occupying at the moment of observation. Our main assumptions are: • particles of type 1 form a critical or subcritical branching process in a random environment, • particles of type 2 form a critical branching process which is independent of the environment.
Let X n and Z n be the numbers of particles of type 1 and of type 2, respectively, present at time n. Throughout this paper it is assumed (unless otherwise specified) that X 0 = 1 and Z 0 = 0. We investigate asymptotics of the survival probability P [X n + Z n > 0] as n → ∞. In all cases addressed here we have Therefore, in view of we focus on the asymptotic behavior of P [Z n > 0]. In Section 2 we recall known facts for constant environments. They will then be compared to the results of this paper on random environments. In Section 3.1 we describe IID environments (Independent and Identically Distributed environmental states), and then in Section 3.2 Markovian environments. The main results of the paper are • Theorem 2 in Section 4 on the critical case with an IID environment, • Theorem 4 in Section 5 on the subcritical case with an IID environment, • Theorem 5 in Section 6 on the critical case with a Markovian environment, • Theorem 6 in Section 7 on subcritical case with a Markovian environment.
Theorems 5 and 6 treating the case of Markovian environment are extensions of Theorems 2 and 4 obtained under rather restrictive conditions and yielding qualitatively the same asymptotic behavior as in the case of IID environment.
Notation: in asymptotic formulae constants denoted by the same letter c are always assumed to be fixed and independent of the parameter that tends to infinity (or zero).

Two-type decomposable branching processes
Consider a two-type BGW-process initiated at time zero by a single individual of type 1. We focus on the decomposable case where type 1 particles may produce particles of types 1 and 2 while the type 2 particles can give birth only to type 2 particles. Put • Y n := the number of type 2 daughters produced by the particles of type 1 present at time n, in particular, Y 0 = Z 1 , • T := the first time n when X n = 0, so that {T > n} = {X n > 0}, • S n := n−1 k=0 X k , so that S T gives the total number ever of type 1 particles.
• W n := n−1 k=0 Y k , so that W T gives the total number of type 2 daughters produced by all S T particles of type 1.
The aim of this section is to summarize what is already known about such branching processes in the case of a constant environment. This will pave our way in terms of notation and basic manipulation with generating functions towards branching processes in IID random and then Markovian environments.
If the environment is constant from generation to generation, two-type decomposable BGW-processes are fully described by a pair of probability generating functions where ξ 1 and ξ 2 represent the numbers of daughters of type 1 and 2 of a mother of type 1, while η stands for the number of daughters (necessarily of type 2) of a mother of type 2. Let be the first two moments of the reproduction laws. Concerning the second type of particles we assume that implying that the probability of extinction (of a single-type BGW-process evolving in constant environment with the probability generating function h(s)) satisfies [8, Ch. I.9] It follows that We will be interested in two kinds of reproduction regimes for particles of type 1, critical and subcritical. In the constant environment setting with µ 2 ∈ (0, ∞), the critical case corresponds to µ 1 = 1 and the subcritical case is given by µ 1 ∈ (0, 1). In the critical case with a constant environment we have and according to [16, Theorem 1] Next we outline a proof of (6) based on the representation preparing for the proof in the random environment case, to be given in Section 4. Thanks to (5) and (1), it is enough to verify that in order to prove (6). However, by the branching property the total progeny of a single-type branching process S T is 1 plus ξ 1 independent daughter copies of S T . In terms of the Laplace transform . As λ → 0, a Taylor expansion of f (φ(λ), 1) as a function of 1 − φ(λ) yields For µ 1 = 1, after removing the negligible terms, we get a quadratic equation whose solution shows that Replacing λ by a n and using (4), we obtain It remains to verify, see (7), that This holds, indeed, since by (5) and for any fixed ǫ > 0 the probability P[T > nǫ] is much smaller than the target value of order c/ √ n. (In [27] and [28] infinite second moments in decomposable two-type critical processes were allowed.) On the other hand, in the subcritical case (8) implies that In view of P [X n > 0] ∼ cµ n 1 we conclude that in the subcritical case See [20] for a comprehensive study of subcritical decomposable branching processes in a constant environment.

Branching processes in a random environment
A randomly changing environment for BGW-processes is modeled by a random sequence of probability generating functions for the offspring distributions of consecutive generations. Throughout this paper we assume that the offspring distribution for type 2 particles is the same across the different states of the environment and characterized by the same generating function h(s). This restriction greatly simplifies analysis still allowing new interesting asymptotic regimes.
We consider two types of stationarily changing environments: IID and Markovian.

IID environment
Our description of the IID environment case starts with a simple illustration based on just two alternative bivariate generating functions f (1) (s 1 , s 2 ) and f (2) (s 1 , s 2 ) with mean offspring numbers (µ 1 ) respectively. We assume that at each time n the environment is say "good" with probability π 1 , so that the type 1 particles reproduce independently according f (1) (s 1 , s 2 ), and with probability π 2 = 1 − π 1 the environment is "bad" and particles of type 1 reproduce according to the f (2) (s 1 , s 2 ) law. In other words, the generating function f (s 1 , s 2 ) should be treated as a random function having distribution In particular, the vector of the mean offspring numbers (µ 1 , θ 1 ) takes values (µ 1 ) with probabilities π 1 and π 2 . More generally, our two-type branching process in an IID random environment is characterized (besides the fixed reproduction law h(s) for the type 2 particles) by a sequence of generating functions {f n (s 1 , s 2 )} ∞ n=0 independently drawn from a certain distribution over probability generating functions so that In this setting the respective conditional moments µ 1 , µ 2 , θ 1 , and θ 2 should be treated as random variables. An important role is played by the random variable ζ := log µ 1 representing the step size of the so-called associated random walk [3] formed by the partial sums ζ 0 + · · · + ζ n−1 with ζ i d = ζ.
Notation: characteristics of the reproduction law in generation n are denoted by adding an extra lower index n to the generic notation, like in (10).

Markovian environment
One way to relax the IID assumption on the environment is to allow for Markovian dependence among its consecutive states. We implement this by modelling changes in terms of an irreducible aperiodic positive recurrent Markov chain {e n } ∞ n=0 with countably many states {1, 2, . . .}. Assuming a stationary initial distribution (π 1 , π 2 , . . .), we associate with each state i of this chain a probability generating function f (i) (s 1 , s 2 ), so that the changing environment for the branching process is governed by the sequence of identically reproduction laws with Markovian dependence. Due to the stationarity we can again write (10) and use the same notation for the marginal moments of the reproduction laws as in the IID case.
To build a bridge to the IID environment case we use an embedding through a sequence of regeneration moments {τ k } ∞ k=0 defined as The times τ k+1 − τ k between consecutive regenerations are independent and all distributed as τ := τ 1 . The embedded process (X n ,Ẑ n ) defined as is a decomposable branching process in an IID environment with two types of particles1 and2 and conditional reproduction generating functionŝ where h k (s) stands for the k-fold iteration of h(s).
Notation: for all characteristics of the embedded process (X n ,Ẑ n ) and related constants appearing in the asymptotic formulae we use the same notation as for the process (X n , Z n ) in the IID case just adding the hat sign.
The key difference from the IID case is that the reproduction law for thê 2-type particles is dependent on the random environment. However, this dependence is of specific nature which we are able to manage using the law of large numbers for renewal processes. Notice that on its own the2-type particles form a so-called degenerate critical branching process in an IID random environment [3]: its conditional offspring mean is deterministicm 1 = 1. Meanwhile, the conditional variance is randomm 2 = τ m 2 .
Taking the first and second order derivatives of (13), we can express the moments of the reproduction law of the embedded process in terms of the moments of the consecutive reproduction laws with Markovian dependence. In what follows we use (11) again, while keeping in mind that the sequence (µ 1,k , µ 2,k , θ 1,k , θ 2,k ) τ −1 k=0 now consists of dependent random vectors. It can be shown thatμ where, as usual, the product of the elements of an empty set is one. Furthermore, setting For the following sum of a random number of random variableŝ a version of the Wald identity holds: Proof. For any state j consider the function According to [13, Th.6.5.2] this defines a stationary measure which is necessarily of the form µ j (i) = c j π i . The constants c j are such that It follows that Developing the example of two environmental states from Section 3.1, let us consider a Markov chain {e n } ∞ n=0 with transition probabilities and a stationary distribution (π 1 , π 2 ). (Notice that d = 1 corresponds to the IID case.) Under stationarity the regeneration time satisfies implying that If (b 1 , b 2 ) are the two possible values for ζ, we can write E[ζ] = π 1 b 1 + π 2 b 2 and in full agreement with Lemma 1.

Critical processes in IID environment
The single type critical branching process with an IID environment displays an asymptotic behavior that is in stark contrast with the constant environment formula (5). According to [17,Th.1], if then for some positive constant c (A much more general limit theorem is obtained in [3].) The following theorem shows that in the decomposable case the difference between constant and random environments is even more striking. For constant environments the survival probability decays as c/ √ n, see (6), but in random environments the decay is like c/ log n. (2) (16), (17), and

Theorem 2. Consider a critical decomposable branching process in an IID environment satisfying
If for some positive α then there exists a constant K 0 such that Before turning to the proof, we make some comments on the conditions and statement of this theorem.
Notation: we will often use the abbreviations x a := (log x) 2+a and n a := (log n) 2+a .
Conditions (20), (21), and (22) are needed for the following properties to hold for any fixed ε > 0, recall notation agreements (11) and (23), Each of them is proven via an intermediate step like relying on the IID assumption for consecutive environmental states. The constant K 0 in the statement of Theorem 2 is the same as in the asymptotic formula from [1] concerning the total number S T of particles of type 1 ever appeared in the process: This constant has a complicated nature and is not further explained here. It is necessary to mention that the representation (27) has been proved in [1] under conditions (16), (17), and (19) only for the case when the probability generating functions f n (s, 1) are linear-fractional with probability 1. However, the latter restriction is easily removed using the results established later on for the general case in [17] and [3]. Our proof of Theorem 2 uses the next lemma.
Lemma 3 Consider conditional moments of the entity W n defined at the beginning of Section 2: Under conditions (16), (20), (21), and (22), For any fixed ǫ > 0, in the notation from (23), Proof. For any fixed ǫ > 0, Notice that according to (18) P T > (log x) 2+ε = o 1 log x , x → ∞, for any fixed ε > 0. (28) Thus, using (24) and (27) we get To obtain a similar estimate from above we write, recalling (23), which together with (25), (27), and (28) yields lim sup Finally, according to (26) P S (2) Proof of Theorem 2. We will show that lim sup using a counterpart of (7) and Lemma 3. First we prove the second inequality in (29). It follows from (30) and the monotonicity of Q n that for any fixed ǫ ∈ (0, 1) with the latter inequality being valid thanks to the monotonicity of the second derivative of the generating function. Therefore, where the last inequality is due to (3). It follows that given S for sufficiently large n. As a result, we see that for large n Now, to finish the proof of the second inequality in (29) it remains to use (28), Lemma 3, and which again due to Lemma 3 follows from the Tauberian theorem [15, Ch. XIII.5, Th.4] applied to the right hand side of T > x e −λx dx.

The subcritical case with an IID environment
We continue studying BGW-processes in IID environment but now assume instead of (16). Results rely upon a theorem from [24] giving the asymptotics for P[W T > x] as x → ∞. It requires an important technical assumption viz. the existence of a constant κ such that If, in addition, for some δ > 0 and either then, according to [24], there exists a constant C κ ∈ (0, ∞) such that It is also known [4,5,6] that under (31) for some positive constant K κ , given by (40) below, where Proof. Referring to (37), put B n := 2 log n log(A −1 ) and notice that From the first equality in (30) and the evident inequality Q n−k ≤ Q n we obtain for n ≥ B n On the other hand, we have a similar upper bound It remains to observe that due to (3) and (36), the same Tauberian theorem [15,Ch. XIII.5,Th.4] applied to the right hand side of 6 The critical case with a Markovian environment As compared to the IID case, Markovian environments require extra conditions on the underlying Markov chain. First we assume that the two-type critical process (X n , Z n ) evolves in a stationary Markovian random environment as defined in Section 3.2. Besides, we suppose the validity of (15) and that for some for an arbitrarily small ε > 0, cf. [19].
Further, for some positive α let Then P [X n > 0] = O(n −1/2 ) and there exists a constantK 0 > 0 such that Proof. The statement is derived in two steps: first and then Fix δ ∈ (0, 1/4) and write Here the last two terms are treated with the help of P τ r δ > r 2δ ≤ ar −δ and (42), while the main term is analyzed by means of ideas from the proof of Theorem 2. LettingŶ k be the number of type2 daughters produced byX k particles of type1 and puttingT := min r :Ẑ r = 0 , we deduce from Hence, applying arguments used to derive (29) in Theorem 2 one can show that for someK 0 > 0 lim sup To demonstrate that (44) follows from (43) observe first that due to we have P X r +Ẑ r > 0 ∼K 0 log r , r → ∞.

It follows that
On the other hand, again by (42) as n → ∞ P N n < a −1 n(1 − ε) = P τ a −1 n(1−ε) > n = o (log n) −1−ρ . Thus, A similar estimate from below follows from Finally, relation (45) is derived from (46) by the law of large numbers argument.

Subcritical processes with a Markovian environment
Assume now that the two-type subcritical process (X n , Z n ) evolves in a stationary Markovian random environment as defined in Section 3.2. Here, similarly to Section 6 the auxiliary branching process X r ,Ẑ r in IID environment with probability generating functions (13) and (14) plays an important role. Single type subcritical processes with a Markovian environment were recently studied in [14]. According to [14] under the conditions of our next theorem one has, similarly to (37), that P [X n > 0] = o (A n ) for some constant A ∈ (0, 1). Theorem 6. Assume that assumption (2) holds, and conditions (32), (33) and either (34) or (35) are valid for the corresponding random variables related to the embedded process (X r ,Ẑ r ) with the key constant κ replaced byκ > 0. Suppose, in addition, that Then, there exists a constantK ≡Kκ > 0, given by (54) and (55) below, such that, see (39), P [Z n > 0] ∼ a min(1,κ)K qκ (n) , n → ∞.
It follows that in view of (1), (51) and (52) one can find N such that P T > N log r + P B (r, ε) = o r −κ .
The corresponding lower bound is obtained similarly.