Long-Time Behavior of Galton–Watson Systems with Circular Mechanism

Abstract

Let $\{Z_n:n\ge 0\}$ be a Galton–Watson system with a circular mechanism $\mathit{a}\ast \mathit{b}$, where $\mathit{a}={\left\{a_j\right\}}_{j=0}^{\infty }$ and $\mathit{b}={\left\{b_j\right\}}_{j=0}^{\infty }$ are probability distributions on ${\mathbf{Z}}_{+}:=\{0,1,2,\cdots \}$. Let $m_a:={\displaystyle \sum _{j=0}^{\infty }}j{a}_j$, $m_b:={\displaystyle \sum _{j=0}^{\infty }}j{b}_j$. The extinction property of such branching systems is first studied. Then, it is proved that there exists ${\Gamma }_n$ such that $W_n={\Gamma }_n^{-1}{Z}_n$ is an integrable martingale and hence converges to some random variable W. Moreover, for the case that $a_0=b_0=0, a_1,b_1\in (0,1)$, the convergence rates to 0 of $P\left(\left|{\displaystyle \frac{Z_{n+1}}{Z_n}}-{\omega }_n\right|>\epsilon \mid Z_0=1\right),P\left(\right|W_n-W|>\epsilon \mid Z_0=1)$ and $P\left(\left|{\displaystyle \frac{Z_{n+1}}{Z_n}}-{\omega }_n\right|>\epsilon \mid W\ge \delta ,Z_0=1\right)$ as $n\to \infty$ are presented for $\epsilon ,\delta >0$ under various moment conditions on $\left\{a_j\right\}$ and $\left\{b_j\right\}$, where ${\omega }_n=m_a$ or $m_b$ for n being even or odd, respectively. It is further shown that the first rate is geometric while the last two rates are supergeometric under a finite moment generating function hypothesis.

1. Introduction

The Markov branching processes play an important role in the classical field of stochastic systems. Branching processes and branching processes with varying mechanisms have been widely applied in practical scenarios across multiple fields, such as biology, ecology, medical and public health, physics and engineering, communication and resource-constrained system optimization. The large deviation theory of such processes can be applied in extreme risk assessment, rare event prediction and system optimization design. Examples include estimating the probability of “explosive population growth” in supercritical branching systems (e.g., large-scale outbreaks of locust plagues and sudden spread of algal blooms) to provide a quantitative basis for disaster early warning; calculating the probability of “unexpected traffic peaks” in communication networks to guide backbone network capacity expansion design; assessing the probability of “extreme risk transmission chains” in financial markets to assist in systemic risk prevention and control; and adjusting the switching parameters of branching systems with varying mechanisms (e.g., reproduction rate adjustment nodes and task splitting thresholds) to minimize the probability of extreme events and improve system robustness. A branching process with a circular mechanism is a special class of branching processes with varying mechanisms. It is characterized by periodic cycles in the reproduction mechanism with generations/states (e.g., periodic switching of reproduction rates and environmental conditions). Branching processes with a circular mechanism can be applied to systems with periodic dynamic changes, especially suited to practical problems of “cycle-driven evolution”, such as seasonal breeding population regulation, population evolution under periodic environmental resource fluctuations and modeling of periodically transmitted infectious diseases. Therefore, studying the long-time properties and large deviations of branching processes with a circular mechanism has significant theoretical significance and great applied implications.

The basic property of a Markov branching system is the branching property; i.e., different individuals act independently when giving birth or dying and the system stops when there is no particle in it. The classical Markov branching systems are very deeply studied; standard references are Anderson [1], Asmussen & Jagers [2], Asmussen & Hering [3], Athreya & Ney [4] and Harris [5]. Based on the standard branching structure, some generalized branching systems are well studied. For example, Yamazato [6] investigated a branching system with immigration, which only occurs at state zero. Chen [7] considered general branching systems with or without resurrection. Sevast’yanov [8] and Vatutin [9] considered interacting branching systems. Liu & Zhang [10] studied large deviation for supercritical branching processes with immigration. Zhang, Li & Geng [11] discussed nonlinear Markov branching processes with immigration and resurrection. Mitov & Yanev [12] considered a class of critical Markov branching processes with non-homogeneous Poisson immigration. Hermann & Pfaffelhuber [13] investigated extinction, survival and duality for Markov branching processes with disasters. Furthermore, Pasha [14] considered the stability of traveling wave solutions for integro-differential equations related to branching Markov processes. Anthony [15] studied the stationary measure for Markov branching processes. Imomov & Murtazaev [16] discussed the Kolmogorov constant explicit form for discrete-time stochastic branching systems. Francisci & Vidyashnkar [17] studied branching processes in random environments with thresholds. Mitov & Yanev [18] considered the critical Markov branching process with infinite variance allowing Poisson immigration with increasing intensity. Simon, Emma, Andreas & Ellen [19] investigated the properties of the non-local branching Markov process. Smorodina & Yarovaya [20] studied the limit behavior of branching random walks.

It is well-known that the evolution of a branching system is controlled by its branching mechanism. However, in realistic situations, such as controlled population models and controlled molecular biology models, the controller may change the branching mechanism at different time or states. Therefore, the evolution behavior of the system will be controlled by all the branching mechanisms involved.

In order to clearly describe the model considered in this paper, we first give the following definitions.

Let $(\Omega ,\mathcal{F},P)$ be a probability space and denote

$$\begin{array}{c}\hfill \mathcal{P}=\{\mathit{a}={\left\{a_k\right\}}_{k=0}^{\infty }:a_k\ge 0,\forall k\ge 0\mathrm{and}\sum _{k=0}^{\infty }{a}_k=1\},\end{array}$$

(1)

i.e., $\mathcal{P}$ is the set of all probability distributions on ${\mathbf{Z}}_{+}:=\{0,1,2,\cdots \}$. Obviously, $\mathcal{P}$ is a Borel subset of Banach space $l_{\infty }$. An element $\mathit{a}\in \mathcal{P}$ is also called a branching mechanism in a branching system, which is the offspring distribution of the particles in the system. For any $\mathit{a}={\left\{a_k\right\}}_{k=0}^{\infty }\in \mathcal{P}$, define

$$\begin{array}{c}\hfill f(\mathit{a};s)=\sum _{k=0}^{\infty }{a}_k{s}^k,\left|s\right|\le 1.\end{array}$$

Denote $m_a=f^{\prime }(\mathit{a};1)={\sum }_{k=0}^{\infty }k{a}_k$, and ${\rho }_a$ is the smallest non-negative root of $f(\mathit{a};s)=s$.

Definition 1. 

Let ${\left\{{\mathit{a}}^{\left(n\right)}\right\}}_{n=0}^{\infty }$ be a sequence of branching mechanisms in $\mathcal{P}$.

(i) 

A ${\mathbf{Z}}_{+}$-valued system $\{Z_n:n\ge 0\}$ is called a Galton–Watson system in a deterministic environment ${\left\{{\mathit{a}}^{\left(n\right)}\right\}}_{n=0}^{\infty }$ if

$$\begin{array}{c}\hfill Z_{n+1}=\sum _{i=1}^{Z_n}{\xi }_{n,i}^{\left({\mathit{a}}^{\left(n\right)}\right)},n\ge 0\end{array}$$

where $\{{\xi }_{n,i}^{\left({\mathit{a}}^{\left(n\right)}\right)}:n\ge 0,i\ge 1\}$ is a sequence of independent ${\mathbf{Z}}_{+}$-valued random variables satisfying $E[s^{{\xi }_{n,i}^{\left({\mathit{a}}^{\left(n\right)}\right)}}]=f({\mathit{a}}^{\left(n\right)};s)$ for $i\ge 1$.

(ii) 

If there exists an integral d such that ${\mathit{a}}^{\left(n\right)}={\mathit{a}}^{\left(k\right)}\mathrm{if}n=kmod(d+1)$, then $\{Z_n:n\ge 0\}$ is called a branching system with a circular mechanism or is simply called an ${\mathit{a}}^{\left(0\right)}\ast \cdots \ast {\mathit{a}}^{\left(d\right)}$-Galton–Watson system.

If ${\mathit{a}}^{\left(n\right)}=\mathit{a}\in \mathcal{P}$, then the process defined in Definition 1 becomes the standard Galton–Watson process. In such case, Athreya [21] considered the above convergence rates for supercritical Galton–Watson systems. Based on Athreya [21], Liu & Zhang [10] studied the convergence rate of (4) for Galton–Watson systems with immigration. Li & Li [22] discussed the convergence rates of (5) and (6) for Galton–Watson systems with immigration and proved that they are supergeometric. Li, Cheng & Li [23] investigated the above convergence rates for single-type continuous-time branching systems.

Recall that the branching mechanism involved in the above references is time-independent. Motivated by the fact that the branching mechanism may be different at different times in realistic situations, in this paper, we mainly consider the long-time behavior of ${\mathit{a}}^{\left(0\right)}\ast \cdots \ast {\mathit{a}}^{\left(d\right)}$-Galton–Watson systems. For simplicity, we only consider the case of $\mathit{a}\ast \mathit{b}$-Galton–Watson systems, where $\mathit{a},\mathit{b}\in \mathcal{P}$. The general case can be similarly discussed.

Let $\{Z_n:n\ge 0\}$ be an $\mathit{a}\ast \mathit{b}$-Galton–Watson system. Denote $m_a:=f^{\prime }(\mathit{a};1),m_b:=f^{\prime }(\mathit{b};1)$ and

$$\begin{array}{c}\hfill m:=m_a{m}_b,{\mathrm{\Gamma }}_n:=\left\{\begin{array}{cc}{m}^{{\displaystyle \frac{n}{2}}},\hfill & \mathrm{if}n\mathrm{is}\mathrm{even},\hfill \\ m^{{\displaystyle \frac{n-1}{2}}}{m}_a,\hfill & \mathrm{if}n\mathrm{is}\mathrm{odd},\hfill \end{array}\right.,{\omega }_n=\left\{\begin{array}{cc}{m}_a,\hfill & \mathrm{if}n\mathrm{is}\mathrm{even},\hfill \\ m_b,\hfill & \mathrm{if}n\mathrm{is}\mathrm{odd}.\hfill \end{array}\right.\end{array}$$

(2)

Then, define

$$\begin{array}{c}\hfill W_n={\mathrm{\Gamma }}_n^{-1}{Z}_n,n\ge 0.\end{array}$$

(3)

More specifically, the main aim of this paper is to discuss the extinction property of $\mathit{a}\ast \mathit{b}$-Galton–Watson systems and the convergence rates of

$$\begin{array}{c}\hfill P\left(\left|{\displaystyle \frac{Z_{n+1}}{Z_n}}-{\omega }_n\right|>\epsilon \mid Z_0=1\right),\end{array}$$

(4)

$$\begin{array}{c}\hfill P\left(\right|W_n-W|>\epsilon \mid Z_0=1),\end{array}$$

(5)

and

$$\begin{array}{c}\hfill P\left(\left|{\displaystyle \frac{Z_{n+1}}{Z_n}}-{\omega }_n\right|>\epsilon \mid W\ge \delta ,Z_0=1\right)\end{array}$$

(6)

as $n\to \infty$ for $\epsilon ,\delta >0$.

The main contribution of this paper is threefold: (i) we derive the extinction probability of Markov branching processes with a circular mechanism; (ii) the explicit expression for the growth rate of the number of individuals in the system is presented; and (iii) the large deviation convergence rate and conditional large deviation convergence rate of the number of individuals in the system are revealed under the condition of non-extinction.

2. Extinction Property of $\mathit{a}\ast \mathit{b}$-Galton–Watson Systems

In this section, we discuss the basic property and the extinction behavior of $\mathit{a}\ast \mathit{b}$-Galton–Watson systems. We first give some preliminaries.

The following Lemma 1 is due to Chapter I of Athreya and Ney [4] (p. 4) and the proof is omitted.

Lemma 1. 

For any $\mathit{a}={\left\{a_k\right\}}_{k=0}^{\infty }\in \mathcal{P}$, $f(\mathit{a};s)$ is a convex increasing function on $[0,1]$. If $m_a\le 1$, then $f(\mathit{a};s)>s$ for all $s\in [0,1)$ and $f(\mathit{a};s)=s$ has exactly one root 1 on $[0,1]$. Furthermore, if $m_a<1$, then 1 is a simple root, while if $m_a=1$, then 1 is a root of multiplicity 2. If $m_a>1$, then $f(\mathit{a};s)=s$ has exactly two roots ${\rho }_a$ and 1 on $[0,1]$ with $0\le {\rho }_a<1$ such that $f(\mathit{a};s)>s$ for $s\in [0,{\rho }_a)$ and $f(\mathit{a};s)<s$ for $s\in ({\rho }_a,1)$. Both ${\rho }_a$ and 1 are simple.

For $\mathit{a}={\left\{a_k\right\}}_{k=0}^{\infty },\mathit{b}={\left\{b_k\right\}}_{k=0}^{\infty }\in \mathcal{P}$, define

$$\begin{array}{c}\hfill {\alpha }_0\left(s\right)=s,{\alpha }_1\left(s\right)=\alpha \left(s\right):=f(\mathit{a};f(\mathit{b};s)),{\alpha }_{n+1}\left(s\right)=\alpha \left({\alpha }_n\left(s\right)\right),n\ge 0,\end{array}$$

and

$$\begin{array}{c}\hfill {\beta }_0\left(s\right)=s,{\beta }_1\left(s\right)=\beta \left(s\right):=f(\mathit{b};f(\mathit{a};s)),{\beta }_{n+1}\left(s\right)=\beta \left({\beta }_n\left(s\right)\right),n\ge 0.\end{array}$$

Let ${\rho }_a$, ${\rho }_b$, ${\rho }_{ab}$ and ${\rho }_{ba}$ denote the smallest non-negative roots of $f(\mathit{a};s)=s$, $f(\mathit{b};s)=s$, $\alpha \left(s\right)=s$ and $\beta \left(s\right)=s$, respectively.

Lemma 2. 

(i) 

${\rho }_{ab}=f(\mathit{a};{\rho }_{ba})$ and ${\rho }_{ba}=f(\mathit{b};{\rho }_{ab})$.

(ii) 

If $m_a{m}_b\le 1$, then ${\rho }_{ab}={\rho }_{ba}=1$.

(iii) 

If $m_a{m}_b>1$ and ${\rho }_a={\rho }_b$, then ${\rho }_{ab}={\rho }_{ba}={\rho }_a$.

(iv) 

If $m_a{m}_b>1$ and ${\rho }_a<{\rho }_b$, then ${\rho }_a<{\rho }_{ab}<{\rho }_{ba}<{\rho }_b$.

Proof. 

It is easy to see that

$${\rho }_{ab}={\alpha }_n\left({\rho }_{ab}\right)=f(\mathit{a};f(\mathit{b};{\alpha }_{n-1}\left({\rho }_{ab}\right)))=f(\mathit{a};{\beta }_{n-1}\left(f(\mathit{b};{\rho }_{ab})\right).$$

Note that ${\beta }_{n-1}\left(f(\mathit{b};{\rho }_{ab})\right)\to {\rho }_{ba}$ yields ${\rho }_{ab}=f(\mathit{a};{\rho }_{ba})$. Similarly, ${\rho }_{ba}=f(\mathit{b};{\rho }_{ab})$. (i) is proved. (ii) follows from ${\alpha }^{\prime }\left(1\right)={\beta }^{\prime }\left(1\right)=m_a{m}_b\le 1$.

If $m_a{m}_b>1$, then ${\alpha }^{\prime }\left(1\right)={\beta }^{\prime }\left(1\right)=m_a{m}_b>1$ and hence ${\rho }_{ab},{\rho }_{ba}<1$. If further ${\rho }_a={\rho }_b$, then ${\rho }_a={\rho }_b<1$ and

$$\begin{array}{c}\hfill \alpha \left({\rho }_a\right)=f(\mathit{a};f(\mathit{b};{\rho }_b))=f(\mathit{a};{\rho }_b)=f(\mathit{a};{\rho }_a)={\rho }_a.\end{array}$$

Similarly, $\beta \left({\rho }_a\right)={\rho }_a$. Hence, ${\rho }_{ab}={\rho }_{ba}={\rho }_a$. (iii) is proved.

Now, we prove (iv). It is obvious that ${\rho }_{ab},{\rho }_{ba}<1$ since $m_a{m}_b>1$. If ${\rho }_a<{\rho }_b<1$, by the property of $f(\mathit{a};s)$ and $f(\mathit{b};s)$,

$$\begin{array}{c}\hfill \alpha \left({\rho }_a\right)=f(\mathit{a};f(\mathit{b};{\rho }_a))>f(\mathit{a};{\rho }_a)={\rho }_a\end{array}$$

and

$$\begin{array}{c}\hfill \alpha \left({\rho }_b\right)=f(\mathit{a};f(\mathit{b};{\rho }_b))=f(\mathit{a};{\rho }_b)<{\rho }_b.\end{array}$$

Hence, ${\rho }_a<{\rho }_{ab}<{\rho }_b$.

If ${\rho }_a<{\rho }_b=1$, by the property of $f(\mathit{a};s)$ and $f(\mathit{b};s)$,

$$\begin{array}{c}\hfill \alpha \left({\rho }_a\right)=f(\mathit{a};f(\mathit{b};{\rho }_a))>f(\mathit{a};{\rho }_a)={\rho }_a\end{array}$$

and hence, ${\rho }_a<{\rho }_{ab}<{\rho }_b=1$. The proof is complete. □

Similarly to the ordinary Galton–Watson case, we give the following definition.

Definition 2. 

An $\mathit{a}\ast \mathit{b}$-Galton–Watson system $\{Z_n:n\ge 0\}$ is called critical, supercritical or subcritical if $m_a{m}_b=1$, $m_a{m}_b>1$ or $m_a{m}_b<1$, respectively.

Let $\{Z_n:n\ge 0\}$ be an $\mathit{a}\ast \mathit{b}$-Galton–Watson system, where $\mathit{a}={\left\{a_n\right\}}_{n=0}^{\infty },\mathit{b}={\left\{b_n\right\}}_{n=0}^{\infty }\in \mathcal{P}$. It is easy to see that $Z_n$ can be rewritten as

$$\begin{array}{c}\hfill Z_{n+1}=\left\{\begin{array}{c}\sum _{i=1}^{Z_n}{\xi }_{n,i}^{\left(a\right)},\mathrm{if}n\mathrm{is}\mathrm{even},\hfill \\ \sum _{i=1}^{Z_n}{\xi }_{n,i}^{\left(b\right)},\mathrm{if}n\mathrm{is}\mathrm{odd},\hfill \end{array}\right.\end{array}$$

(7)

where $\{{\xi }_{n,i}^{\left(a\right)}:k\ge 0,i\ge 1\}$ and $\{{\xi }_{n,i}^{\left(b\right)}:k\ge 0,i\ge 1\}$ are independent identically distributed random variable sequences with probability distribution $P({\xi }_{n,1}^{\left(a\right)}=j)=a_j$ and $P({\xi }_{n,1}^{\left(b\right)}=j)=b_j$, respectively. Moreover, $\{{\xi }_{n,i}^{\left(a\right)}:k\ge 0,i\ge 1\}$ is independent with $\{{\xi }_{n,i}^{\left(b\right)}:k\ge 0,i\ge 1\}$.

Define

$$\begin{array}{c}\hfill \left\{\begin{array}{cc}& f_0(\mathit{ab};s)=s,\hfill \\ & f_{2n+1}(\mathit{ab};s)=f_{2n}(\mathit{ab};f(\mathit{a};s)),n\ge 0,\hfill \\ & f_{2n+2}(\mathit{ab};s)=f_{2n+1}(\mathit{ab};f(\mathit{b};s)),n\ge 0.\hfill \end{array}\right.\end{array}$$

(8)

Theorem 1. 

Let $\{Z_n:n\ge 0\}$ be an $\mathit{a}\ast \mathit{b}$-Galton–Watson system with $Z_0=1$. Then the following hold:

(i) 

The probability generating function of $Z_n$ is given by

$$\begin{array}{c}\hfill E[s^{Z_n}]=f_n(\mathit{ab};s).\end{array}$$

(9)

(ii) 

The mean and variance of $Z_n$ are given by

$$\begin{array}{c}\hfill E[Z_n]={\mathrm{\Gamma }}_n\end{array}$$

(10)

and

$$\begin{array}{c}\hfill Var\left(Z_n\right)=\left\{\begin{array}{c}{\displaystyle \frac{{\sigma }^2{m}^{k-1}(m^k-1)m_a^2}{(m-1)}}+{\sigma }_a^2{m}^k,n=2k+1\hfill \\ {\displaystyle \frac{{\sigma }^2{m}^{k-1}(m^k-1)}{(m-1)}},n=2k\hfill \end{array}\right.\end{array}$$

(11)

where ${\sigma }_a^2={\displaystyle \sum _{j=1}^{\infty }}{j}^2{a}_j-m_a^2$, ${\sigma }_b^2={\displaystyle \sum _{j=1}^{\infty }}{j}^2{b}_j-m_b^2$ and ${\sigma }^2={\sigma }_a^2{m}_b^2+{\sigma }_b^2{m}_a$.

Proof. 

We first prove (i). If $n=1$, then

$$\begin{array}{c}\hfill E[s^{Z_1}]=E[s^{{\xi }_{0,1}^{\left(a\right)}}]={\displaystyle \sum _{k=0}^{\infty }}{a}_k{s}^k=f(\mathit{a};s)=f_1(\mathit{ab};s).\end{array}$$

If $n=2$, then

$$\begin{array}{c}\hfill E[s^{Z_2}]=E\left[s^{{\displaystyle \sum _{i=1}^{Z_1}}{\xi }_{1,i}^{\left(b\right)}}\right]={\displaystyle \sum _{j=0}^{\infty }}P(Z_1=j)·E\left[s^{{\displaystyle \sum _{i=1}^j}{\xi }_{1,i}^{\left(b\right)}}\right]=f(\mathit{a};f(\mathit{b};s))=f_2(\mathit{ab};s).\end{array}$$

Suppose (9) holds true for n. Then, if $n=2k$, we have

$$\begin{array}{c}\hfill E[s^{Z_{2k+1}}]=E\left[s^{{\displaystyle \sum _{i=1}^{Z_{2k}}}{\xi }_{2k,i}^{\left(a\right)}}\right]={\displaystyle \sum _{j=0}^{\infty }}P(Z_{2k}=j)·{\left(E[s^{{\xi }_{2k,1}^{\left(a\right)}}]\right)}^j=f_{2k}(\mathit{ab};f(\mathit{a};s))=f_{2k+1}(\mathit{ab};s).\end{array}$$

Similarly, if $n=2k+1$, then $E[s^{Z_{2k+2}}]=f_{2k+2}(\mathit{ab};s)$. Therefore, (i) is proved.

Now we prove (ii). By (8),

$$\begin{array}{c}\hfill E[Z_{2k+1}]=f_{2k}^{\prime }(\mathit{ab};1)·f^{\prime }(\mathit{a};1)=m_a·f_{2k}^{\prime }(\mathit{ab};1),k\ge 0,\end{array}$$

$$\begin{array}{c}\hfill E[Z_{2k+2}]=f_{2k+1}^{\prime }(\mathit{ab};1)·f^{\prime }(\mathit{b};1)=m_b·f_{2k+1}^{\prime }(\mathit{ab};1),k\ge 0,\end{array}$$

which implies (10) since $E[Z_0]=1$ and $E[Z_1]=m_a$. On the other hand,

$$\begin{array}{c}\hfill {\displaystyle \frac{d^2{f}_{2k+1}(\mathit{ab};s)}{d{s}^2}}=f_{2k}^{\prime \prime }(\mathit{ab};f(\mathit{a};s))·{\left(f^{\prime }(\mathit{a};s)\right)}^2+f_{2k}^{\prime }(\mathit{ab};f(\mathit{a};s))·f^{\prime \prime }(\mathit{a};s),k\ge 0.\end{array}$$

Denote $V_n=E[Z_n(Z_n-1)]$. Then

$$\begin{array}{c}\hfill V_{2k+1}=V_{2k}·m_a^2+m^k·({\sigma }_a^2-m_a+m_a^2),k\ge 0.\end{array}$$

Similarly,

$$\begin{array}{c}\hfill V_{2k+2}=V_{2k+1}·m_b^2+m^k{m}_a·({\sigma }_b^2-m_b+m_b^2),k\ge 0.\end{array}$$

By the above two equalities,

$$\begin{array}{ccc}\hfill V_{2k+2}& =& m^2·V_{2k}+m^k{m}_b^2·({\sigma }_a^2-m_a+m_a^2)+m^k{m}_a·({\sigma }_b^2-m_b+m_b^2)\hfill \\ & =& m^2·V_{2k}+m^k(m_b^2{\sigma }_a^2-m{m}_b+m^2+m_a{\sigma }_b^2-m+m{m}_b)\hfill \\ & =& m^2·V_{2k}+m^k({\sigma }^2+m^2-m),k\ge 0,\hfill \end{array}$$

which implies

$$\begin{array}{c}\hfill V_{2k}={\displaystyle \frac{m^{k-1}(m^k-1){\sigma }^2}{m-1}}+m^{2k}-m^k,k\ge 0\end{array}$$

and

$$\begin{array}{c}\hfill V_{2k+1}={\displaystyle \frac{m^{k-1}(m^k-1)m_a^2{\sigma }^2}{m-1}}+m^{2k}{m}_a^2-m^k{m}_a+m^k{\sigma }_a^2,k\ge 0.\end{array}$$

Hence,

$$\begin{array}{c}\hfill Var\left(Z_{2k}\right)=V_{2k}+m^k-m^{2k}={\displaystyle \frac{m^{k-1}(m^k-1){\sigma }^2}{m-1}},k\ge 0\end{array}$$

and

$$\begin{array}{c}\hfill Var\left(Z_{2k+1}\right)=V_{2k+1}+m^k{m}_a-m^{2k}{m}_a^2={\displaystyle \frac{m^{k-1}(m^k-1)m_a^2{\sigma }^2}{m-1}}+m^k{\sigma }_a^2,k\ge 0.\end{array}$$

The proof is complete. □

Theorem 1 presents the probability distribution and related moments for the number of individuals in the system at time n.

Remark 1. 

By Theorem 1, it is easy to see that $\{Z_{2n}:n\ge 0\}$ is an ordinary Galton–Watson system in which the generating function of the branching mechanism is $f(\mathit{a}\mathit{b};s)$. Hence, by Theorem 1 of Athreya [21], we can obtain the convergence rate of $P(|{\displaystyle \frac{Z_{2n+2}}{Z_{2n}}}-m|>\epsilon \mid Z_0=1)$. However, $\{Z_{2n+1}:n\ge 0\}$ is not a real ordinary Galton–Watson system since $E[s^{Z_3}]=f(\mathit{a};f(\mathit{b};f(\mathit{a};s))$ cannot be a composite function of $E[s^{Z_1}]=f(\mathit{a};s)$ itself.

The following lemma is due to Lemma 3.2 in Chapter I of Athreya and Ney [4] and the proof is omitted.

Lemma 3. 

Suppose that $a_0+a_1<1,b_0+b_1<1$.

(i) 

$\alpha \left(s\right)$ and $\beta \left(s\right)$ are strictly convex and increasing on $[0,1]$.

(ii) 

${\alpha }_n\left(s\right)\uparrow {\rho }_{ab}$ as $n\to \infty$ for $s\in [0,{\rho }_{ab})$, while ${\alpha }_n\left(s\right)\downarrow {\rho }_{ab}$ as $n\to \infty$ for $s\in ({\rho }_{ab},1)$. Similarly, ${\beta }_n\left(s\right)\uparrow {\rho }_{ba}$ as $n\to \infty$ for $s\in [0,{\rho }_{ba})$, while ${\beta }_n\left(s\right)\downarrow {\rho }_{ba}$ as $n\to \infty$ for $s\in ({\rho }_{ba},1)$.

Theorem 2. 

Let $\{Z_n:n\ge 0\}$ be an $\mathit{a}\ast \mathit{b}$-Galton–Watson system with $Z_0=1$. Then the extinction probability of $\{Z_n:n\ge 0\}$ is ${\rho }_{ab}$, which is the smallest non-negative root of $\alpha \left(s\right)=s$.

Proof. 

By Theorem 1, we only need to prove that $\underset{n\to \infty }{lim}{f}_n(\mathit{ab};s)={\rho }_{ab}$ for all $s\in (0,1)$. Without loss of generality, we assume $a_0,b_0<1$. Since

$$\begin{array}{ccc}& & f_1(\mathit{ab};s)=f(\mathit{a};s)\hfill \\ & & f_2(\mathit{ab};s)=f(\mathit{a};f(\mathit{b};s))=\alpha \left(s\right).\hfill \end{array}$$

Recursively,

$$\begin{array}{ccc}& & f_{2n}(\mathit{ab};s)={\alpha }_n\left(s\right),\hfill \\ & & f_{2n+1}(\mathit{ab};s)={\alpha }_n\left(f(\mathit{a};s)\right).\hfill \end{array}$$

Since $f(\mathit{a};s)<1$ for all $s\in [0,1)$, by Lemma 3, we know that

$$\underset{n\to \infty }{lim}{f}_{2n}(\mathit{ab};s)=\underset{n\to \infty }{lim}{f}_{2n+1}(\mathit{ab};s)={\rho }_{ab}.$$

Hence, $\underset{n\to \infty }{lim}{f}_n(\mathit{ab};s)={\rho }_{ab}$ for all $s\in (0,1)$. The proof is complete. □

The above Theorem 2 reveals the extinction behavior and presents the extinction probability.

3. Limit Properties

In this section, we study the limit properties of $Z_n$. Let $W_n$ be defined in (3); i.e.,

$$\begin{array}{c}\hfill W_n={\mathrm{\Gamma }}_n^{-1}{Z}_n,n\ge 0,\end{array}$$

where ${\mathrm{\Gamma }}_n$ is defined by (2).

The following theorem reveals the properties of $W_n$.

Theorem 3. 

Suppose that $m>1,{\sigma }_a^2,{\sigma }_b^2<\infty$ and $Z_0=1$. Then $W_n$ is an integrable martingale and hence converges to some random variable W. Moreover,

(i) 

$\underset{n\to \infty }{lim}E[{(W_n-W)}^2]=0$;

(ii) 

$E[W]=1,Var\left(W\right)={\displaystyle \frac{{\sigma }^2}{m^2-m}}$;

(iii) 

$P(W=0)={\rho }_{ab}$.

Proof. 

By the Markov property and (7), we have

$$\begin{array}{ccc}& & E[W_{n+1}\mid \sigma (Z_0,Z_1,\cdots ,Z_n)]\hfill \\ & =& E\left[{\mathrm{\Gamma }}_{n+1}^{-1}{\displaystyle \sum _{i=1}^{Z_n}}{\xi }_{n,i}^{\left({\mathit{a}}^{\left(n\right)}\right)}\mid Z_n\right]\hfill \\ & =& {\mathrm{\Gamma }}_{n+1}^{-1}{Z}_n{f}^{\prime }({\mathit{a}}^{\left(n\right)};1)\hfill \\ & =& {\mathrm{\Gamma }}_n^{-1}{Z}_n,\hfill \end{array}$$

where ${\mathit{a}}^{\left(n\right)}=\mathit{a}$ or $\mathit{b}$ for n being even or odd, respectively, and we have used the fact that ${\mathrm{\Gamma }}_{n+1}^{-1}{f}^{\prime }({\mathit{a}}^{\left(n\right)};1)={\mathrm{\Gamma }}_n^{-1}$ for all n. Hence, by Theorem 1, $W_n$ is an integrable martingale and hence converges to some random variable W.

It follows from (11) that

$$E[W_n^2]={\mathrm{\Gamma }}_n^{-2}E[Z_n^2]=\left\{\begin{array}{c}{\displaystyle \frac{{\sigma }^2(1-m^{-k})}{m^2-m}}+{\displaystyle \frac{{\sigma }_a^2}{m^k{m}_a^2}}+1,n=2k+1\hfill \\ {\displaystyle \frac{{\sigma }^2(1-m^{-k})}{m^2-m}}+1,n=2k\hfill \end{array}\right.$$

and hence, $\underset{n}{sup}E[W_n^2]<\infty$. Now by standard martingale theory (see Chapter 7 of Doob [24]), (i) and (ii) follow.

If $r=P(W=0)$, then $E[W]=1$ implies $r<1$. Furthermore,

$$r=\underset{n\to \infty }{lim}P(W_n=0)=\underset{n\to \infty }{lim}P(Z_n=0)={\rho }_{ab}.$$

The proof is complete. □

Theorem 3 shows the growth rate of the number of individuals in the system under the notion of convergence in the second moment.

By the proof of Theorem 2, we know that for any $s\in [0,1)$, $f_n(\mathit{ab};s)\to {\rho }_{ab}$ as $n\to \infty$. We shall now consider the convergence rate. Define

$$\begin{array}{c}\hfill Q_n\left(s\right)={\displaystyle \frac{f_n(\mathit{ab};s)-{\rho }_{ab}}{{\gamma }_n}},\end{array}$$

(12)

where

$$\begin{array}{c}\hfill {\gamma }_n=\left\{\begin{array}{cc}{[f^{\prime }(\mathit{a};{\rho }_{ba})·f^{\prime }(\mathit{b};{\rho }_{ab})]}^k,\hfill & n=2k,\hfill \\ [f^{\prime }(\mathit{a};{\rho }_{ba})·f^{\prime }(\mathit{b};{\rho }_{ab}){]}^k·f^{\prime }(\mathit{a},{\rho }_{ba}),\hfill & n=2k+1.\hfill \end{array}\right.\end{array}$$

Theorem 4. 

Suppose that $m>1$. Then

$$\begin{array}{c}\hfill \underset{k\to \infty }{lim}{Q}_{2k}^{\prime }\left(s\right)=:Q^{\prime }\left(s\right)\mathrm{and}\underset{k\to \infty }{lim}{Q}_{2k+1}^{\prime }\left(s\right)=:{\tilde{Q}}^{\prime }\left(s\right)\end{array}$$

(13)

exist for $0\le s<1$ and $Q^{\prime }\left(s\right)>0$ for all $s\in [0,1)$. Furthermore, $\underset{s\to {\rho }_{ab}}{lim}{Q}^{\prime }\left(s\right)=1$ and

$$\begin{array}{c}\hfill {\tilde{Q}}^{\prime }\left(s\right)=Q^{\prime }\left(f(\mathit{a};s)\right)·{\displaystyle \frac{f^{\prime }(\mathit{a};s)}{f^{\prime }(\mathit{a};{\rho }_{ba})}}.\end{array}$$

(14)

Proof. 

It is easy to see that

$$Q_n^{\prime }\left(s\right)={\displaystyle \frac{f_n^{\prime }(\mathit{ab};s)}{{\gamma }_n}}=\left\{\begin{array}{cc}{\displaystyle \frac{{\alpha }_k^{\prime }\left(s\right)}{{[f^{\prime }(\mathit{a};{\rho }_{ba})·f^{\prime }(\mathit{b};{\rho }_{ab})]}^k}},\hfill & n=2k,\hfill \\ {\displaystyle \frac{{\alpha }_k^{\prime }\left(f(\mathit{a};s)\right)}{{[f^{\prime }(\mathit{a};{\rho }_{ba})·f^{\prime }(\mathit{b};{\rho }_{ab})]}^k}}·{\displaystyle \frac{f^{\prime }(\mathit{a};s)}{f^{\prime }(\mathit{a};{\rho }_{ba})}},\hfill & n=2k+1.\hfill \end{array}\right.$$

(15)

Since $m=m_a{m}_b=f^{\prime }(\mathit{a};1)·f^{\prime }(\mathit{b};1)={\alpha }^{\prime }\left(1\right)>1$, by Theorem 11.1 in Chapter 1 of Arthreya [4], we know that

$$\underset{k\to \infty }{lim}{\displaystyle \frac{{\alpha }_k^{\prime }\left(s\right)}{{[f^{\prime }(\mathit{a};{\rho }_{ba})·f^{\prime }(\mathit{b};{\rho }_{ab})]}^k}}=Q^{\prime }\left(s\right)$$

exists for all $s\in [0,1)$ and $Q^{\prime }\left(s\right)>0$ for all $s\in [0,1)$. Furthermore, $\underset{s\to {\rho }_{ab}}{lim}{Q}^{\prime }\left(s\right)=1$. Hence, $\tilde{Q}\left(s\right)$ exists and by (15),

$$\begin{array}{c}\hfill {\tilde{Q}}^{\prime }\left(s\right)>0,s\in [0,1)\mathrm{and}\underset{s\to {\rho }_{ba}}{lim}{\tilde{Q}}^{\prime }\left(s\right)=1.\end{array}$$

The proof is complete. □

Now define

$$Q\left(s\right)={\int }_{{\rho }_{ab}}^s{Q}^{\prime }\left(t\right)dt,for0\le s<1,$$

$$\tilde{Q}\left(s\right)={\int }_{{\rho }_{ba}}^s{\tilde{Q}}^{\prime }\left(t\right)dt,for0\le s<1.$$

Then we have

Corollary 1. 

If $m>1$. Then,

$$\underset{n\to \infty }{lim}{Q}_{2n}\left(s\right)=Q\left(s\right)\mathrm{and}\underset{n\to \infty }{lim}{Q}_{2n+1}\left(s\right)=\tilde{Q}\left(s\right).$$

Proof. 

By the bounded convergence theorem,

$$Q_{2n}\left(s\right)=Q_{2n}\left(s\right)-Q_{2n}\left({\rho }_{ab}\right)={\int }_{{\rho }_{ab}}^s{Q}_{2n}^{\prime }\left(t\right)dt\to Q\left(s\right).$$

and

$$Q_{2n+1}\left(s\right)=Q_{2n+1}\left(s\right)-Q_{2n+1}\left({\rho }_{ba}\right)={\int }_{{\rho }_{ba}}^s{Q}_{2n+1}^{\prime }\left(t\right)dt\to \tilde{Q}\left(s\right).$$

The proof is complete. □

Theorem 5. 

$(Q\left(s\right),\tilde{Q}\left(s\right))$ is the unique solution of the functional equations

$$\left\{\begin{array}{c}Q\left(f(\mathit{a};s)\right)=f^{\prime }(\mathit{a};{\rho }_{ba})·\tilde{Q}\left(s\right),\hfill \\ \tilde{Q}\left(f(\mathit{b};s)\right)=f^{\prime }(\mathit{b};{\rho }_{ab})·Q\left(s\right),\hfill \end{array}\right.$$

(16)

subject to

$$Q\left({\rho }_{ab}\right)=0,\tilde{Q}\left({\rho }_{ba}\right)=0and\underset{s\to {\rho }_{ab}}{lim}{Q}^{\prime }\left(s\right)=1,\underset{s\to {\rho }_{ba}}{lim}{\tilde{Q}}^{\prime }\left(s\right)=1.$$

(17)

Proof. 

Substituting $f(\mathit{a};s)$ or $f(\mathit{b};s)$ for s in the definition of $Q_n\left(s\right)$,

$$Q_{2k}\left(s\right)={\displaystyle \frac{f_{2k}(\mathit{ab};s)-{\rho }_{ab}}{{\gamma }_{2k}}}={\displaystyle \frac{f_{2k-1}(\mathit{ab};f(\mathit{b};s))-{\rho }_{ab}}{{\gamma }_{2k-1}·f^{\prime }(\mathit{b};{\rho }_{ab})}}={\displaystyle \frac{Q_{2k-1}\left(f(\mathit{b};s)\right)}{f^{\prime }(\mathit{b};{\rho }_{ab})}}.$$

Taking limits on both sides yields the second equality of (16). The first equality of (16) can be similarly proved.

As for the uniqueness, note that if $(Q\left(s\right),\tilde{Q}\left(s\right))$ and $(H\left(s\right),\tilde{H}\left(s\right))$ are two solutions of (16) subject to (17), then

$$\begin{array}{cc}|\tilde{Q}\left(s\right)-\tilde{H}\left(s\right)|\hfill & =f^{\prime }{(\mathit{a};{\rho }_{ba})}^{-1}|Q\left(f(\mathit{a};s)\right)-H\left(f(\mathit{a};s)\right)|\hfill \\ & =f^{\prime }{(\mathit{a};{\rho }_{ba})}^{-1}{f}^{\prime }{(\mathit{b};{\rho }_{ab})}^{-1}·|\tilde{Q}\left(f(\mathit{b};f(\mathit{a};s))\right)-\tilde{H}\left(f(\mathit{b};f(\mathit{a};s))\right)|\hfill \\ & ={\gamma }_{2k}^{-1}|\tilde{Q}\left(f_{2k}(\mathit{ba};s)\right)-\tilde{H}\left(f_{2k}(\mathit{ba};s)\right)|\hfill \\ & \le |{\widehat{Q}}_{2k}\left(s\right)|·\left\{\left|1-{\displaystyle \frac{\tilde{Q}\left(f_{2k}(\mathit{ba};s)\right)}{f_{2k}(\mathit{ba};s)-{\rho }_{ba}}}\right|+\left|1-{\displaystyle \frac{\tilde{H}\left(f_{2k}(\mathit{ba};s)\right)}{f_{2k}(\mathit{ba};s)-{\rho }_{ba}}}\right|\right\},\hfill \end{array}$$

(18)

where

$${\widehat{Q}}_n\left(s\right)={\displaystyle \frac{f_n(\mathit{ba};s)-{\rho }_{ba}}{{\gamma }_n^{\prime }}},$$

$$\begin{array}{c}\hfill {\gamma }_n^{\prime }=\left\{\begin{array}{cc}{[f^{\prime }(\mathit{b};{\rho }_{ab})·f^{\prime }(\mathit{a};{\rho }_{ba})]}^k,\hfill & n=2k,\hfill \\ [f^{\prime }(\mathit{b};{\rho }_{ab})·f^{\prime }(\mathit{a};{\rho }_{ba}){]}^k{f}^{\prime }(\mathit{b};{\rho }_{ab}),\hfill & n=2k+1.\hfill \end{array}\right.\end{array}$$

Now for any $s\in [0,1)$, $f_{2k}(\mathit{ba};s)\to {\rho }_{ba}$ and

$$\underset{k\to \infty }{lim}{\displaystyle \frac{\tilde{Q}\left(f_{2k}(\mathit{ba};s)\right)}{f_{2k}(\mathit{ba};s)-{\rho }_{ba}}}=\underset{s\to {\rho }_{ba}}{lim}{\tilde{Q}}^{\prime }\left(s\right)=1,\underset{k\to \infty }{lim}{\displaystyle \frac{\tilde{H}\left(f_{2k}(\mathit{ba};s)\right)}{f_{2k}(\mathit{ba};s)-{\rho }_{ba}}}=\underset{s\to {\rho }_{ba}}{lim}{\tilde{H}}^{\prime }\left(s\right)=1.$$

Similarly, we have

$$|Q\left(s\right)-H\left(s\right)|\le |{\widehat{Q}}_{2k-1}\left(s\right)|·\left\{\left|1-{\displaystyle \frac{Q\left(f_{2k-1}(\mathit{ab};s)\right)}{f_{2k-1}(\mathit{ab};s)-{\rho }_{ab}}}\right|+\left|1-{\displaystyle \frac{H\left(f_{2k-1}(\mathit{ab};s)\right)}{f_{2k-1}(\mathit{ab};s)-{\rho }_{ab}}}\right|\right\},$$

(19)

and

$$\underset{k\to \infty }{lim}{\displaystyle \frac{Q\left(f_{2k-1}(\mathit{ab};s)\right)}{f_{2k-1}(\mathit{ab};s)-{\rho }_{ab}}}=\underset{s\to {\rho }_{ab}}{lim}{Q}^{\prime }\left(s\right)=1,\underset{k\to \infty }{lim}{\displaystyle \frac{H\left(f_{2k-1}(\mathit{ab};s)\right)}{f_{2k-1}(\mathit{ab};s)-{\rho }_{ab}}}=\underset{s\to {\rho }_{ab}}{lim}{H}^{\prime }\left(s\right)=1.$$

On the other hand, by a similar argument to Theorem 4 and Corollary 1, we know that ${lim}_{k\to \infty }{\widehat{Q}}_{2k}\left(s\right)$ and ${lim}_{k\to \infty }{\widehat{Q}}_{2k+1}\left(s\right)$ exist and are finite for all $s\in [0,1)$. Therefore, by (18) and (19), we have $Q\left(s\right)=H\left(s\right)$ and $\tilde{Q}\left(s\right)=\tilde{H}\left(s\right)$ for all $s\in [0,1)$. The proof is complete. □

Since $Q\left(s\right)$ and $\tilde{Q}\left(s\right)$ are limits of power series, we may rewrite them as

$$\begin{array}{c}\hfill Q\left(s\right)=\sum _{k=0}^{\infty }{q}_k{s}^k,s\in [0,1)\mathrm{and}\tilde{Q}\left(s\right)=\sum _{k=0}^{\infty }{\tilde{q}}_k{s}^k,s\in [0,1).\end{array}$$

(20)

4. Large Deviation

In this section, we will discuss the large deviation rates of $\{Z_n:n\ge 0\}$. By Theorem 1, we know that $E\{s^{Z_n}\mid Z_0=1\}=f_n(\mathit{ab};s)$. Therefore, we first study the convergence property of $f_n(\mathit{ab};s)$ and its inverse as $n\to \infty$.

From now on, we assume that

$$\begin{array}{c}\hfill a_0=b_0=0, a_j,b_j\ne 1,\forall j\ge 0\mathrm{and}{m}_a=f^{\prime }(\mathit{a};1),m_b=f^{\prime }(\mathit{b};1)<\infty .\end{array}$$

Proposition 1. 

Let $a_0=b_0=0, a_1,b_1>0$. Then ${\rho }_a={\rho }_{ab}={\rho }_{ba}={\rho }_b=0$ and there exist $0\le q_j<\infty$, $0\le {\tilde{q}}_j<\infty$$(j\ge 1)$ with $q_1={\tilde{q}}_1=1$ such that

$$\underset{n\to \infty }{lim}{\displaystyle \frac{f_{2n}(\mathit{ab};s)}{{\left(a_1{b}_1\right)}^n}}=\sum _{j=1}^{\infty }{q}_j{s}^j=Q\left(s\right)<\infty ,s\in [0,1)$$

(21)

and

$$\underset{n\to \infty }{lim}{\displaystyle \frac{f_{2n+1}(\mathit{ab};s)}{{\left(a_1{b}_1\right)}^n{a}_1}}=\sum _{j=1}^{\infty }{\tilde{q}}_j{s}^j=\tilde{Q}\left(s\right)<\infty ,s\in [0,1).$$

(22)

Furthermore, $(Q\left(s\right),\tilde{Q}\left(s\right))$ is the unique solution of the functional equations

$$\left\{\begin{array}{c}Q\left(f(\mathit{a};s)\right)=a_1\tilde{Q}\left(s\right)\hfill \\ \tilde{Q}\left(f(\mathit{b};s)\right)=b_{1}Q\left(s\right)\hfill \end{array}\right.$$

subject to

$$Q\left(0\right)=0,\tilde{Q}\left(0\right)=0,Q\left(1\right),\tilde{Q}\left(1\right)=\infty \mathrm{and}Q\left(s\right),\tilde{Q}\left(s\right)<\infty ,s\in (0,1).$$

Consequently, for all $1\le i,j<\infty$,

$$\underset{n\to \infty }{lim}{\displaystyle \frac{P(Z_{2n}=j|Z_0=i)}{{\left(a_1{b}_1\right)}^{in}}}=q_j^{\ast \left(i\right)}$$

(23)

and

$$\underset{n\to \infty }{lim}{\displaystyle \frac{P(Z_{2n+1}=j|Z_0=i)}{{\left(a_1{b}_1\right)}^{in}{a}_1^i}}={\tilde{q}}_j^{\ast \left(i\right)},$$

(24)

where $q_j^{\ast \left(i\right)}$, ${\tilde{q}}_j^{\ast \left(i\right)}$ satisfy ${\sum }_{j=1}^{\infty }{q}_j^{\ast \left(i\right)}{s}^j=Q^i\left(s\right)$ and ${\sum }_{j=1}^{\infty }{\tilde{q}}_j^{\ast \left(i\right)}{s}^j={\tilde{Q}}^i\left(s\right)$ for $s\in [0,1)$.

Proof. 

All the assertions excepting (23) and (24) follow directly from Theorems 4 and 5 and Corollary 1, while (23) and (24) follow from (21) and (22) since $E[s^{Z_n}|Z_0=i]={\left(E[s^{Z_n}|Z_0=1]\right)}^i=f_n^i(\mathit{ab};s)$. □

Now, denote $r_a:=max\{s:f(\mathit{a};s)<\infty \}$ and $r_b:=max\{s:f(\mathit{b};s)<\infty \}$. Obviously, $r_a,r_b\ge 1$. Let $g(\mathit{a};s)$ and $g(\mathit{b};s)$ be the inverse functions of $f(\mathit{a};s)$ and of $f(\mathit{b};s)$, respectively, which are defined by

$$f(\Delta ;g(\Delta ;s\left)\right)=s\mathrm{for}0\le s<\infty ,\Delta =\mathit{a},\mathit{b}.$$

It can be easily seen that $g(\mathit{a};s)$ and $g(\mathit{b};s)$ are well defined on $[0,f(\mathit{a};r_a)]$ and $[0,f(\mathit{b};r_b)]$ respectively. Moreover, $g(\mathit{a};s),g(\mathit{b};s)\ge s$ for $s\in [0,1]$ and $g(\mathit{a};s)\le s$ for $s\in [1,r_a]$ (respectively, $g(\mathit{b};s)\le s$ for $s\in [1,r_b]$). Let $g_n(\mathit{ab};s)$ and $g_n(\mathit{ba};s)$ be the inverse functions of $f_n(\mathit{ab};s)$ and $f_n(\mathit{ba};s)$, respectively. It is easy to see that

$$\begin{array}{c}\hfill \left\{\begin{array}{c}{g}_1(\mathit{ab};s)=g(\mathit{a};s)\hfill \\ g_{2n}(\mathit{ab};s)=g(\mathit{b};g_{2n-1}(\mathit{ab};s)),n\ge 1\hfill \\ g_{2n+1}(\mathit{ab};s)=g(\mathit{a};g_{2n}(\mathit{ab};s)),n\ge 0\hfill \end{array}\right.and\left\{\begin{array}{c}{g}_1(\mathit{ba};s)=g(\mathit{b};s)\hfill \\ g_{2n}(\mathit{ba};s)=g(\mathit{a};g_{2n-1}(\mathit{ba};s)),n\ge 1\hfill \\ g_{2n+1}(\mathit{ba};s)=g(\mathit{b};g_{2n}(\mathit{ba};s)),n\ge 0.\hfill \end{array}\right.\end{array}$$

Moreover, $g_n(\mathit{ab};s)$ and $g_n(\mathit{ba};s)$ are nondecreasing with n for $s\in [0,1]$ and nonincreasing with n for $s\in [1,f(\mathit{a};s_0)]$ and $s\in [1,f(\mathit{b};s_0)]$, respectively.

The next proposition shows that the rate of convergence of $g_n(\mathit{ab};·)$ is geometric.

Proposition 2. 

Let $f(\mathit{a};s_0)<\infty$ for some $s_0>1$. Then, for $1\le s\le f(\mathit{a};s_0)$, we have $g_n(\mathit{ab};s),g_n(\mathit{ba};s)\downarrow 1$ and

$$R_n\left(s\right):={\mathrm{\Gamma }}_n·(g_n(\mathit{ab};s)-1)\downarrow R\left(s\right),$$

(25)

$${\tilde{R}}_n\left(s\right):={\tilde{\mathrm{\Gamma }}}_n·(g_n(\mathit{ba};s)-1)\downarrow \tilde{R}\left(s\right),$$

(26)

where

$$\begin{array}{c}\hfill {\tilde{\mathrm{\Gamma }}}_n=\left\{\begin{array}{cc}{m}^{{\displaystyle \frac{n}{2}}},\hfill & n\mathrm{is}\mathrm{even},\hfill \\ m^{{\displaystyle \frac{n-1}{2}}}{m}_b,\hfill & n\mathrm{is}\mathrm{odd}.\hfill \end{array}\right.\end{array}$$

(27)

Moreover, $(R(·),\tilde{R}(·))$ is the unique solution of the functional equations

$$\left\{\begin{array}{c}R\left(f(\mathit{a};s)\right)=m_a\tilde{R}\left(s\right),\hfill \\ \tilde{R}\left(f(\mathit{b};s)\right)=m_{b}R\left(s\right),\hfill \end{array}\right.s\in [1,f(\mathit{a};s_0)]$$

(28)

subject to

$$\begin{array}{c}\hfill \left\{\begin{array}{cc}& 0<R\left(s\right),\tilde{R}\left(s\right)<\infty fors\in [1,f(\mathit{a};s_0)],\hfill \\ & R\left(1\right)=\tilde{R}\left(1\right)=0,R^{\prime }\left(1\right)={\tilde{R}}^{\prime }\left(1\right)=1.\hfill \end{array}\right.\end{array}$$

(29)

The proof of Proposition 2 is similar to that of Theorem 5 and is omitted.

We are now in the position to discuss the convergence rates in (4)–(6). For convenience, we will assume $Z_0=1$ from now on. The following theorem presents the convergence rates in (4).

Theorem 6. 

If $a_0=b_0=0,a_1,b_1>0$ and $f(\mathit{a};s_0)+f(\mathit{b};s_0)<\infty$ for some $s_0>1$. Let $\epsilon >0$. Then there exists $\lambda \in (0,1)$ such that

$${\varphi }_a(n,\epsilon )=P\left(\right|{\overline{X}}_n-m_a|>\epsilon )\le 2{\lambda }^n,n\ge 1,$$

(30)

$${\varphi }_b(n,\epsilon )=P\left(\right|{\overline{Y}}_n-m_b|>\epsilon )\le 2{\lambda }^n,n\ge 1,$$

(31)

with ${\overline{X}}_n={\displaystyle \frac{{\sum }_{i=1}^n{X}_i}{n}}$ being the mean of n i.i.d. r.v.s $\left\{X_i\right\}$ with distribution $\left\{a_j\right\}$ and ${\overline{Y}}_n={\displaystyle \frac{{\sum }_{i=1}^n{Y}_i}{n}}$ being the mean of i.i.d. r.v.s $\left\{Y_i\right\}$ with distribution $\left\{b_j\right\}$. Furthermore,

$$\underset{n\to \infty }{lim}{\displaystyle \frac{1}{a_1^n{b}_1^n}}P\left(\left|{\displaystyle \frac{Z_{2n+1}}{Z_{2n}}}-m_a\right|>\epsilon \right)=\sum _{j=1}^{\infty }{\varphi }_a(j,\epsilon )q_j<\infty$$

(32)

and

$$\underset{n\to \infty }{lim}{\displaystyle \frac{1}{a_1^n{b}_1^{n-1}}}P\left(\left|{\displaystyle \frac{Z_{2n}}{Z_{2n-1}}}-m_b\right|>\epsilon \right)=\sum _{j=1}^{\infty }{\varphi }_b(j,\epsilon ){\tilde{q}}_j<\infty ,$$

(33)

where $\left\{q_j\right\}$ and $\left\{{\tilde{q}}_j\right\}$ are defined via $Q\left(s\right)={\sum }_{j=0}^{\infty }{q}_j{s}^j$ and $\tilde{Q}\left(s\right)={\sum }_{j=0}^{\infty }{\tilde{q}}_j{s}^j(0\le s<1)$, being the unique solution of functional equations

$$\begin{array}{c}\hfill \left\{\begin{array}{cc}& Q\left(f(\mathit{a};s)\right)=a_1\tilde{Q}\left(s\right)\hfill \\ & \tilde{Q}\left(f(\mathit{b};s)\right)=b_{1}Q\left(s\right)\hfill \end{array}\right.\end{array}$$

subject to

$$Q\left(0\right)=0,\tilde{Q}\left(0\right)=0and\underset{s\to 0}{lim}{Q}^{\prime }\left(s\right)=1,\underset{s\to 0}{lim}{\tilde{Q}}^{\prime }\left(s\right)=1.$$

Proof. 

First note that

$$\begin{array}{cc}{\varphi }_a(n,\epsilon )\hfill & =P\left(\right|{\overline{X}}_n-m_a|>\epsilon )\hfill \\ \hfill & \le P\left({\alpha }^{{\sum }_{i=1}^n{X}_i}>{\alpha }^{n(m_a+\epsilon )}\right)+P\left({\beta }^{{\sum }_{i=1}^n{X}_i}>{\beta }^{n(m_a-\epsilon )}\right)\hfill \\ \hfill & \le {[{\alpha }^{-(m_a+\epsilon )}f(\mathit{a};\alpha )]}^n+{[{\beta }^{-(m_a-\epsilon )}f(\mathit{a};\beta )]}^n\hfill \end{array}$$

for any $\alpha >1$ and $\beta <1$. To prove (30), we only need to show that ${\alpha }_0^{-(m_a+\epsilon )}f(\mathit{a};{\alpha }_0)<1$, ${\beta }_0^{-(m_a-\epsilon )}f(\mathit{a};{\beta }_0)<1$ for some ${\alpha }_0>1$ and ${\beta }_0<1$. Indeed, consider

$$F\left(\alpha \right)=f(\mathit{a};\alpha )-{\alpha }^{m_a+\epsilon }.$$

It is easy to know that $F\left(1\right)=0$, and $F^{\prime }\left(\alpha \right)=f^{\prime }(\mathit{a};\alpha )-{\alpha }^{m_a+\epsilon -1}(m_a+\epsilon )$. When $\alpha \downarrow 1$, we have $F^{\prime }\left(\alpha \right)\to m_a-(m_a+\epsilon )<0$. Thus, there exists ${\alpha }_0>1$ such that $f(\mathit{a};{\alpha }_0)<{\alpha }_0^{m_a+\epsilon }$, and hence ${\alpha }_0^{-(m_a+\epsilon )}f(\mathit{a};{\alpha }_0)<1$. Similarly, consider

$$G\left(\beta \right)=f(\mathit{a};\beta )-{\beta }^{m_a-\epsilon }.$$

Then we have $G\left(1\right)=0$, and $G^{\prime }\left(\beta \right)=f^{\prime }(\mathit{a};\beta )-{\beta }^{m_a-\epsilon -1}(m_a-\epsilon )$. When $\beta \uparrow 1$, we have $G^{\prime }\left(\beta \right)\to m_a-(m_a-\epsilon )>0$. Thus, there exists ${\beta }_0<1$ such that ${\beta }_0^{-(m_a-\epsilon )}f(\mathit{a};{\beta }_0)<1$. Take $\lambda =max\{{\alpha }_0^{-(m_a+\epsilon )}f(\mathit{a};{\alpha }_0),{\beta }_0^{-(m_a-\epsilon )}f(\mathit{a};{\beta }_0)\}$. Then $\lambda <1$ and (30) is proved. Similarly, (31) holds true. Furthermore, by adjusting $\lambda$, we know that (30) and (31) hold for the same $\lambda$.

Next prove (32) and (33). By the branching property,

$$\begin{array}{cc}\hfill P\left(\left|{\displaystyle \frac{Z_{2n+1}}{Z_{2n}}}-m_a\right|>\epsilon \right)& =\sum _{j=1}^{\infty }P(Z_{2n}=j)P\left(\left|{\displaystyle \frac{Z_{2n+1}}{Z_{2n}}}-m_a\right|>\epsilon \mid Z_{2n}=j\right)\hfill \\ \hfill & =\sum _{j=1}^{\infty }P(Z_{2n}=j)P\left(\left|{\displaystyle \frac{{\sum }_{i=1}^j{X}_i}{j}}-m_a\right|>\epsilon \right)\hfill \\ \hfill & =\sum _{j=1}^{\infty }P(Z_{2n}=j){\varphi }_a(j,\epsilon ).\hfill \end{array}$$

Let $h_{2n}\left(j\right)={\varphi }_a(j,\epsilon )P(Z_{2n}=j)a_1^{-n}{b}_1^{-n}$. Since ${\varphi }_a(j,\epsilon )\le 2{\lambda }^j$, we have

$$h_{2n}\left(j\right)\le 2{\lambda }^{j}P(Z_{2n}=j)a_1^{-n}{b}_1^{-n}=:r_{2n}\left(j\right).$$

Hence

$$\sum _{j=1}^{\infty }{r}_{2n}\left(j\right)=\sum _{j=1}^{\infty }2{\lambda }^{j}P(Z_{2n}=j)a_1^{-n}{b}_1^{-n}=2f_{2n}(\mathit{ab};\lambda )a_1^{-n}{b}_1^{-n}\to 2Q\left(\lambda \right)<\infty .$$

Thus ${\sum }_{j=1}^{\infty }{h}_{2n}\left(j\right)<\infty$ and $h_{2n}\left(j\right)\to {\varphi }_a(j,\epsilon )q_j$. Secondly,

$$\begin{array}{cc}\hfill P\left(\left|{\displaystyle \frac{Z_{2n}}{Z_{2n-1}}}-m_b\right|>\epsilon \right)& =\sum _{j=1}^{\infty }P(Z_{2n-1}=j)P\left(\left|{\displaystyle \frac{Z_{2n}}{Z_{2n-1}}}-m_b\right|>\epsilon \mid Z_{2n-1}=j\right)\hfill \\ \hfill & =\sum _{j=1}^{\infty }P(Z_{2n-1}=j)P\left(\left|{\displaystyle \frac{{\sum }_{i=1}^j{Y}_i}{j}}-m_b\right|>\epsilon \right)\hfill \\ \hfill & =\sum _{j=1}^{\infty }P(Z_{2n-1}=j){\varphi }_b(j,\epsilon ).\hfill \end{array}$$

Let $h_{2n-1}\left(j\right)={\varphi }_b(j,\epsilon )P(Z_{2n-1}=j)a_1^{-n}{b}_1^{-n+1}$. Since ${\varphi }_b(j,\epsilon )\le 2{\lambda }^j$, we have

$$h_{2n-1}\left(j\right)\le 2{\lambda }^{j}P(Z_{2n-1}=j)a_1^{-n}{b}_1^{-n+1}=:r_{2n-1}\left(j\right).$$

Hence

$$\sum _{j=1}^{\infty }{r}_{2n-1}\left(j\right)=\sum _{j=1}^{\infty }2{\lambda }^{j}P(Z_{2n-1}=j)a_1^{-n}{b}_1^{-n+1}=2f_{2n-1}(ab,\lambda )a_1^{-n}{b}_1^{-n+1}\to 2\tilde{Q}\left(\lambda \right)<\infty .$$

Thus ${\sum }_{j=1}^{\infty }{h}_{2n-1}\left(j\right)<\infty$ and $h_{2n-1}\left(j\right)\to {\varphi }_b(j,\epsilon ){\tilde{q}}_j$. The proof is complete. □

Theorem 6 presents the large deviation convergence rate of the number of individuals in the system.

The next theorem and corollary establish (32) and (33) under conditions weaker than $f(\mathit{a};s_0)+f(\mathit{b};s_0)<\infty$ for some $s_0>1$.

Theorem 7. 

Assume $a_0=b_0=0,a_1,b_1>0$ and that there exist constants $C_{\epsilon }$ and $r>0$ such that $a_1{b}_1{\left(m_a{m}_b\right)}^r>1$, ${\varphi }_a(k,\epsilon ),{\varphi }_b(k,\epsilon )\le C_{\epsilon }{k}^{-r}$ for all k, where ${\varphi }_a(k,\epsilon ),{\varphi }_b(k,\epsilon )$ are defined in (30) and (31). Then (32) and (33) hold.

Proof. 

Notice that

$$P\left(\left|{\displaystyle \frac{Z_{2n+1}}{Z_{2n}}}-m_a\right|>\epsilon \right)=\sum _{k=1}^{\infty }{\varphi }_a(k,\epsilon )P(Z_{2n}=k).$$

By assumption,

$$h_{2n}\left(k\right):={\displaystyle \frac{{\varphi }_a(k,\epsilon )P(Z_{2n}=k)}{a_1^n{b}_1^n}}\le {\displaystyle \frac{C_{\epsilon }}{k^r}}{\displaystyle \frac{P(Z_{2n}=k)}{a_1^n{b}_1^n}}=:h_{2n}^{\prime }\left(k\right),say.$$

By (21),

$$\begin{array}{cc}& h_{2n}\left(k\right)\to q_k{\varphi }_a(k,\epsilon )=:h\left(k\right),say\hfill \\ & h_{2n}^{\prime }\left(k\right)\to C_{\epsilon }{\displaystyle \frac{q_k}{k^r}}.\hfill \end{array}$$

If we show that

$$\sum _{k=1}^{\infty }{h}_{2n}^{\prime }\left(k\right)\to \sum _{k=1}^{\infty }{C}_{\epsilon }{\displaystyle \frac{q_k}{k^r}}<\infty ,\mathrm{as}n\to \infty ,$$

then by a slight modification of Lebesque’s dominated convergence theorem, we get that

$$\left(a_1^{-n}{b}_1^{-n}\right)·P\left(\left|{\displaystyle \frac{Z_{2n+1}}{Z_{2n}}}-m_a\right|>\epsilon \right)=\sum _{k=1}^{\infty }{h}_{2n}\left(k\right)\to \sum _{k=1}^{\infty }h\left(k\right)<\infty .$$

However,

$$\sum _{k=1}^{\infty }{\displaystyle \frac{1}{k^r}}{\displaystyle \frac{P(Z_{2n}=k)}{a_1^n{b}_1^n}}={\displaystyle \frac{E[Z_{2n}^{-r}]}{a_1^n{b}_1^n}}.$$

For any non-negative r.v. X and $0<p<\infty$,

$$\begin{array}{c}\hfill E[X^{-p}]=E\left[{\displaystyle \frac{1}{\mathrm{\Gamma }\left(p\right)}}{\int }_0^{\infty }{e}^{-tX}{t}^{p-1}dt\right]={\displaystyle \frac{1}{\mathrm{\Gamma }\left(p\right)}}{\int }_0^{\infty }E[e^{-tX}]t^{p-1}dt.\end{array}$$

Therefore,

$$\begin{array}{c}\hfill {\displaystyle \frac{E[Z_{2n}^{-r}]}{a_1^n{b}_1^n}}={\displaystyle \frac{1}{\mathrm{\Gamma }\left(r\right)}}{\int }_0^{\infty }{\displaystyle \frac{f_{2n}(\mathit{ab};e^{-t})}{a_1^n{b}_1^n}}{t}^{r-1}dt={\displaystyle \frac{1}{\mathrm{\Gamma }\left(r\right)}}{\int }_0^1{\displaystyle \frac{f_{2n}(\mathit{ab};s)}{a_1^n{b}_1^n}}k\left(s\right)ds,\end{array}$$

where $\mathrm{\Gamma }(·)$ is the Gamma-function and

$$\begin{array}{c}\hfill k\left(s\right)={\displaystyle \frac{{|logs|}^{r-1}}{s}}.\end{array}$$

Since $f_{2n}(\mathit{ab};s)/\left(a_1^n{b}_1^n\right)\uparrow Q\left(s\right)$, by the monotone convergence theorem

$$\mathrm{\Gamma }\left(r\right){\displaystyle \frac{E[Z_{2n}^{-r}]}{a_1^n{b}_1^n}}\uparrow {\int }_0^{1}Q\left(s\right)k\left(s\right)ds.$$

So the proof of (32) will be complete if we show ${\int }_0^{1}Q\left(s\right)k\left(s\right)ds<\infty$. Denote $l\left(s\right):=g(\mathit{b};g(\mathit{a};s\left)\right)$. Then $l\left(s\right):=g(\mathit{b};g(\mathit{a};s\left)\right)$ is the inverse of $\widehat{l}\left(s\right):=f(\mathit{a};f(\mathit{b};s))$. Let $l_m\left(s\right)$ and ${\widehat{l}}_m\left(s\right)$ be the mth iterate of $l\left(s\right)$ and $\widehat{l}\left(s\right)$, respectively. Then, $l_m\left({\widehat{l}}_m\left(s\right)\right)=s,l_{m+1}\left(s\right)\ge l_m\left(s\right)$ and for $0<s<1$, $l_m\left(s\right)\uparrow 1$ and ${\widehat{l}}_m\left(s\right)\downarrow 0$. Fix $0<t_0<1$. Then $t_m=l_m\left(t_0\right)\uparrow 1$. Also since $Q\left(s\right)$ satisfies (16) and (17),

$$\begin{array}{cc}\hfill I_m& ={\int }_{t_m}^{t_{m+1}}Q\left(s\right)k\left(s\right)ds={\int }_{t_m}^{t_{m+1}}{\displaystyle \frac{\tilde{Q}\left(f(\mathit{b};s)\right)}{b_1}}k\left(s\right)ds\hfill \\ & ={\int }_{f(\mathit{b};t_m)}^{f(\mathit{b};t_{m+1})}\tilde{Q}\left(u\right){\displaystyle \frac{k\left(g(\mathit{b};u)\right)g^{\prime }(\mathit{b};u)du}{b_1}}={\int }_{t_{m-1}}^{t_m}Q\left(s\right){\displaystyle \frac{k\left(l\left(s\right)\right)l^{\prime }\left(s\right)}{a_1{b}_1}}ds\hfill \\ & ={\int }_{t_{m-1}}^{t_m}Q\left(s\right)k\left(s\right){\displaystyle \frac{k\left(l\left(s\right)\right)l^{\prime }\left(s\right)}{a_1{b}_{1}k\left(s\right)}}ds.\hfill \end{array}$$

Since $l^{\prime }\left(s\right)=1/{\widehat{l}}^{\prime }\left(s\right)$ and $|logs|/(1-s)\to 1$ as $s\uparrow 1$,

$${\displaystyle \frac{k\left(l\left(s\right)\right)l^{\prime }\left(s\right)}{a_1{b}_{1}k\left(s\right)}}\to {\displaystyle \frac{1}{a_1{b}_1{m}_a^r{m}_b^r}},$$

where $m_a=f^{\prime }(\mathit{a};1)$, $m_b=f^{\prime }(\mathit{b};1)$. Thus if $a_1{b}_1{m}_a^r{m}_b^r>1$, then for any $0<{\left(a_1{b}_1{m}_a^r{m}_b^r\right)}^{-1}<\lambda <1$, there exists an $m_0$ such that $k\left(l\left(s\right)\right)l^{\prime }\left(s\right)/\left(a_1{b}_{1}k\left(s\right)\right)<\lambda$ for $s\ge l_{m_0}\left(t_0\right)$. Thus, $I_m\le \lambda I_{m-1}$ for $m\ge m_0+2$. Hence,

$$\begin{array}{c}\hfill \sum _{m=m_0+2}^{\infty }{I}_m\le I_{m_0+1}\sum _{j=1}^{\infty }{\lambda }^j<\infty .\end{array}$$

Therefore,

$${\int }_0^{1}Q\left(s\right)k\left(s\right)ds\le {\int }_0^{t_{m_0}}Q\left(s\right)k\left(s\right)ds+{\int }_{t_{m_0}}^{1}Q\left(s\right)k\left(s\right)ds<\infty .$$

On the other hand,

$$P\left(\left|{\displaystyle \frac{Z_{2n}}{Z_{2n-1}}}-m_b\right|>\epsilon \right)=\sum _{k=1}^{\infty }{\varphi }_b(k,\epsilon )P(Z_{2n-1}=k).$$

By assumption,

$$h_{2n-1}\left(k\right):={\displaystyle \frac{{\varphi }_b(k,\epsilon )P(Z_{2n-1}=k)}{a_1^n{b}_1^{n-1}}}\le {\displaystyle \frac{C_{\epsilon }}{k^r}}{\displaystyle \frac{P(Z_{2n-1}=k)}{a_1^n{b}_1^{n-1}}}=:h_{2n-1}^{\prime }\left(k\right),say.$$

By (22),

$$\begin{array}{cc}& h_{2n-1}\left(k\right)\to {\tilde{q}}_k{\varphi }_b(k,\epsilon )=:h\left(k\right),say,\hfill \\ & h_{2n-1}^{\prime }\left(k\right)\to C_{\epsilon }{\displaystyle \frac{{\tilde{q}}_k}{k^r}}.\hfill \end{array}$$

If we show that

$$\begin{array}{c}\hfill \sum _k{h}_{2n-1}^{\prime }\left(k\right)\to \sum _k{C}_{\epsilon }{\displaystyle \frac{{\tilde{q}}_k}{k^r}}<\infty ,\end{array}$$

then by a slight modification of Lebesque’s dominated convergence theorem, we get that

$$\begin{array}{c}\hfill a_1^{-n}{b}_1^{-n+1}·P\left(\left|{\displaystyle \frac{Z_{2n}}{Z_{2n-1}}}-m_b\right|>\epsilon \right)=\sum _{k=1}^{\infty }{h}_{2n-1}\left(k\right)\to \sum _{k=1}^{\infty }h\left(k\right)<\infty .\end{array}$$

However,

$$\begin{array}{c}\hfill \sum _{k=1}^{\infty }{\displaystyle \frac{1}{k^r}}{\displaystyle \frac{P(Z_{2n-1}=k)}{a_1^n{b}_1^{n-1}}}={\displaystyle \frac{E[Z_{2n-1}^{-r}]}{a_1^n{b}_1^{n-1}}}.\end{array}$$

For any non-negative r.v. X and $0<p<\infty$,

$$\begin{array}{c}\hfill E{X}^{-p}=E[{\displaystyle \frac{1}{\mathrm{\Gamma }\left(p\right)}}{\int }_0^{\infty }{e}^{-tX}{t}^{p-1}dt]={\displaystyle \frac{1}{\mathrm{\Gamma }\left(p\right)}}{\int }_0^{\infty }E[e^{-tX}]t^{p-1}dt.\end{array}$$

Therefore,

$$\begin{array}{c}\hfill {\displaystyle \frac{E[Z_{2n-1}^{-r}]}{a_1^n{b}_1^{n-1}}}={\displaystyle \frac{1}{\mathrm{\Gamma }\left(r\right)}}{\int }_0^{\infty }{\displaystyle \frac{f_{2n-1}(\mathit{ab};e^{-t})}{a_1^n{b}_1^{n-1}}}{t}^{r-1}dt={\displaystyle \frac{1}{\mathrm{\Gamma }\left(r\right)}}{\int }_0^1{\displaystyle \frac{f_{2n-1}(\mathit{ab};s)}{a_1^n{b}_1^{n-1}}}k\left(s\right)ds,\end{array}$$

where

$$\begin{array}{c}\hfill k\left(s\right)={\displaystyle \frac{{|logs|}^{r-1}}{s}}.\end{array}$$

Since $f_{2n-1}(\mathit{ab};s)/a_1^n{b}_1^{n-1}\uparrow \tilde{Q}\left(s\right)$, by the monotone convergence theorem

$$\begin{array}{c}\hfill \mathrm{\Gamma }\left(r\right){\displaystyle \frac{E[Z_{2n-1}^{-r}]}{a_1^n{b}_1^{n-1}}}\uparrow {\int }_0^1\tilde{Q}\left(s\right)k\left(s\right)ds,n\to \infty .\end{array}$$

So the proof (33) will be complete if we show ${\int }_0^1\tilde{Q}\left(s\right)k\left(s\right)ds<\infty$. Denote $\eta \left(s\right):=g(\mathit{a};g(\mathit{b};s\left)\right)$ and $\widehat{\eta }\left(s\right):=f(\mathit{b};f(\mathit{a};s))$. Then, $\eta \left(s\right)$ is the inverse of $\widehat{\eta }\left(s\right)$. Let ${\eta }_m\left(s\right)$ and ${\widehat{\eta }}_m\left(s\right)$ be the mth iterate of $\eta \left(s\right)$ and $\widehat{\eta }\left(s\right)$. Then, ${\eta }_m\left({\widehat{\eta }}_m\left(s\right)\right)=s,{\eta }_{m+1}\left(s\right)\ge {\eta }_m\left(s\right)$ and for $0<s<1$, ${\eta }_m\left(s\right)\uparrow 1$ and ${\widehat{\eta }}_m\left(s\right)\downarrow 0$. Fix $0<t_0<1$. Then $t_m:={\eta }_m\left(t_0\right)\uparrow 1$. Also since $\tilde{Q}\left(s\right)$ satisfies (16) and (17),

$$\begin{array}{cc}\hfill I_m& ={\int }_{t_m}^{t_{m+1}}\tilde{Q}\left(s\right)k\left(s\right)ds={\int }_{t_m}^{t_{m+1}}{\displaystyle \frac{Q\left(f\right(\mathit{a};s\left)\right)}{a_1}}k\left(s\right)ds\hfill \\ & ={\int }_{f(\mathit{a};t_m)}^{f(\mathit{a};t_{m+1})}Q\left(u\right){\displaystyle \frac{k\left(g(\mathit{a};u)\right)g^{\prime }(\mathit{a};u)du}{a_1}}={\int }_{t_{m-1}}^{t_m}\tilde{Q}\left(s\right){\displaystyle \frac{k\left(\eta \left(s\right)\right){\eta }^{\prime }\left(s\right)}{a_1{b}_1}}ds\hfill \\ & ={\int }_{t_{m-1}}^{t_m}\tilde{Q}\left(s\right)k\left(s\right){\displaystyle \frac{k\left(\eta \left(s\right)\right){\eta }^{\prime }\left(s\right)}{a_1{b}_{1}k\left(s\right)}}ds.\hfill \end{array}$$

Since ${\eta }^{\prime }\left(s\right)=1/{\widehat{\eta }}^{\prime }\left(s\right)$ and $|logs|/(1-s)\to 1$ as $s\uparrow 1$,

$$(k\left(\eta \left(s\right)\right){\eta }^{\prime }\left(s\right)/\left(a_1{b}_{1}k\left(s\right)\right)\to 1/\left(a_1{b}_1{m}_a^r{m}_b^r\right),$$

where $m_a=f^{\prime }(\mathit{a};1)$, $m_b=f^{\prime }(\mathit{b};1)$. Thus if $a_1{b}_1{m}_a^r{m}_b^r>1$, then for any $0<{\left(a_1{b}_1{m}_a^r{m}_b^r\right)}^{-1}<\lambda <1$, there exists an $m_1$ such that $k\left(\eta \left(s\right)\right){\eta }^{\prime }\left(s\right)/\left(a_1{b}_{1}k\left(s\right)\right)<\lambda$ for all $s\ge {\eta }_{m_1}\left(t_0\right)$. Thus, $I_m\le \lambda I_{m-1}$ for $m\ge m_1+2$. Hence,

$$\begin{array}{c}\hfill \sum _{m=m_1+2}^{\infty }{I}_m\le I_{m_1+1}\sum _{j=1}^{\infty }{\lambda }^j<\infty .\end{array}$$

Therefore,

$${\int }_0^1\tilde{Q}\left(s\right)k\left(s\right)ds\le {\int }_0^{t_{m_1}}\tilde{Q}\left(s\right)k\left(s\right)ds+{\int }_{t_{m_1}}^1\tilde{Q}\left(s\right)k\left(s\right)ds<\infty .$$

The proof is complete. □

Remark 2.

(i) 

By Theorem 7, we see that the condition that $f(\mathit{a};s_0)+f(\mathit{b};s_0)<\infty$ for some $s_0>1$ is replaced by the condition that ${\varphi }_a(k,\epsilon ),{\varphi }_b(k,\epsilon )\le C_{\epsilon }{k}^{-r}(k\ge 1)$ provided $a_1{b}_1{\left(m_a{m}_b\right)}^r>1$. The latter is much weaker than the former.

(ii) 

By the proof of Theorem 7, we can obtain that

$$\begin{array}{c}\hfill a_1^{-n}{b}_1^{-n+1}·P\left(\left|{\displaystyle \frac{Z_{2n}}{Z_{2n-1}}}-m_b\right|>\epsilon \right)\le {\tilde{C}}_{\epsilon }\sum _k{\displaystyle \frac{{\tilde{q}}_k}{k^r}}\end{array}$$

for all $n\ge 1$.

Corollary 2. 

Assume $a_1,b_1>0$ and ${\displaystyle \sum _{j=1}^{\infty }}{j}^{2r+\delta }{a}_j+{\displaystyle \sum _{j=1}^{\infty }}{j}^{2r+\delta }{b}_j<\infty$ for some $r\ge 1$ and $\delta >0$ such that $a_1{m}_a^r$, $b_1{m}_b^r>1$. Then (32) and (33) hold.

Proof. 

Since ${\displaystyle \sum _{j=1}^{\infty }}{j}^{2r+\delta }{a}_j+{\displaystyle \sum _{j=1}^{\infty }}{j}^{2r+\delta }{b}_j<\infty$ for some $r\ge 1$ and $\delta >0$, we know that

$$C_{r,1}:=\underset{k}{sup}E\left[{\left|\sqrt{k}{\displaystyle \frac{({\overline{X}}_k-m_a)}{{\sigma }_a}}\right|}^{2r}\right]<\infty$$

and

$$C_{r,2}:=\underset{k}{sup}E\left[{\left|\sqrt{k}{\displaystyle \frac{({\overline{Y}}_k-m_b)}{{\sigma }_b}}\right|}^{2r}\right]<\infty ,$$

where $\{{\overline{X}}_k:k\ge 1\}$ and $\{{\overline{Y}}_k:k\ge 1\}$ are given in Theorem 6. Then by Markov’s inequality, we get

$$\begin{array}{c}\hfill {\varphi }_a(k,\epsilon )\le {\displaystyle \frac{E\left[|\sqrt{k}({\overline{X}}_k-m_a){|}^{2r}\right]}{{\left(\epsilon \sqrt{k}\right)}^{2r}}}\le {\displaystyle \frac{C_{r,1}}{{\epsilon }^{2r}{k}^r}}.\end{array}$$

and

$$\begin{array}{c}\hfill {\varphi }_b(k,\epsilon )\le {\displaystyle \frac{E\left[|\sqrt{k}({\overline{Y}}_k-m_b){|}^{2r}\right]}{{\left(\epsilon \sqrt{k}\right)}^{2r}}}\le {\displaystyle \frac{C_{r,2}}{{\epsilon }^{2r}{k}^r}}.\end{array}$$

Let $C_r=max\{C_{r,1},C_{r,2}\}$. Hence, we have ${\varphi }_b(k,\epsilon ),{\varphi }_a(k,\epsilon )\le {\displaystyle \frac{C_r}{{\epsilon }^{2r}{k}^r}}$. Applying Theorem 7 yields (32) and (33). The proof is complete. □

Now we consider (5), i.e., the long-time behavior of $W_n$. Let $\{{\tilde{Z}}_n:n\ge 0\}$ be the $\mathit{b}\ast \mathit{a}$-Galton–Watson system. Define

$$\begin{array}{c}\hfill {\tilde{W}}_n={\tilde{\mathrm{\Gamma }}}_n^{-1}{\tilde{Z}}_n,n\ge 0,\end{array}$$

where ${\tilde{\mathrm{\Gamma }}}_n$ is given in (27). Similarly to the argument about $W_n$, ${\tilde{W}}_n$ is also an integrable martingale and hence converges to some random variable $\tilde{W}$.

Theorem 8. 

Assume that $f(\mathit{a};e^{{\theta }_0}),f(\mathit{b};e^{{\theta }_0})<\infty$ for some ${\theta }_0>0$. Then there exists ${\theta }_1>0$ such that

$$C_1=\underset{n}{sup}E[exp\left({\theta }_1{W}_n\right)]<\infty$$

(34)

and

$$C_2=\underset{n}{sup}E[exp\left({\theta }_1{\tilde{W}}_n\right)]<\infty .$$

(35)

Proof. 

Since $K:=f(\mathit{a};s_0)<\infty$ for $s_0=e^{{\theta }_0}$, we know that $f_2(\mathit{ab};s)\le K$ if $0\le f(\mathit{b};s)\le s_0$, that is, if $0\le s\le g(\mathit{b};s_0)$. Similarly, $f_3(\mathit{ab};s)\le K$ if $0\le f(\mathit{a};s)\le g(\mathit{b};s_0)$, that is, if $0\le s\le g(\mathit{a};g(\mathit{b};s_0))$. More generally,

$$f_n(\mathit{ab};s)\le Kif0\le s\le g_{n-1}(\mathit{ba};s_0).$$

Now, since $W_n=Z_n/{\mathrm{\Gamma }}_n$, $E[e^{\theta W_n}|Z_0=1]=f_n(\mathit{ab};e^{\theta /{\mathrm{\Gamma }}_n})$. Thus

$$E[exp\left(\theta W_n\right)|Z_0=1]\le K,$$

if $\theta \le {\mathrm{\Gamma }}_{n}log{g}_{n-1}(\mathit{ba};s_0).$ Since $g_n(\mathit{ba};s_0)\downarrow 1$, $log{g}_n(\mathit{ba};s_0)\sim (g_n(\mathit{ba};s_0)-1)$. By Proposition 2, $f(\mathit{a};s_0)<\infty$ for $s_0>1$ implies ${\mathrm{\Gamma }}_{n}log{g}_{n-1}(\mathit{ba};s_0)\to m_a\tilde{R}\left(s_0\right)$, which is positive and finite. Because of $g_n(\mathit{ba};s_0)>1$ for all $n\ge 1$, we can choose

$${\theta }_1=\underset{n}{inf}{\mathrm{\Gamma }}_{n}log{g}_{n-1}(\mathit{ba};s_0)and{C}_1=K.$$

Formula (34) is proved. A similar argument yields (35). The proof is complete. □

The next result shows that the convergence rate of $P\left(\right|W_n-W|>\epsilon \mid Z_0=1)$ is supergeometric.

Theorem 9. 

Let $f(\mathit{a};e^{{\theta }_0})+f(\mathit{b};e^{{\theta }_0})<\infty$ for some ${\theta }_0>0$. Then there exist constants $C_4$ and $\lambda >0$ such that

$$P\left(\right|W_n-W|>\epsilon )\le C_{4}exp(-\lambda {\epsilon }^{2/3}{\mathrm{\Gamma }}_n^{1/3}).$$

(36)

Proof. 

First we need two estimates. Denote

$$\begin{array}{c}\hfill \varphi \left(\theta \right)=E[exp\left(\theta W\right)]and\tilde{\varphi }\left(\theta \right)=E[exp\left(\theta \tilde{W}\right)],\end{array}$$

which are finite for all $\theta \le {\theta }_1$. So, if ${\left\{W^{\left(i\right)}\right\}}_1^{\infty },{\left\{{\tilde{W}}^{\left(i\right)}\right\}}_1^{\infty }$ are $i.i.d.$ copies of W and $\tilde{W}$, respectively, $S_k={\sum }_{i=1}^k(W^{\left(i\right)}-1)$, ${\tilde{S}}_k={\sum }_{i=1}^k({\tilde{W}}^{\left(i\right)}-1)$, then for $\theta \le {\theta }_1$,

$$E[exp\left(\theta (S_k/\sqrt{k})\right)]={\left(\varphi \left({\displaystyle \frac{\theta }{\sqrt{k}}}\right)e^{-\theta /\sqrt{k}}\right)}^k={\left(1+{\displaystyle \frac{1}{k}}{\displaystyle \frac{(\varphi (\theta /\sqrt{k})e^{-\theta /\sqrt{k}}-1)}{({\theta }^2/k)}}{\theta }^2\right)}^k,$$

$$E[exp\left(\theta ({\tilde{S}}_k/\sqrt{k})\right)]={\left(\tilde{\varphi }\left({\displaystyle \frac{\theta }{\sqrt{k}}}\right)e^{-\theta /\sqrt{k}}\right)}^k={\left(1+{\displaystyle \frac{1}{k}}{\displaystyle \frac{(\tilde{\varphi }(\theta /\sqrt{k})e^{-\theta /\sqrt{k}}-1)}{({\theta }^2/k)}}{\theta }^2\right)}^k.$$

However, since

$$\underset{u\to 0}{lim}(\varphi \left(u\right)e^{-u}-1)/u^2={\displaystyle \frac{1}{2}}Var\left(W\right)<\infty$$

and

$$\underset{u\to 0}{lim}(\tilde{\varphi }\left(u\right)e^{-u}-1)/u^2={\displaystyle \frac{1}{2}}Var\left(\tilde{W}\right)<\infty ,$$

we have

$$\underset{\left|u\right|\le 1}{sup}|(\varphi \left(u\right)e^{-u}-1)/u^2|=:c_1<\infty$$

and

$$\underset{\left|u\right|\le 1}{sup}|(\tilde{\varphi }\left(u\right)e^{-u}-1)/u^2|=:c_2<\infty .$$

If ${\theta }_2=min({\theta }_1,1)$, $c=max(c_1,c_2)$, then

$$\underset{\left|\theta \right|\le {\theta }_2}{sup}{\left|\varphi (\theta /\sqrt{k})e^{-\theta /\sqrt{k}}\right|}^k,\underset{\left|\theta \right|\le {\theta }_2}{sup}{\left|\tilde{\varphi }(\theta /\sqrt{k})e^{-\theta /\sqrt{k}}\right|}^k\le e^c=:C_3.$$

Here we have used the fact that for $x>0$, ${(1+x/k)}^k\le e^x$.

Now we proceed with the proof Theorem 9. We begin by noting that (see Theorem I.13.2 in Chapter 1 of Arthreya [4], p. 55)

$$\begin{array}{cc}\hfill W-W_n& =\underset{m\to \infty }{lim}(W_{n+m}-W_n)\hfill \\ & =\left\{\begin{array}{cc}{\mathrm{\Gamma }}_n^{-1}{\displaystyle \sum _{j=1}^{Z_n}}({\tilde{W}}^{\left(j\right)}-1),\hfill & ifnisodd,\hfill \\ {\mathrm{\Gamma }}_n^{-1}{\displaystyle \sum _{j=1}^{Z_n}}(W^{\left(j\right)}-1),\hfill & ifniseven,\hfill \end{array}\right.\hfill \end{array}$$

where $W^{\left(j\right)}$ (or ${\tilde{W}}^{\left(j\right)}$ if n is odd) is the limit $r.v$ in the line of descent initiated by the jth parent of the nth generation of $\left\{Z_n\right\}$. By conditional independence,

$$P(W-W_n>\epsilon |Z_0,Z_1,\cdots ,Z_n)=\left\{\begin{array}{cc}\tilde{\psi }(Z_n,{\mathrm{\Gamma }}_n\epsilon ),\hfill & ifnisodd,\hfill \\ \psi (Z_n,{\mathrm{\Gamma }}_n\epsilon ),\hfill & ifniseven,\hfill \end{array}\right.$$

where $\psi (k,\eta )=P(S_k\ge \eta ),\tilde{\psi }(k,\eta )=P({\tilde{S}}_k\ge \eta )$. However, by the above estimation,

$$\begin{array}{c}\hfill P(S_k\ge \eta )=P\left({\displaystyle \frac{S_k}{\sqrt{k}}}\ge {\displaystyle \frac{\eta }{\sqrt{k}}}\right)\le C_{3}exp(-{\displaystyle \frac{{\theta }_2\eta }{\sqrt{k}}}),\end{array}$$

and

$$P({\tilde{S}}_k\ge \eta )\le C_{3}exp(-{\displaystyle \frac{{\theta }_2\eta }{\sqrt{k}}}).$$

Thus,

$$\begin{array}{cc}\hfill P(W-W_n>\epsilon )& =\left\{\begin{array}{c}E[\tilde{\psi }(Z_n,{\mathrm{\Gamma }}_n\epsilon )],ifnisodd\hfill \\ E[\psi (Z_n,{\mathrm{\Gamma }}_n\epsilon )],ifniseven\hfill \end{array}\right.\hfill \\ \hfill & \le C_{3}E\left[exp(-{\displaystyle \frac{{\theta }_2{\mathrm{\Gamma }}_n\epsilon }{\sqrt{Z_n}}})\right]\hfill \\ \hfill & =C_{3}E\left[exp(-{\theta }_2\epsilon {\mathrm{\Gamma }}_n^{1/2}{\displaystyle \frac{1}{\sqrt{W_n}}})\right].\hfill \end{array}$$

For $\lambda >0$,

$$\begin{array}{cc}\hfill E[exp(-\lambda (1/\sqrt{W_n}))]& =\lambda {\int }_0^{\infty }{e}^{-\lambda u}P\left({\displaystyle \frac{1}{\sqrt{W_n}}}\le u\right)du\hfill \\ \hfill & =\lambda {\int }_0^{\infty }{e}^{-\lambda u}P\left(W_n\ge {\displaystyle \frac{1}{u^2}}\right)du\hfill \\ & \le \lambda C_1{\int }_0^{\infty }{e}^{-\lambda u}exp\left(-{\displaystyle \frac{{\theta }_1}{u^2}}\right)du(byTheorem 8)\hfill \\ \hfill & =C_1{\int }_0^{\infty }{e}^{-t}exp\left(-{\displaystyle \frac{{\theta }_1{\lambda }^2}{t^2}}\right)dt.\hfill \end{array}$$

Thus,

$$P(W-W_n>\epsilon )\le C_3{C}_1{\int }_0^{\infty }{e}^{-t}exp\left(-{\displaystyle \frac{{\theta }_1{\lambda }_n^2}{t^2}}\right)dt,$$

where ${\lambda }_n={\theta }_2\epsilon {\mathrm{\Gamma }}_n^{1/2}$. However, for $\lambda >0$,

$$I\left(\lambda \right):={\int }_0^{\infty }{e}^{-t}{e}^{-{\lambda }^2/t^2}dt={\int }_0^{k\left(\lambda \right)}+{\int }_{k\left(\lambda \right)}^{\infty }\le exp\left(-{\displaystyle \frac{{\lambda }^2}{k^2\left(\lambda \right)}}\right)+e^{-k\left(\lambda \right)}.$$

Choose $k\left(\lambda \right)={\lambda }^{2/3}$. Then $I\left(\lambda \right)\le 2exp(-{\lambda }^{2/3})$. Thus

$$P(W-W_n>\epsilon )\le 2C_3{C}_{1}exp\left(-{\left(\sqrt{{\theta }_1}{\theta }_2\epsilon {\mathrm{\Gamma }}_n^{1/2}\right)}^{2/3}\right)=C_{4}exp\left(-\lambda {\mathrm{\Gamma }}_n^{1/3}{\epsilon }^{2/3}\right),$$

where $C_4=2C_3{C}_1$, $\lambda ={\left(\sqrt{{\theta }_1}{\theta }_2\right)}^{2/3}$. Similar arguments hold for $P(W_n-W>\epsilon )$. Hence, (36) is proved. The proof is complete. □

Theorem 9 reveals the growth rate of the number of individuals in the system under the notion of convergence in probability.

Finally, we consider (6). The next result shows that, conditioned on $W>0$, the convergence rate of $P\left(\left|{\displaystyle \frac{Z_{n+1}}{Z_n}}-D_n\right|>\epsilon \mid W\ge \delta \right)$ is supergeometric.

Theorem 10. 

Let $f(\mathit{a};e^{{\theta }_0})+f(\mathit{b};e^{{\theta }_0})<\infty$ for some ${\theta }_0>0$. Then there exist constants $C_5$ and $\lambda >0$ such that for all $\epsilon >0$, $\delta >0$, we can find $0<I\left(\epsilon \right)<\infty$ such that

$$P\left(\left|{\displaystyle \frac{Z_{n+1}}{Z_n}}-{\omega }_n\right|>\epsilon \mid W\ge \delta \right)\le p_{\delta }\left[C_{5}exp\left(-\delta \gamma I\left(\epsilon \right){\mathrm{\Gamma }}_n\right)+C_{4}exp\left(-\lambda {\left(\delta (1-\gamma )\right)}^{2/3}{\mathrm{\Gamma }}_n^{1/3}\right)\right]$$

(37)

for every $0<\gamma <1$, where $p_{\delta }=1/P(W\ge \delta )$. Hence (for $\gamma =1/2$) there exists constant $C_8>0$ such that

$$\begin{array}{c}\hfill P\left(\left|{\displaystyle \frac{Z_{n+1}}{Z_n}}-{\omega }_n\right|>\epsilon \mid W\ge \delta \right)\le C_6{p}_{\delta }exp\left(-\lambda {(\delta /2)}^{2/3}{\mathrm{\Gamma }}_n^{1/3}\right).\end{array}$$

(38)

Proof. 

It is easy to see that

$$\begin{array}{ccc}& & P\left(\left|{\displaystyle \frac{Z_{n+1}}{Z_n}}-{\omega }_n\right|>\epsilon \mid W\ge \delta \right)\hfill \\ & =& P\left(\left|{\displaystyle \frac{Z_{n+1}}{Z_n}}-{\omega }_n\right|>\epsilon ,W\ge \delta \right){\displaystyle \frac{1}{P(W\ge \delta )}}\hfill \\ & =& p_{\delta }\left[P\left(\left|{\displaystyle \frac{Z_{n+1}}{Z_n}}-{\omega }_n\right|>\epsilon ,W_n\le \delta \gamma ,W\ge \delta \right)+P\left(\left|{\displaystyle \frac{Z_{n+1}}{Z_n}}-{\omega }_n\right|>\epsilon ,W_n\ge \delta \gamma ,W\ge \delta \right)\right]\hfill \\ & =:& p_{\delta }({\delta }_{n1}+{\delta }_{n2}),\hfill \end{array}$$

where $0<\gamma <1$ and $p_{\delta }=1/P(W\ge \delta )$. Clearly,

$${\delta }_{n2}\le P\left(\left|{\displaystyle \frac{Z_{n+1}}{Z_n}}-{\omega }_n\right|>\epsilon ,W_n\ge \delta \gamma \right)\le C_{5}exp(-\delta \gamma I\left(\epsilon \right){\mathrm{\Gamma }}_n),$$

where $C_5$ and $I\left(\epsilon \right)$ are such that $P\left(\right|{\overline{Y}}_k|\ge \epsilon )\le C_5{e}^{-kI\left(\epsilon \right)}$, $P\left(\right|{\overline{X}}_k|\ge \epsilon )\le C_5{e}^{-kI\left(\epsilon \right)}$ and ${\overline{Y}}_k={\displaystyle \frac{{\sum }_{i=1}^k{Y}_i}{k}}$, with $\left\{Y_i\right\}$ being $i.i.d.$ as $Z_1^{\mathit{b}}-m_b$, ${\overline{X}}_k={\displaystyle \frac{{\sum }_{i=1}^k{X}_i}{k}}$ and $\left\{X_i\right\}$ being $i.i.d.$ as $Z_1^{\mathit{a}}-m_a$. (Such $C_5$ and $I\left(\epsilon \right)$ exist by Chernoff-type bounds since $f(\mathit{a};e^{{\theta }_1})$, $f(\mathit{b};e^{{\theta }_1})<\infty$ for some ${\theta }_1>0$.) Now

$$\begin{array}{cc}\hfill {\delta }_{n1}& \le P(W-W_n\ge \delta (1-\gamma ))\hfill \\ & \le C_{4}exp\left(-\lambda {\left(\delta (1-\gamma )\right)}^{2/3}{\mathrm{\Gamma }}_n^{1/3}\right)(byTheorem 9).\hfill \end{array}$$

Therefore,

$$P\left(\left|{\displaystyle \frac{Z_{n+1}}{Z_n}}-D_n\right|>\epsilon |W\ge \delta \right)\le p_{\delta }\left[C_{5}exp(-\delta \gamma I\left(\epsilon \right){\mathrm{\Gamma }}_n)+C_{4}exp\left(-\lambda {\left(\delta (1-\gamma )\right)}^{2/3}{\mathrm{\Gamma }}_n^{1/3}\right)\right].$$

Equation (37) is proved. Since the only condition on $\gamma$ is that $0<\gamma <1$ and the second term goes to zero slower than the first term, we can say that there exist $C_6$ and $\lambda (C_6$ may depend on $\gamma$) such that

$$P\left(\left|{\displaystyle \frac{Z_{n+1}}{Z_n}}-D_n\right|>\epsilon |W\ge \delta \right)\le C_{6}exp\left(-\lambda {\left(\delta (1-\gamma )\right)}^{2/3}{\mathrm{\Gamma }}_n^{1/3}\right).$$

Taking $\gamma ={\displaystyle \frac{1}{2}}$ yields (38). The proof is complete. □

Theorem 10 presents the conditional large deviation convergence rate of the number of individuals in the system.

5. Conclusions

In this paper, we investigated the long-time behavior of Markov branching processes with a circular mechanism, and the main results are as follows:

(1)

We derive the probability distribution of the number of particles in the system at any time and calculate the extinction probability;

(2)

We reveal the growth rate of the number of particles in the system under the notion of convergence in the second moment;

(3)

We obtain the specific expressions for the large deviation convergence rate and conditional large deviation convergence rate of the number of particles in the system.

This paper focuses on the case of a single type of individual. It is worth noting that the large deviation problems of multi-type branching processes with varying mechanisms or branching processes with varying mechanisms and immigration are highly significant and worthy of further consideration.