Some Generalized Entropy Ergodic Theorems for Nonhomogeneous Hidden Markov Models

Abstract

Entropy measures the randomness or uncertainty of a stochastic process, and the entropy rate refers to the limit of the time average of entropy. The generalized entropy rate in the form of delayed averages can overcome the redundancy of initial information while ensuring stationarity. Therefore, it has better practical value. A Hidden Markov Model (HMM) contains two stochastic processes, a stochastic process in which all states can be observed and a Markov chain in which all states cannot be observed. The entropy rate is an important characteristic of HMMs. The transition matrix of a homogeneous HMM is unique, while a Nonhomogeneous Hidden Markov Model (NHMM) requires the transition matrices to be dependent on time variables. From the perspective of model structure, NHMMs are novel extensions of homogeneous HMMs. In this paper, the concepts of the generalized entropy rate and NHMMs are defined and fully explained, a strong limit theorem and limit properties of a norm are presented, and then generalized entropy ergodic theorems with an almost surely convergence for NHMMs are obtained. These results provide concise formulas for the computation and estimation of the generalized entropy rate for NHMMs.

1. Introduction

Shannon introduced entropy, originally a thermodynamic function, into information theory to measure the uncertainty of random phenomena [1]. Shannon’s entropy in information theory has been successfully applied in many engineering fields, including vibration-signal feature extraction [2], chaotic image encryption [3], groundwater quality evaluation [4], etc. In order to accurately compute the average uncertainty of a stochastic process, the existence and properties of the entropy rate should be estimated [5,6]. Many approaches have been adopted to improve the theoretical integrity of the entropy rate. One of the famous works is the Shannon–McMillan–Breiman theorem or entropy ergodic theorem, or the asymptotic equipartition property (AEP), which reflects the almost surely (a.s.) convergence of the entropy rate to a constant. Liu ang Yang [7] proposed an extension of the Shannon–McMillan–Breiman theorem and some limit properties for nonhomogeneous Markov chains. Yang [8] proved the AEP for a nonhomogeneous Markov information source. Ordentlich et al. [9] used the Blackwell measure to compute the entropy rate. The entropy rate for Hidden Markov Models (HMMs) has been expressed in terms of upper and lower bounds [10]. However, even if states of HMMs are finite, the finite expression of the entropy rate does not exist, which is mainly because the set of their predictive features is generically infinite. To solve this problem, Jurgens et al. [11] evolved the mixed state according to the iterated function system and sampled the entropy of the place-dependent probability distribution at each step sequentially. Using an arbitrarily long word, the mean of these entropies can converge to the entropy rate.

The entropy rate is defined by the average of random variables throughout the entire process. In fact, we often encounter time series that appear to be “locally stationary”, so we can take an average of what has happened in some window of the recent past. The generalized entropy rate in the form of delayed averages can overcome the redundancy of initial information while ensuring stationarity and, therefore, has better practical value. Essentially, the generalized entropies are nonnegative functions defined on probability distributions that satisfy continuity, maximality, and expansibility. Delayed averages of random variables were first discussed by Zygmund [12]. Using the limiting behavior of delayed averages, Chow [13] proposed necessary and sufficient conditions for the Borel summability of independent identically distributed random variables. Lai [14] studied analogues of the law of the iterated logarithm for delayed averages of independent random variables. On this basis, Gut and Stradtm$\ddot{\mathrm{u}}$ller [15] studied the strong law of large numbers on delayed averages of random fields. Wang [16] discussed the limit theorems of delayed averages for row-wise conditionally independent stochastic arrays and a class of asymptotic properties of moving averages for Markov chains in Markovian environments. From these studies, it can be seen that the limit of delayed averages has laid a solid theoretical foundation and is naturally applied to the entropy rate.

Combining the generalized entropy rate and nonhomogeneous Markov chains, Wang and Yang [17,18] studied generalized entropy ergodic theorems with a.s. and ${\mathcal{L}}_1$ convergence for time nonhomogeneous Markov chains, and they obtained generalized entropy ergodic theorems for non-null stationary processes using Markov approximation. Shi et al. [19] studied the generalized AEP of higher-order nonhomogeneous Markov information sources by establishing several strong deviation theorems. The entropy rate is an important characteristic of HMMs, and the entropy rate of HMMs plays an important role in applications such as communication decoding, compression, and sorting. Therefore, theoretical research on the entropy rate of HMMs is very necessary. The classical HMMs were first introduced by Baum and Petrie, and they have been widely applied in various fields, including speech recognition, facial expression recognition, gene prediction, gesture recognition, musical composition, bio-informatics, and big data ranking [20,21,22,23]. The power of these models is that they can be very efficiently implemented and simulated. A homogeneous HMM contains two stochastic processes: the observed process is assumed to be conditionally temporally independent given the hidden process, and the hidden process is assumed to evolve according to a first-order Markov chain. A Nonhomogeneous Hidden Markov Model (NHMM) provides the idea by allowing for the transition matrices of the hidden states to be related to a set of observed covariates, that is, the transition matrix of a homogeneous HMM is unique, while an NHMM requires the transition matrices to be dependent on time variables. From the perspective of model structure, NHMMs are novel extensions of homogeneous HMMs. In the last ten years, new theories on NHMMs have emerged. Yang et al. [24] stated the law of large numbers for countable NHMMs. Zhang et al. [25] studied the stability analysis and controller design for a family of nonhomogeneous hidden semi-Markov jump systems with limited information of sojourn-time probability density functions. Shahzadi et al. [26] proposed a class of Nonhomogeneous Hidden Semi-Markov Models for modelling partially observed processes that do not necessarily behave in a stationary and memoryless manner.

Although there have been fruitful achievements in the two fields of generalized entropy ergodic theorems and NHMMs, research on generalized entropy ergodic theorems for NHMMs is limited. Therefore, we consider extending the application scenarios of the generalized entropy rate. Motivated by the above work, the main focus of this paper is to obtain a strong limit theorem of delayed averages of real-number functions and generalized entropy ergodic theorems with an almost surely convergence for NHMMs. These results provide a general idea for the relevant theoretical proof and concise formulas for the computation and estimation of the generalized entropy rate for NHMMs, and they lay the necessary mathematical, theoretical foundation for the reliability of model applications. The rest of this paper is organized as follows: in Section 2, a detailed description of NHMMs and related definitions are introduced. Section 3 presents some limit properties that are used in Section 4. In Section 4, the main results and the proofs are given. Section 5 summarizes the important content and discusses the significance of the research.

2. Preliminaries

This section provides preliminaries for the subsequent important contents. Continuing with the introduction, we state basic concepts and properties of NHMMs, point out the relationship between NHMMs and homogeneous HMMs, and define the generalized entropy, the generalized entropy rate, and other commonly used symbols on this basis.

Firstly, we give the definition and properties of NHMMs. Let $X=\{X_n,n\ge 0\}$ be a nonhomogeneous Markov chain defined on the probability space $\{\mathsf{\Omega },\mathcal{F},\mathbb{P}\}$ taking values in a finite state space $\mathcal{S}=\{s_1,s_2,\dots ,s_N\}$, and $Y=\{Y_n,n\ge 0\}$ a stochastic process defined on the probability space $\{\mathsf{\Omega },\mathcal{F},\mathbb{P}\}$ taking values in a finite state space $\mathcal{L}=\{l_1,l_2,\dots ,l_M\}$. $\{X,Y\}=\{(X_n,Y_n),n\ge 0\}$ is called an NHMM if and only if it meets the following forms and conditions:

• The initial distribution of the nonhomogeneous Markov chain $X=\{X_n,n\ge 0\}$ is

  $$q^{(0)}=(q_0(s_1),q_0(s_2),\cdots ,q_0(s_N)),q_0(s_i)=\mathbb{P}(X_0=s_i),0\le i\le N$$

  (1)

  and the transition matrices are

  $$Q_n=[q_n(s_i;s_j)],$$

  (2)

  where

  $$q_n(s_i;s_j)=\mathbb{P}(X_n=s_j|X_{n-1}=s_i),n\ge 1,s_i,s_j\in \mathcal{S},1\le i,j\le N.$$

  (3)
• For any n

  $$\mathbb{P}\{(Y_0,Y_1,\dots ,Y_n)=(y_0,y_1,\dots ,y_n)|X\}={\displaystyle \prod _{i=0}^n}\mathbb{P}\{Y_i=y_i|X_i\},a.s.$$

  (4)

  where $(y_0,y_1,\dots ,y_n)$ are the realizations of $(Y_0,Y_1,\dots ,Y_n)$.

If for any n, $Q_n=Q,$, and the conditional probabilities $\mathbb{P}(Y_n=y_n|X_n=x_n)$ are independent on n, where Q is a stochastic matrix, $x_n$ and $y_n$ are realizations of $X_n$ and $Y_n$, then $X=\{X_n,n\ge 0\}$ is a homogeneous Markov chain and $\{X,Y\}=\{(X_n,Y_n),n\ge 0\}$ is a homogeneous HMM.

Next, we list some properties for NHMMs, which are also equivalent definitions of the model and play a role in proving the theorems. For any $0\le m<k$, let $X_m^k=(X_m,X_{m+1},\dots ,X_k),Y_m^k=(Y_m,Y_{m+1},\dots ,Y_k)$, and let $x_m^k$ and $y_m^k$ be the realizations of $X_m^k$ and $Y_m^k$, respectively. In addition, For any $m,k\ge 0$, let $X_{m,k}=(X_m,X_{m+1},\dots ,X_{m+k}),$$Y_{m,k}=(Y_m,Y_{m+1},\dots ,Y_{m+k})$.

• $\{X,Y\}=\{(X_n,Y_n),n\ge 0\}$ is an NHMM if and only if for any n,

  $$\mathbb{P}\{Y_0^n=y_0^n|X_0^n=x_0^n\}={\displaystyle \prod _{i=1}^n}\mathbb{P}\{Y_i=y_i|X_i=x_i\}.$$

  (5)
• $\{X,Y\}=\{(X_n,Y_n),n\ge 0\}$ is an NHMM if and only if for any n,

  $$\begin{array}{cc}\hfill & \mathbb{P}\{X_0^n=x_0^n,Y_0^n=y_0^n\}\hfill \\ \hfill =& \mathbb{P}\{X_0=x_0\}\mathbb{P}\{Y_0=y_0|X_0=x_0\}{\displaystyle \prod _{i=1}^n}\mathbb{P}\{X_i=x_i|X_{i-1}=x_{i-1}\}\mathbb{P}\{Y_i=y_i|X_i=x_i\}.\hfill \end{array}$$

  (6)
• $\{X,Y\}=\{(X_n,Y_n),n\ge 0\}$ is an NHMM if and only if for any n,

  $$\begin{array}{c}\hfill \mathbb{P}\{X_n=x_n|X_0^{n-1}=x_0^{n-1},Y_0^{n-1}=y_0^{n-1}\}=\mathbb{P}\{X_n=x_n|X_{n-1}=x_{n-1}\}\end{array}$$

  (7)

  and

  $$\begin{array}{c}\hfill \mathbb{P}\{Y_n=y_n|X_0^n=x_0^n,Y_0^{n-1}=y_0^{n-1}\}=\mathbb{P}\{Y_n=y_n|X_n=x_n\}.\end{array}$$

  (8)

By Equations (7) and (8), we have

$$\begin{array}{cc}& \mathbb{P}\{X_{n+1}=x_{n+1},Y_{n+1}=y_{n+1}|X_0^n=x_0^n,Y_0^n=y_0^n\}\hfill \\ \hfill =& \mathbb{P}\{X_{n+1}=x_{n+1},Y_{n+1}=y_{n+1}|X_n=x_n\}.\hfill \end{array}$$

(9)

In the following text, probability measures are widely used. Therefore, for any $k\ge 1$, denote

$$\begin{array}{c}\hfill p_k(x_k;y_k)=\mathbb{P}(Y_k=y_k|X_k=x_k),\end{array}$$

(10)

$$\begin{array}{c}\hfill p_k(x_{k-1};x_k,y_k)=\mathbb{P}(Y_k=y_k,X_k=x_k|X_{k-1}=x_{k-1}),\end{array}$$

(11)

$$\begin{array}{c}\hfill p(x_0,y_0,\cdots ,x_k,y_k)=\mathbb{P}(X_0=x_0,Y_0=y_0,\cdots ,X_k=x_k,Y_k=y_k),\end{array}$$

(12)

and

$$\begin{array}{cc}& p(x_0,y_0\cdots ,x_{k-1},y_{k-1};x_k,y_k)\hfill \\ \hfill =& \mathbb{P}(X_k=x_k,Y_k=y_k|X_0=x_0,Y_0=y_0,\cdots ,X_{k-1}=x_{k-1},Y_{k-1}=y_{k-1}).\hfill \end{array}$$

(13)

A delayed average is one of the widely known technical indicators used to predict future data in time series analysis, and we define the entropy and the entropy rate in the form of delayed averages. Let ${(f(n))}_{n=0}^{\infty }$ be a sequence of non-negative integers such that $f(n)\to \infty$ as $n\to \infty$.

For simplicity, we use the natural logarithm here; thus, the generalized entropy can be measured. According to information theory, the definition of the entropy of an NHMM $\{X,Y\}=\{(X_n,Y_n),n\ge 0\}$ is

$$\begin{array}{c}\hfill H(X_0,Y_0,X_1,Y_1,\dots ,X_n,Y_n)=E[-\log \mathbb{P}(X_0,Y_0,X_1,Y_1,\dots ,X_n,Y_n,)],\end{array}$$

(14)

where $E[·]$ represents the expectation, and the definition of the entropy rate of an NHMM $\{X,Y\}=\{(X_n,Y_n),n\ge 0\}$ is

$$\begin{array}{c}\hfill \underset{n\to \infty }{\lim }\frac{1}{n}H(X_0,Y_0,X_1,Y_1,\dots ,X_n,Y_n).\end{array}$$

(15)

Combining the concept of the generalized entropy with the properties of NHMMs, we have

$$\begin{array}{cc}\hfill & H(X_{n,f(n)},Y_{n,f(n)})\hfill \\ \hfill =& H(X_n,Y_n,\cdots ,X_{n+f(n)},Y_{n+f(n)})\hfill \\ \hfill =& H(X_n,Y_n)+{\displaystyle \sum _{k=n+1}^{n+f(n)}}H(X_k,Y_k|X_n,Y_n,\cdots ,X_{k-1},Y_{k-1})\hfill \\ \hfill =& H(X_n,Y_n)-{\displaystyle \sum _{k=n+1}^{n+f(n)}}\sum _{x_n,\cdots ,x_k\in \mathcal{S}}\sum _{y_n,\cdots ,y_k\in \mathcal{L}}p(x_n,y_n,\cdots ,x_k,y_k)\log p(x_n,y_n,\cdots ,x_{k-1},y_{k-1};x_k,y_k)\hfill \\ \hfill =& H(X_n,Y_n)-{\displaystyle \sum _{k=n+1}^{n+f(n)}}\sum _{x_{k-1}\in \mathcal{S}}{q}_k(x_{k-1})\sum _{x_k\in \mathcal{S}}\sum _{y_k\in \mathcal{L}}{p}_k(x_{k-1};x_k,y_k)\log p_k(x_{k-1};x_k,y_k)\hfill \\ \hfill =& H(X_n,Y_n)-{\displaystyle \sum _{k=n+1}^{n+f(n)}}\sum _{x_{k-1}\in \mathcal{S}}{q}_k(x_{k-1})\sum _{x_k\in \mathcal{S}}\sum _{y_k\in \mathcal{L}}{q}_k(x_{k-1};x_k)p_k(x_k;y_k)\log q_k(x_{k-1};x_k)p_k(x_k;y_k)\hfill \\ \hfill =& H(X_n,Y_n)+{\displaystyle \sum _{k=n+1}^{n+f(n)}}H(X_k,Y_k|X_{k-1}),\hfill \end{array}$$

(16)

where $X_{n,f(n)}$ and $Y_{n,f(n)}$ denotes $(X_n,X_{n+1},\cdots ,X_{n+f(n)})$ and $(Y_n,Y_{n+1},\cdots ,Y_{n+f(n)})$, respectively, and we use the distribution

$$q^{(k)}=(q_k(s_1),q_k(s_2),\cdots ,q_k(s_N)),q_k(s_i)=\mathbb{P}(X_k=s_i),0\le i\le N,$$

(17)

which is the same below.

To prepare for the following text, we introduce concepts that may frequently appear.

Definition 1. 

Let $\{X,Y\}=\{(X_n,Y_n),n\ge 0\}$ be an NHMM defined as above. Define the generalized entropy density as

$$\begin{array}{cc}& F_{n,f(n)}(\omega )=-\frac{1}{f(n)}\log \mathbb{P}(X_{n,f(n)},Y_{n,f(n)})\hfill \\ \hfill =& -\frac{1}{f(n)}\{\log \mathbb{P}(X_n,Y_n)+{\displaystyle \sum _{k=n+1}^{n+f(n)}}\log \mathbb{P}(X_k,Y_k|X_{k-1},Y_{k-1})\},\hfill \end{array}$$

(18)

where $\omega \in \mathsf{\Omega }$.

In the following text, we prove that the limit of the entropy density is the entropy rate.

Definition 2 

([21]). Let $X=\{X_n,n\ge 0\}$ be a homogeneous Markov chain. Let Q be a transition matrix of $X=\{X_n,n\ge 0\}$. Q is called strongly ergodic if there exists a probability distribution $\pi =({\pi }_1,{\pi }_2,\cdots ,{\pi }_N)$ in $\mathcal{S}$ satisfying

$$\begin{array}{c}\hfill \underset{q^{(0)}}{\sup }\parallel q^{(0)}{Q}^n-\pi \parallel \to 0,asn\to \infty \end{array}$$

(19)

where $q^{(0)}$ is a starting vector. Obviously, Equation (19) implies $\pi Q=\pi$, and π is called the stationary distribution determined by Q.

Definition 3. 

Let $\alpha =({\alpha }_1,{\alpha }_2,\cdots ,{\alpha }_N)$ be a vector, and then the norm of α is defined by

$$\begin{array}{c}\hfill \parallel \alpha \parallel ={\displaystyle \sum _{i=1}^N}|{\alpha }_i|.\end{array}$$

(20)

Let $A={(a_{ij})}_{N\times N}$ be a square matrix, and then the norm of A is defined by

$$\begin{array}{c}\hfill \parallel A\parallel =\underset{i}{\max }{\displaystyle \sum _{j=1}^N}|a_{ij}|.\end{array}$$

(21)

3. Some Limit Properties

In this section, we give some lemmas, which are used to prove the main conclusions. These lemmas include a strong limit theorem for NHMMs and limit properties of a norm. Lemma 1 provides a strong limit theorem for bounded functions of NHMMs. Lemma 2 gives the convergence of bounded functions of real sequences on averages. Lemma 3 points out that under the condition of the convergence of the transition matrices of NHMMs, the counting averages of Markov chains converge to a stationary distribution of irreducible matrices. Lemma 4 points out the convergence of the transition matrix multiplication average. Lemma 5 proves the relationship between the convergence of vector sequences on averages and the convergence of subsequences.

Lemma 1. 

Let $\{X,Y\}=\{(X_n,Y_n),n\ge 0\}$ be an NHMM. Let $g_n(·,·,·)$ be a sequence of bounded real number functions defined on ${\mathcal{S}}^2\times \mathcal{L}$. If for any $ϵ>0$,

$$\begin{array}{c}\hfill {\displaystyle \sum _{n=1}^{\infty }}\exp \{-ϵf(n)\}<\infty ,\end{array}$$

(22)

and there exists a positive number γ such that for any $s\in \mathcal{S},l\in \mathcal{L}$,

$$\begin{array}{c}\hfill \underset{n\to \infty }{\mathrm{lim\; sup}}\underset{t\in \mathcal{S}}{\max }\frac{1}{f(n)}{\displaystyle \sum _{k=n+1}^{n+f(n)}}{g}_k(t,s,l)p_k(t;s,l)\exp \{\gamma |g_k(t,s,l)|\}=C_{\gamma }(s,l)<\infty ,\end{array}$$

(23)

then

$$\begin{array}{c}\hfill \underset{n\to \infty }{\lim }\frac{1}{f(n)}{\displaystyle \sum _{k=n+1}^{n+f(n)}}\{g_k(X_{k-1},X_k,Y_k)-E[g_k(X_{k-1},X_k,Y_k)|X_{k-1}]\}=0,a.s.\end{array}$$

(24)

where $E[·|·]$ represents conditional expectation.

Proof. 

Let u be a nonzero real number, and define

$$\begin{array}{c}\hfill {\mathsf{\Lambda }}_{n,f(n)}(u,\omega )=\frac{\exp \{u{\displaystyle \sum _{k=n+1}^{n+f(n)}}{g}_k(X_{k-1},X_k,Y_k)\}}{{\displaystyle \prod _{k=n+1}^{n+f(n)}}E[e^{g_k(X_{k-1},X_k,Y_k)}|X_{k-1}]},n=1,2,\dots \end{array}$$

(25)

where $\omega \in \mathsf{\Omega }$ and $E[·|·]$ represents conditional expectation. By the properties of conditional expectations, we have

$$\begin{array}{cc}& E{\mathsf{\Lambda }}_{n,f(n)}(u,\omega )\hfill \\ \hfill =& E[E[{\mathsf{\Lambda }}_{n,f(n)}(u,\omega )|X_{0,n+f(n)-1}]]\hfill \\ \hfill =& E[E[{\mathsf{\Lambda }}_{n,f(n)-1}(u,\omega )\frac{e^{u{g}_{n+f(n)-1}(X_{n+f(n)-1},X_{n+f(n)},Y_{n+f(n)})}}{E[e^{g_{n+f(n)-1}}(X_{n+f(n)-1},X_{n+f(n)},Y_{n+f(n)})|X_{n+f(n)-1}]}|X_{0,n+f(n)-1}]]\hfill \\ \hfill =& E[\frac{{\mathsf{\Lambda }}_{n,f(n)-1}(u,\omega )E[e^{u{g}_{n+f(n)-1}(X_{n+f(n)-1},X_{n+f(n)},Y_{n+f(n)})}]|X_{n+f(n)-1}]}{E[e^{g_{n+f(n)-1}}(X_{n+f(n)-1},X_{n+f(n)},Y_{n+f(n)})|X_{n+f(n)-1}]}]\hfill \\ \hfill =& E{\mathsf{\Lambda }}_{n,f(n)-1}(u,\omega )=···=E{\mathsf{\Lambda }}_{n,1}(u,\omega )=1.\hfill \end{array}$$

(26)

For any $ϵ>0$, by Markov inequality and Equation (25), we have

$$\begin{array}{cc}\hfill {\displaystyle \sum _{n=1}^{\infty }}\mathbb{P}[f^{-1}(n)\log {\mathsf{\Lambda }}_{n,f(n)}(u,\omega )\ge ϵ]=& {\displaystyle \sum _{n=1}^{\infty }}\mathbb{P}[{\mathsf{\Lambda }}_{n,f(n)}(u,\omega )\ge \exp (ϵf(n))]\hfill \\ \hfill \le & {\displaystyle \sum _{n=1}^{\infty }}1·\exp \{-ϵf(n)\}<\infty .\hfill \end{array}$$

(27)

Combining the Borel–Cantelli Lemma and the arbitrariness of $ϵ$, we have

$$\begin{array}{c}\hfill \underset{n\to \infty }{\mathrm{lim\; sup}}\frac{1}{f(n)}\log {\mathsf{\Lambda }}_{n,f(n)}(u,\omega )\le 0.a.s.\end{array}$$

(28)

Expanding Equation (28), we have

$$\begin{array}{c}\hfill \underset{n\to \infty }{\mathrm{lim\; sup}}\frac{1}{f(n)}{\displaystyle \sum _{k=n+1}^{n+f(n)}}\{u{g}_k(X_{k-1},X_k,Y_k)-\log E[g_k(X_{k-1},X_k,Y_k)|X_{k-1}]\}\le 0.a.s.\end{array}$$

(29)

Let $0<u<\gamma$. Using the inequalities $\log x\le x-1(x>0)$ and $0\le e^x-1-x\le \frac{1}{2}{x}^2{e}^{|x|}(x\in \mathbb{R})$, we have

$$\begin{array}{cc}& \underset{n\to \infty }{\mathrm{lim\; sup}}\frac{1}{f(n)}{\displaystyle \sum _{k=n+1}^{n+f(n)}}\{g_k(X_{k-1},X_k,Y_k)-E[g_k(X_{k-1},X_k,Y_k)|X_{k-1}]\}\hfill \\ \hfill \le & \underset{n\to \infty }{\mathrm{lim\; sup}}\frac{1}{f(n)}{\displaystyle \sum _{k=n+1}^{n+f(n)}}\{\frac{1}{u}\log E[e^{u{g}_k(X_{k-1},X_k,Y_k)}|X_{k-1}]-E[g_k(X_{k-1},X_k,Y_k)|X_{k-1}]\}\hfill \\ \hfill \le & \underset{n\to \infty }{\mathrm{lim\; sup}}\frac{1}{f(n)}{\displaystyle \sum _{k=n+1}^{n+f(n)}}\{\frac{E[e^{u{g}_k(X_{k-1},X_k,Y_k)}-1-u{g}_k(X_{k-1},X_k,Y_k)|X_{k-1}]}{u}\}\hfill \\ \hfill \le & \underset{n\to \infty }{\mathrm{lim\; sup}}\frac{1}{f(n)}{\displaystyle \sum _{k=n+1}^{n+f(n)}}E[g_k^2(X_{k-1},X_k,Y_k)e^{u|g_k(X_{k-1},X_k,Y_k)|}|X_{k-1}]\hfill \\ \hfill \le & \frac{1}{2}u{C}_{\gamma }(s,l)<\infty .a.s.\hfill \end{array}$$

(30)

Let $u\downarrow 0$ in Equation (30), then

$$\begin{array}{cc}& \underset{n\to \infty }{\mathrm{lim\; sup}}\frac{1}{f(n)}{\displaystyle \sum _{k=n+1}^{n+f(n)}}\{g_k(X_{k-1},X_k,Y_k)-E[g_k(X_{k-1},X_k,Y_k)|X_{k-1}]\}\le 0.a.s.\hfill \end{array}$$

(31)

Similarly, let $-\gamma <u<0$, we have

$$\begin{array}{cc}& \underset{n\to \infty }{\mathrm{lim\; inf}}\frac{1}{f(n)}{\displaystyle \sum _{k=n+1}^{n+f(n)}}\{g_k(X_{k-1},X_k,Y_k)-E[g_k(X_{k-1},X_k,Y_k)|X_{k-1}]\}\ge 0.a.s.\hfill \end{array}$$

(32)

Equation (24) follows immediately from inequalities (31) and (32). □

The key to prove Lemma 1 is to construct the likelihood ratio. By using the approximation of the upper and lower limits, it is possible to obtain an almost surely convergence of bounded functions. This lemma is used to prove Theorem 1.

Lemma 2 

([17]). Let $h(z)$ be a bounded function defined on an real-number interval D and ${\{z_n\}}_{n=0}^{\infty }$ a sequence in D. If $\underset{n\to \infty }{\lim }\frac{1}{f(n)}{\displaystyle \sum _{k=n+1}^{n+f(n)}}|z_k-z|=0$, $h(z)$ is continuous at point z, and Equation (22) holds, then

$$\begin{array}{c}\hfill \underset{n\to \infty }{\lim }\frac{1}{f(n)}{\displaystyle \sum _{k=n+1}^{n+f(n)}}|h(z_k)-h(z)|=0.\end{array}$$

(33)

Lemma 3 

([17]). Let Q be an irreducible transition matrix. If Equation (22) holds, and for any $t,s\in \mathcal{S}$,

$$\begin{array}{c}\hfill \underset{n\to \infty }{\lim }\frac{1}{f(n)}{\displaystyle \sum _{k=n+1}^{n+f(n)}}|q_k(s;t)-q(s;t)|=0,\end{array}$$

(34)

then

$$\begin{array}{c}\hfill \underset{n\to \infty }{\lim }\frac{1}{f(n)}{\displaystyle \sum _{k=n}^{n+f(n)-1}}{1}_{\{s_i\}}(X_k)={\pi }_{i}a.s.\end{array}$$

(35)

holds for any $s_i\in \mathcal{S},1\le i\le N$, where $1_{\{s_i\}}(X_k)$ is an indicative function and $\pi =({\pi }_1,{\pi }_2,\dots ,{\pi }_N)$ is the unique stationary distribution determined by Q.

These two lemmas serve as support for theorem 2. It should be emphasized that Q in Lemma 3 is irreducible, so this lemma still holds for ergodic matrices.

Lemma 4. 

Let $X=\{X_n,n\ge 0\}$ be a nonhomogeneous Markov chain with transition matrices $\{Q_n,n\ge 1\}$. Let Q be a periodic strong ergodic stochastic matrix. Assume that $c=(c_1,c_2,\dots ,c_N)$ is a left eigenvector of Q, the unique solution of equations $cQ=c$. Let B be a constant random matrix, where each row of B is c. If Equation (22) holds and

$$\begin{array}{c}\hfill \underset{n\to \infty }{\lim }\frac{1}{f(n)}{\displaystyle \sum _{k=n+1}^{n+f(n)}}||Q_k-Q||=0,\end{array}$$

(36)

then

$$\begin{array}{c}\hfill \underset{n\to \infty }{\lim }||\frac{1}{f(n)}{\displaystyle \sum _{k=n+1}^{n+f(n)}}{Q}^{(m,m+k-1)}-B||=0\end{array}$$

(37)

holds for any m $\in \mathbb{N}$, where $Q^{(m,m+k-1)}=Q_m\dots Q_{m+k-1},Q^{(m,m)}=Q_m,Q_0=I$ (I is the identical matrix).

The proof process of Lemma 4 is similar to theorem 1 of [27]. It should be emphasized that the matrices appearing in Lemma 4 are composed of constants, so the norm can be calculated. Lemma 4 is one of the prerequisites for Theorem 3.

Lemma 5. 

Let ${\{{\beta }_n\}}_{n=1}^{\infty }$ and β be column vectors with real entries. If Equation (22) holds and

$$\begin{array}{c}\hfill \underset{n\to \infty }{\lim }\frac{1}{f(n)}{\displaystyle \sum _{k=n+1}^{n+f(n)}}||{\beta }_k-\beta ||=0,\end{array}$$

(38)

then there exists a subsequence ${\{{\beta }_{n_k}\}}_{k=1}^{\infty }$ of ${\{{\beta }_n\}}_{n=1}^{\infty }$ such that

$$\begin{array}{c}\hfill \underset{k\to \infty }{\lim }||{\beta }_{n_k}-\beta ||=0.\end{array}$$

(39)

Proof. 

Constructing an inequality, there exists $m\in \mathbb{N}$

$$\begin{array}{c}\hfill \frac{1}{f(n)}{\displaystyle \sum _{k=n+1}^{n+f(n)}}||{\beta }_{m+k}-\beta ||\le \frac{m+f(n)}{f(n)}·\frac{1}{m+f(n)}{\displaystyle \sum _{k=n+1}^{n+f(n)}}||{\beta }_k-\beta ||.\end{array}$$

(40)

From Equation (40), it can be concluded that there exists $m\in \mathbb{N}$, such that

$$\begin{array}{c}\hfill \underset{n\to \infty }{\lim }\frac{1}{f(n)}{\displaystyle \sum _{k=n+1}^{n+f(n)}}||{\beta }_{m+k}-\beta ||=0.\end{array}$$

(41)

holds. Choose a positive sequence $\{b_k\}\downarrow 0$, and by Equation (41), $\exists u,v\in \mathbb{N}$,

$$\begin{array}{c}\hfill \frac{1}{v}{\displaystyle \sum _{k=u+1}^{u+v}}||{\beta }_{m+k}-\beta ||\le b_1.\end{array}$$

(42)

Therefore, there exists $u\le n_1\le u+v$ such that $||g_{n_1}-g||\le b_1$. By (40), we have

$$\begin{array}{c}\hfill \underset{n\to \infty }{\lim }\frac{1}{f(n)}{\displaystyle \sum _{k=n+1}^{n+f(n)}}||{\beta }_{n_1+k}-\beta ||=0.\end{array}$$

(43)

Similarly, there exists $n_2\ge n_1$ such that $||{\beta }_{n_2}-\beta ||\le a_1$. Generally, we can get a subsequence ${\{{\beta }_{n_k}\}}_{k=1}^{\infty }$ of ${\{{\beta }_n\}}_{n=1}^{\infty }$ such that $||{\beta }_{n_k}-\beta ||\le b_k,k=1,2,3,\cdots$. Equation (39) follows immediately. □

The key to the proof of Lemma 5 is to construct inequality (40). According to the properties of the convergent sequence, even if the sequence is not monotonic, there exists $m\in \mathbb{N}$, and inequality (40) holds. Lemma 5 is also one of the prerequisites for Theorem 3.

4. Generalized Entropy Ergodic Theorems

In this section, we give the main results and their proofs. Generalized entropy ergodic theorems with an almost surely convergence for NHMMs are presented. These results provide concise formulas for the computation and estimation of the generalized entropy rate for NHMMs.

Theorem 1. 

Let $\{X,Y\}=\{(X_n,Y_n),n\ge 0\}$ be an NHMM. If for any $ϵ>0$,

$$\begin{array}{c}\hfill {\displaystyle \sum _{n=1}^{\infty }}\exp \{-ϵf(n)\}<\infty ,\end{array}$$

(44)

and there exists a positive number γ, such that for any $s\in \mathcal{S},l\in \mathcal{L}$,

$$\begin{array}{c}\hfill \underset{n\to \infty }{\mathrm{lim\; sup}}\underset{t\in \mathcal{S}}{\max }\frac{1}{f(n)}{\displaystyle \sum _{k=n+1}^{n+f(n)}}{g}_k(t,s,l)p_k(t;s,l)\exp \{\gamma |g_k(t,s,l)|\}=C_{\gamma }(s,l)<\infty ,\end{array}$$

(45)

then

$$\begin{array}{c}\hfill \underset{n\to \infty }{\lim }{F}_{n,f(n)}(\omega )=\underset{n\to \infty }{\lim }-\frac{1}{f(n)}{\displaystyle \sum _{k=n+1}^{n+f(n)}}\sum _{s\in \mathcal{S}}\sum _{l\in \mathcal{L}}{p}_k(X_{k-1};s,l)\log p_k(X_{k-1};s,l).a.s.\end{array}$$

(46)

Proof. 

Set $g_k(X_{k-1};s,l)=-\log p_k(X_{k-1};s,l)$ and $\gamma =\frac{1}{2}$ in Lemma 3. Using the inequality ${(\log x)}^2{x}^{\frac{1}{2}}\le e^{-2}(0\le x\le 1)$, we can conclude that for any $s\in \mathcal{S},l\in \mathcal{L}$,

$$\begin{array}{cc}& C_{\frac{1}{2}}(s,l)\hfill \\ \hfill =& \underset{n\to \infty }{\mathrm{lim\; sup}}\underset{t\in \mathcal{S}}{\max }\frac{1}{f(n)}{\displaystyle \sum _{k=n+1}^{n+f(n)}}{[\log p_k(t;s,l)]}^2{p}_k(t;s,l)\exp \{\frac{1}{2}|p_k(t;s,l)|\}\hfill \\ \hfill \le & 16\exp \{-2\}.\hfill \end{array}$$

(47)

It is not hard to verify that

$$\begin{array}{cc}& \frac{1}{f(n)}{\displaystyle \sum _{k=n+1}^{n+f(n)}}\{g_k(X_{k-1},X_k,Y_k)-\sum _{s\in \mathcal{S}}\sum _{l\in \mathcal{L}}{g}_k(X_{k-1},s,l)p_k(X_{k-1};s,l)\}\hfill \\ \hfill =& -\frac{1}{f(n)}{\displaystyle \sum _{k=n+1}^{n+f(n)}}\{\log p_k(X_{k-1};X_k,Y_k)-\sum _{s\in \mathcal{S}}\sum _{l\in \mathcal{L}}{p}_k(X_{k-1};s,l)\log p_k(X_{k-1};s,l)\}\hfill \\ \hfill =& -\frac{1}{f(n)}{\displaystyle \sum _{k=n+1}^{n+f(n)}}\{\log p_k(X_{k-1};X_k,Y_k)-\sum _{s\in \mathcal{S}}\sum _{l\in \mathcal{L}}{p}_k(X_{k-1};s,l)\log p_k(X_{k-1};s,l)\}\hfill \\ \hfill =& -\frac{1}{f(n)}\log \mathbb{P}(X_n,Y_n)+F_{n,f(n)}(\omega )\hfill \\ \hfill & +\frac{1}{f(n)}{\displaystyle \sum _{k=n+1}^{n+f(n)}}\sum _{s\in \mathcal{S}}\sum _{l\in \mathcal{L}}{p}_k(X_{k-1};s,l)\log p_k(X_{k-1};s,l).\hfill \end{array}$$

(48)

The transition probability falls within the interval [0,1], so its logarithm is bounded. In addition, infinitesimal multiplication by a bounded quantity is still infinitesimal, so the initial distribution is erased. Hence the conclusion can be deduced immediately from Lemma 1. □

Theorem 2. 

Let $\{X,Y\}=\{(X_n,Y_n),n\ge 0\}$ be an NHMM. Assume that for any $ϵ>0$,

$$\begin{array}{c}\hfill {\displaystyle \sum _{n=1}^{\infty }}\exp \{-ϵf(n)\}<\infty ,\end{array}$$

(49)

and there exists a positive number γ, such that for any $s\in \mathcal{S},l\in \mathcal{L}$,

$$\begin{array}{c}\hfill \underset{n\to \infty }{\mathrm{lim\; sup}}\underset{t\in \mathcal{S}}{\max }\frac{1}{f(n)}{\displaystyle \sum _{k=n+1}^{n+f(n)}}{g}_k(t,s,l)p_k(t;s,l)\exp \{\gamma |g_k(t,s,l)|\}=C_{\gamma }(s,l)<\infty .\end{array}$$

(50)

If for any $(t,s,l)\in {\mathcal{S}}^2\times \mathcal{L}$,

$$\begin{array}{cc}& \underset{n\to \infty }{\lim }\frac{1}{f(n)}{\displaystyle \sum _{k=n+1}^{n+f(n)}}|p_k(t;s,l)-p(t;s,l)|\hfill \\ \hfill =& \underset{n\to \infty }{\lim }\frac{1}{f(n)}{\displaystyle \sum _{k=n+1}^{n+f(n)}}|q_k(t;s)p_k(s;l)-q(t;s)p(s;l)|=0\hfill \end{array}$$

(51)

holds, where q represents terms of an irreducible transition matrix Q, then

$$\begin{array}{c}\hfill \underset{n\to \infty }{\lim }{F}_{n,f(n)}(\omega )=-\sum _{t\in \mathcal{S}}\pi (t)\sum _{s\in \mathcal{S}}\sum _{l\in \mathcal{L}}p(t;s,l)\log p(t;s,l),\end{array}$$

(52)

where $\pi (t),t\in \mathcal{S}$ belongs to $\pi =({\pi }_1,{\pi }_2,\dots ,{\pi }_N)$, which is a stationary distribution of Q.

Proof. 

Set $h(z)=z\log z$ in Lemma 2; then, for any $(t,s,l)\in {\mathcal{S}}^2\times \mathcal{L}$,

$$\begin{array}{c}\hfill \underset{n\to \infty }{\lim }\frac{1}{f(n)}{\displaystyle \sum _{k=n+1}^{n+f(n)}}|p_k(t;s,l)\log p_k(t;s,l)-p(t;s,l)\log p(t;s,l)|=0.\end{array}$$

(53)

Using absolute inequality, we have

$$\begin{array}{cc}& |F_{n,f(n)}(\omega )+\sum _{t\in \mathcal{S}}\pi (t)\sum _{s\in \mathcal{S}}\sum _{l\in \mathcal{L}}p(t;s,l)\log p(t;s,l)|\hfill \\ \hfill \le & |F_{n,f(n)}(\omega )+\frac{1}{f(n)}{\displaystyle \sum _{k=n+1}^{n+f(n)}}\sum _{t\in \mathcal{S}}\sum _{s\in \mathcal{S}}\sum _{l\in \mathcal{L}}{1}_{\{t\}}(X_{k-1})p_k(t;s,l)\log p_k(t;s,l)|\hfill \\ \hfill +& |\frac{1}{f(n)}{\displaystyle \sum _{k=n+1}^{n+f(n)}}\sum _{t\in \mathcal{S}}\sum _{s\in \mathcal{S}}\sum _{l\in \mathcal{L}}{1}_{\{t\}}(X_{k-1})[p_k(t;s,l)\log p_k(t;s,l)-p(t;s,l)\log p(t;s,l)]|\hfill \\ \hfill +& |\frac{1}{f(n)}{\displaystyle \sum _{k=n+1}^{n+f(n)}}\sum _{t\in \mathcal{S}}[\pi (t)-1_{\{t\}}(X_{k-1})]\sum _{s\in \mathcal{S}}\sum _{l\in \mathcal{L}}p(t;s,l)\log p(t;s,l)|\hfill \\ \hfill \le & |F_{n,f(n)}(\omega )+\frac{1}{f(n)}{\displaystyle \sum _{k=n+1}^{n+f(n)}}\sum _{t\in \mathcal{S}}\sum _{s\in \mathcal{S}}\sum _{l\in \mathcal{L}}{1}_{\{t\}}(X_{k-1})p_k(t;s,l)\log p_k(t;s,l)|\hfill \\ \hfill +& \sum _{t\in \mathcal{S}}\sum _{s\in \mathcal{S}}\sum _{l\in \mathcal{L}}\frac{1}{f(n)}{\displaystyle \sum _{k=n+1}^{n+f(n)}}|p_k(t;s,l)\log p_k(t;s,l)-p(t;s,l)\log p(t;s,l)|\hfill \\ \hfill +& \sum _{t\in \mathcal{S}}|\pi (t)-\frac{1}{f(n)}{\displaystyle \sum _{k=n+1}^{n+f(n)}}{1}_{\{t\}}(X_{k-1})||\sum _{s\in \mathcal{S}}\sum _{l\in \mathcal{L}}p(t;s,l)\log p(t;s,l)|\hfill \end{array}$$

(54)

where $\pi (t),t\in \mathcal{S}$ belongs to $\pi =({\pi }_1,{\pi }_2,\dots ,{\pi }_N)$, which is a stationary distribution of Q. By Lemma 3, Equation (52) follows from Equation (54). □

The proof of Theorem 1 mainly relies on the construction of inequalities, and the proof of Theorem 2 utilizes the properties of norms. The two theorems explain that the limit of the generalized entropy density is the generalized entropy rate.

Theorem 3. 

Let $\{X,Y\}=\{(X_n,Y_n),n\ge 0\}$ be an NHMM. $Q={(q(t;s))}_{N\times N}$ is another transition matrix, and assume that Q is periodic and strongly ergodic. Let

$$\begin{array}{c}\hfill {\beta }_n(t)=\sum _{s\in \mathcal{S}}\sum _{l\in \mathcal{L}}{q}_n(t;s)p_n(s;l)\log q_n(t;s)p_n(s;l),\end{array}$$

(55)

$$\begin{array}{c}\hfill \beta (t)=\sum _{s\in \mathcal{S}}\sum _{l\in \mathcal{L}}q(t;s)p(s;l)\log q(t;s)p(s;l),\end{array}$$

(56)

where ${\beta }_n(t),\beta (t),t\in \mathcal{S}$ are the elements of column vectors ${\beta }_n$ and β, respectively, and assume that $\{\Vert {\beta }_n\Vert ,n\ge 1\}$ are bounded. If Equation (22) holds,

$$\begin{array}{c}\hfill \underset{n\to \infty }{\lim }\frac{1}{f(n)}{\displaystyle \sum _{k=n+1}^{n+f(n)}}\parallel Q_k-Q\parallel =0,\end{array}$$

(57)

and

$$\begin{array}{c}\hfill \underset{n\to \infty }{\lim }\frac{1}{f(n)}{\displaystyle \sum _{k=n+1}^{n+f(n)}}\parallel {\beta }_k-\beta \parallel =0,\end{array}$$

(58)

then the generalized entropy rate of $\{X,Y\}$ exists, and

$$\begin{array}{c}\hfill \underset{n\to \infty }{\lim }\frac{1}{f(n)}H(X_n,Y_n,\cdots ,X_{n+f(n)},Y_{n+f(n)})=-\pi \beta ,a.s.\end{array}$$

(59)

where $\pi =({\pi }_1,{\pi }_2,\cdots ,{\pi }_N)$ is the unique stationary distribution determined by Q.

Proof. 

Let $q^{(k-1)}$ be a row vector with elements $\mathbb{P}(X_{k-1}=t),t\in \mathcal{S}$. Hence, using the definition and properties of ${\beta }_n$, we have

$$\begin{array}{c}\hfill H(X_k,Y_k,|X_{k-1})=-q^{(k-1)}{\beta }_k.\end{array}$$

(60)

Simply take B as a constant random matrix whose rows are equal to $\pi$. Note that $\pi =q^{(0)}B$, where $q^{(0)}$ is an initial distribution of the Markov chain. Since

$$\begin{array}{cc}& \parallel \frac{1}{f(n)}{\displaystyle \sum _{k=n+1}^{n+f(n)}}{q}^{(k-1)}-\pi \parallel \hfill \\ \hfill =& \parallel \frac{1}{f(n)}{\displaystyle \sum _{k=n+1}^{n+f(n)}}{q}^{(0)}{Q}^{(0,k-1)}-q^{(0)}B\parallel \hfill \\ \hfill \le & \parallel \frac{1}{f(n)}{\displaystyle \sum _{k=n+1}^{n+f(n)}}{Q}^{(0,k-1)}-B\parallel ,\hfill \end{array}$$

(61)

where $Q^{(0,k-1)}=Q_1\cdots Q_{k-1}$, $Q^{(0,0)}=I$ (I is the identical matrix), by Equations (57) and (61) and Lemma 5, we have

$$\begin{array}{c}\hfill \underset{n\to \infty }{\lim }\parallel \frac{1}{f(n)}{\displaystyle \sum _{k=n+1}^{n+f(n)}}{q}^{(k-1)}-\pi \parallel =0.\end{array}$$

(62)

By Equation (58) and Lemma 5, there exists a subsequence ${\{{\beta }_{n_k}\}}_{k=1}^{\infty }$ of ${\{{\beta }_n\}}_{n=1}^{\infty }$ such that

$$\begin{array}{c}\hfill \underset{k\to \infty }{\lim }\parallel {\beta }_{n_k}-\beta \parallel =0.\end{array}$$

(63)

Hence, $\parallel \beta \parallel$ is finite. By Equation (62) and properties of entropy, we have

$$\begin{array}{cc}& |\frac{1}{f(n)}H(X_n,Y_n,\cdots ,X_{n+f(n)},Y_{n+f(n)})+\pi \beta |\hfill \\ \hfill =& |\frac{1}{f(n)}H(X_n,Y_n)+\frac{1}{f(n)}{\displaystyle \sum _{k=n+1}^{n+f(n)}}H(X_k,Y_k|X_{k-1})+\pi \beta |\hfill \\ \hfill \le & |\frac{1}{f(n)}H(X_n,Y_n)|+|\frac{1}{f(n)}{\displaystyle \sum _{k=n+1}^{n+f(n)}}{q}^{(k-1)}{\beta }_k-\pi \beta |\hfill \\ \hfill \le & |\frac{1}{f(n)}H(X_n,Y_n)|+|\frac{1}{f(n)}{\displaystyle \sum _{k=n+1}^{n+f(n)}}{q}^{(k-1)}{\beta }_k-\frac{1}{f(n)}{\displaystyle \sum _{k=n+1}^{n+f(n)}}{q}^{(k-1)}\beta |+|\frac{1}{f(n)}{\displaystyle \sum _{k=n+1}^{n+f(n)}}{q}^{(k-1)}\beta -\pi \beta |\hfill \\ \hfill \le & \frac{1}{f(n)}|H(X_n,Y_n)|+\frac{1}{f(n)}{\displaystyle \sum _{k=n+1}^{n+f(n)}}\parallel {\beta }_k-\beta \parallel +\parallel \beta \parallel \parallel \frac{1}{f(n)}{\displaystyle \sum _{k=n+1}^{n+f(n)}}{q}^{(k-1)}-\pi \parallel \hfill \\ \hfill \to & 0\mathrm{as}n\to \infty .\hfill \end{array}$$

(64)

This completes the proof of Theorem 3. □

Theorem 3 gives a method to compute the generalized entropy rate of an NHMM under some mild conditions, and when the model degenerates into a homogeneous HMM and a nonhomogeneous Markov chain, the results are still the same. The two corollaries are existing results, which indirectly demonstrate the correctness of this theorem.

Corollary 1. 

Let $\{X,Y\}=\{(X_n,Y_n),n\ge 0\}$ be an NHMM with a periodic and strongly ergodic transition matrix $Q={(q(t;s))}_{N\times N}$ and an emission probability matrix $P={(p(s;l))}_{N\times M}$, where $t,s\in \mathcal{S},l\in \mathcal{L}$. Let

$$\begin{array}{c}\hfill \beta (t)=\sum _{s\in \mathcal{S}}\sum _{l\in \mathcal{L}}q(t;s)p(s;l)\log q(t;s)p(s;l),\end{array}$$

(65)

where $\beta (t),t\in \mathcal{S}$ are the elements of column vector β. Assume that $\Vert \beta \Vert$ is bounded. If Equation (22) holds, then the generalized entropy rate of $\{X,Y\}$ exists, and

$$\begin{array}{c}\hfill \underset{n\to \infty }{\lim }\frac{1}{f(n)}H(X_n,Y_n,\cdots ,X_{n+f(n)},Y_{n+f(n)})=-\pi \beta ,a.s.\end{array}$$

(66)

where $\pi =({\pi }_1,{\pi }_2,\cdots ,{\pi }_N)$ is the unique stationary distribution determined by Q.

Corollary 2. 

Let $X=\{(X_n),n\ge 0\}$ be a nonhomogeneous Markov chain. $Q={(q(t;s))}_{N\times N}$ is another transition matrix, and assume that Q is periodic and strongly ergodic, where $t,s\in \mathcal{S}$. Let

$$\begin{array}{c}\hfill {\beta }_n(t)=\sum _{s\in \mathcal{S}}{q}_n(t;s)\log q_n(t;s),\end{array}$$

(67)

$$\begin{array}{c}\hfill \beta (t)=\sum _{s\in \mathcal{S}}q(t;s)\log q(t;s),\end{array}$$

(68)

where ${\beta }_n(t),\beta (t),t\in \mathcal{S}$ are the elements of column vectors ${\beta }_n$ and β, respectively. Assume that $\{\Vert {\beta }_n\Vert ,n\ge 1\}$ are bounded. If Equation (22) holds,

$$\begin{array}{c}\hfill \underset{n\to \infty }{\lim }\frac{1}{f(n)}{\displaystyle \sum _{k=n+1}^{n+f(n)}}\parallel Q_k-Q\parallel =0,\end{array}$$

(69)

and

$$\begin{array}{c}\hfill \underset{n\to \infty }{\lim }\frac{1}{f(n)}{\displaystyle \sum _{k=n+1}^{n+f(n)}}\parallel {\beta }_k-\beta \parallel =0,\end{array}$$

(70)

then the generalized entropy rate of X exists, and

$$\begin{array}{c}\hfill \underset{n\to \infty }{\lim }\frac{1}{f(n)}H(X_n,\cdots ,X_{n+f(n)})=-\pi \beta ,a.s.\end{array}$$

(71)

where $\pi =({\pi }_1,{\pi }_2,\cdots ,{\pi }_N)$ is the unique stationary distribution determined by Q.

5. Conclusions

Entropy ergodic theorems reflect the almost surely convergence of the entropy rate to a constant. The generalized entropy rate in the form of delayed averages can overcome the redundancy of initial information while ensuring stationarity and, therefore, it has better practical value. An NHMM provides the idea by allowing the transition matrix of the hidden states in a homogeneous HMM to be related to a set of observed covariates. Although there have been fruitful achievements in the two fields of generalized entropy ergodic theorems and NHMMs, research on the generalized entropy ergodic theorems for NHMMs is limited. Therefore, we consider extending the application scenarios of the generalized entropy rate. In this paper, we give the basic concepts of NHMMs and the entropy rate in the form of delayed averages, list some lemmas including a strong limit theorem and limit properties of a norm, and prove some generalized entropy ergodic theorems with an almost surely convergence for NHMMs. These results, which are generalizations of previous findings of [17,27], provide concise formulas for the computation and estimation of the generalized entropy rate in the form of delayed averages. Theoretical analysis has proven the existence and limit formula of entropy rate, but its numerical calculation is also very complex and challenging. Specifically, this process requires calculating the integration on a certain measure, which is related to the parameters of HMMs. Many methods have been proposed for this, such as approximate formulas, series expansion, and statistical calculation methods. Due to the fact that approximate formulas are not applicable to any parameters, and their accuracy is difficult to estimate, the convergence conditions of the series are difficult to prove, and the sample size required for statistical convergence is difficult to estimate. Therefore, approximating with upper and lower limits is a reliable method, and its accuracy can be estimated, with relatively lenient requirements for HMM parameters. In the future, we will conduct numerical case studies based on the theoretical analysis above.