Exponential Strong Converse for Source Coding with Side Information at the Decoder ^†

Abstract

We consider the rate distortion problem with side information at the decoder posed and investigated by Wyner and Ziv. Using side information and encoded original data, the decoder must reconstruct the original data with an arbitrary prescribed distortion level. The rate distortion region indicating the trade-off between a data compression rate R and a prescribed distortion level $\Delta$ was determined by Wyner and Ziv. In this paper, we study the error probability of decoding for pairs of $(R,\Delta )$ outside the rate distortion region. We evaluate the probability of decoding such that the estimation of source outputs by the decoder has a distortion not exceeding a prescribed distortion level $\Delta$. We prove that, when $(R,\Delta )$ is outside the rate distortion region, this probability goes to zero exponentially and derive an explicit lower bound of this exponent function. On the Wyner–Ziv source coding problem the strong converse coding theorem has not been established yet. We prove this as a simple corollary of our result.

1. Introduction

For single or multi terminal source coding systems, the converse coding theorems state that at any data compression rates below the fundamental theoretical limit of the system the error probability of decoding cannot go to zero when the block length n of the codes tends to infinity. On the other hand, the strong converse theorems state that, at any transmission rates exceeding the fundamental theoretical limit, the error probability of decoding must go to one when n tends to infinity. The former converse theorems are sometimes called the weak converse theorems to distinguish them with the strong converse theorems.

In this paper, we study the strong converse theorem for the rate distortion problem with side information at the decoder posed and investigated by Wyner and Ziv [1]. We call the above source coding system the Wyner and Ziv source coding system (the WZ system). The WZ system is shown in Figure 1. In this figure, the WZ system corresponds to the case where the switch is close. In Figure 1, the sequence $(X^n,Y^n)$ represents independent copies of a pair of dependent random variables $(X,Y)$ which take values in the finite sets $\mathcal{X}$ and $\mathcal{Y}$, respectively. We assume that $(X,Y)$ has a probability distribution denoted by $p_{XY}$. The encoder ${\phi }^{\left(n\right)}$ outputs a binary sequence which appears at a rate R bits per input symbol. The decoder function ${\psi }^{\left(n\right)}$ observes ${\phi }^{\left(n\right)}\left(X^n\right)$ and $Y^n$ to output a sequence $Z^n$. The t-th component $Z_t$ of $Z^n$ for $t=1,2,\cdots ,n$ take values in the finite reproduction alphabet $\mathcal{Z}$. Let $d:\mathcal{X}\times \mathcal{Z}$$\to [0,\infty )$ be an arbitrary distortion measure on $\mathcal{X}\times \mathcal{Z}$. The distortion between $x^n\in {\mathcal{X}}^n$ and $z^n\in {\mathcal{Z}}^n$ is defined by

$$d(x^n,z^n):=\sum _{t=1}^{n}d(x_t,z_t).$$

In general, we have two criteria on $d(X^n,Z^n)$. One is the excess-distortion probability of decoding defined by

$${\mathrm{P}}_{\mathrm{e}}^{\left(n\right)}({\phi }^{\left(n\right)},{\psi }^{\left(n\right)};\Delta ):=Pr\left\{\frac{1}{n}d(X^n,Z^n)\ge \Delta \right\}.$$

(1)

The other is the average distortion defined by

$$\begin{array}{cc}\hfill {\Delta }_n:=& \mathrm{E}\left[\frac{1}{n}d\left(X_{,}^n{Z}^n\right)\right]:=\sum _{(x^n,z^n)\in {\mathcal{X}}^n\times {\mathcal{Z}}^n}\left[\frac{1}{n}\sum _{k=1}^{n}d(x_k,z_k)\right]\mathrm{Pr}\{X^n=x^n,Z^n=z^n\}\hfill \\ \hfill =& \frac{1}{n}\sum _{k=1}^n\sum _{(x_k,z_k)\in \mathcal{X}\times \mathcal{Z}}d(x_k,z_k)\mathrm{Pr}\left\{X_k=x_k,Z_k=z_k\right\}.\hfill \end{array}$$

A pair $(R,\Delta )$ is $\epsilon$-achievable for $p_{XY}$ if there exist a sequence of pairs $\left\{\right({\phi }^{\left(n\right)},$ ${\psi }^{\left(n\right)}{\left)\right\}}_{n\ge 1}$ such that for any $\delta >0$ and any n with $n\ge n_0=n_0(\epsilon ,\delta )$

$$\begin{array}{c}\frac{1}{n}log\parallel {\phi }^{\left(n\right)}\parallel \le R+\delta ,{\mathrm{P}}_{\mathrm{e}}^{\left(n\right)}({\phi }^{\left(n\right)},{\psi }^{\left(n\right)};\Delta )\le \epsilon ,\hfill \end{array}$$

where $\parallel {\phi }^{\left(n\right)}\parallel$ stands for the range of cardinality of ${\phi }^{\left(n\right)}$. The rate distortion region ${\mathcal{R}}_{\mathrm{WZ}}\left(\epsilon \right|p_{XY})$ is defined by

$$\begin{array}{ccc}\hfill {\mathcal{R}}_{\mathrm{WZ}}\left(\epsilon \right|p_{XY})& =& \left\{(R,\Delta ):(R,\Delta )\mathrm{is}\epsilon \text{-}\mathrm{achievable}\mathrm{for}{p}_{XY}\right\}.\hfill \end{array}$$

Furthermore, set

$${\mathcal{R}}_{\mathrm{WZ}}\left(p_{XY}\right):=\bigcap _{\epsilon >0}{\mathcal{R}}_{\mathrm{WZ}}\left(\epsilon \right|p_{XY}).$$

On the other hand, we can define a rate distortion region based on the average distortion criterion, a formal definition of which is the following. A pair $(R,\Delta )$ is achievable for $p_{XY}$ if there exist a sequence of pairs $\left\{\right({\phi }^{\left(n\right)},$${\psi }^{\left(n\right)}{\left)\right\}}_{n\ge 1}$ such that for any $\delta >0$ and any n with $n\ge n_0$$=n_0\left(\delta \right)$,

$$\begin{array}{c}\frac{1}{n}log\parallel {\phi }^{\left(n\right)}\parallel \le R+\delta ,{\Delta }^{\left(n\right)}\le \Delta +\delta .\hfill \end{array}$$

The rate distortion region ${\tilde{\mathcal{R}}}_{\mathrm{WZ}}\left(p_{XY}\right)$ is defined by

$$\begin{array}{ccc}\hfill {\tilde{\mathcal{R}}}_{\mathrm{WZ}}\left(p_{XY}\right)& :=& \left\{(R,\Delta ):(R,\Delta )\mathrm{is}\mathrm{achievable}\mathrm{for}{p}_{XY}\right\}.\hfill \end{array}$$

If the switch is open, then the side information is not available to the decoder. In this case the communication system corresponds to the source coding for the discrete memoryless source (DMS) specified with $p_X$. We define the rate distortion region ${\tilde{\mathcal{R}}}_{\mathrm{DMS}}\left(p_X\right)$ in a similar manner to the definition of ${\tilde{\mathcal{R}}}_{\mathrm{WZ}}\left(p_{XY}\right)$. We further define the region ${\mathcal{R}}_{\mathrm{DMS}}\left(\epsilon \right|p_X),\epsilon \in (0,1)$ and ${\mathcal{R}}_{\mathrm{DMS}}\left(p_X\right)$, respectively in a similar manner to the definition of ${\mathcal{R}}_{\mathrm{WZ}}\left(\epsilon \right|p_{XY})$ and ${\mathcal{R}}_{\mathrm{WZ}}\left(p_{XY}\right)$.

Previous works on the characterizations of ${\tilde{\mathcal{R}}}_{\mathrm{DMS}}\left(p_X\right)$, ${\mathcal{R}}_{\mathrm{DMS}}\left(\epsilon \right|p_X),\epsilon \in (0,1)$, and ${\mathcal{R}}_{\mathrm{DMS}}\left(p_X\right)$ are shown in

Table 1. Previous results on the converse coding theorems for DMS, WZ. Main results in the present paper on WZ are also included.

	Characterization of the Rate Distortion Region	Strong Converse	Strong Converse Exponent
DMS	Shannon [2] (1959) (Explicit form of ${\tilde{\mathcal{R}}}_{\mathrm{DMS}}\left(p_X\right)$) Wolfowitz [3] (1966) (${\tilde{\mathcal{R}}}_{\mathrm{DMS}}\left(p_X\right)={\mathcal{R}}_{\mathrm{DMS}}\left(p_X\right)$)	Wolfowitz [3] (1966) (${\mathcal{R}}_{\mathrm{DMS}}\left(\epsilon \right|p_X)={\mathcal{R}}_{\mathrm{DMS}}\left(p_X\right)$ for any $\epsilon \in (0,1)$)	Csiszár and Körner [4] (1981) (The optimal exponent)
WZ	Wyner and Ziv [1] (1976) (Explicit form of ${\tilde{\mathcal{R}}}_{\mathrm{WZ}}\left(p_{XY}\right)$) Csiszár and Körner [4] (1981) (${\tilde{\mathcal{R}}}_{\mathrm{WZ}}\left(p_{XY}\right)={\mathcal{R}}_{\mathrm{WZ}}\left(p_{XY}\right)$)	Corollary 2 (Outer bound with $O\left(1/\sqrt{n}\right)$ gap from the rate distortion region, ${\mathcal{R}}_{\mathrm{WZ}}\left(\epsilon \right|p_{XY})={\mathcal{R}}_{\mathrm{WZ}}\left(p_{XY}\right)$ for any $\epsilon \in (0,1)$)	Theorem 3 (Lower bound F of the opt. exp. G)

. Shannon [2] determined ${\tilde{\mathcal{R}}}_{\mathrm{DMS}}\left(p_X\right)$. Subsequently, Wolfowiz [3] proved that ${\tilde{\mathcal{R}}}_{\mathrm{DMS}}\left(p_X\right)={\mathcal{R}}_{\mathrm{DMS}}\left(p_X\right).$ Furthermore, he proved the strong converse theorem. That is, if $(R,\Delta )\notin$ ${\tilde{\mathcal{R}}}_{\mathrm{DMS}}\left(p_X\right)$, then for any sequence ${\left\{({\phi }^{\left(n\right)},{\psi }^{\left(n\right)})\right\}}_{n=1}^{\infty }$ of encoder and decoder functions satisfying the condition

$$\underset{n\to \infty }{lim\; sup}\frac{1}{n}log\left|\right|{\phi }^{\left(n\right)}\left|\right|\le R,$$

(2)

we have

$$\underset{n\to \infty }{lim}{\mathrm{P}}_{\mathrm{e}}^{\left(n\right)}({\phi }^{\left(n\right)},{\psi }^{\left(n\right)};\Delta )=\underset{n\to \infty }{lim}Pr\left\{\frac{1}{n_k}d(X^n,Z^n)\ge \Delta \right\}=1.$$

(3)

The above strong converse theorem implies that, for any $\epsilon \in (0,1)$,

$${\tilde{\mathcal{R}}}_{\mathrm{DMS}}\left(p_X\right)={\mathcal{R}}_{\mathrm{DMS}}\left(p_X\right)={\mathcal{R}}_{\mathrm{DMS}}\left(\epsilon \right|p_X).$$

Csiszár and Körner proved that in Equation (3), the probability ${\mathrm{P}}_{\mathrm{e}}^{\left(n\right)}({\phi }^{\left(n\right)},{\psi }^{\left(n\right)};\Delta )$ converges to one exponentially and determined the optimal exponent as a function of $(R,\Delta )$.

The previous works on the coding theorems for the WZ system are summarized in Table 1. The rate distortion region ${\tilde{\mathcal{R}}}_{\mathrm{WZ}}\left(p_{XY}\right)$ was determined by Wyner and Ziv [1]. Csiszár and Körner [4] proved that ${\tilde{\mathcal{R}}}_{\mathrm{WZ}}\left(p_{XY}\right)$ $={\mathcal{R}}_{\mathrm{WZ}}\left(p_{XY}\right)$. On the other hand, we have had no result on the strong converse theorem for the WZ system.

Main results of this paper are summarized in Table 1. For the WZ system, we prove that if $(R,\Delta )$ is out side the rate distortion region ${\mathcal{R}}_{\mathrm{WZ}}\left(p_{XY}\right)$, then we have that for any sequence ${\left\{({\phi }^{\left(n\right)},{\psi }^{\left(n\right)})\right\}}_{n=1}^{\infty }$ of encoder and decoder functions satisfying the conditionin Equation (2), the quantity ${\mathrm{P}}_{\mathrm{e}}^{\left(n\right)}({\phi }^{\left(n\right)},{\psi }^{\left(n\right)};\Delta )$ goes to zero exponentially and derive an explicit lower bound of this exponent function. This result corresponds to Theorem 3 in Table 1. As a corollary from this theorem, we obtain the strong converse result, which is stated in Corollary 2 in Table 1. This results states that we have an outer bound with $O\left(1\sqrt{n}\right)$ gap from the rate distortion region ${\mathcal{R}}_{\mathrm{WZ}}\left(p_{XY}\right)$.

To derive our result, we use a new method called the recursive method. This method is a general powerful tool to prove strong converse theorems for several coding problems in information theory. In fact, the recursive method plays important roles in deriving exponential strong converse exponent for communication systems treated in [5,6,7,8].

2. Source Coding with Side Information at the Decoder

In the following argument, the operations ${\mathrm{E}}_p[·]$ and ${\mathrm{Var}}_p[·]$, respectively, stand for the expectation and the variance with respect to a probability distribution p. When the value of p is obvious from the context, we omit the suffix p in those operations to simply write $\mathrm{E}[·]$ and $\mathrm{Var}[·]$. Let $\mathcal{X}$ and $\mathcal{Y}$ be finite sets and ${\left\{(X_t,Y_t)\right\}}_{t=1}^{\infty }$ be a stationary discrete memoryless source. For each $t=1,2,\cdots$, the random pair $(X_t,Y_t)$ takes values in $\mathcal{X}\times \mathcal{Y}$, and has a probability distribution

$$p_{XY}={\left\{p_{XY}(x,y)\right\}}_{(x,y)\in \mathcal{X}\times \mathcal{Y}}.$$

We write n independent copies of ${\left\{X_t\right\}}_{t=1}^{\infty }$ and ${\left\{Y_t\right\}}_{t=1}^{\infty }$, respectively, as

$$X^n=X_1,X_2,\cdots ,X_n \mathrm{and} Y^n=Y_1,Y_2,\cdots ,Y_n.$$

We consider a communication system depicted in Figure 2. Data sequences $X^n$ is separately encoded to ${\phi }^{\left(n\right)}\left(X^n\right)$ and is sent to the information processing center. At the centerm the decoder function ${\psi }^{\left(n\right)}$ observes ${\phi }^{\left(n\right)}\left(X^n\right)$ and $Y^n$ to output the estimation $Z^n$ of $X^n$. The encoder function ${\phi }^{\left(n\right)}$ is defined by

$$\begin{array}{c}{\phi }^{\left(n\right)}:{\mathcal{X}}^n\to {\mathcal{M}}_n=\left\{1,2,\cdots ,M_n\right\}.\hfill \end{array}$$

(4)

Let $\mathcal{Z}$ be a reproduction alphabet. The decoder function ${\psi }^{\left(n\right)}$ is defined by

$${\psi }^{\left(n\right)}:{\mathcal{M}}_n\times {\mathcal{Y}}^n\to {\mathcal{Z}}^n.$$

(5)

Let $d:\mathcal{X}\times \mathcal{Z}$$\to [0,\infty )$ be an arbitrary distortion measure on $\mathcal{X}\times \mathcal{Z}$. The distortion between $x^n\in {\mathcal{X}}^n$ and $z^n\in {\mathcal{Z}}^n$ is defined by

$$d(x^n,z^n):=\sum _{t=1}^{n}d(x_t,z_t).$$

The excess-distortion probability of decoding is

$${\mathrm{P}}_{\mathrm{e}}^{\left(n\right)}({\phi }^{\left(n\right)},{\psi }^{\left(n\right)};\Delta )=Pr\left\{\frac{1}{n}d(X^n,Z^n)\ge \Delta \right\},$$

(6)

where $Z^n={\psi }^{\left(n\right)}({\phi }^{\left(n\right)}\left(X^n\right),Y^n)$. The average distortion ${\Delta }^{\left(n\right)}$ between $X^n$ and $Z^n$ is defined by

$${\Delta }^{\left(n\right)}:=\frac{1}{n}\mathrm{E}\left[d(X^n,Z^n)\right]:=\frac{1}{n}\sum _{t=1}^n\mathrm{E}d(X_t,Z_t).$$

In the previous section, we gave the formal definitions of ${\mathcal{R}}_{\mathrm{WZ}}\left(\epsilon \right|$$p_{XY})$, $\epsilon \in (0,1)$, ${\mathcal{R}}_{\mathrm{WZ}}\left(p_{XY}\right)$, and ${\tilde{\mathcal{R}}}_{\mathrm{WZ}}\left(p_{XY}\right)$. We can show that the above three rate distortion regions satisfy the following property.

Property 1.

(a) 

The regions ${\mathcal{R}}_{\mathrm{WZ}}\left(\epsilon \right|p_{XY})$, $\epsilon \in (0,1)$, ${\mathcal{R}}_{\mathrm{WZ}}\left(p_{XY}\right)$, and ${\tilde{\mathcal{R}}}_{\mathrm{WZ}}\left(p_{XY}\right)$ are closed convex sets of ${\mathbb{R}}_{+}^2$, where

$$\begin{array}{ccc}\hfill {\mathbb{R}}_{+}^2& :=& \left\{\right(R,\Delta ):R\ge 0,\Delta \ge 0\}.\hfill \end{array}$$

(b) 

${\mathcal{R}}_{\mathrm{WZ}}\left(\epsilon \right|p_{XY})$ has another form using $(n,\epsilon )$-rate distortion region, the definition of which is as follows. We set

$$\begin{array}{ccc}\hfill {\mathcal{R}}_{\mathrm{WZ}}(n,\epsilon |p_{XY})& =& \{(R,\Delta ):There exists({\phi }^{\left(n\right)},{\psi }^{\left(n\right)})such that\hfill \\ & & \frac{1}{n}log\left|\right|{\phi }^{\left(n\right)}\left|\right|\le R,{\mathrm{P}}_{\mathrm{e}}^{\left(n\right)}({\phi }^{\left(n\right)},{\psi }^{\left(n\right)};\Delta )\le \epsilon \},\hfill \end{array}$$

which is called the $(n,\epsilon )$-rate distortion region. Using ${\mathcal{R}}_{\mathrm{WZ}}(n,\epsilon |p_{XY})$, ${\mathcal{R}}_{\mathrm{WZ}}\left(\epsilon \right|p_{XY})$ can be expressed as

$$\begin{array}{ccc}\hfill {\mathcal{R}}_{\mathrm{WZ}}\left(\epsilon \right|p_{XY})& =& \mathrm{cl}\left(\bigcup _{m\ge 1}\bigcap _{n\ge m}{\mathcal{R}}_{\mathrm{WZ}}(n,\epsilon |p_{XY})\right),\hfill \end{array}$$

where $\mathrm{cl}(·)$ stands for the closure operation.

Proof of this property is given in Appendix A.

It is well known that ${\tilde{\mathcal{R}}}_{\mathrm{WZ}}\left(p_{XY}\right)$ was determined by Wyner and Ziv [1]. To describe their result we introduce auxiliary random variables U and Z, respectively, taking values in finite sets $\mathcal{U}$ and $\mathcal{Z}$. We assume that the joint distribution of $(U,X,Y,Z)$ is

$$\begin{array}{c}{p}_{UXYZ}(u,x,y,z)=p_U\left(u\right)p_{X|U}\left(x\right|u)p_{Y|X}\left(y\right|x)p_{Z|UY}\left(z\right|u,y).\hfill \end{array}$$

The above condition is equivalent to

$$U\leftrightarrow X\leftrightarrow Y,X\leftrightarrow (U,Y)\leftrightarrow Z.$$

Define the set of probability distribution $p=p_{UXYZ}$ by

$$\begin{array}{ccc}\hfill \mathcal{P}\left(p_{XY}\right)& :=& \{p=p_{UXYZ}:|\mathcal{U}|\le |\mathcal{X}|+1,U\leftrightarrow X\leftrightarrow Y,X\leftrightarrow (U,Y)\leftrightarrow Z\},\hfill \\ \hfill {\mathcal{P}}^{*}\left(p_{XY}\right)& :=& \{p=p_{UXYZ}:|\mathcal{U}|\le |\mathcal{X}|+1,U\leftrightarrow X\leftrightarrow Y,Z=\varphi (U,Y)\mathrm{for} \mathrm{some}\varphi :\mathcal{U}\times \mathcal{Y}\to \mathcal{Z}\}.\hfill \end{array}$$

By definitions, it is obvious that ${\mathcal{P}}^{*}\left(p_{XY}\right)\subseteq \mathcal{P}\left(p_{XY}\right)$. Set

$$\begin{array}{ccc}\hfill \mathcal{R}\left(p\right)& :=& \begin{array}{c}\{(R,\Delta ):R,\Delta \ge 0,\begin{array}{ccc}\hfill R& \ge & I_p(X;U|Y),\Delta \ge {\mathrm{E}}_{p}d(X,Z)\},\hfill \end{array}\hfill \end{array}\hfill \\ \hfill \mathcal{R}\left(p_{XY}\right)& :=& \bigcup _{p\in \mathcal{P}\left(p_{XY}\right)}\mathcal{R}\left(p\right),{\mathcal{R}}^{*}\left(p_{XY}\right):=\bigcup _{p\in {\mathcal{P}}^{*}\left(p_{XY}\right)}\mathcal{R}\left(p\right).\hfill \end{array}$$

We can show that the above functions and sets satisfy the following property:

Property 2.

(a) 

The region $\mathcal{R}\left(p_{XY}\right)$ is a closed convex set of ${\mathbb{R}}_{+}^2$.

(b) 

For any $p_{XY}$, we have

$$\mathcal{R}\left(p_{XY}\right)={\mathcal{R}}^{*}\left(p_{XY}\right).$$

Proof of Property 2 is given in Appendix C. In Property 2 Part (b), $\mathcal{R}\left(p_{XY}\right)$ is regarded as another expression of ${\mathcal{R}}^{*}\left(p_{XY}\right)$. This expression is useful for deriving our main result. The rate region ${\mathcal{R}}_{\mathrm{WZ}}\left(p_{XY}\right)$ was determined by Wyner and Ziv [1]. Their result is the following:

Theorem 1 

(Wyner and Ziv [1]).

$$\begin{array}{c}{\tilde{\mathcal{R}}}_{\mathrm{WZ}}\left(p_{XY}\right)={\mathcal{R}}^{*}\left(p_{XY}\right)=\mathcal{R}\left(p_{XY}\right).\hfill \end{array}$$

On ${\mathcal{R}}_{\mathrm{WZ}}\left(p_{XY}\right)$, Csiszár and Körner [4] obtained the following result.

Theorem 2 

(Csiszár and Körner [4]).

$$\begin{array}{c}{\mathcal{R}}_{\mathrm{WZ}}\left(p_{XY}\right)={\tilde{\mathcal{R}}}_{\mathrm{WZ}}\left(p_{XY}\right)={\mathcal{R}}^{*}\left(p_{XY}\right)=\mathcal{R}\left(p_{XY}\right).\hfill \end{array}$$

We are interested in an asymptotic behavior of the error probability of decoding to tend to one as $n\to \infty$ for $(R,\Delta )\notin {\mathcal{R}}_{\mathrm{WZ}}\left(p_{XY}\right)$. To examine the rate of convergence, we define the following quantity. Set

$$\begin{array}{c}{\mathrm{P}}_{\mathrm{c}}^{\left(n\right)}({\phi }^{\left(n\right)},{\psi }^{\left(n\right)};\Delta ):=1-{\mathrm{P}}_{\mathrm{e}}^{\left(n\right)}({\phi }^{\left(n\right)},{\psi }^{\left(n\right)};\Delta ),\hfill \\ G^{\left(n\right)}(R,\Delta |p_{XY}):=\underset{\genfrac{}{}{0pt}{}{({\phi }^{\left(n\right)},{\psi }^{\left(n\right)}):}{(1/n)log\parallel {\phi }^{\left(n\right)}\parallel \le R}}{min}\left(-\frac{1}{n}\right)log{\mathrm{P}}_{\mathrm{c}}^{\left(n\right)}({\phi }^{\left(n\right)},{\psi }^{\left(n\right)};\Delta ).\hfill \end{array}$$

By time sharing, we have that

$$\begin{array}{ccc}\hfill G^{(n+m)}\left(\left.\frac{nR+m{R}^{\prime }}{n+m},\frac{n\Delta +m{\Delta }^{\prime }}{n+m}\right|p_{XY}\right)& \le & \frac{n{G}^{\left(n\right)}(R,\Delta |p_{XY})+m{G}^{\left(m\right)}(R^{\prime },{\Delta }^{\prime }|p_{XY})}{n+m}.\hfill \end{array}$$

(7)

Choosing $R=R^{\prime }$ and $\Delta ={\Delta }^{\prime }$ in Equation (7), we obtain the following subadditivity property on $\{G^{\left(n\right)}(R,\Delta |p_{XY})$${\}}_{n\ge 1}$:

$$\begin{array}{ccc}\hfill G^{(n+m)}(R,\Delta |p_{XY})& \le & \frac{n{G}^{\left(n\right)}(R,\Delta |p_{XY})+m{G}^{\left(m\right)}(R,\Delta |p_{XY})}{n+m},\hfill \end{array}$$

which together with Fekete’s lemma yields that $G^{\left(n\right)}(R,$$\Delta |p_{XY})$ exists and satisfies the following:

$$\begin{array}{c}\hfill \underset{n\to \infty }{lim}{G}^{\left(n\right)}(R,\Delta |p_{XY})=\underset{n\ge 1}{inf}{G}^{\left(n\right)}(R,\Delta |p_{XY}).\end{array}$$

Set

$$\begin{array}{c}G(R,\Delta |p_{XY}):=\underset{n\to \infty }{lim}{G}^{\left(n\right)}(R,\Delta |p_{XY}),\hfill \\ \mathcal{G}\left(p_{XY}\right):=\{(R,\Delta ,G):G\ge G(R,\Delta |p_{XY})\}.\hfill \end{array}$$

The exponent function $G(R,\Delta |p_{XY})$ is a convex function of $(R,\Delta )$. In fact, from Equation (7), we have that for any $\alpha \in [0,1]$

$$\begin{array}{ccc}\hfill G(\alpha R+\overline{\alpha }{R}^{\prime },\alpha \Delta +\overline{\alpha }{\Delta }^{\prime }|p_{XY})& \le & \alpha G(R,\Delta |p_{XY})+\overline{\alpha }G(R^{\prime },{\Delta }^{\prime }|p_{XY}),\hfill \end{array}$$

where $\overline{\alpha }=1-\alpha$. The region $\mathcal{G}\left(p_{XY}\right)$ is also a closed convex set. Our main aim is to find an explicit characterization of $\mathcal{G}\left(p_{XY}\right)$. In this paper, we derive an explicit outer bound of $\mathcal{G}$ $\left(p_{XY}\right)$ whose section by the plane $G=0$ coincides with ${\mathcal{R}}_{\mathrm{WZ}}\left(p_{XY}\right)$.

3. Main Results

In this section, we state our main results. We first explain that the rate distortion region $\mathcal{R}\left(p_{XY}\right)$ can be expressed with two families of supporting hyperplanes. To describe this result, we define two sets of probability distributions on $\mathcal{U}\times \mathcal{X}\times \mathcal{Y}\times \mathcal{Z}$ by

$$\begin{array}{ccc}\hfill {\mathcal{P}}_{\mathrm{sh}}\left(p_{XY}\right)& :=& \{p_{UXYZ}:|\mathcal{U}|\le |\mathcal{X}|,U\leftrightarrow X\leftrightarrow Y,X\leftrightarrow (U,Y)\leftrightarrow Z\}.\hfill \\ \hfill \mathcal{Q}& :=& \{q=q_{UXYZ}:|\mathcal{U}|\le |\mathcal{X}\left|\right\}.\hfill \end{array}$$

Let $\overline{\mu }=1-\mu$. We set

$$\begin{array}{c}{R}^{\left(\mu \right)}\left(p_{XY}\right):=\underset{p\in {\mathcal{P}}_{\mathrm{sh}}\left(p_{XY}\right)}{min}\left\{\overline{\mu }{I}_p(X;U|Y)+\mu {\mathrm{E}}_{p}d(X;Z)\right\},\hfill \\ {\mathcal{R}}_{\mathrm{sh}}\left(p_{XY}\right):=\bigcap _{\mu \in [0,1]}\{(R,\Delta ):\overline{\mu }R+\mu \Delta \ge R^{\left(\mu \right)}\left(p_{XY}\right)\}.\hfill \end{array}$$

Then, we have the following property:

Property 3.

For any $p_{XY}$, we have

$${\mathcal{R}}_{\mathrm{sh}}\left(p_{XY}\right)=\mathcal{R}\left(p_{XY}\right).$$

(8)

Proof of Property 3 is given in Appendix D. For $\mu \in [0,1]$ and $\lambda ,\alpha \ge 0$, define

$$\begin{array}{c}{\omega }_{q\left|\right|p}^{(\mu ,\lambda )}(x,y,z|u)\hfill \\ :=log\left[\frac{q_X\left(x\right)q_{Y|XU}\left(y\right|x,u)q_{Z|UYX}\left(z\right|u,y,x)}{p_X\left(x\right)p_{Y|X}\left(y\right|x)q_{Z|UY}\left(z\right|u,y)}\right]+\lambda \left[\overline{\mu }log\frac{q_{X|YU}\left(x\right|y,u)}{p_{X|Y}\left(x\right|y)}+\mu d(x,z)\right],\hfill \\ {\mathsf{\Omega }}^{(\mu ,\lambda ,\alpha )}\left(q\right|p_{XY}):=-log{\mathrm{E}}_q\left[exp\left\{-\alpha {\omega }_{q\left|\right|p}^{(\mu ,\lambda )}(X,Y,Z|U)\right\}\right],\hfill \\ {\mathsf{\Omega }}^{(\mu ,\lambda ,\alpha )}\left(p_{XY}\right):=\underset{\genfrac{}{}{0pt}{}{}{q\in \mathcal{Q}}}{min}{\mathsf{\Omega }}^{(\mu ,\lambda ,\alpha )}\left(q\right|p_{XY}),\hfill \\ F^{(\mu ,\lambda ,\alpha )}(\overline{\mu }R+\mu \Delta |p_{XY}):=\frac{{\mathsf{\Omega }}^{(\mu ,\lambda ,\alpha )}\left(p_{XY}\right)-\lambda \alpha (\overline{\mu }R+\mu \Delta )}{1+(4+\lambda \overline{\mu })\alpha }.\hfill \end{array}$$

Furthermore, set

$$\begin{array}{c}F(R,\Delta |p_{XY}):=\underset{\mu \in [0,1],\lambda ,\alpha \ge 0}{sup}{F}^{(\mu ,\lambda ,\alpha )}(\overline{\mu }R+\mu \Delta |p_{XY}),\hfill \\ \overline{\mathcal{G}}\left(p_{XY}\right):=\left\{(R,\Delta ,G):G\ge F(R,\Delta |p_{XY})\right\}.\hfill \end{array}$$

We next define a functions serving as a lower bound of $F(R,\Delta |p_{XY})$. For each $p=p_{UXYZ}\in {\mathcal{P}}_{\mathrm{sh}}\left(p_{XY}\right)$, define

$$\begin{array}{c}{\tilde{\omega }}_p^{\left(\mu \right)}(x,y,z|u):=\left[\overline{\mu }log\frac{p_{X|YU}\left(x\right|y,u)}{p_{X|Y}\left(x\right|y)}+\mu d(x,z)\right],\hfill \\ {\tilde{\mathsf{\Omega }}}^{(\mu ,\lambda )}\left(p\right):=-log{\mathrm{E}}_p\left[exp\left\{-\lambda {\omega }_p^{\left(\mu \right)}(X,Y,Z|U)\right\}\right].\hfill \end{array}$$

Furthermore, set

$$\begin{array}{c}{\tilde{\mathsf{\Omega }}}^{(\mu ,\lambda )}\left(p_{XY}\right):=\underset{\genfrac{}{}{0pt}{}{}{p\in {\mathcal{P}}_{\mathrm{sh}}\left(p_{XY}\right)}}{min}{\tilde{\mathsf{\Omega }}}^{(\mu ,\lambda )}\left(p\right),\hfill \\ {\tilde{F}}^{(\mu ,\lambda )}(\mu R+\overline{\mu }\Delta |p_{XY}):=\frac{{\tilde{\mathsf{\Omega }}}^{(\mu ,\lambda )}\left(p_{XY}\right)-\lambda (\overline{\mu }R+\mu \Delta )}{5+\lambda (1+\overline{\mu })},\hfill \end{array}$$

$$\begin{array}{c}\tilde{F}(R,\Delta |p_{XY}):=\underset{\lambda \ge 0,\mu \in [0,1]}{sup}{\tilde{F}}^{(\mu ,\lambda )}(\overline{\mu }R+\mu \Delta |p_{XY}).\hfill \end{array}$$

We can show that the above functions satisfies the following properties:

Property 4.

(a) 

The cardinality bound $\left|\mathcal{U}\right|\le \left|\mathcal{X}\right|$ appearing in $\mathcal{Q}$ is sufficient to describe the quantity ${\mathsf{\Omega }}^{(\mu ,\lambda ,\alpha )}\left(p_{XY}\right)$. Furthermore, the cardinality bound $\left|\mathcal{U}\right|\le \left|\mathcal{X}\right|$ in ${\mathcal{P}}_{\mathrm{sh}}$ is sufficient to describe the quantity ${\tilde{\mathsf{\Omega }}}^{(\mu ,\lambda )}\left(p_{XY}\right)$.

(b) 

For any $R,\Delta \ge 0$, we have

$$F(R,\Delta |p_{XY})\ge \tilde{F}(R,\Delta |p_{XY}).$$

(c) 

Fix any $p=p_{UXY}\in {\mathcal{P}}_{\mathrm{sh}}\left(p_{XY}\right)$ and $\mu \in [0,1]$. For $\lambda \in [0,1]$, ${\tilde{\mathsf{\Omega }}}^{(\mu ,\lambda )}\left(p\right)$ exists and is nonnegative. For $p=$$p_{UXYZ}\in {\mathcal{P}}_{\mathrm{sh}}\left(p_{XY}\right)$, define a probability distribution $p^{\left(\lambda \right)}=p_{UXYZ}^{\left(\lambda \right)}$ by

$$\begin{array}{ccc}\hfill p^{\left(\lambda \right)}(u,x,y,z)& :=& \frac{p(u,x,y,z)exp\left\{-\lambda {\tilde{\omega }}_p^{\left(\mu \right)}(x,y,z|u)\right\}}{{\mathrm{E}}_p\left[exp\left\{-\lambda {\tilde{\omega }}_p^{\left(\mu \right)}(X,Y,Z|U)\right\}\right]}.\hfill \end{array}$$

Then, for $\lambda \in [0,1/2]$, ${\tilde{\mathsf{\Omega }}}^{(\mu ,\lambda )}\left(p\right)$ is twice differentiable. Furthermore, for $\lambda \in [0,1/2]$, we have

$$\begin{array}{c}\frac{\mathrm{d}}{\mathrm{d}\lambda }{\tilde{\mathsf{\Omega }}}^{(\mu ,\lambda )}\left(p\right)={\mathrm{E}}_{p^{\left(\lambda \right)}}\left[{\omega }_p^{\left(\mu \right)}(X,Y,Z|U)\right],\hfill \\ \frac{{\mathrm{d}}^2}{\mathrm{d}{\lambda }^2}{\tilde{\mathsf{\Omega }}}^{(\mu ,\lambda )}\left(p\right)=-{\mathrm{Var}}_{p^{\left(\lambda \right)}}\left[{\tilde{\omega }}_p^{\left(\mu \right)}(X,Y,Z|U)\right].\hfill \end{array}$$

The second equality implies that ${\tilde{\mathsf{\Omega }}}^{(\mu ,\lambda )}\left(p\right)$ is a concave function of $\lambda \in [0,1/2]$.

(d) 

For $(\mu ,\lambda )\in [0,1]\times [0,1/2]$, define

$$\begin{array}{c}{\rho }^{(\mu ,\lambda )}\left(p_{XY}\right):=\underset{\genfrac{}{}{0pt}{}{(\nu ,p)\in [0,\lambda ]}{\genfrac{}{}{0pt}{}{\times {\mathcal{P}}_{\mathrm{sh}}\left(p_{XY}\right):}{\genfrac{}{}{0pt}{}{{\tilde{\mathsf{\Omega }}}^{(\mu ,\lambda )}\left(p\right)}{={\tilde{\mathsf{\Omega }}}^{(\mu ,\lambda )}\left(p_{XY}\right)}}}}{max}{\mathrm{Var}}_{p^{\left(\nu \right)}}\left[{\tilde{\omega }}_p^{\left(\mu \right)}(X,Y,Z|U)\right],\hfill \end{array}$$

and set

$$\begin{array}{c}\rho =\rho \left(p_{XY}\right):=\underset{(\mu ,\lambda )\in [0,1]\times [0,1/2]}{max}{\rho }^{(\mu ,\lambda )}\left(p_{XY}\right).\hfill \end{array}$$

Then, we have $\rho \left(p_{XY}\right)<\infty$. Furthermore, for any $(\mu ,\lambda )\in [0,1]\times [0,1/2]$,

$${\tilde{\mathsf{\Omega }}}^{(\mu ,\lambda )}\left(p_{XY}\right)\ge \lambda R^{\left(\mu \right)}\left(p_{XY}\right)-\frac{{\lambda }^2}{2}\rho \left(p_{XY}\right).$$

(9)

(e) 

For every $\tau \in (0,(1/2)\rho \left(p_{XY}\right))$, the condition $(R+\tau ,\Delta +\tau )\notin \mathcal{R}\left(p_{XY}\right)$ implies

$$\tilde{F}(R,\Delta |p_{XY})>\frac{\rho \left(p_{XY}\right)}{10}·g^2\left(\frac{\tau }{\rho \left(p_{XY}\right)}\right)>0,$$

where g is the inverse function of $\vartheta \left(a\right):=a+(1/5)a^2,a\ge 0$.

Proof of Property 4 Part (a) is given in Appendix B. Proof of Property 4 Part (b) is given in Appendix E. Proofs of Property 4 Parts (c), (d), and (e) are given in Appendix F.

Our main result is the following:

Theorem 3.

For any $R,\Delta \ge 0$, any $p_{XY}$, and for any $({\phi }^{\left(n\right)},$${\psi }^{\left(n\right)})$ satisfying $(1/n)log\left|\right|{\phi }^{\left(n\right)}\left|\right|\le R,$ we have

$${\mathrm{P}}_{\mathrm{c}}^{\left(n\right)}({\phi }^{\left(n\right)},{\psi }^{\left(n\right)};\Delta )\le 5exp\left\{-nF(R,\Delta |p_{XY})\right\}.$$

(10)

It follows from Theorem 3 and Property 4 Part (d) that if $(R,\Delta )$ is outside the rate distortion region, then the error probability of decoding goes to one exponentially and its exponent is not below $F(R,\Delta |p_{XY})$.

It immediately follows from Theorem 3 that we have the following corollary.

Corollary 1.

For any $R,\Delta \ge 0$ and any $p_{XY}$, we have

$$\begin{array}{c}G(R,\Delta |p_{XY})\ge F(R,\Delta |p_{XY}).\hfill \end{array}$$

(11)

Furthermore, for any $p_{XY}$, we have

$$\begin{array}{c}\mathcal{G}\left(p_{XY}\right)\subseteq \overline{\mathcal{G}}\left(p_{XY}\right):=\left\{(R,\Delta ,G):G\ge F(R,\Delta |p_{XY})\right\}.\hfill \end{array}$$

(12)

Proof of Theorem 3 will be given in the next section. The exponent function in the case of $\Delta =0$ can be obtained as a corollary of the result of Oohama and Han [9] for the separate source coding problem of correlated sources [10]. The techniques used by them is a method of types [4], which is not useful for proving Theorem 3. In fact, when we use this method, it is very hard to extract a condition related to the Markov chain condition $U\leftrightarrow X\leftrightarrow Y$, which the auxiliary random variable $U\in \mathcal{U}$ must satisfy when $(R,\Delta )$ is on the boundary of the set $\mathcal{R}\left(p_{XY}\right)$. Some novel techniques based on the information spectrum method introduced by Han [11] are necessary to prove this theorem.

From Theorem 3 and Property 4 Part (e), we can obtain an explicit outer bound of ${\mathcal{R}}_{\mathrm{WZ}}\left(\epsilon \right|p_{XY})$ with an asymptotically vanishing deviation from ${\mathcal{R}}_{\mathrm{WZ}}\left(p_{XY}\right)$ $=\mathcal{R}\left(p_{XY}\right)$. The strong converse theorem immediately follows from this corollary. To describe this outer bound, for $\kappa >0$, we set

$$\begin{array}{ccc}\hfill \mathcal{R}\left(p_{XY}\right)-\kappa (1,1)& :=& \{(R-\kappa ,\Delta -\kappa ):(R,\Delta )\in \mathcal{R}\left(p_{XY}\right)\},\hfill \end{array}$$

which serves as an outer bound of $\mathcal{R}\left(p_{XY}\right)$. For each fixed $\epsilon \in (0,1)$, we define ${\kappa }_n$$={\kappa }_n(\epsilon ,\rho \left(p_{XY}\right))$ by

$$\begin{array}{ccc}\hfill {\kappa }_n& :=& \rho \left(p_{XY}\right)\vartheta \left(\sqrt{\frac{10}{n\rho \left(p_{XY}\right)}log\left(\frac{5}{1-\epsilon }\right)}\right)\hfill \\ & \stackrel{\left(\mathrm{a}\right)}{=}& \sqrt{\frac{10\rho \left(p_{XY}\right)}{n}log\left(\frac{5}{1-\epsilon }\right)}+\frac{2}{n}log\left(\frac{5}{1-\epsilon }\right).\hfill \end{array}$$

(13)

Step (a) follows from $\vartheta \left(a\right)=a+(1/5)a^2$. Since ${\kappa }_n\to 0$ as $n\to \infty$, we have the smallest positive integer $n_0=n_0(\epsilon ,\rho \left(p_{XY}\right))$ such that ${\kappa }_n\le (1/2)\rho \left(p_{XY}\right)$ for $n\ge n_0$. From Theorem 3 and Property 4 Part (e), we have the following corollary.

Corollary 2.

For each fixed ε$\in (0,1)$, we choose the above positive integer $n_0=$$n_0(\epsilon ,\rho \left(p_{XY}\right))$ Then, for any $n\ge n_0$, we have

$$\begin{array}{c}{\mathcal{R}}_{\mathrm{WZ}}(n,\epsilon |p_{XY})\subseteq \mathcal{R}\left(p_{XY}\right)-{\kappa }_n(1,1).\hfill \end{array}$$

The above result together with

$$\begin{array}{ccc}\hfill {\mathcal{R}}_{\mathrm{WZ}}\left(\epsilon \right|p_{XY})& =& \mathrm{cl}\left(\bigcup _{m\ge 1}\bigcap _{n\ge m}{\mathcal{R}}_{\mathrm{WZ}}(n,\epsilon |p_{XY})\right),\hfill \end{array}$$

yields that for each fixed $\epsilon \in (0,1)$, we have

$$\begin{array}{c}{\mathcal{R}}_{\mathrm{WZ}}\left(\epsilon \right|p_{XY})={\mathcal{R}}_{\mathrm{WZ}}\left(p_{XY}\right)=\mathcal{R}\left(p_{XY}\right).\hfill \end{array}$$

Proof of this corollary will be given in the next section.

The direct part of coding theorem, i.e., the inclusion of $\mathcal{R}\left(p_{XY}\right)$⊆${\mathcal{R}}_{\mathrm{WZ}}\left(\epsilon \right|p_{XY})$ was established by Csiszár and Körner [4]. They proved a weak converse theorem to obtain the inclusion ${\mathcal{R}}_{\mathrm{WZ}}\left(p_{XY}\right)$ $\subseteq \mathcal{R}\left(p_{XY}\right)$. Until now, we have had no result on the strong converse theorem. The above corollary stating the strong converse theorem for the Wyner–Ziv source coding problem implies that a long standing open problem since Csiszár and Körner [4] has been resolved.

4. Proof of the Main Results

In this section, we prove Theorem 3 and Corollary 2. We first present a lemma which upper bounds the correct probability of decoding by the information spectrum quantities. We set

$$S_n:={\phi }^{\left(n\right)}\left(X^n\right),Z^n:={\psi }_n({\phi }^{\left(n\right)}\left(X^n\right),Y^n).$$

It is obvious that

$$S_n\leftrightarrow X^n\leftrightarrow Y^n,X^n\leftrightarrow (S_n,Y^n)\leftrightarrow Z^n.$$

Then, we have the following:

Lemma 1.

For any $\eta >0$ and for any $({\phi }^{\left(n\right)}$, ${\psi }^{\left(n\right)})$ satisfying $(1/n)log\left|\right|{\phi }^{\left(n\right)}\left|\right|\le R,$ we have

$$\begin{array}{c}\hfill {\displaystyle {\mathrm{P}}_{\mathrm{c}}^{\left(n\right)}({\phi }^{\left(n\right)},{\psi }^{\left(n\right)};\Delta )\le p_{S_n{X}^n{Y}^n{Z}^n}\{}\end{array}$$

$$\begin{array}{ccc}\hfill \eta & \ge & \frac{1}{n}log\frac{Q_{X^n}^{\left(\mathrm{i}\right)}\left(X^n\right)}{p_{X^n}\left(X^n\right)},\hfill \end{array}$$

(14)

$$\begin{array}{ccc}\hfill \eta & \ge & \frac{1}{n}log\frac{Q_{Y^n|S_n{X}^n}^{\left(\mathrm{ii}\right)}\left(Y^n\right|S_n,X^n)}{p_{Y^n|X^n}\left(Y^n\right|X^n)},\hfill \end{array}$$

(15)

$$\begin{array}{ccc}\hfill \eta & \ge & \frac{1}{n}log\frac{Q_{X^n|S_n{Y}^n{Z}^n}^{\left(\mathrm{iii}\right)}\left(X^n\right|S_n,Y^n,Z^n)}{p_{X^n|S_n{Y}^n}\left(X^n\right|S_n,Y^n)},\hfill \end{array}$$

(16)

$$\begin{array}{ccc}\hfill R+\eta & \ge & \frac{1}{n}log\frac{Q_{X^n|S_n{Y}^n}^{\left(\mathrm{iv}\right)}\left(X^n\right|S_n,Y^n)}{p_{X^n|Y^n}\left(X^n\right|Y^n)},\hfill \end{array}$$

(17)

$$\begin{array}{ccc}\hfill \Delta & \ge & \left.\frac{1}{n}logexp\left\{d(X^n,Z^n)\right\}\right\}+4{\mathrm{e}}^{-n\eta }.\hfill \end{array}$$

(18)

The probability distribution and stochastic matrices appearing in the right members of Equation (18) have a property that we can select them arbitrary. In Equation (14), we can choose any probability distribution $Q_{X^n}^{\left(\mathrm{i}\right)}$ on ${\mathcal{X}}^n$. In Equation (15), we can choose any stochastic matrix $Q_{Y^n|S_n{X}^n}^{\left(\mathrm{ii}\right)}:$${\mathcal{M}}_n\times$ ${\mathcal{X}}^n$$\to {\mathcal{Y}}^n$. In Equation (16), we can choose any stochastic matrix $Q_{X^n|S_n{Y}^n{Z}^n}^{\left(\mathrm{iii}\right)}:$ ${\mathcal{M}}_n\times$${\mathcal{Y}}^n$×${\mathcal{Z}}^n$$\to {\mathcal{X}}^n$. In Equation (17), we can choose any stochastic matrix $Q_{X^n|S_n{Y}^n}^{\left(\mathrm{iv}\right)}:$${\mathcal{M}}_n$×${\mathcal{Y}}^n$$\to {\mathcal{X}}^n$.

Proof of this lemma is given in Appendix G.

Lemma 2.

Suppose that, for each $t=1,2,\cdots ,n$, the joint distribution $p_{S_n{X}^t{Y}^n}$ of the random vector $S_n{X}^t{Y}^n$ is a marginal distribution of $p_{S_n{X}^n{Y}^n}$. Then, for $t=1,2,\cdots ,n$, we have the following Markov chain:

$$X_t\leftrightarrow S_n{X}^{t-1}{Y}_t^n\leftrightarrow Y^{t-1}$$

(19)

or equivalently that $I(X_t;Y^{t-1}|S_n{X}^{t-1}{Y}_t^n)=0$.

Proof of this lemma is given in Appendix H. For $t=1,2,\cdots ,n$, set $u_t:=(s,x^{t-1},y_{t+1}^n)$. Let $U_t:=$$(S_n,X^{t-1},$$Y_{t+1}^n)$ be a random vector taking values in ${\mathcal{M}}_n$ ×${\mathcal{X}}^{t-1}$ ×${\mathcal{Y}}_{t+1}^n$. From Lemmas 1 and 2, we have the following:

Lemma 3.

For any $\eta >0$ and for any $({\phi }^{\left(n\right)}$, ${\psi }^{\left(n\right)})$ satisfying $(1/n)log\left|\right|{\phi }^{\left(n\right)}\left|\right|\le R,$ we have the following:

$$\begin{array}{c}{\mathrm{P}}_{\mathrm{c}}^{\left(n\right)}({\phi }^{\left(n\right)},{\psi }^{\left(n\right)};\Delta )\le p_{S_n{X}^n{Y}^n{Z}^n}\{\\ \begin{array}{ccc}\hfill \eta & \ge & \frac{1}{n}{\displaystyle \sum _{t=1}^n}log\frac{Q_{X_t}^{\left(\mathrm{i}\right)}\left(X_t\right)}{p_{X_t}\left(X_t\right)},\hfill \\ \hfill \eta & \ge & \frac{1}{n}{\displaystyle \sum _{t=1}^n}log\frac{Q_{Y_t|U_t{X}_t}^{\left(\mathrm{ii}\right)}\left(Y_t\right|U_t,X_t)}{p_{Y_t|X_t}\left(Y_t\right|X_t)},\hfill \\ \hfill \eta & \ge & \frac{1}{n}{\displaystyle \sum _{t=1}^n}log\frac{Q_{X_t|U_t{Y}_t{Z}_t}^{\left(\mathrm{iii}\right)}\left(X_t\right|U_t,Y_t,Z_t)}{p_{X_t|U_t{Y}_t}\left(X_t\right|U_t,Y_t)},\hfill \\ \hfill R+\eta & \ge & \frac{1}{n}{\displaystyle \sum _{t=1}^n}log\frac{Q_{X_t|U_t{Y}_t}^{\left(\mathrm{iv}\right)}\left(X_t\right|U_t,Y_t)}{p_{X_t|Y_t}\left(X_t\right|Y_t)},\hfill \\ \hfill \Delta & \ge & \left.\frac{1}{n}{\displaystyle \sum _{t=1}^n}log{\mathrm{e}}^{d(X_t,Z_t)}\right\}+4{\mathrm{e}}^{-n\eta },\hfill \end{array}\end{array}$$

(20)

where for each $t=1,2,\cdots ,n$, the following probability distribution and stochastic matrices:

$$Q_{X_t}^{\left(\mathrm{i}\right)},Q_{Y_t|U_t{X}_t}^{\left(\mathrm{ii}\right)},Q_{X_t|U_t{Y}_t{Z}_t}^{\left(\mathrm{iii}\right)}, and Q_{X_t|U_t{Y}_t}^{\left(\mathrm{iv}\right)}$$

appearing in the first term in the right members of Equation (21) have a property that we can choose their values arbitrary.

Proof. 

On the probability distributions appearing in the right members of Equation (18), we take the following choices. In Equation (14), we choose $Q_{X^n}^{\left(\mathrm{i}\right)}$ so that

$$Q_{X^n}^{\left(\mathrm{i}\right)}\left(X^n\right)=\prod _{t=1}^n{Q}_{X_t}^{\left(\mathrm{i}\right)}\left(X_t\right).$$

(21)

In Equation (15), we choose $Q_{Y^n|S_n{X}^n}^{\left(\mathrm{ii}\right)}$ so that

$$\begin{array}{ccc}\hfill Q_{Y^n|S_n{X}^n}^{\left(\mathrm{ii}\right)}\left(Y^n\right|S_n,X^n)& =& \prod _{t=1}^n{Q}_{Y_t|S_n{X}^t{Y}_{t+1}^n}^{\left(\mathrm{ii}\right)}\left(Y_t\right|S_n,X^t,Y_{t+1}^n)=\prod _{t=1}^n{Q}_{Y_t|X_t{U}_t}^{\left(\mathrm{ii}\right)}\left(Y_t\right|U_t{X}_t).\hfill \end{array}$$

(22)

In Equation (16), we choose $Q_{X^n|S_n{Y}^n{Z}^n}^{\left(\mathrm{iii}\right)}$ so that

$$\begin{array}{ccc}\hfill Q_{X^n|S_n{Y}^n{Z}^n}^{\left(\mathrm{iii}\right)}\left(X^n\right|S_n,Y^n,Z^n)& =& \prod _{t=1}^n{Q}_{X_t|S_n{X}^{t-1}{Y}_t^n{Z}_t}^{\left(\mathrm{iii}\right)}\left(X_t\right|S_n{X}^{t-1},Y_t^n,Z_t)\hfill \\ & =& \prod _{t=1}^n{Q}_{X_t|U_t{Y}_t{Z}_t}^{\left(\mathrm{iii}\right)}\left(X_t\right|U_t{Y}_t{Z}_t).\hfill \end{array}$$

(23)

In Equation (16), we note that

$$\begin{array}{ccc}\hfill p_{X^n|S_n{Y}^n}\left(X^n\right|S_n,Y^n)& =& \prod _{t=1}^n{p}_{X_t|S_n{X}^{t-1}{Y}^n}\left(X_t\right|S_n,X^{t-1},Y^n)\stackrel{\left(\mathrm{a}\right)}{=}\prod _{t=1}^n{p}_{X_t|S_n{X}^{t-1}{Y}_t^n}\left(X_t\right|S_n,X^{t-1},Y_t^n)\hfill \\ & =& \prod _{t=1}^n{p}_{X_t|U_t{Y}_t}\left(X_t\right|U_t,Y_t).\hfill \end{array}$$

(24)

Step (a) follows from Lemma 2. In Equation (17), we choose $Q_{X^n|S_n{Y}^n}^{\left(\mathrm{iv}\right)}$ so that

$$\begin{array}{c}{Q}_{X^n|S_n{Y}^n}^{\left(\mathrm{iv}\right)}\left(X^n\right|S_n,Y^n)=\prod _{t=1}^n{Q}_{X_t|S_n{X}^{t-1}{Y}_t^n}^{\left(\mathrm{iv}\right)}\left(X_t\right|S_n,X^{t-1},Y_t^n)=\prod _{t=1}^n{Q}_{X_t|U_t{Y}_t}^{\left(\mathrm{iv}\right)}\left(X_t\right|U_t,Y_t).\hfill \end{array}$$

(25)

From Lemma 1 and Equations (21)–(25), we have the bound of Equation (21) in Lemma 3. ☐

To evaluate an upper btound of Equation (21) in Lemma 3, we use the following lemma, which is well known as the Cramér’s bound in the large deviation principle.

Lemma 4.

For any real valued random variable A and any $\theta \ge 0$, we have

$$Pr\{A\le a\}\le exp\left[\left(\theta a+log\mathrm{E}[exp(-\theta A\left)\right]\right)\right].$$

Here, we define a quantity which serves as an exponential upper bound of ${\mathrm{P}}_{\mathrm{c}}^{\left(n\right)}({\phi }^{\left(n\right)},$ ${\psi }^{\left(n\right)})$. For each $t=1,2,\cdots ,n$, let ${\underline{\mathcal{Q}}}_t$ be a set of all

$$\begin{array}{ccc}\hfill {\underline{Q}}_t& =& (Q_{X_t}^{\left(\mathrm{i}\right)},Q_{Y_t|U_t{X}_t}^{\left(\mathrm{ii}\right)},Q_{X_t|U_t{Y}_t{Z}_t}^{\left(\mathrm{iii}\right)},Q_{X_t|U_t{Y}_t}^{\left(\mathrm{iv}\right)}).\hfill \end{array}$$

Set

$$\begin{array}{ccc}\hfill {\underline{\mathcal{Q}}}^n& :=& \prod _{t=1}^n{\underline{\mathcal{Q}}}_t,{\underline{Q}}^n:={\left\{{\underline{Q}}_t\right\}}_{t=1}^n\in {\underline{\mathcal{Q}}}^n.\hfill \end{array}$$

Let ${\mathcal{P}}^{\left(n\right)}\left(p_{XY}\right)$ be a set of all probability distributions $p_{S_n{X}^n{Y}^n{Z}^n}$ on ${\mathcal{M}}_n\times {\mathcal{X}}^n\times {\mathcal{Y}}^n\times {\mathcal{Z}}^n$ having the form:

$$\begin{array}{ccc}\hfill p_{S_n{X}^n{Y}^n{Z}^n}(s,x^n,y^n,z^n)& =& p_{S_n|X^n}\left(s\right|x^n)\left\{\prod _{t=1}^n{p}_{X_t{Y}_t}(x_t,y_t)\right\}{p}_{Z^n|Y^n{S}_n}\left(z^n\right|y^n,s).\hfill \end{array}$$

For simplicity of notation, we use the notation $p^{\left(n\right)}$ for $p_{S_n{X}^n{Y}^n{Z}^n}$$\in {\mathcal{P}}^{\left(n\right)}$ $\left(p_{XY}\right)$. We assume that $p_{U_t{X}_t{Y}_t{Z}_t}=p_{S_n{X}^t{Y}_t^n{Z}_t}$ is a marginal distribution of $p^{\left(n\right)}$. For $t=1,2,\cdots ,n$, we simply write $p_t=$$p_{U_t{X}_t{Y}_t{Z}_t}$. For $p^{\left(n\right)}$ $\in {\mathcal{P}}^{\left(n\right)}\left(p_{XY}\right)$ and ${\underline{Q}}^n$$\in {\underline{\mathcal{Q}}}^n$, we define

$$\begin{array}{c}{\mathsf{\Omega }}_{}^{(\mu ,\lambda ,\theta )}(p^{\left(n\right)},{\underline{Q}}^n|p_{XY}):=-log{\mathrm{E}}_{p^{\left(n\right)}}\left[\left(\prod _{t=1}^n\left\{\frac{p_{X_t}\left(X_t\right)}{Q_{X_t}^{\left(\mathrm{i}\right)}\left(X_t\right)}\right.\right.\right.\left.{\left.\frac{p_{Y_t|X_t}\left(Y_t\right|X_t)}{Q_{Y_t|X_t{U}_t}^{\left(\mathrm{ii}\right)}\left(Y_t\right|X_t,U_t)}\right\}}^{\theta }\right)\hfill \\ \times \left(\prod _{t=1}^n{\left\{\frac{p_{X_t|U_t{Y}_t}\left(X_t\right|U_t,Y_t)}{Q_{X_t|U_t{Y}_t{Z}_t}^{\left(\mathrm{iii}\right)}\left(X_t\right|U_t,Y_t,Z_t)}\right\}}^{\theta }\right)\left.\left(\prod _{t=1}^n{\left\{{\left(\frac{p_{X_t|Y_t}\left(X_t\right|Y_t)}{Q_{X_t|Y_t{U}_t}^{\left(\mathrm{iv}\right)}\left(X_t\right|U_t,Y_t)}\right)}^{\overline{\mu }}{\mathrm{e}}^{-\mu d(X_t,Z_t)}\right\}}^{\lambda \theta }\right)\right],\hfill \end{array}$$

where, for each $t=1,2,\cdots ,n$, the following probability distribution and stochastic matrices:

$$Q_{X_t}^{\left(\mathrm{i}\right)},Q_{X_t|U_t{Y}_t}^{\left(\mathrm{ii}\right)},Q_{X_t|U_t{Y}_t{Z}_t}^{\left(\mathrm{iii}\right)},Q_{Y_t|X_t{U}_t}^{\left(\mathrm{iv}\right)}$$

appearing in the definition of ${\mathsf{\Omega }}_{}^{(\mu ,\lambda ,\theta )}(p^{\left(n\right)},{\underline{Q}}^n|p_{XY})$ are chosen so that they are induced by the joint distribution $q_t=q_{U_t{X}_t{Y}_t{Z}_t}\in {\mathcal{Q}}_t$.

By Lemmas 3 and 4, we have the following proposition:

Proposition 1.

For any $\mu \in [0,1],$ $\lambda ,$$\theta \ge 0$, any $q^n\in {\mathcal{Q}}^n$, and any $({\phi }^{\left(n\right)},$${\psi }^{\left(n\right)})$ satisfying $(1/n)log\left|\right|{\phi }^{\left(n\right)}\left|\right|\le R,$ we have

$$\begin{array}{ccc}\hfill {\mathrm{P}}_{\mathrm{c}}^{\left(n\right)}({\phi }^{\left(n\right)},{\psi }^{\left(n\right)};\Delta )& \le & 5exp\{-n{\left[1+(3+\lambda \overline{\mu })\theta \right]}^{-1}\left.\left[\frac{1}{n}{\mathsf{\Omega }}_{}^{(\mu ,\lambda ,\theta )}(p^{\left(n\right)},{\underline{Q}}^n|p_{XY})-\lambda \theta (\overline{\mu }R+\mu \Delta )\right]\right\}.\hfill \end{array}$$

Proof. 

When ${\mathsf{\Omega }}_{}^{(\mu ,\lambda ,\theta )}(p^{\left(n\right)},{\underline{Q}}^n|p_{XY})\le n\lambda \theta (\overline{\mu }R+\mu \Delta )$, the bound we wish to prove is obvious. In the following argument, we assume that ${\mathsf{\Omega }}_{}^{(\mu ,\lambda ,\theta )}(p^{\left(n\right)},{\underline{Q}}^n|p_{XY})>$ $n\lambda \theta (\overline{\mu }R+\mu \Delta ).$ We define five random variables $A_i,$$i=1,2,\cdots ,5$ by

$$\begin{array}{c}{A}_1=\frac{1}{n}\sum _{t=1}^{n}log\frac{Q_{X_t}^{\left(\mathrm{i}\right)}\left(X_t\right)}{p_{X_t}\left(X_t\right)},A_2=\frac{1}{n}\sum _{t=1}^{n}log\frac{Q_{Y_t|X_t{U}_t}^{\left(\mathrm{ii}\right)}\left(Y_t\right|X_t,U_t)}{p_{Y_t|X_t}\left(Y_t\right|X_t)},\hfill \\ A_3=\frac{1}{n}\sum _{t=1}^{n}log\frac{Q_{X_t|U_t{Y}_t{Z}_t}^{\left(\mathrm{iii}\right)}\left(X_t\right|U_t,Y_t,Z_t)}{p_{X_t|U_t{Y}_t}\left(X_t\right|U_t,Y_t)},\hfill \\ A_4=\frac{1}{n}\sum _{t=1}^{n}log\frac{Q_{X_t|U_t{Y}_t}^{\left(\mathrm{iv}\right)}\left(X_t\right|U_t,Y_t)}{p_{X_t|Y_t}\left(X_t\right|Y_t)},A_5=\frac{1}{n}\sum _{t=1}^{n}log{\mathrm{e}}^{d(X_t,Z_t)}.\hfill \end{array}$$

By Lemma 3, for any $({\phi }^{\left(n\right)},{\psi }^{\left(n\right)})$ satisfying $(1/n)log\left|\right|{\phi }^{\left(n\right)}\left|\right|$$\le R,$ we have

$$\begin{array}{c}{\mathrm{P}}_{\mathrm{c}}^{\left(n\right)}({\phi }^{\left(n\right)},{\psi }^{\left(n\right)};\Delta )\le p_{S_n{X}^n{Y}^n{Z}^n}\{A_i\le \eta fori=1,2,3,A_4\le R+\eta ,A_5\le \Delta \}+4{\mathrm{e}}^{-n\eta }\hfill \\ \le p_{S_n{X}^n{Y}^n{Z}^n}\{A_1+A_2+A_3+\lambda (\overline{\mu }{A}_4+\mu A_5)\le \lambda (\overline{\mu }R+\mu \Delta )+(3+\lambda \overline{\mu })\eta \}+4{\mathrm{e}}^{-n\eta }\hfill \\ =p_{S_n{X}^n{Y}^n{Z}^n}\{A\le a\}+4{\mathrm{e}}^{-n\eta },\hfill \end{array}$$

(26)

where we set

$$\begin{array}{ccc}\hfill A& :=& A_1+A_2+A_3+\lambda (\overline{\mu }{A}_4+\mu A_5),\hfill \\ \hfill a& :=& \lambda (\overline{\mu }R+\mu \Delta )+(3+\lambda \overline{\mu })\eta .\hfill \end{array}$$

Applying Lemma 4 to the first term in the right member of Equation (26), we have

$$\begin{array}{cc}& {\mathrm{P}}_{\mathrm{c}}^{\left(n\right)}({\phi }^{\left(n\right)},{\psi }^{\left(n\right)};\Delta )\le exp\left[\left(\theta a+log{\mathrm{E}}_{p^{\left(n\right)}}[exp(-\theta A)]\right)\right]+4{\mathrm{e}}^{-n\eta }\hfill \\ =& exp\left[n\right\{\lambda \theta (\overline{\mu }R+\mu \Delta )+(3+\lambda \overline{\mu })\theta \eta \left.\left.-\frac{1}{n}{\mathsf{\Omega }}_{}^{(\mu ,\lambda ,\theta )}(p^{\left(n\right)},{\underline{Q}}^n|p_{XY})\right\}\right]+4{\mathrm{e}}^{-n\eta }.\hfill \end{array}$$

(27)

We choose $\eta$ so that

$$\begin{array}{ccc}\hfill -\eta & =& \lambda \theta (\overline{\mu }R+\mu \Delta )+\theta (3+\lambda \overline{\mu })\eta -\frac{1}{n}{\mathsf{\Omega }}_{}^{(\mu ,\lambda ,\theta )}(p^{\left(n\right)},{\underline{Q}}^n|p_{XY}).\hfill \end{array}$$

(28)

Solving Equation (28) with respect to $\eta$, we have

$$\begin{array}{c}\hfill \eta =\frac{\frac{1}{n}{\mathsf{\Omega }}_{}^{(\mu ,\lambda ,\theta )}(p^{\left(n\right)},{\underline{Q}}^n|p_{XY})-\lambda \theta (\overline{\mu }R+\mu \Delta )}{1+(3+\lambda \overline{\mu })\theta }.\end{array}$$

For this choice of $\eta$ and Equation (27), we have

$$\begin{array}{cc}& {\mathrm{P}}_{\mathrm{c}}^{\left(n\right)}({\phi }^{\left(n\right)},{\psi }^{\left(n\right)};\Delta )\le 5{\mathrm{e}}^{-n\eta }\hfill \\ =& 5exp\{-n{\left[1+(3+\lambda \overline{\mu })\theta \right]}^{-1}\left.\left[\frac{1}{n}{\mathsf{\Omega }}_{}^{(\mu ,\lambda ,\theta )}(p^{\left(n\right)},{\underline{Q}}^n|p_{XY})-\lambda \theta (\overline{\mu }R+\mu \Delta )\right]\right\},\hfill \end{array}$$

completing the proof. ☐

Set

$$\begin{array}{ccc}\hfill {\underline{\mathsf{\Omega }}}^{(\mu ,\lambda ,\theta )}\left(p_{XY}\right)& :=& \underset{n\ge 1}{inf}\underset{p^{\left(n\right)}\in {\mathcal{P}}^{\left(n\right)}\left(p_{XY}\right)}{min}\underset{{\underline{Q}}^n\in {\underline{\mathcal{Q}}}^n}{max}\frac{1}{n}{\mathsf{\Omega }}_{}^{(\mu ,\lambda ,\theta )}(p^{\left(n\right)},{\underline{Q}}^n|p_{XY}).\hfill \end{array}$$

By Proposition 1, we have the following corollary.

Corollary 3.

For any $\mu \in [0,1],\lambda \ge 0$, for any $\theta \ge 0$, and for any $({\phi }^{\left(n\right)},$${\psi }^{\left(n\right)})$ satisfying $(1/n)log\left|\right|{\phi }^{\left(n\right)}\left|\right|\le R,$ we have

$$\begin{array}{ccc}\hfill {\mathrm{P}}_{\mathrm{c}}^{\left(n\right)}({\phi }^{\left(n\right)},{\psi }^{\left(n\right)};\Delta )& \le & 5exp\left\{-n\left[\frac{{\underline{\mathsf{\Omega }}}^{(\mu ,\lambda ,\theta )}\left(p_{XY}\right)-\lambda \theta (\overline{\mu }R+\mu \Delta )}{1+(3+\lambda \overline{\mu })\theta }\right]\right\}.\hfill \end{array}$$

We shall call ${\underline{\mathsf{\Omega }}}^{(\mu ,\lambda ,\theta )}\left(p_{XY}\right)$ the communication potential. The above corollary implies that the analysis of ${\underline{\mathsf{\Omega }}}^{(\mu ,\lambda ,\theta )}\left(p_{XY}\right)$ leads to an establishment of a strong converse theorem for Wyner–Ziv source coding problem. In the following argument, we drive an explicit lower bound of ${\underline{\mathsf{\Omega }}}^{(\mu ,\lambda ,\theta )}\left(p_{XY}\right)$. We use a new technique we call the recursive method. The recursive method is a powerful tool to drive a single letterized exponent function for rates below the rate distortion function. This method is also applicable to prove the exponential strong converse theorems for other network information theory problems [5,6,7]. Set

$${\mathcal{F}}_t:=(p_{X_t|U_t{Y}_t},{\underline{Q}}_t),{\mathcal{F}}^t:={\left\{{\mathcal{F}}_i\right\}}_{i=1}^t.$$

For each $t=1,2,\cdots ,n$, define a function of $(u_t,x_t,y_t,z_t)$ $\in {\mathcal{U}}_t$ $\times \mathcal{X}$ $\times \mathcal{Y}$ $\times \mathcal{Z}$ by

$$\begin{array}{c}{f}_{{\mathcal{F}}_t}^{(\mu ,\lambda ,\theta )}(x_t,y_t,z_t|u_t)\hfill \\ :=\left\{\frac{p_{X_t}\left(x_t\right)}{Q_{X_t}^{\left(\mathrm{i}\right)}\left(x_t\right)}\frac{p_{Y_t|X_t}\left(y_t\right|x_t)}{Q_{Y_t|X_t{U}_t}^{\left(\mathrm{ii}\right)}\left(y_t\right|x_t,u_t)}\right.{\left.\frac{p_{X_t|U_t{Y}_t}\left(x_t\right|u_t,y_t)}{Q_{X_t|U_t{Y}_t{Z}_t}^{\left(\mathrm{iii}\right)}\left(x_t\right|u_t,y_t,z_t)}{\left(\frac{p_{X_t|Y_t}\left(x_t\right|y_t)}{Q_{X_t|Y_t{U}_t}^{\left(\mathrm{iv}\right)}\left(x_t\right|u_t,y_t)}\right)}^{\lambda \overline{\mu }}{\mathrm{e}}^{-\lambda \mu d(x_t,z_t)}\right\}}^{\theta }.\hfill \end{array}$$

By definition, we have

$$\begin{array}{cc}& exp\left\{-{\mathsf{\Omega }}_{}^{(\mu ,\lambda ,\theta )}(p^{\left(n\right)},{\underline{Q}}^n|p_{XY})\right\}\hfill \\ =& \sum _{s,y^n}{p}_{S_n{Y}^n}(s,y^n)\sum _{x^n,z^n}{p}_{X^n{Z}^n|S_n{Y}^n}(x^n,z^n|s,y^n)\prod _{t=1}^n{f}_{{\mathcal{F}}_t}^{(\mu ,\lambda ,\theta )}(x_t,y_t,z_t|u_t).\hfill \end{array}$$

(29)

For each $t=1,2,\cdots ,n$, we define the conditional probability distribution

$$\begin{array}{ccc}\hfill p_{X^t{Z}^t|S_n{Y}^n;{\mathcal{F}}^t}^{(\mu ,\lambda ,\theta )}& :=& {\left\{p_{X^t{Z}^t|S_n{Y}^n;{\mathcal{F}}^t}^{(\mu ,\lambda ,\theta )}(x^t,z^t|s,y^n)\right\}}_{(x^t,z^t,s,y^n)\in {\mathcal{X}}^t\times {\mathcal{Z}}^t\times {\mathcal{M}}_n\times {\mathcal{Y}}^n}\hfill \end{array}$$

by

$$\begin{array}{ccc}\hfill p_{X^t{Z}^t|S_n{Y}^n;{\mathcal{F}}^t}^{(\mu ,\lambda ,\theta )}(x^t,z^t|s,y^n)& :=& C_t^{-1}(s,y^n)p_{X^t{Z}^t|S_n{Y}^n}(x^t,z^t|s,y^n)\prod _{i=1}^t{f}_{{\mathcal{F}}_i}^{(\mu ,\lambda ,\theta )}(x_i,y_i,z_i|u_i)\hfill \end{array}$$

where

$$\begin{array}{ccc}\hfill C_t(s,y^n)& :=& \sum _{x^t,z^t}{p}_{X^t{Z}^t|S_n{Y}^n}(x^t,z^t|s,y^n)\prod _{i=1}^t{f}_{{\mathcal{F}}_i}^{(\mu ,\lambda ,\theta )}(x_i,y_i,z_i|u_i)\hfill \end{array}$$

(31)

are constants for normalization. For $t=1,2,\cdots ,n$, define

$${\mathsf{\Phi }}_{t,{\mathcal{F}}^t}^{(\mu ,\lambda ,\theta )}(s,y^n):=C_t(s,y^n)C_{t-1}^{-1}(s,y^n),$$

(31)

where we define $C_0(s,y^n)=1$ for $(s,y^n)\in {\mathcal{M}}_n$$\times {\mathcal{Y}}^n.$ Then, we have the following lemma:

Lemma 5.

For each $t=1,2,\cdots ,n$, and for any $(s,$$y^n$$x^t,z^t)\in {\mathcal{M}}_n$ $\times {\mathcal{Y}}^n$ $\times {\mathcal{X}}^t$ $\times {\mathcal{Z}}^t$, we have

$$\begin{array}{c}{p}_{X^t{Z}^t|S_n{Y}^n;{\mathcal{F}}^t}^{(\mu ,\lambda ,\theta )}(x^t,z^t|s,y^n)={\left({\mathsf{\Phi }}_{t,{\mathcal{F}}^t}^{(\mu ,\lambda ,\theta )}(s,y^n)\right)}^{-1}{p}_{X^{t-1}{Y}^{t-1}|S_n{Y}^n;{\mathcal{F}}^{t-1}}^{(\mu ,\lambda ,\theta )}(x^{t-1},z^{t-1}|s,y^n)\hfill \end{array}$$

$$\begin{array}{c}\times p_{X_t{Z}_t|S_n{X}^{t-1}{Y}^n}(x_t,z_t|s,x^{t-1},z^{t-1},y^n)f_{{\mathcal{F}}_t}^{(\mu ,\lambda ,\theta )}(x_t,y_t,z_t|u_t).\hfill \\ {\mathsf{\Phi }}_{t,{\mathcal{F}}^t}^{(\mu ,\lambda ,\theta )}(s,y^n)=\sum _{x^t,z^t}{p}_{X^{t-1}{Z}^{t-1}|S_n{Y}^n;{\mathcal{F}}^{t-1}}^{(\mu ,\lambda ,\theta )}(x^{t-1},z^{t-1}|s,y^n)\hfill \end{array}$$

(32)

$$\begin{array}{c}\times p_{X_t{Z}_t|S_n{X}^{t-1}{Y}^n}(x_t,z_t|s,x^{t-1},z^{t-1},y^n)f_{{\mathcal{F}}_t}^{(\mu ,\lambda ,\theta )}(x_t,y_t,z_t|u_t).\hfill \end{array}$$

(33)

Furthermore, we have

$$\begin{array}{ccc}\hfill exp\left\{-{\mathsf{\Omega }}_{}^{(\mu ,\lambda ,\theta )}(p^{\left(n\right)},{\underline{Q}}^n|p_{XY})\right\}& =& \sum _{s,y^n}{p}_{S_n{Y}^n}(s,y^n)\prod _{t=1}^n{\mathsf{\Phi }}_{t,{\mathcal{F}}^t}^{(\mu ,\lambda ,\theta )}(s,y^n).\hfill \end{array}$$

(34)

The equality in Equation (34) in Lemma 5 is obvious from Equations (29)–(31). Proofs of Equations (32) and (33) in this lemma are given in Appendix I. Next, we define a probability distribution of the random pair $(S_n,Y^n)$ taking values in ${\mathcal{M}}_n$$\times {\mathcal{Y}}^n$ by

$$\begin{array}{c}{p}_{S_n{Y}^n;{\mathcal{F}}^t}^{(\mu ,\lambda ,\theta )}(s,y^n)={\tilde{C}}_t^{-1}{p}_{S_n{Y}^n}(s,y^n)\prod _{i=1}^t{\mathsf{\Phi }}_{i,{\mathcal{F}}^i}^{(\mu ,\lambda ,\theta )}(s,y^n),\hfill \end{array}$$

(35)

where ${\tilde{C}}_t$ is a constant for normalization given by

$${\tilde{C}}_t=\sum _{s,y^n}{p}_{S_n{Y}^n}(s,y^n)\prod _{i=1}^t{\mathsf{\Phi }}_{i,{\mathcal{F}}^i}^{(\mu ,\lambda ,\theta )}(s,y^n).$$

For $t=1,2,\cdots ,n$, define

$${\mathsf{\Lambda }}_{t,{\mathcal{F}}^t}^{(\mu ,\lambda ,\theta )}:=\widetilde{C_t}{\tilde{C}}_{t-1}^{-1},$$

(36)

where we define ${\tilde{C}}_0=1$. Set

$$\begin{array}{c}{p}_{S_n{X}^t{Y}_t^n{Z}_t;{\mathcal{F}}^{t-1}}^{(\mu ,\lambda ,\theta )}(s,x^t,y_t^n,z_t)=p_{U_t{X}_t{Y}_t{Z}_t;{\mathcal{F}}^{t-1}}^{(\mu ,\lambda ,\theta )}(u_t,x_t,y_t,z_t)\hfill \\ :=\sum _{y^{t-1},z^{t-1}}{p}_{S_n{Y}^n;{\mathcal{F}}^{t-1}}^{(\mu ,\lambda ,\theta )}(s,y^n)p_{X^{t-1}{Z}^{t-1}|S_n{Y}^n;{\mathcal{F}}^{t-1}}^{(\mu ,\lambda ,\theta )}(x^{t-1},z^{t-1}|s,y^n)\hfill \\ \times p_{X_t{Z}_t|X^{t-1}{Z}^{t-1}{S}_n{Y}^n}(x_t,z_t|x^{t-1},z^{t-1},s,y^n).\hfill \end{array}$$

(37)

Then, we have the following:

Lemma 6.

$$\begin{array}{c}exp\left\{-{\mathsf{\Omega }}_{}^{(\mu ,\lambda ,\theta )}(p^{\left(n\right)},{\underline{Q}}^n|p_{XY})\right\}=\prod _{t=1}^n{\mathsf{\Lambda }}_{t,{\mathcal{F}}^t}^{(\mu ,\lambda ,\theta )},\hfill \end{array}$$

(38)

$$\begin{array}{c}{\mathsf{\Lambda }}_{t,{\mathcal{F}}^t}^{(\mu ,\lambda ,\theta )}=\sum _{u_t,x_t,y_t,z_t}{p}_{U_t{X}_t{Y}_t{Z}_t;{\mathcal{F}}^{t-1}}^{(\mu ,\lambda ,\theta )}(u_t,x_t,y_t,z_t)f_{{\mathcal{F}}_t}^{(\mu ,\lambda ,\theta )}(x_t,y_t,z_t|u_t).\hfill \end{array}$$

(39)

Proof. 

By the equality Equation (34) in Lemma 5, we have

$$\begin{array}{c}exp\left\{-{\mathsf{\Omega }}_{}^{(\mu ,\lambda ,\theta )}(p^{\left(n\right)},{\underline{Q}}^n|p_{XY})\right\}={\tilde{C}}_n=\prod _{t=1}^n{\tilde{C}}_t{\tilde{C}}_{t-1}^{-1}\stackrel{\left(\mathrm{a}\right)}{=}\prod _{t=1}^n{\mathsf{\Lambda }}_{t,{\mathcal{F}}^t}^{(\mu ,\lambda ,\theta )}.\hfill \end{array}$$

(40)

Step (a) follows from the definition in Equation (36) of ${\mathsf{\Lambda }}_{t,{\mathcal{F}}^t}^{(\mu ,\lambda ,\nu ,\theta )}.$ We next prove Equation (39) in Lemma 6. Multiplying ${\mathsf{\Lambda }}_{t,{\mathcal{F}}^t}^{(\mu ,\lambda ,\theta )}=$${\tilde{C}}_t/{\tilde{C}}_{t-1}$ to both sides of Equation (35), we have

$$\begin{array}{cc}& {\mathsf{\Lambda }}_{t,{\mathcal{F}}^t}^{(\mu ,\lambda ,\theta )}{p}_{S_n{Y}^n;{\mathcal{F}}^t}^{(\mu ,\lambda ,\theta )}(s,y^n)\hfill \\ =& {\tilde{C}}_{t-1}^{-1}{p}_{S_n{Y}^n}(s,y^n)\prod _{i=1}^t{\mathsf{\Phi }}_{i,{\mathcal{F}}^i}^{(\mu ,\lambda ,\theta )}(s,y^n)\hfill \end{array}$$

(41)

$$\begin{array}{cc}=& p_{S_n{Y}^n;{\mathcal{F}}^{t-1}}^{(\mu ,\lambda ,\theta )}(s,y^n){\mathsf{\Phi }}_{t,{\mathcal{F}}^t}^{(\mu ,\lambda ,\theta )}(s,y^n).\hfill \end{array}$$

(42)

Taking summations of Equations (41) and (42) with respect to $(s,y^n)$, we have

$$\begin{array}{c}{\mathsf{\Lambda }}_{t,{\mathcal{F}}^t}^{(\mu ,\lambda ,\theta )}=\sum _{s,y^n}{p}_{S_n{Y}^n;{\mathcal{F}}^{t-1}}^{(\mu ,\lambda ,\theta )}(s,y^n){\mathsf{\Phi }}_{t,{\mathcal{F}}^t}^{(\mu ,\lambda ,\theta )}(s,y^n)\hfill \\ \stackrel{\left(\mathrm{a}\right)}{=}\sum _{s,y^n}{p}_{S_n{Y}^n;{\mathcal{F}}^{t-1}}^{(\mu ,\lambda ,\theta )}(s,y^n)\sum _{x^t,z^t}{p}_{X^{t-1}{Z}^{t-1}|S_n{Y}^n;{\mathcal{F}}^{t-1}}^{(\mu ,\lambda ,\theta )}(x^{t-1},z^{t-1}|s,y^n)\hfill \\ \times p_{X_t{Z}_t|X^{t-1}{Z}^{t-1}{S}_n{Y}^n}(x_t,z_t|x^{t-1},z^{t-1},s,y^n)f_{{\mathcal{F}}_t}^{(\mu ,\lambda ,\theta )}(x_t,y_t,z_t|u_t)\hfill \\ =\sum _{s,x^t,y_t^n,z_t}\sum _{y^{t-1},z^{t-1}}{p}_{S_n{Y}^n;{\mathcal{F}}^{t-1}}^{(\mu ,\lambda ,\theta )}(s,y^n)p_{X^{t-1}{Z}^{t-1}|S_n{Y}^n;{\mathcal{F}}^{t-1}}^{(\mu ,\lambda ,\theta )}(x^{t-1},z^{t-1}|s,y^n)\hfill \\ \times p_{X_t{Z}_t|X^{t-1}{Z}^{t-1}{S}_n{Y}^n}(x_t,z_t|x^{t-1},z^{t-1},s,y^n)f_{{\mathcal{F}}_t}^{(\mu ,\lambda ,\theta )}(x_t,y_t,z_t|u_t).\hfill \\ \stackrel{\left(\mathrm{b}\right)}{=}\sum _{u_t,x_t,y_t,z_t}{p}_{U_t{X}_t{Y}_t{Z}_t;{\mathcal{F}}^{t-1}}^{(\mu ,\lambda ,\theta )}(u_t,x_t,y_t,z_t)f_{{\mathcal{F}}_t}^{(\mu ,\lambda ,\theta )}(x_t,y_t,z_t|u_t).\hfill \end{array}$$

Step (a) follows from Equation (33) in Lemma 5. Step (b) follows from the definition in Equation (37) of $p_{U_t{X}_t{Y}_t{Z}_t;{\mathcal{F}}^{t-1}}^{(\mu ,\lambda ,\theta )}$. ☐

The following proposition is a mathematical core to prove our main result.

Proposition 2.

For $\theta \in [0,1)$, we choose the parameter α such that

$$\alpha =\frac{\theta }{1-\theta }\iff \theta =\frac{\alpha }{1+\alpha }.$$

(43)

Then, for any $\lambda \ge 0,\mu \in [0,1]$ and for any $\theta \in [0,1)$, we have

$${\underline{\mathsf{\Omega }}}^{(\mu ,\lambda ,\theta )}\left(p_{XY}\right)\ge \frac{{\mathsf{\Omega }}^{(\mu ,\lambda ,\alpha )}\left(p_{XY}\right)}{1+\alpha }.$$

(44)

Proof. 

Set

$$\begin{array}{c}{\widehat{\mathcal{Q}}}_n:=\{q=q_{UXYZ}:\left|\mathcal{U}\right|\le |{\mathcal{M}}_n\left|\right|{\mathcal{X}}^{n-1}\left|\right|{\mathcal{Y}}^{n-1}\left|\right\},\hfill \\ {\widehat{\mathsf{\Omega }}}_n^{(\mu ,\lambda ,\alpha )}\left(p_{XY}\right):=\underset{\genfrac{}{}{0pt}{}{}{q\in {\widehat{\mathcal{Q}}}_n}}{min}{\mathsf{\Omega }}^{(\mu ,\lambda ,\alpha )}\left(q\right|p_{XY}).\hfill \end{array}$$

Then, by Lemma 6, we have

$$\begin{array}{ccc}\hfill {\mathsf{\Lambda }}_{t,{\mathcal{F}}^t}^{(\mu ,\lambda ,\theta )}& =& \sum _{u_t,x_t,y_t,z_t}{p}_{U_t{X}_t{Y}_t{Z}_t;{\mathcal{F}}^{t-1}}^{(\mu ,\lambda ,\theta )}(u_t,x_t,y_t,z_t)f_{{\mathcal{F}}_t}^{(\mu ,\lambda ,\theta )}(x_t,y_t,z_t|u_t).\hfill \end{array}$$

For each $t=1,2,\cdots ,n$, we recursively choose $q_t=q_{U_t{X}_t{Y}_t{Z}_t}$ so that $q_{U_t{X}_t{Y}_t{Z}_t}=p_{U_t{X}_t{Y}_t{Z}_t;{\mathcal{F}}^{t-1}}^{(\mu ,\lambda ,\theta )}$ and choose $Q_{X_t}^{\left(\mathrm{i}\right)}$, $Q_{Y_t|X_t{U}_t}^{\left(\mathrm{ii}\right)}$, $Q_{X_t|U_t{Y}_t{Z}_t}^{\left(\mathrm{iii}\right)}$, and $Q_{X_t|Y_t{U}_t}^{\left(\mathrm{iv}\right)}$ appearing in

$$\begin{array}{c}{f}_{{\mathcal{F}}_t}^{(\mu ,\lambda ,\theta )}(x_t,y_t,z_t|u_t)\hfill \\ :={\left\{\frac{p_{X_t}\left(x_t\right)}{Q_{X_t}^{\left(\mathrm{i}\right)}\left(x_t\right)}\frac{p_{Y_t|X_t}\left(y_t\right|x_t)}{Q_{Y_t|X_t{U}_t}^{\left(\mathrm{ii}\right)}\left(y_t\right|x_t,u_t)}\frac{p_{X_t|U_t{Y}_t}\left(x_t\right|u_t,y_t)}{Q_{X_t|U_t{Y}_t{Z}_t}^{\left(\mathrm{iii}\right)}\left(x_t\right|u_t,y_t,z_t)}{\left(\frac{p_{X_t|Y_t}\left(x_t\right|y_t)}{Q_{X_t|Y_t{U}_t}^{\left(\mathrm{iv}\right)}\left(x_t\right|u_t,y_t)}\right)}^{\lambda \overline{\mu }}{\mathrm{e}}^{-\lambda \mu d(x_t,z_t)}\right\}}^{\theta }\hfill \end{array}$$

such that they are the distributions induced by $q_{U_t{X}_t{Y}_t{Z}_t}$. Then, for each $t=1,2,$⋯, n, we have the following chain of inequalities:

$$\begin{array}{c}{\mathsf{\Lambda }}_{t,{\mathcal{F}}^t}^{(\mu ,\lambda ,\theta )}={\mathrm{E}}_{q_t}\left[{\left\{\frac{p_{X_t}\left(X_t\right)}{q_{X_t}\left(X_t\right)}\frac{p_{Y_t|X_t}\left(Y_t\right|X_t)}{q_{Y_t|X_t{U}_t}\left(Y_t\right|X_t,U_t)}\frac{p_{X_t|U_t{Y}_t}\left(X_t\right|U_t,Y_t)}{q_{X_t|U_t{Y}_t{Z}_t}\left(X_t\right|U_t,Y_t,Z_t)}\frac{p_{X_t|Y_t}^{\lambda \overline{\mu }}\left(X_t\right|Y_t){\mathrm{e}}^{-\lambda \mu d(X_t,Z_t)}}{q_{X_t|Y_t{U}_t}^{\lambda \overline{\mu }}\left(X_t\right|U_t,Y_t)}\right\}}^{\theta }\right]\hfill \\ ={\mathrm{E}}_{q_t}\left[{\left\{\frac{p_{X_t}\left(X_t\right)}{q_{X_t}\left(X_t\right)}\frac{p_{Y_t|X_t}\left(Y_t\right|X_t)}{q_{Y_t|X_t{U}_t}\left(Y_t\right|X_t,U_t)}\frac{q_{X_t|U_t{Y}_t}\left(X_t\right|U_t,Y_t)}{q_{X_t|U_t{Y}_t{Z}_t}\left(X_t\right|U_t,Y_t,Z_t)}\frac{p_{X_t|Y_t}^{\lambda \overline{\mu }}\left(X_t\right|Y_t){\mathrm{e}}^{-\lambda \mu d(X_t,Z_t)}}{q_{X_t|Y_t{U}_t}^{\lambda \overline{\mu }}\left(X_t\right|U_t,Y_t)}\right\}}^{\theta }\right.\hfill \\ \times \left.{\left\{\frac{p_{X_t|U_t{Y}_t}\left(X_t\right|U_t,Y_t)}{q_{X_t|U_t{Y}_t}\left(X_t\right|U_t,Y_t)}\right\}}^{\theta }\right]\hfill \\ \stackrel{\left(\mathrm{a}\right)}{\le }{\left({\mathrm{E}}_{q_t}\left[{\left\{\frac{p_{X_t}^{}\left(X_t\right)}{q_{X_t}^{}\left(X_t\right)}\frac{p_{Y_t|X_t}\left(Y_t\right|X_t)}{q_{Y_t|X_t{U}_t}\left(Y_t\right|X_t,U_t)}\frac{q_{Z_t|U_t{Y}_t}\left(Z_t\right|U_t,Y_t)}{q_{Z_t|U_t{X}_t{Y}_t}\left(Z_t\right|U_t,X_t,Y_t)}\frac{p_{X_t|Y_t}^{\lambda \overline{\mu }}\left(X_t\right|Y_t){\mathrm{e}}^{-\lambda \mu d(X_t,Z_t)}}{q_{X_t|Y_t{U}_t}^{\lambda \overline{\mu }}\left(X_t\right|U_t,Y_t)}\right\}}^{\frac{\theta }{1-\theta }}\right]\right)}^{1-\theta }\hfill \\ \times {\left({\mathrm{E}}_{q_t}\left\{\frac{p_{X_t|U_t{Y}_t}\left(X_t\right|U_t,Y_t)}{q_{X_t|U_t{Y}_t}\left(X_t\right|U_t,Y_t)}\right\}\right)}^{\theta }=exp\left\{-(1-\theta ){\mathsf{\Omega }}^{(\mu ,\lambda ,\frac{\theta }{1-\theta })}\left(q_t\right|p_{XY})\right\}\hfill \\ \stackrel{\left(\mathrm{b}\right)}{=}exp\left\{-\frac{{\mathsf{\Omega }}^{(\mu ,\lambda ,\alpha )}\left(q_t\right|p_{XY})}{1+\alpha }\right\}\stackrel{\left(\mathrm{c}\right)}{\le }exp\left\{-\frac{{\widehat{\mathsf{\Omega }}}_n^{(\mu ,\lambda ,\alpha )}\left(p_{XY}\right)}{1+\alpha }\right\}\stackrel{\left(\mathrm{d}\right)}{=}exp\left\{-\frac{{\mathsf{\Omega }}^{(\mu ,\lambda ,\alpha )}\left(p_{XY}\right)}{1+\alpha }\right\}.\hfill \end{array}$$

(45)

Step (a) follows from Hölder’s inequality and the following identity:

$$\begin{array}{c}\frac{q_{X_t|U_t{Y}_t}\left(X_t\right|U_t,Y_t)}{q_{X_t|U_t{Y}_t{Z}_t}\left(X_t\right|U_t,Y_t,Z_t)}=\frac{q_{Z_t|U_t{Y}_t}\left(Z_t\right|U_t,Y_t)}{q_{Z_t|U_t{X}_t{Y}_t}\left(Z_t\right|U_t,X_t,Y_t)}\mathrm{for} t=1,2,\cdots ,n.\hfill \end{array}$$

Step (b) follows from Equation (43). Step (c) follows from the definition of ${\widehat{\mathsf{\Omega }}}_n^{(\mu ,\lambda ,\alpha )}$$\left(p_{XY}\right)$. Step (d) follows from that by Property 4 Part (a), the bound $\left|\mathcal{U}\right|\le \left|\mathcal{X}\right|$, is sufficient to describe ${\widehat{\mathsf{\Omega }}}_n^{(\mu ,\lambda ,\alpha )}\left(p_{XY}\right)$. Hence, we have the following:

$$\begin{array}{c}\underset{q^n\in {\mathcal{Q}}^n}{max}\frac{1}{n}{\mathsf{\Omega }}_{}^{(\mu ,\lambda ,\theta )}(p^{\left(n\right)},{\underline{Q}}^n|p_{XY})\hfill \\ \ge \frac{1}{n}{\mathsf{\Omega }}_{}^{(\mu ,\lambda ,\theta )}(p^{\left(n\right)},{\underline{Q}}^n|p_{XY})\stackrel{\left(\mathrm{a}\right)}{=}-\frac{1}{n}\sum _{t=1}^{n}log{\mathsf{\Lambda }}_{t,{\mathcal{F}}^t}^{(\mu ,\lambda ,\theta )}\stackrel{\left(\mathrm{b}\right)}{\ge }\frac{{\mathsf{\Omega }}^{(\mu ,\lambda ,\alpha )}\left(p_{XY}\right)}{1+\alpha }.\hfill \end{array}$$

(46)

Step (a) follows from Equation (38) in Lemma 6. Step (b) follows from Equation (45). Since Equation (46) holds for any $n\ge 1$ and any $p^{\left(n\right)}\in {\mathcal{P}}^{\left(n\right)}$$\left(p_{XY}\right)$, we have

$${\underline{\mathsf{\Omega }}}^{(\mu ,\lambda ,\theta )}\left(p_{XY}\right)\ge \frac{{\mathsf{\Omega }}^{(\mu ,\lambda ,\alpha )}\left(p_{XY}\right)}{1+\alpha }.$$

Thus, we have Equation (44) in Proposition 2. ☐

Proof of Theorem 3:

For $\theta \in [0,1)$, set

$$\alpha =\frac{\theta }{1-\theta }\iff \theta =\frac{\alpha }{1+\alpha }.$$

(47)

Then, we have the following:

$$\begin{array}{cc}& \frac{1}{n}log\left\{\frac{5}{{\mathrm{P}}_{\mathrm{c}}^{\left(n\right)}({\phi }^{\left(n\right)},{\psi }^{\left(n\right)};\Delta )}\right\}\stackrel{\left(\mathrm{a}\right)}{\ge }\frac{{\underline{\mathsf{\Omega }}}^{(\mu ,\lambda ,\theta )}\left(p_{XY}\right)-\lambda \theta (\overline{\mu }R+\mu \Delta )}{1+\theta (3+\lambda \overline{\mu })}\hfill \\ \stackrel{\left(\mathrm{b}\right)}{\ge }& \frac{\frac{1}{1+\alpha }{\mathsf{\Omega }}^{(\mu ,\lambda ,\alpha )}\left(p_{XY}\right)-\frac{\lambda \alpha }{1+\alpha }(\overline{\mu }R+\mu \Delta )}{1+\frac{\alpha }{1+\alpha }(3+\lambda \overline{\mu })}=\frac{{\mathsf{\Omega }}^{(\mu ,\lambda ,\alpha )}\left(p_{XY}\right)-\lambda \alpha (\overline{\mu }R+\mu \Delta )}{1+\alpha +\alpha (3+\lambda \overline{\mu })}\hfill \\ =& F^{(\mu ,\lambda ,\alpha )}(\overline{\mu }R+\mu \Delta |p_{XY}).\hfill \end{array}$$

Step (a) follows from Corollary 3. Step (b) follows from Proposition 2 and Equation (47). Since the above bound holds for any positive $\lambda \ge 0$, $\mu \in [0,1],$ and $\alpha \ge 0$, we have

$$\frac{1}{n}log\left\{\frac{5}{{\mathrm{P}}_{\mathrm{c}}^{\left(n\right)}({\phi }^{\left(n\right)},{\psi }^{\left(n\right)};\Delta )}\right\}\ge F(R,\Delta |p_{XY}).$$

Thus, Equation (10) in Theorem 3 is proved. ☐

Proof of Corollary 2:

Since g is an inverse function of $\vartheta$, the definition in Equation (13) of ${\kappa }_n$ is equivalent to

$$g\left(\frac{{\kappa }_n}{\rho \left(p_{XY}\right)}\right)=\sqrt{\frac{10}{n\rho \left(p_{XY}\right)}log\left(\frac{5}{1-\epsilon }\right)}.$$

(48)

By the definition of $n_0=n_0(\epsilon ,\rho \left(p_{XY}\right))$, we have that ${\kappa }_n\le (1/2)\rho \left(p_{XY}\right)$ for $n\ge n_0$. We assume that for $n\ge n_0$, $(R,\Delta )\in {\mathcal{R}}_{\mathrm{WZ}}(n,\epsilon |p_{XY}).$ Then, there exists a sequence $\left\{\right({\phi }^{\left(n\right)},$${\psi }^{\left(n\right)})$ ${\}}_{n\ge n_0}$ such that for $n\ge n_0$, we have

$$\frac{1}{n}log\left|\right|{\phi }^{\left(n\right)}\left|\right|\le R,{\mathrm{P}}_{\mathrm{e}}^{\left(n\right)}({\phi }^{\left(n\right)},{\psi }^{\left(n\right)};\Delta )\le \epsilon .$$

Then, by Theorem 3, we have

$$\begin{array}{ccc}\hfill 1-\epsilon & \le & {\mathrm{P}}_{\mathrm{c}}^{\left(n\right)}({\phi }^{\left(n\right)},{\psi }^{\left(n\right)};\Delta )\le 5exp\left\{-nF(R,\Delta |p_{XY})\right\}\hfill \end{array}$$

(49)

for any $n\ge n_0$. We claim that for $n\ge n_0$, we have $(R+{\kappa }_n,\Delta +{\kappa }_n)$ ∈ $\mathcal{R}\left(p_{XY}\right)$. To prove this claim, we suppose that $(R+{\kappa }_{n^{*}},\Delta +{\kappa }_{n^{*}})$ does not belong to $\mathcal{R}\left(p_{XY}\right)$ for some $n^{*}\ge n_0$. Then, we have the following chain of inequalities:

$$\begin{array}{cc}& 5exp\left[-n^{*}F(R,\Delta |p_{XY})\right]\stackrel{\left(\mathrm{a}\right)}{<}5exp\left[-\frac{n^{*}\rho \left(p_{XY}\right)}{10}·g^2\left(\frac{{\kappa }_{n^{*}}}{\rho \left(p_{XY}\right)}\right)\right]\hfill \\ \stackrel{\left(\mathrm{b}\right)}{=}& 5exp\left[-\frac{n^{*}\rho \left(p_{XY}\right)}{10}\frac{10}{n^{*}\rho \left(p_{XY}\right)}log\left(\frac{5}{1-\epsilon }\right)\right]=5exp\left[log\left(\frac{1-\epsilon }{5}\right)\right]=1-\epsilon .\hfill \end{array}$$

(50)

Step (a) follows from ${\kappa }_{n^{*}}\le (1/2)\rho \left(p_{XY}\right)$ and Property 4 Part (e). Step (b) follows from Equation (48). The bound of Equation (50) contradicts Equation (49). Hence, we have $(R+{\kappa }_n,\Delta +$${\kappa }_n)$ ∈ $\mathcal{R}\left(p_{XY}\right)$ or equivalent to

$$(R,\Delta )\in \mathcal{R}\left(p_{XY}\right)-{\kappa }_n(1,1)$$

for $n\ge n_0$, which implies that for $n\ge n_0$,

$${\mathcal{R}}_{\mathrm{WZ}}(n,\epsilon |p_{XY})\subseteq \mathcal{R}\left(p_{XY}\right)-{\kappa }_n(1,1),$$

completing the proof. ☐

5. Conclusions

For the WZ system, we have derived an explicit lower bound of the optimal exponent function on the correct probability of decoding for for $(R,\Delta )\notin {\mathcal{R}}_{\mathrm{WZ}}\left(p_{XY}\right)$. We have described this result in Theorem 3. The determination problem of the optimal exponent remains to be resolved. This problem is our future work.

In this paper, we have treated the case where $\mathcal{X}$ and $\mathcal{Y}$ are finite sets. Extension of Theorem 3 to the case where $\mathcal{X}$ and $\mathcal{Y}$ are arbitrary sets is also our future work. Wyner [12] investigated the characterization of the rate distortion region in the case where $\mathcal{X}$ and $\mathcal{Y}$ are general sets and ${\left\{(X_t,Y_t)\right\}}_{t=1}^{\infty }$ is a correlated stationary memoryless source. This work may provide a good tool to investigate the second future work.