Source Coding with a Causal Helper

Abstract

A multi-terminal network, in which an encoder is assisted by a side-information-aided helper, describes a memoryless identically distributed source to a receiver, is considered. The encoder provides a non-causal one-shot description of the source to both the helper and the receiver. The helper, which has access to causal side-information, describes the source to the receiver sequentially by sending a sequence of causal descriptions depending on the message conveyed by the encoder and the side-information subsequence it has observed so far. The receiver reconstructs the source causally by producing on each time unit an estimate of the current source symbol based on what it has received so far. Given a reconstruction fidelity measure and a maximal allowed distortion, we derive the rates-distortion region for this setting and express it in terms of an auxiliary random variable. When the source and side-information are drawn from an independent identically distributed Gaussian law and the fidelity measure is the squared-error distortion we show that for the evaluation of the rates-distortion region it suffices to choose the auxiliary random variable to be jointly Gaussian with the source and side-information pair.

1. Introduction

In the classical source coding with decoder side information problem, the source and side information are generated by independent drawings $(X_k,Y_k)$ of the pair $(X,Y)\sim P_{XY}$. The encoder forms a description of the source sequence $X^n=(X_1,\dots ,X_n)$ using a map $f^{\left(n\right)}:{\mathcal{X}}^n\to \{1,\dots ,\lfloor 2^{nR}\rfloor \}$, while the decoder forms its reconstruction ${\widehat{X}}^n$ depending on both the side-information sequence $Y^n$ and the index $T\in \{1,\dots ,\lfloor 2^{nR}\rfloor \}$ conveyed by the encoder. In their seminal work [1], Wyner and Ziv derived the rate distortion function for this setting, given a fidelity measure, when ${\widehat{X}}^n$ can depend on $Y^n$ in an arbitrary manner. Yet, Wyner–Ziv source coding with non-causal decoder side-information involves binning the implementation of which is complex.

A successive refinement for the Wyner–Ziv problem with side-information $(Y,Z)$ is a variant of the Wyner–Ziv model, in which the encoder provides a two-layer description of the source sequence $X^n$ to a pair of decoders. Decoder 1, which obtains just the course description layer, has available as non-causal side-information the memoryless vector $Z^n$, while Decoder 2, which obtains both description layers, has available as non-causal side-information the memoryless vector $Y^n$. It is further assumed that the reconstruction formed by Decoder 2 should be of smaller distortion compared to that formed by Decoder 1. Such a model has been considered in [2], wherein a complete characterization of the rates-distortion region has been obtained for the case where Z is stochastically degraded to Y with respect to X—i.e., $X\leftrightarrow Y\leftrightarrow Z$ forms a Markov chain.

The work [3] studies an extension of the model of [2] where a conference link of given capacity allows the unidirectional cooperation between Decoder 1 and Decoder 2—i.e., Decoder 1 functions also as a helper. The results in [3] are partially tight in the sense that the characterization of the encoder rates is conclusive, the remaining gap being in the characterization of the helper’s rate. Thus, with non-causal side-information at both decoders, the successive refinement for the Wyner–Ziv problem with a helper is yet unsolved.

Motivated by practical delay-constrained sequential source coding with decoder side information Weissman and El Gamal considered in [4], a scheme with causal side information at the decoder, where the sequence of reconstructions ${\widehat{X}}^n=({\widehat{X}}_1,\dots ,{\widehat{X}}_n)$ is formed sequentially in a causal manner according to ${\widehat{X}}_k={\widehat{X}}_k(T,Y^k)$, and derived the corresponding rate distortion function. Similar to [1], the rate distortion function in [4] is expressed in terms of an auxiliary random variable, thus leaving the optimal choice of which an open issue depending on the specifics of the model. For the Gaussian setting where $(X,Y)$ are Gaussian, the authors in [4] compute an upper bound on the rate distortion function by choosing the auxiliary random variable to be jointly Gaussian with X, while leaving the question of whether this choice is optimal yet an open problem.

With the vision that modern network design will support the use of cooperation links in favor of reduction of encoding/decoding complexity and network deployment constraints, this work considers an extension of the model [4], involving a causal helper and causal side information at the decoder, which is described as follows. The components of a trivariate independent identically distributed (IID) finite-alphabet source ${\left\{X_k,Y_k,Z_k\right\}}_{k=1}^{\infty }$ are observed by three terminals. The source component ${\left\{X_k\right\}}_{k=1}^{\infty }$ is observed by the encoder, while the source component ${\left\{Y_k\right\}}_{k=1}^{\infty }$ is observed by the helper. Both the encoder and the helper describe the length-n source sequence $X^n$ according to a given fidelity to the decoder in two steps. First, the encoder when given $X^n$ sends a rate-R description of it to both the helper and the decoder. Then, the helper sends to the decoder, per each source symbol $Y_k$, a causal description depending on the message it had received from the encoder and the source subsequence $Y^k$ it had observed so far, with the aggregate rate not exceeding $R_{\mathrm{h}}$. The decoder, which observes the source component ${\left\{Z_k\right\}}_{k=1}^{\infty }$ in a causal manner, per channel use k, uses the descriptions it had received so far and the source subsequence $Z^k$ to form its reconstruction ${\widehat{X}}_k$ for the source symbol $X_k$. Given a fidelity measure and a maximal allowed distortion, the goal is to determine the set of all rate pairs $(R,R_{\mathrm{h}})$ that satisfy the distortion constraint.

Causal decoder side information has been considered as well in context with successive refinement. With the aim of reducing encoder/decoder complexity, a two layer description model with successive refinement has been considered in [5], under the setting that the side information is available causally at each of the decoders. A single-letter characterization of the rates-distortion region is obtained in [5], irrespective of the relative ordering of the side information quality at the decoders. Furthermore, the direct part in [5] demonstrates that similarly to [4] with causal side-information at the decoders the optimal code avoids binning, hence its implementation is practically appealing. The extension of the model [5] with a causal helper has recently been studied in [6].

2. Problem Formulation

Formally, our problem can be stated as follows. A discrete memoryless source (DMS) $(\mathcal{X},P_X)$ is an infinite sequence ${\left\{X_i\right\}}_{i=1}^{\infty }$ of independent copies of a random variable X taking values in the finite set $\mathcal{X}$ with generic law $P_X$. Similarly, a triple source $(\mathcal{X}\mathcal{Y}\mathcal{Z},P_{XYZ})$ is an infinite sequence of independent copies of the triplet of random variables $(X,Y,Z)$, taking values in the finite sets $\mathcal{X},\mathcal{Y}$ and $\mathcal{Z}$, respectively, with generic joint law $P_{XYZ}$. Since our goal is to encode the source X, let $\widehat{\mathcal{X}}$ denote any finite reconstruction alphabet and let $d:\mathcal{X}\times \widehat{\mathcal{X}}\to [0,\infty )$ be a single-letter distortion measure. The vector distortion measure is defined as

$$d(x,\widehat{x})=\frac{1}{n}\sum _{i=1}^{n}d(x_i,{\widehat{x}}_i),\forall x\in {\mathcal{X}}^n,\widehat{x}\in {\widehat{\mathcal{X}}}^n.$$

A system diagram appears in Figure 1.

Figure 1. The causal listening-helper model.

Definition 1.

An$\left(n,M^{\left(n\right)},{\prod }_{k=1}^n{L}_k^{\left(n\right)},D\right)$code for the source X with causal side-information (SI)$(Y,Z)$and causal helper, consists of:

1. 

An encoder mapping

$$\begin{array}{c}\hfill f^{\left(n\right)}:{\mathcal{X}}^n\to \{1,\dots ,M^{\left(n\right)}\}.\end{array}$$

(1)

2. 

A unidirectional conference between the helper and the decoder consisting of a sequence of causal descriptions

$$\begin{array}{c}\hfill f_{1,k}^{\left(n\right)}:\{1,\dots ,M^{\left(n\right)}\}\times {\mathcal{Y}}^k\to \{1,\dots ,L_k^{\left(n\right)}\},k=1,\dots ,n\end{array}$$

(2)

and

3. 

A sequence $g_1^{\left(n\right)},\dots ,g_n^{\left(n\right)}$ of decoder reconstructions

$$\begin{array}{c}\hfill g_k^{\left(n\right)}:\{1,\dots ,M^{\left(n\right)}\}\times \{1,\dots ,L_1^{\left(n\right)}\}\times \dots \times \{1,\dots ,L_k^{\left(n\right)}\}\times {\mathcal{Z}}^k\to \widehat{\mathcal{X}},k=1,\dots ,n\end{array}$$

(3)

such that

$$\begin{array}{c}\hfill \mathbb{E}d[X^n,(g_1^{\left(n\right)}\left(f^{\left(n\right)}\left(X^n\right),f_{1,1}^{\left(n\right)}(f^{\left(n\right)}\left(X^n\right),Y_1),Z_1\right),\dots ,\\ \hfill g_k^{\left(n\right)}\left(f^{\left(n\right)}\left(X^n\right),{\left\{f_{1,j}^{\left(n\right)}(f^{\left(n\right)}\left(X^n\right),Y^j)\right\}}_{j=1}^k,Z^k\right),\dots ,\\ \hfill g_n^{\left(n\right)}\left(f^{\left(n\right)}\left(X^n\right),{\left\{f_{1,j}^{\left(n\right)}(f^{\left(n\right)}\left(X^n\right),Y^j)\right\}}_{j=1}^n,Z^n\right)\left)\right]\le D.\end{array}$$

(4)

The rate tuple $(R,R_{\mathrm{h}})$ of the code is

$$\begin{array}{ccc}\hfill R& =& \frac{1}{n}log{M}^{\left(n\right)}\hfill \\ \hfill R_{\mathrm{h}}& =& \frac{1}{n}\sum _{k=1}^{n}log{L}_k^{\left(n\right)}.\hfill \end{array}$$

(5)

Given a non-negative distortion D, the tuple $(R,R_{\mathrm{h}})$ is said to be D-achievable for X with causal SI $(Y,Z)$ if, for any $\delta >0$, $ϵ>0$, and sufficiently large n, there exists an $(n,exp\left[n(R+\delta )\right],exp\left[n(R_{\mathrm{h}}+\delta )\right],D+ϵ)$ code for the source X with causal SI $(Y,Z)$ and causal helper. The collection of all D-achievable rate tuples is the achievable source-coding region and is denoted by $\mathcal{R}\left(D\right)$.

In this work, we provide a single-letter characterization for $\mathcal{R}\left(D\right)$. In contrast to [5], a consequence of Definition 1 is that $\mathcal{R}\left(D\right)$ may depend on the joint law of the triple $P_{XYZ}$, not only through the marginal laws $P_{XY}$ and $P_{XZ}$. This is due to the fact that the decoder acquires, in addition to its private side information Z, some additional side information on Y via the conference link. As a result, the expectation in (4), which takes into account the mapping $g_k^{\left(n\right)}(·)$, is taken over the joint law $P_{XYZ}$ and not just over the marginal law $P_{XZ}$, as is the case in [5].

Finally, although not of the finite alphabet, of particular interest to us is the Gaussian source. This is a memoryless source, where $(X,Y,Z)$ are centered jointly Gaussians with each pair $(X_i,Y_i)$ drawn such that $P_{XY}$ satisfies

$$\begin{array}{c}\hfill Y_i=\sqrt{\rho }{X}_i+W_i\end{array}$$

(6)

where $\rho >0$ is a fixed constant, $X_i\sim \mathcal{N}(0,1)$ and $W_i\sim \mathcal{N}(0,1)$ are mutually independent. Moreover, $Z_i$ is drawn according to

$$\begin{array}{c}\hfill Z_i=a{X}_i+b{Y}_i+T_i\end{array}$$

(7)

where a and b are real numbers and $T_i\sim \mathcal{N}(0,1)$ is independent of $(X_i,Y_i)$. Furthermore, in this case, the fidelity measure will be

$$\mathbb{E}d[X^n,{\widehat{X}}^n]\triangleq \frac{1}{n}\sum _{i=1}^n\mathbb{E}\left[{(X_i-{\widehat{X}}_i)}^2\right],$$

(8)

in which case we may restrict the reproduction functions $g_k^{\left(n\right)}$ to be the MMSE estimates of $X_k,k=1,\dots ,n$ given $f^{\left(n\right)}\left(X^n\right)$, the sequence of causal descriptions ${\left\{f_{1,j}^{\left(n\right)}(f^{\left(n\right)}\left(X^n\right),Y^j)\right\}}_{j=1}^k$, and the side-information $Z^k$. That is,

$$\begin{array}{c}\hfill {\widehat{X}}_k=\mathbb{E}[X_k\left|f^{\left(n\right)}\left(X^n\right),{\left\{f_{1,j}^{\left(n\right)}(f^{\left(n\right)}\left(X^n\right),Y^j)\right\}}_{j=1}^k,Z^k\right].\end{array}$$

In the Gaussian network setting, our focus will be on determining the optimal choice of the auxiliary random variable by means of which the rates-distortion region is defined. Specifically, we will show that choosing it to be jointly Gaussian with X is optimal.

3. Main Results

Given a maximal allowed distortion D, define ${\mathcal{R}}^{*}\left(D\right)$ to be the set of all rate pairs $(R,R_{\mathrm{h}})$ for which there exist random variables $(U,V)$, taking values in finite alphabets $\mathcal{U},\mathcal{V}$, respectively, such that

• $U\leftrightarrow X\leftrightarrow (Y,Z)$ forms a Markov chain.
• Conditioned on U, $X\leftrightarrow Y\leftrightarrow V$ forms a Markov chain.
• There exist deterministic maps

  $$\begin{array}{ccc}\hfill {\tilde{f}}_1& :& \mathcal{U}\times \mathcal{Y}\to \mathcal{V}\hfill \\ \hfill g& :& \mathcal{U}\times \mathcal{V}\times \mathcal{Z}\to \widehat{\mathcal{X}}\hfill \end{array}$$

  (9)

  such that, with $V\triangleq {\tilde{f}}_1(U,Y)$,

  $$\mathbb{E}d(X,g(U,V,Z\left)\right)\le D.$$

  (10)
• The alphabets $\mathcal{U},\mathcal{V}$ satisfy

  $$\begin{array}{cc}\hfill \left|\mathcal{U}\right|& \le \left|\mathcal{X}\right|+2\hfill \\ \hfill \left|\mathcal{V}\right|& \le \left|\mathcal{X}\right|\left(\right|\mathcal{X}|+2)+1.\hfill \end{array}$$

  (11)
• The rates R and $R_{\mathrm{h}}$ satisfy

  $$\begin{array}{ccc}\hfill R& \ge & I(X;U)\hfill \end{array}$$

  (12a)

  $$\begin{array}{ccc}\hfill R_{\mathrm{h}}& \ge & H\left(V\right|U)=I(Y;V\left|U\right).\hfill \end{array}$$

  (12b)

Our first result is a single-letter characterization of the rates-distortion region.

Theorem 1.

$\mathcal{R}\left(D\right)={\mathcal{R}}^{*}\left(D\right)$.

Proof. 

See Section 4.1. □

Remark 1.

The converse holds as well for the setting where the causal helper and the reconstructor benefit from causal disclosure—i.e., are cognizant of the past realizations of the source sequence, hence they are allowed to depend also on $X^{k-1}$, when forming $f_{1,k}^{\left(n\right)}$ and ${\widehat{X}}_k$, respectively. That is,

$$\begin{array}{ccc}\hfill f_{1,k}^{\left(n\right)}& =& f_{1,k}^{\left(n\right)}(f^{\left(n\right)}\left(X^n\right),Y^k,X^{k-1}),\hfill \\ \hfill {\widehat{X}}_k& =& g_k^{\left(n\right)}\left(f^{\left(n\right)}\left(X^n\right),{\left\{f_{1,j}^{\left(n\right)}(f^{\left(n\right)}\left(X^n\right),Y^i,X^{i-1})\right\}}_{j=1}^k,Z^k,X^{k-1}\right).\hfill \end{array}$$

(13)

3.1. The Gaussian Setting with $Z=\varnothing$.

To simplify the presentation, we consider first the Gaussian setting with $Z=\varnothing$. In this case, the region $\mathcal{R}\left(D\right)$ is defined as the set of rate pairs $(R,R_{\mathrm{h}})$ for which there exist random variables $(U,V)$, taking real values, such that

• $U\leftrightarrow X\leftrightarrow Y$ forms a Markov chain.
• Conditioned on U, $X\leftrightarrow Y\leftrightarrow V$ forms a Markov chain.
• There exist deterministic maps

  $$\begin{array}{ccc}\hfill {\tilde{f}}_1& :& \mathcal{U}\times \mathcal{Y}\to \mathcal{V}\hfill \\ \hfill g& :& \mathcal{U}\times \mathcal{V}\to \widehat{\mathcal{X}}\hfill \end{array}$$

  (14)

  such that, with $V\triangleq {\tilde{f}}_1(U,Y)$,

  $$D\ge \mathbb{E}\left[{\left(X-g(U,V)\right)}^2\right]\triangleq {\sigma }_{X|UV}^2.$$

  (15)
• The rates R and $R_{\mathrm{h}}$ satisfy (12a) and (12b), respectively.

Our second result characterizes the optimal choice of $P_{XU}$ in the Gaussian setting.

Theorem 2.

For the evaluation of $\mathcal{R}\left(D\right)$ when $(X,Y)$ are Gaussian, it suffices to assume that $(X,U)$ are jointly Gaussian.

Proof. 

For the treatment to follow, let us define the Gaussian channel $Y=\sqrt{\varrho }X+W$ where X has an arbitrary law with $\mathbb{E}\left[X^2\right]\le 1$, $W\sim \mathcal{N}(0,1)$ is independent of X, and $\varrho >0$ is the channel signal-to-noise ratio. We shall denote by ${\mathbb{G}}_{Y|X}^{\left(\varrho \right)}$ the conditional law of Y given X for this additive Gaussian model. Furthermore, the notation $(U_n,X_n,Y_n)\sim P_{U_n{X}_n}{\mathbb{G}}_{Y_n|X_n}^{\left(\varrho \right)}$ would imply that $(U_n,X_n)\sim P_{U_n{X}_n}$ and $Y_n=\sqrt{\varrho }{X}_n+W_n$ with $W_n\sim \mathcal{N}(0,1)$ independent of $(U_n,X_n)$. Using this notation, the law $P_{UXYV}$ defining $\mathcal{R}\left(D\right)$ (see also (45) ahead) may equivalently be expressed as

$$P_{XUYV}=P_{XU}{\mathbb{G}}_{Y|X}^{\left(\varrho \right)}{P}_{V|YU}.$$

(16)

Henceforth, we denote a law which factors as in (16) by ${X\leftrightarrow Y\leftrightarrow V|}_U$—i.e., conditioned on U, $X\leftrightarrow Y\leftrightarrow V$ forms a Markov chain and, furthermore, ${\mathbb{P}}_{Y|X}={\mathbb{G}}_{Y|X}^{\left(\varrho \right)}$ independently of the rest.

Define the region (12a) and (12b) subject to constraint (15) by ${\mathcal{O}}_K$, where the subscript K denotes the covariance constraint (15). The region ${\mathcal{O}}_K$ is a closed convex set.

In line with [7,8], we define a $\lambda$-parametrized family of functions which are related to the sum rate associated with $\mathcal{R}\left(D\right)$.

Fix some $\lambda >1$, and consider the minimization of the $\lambda$-sum rate defined as

$$\underset{(R,R_{\mathrm{h}})\in {\mathcal{O}}_K}{min}R+\lambda R_{\mathrm{h}}.$$

(17)

Observe that

$$\begin{array}{c}\hfill \underset{(R,R_{\mathrm{h}})\in {\mathcal{O}}_K}{min}R+\lambda R_{\mathrm{h}}\stackrel{\left(a\right)}{\ge }\underset{\genfrac{}{}{0.0pt}{}{P_{XUYV}{:X\leftrightarrow Y\leftrightarrow V|}_U}{{\sigma }_{X|UV}^2\le D}}{inf}I(X;U)+\lambda I(Y;V|U),\end{array}$$

(18)

where $\left(a\right)$ follows using the lower bounds (12a) and (12b). Since the marginal law of X in our model (6) is Gaussian, the differential entropy $h\left(X\right)$ is fixed, thus for the minimization of (18) over a law of the form (16), we define the following functional of $P_{XU}$ (i.e., of the conditional law $P_{X|U}$)

$$\begin{array}{c}\hfill {\mathsf{s}}_{\lambda }(X,\varrho |U)\triangleq -h\left(X\right|U)+\underset{\genfrac{}{}{0.0pt}{}{{V:X\leftrightarrow Y\leftrightarrow V|}_U}{{\sigma }_{X|UV}^2\le D}}{inf}\lambda I(Y;V|U).\end{array}$$

(19)

Thus, the minimum in (17) may be expressed as

$${\mathsf{V}}_{\lambda }\left(\varrho \right)\triangleq \underset{P_{XU}:E\left[X^2\right]\le 1}{inf}{\mathsf{s}}_{\lambda }(X,\varrho |U).$$

(20)

Henceforth, the set of laws $P_{UXYV}$ defined by (16) which satisfy ${\sigma }_{X|UV}^2\le D$ will be denoted by $\mathcal{Q}$ and will be attributed as the feasible set.

As shown below, with a proper choice of $\lambda$, the functional (19) exhibits a “pair grouping” property with respect to the input X in the sense that the value of ${\mathsf{s}}_{\lambda }$ does not increase under this operation. Having established that, we follow the same steps as in the proof of ([9] Theorem 9) to establish that the objective (20) is attained when $P_{X|U}$ is Gaussian. More specifically,

• Lemma 1 shows that the value of ${\mathsf{s}}_{\lambda }$ “improves” under the pair grouping operation.
• With the proper time-sharing of two distributions attaining the infimum in (20) and satisfying the extremal property defined in (23) ahead, Lemma 2 proves that the pair grouping operation exhibits Gaussianity in the sense of Bernstein’s characterization.

Lemma 1.

Let $P_{XU}$ be a law on $\mathcal{X}\times \mathcal{U}$, let $P_{UXY}=P_{XU}{\mathbb{G}}_{Y|X}^{\left(\varrho \right)}$, and let $(U_1,X_1,Y_1)$ and $(U_2,X_2,Y_2)$ be two independent copies of $(U,X,Y)$. Define

$$\begin{array}{c}\hfill X_{+}\triangleq \frac{X_1+X_2}{\sqrt{2}},X_{-}\triangleq \frac{X_1-X_2}{\sqrt{2}}\end{array}$$

(21)

and similarly

$$\begin{array}{c}\hfill Y_{+}\triangleq \frac{Y_1+Y_2}{\sqrt{2}},Y_{-}\triangleq \frac{Y_1-Y_2}{\sqrt{2}}.\end{array}$$

with $\mathbf{U}=(U_1,U_2)$, there exists some ${\lambda }^{*}>1$ such that for any $\lambda \ge {\lambda }^{*}$, the following inequalities hold

$$\begin{array}{ccc}\hfill 2{\mathsf{s}}_{\lambda }(X,\varrho |U)& \ge & {\mathsf{s}}_{\lambda }(X_{+},\varrho |X_{-},\mathbf{U})+{\mathsf{s}}_{\lambda }(X_{-},\varrho |Y_{+},\mathbf{U})\hfill \end{array}$$

(22a)

$$\begin{array}{ccc}\hfill 2{\mathsf{s}}_{\lambda }(X,\varrho |U)& \ge & {\mathsf{s}}_{\lambda }(X_{+},\varrho |Y_{-},\mathbf{U})+{\mathsf{s}}_{\lambda }(X_{-},\varrho |X_{+},\mathbf{U}).\hfill \end{array}$$

(22b)

Proof. 

See Section 4.2. □

Assume that the infimum in (20) is attained, and let $\mathcal{P}$ denote the subset of laws $P_{XU}$ achieving the minimum. Suppose further that there exists a law $P_{XU}\in \mathcal{P}$ such that, for $(Y,X,U)\sim P_{XU}{\mathbb{G}}_{Y|X}^{\left(\varrho \right)}$, for any other $P_{X^{\prime }{U}^{\prime }}\in \mathcal{P}$ where $(Y^{\prime },X^{\prime },U^{\prime })\sim P_{X^{\prime }{U}^{\prime }}{\mathbb{G}}_{Y^{\prime }|X^{\prime }}^{\left(\varrho \right)}$

$$h\left(Y\right|U)-h\left(X\right|U)\le h\left(Y^{\prime }\right|U^{\prime })-h\left(X^{\prime }\right|U^{\prime }).$$

(23)

Denote the value of the LHS of (23) by $g^{*}\left(\varrho \right)$.

Lemma 2.

Fix $ϵ>0$. Let $P_{XU}$ be an admissible law such that

$$\begin{array}{ccc}\hfill {\mathsf{s}}_{\lambda }(X,\varrho |U)& \le & {\mathsf{V}}_{\lambda }\left(\varrho \right)+ϵ\hfill \end{array}$$

(24a)

$$\begin{array}{ccc}\hfill h\left(Y\right|U)-h(X\left|U\right)& \le & g^{*}\left(\varrho \right)+ϵ.\hfill \end{array}$$

(24b)

There exists a law $(Y^{\prime },X^{\prime },U^{\prime })\sim P_{X^{\prime }{U}^{\prime }}{\mathbb{G}}_{Y^{\prime }|X^{\prime }}^{\left(\varrho \right)}$ satisfying

$${\mathsf{s}}_{\lambda }(X^{\prime },\varrho |U^{\prime })\le {\mathsf{V}}_{\lambda }\left(\varrho \right)+2ϵ$$

(25)

and

$$\begin{array}{c}\hfill h\left(Y^{\prime }\right|U^{\prime })-h\left(X^{\prime }\right|U^{\prime })+\frac{1}{2}I(X_1+X_2;X_1-X_2|Y_1,Y_2,U_1,U_2)\le g^{*}\left(\varrho \right)+ϵ,\end{array}$$

(26)

where $(U_1,X_1,Y_1)$ and $(U_2,X_2,Y_2)$ denote independent copies of $(U,X,Y)$.

Proof. 

See Section 4.3. □

Inequality (26) combined with assumption (23) and Lemma 6 in ([9] Section VI.B) establish the following.

Lemma 3.

There exists a sequence $\{X_n,U_n\}$, such that for each $n\ge 1$, $(X_n,U_n)$ is feasible,

$$\underset{n\to \infty }{lim}{\mathsf{s}}_{\lambda }(X_n,\varrho |U_n)={\mathsf{V}}_{\lambda }\left(\varrho \right)$$

(27)

and there exists a feasible law $P_{X_{*}{U}_{*}}$ on $\mathbb{R}\times \mathcal{U}$ such that

$$(X_n,U_n)\stackrel{\mathcal{D}}{\to }(X_{*},U_{*})\sim P_{X_{*}{U}_{*}}$$

and with ${\mathbf{U}}_n\triangleq (U_{1,n},U_{2,n})$, ${\mathbf{Y}}_n\triangleq (Y_{1,n},Y_{2,n})$, where the tuple $(U_{1,n},X_{1,n},Y_{1,n})$ and the tuple $(U_{2,n},X_{2,n},Y_{2,n})$ are two independent copies of $(U_n,X_n,Y_n)\sim P_{U_n{X}_n}{\mathbb{G}}_{Y_n|X_n}^{\left(\varrho \right)}$, we have

$$\begin{array}{c}\hfill \underset{n\to \infty }{lim\; inf}I(X_{1,n}+X_{2,n};X_{1,n}-X_{2,n}|{\mathbf{Y}}_n,{\mathbf{U}}_n=\mathbf{u})=0\mathrm{for}{P}_{U_{*}}\times P_{U_{*}}-a.e\mathbf{u}.\end{array}$$

(28)

Finally, we apply to identity (28) the extended form of Bernstein’s theorem (see [10,11]), which is proved in ([9] Appendix A), to conclude that $X_{1,*}{|}_{\{U_{1,*}=u_1\}}$ (i.e., the conditional random variable: $X_{1,*}$ conditioned on $U_{1,*}=u_1$) and $X_{2,*}{|}_{\{U_{2,*}=u_2\}}$ are independent Gaussian random variables of equal variance.

Since the marginal of X is Gaussian, with no loss in generality, we may choose $(X,U)$ to be jointly Gaussian, which establishes our claim in Theorem 2. □

3.2. The Gaussian Setting with Decoder Side-Information Z

In the Gaussian setting where the decoder side-information Z in non-void, using our previous definition $Y=\sqrt{\rho }X+W$, let

$$Z=aX+bY+T$$

(29)

for a pair of real numbers $(a,b)$ where $T\sim \mathcal{N}(0,1)$ is independent of $(X,Y)$ and U. For arbitrary X with $\mathbb{E}\left[X^2\right]\le 1$, let us define by ${\tilde{\mathbb{G}}}_{YZ|X}^{(\varrho ,a,b)}$ the conditional law obtained by forming the pair $(Y,Z)$, when given X, via the additive independent Gaussian pair $(W,T)$ as described above.

The rates-distortion region $\mathcal{R}\left(D\right)$ is defined in Theorem 1 with

$$\mathbb{E}d(X,g(U,V,Z))=\mathbb{E}\left[{\left(X-g(U,V,Z)\right)}^2\right]\triangleq {\sigma }_{X|UVZ}^2\le D$$

(30)

and it is evaluated over a law of the form

$$P_{XUYZV}=P_{XU}{\tilde{\mathbb{G}}}_{YZ|X}^{(\varrho ,a,b)}{P}_{V|YU}.$$

(31)

Consequently, for the minimization of (20), we consider the functional

$$\begin{array}{c}\hfill {\mathsf{s}}_{\lambda }(X,\varrho |U)\triangleq -h\left(X\right|U)+\underset{\genfrac{}{}{0.0pt}{}{{V:X\leftrightarrow Y\leftrightarrow V|}_U}{{\sigma }_{X|UVZ}^2\le D}}{inf}\lambda I(Y;V|U).\end{array}$$

(32)

under a law of the form (31). Next, in Lemma 1, for $P_{XU}$, a law on $\mathcal{X}\times \mathcal{U}$ and $P_{UXYZ}=P_{XU}{\tilde{\mathbb{G}}}_{YZ|X}^{(\varrho ,a,b)}$ we let $(U_1,X_1,Y_1,Z_1)$ and $(U_2,X_2,Y_2,Z_2)$ be two independent copies of $(U,X,Y,Z)$.

Upon defining $(X_{-},X_{+})$ and $(Y_{-},Y_{+})$ as before, we also define

$$\begin{array}{c}\hfill Z_{+}\triangleq \frac{Z_1+Z_2}{\sqrt{2}},Z_{-}\triangleq \frac{Z_1-Z_2}{\sqrt{2}}.\end{array}$$

It can be verified that

$$\begin{array}{c}\hfill Z_{+}=a{X}_{+}+b{Y}_{+}+\frac{1}{\sqrt{2}}(T_1+T_2)\triangleq a{X}_{+}+b{Y}_{+}+T_{+}\\ \hfill Z_{-}=a{X}_{-}+b{Y}_{-}+\frac{1}{\sqrt{2}}(T_1-T_2)\triangleq a{X}_{-}+b{Y}_{-}+T_{-}\end{array}$$

(33)

where $T_{-}$ and $T_{+}$ are independent. Furthermore, the pair $(T_{-},T_{+})$ is independent of $(U,X_{-},X_{+},Y_{-},Y_{+})$ and is equal in distribution to the pair $(T_1,T_2)$. Thus, the simultaneous unitary transformation $(Y_1,Y_2)\mapsto (Y_{-},Y_{+})$ and $(Z_1,Z_2)\mapsto (Z_{-},Z_{+})$ preserves the Gaussian nature of the channel and factors according to

$$P_{U{X}_{-}{X}_{+}{Y}_{-}{Y}_{+}{Z}_{-}{Z}_{+}V}=P_{X_{-}{X}_{+}U}{\tilde{\mathbb{G}}}_{Y_{-}{Z}_{-}|X_{-}}^{(\varrho ,a,b)}{\tilde{\mathbb{G}}}_{Y_{+}{Z}_{+}|X_{+}}^{(\varrho ,a,b)}{P}_{V|U{Y}_{-}{Y}_{+}}.$$

(34)

Observe that with the choice of $U\triangleq \mathbf{U}=(U_1,U_2)$ and $V=(V_1,V_2)$ where $(U_1,X_1,Y_1,Z_1,V_1)$ and $(U_2,X_2,Y_2,Z_2,V_2)$ are two independent copies of $(\tilde{U},X,Y,Z,\tilde{V})$ such that $X\leftrightarrow Y\leftrightarrow \tilde{V}{|}_{\tilde{U}}$ and ${\sigma }_{X|\tilde{U}\tilde{V}Z}^2\le D$ –i.e., $P_{\tilde{U}XYZ\tilde{V}}$ is feasible for the minimization of (20), we have

$$\begin{array}{ccc}\hfill {\sigma }_{X_{+}|U{Y}_{-}{Z}_{+}V}^2& \le & {\sigma }_{X_{+}|UV{Z}_{+}}^2\le D\hfill \\ \hfill {\sigma }_{X_{-}|U{Y}_{+}{Z}_{-}V}^2& \le & {\sigma }_{X_{-}|UV{Z}_{-}}^2\le D.\hfill \end{array}$$

(35)

Thus, Lemma 1 holds for this setting with decoder side-information Z as well.

4. Proofs

4.1. Proof of Theorem 1

Converse: Assume that the pair $(R,R_{\mathrm{h}})$ is D-achievable. For $j=1,\dots ,n$ with the convention $Z^0\triangleq \varnothing$, define the RV’s

$$\begin{array}{ccc}\hfill T& \triangleq & f^{\left(n\right)}\left(X^n\right)\hfill \end{array}$$

(36a)

$$\begin{array}{ccc}\hfill V_j& \triangleq & f_{1,j}^{\left(n\right)}(f^{\left(n\right)}\left(X^n\right),Y^j),j=1,\dots ,n\hfill \end{array}$$

(36b)

$$\begin{array}{ccc}\hfill U_j& \triangleq & (T,X^{j-1},Y^{j-1},Z^{j-1}),j=1,\dots ,n.\hfill \end{array}$$

(36c)

The rate R is lower bounded as follows

$$\begin{array}{ccc}\hfill nR& \ge & log{M}^{\left(n\right)}\ge H\left(T\right)\hfill \\ & =& I(X^n;T)=\sum _{k=1}^{n}I(X_k;T|X^{k-1})\hfill \\ & \stackrel{\left(a\right)}{=}& \sum _{k=1}^{n}I(X_k;T{X}^{k-1})\hfill \\ & \stackrel{\left(b\right)}{=}& \sum _{k=1}^{n}I(X_k;T{X}^{k-1}{Y}^{k-1}{Z}^{k-1})\hfill \\ & =& \sum _{k=1}^{n}I(X_k;U_k).\hfill \end{array}$$

(37)

Here, $\left(a\right)$ follows since $X^n$ is memoryless, and $\left(b\right)$ follows since $X_k\leftrightarrow (T,X^{k-1})\leftrightarrow (Y^{k-1},Z^{k-1})$ forms a Markov chain.

We may now lower bound $R_{\mathrm{h}}$ as follows

$$\begin{array}{ccc}\hfill n{R}_{\mathrm{h}}& \ge & \sum _{k=1}^{n}log{L}_k^{\left(n\right)}\ge H(V_1,V_2,\dots ,V_n)\hfill \\ & \ge & H(V_1,V_2,\dots ,V_n|T)\hfill \\ & =& \sum _{k=1}^{n}H\left(V_k\right|T{V}^{k-1})\hfill \\ & \ge & \sum _{k=1}^{n}H\left(V_k\right|T{V}^{k-1}{X}^{k-1}{Y}^{k-1}{Z}^{k-1})\hfill \\ & \stackrel{\left(c\right)}{=}& \sum _{k=1}^{n}H\left(V_k\right|T{X}^{k-1}{Y}^{k-1}{Z}^{k-1})\hfill \\ & \stackrel{\left(c\right)}{=}& \sum _{k=1}^{n}I(Y_k;V_k|T{X}^{k-1}{Y}^{k-1}{Z}^{k-1})\hfill \\ & =& \sum _{k=1}^{n}I(Y_k;V_k|U_k),\hfill \end{array}$$

(38)

where $\left(c\right)$ follows since $V_j$ is a deterministic function of $(T,Y^j)$.

The sum-rate can be lower bounded as follows

$$\begin{array}{ccc}\hfill n(R+R_{\mathrm{h}})& \ge & log{M}^{\left(n\right)}+\sum _{k=1}^{n}log{L}_k^{\left(n\right)}\hfill \\ & \ge & H\left(T\right)+H(V_1,\dots ,V_n)\hfill \\ & \ge & H\left(T\right)+H(V_1,\dots ,V_n|T)\hfill \\ & =& I(T;X^n)+H(V_1,\dots ,V_n|T)\hfill \\ & =& \sum _{k=1}^{n}I(X_k;T|X^{k-1})+H(V_1,\dots ,V_n|T)\hfill \\ & \stackrel{\left(d\right)}{\ge }& \sum _{k=1}^n\left[I(X_k;U_k)+I(Y_k;V_k|U_k)\right],\hfill \end{array}$$

(39)

where $\left(d\right)$ follows by equality (37) and inequality (38).

Draw J uniformly from $\{1,\dots ,n\}$ independently of ${\left\{(X_k,Y_k,Z_k,V_k,U_k)\right\}}_{k=1}^n$, and define the RV’s $U=(U_J,J)$, $V=V_J$, $Z=Z_J$, $Y=Y_J$, and $X=X_J$. Using J, we may express (37) as follows

$$\begin{array}{ccc}\hfill R& \ge & \frac{1}{n}\sum _{k=1}^{n}I(X_k;U_k)=I(X_J;U_J|J)\hfill \\ & =& I(X_J;U_J,J)-I(X_J;J)\hfill \\ & =& I(X_J;U_J,J)=I(X;U),\hfill \end{array}$$

(40)

and we may express (38) as follows

$$\begin{array}{ccc}\hfill R_{\mathrm{h}}& \ge & \frac{1}{n}\sum _{k=1}^{n}I(Y_k;V_k|U_k)\hfill \\ & =& I(Y_J;V_J|U_J,J)=I(Y;V|U)=H\left(V\right|U).\hfill \end{array}$$

(41)

With regard to the expected distortion, we may write

$$\begin{array}{ccc}\hfill D& \ge & \frac{1}{n}\sum _{i=1}^n\mathbb{E}\left[d(X_i,{\widehat{X}}_i)\right]\hfill \\ & =& \frac{1}{n}\sum _{i=1}^n\mathbb{E}\left[d(X_i,g_i^{\left(n\right)}(T,V_1,\dots ,V_i,Z^i))\right]\hfill \\ & \stackrel{\left(e\right)}{\ge }& \frac{1}{n}\sum _{i=1}^n\mathbb{E}\left[d(X_i,g_i^{*}(T,X^{i-1},Y^{i-1},Z^{i-1},V_i,Z_i))\right]\hfill \\ & =& \frac{1}{n}\sum _{i=1}^n\mathbb{E}\left[d(X_i,g_i^{*}(U_i,V_i,Z_i))\right]\hfill \\ & =& \mathbb{E}\left[d(X_J,g(U_J,J,V_J,Z_J))\right]\hfill \\ & =& \mathbb{E}\left[d\right(X,g(U,V,Z)\left)\right].\hfill \end{array}$$

(42)

Step $\left(e\right)$ is justified as follows: Since $V_1,\dots ,V_{i-1}$ are deterministic functions of $(T,Y^{i-1})$

$$(X^{i-1},Z^{i-1},Z_i)\leftrightarrow (T,Y^{i-1})\leftrightarrow (V_1,\dots ,V_{i-1})$$

is a Markov chain and, given $(U_k,V_k,Z_k)$, the tuple $(T,V_1,\dots ,V_k,Z^k)$ is independent of $(V_1,\dots ,V_{k-1})$. As a consequence of that, Lemma 1 in ([12] Section II.B) guarantees the existence of a reconstruction ${\widehat{X}}_k^{*}(U_k,V_k,Z_k)$ which dominates ${\widehat{X}}_k$ in the sense that

$$\begin{array}{c}\hfill \mathbb{E}\left[d(X_k,{\widehat{X}}_k^{*}(U_k,V_k,Z_k))\right]\le \mathbb{E}\left[d(X_k,{\widehat{X}}_k(T,V_1,\dots ,V_k,Z_k,Z^{k-1}))\right].\end{array}$$

(43)

This observation interpreted as the “data processing inequality” for estimation has already been made in ([12] Lemma 1).

Furthermore,

$$\begin{array}{ccc}\hfill V_J& =& f_{1,J}^{\left(n\right)}(f^{\left(n\right)}\left(X^n\right),Y^J)=f_{1,J}^{\left(n\right)}(f^{\left(n\right)}\left(X^n\right),Y^{J-1},Y_J)\hfill \\ & =& {\tilde{f}}_{1,J}^{\left(n\right)}(T,X^{J-1},Y^{J-1},Z^{J-1},Y_J)={\tilde{f}}_{1,J}^{\left(n\right)}(U_J,J,Y_j).\hfill \end{array}$$

(44)

By (1) and the memoryless property of the sequence $(X_k,Y_k,Z_k),k=1,\dots ,n$ one can verify the Markov relation $U_k\leftrightarrow X_k\leftrightarrow (Y_k,Z_k)$ which implies the Markov relation $U\leftrightarrow X\leftrightarrow (Y,Z)$. Similarly, the definitions of $U_k$ and $V_k$ and the memoryless property of $(X_k,Y_k),k=1,\dots ,n$ imply that, conditioned on $U_k$, $X_k\leftrightarrow Y_k\leftrightarrow V_k$ forms a Markov chain.

Thus, conditioned on U, $X\leftrightarrow Y\leftrightarrow V$ forms a Markov chain, hence

$$P_{XUYV}=P_{XU}{\mathbb{P}}_{Y|X}{P}_{V|YU},$$

(45)

where ${\mathbb{P}}_{Y|X}$ denotes the conditional law induced by the marginal law $P_{XY}$. The combination of (40)–(42) and (44) together with the latter Markov relations establish the converse.

We shall now obtain an alternative characterization for the lower bound (38). For a law

$$P_{UXZYV}=P_U{P}_{X|U}{P}_{ZY|X}{P}_{V|YU},$$

and its induced conditional law $P_{XZYV|U}$, let

$$\begin{array}{c}\hfill Q(P_{XZYV|U},\tilde{D})\triangleq \underset{g^{*}:\mathcal{U}\times \mathcal{V}\times \mathcal{Z}\to \widehat{\mathcal{X}}:\mathbb{E}\left[d(X,g^{*}(U,V,Z))\right|U=u]\le \tilde{D}}{inf}H\left(V\right|U=u),\end{array}$$

and let $\overline{Q}(P_{XZYV|U},·)$ denote the lower convex envelope of $Q(P_{XZYV|U},·)$.

Define

$$\begin{array}{c}\hfill Q_s(P_{UXZYV},D)\triangleq \underset{\rho \left(u\right):\int \rho \left(u\right)d{P}_U\left(u\right)\le D}{inf}{\int }_{\mathcal{U}}\overline{Q}(P_{XZYV|U},\rho \left(u\right))d{P}_U\left(u\right),\end{array}$$

then by ([13] Section III.C, Lemma 1) $Q_s(P_{UXZYV},·)$ is convex.

Note that

$$\begin{array}{ccc}\hfill H\left(V_k\right|T,X^{k-1},Y^{k-1},Z^{k-1})& =& H\left(f_{1,k}^{\left(n\right)}(T,Y^{k-1},Y_k)|U_k\right)\hfill \\ & =& \int H\left(f_{1,k}^{\left(n\right)}(t,y^{k-1},Y_k)|U_k=u_k\right)d\mu \left(u_k\right).\hfill \end{array}$$

(46)

The integrand on the RHS of (46) is the entropy of the scalar quantizer of $V_k\triangleq f_{1,k}^{\left(n\right)}(T,Y^{k-1},Y_k)$ conditioned on $U_k=u_k$ where $U_k$ is defined in (36c). Now, conditioned on $U_k=u_k$, Lemma 1 in ([12] Section II.B) ensures that

$$\begin{array}{c}\hfill \mathbb{E}\left[d(X_k,{\widehat{X}}_k(t,y^{k-1},f_{1,k}^{\left(n\right)}(t,y^{k-1},Y_k),Z^k))|U_k=u_k\right]\\ \hfill \ge \mathbb{E}\left[d(X_k,g_k^{*}(U_k,V_k,Z_k))|U_k=u_k\right].\end{array}$$

Consequently, we may lower bound the RHS of (46) as follows

$$\begin{array}{c}\hfill \int H\left(f_{1,k}^{\left(n\right)}(t,y^{k-1},Y_k)|U_k=u_k\right)d\mu \left(u_k\right)\stackrel{\left(a\right)}{\ge }\int \overline{Q}(P_{XZYV|U},\mathbb{E}\left[d(X_k,{\widehat{X}}_k)\right|U_k=u_k])d\mu \left(u_k\right)\\ \hfill \ge \int \left[\int \overline{Q}(P_{XZYV|U},\mathbb{E}\left[d(X_k,g(u,V,z))\right|U_k=u])d\mu \left(z\right|u)\right]d\mu \left(u\right)\\ \hfill \stackrel{\left(b\right)}{\ge }\int \overline{Q}(P_{XZYV|U},\mathbb{E}\left[d(X_k,g(u,V,Z))\right|U_k=u])d\mu \left(u\right)\\ \hfill \ge Q_s(P_{UXZYV},D),\end{array}$$

(47)

where in $\left(a\right)$ we used the definition of Q and in $\left(b\right)$ its convexity. The lower bound (47) may be interpreted as follows. Fix $\rho :\mathcal{U}\to {\mathbb{R}}^{+}$ and consider, for each $u\in \mathcal{U}$, time shraing of at most two scalar quantizers for the “source” $P_{V|U=u}$ attaining a distortion level $\rho \left(u\right)$. The optimal helper time shares side-information-dependent scalar quantizers of $V_k$ (at most two per each side-information symbol $U_k$), while the reconstruction at the decoder is a function of $(U,V,Z)$.

Direct: To establish the achievability of ${\mathcal{R}}^{*}\left(D\right)$, consider the codebook construction as follows. The codebook $\mathcal{A}=\{{\mathit{u}}_1,\dots ,{\mathit{u}}_M\},{\mathit{u}}_k\in {\mathcal{U}}^n$ is obtained by drawing the n-length sequences ${\mathit{u}}_k$ independently of $T_{P_U}^{\delta }$. (For the definition of $T_{P_U}^{\delta }$—the set of $\delta$-strongly typical n-sequences corresponding to a marginal law $P_U$ and a few properties of these sequences see [14,15,16]).

Given the source sequence x, $f^{\left(n\right)}\left(x\right)$ is defined as follows.

• If $\mathit{x}\in T_{P_X}^{\delta }$ the encoder searches for the first sequence ${\mathit{U}}_k=\mathit{u}$ in $\mathcal{A}$ such that (s.t.) $(\mathit{x},\mathit{u})\in T_{P_{XU}}^{2\delta }$ and sets $f^{\left(n\right)}\left(\mathit{x}\right)=k$.
• If $\mathit{x}\notin T_{P_X}^{\delta }$, or $\nexists {\mathit{U}}_k\in \mathcal{A}$ s.t. $(\mathit{x},\mathit{u})\in T_{P_{XU}}^{2\delta }$, an encoding error is declared.

Given $f^{\left(n\right)}\left(\mathit{x}\right)=k$, the helper forms the sequence of descriptions $V_i={\tilde{f}}_1(U_{k,i},Z_i),V_i\in [1,\dots ,L_i]$ that is sent causally to the decoder.

Decoding: Given $f^{\left(n\right)}\left(\mathit{x}\right)=k$ as well as the sub-sequence $V_1,\dots ,V_i$, the decoder forms the reconstruction sequence ${\widehat{X}}_i=g(U_{k,i},V_i,Z_i),i\in [1,\dots ,n]$.

Given that $(\mathit{U},\mathit{X})$ are jointly typical since $(\mathit{X},\mathit{Y},\mathit{Z})$ is memoryless, the Markov lemma guarantees that, for large n, with high probability $(\mathit{U},\mathit{X},\mathit{Y},\mathit{Z})$ are also jointly typical. Since $\mathit{X}\leftrightarrow (\mathit{U},\mathit{Y})\leftrightarrow \mathit{V}$ forms a Markov chain, by the Markov lemma, with high probability, $(\mathit{X},\mathit{V})$ as well as $(\mathit{X},\mathit{V},\mathit{Z})$ are jointly typical. Thus, with high probability, $(\mathit{X},\widehat{\mathit{X}})$ are jointly typical, hence the distortion constraint (10) is fulfilled for large n. That the sequence $V_1,\dots ,V_n$ can be described at a conditional entropy rate satisfying (12b)–(47) can be established along similar lines as the proof of the direct part of Theorem 2 in ([13] Section III.C). Finally, standard error probability analysis verifies that, with high probability, $(\mathit{U},\mathit{X})$ are jointly typical as long as (12a) holds.

4.2. Proof of Lemma 1

If $Y_i=\sqrt{\rho }{X}_i+W_i,i=1,2$, then

$$\begin{array}{ccc}\hfill Y_{+}& =& \frac{\sqrt{\rho }}{\sqrt{2}}(X_1+X_2)+\frac{1}{\sqrt{2}}(W_1+W_2)\triangleq \sqrt{\rho }{X}_{+}+W_{+}\hfill \end{array}$$

(48a)

$$\begin{array}{ccc}\hfill Y_{-}& =& \frac{\sqrt{\rho }}{\sqrt{2}}(X_1-X_2)+\frac{1}{\sqrt{2}}(W_1-W_2)\triangleq \sqrt{\rho }{X}_{-}+W_{-}\hfill \end{array}$$

(48b)

where $W_{-}$ and $W_{+}$ are independent and the pair $(W_{-},W_{+})$ is equal in distribution to the pair $(W_1,W_2)$. Thus, the unitary transformation $(Y_1,Y_2)\mapsto (Y_{-},Y_{+})$ preserves the Gaussian nature of the channel and factors according to (see (53) ahead)

$$P_{U{X}_{-}{X}_{+}{Y}_{-}{Y}_{+}V}=P_{X_{-}{X}_{+}U}{\mathbb{G}}_{Y_{-}|X_{-}}^{\left(\varrho \right)}{\mathbb{G}}_{Y_{+}|X_{+}}^{\left(\varrho \right)}{P}_{V|U{Y}_{-}{Y}_{+}}.$$

(49)

To show (22a), consider the sequence of identities

$$\begin{array}{c}\hfill \lambda I(Y_{+}{Y}_{-};V|U)-h\left(X_{+}{X}_{-}\right|U)\\ \hfill =\lambda I(Y_{+};V|U)+\lambda I(Y_{-};V|U{Y}_{+})-h\left(X_{-}\right|U{Y}_{+})-h\left(X_{+}\right|U{X}_{-})-I(X_{-};Y_{+}|U)\\ \hfill =\lambda I(Y_{+};V|U{X}_{-})+\lambda I(Y_{-};V|U{Y}_{+})-h\left(X_{+}\right|U{X}_{-})-h\left(X_{-}\right|U{Y}_{+})\\ \hfill +(\lambda -1)\left[I(Y_{+};X_{-}|U)-I(Y_{+};X_{-}|UV)\right]-I(Y_{+};X_{-}|UV).\end{array}$$

(50)

Moreover, to show (22b), consider the sequence of identities

$$\begin{array}{c}\hfill \lambda I(Y_{+}{Y}_{-};V|U)-h\left(X_{+}{X}_{-}\right|U)\\ \hfill =\lambda I(Y_{-};V|U)+\lambda I(Y_{+};V|U{Y}_{-})-h\left(X_{+}\right|U{Y}_{-})-h\left(X_{-}\right|U{X}_{+})-I(X_{+};Y_{-}|U)\\ \hfill =\lambda I(Y_{-};V|U{X}_{+})+\lambda I(Y_{+};V|U{Y}_{-})-h\left(X_{+}\right|U{Y}_{-})-h\left(X_{-}\right|U{X}_{+})\\ \hfill +(\lambda -1)\left[I(Y_{-};X_{+}|U)-I(Y_{-};X_{+}|UV)\right]-I(Y_{-};X_{+}|UV).\end{array}$$

(51)

Starting with (50), consider the difference

$$I(Y_{+};X_{-}|U)-I(Y_{+};X_{-}|UV)=I(X_{-};V|U)-I(X_{-};V|U{Y}_{+})$$

(52)

under a law of the form

$$P_{U{X}_{-}{X}_{+}{Y}_{-}{Y}_{+}V}=P_{X_{-}{X}_{+}U}{P}_{Y_{-}|X_{-}}{P}_{Y_{+}|X_{+}}{P}_{V|U{Y}_{-}{Y}_{+}}$$

(53)

i.e., that $X_{-}\leftrightarrow Y_{-}{\leftrightarrow V|}_{(U,Y_{+})}$ forms a Markov chain (see also ([9] Section VI.A, Lemma 4)).

We distinguish between the two cases:

(1)

In case that $I(X_{-};Y_{+}|UV)=0$, the non-negativity of mutual information implies that the expression (52) is non-negative, hence the inequalities (63) ahead hold for any $\lambda >1$.

(2)

In case that $I(X_{-};Y_{+}|UV)>0$, we prove first that for the set of laws which are feasible for the optimization problem (20), the expression (52) is strictly positive.

Observe that with the choice of $U\triangleq \mathbf{U}=(U_1,U_2)$ and $V=(V_1,V_2)$ where $(U_1,X_1,Y_1,V_1)$ and $(U_2,X_2,Y_2,V_2)$ are two independent copies of $(\tilde{U},X,Y,\tilde{V})$ such that $X\leftrightarrow Y\leftrightarrow \tilde{V}{|}_{\tilde{U}}$ and ${\sigma }_{X|\tilde{U}\tilde{V}}^2\le D$ i.e., $P_{\tilde{U}XY\tilde{V}}\in \mathcal{Q}$, we have

$$\begin{array}{ccc}\hfill {\sigma }_{X_{+}|U{Y}_{-}V}^2& \le & {\sigma }_{X_{+}|UV}^2=\frac{1}{2}{\sigma }_{X_1|U_1{V}_1}^2+\frac{1}{2}{\sigma }_{X_2|U_2{V}_2}^2\le D\hfill \\ \hfill {\sigma }_{X_{-}|U{Y}_{+}V}^2& \le & {\sigma }_{X_{-}|UV}^2=\frac{1}{2}{\sigma }_{X_1|U_1{V}_1}^2+\frac{1}{2}{\sigma }_{X_2|U_2{V}_2}^2\le D.\hfill \end{array}$$

(54)

Thus, the unitary transformation $(Y_1,Y_2)\mapsto (Y_{-},Y_{+})$ picks a pair of independent copies of a law $P_{\tilde{U}XY\tilde{V}}\in \mathcal{Q}$ and “creates” a pair of laws, $X_{-}\leftrightarrow Y_{-}{\leftrightarrow V|}_{U{Y}_{+}}$ and $X_{+}\leftrightarrow Y_{+}{\leftrightarrow V|}_{U{Y}_{-}}$, which factor jointly according to (53) with

$$\begin{array}{ccc}\hfill P_{Y_{-}|X_{-}}& =& {\mathbb{G}}_{Y_{-}|X_{-}}^{\left(\varrho \right)}\hfill \\ \hfill P_{Y_{+}|X_{+}}& =& {\mathbb{G}}_{Y_{+}|X_{+}}^{\left(\varrho \right)},\hfill \end{array}$$

hence are symmetric w.r.t. the inputs $X_{-}$ and $X_{+}$. We shall define the latter set of laws by ${\mathcal{P}}^{*}$ and, as shown above, ${\mathcal{P}}^{*}\subseteq \mathcal{Q}$.

Remark 2.

Suppose that $(\tilde{U},X,Y)\sim P_{X\tilde{U}}{\mathbb{G}}_{Y|X}^{\left(\varrho \right)}$ with $\tilde{U}\in \mathcal{U}$ then, for both laws $X_{-}\leftrightarrow Y_{-}{\leftrightarrow V|}_{U{Y}_{+}}$ and $X_{+}\leftrightarrow Y_{+}{\leftrightarrow V|}_{U{Y}_{-}}$, we have $(U,Y_{+})\in \mathcal{U}\times \mathcal{U}\times \mathbb{R}$ and $(U,Y_{-})\in \mathcal{U}\times \mathcal{U}\times \mathbb{R}$. Since the image of the map

$$P_{X\tilde{U}}\mapsto (\mathbb{E}\left[X^2\right],{\sigma }_{X|\tilde{U}V}^2,{\mathsf{s}}_{\lambda }(X,\varrho |\tilde{U}))$$

(55)

is a convex set, standard dimensionality reduction argument can be used to establish the existence of a law $P_{XU}$ where $P_U$ is supported on a finite set and achieves any point in the image of the map (55) (See ([9] Section IV, Remark 2)).

By rate-distortion theory, the constraint ${\sigma }_{X_{-}|U,V}^2\le D$ with both U and V non-void implies that

$$\begin{array}{c}\hfill I(X_{-};U)<I(X_{-};UV)\Rightarrow I(X_{-};V|U)>0.\end{array}$$

(56)

Now, for any $P_{U{X}_{-}{X}_{+}{Y}_{-}{Y}_{+}V}$ as per (49), conditioned on $(U,Y_{+})$, the random variable V is dependent on $Y_{-}$ i.e. $I(Y_{-};V|U{Y}_{+})>0$. As a consequence of that, since the mutual information $I(X_{-};Y_{-})$ is strictly positive and, conditioned on $(U,Y_{+})$, $X_{-}\leftrightarrow Y_{-}\leftrightarrow V$ forms a Markov chain

$$I(X_{-};V|U{Y}_{+})>0.$$

(57)

Since a law of the form (53) dictates

$$I(X_{-};V|U{Y}_{-}{Y}_{+})=0,$$

(58)

the combination of (56)–(58) yields that

$$I(X_{-};V|U)\ge I(X_{-};V|U{Y}_{+})>I(X_{-};V|U{Y}_{-}{Y}_{+})=0.$$

Thus, the conditional mutual information $I(X_{-};V|U)$ is non-increasing under the conditioning on $(Y_{-},Y_{+})$.

By the symmetry of the pair of laws $X_{-}\leftrightarrow Y_{-}{\leftrightarrow V|}_{U{Y}_{+}}$ and $X_{+}\leftrightarrow Y_{+}{\leftrightarrow V|}_{U{Y}_{-}}$ induced by $P_{U{X}_{-}{X}_{+}{Y}_{-}{Y}_{+}V}$, conditioned on $(U,Y_{-})$, the random variable V is dependent on $Y_{+}$, hence

$$I(Y_{+};V|U{Y}_{-})>0.$$

(59)

Now

$$\begin{array}{ccc}\hfill P_{U{X}_{-}{X}_{+}{Y}_{-}{Y}_{+}V}& =& P_{X_{-}{X}_{+}U}{P}_{Y_{-}|X_{-}}{P}_{Y_{+}|X_{+}}{P}_{V|U{Y}_{-}{Y}_{+}}\hfill \\ & =& P_{X_{-}{X}_{+}U}{P}_{Y_{+}|X_{+}}{P}_{V{Y}_{-}|U{X}_{-}{Y}_{+}},\hfill \end{array}$$

(60)

hence

$$\begin{array}{ccc}\hfill P_{U{X}_{-}{X}_{+}{Y}_{+}V}& =& \sum _{Y_{-}}{P}_{X_{-}{X}_{+}U{Y}_{-}{Y}_{+}V}\hfill \\ & =& \sum _{Y_{-}}{P}_{X_{-}{X}_{+}U}{P}_{Y_{+}|X_{+}}{P}_{V{Y}_{-}|U{X}_{-}{Y}_{+}}\hfill \\ & =& P_{X_{-}{X}_{+}U}{P}_{Y_{+}|X_{+}}{P}_{V|U{X}_{-}{Y}_{+}}.\hfill \end{array}$$

(61)

However, with the latter factorization, if indeed $I(X_{-};V|U)=I(X_{-};V|U{Y}_{+})$, i.e.,

$$I(X_{-};V|U)=I(X_{-};V|U{Y}_{+})>I(X_{-};V|U{Y}_{-}{Y}_{+})=0$$

then, since the conditional mutual information $I(X_{-};V|U)$ is non-increasing under the conditioning on $(Y_{-},Y_{+})$, it follows that $P_{V|U{X}_{-}{Y}_{+}}=P_{V|U{X}_{-}}$, which is in contradiction with (59). Consequently, for $P_{U{X}_{-}{X}_{+}{Y}_{-}{Y}_{+}V}\in {\mathcal{P}}^{*}$, $I(X_{-};V|U)-I(X_{-};V|U{Y}_{+})=\Delta >0$, thus establishing that the expression (52) is positive.

Consequently, there exists some ${\lambda }^{*}>1$, such that for any $\lambda \ge {\lambda }^{*}\triangleq \frac{I(Y_{+};X_{-}|UV)}{\Delta }+1$

$$(\lambda -1)\left[I(Y_{+};X_{-}|U)-I(Y_{+};X_{-}|UV)\right]-I(Y_{+};X_{-}|UV)>0.$$

(62)

The combination of (50) and (62) implies that, for any $\lambda \ge {\lambda }^{*}$,

$$\begin{array}{c}\hfill \lambda I(Y_{+}{Y}_{-};V|\mathbf{U})-h\left(X_{+}{X}_{-}\right|\mathbf{U})\\ \hfill \stackrel{\left(a\right)}{\ge }-[h\left(X_{+}\right|\mathbf{U}{X}_{-})+h\left(X_{-}\right|\mathbf{U}{Y}_{+})]+\lambda [I(Y_{+};V|\mathbf{U}{X}_{-})+I(Y_{-};V|\mathbf{U}{Y}_{+})]\\ \hfill \stackrel{\left(b\right)}{\ge }{\mathsf{s}}_{\lambda }(X_{+},\varrho |X_{-},\mathbf{U})+{\mathsf{s}}_{\lambda }(X_{-},\varrho |Y_{+},\mathbf{U}).\end{array}$$

(63)

Here, $\left(a\right)$ follows by (50) and (62), and $\left(b\right)$ is true, since the set of laws over which the infimum on the RHS of (63) is evaluated is not empty. Indeed, the choice of $\mathbf{U}=(U_1,U_2)$ and $V=(V_1,V_2)$ where $(U_1,X_1,Y_1,V_1)$ and $(U_2,X_2,Y_2,V_2)$ are two independent copies of $(\tilde{U},X,Y,\tilde{V})$ such that $X\leftrightarrow Y\leftrightarrow \tilde{V}{|}_{\tilde{U}}$ and ${\sigma }_{X|\tilde{U}\tilde{V}}^2\le D$ –i.e., $P_{\tilde{U}XY\tilde{V}}\in \mathcal{Q}$ (hence it is feasible for ${\mathsf{s}}_{\lambda }(X,\varrho |U)$ in (19)), satisfies (54), hence it belongs to the feasible set.

On the other hand, since both mappings $(X_1,X_2)\mapsto (X_{+},X_{-})$ and $(Y_1,Y_2)\mapsto (Y_{+},Y_{-})$ are invertible, then with $\mathbf{U}=(U_1,U_2)$

$$\begin{array}{ccc}\hfill \lambda I(Y_{+}{Y}_{-};V|\mathbf{U})-h\left(X_{+}{X}_{-}\right|\mathbf{U})& =& \lambda I(Y_1{Y}_2;V|\mathbf{U})-h\left(X_1{X}_2\right|\mathbf{U})\hfill \\ & =& \lambda I(Y_1;V|\mathbf{U})+\lambda I(Y_2;V|\mathbf{U}{Y}_1)-h\left(X_1\right|\mathbf{U})-h\left(X_2\right|\mathbf{U}{X}_1)\hfill \\ & \stackrel{\left(a\right)}{=}& \lambda I(Y_1;V|\mathbf{U})+\lambda I(Y_2;V|\mathbf{U}{Y}_1)-h\left(X_1\right|\mathbf{U})-h\left(X_2\right|\mathbf{U})\hfill \\ & =& \lambda I(Y_1;V|\mathbf{U})+\lambda I(Y_2;V|\mathbf{U})-h\left(X_1\right|\mathbf{U})-h\left(X_2\right|\mathbf{U})\hfill \\ & & -\lambda [I(Y_2;V|\mathbf{U})-I(Y_2;V|\mathbf{U}{Y}_1)]\hfill \\ & =& \lambda I(Y_1;V|\mathbf{U})+\lambda I(Y_2;V|\mathbf{U})-h\left(X_1\right|\mathbf{U})-h\left(X_2\right|\mathbf{U})\hfill \\ & & -\lambda [I(Y_2;V|\mathbf{U})-h\left(Y_2\right|\mathbf{U}{Y}_1)+h(Y_2;|\mathbf{U}V{Y}_1)]\hfill \\ & \stackrel{\left(b\right)}{=}& \lambda I(Y_1;V|\mathbf{U})+\lambda I(Y_2;V|\mathbf{U})-h\left(X_1\right|\mathbf{U})-h\left(X_2\right|\mathbf{U})\hfill \\ & & -\lambda [I(Y_2;V|\mathbf{U})-h\left(Y_2\right|\mathbf{U})+h(Y_2;|\mathbf{U}V{Y}_1)]\hfill \\ & =& \lambda I(Y_1;V|\mathbf{U})+\lambda I(Y_2;V|\mathbf{U})-h\left(X_1\right|\mathbf{U})-h\left(X_2\right|\mathbf{U})\hfill \\ & & +\lambda I(Y_2;Y_1|\mathbf{U}V).\hfill \end{array}$$

(64)

Here, $\left(a\right)$ follows since, conditioned on $\mathbf{U}$, $X_1$ and $X_2$ are independent, and $\left(b\right)$ follows since, conditioned on $\mathbf{U}$, $Y_1$ and $Y_2$ are independent.

When $\mathbf{U}=(U_1,U_2)$ and $V=(V_1,V_2)$ where $(U_1,X_1,Y_1,V_1)$ and $(U_2,X_2,Y_2,V_2)$ are two independent copies of $(\tilde{U},X,Y,\tilde{V})$, such that $X\leftrightarrow Y\leftrightarrow \tilde{V}{|}_{\tilde{U}}$ and ${\sigma }_{X|\tilde{U}\tilde{V}}^2\le D$, then $Y_2\leftrightarrow (\mathbf{U},V)\leftrightarrow Y_1$ forms a Markov chain. Thus, $I(Y_1;Y_2|\mathbf{U}V)=0$ and the RHS of (64) becomes $\lambda I(Y_1;V_1|U_1)+\lambda I(Y_2;V_2|U_2)-h\left(X_1\right|U_1)-h\left(X_2\right|U_2)$ thus establishing that,

$$\begin{array}{c}\hfill \underset{\genfrac{}{}{0.0pt}{}{{V:X\leftrightarrow Y\leftrightarrow V|}_{\mathbf{U}}}{{\sigma }_{X|\mathbf{U}V}^2\le D}}{inf}\left\{-h(X_1,X_2|\mathbf{U})+\lambda I(Y_1,Y_2;V|\mathbf{U})\right\}\\ \hfill \le \sum _{i=1}^2\underset{\genfrac{}{}{0.0pt}{}{V:X_i\leftrightarrow Y_i{\leftrightarrow V|}_{U_i}}{{\sigma }_{X_i|U_{i}V}^2\le D}}{inf}\left\{-h\left(X_i\right|U_i)+\lambda I(Y_i;V|U_i)\right\}=2{\mathsf{s}}_{\lambda }(X,\varrho |U),\end{array}$$

(65)

where the inequality follows, since the infimum on the LHS is taken over a larger set than that on the RHS. The combination of (63) and (65) establishes (22a) for $\lambda \ge {\lambda }^{*}$.

Now, returning to (51), consider the difference

$$I(Y_{-};X_{+}|U)-I(Y_{-};X_{+}|UV)=I(X_{+};V|U)-I(X_{+};V|U{Y}_{-})$$

(66)

under the law (53) i.e., that $X_{+}\leftrightarrow Y_{+}{\leftrightarrow V|}_{(U,Y_{-})}$ forms a Markov chain. By rate-distortion theory, the constraint ${\sigma }_{X_{+}|U,V}^2\le D$ with both U and V non-void implies that

$$\begin{array}{c}\hfill I(X_{+};U)<I(X_{+};UV)\Rightarrow I(X_{+};V|U)>0.\end{array}$$

(67)

Moreover, an argument similar to that leading to the conclusion that (52) is non-negative establishes that (66) is non-negative.

Consequently, there exist some ${\lambda }^{*}>1$ such that for any $\lambda \ge {\lambda }^{*}$

$$(\lambda -1)\left[I(Y_{-};X_{+}|U)-I(Y_{-};X_{+}|UV)\right]-I(Y_{-};X_{+}|UV)>0.$$

(68)

In addition, the combination of (51) and (68) implies that, for any $\lambda \ge {\lambda }^{*}$,

$$\begin{array}{c}\hfill \lambda I(Y_{+}{Y}_{-};V|\mathbf{U})-h\left(X_{+}{X}_{-}\right|\mathbf{U})\ge {\mathsf{s}}_{\lambda }(X_{-},\varrho |X_{+},\mathbf{U})+{\mathsf{s}}_{\lambda }(X_{+},\varrho |Y_{-},\mathbf{U}).\end{array}$$

(69)

The combination of (69) and (65) establishes (22b) for $\lambda \ge {\lambda }^{*}$.

4.3. Proof of Lemma 2

Consider the tuple $(X_{+},X_{-},Y_{+},Y_{-},\mathbf{U})$ as defined in Lemma 1 that is constructed from independent copies of $(U,X,Y)\sim P_{XU}{\mathbb{G}}_{Y|X}^{\left(\varrho \right)}$. With the transformation (21) $X_{+}$ and $X_{-}$ preserve the variance of $X_i,i=1,2$ and $W_{+}$ and $W_{-}$ preserve the variance of $W_i,i=1,2$. Furthermore, as shown in the proof of Lemma 1, the unitary transformation $(Y_1,Y_2)\mapsto (Y_{-},Y_{+})$ picks a pair of independent copies of a law $P_{UXYV}\in \mathcal{Q}$ and “creates” a pair of laws, $X_{-}\leftrightarrow Y_{-}{\leftrightarrow V|}_{U{Y}_{+}}$ and $X_{+}\leftrightarrow Y_{+}{\leftrightarrow V|}_{U{Y}_{-}}$, which factor jointly according to (53) with

$$\begin{array}{ccc}\hfill P_{Y_{-}|X_{-}}& =& {\mathbb{G}}_{Y_{-}|X_{-}}^{\left(\varrho \right)}\hfill \\ \hfill P_{Y_{+}|X_{+}}& =& {\mathbb{G}}_{Y_{+}|X_{+}}^{\left(\varrho \right)},\hfill \end{array}$$

hence are symmetric w.r.t., the inputs $X_{-}$ and $X_{+}$ and both are in the feasible set. Similarly, the unitary transformation $(Y_1,Y_2)\mapsto (Y_{-},Y_{+})$ picks a pair of independent copies of a law $P_{UXYV}\in \mathcal{Q}$ and “creates” a pair of laws, $X_{-}\leftrightarrow Y_{-}{\leftrightarrow V|}_{U{X}_{+}}$ and $X_{+}\leftrightarrow Y_{+}{\leftrightarrow V|}_{U{X}_{-}}$, which are both in the feasible set (see the factorization (61)).

Consequently, by (20), each of the quantities ${\mathsf{s}}_{\lambda }(X_{+},\varrho |X_{-},\mathbf{U})$, ${\mathsf{s}}_{\lambda }(X_{-},\varrho |Y_{+},\mathbf{U})$, ${\mathsf{s}}_{\lambda }(X_{-},\varrho |X_{+},\mathbf{U})$ and ${\mathsf{s}}_{\lambda }(X_{+},\varrho |Y_{-},\mathbf{U})$ is at least ${\mathsf{V}}_{\lambda }\left(\varrho \right)$. The assumption $P_{XU}\in \mathcal{P}$ leads by (22) of Lemma 1 to an opposite conclusion. Therefore, all four quantities are equal and coincide with the minimum, i.e.,

$$\begin{array}{ccc}\hfill {\mathsf{s}}_{\lambda }(X_{-},\varrho |Y_{+},\mathbf{U})& =& {\mathsf{s}}_{\lambda }(X_{+},\varrho |Y_{-},\mathbf{U})={\mathsf{V}}_{\lambda }\left(\varrho \right)\hfill \end{array}$$

(70a)

$$\begin{array}{ccc}\hfill {\mathsf{s}}_{\lambda }(X_{-},\varrho |X_{+},\mathbf{U})& =& {\mathsf{s}}_{\lambda }(X_{+},\varrho |X_{-},\mathbf{U})={\mathsf{V}}_{\lambda }\left(\varrho \right).\hfill \end{array}$$

(70b)

Let $B\sim Ber(1/2)$ be a Bernoulli random variable with values in the set $\{+,-\}$ independent of $(X_{-},X_{+},Y_{-},Y_{+},\mathbf{U})$. Let $\overline{B}$ be the complement of B in the set $\{+,-\}$ and define $(\tilde{X},\tilde{\mathbf{U}})$ by $\tilde{X}\triangleq X_B$ and $\tilde{\mathbf{U}}\triangleq (B,Y_{\overline{B}},\mathbf{U})$. Then

$$\begin{array}{ccc}\hfill {\mathsf{s}}_{\lambda }(\tilde{X},\varrho |\tilde{\mathbf{U}})& =& \frac{1}{2}{\mathsf{s}}_{\lambda }(X_{-},\varrho |Y_{+},\mathbf{U})+\frac{1}{2}{\mathsf{s}}_{\lambda }(X_{+},\varrho |Y_{-},\mathbf{U})\hfill \\ & \le & {\mathsf{V}}_{\lambda }\left(\varrho \right)+2ϵ,\hfill \end{array}$$

(71)

where the inequality follows by assumption (24a).

Furthermore, $\mathbb{E}[{(\tilde{X})}^2]=\mathbb{E}\left[X^2\right]$ and by (54)

$${\sigma }_{\tilde{X}|\tilde{\mathbf{U}}V}^2\le \frac{1}{2}{\sigma }_{X_{+}|\mathbf{U}V}^2+\frac{1}{2}{\sigma }_{X_{-}|\mathbf{U}V}^2\le D.$$

(72)

Next

$$\begin{array}{ccc}\hfill h\left(\tilde{Y}\right|\tilde{\mathbf{U}})& =& h\left(Y_B\right|B,Y_{\overline{B}},\mathbf{U})=\frac{1}{2}h\left(Y_{+}\right|Y_{-}\mathbf{U})+\frac{1}{2}h\left(Y_{-}\right|Y_{+}\mathbf{U})\hfill \\ \hfill h\left(\tilde{X}\right|\tilde{\mathbf{U}})& =& h\left(X_B\right|B,Y_{\overline{B}},\mathbf{U})=\frac{1}{2}h\left(X_{+}\right|Y_{-}\mathbf{U})+\frac{1}{2}h\left(X_{-}\right|Y_{+}\mathbf{U}).\hfill \end{array}$$

Thus

$$\begin{array}{ccc}\hfill h\left(\tilde{Y}\right|\tilde{\mathbf{U}})-h\left(\tilde{X}\right|\tilde{\mathbf{U}})& =& \frac{1}{2}\left[h\left(Y_{+}\right|Y_{-}\mathbf{U})+h\left(Y_{-}\right|Y_{+}\mathbf{U})\right]-\frac{1}{2}\left[h\left(X_{+}\right|Y_{-}\mathbf{U})+h\left(X_{-}\right|Y_{+}\mathbf{U})\right]\hfill \\ & =& \frac{1}{2}\left[h\left(Y_{+}{Y}_{-}\right|\mathbf{U})-h\left(Y_{-}\right|\mathbf{U})+h\left(Y_{-}\right|Y_{+}\mathbf{U})\right]-\frac{1}{2}\left[h\left(X_{+}\right|Y_{-}\mathbf{U})+h\left(X_{-}\right|Y_{+}\mathbf{U})\right]\hfill \\ & =& \frac{1}{2}\left[h\left(Y_{+}{Y}_{-}\right|\mathbf{U})-h\left(X_{+}{X}_{-}\right|\mathbf{U})\right]-\frac{1}{2}I(Y_{-};Y_{+}|\mathbf{U})\hfill \\ & & +\frac{1}{2}\left[h\left(X_{+}\right|\mathbf{U})+h\left(X_{-}\right|X_{+}\mathbf{U})\right]-\frac{1}{2}\left[h\left(X_{+}\right|Y_{-}\mathbf{U})+h\left(X_{-}\right|Y_{+}\mathbf{U})\right]\hfill \\ & \stackrel{\left(a\right)}{=}& \frac{1}{2}\left[h\left(Y_{+}{Y}_{-}\right|\mathbf{U})-h\left(X_{+}{X}_{-}\right|\mathbf{U})\right]-\frac{1}{2}I(Y_{-};Y_{+}|\mathbf{U})+\frac{1}{2}I(X_{+};Y_{-}|\mathbf{U})\hfill \\ & & +\frac{1}{2}\left[h\left(X_{-}\right|X_{+}{Y}_{+}\mathbf{U})-h\left(X_{-}\right|Y_{+}\mathbf{U})\right]\hfill \\ & \stackrel{\left(b\right)}{=}& \frac{1}{2}\left[h\left(Y_{+}{Y}_{-}\right|\mathbf{U})-h\left(X_{+}{X}_{-}\right|\mathbf{U})\right]+\frac{1}{2}\left[h\left(Y_{-}\right|Y_{+}\mathbf{U})-h\left(Y_{-}\right|X_{+}{Y}_{+}\mathbf{U})\right]\hfill \\ & & -\frac{1}{2}I(X_{-};X_{+}|Y_{+}\mathbf{U})\hfill \\ & =& \frac{1}{2}\left[h\left(Y_{+}{Y}_{-}\right|\mathbf{U})-h\left(X_{+}{X}_{-}\right|\mathbf{U})\right]+\frac{1}{2}I(Y_{-};X_{+}|Y_{+}\mathbf{U})-\frac{1}{2}I(X_{-};X_{+}|Y_{+}\mathbf{U})\hfill \\ & \stackrel{\left(c\right)}{=}& \frac{1}{2}\left[h\left(Y_{+}{Y}_{-}\right|\mathbf{U})-h\left(X_{+}{X}_{-}\right|\mathbf{U})\right]+\frac{1}{2}I(Y_{-};X_{+}|Y_{+}\mathbf{U})-\frac{1}{2}I(X_{-}{Y}_{-};X_{+}|Y_{+}\mathbf{U})\hfill \\ & =& \frac{1}{2}\left[h\left(Y_{+}{Y}_{-}\right|\mathbf{U})-h\left(X_{+}{X}_{-}\right|\mathbf{U})\right]-\frac{1}{2}I(X_{-};X_{+}|Y_{-}{Y}_{+}\mathbf{U})\hfill \\ & =& h\left(Y\right|U)-h\left(X\right|U)-\frac{1}{2}I(X_{-};X_{+}|Y_{-}{Y}_{+}\mathbf{U}).\hfill \end{array}$$

(73)

Here,

$\left(a\right)$

follows since $X_{-}\leftrightarrow \left(X_{+}\mathbf{U}\right)\leftrightarrow Y_{+}$ forms a Markov chain,

$\left(b\right)$

follows since $Y_{-}\leftrightarrow \left(X_{+}\mathbf{U}\right)\leftrightarrow Y_{+}$ forms a Markov chain, and

$\left(c\right)$

follows since $Y_{-}\leftrightarrow \left(X_{-}{Y}_{+}\mathbf{U}\right)\leftrightarrow X_{+}$ forms a Markov chain.

The combination of identity (73) with inequality (24b) proves (26).

Since $\tilde{X}\in \mathbb{R}$ while $\tilde{\mathbf{U}}\in \{+,-\}\times \mathbb{R}\times \mathcal{U}\times \mathcal{U}$ a dimensionality reduction argument as in Remark 2 establishes the existence of a law $P_{X^{\prime }{U}^{\prime }}$, where $U^{\prime }$ has finite support, such that (25) and (26) are fulfilled, hence $(U^{\prime },X^{\prime })$ are in the feasible set.