The State-Dependent Channel with a Rate-Limited Cribbing Helper

Abstract

The capacity of a memoryless state-dependent channel is derived for a setting in which the encoder is provided with rate-limited assistance from a cribbing helper that observes the state sequence causally and the past channel inputs strictly causally. Said cribbing may increase capacity but not to the level achievable by a message-cognizant helper.

1. Introduction

An encoder for a state-dependent channel is said to have causal state information if the channel input $X_i$ it produces at time i may depend, not only on the message m it wishes to transmit, but also on the present and past channel states $S_i$ and $S^{i-1}$ (where $S^{i-1}$ stands for the states $S_1,\dots ,S_{i-1}$). Its state information is noncausal if, in addition to depending on the message, its time i input may depend on all the channel states: past $S^{i-1}$, present $S_i$, and future $S_{i+1}^n$ (where n denotes the blocklength, and $S_{i+1}^n$ stands for $S_{i+1},\dots ,S_n$).

The former case was studied by Shannon [1], who showed that capacity can be achieved by what-we-now-call Shannon strategies. The latter was studied by Gel’fand and Pinsker [2], who showed that the capacity, in this case, can be achieved using binning [3].

As of late, there has been renewed interest in the causal case, but when the state information must be quantized before it is provided to the encoder [4]. While still causal, the encoder is not provided now with the state sequence $\left\{S_i\right\}$ directly, but rather with some “assistance sequence” $\left\{T_i\right\}$ describing it. Its time i output $X_i$ is now determined by the message m and by the present and past assistances $T^i$. The assistance sequence is produced by a helper, which observes the state sequence causally and produces the time i assistance $T_i$ based on the present and past states $S^i$ subject to the additional constraint that $T_i$ take values in a given finite set $\mathcal{T}$ whose cardinality is presumably smaller than that of the state alphabet $\mathcal{S}$. (If the cardinality of $\mathcal{T}$ is one, the problem reduces to the case of no assistance; if it exceeds or equals the cardinality of $\mathcal{S}$, the problem reduces to Shannon’s original problem because, in this case, $T_i$ can describe $S_i$ unambiguously.) We refer to the base-2 logarithm of the cardinality of $\mathcal{T}$ as the “help rate” and denote it $R_h$:

$$\begin{array}{ccc}\hfill R_{\mathrm{h}}& =\hfill & {log}_2|\mathcal{T}|.\end{array}$$

(1)

Three observations in [4] inspired the present paper:

• Symbol-by-symbol quantizers are suboptimal: restricting $T_i$ to be a function of $S_i$ may reduce capacity.
• Allowing $T_i$ to depend not only on $S^i$ but also on the message m may increase capacity.
• If $T_i$ is allowed to depend on $S^i$, as well as on the transmitted message, then message-cognizant symbol-by-symbol helpers achieve capacity: there is no loss in capacity in restricting $T_i$ to be a function of $(m,S_i)$.

Sandwiched between the message-oblivious helper and the message-cognizant helper is the cribbing helper whose time-i assistance $T_i$ depends on $S^i$ and on the past symbols produced by the encoder

$$\begin{array}{ccc}\hfill T_i& =\hfill & T_i\left(S^i,X^{i-1}\right).\end{array}$$

(2)

Such a helper is depicted in Figure 1.

Figure 1. Communication over a state-dependent channel with a rate-limited causal cribbing helper.

Since one can reproduce the channel inputs from the states and message, the cribbing helper cannot outperform the message-cognizant helper. And since the helper can ignore the past channel inputs, the cribbing capacity must be at least as high as that of the message-oblivious helper.

Here, we shall characterize the capacity with a cribbing helper and show that the above inequalities can be strict: the message-cognizant helper may outperform the cribbing helper, and the latter may outperform the message-oblivious helper (presumably because, thanks to the cribbing, it can learn something about the message). We further show that the capacity of the cribbing helper can be achieved using a block Markov coding scheme with backward decoding.

It is important to note that allowing the helper to crib does not render it a relay [5] because the helper does not communicate with the receiver. Therefore, our results do not have any bearing on the Relay problem.

It is also noteworthy that message-cognizant helpers are also advantageous in the noncausal case. For such helpers, capacity was recently computed in [6,7]. Cribbing, however, is somewhat less natural in this setting.

2. Problem Statement and Main Result

We are given a state-dependent discrete memoryless channel $W_{Y|XS}$ of finite input, output, and state alphabets $\mathcal{X}$, $\mathcal{Y}$, and $\mathcal{S}$. When its input is $x\in \mathcal{X}$ and its state is $s\in \mathcal{S}$, the probability of its output being $y\in \mathcal{Y}$ is $W_{Y|XS}(y|x,s)$. The states $\left\{S_i\right\}$ are drawn IID $\sim P_S$, where $P_S$ is some given probability mass function (PMF) on the state alphabet $\mathcal{S}$. Also given is some finite set $\mathcal{T}$ that we call the description alphabet. We shall assume throughout that its cardinality is at least 2

$$|\mathcal{T}|\ge 2$$

(3)

because otherwise the helper cannot provide any assistance.

Given some blocklength n, a rate-R message set is a set $\mathcal{M}$ whose cardinality is $2^{nR}$ (where we ignore the fact that the latter need not be an integer). A blocklength-n encoder for our channel comprises n mappings

$$\begin{array}{c}\hfill f_i:\mathcal{M}\times {\mathcal{T}}^i\to \mathcal{X},i=1,\dots ,n\end{array}$$

(4)

with the understanding that if the message to be transmitted is $m\in \mathcal{M}$, and if the assistance sequence produced by the helper is $t^n\in {\mathcal{T}}^n$, then the time i channel input produced by the encoder is

$$\begin{array}{ccc}\hfill x_i& =\hfill & f_i(m,t^i)\end{array}$$

(5)

which we also denote $x_i(m,t^i)$. Here, ${\mathcal{T}}^i$ denotes the i-fold Cartesian product

$$\begin{array}{c}\hfill T^i=\underset{i\mathrm{times}}{\underbrace{\mathcal{T}\times \mathcal{T}\times \cdots \times \mathcal{T}}}\end{array}$$

(6)

and $t^j$ denotes $t_1,\dots ,t_j$. A blocklength-n cribbing helper comprises n mapping

$$\begin{array}{c}\hfill h_i:{\mathcal{X}}^{i-1}\times {\mathcal{S}}^i\to \mathcal{T},i=1,\dots ,n\end{array}$$

(7)

with the understanding that—after observing the channel inputs $x_1,\dots ,x_{i-1}$ and the states $s_1,\dots ,s_i$—the helper produces the time i assistance

$$\begin{array}{ccc}\hfill t_i& =\hfill & h_i\left(x^{i-1},s^i\right)\end{array}$$

(8)

which we also denote $t_i\left(x^{i-1},s^i\right)$.

Communication proceeds as follows: the helper produces the time-1 assistance $t_1$ that is given by $h_1\left(s_1\right)$, and the encoder then produces the first channel input $x_1=f_1(m,t_1)$. The helper then produces the time-2 assistance $t_2$ that is given by $h_2(x_1,s^2)$, and the encoder then produces the second channel input $x_2=f_2(m,t^2)$, and so on.

The decoder is cognizant neither of the state sequence $s^n$ nor of the assistance sequence $t^n$: it is thus a mapping of the form

$$\varphi :{\mathcal{Y}}^n\to \mathcal{M}$$

(9)

with the understanding that, upon observing the output sequence $Y^n$, the decoder guesses that the transmitted message is $\varphi \left(Y^n\right)$.

Let $P_e=Pr\left(\varphi \left(Y^n\right)\ne M\right)$ denote the probability of a decoding error when the message M is drawn uniformly from $\mathcal{M}$. If $P_e<ϵ$, then we say that the coding scheme is of parameters $(n,2^{nR},|\mathcal{T}|,ϵ)$ or that it is a $(n,2^{nR},|\mathcal{T}|,ϵ)$-scheme. A rate R is said to be achievable if, for every $ϵ>0$, there exist, for all sufficiently large n, schemes as above with $P_e<ϵ$. The capacity of the channel is defined as the supremum of all achievable rates R and is denoted C.

Define

$$C^{\left(I\right)}=maxmin\left\{I(UV;Y),I(U;X|VT)\right\}$$

(10)

where the maximum is over all finite sets $\mathcal{U}$ and $\mathcal{V}$ and over all joint distributions of the form

$$P_S{P}_{UV}{P}_{T|VS}{P}_{X|UVT}{W}_{Y|XS}$$

(11)

with T taking values in the assistance alphabet $\mathcal{T}$. (When writing Markov conditions and information theoretic quantities such as entropy and mutual information, we do not separate the variables with commas. We thus write $H\left(XY\right)$, and not $H(X,Y)$, for the joint entropy of X and Y. We do, however, introduce commas when this convention can lead to ambiguities; see, for example, (62).)

Our main result is stated next.

Theorem 1.

The capacity C of the memoryless state-dependent channel with rate-limited cribbing helper equals $C^{\left(I\right)}$:

$$C=C^{\left(I\right)}.$$

(12)

Moreover, the maximum in (10) can be achieved when:

1. 

$P_{T|VS}$ and $P_{X|UVT}$ are both zero-one laws.

2. 

The alphabet sizes of U and V are restricted to

$$\begin{array}{ccc}\hfill |\mathcal{V}|& \le \hfill & L^2|\mathcal{S}|\left(|\mathcal{T}|-1\right)+L\\ \hfill |\mathcal{U}|& \le \hfill & L^3|\mathcal{T}|\left(|\mathcal{X}|-1\right)+L\end{array}$$

where $L=|\mathcal{X}||\mathcal{T}||\mathcal{S}|+1$.

3. 

The chain

$$V⊸\text{–} U⊸\text{–} \left(XTS\right)⊸\text{–} Y$$

(13)

is a Markov chain.

(Henceforth, we use $A⊸\text{–} B⊸\text{–} C$ to indicate that A and C are conditionally independent given B and, more generally, $A⊸\text{–} B⊸\text{–} C⊸\text{–} D$ to indicate that $A,B,C,D$ forms a Markov chain.)

The proof is given in Section 4.

Remark 1.

The assumption of (3) notwithstanding, the theorem also holds in case $|\mathcal{T}|=1$, which corresponds to no help.

Proof of Remark 1. 

When T is deterministic, $P_{X|UVT}$ equals $P_{X|UV}$, and the data processing inequality implies that $I(UV;Y)\le I(X;Y)$, thus establishing that, in this case, $C^{\left(I\right)}$ is upper bounded by the capacity without state information, i.e.,

$$\begin{array}{ccc}\hfill C^{\left(I\right)}& \le \hfill & {\displaystyle \underset{P_X}{max}}I(X;Y).\end{array}$$

(14)

Equality can be established by choosing V as null and U as X, a choice that results in $I(UV;Y)$ being $I(X;Y)$ and in $I(U;X|VT)$ being $H\left(X\right)$. □

Remark 2.

As is to be expected, when $|\mathcal{T}|\ge |\mathcal{S}|$, i.e., when T can describe S precisely, $C^{\left(I\right)}$ reduces to the Shannon strategies capacity $C^{\left(Sh\right)}$ of the channel with perfect causal state information at the transmitter:

$$C^{\left(\mathrm{Sh}\right)}=maxI(U;Y)$$

(15)

where the maximization is over all the joint PMFs of the form $P_S{P}_U{P}_{X|US}{W}_{Y|XS}$ (and where, without altering the result of the maximization, we can restrict $P_{X|US}$ to be zero–one valued).

Proof of Remark 2. 

We first establish that $C^{\left(I\right)}$ is greater-equal $C^{\left(\mathrm{Sh}\right)}$. To that end, we set T to equal S and V to be null and then argue that, with this choice, the minimum of the two terms in (10) is the first, i.e., $I(UV;Y)$ (which, because V is null, equals $I(U;Y)$). To that end, we calculate

$$\begin{array}{}\hfill \mathrm{(16)}& \begin{array}{ccc}\hfill I(U;X|VT)& =\hfill & I(U;X|VT)\hfill \end{array}\hfill \hfill \mathrm{(17)}& \begin{array}{ccc} & =\hfill & I(U;X|S)\end{array}\hfill \hfill \mathrm{(18)}& \begin{array}{ccc} & =\hfill & I(U;XS)\end{array}\hfill \hfill \mathrm{(19)}& \begin{array}{ccc} & =\hfill & I(U;Y)\end{array}\hfill \end{array}$$

where the first equality holds because V is null, the second because T equals S, the third because U is independent of S, and the final inequality follows from the Data Processing inequality.

It remains to prove that

$$\begin{array}{ccc}\hfill C^{\left(I\right)}& \le \hfill & C^{\left(\mathrm{Sh}\right)}\end{array}$$

(20)

which always holds. To simplify our analysis, we assume the Markov condition (13), and we then upper-bound $I(UV;Y)$ (which is an upper bound on the minimum in the definition (10) of $C^{\left(I\right)}$). Under this Markov condition, the maximum of $I(UV;Y)$ can be achieved with V null, which we proceed to assume. The joint PMF of the remaining variables is then of the form

$$P_S{P}_U{P}_{T|S}{P}_{X|UT}{W}_{Y|XS}.$$

(21)

We will show that—for every fixed $P_U$—to any choice of $P_{T|S}$ and $P_{X|UT}$ there corresponds a choice of $P_{X|US}$, which is feasible for the maximization defining $C^{\left(\mathrm{Sh}\right)}$ in (15) and that thus proves that $C^{\left(\mathrm{Sh}\right)}\ge I(U;Y)$.

To this end, we begin by expressing the channel from U to Y using (21) as

$$\begin{array}{}\hfill \mathrm{(22)}& \begin{array}{ccc}\hfill P_{Y|U}& =\hfill & {\displaystyle \sum _s}{P}_S{P}_{X|US}{W}_{Y|XS}\end{array}\hfill \hfill \mathrm{(23)}& \begin{array}{ccc} & =\hfill & {\displaystyle \sum _s}{P}_S\left({\displaystyle \sum _t}{P}_{T|US}{P}_{X|UST}\right)W_{Y|XS}\end{array}\hfill \hfill \mathrm{(24)}& \begin{array}{ccc} & =\hfill & {\displaystyle \sum _s}{P}_S\left({\displaystyle \sum _t}{P}_{T|S}{P}_{X|UT}\right)W_{Y|XS}.\end{array}\hfill \end{array}$$

We then note that, for a fixed $P_U$, the mutual information $I(U;Y)$ is thus determined by the $|\mathcal{S}|·|\mathcal{U}|$ conditional PMFs of X given $(S,U)=(s,u)$

$${\left\{\sum _t{P}_{T|US}(t|u,s)P_{X|UT}(x|u,t)\right\}}_{(s,u)\in \mathcal{S}\times \mathcal{U}}.$$

These conditional PMFs are feasible for the maximization defining $C^{\left(\mathrm{Sh}\right)}$ in (15), thus demonstrating that $C^{\left(\mathrm{Sh}\right)}\ge I(U;Y)$. □

3. Example

We next present an example where the message-cognizant helper outperforms the cribbing helper and the latter outperforms the plain-vanilla causal helper. It is trivial to find cases where the three perform identically, e.g., when the state does not affect the channel. The example is borrowed from ([4], Example 7) (from which we also lift the notation).

The channel inputs, states, and outputs are binary tuples

$$\mathcal{X}=\mathcal{S}=\mathcal{Y}=\{0,1\}\times \{0,1\}$$

(25)

and are denoted $(A,B)$, $\left(S^{\left(0\right)},S^{\left(1\right)}\right)$, and $\left(Y^{\left(0\right)},Y^{\left(1\right)}\right)$ respectively. The two components of the state are IID, each taking on the values 0 and 1 equiprobably. Given the state and input, the channel output is deterministically given by

$$Y=\left(A,B\oplus S^{\left(A\right)}\right).$$

(26)

The assistance is one-bit assistance, so $\mathcal{T}=\{0,1\}$.

As shown in ([4], Claim 8), the capacity with a message-cognizant helper is 2 bits, and with a message-oblivious helper $log3$. Here, we show that the capacity with a cribbing helper is strictly smaller than 2 bits and strictly larger than $log3$. All logarithms in this section are base-2 logarithms, and all rates are in bits.

We begin by showing the former. Recall that if R is achievable, then it must satisfy the constraints

$$\begin{array}{ccc}\hfill R& \le \hfill & I(UV;Y)\end{array}$$

(27)

$$\begin{array}{ccc}\hfill R& \le \hfill & I(U;X|VT).\end{array}$$

(28)

Recall also the form of the joint PMF

$$P_S{P}_V{P}_{T|VS}{P}_{U|V}{P}_{X|UVT}{W}_{Y|XS}$$

(29)

and that we may assume that $P_{X|UVT}(x|u,v,t)$ is zero-one valued. Note that (29) implies

$$ST⊸\text{–} V⊸\text{–} U$$

(30)

and consequently

$$S⊸\text{–} TV⊸\text{–} U.$$

(31)

We will show that the above constraints cannot be both satisfied if $R=2$. To that end, we assume that

$$\begin{array}{c}\hfill I(U;X|VT)=2\end{array}$$

(32)

(it cannot be larger because $\left|\mathcal{X}\right|=4$) and prove that

$$\begin{array}{c}\hfill I(UV;Y)<2.\end{array}$$

(33)

Since $\mathcal{Y}$ is of cardinality 4, it suffices to show that

$$H(Y|UV)>0.$$

(34)

In fact, it suffices to show that

$$H(Y|UVT)>0$$

(35)

i.e., that there exist $u^{★},v^{★},t^{★}$ of positive probability for which

$$H(Y|U=u^{★},V=v^{★},T=t^{★})>0.$$

(36)

This is what we proceed to do. We first show the existence of $v^{★}$ and $t^{★}$ for which $H(S|V=v^{★},T=t^{★})\ge 1$. Once this is established, we proceed to pick $u^{★}$.

Since $\left|\mathcal{X}\right|=4$, (32) implies that

$$P_{X|V=v,T=t}\mathrm{is} \mathrm{uniform} \forall (v,t).$$

(37)

Fix any $v^{★}$ (of positive probability). As we next argue, there must exist some $t^{★}$ for which $P_{S|V=v^{★},T=t^{★}}$ is not zero–one valued. Indeed, by (29), $V\perp \perp S$, so $H(S|V=v^{★})=H\left(S\right)=2$ and

$$\begin{array}{}\hfill \mathrm{(38)}& \begin{array}{ccc}\hfill H(S|T,V=v^{★})& =\hfill & H(S|V=v^{★})-I(S;T|V=v^{★})\end{array}\hfill \hfill \mathrm{(39)}& \begin{array}{ccc} & =\hfill & H\left(S\right)-I(S;T|V=v^{★})\end{array}\hfill \hfill \mathrm{(40)}& \begin{array}{ccc} & \ge \hfill & H\left(S\right)-H(T|V=v^{★})\end{array}\hfill \hfill \mathrm{(41)}& \begin{array}{ccc} & \ge \hfill & 2-log\left|\mathcal{T}\right|\end{array}\hfill \hfill \mathrm{(42)}& \begin{array}{ccc} & =\hfill & 1\end{array}\hfill \end{array}$$

so there must exist some $t^{★}$ for which

$$H(S|V=v^{★},T=t^{★})\ge 1.$$

(43)

We next choose $u^{★}$ as follows. Conditional on $V=v^{★},T=t^{★}$, the chance variable U has some PMF $P_{U|V=v^{★},T=t^{★}}$ (equal to $P_{U|V=v^{★}}$ by (29)) under which $X(U,v^{★},t^{★})$ is uniform; see (37). It follows that there exist $u_0$ and $u_1$ (both of positive conditional probability) such that

$$\begin{array}{ccc}\hfill A(u_0,v^{★},t^{★})& =\hfill & 0\end{array}$$

(44)

$$\begin{array}{ccc}\hfill A(u_1,v^{★},t^{★})& =\hfill & 1\end{array}$$

(45)

where we introduced the notation

$$X(u,v^{★},t^{★})=\left(A(u,v^{★},t^{★}),B(u,v^{★},t^{★})\right).$$

(46)

Returning to (43), we note that it implies that

$$\begin{array}{c}\hfill H\left(S^{\left(0\right)}|V=v^{★},T=t^{★}\right)>0\mathrm{or}H\left(S^{\left(1\right)}|V=v^{★},T=t^{★}\right)>0.\end{array}$$

(47)

In the former case, $H(Y|U=u_0,V=v^{★},T=t^{★})$ is positive, and in the latter, $H(Y|U=u_1,V=v^{★},T=t^{★})$ is positive. This establishes the existence of a triple $(u^{★},v^{★},t^{★})$ for which (36) holds, and thus concludes the proof that the capacity with a cribbing encoder is smaller than 2. We next show that it exceeds $log3$.

To that end, we consider choosing

$$U=(A,\tilde{U})$$

(48)

to be uniform over $\{0,1\}\times \{0,1\}$, and we let $\sigma$ be a Bernoulli–$\alpha$ random variable that is independent of U and of the channel, for some $\alpha \in [0,1]$ to be specified later. We further define

$$\begin{array}{c}\hfill \tilde{V}=\left\{\begin{array}{ccc}A\hfill & \mathrm{if}& \sigma =1\hfill \\ 0\left(\mathrm{null}\right)\hfill & \mathrm{if}& \sigma =0\hfill \end{array}\right.\end{array}$$

(49)

and

$$V=(\tilde{V},\sigma ).$$

(50)

We choose the helper function $h(s,v)$—which can also be written as $h\left((s^{\left(0\right)},s^{\left(1\right)}),(\tilde{v},\sigma )\right)$—to equal $s^{\left(\tilde{v}\right)}$, so

$$T=S^{\left(\tilde{V}\right)}$$

(51)

and

$$T=\left\{\begin{array}{cc}{S}^{\left(A\right)}\hfill & \mathrm{w}.\mathrm{p}.\alpha \hfill \\ S^{\left(0\right)}\hfill & \mathrm{w}.\mathrm{p}.1-\alpha .\hfill \end{array}\right.$$

(52)

Our encoder function $f(u,v,t)$ ignores v and results in

$$X^{\left(0\right)}=A,X^{\left(1\right)}=\tilde{U}\oplus T$$

(53)

where $X=(X^{\left(0\right)},X^{\left(1\right)})$. That is,

$$f\left((A,\tilde{U}),T\right)=\left(A,\tilde{U}\oplus T\right).$$

(54)

Note that with the variables defined in (49)–(53), the Markov relations in item 3 of Theorem 1 hold.

We now proceed to calculate the rate bounds. For the RHS of (27), we have

$$\begin{array}{ccc}\hfill I(UV;Y)& =\hfill & I(U\tilde{V}\sigma ;Y)\hfill \\ & \ge \hfill & I(U\tilde{V};Y|\sigma )\hfill \\ & =\hfill & \alpha I(U\tilde{V};Y|\sigma =1)+(1-\alpha )I(U\tilde{V};Y|\sigma =0)\hfill \\ & =\hfill & \alpha I(A\tilde{U};Y|\sigma =1)+(1-\alpha )I(A\tilde{U};Y|\sigma =0)\hfill \end{array}$$

(55)

where the last equality holds because if $\sigma =0$, then $\tilde{V}$ is null.

We next evaluate each of the terms on the RHS of (55) separately. When $\sigma =1$,

$$T=S^{\left(A\right)}$$

$$T=S^{\left(A\right)}{X}^{\left(1\right)}=\tilde{U}\oplus S^{\left(A\right)}$$

$$Y^{\left(1\right)}=X^{\left(1\right)}\oplus S^{\left(A\right)}=\tilde{U}\oplus S^{\left(A\right)}\oplus S^{\left(A\right)}=\tilde{U}$$

(56)

so

$$Y=(Y^{\left(0\right)},Y^{\left(1\right)})=(A,\tilde{U})$$

(57)

and

$$\begin{array}{ccc}\hfill I(A\tilde{U};Y|\sigma =1)& =\hfill & H(U|\sigma =1)\hfill \\ & =\hfill & H\left(U\right)\hfill \\ & =\hfill & 2\hfill \end{array}$$

(58)

where the second equality holds because $\sigma$ is independent of U.

When $\sigma =0$,

$$\begin{array}{ccc}\hfill T& =\hfill & S^{\left(0\right)}\end{array}$$

$$\begin{array}{ccc}\hfill X& =\hfill & (A,\tilde{U}\oplus S^{\left(0\right)})\end{array}$$

$$\begin{array}{ccc}\hfill Y& =\hfill & (A,\tilde{U}\oplus S^{\left(0\right)}\oplus S^{\left(1\right)})\end{array}$$

(59)

so

$$\begin{array}{ccc}\hfill I(A\tilde{U};Y|\sigma =0)& =\hfill & I(A\tilde{U};Y^{\left(0\right)}{Y}^{\left(1\right)}|\sigma =0)\hfill \\ & =\hfill & I(A\tilde{U};A,\tilde{U}\oplus S^{\left(0\right)}\oplus S^{\left(A\right)})\hfill \\ & =\hfill & I(A\tilde{U};A)+I(A\tilde{U};\tilde{U}\oplus S^{\left(0\right)}\oplus S^{\left(A\right)}|A)\hfill \\ & =\hfill & H\left(A\right)+\frac{1}{2}I(\tilde{U};\tilde{U}\oplus S^{\left(0\right)}\oplus S^{\left(0\right)}|A=0)+\frac{1}{2}I(\tilde{U};\tilde{U}\oplus S^{\left(0\right)}\oplus S^{\left(1\right)}|A=1)\hfill \\ & =\hfill & H\left(A\right)+\frac{1}{2}H\left(\tilde{U}\right)+0=\frac{3}{2}.\hfill \end{array}$$

(60)

From (58), (60), and (55), we obtain that the RHS of (27) satisfies

$$I(UV;Y)\ge 2\alpha +(1-\alpha )\frac{3}{2}=(\alpha +3)/2.$$

(61)

Next, we evaluate the RHS of (28):

$$\begin{array}{ccc}\hfill I(U;X|VT)& =\hfill & I(U;X|\tilde{V},\sigma ,T)\hfill \\ & =\hfill & \alpha I(U;X|\tilde{V},\sigma =1,T)+(1-\alpha )I(U;X|\tilde{V},\sigma =0,T)\hfill \\ & =\hfill & \alpha I(A\tilde{U};X|A,S^{\left(A\right)},\sigma =1)+(1-\alpha )I(A\tilde{U};A,\tilde{U}\oplus S^{\left(0\right)}|S^{\left(0\right)},\sigma =0)\hfill \\ & =\hfill & \alpha I(\tilde{U};A,\tilde{U}\oplus T|A{S}^{\left(A\right)},\sigma =1)+(1-\alpha )I(A\tilde{U};A,\tilde{U}\oplus S^{\left(0\right)}|S^{\left(0\right)},\sigma =0)\hfill \\ & =\hfill & \alpha I(\tilde{U};\tilde{U}\oplus T|A{S}^{\left(A\right)},\sigma =1)+(1-\alpha )H(A,\tilde{U})\hfill \\ & =\hfill & \alpha H\left(\tilde{U}\right)+(1-\alpha )H(A,\tilde{U})\hfill \\ & =\hfill & \alpha +(1-\alpha )2\hfill \\ & =\hfill & 2-\alpha .\hfill \end{array}$$

(62)

In view of (61) and (62), any rate R satisfying

$$R\le min\left\{\right(\alpha +3)/2,2-\alpha \}$$

(63)

is achievable. Choosing $\alpha =1/3$ (which maximizes the RHS of (63)), demonstrates the achievability of

$$R=5/3$$

(64)

which exceeds $log3$.

4. Proof of Theorem 1

4.1. Direct Part

Pick a distribution as in (11), where $P_{T|SV}$ and $P_{X|UVT}$ are 0–1 laws, so

$$\begin{array}{ccc}\hfill x& =\hfill & f(u,v,t)\end{array}$$

(65)

$$\begin{array}{ccc}\hfill t& =\hfill & h(s,v)\end{array}$$

(66)

for some deterministic functions f and h. Extend these functions to act on n-tuples componentwise so that if $s,v$ are n-tuples in ${\mathcal{S}}^n$ and ${\mathcal{V}}^n$, then $\mathit{t}=h(s,v)$ indicates that t is an n-tuple in ${\mathcal{T}}^n$ whose i-th component $t_i$ equals $h(s_i,v_i)$, where $s_i$ and $v_i$ are the corresponding components of s and v. Likewise, we write $\mathit{x}=f(\mathit{u},\mathit{v},\mathit{t})$.

To prove achievability, we propose a block Markov coding scheme with the receiver performing backward decoding. Although only the receiver is required to decode the message, in our scheme, the helper does too (but not with backward decoding, which would violate causality).

The transmission comprises Bn-length sub-blocks, for a total of $Bn$ channel uses. The transmitted message m is represented by $B-1$ sub-messages $m_1,\dots ,m_{B-1}$, with each of the sub-messages taking values in the set $\mathcal{M}\stackrel{\Delta }{=}\{1,2,\dots ,2^{nR}\}$. The overall transmission rate is thus $R(B-1)/B$, which can be made arbitrarily close to R by choosing B very large. The $B-1$ sub-messages are transmitted in the first $B-1$ sub-blocks, with $m_b$ transmitted in sub-block b (for $b\in [1:B-1]$). Hereafter, we use $s^{\left(b\right)}$ to denote the state n-tuple affecting the channel in sub-block b and use $s_i^{\left(b\right)}$ to denote its i-component (with $i\in [1:n]$). Similar notation holds for ${\mathit{x}}^{\left(b\right)}$, ${\mathit{y}}^{\left(b\right)}$, etc.

We begin with an overview of the scheme, where we focus on the transmission in sub-blocks 2 through $B-1$: the first and last sub-blocks must account for some edge effects that we shall discuss later. Let b be in this range. The coding we use in sub-block b is superposition coding with the cloud center determined by $m_{b-1}$ and the satellite by $m_b$.

Unlike the receiver, the helper, which must be causal, cannot employ backward decoding: it decodes each sub-message at the end of the sub-block in which it is transmitted. Consequently, when sub-block b begins, it already has a reliable guess ${\widehat{m}}_{b-1}$ of $m_{b-1}$ (based on the previous channel inputs $x^{(b-1)}$ it cribbed). The encoder, of course, knows $m_{b-1}$, so the two can agree on the cloud center $v^{\left(b\right)}\left(m_{b-1}\right)$ indexed by $m_{b-1}$. (We ignore for now the fact that ${\widehat{m}}_{b-1}$ may, with small probability, differ from $m_{b-1}$.) The satellite is computed by the encoder as ${\mathit{u}}^{\left(b\right)}(m_b|m_{b-1})$; it is unknown to the helper. The helper produces the sub-block b assistance ${\mathit{t}}^{\left(b\right)}$ based on the state sequence and the cloud center

$${\mathit{t}}^{\left(b\right)}=h\left(s^{\left(b\right)},v^{\left(b\right)}\left(m_{b-1}\right)\right).$$

(67)

(Since $h(·,·)$ acts componentwise, this help is causal with the i-th component of $t^{\left(b\right)}$ being a function of the corresponding component $s_i^{\left(b\right)}$ of the state sequence and $v^{\left(b\right)}\left(m_{b-1}\right)$; it does not require knowledge of future states.)

For its part, the encoder produces the n-tuple

$${\mathit{x}}^{\left(b\right)}=f\left({\mathit{u}}^{\left(b\right)}(m_b|m_{b-1}),{\mathit{v}}^{\left(b\right)}\left(m_{b-1}\right),{\mathit{t}}^{\left(b\right)}\right)$$

(68)

with causality preserved because ${\mathit{u}}^{\left(b\right)}(m_b|m_{b-1})$ and $v^{\left(b\right)}\left(m_{b-1}\right)$ can be computed from $m_{b-1}$ and $m_b$ ahead of time, and because t is presented to the encoder causally and $f(·)$ operates componentwise.

As to the first and last sub-blocks: In the first, we set $m_0$ as constant (e.g., $m_0=1$), so we have only one cloud center. In sub-block B, we send no fresh information, so each cloud center has only one satellite.

We now proceed to a more formal exposition. For this, we will need some notation. Given a joint distribution $P_{XYZ}$, we denote by ${\mathcal{T}}_{XY}$ the set of all jointly typical sequences $(x,y)$, where the length n is understood from the context, and we adopt the $\delta$-convention of [8]. Similarly, given a sequence z, ${\mathcal{T}}_{XYZ}\left(z\right)$ stands for the set of all pairs $(x,y)$ that are jointly typical with the given sequence z.

To describe the first and last sub-blocks, we define $m_0=1$ and $m_B=1$, respectively. The proof of the direct part is based on random coding and joint typicality decoding.

4.1.1. Code Construction

We construct B codebooks $\left\{{\mathcal{C}}_b\right\}$, $b\in [1:B]$, each of length n. Each codebook ${\mathcal{C}}_b$ is generated randomly and independently of the other codebooks as follows:

• For every $b\in [1:B]$, generate $2^{nR}$ length-n cloud centers $\left\{v^{\left(b\right)}\left(j\right)\right\}$, $j\in \mathcal{M}$ independently, each with IID $\sim P_V$ components.
• For every $b\in [1:B]$ and $j\in \mathcal{M}$, generate $2^{nR}$ length-n satellites $\left\{u^{\left(b\right)}(m|j)\right\}$, $m\in \mathcal{M}$ conditionally independently given $v^{\left(b\right)}\left(j\right)$, each according to

  $$\prod _{i=1}^n{P}_{U|V}\left(·|v_i^{\left(b\right)}\left(j\right)\right).$$

  (69)

The codebook ${\mathcal{C}}_b$ is the collection

$$\left\{v^{\left(b\right)}\left(j\right),u^{\left(b\right)}(m|j)\right\},(j,m)\in \mathcal{M}\times \mathcal{M}.$$

(70)

Reveal the codebooks to the encoder, decoder, and helper.

4.1.2. Operation of the code

We first describe the operation of the helper and encoder in the first sub-block.

Helper. In the first sub-block, $b=1$, the helper produces

$${\mathit{t}}^{\left(1\right)}=(t_1^{\left(1\right)},t_2^{\left(1\right)},\dots ,t_n^{\left(1\right)})$$

(71)

where

$${\mathit{t}}_i^{\left(1\right)}=h(s_i^{\left(1\right)},v_i^{\left(1\right)}\left(m_0\right)),1\le i\le n.$$

(72)

Note that ${\mathit{t}}^{\left(1\right)}$ is causal in ${\mathit{s}}^{\left(1\right)}$.

Encoder. Set ${\mathit{u}}^{\left(1\right)}={\mathit{u}}^{\left(1\right)}(m_1|m_0)$ and ${\mathit{v}}^{\left(1\right)}={\mathit{v}}^{\left(1\right)}\left(m_0\right)$. The input to the channel is

$${\mathit{x}}^{\left(1\right)}=\left(x_1^{\left(1\right)},x_2^{\left(1\right)},\dots ,x_n^{\left(1\right)}\right)$$

(73)

where

$$\begin{array}{cc}\hfill x_i^{\left(1\right)}& =f\left(u_i^{\left(1\right)}(m_1|m_0),v_i^{\left(1\right)}\left(m_0\right),t_i^{\left(1\right)}\left(s_i^{\left(1\right)},v_i^{\left(1\right)}\left(m_0\right)\right)\right)\hfill \\ & =f\left(u_i^{\left(1\right)},v_i^{\left(1\right)},t_i^{\left(1\right)}\right),1\le i\le n.\hfill \end{array}$$

(74)

Note that ${\mathit{x}}^{\left(1\right)}$ is causal in ${\mathit{t}}^{\left(1\right)}$.

Helper at the end of the sub-block. Thanks to its cribbing, at the end of sub-block 1, the helper is cognizant of ${\mathit{x}}^{\left(1\right)}$. In addition, it knows ${\mathit{v}}^{\left(1\right)}$ (since it is determined by $m_0$, which was set a priori) and ${\mathit{t}}^{\left(1\right)}$ (since it was produced by itself). The helper now decodes the message $m_1$ by looking for an index $j\in \mathcal{M}$ such that

$$\left({\mathit{u}}^{\left(1\right)}(j|m_0),x^{\left(1\right)}\right)\in T_{UXVT}({\mathit{v}}^{\left(1\right)},{\mathit{t}}^{\left(1\right)}).$$

(75)

If such an index j exists and is unique, the helper sets ${\widehat{m}}_1=j$. Otherwise, an error is declared. By standard results, the probability of error is vanishingly small provided that

$$R<I(U;X|VT).$$

(76)

Denote by ${\widehat{m}}_1$ the message decoded by the helper at the end of sub-block 1. We proceed to describe the operation of the helper and encoder in sub-block b, when $2\le b\le B-1$.

Helper, $2\le b\le B-1$. Denote by ${\widehat{m}}_{b-1}$ the message decoded by the helper at the end of sub-block $(b-1)$. In sub-block b, the helper produces

$${\mathit{t}}^{\left(b\right)}=(t_1^{\left(b\right)},t_2^{\left(b\right)},\dots ,t_n^{\left(b\right)})$$

(77)

where

$$t_i^{\left(b\right)}=h\left(s_i^{\left(b\right)},v_i^{\left(b\right)}\left({\widehat{m}}_{b-1}\right)\right),1\le i\le n.$$

(78)

Encoder, $2\le b\le B-1$. Set ${\mathit{u}}^{\left(b\right)}={\mathit{u}}^{\left(b\right)}(m_b|m_{b-1})$ and ${\mathit{v}}^{\left(b\right)}={\mathit{v}}^{\left(b\right)}\left({\widehat{m}}_{b-1}\right)$. The input to the channel is

$${\mathit{x}}^{\left(b\right)}=\left(x_1^{\left(b\right)},x_2^{\left(b\right)},\dots ,x_n^{\left(b\right)}\right)$$

(79)

where

$$\begin{array}{cc}\hfill x_i^{\left(b\right)}& =f\left(u_i^{\left(b\right)}(m_b|m_{b-1}),v_i^{\left(b\right)}\left(m_{b-1}\right),t_i^{\left(b\right)}\left(s_i^{\left(b\right)},v_i^{\left(b\right)}\left({\widehat{m}}_{b-1}\right)\right)\right)\hfill \\ & =f\left(u_i^{\left(b\right)},v_i^{\left(b\right)},t_i^{\left(b\right)}\right),1\le i\le n.\hfill \end{array}$$

(80)

Note that $t^{\left(b\right)}$ and $x^{\left(b\right)}$ are causal in $s^{\left(b\right)}$ and $t^{\left(b\right)}$, respectively.

Helper at the end of the sub-block, $2\le b\le B-1$. At the end of sub-block b the helper has ${\mathit{x}}^{\left(b\right)}$ at hand. In addition, it has ${\mathit{v}}^{\left(b\right)}\left({\widehat{m}}_{b-1}\right)$ (since ${\widehat{m}}_{b-1}$ was decoded at the end of the previous sub-block) and ${\mathit{t}}^{\left(b\right)}$ (since it was produced by itself). The helper now decodes the message $m_b$. Assuming that ${\widehat{m}}_{b-1}$ was decoded correctly, this can be done with a low probability of error if (37) is satisfied.

We proceed to the last sub-block, where no fresh information is sent. Here $m_B=1$, and the operations of the helper and encoder proceed exactly as in (77)–(80), with $b=B$. Note that in sub-block B, the helper need not decode $m_B$ since it is set a priori and known to all.

4.1.3. Decoding

At the destination, we employ backward decoding. Starting at sub-block B with $m_B=1$, the decoder looks for an index $j\in \mathcal{M}$ such that

$$\left({\mathit{u}}^{\left(B\right)}(1|j),{\mathit{v}}^{\left(B\right)}\left(j\right),{\mathit{y}}^{\left(B\right)}\right)\in {\mathcal{T}}_{UVY}.$$

(81)

If such an index exists and is unique, the decoder sets ${\widehat{\widehat{m}}}_{B-1}=j$. Otherwise, an error is declared. By standard result, the decoding is correct with probability approaching 1 provided

$$R<I(UV;Y).$$

(82)

In the next (backward) decoding sub-blocks, the decoding proceeds as in (81), with the exception that the estimate ${\widehat{\widehat{m}}}_b$ replaces the default value $m_B=1$ in (81). Thus, in sub-block $B-1$, the decoder has at hand the estimate ${\widehat{\widehat{m}}}_{B-1}$, and the channel output $y^{(B-1)}$. It looks for an index j such that

$$\left({\mathit{u}}^{(B-1)}({\widehat{\widehat{m}}}_{B-1}|j),{\mathit{v}}^{(B-1)}\left(j\right),{\mathit{y}}^{(B-1)}\right)\in {\mathcal{T}}_{UVY}.$$

(83)

Similarly, for $2\le b\le B-1$, the decoder looks for an index j such that

$$\left({\mathit{u}}^{\left(b\right)}({\widehat{\widehat{m}}}_b|j),{\mathit{v}}^{\left(b\right)}\left(j\right),{\mathit{y}}^{\left(b\right)}\right)\in {\mathcal{T}}_{UVY}.$$

(84)

If such an index j exists and is unique, the decoder sets ${\widehat{\widehat{m}}}_{b-1}=j$. Otherwise, an error is declared. Assuming that $m_b$ was decoded correctly in the previous decoding stage, i.e., ${\widehat{\widehat{m}}}_b=m_b$, the decoding of $m_{b-1}$ in sub-block b is correct with probability close to 1 provided that (82) holds. Note that $m_1$ is decoded in sub-block $b=2$, that is, ${\mathit{y}}^{\left(1\right)}$ is not used at the destination. However, the transmission in sub-block 1 is not superfluous, as it is used by the helper to decode $m_1$ at the end of the first sub-block. Since (76) and (82) are the two terms in (10), this concludes the proof of the direct part.

4.2. Converse Part

Fix $|\mathcal{T}|$, and consider $(n,2^{nR},|\mathcal{T}|,{\widetilde{ϵ}}_n)$-codes with ${\widetilde{ϵ}}_n\downarrow 0$. For each n, feed a random message M that is drawn equiprobably from the set $\{1,2,\dots ,2^{nR}\}$ to the encoder. By the channel model,

$$M⊸\text{–} \left(X^n{S}^n\right)⊸\text{–} Y^n.$$

(85)

Fano’s inequality and the fact that ${\widetilde{ϵ}}_n\downarrow 0$ imply the existence of a sequence ${ϵ}_n\downarrow 0$ for which

$$\begin{array}{ccc}\hfill n(R-{ϵ}_n)& \le \hfill & I(M;Y^n)\hfill \\ & \stackrel{\left(a\right)}{\le }\hfill & I(M;X^n{S}^n)\hfill \\ & =\hfill & I(M;X^n|S^n)\hfill \\ & =\hfill & {\displaystyle \sum _{i=1}^n}I(M;X_i|S^n{X}^{i-1})\hfill \\ & \stackrel{\left(b\right)}{=}\hfill & {\displaystyle \sum _{i=1}^n}I(M;X_i|S^n{X}^{i-1}{T}_i)\hfill \\ & \le \hfill & {\displaystyle \sum _{i=1}^n}I(M{S}_i^n;X_i|S^{i-1}{X}^{i-1}{T}_i)\hfill \\ & \stackrel{\left(c\right)}{=}\hfill & {\displaystyle \sum _{i=1}^n}I(M;X_i|S^{i-1}{X}^{i-1}{T}_i)\hfill \\ & \le \hfill & {\displaystyle \sum _{i=1}^n}I(M{Y}^{i-1};X_i|S^{i-1}{X}^{i-1}{T}_i)\hfill \end{array}$$

(86)

where $\left(a\right)$ follows from (85); $\left(b\right)$ holds because $T_i$ is a function of $X^{i-1}{S}^i$ (8); and $\left(c\right)$ holds because $X_i$ is a function of $M{T}^i$ and hence of $M{S}^{i-1}{X}^{i-1}{T}_i$ (so $I(S_i^n;X_i|M{S}^{i-1}{X}^{i-1}{T}_i)$ must be zero).

We proceed to derive the second bound. Starting again with Fano’s inequality,

$$\begin{array}{ccc}\hfill n(R-{ϵ}_n)& \le \hfill & I(M;Y^n)\hfill \\ & =\hfill & {\displaystyle \sum _{u=1}^n}I(M;Y_i|Y^{i-1})\hfill \\ & \le \hfill & {\displaystyle \sum _{u=1}^n}I(M{Y}^{i-1};Y_i).\hfill \end{array}$$

(87)

Defining

$$\begin{array}{ccc}\hfill U_i& =\hfill & M{Y}^{i-1}\end{array}$$

(88)

$$\begin{array}{ccc}\hfill V_i& =\hfill & S^{i-1}{X}^{i-1}\end{array}$$

(89)

we can rewrite (86) and (87) as

$$\begin{array}{ccc}\hfill R-{ϵ}_n& \le \hfill & {\displaystyle \sum _{i=1}^n}I(U_i;X_i|V_i{T}_i)\end{array}$$

(90)

$$\begin{array}{ccc}\hfill R-{ϵ}_n& \le \hfill & {\displaystyle \sum _{i=1}^n}I(U_i;Y_i)\end{array}$$

(91)

Moreover, with $U_i$ and $V_i$ defined as above, $U_i{V}_i$ and $S_i$ are independent

$$\left(U_i{V}_i\right)\perp \perp S_i$$

(92)

and

$$\begin{array}{ccc}\hfill T_i& =\hfill & h_i(S_i,V_i)\end{array}$$

(93)

$$\begin{array}{ccc}\hfill X_i& =\hfill & f_i(U_i,V_i,T_i)\end{array}$$

(94)

where $h_i$ and $f_i$ are (blocklength dependent) deterministic functions. Indeed, $X_i$ can be determined from $(U_i,V_i,T_i)$ because $U_i$ determines the message M and $V_i$ determined $T^{i-1}$, so $(U_i,V_i,T_i)$ determines $(M,T^i)$ from which $X_i$ can be computed using (5).

We next do away with the sums by conditioning on a time-sharing random variable. Let Q be a random variable uniformly distributed over $\{1,2,\dots ,n\}$ independently of the channel and the state. Using Q, we can express the bounds (90), (91) as

$$\begin{array}{ccc}\hfill R-{ϵ}_n& \le \hfill & I(U_Q;X_Q|V_Q{T}_{Q}Q)\hfill \\ & =\hfill & I(U_{Q}Q;X_Q|V_Q{T}_{Q}Q)\hfill \\ & =\hfill & I(\tilde{U};X|VT)\hfill \\ & =\hfill & I(\tilde{U}V;X|VT)\hfill \\ & =\hfill & I(U;X|VT)\hfill \end{array}$$

(95)

$$\begin{array}{ccc}\hfill R-{ϵ}_n& \le \hfill & I(U_Q;Y_Q|Q)\hfill \\ & \le \hfill & I(U_{Q}Q;Y_Q)\hfill \\ & =\hfill & I(\tilde{U};Y)\hfill \\ & \le \hfill & I(\tilde{U}V;Y)\hfill \\ & =\hfill & I(U;Y)\hfill \end{array}$$

(96)

where we define

$$X=X_Q,Y=Y_Q,T=T_Q,S=S_Q$$

(97)

and the auxiliaries

$$\begin{array}{ccc}\hfill V& =\hfill & \left(V_{Q}Q\right)\end{array}$$

(98)

$$\begin{array}{ccc}\hfill \tilde{U}& =\hfill & \left(U_{Q}Q\right)\end{array}$$

(99)

$$\begin{array}{ccc}\hfill U& =\hfill & (\tilde{U},V)=\left(U_Q{V}_{Q}Q\right).\end{array}$$

(100)

Note that the conditional law of Y given $\left(XTS\right)$ is that of the channel, namely, $W_{Y|XS}$ and that S is distributed like the channel state. Moreover,

$$V⊸\text{–} U⊸\text{–} \left(XTS\right)⊸\text{–} Y.$$

(101)

Since U and V contain the time sharing random variable Q, (93) and (94) imply that,

$$\begin{array}{ccc}\hfill T& =\hfill & h(S,V)\hfill \end{array}$$

(102)

$$\begin{array}{ccc}\hfill X& =\hfill & \tilde{f}(\tilde{U},V,T)=f(U,T)\hfill \end{array}$$

(103)

for some deterministic functions h and f. Therefore, the joint distribution under which the RHS of (95) and the RHS of (96) are computed is of the form

$$P_{S\tilde{U}VTXY}=P_S{P}_{\tilde{U}V}{P}_{T|SV}{P}_{X|\tilde{U}VT}{W}_{Y|XS}$$

(104)

where $P_{T|SV}$ and $P_{X|\tilde{U}VT}$ are zero-one laws, or

$$P_{SUVTXY}=P_S{P}_U{P}_{V|U}{P}_{T|SV}{P}_{X|UT}{W}_{Y|XS}$$

(105)

where $P_{T|SV}$, $P_{X|UT}$, and $P_{V|U}$ are zero-one laws.

The form (105) and the inequalities (95), (96) establish the converse.

4.3. Cardinality Bounds

We next proceed to bound the alphabet sizes of $U,V$ in two steps. In the first, we do so while relaxing the zero-one-law requirements. In the second, we enlarge the alphabet to fulfill said requirements. Let

$$L=|\mathcal{X}||\mathcal{T}||\mathcal{S}|+1.$$

(106)

Fix a conditional distribution $p(x,t,s|u)$, and define the L functions of $p(u|v)$:

$$\begin{array}{ccc}\hfill p(x,t,s|v)& =\hfill & {\displaystyle \sum _u}p(x,t,s|u)p(u|v)(L-2\mathrm{functions})\end{array}$$

(107)

$$\begin{array}{c}I(U;X|T,V=v)\end{array}$$

$$\begin{array}{c}I(U;Y|V=v)\end{array}$$

(with the $L-2$ functions corresponding to all by one of the tuples $(x,t,s)$). By the support lemma [5,8], there exists a random variable $V^{\prime }$ with alphabet $|{\mathcal{V}}^{\prime }|\le L$, such that $P_{XTS}$, $I(U;X|TV)$, and $I(U;Y)$ are preserved. Denote by $U^{\prime }$ the resulting random variable U, i.e.,

$$P_{U^{\prime }}\left(u^{\prime }\right)=\sum _{v^{\prime }}p(u^{\prime }|v)P_{V^{\prime }}\left(v^{\prime }\right).$$

(108)

We next bound the alphabet size of $U^{\prime }$. For each $v^{\prime }\in {\mathcal{V}}^{\prime }$, we define the L functions

$$\begin{array}{c}p(x,t,s|v^{\prime },u^{\prime }\left)(L-2\mathrm{functions}\right)\hfill \end{array}$$

(109)

$$\begin{array}{c}I(U^{\prime };X|T,V^{\prime })\end{array}$$

(110)

$$\begin{array}{c}I(U^{\prime };Y|V^{\prime }).\end{array}$$

(111)

Applying again the support lemma, for every $v^{\prime }$ there exists a random variable $U^{″}$ with alphabet $|{\mathcal{U}}^{″}|\le L$ such that (109)–(111) are preserved. If we multiply ${\mathcal{U}}^{″}$$|{\mathcal{V}}^{\prime }|$ times we can, with proper labeling of the elements of ${\mathcal{U}}^{″}$, retain a Markov structure like (101). Now the alphabet sizes are fixed and independent of n. Thus, substituting $V^{\prime },U^{″}$ in (95), (96) and taking the limit $n\to \infty$ we have the upper bound

$$\begin{array}{ccc}\hfill R& \le \hfill & I(U^{″};X|V^{\prime }T)\end{array}$$

(112)

$$\begin{array}{ccc}\hfill R& \le \hfill & I(U^{″};Y)\end{array}$$

(113)

where

$$\begin{array}{ccc}\hfill P_{S{U}^{″}{V}^{\prime }TXY}& =\hfill & P_S{P}_{U^{″}{V}^{\prime }}{P}_{T|S{V}^{\prime }}{P}_{X|U^{″}{V}^{\prime }T}{W}_{Y|XS}\end{array}$$

(114)

$$\begin{array}{c}|{\mathcal{V}}^{\prime }|\le L,|{\mathcal{U}}^{″}|\le L^2\end{array}$$

(115)

and the following Markov chain holds:

$$V^{\prime }⊸\text{–} U^{″}⊸\text{–} \left(XTS\right)⊸\text{–} Y.$$

(116)

Note, however, that $P_{T|S{V}^{\prime }}$ and $P_{X|U^{″}{V}^{\prime }T}$ are no longer zero-one laws. We remedy this using the Functional Representation lemma (FRL) [5] (at the cost of increasing the alphabet sizes): a standard convexity argument will not do because—although $I(U;X|VT)$ is a convex function of $P_{T|SV}$ and also a convex function of $P_{X|UVT}$ and likewise $I(U;Y)$—the minimum of two convex functions need not be convex.

The Functional Representation lemma implies that—without altering the conditional law of T given $S{V}^{\prime }$ nor of X given $U^{″}{V}^{\prime }T$—the random variables T and X can be represented as

$$\begin{array}{ccc}\hfill T& =\hfill & {\tilde{g}}_1(S{V}^{\prime },Z_1)\end{array}$$

(117)

$$\begin{array}{ccc}\hfill X& =\hfill & {\tilde{g}}_2(U^{″}{V}^{\prime }T,Z_2)\end{array}$$

(118)

where ${\tilde{g}}_1$, ${\tilde{g}}_2$ are deterministic functions; $Z_1$ and $Z_2$ are independent random variables that are independent of $(S{V}^{\prime },U^{″}{V}^{\prime }T)$; and their alphabets satisfy

$$\begin{array}{ccc}\hfill |{\mathcal{Z}}_1|& \le \hfill & |\mathcal{S}||{\mathcal{V}}^{\prime }|\left(|\mathcal{T}|-1\right)+1\end{array}$$

(119)

$$\begin{array}{ccc}\hfill |{\mathcal{Z}}_2|& \le \hfill & |{\mathcal{U}}^{″}||{\mathcal{V}}^{\prime }||\mathcal{T}|\left(|\mathcal{X}|-1\right)+1.\end{array}$$

(120)

At the expense of increased alphabets sizes, we now append $Z_1$ to $V^{\prime }$ and $Z_2$ to $U^{″}$ to form the new auxiliary random variables

$$\begin{array}{ccc}\hfill \widehat{V}& =\hfill & \left(V^{\prime }{Z}_1\right)\end{array}$$

(121)

$$\begin{array}{ccc}\hfill \widehat{U}& =\hfill & \left(U^{″}{Z}_2\right)\end{array}$$

(122)

with alphabet sizes

$$\begin{array}{ccc}\hfill |\widehat{\mathcal{V}}|& \le \hfill & |\mathcal{S}||{\mathcal{V}}^{\prime }{|}^2\left(|\mathcal{T}|-1\right)+|{\mathcal{V}}^{\prime }|\end{array}$$

(123)

$$\begin{array}{ccc}\hfill |\widehat{\mathcal{U}}|& \le \hfill & |{\mathcal{U}}^{″}{|}^2|{\mathcal{V}}^{\prime }||\mathcal{T}|\left(|\mathcal{X}|-1\right)+|{\mathcal{U}}^{″}|.\end{array}$$

(124)

We set

$$P_{X|\widehat{U}\widehat{V}T}(x|u^{″},z_2,v^{\prime },z_1,t)=\mathbb{1}\left\{x={\tilde{g}}_2(u^{″},z_2,v^{\prime },t)\right\}$$

(125)

(irrespective of $z_1$) and

$$P_{T|\widehat{V}S}(t|v^{\prime },z_1,t)=\mathbb{1}\left\{t=g_1(s,v^{\prime },z_1)\right\}$$

(126)

where $\mathbb{1}\left\{\mathrm{statement}\right\}$ equals 1 if the statement is true and equals 0 otherwise.

As we next argue, these auxiliary random variables and the above zero–one laws do not decrease the relevant mutual information expressions.

Beginning with $I(\widehat{U};X|\widehat{V}T)$, we note that $H(X|\widehat{V}T)=H(X|V^{\prime }T)$ because we have preserved the joint law of $V^{\prime }T$ and because $Z_1$ does not influence the mapping (54) to X. Since $H(X|U^{″}{Z}_{2}VT)\le H(X|H(X|U^{″}VT)$, this establishes that

$$I(\widehat{U};X|\widehat{V}T)\ge I(U^{″};X|V^{\prime }T).$$

(127)

Likewise, our new auxiliary random variables and zero–one laws do not alter $H\left(Y\right)$, but $H(Y|\widehat{U})\le H(Y|U^{″})$, so

$$I(\widehat{U};Y)\ge I(U^{″};Y).$$

(128)

This completes the proof of Theorem 1.