Upper Bound on the Joint Entropy of Correlated Sources Encoded by Good Lattices

Abstract

Lattices provide useful structure for distributed coding of correlated sources. A common lattice encoder construction is to first round an observed sequence to a ‘fine’ lattice with dither, then produce the result’s modulo to a ‘coarse’ lattice as the encoding. However, such encodings may be jointly-dependent. A class of upper bounds is established on the conditional entropy-rates of such encodings when sources are correlated and Gaussian and the lattices involved are a from an asymptotically-well-behaved sequence. These upper bounds guarantee existence of a joint–compression stage which can increase encoder efficiency. The bounds exploit the property that the amount of possible values for one encoding collapses when conditioned on other sufficiently informative encodings. The bounds are applied to the scenario of communicating through a many-help-one network in the presence of strong correlated Gaussian interferers, and such a joint–compression stage is seen to compensate for some of the inefficiency in certain simple encoder designs.

1. Introduction

Lattice codes are a useful tool for information theoretic analysis of communications networks. Sequences of lattices can be designed to posess certain properties which make them useful for noisy channel coding or source coding in limit with dimension. These properties have been termed ‘good for channel coding’ and ‘good for source coding’ [1]. Sequences posessing both such properties exist, and an arbitrary number of sequences can be nested [2]. One application of ‘good’ sequences of nested lattices is in construction of distributed source codes for Gaussian signals. Well designed codes for such a scenario built off of such lattices enables encoders to produce a more efficient representation of their observations than would be possible without joint code design [3]. Such codes can provide optimal or near-optimal solutions to coding problems [4,5,6]. Despite their demonstrated ability to compress signals well in these cases, literature has identified redundancies across lattice encodings in other contexts [7,8,9,10]. In these cases, further compression of encodings is possible. This paper studies the correlation between lattice encodings of a certain design.

A class of upper bounds on the conditional Shannon entropies between lattice encodings of correlated Gaussian sources is produced by exploiting linear relations between lattice encodings and their underlying signals’ covariances. The key idea behind the analysis is that when the lattice-modulo of one random signal is conditioned on the lattice-modulo of a related signal, the region of feasible points for the first modulo collapses. A sketch of this support reduction is shown in Figure 1. This process is repeated until all information from the conditionals is integrated into the estimate of the support set. The upper bound establishes stronger performance limits for such coding structures since it demonstrates that encoders are able to convey the same encodings at lower messaging rates.

Figure 1. Collapse of the support of a random signal’s modulo after conditioning on the modulo of a related signal. Modulo is shown to some lattice L with base region B. Consider a signal comprised of two independent random components, $\overrightarrow{a}$ and $\overrightarrow{b}$, equaling $\beta \overrightarrow{a}+\overrightarrow{b}$. A possible outcome is drawn on the far left. Unconditioned, the support for $mod(\beta \overrightarrow{a}+\overrightarrow{b})$ is the entire base region B, shown fully shaded in gray. Once $mod\left(\overrightarrow{a}\right)$ is observed, the component $\beta \overrightarrow{a}$ is known up to an additive factor in $\beta L$. If further the powers of $\overrightarrow{a}$ and $\overrightarrow{b}$ are bounded above, this leaves feasible points for $mod(\beta \overrightarrow{a}+\overrightarrow{b})$ as a subset of those of the unconditioned variable. This subset is shaded yellow on the far right.

1.1. Contributions

The following novel contributions are provided:

• A class of upper bounds on conditional entropy-rates of appropriately designed lattice encoded Gaussian signals.
• An application of the bounds to the problem of point-to-point communication through a many-help-one network in the presence of interference. This strategy takes advantage of a specially designed transmitter codebook’s lattice structure.
• A numerical experiment demonstrating the behavior of these bounds. It is seen that a joint–compression stage can partially alleviate inefficiencies in lattice encoder design.

1.2. Background

The redundancy of lattice-modulo-encoded messages has been noticed before, usually in the context of the following many-help-one problem: many `helpers’ observe correlated Gaussian signals and forward messages to a decoder which is interested in recovering a linear combination of said signals. Towards this end, Wagner in [7] provides an upper and lower bound on conditional entropies such as those here for a case with two lattice encodings. Yang in [8] realized a similar compression scheme for such encodings using further lattice processing on them and presents an insightful `coset planes’ abstraction. It was further noticed by Yang in [9] that improvement towards the many-help-one problem is obtained by splitting helper messages into two parts: one part a coarse quantization of the signal, compressed across helpers via Slepian–Wolf joint–compression (these message parts corresponding to the ‘high bit planes’), and another a lattice-modulo-encoding representing signal details (corresponding to ‘low bit planes’). This paper extends these ideas to a general quantity of helpers, and treats a case where a single component of the observations is known to have lattice structure.

Most recently, a joint–compression scheme for lattice encodings called ‘Generalized Compute Compress and Forward’ was introduced in [10], towards coding for a multi-user additive white Gaussian noise channel where a decoder seeks to recover all user’s messages and is informed by helpers. The scheme in [10] makes use of concepts from [9]. In the scheme each lattice message is split into a combination of multiple components, each component from a different coset plane. Design of which coset planes are used yields different performance results. Section 3 in the present work follows along the same lines, although for a network with one user and where many interferers without codebook structure are also present.

Throughout the paper, terminology and basic lattice theory results are taken from [1]. The lattice encoders studied are built from an ensemble of nested lattices, all both ‘good for quantization’ (Rogers-good) and ‘good for coding’ (Poltyrev-good). Such a construction is provided in [2]. An algorithm from [3] is also used which takes as an argument the structure of some lattice modulo encodings and returns linear combinations of the underlying signals recoverable by a certain type of processing on such encodings. This algorithm is listed here as Stages*$\left(·\right)$ and is shown in Appendix A.

1.3. Outline

The main theorem providing upper bounds on conditional entropies of lattice messages, along with an overview of its proof is stated in Section 2. The theorem is slightly strengthened for an application to the problem of communicating over a many-help-one network in Section 3. A numerical analysis of the bounds is given in Section 3.2. A conclusion and discussion on the bound’s remaining inefficiencies is given in Section 4. A table of notation is provided in Table 1. A key for the interpretation of significant named variables is given in Table 2.

Table 1. Symbols and notation.

$a:=b$	Define a to equal b
$\left[n\right]$	Integers from 1 to n
$\mathit{A},\overrightarrow{a},\overrightarrow{A}$	Matrix, column vector, vector, random vector
${\mathit{A}}^{†},{\overrightarrow{a}}^{†}$	Transpose (All matrices involved are real)
${\left[\mathit{A}\right]}_{S,T}$	Submatrix corresponding to rows S, columns T of $\mathit{A}$
${\overrightarrow{Y}}_S$	an $\left|S\right|$-vector, the sub-vector of $\overrightarrow{Y}$ including components with indices in S. If S has order then this vector respects S’s order.
${\mathbf{I}}_K$	$K\times K$ identity matrix
$0_K$	$K\times 1$ zero vector
$diag\overrightarrow{a}$	Square diagonal matrix with diagonals $\overrightarrow{a}$
$pinv(·)$	Moore-Penrose pseudoinverse
$\mathcal{N}(0,\mathsf{\Sigma })$	Normal distribution with zero mean, covariance $\mathsf{\Sigma }$
$X\sim f$	X is a random variable distributed like f
$X^n,f\left(x^n\right)$	Vector of n independent trials of a random variable distributed like X, a function whose input is intended to be such a variable
$var\left(a\right)$	Variance (or covariance matrix) of (components of) a, averaged over time index.
$var\left(a\right|b)$	Conditional variance (or covariance matrix) of (components of) a given observation b, averaged over time index.
$cov(a,b),cov\left(a,b|c\right)$	Covariance between a and $b,$, covariance between a and b conditioned on c, averaged over time index.
$\mathcal{E}\left(a|b\right)$	Linear MMSE estimate of a given observations b
${\mathcal{E}}_{\perp }\left(a|b\right)$	Complement of $\mathcal{E}\left(a|b\right)$, i.e., ${\mathcal{E}}_{\perp }\left(a|b\right):=a-\mathcal{E}\left(a|b\right)$. An important property is that $\mathcal{E}\left(a|b\right)$ and ${\mathcal{E}}_{\perp }\left(a|b\right)$ are uncorrelated.
${round}_L(·),{mod}_L(·)$	Lattice round, modulo to a lattice L (when it is clear what base region is associated with L).

Table 2. Description of variables.

K	Number of lattice encodings in current context.
n	Scheme blocklength
$X_k^n$	Observation at receiver k
$W_k$	Lattice dither k
$U_k$	Lattice encoding k
$Y_k$	Quantization of $X_k^n$
${\overrightarrow{Y}}_c$	Ensemble of lattice quantizations, sans modulo
$\mathsf{\Sigma }$	$K\times K$ time-averaged covariance between observations $X_1^n,\dots ,X_K^n$
${\mathsf{\Sigma }}_Q$	$K\times K$ time-averaged covariance between quantizations $Y_1,\dots ,Y_K$
$r_1,\dots ,r_K$	Nesting ratios for coarse lattice $L_c$ in the fine lattices $L_1,\dots ,L_K$, equivalent to the encoding rates of lattice codes when joint compression is not used
$R_1,\dots ,R_K$	Messaging rates for helpers in the Section 3 communications scenario
$r_{\mathrm{msg}}$	Nesting ratio for codebook coarse lattice $L_{c,\mathrm{msg}}$ in codebook fine lattice $L_{f,\mathrm{msg}}$ in Section 3, equivalent to codebook rate
${\overrightarrow{h}}_{\mathrm{msg}}$	Covariance between codeword and quantizations in Section 3
${\overrightarrow{\alpha }}_s$	Integer combination of ${\overrightarrow{Y}}_c$ to analyze in step s of Appendix B
${\delta }_s^2$	Variance of ${\overrightarrow{\alpha }}_s^{†}{\overrightarrow{Y}}_c$ after removing prior knowledge in Appendix B
${\sigma }_s^2$	Variance of $Y_K$ uncorrelated with prior knowledge and ${\overrightarrow{\alpha }}_s^{†}{\overrightarrow{Y}}_c$ in Appendix B
${\beta }_s$	Regression coefficient for ${\overrightarrow{\alpha }}_s^{†}{\overrightarrow{Y}}_c$ in $Y_K$ after including prior knowledge at step s in Appendix B

2. Main Results

The main results are as follows:

Theorem 1.

For covariance $\mathsf{\Sigma }\in {\mathbb{R}}^{K\times K}$, take ${\overrightarrow{X}}^n=(X_1^n,\dots ,X_K^n)$ to be n independent draws from the joint-distribution $\mathcal{N}(0,\mathsf{\Sigma }).$ Take rates $r_1,\dots ,r_K>0$ and any $\epsilon >0$. If n is large enough, an ensemble of nested lattices $L_c\subset L_1,\dots ,L_K$ (with base regions $B_c\supset B_1,\dots ,B_K$) from [2] (Theorem 1) can be designed so that the following holds. First fix independent dithers $W_k\sim unif{B}_k$. These dithers have $var{W}_k=2^{-2r_k}.$ Also fix $Y_k:={round}_{B_k}(X_k^n+W_k)-W_k$ and lattice modulo encodings $U_k:={mod}_{B_c}\left({round}_{B_k}(X_k^n+W_k)\right)$.

Now for any ${\overrightarrow{\alpha }}_0\in {\mathbb{Z}}^{K-1}$, number $n_0\in \mathbb{N}$, basis $\{{\overrightarrow{\alpha }}_1,\dots ,{\overrightarrow{\alpha }}_K\}\subset {\mathbb{Z}}^K,$ fix variables:

$$\begin{array}{cc}\hfill Y_0& :=Y_K+\frac{1}{n_0}{\overrightarrow{\alpha }}_0^{†}{\overrightarrow{Y}}_{[K-1]},\hfill \\ \hfill {\overrightarrow{Y}}_c& :=(Y_0-Y_K,Y_1,\dots ,Y_{K-1}),\hfill \\ \hfill {\delta }_0^2& :=n_0^2,\hfill \\ \hfill {\sigma }_k^2& :=var\left(Y_0|{\mathrm{S}\mathrm{T}\mathrm{A}\mathrm{G}\mathrm{E}\mathrm{S}}^{*}{\left(var\left({\overrightarrow{Y}}_c|{\left({\overrightarrow{\alpha }}_j^{†}{\overrightarrow{Y}}_c\right)}_{0<j\le k}\right)\right)}^{†}{\overrightarrow{Y}}_c\right),k\in \left\{0\right\}\cup \left[K\right],\hfill \\ \hfill {\delta }_k^2& :=var\left({\overrightarrow{\alpha }}_k^{†}{\overrightarrow{Y}}_c|{\mathrm{S}\mathrm{T}\mathrm{A}\mathrm{G}\mathrm{E}\mathrm{S}}^{*}{\left(var\left({\overrightarrow{Y}}_c|{\left({\overrightarrow{\alpha }}_j^{†}{\overrightarrow{Y}}_c\right)}_{0<j<k}\right)\right)}^{†}{\overrightarrow{Y}}_c\right),k\in \left[K\right].\hfill \end{array}$$

Then the conditional entropy-rate is bounded:

$$\begin{array}{cc}\hfill \frac{1}{n}H\left({\overrightarrow{U}}_K|{\overrightarrow{U}}_{[K-1]},\overrightarrow{W}\right)& \le \underset{k\in \left\{0\right\}\cup \left[K\right]}{min}\left[r_K+\frac{1}{2}log{\sigma }_k^2+\sum _{j=0}^{k}max\{\frac{1}{2}log{\delta }_j^2,0\}\right]+K^2·\epsilon .\hfill \end{array}$$

Bounds of this form hold simultaneously for any subset and reordering of message indices $1,\dots ,K$.

Proof for Theorem 1 is given in Appendix B. The proof is built from [3] (Theorem 1), its associated algorithm ${\mathrm{S}\mathrm{T}\mathrm{A}\mathrm{G}\mathrm{E}\mathrm{S}}^{*}\left(·\right)$ (listed here in Appendix A) and two lemmas which provide useful decompositions of the involved random variables.

Lemma 1.

Take variables as in the statement of Theorem 1. Then, the ensemble of lattices described can include an ‘auxiliary lattice’ ${\widehat{L}}^{\prime }\subset L_K$ with base region ${\widehat{B}}^{\prime }$, nesting ratio $\frac{1}{n}log|{\widehat{B}}^{\prime }\cap L_K|\to \frac{1}{2}log{\sigma }^2+\epsilon$ so that

$$\begin{array}{cc}\hfill U_K& ={mod}_{B_c}\left({\mathcal{C}}_{}+\frac{1}{n_0}\tilde{Y}+{\tilde{Y}}_{\perp }\right),\hfill \end{array}$$

where ${\mathcal{C}}_{},{\mathcal{D}}_{}$ are functions of $({\overrightarrow{U}}_{\left[K\right]},\overrightarrow{W})$, and with high probability

$$\begin{array}{cc}\hfill \tilde{Y}& =-{\overrightarrow{\alpha }}_0^{†}{\overrightarrow{Y}}_{[K-1]}\in ({\mathcal{D}}_{}+L_c),\hfill \end{array}$$

$$\begin{array}{cc}\hfill {\tilde{Y}}_{\perp }& ={\mathcal{E}}_{\perp }\left(Y_0|\overrightarrow{A}\right)\in \widehat{B},\hfill \end{array}$$

$$\begin{array}{cc}\hfill \overrightarrow{A}& ={\mathrm{S}\mathrm{T}\mathrm{A}\mathrm{G}\mathrm{E}\mathrm{S}}^{*}{\left(var{\overrightarrow{Y}}_c\right)}^{†}{\overrightarrow{Y}}_c.\hfill \end{array}$$

In addition, ${\sigma }^2=max\{2^{-2r_K},var{\tilde{Y}}_{\perp }\}.$

Lemma 2.

Take variables as in the statement of Theorem 1. Then, the ensemble of lattices described can include ‘auxiliary lattices’ $\widehat{L}\subset L_c,{\widehat{L}}^{\prime }\subset L_K$ with base regions $\widehat{B},{\widehat{B}}^{\prime }$, nesting ratios $\frac{1}{n}log|\widehat{B}\cap L_c|\to \frac{1}{2}log{\delta }^2+\epsilon ,\frac{1}{n}log|{\widehat{B}}^{\prime }\cap L_K|\to \frac{1}{2}log{\sigma }^2+\epsilon$ so that, for any linear combination Y of ${\overrightarrow{Y}}_{\left[K\right]}$, vector $\overrightarrow{\alpha }\in {\mathbb{Z}}^K$, matrix $\mathit{A}\in {\mathbb{R}}^{*\times K}$ and $\overrightarrow{A}=\mathit{A}{\overrightarrow{Y}}_c$, then

$$\begin{array}{c}\hfill Y={\mathcal{C}}_{}+\beta \tilde{Y}+{\tilde{Y}}_{\perp },\end{array}$$

where ${\mathcal{C}}_{},{\mathcal{D}}_{}$ are functions of $(\overrightarrow{A},{mod}_{n_0{B}_c}\left(Y_0\right),{\overrightarrow{U}}_{\left[K\right]},\overrightarrow{W})$, β is some scalar estimation coefficient, and with high probability

$$\begin{array}{cc}\hfill \tilde{Y}& ={\mathcal{E}}_{\perp }\left({\overrightarrow{\alpha }}^{†}{Y}_c|\overrightarrow{A}\right)\in ({\mathcal{D}}_{}+L_c)\cap \widehat{B},\hfill \end{array}$$

$$\begin{array}{cc}\hfill {\tilde{Y}}_{\perp }& ={\mathcal{E}}_{\perp }\left(Y|\overrightarrow{A},\tilde{Y}\right)\in {\widehat{B}}^{\prime }.\hfill \end{array}$$

In addition, ${\delta }^2=var\tilde{Y}$, ${\sigma }^2=max\{2^{-2r_K},var{\tilde{Y}}_{\perp }\}$.

Proofs for Lemmas 1, 2 are given in Appendix B. These lemmas do not strictly require that the sources be multivariate normal. This technical generalization is relevant in the application to the communication strategy in Section 3. Broadly, the proof of Theorem 1 goes as follows.

• Choose some ${\overrightarrow{\alpha }}_0\in {\mathbb{Z}}^{K-1},n_0\in \mathbb{N}.$ Apply Lemma 1 to $U_K$. Call ${\tilde{Y}}_{\perp }$ a ‘residual.’
• Choose some ${\overrightarrow{\alpha }}_{}\in {\mathbb{Z}}^K.$ Apply Lemma 2 to the residual to break the residual ${\tilde{Y}}_{\perp }$ up into the sum of a lattice part due to ${\overrightarrow{\alpha }}_{}^{†}{\overrightarrow{Y}}_{[K-1]}$ and a new residual, whatever is left over.
• Repeat the previous step until the residual vanishes (up to $K-1$ times). Notice that this process has given several different ways of writing $U_K$; by stopping at any amount of steps, $U_K$ is the modulo sum of several lattice components and a residual.
• Design the lattice ensemble for the encoders such that the log-volume contributed to the support of $U_K$ by each component can be estimated. The discrete parts will each contribute log-volume $\frac{1}{2}log{\delta }^2$ and residuals log-volume $r_K+\frac{1}{2}log{\sigma }^2.$
• Recognize the entropy of $U_K$ is no greater than the log-volume of its support. Choose the lowest support log-volume estimate of those just found.

Notice that each lemma application involves choice of some integer parameters. Choices which yield the strongest bound are unknown. Possible schemes for these decisions are the subroutines Alpha0(·), Alpha(·), listed in Appendix A. As implemented, Alpha0(·) chooses $n_0=1$ and the integer linear combination ${\overrightarrow{\alpha }}_0$ which leaves the least residual. As implemented, $\mathrm{A}\mathrm{L}\mathrm{P}\mathrm{H}\mathrm{A}\left(·\right)$ chooses the integer linear combination ${\overrightarrow{\alpha }}_{}$ for which ${\overrightarrow{\alpha }}_{}^{†}{\overrightarrow{Y}}_{[K-1]}$ is closest to being recoverable from current knowledge at each lemma application. It produces the combination for which the entropy $\frac{1}{2}log{\delta }^2$ of the unknown part of ${\overrightarrow{\alpha }}_{}^{†}{\overrightarrow{Y}}_{[K-1]}$ is minimized. This may be a suboptimal choice since, while such combinations are close to recoverable, they may not be very pertinent to a description of $U_K$. Nonetheless, it is still a good enough rule to produce nontrivial entropy bounds, as seen in Section 3.2.

3. Lattice-Based Strategy for Communication via Decentralized Processing

Consider a scenario where a decoder seeks to decode a message from a single-antenna broadcaster in an additive white Gaussian noise (AWGN). The decoder does not observe a signal directly but instead is provided information by a collection of distributed observers (‘helpers’) which forward it digital information, each observer-to-decoder link supporting a different communications rate. This network is depicted in Figure 2. A block diagram is shown in Figure 3. This is the problem of a single-antenna transmitter communicating to a decoder informed out-of-band by a network of helpers in the presence of additive white Gaussian noise and interference.

Figure 2. High level overview of the communications scenario in Section 3. A transmitter seeks to communicate digital information to a decoder through a Gaussian channel in the presence of Gaussian interference (one interferer drawn). The decoder is informed of the transmitter’s signal through helpers which pass it digital information through an out-of-band local area network (LAN).

Figure 3. Block diagram of the communications scenario. A signal $X_{\mathrm{msg}}$ from a codebook is broadcast through an additive white Gaussian noise (AWGN) channel in the presence of independent Gaussian interference, creating correlated additive noise $(W_1,\dots ,W_K)$. The signal is observed at K receivers labeled $Q_1,\dots ,Q_K$. The k-th reciver observes $X_{k,\mathrm{raw}}$ and processes its observation into a rate-$R_k$ message $U_k$. The messages are forwarded losslessly over a local area side channel to a decoder which attempts to recover the message.

Note that this problem is different from the problem of distributed source coding of a linear function [3,7,8,9,11]. In contrast to the source coding problem, the signal being preserved by the many-help-one network in the present case has a codebook structure. This structure can be exploited to improve the source-to-decoder communications rate. This problem has been studied [12,13], but the best achievable rate is still unknown. In this section, we present a strategy that takes advantage of this codebook structure.

The core of the strategy is to apply a slight modification of Theorem 1 to the network. The transmitter modulates its communications message using a nested lattice codebook such as one in [4]. The helpers employ lattice encoders such as those from Theorem 1, and then perform Slepian–Wolf distributed lossless compression [14] (Theorem 10.3) on their encodings to further reduce their rate. Because the codeword appears as a component of all the helper’s observations, the bound on the message’s joint entropy obtained from Theorem 1 can be strengthened, allowing one to use a more aggressive compression stage.

3.1. Description of the Communication Scheme

It is well known that a nested lattice codebook with dither achieves Shannon information capacity in a point-to-point AWGN channel with a power-constrained transmitter [4]. One interesting aspect of the point-to-point communications scheme described in [4] is that decoding of the noisy signal is done in modulo space. We will see in this section how lattice encodings like those in Theorem 1 can be used to provide such a decoder enough information to recover a communications message.

Without loss of generality, assume that the transmitter is limited to have average transmission power 1. The scheme’s codebook is designed from nested lattices $L_{f,\mathrm{msg}}\supset L_{c,\mathrm{msg}}$ with base regions $B_{f,\mathrm{msg}},B_{c,\mathrm{msg}}$. $L_{f,\mathrm{msg}}$ is chosen to be good for coding and $L_{c,\mathrm{msg}}$ good for quantization. The messaging rate of this codebook is determined by the nesting ratio of $L_{c,\mathrm{msg}}$ in $L_{f,\mathrm{msg}}$:

$$R_{\mathrm{msg}}:=\frac{1}{n}log\left|L_{f,\mathrm{msg}}\cap B_{c,\mathrm{msg}}\right|.$$

Lattices can be designed with nesting ratios such that any rate above zero can be formed. Taking a message $M\in L_{f,\mathrm{msg}}\cap B_{c,\mathrm{msg}}$ and choosing a dither $W_{\mathrm{msg}}\sim -B_{c,\mathrm{msg}}$ of which the decoder is informed, then the codeword associated with M takes the form:

$$\begin{array}{c}\hfill X_{\mathrm{msg}}^n\left(M\right):=\frac{{mod}_{L_{c,\mathrm{msg}}}\left(M+W_{\mathrm{msg}}\right)}{\sqrt{var{W}_{\mathrm{msg}}}}\in \frac{B_{L_{c,\mathrm{msg}}}}{\sqrt{var{W}_{\mathrm{msg}}}}\subset {\mathbb{R}}^n.\end{array}$$

We now describe observations of such a signal by helpers in the presence of AWGN interferers. For covariance ${\mathsf{\Sigma }}_{\mathrm{noise}}\in {\mathbb{R}}^{K\times K}$, take

$${\overrightarrow{X}}_{\mathrm{noise}}^n=(X_{\mathrm{noise},1}^n,\dots ,X_{\mathrm{noise},K}^n)\in {\left({\mathbb{R}}^n\right)}^K$$

to be n independent draws from the joint-distribution $\mathcal{N}(0,{\mathsf{\Sigma }}_{\mathrm{noise}}).$ In addition, take a random vector $X_{\mathrm{msg}}^n$ as described at the beginning of Section 3.1 and a vector ${\mathit{c}}_{\mathrm{msg}}\in {\mathbb{R}}^K$ and define ${\mathsf{\Sigma }}_{\mathrm{msg}}:={\mathit{c}}_{\mathrm{msg}}{\mathit{c}}_{\mathrm{msg}}^{†}$. Now, the k-th helper observes the vector:

$$X_k^n={\left[{\mathit{c}}_{\mathrm{msg}}\right]}_k{X}_{\mathrm{msg}}^n+X_{\mathrm{noise},k}^n\in {\mathbb{R}}^n.$$

Form an observations vector:

$${\overrightarrow{X}}^n:={\mathit{c}}_{\mathrm{msg}}\left(X_{\mathrm{msg}}^n\right)+{\overrightarrow{X}}_{\mathrm{noise}}^n\in {\left({\mathbb{R}}^n\right)}^K,$$

and finally form a cumulative time-averaged covariance matrix as

$$\mathsf{\Sigma }:=var{\overrightarrow{X}}^n={\mathit{c}}_{\mathrm{msg}}{\mathit{c}}_{\mathrm{msg}}^{†}+{\mathsf{\Sigma }}_{\mathrm{noise}}\in {\mathbb{R}}^{K\times K}.$$

If helpers are informed of message dither $W_{\mathrm{msg}}$, then they are informed of the codebook for $X_{\mathrm{msg}}$ and its lattice structure. Using lattice encoders such as those described in Theorem 1, this codebook information can be used to strengthen the upper bound on conditional entropies between the messages.

Theorem 2.

In the context of the channel description given in Section 3.1, entropy bounds identical to those from Theorem 1 hold for its described observer encodings. The bounds also hold re-defining:

$$\begin{array}{cc}\hfill Y_0& :=X_{msg},\hfill \end{array}$$

defining the rest of the variables in the theorem as stated. The bounds also hold instead re-defining:

$$\begin{array}{cc}\hfill Y_c& :=(Y_0-Y_K,Y_1,\dots ,Y_{K-1},X_{src}),\hfill \end{array}$$

vectors $\{{\overrightarrow{\alpha }}_1,\dots ,{\overrightarrow{\alpha }}_{K+1}\}\subset {\mathbb{Z}}^{K+1}$ a basis where all vectors but one ${\overrightarrow{\alpha }}_s,s\in [K+1]$ have 0 as their $(K+1)$-th component and ${\overrightarrow{\alpha }}_s={[0,0,\dots ,0,1]}^{†},$ taking

$$\begin{array}{cc}\hfill {\overrightarrow{a}}_{\mathbb{R}}^{\left(msg\right)}& \in image{\mathrm{S}\mathrm{T}\mathrm{A}\mathrm{G}\mathrm{E}\mathrm{S}}^{*}\left(var\left({\left[{\overrightarrow{Y}}_c\right]}_{\left[K\right]}|{\left({\overrightarrow{\alpha }}_j^{†}{\overrightarrow{Y}}_c\right)}_{0<j<s}\right)\right),\hfill \\ \hfill {\overrightarrow{a}}_{\mathbb{Z}}^{\left(msg\right)}& \in {\mathbb{Z}}^K,\hfill \\ \hfill {\lambda }^{\left(msg\right)}& :=cov(X_{\mathit{msg}}^n,{({\overrightarrow{a}}_{\mathbb{R}}^{\left(msg\right)}+{\overrightarrow{a}}_{\mathbb{Z}}^{\left(msg\right)})}^{†}{\left[{\overrightarrow{Y}}_c\right]}_{\left[K\right]}),\hfill \\ \hfill Y_{\perp }^{\left(msg\right)}& :=\mathcal{E}\left({({\overrightarrow{a}}_{\mathbb{R}}^{\left(msg\right)}+{\overrightarrow{a}}_{\mathbb{Z}}^{\left(msg\right)})}^{†}{\left[{\overrightarrow{Y}}_c\right]}_{\left[K\right]}|X_{\mathit{msg}}^n\right),\hfill \\ \hfill {\delta }_{\left(msg\right)}^2& :={\left(\frac{{\lambda }^{\left(msg\right)}}{{\gamma }_n}-1\right)}^2+var{Y}_{\perp }^{\left(msg\right)},\hfill \\ \hfill {\delta }_s^2& :=max\{1,\frac{{\delta }_{\left(msg\right)}^2}{2^{-2r_{msg}}}+\epsilon \},\hfill \end{array}$$

and taking the rest of the variables in the theorem as stated over range $k\in [K+1].$

A sketch for Theorem 2 is provided in Appendix C. The theorem’s statement can be broadly understood in terms of the proof of Theorem 1. After a number of steps s in the support analysis for Theorem 1, the codebook component $X_{msg}^n$ can be partially decoded yielding tighter estimation of that component’s contribution to the support of $U_K$. The variables ${\lambda }^{\left(msg\right)},{\overrightarrow{a}}_{\mathbb{R}}^{\left(msg\right)},{\overrightarrow{a}}_{\mathbb{Z}}^{\left(msg\right)}$ are parameters for this partial decoding. Lattice modulo messages such as those described in Theorem 2 can be recombined in a useful way:

Lemma 3.

For $\epsilon >0$ and vectors ${\mathbf{a}}_{\mathbb{Z}}\in {\mathbb{Z}}^K$, ${\mathbf{a}}_{\mathbb{R}}\in image{\mathrm{S}\mathrm{T}\mathrm{A}\mathrm{G}\mathrm{E}\mathrm{S}}^{*}\left(\mathsf{\Sigma }\right)\subset {\mathbb{R}}^K$, then lattice modulo encodings ${\overrightarrow{U}}_{\left[K\right]}$ from Theorem 2 can be processed into:

$$U_{\mathit{proc}}:={mod}_{L_{c,\mathit{msg}}}\left(\lambda X_{\mathit{msg}}+Y_{\mathit{noise}}\right),$$

(1)

where $\lambda \in \mathbb{R}$ is some constant:

$$\lambda :=cov\left(X_{\mathit{msg}}^n,{({\mathbf{a}}_{\mathbb{Z}}+{\mathbf{a}}_{\mathbb{R}})}^{†}{\overrightarrow{Y}}_{\left[K\right]}\right)$$

and the noise term $Y_{\mathit{noise}}$ has the following properties:

• ${\sigma }_{\mathit{noise}}^2:=var{Y}_{\mathit{noise}}=var\left({({\mathbf{a}}_{\mathbb{Z}}+{\mathbf{a}}_{\mathbb{R}})}^{†}{\overrightarrow{Y}}_{\left[K\right]}|X_{\mathit{msg}}^n\right),$
• $Y_{\mathit{noise}}\perp (X_{\mathit{msg}},M,W_{\mathit{msg}}),$
• $Y_{\mathit{noise}}$ is with high probability in the base cell of any lattice good for coding semi norm-ergodic noise up to power${\sigma }_{\mathit{noise}}^2+\epsilon$.

Lemma 3 is demonstrated in Appendix D. Notice that Equation (1) is precisely the form of signal processed by the communications decoder described in [4]. The following result summarizes the performance of this communications strategy.

Corollary 1.

Fix a codebook rate $r_{\mathit{msg}}>0.$ As long as helper-to-decoder messaging rates $R_1,\dots ,R_K>0$ satisfy all the following criteria:

$$\forall S\subset \left[K\right],\sum _{k\in S}{R}_k>\tilde{H}\left(S\right|\left[K\right]\backslash S)+\epsilon ,$$

(2)

each $\tilde{H}\left(S\right|\left[K\right]\backslash S)$ being any entropy-rate bound obtained from Theorem 2, then the following communications rate from source to decoder is achievable, taking ${\mathbf{a}}_{\mathbb{Z}},{\mathbf{a}}_{\mathbb{R}},\lambda ,{\sigma }_{\mathit{noise}}^2$ from their definitions in Lemma 3:

$$R_{\mathit{msg}}<min\left\{r_{\mathit{msg}},\underset{{\mathbf{a}}_{\mathbb{Z}},{\mathbf{a}}_{\mathbb{R}}}{sup}\underset{{\gamma }^2\in (0,1]}{max}\frac{1}{2}log\left[\frac{{\gamma }^2}{{(\lambda -\gamma )}^2+{\sigma }_{\mathit{noise}}^2}\right]\right\}.$$

(3)

Proof for Corollary 1 is given in Appendix E, and evaluation of the achieved communications rates for certain lattice code designs is shown in Section 3.2.

3.2. Numerical Results

The achievable rate given in Corollary 1 depends on the design of the lattice encoding scheme at the helpers. Identification of the best such lattice encoders for such a system is closely tied to a receivers’ covariance structure [3]. For this reason and for the purpose of evaluating the effect of joint compression stage, we restrict our attention to a particular channel structure and lattice encoder design.

The line-of-sight configuration shown in Figure 4 is considered. It yields helper observations with the following covariance structure, labeling interferer signals in Figure 4 from top to bottom as $(W_{I1},W_{I2},W_{I3})$ and indexing helpers from top to bottom:

$$\begin{array}{cc}\hfill X_{raw,1}& =\frac{\sqrt{P_S}}{\parallel 1+\left(\frac{2}{3}\right)e^{i\pi ·1/2}\parallel }{X}_{msg}+W_1+\dots \hfill \\ & +\frac{\sqrt{P_I}}{\parallel \left(\frac{2}{3}\right)(e^{i\pi ·1/2}-e^{i\pi ·2/3})\parallel }{W}_{I1}+\frac{\sqrt{P_I}}{\parallel \left(\frac{2}{3}\right)(e^{i\pi ·1/2}-e^{i\pi ·1})\parallel }{W}_{I2}+\frac{\sqrt{P_I}}{\parallel \left(\frac{2}{3}\right)(e^{i\pi ·1/2}-e^{i\pi ·4/3})\parallel }{W}_{I3},\hfill \\ \hfill X_{raw,2}& =\frac{\sqrt{P_S}}{\parallel 1+\left(\frac{2}{3}\right)e^{i\pi ·5/6}\parallel }{X}_{msg}+W_2+\dots \hfill \\ & +\frac{\sqrt{P_I}}{\parallel \left(\frac{2}{3}\right)(e^{i\pi ·5/6}-e^{i\pi ·2/3})\parallel }{W}_{I1}+\frac{\sqrt{P_I}}{\parallel \left(\frac{2}{3}\right)(e^{i\pi ·5/6}-e^{i\pi ·1})\parallel }{W}_{I2}+\frac{\sqrt{P_I}}{\parallel \left(\frac{2}{3}\right)(e^{i\pi ·5/6}-e^{i\pi ·4/3})\parallel }{W}_{I3},\hfill \\ \hfill X_{raw,3}& =\frac{\sqrt{P_S}}{\parallel 1+\left(\frac{2}{3}\right)e^{i\pi ·7/6}\parallel }{X}_{msg}+W_3+\dots \hfill \\ & +\frac{\sqrt{P_I}}{\parallel \left(\frac{2}{3}\right)(e^{i\pi ·7/6}-e^{i\pi ·2/3})\parallel }{W}_{I1}+\frac{\sqrt{P_I}}{\parallel \left(\frac{2}{3}\right)(e^{i\pi ·7/6}-e^{i\pi ·1})\parallel }{W}_{I2}+\frac{\sqrt{P_I}}{\parallel \left(\frac{2}{3}\right)(e^{i\pi ·7/6}-e^{i\pi ·4/3})\parallel }{W}_{I3},\hfill \\ \hfill X_{raw,4}& =\frac{\sqrt{P_S}}{\parallel 1+\left(\frac{2}{3}\right)e^{i\pi ·3/2}\parallel }{X}_{msg}+W_4+\dots \hfill \\ & +\frac{\sqrt{P_I}}{\parallel \left(\frac{2}{3}\right)(e^{i\pi ·3/2}-e^{i\pi ·2/3})\parallel }{W}_{I1}+\frac{\sqrt{P_I}}{\parallel \left(\frac{2}{3}\right)(e^{i\pi ·3/2}-e^{i\pi ·1})\parallel }{W}_{I2}+\frac{\sqrt{P_I}}{\parallel \left(\frac{2}{3}\right)(e^{i\pi ·3/2}-e^{i\pi ·4/3})\parallel }{W}_{I3},\hfill \\ \hfill W_k& \sim \mathcal{N}(0,1)\mathrm{i}.\mathrm{i}.\mathrm{d}.\hfill \end{array}$$

(4)

where $P_S,P_I>0$ are signal, interferer powers, respectively. Choice of this channel is arbitrary but provides an instance where the decoder would not be able to recover the signal of interest if it observed directly without the provided helper messages.

Figure 4. The line-of-sight channel considered. A black transmit node at $(0,0)$ seeks to communicate with a black decoder node at $(1,0)$. Three red ‘interferer’ nodes broadcast an independent Gaussian signal, each interferer has its own signal. The decoder does not observe any signal directly but is forwarded messages from four blue ‘helper’ nodes which observe signals through a line-of-sight additive-white-Gaussian noise channel. The interferers and helpers are oriented alternatingly and equispaced about a radius-$2/3$ semicircle towards the encoder with center $(1,0)$.

3.2.1. Communications Schemes

First, we describe a class of lattice encoders the four helpers could employ:

• Fix some $c\in (0,3)$. If helper $k\in \left[4\right]$ in the channel from Figure 4 observes $X_{\mathrm{raw},k}^n$, then it encodes a normalized version of the signal:

  $$X_k^n:=\frac{c}{\sqrt{var{X}_{\mathrm{raw},k}^n}}{X}_{\mathrm{raw},k}^n.$$
• Fix equal lattice encoding rates per helper $r=r_1=r_2=r_3=r_4$, and take lattice encoders as described in Theorem 1. Note that these rates may be distinct from the helper-to-base rates $R_1,\dots ,R_4$ if post-processing of the encodings is involved.

Communications schemes involving lattice encoders of this form are compared in Figure 5 over an ensemble of choices for lattice encoder rates r and scales $c\in (0,3).$ Achieved transmitter-to-decoder communication rate versus sum-rate from helpers to decoder are plotted. The following quantities are plotted:

Figure 5. Communications rate versus helper-sum-rate for 1000 randomly chosen encoding schemes as described in Section 3.2.1 in the line-of-sight channel from Figure 4, Equation (4). In each subplot, the transmitter broadcasts with power such that the average SNR seen across helpers is the given ‘transmitter’ dB figure. Each interferer broadcasts its own signal with its power the given ‘interferer’ dB stronger than the transmitter’s power. Notice that, although the uncompressed lattice scheme is often outperformed by plain Quantize & Forward for the same helper message rates, adding a properly configured compression stage can more than make up for the sum-rate difference. In certain regimes, even the compressed lattice scheme performs worse or practically the same as Quantize & Forward, indicating the given lattice encoder design is weak; uncompressed lattice encoders can be configured to implement the Quantize & Forward scheme.

• Upper Bound: An upper bound on the achievable transmitter-to-decoder communications rate, corresponding to helpers which forward with infinite rate. This bound is given by the formula $I(X_{msg};{\left(X_{\mathrm{raw},k}\right)}_{k\in \left[4\right]})$.
• Corollary 1 The achievable communications rate from Corollary 1, where each helper computes the lattice encoding described above, then employs a joint–compression stage to reduce its messaging rate. The sum-helpers-to-decoder rate for this scheme is given by Equation (2), taking $S=\left[4\right].$ The achieved messaging rate is given by the right-hand-side of Equation (3).
• Uncompressed Lattice: The achievable communications rate from Corollary 1, with each helper forwarding to the decoder its entire lattice encoding without joint–compression. The sum-helpers-to-decoder rate for this scheme is $4r$ since in this scheme each helper forwards to the base at rate $R_k=r$. The achieved messaging rate is given by the right-hand-side of Equation (3).
• Quantize & Forward: An achievable communications rate where helper-to-decoder rates $R_k,k\in \left[4\right]$ are chosen so that $R_1+R_2+R_3+R_4=R_{sum}$ and each helper forwards a rate-distortion-optimal quantization of its observation to the decoder. The decoder processes these quantizations into an estimate of $X_{\mathrm{msg}}$ and decodes. This is discussed in more detail in [13]. The sum-helpers-to-decoder rate for this scheme is $R_{sum}$. The achieved messaging rate is $I(X_{msg};{(X_{raw,k}+Z_k)}_{k\in \left[4\right]})$, where $Z_k\sim \mathcal{N}(0,var\left(X_{raw,k}\right)·2^{-2R_k})$.

Performance of these strategies for different broadcaster powers is shown in Figure 5. It is seen that, although the lattice encoder designs are poor, the joint–compression stage partially compensates for this, and with joint compression the scheme outperforms the plain ‘Quantize & Forward’ scheme. Notice that none of the strategies produce convex rate regions, indicating that time-sharing can be used to achieve better rates in some regimes.

In all figures shown, the gap between achieved rates from the joint–compression bound given from Theorem 1 and Theorem 2 (the latter being an improvement) were often nonzero but too small to noticeably change the graphs in Figure 5. For this reason only, achievable rates for the strategy from Corollary 1 are plotted. The gain from involving codebook knowledge in lattice encoding compression is either insignificant for the tested scenario, or choices in computing the upper bounds are too poor to reveal its performance gains. Sub-optimality of the algorithm implementations here are all summarized and discussed in Section 4.

4. Conclusions

A class of upper bounds on the joint entropy of lattice-modulo encodings of correlated Gaussian signals was presented in Theorem 1. Proof of these bounds involves reducing the problem to the entropy of one lattice message, say, $U_K$ conditioned on the rest, ${\overrightarrow{U}}_{[K-1]}$. The upper bound for this reduced case involves an iterative construction where in each step a suitable integer vector is chosen. Choice of vectors in these steps determines the order in which the observed lattice-modulo components are integrated into an estimate of $U_K$’s support. Different choice of vectors at each step yields a different bound, and the strongest sequence of choices is unknown. For numerical results in Section 3.2, a certain suboptimal was used although there is no guarantee that this choice is optimal.

The upper bounds were applied to the problem of communicating through a many-help-one network, and these bounds were evaluated for a rendition of the problem using lattice codes of simple structure. The bounds in Theorem 1 can be strengthened in this scenario by integrating codebook knowledge. This strengthening is described in Theorem 2.

In spite of the suboptimal lattice encoder designs analyzed, it was seen in Section 3.2 that jointly-compressed lattice encoders are able to significantly outperform more basic schemes in the presence of heavy interference, even when the joint compression stage uses the weaker entropy bounds from Theorem 1. In the numerical experiments tried, the strengthening in Theorem 2 was not seen to significantly improve compression. Whether this is typically true or just an artifact of poor design of the joint-compression stage is unknown. In either case, the simpler joint-compression strategy without codebook knowledge was seen to improve performance.

The most immediate forwards steps to the presented results is in characterization of the search problems posed by Theorems 1, 2. Although not discussed, corner-points of joint compression described here are implementable using further lattice processing on the encodings $U_1,\dots ,U_K$ and their dithers $\overrightarrow{W}$. Such a process might mimic the compression procedure described in [10]. Tightness arguments from this work may also apply to the present less structured channel.

Finally, according to the transmission method in [10], the achievable rate in Corollary 1 may be improvable by breaking the transmitter’s message M up into a sum of multiple components, each from a finer lattice. Joint–compression for such a transmission could integrate codebook information from each message component separately, allowing for more degrees of freedom in the compression stage’s design, possibly improving the achievable rate. This is an extension of the argument in Appendix D. These improvements are out of this paper’s scope but provide meaningful paths forward.

Appendix B.1. Upper Bound for Singleton S

Take a nested lattice construction from [20] (Theorem 1), involving the following sets:

• Coarse and fine encoding lattices$L_c,L_1,\dots ,L_K$ (base regions $B_c,B_1,\dots ,B_K$) with each k has $L_c\subset L_k$ designed with nesting ratio $\frac{1}{n}log|B_c\cap L_k|\to r_k$.
• Discrete part auxiliary lattices${\widehat{L}}_1,\dots ,{\widehat{L}}_K$ (base regions ${\widehat{B}}_1,\dots ,{\widehat{B}}_K$) with each ${\widehat{L}}_k\subset L_c$ having nesting ratio $\frac{1}{n}log|B_c\cap {\widehat{L}}_k|\to \frac{1}{2}log{\delta }_k^2$.
• Initial residual part auxiliary lattice${\widehat{L}}_0^{\prime }$ (base region ${\widehat{B}}_0^{\prime }$) with ${\widehat{L}}_0^{\prime }\subset L_K$, nesting ratio $\frac{1}{n}log|{\widehat{B}}_0^{\prime }\cap L_K|\to \frac{1}{2}log{\sigma }_0^2.$
• Residual part auxiliary lattices${\widehat{L}}_1^{\prime },\dots ,{\widehat{L}}_K^{\prime }$ (base regions ${\widehat{B}}_1^{\prime },\dots ,{\widehat{B}}_K^{\prime }$) with each ${\widehat{L}}_k^{\prime }\subset L_K$, having nesting ratio $\frac{1}{n}log|{\widehat{B}}_k^{\prime }\cap L_K|\to \frac{1}{2}log{\sigma }_k^2$.

The specified nesting ratios for the auxiliary lattices, ${\sigma }_0^2,{\sigma }_1^2,\dots ,{\sigma }_K^2,{\delta }_1^2,\dots ,{\delta }_K^2$ will be specified later.

Initialization 

Apply Lemma 1 to $U_K$, and label the resulting variables ${\tilde{Y}}_0:=\tilde{Y},{\tilde{Y}}_{0\perp }:={\tilde{Y}}_{\perp },({\widehat{L}}_0^{\prime },{\widehat{B}}_0^{\prime }):=({\widehat{L}}^{\prime },{\widehat{B}}^{\prime }),{\mathcal{D}}_0:=n_0{\mathcal{D}}_{},{\mathcal{C}}_0:={\mathcal{C}}_{},{\sigma }_0^2:={\sigma }^2.$ In addition, define ${\delta }_0^2:=n_0^2,{\beta }_0=\frac{1}{n_0},{\widehat{B}}_0:=n_0{B}_c$ Now,

$$U_K={mod}_{B_c}\left({\mathcal{C}}_0+{\beta }_0{\tilde{Y}}_0+{\tilde{Y}}_{0\perp }\right)$$

so the support of $U_K$ is contained within:

$$\begin{array}{cc}\hfill {\mathcal{S}}_0:=& [{\mathcal{C}}_0+{\widehat{B}}_0^{\prime }+(L_c/n_0+{\mathcal{D}}_0)]\cap (B_c\cap L_K)\hfill \\ \hfill =& [{\mathcal{C}}_0+{\widehat{B}}_0^{\prime }+{\beta }_0[({\mathcal{D}}_0+L_c)\cap {\widehat{B}}_0]\cap (B_c\cap L_K).\hfill \end{array}$$

Support Reduction

Iterate over steps $s=1,\dots ,K$. For step s, condition on any event of the form ${\tilde{Y}}_{(s-1)}={\ell }_s\in ({\mathcal{D}}_{s-1}+L_c)\cap {\widehat{B}}_{s-1}$, of which there are no more than $2^{n·(log\left({\delta }_{s-1}^2\right)+\epsilon )}$ choices due to the nesting ratio for ${\widehat{L}}_{s-1}$ in $L_c$. Take ${\mathit{A}}_s:={\mathrm{S}\mathrm{T}\mathrm{A}\mathrm{G}\mathrm{E}\mathrm{S}}^{*}\left(var\left({\overrightarrow{Y}}_c|{[{\overrightarrow{\alpha }}_0,\dots ,{\overrightarrow{\alpha }}_{s-1}]}^{†}{\overrightarrow{Y}}_c\right)\right).$ By [3] (Theorem 1), ${\overrightarrow{A}}_s:={\mathit{A}}_s^{†}{\overrightarrow{Y}}_c$ is recoverable by processing $({\overrightarrow{A}}_{s-1},{mod}_{n_0{B}_c}{Y}_0{\overrightarrow{U}}_{\left[K\right]},\overrightarrow{W})$.

Now, apply Lemma 2 to $(Y,{\overrightarrow{\alpha }}_{},\mathit{A})=({\tilde{Y}}_{(s-1)\perp },{\overrightarrow{\alpha }}_s,{\mathit{A}}_s)$, and label the resulting variables ${\tilde{Y}}_s:=\tilde{Y},{\tilde{Y}}_{s\perp }:={\tilde{Y}}_{\perp },({\widehat{L}}_s,{\widehat{B}}_s):=(\widehat{L},\widehat{B}),({\widehat{L}}_s^{\prime },{\widehat{B}}_s^{\prime }):=({\widehat{L}}^{\prime },{\widehat{B}}^{\prime }),{\mathcal{D}}_s:={\mathcal{D}}_{},{\mathcal{C}}_s:={\mathcal{C}}_{},{\beta }_s:=\beta ,{\sigma }_s^2:={\sigma }^2,{\delta }_s^2:={\delta }^2.$ Now,

$$U_K={mod}_{B_c}\left(\left[\sum _{t=0}^s{\mathcal{C}}_t+{\beta }_t{\tilde{Y}}_t\right]+{\tilde{Y}}_{s\perp }\right)$$

so the support of $U_K$ is contained within:

$${\mathcal{S}}_s:=\left[\left[\sum _{t=0}^s{\mathcal{C}}_t+{\beta }_t\left[({\mathcal{D}}_t+L_c)\cap {\widehat{B}}_t\right]\right]+{\widehat{B}}_s^{\prime }\right]\cap (B_c\cap L_K).$$

Count Points in Estimated Supports

By design, there are no more than ${\prod }_{t=0}^s{2}^{n·(\frac{1}{2}log\left({\delta }_t^2\right)+\epsilon )}$ possible choices for ${\mathcal{S}}_s.$ Each ${\mathcal{S}}_s$ has no more than $|{\widehat{B}}_s^{\prime }\cap (B_c\cap L_K)|\le 2^{n·(r_K+\frac{1}{2}log\left({\sigma }_s^2\right)+\epsilon )}$ points. Then,

$$H\left(U_K\right|{\overrightarrow{U}}_{[K-1]},\overrightarrow{W})\le \underset{s\in \left\{0\right\}\cup \left[K\right]}{min}n·\left(r_s+\frac{1}{2}log\left({\sigma }_s^2\right)+\sum _{t=0}^s\frac{1}{2}log\left({\delta }_t^2\right)+K\epsilon \right).$$

Bound Simultaneity

An argument is given in Section B.1 for an upper bound on the singleton case. The argument uses a Zamir-good nested lattice construction with a finite amount of nesting criteria, and conditions on a finite amount of high-probability events. Then, the argument holds for all cases of this form simultaneously by using a Zamir-good nested lattice construction satisfying all of each case’s nesting criteria and conditioning on all of each case’s high-probability events.

The entropy for the general case $S=\{s_1,\dots ,s_{\left|S\right|}\},T=\{t_1,\dots ,t_{\left|T\right|}\}$ can be rewritten using the chain rule:

$$\begin{array}{cc}\hfill H\left({\overrightarrow{U}}_S|{\overrightarrow{U}}_T,\overrightarrow{W}\right)& =\sum _{p=1}^{\left|S\right|}H\left({\overrightarrow{U}}_{s_p}|{\overrightarrow{U}}_{\{s_m:m<p\}\cup T},\overrightarrow{W}\right).\square \hfill \end{array}$$