Distributed Quantization for Partially Cooperating Sensors Using the Information Bottleneck Method

Abstract

This paper addresses the optimization of distributed compression in a sensor network with partial cooperation among sensors. The widely known Chief Executive Officer (CEO) problem, where each sensor has to compress its measurements locally in order to forward them over capacity limited links to a common receiver is extended by allowing sensors to mutually communicate. This extension comes along with modified statistical dependencies among involved random variables compared to the original CEO problem, such that well-known outer and inner bounds do not hold anymore. Three different inter-sensor communication protocols are investigated. The successive broadcast approach allows each sensor to exploit instantaneous side-information of all previously transmitting sensors. As this leads to dimensionality problems for larger networks, a sequential point-to-point communication scheme is considered forwarding instantaneous side-information to only one successor. Thirdly, a two-phase transmission protocol separates the information exchange between sensors and the communication with the common receiver. Inspired by algorithmic solutions for the original CEO problem, the sensors are optimized in a greedy manner. It turns out that partial communication among sensors improves the performance significantly. In particular, the two-phase transmission can reach the performance of a fully cooperative CEO scenario, where each sensor has access to all measurements and the knowledge about all channel conditions. Moreover, exchanging instantaneous side-information increases the robustness against bad Wyner–Ziv coding strategies, which can lead to significant performance losses in the original CEO problem.

1. Introduction

This contribution considers a special case of the distributed source coding problem where each sensor observes the same source signal. In order to forward their measurements over capacity limited links to a common receiver, the sensors have to compress their measurements. In the case where a direct communication among operating sensors is not possible, this problem is termed as the CEO problem. Here, the compression at each sensor is optimized according to the Wyner–Ziv coding principle exploiting only statistical side-information. Within this paper, the term CEO problem always stands for this non-cooperative CEO problem, which means that sensors cannot communicate with each other during runtime.

We extend this scenario and allow sensors to cooperate with each other by exchanging instantaneous side-information. The fully cooperative Chief Executive Officer (fcCEO) problem is obtained if sensors can forward their uncompressed observations over inter-sensor links to all other sensors. The partially cooperative Chief Executive Officer (pcCEO) problem represents a scenario where instantaneous side-information is compressed before it is forwarded to other sensors.

1.1. The CEO Problem

The CEO problem has been investigated for various system assumptions which mainly differ in the distribution of the variable of interest and the applied distortion measure. The quadratic Gaussian CEO problem considers jointly Gaussian signals and the mean squared error (MSE) distortion measure [1,2,3,4,5,6]. Using an infinite number of encoders, Oohama analytically derived an asymptotic version of the sum-rate distortion in [1]. In [2], the authors investigated the influence of cooperating and non-cooperating encoders on the distortion measure. It turns out that the distortion decreases asymptotically with the reciprocal sum-rate R for non-cooperating encoder. In scenarios with cooperating sensors, the MSE distortion decays exponentially with $2^{-2R}$. The non-asymptotic case was first investigated in [3] where the authors derived an upper bound on the sum-rate distortion function. Moreover, they showed that this bound is tight when each encoder has the same measurement SNR. Prabhakaran et al. [4], Oohama [5] and Wagner et al. [6] characterized the complete rate-region for this quadratic Gaussian CEO problem. Results also exist for multivariate Gaussian relevant processes [7,8]. The CEO problem for arbitrary discrete source distributions and the logarithmic loss distortion measure has been analyzed in [9,10,11]. For this scenario, Courtade and Weissman completely characterized the CEO rate-region in [9]. In [10,11], asymptotic analyses for an infinite number of sensors have been performed. Using the Hamming distance as a distortion measure, in [10] an inevitable loss in error rate performance due to non-cooperating sensors has been discovered. For arbitrary distortion measures, a scaling law on the sum-rate distortion function has been derived in [11]. In [12,13], the authors developed a variational bound on the optimal trade-off between relevance and complexity of the CEO setup and used neural networks for encoder and decoder to compute this bound.

There exist several algorithmic approaches to solve the CEO problem [8,14,15,16,17]. In our previous work [14,15], a Greedy Distributed Information Bottleneck (GDIB) algorithm based on the inner bound of the CEO problem with log-loss distortion measure defined in [9] has been introduced to determine extreme points of the contra-polymatroid solution space. Equivalently to Wyner–Ziv coding, the algorithm optimizes the quantizer of a specific sensor using the mappings of previously designed quantizers as statistical side-information. We showed that the GDIB algorithm outperforms an individual scalar information bottleneck optimization of sensors without Wyner–Ziv coding, especially for larger networks. For asymmetric scenarios, we demonstrated the dependency of the performance on the optimization order, i.e., the Wyner–Ziv coding strategy. Since the memory complexity of this optimization algorithm depends exponentially on the network size, we introduced a way to compress the Wyner–Ziv side-information by means of the information bottleneck principle. However, there still remains a large performance gap between non-cooperative and fully cooperative distributed compression.

1.2. Partially Cooperating Sensors

In order to close the gap between non-cooperative and fully cooperative distributed compression, we consider an extension of the CEO system model allowing partial cooperation among sensors. While a rich literature can be found on the classical CEO problem, less results are known for partially cooperating sensors. Most work has been done for jointly Gaussian signals because they allow an analytical treatment at least in parts. In [18], it is shown that cooperation among sensors can reduce the compression sum-rate except for the quadratic Gaussian CEO problem. In [19], the authors consider estimation problems under communication constraints and propose coding strategies for tree-structured sensor networks. Exploiting the Wyner–Ziv coding principle, the authors developed solutions for general trees and provide particular results for serial and parallel networks. A two sensor system with a Gaussian source was investigated in [20] for two transmission scenarios: orthogonal but rate-limited links between sensors and the common receiver as well as the Gaussian multiple access channel. It was shown that cooperation between the sensors over rate-limited inter-sensor links leads in both cases to substantial gains in terms of the compression sum-rate. Finally, a simple three node network consisting of encoder, helper and decoder was analyzed in [21]. The authors showed that side-information provided by the helper to the encoder need not exceed that given to the decoder.

1.3. Structure and Notation

This paper is structured as follows: Section 2 gives a short introduction to the information bottleneck principle. Section 3 and Section 4 introduce the non-cooperative distributed sensing scenario with the Greedy Distributed Information Bottleneck as an algorithmic solution defined in [14,15] and the fully cooperative distributed sensing scenario being equivalent to a centralized quantization approach, respectively. The main contribution of this paper can be found in Section 5, which introduces the partially cooperative distributed sensing scenario containing three different inter-sensor communication protocols, i.e., successive broadcasting, successive point to point transmission and two-phase transmission. Section 6 concludes this paper.

Throughout this paper, the following notation is used. Calligraphic letters $𝒳,𝒴,Ƶ$ denote random variables with realizations $x,y,z$, which are elements of the sets $𝕏,𝕐,\mathbb{Z}$ with cardinalities $\left|𝕏\right|$, $\left|𝕐\right|$ and $|\mathbb{Z}|$, respectively. Bold letters $\mathbf{y}={[y_1\dots y_M]}^{\mathrm{T}}$ denote vectors while boldface calligraphic letters $\mathit{𝓨},\mathit{Ƶ}$ denote multivariate random variables. Note that ${\mathit{Ƶ}}_{<m}$ covers only the processes ${Ƶ}_1$ to ${Ƶ}_{m-1}$. $I(𝒳;𝒴)$ represents the mutual information between the random variables $𝒳$ and $𝒴$. Conditional and joint probability mass functions (pmfs) are termed $p\left(y\right|x)$ and $p(x,y)$, respectively. The Kullback-Leibler (KL) divergence is given as $D_{\mathrm{K}\mathrm{L}}[·\parallel ·]$. Finally, the expectation of a function $f\left(x\right)$ with respect to the random variable $𝒳$ is denoted as ${𝔼}_{𝒳}[f\left(𝒳\right)]$.

2. The Information Bottleneck Principle

The information bottleneck (IB) principle was first introduced by Tishby et al. in [22,23] and defines a clustering framework based on information theoretic measures. An overview about algorithmic solutions for this basic optimization problem is given in [23,24]. The IB principle finds application in various fields in communications [25,26,27,28,29,30].

The general IB setup is depicted in Figure 1. It contains the relevant process $𝒳$, a noisy observation $𝒴$ of $𝒳$ and a compressed version Ƶ of $𝒴$. The IB approach aims to optimize the mapping $p\left(z\right|y)$ in order to preserve as much information about the relevant process $𝒳$ in Ƶ as possible. More precisely, it tries to maximize the relevant mutual information $I(𝒳;Ƶ)$ while fulfilling a rate constraint $I(𝒴;Ƶ)\le C$. This general goal is summarized in Figure 2a. The optimization can be formulated as a maximization of the Lagrangian function

$$\begin{array}{cc}\hfill L_{\mathrm{IB}}& =I(𝒳;Ƶ)-\beta I(𝒴;Ƶ).\hfill \end{array}$$

(1)

It turns out to be a non-convex optimization problem, since $I(𝒳;Ƶ)$ and $I(𝒴;Ƶ)$ are both convex functions of the mapping $p\left(z\right|y)$. The parameter $\beta$ is a trade-off parameter steering the focus between the preservation of relevant information and the compression of the observation. In the case of $\beta =0$, the focus only lies on preservation of relevant information. By increasing $\beta$ the compression becomes more and more important up to the case of $\beta \to \infty$. Here, the functional in (1) becomes maximal if $I(𝒴;Ƶ)=0$, which means that all information is compressed to a single cluster. Therefore, the parameter $\beta$ can be used to adjust the compression rate $I(𝒴;Ƶ)$ in order to fulfill a desired rate constraint $I(𝒴;Ƶ)\le C$. Since the compression-rate curve is a monotonic increasing function in $\frac{1}{\beta }$, a simple bi-section search can be applied. The optimization problem in (1) can be solved by taking the derivative with respect to the mapping $p\left(z\right|y)$ and equating it to zero. It results in the implicit update equation

$$p\left(z\right|y)=\frac{e^{-d_{\beta }(y,z)}}{{\sum }_z{e}^{-d_{\beta }(y,z)}}$$

(2)

with

$$\begin{array}{cc}\hfill d_{\beta }(y,z)& =\frac{1}{\beta }{D}_{\mathrm{K}\mathrm{L}}[p\left(x\right|y)\parallel p(x\left|z\right)]-logp\left(z\right)\hfill \\ \hfill & =\frac{1}{\beta }{𝔼}_{𝒳|y}[log\frac{p\left(x\right|y)}{p\left(x\right|z)}]-logp\left(z\right).\hfill \end{array}$$

(3)

In (3), $D_{\mathrm{K}\mathrm{L}}[p\left(x\right|y)\parallel p(x\left|z\right)]$ denotes the Kullback–Leibler divergence. This implicit solution can be solved by an iterative Blahut–Arimoto-like algorithm.

In the case of focusing solely on preservation of relevant information with $\beta =0$, the optimization algorithm yields a deterministic clustering $p\left(z\right|y)\in \{0,1\}$. For $\beta >0$, the clustering $p\left(z\right|y)\in [0,1]$ is generally stochastic. The IB method can easily be extended to multiple input values. A graphical tool for visualization are IB graphs [31]. Figure 2b illustrates an example where the observations $y_1,y_2$ are compressed into the cluster index z. The trapezoid represents the IB compression with respect to the relevant variable written inside the trapezoid.

3. Non-Cooperative Distributed Sensing System

Figure 3 illustrates the CEO system model without communication among operating sensors. Here, M sensors observe noisy versions $y_m$ of the same relevant signal x. The measurement processes can be modeled as statistically independent memoryless channels (MCs).

Exemplarily, a measurement $y_m=x+w_m$ represents the relevant signal x corrupted by zero mean white Gaussian measurement noise $w_m$ with measurement signal-to-noise-ratio (SNR) ${\gamma }_m=\frac{{\sigma }_x^2}{{\sigma }_{w_m}^2}$, where ${\sigma }_x^2$, ${\sigma }_{w_m}^2$ denote signal and noise variances, respectively. In order to be able to forward the measurements over capacity limited links with capacities $C_1,\dots ,C_M$, each sensor has to compress its observations using a specific encoding process. More precisely, each sensor compresses its observations $y_m$ to a cluster index $z_m$ using the mapping $p\left(z_m\right|y_m)$ leading to the Markov property:

$$\begin{array}{c}\hfill p(\mathbf{z},\mathbf{y},x)=\prod _{m=1}^{M}p\left(z_m\right|y_m)p\left(y_m\right|x)p\left(x\right).\end{array}$$

(4)

The encoding process contains a second lossy compression step if the mapping $p\left(z_m\right|y_m)$ is stochastic and lossless entropy coding if the mapping $p\left(z_m\right|y_m)$ is deterministic. Therefore, a compressed version of the index $z_m$ is transmitted without any further loss to the common receiver. The optimization of $p\left(z_m\right|y_m)$ for each sensor is done offline.

The mathematical analysis of the CEO problem and the structure of its rate-region for discrete input alphabets and the log-loss distortion measure was presented in [9] and exploits (4). It was proved that the extreme points of the contra-polymatroid solution space can be determined by greedy algorithms as the one described next. Since the communication among sensors during run-time is not possible in this approach, the solution represents a lower bound on the performance of cooperative distributed compression in this paper.

An algorithmic solution to solve the CEO problem has previously been proposed in [14,15] as the so called Greedy Distributed Information Bottleneck (GDIB) algorithm. It is based on the inner bound of the CEO rate-region for the logarithmic loss distortion measure [9] and optimizes the quantization at the sensors successively. Replacing the logarithmic loss function $H\left(𝒳\right|\mathit{Ƶ})$ by the relevant mutual information $I(𝒳;\mathit{Ƶ})$ delivers the optimization problem

$$\begin{array}{c}\hfill \underset{\mathbf{P}}{max}I(𝒳;\mathit{Ƶ})\mathrm{s}.\mathrm{t}.I({\mathit{𝓨}}_{𝕊};{\mathit{Ƶ}}_{𝕊}|{\mathit{Ƶ}}_{\overline{𝕊}})\le \sum _{m\in 𝕊}{C}_m\\ \hfill \forall 𝕊\subseteq \{1,2,\dots ,M\}.\end{array}$$

(5)

The set $\mathbf{P}= \left[\begin{array}{ccc}p\left(z_1\right|y_1)& \cdots & p\left(z_M\right|y_M)\end{array}\right]$ defines the set of all mappings. According to [9], the compression rates $I({\mathit{𝓨}}_{𝕊};{\mathit{Ƶ}}_{𝕊}|{\mathit{Ƶ}}_{\overline{𝕊}})$ are supermodular set functions with respect to the sets 𝕊 [32], while the relevant information $I(𝒳;\mathit{Ƶ})$ does not depend on 𝕊. Therefore, the greedy optimization structure of the GDIB algorithm is optimal and finds the extreme points of the solution space. It has to be emphasized that since the GDIB algorithm is based on the inner bound of the rate-region, it does not find the complete rate-region of the CEO problem. Following this approach, M IB related Lagrangian optimization problems are obtained, one for each sensor.

$$\begin{array}{cc}\hfill L_{\mathrm{GDIB}}^{\left(1\right)}& =I(𝒳;{Ƶ}_1)-{\beta }_{1}I({𝒴}_1;{Ƶ}_1)\hfill \\ \hfill & ⋮\hfill \end{array}$$

(6)

$$\begin{array}{cc}\hfill L_{\mathrm{GDIB}}^{\left(M\right)}& =I(𝒳;{Ƶ}_M|{\mathit{Ƶ}}_{<M})-{\beta }_{M}I({𝒴}_M;{Ƶ}_M|{\mathit{Ƶ}}_{<M})\hfill \end{array}$$

(7)

Obviously, the optimization problem of the first sensor resembles the optimization problem for the scalar IB problem given in (1) since there is no predecessor. Subsequent sensors exploit the mappings of previously designed quantizers as statistical side-information leading to the well-known Wyner–Ziv coding strategy. Naturally, each Lagrange multiplier ${\beta }_m$ has to be chosen such that the corresponding compression rate fulfills the individual rate constraint $I({𝒴}_m;{Ƶ}_m|{\mathit{Ƶ}}_{<m})\le C_m$. The objectives in (6) and (7) can be solved by equating the derivative with respect to the mapping $p\left(z_m\right|y_m)$ to zero delivering the update rule

$$\begin{array}{c}\hfill p\left(z_m\right|y_m)=\frac{e^{-d_{{\beta }_m}(y_m,z_m)}}{{\sum }_{z_m}{e}^{-d_{{\beta }_m}(y_m,z_m)}}\end{array}$$

(8)

with the exponent

$$\begin{array}{cc}\hfill d_{{\beta }_m}(y_m,z_m)≔{\mathbb{E}}_{{\mathit{Ƶ}}_{<m}|y_m}& \left[\frac{1}{{\beta }_m}{D}_{\mathrm{K}\mathrm{L}}[p\left(x\right|y_m,{\mathbf{z}}_{<m})\parallel p\left(x\right|{\mathbf{z}}_{\le m})]-logp\left(z_m\right|{\mathbf{z}}_{<m})\right].\hfill \end{array}$$

(9)

Similar to the scalar IB optimization, the implicit expression in (8) can be solved using a Blahut–Arimoto like algorithm, providing local optimal solutions. It has to be mentioned that for asymmetric scenarios, this optimization has to be performed for all $M!$ possible permutations of the optimization order to find the best solution.

A detailed derivation and performance analysis of this algorithm can be found in [14,15]. If the capacity is equally distributed over all sensors in the network, e.g., sensors share the same channel in an orthogonal way and a round robin fashion, numerical results demonstrate that the GDIB algorithm outperforms an individual scalar IB optimization at each sensor. However, there is still a large gap to the performance of a fcCEO scenario, which is defined in Section 4. Moreover, in asymmetric scenarios, the performance highly depends on the optimization order. Although no clear conclusion about the optimal Wyner–Ziv coding strategy can be drawn, a good solution can be expected when starting the optimization with the best forward channel conditions, i.e., the lowest compression (highest compression rate).

4. Fully Cooperative Distributed Sensing—A Centralized Quantization Approach

This section introduces the fcCEO scenario, which considers distributed sensors being able to forward their uncompressed observations to all other sensors in the network over ideal noiseless inter-sensor links. In this case, sensors can perfectly exchange their measurements $y_m$ before they jointly compress the received signals taking into account the rate constraints of all individual forward channels. Naturally, the exchange has to be done by a two-phase transmission protocol, consisting of a cooperation phase and a transmission phase. During the cooperation phase sensors exchange information until every sensor knows measurements $\mathbf{y}={[y_1\dots y_M]}^{\mathrm{T}}$ of all M sensors. The actual forwarding of the compressed observations to the common receiver is performed during the transmission phase. This full cooperation is equivalent to a single central quantizer having access to all measurements y as depicted in Figure 4. Applying the IB principle, this central quantizer can be designed in order to compress the vector y onto a cluster index z using the mapping $p\left(z\right|\mathbf{y})$, which motivates the name centralized IB (CIB) for the algorithmic solution in a fcCEO scenario. The optimization problem can be formulated as the maximization of

$$\begin{array}{c}\hfill L_{\mathrm{CIB}}=I(𝒳;Ƶ)-\beta I(\mathit{𝓨};Ƶ)\end{array}$$

(10)

and is solved using update Equation (2) with (3) substituting the scalar y by vector y. The number of output clusters $|\mathbb{Z}|$ has to be chosen to $|\mathbb{Z}|={\prod }_{m=1}^M\left|{\mathbb{Z}}_m\right|$ while the single link from the imaginary central quantizer to the receiver in Figure 4 has a channel capacity of $C_{\mathrm{sum}}={\sum }_{m=1}^M{C}_m$. The actual transmission over the M links has to be coordinated such that each sensor m transmits a specific part of the bits corresponding to its link capacity $C_m$.

In the special case of the measurement process being modeled as additive noise, the algorithm can be simplified to a scalar optimization problem where maximum ratio combining of all inputs $y_m$ delivers a scalar sufficient statistics

$$\overline{y}=\sum _{m=1}^M{\gamma }_m·y_m$$

of the desired relevant signal x with an overall SNR $\gamma ={\sum }_m{\gamma }_m$. The solution of the fcCEO scenario serves as an upper bound in this paper.

5. Partially Cooperative Distributed Sensing

In order to investigate how the gap between non-cooperative and fully-cooperative distributed compression can be reduced, partially cooperating sensors shall now be considered. Partial cooperation means a limited exchange of instantaneous side-information among the sensors during runtime due to a rate-limitation of inter-sensor links. Non-cooperative CEO and fully-cooperative CEO problems represent the extreme cases for zero rate and unlimited rate inter-sensor links, respectively. The rate limitation requires the compression of instantaneous side-information before forwarding it to other sensors.

In this paper, only deterministic mappings are considered for this compression, while indexes $z_m$ are still obtained by stochastic mappings. This is motivated by the fact that deterministic mappings do not require further lossy compression and the resulting side-information indices $s_m$ can be exploited at other sensors by choosing a particular mapping $p\left(z_m\right|y_m)$ from a list of possible mappings designed offline in advance. As a consequence, the compression rates for instantaneous side-information can only be adjusted by changing the cardinalities $|{𝕊}_m|$. For all results presented below, inter-sensor links are modeled as bit pipes being able to deliver $s_m$ reliably.

The GDIB algorithm to solve the non-cooperative CEO problem is based on the inner bound (5) of the CEO rate-region. Moreover, a greedy optimization approach is optimal due to the supermodularity of the compression rates in (5). Both require the Markovian structure in (4). However, cooperation among sensors changes the Markovian structure and implies different statistical dependencies among involved random variables. As (4) does not hold anymore in pcCEO scenarios, the inner bound on the rate-region in (5) cannot be utilized to find solutions of the pcCEO scenario. To the knowledge of the authors, tight bounds on the rate-region are not available for the cooperative case. Therefore, a heuristic approach based on the greedy optimization structure of the GDIB algorithm will be applied to solve the pcCEO scenario, which is not proven to be optimal. Nevertheless, the numerical evaluation of the found solutions demonstrate their usefulness. However, the computation of required pmfs becomes more challenging and results in recursive calculations given in Appendix A.1 and Appendix A.2 because the Markovian structure of (4) does not hold anymore in pcCEO scenarios.

This paper introduces three different inter-sensor communication protocols for exchanging this instantaneous side-information: successive broadcasting, a successive point-to-point transmission and a two-phase transmission. The first two protocols perform the exchange of instantaneous side-information $s_m$ with other sensors and the forwarding of compressed versions of $z_m$ to the common receiver in the same time slot. Contrarily, the two-phase transmission protocol separates the exchange of instantaneous side-information among sensors and the communication with the common receiver into two distinct phases. The latter starts after the exchange among sensors has been completed such that all sensors have (approximately) the same amount of side-information.

5.1. Successive Broadcasting Protocol

The system model for the successive broadcasting protocol is illustrated in Figure 5. In the same time slot, sensor $m-1$ not only forwards a compressed version of the quantization index $z_{m-1}$ to the common receiver, but also broadcasts instantaneous side-information $s_{m-1}$ to all other sensors. However, due to the greedy optimization structure, only subsequent sensors can exploit this instantaneous side-information. Thus, sensor m can exploit indices ${\mathbf{s}}_{<m}$ of all previously transmitting sensors in order to select its quantization index $z_m$ as well as a new instantaneous side-information index $s_m$. This scenario leads to the Markov model

$$\begin{array}{cc}\hfill p(x,\mathbf{y},\mathbf{z},\mathbf{s})& =\prod _{m=1}^{M}p\left(z_m\right|y_m,{\mathbf{s}}_{<m})p\left(s_m\right|y_m,{\mathbf{s}}_{<m})p\left(y_m\right|x)p\left(x\right).\hfill \end{array}$$

(11)

The indices $s_m$ and $z_m$ are obtained by deterministic mappings $p\left(s_m\right|y_m,{\mathbf{s}}_{<m})\in \{0,1\}$ and stochastic mappings $p\left(z_m\right|y_m,{\mathbf{s}}_{<m})\in [0,1]$, respectively. The design of these mappings can be performed offline leveraging the IB principle as illustrated in Figure 6. It combines the observation $y_m$ and the instantaneous side-information ${\mathbf{s}}_{<m}$ to indexes $s_m$ and $z_m$ while maintaining as much information as possible about the relevant signal x.

5.1.1. Generation of Broadcast Side-Information

The design of $p\left(s_m\right|y_m,{\mathbf{s}}_{<m})$ is inspired by the general GDIB algorithm, i.e., the optimization is done in a greedy manner. Again, there emerges one optimization problem for each sensor:

$$\begin{array}{cc}\hfill L_{\mathrm{BC}-\mathrm{SIDE}}^{\left(1\right)}& =I(𝒳;{𝒮}_1)-\beta I({𝒴}_1;{𝒮}_1)\hfill \\ \hfill & ⋮\hfill \end{array}$$

(12)

$$\begin{array}{cc}\hfill L_{\mathrm{BC}-\mathrm{SIDE}}^{(M-1)}& =I(𝒳;{𝒮}_{M-1}|{\mathit{𝓢}}_{<M-1})-\beta I({𝒴}_{M-1};{𝒮}_{M-1}|{\mathit{𝓢}}_{<M-1}).\hfill \end{array}$$

(13)

The optimization problem of the first sensor equals the individual scalar optimization without any side-information at all, as described in Section 2. Subsequent sensors combine the instantaneous side-information of all previously transmitting sensors ${\mathbf{s}}_{<m}$ with its observation $y_m$. The relevant mutual information and the compression rate of sensor m are conditioned on ${\mathit{𝓢}}_{<m}$ since broadcasting instantaneous side-information ensures all successive sensors to have access to ${\mathbf{s}}_{<m}$ allowing Wyner–Ziv coding for generating $s_m$. Each optimization problem given in (12) and (13) can be solved by taking the derivative with respect to the mapping $p\left(s_m\right|y_m,{\mathbf{s}}_{<m})$ and equating it to zero. This results in the implicit update equation

$$p\left(s_m\right|y_m,{\mathbf{s}}_{<m})=\frac{e^{-d_{{\beta }_m}(y_m,s_m,{\mathbf{s}}_{<m})}}{{\sum }_{s_m}{e}^{-d_{{\beta }_m}(y_m,s_m,{\mathbf{s}}_{<m})}}$$

(14)

with

$$\begin{array}{c}\hfill d_{{\beta }_m}(y_m,s_m,{\mathbf{s}}_{<m})≔\frac{1}{{\beta }_m}{D}_{\mathrm{K}\mathrm{L}}\left[p\left(x\right|y_m,{\mathbf{s}}_{<m})\parallel p\left(x\right|{\mathbf{s}}_{\le m})\right]-logp\left(s_m\right|{\mathbf{s}}_{<m}).\end{array}$$

(15)

As in the general GDIB algorithm, the implicit update equation in (14) can be solved using a Blahut–Arimoto like algorithm resulting in local optimal solutions.

Algorithmic pcCEO Solution for the Successive Broadcasting Protocol

After designing the mapping for the instantaneous side-information, the mapping $p\left(z_m\right|y_m,{\mathbf{s}}_{<m})$ can be optimized, again by means of the IB principle. Therefore, the original GDIB algorithm is modified to exploit the broadcasted instantaneous side-information, defining the GDIB-BC algorithm. The optimization problem for each sensor is given as

$$\begin{array}{cc}\hfill L_{\mathrm{GDIB}-\mathrm{BC}}^{\left(1\right)}& =I(𝒳;{Ƶ}_1)-{\beta }_{1}I({𝒴}_1;{Ƶ}_1)\hfill \\ \hfill & ⋮\hfill \end{array}$$

(16)

$$\begin{array}{cc}\hfill L_{\mathrm{GDIB}-\mathrm{BC}}^{\left(M\right)}& =I(𝒳;{Ƶ}_M|{\mathit{Ƶ}}_{<M})-{\beta }_{M}I({𝒴}_M,{\mathit{𝓢}}_{<M};{Ƶ}_M|{\mathit{Ƶ}}_{<M}).\hfill \end{array}$$

(17)

The main difference to the original GDIB optimization problem in (6) and (7) lies in the definition of the compression rate $I({𝒴}_m,{\mathit{𝓢}}_{<m};{Ƶ}_m|{\mathit{Ƶ}}_{<m})$ which emerges from the combination of the observation $y_m$ and the instantaneous side-information ${\mathrm{s}}_{<m}$. Taking the derivative of the optimization problem for sensor m with respect to the mapping $p\left(z_m\right|y_m,{\mathbf{s}}_{<m})$ and equating it to zero delivers

$$p\left(z_m\right|y_m,{\mathbf{s}}_{<m})=\frac{e^{-d_{{\beta }_m}(y_m,z_m,{\mathbf{s}}_{<m})}}{{\sum }_{z_m}{e}^{-d_{{\beta }_m}(y_m,z_m,{\mathbf{s}}_{<m})}}$$

(18)

with

$$\begin{gathered} d_{{\beta }_m}(y_m,z_m,{\mathbf{s}}_{<m})≔{\mathbb{E}}_{{\mathit{Ƶ}}_{<m}|y_m,{\mathbf{s}}_{<m}}[\frac{1}{{\beta }_m}· \\ {D}_{\mathrm{K}\mathrm{L}}[p\left(x\right|y_m,{\mathbf{s}}_{<m},{\mathbf{z}}_{<m})\parallel p\left(x\right|{\mathbf{z}}_{\le m})]-logp\left(z_m\right|{\mathbf{z}}_{<m})]. \end{gathered}$$

(19)

Again, the implicit update equation in (18) can be solved using a Blahut–Arimoto like algorithm. The extended Blahut–Arimoto like algorithm to design the mapping $p\left(z_m\right|y_m,{\mathbf{s}}_{<m})$ of sensor m for a specific Lagrange parameter ${\beta }_m$ and instantaneous side-information ${\mathbf{s}}_{<m}$ is given in Algorithm 1. The input pmf $p(y_{m-1},{\mathbf{s}}_{<m-1},{\mathbf{z}}_{<m-1},x)$ can be computed during the optimization of previous sensors. Lines 3 to 5 determine the required pmfs for the calculation of the KL-divergence of (19) in lines 6 to 9. The statistical distance of (19) is determined in lines 10 to 14. It is used to update the quantizer mapping $p\left(z_m\right|y_m,{\mathbf{s}}_{<m})$ of sensor m. This procedure is repeated until no significant changes of the desired mappings occur anymore. The algorithm returns the updated mapping $p\left(z_m\right|y_m,{\mathbf{s}}_{<m})$ as well as the pmf $p(y_m,{\mathbf{s}}_{<m},{\mathbf{z}}_{<m},x)$, which is used as an input for the successive sensor.

Algorithm 1: Extended Blahut–Arimoto algorithm for broadcast cooperating sensors.

The parameter ${\beta }_m$, which determines the compression rate at sensor m, has to be adjusted such that $I({𝒴}_m,{\mathit{𝓢}}_{<m};{Ƶ}_m|{\mathit{Ƶ}}_{<m})\le C_m$ is fulfilled. Similar to the original GDIB algorithm, the GDIB-BC algorithm has to be performed for each sensor and all possible optimization orders.

5.1.2. Evolution of Instantaneous Side-Information

Figure 7 illustrates the amount of instantaneous side-information available at the different sensors in a network of size $M=6$ considering the broadcast of side-information. It depicts the relevant mutual information $I(𝒳;{\mathit{𝓢}}_{\le m})$ versus the sensor number m for different cardinalities $|{𝕊}_m|$ and SNRs ${\gamma }_m$. The relevant signal is chosen to be a uniformly distributed 4-ASK signal leading to $\left|𝕏\right|=4$. As expected, the amount of available instantaneous side-information increases with each additional sensor for all $|{𝕊}_m|$ and ${\gamma }_m$. To be more specific, the resolution and the quality of instantaneous side-information available at sensor m increases with growing m. In the considered symmetric scenario, the amount of information $I(𝒳;{𝒮}_m|{\mathit{𝓢}}_{<m})$ a sensor can contribute to $I(𝒳;{\mathit{𝓢}}_{\le m})$ gets smaller for each additional sensor and the slopes of the curves decrease. Since one bit is not enough to represent the information of $\left|𝕏\right|=4$, the largest gain can be observed between $|{𝕊}_m| =2$ and $|{𝕊}_m| =4$. Increasing the cardinality further to $|{𝕊}_m| =8$ results only in a small additional improvement. Certainly, this observation depends on the relevant signal $𝒳$ and can not be generalized.

5.1.3. Performance for Different Network Sizes

Figure 8 and Figure 9 illustrate the overall performance of the GDIB-BC approach when broadcasting instantaneous side-information for different network sizes. The relevant mutual information $I(𝒳;\mathit{Ƶ})$ is depicted versus the number of sensors M in the network. The gray colored area represents the non-achievable region, since $I(𝒳;\mathit{Ƶ})$ cannot exceed $I(𝒳;\mathit{𝓨})$ due to the data-processing inequality. Both figures consider a scenario where all sensors in the network share the same channel to the common receiver with a fixed sum-rate $C_{\mathrm{sum}}$ in an orthogonal way and a round robin fashion. Consequently, larger network sizes correspond to smaller individual capacities $C_m=\frac{C_{\mathrm{sum}}}{M}$ for each forward link. The performance of partially cooperating sensors broadcasting instantaneous side-information (pcCEO-BC) is compared to the non-cooperative case (CEO) of Section 3 and the fully cooperative case (fcCEO) of Section 4. As already mentioned, these two scenarios provide upper and lower bounds. In general, it can be observed that increasing the number of sensors in the network also increases the overall relevant mutual information $I(𝒳;\mathit{Ƶ})$. This holds even for the case without cooperation, since each sensor applies Wyner–Ziv coding and exploits the mapping of previously designed quantizers as statistical side-information [14]. Independent of the cardinality $|{𝕊}_m|$, the performance of the pcCEO-BC scenario is superior to the case without cooperation among sensors. This difference grows for larger network sizes because the amount of information ${\mathbf{s}}_{<m}$ has about the relevant variable x increases. As expected from Figure 7, increasing the cardinality $|{𝕊}_m|$ not only improves the relevant information $I(𝒳;{\mathit{𝓢}}_{\le m})$, but also the overall performance measured by $I(𝒳;\mathit{Ƶ})$. However, it can be observed that even for large $|{𝕊}_m|$ there remains a gap to the fcCEO upper bound, especially for smaller network sizes or lower SNRs. This gap can be explained by the successive transmission protocol resulting in a gradually increasing amount of instantaneous side-information at the sensors. For instance, the first sensors does not profit at all from the partial cooperation in contrast to the fcCEO scenario where all sensors exploit almost the same amount of side-information.

Considering the pmfs in Algorithm 1, it becomes obvious that larger networks might suffer from the curse of dimensionality. More precisely, pmfs like $p(y_m,{\mathbf{s}}_{<m},{\mathbf{z}}_{<m},x)$ can become very large during the optimization for larger network sizes. Moreover, the mapping $p\left(z_m\right|y_m,{\mathbf{s}}_{<m})$ also depends on the network size, i.e., this problem does not only occur during the optimization, but also when storing the already optimized mapping. This numerical issue is the reason why there is no result for $|{𝕊}_m| =8$ and a network size of $M=6$ in Figure 8 and Figure 9. In this case, it requires 2024 GiB (1 GiB = 1024 MiB, 1 MiB = 1024 KiB, 1 KiB = 1024 byte) just for storing a single instance of the pmf $p(y_m,{\mathbf{s}}_{<m},{\mathbf{z}}_{<m},x)$.

5.2. Successive Point-to-Point Protocol

For larger network sizes, broadcasting side-information might not be feasible anymore, since the dimensions of the mappings $p\left(z_m\right|y_m,{\mathbf{s}}_{<m})$ and $p\left(s_m\right|y_m,{\mathbf{s}}_{<m})$ as well as intermediate pmfs used within the optimization become huge. In order to relax this curse of dimensionality, the successive way of cooperation is exploited and the instantaneous side-information of sensor m shall only be forwarded to the direct successor $m+1$ as depicted in Figure 10. Hence, a sequential chain is established from the first to the last sensor leading to the Markov Model:

$$\begin{array}{c}\hfill p(x,\mathbf{y},\mathbf{z},\mathbf{s})=\prod _{m=1}^{M}p\left(z_m\right|y_m,s_{m-1})p\left(s_m\right|y_m,s_{m-1})p\left(y_m\right|x)p\left(x\right).\end{array}$$

(20)

Again, the instantaneous side-information is obtained by a deterministic mapping optimized by means of the information bottleneck principle, illustrated in Figure 11. With each step in the sequential chain, the information $s_m$ has about the relevant signal x increases.

5.2.1. Generation of Point-to-Point Side-Information

Similar to the broadcast case, the design of $p\left(s_m\right|y_m,s_{m-1})$ is inspired by the original GDIB algorithm. The optimization problem can be formulated in a greedy manner as

$$\begin{array}{cc}\hfill L_{\mathrm{PTP}-\mathrm{SIDE}}^{\left(1\right)}& =I(𝒳;{𝒮}_1)-\beta I({𝒴}_1;{𝒮}_1)\hfill \\ \hfill & ⋮\hfill \end{array}$$

(21)

$$\begin{array}{cc}\hfill L_{\mathrm{PTP}-\mathrm{SIDE}}^{(M-1)}& =I(𝒳;{𝒮}_{M-1})-\beta I({𝒴}_{M-1},{𝒮}_{M-2};{𝒮}_{M-1}),\hfill \end{array}$$

(22)

where Equation (21) equals the individual scalar optimization without any side-information. Subsequent sensors combine the instantaneous side-information $s_{m-1}$ sent by the previous sensor with its observation $y_m$. In contrast to the broadcast case, the relevant mutual information is not conditioned on ${\mathit{𝓢}}_{<m}$ as in (12) and (13) because sensor m will only have access to $s_{m-1}$ and not to indices of any other sensor. Therefore, Wyner–Ziv coding cannot be applied for exchanging instantaneous side-information with the successive point-to-point protocol. The optimization problems can be solved by taking the derivative with respect to the mapping $p\left(s_m\right|y_m,s_{m-1})$ and equating it to zero, resulting in the implicit update equation

$$\begin{array}{c}\hfill p\left(s_m\right|y_m,s_{m-1})=\frac{e^{-d_{{\beta }_m}(y_m,s_m,s_{m-1})}}{{\sum }_{s_m}{e}^{-d_{{\beta }_m}(y_m,s_m,s_{m-1})}}\end{array}$$

(23)

with

$$\begin{array}{c}\hfill d_{{\beta }_m}(y_m,s_m,s_{m-1})≔\frac{1}{{\beta }_m}·D_{\mathrm{K}\mathrm{L}}\left[p\left(x\right|y_m,s_{m-1})\parallel p\left(x\right|s_m)\right]-logp\left(s_m\right).\end{array}$$

(24)

As in the broadcast case, using a Blahut–Arimoto like algorithm to solve the update Equation (23) results in local optimal solutions.

5.2.2. Algorithmic pcCEO Solution Applying the Successive Point-to-Point Protocol

After the optimization of the mapping for instantaneous side-information, the mapping $p\left(z_m\right|y_m,s_{m-1})$ can be designed by means of the information bottleneck principle. Inspired by the original GDIB algorithm, the optimization problem can be formulated as

$$\begin{array}{cc}\hfill L_{\mathrm{GDIB}-\mathrm{PTP}}^{\left(1\right)}& =I(𝒳;{Ƶ}_1)-{\beta }_{1}I({𝒴}_1;{Ƶ}_1)\hfill \\ \hfill & ⋮\hfill \end{array}$$

(25)

$$\begin{array}{cc}\hfill L_{\mathrm{GDIB}-\mathrm{PTP}}^{\left(M\right)}& =I(𝒳;{Ƶ}_M|{\mathit{Ƶ}}_{<M})-{\beta }_{M}I({𝒴}_M,{𝒮}_{M-1};{Ƶ}_M|{\mathit{Ƶ}}_{<M}).\hfill \end{array}$$

(26)

The main difference to the original GDIB optimization problem in (6) and (7) now lies in the compression rate $I({𝒴}_m,{𝒮}_{m-1};{Ƶ}_m|{\mathit{Ƶ}}_{<m})$, which emerges from the combination of the instantaneous side-information $s_{m-1}$ of the previous sensor $m-1$ and the observation $y_m$ of sensor m. Taking the derivative with respect to the mapping $p\left(z_m\right|y_m,s_{m-1})$ and equating it to zero, the optimization problem for sensor m can be solved, leading to the implicit update equation

$$\begin{array}{c}\hfill p\left(z_m\right|y_m,s_{m-1})=\frac{e^{-d_{{\beta }_m}(y_m,z_m,s_{m-1})}}{{\sum }_{z_m}{e}^{-d_{{\beta }_m}(y_m,z_m,s_{m-1})}}\end{array}$$

(27)

with

$$\begin{array}{cc}\hfill & d_{{\beta }_m}(y_m,z_m,s_{m-1})≔{\mathbb{E}}_{{\mathit{Ƶ}}_{<m}|y_m,s_{m-1}}[\frac{1}{{\beta }_m}·\hfill \\ & D_{\mathrm{K}\mathrm{L}}[p\left(x\right|y_m,s_{m-1},{\mathbf{z}}_{<m})\parallel p\left(x\right|{\mathbf{z}}_{\le m})]-logp\left(z_m\right|{\mathbf{z}}_{<m})].\hfill \end{array}$$

(28)

Thus, the mapping $p\left(z_m\right|y_m,s_{m-1})$ can be optimized using a Blahut–Arimoto-like algorithm. The specific algorithm for a given sensor m and a Lagrange parameter ${\beta }_m$ is given in Algorithm 2. The input pmfs $p\left(z_i\right|y_i,s_{i-1})\forall i<m$ and $p\left(s_i\right|{\mathbf{z}}_{\le i},x)\forall i<m$ as well as $p({\mathbf{z}}_{<m-1},x)$ are calculated in advance by previous sensor optimizations. Lines 3 to 7 calculates required pmfs as given the Appendix A.2. The KL-divergence is calculated in lines 8 to 11. Using this, the statistical distance $d_{{\beta }_m}(z_m,y_m,s_{m-1})$ of (28) can be calculated in lines 12 to 16, which is then used to update the quantizer mapping $p\left(z_m\right|y_m,s_{m-1})$. The algorithm stops if this mapping does not change significantly anymore during subsequent iterations. Finally, the output pmfs $p\left(s_m\right|{\mathbf{z}}_{\le m},x)$ and $p({\mathbf{z}}_{<m},x)$ need to be calculated in lines 21 to 25 for their usage in the optimization of the next sensor.

Similar to the original GDIB algorithm, the optimization needs to be done for all possible optimization orders. A simple bisection search can be applied to find the rate-fulfilling parameter ${\beta }_m$, such that $I({𝒴}_m,{𝒮}_{m-1};{Ƶ}_m|{\mathit{Ƶ}}_{<m})\le C_m$ holds.

5.2.3. Evolution of Instantaneous Side-Information

Figure 12 illustrates the amount of instantaneous side-information $I(𝒳;{𝒮}_m)$ at a specific sensor m in a network of size $M=6$ using the successive point-to-point transmission protocol for different cardinalities $|{𝕊}_m|$. Obviously, $I(𝒳;{𝒮}_m)$ increases with each further sensor. The main difference to the broadcast case is that the instantaneous side-information provided to sensor m is represented by a single highly compressed index $s_{m-1}$ with cardinality $|{𝕊}_{m-1}|$. While the resolution $|{𝕊}_{<m}|$ of the available instantaneous side-information ${\mathbf{s}}_{<m}$ increases with m in the broadcast case, it remains the same for the successive point-to-point protocol. Therefore, a higher cardinality $|{𝕊}_m|$ is required compared to the broadcast case to avoid additional compression losses.

Figure 12. Evolution of $I(𝒳;{𝒮}_m)$ for sensor m in a network with $M=6$ sensors and different cardinalities $|{𝕊}_m|$ for the successive point-to-point transmission protocol; $\left|𝕏\right|=4$, $|{𝕐}_m| =64$.

Figure 12. Evolution of $I(𝒳;{𝒮}_m)$ for sensor m in a network with $M=6$ sensors and different cardinalities $|{𝕊}_m|$ for the successive point-to-point transmission protocol; $\left|𝕏\right|=4$, $|{𝕐}_m| =64$.

Algorithm 2: Extended Blahut–Arimoto algorithm for the successive point-to-point protocol.

5.2.4. Performance for Different Network Sizes

Figure 13 and Figure 14 illustrate the overall performance of the pcCEO system with point-to-point exchanged instantaneous side-information where all sensors share the same channel to the common receiver with a fixed sum-rate $C_{\mathrm{sum}}={\sum }_{m=1}^M{C}_m$ in an orthogonal way and a round robin fashion. Again, the black curves represent the non-cooperative CEO scenario and the fcCEO scenario. Hence, they serve as lower and upper bound, respectively. In general, the curves are very similar to those for broadcasting instantaneous side-information in Figure 8 and Figure 9. Independent of the SNR or the sum-rate $C_{\mathrm{sum}}$, the relevant mutual information $I(𝒳;\mathit{Ƶ})$ increases for larger networks and even a single bit as instantaneous side-information $|{𝕊}_m| =2$ leads to slight improvements compared to the non-cooperative case. However, there still remains a gap to the fcCEO scenario even for large $|{𝕊}_m|$, which results from the successive communication strategy, since sensors at the beginning of the optimization chain can exploit no or little instantaneous side-information.

5.2.5. Performance for Different Sum-Rates

Figure 15 illustrates the influence of the sum-rate $C_{\mathrm{sum}}={\sum }_{m=1}^M{C}_m$ for a scenario with $M=5$ sensors. Naturally, larger sum-rates correlate with higher individual link capacities. Again, CEO and fcCEO scenarios provide lower and upper bounds, respectively. For a cardinality of $|{𝕊}_m| =2$, only a small gain compared to the non-cooperative CEO scenario can be observed. However, the gain gets more and more significant with increasing $|{𝕊}_m|$. Comparing the results to the upper fcCEO bound illuminates the loss due to limited available side-information at early transmitting sensors. The largest difference can be observed for sum-rates between $2\le C_{\mathrm{sum}}\le 4$ bit/s/Hz.

5.2.6. Asymmetric Scenarios

A very important part is the investigation of asymmetric scenarios. As the achievable relevant information $I(𝒳;\mathit{Ƶ})$ of a non-cooperative CEO scenario is very sensitive to the optimization order, i.e., the Wyner–Ziv coding strategy, in asymmetric scenarios [14] the question arises if the exchange of instantaneous side-information can improve the robustness against bad optimization orders. Therefore, the same two asymmetric setups as in [14] are analyzed. Scenario 1 considers the case where sensors with low SNRs ${\gamma }_m$ have low link capacities $C_m$ while sensors with high SNRs ${\gamma }_m$ have high link capacities $C_m$. Scenario 2 considers the opposite case, where sensors with low SNRs have high link capacities and vice versa.

Figure 16 illustrates the relevant mutual information $I(𝒳;\mathit{Ƶ})$ for all $M!=24$ sensor permutations for a network of $M=4$ sensors. The dots represent the results from [14] for a non-cooperative CEO scenario. Blue dots show Scenario 1 while the red dots represent Scenario 2. The results for the pcCEO scenario with successive point-to-point side-information exchange is depicted as bars.

Comparing the non-cooperative case with the successive point-to-point exchange of side-information for Scenario 1, we observe a slight increase of the overall relevant mutual information $I(𝒳;\mathit{Ƶ})$ for partial cooperation and this particular scenario. Moreover, the influence of the Wyner–Ziv coding strategy (optimization order) becomes smaller due to cooperation. The performance for Scenario 2 is worse than the performance for Scenario 1, again for both the cooperative and the non-cooperative case. In this scenario, accurate measurements have to be strongly compressed in order to forward them to the common receiver while unreliable measurements cannot contribute much to the overall performance although they can be forwarded to the common receiver at high rates. However, the loss due to bad optimization orders is much lower for partial cooperation. A sensor with a bad forward channel and a high SNR can still forward its information to the next sensor, which might have a better forward channel. Therefore, exchanging instantaneous side-information can improve the robustness against bad optimization orders.

5.3. Two-Phase Transmission Protocol with Artificial Side-Information

Previous subsections revealed that partial cooperation by exchanging instantaneous side-information improves the overall performance. However, a gap to the fcCEO scenario still remains, and we claimed that the successive exchange of instantaneous side-information is the reason for this difference. Due to the sequential forwarding protocols considered so far, early sensors have no or little instantaneous side-information. They hardly profit from the cooperation as opposed to the full cooperation case where all sensors have access to the complete information. In order to substantiate this statement, a third transmission protocol consisting of two phases is considered. Inspired by the fcCEO scenario, the first cooperation phase is used to exchange instantaneous side-information between all sensors, while the transmission phase is used to forward the information to the common receiver in the usual way. The difference to the fcCEO scenario is that only compressed versions of the observations can be exchanged during the cooperation phase.

For simplicity, we assume that each sensor obtains the same instantaneous side-information represented by $s^{*}$, independent of its position in the optimization chain, see Figure 17. Moreover, we pursue the EXIT chart philosophy [33], where extrinsic information is artificially created to analyze the information exchange between decoders in concatenated coding schemes. In the pcCEO context, the artificial side-information can be interpreted as extrinsic information about the relevant signal x being generated by adding AWGN to x. The noise variance is adapted to obtain a specific SNR ${\gamma }_{\mathrm{extr}}$ or equivalently a desired mutual side-information $I(𝒳;{𝒮}^{*})$. It has to be emphasized that ${\gamma }_{\mathrm{extr}}$ can be chosen independently from the measurement SNRs at the sensors in order to obtain general conclusions. Since the instantaneous side-information is created artificially, $s^{*}$ is assumed to be independent of the indexes $y_m$ given the relevant signal x, i.e., $p(y_m,s^{*}|x)=p\left(y_m\right|x)p\left(s^{*}\right|x)$ holds. This simplifies the Markovian structure of the optimization problem which equals the one of the original CEO problem. With the same argumentation as in the original CEO problem, we claim that the supermodularity holds and the greedy optimization structure is optimal. This model leads to the modified optimization problem

$$\begin{array}{cc}\hfill L_{\mathrm{GDIB}-\mathrm{TP}}^{\left(1\right)}& =I(𝒳;{Ƶ}_1)-{\beta }_{1}I({𝒴}_1,{𝒮}^{*};{Ƶ}_1)\hfill \\ \hfill & ⋮\hfill \end{array}$$

(29)

$$\begin{array}{cc}\hfill L_{\mathrm{GDIB}-\mathrm{TP}}^{\left(M\right)}& =I(𝒳;{Ƶ}_M|{\mathit{Ƶ}}_{<M})-{\beta }_{M}I({𝒴}_M,{𝒮}^{*};{Ƶ}_M|{\mathit{Ƶ}}_{<M}).\hfill \end{array}$$

(30)

The optimization problem can be solved using the same strategy as described in previous Section 5.1 and Section 5.2. This leads to the implicit update equation

$$\begin{array}{c}\hfill p\left(z_m\right|y_m,s^{*})=\frac{e^{-d_{{\beta }_m}(y_m,z_m,s^{*})}}{{\sum }_{z_m}{e}^{-d_{{\beta }_m}(y_m,z_m,s^{*})}}\end{array}$$

(31)

with

$$\begin{array}{cc}\hfill & d_{{\beta }_m}(y_m,z_m,s^{*})≔{\mathbb{E}}_{{\mathit{Ƶ}}_{<m}|y_m,s^{*}}[\frac{1}{{\beta }_m}·\hfill \\ \hfill & D_{\mathrm{K}\mathrm{L}}[p\left(x\right|y_m,s^{*},{\mathbf{z}}_{<m})\parallel p\left(x\right|{\mathbf{z}}_{\le m})]-logp\left(z_m\right|{\mathbf{z}}_{<m})].\hfill \end{array}$$

(32)

5.3.1. Performance of Two-Phase Transmission

Figure 18 illustrates the same experiment as in Figure 8 or Figure 13, but for the two-phase transmission protocol. The extrinsic information is chosen independent of the measurement SNR and has its own SNR ${\gamma }_{\mathrm{ext}}$ represented by different colors in Figure 18. The cardinality of the extrinsic information is chosen as $|{𝕊}^{*}| =512$ to not introduce any compression losses. As before, the black dashed line represents the fcCEO scenario. The curve for ${\gamma }_{\mathrm{extr}}={\gamma }_m=8$ dB represents the case where each sensor forwards instantaneous side-information whose quality corresponds to its measurement SNR. We observe the same performance as for the fcCEO scenario. This demonstrates that the remaining performance gap to the fcCEO scenario disappears completely for appropriate cooperation among sensors. Naturally, decreasing the SNR of the extrinsic information ${\gamma }_{\mathrm{ext}}$ or equivalently $I(𝒳;{𝒮}^{*})$ leads to a lower overall performance $I(𝒳;\mathit{Ƶ})$.

5.3.2. Influence of Extrinsic Information

The influence of extrinsic information is depicted in Figure 19 for ${\gamma }_m=8$ dB and ${\gamma }_m=3$ dB. Therefore, the overall relevant mutual information $I(𝒳;\mathit{Ƶ})$ is depicted versus the mutual information of the extrinsic information $I(𝒳;{𝒮}^{*})$ for different network sizes. As before, all sensors share the same forward channel with $C_{\mathrm{sum}}=2.5$ bit/s/Hz and $C_m=\frac{C_{\mathrm{sum}}}{M}$. Providing no extrinsic information, i.e., $I(𝒳;{𝒮}^{*})=0$ delivers the same result as the non-cooperative CEO scenario of Section 3. Naturally, enhancing the quality of the extrinsic information increases the overall relevant mutual information $I(𝒳;\mathit{Ƶ})$ up to the maximum of 2 bit/s/Hz.

6. Conclusions

This paper extends the non-cooperative CEO scenario allowing partial cooperation among sensors in the network. Therefore, it extends the algorithmic solution introduced in [14] for three different inter-sensor communication protocols: successive broadcasting, successive point-to-point communication and a two-phase transmission protocol. The first two protocols perform the exchange of instantaneous side-information and forwarding information to the common receiver at the same time step. Therefore, successive broadcasting exploits the instantaneous side-information of all previous sensors within the optimization chain. Since this may cause dimensionality problems during the optimization, the successive point-to-point transmission protocol forwards the instantaneous side-information only to the next sensor. It turns out that allowing this partial communication outperforms the non-cooperative compression where no communication among sensors is possible. Moreover, cooperative compression shows a larger robustness to suboptimal Wyner–Ziv coding strategies in asymmetric scenarios. However, a small performance gap to the fcCEO scenario still remains for the proposed successive broadcasting and successive point-to-point transmission protocols. This gap can be closed by a third protocol separating the cooperation from the forwarding phase and allowing each sensor to access the maximal available side-information. Although no formal conclusion about the optimality of the pcCEO can be drawn, the closeness to the fcCEO scenario in the investigated simulations reveals that solutions found by the proposed greedy algorithms are at least close to optimal.