Broadcast Channel Cooperative Gain: An Operational Interpretation of Partial Information Decomposition

Abstract

Partial information decomposition has recently found applications in biological signal processing and machine learning. Despite its impacts, the decomposition was introduced through an informal and heuristic route, and its exact operational meaning is unclear. In this work, we fill this gap by connecting partial information decomposition to the capacity of the broadcast channel, which has been well studied in the information theory literature. We show that the synergistic information in the decomposition can be rigorously interpreted as the cooperative gain, or a lower bound of this gain, on the corresponding broadcast channel. This interpretation can help practitioners to better explain and expand the applications of the partial information decomposition technique.

1. Introduction

Shannon’s mutual information has been widely accepted as a measure to gauge the amount of information that can be revealed by one random variable regarding another random variable. Partial information decomposition (PID) is an approach to refine and further decompose this fundamental quantity to explain the effect of interactions among several random variables. Recently, this approach has found applications in biological information processing [1,2,3,4,5,6,7] and machine learning [8,9,10].

There exist different approaches to decompose the total information [11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,26,27], but the general idea in the case with two observables is as follows: the total information revealed by X and Y regarding a third quantity T is the mutual information between $(X,Y)$ and T, i.e., $I(X,Y;T)$ (see [28]), and it needs to be decomposed into four non-negative parts:

• The common (or redundancy) information in X and Y, regarding T;
• The unique information in X, but not in Y, regarding T;
• The unique information in Y, but not in X, regarding T;
• The synergistic information (or complementary) information in X and Y, regarding T, which only becomes useful when combined, but is useless otherwise.

This decomposition helps to explain the effect of combining X and Y, or separately using X or Y, to infer T. For example, in multi-modality machine learning, X can represent one modality such as the vision image of the event, Y another modality such as the soundtrack of the event, and T the event’s label. In this case, we can use the amounts of synergistic and unique information to determine which multi-modality models will be the most effective in capturing the interactions and therefore are most likely to be accurate. In neural signal processing, the quantities can be used in a similar manner to interpret different biological signals.

One of the most influential notions of partial information decomposition was proposed by Bertschinger et al. [13], sometimes referred to as the BROJA PID (the initials of the authors; we will simply refer to it as partial information decomposition as it is the only decomposition we consider). The desirable properties of this definition were thoroughly studied in [13], but in terms of the operation meaning, only a qualitative justification in a decision-making setting was given. The intuition and motivation were that these quantities can help explain how rewards can be optimized via decision-making in such a setting, either using only X (or only Y) as the observations, or using X and Y jointly as the observations (a refinement was given in [29]). However, this qualitative justification is far from satisfying, as it does not provide the exact quantitative interpretation and requires an artificially introduced utility function, which makes its meaning quite vague. In this work, we fill this gap by connecting PID to the well-studied broadcast channel in information theory [30,31,32], and show that there is indeed a quantitative connection between PID and the sum-rate capacity of the broadcast channel. More precisely, the synergistic information defined in [13] can be interpreted as either the cooperative gain or a lower bound of the cooperative gain in the broadcast channel, under a fixed signaling distribution.

2. Partial Information Decomposition

2.1. A Few Examples

Before formally introducing the PID defined in [13], let us consider a few simple settings to understand intuitively how the decomposed information should behave.

1.

Common information dominant: X is a uniformly distributed Bernoulli random variable and $Y=T=X$. It is clear that $I(X,Y;T)=1$. In this case, the common information should be one, since X and Y are exactly identical. The other components should all be 0, since there is no unique information in either X or Y regarding T, and there is no synergistic information when combining X and Y.

2.

Synergistic information dominant: X and Y are uniformly distributed Bernoulli random variables independent of each other, and $T=X\oplus Y$, where ⊕ is the XOR operation. Here, the synergistic information should be 1, and the other components should be 0. This is because X or Y alone do not reveal any information regarding T, and since they are completely independent, they do not share any common information either, but their combination reveals the full information on T.

3.

Component-wise decomposition: $X=(X_1,X_2)$ and $Y=(Y_1,Y_2)$, where $X_1,X_2,Y_1,Y_2$ are all uniformly-distributed Bernoulli random variables, mutually independent of each other. Let $T=(X_1,Y_1,X_2\oplus Y_2)$. Clearly, in this case, the common information is still 0, but the two kinds of unique information are both 1, and the synergistic information is also 1.

2.2. Partial Information Decomposition

We next introduce the formal definition for the PID given in [13]. Let $X,Y,$ and T be three random variables in their respective alphabet $\mathcal{X}$, $\mathcal{Y}$, and $\mathcal{T}$, which follow the joint distribution $P_{X,Y,T}$. The total information between $X,Y,$ and T is their mutual information, i.e., $I(T;X,Y)$; however, from here on out, we shall write it as $I_P(T;X,Y)$ to make the dependence on the distribution more explicit. As mentioned earlier, the decomposition first needs to satisfy the total information rule:

$$\begin{array}{cc}& \mathbf{Total} \mathbf{information}: \hfill \\ & I_P(T;X,Y)=I^{\left(C\right)}(T;X,Y)+I^{\left(U_X\right)}(T;X,Y)+I^{\left(U_Y\right)}(T;X,Y)+I^{\left(S\right)}(T;X,Y),\hfill \end{array}$$

(1)

where the superscript C indicates common (or redundancy) information, $U_X$ indicates unique information from X, $U_Y$ indicates unique information from Y, and S indicates synergistic (or complementary) information. This condition simply states that the total information is the summation of four parts. The decomposition also needs to satisfy the individual information rule:

$$\begin{array}{c}\hfill \mathbf{Individual} \mathbf{information}: \begin{array}{c}{I}_P(T;X)=I^{\left(C\right)}(T;X,Y)+I^{\left(U_X\right)}(T;X,Y)\hfill \\ I_P(T;Y)=I^{\left(C\right)}(T;X,Y)+I^{\left(U_Y\right)}(T;X,Y).\hfill \end{array}\end{array}$$

(2)

This rule dictates that the information that can be revealed from X regarding T needs to be the summation of the unique information in X, and the common information in both X and Y. Note that the synergistic information is not included here, since it can only manifest when combining X and Y.

We now already have three linear equations for the four quantities in the decomposition, and the authors of [13] introduced the following definition of synergistic information:

$$\begin{array}{c}\hfill I^{\left(S\right)}(T;X,Y)\triangleq I_P(T;X,Y)-\underset{Q\in \mathcal{Q}}{min}{I}_Q(T;X,Y),\end{array}$$

(3)

where the set of distributions $\mathcal{Q}$ is defined as

$$\begin{array}{c}\hfill \mathcal{Q}=\{Q\in {\Delta }_{\mathcal{X}\times \mathcal{Y}\times \mathcal{T}}:Q_{T,X}=P_{T,X},Q_{T,Y}=P_{T,Y}\},\end{array}$$

(4)

where ${\Delta }_{\mathcal{X}\times \mathcal{Y}\times \mathcal{T}}$ is the probability simplex on $\mathcal{X}\times \mathcal{Y}\times \mathcal{T}$; in other words, $\mathcal{Q}$ is the set of all joint distributions such that the pairwise marginals of $P_{T,X}$ and $P_{T,Y}$ are preserved. As a consequence, this definition, together with (1) and (2), also specifies the values of the other three quantities $I^{\left(C\right)}(T;X,Y)$, $I^{\left(U_X\right)}(T;X,Y)$, and $I^{\left(U_Y\right)}(T;X,Y)$. It can be easily verified that in the examples discussed earlier, this decomposition gives the quantities matching our expectations.

It was shown in [13] that this PID definition enjoys several desirable properties, for example, the following: (1) All the quantities $I^{\left(C\right)}(T;X,Y)$, $I^{\left(U_X\right)}(T;X,Y)$, $I^{\left(U_Y\right)}(T;X,Y)$, and $I^{\left(S\right)}(T;X,Y)$ are non-negative. (2) They satisfy certain extremal relations in the sense that the shared information $I^{\left(S\right)}(T;X,Y)$ is the minimum among all possible definition under an additional axiom that the unique information only depends on the two marginal distributions. (3) They satisfy the symmetry, self-redundancy, and monotonicity axioms stipulated in [11].

3. Broadcast Channels

The broadcast channel is a well-studied communication system in classic information theory [30], where a transmitter wishes to send two independent messages to two receivers through a channel with a single input signal that induces two separate output signals; see Figure 1a. To accomplish this goal, the transmitter must encode the messages in such a way that the individual messages can be decoded by the intended receivers. Clearly, if the first receiver is completely ignored, the transmitter can send more information to the second receiver, and vice versa. In other words, there is a tradeoff between the amounts of information that can be sent to the two receivers. One particular important quantity here is the sum of the rates that can be supported on this channel, often referred to as the sum-rate capacity.

Next, we provide a rigorous definition of the broadcast channel capacity region, where the channel is allowed to be used many times at a time, i.e., a block code where the block length approaches infinity. A two-user broadcast channel is specified by an input alphabet $\mathcal{T}$, two output alphabets $\mathcal{X}$ and $\mathcal{Y}$, and a conditional probability distribution $P_{X,Y\mid T}$ that gives the channel law for each symbol $t\in \mathcal{T}$ and $(x,y)\in \mathcal{X}\times \mathcal{Y}$. The alphabets are usually assumed to be finite, though the results usually hold under more general assumptions [33].

Definition 1.

For a blocklength $n\in \mathbb{N}$, let $M_1$ and $M_2$ be two positive integers. An $(n,M_1,M_2)$-code for the broadcast channel $P_{X,Y\mid T}$ consists of the following:

• Two message sets:

  $${\mathcal{M}}_1=\{1,2,\dots ,M_1\}, {\mathcal{M}}_2=\{1,2,\dots ,M_2\}.$$
• An encoding function

  $$f:{\mathcal{M}}_1\times {\mathcal{M}}_2\to {\mathcal{T}}^n,$$

  which assigns to each pair of messages $(m_1,m_2)$ a length-n input sequence $T^n=(T_1,T_2,\dots ,T_n)$.
• Two decoding functions:

  $$g_1:{\mathcal{X}}^n\to {\mathcal{M}}_1, g_2:{\mathcal{Y}}^n\to {\mathcal{M}}_2,$$

  where ${\widehat{M}}_1=g_1\left(X^n\right)$ is the estimate of message $M_1$ by receiver 1, and ${\widehat{M}}_2=g_2\left(Y^n\right)$ is the estimate of message $M_2$ by receiver 2.

Definition 2.

For an $(n,M_1,M_2)$-code, let $(M_1,M_2)$ be chosen uniformly from ${\mathcal{M}}_1\times {\mathcal{M}}_2$. The average probability of error is defined as

$$P_e^{\left(n\right)}=Pr\left\{({\widehat{M}}_1,{\widehat{M}}_2)\ne (M_1,M_2)\right\}.$$

A sequence of codes (indexed by n) is said to have vanishing error probability if $P_e^{\left(n\right)}\to 0$ as $n\to \infty$.

Definition 3.

A rate pair$(R_1,R_2)$ is said to be achievable for the broadcast channel $P_{X,Y\mid T}$ if there exists a sequence of $\left(n,M_1^{\left(n\right)},M_2^{\left(n\right)}\right)$-codes with vanishing error probability such that

$$\underset{n\to \infty }{lim\; inf}{\displaystyle \frac{1}{n}}log{M}_1^{\left(n\right)}\ge R_1, \underset{n\to \infty }{lim\; inf}{\displaystyle \frac{1}{n}}log{M}_2^{\left(n\right)}\ge R_2.$$

The capacity region $\mathcal{C}$ of the two-user broadcast channel is the closure of the set of all achievable rate pairs. The sum-rate capacity is $C_{\mathrm{sum}}\triangleq {max}_{(R_1,R_2)\in \mathcal{C}}{R}_1+R_2$.

4. Main Result: An Operational Interpretation of PID

In this section, we provide the main result of this work, which is an operational interpretation of partial information decomposition.

4.1. PID via Sato’s Outer Bound

Researchers in the information theory community have made numerous efforts to identify a computable characterization of the capacity region of general broadcast channels (see the textbook [33] for a historical summary); yet, at this time, a complete solution is still elusive. Nevertheless, significant progress has indeed been made toward this goal. Particularly, Sato [34] provided an outer bound for $\mathcal{C}$, and it can be specialized to yield an upper bound for the sum-rate capacity of the general broadcast channel as follows:

$$\begin{array}{c}\hfill C_{\mathrm{sum}}\le C_{\mathrm{Sato}}\triangleq \underset{Q\in {\mathcal{Q}}^{\prime }}{min}\underset{P_T}{max}{I}_{P_T{Q}_{X,Y\mid T}}(X,Y;T),\end{array}$$

(5)

where the set ${\mathcal{Q}}^{\prime }$ is defined as

$$\begin{array}{c}\hfill {\mathcal{Q}}^{\prime }=\{Q\in {\Delta }_{\mathcal{X}\times \mathcal{Y}\mid \mathcal{T}}:Q_{X|T}=P_{X|T},Q_{Y|T}=P_{Y|T}\},\end{array}$$

(6)

i.e., the set of conditional distributions for which the marginal conditional distributions $P_{X|T}$ and $P_{Y|T}$ are preserved. The inner maximization is over the possible marginal distribution of the random variable $P_T$ in the alphabet $\mathcal{T}$. The form already bears a certain similarity to (4). Note that for channels on general alphabets (i.e., not necessarily optimized on a compact space), the maximization should be replaced by the supremum and the minimization by the infimum. Due to the minimax form, the meaning is not yet clear, but the max–min inequality (weak duality) implies that

$$\begin{array}{c}\hfill \underset{Q\in {\mathcal{Q}}^{\prime }}{min}\underset{P_T}{max}{I}_{P_T{Q}_{X,Y\mid T}}(X,Y;T)\ge \underset{P_T}{max}\underset{Q\in {\mathcal{Q}}^{\prime }}{min}{I}_{P_T{Q}_{X,Y\mid T}}(X,Y;T)=\underset{P_T}{max}\underset{Q\in \mathcal{Q}}{min}{I}_{Q_{X,Y,T}}(X,Y;T),\end{array}$$

(7)

where the equality is by the definition of ${\mathcal{Q}}^{\prime }$, and the set $\mathcal{Q}$ is exactly the one defined in (4) with $Q_{X,T}=P_T{Q}_{X|T}$ and $Q_{Y,T}=P_T{Q}_{Y|T}$. The inner minimization of this form is exactly the same as the second term in (3). Though the max–min form does not yield a true upper bound of the sum-rate capacity, in the PID setting we consider, $P_T$ is always fixed; therefore, the max–min and min–max forms are in fact equivalent in this setting. The equivalence in mathematical forms does not fully explain the significance of this connection, and we will need to consider Sato’s bound more carefully. Let us define the following quantity for notational simplicity:

$$\begin{array}{c}\hfill R_{P_T}\triangleq \underset{Q\in {\mathcal{Q}}^{\prime }}{min}{I}_{P_T{Q}_{X,Y\mid T}}(X,Y;T).\end{array}$$

(8)

Sato’s outer bound was derived using the following argument. For a channel that has a single input signal T and a single output signal Y on a channel $P_{Y|T}$, Shannon’s channel coding theorem [28] states that the channel capacity is given by ${max}_{P_T}{I}_{P_T{P}_{Y|T}}(Y;T)$. Moreover, for any fixed distribution $P_T$, the rate $I_{P_T{P}_{Y|T}}(Y;T)$ is achievable, where the probability distribution $P_T$ represents the statistical signaling pattern of the underlying codes. Turning our attention back to the broadcast channel with a transition probability $P_{X,Y|T}$, if the receivers are allowed to cooperate fully and share the two output signals X and Y—i.e., they become a single virtual receiver (see Figure 1b)—then clearly the maximum rate achievable would be ${max}_{P_T}{I}_{P_T{Q}_{X,Y\mid T}}(X,Y;T)$. However, Sato further observed that the error probability of any code should not differ on any broadcast channel $Q_{X,Y\mid T}\in {\mathcal{Q}}^{\prime }$, even if the transition distribution $Q_{X,Y\mid T}$ is different from the true broadcast channel transition probability $P_{X,Y\mid T}$. This is because the channel outputs only depend on the marginal transition probabilities $P_{X|T}$ and $P_{Y|T}$, respectively, and the decoders only use their respective channel outputs to decode. Therefore, we can obtain an upper bound by choosing the worst channel configuration $Q\in \mathcal{Q}$, i.e., the outer minimization in (5).

With the interpretation of Sato’s upper bound above, it becomes clear that $R_{P_T}$ is essentially an upper bound on the sum rate of the broadcast channel where the receivers are not allowed to cooperate, when the input signaling pattern is fixed to follow $P_T$. On the other hand, the quantity $I_{P_T{P}_{X,Y\mid T}}(X,Y;T)$ is the rate that can be achieved by allowing the two receivers to fully cooperate, also with $P_T$ being the input signaling pattern. In this sense, $I^{\left(S\right)}(T;X,Y)$ defined in (3) is a lower bound on the difference between the sum rate with full cooperation and that without any cooperation, with the input signaling pattern following $P_T$.

This connection provides an operational interpretation of PID for general distributions. Essentially, synergistic information can be viewed as a surrogate of the cooperative gain. When this lower bound is in fact also achievable, $I^{\left(S\right)}(T;X,Y)$ would be exactly equal to the cooperative gain. In the corresponding learning setting, it is the difference between what can be inferred about T by using both X and Y in a non-cooperative manner, and what can be inferred by using them jointly. This indeed matches our expectations for the synergistic information. In the next subsection, we consider a special case when Sato’s bound is indeed achievable, and the lower bound mentioned above becomes exact.

In one sense, this operational interpretation is quite intuitive as explained above, but on the other hand, it is also quite surprising. For example, in broadcast channels, a more general setup allows the transmitter to also send a common message to both receivers [35,36], in addition to the two individual messages to the two respective receivers. It would appear plausible to expect this generalized setting to be more closely connected to the PID setting with the common message related to the common information, yet this turns out to be not the case here. Moreover, a dual communication problem studied in information theory is the multiple access channel (see, e.g., [28]), where two transmitters wish to communicate to the same receiver simultaneously. The readers may also wonder if an operational meaning should be extracted on this channel, instead of on the broadcast channel. However, note that in the PID setting, we are inferring T from X and Y, which is similar to the decoding process in the broadcast channel, instead of the multiple access channel. Moreover, in the multiple access channel, the joint distribution of the two transmitters’ inputs is always independent when the two transmitters cannot cooperate, and this will not match the PID setting under consideration. Another seemingly related problem setting studied in the information theory literature is the common information between two random variables [37,38]; however, for the PID defined in [13], this approach also does not yield a meaningful interpretation.

4.2. Gaussian MIMO Broadcast Channel and Gaussian PID

One setting where a full capacity region characterization is indeed known is the Gaussian multiple-input multiple-output (MIMO) channel [32,39]. In the two-user Gaussian MIMO broadcast channel, the channel transition probability $P_{\overrightarrow{X}|\overrightarrow{T}}$ and $P_{\overrightarrow{Y}|\overrightarrow{T}}$ are given, with $\overrightarrow{T}$ being the transmitter input variable, and $\overrightarrow{X},\overrightarrow{Y}$ the channel outputs at the two individual receivers. The channel is usually defined as follows:

$$\begin{array}{cc}& \overrightarrow{X}=H_X\overrightarrow{T}+{\overrightarrow{n}}_X\hfill \end{array}$$

(9)

$$\begin{array}{cc}& \overrightarrow{Y}=H_Y\overrightarrow{T}+{\overrightarrow{n}}_Y,\hfill \end{array}$$

(10)

where $H_X$ and $H_y$ are two channel matrices, the additive noise vector ${\overrightarrow{n}}_X$ is independent of $\overrightarrow{T}$, and similarly, ${\overrightarrow{n}}_Y$ is independent of $\overrightarrow{T}$. For a fixed input signaling distribution $P_T$, the pairwise marginal distributions $P_{\overrightarrow{T},\overrightarrow{X}}$ and $P_{\overrightarrow{T},\overrightarrow{Y}}$ are well specified. Conversely, for any joint distribution $P_{\overrightarrow{T},\overrightarrow{X},\overrightarrow{Y}}$, where the marginals $P_{\overrightarrow{T},\overrightarrow{X}}$ and $P_{\overrightarrow{T},\overrightarrow{Y}}$ are jointly Gaussian, respectively, we can represent their relation in the form above via a Gram–Schmidt orthogonalization. Note that the joint distribution of ${\overrightarrow{n}}_X,{\overrightarrow{n}}_Y$ is not fully specified here, as the noise vectors ${\overrightarrow{n}}_X$ and ${\overrightarrow{n}}_Y$ are not necessarily jointly Gaussian, but can be dependent in a more sophisticated manner. The standard Gaussian MIMO broadcast problem usually specifies the noises zero-mean Gaussian with certain fixed covariances, and there is also a covariance constraint on the transmitter’s signaling $\overrightarrow{T}$. The problem can be further simplified using certain linear transformations, as discussed below.

Let us assume ${\Sigma }_{{\overrightarrow{n}}_X}$ and ${\Sigma }_{{\overrightarrow{n}}_Y}$ are full rank for now (when ${\Sigma }_{{\overrightarrow{n}}_X}$ and ${\Sigma }_{{\overrightarrow{n}}_Y}$ are not fully rank, a limiting argument can be invoked to show the same conclusion holds), and in this case, it is clearly without loss of generality to assume that ${\Sigma }_{{\overrightarrow{n}}_i}$’s are in fact identity matrices, since otherwise, we can perform receiver-side linear transforms to make them so, i.e., through a transformation based on the eigenvalue decomposition of ${\Sigma }_{{\overrightarrow{n}}_X}$ and ${\Sigma }_{{\overrightarrow{n}}_Y}$, respectively. For the same reason, we can assume ${\Sigma }_{\overrightarrow{T}}$ is an identity matrix, through a linear transformation at the transmitter. These reductions to independent noise and independent channel input are often referred to as the transmitter precoding transformation and the receiver precoding transformations in the communication literature; see, e.g., [39].

For the two-user Gaussian broadcast channel, the worst channel configuration problem we discussed in the general setting is essentially the least favorable noise problem considered by Yu and Cioffi [39] with the simplification above, where the noise relation between ${\overrightarrow{n}}_X$ and ${\overrightarrow{n}}_Y$ needs to be identified for a channel that makes it the hardest to communicate. It was shown in [39] that the least favorable noise problem can be recast as an optimization problem:

$$\begin{array}{cc}\hfill \mathrm{minimize}: & log{\displaystyle \frac{|H{\Sigma }_{\overrightarrow{T}}{H}^T+{\Sigma }_{\overrightarrow{n}}|}{|{\Sigma }_{\overrightarrow{n}}|}}\hfill \end{array}$$

(11)

$$\begin{array}{cc}\hfill \mathrm{subject} \mathrm{to}: & {\Sigma }_{{\overrightarrow{n}}_x}=I,\hfill \end{array}$$

(12)

$$\begin{array}{cc}& {\Sigma }_{{\overrightarrow{n}}_y}=I,\hfill \end{array}$$

(13)

$$\begin{array}{cc}\hfill & {\Sigma }_{\overrightarrow{n}}⪰0,\hfill \end{array}$$

(14)

where $H^T=[H_X^T,H_Y^T]$, and $\overrightarrow{n}=({\overrightarrow{n}}_X,{\overrightarrow{n}}_Y)$, when $H{\Sigma }_{\overrightarrow{T}}{H}^T$ is nonsingular. It can be shown that the problem is convex.

Yu and Cioffi also showed that in this setting, Sato’s upper bound is achievable—i.e., it is exactly the sum-rate capacity. Moreover, for any input signaling $P_{\overrightarrow{T}}$ that is Gaussian-distributed, the corresponding $R_{P_T}$ can be achieved on this broadcast channel through a more sophisticated scheme known as dirty-paper coding [40]. Therefore, when the pairwise marginals $P_{\overrightarrow{T},\overrightarrow{X}}$ and $P_{\overrightarrow{T},\overrightarrow{X}}$ are jointly Gaussian, respectively, the synergistic information $I^{\left(S\right)}(\overrightarrow{T};\overrightarrow{X},\overrightarrow{Y})$ is exactly the cooperative gain of the corresponding Gaussian broadcast channel using this specific input signaling pattern $P_{\overrightarrow{T}}$. The connection in the Gaussian setting is of particular interest, given the practical importance of the Gaussian PID, which was thoroughly explored in [41].

4.3. Revisiting the Examples

Let us now revisit the examples given earlier, and attempt to understand them in the broadcast channel setting:

1.

Common information dominant: Here, the two receivers both know the transmitted signal completely, and therefore, there is no difference even if they are allowed to cooperate, and the cooperative gain is 0.

2.

Synergistic information dominant: Clearly, here, T is a uniformly distributed Bernoulli random variable. When the two receivers are not allowed to cooperate, they cannot decode anything because the channel $T\to X$ is completely noisy, and similarly for the channel $T\to Y$. When the receivers are allowed to cooperate, then from X and Y, we can completely recover T—i.e., the channel becomes noiseless. Therefore, the full information in T can be decoded. The cooperative gain is then one, matching the synergistic information.

3.

Component-wise decomposition: Here, T has three uniformly distributed Bernoulli random variable components, mutually independent of each other. In the channel $T\to X$, the first component is noiseless, and the second component is completely noisy. This is similarly the case for $T\to Y$. When the receivers are not allowed to cooperate, the sum rate on the channel is two. However, when the channels are allowed to cooperate, the last component in T becomes useful (noiseless), and the total communication rate becomes three. Therefore the cooperative gain is $3-2=1$, equal to the synergistic information.

5. Conclusions

We provide an operational interpretation of the PID given in [13] via a connection to the well-studied broadcast channel capacity in the information theory literature. The synergistic information is directly connected to the cooperative gain on the corresponding broadcast channel, either quantitatively exactly equal or being a lower bound. This interpretation can help us better understand why such decomposition can be used to guide model selection in machine learning and in the analysis of biological signals, and potentially design new methods to utilize it.

We note that in the information theory literature, broadcast channels with cooperative receivers have been studied carefully [42], where the two receivers are allowed to communicate on rate-limited links. It may be of interest to consider the corresponding notion of partial information decompositions, when the synergistic information can be parametrized by the degree of cooperation between the receivers. Moreover, the concept of Renyi information has found applications in many fields [43], and the decomposition of Renyi information has not been explored; it may be beneficial to study how our approach can be applied to such decompositions.