An Umbrella Converse for Data Exchange: Applied to Caching, Computing, and Shuffling ^†

Abstract

The problem of data exchange between multiple nodes with storage and communication capabilities models several current multi-user communication problems like Coded Caching, Data Shuffling, Coded Computing, etc. The goal in such problems is to design communication schemes which accomplish the desired data exchange between the nodes with the optimal (minimum) amount of communication load. In this work, we present a converse to such a general data exchange problem. The expression of the converse depends only on the number of bits to be moved between different subsets of nodes, and does not assume anything further specific about the parameters in the problem. Specific problem formulations, such as those in Coded Caching, Coded Data Shuffling, and Coded Distributed Computing, can be seen as instances of this generic data exchange problem. Applying our generic converse, we can efficiently recover known important converses in these formulations. Further, for a generic coded caching problem with heterogeneous cache sizes at the clients with or without a central server, we obtain a new general converse, which subsumes some existing results. Finally we relate a “centralized” version of our bound to the known generalized independence number bound in index coding and discuss our bound’s tightness in this context.

1. Introduction and Main Result

Consider a system of K nodes, denoted by $\left[K\right]\triangleq \{1,\cdots ,K\}$, each of which have (not necessarily uniform) storage. The nodes can communicate with each other through a noiseless bus link, in which transmissions of any node is received by all others. Each node possesses a collection of data symbols (represented in bits) in its local storage and demands another set of symbols present in other nodes. We formalize this as a data exchange problem.

Definition 1.

A data exchange problem on a set of K nodes involving a collection B of information bits is given by the following:

• a collection $\{C_i:i\in \left[K\right]\},$ where $C_i\subset B$ denotes the subset of data present in node i,
• a collection $\{D_i:i\in \left[K\right]\}$ where $D_i\subset {\cup }_{j\ne i}{C}_j\setminus C_i$ denotes the set of bits demanded by node i.

The above data exchange problem models a number of cache-enabled multi-receiver communication problems studied recently in the coding theory community, including Coded Caching [1], Coded Distributed Computing [2,3], Coded Data Shuffling [4,5,6], and Coded Data Rebalancing [7]. In [8], a special case of our general problem here was considered in the name of cooperative data exchange, where the goal was to reach a state in which all nodes have all the data in the system.

A solution to a given data exchange problem involves communication between the nodes. Each node i encodes the symbols in $C_i$ into a codeword of length $l_i$ and sends it to all other nodes. The respective demanded symbols at any node is then to be decoded using the received transmissions from all the other nodes and the node’s own content.

Formally, a communication scheme for the given data exchange problem consists of a set of encoding functions $\mathsf{\Phi }\triangleq \{{\varphi }_i:i\in \left[K\right]\}$ and decoding functions $\mathsf{\Psi }\triangleq \{{\psi }_i:i\in \left[K\right]\}$, defined as follows.

$$\begin{array}{cc}\hfill {\varphi }_i:& {\{0,1\}}^{|C_i|}\to {\{0,1\}}^{l_i},(\mathrm{for} \mathrm{some} \mathrm{non-negative} \mathrm{integer}{l}_i)\hfill \\ \hfill {\psi }_i:& {\{0,1\}}^{|C_i|}\times {\{0,1\}}^{{\displaystyle \sum _{j\ne i}}{l}_j}\to {\{0,1\}}^{|D_i|},\hfill \end{array}$$

such that

$$\begin{array}{c}\hfill {\psi }_i\left(C_i,\{{\varphi }_j\left(C_j\right):j\ne i\}\right)=D_i.\end{array}$$

The communication load of the above scheme is defined as the total number of bits communicated, i.e.,

$$L(\mathsf{\Phi },\mathsf{\Psi })\triangleq \sum _{i\in \left[K\right]}{l}_i.$$

The optimal communication load is then denoted by

$$L^{*}\triangleq \underset{\mathsf{\Phi },\mathsf{\Psi }}{min}L(\mathsf{\Phi },\mathsf{\Psi }).$$

The central result in this work is Theorem 1 in Section 1.1, which is a lower bound on the optimal communication load $L^{*}$. Using this lower bound, we recover several important converse results of cache-enabled communication problems studied in the literature, including Coded Caching (Section 2), Data Shuffling (Section 3), and Distributed Computing (Section 4). In each of these sections, we briefly review each setting and then apply Theorem 1 to recover the respective converses. As a result, the proofs of these existing converses are also made simpler than what is already available in the literature for the respective settings. The generic structure of the converse proofs obtained using our data exchange bound is presented in Section 1.2. This structure includes three steps, which we also highlight at the appropriate junctures within the proofs themselves. The close relationship between these problems is quite widely known. This work gives a further formal grounding to this connection, by abstracting the common structure of these converses into a general form, which can potentially be applied to other new data exchange problems as well.

Apart from recovering existing results, more importantly we also use our data exchange lower bound to obtain new tight converse results for some settings, while improving tightness results of some known bounds. Specifically, we present a new converse for a generic coded caching setting with multi-level cache sizes. Using this, we are able to close the gap to optimality for some known special cases of this generic setting (Section 2.1). In Section 5, we show the relationship between a “centralized” version of our data exchange lower bound and an existing bound for index coding known as the $\alpha$-bound or the generalized independence number bound [9]. In general, we find that our bound is weaker than the $\alpha$-bound. However, for unicast index coding problems, we identify the precise conditions under which our data exchange bound is equal to the $\alpha$-bound. In Section 6, we discuss the application of our data exchange lower bound to more generalized index coding settings, specifically distributed index coding [10,11] and embedded index coding [12].

Notation: For positive integer a, let $\left[a\right]\triangleq \{1,\cdots ,a\}$. For a set S, we denote by $S\setminus k$ the set of items in S except for the item k, and represent the union $S\cup \left\{k\right\}$ as $S\cup k$. The binomial coefficient is denoted by $\left(\genfrac{}{}{0pt}{}{n}{k}\right)$, which is zero if $k>n.$ The set of all t-sized subsets of a set A is denoted by $\left(\genfrac{}{}{0pt}{}{A}{t}\right).$

1.1. A Converse for the Data Exchange Problem

In this subsection, we will obtain a lower bound on the optimal communication load of the general data exchange problem defined in Section 1. This is the central result of this work. The predecessor to the proof technique of our data exchange lower bound is in [3], which first presented an induction based approach for the converse of the coded distributed computing setting. Our proof uses a similar induction technique.

Given a data exchange problem and for $P,Q\subset \left[K\right]$ such that $P\ne \varnothing$, let $a_P^Q$ denote the number of bits which are stored in every node in the subset of nodes Q and stored in no other node, and demanded by every node in the subset P and demanded by no other node, i.e.,

$$\begin{array}{c}\hfill a_P^Q\triangleq |\left({\cap }_{i\in P}{D}_i\right)\cap \left({\cap }_{j\in Q}{C}_j\right)\setminus \left({\cup }_{j\notin Q}{C}_j)\cup \left({\cup }_{i\notin P}{D}_i\right)\right)|.\end{array}$$

(1)

Note that, by definition, $a_P^Q=0$ under the following conditions.

• If $P\cap Q\ne \varnothing$, as the bits demanded by any node are absent in the same node.
• If $Q=\varnothing$, by Definition 1.

Theorem 1 gives a lower bound on the optimal communication load of a given data exchange problem. The proof of the theorem is relegated to Appendix A. The idea of the proof is as follows. If we consider only two nodes in the system, say $\left[K\right]=\{1,2\}$, then each of the 2 nodes has to transmit whatever bits it has which are demanded by the other node, i.e., $L^{*}\ge a_{\left\{2\right\}}^{\left\{1\right\}}+a_{\left\{1\right\}}^{\left\{2\right\}}$. The proof of the theorem uses this as a base case and employs an induction technique to obtain a sequence of cut-set bounds leading to the final expression.

Theorem 1.

$$L^{*}\ge \sum _{P\subset \left[K\right]}\sum _{Q\subset \left[K\right]\setminus P}\frac{\left|P\right|}{\left|P\right|+\left|Q\right|-1}{a}_P^Q.$$

Theorem 1, along with the observation that $a_{\varnothing }^Q=0=a_P^{\varnothing }$ gives us the following corollary, which is a restatement of Theorem 1.

Corollary 1.

Let

$$n(p,q)\triangleq \sum _{\begin{array}{c}P,Q\subset \left[K\right]:\\ \left|P\right|=p,\left|Q\right|=q,P\cap Q=\varnothing \end{array}}{a}_P^Q$$

denote the total number of bits present exactly in q nodes and demanded exactly by p (other) nodes. Then,

$$\begin{array}{c}\hfill L^{*}\ge \sum _{p=1}^{K-1}\sum _{q=1}^{K-p}\frac{p}{p+q-1}n(p,q).\end{array}$$

(2)

Remark 1.

In [13], the authors presented an essentially identical bound (Lemma 1, [13]) as Corollary 1 in the setting of coded distributed computing. The proof given in [13] for this lemma also generalizes the arguments presented in [3], as does this work. Our present work considers a general data exchange problem and derives the lower bound in Theorem 1 for the communication load in such a setting. We had derived this lower bound independently in the conference version of this paper [14], and only recently came to know about the bound in [13]. In subsequent sections, we show how to use this bound to recover converses for various multi-terminal communication problems considered in the literature in recent years, and also obtain new converses for some settings. We also discuss, in Section 5, the looseness of Theorem 1 by considering a centralized version of the data exchange problem and comparing our bound with the generalized independence number bound in index coding. In Section 6, we discuss the application of our data exchange bound to more generalized index coding settings. These are the novel features of our present work, compared to the bound in Lemma 1 of [13].

1.2. A Generic Outline of the Converse Proofs Presented in This Paper

In this work, we derive converse bounds for various settings in coded caching, coded distributed computing, and coded data shuffling using the bound in Theorem 1. Some of these converse bounds are already available in the literature, while others are novel. Each setting enjoins some constraints on the size of the demands and the size of the pre-stored content at each node. The bound in Theorem 1 applies for the setting in which the nodes have some predetermined local storage and some specific demanded bits. However, the settings of coded caching, coded distributed computing, and coded data shuffling permit the design of the initial storage so that the communication load is minimized. Further, the optimal communication load as defined in the literature for some of these settings involves maximization over all possible demand configurations, keeping only the size of the demands fixed. Keeping with these specifics, our bound in Theorem 1 must be tuned for each setting to obtain the respective converse, as captured by the three following steps which describe the generic structure behind our converse proofs.

• Applying Theorem 1 to the present setting, we obtain a lower bound expression on the communication load, assuming an arbitrary choice of demands across the nodes and some arbitrary but fixed storage across the nodes.
• “Symmetrization” step: In this step, the lower bound expression obtained in the previous step is averaged over some carefully chosen configurations of demanded bits at the nodes. This step helps to remove the dependency of the lower bound on the specific choice of demands.
• Refine the averaged bound by imposing the constraints on the size of the initial storage at the nodes, and using convexity of terms inside the averaged bound to obtain the final expression of the bound. This step helps to remove the dependency of the converse on the specific initial storage configuration at the nodes.

These three steps enable us to give simpler proofs to those in the literature for known converses, and also obtain novel converses for some variants of the same problems. Further, it also illustrates the generic nature of the data exchange bound of Theorem 1. In the converse proofs that are to follow in this paper, we will highlight these steps at the appropriate junctures.

2. Coded Caching

In this section, we apply Theorem 1 to recover the lower bound obtained in [15] for the problem of coded caching introduced in [1]. Further, using Theorem 1, we prove in Section 2.1 a new converse for a generic coded caching problem under multiple cache size settings. This provides new converses for some existing settings in literature, and also tightens bounds in some others. In Section 2.2, we recover a converse for coded caching with multiple file requests. In Section 2.3, we recover the converse for coded caching with decentralized cache placement.

We now describe the main setting of this section. In the coded caching system introduced in [1], there is one server connected via a noiseless broadcast channel to K clients indexed as $\left[K\right]$. The server possesses N files, each of size F bits, where the files are indexed as $W_i:i\in \left[N\right]$. Each client contains local storage, or a cache, of size $MF$ bits, for some $M\le N$. We call this a $(K,M,N,F)$ coded caching system. Figure 1 illustrates this system model.

The coded caching system operates in two phases: in the caching phase which occurs during the low-traffic periods, the caches of the clients are populated by the server with some (uncoded) bits of the file library. This is known as uncoded prefetching. In this phase, the demands of the clients are not known. We denote the caching function for node k as ${\zeta }_k,$ and thus the cache content at client k at the end of the caching phase is denoted as $Z_k\triangleq {\zeta }_k\left(\{W_i:i\in \left[N\right]\}\right)$.

In the delivery phase which occurs during the high-traffic periods, each client demands one file from the server, and the server makes transmissions via the broadcast channel to satisfy the client demands. Let the demanded file at client k be $W_{d_k}$, where $d_k\in \left[N\right].$ The server uses an encoding function $\varphi$ to obtain coded transmissions $X=\varphi \left(\{W_{d_k}:k\in \left[K\right]\}\right)$ such that each client $k\in \left[K\right]$ can employ a decoding function ${\psi }_k$ to decode its demanded file using the coded transmissions and its cache content, i.e., ${\psi }_k(X,Z_k)=W_{d_k}.$

The communication load $L_c(\{{\zeta }_k:k\in \left[K\right]\},\varphi ,\{{\psi }_k:k\in \left[K\right]\})$ of the above coded caching scheme is the number of bits transmitted in the delivery phase (i.e., the length of X) in the worst case (where “worst case” denotes maximization across all possible demands). The optimal communication load denoted by $L_c^{*}$, is then defined as

$$\begin{array}{c}\hfill L_c^{*}\triangleq \underset{\{{\zeta }_k:k\in \left[K\right]\},\varphi ,\{{\psi }_k:k\in \left[K\right]\}}{min}{L}_c(\{{\zeta }_k:k\in \left[K\right]\},\varphi ,\{{\psi }_k:k\in \left[K\right]\}).\end{array}$$

(3)

For this system model, when $\frac{MK}{N}\in \mathbb{Z}$,the work in [1] proposed a caching and delivery scheme which achieves a communication load (normalized by the size of the file F) given by $K(1-\frac{M}{N})min\left\{\frac{1}{1+\frac{MK}{N}},\frac{N}{K}\right\}$. In [15], it was shown that, for any coded caching scheme with uncoded cache placement, the optimal communication load is lower bounded by $L_c^{*}\ge \frac{K(1-\frac{M}{N})F}{1+MK/N}$. Therefore, it was shown that, when $K\le N$ and $\frac{MK}{N}\in \mathbb{Z}$, the scheme in [1] is optimal.

In the present section, we give another proof of the lower bound for coded caching derived in [15]. We later discuss the case of arbitrary $K,N$ in Remark 2.

We now proceed with restating the lower bound from [15]. Note that these converses are typically normalized by the file size in literature, however we recall them in their non-normalized form, in order to relate them with our data exchange problem setting.

Theorem 2

([15]). Consider a $(K,M,N,F)$ coded caching system with $K\le N$. The optimal communication load $L_c^{*}$ in the delivery phase satisfies

$$L_c^{*}\ge \frac{K(1-M/N)}{1+MK/N}F.$$

Proof based on Theorem 1.

We assume that the caching scheme and delivery scheme of the coded caching scheme are designed such that the communication load $L_c$ is exactly equal to the optimal load $L_c^{*}.$ Let the K client demands in the delivery phase be represented by a demand vector $\mathit{d}=(d_1,\cdots ,d_K)$, where $d_k\in \left[N\right]$ denotes the index of the demanded file of the client k. We are interested in the worst case demands scenario; this means we can assume that all the demanded files are distinct, i.e., $d_k\ne d_{k^{\prime }}$ for all $k\ne k^{\prime }$ to bound $L_c^{*}$ from below, without loss of generality.

We observe that a $(K,M,N,F)$ coded caching problem during the delivery phase satisfies Definition 1 of a data exchange problem on $K+1$ nodes indexed as $\{0,1,\cdots ,K\}$, where we give the index 0 to the server node and include this in the data exchange system. Before proceeding, we remark that the below proof gives a lower bound where all $K+1$ nodes in the system may transmit, whereas in the coded caching system of [1] only the server can transmit. Thus, any lower bound that we obtain in this proof applies to the setting in [1] also.

Clearly in the equivalent data exchange problem, the node 0 (the server) does not demand anything, but has a copy of all the bits in the entire system. With these observations, we have by definition of $a_P^Q$ in (1)

$$\begin{array}{c}\hfill a_P^Q=0,\mathrm{if}0\notin Q\mathrm{or}\mathrm{if}P\notin \left(\genfrac{}{}{0pt}{}{\left[K\right]}{1}\right),\end{array}$$

(4)

where the quantities $a_P^Q$ clearly depend on the demand vector $\mathit{d}$.

We thus use a new set of variables: for each $k\in \left[K\right]$, $Q\subset \left[K\right]$, and given demands $\mathit{d}=(d_1,\cdots ,d_K)$, let $c_k^Q\left(\mathit{d}\right)$ denote the number of bits demanded by receiver node k that are available only at the nodes $Q\cup \left\{0\right\}$, i.e.,

$$\begin{array}{c}\hfill c_k^Q\left(\mathit{d}\right)\triangleq a_{\left\{k\right\}}^{Q\cup 0}.\end{array}$$

(5)

Using these definitions, we proceed following the three steps given in Section 1.2.

Applying Theorem 1: By Theorem 1, we have the following lower bound for demand vector $\mathit{d}$

$$\begin{array}{cc}\hfill L_c^{*}=L_c& \ge \sum _{P\subset \left[K\right]\cup \left\{0\right\}}\sum _{Q^{\prime }\subset \left[K\right]\cup \left\{0\right\}\setminus P}\frac{\left|P\right|}{\left|P\right|+|Q^{\prime }|-1}{a}_P^{Q^{\prime }}\hfill \\ \hfill & =\sum _{k=1}^K\sum _{Q\subset \left[K\right]\setminus k}\frac{1}{\left|Q\right|+1}{c}_k^Q\left(\mathit{d}\right),\hfill \end{array}$$

(6)

where (6) is obtained from (4) and (5).

“Symmetrizing” (6) over carefully chosen demand vectors: We now consider the averaging of bounds of type (6) over a chosen collection of N demand vectors, given by

$$\begin{array}{cc}\hfill \mathcal{D}& \triangleq \left\{\left(j{\oplus }_{N}0,j{\oplus }_{N}1,\cdots ,j{\oplus }_N(K-1)\right):j=0,\cdots ,N-1\right\}\hfill \end{array}$$

(7)

where $j{\oplus }_{N}i\triangleq ((j+i)modN)+1.$

That is, $\mathcal{D}$ contains the demand vectors consisting of consecutive K files, starting with each of the N files as the demand of the first client.

Averaging (6) through the set of N demand vectors in $\mathcal{D}$, the lower bound we obtain is

$$\begin{array}{cc}\hfill L_c^{*}& \ge \frac{1}{N}\sum _{\mathit{d}\in \mathcal{D}}\sum _{k=1}^K\sum _{Q\subset \left[K\right]\setminus k}\frac{1}{\left|Q\right|+1}{c}_k^Q\left(\mathit{d}\right).\hfill \end{array}$$

(8)

Let $b_n^Q$ denote the number of bits of file n stored only in $Q\cup \left\{0\right\}$. Then, in the above sum, $b_n^Q=c_k^Q\left(\mathit{d}\right)$ if and only if $d_k=n$. This happens precisely once in the collection of N demand vectors in $\mathcal{D}$. Thus, we have

$$\begin{array}{cc}\hfill L_c^{*}& \ge \frac{1}{N}\sum _{k=1}^K\sum _{Q\subset \left[K\right]\setminus k}\sum _{\mathit{d}\in \mathcal{D}}\frac{1}{\left|Q\right|+1}{c}_k^Q\left(\mathit{d}\right)\hfill \end{array}$$

$$\begin{array}{cc}\hfill & =\frac{1}{N}\sum _{k=1}^K\sum _{Q\subset \left[K\right]\setminus k}\sum _{n=1}^N\frac{1}{\left|Q\right|+1}{b}_n^Q\hfill \end{array}$$

(9)

$$\begin{array}{cc}\hfill & =F\sum _{Q\subset \left[K\right]}\sum _{n=1}^N\left(\frac{K-\left|Q\right|}{\left|Q\right|+1}\right)\frac{b_n^Q}{NF}\hfill \end{array}$$

(10)

where (10) follows as for a fixed n and Q, $k\in \left[K\right]\setminus Q$ in (9), and by multiplying and dividing by F.

Refining the bound (10) by using the constraints of the setting: Now, by definition, ${\sum }_n{\sum }_{Q\subset \left[K\right]}{b}_n^Q=NF$, and thus $b_n^Q/NF:n\in \left[N\right],Q\subset \left[K\right]$, denotes a probability mass function. Furthermore, ${\sum }_{Q\subset \left[K\right]}\left|Q\right|b_n^Q\le KMF.$ As $(K-x)/(1+x)$ is a convex decreasing function for $x\ge 0$, using Jensen’s inequality, we have $L_c^{*}\ge (K-x)/(1+x)$, where

$$x=\sum _n\sum _{Q\subset \left[K\right]}\left|Q\right|\frac{b_n^Q}{NF}\le \frac{KMF}{NF}=\frac{KM}{N}.$$

Thus, we get $L_c^{*}\ge \frac{K(1-M/N)}{1+MK/N}F,$ which completes the proof. □

Remark 2.

In the previous part of this section, we have shown the converse for the worst case communication load $L_c^{*}$ for coded caching in the regime of $K\le N$. We now consider a general coded caching setup with arbitrary $K,N$ values and cache size M. Consider a positive integer $N_u\le min\{N,K\}.$ For a fixed caching scheme denoted by $\zeta =\{{\zeta }_k:k\in \left[K\right]\},$ let the minimum communication load for satisfying the clients, maximized across all possible demand vectors with exactly $N_u$ distinct files in each of the demand vectors, be denoted as $L_c^{*}(N_u,\zeta ).$

In the work [16], it was shown that for $t\triangleq \frac{MK}{N},$

$$\begin{array}{c}\hfill L_c^{*}(N_u,\zeta )\ge g_{{}_{N_u}}\left(t\right),\end{array}$$

(11)

where $g_{{}_{N_u}}\left(x\right)$ is defined as the lower convex envelope of the points

$$\mathcal{P}\left(N_u\right)=\left\{\left(x,\frac{\left(\genfrac{}{}{0pt}{}{K}{x+1}\right)-\left(\genfrac{}{}{0pt}{}{K-N_u}{x+1}\right)}{\left(\genfrac{}{}{0pt}{}{K}{x}\right)}F\right):x\in \{0,\cdots ,K\}\right\}.$$

Note that $g_{{}_{N_u}}\left(t\right)$ is independent of $\zeta .$ For this general setting, the optimal worst case load $L_c^{*}$, as defined in (3), satisfies

$$L_c^{*}=\underset{\zeta }{min}{L}_c^{*}(min\{N,K\},\zeta ).$$

Thus, from (11), we get

$$\begin{array}{c}\hfill L_c^{*}\ge g_{{}_{min\{N,K\}}}\left(t\right),\end{array}$$

(12)

which is the converse bound on the worst case communication load proved in [16] for this general scenario. In Appendix B, we use our data exchange bound in Theorem 1 to recover (11), which therefore shows (12).

2.1. Server-Based and Server-Free Coded Caching with Heterogeneous Cache Sizes at Clients

So far we have discussed the coded caching scenario where there is a central server containing the entire file library and the client cache sizes are homogeneous, i.e., the same at all clients. We now describe a generalization of the result in Theorem 2 to the case of systems in which the clients have heterogeneous cache sizes, with either a centralized server present or absent. The proof of this is easily obtained from our data exchange bound in Theorem 1. To the best of our knowledge, a converse for this general setting is not known in the literature. Using this converse, we can derive new converses and tighten existing converses for various special cases of this setting, which include widely studied coded caching settings, such as device-to-device coded caching [17].

Consider a coded caching system with N files (each of size F) with K client nodes denoted by a set ${\mathcal{K}}_T$. We shall indicate by the value $\gamma$ the presence ($\gamma =1$) or absence ($\gamma =0$) of a centralized server in the system containing the file library. For the purpose of utilizing our data exchange bound, we assume that all the nodes in the system are capable of transmissions; thereby, any converse for this scenario is also valid for the usual coded caching scenario in which only the server (if it is present) does transmissions in the delivery phase. The set of clients ${\mathcal{K}}_T$ is partitioned into subsets ${\mathcal{K}}_{T_i}:i=1,\cdots ,t$ where the nodes in subset ${\mathcal{K}}_{T_i}$ can store a fraction ${\gamma }_{T_i}$ of the file library. Let $|{\mathcal{K}}_{T_i}|=K_{T_i}.$ We now give our converse for this setting. The caching and the delivery scheme, as well as the optimal communication load $L_c^{*}$, are defined as in the case of coded caching with homogeneous cache sizes.

Proposition 1.

For the above heterogeneous cache sizes setting, assuming $K\le N$, the optimal communication load $L_c^{*}$ for uncoded cache placement is lower bounded as follows.

$$\begin{array}{c}\hfill L_c^{*}\ge \left(\frac{K-{\sum }_{i=1}^t{K}_{T_i}{\gamma }_{T_i}}{\gamma +{\sum }_{i=1}^t{K}_{T_i}{\gamma }_{T_i}}\right)F.\end{array}$$

(13)

Before giving the proof of Proposition 1, we give the following remarks regarding the generality of Proposition 1, the new results which arise by applying Proposition 1 and various results from existing literature that are subsumed or improved by it.

• Heterogeneous Cache Sizes: There exists a number of works discussing distinct or heterogenous client cache sizes, for instance, in [18,19]. However, closed form expressions for the lower bound on the load seem to be lacking for such scenarios, to the best of our knowledge. Proposition 1 gives a lower bound for all such settings.
• Device-to-Device Coded Caching: Suppose there is no designated server in a coded caching setup, but the client nodes themselves are responsible for exchanging the information to satisfy their demands. This corresponds to the case of Device-to-Device (D2D) coded caching, first explored in [17]. In [17], an achievable scheme was presented for the case when each (client) node has equal cache fraction $\frac{M}{N}$, and this scheme achieves a communication load of $(\frac{N}{M}-1)F$ bits. In the work [20], it was shown that this communication load is optimal (for the regime of $K\le N$) over all possible “one shot” schemes (where “one shot” refers to those schemes in which each demanded bit is decoded using the transmission only from one server), and further it was shown that the load is within a multiplicative factor of 2 of the optimal communication load under the constraint for uncoded cache placement. We remark that the D2D setting of [17] corresponds to the special case of our current setting, with $\gamma =0,t=1,K_{T_1}=K,$ and ${\gamma }_{T_1}=M/N$. By this correspondence, by applying Proposition 1, we see that the load in this case is lower bounded as $\left(\frac{N}{M}-1\right)F,$ thus showing that the achievable scheme in [17] is exactly optimal under uncoded cache placement. The D2D scenario with heterogeneous cache sizes was explored in [21], in which the optimal communication load was characterized as the solution of an optimization problem. However, no closed form expression of the load for such a scenario is mentioned. Clearly, our Proposition 1 gives such a lower bound, when we fix $\gamma =0$, for any number of levels t of the client-side cache sizes.

Further, the result for coded caching with a server and equal cache sizes at receivers, as in Theorem 2, is clearly obtained as a special case of Proposition 1 with $\gamma =1,t=1,K_{T_1}=K$ and ${\gamma }_{T_1}=\frac{M}{N}$.

We now proceed to prove Proposition 1. The proof is similar to that of Theorem 2.

Proof of Proposition 1.

As in the proof of Theorem 2, we will denote the server node as the node 0 and assume a caching and delivery scheme which achieves the optimal load $L_c^{*}$ for worst case client demands.

Applying Theorem 1, for our setting, we have

$$\begin{array}{c}\hfill L_c^{*}\ge \left\{\begin{array}{cc}\sum _{P\subset {\mathcal{K}}_T}\sum _{Q\subset {\mathcal{K}}_T\setminus P}\frac{\left|P\right|}{\left|P\right|+\left|Q\right|-1}{a}_P^Q\hfill & \mathrm{if}\gamma =0,\hfill \\ \sum _{P\subset {\mathcal{K}}_T}\sum _{Q^{\prime }\subset {\mathcal{K}}_T\cup 0\setminus P}\frac{\left|P\right|}{\left|P\right|+|Q^{\prime }|-1}{a}_P^{Q^{\prime }}\hfill & \mathrm{if}\gamma =1.\hfill \end{array}\right.\end{array}$$

Note that if $\gamma =1$ (i.e., the server is present), then $a_P^{Q^{\prime }}=0$ whenever $0\notin Q^{\prime }.$

For a specific demand vector $\mathit{d}=(d_1,\cdots ,d_K)$ consisting of distinct demands and for some $Q\subset {\mathcal{K}}_T$, we define the quantity $c_k^Q\left(\mathit{d}\right)$ as follows.

$$\begin{array}{c}\hfill c_k^Q\left(\mathit{d}\right)=\left\{\begin{array}{cc}{a}_{\left\{k\right\}}^Q=\mathrm{Number}\mathrm{of}\mathrm{bits}\mathrm{demanded}\mathrm{by}k\hfill & \mathrm{if}\gamma =0,\hfill \\ \mathrm{available}\mathrm{exclusively}\mathrm{in}Q\hfill & \\ a_{\left\{k\right\}}^{Q\cup 0}=\mathrm{Number}\mathrm{of}\mathrm{bits}\mathrm{demanded}\mathrm{by}k\hfill & \mathrm{if}\gamma =1.\hfill \\ \mathrm{available}\mathrm{exclusively}\mathrm{in}Q\cup 0\hfill \end{array}\right.\end{array}$$

Symmetrization over appropriately chosen demand vectors: Choosing the same special set of demand vectors $\mathcal{D}$ as in (7) and averaging the above lower bound over the demand vectors in $\mathcal{D}$ similar to the proof of Theorem 2, we obtain a bound similar to (8):

$$\begin{array}{cc}\hfill L_c^{*}& \ge \left\{\begin{array}{cc}\frac{1}{N}\sum _{\mathit{d}\in \mathcal{D}}\sum _{k\in {\mathcal{K}}_T}\sum _{Q\subset {\mathcal{K}}_T\setminus k}\frac{c_k^Q\left(\mathit{d}\right)}{\left|Q\right|}\hfill & \mathrm{if}\gamma =0,\hfill \\ \frac{1}{N}\sum _{\mathit{d}\in \mathcal{D}}\sum _{k\in {\mathcal{K}}_T}\sum _{Q\subset {\mathcal{K}}_T\setminus k}\frac{c_k^Q\left(\mathit{d}\right)}{|Q\cup 0|}\hfill & \mathrm{if}\gamma =1.\hfill \end{array}\right.\hfill \end{array}$$

(14)

Combining the two expressions in (14), we can write a single equation which holds for $\gamma \in \{0,1\}$,

$$\begin{array}{cc}\hfill L_c^{*}& \ge \frac{1}{N}\sum _{\mathit{d}\in \mathcal{D}}\sum _{k\in {\mathcal{K}}_T}\sum _{Q\subset {\mathcal{K}}_T\setminus k}\frac{c_k^Q\left(\mathit{d}\right)}{\gamma +\left|Q\right|}.\hfill \end{array}$$

(15)

We now define the term $b_n^Q$ as follows.

$$\begin{array}{c}\hfill b_n^Q=\left\{\begin{array}{cc}\mathrm{Number}\mathrm{of}\mathrm{bits}\mathrm{of}\mathrm{file}n\mathrm{available}\mathrm{exclusively}\mathrm{in}Q\hfill & \mathrm{if}\gamma =0,\hfill \\ \mathrm{Number}\mathrm{of}\mathrm{bits}\mathrm{of}\mathrm{file}n\mathrm{available}\mathrm{exclusively}\mathrm{in}Q\cup 0\hfill & \mathrm{if}\gamma =1.\hfill \end{array}\right.\end{array}$$

(16)

Using the above definition of $b_n^Q$ and observing that each demand vector in $\mathcal{D}$ has distinct components, Equation (15) can be written as

$$\begin{array}{cc}\hfill L_c^{*}& \ge \frac{1}{N}\sum _{k\in {\mathcal{K}}_T}\sum _{Q\subset {\mathcal{K}}_T\setminus k}\sum _{\mathit{d}\in \mathcal{D}}\frac{c_k^Q\left(\mathit{d}\right)}{\gamma +\left|Q\right|}\hfill \end{array}$$

(17)

$$\begin{array}{cc}\hfill & =\frac{1}{N}\sum _{k\in {\mathcal{K}}_T}\sum _{Q\subset {\mathcal{K}}_T\setminus k}\sum _{n=1}^N\frac{b_n^Q}{\gamma +\left|Q\right|}\hfill \end{array}$$

(18)

$$\begin{array}{cc}\hfill & =\frac{1}{N}\sum _{Q\subset {\mathcal{K}}_T}\sum _{n=1}^N\frac{(K-|Q\left|\right)b_n^Q}{\gamma +\left|Q\right|}.\hfill \end{array}$$

(19)

Refining the bound in (19) using setting constraints and convexity: By the definition of $b_n^Q$ in (16), we have ${\sum }_n{\sum }_{Q\subset {\mathcal{K}}_T}{b}_n^Q=NF$. Further, ${\sum }_{Q\subset {\mathcal{K}}_T}\left|Q\right|b_n^Q\le {\displaystyle \sum _{i=1}^t}{K}_{T_i}{\gamma }_{T_i}NF$. Furthermore, for $\gamma \ge 0$, the function $\frac{K-x}{\gamma +x}$ is a convex decreasing function in x for $x>0.$ Thus, using Jensen’s inequality, we have $L_c^{*}\ge \frac{K-x}{\gamma +x},$ where

$$\begin{array}{c}\hfill x=\sum _n\sum _{Q\subset {\mathcal{K}}_T}\left|Q\right|\frac{b_n^Q}{NF}\le \frac{{\displaystyle \sum _{i=1}^t}{K}_{T_i}{\gamma }_{T_i}NF}{NF}=\sum _{i=1}^t{K}_{T_i}{\gamma }_{T_i}.\end{array}$$

This completes the proof. □

Remark 3.

Proposition 1 holds when $N\ge K$. This scenario is the most studied case in the literature and is practically more relevant than the case $K>N$. We now provide lower bounds for the heterogeneous cache sizes setting for general values of $K,N$, which includes the case $K>N$. As before, we consider two cases: $\gamma =1$ indicates the presence of a centralized server in the system and $\gamma =0$ indicates its absence.

Case 1, $\gamma =1$: For the case where a centralized server is present, i.e., $\gamma =1$, we have

$$\begin{array}{c}\hfill L_c^{*}\ge g_{min\{N,K\}}\left(\sum _{i=1}^t{K}_{T_i}{\gamma }_{T_i}\right),\end{array}$$

(20)

where the function $g_{min\{N,K\}}$ is defined in Remark 2. The derivation of this lower bound follows the steps in Appendix B until (A21), where we choose $N_u=min\{N,K\}$. Without loss of generality, we assume that all caches are fully populated with uncoded bits from the library, thus the total memory occupied by the cached bits ${\sum }_{Q\subset \left[K\right]}\left|Q\right|a^Q$ is equal to the sum of all the cache memory available in the system ${\sum }_{i=1}^t{K}_{T_i}{\gamma }_{T_i}NF$. Applying Jensen’s inequality on (A21) and using the fact ${\sum }_{Q\subset \left[K\right]}\left|Q\right|\frac{a^Q}{NF}={\sum }_{i=1}^t{K}_{T_i}{\gamma }_{T_i}$, we immediately arrive at the lower bound (20).

Case 2, $\gamma =0$: In this case, the optimal worst-case communication load can be lower bounded as follows:

$$\begin{array}{c}\hfill L_c^{*}\ge \frac{min\{N,K\}}{K}\left(\frac{K-{\sum }_{i=1}^t{K}_{T_i}{\gamma }_{T_i}}{{\sum }_{i=1}^t{K}_{T_i}{\gamma }_{T_i}}\right)F.\end{array}$$

(21)

The proof of this lower bound follows similar approach as Appendix B and is outlined in Appendix C. Note that when $N\ge K$, both (20) and (21) become identical to the inequality in Proposition 1.

2.2. Coded Caching with Multiple File Requests

In [22], coded caching with multiple file requests was considered, in which each client requests any $\mathsf{\Delta }$ files out of the N files in the delivery phase. It was shown in [22] (Section V.A) that if the $\mathsf{\Delta }K\le N$, then the optimal worst case communication load can be lower bounded as

$$\begin{array}{c}\hfill L_c^{*}\ge \frac{K\mathsf{\Delta }(1-M/N)}{1+MK/N}F.\end{array}$$

(22)

The work in [22] also gives an achievable scheme based on the scheme in [1] which meets the above bound. The same lower bound can be derived using Theorem 1 also, by following a similar procedure as that of the proof of Theorem 2.

Applying Theorem 1, we give the proof in brief. The demand vector assumed in proof of Theorem 1 becomes a $K\mathsf{\Delta }$-length vector in this case, consisting of K subvectors, each of length $\mathsf{\Delta }$, capturing $\mathsf{\Delta }$ distinct demands for each client. The proof proceeds as is until (6).

Symmetrization: The set $\mathcal{D}$ in (7) now contains the $K\mathsf{\Delta }$-length vectors of consecutive file indices, cyclically constructed, starting from $(1,\cdots ,K\mathsf{\Delta })$, i.e.,

$$\begin{array}{cc}\hfill \mathcal{D}& \triangleq \left\{\left(j{\oplus }_{N}0,j{\oplus }_{N}1,\cdots ,j{\oplus }_N(K\mathsf{\Delta }-1)\right):j=0,\cdots ,N-1\right\}.\hfill \end{array}$$

(23)

Thus, if the demand vector considered is $\mathit{d}\left(j\right)\triangleq \left(j{\oplus }_{N}0,j{\oplus }_{N}1,\cdots ,j{\oplus }_N(K\mathsf{\Delta }-1)\right)\in \mathcal{D},$ then the indices of the demanded files at client $k\in \left[K\right]$, denoted by ${\mathit{d}}_k\left(j\right),$ is given by

$$\begin{array}{c}\hfill {\mathit{d}}_k\left(j\right)\triangleq \{j{\oplus }_N(k-1)\mathsf{\Delta },j{\oplus }_N(k-1)\mathsf{\Delta }+1,\cdots ,j{\oplus }_N(k\mathsf{\Delta }-1)\}.\end{array}$$

The averaged lower bound expression similar to (8) is then obtained as

$$\begin{array}{cc}\hfill L_c^{*}& \ge \frac{1}{N}\sum _{j=0}^{N-1}\sum _{k=1}^K\sum _{Q\subset \left[K\right]\setminus k}\frac{1}{\left|Q\right|+1}{c}_k^Q\left(\mathit{d}\left(j\right)\right).\hfill \end{array}$$

(24)

In this expression, we have $c_k^Q\left(\mathit{d}\left(j\right)\right)$ which now indicates the number of bits of $\mathsf{\Delta }$ distinct and consecutive files indexed by ${\mathit{d}}_k\left(j\right)$ and available exclusively at the nodes in $Q\cup 0$ (0 denoting the server).

Observation:$c_k^Q\left(\mathit{d}\left(j\right)\right)={\displaystyle \sum _{n^{\prime }\in {\mathit{d}}_k\left(j\right)}}{b}_{n^{\prime }}^Q$ where $b_{n^{\prime }}^Q$ denotes the number of bits of file $n^{\prime }$ available exclusively in the nodes $Q\cup 0$, as in the proof of Theorem 2.

Now, $n^{\prime }\in {\mathit{d}}_k\left(j\right)$ if and only if the file $n^{\prime }$ is demanded by client k. By definition of $\mathcal{D}$, the event $n^{\prime }\in {\mathit{d}}_k\left(j\right)$ happens for precisely $\mathsf{\Delta }$ values of index j. From (24), applying the above observation, we have the following.

$$\begin{array}{cc}\hfill L_c^{*}& \ge \frac{1}{N}\sum _{k=1}^K\sum _{Q\subset \left[K\right]\setminus k}\sum _{j=0}^{N-1}\frac{1}{\left|Q\right|+1}{c}_k^Q\left(\mathit{d}\left(j\right)\right)\hfill \\ \hfill & =\frac{1}{N}\sum _{k=1}^K\sum _{Q\subset \left[K\right]\setminus k}\frac{1}{\left|Q\right|+1}\left(\sum _{j=0}^{N-1}\sum _{n^{\prime }\in {\mathit{d}}_k\left(j\right)}{b}_{n^{\prime }}^Q\right)\hfill \\ \hfill & =\frac{1}{N}\sum _{k=1}^K\sum _{Q\subset \left[K\right]\setminus k}\sum _{n^{\prime }=1}^N\frac{\mathsf{\Delta }}{\left|Q\right|+1}{b}_{n^{\prime }}^Q.\hfill \end{array}$$

(25)

Refining the bound in (25) using the setting constraints: We use the constraints of the setting and the convexity of the resultant expression to refine (25). This refinement essentially follows similar subsequent steps as in the proof of Theorem 2 following (9), and leads finally to (22).

Remark 4.

The work in [23] considers a coded caching setup in which Λ caches ($\mathsf{\Lambda }\le K$) are shared between the K clients. The special case when Λ divides K and each cache is serving exactly $\frac{K}{\mathsf{\Lambda }}$ clients is equivalent to the scenario of the multiple file requests in [22] with Λ clients, each demanding $\frac{K}{\mathsf{\Lambda }}$ files. The above proof then recovers the converse for this setting, which is obtained in [23] (Section III.A in [23]).

2.3. Coded Caching with Decentralized Caching

Theorem 2 and the subsequent results discussed above hold for the centralized caching framework, in which the caching phase is designed carefully in a predetermined fashion. In [24], the idea of decentralized placement was introduced, in which the caching phase is not coordinated centrally (this was called “decentralized coded caching” in [24]). In this scenario, each client, independently of others, caches a fraction $\gamma =\frac{M}{N}$ of the bits in each of the N files in the file library, chosen uniformly at random. For this scenario, the server (which has the file library) is responsible for the delivery phase. The optimal communication load $L_c^{*}$ is defined as the minimum worst case communication load over all possible delivery schemes for a given caching configuration, randomly constructed as given above. For the case of $K\le N$, the authors of [24] show a scheme which achieves the worst case communication load $L_c=KF(1-M/N)\frac{\left(1-{(1-M/N)}^K\right)}{MK/N}$. This was shown to be optimal for large F in [16] and also in [25] via a connection to index coding. In the following, we show that the same optimality follows easily via our Theorem 1.

Assume that we have distinct demands at the K clients, as in the proof of Theorem 1, given by the demand vector $\mathit{d}$. We first note that by the law of large numbers, as F increases, for the decentralized cache placement, for any $k\in \left[K\right],Q\subset \left[K\right]\setminus k$, we have

$$c_k^Q\left(\mathit{d}\right)=F{\left(\frac{M}{N}\right)}^{\left|Q\right|}{\left(1-\frac{M}{N}\right)}^{K-\left|Q\right|},$$

with probability close to 1, where $c_k^Q\left(\mathit{d}\right)$ is as defined in (5). This observation enables us to avoid the steps 2 and 3 mentioned in Section 1.2, as the value of $c_k^Q\left(\mathit{d}\right)$ is independent of the specific random cache placement or the demands chosen (as long as they are distinct). Using this in (6), we get

$$\begin{array}{cc}\hfill L_c^{*}& \ge \sum _{k=1}^K\sum _{Q\subset \left[K\right]\setminus k}\frac{1}{\left|Q\right|+1}{c}_k^Q\left(\mathit{d}\right)\hfill \\ \hfill & \ge \sum _{k=1}^K\sum _{Q\subset \left[K\right]\setminus k}\frac{F}{\left|Q\right|+1}{\left(\frac{M}{N}\right)}^{\left|Q\right|}{\left(1-\frac{M}{N}\right)}^{K-\left|Q\right|}\hfill \\ \hfill & =F\sum _{Q\subset \left[K\right]}\left(\frac{K-\left|Q\right|}{\left|Q\right|+1}\right){\left(\frac{M}{N}\right)}^{\left|Q\right|}{\left(1-\frac{M}{N}\right)}^{K-\left|Q\right|}\hfill \\ \hfill & =F\sum _{i=0}^K\left(\genfrac{}{}{0pt}{}{K}{i}\right)\left(\frac{K-i}{i+1}\right){\left(\frac{M}{N}\right)}^i{\left(1-\frac{M}{N}\right)}^{K-i}\hfill \\ \hfill & =F\sum _{i=0}^K\left(\genfrac{}{}{0pt}{}{K}{i+1}\right){\left(\frac{M}{N}\right)}^i{\left(1-\frac{M}{N}\right)}^{K-i}\hfill \\ \hfill & =F\sum _{j=1}^K\left(\genfrac{}{}{0pt}{}{K}{j}\right){\left(\frac{M}{N}\right)}^{j-1}{\left(1-\frac{M}{N}\right)}^{K-j+1}\hfill \\ \hfill & =F\frac{N}{M}\left(1-\frac{M}{N}\right)\sum _{j=1}^K\left(\genfrac{}{}{0pt}{}{K}{j}\right){\left(\frac{M}{N}\right)}^j{\left(1-\frac{M}{N}\right)}^{K-j}\hfill \\ \hfill & =\frac{NF}{M}\left(1-\frac{M}{N}\right)\left(1-{\left(1-\frac{M}{N}\right)}^K\right),\hfill \end{array}$$

where the last step follows as ${\sum }_{j=0}^K\left(\genfrac{}{}{0pt}{}{K}{j}\right){\left(\frac{M}{N}\right)}^j{\left(1-\frac{M}{N}\right)}^{K-j}={\left(\frac{M}{N}+\left(1-\frac{M}{N}\right)\right)}^K=1.$ Thus, we have given an alternate proof of the optimality of the decentralized scheme in [24].

3. Decentralized Coded Data Shuffling

In distributed machine learning systems consisting of a master and multiple worker nodes, data are distributed to the workers by the master in order to perform training of the machine learning model in a distributed manner. In general, this training process takes multiple iterations, with the workers doing some processing (like computing gradients) on their respective training data subsets. In order to ensure that the training data subset at each node are sufficiently representative of the data, and to improve the statistical performance of machine learning algorithms, shuffling of the training data between the worker nodes is implemented after every training iteration. This is known as data shuffling.

A coding theoretic approach to data shuffling, which involves the master communicating coded data to the workers was presented in [4]. The setting in [4] was centralized, which meant that there is a master node communicating to the servers to perform the data shuffling.

The work in [5] considered the data shuffling problem in which there is no master node, but the worker nodes exchange the training data among themselves, without involving the master node, to create a new desired partition in the next iteration. This was termed as decentralized data shuffling in [5]. Note that these notions of “centralized” and “decentralized” in the data shuffling problem are different from those in the coded caching [24], in which these terms were used to define the deterministic and random design of the caching phase, respectively. In this section, we look at the work in [5] and give a new simpler proof of the lower bound on the communication load for decentralized data shuffling.

We first review the setting in [5]. Consider K workers in the system, where each worker node is required to process q data units at any given time. The total dataset $F_1\cup \cdots \cup F_N$ consists of $N=Kq$ data units $F_1,\cdots ,F_N$, with a size of B bits per data unit. The collection of data units to be processed by worker node k at time t is denoted as $A_{k,t}$. The collection of data units $A_{1,t},\cdots ,A_{K,t}$ must form a partition of the dataset $F_1\cup \cdots \cup F_N$ for every time instant t, i.e., for any time t and any choice of $k,k^{\prime }\in \left[K\right]$ with $k\ne k^{\prime }$ we have

$$\begin{array}{c}\hfill A_{k,t}\subset F_1\cup \cdots \cup F_N,\left|A_{k,t}\right|=qB\mathrm{and}{A}_{k,t}\cap A_{k^{\prime },t}=\varnothing .\end{array}$$

Each node k has a local cache of size $MB$ bits (such that $q\le M\le Kq$) that can hold M data units. Out of these M units q units are the current “active” data $A_{k,t}$ at any time step which are required to be processed by the node k. The contents of the cache of node k at time t is denoted as $Z_{k,t}$. Therefore, for each choice of $k\in \left[K\right]$ and any time t, we have

$$\begin{array}{c}\hfill |Z_{k,t}|=MB\mathrm{and}{A}_{k,t}\subset Z_{k,t}.\end{array}$$

At each time instance t, a new partition $\{A_{k,t}:k\in \left[K\right]\}$ is to be made active at the nodes $\left[K\right]$, where this new partition is made known to the workers only at time step t. Note that the contents of the nodes at time $t-1$ are $Z_{1,t-1},\cdots ,Z_{K,t-1}$, and the active partition at time $t-1$ is $A_{1,t-1},\cdots ,A_{K,t-1}$. The worker nodes communicate with each other over a common broadcast link, as shown in Figure 2, to achieve the new partition. The decentralized data shuffling problem is to find a delivery scheme (between workers) to shuffle the collection of active data units $\{A_{k,t-1}:k\in \left[K\right]\}$ to a new partition $\{A_{k,t}:k\in \left[K\right]\}$. Each worker k computes a function ${\varphi }_k\left(Z_{k,t-1}\right)$ of its cache contents and broadcasts it to the other workers. Using these transmissions and the locally available cache content $Z_{k,t-1},$ each node k is required to decode $A_{k,t}.$ As in the case of coded caching, one seeks to reduce the worst-case communication load by designing the initial storage and coded transmissions carefully. The communication load of this data shuffling scheme, denoted by $L_{ds}$, is the sum of the number of bits broadcast by all the K nodes in the system, i.e., $L_{ds}={\sum }_{k\in \left[K\right]}\left|{\varphi }_k\left(Z_{k,t-1}\right)\right|$. The optimal communication load of data shuffling $L_{ds}^{*}$ (for the worst case data shuffle) is defined as

$$L_{ds}^{*}=minmax{L}_{ds}$$

where the maximization is over all possible choices for $A_{k,t-1}:k\in \left[K\right]$ and $A_{k,t}:k\in \left[K\right]$, and the minimization is over all possible choices for the cache placement $\{Z_{k,t-1}:k\in \left[K\right]\}$ and the delivery scheme $\{{\varphi }_k:k\in \left[K\right]\}$.

For the above setting, the following bound on the communication load $L_d^{*}$ was shown in [5].

$$\begin{array}{c}\hfill L_{ds}^{*}\ge \frac{Kq}{K-1}.\frac{K-M/q}{M/q}B.\end{array}$$

(26)

The above bound was shown to be optimal for some special cases of the parameters, and order-optimal otherwise.

Proof of the Decentralized Data Shuffling Converse

We now recover the bound (26) by a simple proof using our generic lower bound in Theorem 1. We assume that the cache placement and delivery scheme of the data shuffling scheme are designed such that the communication load of the data shuffling scheme is exactly equal to $L_{ds}^{*}.$ We proceed as per the three steps in Section 1.2.

Applying Theorem 1: For $k\in \left[K\right]$ and $Q\subset \left[K\right],$ let $A_{k,t}^Q$ denote the subset of bits of $A_{k,t}$ available exactly at the nodes in Q and not anywhere else. Note that $|A_{k,t}^Q|=0$ if $Q=\varnothing$, as each bit is necessarily present in at least one of the K nodes.

As per our bound in Theorem 1, we have

$$\begin{array}{c}\hfill L_{ds}^{*}=L_{ds}\ge \sum _{k\in \left[K\right]}\sum _{Q\subset \left[K\right]\setminus k}\frac{|A_{k,t}^Q|}{\left|Q\right|}.\end{array}$$

Symmetrization by averaging over appropriately chosen set of shuffles: Let the set of circular permutations of $(1,2,\cdots ,K)$, apart from the identity permutation, be denoted by $\mathsf{\Gamma }$. There are $K-1$ of them clearly. We denote an arbitrary permutation in $\mathsf{\Gamma }$ by $\gamma$, and by ${\gamma }_k$ we denote the $kth$ coordinate of $\gamma$.

Now, consider the shuffle given by $\gamma \in \mathsf{\Gamma }$, i.e., for each k, $A_{k,t}=A_{{\gamma }_k,t-1}$. For this shuffle, we have by the above equation that

$$\begin{array}{cc}\hfill L_{ds}^{*}& \ge \sum _{k\in \left[K\right]}\sum _{Q\subset \left[K\right]\setminus k}\frac{|A_{{\gamma }_k,t-1}^Q|}{\left|Q\right|}\hfill \end{array}$$

(27)

$$\begin{array}{cc}\hfill & =\sum _{Q\subset \left[K\right]}\sum _{k\in \left[K\right]\setminus Q}\frac{|A_{{\gamma }_k,t-1}^Q|}{\left|Q\right|}.\hfill \end{array}$$

(28)

Now, averaging (28) over all permutations in $\mathsf{\Gamma }$, we get

$$\begin{array}{cc}\hfill L_{ds}^{*}& \ge \frac{1}{K-1}\sum _{\gamma \in \mathsf{\Gamma }}\sum _{Q\subset \left[K\right]}\sum _{k\in \left[K\right]\setminus Q}\frac{|A_{{\gamma }_k,t-1}^Q|}{\left|Q\right|},\hfill \end{array}$$

(29)

$$\begin{array}{cc}\hfill & =\frac{1}{K-1}\sum _{Q\subset \left[K\right]}\sum _{k\in \left[K\right]\setminus Q}\sum _{\gamma \in \mathsf{\Gamma }}\frac{|A_{{\gamma }_k,t-1}^Q|}{\left|Q\right|}.\hfill \end{array}$$

(30)

As we go through all choices of $\gamma \in \mathsf{\Gamma },$ we see that ${\gamma }_k$ takes every value except k, i.e., ${\gamma }_k$ assumes each value in $\left[K\right]\setminus k$ exactly once. Moreover, $A_{k^{\prime },t-1}^Q$ is the collection of bits of $A_{k^{\prime },t-1}$ present only in Q. However, the bits $A_{k^{\prime },t-1}$ are already presented in $k^{\prime }$. Hence, $|A_{k^{\prime },t-1}^Q|=0$ if $k^{\prime }\notin Q$. Therefore, we have

$$\begin{array}{cc}\hfill L_{ds}^{*}& \ge \frac{1}{K-1}\sum _{Q\subset \left[K\right]}\sum _{k\in \left[K\right]\setminus Q}\sum _{k^{\prime }\in Q}\frac{|A_{k^{\prime },t-1}^Q|}{\left|Q\right|},\hfill \end{array}$$

(31)

$$\begin{array}{cc}\hfill & =\frac{1}{K-1}\sum _{Q\subset \left[K\right]}\sum _{k^{\prime }\in Q}\frac{|A_{k^{\prime },t-1}^Q\left|\right(K-\left|Q\right|)}{\left|Q\right|}\hfill \end{array}$$

(32)

$$\begin{array}{cc}\hfill & =\frac{1}{K-1}\sum _{k^{\prime }\in \left[K\right]}\sum _{Q\subset \left[K\right]:k^{\prime }\in Q}\frac{|A_{k^{\prime },t-1}^Q\left|\right(K-\left|Q\right|)}{\left|Q\right|}.\hfill \end{array}$$

(33)

Refining the bound using setting constraints and convexity: Now, we have the following observations as $A_{k^{\prime },t-1}^Q:k^{\prime }\in \left[K\right],\{Q^{\prime }\subset \left[K\right]:k^{\prime }\in Q\}$ form a partition of all the $NB$ bits.

$$\begin{array}{cc}\hfill \sum _{Q\subset \left[K\right]}\sum _{k^{\prime }\in Q}\left|A_{k^{\prime },t-1}^Q\right|& =NB=KqB\hfill \\ \hfill \sum _{Q\subset \left[K\right]}\sum _{k^{\prime }\in Q}|A_{k^{\prime },t-1}^Q||Q|& \le KMB.\hfill \end{array}$$

Utilizing the above, and the fact that $\frac{K-\left|Q\right|}{\left|Q\right|}$ is a convex decreasing function in $\left|Q\right|$ (for $\left|Q\right|\ge 0$), we have

$$\begin{array}{cc}\hfill L_{ds}^{*}& \ge \frac{KqB}{K-1}.\frac{(K-{\sum }_{Q\subset \left[K\right]}{\sum }_{k^{\prime }\in Q}\frac{|A_{k^{\prime },t-1}^Q|}{NB}|Q|)}{{\sum }_{Q\subset \left[K\right]}{\sum }_{k^{\prime }\in Q}\frac{|A_{k^{\prime },t-1}^Q|}{NB}\left|Q\right|}\hfill \end{array}$$

(34)

$$\begin{array}{cc}\hfill & \ge \frac{KqB}{K-1}.\frac{K-KM/N}{KM/N}\hfill \end{array}$$

(35)

$$\begin{array}{cc}\hfill & =\frac{KqB}{K-1}.\frac{K-M/q}{M/q}\hfill \end{array}$$

(36)

Thus, we have recovered (26).

Remark 5.

We have considered the decentralized version of the coded data shuffling problem in this subsection. The centralized version of the data shuffling problem was introduced in [4] and its information theoretic limits were studied elaborately in [6]. Our data exchange bound, when applied to the setting in [6], results in a looser converse result than that in [6]. The reasons for this is explored in Section 5 using the connection between our data exchange bound and the bound for index coding known in literature.

4. Coded Distributed Computing

In a distributed computing setting, there are N files on which the distributed computing task has to be performed by K nodes. The job at hand is divided into three phases: Map, Shuffle, and Reduce. In the shuffle phase, the nodes that are assigned to perform the distributed computing task exchange data. In [3], the authors proposed coded communication during the shuffle phase to reduce the communication load. We recollect the setting and the main converse result from [3], which we recover using our data exchange bound.

A subset ${\mathcal{M}}_i$ of N files is assigned to $i\mathrm{th}$ node and the $i\mathrm{th}$ node computes the map functions on this subset in the map phase (see Figure 3). We assume that the total number of map functions computed at the K nodes is $rN$, where r is referred to as the computation load. In the reduce phase, a total of W reduce functions is to be computed across the K nodes corresponding to the N files. Each node is assigned the same number of functions. Obtaining the output of the reduce functions at all the nodes will complete the distributed computing task. In this work, as in [3], we consider two scenarios: in the first one, each reduce function is computed exactly at one node and in the second, each reduce function is computed at s nodes, where $s\ge 2$.

Each map function output (also referred to as intermediate output) corresponds to a particular file and a particular reduce function. For each file and each reduce function, an intermediate output of T bits is obtained. To compute an assigned reduce function, each node requires the intermediate outputs of all the files corresponding to the assigned reduce function. This means each node is missing the intermediate outputs (corresponding to the assigned reduce functions) of those files that are not assigned to it in the map phase.

The intermediate outputs of each file assigned to node i corresponding to all the reduce functions are available at node i at the end of the map phase and denoted by $v_{1:Q,{\mathcal{M}}_i}$. These intermediate outputs at the end of the map phase are encoded as follows: $X_i={\varphi }_i\left(v_{1:Q,{\mathcal{M}}_i}\right)$ and broadcasted to the remaining nodes, in the shuffle phase (in order to deliver the missing intermediate outputs at the nodes). Let $L_{dc}^{*}$ be the total number of bits broadcasted by the K nodes in the shuffle phase, minimized over all possible map function assignments, reduce function assignments, and shuffling schemes, with a computation load r. We refer to $L_{dc}^{*}$ as the minimum communication load.

To obtain similar expressions for the communication load as in [3], we normalize the communication load by the total number of intermediate output bits (=$WNT$). We consider the first scenario now, where each reduce function is computed exactly at one node.

Theorem 3

([3]). The minimum communication load $L_{dc}^{*}$ incurred by a distributed computing system of K nodes for a given computation load r, where every reduce function is computed at exactly one node and each node computes $\frac{W}{K}$ reduce functions, is bounded as

$$\frac{L_{dc}^{*}}{WNT}\ge \frac{1}{r}\left(1-\frac{r}{K}\right).$$

(37)

Proof. 

We resort to two of the three steps of Section 1.2 to complete this proof. The symmetrization step, which involves averaging over demand configurations, is not applicable in the present setting because the definition of $L_{dc}^{*}$ involves minimization over the reduce function assignment as well.

Applying Theorem 1: Let $\mathcal{M}=({\mathcal{M}}_1,\dots ,{\mathcal{M}}_K)$ denote a given map function assignment to the nodes, where ${\mathcal{M}}_i\subset \left[N\right]$. Let $L_{\mathcal{M}}$ denote the communication load associated with the map function assignment $\mathcal{M}$. We will prove that

$$\frac{L_{\mathcal{M}}}{WNT}\ge \sum _{j=1}^K\frac{{\tilde{a}}_{\mathcal{M}}^j}{N}\frac{K-j}{Kj},$$

where ${\tilde{a}}_{\mathcal{M}}^j$ denotes the number of files which are mapped at exactly j nodes in $\left[K\right]$. It is easy to see that ${\sum }_{j=1}^K{\tilde{a}}_{\mathcal{M}}^j=N$ and ${\sum }_{j=1}^{K}j{\tilde{a}}_{\mathcal{M}}^j=rN$. We will apply Theorem 1 to this setting. Recall that each reduce function is computed exactly at one node in our present setup. To apply Theorem 1, we need to ascertain the quantities $a_P^Q$ for $P,Q$ being disjoint subsets of $\left[K\right]$. To do this, we first denote by ${\tilde{a}}^Q$ the number of files whose intermediate outputs are demanded by some node k and available exclusively in the nodes of Q. Note that ${\tilde{a}}^Q$ is the same for any $k\in \left[K\right]\setminus Q$, as each node demands intermediate outputs of all the files that are not mapped at the node itself.

As the number of reduce functions assigned to node k is $\frac{W}{K}$ (as each reduce function is computed at exactly one node) and each intermediate output is T bits, the number of intermediate output bits which are demanded by any node k and available exclusively in the nodes of Q are $\frac{WT}{K}{\tilde{a}}^Q$. Thus, for any $Q\subset \left[K\right]$, the quantities $a_P^Q$ in Theorem 1 are given as follows.

$$a_P^Q=\left\{\begin{array}{cc}\frac{WT}{K}{\tilde{a}}^Q\hfill & \mathrm{if}P=\left\{k\right\}\mathrm{for}\mathrm{some}k\in \left[K\right]\mathrm{such}\mathrm{that}k\notin Q\hfill \\ 0\hfill & \mathrm{otherwise}\hfill \end{array}\right.$$

Further note that ${\sum }_{Q\subset \left[K\right]:\left|Q\right|=j}{\tilde{a}}^Q={\tilde{a}}_{\mathcal{M}}^j$ by definition of ${\tilde{a}}_{\mathcal{M}}^j.$ Using these and applying Theorem 1 with the normalization factor $WNT$, we have the following inequalities.

$$\begin{array}{cc}\hfill \frac{L_{\mathcal{M}}}{WNT}& \ge \frac{1}{WNT}\sum _{k=1}^K\sum _{Q\subset \left[K\right]\setminus \left\{k\right\}}\frac{1}{\left|Q\right|}{\tilde{a}}^Q\frac{WT}{K}\hfill \\ \hfill & =\frac{1}{KN}\sum _{j=1}^K\sum _{Q\subset \left[K\right]:\left|Q\right|=j}\sum _{k\in \left[K\right]\setminus Q}\frac{1}{j}{\tilde{a}}^Q\hfill \\ \hfill & =\frac{1}{KN}\sum _{j=1}^K\sum _{Q\subset \left[K\right]:\left|Q\right|=j}\frac{K-j}{j}{\tilde{a}}^Q\hfill \\ \hfill & =\frac{1}{KN}\sum _{j=1}^K\frac{K-j}{j}\left(\sum _{Q\subset \left[K\right]:\left|Q\right|=j}{\tilde{a}}^Q\right)\hfill \\ \hfill & =\frac{1}{KN}\sum _{j=1}^K\frac{K-j}{j}{\tilde{a}}_{\mathcal{M}}^j.\hfill \end{array}$$

Refining the bound using convexity and setting constraints: Using definition of $L_{dc}^{*}$, noting that $\frac{K-j}{j}$ is a convex decreasing function of j and that $\frac{{\sum }_{j=1}^K{\tilde{a}}_{\mathcal{M}}^j}{N}=1$, we have that

$$\begin{array}{cc}\hfill \frac{L_{dc}^{*}}{WNT}& \ge \frac{1}{K}\frac{K-{\sum }_{j=1}^K\frac{j{\tilde{a}}_{\mathcal{M}}^j}{N}}{{\sum }_{j=1}^K\frac{j{\tilde{a}}_{\mathcal{M}}^j}{N}}\hfill \\ \hfill & =\frac{1}{K}\frac{K-r}{r}=\frac{1}{r}\left(1-\frac{r}{K}\right).\hfill \end{array}$$

□

Now, we consider the case in which each reduce function has to be computed at s nodes. The total number of reduce functions is assumed to be W. In addition, the following assumption is made to keep the problem formulation symmetric with respect to reduce functions: every possible s sized subset of K nodes is assigned $\frac{W}{\left(\genfrac{}{}{0pt}{}{K}{s}\right)}$ reduce functions (we assume $\left(\genfrac{}{}{0pt}{}{K}{s}\right)$ divides W). As in the previous case, we will denote the communication load for a given map function assignment by $L_{\mathcal{M}}\left(s\right)$ and the optimal communication load with computation load r by $L_{dc}^{*}\left(s\right)$. We will prove the following result which gives a lower bound on $L_{\mathcal{M}}\left(s\right)$.

Proposition 2

([3]). The communication load corresponding to a map function assignment $\mathcal{M}$ when each reduce function has to be computed at s nodes is lower bounded as

$$\frac{L_{\mathcal{M}}\left(s\right)}{WNT}\ge \sum _{j=1}^K\frac{{\tilde{a}}_{\mathcal{M}}^j}{N}\sum _{l=max(0,s-j)}^{min(K-j,s)}\frac{\left(\genfrac{}{}{0pt}{}{K-j}{l}\right)\left(\genfrac{}{}{0pt}{}{j}{s-l}\right)}{\left(\genfrac{}{}{0pt}{}{K}{s}\right)}\frac{l}{l+j-1}.$$

(38)

Proof. 

As before, we will denote by ${\tilde{a}}^Q$ the number of files whose map function outputs are available exclusively in the nodes of Q. Furthermore, we will denote the number of intermediate output bits which are demanded exclusively by the nodes in P and available exclusively in the nodes of Q by $b_P^Q$. Then, applying Theorem 1, the lower bound on the communication load in terms of $\left\{b_P^Q\right\}$ is given by

$$\begin{array}{ccc}\hfill \frac{L_{\mathcal{M}}\left(s\right)}{WNT}& \ge & \frac{1}{WNT}\sum _{P\subset \left[K\right]}\sum _{Q\subset \left[K\right]\setminus P}\frac{\left|P\right|}{\left|P\right|+\left|Q\right|-1}{b}_P^Q.\hfill \end{array}$$

We first interchange the above summation order and consider all sets Q with $\left|Q\right|=j$ and all sets P such that $\left|P\right|=l$. For $\left|Q\right|=j$, we need to count the subsets of size s, which form a subset of $P\cup Q$. Thus, for a fixed j, we can see that the range of l can vary from $max(0,s-j)$ to $min(K-j,s)$. For a given subset P of size l, the number of s sized subsets which are contained within $P\cup Q$ and contain P are $\left(\genfrac{}{}{0pt}{}{j}{s-l}\right)$. Therefore, the number of intermediate output bits demanded exclusively by the nodes in P and available exclusively in Q, $b_P^Q$, is given by $b_P^Q={\tilde{a}}^{Q}WT\frac{\left(\genfrac{}{}{0pt}{}{j}{s-l}\right)}{\left(\genfrac{}{}{0pt}{}{K}{s}\right)}$. This is because each of the s-sized subset has to reduce ${\tilde{a}}^Q\frac{W}{\left(\genfrac{}{}{0pt}{}{K}{s}\right)}$ functions. Using this relation, the above inequality can be rewritten as follows.

$$\begin{array}{ccc}\hfill \frac{L_{\mathcal{M}}\left(s\right)}{WNT}& \ge & \sum _{j=1}^K\frac{1}{N}\left(\sum _{Q\subset \left[K\right]:\left|Q\right|=j}{\tilde{a}}^Q\right)\sum _{l=max(0,s-j)}^{min(K-j,s)}\frac{l}{l+j-1}\left(\sum _{P\subset \left[K\right]\setminus Q:\left|P\right|=l}\frac{\left(\genfrac{}{}{0pt}{}{j}{s-l}\right)}{\left(\genfrac{}{}{0pt}{}{K}{s}\right)}\right)\hfill \end{array}$$

(39)

$$\begin{array}{ccc}\hfill & =& \sum _{j=1}^K\frac{{\tilde{a}}_{\mathcal{M}}^j}{N}\sum _{l=max(0,s-j)}^{min(K-j,s)}\frac{l}{l+j-1}\frac{\left(\genfrac{}{}{0pt}{}{K-j}{l}\right)\left(\genfrac{}{}{0pt}{}{j}{s-l}\right)}{\left(\genfrac{}{}{0pt}{}{K}{s}\right)},\hfill \end{array}$$

(40)

where (40) follows as ${\tilde{a}}_{\mathcal{M}}^j={\sum }_{Q\subset \left[K\right]:\left|Q\right|=j}{\tilde{a}}^Q$. This completes the proof. □

The above lemma along with certain convexity arguments resulting from the constraints imposed by the computation load can be used to prove the lower bound on $L_{dc}^{*}\left(s\right)$. The interested reader is referred to the converse proof of Theorem 2 in [3] for the same.

5. Relation to Index Coding Lower Bound

We now consider the “centralized” version of the data exchange problem, where one of the nodes has a copy of all the information bits and is the lone transmitter in the system. We will use the index 0 for this server node, and assume that there are K other nodes in the system, with index set $\left[K\right]$, acting as clients. In terms of Definition 1, this system is composed of $K+1$ nodes $\left\{0\right\}\cup \left[K\right]$, the demand $D_0$ of the server is empty, while the demands $D_i$ and the contents $C_i$ of all the clients are subsets of the contents of the server, i.e., $C_i,D_i\subset C_0$ for all $i\in \left[K\right]$. Without loss of generality, we assume that only the server performs all the transmissions as any coded bit that can be generated by any of the client nodes can be generated at the server itself. Clearly, this is an index coding problem [26] with K clients or receivers, the demand of the $i\mathrm{th}$ receiver is $D_i$, and its side information is $C_i$. When applied to this scenario, our main result Theorem 1 therefore provides a lower bound on the index coding communication cost.

The maximum acyclic induced subgraph (MAIS) and its generalization, which is known as the generalized independence number or the $\alpha$-bound, are well-known lower bounds in index coding [9,26]. In this section, we describe the relation between the $\alpha$-bound of index coding and the centralized version of Theorem 1. We show that the latter is in general weaker, and identify the scenarios when these two bounds are identical. We then use these observations to explain why Theorem 1 cannot provide a tight lower bound for the centralized data shuffling problem [6].

Let us first apply Theorem 1 to the centralized data exchange problem. As node 0 contains all the information bits and its demand is empty, we have $a_P^{Q^{\prime }}=0$ if $0\notin Q^{\prime }$ or $0\in P$. Using $Q=Q^{\prime }\setminus \left\{0\right\}$ and defining the variable $c_P^Q=a_P^{Q\cup \left\{0\right\}}=a_P^{Q^{\prime }}$, we obtain

Theorem 4.

The centralized version of our main result Theorem 1 is

$$\begin{array}{cc}\hfill L^{*}& \ge \sum _{P\subset \left[K\right]}\sum _{\begin{array}{c}{Q}^{\prime }\subset \left\{0\right\}\cup \left[K\right]\\ 0\in Q^{\prime },P\cap Q^{\prime }=\varnothing \end{array}}\frac{\left|P\right|}{\left|P\right|+|Q^{\prime }|-1}{a}_P^{Q^{\prime }}\hfill \\ \hfill & =\sum _{P\subset \left[K\right]}\sum _{Q\subset \left[K\right]\setminus P}\frac{\left|P\right|}{\left|P\right|+\left|Q\right|}{c}_P^Q.\hfill \end{array}$$

Note that it is possible to have $c_P^Q=a_P^{Q\cup \left\{0\right\}}>0$ when $Q=\varnothing$.

In Section 5.1, we express the generalized independence number $\alpha$ in terms of the parameters $c_P^Q$, and in Section 5.2, we identify the relation between our lower bound Theorem 4 and the index coding lower bound $\alpha$.

5.1. The Generalized Independence Number Bound

Let $\gamma =({\gamma }_1,\cdots ,{\gamma }_K)$ be any permutation of $\left[K\right]$, where ${\gamma }_i$ is the $i\mathrm{th}$ coordinate of the permutation. Applying similar ideas as in the proof of Theorem 1 to the centralized scenario, we obtain the following lower bound on $L^{*}$. This lower bound considers the nodes in the order ${\gamma }_1,\cdots ,{\gamma }_K$, and for each node in this sequence it counts the number of bits that are demanded by this node which are neither demanded by and nor available as side information in any of the earlier nodes.

Proposition 3.

For any permutation γ of $\left[K\right]$,

$$L^{*}\ge \sum _{i=1}^K\sum _{\begin{array}{c}P\subset \{{\gamma }_i,\cdots ,{\gamma }_K\}\\ {\gamma }_i\in P\end{array}}\sum _{Q\subset \{{\gamma }_{i+1},\cdots ,{\gamma }_K\}}{c}_P^Q.$$

(41)

Proof. 

See Appendix D. □

A direct consequence of Proposition 3 is

$$L^{*}\ge \underset{\gamma }{max}\sum _{i=1}^K\sum _{\begin{array}{c}P\subset \{{\gamma }_i,\cdots ,{\gamma }_K\}\\ {\gamma }_i\in P\end{array}}\sum _{Q\subset \{{\gamma }_{i+1},\cdots ,{\gamma }_K\}}{c}_P^Q$$

(42)

where the maximization is over all possible permutations on $\left[K\right]$.

We now recall the definition of the generalized independence number [9]. Denote the collection of the $c_P^Q$ information bits available exclusively at the nodes $Q\cup \left\{0\right\}$ and demanded exclusively by the nodes P as $\{w_{P,m}^Q:m=1,\cdots ,c_P^Q\}$. Therefore, the set of all the information bits present in the system is

$$B=\bigcup _{P\subset \left[K\right]}\bigcup _{Q\subset \left[K\right]\setminus P}\left\{w_{P,m}^Q:m=1,\cdots ,c_P^Q\right\}.$$

Note that each bit is identified by a triple $(P,Q,m)$.

Definition 2.

A subset $\mathcal{H}$ of B is a generalized independent set if and only if every subset $\mathcal{I}\subset \mathcal{H}$ satisfies the following:

• there exists a node $k\in \left[K\right]$ and an information bit in $\mathcal{I}$ such that this information bit is demanded by k (and possibly some other nodes), and none of the other bits in $\mathcal{I}$ are available as side information at k.

The generalized independence number α is the size of the largest generalized independent set.

We next show that the lower bound in (42) is in fact equal to the generalized independence number $\alpha$ of this index coding problem.

Theorem 5.

The generalized independence number α satisfies

$$\alpha =\underset{\gamma }{max}\sum _{i=1}^K\sum _{\begin{array}{c}P\subset \{{\gamma }_i,\cdots ,{\gamma }_K\}\\ {\gamma }_i\in P\end{array}}\sum _{Q\subset \{{\gamma }_{i+1},\cdots ,{\gamma }_K\}}{c}_P^Q,$$

(43)

where the maximization is over all $K!$ permutations of $\left[K\right]$.

Proof. 

See Appendix E. □

5.2. Relation to the Index Coding Lower Bound

Proposition 3 serves as the platform for comparing Theorem 4 and the $\alpha$-bound. While $\alpha$ equals the maximum value of the bound in Proposition 3 over all permutations on $\left[K\right]$, our bound in Theorem 4 equals the average value of the lower bound given in Proposition 3 over all permutations on $\left[K\right]$. We will show this relation between Theorem 4 and Proposition 3 now.

Taking the average of the right hand side of (41) with respect to all $\gamma$, we obtain

$$\frac{1}{K!}\sum _{\gamma }\sum _{i=1}^K\sum _{\begin{array}{c}P\subset \{{\gamma }_i,\cdots ,{\gamma }_K\}\\ {\gamma }_i\in P\end{array}}\sum _{Q\subset \{{\gamma }_{i+1},\cdots ,{\gamma }_K\}}{c}_P^Q.$$

For each choice of $P,Q\subset \left[K\right]$ with $P\cap Q=\varnothing$, we now count the number of times $c_P^Q$ appears in this sum. For a given $\gamma$, the inner summations include the term $c_P^Q$ if and only if the following holds:

$${\gamma }_i\in P,\mathrm{where}i=min\{j\in \left[K\right]:{\gamma }_j\in P\cup Q\},$$

i.e., if we consider the elements ${\gamma }_1,\cdots ,{\gamma }_K$ in that order, the first element from $P\cup Q$ to be observed in this sequence belongs to P. Thus, for a given pair $P,Q$ the probability that a permutation $\gamma$ chosen uniformly at random includes the term $c_P^Q$ in the inner summation is $\left|P\right|/\left(\right|P|+|Q\left|\right)$. Therefore, the average of the lower bound in Proposition 3 over all possible $\gamma$ is

$$\sum _{P\subset \left[K\right]}\sum _{\begin{array}{c}Q\subset \left[K\right]\\ P\cap Q=\varnothing \end{array}}\frac{\left|P\right|}{\left|P\right|+\left|Q\right|}{c}_P^Q,$$

which is exactly the bound in Theorem 4.

As the bound in Theorem 4 is obtained by averaging over all $\gamma$, instead of maximizing over all $\gamma$, we conclude that this is in general weaker than the $\alpha$-bound of index coding. The two bounds are equal if and only if the bound in Proposition 3 has the same value for every permutation $\gamma$.

Although weaker in general, we note that the bound of Theorem 4 is easier to use than the $\alpha$-bound. As demonstrated by (2), in order to use Theorem 4, we only need to know, for each information bit, the number of nodes that contain this bit and the number of nodes that demand this bit. In comparison, this information is insufficient to evaluate the $\alpha$-bound, which also requires the identities of these nodes.

5.3. On the Tightness of Theorem 4

We now consider the class of unicast problems, i.e., problems where each bit is demanded by exactly one of the nodes. For this class of problems, we characterize when Theorem 4 yields a tight bound.

Theorem 6.

For unicast problems the bound in Theorem 4 equals $L^{*}$ if and only if every $S\subset \left[K\right]$ with $\left|S\right|\ge 2$ satisfies the following, $c_{\left\{k\right\}}^{S\setminus k}=c_{\left\{k^{\prime }\right\}}^{S\setminus k^{\prime }}$ for every $k,k^{\prime }\in S$.

Proof. 

See Appendix F. When the lower bound of Theorem 4 is tight, the clique-covering based index coding scheme (see in [26,27]) yields the optimal communication cost. □

Our main result in Theorem 1, or equivalently, Theorem 4, does not provide a tight lower bound for centralized data shuffling problem [6], because this problem involves scenarios that do not satisfy the tightness condition of Theorem 6. For instance, consider the simple canonical data shuffling setting, where the system has exactly K files, all of equal size F bits, and each node stores exactly one of these files, i.e., the entirety of the contents of the $k\mathrm{th}$ node $C_k$ is the $k\mathrm{th}$ file. Here, $|C_k|=F$ for all $k\in \left[K\right]$, and $C_i\cap C_j=\varnothing$ for all $i\ne j$. Assume that the shuffling problem is to move the file $C_{k+1}$ to node k, i.e., $D_k=C_{k+1}$, where we consider the index $K+1$ to be equal to 1. This is a worst-case demand for data shuffling incurring the largest possible communication cost. For this set of demands, we have $c_k^{\{k+1\}}=F$ for all $k\in \left[K\right]$, and $c_k^Q=0$ for all other choices of $k,Q$. In particular, $c_{k+1}^{\left\{k\right\}}=0\ne c_k^{\{k+1\}}$. Clearly, the condition in Theorem 6 does not hold for $S=\{k,k+1\}$. Therefore, our lower bound is strictly less than $L^{*}$ for this data shuffling problem, and therefore is not tight.

6. Relationship to Other Index Coding Settings

We now comment on the application of our data exchange bound to a couple of other important index coding settings known in literature, (a) distributed index coding studied in [10] (which is equivalent to the cooperative multi-sender index coding setting considered in [11]), and (b) embedded index coding, presented in [12].

6.1. Distributed Index Coding

In [10], the authors consider a generalization of the single-server index coding problem (which we studied in Section 5) called distributed index coding. The specific setting in [10] is as follows. There are n messages denoted by $x_j:j\in \left[n\right],$ where $x_j\in {\{0,1\}}^{t_j}$ (for some positive integer $t_j$). There is a corresponding set of n receivers indexed by $\left[n\right]$. The receiver $j\in \left[n\right]$ contains as side-information the subset of messages indexed by $A_j\subset \left[n\right]$ (i.e., receiver j knows $\{x_i:i\in A_j\}$) and demands the message $x_j$. There are $2^n-1$ servers in the system, indexed by the sets $\mathcal{J}=\{J:J\subset [n],J\ne \varnothing \}$. The server J contains the messages $\{x_i:i\in J\}.$ The servers do not demand any messages and are responsible only for transmissions that satisfy the receivers. The server J is connected to the n receivers via a broadcast link with capacity $C_J$ bits. In order to satisfy the demands, each server J sends a message $y_J\in {\{0,1\}}^{s_J}$ to all the receivers, where $s_J$ is some positive integer.

Definition 3

([10]). The rate-capacity tuple $((R_j:j\in \left[n\right]),(C_J:J\in \mathcal{J}))$ is said to be achievable if there exists some positive integer r such that $t_j\ge r{R}_j,\forall j$ and $s_J\le r{C}_J,\forall J,$ and there exists valid encoding functions (encoding the messages of lengths $(t_j:j\in \left[n\right])$ into codewords of lengths $(s_J:J\in \mathcal{J})$) and decoding functions, such that all receivers can decode their respective demands.

Slightly abusing Definition 2, for some $T\subset \left[n\right]$, we call a set $S\subset T$ of message indices as a generalized independent set of T, if for every subset $S^{\prime }\subset S,$ there is some $j\in S^{\prime }$ such that $A_j\cap (S^{\prime }\setminus j)=\varnothing .$

Let $((R_j:j\in \left[n\right]),(C_J:J\in \mathcal{J}))$ be an achievable rate-capacity tuple. For any non-empty subset $T\subset \left[n\right]$, let $S_T$ be a generalized independent set of T. In Corollary 2 of [10], it is shown that

$$\begin{array}{c}\hfill \sum _{j\in S_T}{R}_j\le \sum _{J:J\cap T\ne \varnothing }{C}_J.\end{array}$$

(44)

Remark 6.

The above bound in (44) is given in [10] using the terminology of the side-information graph defining the index coding problem and its acyclic induced subgraphs. However, we have used generalized independent sets to state the same bound. The reader can easily confirm that the acyclic induced subgraph of the side-information graph as defined in [10] is the same as a generalized independent set we have used in this work. Therefore, (44) is the same as the bound in Corollary 2 of [10].

Let

$$S_{max}=arg\underset{S}{max}\sum _{j\in S}{t}_j,$$

where the maximization is over all generalized independent sets S of $\left[n\right]$.

Then, we have by (44),

$$\begin{array}{c}\hfill \sum _{j\in S_{max}}{R}_j\le \sum _{J\in \mathcal{J}}{C}_J.\end{array}$$

(45)

In order to relate the bound in (44) with our data exchange bound, we fix $R_j=t_j,\forall j\in \left[n\right]$. This means that we should have $r=1$ in Definition 3. For these parameters, let $s_J^{*}:J\in \mathcal{J}$ be a choice of integers $s_J:J\in \mathcal{J}$ such that the rate-capacity tuple $((R_j=t_j:j\in \left[n\right]),(s_J:J\in \mathcal{J}))$ is achievable and ${\sum }_{J\in \mathcal{J}}{s}_J$ is minimized. Note that such integers $s_J^{*}:J\in \mathcal{J}$ will exist as each index coding problem has at least one solution, namely, the trivial solution consisting of uncoded transmissions of $x_j:j\in \left[n\right]$.

Then, applying (45), we have

$$\begin{array}{c}\hfill \sum _{j\in S_{max}}{t}_j\le \sum _{J\in \mathcal{J}}{s}_J^{*}.\end{array}$$

(46)

Note that ${\sum }_{J\in \mathcal{J}}{s}_J^{*}$ is exactly the minimum number of bits to be communicated by the servers for satisfying receiver demands.

For $Q\subset \left[n\right]$, define

$$\begin{array}{c}\hfill f_j^Q\triangleq \left\{\begin{array}{cc}1\hfill & \mathrm{if}j\in \left({\cap }_{k\in Q}{A}_k\right)\setminus \left({\cup }_{k^{\prime }\in \left[n\right]\setminus Q}{A}_{k^{\prime }}\right),\hfill \\ 0\hfill & \mathrm{otherwise}.\hfill \end{array}\right.\end{array}$$

By arguments similar to that of the proof of Theorem 5, we can verify that

$$\begin{array}{c}\hfill \sum _{j\in S_{max}}{t}_j=\underset{\gamma }{max}\sum _{j=1}^n\sum _{Q\subset \{{\gamma }_{j+1},\cdots ,{\gamma }_n\}}{f}_{{\gamma }_j}^Q{t}_{{\gamma }_j},\end{array}$$

(47)

where the maximization is over all possible permutations $\gamma =({\gamma }_1,\cdots ,{\gamma }_n)$ of $(1,\cdots ,n).$ We thus have by (46) and (47),

$$\begin{array}{c}\hfill \sum _{J\in \mathcal{J}}{s}_J^{*}\ge \underset{\gamma }{max}\sum _{j=1}^n\sum _{Q\subset \{{\gamma }_{j+1},\cdots ,{\gamma }_n\}}{f}_{{\gamma }_j}^Q{t}_{{\gamma }_j}.\end{array}$$

(48)

Finally, we apply our data exchange bound in Theorem 1 to the distributed index coding setting. To do this, we first observe that if we replaced all servers by a single “virtual” central server containing all the messages, $x_j:j\in \left[n\right],$ then ${\sum }_{J\in \mathcal{J}}{s}_J^{*}$ is the minimum number of bits to be transmitted by this virtual central server to satisfy the receiver demands. Any lower bound on the communication cost for this transformed setting with the virtual server will thus continue to apply for the original distributed setting with messages of length $t_j:j\in \left[n\right]$. Now, utilizing the centralized version of Theorem 1 shown in Theorem 4 and by the discussion in Section 5.2, we get

$$\begin{array}{c}\hfill \sum _{J\in \mathcal{J}}{s}_J^{*}\ge \frac{1}{n!}\sum _{\gamma }\sum _{j=1}^n\sum _{Q\subset \{{\gamma }_{j+1},\cdots ,{\gamma }_n\}}{f}_{{\gamma }_j}^Q{t}_{{\gamma }_j}.\end{array}$$

(49)

Therefore, we see that the generalized independent set based bound in (48) is in general better than (49), as (48) involves a maximization over all permutations $\gamma ,$ while (49) involves the average.

6.2. Embedded Index Coding

We now consider the embedded index coding problem, introduced in [12], motivated by device-to-device communications. The embedded index coding setting consists of a set of m data blocks (each a binary vector of length t) distributed across a set of n nodes. Each node stores (as side information) a subset of the data blocks and demands another subset which it already does not have. This setting is different from [26,27] or distributed index coding [10], as there are no dedicated servers by default here. Each node transmits a codeword obtained by encoding its data blocks, and each demanded data block at any node is decoded from the codewords obtained from other nodes and the side information at the node itself. An embedded index code consists of a collection of such encoding functions and decoding functions at the nodes, such that all demanded blocks are decoded at the respective nodes. The communication cost of embedded index coding is the total number of bits transmitted between the nodes to satisfy the node demands. The work [12] generalizes the notion of minrank [26] of single-server index codes to define the optimal length of linear embedded index codes. Further, the authors also present heuristic constructions for general and specialized linear codes which have some nice properties.

As the embedded index coding problem clearly has a direct mapping with the data exchange problem considered in the present work, we can apply our data exchange bound directly to obtain a new lower bound for the communication cost of embedded index coding. The expression of this bound would be in the same form (up to only the change in notation) as Theorem 1 itself. As our bound holds in the information-theoretic sense, it would apply to not just the linear codes considered in [12] but nonlinear embedded index codes as well.

7. Conclusions

We have presented an information theoretic converse result for a generic data exchange problem, where the terminals contain some data in their local storage and want other data available at the local storage of other nodes. As a number of recently studied multi-terminal communication problems fall under this setting, we have used our general converse to obtain converses in many such settings, thus recovering many existing results and presenting some new results as well. Using a connection with index coding, we also presented some ideas on why and when our data exchange based converse can be loose in the index coding setting. It would be quite interesting to see if our converse result can be tightened further while still retaining a closed form expression, so as to cover all known bounds for any existing setting that can be modeled in the data exchange framework. A lower bound for the communication load in a generic data exchange setting in the presence of coded storage bits would also be a prospective direction for future research in this area.