Efficient Algorithms for Coded Multicasting in Heterogeneous Caching Networks

Abstract

Coded multicasting has been shown to be a promising approach to significantly improve the performance of content delivery networks with multiple caches downstream of a common multicast link. However, the schemes that have been shown to achieve order-optimal performance require content items to be partitioned into several packets that grows exponentially with the number of caches, leading to codes of exponential complexity that jeopardize their promising performance benefits. In this paper, we address this crucial performance-complexity tradeoff in a heterogeneous caching network setting, where edge caches with possibly different storage capacity collect multiple content requests that may follow distinct demand distributions. We extend the asymptotic (in the number of packets per file) analysis of shared link caching networks to heterogeneous network settings, and present novel coded multicast schemes, based on local graph coloring, that exhibit polynomial-time complexity in all the system parameters, while preserving the asymptotically proven multiplicative caching gain even for finite file packetization. We further demonstrate that the packetization order (the number of packets each file is split into) can be traded-off with the number of requests collected by each cache, while preserving the same multiplicative caching gain. Simulation results confirm the superiority of the proposed schemes and illustrate the interesting request aggregation vs. packetization order tradeoff within several practical settings. Our results provide a compelling step towards the practical achievability of the promising multiplicative caching gain in next generation access networks.

1. Introduction

Recent information-theoretic studies [1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,26,27,28,29,30,31,32,33,34,35,36,37,38,39,40,41,42,43,44,45,46,47,48,49] have characterized the fundamental limiting performance of several caching networks of practical relevance, in which throughput scales linearly with cache size, showing great promise to accommodate the exponential traffic growth experienced in today’s communication networks [50]. In this context, a caching scheme is defined in terms of two phases: the cache placement phase, which operates at a large time-scale and determines the content to be placed at the network caches, and the delivery phase, during which user requests are served from the content caches and sources in the network. Some of the network topologies studied include shared link caching networks [1,2,8,9,10,11,12,13,14], device-to-device (D2D) caching networks [17,18,19,33,34], hierarchical caching networks [24], multi-server caching networks [29], and combination caching networks [36,37,38,39,40,41,42].

Consider a network with one source (e.g., base station) having access to m files, and n users (e.g., small-cell base stations or end user devices), each with a cache memory of M files. In [17], the authors showed that if the users can communicate between each other via D2D communications, a simple distributed random caching policy and TDMA-based unicast D2D delivery achieves the order-optimal throughput $\mathsf{\Theta }\left(max\{\frac{M}{m},\frac{1}{m},\frac{1}{n}\}\right)$ whose linear scaling with M when $Mn\ge m$ exhibits a remarkable multiplicative caching gain, in the sense that the per-user throughput grows proportionally to the cache size M for fixed library size m, and it is independent of the number of users n in the system. Moreover, in this scheme each user caches entire files without the need for partitioning files into packets, and missing files are delivered via unicast transmissions between neighbor nodes, making it efficiently implementable in practice. We recall that order-optimality refers to the fact that the multiplicative gap between information-theoretic converse and achievable performance can be bounded by a constant number when $m,n\to \infty$.

In the case that users cannot communicate between each other, but share a multicast link from the content source, the authors in [8,9] showed that the use of coded multicasting (also referred to as index coding [51]) allows achieving the same order-optimal worst-case throughput as in the D2D caching network. In this case, however, in order to create enough coding opportunities during the delivery phase, requested files are required to be partitioned into a number of packets that grows exponentially with the number of users, leading to coding schemes of exponential complexity [8,9,21].

In [10,12], the authors considered the same shared link caching network, but under random demands characterized by a probability distribution, and proposed a scheme consisting of random aggregate popularity (RAP) placement and chromatic number index coding (CIC) delivery, referred to as RAP-CIC, proved to be order-optimal in terms of average throughput. The authors further provided optimal average rate scaling laws under Zipf [52] demand distributions, whose analytical characterization required resorting to a polynomial-time approximation of CIC, referred to as greedy constrained coloring (GCC). Using RAP-GCC, the authors further established the regions of the system parameters, in which multiplicative caching gains are potentially achievable. While GCC exhibits polynomial complexity in the number of users and packets, the order-optimal performance guarantee still requires, in general, the packetization order (number of packets per file) to grow exponentially with the number of users, as showed in [21].

It is then key to understand if the promised multiplicative caching gain, shown to be asymptotically achievable by the above-referenced schemes, can be preserved in practical settings of finite packetization order. In this context, we shall differentiate between coded multicast schemes that assume a deterministic vs. a random cache placement phase. Deterministic placement policies determine where to store file packets according to a deterministic procedure that takes into account the ID of each packet. In contrast, random placement policies, after determining the number of packets to be cached of each file at each cache, choose the exact packet IDs uniformly at random. While the increased structure of deterministic placement policies can be exploited to design more efficient coded multicast algorithms, random placement policies are desirable in practice, as they provide increased robustness by requiring less cache configuration changes under system dynamics.

The seminal work of [21] showed that all previously proposed schemes (based on both deterministic and random cache placement) required exponential packetization, and that under random placement, no graph-coloring-based coded multicast algorithm can achieve multiplicative caching gains with sub-exponential packetization. Since the fundamental results of [21], several works have studied the now central problem in caching of finite file packetization. The authors in [53] connect the caching problem to resolvable combinatorial designs and derive a scheme that while improving exponentially over previous schemes [8,9,21], still requires exponential packetization. In [54], the authors introduce the combinatorial concept of Placement Delivery Array (PDA) and derive a caching scheme where the packetization scales super-polynomially with the number of users. The work in [22] establishes a connection with the construction of hypergraphs with extremal properties, and provides the first sub-exponential (but still intractable) scheme. Somewhat surprisingly, some of the authors of [21] introduced a new combinatorial design based on Ruzsa-Szeméredi graphs in [30] and showed that a linear scaling of the number of packets per file with n can be achieved for a throughput of $\mathsf{\Theta }\left(n^{-\delta }\right)$, where $\delta$ can be arbitrarily small. However, all the above studies focus on coded multicast algorithms that assume a deterministic cache placement phase. Under random cache placement, several coded multicast algorithms have been proposed in the context of homogenous shared link caching networks [55,56,57,58,59,60], including our previous work that serves as the basis for this paper.

In this work, we address the important problem of finite-length coded multicasting under random cache placement, focusing on a more general heterogeneous shared link caching network, in which caches with possibly different sizes collect possibly multiple requests according to possibly different demand distributions (see Figure 1). As shown in Figure 1, this scenario can be motivated by the presence of both end user caches and cache-enabled small-cell base stations or WLAN access points sharing a common multicast link. In this case, each small-cell base station can be modeled as a user cache placing multiple requests. In addition, multiple requests per user also arise in the presence of delay-tolerant content requests (e.g., file downloading). While there have been several information-theoretic studies of shared link caching networks with distinct cache sizes [61,62,63], and with multiple per-user requests [13,14,34,64,65], none of these works considered the finite-length regime nor addressed the joint effect of random demands, heterogenous cache sizes, and multiple per-user requests.

The contributions of this paper are as follows:

• We provide a generalized model for heterogeneous shared link caching networks, in which users can have different cache sizes and make different number of requests according to different demand distributions.
• We design two novel coded multicast algorithms based on local graph coloring, referred to as Greedy Local Coloring (GLC) and Hierarchical Greedy Local Coloring (HgLC) that exhibit polynomial-time complexity in both the number of caches and the packetization order. In combination with the Random Aggregate Popularity (RAP) placement policy of [10,12], we show that the overall schemes RAP-GLC and RAP-HgLC are order-optimal in the asymptotic file-length regime.
• Focusing on the finite-length regime, in which content items can be partitioned into a finite number of packets, we show how the general advantage of local graph coloring is especially relevant when the number of per-user requests grow. We validate via simulations the superiority of RAP-GLC, especially with high number of per-user requests. We then show how RAP-HgLC, with a slight increase in the polynomial complexity order, further improves the caching gain of RAP-GLC, remarkably approaching the multiplicative gain that existing schemes can only guarantee in the asymptotic file-length regime.
• We demonstrate that there is a tradeoff between the required packetization order and the number of requested files per user. In particular, for a given target gain, if the number of requests increases, then the number of packets per file can be reduced, while preserving the target gain. We further quantify the regime of per-user requests for which a caching scheme with unit packetization order (i.e., a scheme that treats only whole files) is order-optimal. Our analysis illustrates the key impact of content request aggregation in time and space on caching performance. That is, if edge caches can wait for collecting multiple requests over time and/or aggregate requests from multiple users, the same performance can be achieved with lower packetization order, and hence lower computational complexity.

The paper is organized as follows. Section 2 introduces the network model and problem formulation. Section 3 describes the construction of coded multicast algorithms using graph coloring, with special focus on the advantages of local graph coloring. Section 4 presents novel polynomial-time local-graph-coloring-based coded multicast schemes. Section 5 analyzes the effect of request aggregation on the performance–complexity tradeoff. Section 6 presents simulation results and related discussions. Finally, concluding remarks are given in Section 7.

2. Network Model and Problem Formulation

We consider a caching network formed by a source node with access to a content library, connected to several caching nodes/users via a single shared (multicast) link. Similar to previous works [8,9,10,12,13,14,21,22,30], we define a caching scheme in terms of two phases:

• Placement phase, which operates at a large time-scale and determines the content to be placed at the caching nodes,
• Delivery phase, during which users requests are served from the content caches and sources in the network.

However, differently from previous works, we generalize the model to a heterogeneous system in which each caching node has a possibly different cache size and requests a possibly different number of files. A practical example of our setting can be represented by a macro base station connected to several cache-enabled small-cell base stations, and a number of user devices served either by the macro base station or by the small-cell base stations. In this setting, each small cell acts as a super user requesting multiple files resulting from the requests of the users it serves.

Specifically, the heterogeneous caching network consists of a single source node storing a library of files $\mathcal{F}=\{1,\cdots ,m\}$, each with entropy F bits, and n user nodes $\mathcal{U}=\{1,\cdots ,n\}$, each with a cache of storage capacity $M_{u}F$ bits (i.e., each user caches up to $M_u$ files). Each user u can requests $L_u$$(1\le L_u\le m)$ different files according to its individual request probability distribution. We assume that the library files have finite length and consequently a finite packetization order. Our main objective is to design a caching scheme that minimizes the number of transmissions required to satisfy the demands of all users.

In a homogeneous network setting with infinite packetization order, recent works [8,9,10,12,13,14] have shown that it is possible to satisfy a scaling number of users with only a constant number of multicast transmissions. The achievable schemes configure user caches with complementary (side) information during the caching phase, such that the resulting coded multicasting opportunities that arise during the delivery phase can be used to minimize the transmission rate (or load) over the shared multicast link. Specifically, reference [12] showed that under Zipf file popularity, a properly optimized random fractional placement policy, referred to as Random Aggregate Popularity (RAP) caching, achieves order-optimality when combined with a graph-coloring-based coded multicast scheme. Unfortunately, even in the homogenous setting, it was shown in [21] that a central limitation of all previous works is that they require infinite packetization order: all existing caching schemes achieve at most a factor of two gain when the packetization order is finite.

In this work, inspired by the fundamental throughput-delay-memory tradeoff derived in [21], our goal is to design computationally efficient schemes that provide good performance in the finite packetization regime. For the caching phase, (1) we restrict our placement policies to the class of random fractional schemes described in [9,10,12,13,14], proved to be order-optimal in the homogeneous setting. For the delivery phase, (2) we focus on the class of graph-coloring-based index coding schemes, and design two novel polynomial-time algorithms that employ local graph coloring on the (index coding) conflict graph [51].

2.1. Random Fractional Cache Placement

The class of random fractional placement schemes is described as follows:

• Packetization: Each file is partitioned into B packets of equal-size $F/B$ bits, where the integer B is referred to as the packetization order. Each packet is represented by a symbol in finite field ${\mathbb{F}}_{2^{F/B}}$, where we assume that $F/B$ is large enough.
• Random Placement: Each user u caches $p_{f,u}{M}_{u}B$ packets independently at random from each file f, where $p_{f,u}$ is the probability that file f is cached at user u, and satisfies $0\le p_{f,u}\le 1/M_u,\forall f\in \mathcal{F}$ such that ${\sum }_{f=1}^m{p}_{f,u}=1,\forall u\in \mathcal{U}$.

We introduce a caching distribution matrix $\mathbf{P}=\left[p_{f,u}\right]\in {\mathbb{R}}_{+}^{m\times n}$, where $f\in \mathcal{F}$ and $u\in \mathcal{U}$. Please note that the number of packets of file f cached at user u, $p_{f,u}{M}_{u}B$, can be directly determined from the caching distribution matrix $\mathbf{P}$. As described in [10,12,13,14], the caching distribution must be properly optimized to balance the gains from local cache hits (where requested packets are served by the local cache) and coded multicast opportunities (where requested packets are served by coded transmissions that simultaneously satisfy distinct user requests). When this is the case, we refer to the cache placement scheme as Random Aggregate Popularity (RAP) caching (see e.g., [10,12,13,14]). Given the number of packets to be cached of a given file, the actual indices of the packets to be cached are chosen uniformly at random, and independently across users. We use ${\mathbf{C}}_{u,f}$ to denote the set of packets of file f cached at user u and $\mathbf{C}=\left\{{\mathbf{C}}_{u,f}\right\}$ with $u\in \mathcal{U}$ and $f\in \mathcal{F}$ to denote the packet-level cache placement realization.

The goal of the placement phase is to configure the user caches to create coding opportunities during the delivery phase that allow serving distinct user requests via common multicast transmissions. Compared to deterministic placement [8], random placement schemes allow configuring user caches with lower complexity and increased robustness, i.e., changes in system parameters (e.g., number of users, number files, file popularity) require less changes in users’ cache configurations [12].

Recall that the placement phase operates at a much larger time-scale than the delivery phase, and hence is unaware of the requests in the subsequent delivery rounds. Therefore, the placement phase can be designed according to the demand distribution, but must be independent of the requests realizations.

2.2. Random Multiple Requests

Each user $u\in \mathcal{U}$ requests $L_u$$(1\le L_u\le m)$ files independently from other users, following a probability distribution $q_{f,u}$ with $q_{f,u}\in [0,1]$ and ${\sum }_{f=1}^m{q}_{f,u}=1$ (i.e., for each request of user u, file f is chosen with probability $q_{f,u}$). We introduce a demand distribution matrix $\mathbf{Q}=\left[q_{f,u}\right]\in {\mathbb{R}}_{+}^{m\times n}$, where $f\in \mathcal{F}$ and $u\in \mathcal{U}$. In the following, we use $\mathbf{W}=\left\{{\mathbf{W}}_{u,f}\right\}$, with $u\in \mathcal{U}$ and $f\in \mathcal{F}$, to denote the packet-level demand realization where ${\mathbf{W}}_{u,f}$ denotes the packets of file f requested by user u.

The multiple-request parameters $\left\{L_u\right\}$ have a key operational meaning, in that it captures the possibility of edge caches to collect requests across time and space. That is, $L_u$ may represent the amount of requests collected over time (given the delay tolerance of some content requests) as well as the amount of requests collected across space from users served by the given edge cache (e.g., when edge caches are located at helper nodes or small-cell base stations serving multiple individual users).

2.3. Performance Metric

For given realizations of the random fractional cache placement and the random multiple requests, the goal is to design a delivery scheme that minimizes the rate over the shared multicast link required to satisfy all user requests. Since one placement phase is followed by an arbitrarily large number of delivery rounds (each characterized by a new independent request realization), the rate (or load) of the system refers only to the delivery phase (i.e., asymptotically the cache placement costs no rate). Furthermore, it makes sense to consider the average rate, where averaging with respect to the users request distribution takes on the meaning of a time-averaged rate, invoking an ergodicity argument.

At each request round, let $\mathbf{F}=\{{\mathbf{f}}_1,{\mathbf{f}}_2,\cdots ,{\mathbf{f}}_n\}$ be the demand realization, where ${\mathbf{f}}_u=\{f_{1,u},f_{2,u},\cdots ,f_{L_u,u}\},u\in \mathcal{U}$. The source node computes a multicast codeword as a function of the library and the demand realization $\mathbf{F}$. We assume that the source node communicates to the user nodes through an error-free deterministic shared multicast link.

Given the demand realization $\mathbf{F}$, let the total number of bits transmitted by the source node be $J\left(\mathbf{F}\right)$. We are interested in the average performance of the coded multicast scheme, and hence define the average rate (or load) as the number of transmitted bits normalized by the file size:

$$R=\frac{\mathcal{E}\left[J\left(\mathbf{F}\right)\right]}{F},$$

(1)

where the expectation is over the random demand distribution.

3. Graph-Coloring-Based Coded Multicast Delivery

It is important to note that for given cache placement and demand realizations, the delivery phase of a caching scheme reduces to an index coding problem with a twist. The only difference with the conventional index coding problem introduced in [51] is that the cache information may contain part of (as opposed to entire) requested files, and that users may request multiple (as opposed to single) files. Nevertheless, as in index coding, the problem can still be represented by a conflict graph [10,12,13,14], where vertices represent requested packets, and an edge between two vertices indicates a conflict, in the sense that the packet represented by one vertex is not present in the cache of the user requesting the packet represented by the other vertex. By construction, packets with no conflict in the graph can be simultaneously transmitted via an XOR operation. Performing graph coloring on the conflict graph and transmitting the packets via proper XOR operations, according to the graph coloring, results in an achievable linear index coding scheme, which we refer to as a coded multicast scheme.

In the following, we first illustrate how to construct the conflict graph, we then review classical linear index coding schemes, and then describe our proposed graph-coloring-based coded multicast schemes.

3.1. Conflict Graph Construction

Given cache placement realization $\mathbf{C}$ and demand realization $\mathbf{W}$, the directed conflict graph ${\mathcal{H}}_{\mathbf{C},\mathbf{W}}^d=(\mathcal{V},\mathcal{E})$ can be constructed as follows:

• Vertices: For each packet request in $\mathbf{W}$, there is a vertex in ${\mathcal{H}}_{\mathbf{C},\mathbf{W}}^d$. Each vertex $v\in \mathcal{V}$ is uniquely identified or labeled by a packet-user pair $\left\{\rho \right(v),\mu (v\left)\right\}$, where $\rho \left(v\right)$ denotes the identity of the packet, and $\mu \left(v\right)$ the user requesting it. Hence, if a packet is requested by multiple users, such a packet is represented in as many vertices as the number of users requesting it. Such vertices have the same packet label $\rho \left(v\right)$, but different user label $\mu \left(v\right)$.
• Arcs: For any $v_1,v_2\in \mathcal{V}$, there is an edge $(v_2,v_1)\in \mathcal{E}$ with direction from $v_2$ to $v_1$ if and only if $\rho \left(v_1\right)\ne \rho \left(v_2\right)$ and packet $\rho \left(v_1\right)$ is not in the cache of user $\mu \left(v_2\right)$.

To better understand the rationale behind the conflict graph and its construction, note that for any two vertices $v_1$ and $v_2$ that are labeled as $\{\rho \left(v_1\right),\mu \left(v_1\right)\}$ and $\{\rho \left(v_2\right),\mu \left(v_2\right)\}$, respectively, we have the following three possible cases:

• $\rho \left(v_1\right)\ne \rho \left(v_2\right)$ and $\mu \left(v_1\right)=\mu \left(v_2\right)$: This indicates that two different packets are requested by the same user. Then, $v_1$ and $v_2$ are mutually conflicting, in the sense that if sent within the same time-frequency resource they interfere with each other. Hence, in the conflict graph, they are connected with two directed edges, $(v_1,v_2)\in \mathcal{E}$ and $(v_2,v_1)\in \mathcal{E}$;
• $\rho \left(v_1\right)=\rho \left(v_2\right)$ and $\mu \left(v_1\right)\ne \mu \left(v_2\right)$: This indicates that the same packet is requested by two different users. Then, $v_1$ and $v_2$ are not conflicting, and hence not connected in the conflict graph; i.e., $(v_1,v_2)\notin \mathcal{E}$ and $(v_2,v_1)\notin \mathcal{E}$;
• $\rho \left(v_1\right)\ne \rho \left(v_2\right)$ and $\mu \left(v_1\right)\ne \mu \left(v_2\right)$: This indicates that two different packets are requested by two different users. In this case, if packet $\rho \left(v_1\right)$ is in the cache of user $\mu \left(v_2\right)$, then, even if $\rho \left(v_1\right)$ and $\rho \left(v_2\right)$ are sent within the same time-frequency resource, user $\mu \left(v_2\right)$ will not suffer from interference, since, using its cache information, it can cancel out the undesired packet $\rho \left(v_1\right)$ from the received signal. On the other hand, if packet $\rho \left(v_1\right)$ is not in the cache of user $\mu \left(v_2\right)$, then $v_1$ conflicts with $v_2$, and a directed edge is drawn from $v_2$ to $v_1$. Similarly, $(v_1,v_2)\in \mathcal{E}$ if and only if $\rho \left(v_2\right)\notin {\mathbf{C}}_{\mu \left(v_1\right)}$.

Based on the above construction, it follows that the number of interference dimensions faced by a given node is at most the number of its outgoing neighbors.

To illustrate the construction of the directed conflict graph ${\mathcal{H}}_{\mathbf{C},\mathbf{W}}^d$, we present the following example.

Example 1.

We consider a network with $n=3$ users denoted as $\mathcal{U}=\{1,2,3\}$ and $m=3$ files denoted as $\mathcal{F}=\{\mathrm{A},\mathrm{B},\mathrm{C}\}$. We assume $M_u=1,\forall u\in \mathcal{U}$ and partition each file into three packets. For example, $\mathrm{A}=\{{\mathrm{A}}_1,{\mathrm{A}}_2,{\mathrm{A}}_3\}$. Let $p_{\mathrm{A},u}=p_{\mathrm{B},u}=p_{\mathrm{C},u}=\frac{1}{3}$ for $u\in \mathcal{U}$, which means that one packet from each of $\mathrm{A},\mathrm{B},\mathrm{C}$ is stored in each user’s cache. For the sake of notational convenience, we assume a symmetric caching realization, where the caching configuration $\mathbf{C}$ is given by ${\mathbf{C}}_{u,\mathrm{A}}=\left\{{\mathrm{A}}_u\right\},{\mathbf{C}}_{u,\mathrm{B}}=\left\{{\mathrm{B}}_u\right\},{\mathbf{C}}_{u,\mathrm{C}}=\left\{{\mathrm{C}}_u\right\}$). That is, the cache configuration of each user $u\in \mathcal{U}$ is ${\mathbf{C}}_u=\{{\mathrm{A}}_u,{\mathrm{B}}_u,{\mathrm{C}}_u\}$. We let each user make two requests, i.e., $L_u=2$$(\forall u\in \mathcal{U})$. Specifically, we let user 1 request $\mathrm{A},\mathrm{B}$, user 2 request $\mathrm{B},\mathrm{C}$, and user 3 request $\mathrm{C},\mathrm{A}$, i.e., ${\mathbf{f}}_1=\{\mathrm{A},\mathrm{B}\},{\mathbf{f}}_2=\{\mathrm{B},\mathrm{C}\},{\mathbf{f}}_3=\{\mathrm{C},\mathrm{A}\}$), such that ${\mathbf{W}}_1=\{{\mathrm{A}}_2,{\mathrm{A}}_3,{\mathrm{B}}_2,{\mathrm{B}}_3\},{\mathbf{W}}_2=\{{\mathrm{B}}_1,{\mathrm{B}}_3,{\mathrm{C}}_1,{\mathrm{C}}_3\},{\mathbf{W}}_3=\{{\mathrm{A}}_1,{\mathrm{A}}_2,{\mathrm{C}}_1,{\mathrm{C}}_2\}$. The associated directed conflict graph is shown in Figure 2.

3.2. Code Construction

Let ${\omega }_v$ denote the content (or realization) of packet $\rho \left(v\right)$, $v\in \mathcal{V}$, represented by a symbol in ${\mathbb{F}}_q$. In general, in a linear index coding scheme of length ℓ, every vertex v is associated with a “coding" vector ${\mathbf{g}}_v\in {\mathbb{F}}_q^{\ell \times 1}$ where $v\in [1:|\mathcal{V}\left|\right]$. Let $\mathbf{G}=[{\mathbf{g}}_1,\cdots {\mathbf{g}}_{\left|\mathcal{V}\right|}]$ and $\mathbf{\omega }={[{\omega }_1,\cdots ,{\omega }_{\left|\mathcal{V}\right|}]}^{\mathrm{T}}$. Then, the transmitted codeword, $\mathbf{x}\in {\mathbb{F}}_q^{\ell \times 1}$, is built as follows:

$$\begin{array}{c}\hfill \mathbf{x}=\sum _{v\in \mathcal{V}}{\omega }_v{\mathbf{g}}_v=\mathbf{G}\mathbf{\omega },\end{array}$$

(2)

Let $\mathcal{N}\left(v\right)=\{w:(v,w)\in \mathcal{E}\}$ be the out-neighborhood of v. For any feasible scalar linear index coding scheme of the form (2), the following interference alignment condition is necessary: For every vertex v, the coding vector ${\mathbf{g}}_v$ should be linearly independent of all the coding vectors assigned to the out-neighborhood of v.

In the following, we describe how to construct coding vectors satisfying the interference alignment condition for every vertex. For ease of notation, we use ${\mathcal{H}}^d$ to denote the directed conflict graph, and $\mathcal{H}$ to represent its underlying undirected skeleton, where the direction of edges is ignored. Recall that an undirected skeleton of a directed graph ${\mathcal{H}}^d$ is an undirected graph where there is an undirected edge between $v_1$ and $v_2$ if, between $v_1$ and $v_2$, there is a directed edge in either or both directions in ${\mathcal{H}}^d$.

3.2.1. Graph Coloring and Chromatic Number

A well-known procedure to construct the coding vectors $\{{\mathbf{g}}_v,v\in \mathcal{N}\left(v\right)\}$ is the coloring of ${\mathcal{H}}^d$. In the following, when used without any qualification, a coloring of a directed graph is considered to be a proper (vertex) coloring of its underlying undirected skeleton $\mathcal{H}$, where a proper coloring is a labeling of the graph’s vertices with colors, such that no two vertices sharing the same edge have the same color. Please note that by definition, any subset of nodes with the same color in a proper coloring form an independent set (i.e., a subset of nodes in a graph, no two of which share the same edge). A coloring using at most k colors is called a (proper) k-coloring. The smallest number of colors needed in a proper coloring of ${\mathcal{H}}^d$ is called its chromatic number, and is denoted by $\chi \left({\mathcal{H}}^d\right)$. In the following, we explain why a coloring of ${\mathcal{H}}^d$ provides a way to design the coding vectors $\{{\mathbf{g}}_v,v\in \mathcal{N}\left(v\right)\}$. Let $\xi$ be the total number of colors in a given coloring of ${\mathcal{H}}^d$. Let ${\mathbf{e}}_i$ be the i-th unit vector in the space ${\mathbb{F}}_q^{\ell \times 1}$, with $\ell =\xi$, i.e., ${\mathbf{e}}_i={[0,0,\cdots ,1,\cdots ,0,0]}^{\mathrm{T}}$, where the 1 is in the i-th position. Now, if vertex v is colored with color i, then, its coding vector is ${\mathbf{g}}_v={\mathbf{e}}_i$. Making this choice for the coding vectors, the associated achievable rate is given by $\frac{\xi }{B}$. Since neighbors are assigned different colors, the interference alignment condition is satisfied for every vertex. Recalling the definition of $\chi \left({\mathcal{H}}^d\right)$, it is immediate to see that the best achievable rate due to conflict graph coloring is given by $\frac{\chi \left({\mathcal{H}}^d\right)}{B}$, and, according to the construction of the conflict graph, it is loosely bounded by:

$$\begin{array}{c}\hfill \sum _{f\in {\mathbf{f}}_u}(1-p_{f,u}{M}_u)\le \frac{\chi \left({\mathcal{H}}^d\right)}{B}\le \sum _{u=1}^n\sum _{f\in {\mathbf{f}}_u}(1-p_{f,u}{M}_u),\end{array}$$

(3)

indicating that the achievable rate is a constant with regards to B. A much tighter bound will be given in Section 4.1.

3.2.2. Local Graph Coloring and Local Chromatic Number

More efficient sets of coding vectors can be constructed using the approach proposed in [66], which exploits the direction information in ${\mathcal{H}}_{\mathbf{C},\mathbf{W}}^d$, resulting in the following advanced coding scheme:

Definition 1 (Local Coloring Number).

Given a proper coloring $\mathbf{c}$ of ${\mathcal{H}}^d$, the associated local chromatic number is defined as:

$$\begin{array}{c}{\xi }_{\mathrm{lc}}\left(\mathbf{c}\right)=\underset{v\in \mathcal{V}}{max}\left|\mathbf{c}\left({\mathcal{N}}^{+}\left(v\right)\right)\right|\hfill \end{array}$$

(4)

where ${\mathcal{N}}^{+}\left(v\right)$ is the closed out-neighborhood of vertex v (i.e., vertex v and all its ongoing neighbors $\mathcal{N}\left(v\right)$) and $\left|\mathbf{c}\right({\mathcal{N}}^{+}\left(v\right)\left)\right|$ is the total number of colors in ${\mathcal{N}}^{+}\left(v\right)$ for a given proper color assignment $\mathbf{c}$.

The minimum local coloring number over all proper colorings is referred to as the local chromatic number and is formally defined as follows:

Definition 2 (Local Chromatic Number).

The directed local chromatic number of a directed graph ${\mathcal{H}}^d$ is defined as:

$$\begin{array}{c}{\chi }_{\mathrm{lc}}\left({\mathcal{H}}^d\right)=\underset{\mathbf{c}\in \mathcal{C}}{min}{\xi }_{\mathrm{lc}}\left(\mathbf{c}\right)\hfill \end{array}$$

(5)

where $\mathcal{C}$ denotes the set of all proper coloring assignments of ${\mathcal{H}}^d$, ${\mathcal{N}}^{+}\left(v\right)$ is the closed out-neighborhood of vertex v, and $\left|\mathbf{c}\right({\mathcal{N}}^{+}\left(v\right)\left)\right|$ is the total number of colors in ${\mathcal{N}}^{+}\left(v\right)$ for a given proper color assignment $\mathbf{c}$.

Encoding Scheme: For a given realization of the cache placement ($\mathbf{C}$) and user requests ($\mathbf{W}$), let us consider the conflict graph ${\mathcal{H}}_{\mathbf{C},\mathbf{W}}^d$ as in Section 3.1. Given a (proper) $\xi$-coloring (i.e., a proper coloring of graph ${\mathcal{H}}_{\mathbf{C},\mathbf{W}}^d$ with $\xi$ colors), we compute the associated local coloring number ${\xi }_{\mathrm{lc}}$. Set $\ell ={\xi }_{\mathrm{lc}}$ and $p=\xi$. Then, consider the columns of the generator $\mathbf{H}$ of an $\ell \times p$ Maximum Distance Separable (MDS) [67] code over the field ${\mathbb{F}}_q:q>p$. If the color of a vertex v is i, then the coding vector ${\mathbf{g}}_v$ assigned to vertex v is given by i-th column ${\mathbf{h}}_i$ of $\mathbf{H}$. Then, the transmitted multicast codeword, $\mathbf{x}\in {\mathbb{F}}_q^{\ell \times 1}$, is given by (2).

Decoding Scheme: In any closed out-neighborhood, there are at most ℓ different colors (from the definition of local coloring). Since every ℓ columns of $\mathbf{H}$ are linearly independent (from the defining property of MDS codes), the coding vectors in any closed out-neighborhood have full rank, satisfying the interference alignment condition. The message ${\omega }_v$ at vertex v is obtained at user v as follows: (1) Using side information at user v, cancel out message parts corresponding to all vertices outside ${\mathcal{N}}^{+}\left(v\right)$, i.e., ${\mathbf{x}}^{\prime }=\mathbf{x}-{\displaystyle \sum _{u\notin {\mathcal{N}}^{+}\left(v\right)}}{\omega }_u{\mathbf{g}}_u$. This is possible because, by the definition of the conflict graph ${\mathcal{H}}^d$, the messages ${\left\{{\omega }_u\right\}}_{u\notin {\mathcal{N}}^{+}\left(v\right)}$ are available as side information at user v and the encoding mechanism is known to all the users. (2) Find a vector $\mathbf{z}$ in the dual space of ${\left\{{\mathbf{g}}_u\right\}}_{u\in {\mathcal{N}}^{+}\left(v\right)\backslash \left\{v\right\}}$ such that ${\mathbf{z}}^T{\mathbf{x}}^{\prime }\ne 0$ (this is possible since ${\mathbf{g}}_v$ is linearly independent of ${\left\{{\mathbf{g}}_u\right\}}_{u\in {\mathcal{N}}^{+}\left(v\right)\backslash \left\{v\right\}}$ because of the local chromatic number-based construction). Now, ${\mathbf{z}}^T{\mathbf{x}}^{\prime }=\left({\mathbf{z}}^T{\mathbf{g}}_v\right){\omega }_v$. Therefore, user v recovers its own message. It follows that all users can recover all the requested packets employing such linear scheme.

Achievable Rate: The coding scheme constructed as described above achieves a rate given by ${\xi }_{\mathrm{lc}}/B$, where B is the number of packets per file.

Example 2.

We consider an example shown in Figure 3. First, we assign colors to each vertex such that the total number of colors $\xi =5$, and count the local coloring number, which is ${\xi }_{\mathrm{lc}}=4$. Then, we construct the generator matrix $\mathbf{A}$ of a $(\xi =5,{\xi }_{\mathrm{lc}}=4)$ MDS code, which is given by

$$\mathbf{A}=\left(\begin{array}{ccccc}1& 0& 0& 0& 1\\ 0& 1& 0& 0& 1\\ 0& 0& 1& 0& 1\\ 0& 0& 0& 1& 1\end{array}\right).$$

(6)

After that, we assign the columns of $\mathbf{A}$ to ${\mathbf{g}}_v$, corresponding from the left to the right to the vertices with the packets $\{{\mathrm{A}}_2,{\mathrm{A}}_3,{\mathrm{B}}_2,{\mathrm{B}}_1,{\mathrm{A}}_1\}$, as shown in Figure 3. Finally, the transmitted codewords can be generated which are the rows of the right-hand side of (2)

$$X\left(1\right)={\mathrm{A}}_1\oplus {\mathrm{A}}_2,X\left(2\right)={\mathrm{A}}_1\oplus {\mathrm{A}}_3$$

(7)

$$X\left(3\right)={\mathrm{A}}_1\oplus {\mathrm{B}}_1,X\left(4\right)={\mathrm{A}}_1\oplus {\mathrm{B}}_2$$

(8)

where the length of the code is ${\xi }_{\mathrm{lc}}/B=4/3$ file units. It can be easily verified that every user can decoded its desired packets with the cached ones.

It is immediate to see that the best achievable rate due to local coloring is obtained by computing the local chromatic number of ${\mathcal{H}}_{\mathbf{C},\mathbf{W}}^d$ and using its associated coloring to design the coding vectors, yielding a rate ${\chi }_{\mathrm{lc}}/B$. However, note that to compute ${\chi }_{\mathrm{lc}}$, we must optimize over all proper colorings to find the local chromatic number. As with the chromatic number, this can be cast as an Integer Program and it is hence an NP-hard problem. To overcome this limitation, in Section 4, we propose a greedy approach that (i) exhibits polynomial-time complexity in all the system parameters, (ii) achieves close to optimal performance for finite packetization order, and (iii) is asymptotically (i.e., for infinite packetization order) order-optimal.

3.3. Benefits of Local Coloring

Consider the following relation established for general directed graphs in [66]:

$${\chi }_{\mathrm{lc}}\left({\mathcal{H}}^d\right)\le \chi \left({\mathcal{H}}^d\right)\le {\chi }_{\mathrm{lc}}\left({\mathcal{H}}^d\right)O(logn).$$

(9)

Focusing on the conflict graph of interest ${\mathcal{H}}_{\mathbf{C},\mathbf{W}}^d$, the number of vertices can be as large as $B{\sum }_{u\in \mathcal{U}}{L}_u$. It then follows from (9) that the gap between the local chromatic number and the chromatic number can be as large as $log\left(B{\sum }_{u\in \mathcal{U}}{L}_u\right)$. Please note that this multiplicative factor grows with the number of packets per file B and the number of per-user requests $L_u$, supporting the extra benefit of local coloring in the multiple-request scenario. In addition, the higher the number of per-user requests, the higher the directionality of the conflict graph, which is the main factor exploited by local coloring to reduce the achievable rate (see Section 3.2.2), further supporting the suitability of local coloring in increasingly practical settings where there is some form of spatial or temporal request aggregation.

4. Proposed Algorithms and Performance Analysis

As stated earlier, computing the local chromatic number is NP-hard. To circumvent this challenge, in this section, we propose two greedy coded multicast schemes, which together with the cache placement described in Section 2.1, yield the following two caching schemes: Randomized Aggregate Popularity-Greedy Local Coloring (RAP-GLC) and Randomized Aggregate Popularity-Hierarchical greedy Local Coloring (RAP-HgLC). In both cases, the steps for obtaining the coded multicast scheme are as follow:

• Given a realization of the cache placement ($\mathbf{C}$) and of the user requests ($\mathbf{W}$), build the conflict graph ${\mathcal{H}}_{\mathbf{C},\mathbf{W}}^d$ as in Section 3.1.
• Use any of the above algorithms (GLC or HgLC) to compute a proper coloring. Let $\xi$ denote the number of colors used by either of the above algorithms to color ${\mathcal{H}}_{\mathbf{C},\mathbf{W}}^d$. Let ${\xi }_{\mathrm{lc}}$ be the associated local coloring number.
• Consider a $(\xi ,{\xi }_{\mathrm{lc}})$ MDS code and compute the corresponding coded multicast scheme as described in Section 3.2.2.

4.1. Randomized Aggregate Popularity-Greedy Local Coloring (RAP-GLC)

The RAP-GLC algorithm generalizes the RAP-GCC (Random Aggregate Popularity-Greedy Constrained Coloring) algorithm introduced in [12]. RAP-GCC is a caching scheme based on random fractional caching for the placement phase and a coded multicast scheme built on greedy-graph-coloring -based linear index coding [51,68] for the delivery phase. RAP-GLC is more general than RAP-GCC in two aspects: (1) conventional coloring is replaced by local coloring to leverage possible gains in the multiple-request scenario, as described in Section 3.2.1 and Section 3.3, and (2) RAP-GLC adaptively (depending on the demand realization) chooses between naive or coded multicasting according to a threshold parameter, instead of sticking to one of them (as in RAP-GCC).

4.1.1. RAP-GLC Algorithm Description

The algorithm associates to each vertex v a label or tag, composed of two fields i.e., ${\mathcal{K}}_v\equiv ({\mathcal{T}}_D\left(v\right),{\mathcal{T}}_C\left(v\right))$ with ${\mathcal{T}}_C\left(v\right)$ denoting the subset of users caching the packet associated with vertex v, i.e.,

$${\mathcal{T}}_C\left(v\right)\stackrel{\Delta }{=}\{u\in \mathcal{U}:\rho \left(v\right)\in {\mathbf{C}}_u\},$$

(10)

and ${\mathcal{T}}_D\left(v\right)$ denoting the subset of users requesting the packet associated with vertex v, i.e.,

$${\mathcal{T}}_D\left(v\right)\stackrel{\Delta }{=}\{u\in \mathcal{U}:\rho \left(v\right)\in {\mathbf{W}}_u\},$$

(11)

which includes the user itself $\mu \left(v\right)$ who requests $\rho \left(v\right)$ and all the others requesting $\rho \left(v\right)$. Please note that the cardinality of ${\mathcal{T}}_D\left(v\right)$ indicates the popularity of packet $\rho \left(v\right)$. Furthermore, let

$${\mathcal{T}}_v=\left\{\mu \left(v\right)\right\}\cup {\mathcal{T}}_C\left(v\right).$$

Given a vertex v, if the cardinality of ${\mathcal{T}}_D\left(v\right)$ is higher than a predetermined threshold parameter $t\in \{0\cdots ,n\}$ i.e., $|{\mathcal{T}}_D\left(v\right)|>t$, then all vertices $v^{\prime }$ such that $\rho \left(v\right)=\rho \left(v^{\prime }\right)$ are colored with the same color, leading to a naive multicast transmission scheme. If $|{\mathcal{T}}_D\left(v\right)|\le t$, then RAP-GLC greedily looks for a maximal set of vertices with the same ${\mathcal{T}}_v$ (Algorithm 1, Line 14) and colors them with the same color if there is no conflict among the vertices (Algorithm 1, Line 15). The threshold parameter t is subject to optimization, as described in Section 4.1.2.

Doing this, RAP-GLC computes a valid coloring of the conflict graph $\mathcal{H}$. Finally, the algorithm computes its associated local coloring number (Algorithm 1, Line 24). The coding scheme employed is based on the MDS code described in Section 3.2.1 associated with the above local coloring.

Algorithm 1 RAP-GLC

1: Let $\mathcal{C}=\varnothing$;  2: Let $\mathbf{c}=\varnothing$;  3: while $\mathcal{V}\ne \varnothing$ do  4:  Pick an arbitrary vertex v in $\mathcal{V}$; Let $\mathcal{I}=\left\{v\right\}$;  5:  Let ${\mathcal{V}}^{\prime }=\mathcal{V}\backslash \left\{v\right\}$;  6:  if { $|{\mathcal{T}}_D\left(v\right)|>t$ } then  7:   for all $v^{\prime }\in {\mathcal{V}}^{\prime }$ with $\rho \left(v^{\prime }\right)=\rho \left(v\right)$ do  8:    $\mathcal{I}=\mathcal{I}\cup v^{\prime }$;  9:   end for 10:   Color all the vertices in $\mathcal{I}$ by $c\notin \mathcal{C}$; 11:   Let $\mathbf{c}\left[{\mathcal{I}}^{\prime }\right]=c$; 12:   $\mathcal{V}=\mathcal{V}\backslash {\mathcal{I}}^{\prime }$. 13:  else 14:   for all $v^{\prime }\in {\mathcal{V}}^{\prime }$ with ${\mathcal{T}}_{v^{\prime }}\equiv {\mathcal{T}}_v$ do 15:    if {There is no edge between $v^{\prime }$ and $\mathcal{I}$} then 16:     $\mathcal{I}=\mathcal{I}\cup v^{\prime }$; 17:    end if 18:   end for 19:   Color all the vertices in $\mathcal{I}$ by $c\notin \mathcal{C}$; 20:   Let $\mathbf{c}\left[\mathcal{I}\right]=c$; 21:   $\mathcal{V}=\mathcal{V}\backslash \mathcal{I}$. 22:  end if 23: end while 24: return the local coloring number ${max}_{v\in \mathcal{V}}\left|\mathbf{c}\left({\mathcal{N}}^{+}\left(v\right)\right)\right|$ and the corresponding color assignment $\mathbf{c}\left({\mathcal{N}}^{+}\left(v\right)\right)$ for each v;

Time Complexity: In Algorithm 1, both the outer while-loop starting at Line 3, and the inner for-loop starting at Line 6 iterate at most $\left|\mathcal{V}\right|$ times, and all other operations inside the loops take constant time. Therefore, the complexity of RAP-GLC is $O\left(\right|\mathcal{V}{|}^2)$ or, equivalently, $O\left(n^2{B}^2\right)$, since $\left|\mathcal{V}\right|\le nB$, which is polynomial in $\left|\mathcal{V}\right|$ (or $n,B$).

4.1.2. RAP-GLC Performance Analysis

In the following, we quantify the performance of RAP-GLC in the asymptotic regime when the number of users and files is kept constant while the packetization order is sent to infinity. Denoting by $\mathbb{E}\left[R^{\mathrm{RAP}-\mathrm{GLC}}(\mathbf{P},\mathbf{Q},t)\right]$ the asymptotic average achievable rate of RAP-GLC for a fixed threshold t, the threshold parameter t is optimized to minimize $\mathbb{E}\left[R^{\mathrm{RAP}-\mathrm{GLC}}(\mathbf{P},\mathbf{Q},t)\right]$. Hence, denoting by ${\overline{R}}^{\mathrm{RAP}-\mathrm{GLC}}$ the average rate achieved by RAP-GLC with optimized t, i.e.,

$${\overline{R}}^{\mathrm{RAP}-\mathrm{GLC}}=\underset{t}{min}\mathbb{E}\left[R^{\mathrm{RAP}-\mathrm{GLC}}(\mathbf{P},\mathbf{Q},t)\right],$$

we have that

$${\overline{R}}^{\mathrm{RAP}-\mathrm{GLC}}\le min\{\mathbb{E}\left[R^{\mathrm{RAP}-\mathrm{GLC}}(\mathbf{P},\mathbf{Q},n)\right],\mathbb{E}\left[R^{\mathrm{RAP}-\mathrm{GLC}}(\mathbf{P},\mathbf{Q},0)\right].$$

Since $\mathbb{E}\left[R^{\mathrm{RAP}-\mathrm{GLC}}(\mathbf{P},\mathbf{Q},0)\right]$ is just the rate achieved via naive multicasting, then, an upper bound on the average asymptotic performance of RAP-GLC can be obtained by upper bounding $\mathbb{E}\left[R^{\mathrm{RAP}-\mathrm{GLC}}(\mathbf{P},\mathbf{Q},n)\right]$ which can be obtained by generalizing the asymptotic performance analysis of RAP-GCC derived in [10,12] using conventional graph coloring in the homogeneous shared link caching network to the case of using local coloring in the heterogeneous caching network. Specifically, we extend the order-optimality analysis under single per-user requests ($L=1$) in the asymptotic regime of $B\to \infty$ [10,12], to that under multiple $L>1$ per-use requests [13,14]. These theoretical results will serve as rate lower bounds for the finite-length performance of our proposed algorithms.

Let $L={max}_u{L}_u$ and order $L_u,u\in \mathcal{U}$ as a decreasing sequence $L_{\left[1\right]}\ge L_{\left[2\right]}\ge L_{\left[3\right]},\cdots ,L_{\left[n\right]}$, where $L_{\left[i\right]}$ is the i-th largest $L_u$ and $\left[i\right]=u$ for some $u\in \mathcal{U}$. It can be seen that $L_{\left[1\right]}={max}_u{L}_u$ and $L_{\left[n\right]}={min}_u{L}_u$. Let $n_j={\sum }_{\left[i\right]}1\{L_{\left[i\right]}-j\ge 0\}>0$, where $1\le j\le L_{\left[1\right]}$ and $1\{·\}$ is the indicator function. Let ${\mathcal{U}}_{n_j}=\{\left[i\right]\in \mathcal{U}:1\{L_{\left[i\right]}-j\ge 0\}\}$. In the next theorem, we provide a performance guarantee of the RAP-GLC algorithm.

Theorem 1.

For any given m, n, $M_u$, the random caching distribution $\mathbf{P}$ and the random request distribution $\mathbf{Q}$, the average achievable rate of the RAP-GLC algorithm, ${\overline{R}}^{\mathrm{RAP}-\mathrm{GLC}}$ satisfies

$$\begin{array}{c}\hfill {\overline{R}}^{\mathrm{RAP}-\mathrm{GLC}}\le min\{\psi (\mathbf{P},\mathbf{Q}),\overline{m}-\overline{M}\},\end{array}$$

(12)

when $B\to \infty$, where,

$$\begin{array}{cc}\hfill \overline{m}& =\sum _{f=1}^m\left(1-\prod _{u=1}^n{\left(1-q_{f,u}\right)}^{L_u}\right),\hfill \end{array}$$

(13)

$$\begin{array}{cc}\hfill \overline{M}& =\sum _{f=1}^m\left(1-\prod _{u=1}^n{\left(1-q_{f,u}\right)}^{L_u}\right)\underset{u}{min}{p}_{f,u}{M}_u,\hfill \end{array}$$

(14)

$$\begin{array}{cc}\hfill \psi (\mathbf{P},\mathbf{Q})& =\sum _{j=1}^L\sum _{\ell =1}^n\sum _{{\mathcal{U}}^{\ell }\subset {\mathcal{U}}_{n_j}}\sum _{f=1}^m\sum _{u\in {\mathcal{U}}^{\ell }}{\rho }_{f,u,{\mathcal{U}}^{\ell }}\lambda (u,f,{\mathcal{U}}^{\ell }),\hfill \end{array}$$

with ${\mathcal{U}}^{\ell }$ denoting a set of users with cardinality ℓ,

$$\begin{array}{c}\hfill \lambda (u,f_u,{\mathcal{U}}^{\ell })=(1-p_{f_u,u}{M}_u)\times \prod _{k\in {\mathcal{U}}^{\ell }\backslash \left\{u\right\}}\left(p_{f_u,k}{M}_k\right)\prod _{k\in \mathcal{U}\backslash {\mathcal{U}}^{\ell }}(1-p_{f_u,k}{M}_k)\end{array}$$

(15)

and

$$\begin{array}{c}\hfill {\rho }_{f,u,{\mathcal{U}}^{\ell }}\stackrel{\Delta }{=}\mathbb{P}(f=\arg \underset{f_u\in \mathbf{f}\left({\mathcal{U}}^{\ell }\right)}{max}\lambda (u,f_u,{\mathcal{U}}^{\ell })),\end{array}$$

denoting the probability that f is the file whose $p_{f,u}$ maximizes the term $\lambda (u,f_u,{\mathcal{U}}^{\ell })$ among $\mathbf{f}\left({\mathcal{U}}^{\ell }\right)$ (the set of files requested by ${\mathcal{U}}^{\ell }$).

Proof. 

See Appendix A. □

Using the explicit expression for ${\overline{R}}^{\mathrm{RAP}-\mathrm{GLC}}$ in Theorem 1, we can optimize the caching distribution for a wide class of heterogeneous network models to minimize the number of transmissions. We use ${\mathbf{P}}^{*}$ to denote the caching distribution that minimizes $R^{\mathrm{RAP}-\mathrm{GLC}}$.

Remark 1.

For the sake of the numerical evaluation of $\psi (\mathbf{q},\mathbf{p})$, it is worthwhile to note that the probabilities ${\rho }_{f,u,{\mathcal{U}}^{\ell }}$ can be easily computed as follows. Given the subset of users, ${\mathcal{U}}^{\ell }$ of cardinality ℓ, let $J_{u_1},\dots ,J_{u_{\ell }}$ denote ℓ i.i.d. random variables each of them distributed over $\mathcal{F}$ with pmf ${\mathbf{q}}_{u_i}$, with $i=1,\dots ,\ell$. Since $\lambda (u_1,J_{u_1}{\mathcal{U}}^{\ell }),\cdots ,\lambda (u_{\ell },J_{u_{\ell }},{\mathcal{U}}^{\ell })$ are i.i.d., the CDF of $Y_{\ell }\stackrel{\Delta }{=}max\{\lambda (u_1,J_{u_1},{\mathcal{U}}^{\ell }),\cdots ,\lambda (u_{\ell },J_{u_{\ell }},{\mathcal{U}}^{\ell })\}$ is given by

$$\begin{array}{cc}\hfill \mathbb{P}\left(Y_{\ell }\le y\right)& =\prod _{i=1}^{\ell }\mathbb{P}\left(\lambda (u_i,J_{u_i},{\mathcal{U}}^{\ell })\le y\right)\hfill \\ & =\prod _{i=1}^{\ell }\left(\sum _{j\in \mathcal{F}:\lambda (u_i,j,{\mathcal{U}}^{\ell })\le y}{q}_{u_i,j}\right).\hfill \end{array}$$

(16)

Hence, it follows that

$$\begin{array}{cc}\hfill {\rho }_{f,u,{\mathcal{U}}^{\ell }}& =\mathbb{P}(Y_{\ell }=\lambda (u,f,{\mathcal{U}}^{\ell }))\hfill \\ & =\prod _{i=1}^{\ell }\left(\sum _{j\in \mathcal{F}:g_{\ell }\left(j\right)\le \lambda (u,f,{\mathcal{U}}^{\ell })}{q}_{u_i,j}\right)-\prod _{i=1}^{\ell }\left(\sum _{j\in \mathcal{F}:g_{\ell }\left(j\right)<\lambda (u,f,{\mathcal{U}}^{\ell })}{q}_{u_i,j}\right),\hfill \end{array}$$

(17)

which can be easily computed by sorting the values $\{\lambda (u_i,j,{\mathcal{U}}^{\ell }):j\in \mathcal{F},u_i\in {\mathcal{U}}^{\ell }\}$.

Nevertheless, as shown in [21], when B is finite or is not exponential in n, the performance of RAP-GLC can degrade significantly, compromising the promising multiplicative caching gain, although it is already an improved version of RAP-GCC in [12]. This brings us to the other main contribution where we propose a new algorithm that preserves the gain due to coded multicasting even when B is finite.

4.2. Randomized Aggregate Popularity-Hierarchical Greedy Local Coloring (RAP-HgLC) for Finite-Length Packetization

Similarly to RAP-GLC, RAP-HgLC has a predetermined parameter $t\in \{0,\cdots ,n\}$ that is optimized to minimize its associated average achievable rate. However, in the RAP-HgLC algorithm, we arrange the vertices in a hierarchy and use this to design a more careful coloring algorithm. The key idea of RAP-HgLC is to exploit the labeling of each vertex more efficiently. More specifically, as in RAP-GLC, RAP-HgLC associates to each vertex v a label or tag, composed by the two fields ${\mathcal{K}}_v\equiv ({\mathcal{T}}_D\left(v\right),{\mathcal{T}}_C\left(v\right))$, defined in (10) and (11).

4.2.1. RAP-HgLC Algorithm Description

Before jumping into the algorithm, we introduce the following useful notations and their definitions.

• ${\mathcal{G}}_i$: The i-th layer, ${\mathcal{G}}_i$ is initialized with the set of vertices $\{v:|{\mathcal{T}}_v|=i\}$ and at any point in the algorithm contains only vertices with $\left|{\mathcal{K}}_v\right|\ge i$. ${\mathcal{G}}_i$ is updated continuously in the algorithm. Therefore, higher numbered layers contain vertices with greater popularity.
• ${\mathcal{W}}_1\subset {\mathcal{G}}_i$: a subset of ${\mathcal{G}}_i$ consists of all the vertices with $|{\mathcal{K}}_v|=i$ as well as a certain number of vertices with higher popularity (if available at any iteration), defined as

  $$\begin{array}{c}\hfill {\mathcal{W}}_1=\left\{v\in {\mathcal{G}}_i:\underset{v\in {\mathcal{G}}_i}{min}|{\mathcal{K}}_v|\le |{\mathcal{K}}_v|\le \underset{v\in {\mathcal{G}}_i}{min}\left|{\mathcal{K}}_v\right|\right.\left.+⌊a\left(\underset{v\in {\mathcal{G}}_i}{max}|{\mathcal{K}}_v|-\underset{v\in {\mathcal{G}}_i}{min}\left|{\mathcal{K}}_v\right|\right)⌋\right\},\end{array}$$

  (18)

  where $a\in [0,1]$ is a design parameter and ${\mathcal{W}}_1$ is updated with every iteration.
• ${\mathcal{Q}}_i$ (see Algorithm 2): another subset of ${\mathcal{G}}_i$ that is updated every iteration.
• ${\mathcal{W}}_2\subset {\mathcal{Q}}_i$: a subset of vertices in ${\mathcal{Q}}_i$ defined as:

  $$\begin{array}{c}\hfill {\mathcal{W}}_2=\left\{v^{\prime }\in {\mathcal{Q}}_i:\underset{v^{\prime }\in {\mathcal{Q}}_i}{min}|{\mathcal{K}}_{v^{\prime }}|\le |{\mathcal{K}}_{v^{\prime }}|\le \underset{v^{\prime }\in {\mathcal{Q}}_i}{min}\left|{\mathcal{K}}_{v^{\prime }}\right|\right.+\left.⌊b\left(\underset{v^{\prime }\in {\mathcal{Q}}_i}{max}|{\mathcal{K}}_{v^{\prime }}|-\underset{v^{\prime }\in {\mathcal{Q}}_i}{min}\left|{\mathcal{K}}_{v^{\prime }}\right|\right)⌋\right\},\end{array}$$

  (19)

  where $b\in [0,1]$ is another design parameter.

Based on the above definitions, it follows that the total set vertices $\mathcal{V}$ forms an n-layer hierarchy with the i-th layer composed of the set of vertices ${\mathcal{G}}_i$.

Key Idea: Starting from layer n, at any layer $i\le n$, the RAP-HgLC algorithm attempts to form an independent set of size at least i; when there are no more such independent sets, all remaining packets are dropped to layer $i-1$, and transmission actions on those packets are deferred to later layers. This is the key difference between RAP-HgLC and RAP-GLC. That is, RAP-HgLC makes an extra effort to place nodes with large labels into large independent sets.

We will now describe how the above key idea is implemented in RAP-HgLC. The  RAP-HgLC algorithm forms large independent sets in a “top-down” fashion, starting with the highest layer, and iteratively moving to lower layers until layer 1. The following two steps are performed at each layer:

• Step I: The first step is similar to that in RAP-GLC algorithm. Given a vertex v, the algorithm first checks if the cardinality of ${\mathcal{T}}_D\left(v\right)$ is higher than t, i.e., $|{\mathcal{T}}_D\left(v\right)|>t$ then all the vertices $v^{\prime }$ such that $\rho \left(v\right)=\rho \left(v^{\prime }\right)$ are colored with the same color. If $|{\mathcal{T}}_D\left(v\right)|\le t$ then the algorithm greedily finds independent sets of size i, where every vertex v in the independent set (Algorithm 2, Line 20) has the same ${\mathcal{K}}_v$ (Algorithm 2, Line 19). After removing these vertices, the rest of the vertices in ${\mathcal{G}}_i$ are left for the second step.
• Step II: A candidate pool of vertices ${\mathcal{W}}_1\subseteq {\mathcal{G}}_i$ is created. This set contains vertices v such that $|{\mathcal{K}}_v|$ being close to the smallest available $|{\mathcal{K}}_v|$’s. We randomly pick a vertex v from ${\mathcal{W}}_1$ (Algorithm 2, Line 31). The design parameter a determines how close is the picked $|{\mathcal{K}}_v|$ to the smallest available ones. We gradually form an independent set of size i with v included as follows: Form another set ${\mathcal{W}}_2$ (Algorithm 2, Line 34), excluding v, whose vertices have $|{\mathcal{K}}_{v^{\prime }}|$ that is bigger but closer to that of v determined by b, sample repeatedly with replacement from it to grow the independent set. If an independent set of size at least i cannot be formed, we drop the vertex v to the lower layer ${\mathcal{G}}_{i-1}$, and take it into account in the next layer iteration. Otherwise, we assign a color to the independent set. ${\mathcal{W}}_1$ is repeatedly formed and random sampling from ${\mathcal{W}}_1$ repeated till every vertex in ${\mathcal{G}}_i$ is dropped or colored.

Algorithm 2 HgLC

1: $\mathcal{C}=\varnothing$; 2: $\mathbf{c}=\varnothing$;  3: choose $a\in [0,1]$  4: choose $b\in [0,1]$  5: for all $i=n,n-1,\cdots ,2,1$ do  6:  for all $v\in {\mathcal{G}}_i$ and $|{\mathcal{K}}_v|=i$ do  7:   $\mathcal{I}=\left\{v\right\}$;  8:   Let ${\mathcal{V}}^{\prime }=\mathcal{V}\backslash \left\{v\right\}$;  9:   if { $|{\mathcal{T}}_D\left(v\right)|>t$ } then 10:    for all $v^{\prime }\in {\mathcal{V}}^{\prime }$ with $\rho \left(v^{\prime }\right)=\rho \left(v\right)$ do 11:     $\mathcal{I}=\mathcal{I}\cup v^{\prime }$; 12:    end for 13:    Color all the vertices in $\mathcal{I}$ by $c\notin \mathcal{C}$; 14:    Let $\mathbf{c}\left[\mathcal{I}\right]=c$; 15:    for all $i=n,n-1,\cdots ,2,1$ do 16:     ${\mathcal{G}}_i={\mathcal{G}}_i\backslash \mathcal{I}$; 17:    end for 18:   else 19:    for all $v^{\prime }\in {\mathcal{G}}_i\backslash \mathcal{I}$ with ${\mathcal{K}}_{v^{\prime }}\equiv {\mathcal{K}}_v$ do 20:     if {There is no edge between $v^{\prime }$ and $\mathcal{I}$} then 21:      $\mathcal{I}=\mathcal{I}\cup v^{\prime }$; 22:     end if 23:    end for 24:    if $\left|\mathcal{I}\right|=i$ then 25:     Color all the vertices in $\mathcal{I}$ by $c\notin \mathcal{C}$; 26:     $\mathbf{c}\left[\mathcal{I}\right]=c$, $\mathcal{C}=\mathcal{C}\cup c$; 27:     ${\mathcal{G}}_i={\mathcal{G}}_i\backslash \mathcal{I}$; 28:    end if 29:   end if 30:  end for 31:  for all $v\in {\mathcal{G}}_i$ with v randomly picked from ${\mathcal{W}}_1\subset {\mathcal{G}}_i$ do 32:   $\mathcal{I}=\left\{v\right\}$; 33:   ${\mathcal{Q}}_i={\mathcal{G}}_i\backslash \mathcal{I}$; 34:   for all $v^{\prime }\in {\mathcal{Q}}_i$ with $v^{\prime }$ randomly picked from ${\mathcal{W}}_2\subset {\mathcal{Q}}_i$. do 35:    if {${\mathcal{K}}_{v^{\prime }}\supset {\mathcal{K}}_v$ } ∩ {No edge between $v^{\prime }$ and $\mathcal{I}$} then 36:     $\mathcal{I}=\mathcal{I}\cup v^{\prime }$; 37:     ${\mathcal{Q}}_i={\mathcal{Q}}_i\backslash \left\{v^{\prime }\right\}$; 38:    else 39:     ${\mathcal{Q}}_i={\mathcal{Q}}_i\backslash \left\{v^{\prime }\right\}$; 40:    end if 41:   end for 42:   if $\left|\mathcal{I}\right|\ge i$ then 43:    Color all the vertices in $\mathcal{I}$ by $c\notin \mathcal{C}$; 44:    $\mathbf{c}\left[\mathcal{I}\right]=c$, $\mathcal{C}=\mathcal{C}\cup c$; 45:    ${\mathcal{G}}_i={\mathcal{G}}_i\backslash \mathcal{I}$; 46:   else 47:    ${\mathcal{G}}_i={\mathcal{G}}_i\backslash \left\{v\right\}$, ${\mathcal{G}}_{i-1}={\mathcal{G}}_{i-1}\cup \left\{v\right\}$; 48:   end if 49:  end for 50: end for 51: $\mathbf{c}=$ LocalSearch(${\mathcal{H}}_{\mathbf{C},\mathbf{W}},\mathbf{c},\mathcal{C}$); 52: return the local coloring number ${max}_{v\in \mathcal{V}}\left|\mathbf{c}\left({\mathcal{N}}^{+}\left(v\right)\right)\right|$ and the corresponding color assignment $\mathbf{c}\left({\mathcal{N}}^{+}\left(v\right)\right)$ for each v;

Remark 2.

Please note that RAP-GLC goes through the same Step I as RAP-HgLC, and then simply assigns a different color to each remaining uncolored vertex. On the other hand, Step II in RAP-HgLC tries to find further independent sets among the remaining uncolored vertices. It is this extra step that guarantees the performance of RAP-HgLC to be no worse than that of RAP-GLC.

The RAP-HgLC algorithm, when operating on the i-th layer, always colors at least i vertices with the same color. Please note that if there are remaining vertices when reaching layer 1, all such vertices will be colored, each with a different color.

To further reduce the required number of colors, we use a function called LocalSearch (Algorithm 2, Line 51), which is described in Algorithm 3. It works in an iterative fashion by replacing the current solution with a better one if there exists. It terminates when no better solutions can be found. In particular, the local search algorithm has the purpose of checking the redundancy of each color $c\in \mathcal{C}$, to eventually decrease the current objective function value $\left|\mathcal{C}\right|$. In more detail, the local search computes, iteratively for each color $c\in \mathcal{C}$, the set ${\mathcal{J}}_c$ of all vertices colored with color c, and performs the following steps:

• For each vertex $i\in {\mathcal{J}}_c$, if there is a color $c^{\prime }\in \mathcal{C}$, $c^{\prime }\ne c$ that is not assigned to any adjacent vertex $j\in Adj\left(i\right)$, then assign vertex i with color $c^{\prime }$;
• Color c is removed from the set $\mathcal{C}$ if and only if in the previous step it has been possible to replace c with some color $c^{\prime }\ne c$ for all vertices in ${\mathcal{J}}_c$.

Finally, in Algorithm 2, Line 52, we compute the local coloring number.

Algorithm 3 LocalSearch(${\mathcal{H}}_{\mathbf{C},\mathbf{W}},\mathbf{c},\mathcal{C}$)

1: for all $c\in \mathcal{C}$ do  2:  Let ${\mathcal{J}}_c$ be the set of vertices whose color is c;  3:  Let $\mathcal{B}=\varnothing$;  4:  Let $\widehat{\mathbf{c}}=\mathbf{c}$;  5:  for all $i\in {\mathcal{J}}_c$ do  6:   $\mathcal{A}=\varnothing$;  7:   for all $j\in \mathcal{N}\left(i\right)$ do  8:    $\mathcal{A}=\mathcal{A}\cup \mathbf{c}\left[j\right]$;  9:    if $\mathcal{C}\backslash \mathcal{A}\ne \varnothing$ then 10:     $c^{\prime }$ is chosen uniformly at random from $\mathcal{C}\backslash \mathcal{A}$; 11:     $\widehat{\mathbf{c}}\left[i\right]=c^{\prime }$; 12:     $\mathcal{B}=\mathcal{B}\cup \left\{i\right\}$; 13:    end if 14:   end for 15:   if $\left|\mathcal{B}\right|=|{\mathcal{J}}_c|$ then 16:    $\mathbf{c}=\widehat{\mathbf{c}}$; 17:    $\mathcal{C}=\mathcal{C}\backslash c$; 18:   end if 19:  end for 20: end for 21: return $\mathbf{c}$;

To illustrate the RAP-HgLC algorithm, we present the following example.

Example 3.

Consider a shared link network with $n=3$ users: $\mathcal{U}=\{1,2,3\}$, and $m=3$ files: $\mathcal{F}=\{\mathrm{A},\mathrm{B},\mathrm{C}\}$. Each file is partitioned into 4 packets. For example, $\mathrm{A}=\{{\mathrm{A}}_1,{\mathrm{A}}_2,{\mathrm{A}}_3,{\mathrm{A}}_4\}$. For the caching part, let user 1 cache $\{{\mathrm{A}}_1,{\mathrm{B}}_1,{\mathrm{B}}_2,{\mathrm{B}}_3,{\mathrm{B}}_4,{\mathrm{C}}_2\}$, user 2 cache $\{{\mathrm{A}}_1,{\mathrm{A}}_2,{\mathrm{A}}_3,{\mathrm{A}}_4,{\mathrm{B}}_1,{\mathrm{C}}_2,{\mathrm{C}}_3\}$, user 3 cache $\{{\mathrm{A}}_1,{\mathrm{A}}_2,{\mathrm{A}}_3,{\mathrm{C}}_1,{\mathrm{B}}_2,{\mathrm{B}}_3\}$. Then, let users $\{1,2,3\}$ request files $\{\mathrm{A},\mathrm{B},\mathrm{C}\}$ respectively. Equivalently, user 1 requests ${\mathrm{A}}_2,{\mathrm{A}}_3,{\mathrm{A}}_4$; user 2 requests ${\mathrm{B}}_2,{\mathrm{B}}_3,{\mathrm{B}}_4$; user 3 requests ${\mathrm{C}}_2,{\mathrm{C}}_3,{\mathrm{C}}_4$. Then, we have ${\mathcal{K}}_{{\mathrm{A}}_2}=\{1,23\}$; ${\mathcal{K}}_{{\mathrm{A}}_3}=\{1,23\}$; ${\mathcal{K}}_{{\mathrm{A}}_4}=\{1,2\}$; ${\mathcal{K}}_{{\mathrm{B}}_2}=\{2,13\}$; ${\mathcal{K}}_{{\mathrm{B}}_2}=\{2,13\}$; ${\mathcal{K}}_{{\mathrm{B}}_4}=\{2,1\}$; ${\mathcal{K}}_{{\mathrm{C}}_2}=\{3,12\}$; ${\mathcal{K}}_{{\mathrm{C}}_3}=\{3,2\}$; ${\mathcal{K}}_{{\mathrm{C}}_4}=\left\{3\right\}$ (here ${\mathrm{C}}_4$ is requested by user 3 and not cached anywhere).

The RAP-HgLC algorithm works as follows. For $i=n=3$, ${\mathcal{G}}_3=\{{\mathrm{A}}_2,{\mathrm{A}}_3,{\mathrm{B}}_2,{\mathrm{B}}_3,{\mathrm{C}}_2\}$, let $v={\mathrm{A}}_2$, then it can be found that ${\mathrm{B}}_2$ and ${\mathrm{C}}_2$ would be in $\mathcal{I}$, hence $\mathcal{I}=\{{\mathrm{A}}_2,{\mathrm{B}}_2,{\mathrm{C}}_2\}$. Now since $\left|\mathcal{I}\right|=n=3$, we color ${\mathrm{A}}_2,{\mathrm{B}}_2,{\mathrm{C}}_2$ by black (see Figure 4). Then ${\mathcal{G}}_i={\mathcal{G}}_i\backslash \mathcal{I}=\{{\mathrm{A}}_3,{\mathrm{B}}_3\}$. In the following loop, since we cannot find a set $\mathcal{I}$ with $\left|\mathcal{I}\right|=n=3$, we move to Line 19. Then since we cannot find a $\mathcal{I}$ with $\left|\mathcal{I}\right|\ge n=3$, then we do ${\mathcal{G}}_2={\mathcal{G}}_2\cup \left\{{\mathrm{A}}_3\right\}$, and then ${\mathcal{G}}_2={\mathcal{G}}_2\cup \left\{{\mathrm{B}}_3\right\}$. Therefore, we obtain ${\mathcal{G}}_2=\{{\mathrm{A}}_3,{\mathrm{A}}_4,{\mathrm{B}}_3,{\mathrm{B}}_4,{\mathrm{C}}_3\}$. Now we go to Line 5 (start next loop). For $i=n-1=2$, in this loop, we first pick $v={\mathrm{A}}_4$, then we can find $\mathcal{I}=\{{\mathrm{A}}_4,{\mathrm{B}}_4\}$. We color $\{{\mathrm{A}}_4,{\mathrm{B}}_4\}$ by blue (see Figure 4). Now ${\mathcal{G}}_2={\mathcal{G}}_2\backslash \{{\mathrm{A}}_4,{\mathrm{B}}_4\}=\{{\mathrm{A}}_3,{\mathrm{B}}_3,{\mathrm{C}}_3\}$. Then in Line 19, we find the vertex with smallest length of ${\mathcal{K}}_v$ (let $a=0$), which is ${\mathrm{C}}_3$ with ${\mathcal{K}}_{{\mathrm{C}}_3}=\{3,2\}$, then we have $\mathcal{I}=\left\{{\mathrm{C}}_3\right\}$ and ${\mathcal{Q}}_2=\{{\mathrm{A}}_3,{\mathrm{B}}_3\}$, then in the next loop, we can find $\mathcal{I}=\{{\mathrm{C}}_3,{\mathrm{B}}_3\}$. We color $\mathcal{I}=\{{\mathrm{C}}_3,{\mathrm{B}}_3\}$ by red (see Figure 4). Now ${\mathcal{G}}_2={\mathcal{G}}_2\backslash \{{\mathrm{C}}_3,{\mathrm{B}}_3\}=\left\{{\mathrm{A}}_3\right\}$. Since there is no $\mathcal{I}$ with $\left|\mathcal{I}\right|\ge 2$, then we do ${\mathcal{G}}_1={\mathcal{G}}_1\cup \left\{{\mathrm{A}}_3\right\}=\{{\mathrm{C}}_4,{\mathrm{A}}_3\}$. Then we go to next loop $i=n-2=1$. Then we can see that $\mathcal{I}=\left\{{\mathrm{C}}_4\right\}$, and we color $\left\{{\mathrm{C}}_4\right\}$ by purple (see Figure 4). Then ${\mathcal{G}}_1={\mathcal{G}}_1\backslash \left\{{\mathrm{C}}_4\right\}=\left\{{\mathrm{A}}_3\right\}$. Hence, we can find $\mathcal{I}=\left\{{\mathrm{A}}_3\right\}$ and we color $\left\{{\mathrm{A}}_3\right\}$ by brown.

According to Figure 4, the total number of required colors is 5, while the maximum number of colors required locally by each user is 4. For the naive multicasting, since it only allows the vertices represented the same packet to be colored by the same color, the total number of required colors is 9. The corresponding rate is given by $9/4$. Hence, the final rate achieved by RAP-HgLC with local coloring is no more than $min\{4/4,9/4\}=1$. For the interested reader, it can be verified that if the GCC algorithm, designed for $B\to \infty$, as proposed in [10], is used, the corresponding number of required colors is 6.

The complexity of RAP-HgLC can be computed as follows. For the hierarchical coloring procedure (Line 5–50 in Algorithm 2), the complexity is $O\left(n\right|\mathcal{V}{|}^2)$, and the complexity of local search procedure is $O\left(\right|\mathcal{E}\left|\right)$. Therefore, the running time complexity of RAP-HgLC is given by ${O\left(n\right|\mathcal{V}|}^2+\left|\mathcal{E}\right|)={O\left(n\right|\mathcal{V}|}^2)$. Since $\left|\mathcal{V}\right|\le nB$, the running time complexity of RAP-HgLC is $O\left(n^3{B}^2\right)$.

4.2.2. RAP-HgLC Performance Analysis

For the general heterogeneous network setting, tight upper bounds on the asymptotic ($B\to \infty$) average achievable rate of RAP-HgLC are quite complex to derive, even though a simple (but not necessarily tight) upper bound on the asymptotic performance can be obtained considering the asymptotic average rate of RAP-GLC (see Remark 2).

Regarding the finite-length regime, in [21] we derived a tight upper bound on the performance of RAP-HgLC for the simpler case of homogenous networks under worst-case demands. Specifically, the bound in [21], requires B to be $\tilde{O}\left({\left(\frac{m}{M}\right)}^{g+2}\right)$ (where $\tilde{O}$ hides some poly log terms) to achieve a worst-case rate of at most $\frac{n}{g}$. This approximately matches a lower bound of $\tilde{O}\left({\left(\frac{m}{M}\right)}^g\right)$ derived in the same work for any coloring algorithm, showing, for the simpler homogenous network setting, the optimality of RAP-HgLC among all graph-coloring-based algorithms.

For the more complex setting where demands arise from popularity distributions and every user requests multiple files, the finite-length performance of RAP-HgLC is investigated in Section 6 via numerical analysis, where we show how the RAP-HgLC is able to recover most of the multiplicative caching gain even with very moderate packetization order.

5. Tradeoff between Number of Requests and Code Length

As mentioned earlier, in the simpler homogenous scenario, the authors in [21] showed that under worst-case demands, to achieve a gain g over conventional naive multicasting, it is necessary for B to grow exponentially with g. Intuitively, this is because a sufficiently large B is needed to create coded multicast transmissions that are useful for multiple users. However, when each user makes multiple requests, the number of requests $L_u=L$ can play a similar role to that of B, such that the requirement for B, and hence the resulting computational complexity can be reduced. For ease of analysis, in this section, we assume that all users place the same number of requests ($L_u=L$).

In the following, under either worst-case or uniform demands, we show the sufficient conditions on B and L that guarantee achieving a gain $g=\frac{Mn}{m}$. From this result, we can obtain the regime where B and L are interchangeable (L plays an equivalent role to B). Note that it can be shown that the number of file transmissions under both worst-case and uniform demands have the same order.

We consider two cases for the range of B: the case of $B=1$, and the case of $B=\omega \left(\frac{m}{M}\right)$. The regime where $1<B=O\left(\frac{m}{M}\right)$ is out of the scope of this paper.

When $B=1$, the cache placement algorithm becomes scalar uniform cache placement (SUP), in which each user caches M entire files chosen uniformly at random. For simplicity, we let M be a positive integer. Then, as shown in [14], letting $L\to \infty$ as a function of $n,m,M$, we obtain the following theorem.

Theorem 2.

When $B=1$ and $M=\omega \left(1\right)$, for the shared link caching network with n users, library size m, storage capacity M, and L distinct per-user requests ($nL\le m$), if (i) $\frac{M}{m}\le \frac{1}{2}$ and

$$\begin{array}{cc}\hfill L& =\omega \left(max\left\{\frac{nM}{m}{\left(\frac{m}{M}\right)}^n\frac{1}{\left(1-\frac{M}{m}\right)},\left(\frac{{\left(nM\right)}^{\frac{1}{2(1-\epsilon )}}}{\left(\frac{m}{M}-1\right)\left(1-{\left(1-\frac{M}{m}\right)}^n\right)}\right)\right\}\right),\hfill \end{array}$$

(20)

or (ii) $\frac{M}{m}\ge \frac{e}{1+e}$ and

$$\begin{array}{cc}\hfill L& =\omega \left(max\left\{{\left(\frac{m}{m-M}\right)}^n,\left(\frac{{\left(nM\right)}^{\frac{1}{2(1-\epsilon )}}}{\left(\frac{m}{M}-1\right)\left(1-{\left(1-\frac{M}{m}\right)}^n\right)}\right)\right\}\right),\hfill \end{array}$$

(21)

where ε is an arbitrarily small number,

then, the achievable rate of RAP-GLC is upper bounded by

$$\begin{array}{cc}\hfill & \underset{n,m\to \infty }{lim}\mathbb{P}\left(R^{\mathrm{SUP}-\mathrm{GLC}}\le (1+o\left(1\right))min\left\{L\left(\frac{m}{M}-1\right),Ln,m-M\right\}\right)=1.\hfill \end{array}$$

Proof. 

See Appendix B. □

From Theorem 2, we can see that when L and M are large enough, instead of requiring a large B and packet-level coding, a simpler file-level coding scheme is sufficient to achieve the same order-optimal rate. We remark, however, that the range of the parameter regimes in which this result holds is limited due to the requirement of a large M and L. Next, we focus on another parameter regime, when $B=\omega \left(\frac{m}{M}\right)$, and find the achievable tradeoff between B and L.

Theorem 3.

When $B=\omega \left(\frac{m}{M}\right)$, for the shared link caching network with n users, library size m, storage capacity M, and L distinct per-user requests ($nL\le m$), if (i) $\frac{M}{m}\le \frac{1}{2}$, and

$$\begin{array}{cc}\hfill B& =\omega \left(max\left\{\frac{nM}{Lm}{\left(\frac{m}{M}\right)}^n\frac{1}{\left(1-\frac{M}{m}\right)},\frac{{\left(nM\right)}^{\frac{1}{2(1-\epsilon )}}}{L\left(\frac{m}{M}-1\right)\left(1-{\left(1-\frac{M}{m}\right)}^n\right)}\right\}\right),\hfill \end{array}$$

(22)

or (ii) $\frac{M}{m}\ge \frac{e}{1+e}$, and

$$\begin{array}{cc}\hfill B& =\omega \left(max\left\{\frac{1}{L}{\left(\frac{m}{m-M}\right)}^n,\frac{{\left(nM\right)}^{\frac{1}{2(1-\epsilon )}}}{L\left(\frac{m}{M}-1\right)\left(1-{\left(1-\frac{M}{m}\right)}^n\right)}\right\}\right),\hfill \end{array}$$

(23)

where ε is an arbitrarily small number,

Then, the achievable rate of RAP-GLC is upper bounded by

$$\begin{array}{c}\hfill \underset{n,m\to \infty }{lim}\mathbb{P}\left(R^{\mathrm{SUP}-\mathrm{GLC}}\le (1+o\left(1\right))min\left\{L\left(\frac{m}{M}-1\right),Ln,m-M\right\}\right)=1.\end{array}$$

(24)

Proof. 

See Appendix C. □

If we particularize Theorem 1 to the homogenous network setting under uniform demands, we see that the rate achieved by RAP-GLC is upper bounded by the same expression given in (24). Hence, from Theorem 2, we can see that when L is large enough, instead of requiring a very large B, an intermediate value of $B=\omega \left(\frac{m}{M}\right)$ is sufficient to achieve the same order-optimal rate. In practice, it is important to find the right balance and tradeoff between B and L given the remaining system parameters. In Section 6, we show via simulation that a similar tradeoff holds also for RAP-HgLC.

6. Simulations and Discussions

In this section, we numerically evaluate the performance of the two polynomial-time algorithms described in Section 4, RAP-GLC and RAP-HgLC, in the finite-length regime characterized by the number of packets per file B.

Recall that the caching distribution ${\mathbf{P}}^{*}$ is to be optimized to minimize the number of transmissions. Since the distribution ${\mathbf{P}}^{*}$ resulting from minimizing the right-hand side of (12) may not admit an analytically tractable expression in general, in the following numerical results, we restrict the caching distribution to take the form of a truncated uniform distribution ${\tilde{\mathbf{p}}}_u$, as described in [12]:

$$\begin{array}{c}{\tilde{p}}_{f,u}={\frac{1}{\tilde{m}}}_u,f\le {\tilde{m}}_u\hfill \\ {\tilde{p}}_{f,u}=0,f\ge \tilde{m}+1\hfill \end{array}$$

(25)

where the cut-off index ${\tilde{m}}_u\ge M$ is a function of the system parameters that is optimized to minimize the right-hand side of (12). The intuition behind the form of ${\tilde{\mathbf{p}}}_u$ in (25) is that each user caches the same fraction of (randomly selected) packets from each of the most ${\tilde{m}}_u$ popular files, and does not cache any packet from the remaining $m-{\tilde{m}}_u$ least popular files. We point out that when ${\tilde{m}}_u=M$, this cache placement coincides with the LFU (Least Frequently Used) caching policy. Thus, this cache placement is referred to as Random LFU (RLFU) [12], and the corresponding caching algorithms as RLFU-GLC and RLFU-HgLC. Recall that LFU discards the least frequently requested file upon the arrival of a new file to a full cache of size $M_u$ files. In the long run, this is equivalent to caching the $M_u$ most popular files [69].

In Figure 5 and Figure 6, we plot the average achievable rate, i.e., the average number of transmissions (normalized by the file size) as a function of the cache size for RLFU-GLC and RLFU-HgLC. For comparison, we also simulate the following algorithms:

• LFU, which has been shown to be optimal in single cache networks;
• RLFU-GLC with infinite file packetization ($B\to \infty$), whose performance guarantee is given in Theorem 1, and it is shown to be order optimal.

Regarding the LFU algorithm, the average achievable rate is given by

$$\mathbb{E}\left[R^{\mathrm{LFU}}\right]=\sum _{f={min}_u\left\{M_u\right\}+1}^m\left(1-\prod _{u\in {\mathcal{U}}_{\{M_u<f\}}}{(1-q_{f,u})}^{L_u}\right),$$

(26)

where ${\mathcal{U}}_{\{M_u<f\}}$ denotes the set of users with $M_u<f$.

For simplicity, and to better illustrate the effectiveness of the proposed algorithms, especially under multiple per-user requests, we consider a scenario in which all users request files according to a Zipf demand distribution with parameter $\gamma \in \{0.2,0,4,0.5\}$, and all caches have size M files. Under Zipf demands, file f is requested with probability $\frac{f^{-\gamma }}{{\sum }_{i=1}^m{i}^{-\gamma }}$.

We consider two types of users. In Figure 5a and Figure 6a, users represent end devices requesting only one file each ($L=1$); while in Figure 5b and Figure 6b, they represent helpers/small-cells, each serving 10 end user devices, and consequently collecting $L=10$ requests.

In Figure 5a,b, we fix the total number of users n and the product between L and B ($L\times B=1000$). Figure 5a plots the average rate for a network with $n=40$ users, $\gamma =0.5$, $L=1$, and $B=1000$. It is immediate to observe the impact of finite packetization on the multiplicative caching gain. In fact, as predicted by the theory (see [21]), the significant caching gain (with respect to LFU) quantified by the asymptotic performance of RAP-GLC (GLC with $B=\infty$) is completely lost when using RAP-GLC with finite packetization (GLC with $B=1000$). On the other hand, RAP-HgLC remarkably preserves, at the expense of a slight increase in computational complexity, most of the multiplicative caching gain for the same value of file packetization. For example, in Figure 5a, if M doubles from $M=200$ to $M=400$, then the rate achieved by RAP-HgLC reduces from 15 to $5.7$. Furthermore, RAP-HgLC can achieve a factor of $3.5$ rate reduction from LFU for $M=500$. For the same regime, it is straightforward to verify that neither RAP-GLC nor LFU exhibit this property. Note from Figure 5a that to guarantee a rate of 10, RAP-GLC requires a cache size of $M=500$, while RAP-HgLC can reduce the cache size requirement to $M=250$, a $2\times$ cache size reduction. Furthermore, while LFU can only provide an additive caching gain, additive and multiplicative gains may show indistinguishable when M is comparable to the library size m. Hence, one needs to pick a reasonably small M ($\frac{m}{n}<M\ll m$) to observe the multiplicative caching gain of RAP-HgLC.

Figure 5b shows the average rate for a network with $n=40$ helpers/small-cells, each serving 10 users making requests according to a Zip distribution with $\gamma =0.5$. Hence, the total number of distinct requests per helper is up to $L_u=10,\forall u\in \{1,\dots 20\}$. In this case, we assume $B=100$ (instead of $B=1000$ in Figure 5a). In order to make easier the comparison with Figure 5a, we normalize the achievable rate (number of transmissions) by the file size and the number of requests.

Note from Figure 5a,b that as predicted by Theorem 3, when $L_u$ increases (from $L_u=1$ to $L_u=10$), almost the same multiplicative caching gain can be achieved with a smaller B (from $B=1000$ to $B=100$). In fact, from Figure 5a,b, we see that under RAP-HgLC, the average rate per request for $B=100$ and $L=10$ is almost the same as the average rate per request for $B=1000$ and $L=1$. This confirms the interesting tradeoff between B and L established in Theorem 3.

We can observe a similar behavior in Figure 6a,b. Figure 6a plots the average rate for a network with $n=80$ users, $\gamma =0.4$, $L=1$, and $B=200$. RAP-HgLC is able to preserve most of the multiplicative caching gain for the same values of file packetization. For example, in Figure 6a, if M doubles from $M=200$ to $M=400$, then the rate achieved by RAP-HgLC essentially halves from 20 to 10. Furthermore, RAP-HgLC can achieve a factor of 5 rate reduction from LFU for $M=500$. Note from Figure 6a that to guarantee a rate of 20, RAP-GLC requires a cache size of $M=500$, while RAP-HgLC can reduce the cache size requirement to $M=200$, a $2.5\times$ cache size reduction.

Figure 6b plots the average rate for a network with $n=20$ helpers/small-cells, each serving 10 users making requests according to a Zip distribution with $\gamma =0.2$. Hence, the total number of distinct requests per helper is up to $L_u=10,\forall u\in \{1,\dots 20\}$. In this case, we assume $B=100$. Differently from Figure 5b, here we plot the average rate without normalizing it by the number of requests.

Note from Figure 6a,b that, as predicted by Theorem 3, when $L_u$ increases (from $L_u=1$ to $L_u=10$), almost the same multiplicative caching gain can be achieved with a smaller B (from $B=200$ to $B=100$). In fact, from Figure 6a,b, we see that under RAP-HgLC, the average rate per request for $B=100$ and $L=10$ is almost the same as the average rate per request for $B=200$ and $L=1$. For example, for $M=200$, $B=100$, and $L=10$, the per request average rate achieved by RAP-HgLC is $0.3$, while for $M=200$ and $B=200$, is $0.25$. This again confirms the tradeoff between B and L stated in Theorem 3.

Furthermore, from Figure 5 and Figure 6, we notice that increasing the Zip parameter reduces the gains with respect to LFU. This is explained by the fact that when aggregating multiple requests, there is a higher number of overlapping requests, which increases the opportunities for naive multicasting (as clearly characterized in [13]). Note, however, that RAP-HgLC can remarkably keep similar gains with respect to LFU in this multiple-request setting, and approach the asymptotic performance even with just $B=100$ packets per file, confirming the effectiveness of the local graph coloring and extra processing procedures in RAP-HgLC.

7. Conclusions

Coded multicasting has been shown to be a promising approach to significantly reduce the traffic load in wireless caching networks. However, most existing schemes require the number of packets per file to grow exponentially with the number of users. To address this challenge, in this paper we focused on a heterogeneous shared link caching network model and designed novel coded multicast algorithms based on local graph coloring that exhibit polynomial-time complexity in all the system parameters, and preserve the asymptotically proven multiplicative caching gain for finite file packetization. We also demonstrated that the number of packets per file can be traded-off with the number of requests collected by each cache, such that the same multiplicative caching gain can be preserved. Simulation results confirm the superiority of the proposed schemes and illustrate the tradeoff between request aggregation and computational complexity (driven by the packetization order), shedding light into the practical achievability of the promising multiplicative caching gain in next generation wireless networks.