Adaptive Privacy-Preserving Coded Computing with Hierarchical Task Partitioning

Abstract

Coded computing is recognized as a promising solution to address the privacy leakage problem and the straggling effect in distributed computing. This technique leverages coding theory to recover computation tasks using results from a subset of workers. In this paper, we propose the adaptive privacy-preserving coded computing (APCC) strategy, designed to be applicable to various types of computation tasks, including polynomial and non-polynomial functions, and to adaptively provide accurate or approximated results. We prove the optimality of APCC in terms of encoding rate, defined as the ratio between the computation loads of tasks before and after encoding, based on the optimal recovery threshold of Lagrange Coded Computing. We demonstrate that APCC guarantees information-theoretical data privacy preservation. Mitigation of the straggling effect in APCC is achieved through hierarchical task partitioning and task cancellation, which further reduces computation delays by enabling straggling workers to return partial results of assigned tasks, compared to conventional coded computing strategies. The hierarchical task partitioning problems are formulated as mixed-integer nonlinear programming (MINLP) problems with the objective of minimizing task completion delay. We propose a low-complexity maximum value descent (MVD) algorithm to optimally solve these problems. The simulation results show that APCC can reduce the task completion delay by a range of $20.3\%$ to $47.5\%$ when compared to other state-of-the-art benchmarks.

1. Introduction

Under the vision of “Internet of Everything”, intelligence-enabled applications are essential, leading to a variety of crucial computation tasks, such as the training and inference of complex machine learning models based on extensive datasets [1,2,3]. However, executing these computation-intensive tasks on a single device with limited computation capability and power resources presents significant challenges. To this end, distributed computing emerges as a practical solution, where a central node, referred to as master, manages task division, assignment, and result collection, while multiple distributed computing nodes, called workers, process the assigned partial computation tasks in parallel [4].

Nevertheless, while distributed computing accelerates the computation process by employing multiple workers for parallel processing, the total delay is dominated by the slowest worker, as the master must wait for all workers to complete their assigned tasks [5]. As demonstrated in the experimental results of [6], the delay of the slowest worker can exceed five times that of the others, which significantly prolongs the total delay. Moreover, due to the randomness of delays, identifying slow workers in advance is challenging. To tackle this so-called straggling effect, coded computing has emerged as a promising solution [6,7,8,9,10,11,12]. As Figure 1 shows, this approach combines coding theory with distributed computing and reduces delays by introducing structured computational redundancies. Through the incorporation of redundancy during the encoding process, computation tasks can be completed using results from a subset of workers, thereby reducing total delays.

In coded computing, workers are tasked with processing input data and returning results, but the involved tasks may contain sensitive information, such as patient medical data, customer personal information, and proprietary company data [13,14]. Consequently, it is essential to maintain the data privacy against colluding workers, those who return correct results but may communicate with one another to share input data from the master, so as to infer some private information of the master. Recent research studies have aimed to develop coded computing strategies that address not only the straggling effect but also privacy concerns, such as combining additional random data insertion with prevalent polynomial coded computing methods [15,16,17,18,19,20,21,22,23,24,25]. This approach enhances the robustness of the system against straggling workers while also improving privacy and security by obscuring the original data.

In the majority of existing studies, matrix multiplication is treated as the primary application of coded computing, and its performance has been extensively validated. However, real-world computation tasks are often more diverse than mere matrix multiplication. For instance, in a linear regression task, the iterative process of solving weights involves calculating previous weights multiplied by the quadratic power of the input data. This implies that coded computing schemes for matrix multiplication must be executed twice in each step, and computation becomes considerably more complex when considering other tasks, such as inference tasks of neural networks.

In terms of extending the applicability of coded computing, one state-of-the-art approach is Lagrange Coded Computing (LCC) [15]. LCC employs Lagrange polynomial interpolation to transform input data before and after encoding into interpolation points on the computation function. This allows the recovery of desired results through the reconstruction of the interpolation function. LCC is compatible with various computation tasks, ranging from matrix multiplication to polynomial functions, and offers an optimal recovery threshold concerning the degree of polynomial functions. In [21,25,26], the problem of using matrix data as input and polynomial functions as computation tasks is also explored.

However, LCC still suffers from several shortcomings [27]. First, its recovery threshold is proportional to degrees of polynomial functions, which can be prohibitively large for complex tasks and thereby make it difficult to achieve successful recovery. Second, Lagrange polynomial interpolation can be ill-conditioned, making it challenging to ensure numerical stability, unless one embeds the computation to a finite field. In [27], Berrut’s Approximated Coded Computing (BACC) is proposed to address these shortcomings and further expand the scope of computation tasks to arbitrary functions. However, BACC only yields approximated computing results and does not guarantee privacy preservation. Other related works [28,29,30,31,32] also focus on approximated results while attempting to maintain the numerical stability of coded computing. As far as we know, there is still a lack of a versatile coded computing strategy suitable for various computational tasks. This strategy should be capable of achieving privacy preservation while providing accurate or approximated results based on specific demands.

On the other hand, opportunities exist to enhance mitigation of the straggling effect and further reduce delays. This is because prior studies commonly discard results from straggling workers, leading to the inefficient utilization of computational resources. In [11], a hierarchical task partitioning structure is proposed, where divided tasks are further partitioned into multiple layers, and workers process their assigned tasks in the order of layer indices. Consequently, straggling workers can return results from lower layers instead of none, while fast workers can reach higher layers and return more results. Similar performance improvements are achieved through multi-message communications (MMC) [33,34,35], where workers are permitted to return partial results of assigned tasks in each time slot, enabling straggling workers to contribute to the system.

Essentially, three ways exist to alleviate the straggling effect, given the total number of workers. First, minimize the recovery threshold of coded computing schemes, as a smaller recovery threshold implies fewer workers to wait for [9,10,15,16,36,37,38,39]. As a result, the master can recover desired computing results even with more straggling workers. Second, the computation loads for each worker should be carefully designed to allow them to complete varying amounts of computation based on their capabilities, which is formulated as optimization problems in [4,40,41,42,43]. This approach narrows the gap between the delays of fast and slow workers. Third, workers should be capable of returning partial results of assigned tasks, rather than the scenario where fast workers complete all assigned tasks, leaving straggling workers to contribute virtually nothing. The third point aligns with the idea of a hierarchical task partitioning structure and MMC.

In this work, we consider a distributed system with one master and multiple workers, and propose an adaptive privacy-preserving coded computing (APCC) strategy. The strategy primarily focuses on the applicability for diverse computation tasks, the privacy preservation of input data, and the mitigation of the straggling effect. Moreover, based on the hierarchical task partitioning structure in APCC, we propose an operation called cancellation to prevent slower workers from processing completed tasks, reducing resource waste and improving delay performance. Specifically, the main contributions are summarized as follows:

• We propose the APCC framework, which effectively mitigates the straggling effect and fully preserves data privacy. APCC is applicable to various computation tasks, including polynomial and non-polynomial functions, and can adaptively provide accurate results or approximated results with controllable error.
• We rigorously prove the information-theoretical privacy preservation of the input data in APCC, as well as the optimality of APCC in terms of the encoding rate based on the optimal recovery threshold of LCC. The encoding rate is defined as the ratio between the computation loads of tasks before and after encoding, serving as an indicator of the performance of coded computing schemes in mitigating the straggling effect.
• Considering the randomness of task completion delay, we formulate hierarchical task partitioning problems in APCC, with or without cancellation, as mixed-integer nonlinear programming (MINLP) problems with the objective of minimizing task completion delay. We propose a maximum value descent (MVD) algorithm to optimally solve the problems with linear complexity.
• Extensive simulations demonstrate improvements in delay performance offered by APCC when compared to other state-of-the-art coded computing benchmarks. Notably, APCC achieves a reduction in task completion delay ranging from $20.3\%$ to $47.5\%$ compared to LCC [15] and BACC [27]. Simulations also explore the trade-off between task completion delay and the level of privacy preservation.

The remainder of the paper is structured as follows. Section 2 presents the system model. In Section 3, we propose the adaptive privacy-preserving coded computing strategy, namely APCC. In Section 4, the performance of APCC is further analyzed in terms of encoding rate, privacy preservation, approximation error, numerical stability, communication costs, and encoding and decoding complexity. In Section 5, we proposed the MVD algorithm to address the hierarchical task partitioning optimization problem with or without cancellation. The simulation results are provided in Section 6, and conclusions are drawn in Section 7.

2. System Model

As shown in Figure 2, we consider the distributed computing system consisting of one master and N workers. The goal is to complete a computation task on the master with the help of N workers. The task is represented by a function f, operating over an equally pre-divided input dataset $\mathit{D}=\{{\mathit{D}}_k\in {\mathbb{R}}^{p\times q}|k\in [0:K-1]\}$. The master aims to evaluate the results ${\left\{f\left({\mathit{D}}_k\right)\right\}}_{k=0}^{K-1}$, whose dimensions are decided by the task function f. To achieve this, we employ the proposed APCC strategy. Note that we consider the computation of ${\left\{f\left({\mathit{D}}_k\right)\right\}}_{k=0}^{K-1}$ as the entire task and the computation of $f\left({\mathit{D}}_k\right)$ as a subtask.

In APCC, the K equally pre-divided input data ${\left\{{\mathit{D}}_k\right\}}_{k=0}^{K-1}$ are not directly encoded like conventional coded computing strategies. Instead, they are firstly partitioned into r sets. Subsequently, the input data in each set are encoded into N parts, which are then assigned to N workers for parallel computation. This hierarchical task partitioning structure enables workers to return partial results of assigned subtasks, further mitigating the straggling effect and reducing delays. After the task assignment, the master leverages the results obtained from a subset of workers in each set and employs interpolation methods to reconstruct the original function f, thereby achieving the recovery of ${\left\{f\left({\mathit{D}}_k\right)\right\}}_{k=0}^{K-1}$. A comprehensive description of the APCC strategy is presented in Section 3.

Taking into account the unreliable channels and uncertain computation capabilities of workers, some of them may fail to return results to the master in time. These straggling workers are referred to as stragglers. Additionally, we assume that workers are honest but curious. This means they will send back the correct computation results, but there could be up to L$(L<N)$ colluding workers who can communicate with each other and attempt to learn information about the input data ${\left\{{\mathit{D}}_k\right\}}_{k=0}^{K-1}$. These workers are called colluders.

3. Adaptive Privacy-Preserving Coded Computing

In this section, we propose the adaptive privacy-preserving coded computing (APCC) strategy, which is suitable for diverse computation tasks including polynomial functions and non-polynomial functions, and can adaptively provide accurate results or approximated results. We begin with a general description to explain how APCC works and then provide an illustrative example for accurate results case without loss of generality. Lastly, we introduce the hierarchical task partitioning structure of APCC, and the cancellation of completed subtasks based on this hierarchical structure.

3.1. General Description

In this subsection, we provide a general description of the proposed APCC strategy. As shown in Figure 2, the inputs of the function f are first equally pre-divided into K parts ${\left\{{\mathit{D}}_k\right\}}_{k=0}^{K-1}$, and K corresponding subtasks ${\left\{f\left({\mathit{D}}_k\right)\right\}}_{k=0}^{K-1}$ are formed. The APCC strategy then follows three steps: (1) Encoding; (2) Assignment; (3) Decoding, and obtains accurate or approximated computing results of ${\left\{f\left({\mathit{D}}_k\right)\right\}}_{k=0}^{K-1}$.

3.1.1. Encoding

In the initialization step, the K subtasks are further partitioned into r sets, with set i ($i\in [0:r-1]$) containing $K_i$ subtasks $\{f{\left({\mathit{D}}_{i,j}\right\}}_{j=0}^{K_i-1}$, where ${\mathit{D}}_{i,j}\in \left\{{\mathit{D}}_k\right|k\in [0:K-1]\}$. Consequently, the desired results of the master are

$$\begin{array}{c}\hfill {\left\{f\left({\mathit{D}}_k\right)\right\}}_{k=0}^{K-1}=\left\{{\left\{f\left({\mathit{D}}_{i,j}\right)\right\}}_{j=0}^{K_i-1}|i\in [0:r-1]\right\},\end{array}$$

(1)

where $K_i$ should satisfy ${\sum }_{i=0}^{r-1}{K}_i=K$. The specific values of ${\left\{K_i\right\}}_{i=0}^{r-1}$ will be formulated as optimization problems in Section 5. We refer to the partition of these sets as hierarchical task partitioning.

Inspired by Barycentric polynomial interpolation [27,44], the input data ${\left\{{\mathit{D}}_{i,j}\right\}}_{j=0}^{K_i-1}$ for set i is linearly encoded through function $g_i\left(x\right)$ as:

$$\begin{array}{cc}\hfill g_i\left(x\right)& =\sum _{j=0}^{K_i-1}{\displaystyle \frac{w_{i,j}{\prod }_{k=0,k\ne j}^{K_i+L-1}(x-{\alpha }_{i,k})}{{\sum }_{k=0}^{K_i+L-1}{w}_{i,k}{\prod }_{l=0,l\ne k}^{K_i+L-1}(x-{\alpha }_{i,l})}}{\mathit{D}}_{i,j}\hfill \\ \hfill & +\sum _{j=K_i}^{K_i+L-1}{\displaystyle \frac{w_{i,j}{\prod }_{k=0,k\ne j}^{K_i+L-1}(x-{\alpha }_{i,k})}{{\sum }_{k=0}^{K_i+L-1}{w}_{i,k}{\prod }_{l=0,l\ne k}^{K_i+L-1}(x-{\alpha }_{i,l})}}{\mathit{Z}}_{i,j},\hfill \end{array}$$

(2)

where $\{{\mathit{Z}}_{i,j}\in \mathbb{V}|j\in [K_i:K_i+L-1]\}$ are L random matrices added to preserve the privacy, each element in ${\mathit{Z}}_{i,j}$ follows a uniform distribution, and $x\in \mathbb{R}$ is the encoding parameter. ${\left\{{\alpha }_{i,j}\right\}}_{j=0}^{K_i+L-1}$ are distinct values selected as Chebyshev points of the first kind:

$$\begin{array}{c}\hfill {\alpha }_{i,j}=cos{\displaystyle \frac{(2j+1)\pi }{2(K_i+L)}},j\in [0:K_i+L-1].\end{array}$$

(3)

$w_{i,j}$ is a constant related to ${\alpha }_{i,j}$ and calculated as:

$$\begin{array}{c}\hfill w_{i,j}={\displaystyle \frac{1}{{\prod }_{k=0,k\ne j}^{K_i+L-1}({\alpha }_{i,j}-{\alpha }_{i,k})}},j\in [0:K_i+L-1].\end{array}$$

(4)

Note that the form of $g_i\left(x\right)$ is a Barycentric polynomial [27,44], which can avoid overflows and underflows in floating-point arithmetic and requires a lower computation complexity compared to its similar form of Lagrange polynomial in LCC [15]. Furthermore, Equation (2) guarantees that

$$\begin{array}{c}\hfill g_i\left({\alpha }_{i,j}\right)={\mathit{D}}_{i,j},j\in [0:K_i-1].\end{array}$$

(5)

because the coefficient term before ${\mathit{D}}_{i,j}$ and ${\mathit{Z}}_{i,j}$

$$\begin{array}{c}\hfill {\displaystyle \frac{w_{i,j}{\prod }_{k=0,k\ne j}^{K_i+L-1}(x-{\alpha }_{i,k})}{{\sum }_{k=0}^{K_i+L-1}{w}_{i,k}{\prod }_{l=0,l\ne k}^{K_i+L-1}(x-{\alpha }_{i,l})}}=\left\{\begin{array}{cc}1,\hfill & \mathrm{if}x={\alpha }_{i,j},\hfill \\ 0,\hfill & \mathrm{if}x={\alpha }_{i,k},k\ne j.\hfill \end{array}\right.\end{array}$$

(6)

The encoded input data ${\left\{{\tilde{\mathit{D}}}_{i,n}\right\}}_{n=0}^{N-1}$ are obtained as:

$$\begin{array}{c}\hfill {\tilde{\mathit{D}}}_{i,n}=g_i\left({\beta }_n\right),n\in [0:N-1].\end{array}$$

(7)

${\left\{{\beta }_n\right\}}_{n=0}^{N-1}$ are selected as Chebyshev points of the second kind:

$$\begin{array}{c}\hfill {\beta }_n=cos{\displaystyle \frac{n\pi }{N-1}},n\in [0:N-1].\end{array}$$

(8)

3.1.2. Assignment

For set i, the encoded data ${\tilde{\mathit{D}}}_{i,n}=g_i\left({\beta }_n\right)$ is assigned to worker n. Consequently, as Figure 2 shows, each worker receives r encoded subtasks ${\left\{f\left({\tilde{\mathit{D}}}_{i,n}\right)\right\}}_{i=0}^{r-1}$ and executes them in the order of set indices. Once completed, the result of each encoded subtask $f\left({\tilde{\mathit{D}}}_{i,n}\right)$ is returned to the master. In other words, after the original K subtasks are partitioned into multiple sets, each set is transformed into N encoded subtasks assigned to N workers for processing.

3.1.3. Decoding

For set i, the master decodes using function $r_i\left(x\right)$, which is constructed by interpolation [27,44] as:

$$\begin{array}{c}\hfill r_i\left(x\right)=\sum _{n=0}^{R_i-1}{\displaystyle \frac{\frac{{\tilde{w}}_n}{x-{\tilde{x}}_n}}{{\sum }_{m=0}^{R_i-1}\frac{{\tilde{w}}_m}{x-{\tilde{x}}_m}}}f\left(g_i\left({\tilde{x}}_n\right)\right),\end{array}$$

(9)

where $\left\{f\left(g_i\left({\tilde{x}}_n\right)\right)\right|n\in [0:R_i-1]\}$ are the first $R_i$ received results in ${\left\{f\left({\tilde{\mathit{D}}}_{i,n}\right)\right\}}_{n=0}^{N-1}$ for set i, ${\tilde{x}}_n$ is the corresponding encoding parameter that belongs to $\left\{{\beta }_n\right|n\in [0,N-1]\}$, and the parameter ${\tilde{w}}_n$ is adaptive for different cases, as follows.

Case 1: Accurate results. If f is a polynomial function of degree d, where the degree d of a polynomial function is defined as the maximum order of its monomials, the adaptive parameters ${\tilde{w}}_n$ is determined as:

$$\begin{array}{c}\hfill {\tilde{w}}_n={\displaystyle \frac{1}{{\prod }_{m=0,m\ne n}^{R_i-1}({\tilde{x}}_n-{\tilde{x}}_m)}},n\in [0:R_i-1].\end{array}$$

(10)

In this case, $r_i\left(x\right)$ is a Barycentric polynomial interpolation function [44] for $f\left(g_i\left(x\right)\right)$. The degree of $g_i\left(x\right)$ equals $(K_i+L-1)$, so that $f\left(g_i\left(x\right)\right)$ remains a polynomial function, and its degree satisfies $\mathrm{deg}f\left(g_i\left(x\right)\right)\le d(K_i+L-1)$. Consequently, if the number of received results $R_i$ for set i satisfies:

$$\begin{array}{c}\hfill R_i=d(K_i+L-1)+1,\end{array}$$

(11)

it implies that sufficient interpolation points have been obtained to precisely recover $f\left(g_i\left(x\right)\right)$ through $r_i\left(x\right)$, and the entire computation process is completed with

$$\begin{array}{c}\hfill f\left({\mathit{D}}_{i,j}\right)=f\left(g_i\left({\alpha }_{i,j}\right)\right)=r_i\left({\alpha }_{i,j}\right),\end{array}$$

(12)

for any $i\in [0:r-1],j\in [0:K_i-1]$.

Note that Equation (11) means that the accurate result case of APCC has the same recovery threshold as LCC [15]. Furthermore, similar to LCC, when there is no need for privacy preservation, which means $L=0$, we can also provide an uncoded version of APCC by selecting the value of $\left\{{\beta }_n\right\}$ from $\left\{{\alpha }_{i,j}\right\}$. Thus, the new recovery threshold becomes:

$$\begin{array}{c}\hfill R_i=N-⌊{\displaystyle \frac{N}{K_i}}⌋+1.\end{array}$$

(13)

Case 2: Approximated results. If f is an arbitrary function, the adaptive parameter ${\tilde{w}}_n$ is calculated as:

$$\begin{array}{c}\hfill {\tilde{w}}_n={(-1)}^n,n\in [0:R_i-1].\end{array}$$

(14)

In this case, $r_i\left(x\right)$ is a Berrut’s rational interpolation function for $f\left(g_i\left(x\right)\right)$, as discussed in [27,45]. The computed results $f\left(g_i\left({\tilde{x}}_n\right)\right)$ serve as the interpolation points of $f\left(g_i\left(x\right)\right)$, and they satisfy $r_i\left({\tilde{x}}_n\right)=f\left(g_i\left({\tilde{x}}_n\right)\right)$ due to the property of Berrut’s rational interpolation [45]. Therefore, the master can regard $r_i\left(x\right)$ as an approximation of $f\left(g_i\left(x\right)\right)$, which means that

$$\begin{array}{c}\hfill f\left({\mathit{D}}_{i,j}\right)=f(g_i\left({\alpha }_{i,j}\right)\approx r_i\left({\alpha }_{i,j}\right),\end{array}$$

(15)

for any $i\in [0:r-1],j\in [0:K_i-1]$. In addition, the approximation using $r_i\left(x\right)$ becomes more accurate as $R_i$ increases. Thus, if the master desires more accurate computations, it simply needs to wait for more results.

3.2. An Illustrating Example

In this subsection, we present an illustrative example for the case of accurate results without loss of generality. Specifically, we consider a linear regression problem. The feature data $\mathit{D}\in {\mathbb{R}}^{12000\times 10}$ contains 12,000 data samples with 10 features, and the label vector is denoted by $\mathit{y}\in {\mathbb{R}}^{12000\times 1}$. The objective is to find the weighting vector $\mathit{w}\in {\mathbb{R}}^{10\times 1}$ that minimizes the loss $\left|\right|\mathit{Dw}-{\mathit{y}\left|\right|}^2$. To solve this problem, the gradient descent method updates the weights iteratively along the negative gradient direction as follows:

$$\begin{array}{c}\hfill {\mathit{w}}^{(t+1)}={\mathit{w}}^{\left(t\right)}-{\displaystyle \frac{2\eta }{p^{\prime }}}{\mathit{D}}^T(\mathit{D}{\mathit{w}}^{\left(t\right)}-\mathit{y}),\end{array}$$

(16)

where $\eta$ is the learning rate and t represents the iteration index.

In order to apply the aforementioned update process to a distributed system with one master and $N=10$ workers, for instance, the feature data $\mathit{D}$ is first equally divided into $K=12$ sub-matrices ${({\mathit{D}}_0,{\mathit{D}}_1,\dots ,{\mathit{D}}_{11})}^T,{\mathit{D}}_k\in {\mathbb{R}}^{1000\times 10},k\in [0:11]$. As ${\mathit{w}}^{\left(t\right)}$ is known by the workers and ${\mathit{D}}^T\mathit{y}$ can be precomputed by the master, the computation function (subtask) of the master in each iteration can be expressed as $f\left({\mathit{D}}_k\right)={\mathit{D}}_k^T{\mathit{D}}_k\mathit{w}\in {\mathbb{R}}^{10\times 1},k\in [0:11]$. After obtaining the results of the entire task ${\left\{f\left({\mathit{D}}_k\right)\right\}}_{k=0}^{11}$, the gradient update is computed as ${\mathit{D}}^T\mathit{D}\mathit{w}={\sum }_{k=0}^{11}{\mathit{D}}_k^T{\mathit{D}}_k\mathit{w}$.

We now illustrate how APCC can be applied in the above problem, to obtain $f\left({\mathit{D}}_k\right)={\mathit{D}}_k^T{\mathit{D}}_k\mathit{w},k\in [0:11]$.

3.2.1. Encoding

As depicted in Figure 3a, since there are 12 subtasks $f\left({\mathit{D}}_k\right),k\in [0:11]$, the master further partitions them into $r=3$ sets before encoding input data, and set i ($i=0,1,2$) contains $K_i$ subtasks. Here, for instance, we assume that $K_0=5$, $K_1=4$, and $K_2=3$, and they satisfy $K_0+K_1+K_2=K=12$. After this hierarchical task partitioning, the input of the j-th subtask in set i is denoted as ${\mathit{D}}_{i,j}\in {\mathbb{R}}^{1000\times 10}$ instead of the previous ${\mathit{D}}_k$.

Next, the $K_i$ input data $\{{\mathit{D}}_{i,j}\in {\mathbb{R}}^{1000\times 10}|j=0,\dots ,K_i-1\}$ in set i are encoded into $N=10$ parts $\{{\tilde{\mathit{D}}}_{i,n}\in {\mathbb{R}}^{1000\times 10}|n=0,\dots ,9\}$ through $g_i\left(x\right)$, where ${\tilde{\mathit{D}}}_{i,n}=g_i\left({\beta }_n\right),n\in [0:9]$. Moreover, $g_i\left(x\right)$ is a polynomial function with a degree of $(K_i+L-1)$, and its form ensures that the parameters $\left\{{\alpha }_{i,j}\right\}$ satisfy $g_i\left({\alpha }_{i,j}\right)={\mathit{D}}_{i,j}$.

3.2.2. Assignment

As Figure 3b shows, for each set, the 10 encoded input data ${\left\{{\tilde{\mathit{D}}}_{i,n}\right\}}_{n=0}^9$ are assigned to the 10 workers. Subsequently, each worker applies function f to compute and return the results to the master. As can be observed, the $K_i$ original subtasks in set i are transformed into 10 subtasks performed on the 10 workers in parallel. Since there are 3 sets, each worker is assigned 3 subtasks. These subtasks are executed in the order of set indices, which implies $f\left({\tilde{\mathit{D}}}_{0,n}\right)$ is computed first, followed by $f\left({\tilde{\mathit{D}}}_{1,n}\right)$, and so on.

3.2.3. Decoding

As illustrated in Figure 3b, following the assignment of encoded input to workers, the master continuously awaits the subtask results from workers and creates a decoding function $r_i\left(x\right)$ for set i. This decoding function is constructed using interpolation to recover the original function $f\left({\mathit{D}}_{i,j}\right)=f\left(g_i\left({\alpha }_{i,j}\right)\right)$. Consequently, each received result, $f\left(g_i\left({\beta }_n\right)\right)$, can be regarded as an interpolation point for $f\left(g_i\left(x\right)\right)$, and $r_i\left(x\right)$ is precisely the interpolation function of $f\left(g_i\left(x\right)\right)$.

Presently, $f\left({\mathit{D}}_{i,j}\right)={\mathit{D}}_{i,j}^T{\mathit{D}}_{i,j}\mathit{w}$ is a polynomial function of degree $d=2$, where the degree d of a polynomial function f is defined as the maximum order of its monomials. We have illustrated how to complete the decoding process in Subsection III.A. By setting the number of received results to $R_i=d(K_i+L-1)+1$, sufficient interpolation points are obtained to accurately recover $f\left(g_i\left(x\right)\right)$ through $r_i\left(x\right)$, i.e., $f\left({\mathit{D}}_{i,j}\right)=f\left(g_i\left({\alpha }_{i,j}\right)\right)=r_i\left({\alpha }_{i,j}\right)$, for any $i\in [0:2]$ and $j\in [0:K_i-1]$. To further illustrate APCC, we also provide a corresponding pseudo-code, as presented in Algorithm 1.

Algorithm 1: APCC

3.3. Hierarchical Task Partitioning and Cancellation

In Figure 3, the hierarchical task partitioning in APCC aims to maximize the utility of computing results from straggling workers. This is achieved through a well-designed structure and appropriate choice of $K_i$ values. Although the same number of encoded subtasks are assigned to all workers, the number of successfully returned results from each worker can differ due to varying processing speeds. As a result, straggling workers may return fewer computing results than faster workers, but they can still make valuable contributions to task completion instead of being completely discarded.

Furthermore, the illustration in Figure 3 suggests that $K_{i-1}$ should exceed $K_i$ [11]. This assertion is explained as follows: The “completion time” of set i” is defined as the moment when a sufficient number of encoded subtask results within set i are obtained. The overarching objective is to minimize the delay in completing the entire task, which must necessarily exceed the “completion time” of any set since the entire task remains incomplete until all r sets are recovered. Given that subtasks are executed in order of set indices, when set r is recovered, the master must have acquired results for the smaller-index sets equal to or greater than $K_i$. Opting for smaller values of $K_i$ for the smaller-index sets would result in more workers experiencing straggling, a situation that should be averted. Further details are expounded in Section 5.

Based on the hierarchical structure, we propose an alternative method to further accelerate the coded computing process. As depicted in Figure 4, the subtasks ${\left\{f\left({\tilde{\mathit{D}}}_{i,n}\right)\right\}}_{i=0}^{r-1}$ to be computed on each worker form an execution sequence. Once enough results for set i are obtained, the master can instruct workers that have not completed the computation of $f\left({\tilde{\mathit{D}}}_{i,n}\right)$ to terminate or skip this part of the computation and proceed to compute the next subtasks $f\left({\tilde{\mathit{D}}}_{i+1,n}\right)$ of the subsequent set. This operation, called “Cancellation”, prevents computation resources from being wasted on completed sets. Considering the presence of non-persistent stragglers, cancellation increases the probability of them overcoming the previous straggling effect and avoiding becoming stragglers again.

4. Performance Analysis

In this section, we first define a metric called encoding rate to evaluate the efficiency performance of coded computing schemes, in terms of utilizing computation resources of workers as efficiently as possible. Then, based on the optimal recovery threshold of LCC [15], we rigorously prove APCC for accurate results is also an optimal polynomial coding in terms of the encoding rate. Furthermore, an information-theoretic guarantee to completely preserve the privacy of input data ${\left\{{\mathit{D}}_k\right\}}_{k=0}^{K-1}$ in APCC is proved. Subsequently, we present an analysis of the approximation error for Case 2 of APCC, along with a discussion of numerical stability. At the end of this section, we provide an analysis of encoding and decoding complexity for APCC and compare it with other state-of-the-art strategies.

4.1. Optimality of APCC in Terms of Encoding Rate

To evaluate the performance of various coded computing schemes, a metric known as the encoding rate $R_{\mathrm{encode}}$ is used. This metric is defined as:

$$\begin{array}{c}\hfill R_{\mathrm{encode}}={\displaystyle \frac{K}{N-S}},\end{array}$$

(17)

where K is the number of subtasks before encoding, N is the number of subtasks after encoding (which is equivalent to the number of workers), and S represents the number of straggling workers that failed to return results before the task was completed. Similar metrics, such as those found in [17,20,46], have also been developed.

Furthermore, since the recovery threshold, denoted by H, is defined as the minimum number of results needed to guarantee decodability, we have $H=N-S$ and thus $R_{\mathrm{encode}}={\displaystyle \frac{K}{H}}$. It is important to note that the encoding rate only applies when decodability is guaranteed.

The physical significance of the encoding rate is the ratio between the computation load of tasks before encoding and that required after encoding. For instance, given a task with a computation load of $O\left(\gamma \right)$, each subtask has a corresponding load of $O\left({\displaystyle \frac{\gamma }{K}}\right)$. As $(N-S)$ subtasks are successfully completed, the required computation load is $O\left({\displaystyle \frac{\gamma (N-S)}{K}}\right)$. Since coded computing essentially trades computation redundancy for reduced delay to mitigate the straggling effect, it is reasonable to use this metric to evaluate the efficiency of different schemes.

Before demonstrating the optimality of APCC in terms of encoding rate, we present the definitions of capacity and linear coded computing schemes.

Definition 1. 

A linear coded computing scheme is one in which the encoded data is a linear combination of the original input data as follows:

$$\begin{array}{c}\hfill {\tilde{\mathit{D}}}_n=\sum _{j=0}^{K-1}{G}_{n,j}{\mathit{D}}_j+{\tilde{\mathit{Z}}}_n,n\in [0:N-1],\end{array}$$

(18)

where $\mathit{G}=\left\{G_{n,j}\right\}\in {\mathbb{R}}^{N\times K}$ is the encoding matrix and ${\tilde{\mathit{Z}}}_n$ are additive random real matrices.

For example, according to Equation (2) in APCC, $G_{n,j}={\displaystyle \frac{w_{i,j}{\prod }_{k=0,k\ne j}^{K_i+L-1}({\beta }_n-{\alpha }_{i,k})}{{\sum }_{k=0}^{K_i+L-1}{w}_{i,k}{\prod }_{l=0,l\ne k}^{K_i+L-1}({\beta }_n-{\alpha }_{i,l})}}$ are the coefficient terms before ${\mathit{D}}_{i,j}$, and ${\tilde{\mathit{Z}}}_n={\sum }_{j=K_i}^{K_i+L-1}{\displaystyle \frac{w_{i,j}{\prod }_{k=0,k\ne j}^{K_i+L-1}({\beta }_n-{\alpha }_{i,k})}{{\sum }_{k=0}^{K_i+L-1}{w}_{i,k}{\prod }_{l=0,l\ne k}^{K_i+L-1}({\beta }_n-{\alpha }_{i,l})}}{\mathit{Z}}_{i,j},$ represents the sum of added random matrices in $g_i\left(x\right)$. The index i corresponds to the set index of the hierarchical task partitioning structure of APCC and can be discarded in other coded computing strategies.

Definition 2. 

For a coded computing problem $(N,S,L,f)$, where N is the number of workers, S and L denote the number of stragglers and colluders, respectively, and the computation function f on the master is a polynomial function of degree d, the capacity C is defined as the supremum of the encoding rate $R_{\mathrm{encode}}$ as:

$$\begin{array}{c}\hfill C=sup{R}_{\mathrm{encode}}(N,S,L,d),\end{array}$$

(19)

over all feasible linear coded computing schemes that can address up to L colluders and S stragglers.

As illustrated in Section 3, APCC is a linear coded computing scheme and its hierarchical structure results in different $K_i$ and $S_i$ for each set, with $K_i$ and $S_i$ representing the number of subtasks before encoding and that of straggling workers, respectively. For set i, $R_i$ represents the number of workers that have successfully returned results in time, implying that the number of stragglers is $S_i=N-R_i$. Moreover, according to Equation (11), set i is considered complete when $R_i=d(K_i+L-1)+1$. Hence, the encoding rate of APCC can be calculated as:

$$\begin{array}{c}\hfill R_{\mathrm{encode}}^{\left[\mathrm{APCC}\right]}={\displaystyle \frac{K_i}{N-S_i}}={\displaystyle \frac{N-S_i-d(L-1)-1}{d(N-S_i)}},\end{array}$$

(20)

or the uncoded version for $L=0$:

$$\begin{array}{c}\hfill R_{\mathrm{encode}}^{\left[\mathrm{APCC}\right]}={\displaystyle \frac{K_i}{N-S_i}}\le {\displaystyle \frac{N}{(N-S_i)(S_i+1)}},\end{array}$$

(21)

where the equality holds when N can be divided by $K_i$.

The following theorem shows that the encoding rate of APCC achieves the capacity, thereby establishing its optimality. In fact, the optimality of APCC in encoding rate is attributed to its identical polynomial coding structure when compared to LCC [15], despite having different function expressions. Specifically, for the accurate results case of APCC, the encoding and decoding processes are achieved through Barycentric polynomial interpolation; for LCC, the processes are achieved through Lagrange polynomial interpolation. Although these two formats can be transformed into each other, the Barycentric polynomial format requires less computational complexity and has stronger numerical stability [27,44]. For the sake of clarity, we omit the set index i in APCC and focus on a specific set, without loss of generality.

Theorem 1. 

For a coded computing problem $(N,S,L,f)$, where N is the number of workers, S and L denote the number of stragglers and colluders, respectively, and the computation function f on the master is an arbitrary polynomial function of degree d, the capacity C is given by:

$$\begin{array}{c}\hfill C=\left\{\begin{array}{cc}{\displaystyle \frac{N-S-d(L-1)-1}{d(N-S)}},\hfill & \mathrm{if}L>0,\hfill \\ max\{{\displaystyle \frac{N-S+d-1}{d(N-S)}},{\displaystyle \frac{N}{(N-S)(S+1)}}\}\hfill & \mathrm{if}L=0.\hfill \end{array}\right.\end{array}$$

(22)

Proof. 

To prove Theorem 1, a lower bound on the capacity C is first established, which follows the encoding rate of APCC in (20) and (21). To establish the upper bound, we leverage the optimality statement of LCC, as illustrated in Theorems 1 and 2 of [15], which shows that polynomial coded computing strategies are able to decode returned computing results successfully only if the following condition is met:

$$\begin{array}{c}\hfill N\ge \left\{\begin{array}{cc}d(K+L-1)+1+S,\hfill & \mathrm{if}L>0,\hfill \\ min\left\{d\right(K-1)+1+S,K(S+1\left)\right\}\hfill & \mathrm{if}L=0.\hfill \end{array}\right.\end{array}$$

(23)

Therefore, we have:

$$\begin{array}{c}\hfill K\le \left\{\begin{array}{cc}{\displaystyle \frac{N-S-1}{d}}-L+1,\hfill & \mathrm{if}L>0,\hfill \\ max\{{\displaystyle \frac{N-S+d-1}{d}},{\displaystyle \frac{N}{(S+1)}}\}\hfill & \mathrm{if}L=0.\hfill \end{array}\right.\end{array}$$

(24)

Equation (24) presents the maximum number of task divisions permissible to ensure decodability, given the numbers of workers N, stragglers S, and colluders L. The reason is that the more divisions there are, the more results are needed from workers. However, there are at most N workers, including S stragglers, to return results. Based on (24), an upper bound on the encoding rate can be derived as:

$$\begin{array}{cc}\hfill & R_{\mathrm{encode}}={\displaystyle \frac{K}{N-S}}\hfill \\ \hfill & \le \left\{\begin{array}{cc}{\displaystyle \frac{N-S-d(L-1)-1}{d(N-S)}},\hfill & \mathrm{if}L>0,\hfill \\ max\{{\displaystyle \frac{N-S+d-1}{d(N-S)}},{\displaystyle \frac{N}{(N-S)(S+1)}}\}\hfill & \mathrm{if}L=0.\hfill \end{array}\right.\hfill \end{array}$$

(25)

Since the capacity C is the supremum of $R_{\mathrm{encode}}$, it also has the same upper bound. With the lower bound provided previously, we can conclude that APCC is an optimal coded computing strategy that can reach the capacity in (22).    □

To enhance clarity, the fundamental proof for the derivation of (23) is briefly introduced in Appendix A, following the same steps as outlined in [15].

Please note that the conclusion presented in this subsection pertains only to accurately coded computing. For approximated coded computing, the use of different approximation methods can lead to varying errors, making it challenging to compare and analyze their impact on the encoding rate and capacity in a qualitative manner.

4.2. Guarantee of the Privacy Preservation

Recall that colluders are those workers who can communicate with each other and attempt to learn something about the original input data. Since the system can tolerate at most L colluders, we assume that there are $L^{\prime }$ colluders, where $L^{\prime }\le L$ and the user does not know which workers are colluding. We use the index set $\mathcal{L}=\{l_0,l_1,\dots ,l_{L^{\prime }-1}\}\subseteq \{0,\dots ,N-1\}$ to denote the colluding workers, where $\left|\mathcal{L}\right|=L^{\prime }$.

Assuming that the input data ${\left\{{\mathit{D}}_{i,j}\right\}}_{j=0}^{K_i-1}$ are independent of each other, we denote the encoded input data sent to workers in the colluding set $\mathcal{L}$ for set i as:

$$\begin{array}{c}\hfill {\tilde{\mathit{D}}}_{i,\mathcal{L}}\triangleq ({\tilde{\mathit{D}}}_{i,l_0},{\tilde{\mathit{D}}}_{i,l_1},\dots ,{\tilde{\mathit{D}}}_{i,l_{L^{\prime }-1}}).\end{array}$$

(26)

Therefore, the information-theoretic privacy-preserving constraint can be expressed as:

$$I({\mathit{D}}_{i,0},{\mathit{D}}_{i,1},\dots ,{\mathit{D}}_{i,K_i-1};{\tilde{\mathit{D}}}_{i,\mathcal{L}})=0,\forall i\in [0,r-1],$$

(27)

where $I(·)$ represents the mutual information function.

With the assumption of finite precision floating point arithmetic, the values of elements in the data matrices such as ${\mathit{D}}_{i,j}$, ${\tilde{\mathit{D}}}_{i,n}$, and ${\mathit{Z}}_{i,j}$ come from a sufficiently large finite field $\mathbb{F}$. Assuming that the size of these data matrices is $m\times m^{\prime }$, we have

$$\begin{array}{cc}\hfill & I({\mathit{D}}_{i,0},{\mathit{D}}_{i,1},\dots ,{\mathit{D}}_{i,K_i-1};{\tilde{\mathit{D}}}_{i,\mathcal{L}})=H({\tilde{\mathit{D}}}_{i,l_0},\dots ,{\tilde{\mathit{D}}}_{i,l_{L^{\prime }-1}})\hfill \\ \hfill & -H({\tilde{\mathit{D}}}_{i,l_0},\dots ,{\tilde{\mathit{D}}}_{i,l_{L^{\prime }-1}}|{\mathit{D}}_{i,0},\dots ,{\mathit{D}}_{i,K_i-1})\hfill \\ \hfill & \stackrel{\left(a\right)}{=}H({\tilde{\mathit{D}}}_{i,l_0},\dots ,{\tilde{\mathit{D}}}_{i,l_{L^{\prime }-1}})-H({\mathit{Z}}_{i,K_i},\dots ,{\mathit{Z}}_{i,K_i+L-1})\hfill \\ \hfill & \stackrel{\left(b\right)}{=}H({\tilde{\mathit{D}}}_{i,l_0},\dots ,{\tilde{\mathit{D}}}_{i,l_{L^{\prime }-1}})-Lm{m}^{\prime }log\left|\mathbb{F}\right|\hfill \\ \hfill & \le H\left({\tilde{\mathit{D}}}_{i,l_0}\right)+\dots +H\left({\tilde{\mathit{D}}}_{i,l_{L^{\prime }-1}}\right)-Lm{m}^{\prime }log\left|\mathbb{F}\right|\hfill \\ \hfill & \stackrel{\left(c\right)}{\le }{L}^{\prime }m{m}^{\prime }log\left|\mathbb{F}\right|-Lm{m}^{\prime }log\left|\mathbb{F}\right|=0,\forall i\in [0,r-1],\hfill \end{array}$$

(28)

where $\left(a\right)$ is due to the fact that all random matrices ${\left\{{\mathit{Z}}_{i,j}\right\}}_{j=K_i}^{K_i+L-1}$ are independent of the input data ${\left\{{\mathit{D}}_{i,j}\right\}}_{j=0}^{K_i-1}$. $\left(b\right)$ is because the entropy of each element in the random matrices equals $log\left|\mathbb{F}\right|$, and $\left(c\right)$ follows from the upper bound of the entropy of each element in ${\tilde{\mathit{D}}}_{i,l_{(·)}}$ being $log\left|\mathbb{F}\right|$. Since the mutual information is non-negative, it must be 0, which guarantees complete privacy preservation.

Note that the analysis in this subsection is applicable to both accurate and approximated cases. This is because the analysis only involves the encoding and assignment steps of APCC, and both cases require the same two initial steps. The key difference between the two aforementioned cases is reflected in the decoding functions with distinct adaptive parameters ${\tilde{w}}_n$, which correspond to Barycentric polynomial interpolation and Berrut’s rational interpolation, respectively.

4.3. Analysis of Approximation Error for Case 2

According to the discussion in [27], the approximation error of Berrut’s rational polynomial interpolation used for Case 2 in APCC is provided as the following theorem:

Theorem 2 

([27]). Let the interpolating objective function $h_i\left(x\right)=f\left(g_i\left(x\right)\right)$ have a continuous second derivative on $[-1,1]$, and the number of received results $R_i>3$, the approximation error is upper bounded as:

$$\begin{array}{cc}\hfill & \left|\right|r_i\left(x\right)-h_i\left(x\right)\left|\right|\le \hfill \\ \hfill & 2(1+\Gamma )sin{\displaystyle \frac{(N-R_i+1)\pi }{2(N-1)}}\left|\right|h_i^{\prime \prime }\left(x\right)\left|\right|,\hfill \end{array}$$

(29)

if $R_i$ is even, and

$$\begin{array}{cc}\hfill & \left|\right|r_i\left(x\right)-h_i\left(x\right)\left|\right|\le \hfill \\ \hfill & 2(1+\Gamma )sin{\displaystyle \frac{(N-R_i+1)\pi }{2(N-1)}}\left(\right||h_i^{\prime \prime }\left(x\right)\left|\right|+\left|\right|h_i^{\prime }\left(x\right)\left|\right|),\hfill \end{array}$$

(30)

if $R_i$ is odd, where $\Gamma \triangleq {\displaystyle \frac{(N-R_i+1)(N-R_i+3){\pi }^2}{4}}$.

Consequently, for set i and a fixed total number of workers N, the approximation using $r_i\left(x\right)$ becomes more accurate as the number of received results $R_i$ increases.

4.4. Numerical Stability

In coded computing, the issue of numerical stability typically arises from the decoding part, which is based on solving a system of linear equations involving a Vandermonde matrix. As previously discussed, Cases 1 and 2 of APCC employ Barycentric polynomial interpolation and Berrut’s rational interpolation as decoding methods, respectively. For Case 1, Barycentric polynomial interpolation demonstrates good performance in addressing errors caused by floating-point arithmetic [44]. Regarding Case 2, it has been shown in [27] that the Lebesgue Constant of Berrut’s rational interpolation grows logarithmically with the number of received results from workers, rendering it both forward and backward stable.

4.5. Encoding and Decoding Complexity

In this subsection, we provide the analysis of encoding and decoding complexity. Intuitively, APCC utilizes the hierarchical task partitioning structure to enhance delay performance. However, it does so at the cost of requiring multiple encoding and decoding operations, specifically r times for the r sets, when compared to LCC [15] and BACC [27].

In LCC and BACC, the encoding operations take N times, corresponding to the number of workers, while the decoding operations take $K^{\prime }$ times, equivalent to the number of task divisions. On the other hand, in the case of APCC, which features r partitioned sets, the encoding and decoding operations entail $Nr$ and ${\sum }_{i=0}^r{K}_i=K$, respectively. When the computation loads per worker in all strategies are equal, i.e., $K^{\prime }={\displaystyle \frac{K}{r}}$, it can be deduced that the encoding and decoding operations in APCC are r times those of LCC and BACC.

5. Hierarchical Task Partitioning

In this section, the hierarchical task partitioning is formulated as an optimization problem with the objective of minimizing the task completion delay. The problem is divided into two cases for consideration: with and without cancellation. Through derivations, two mixed integer non-linear programming problems are obtained, and we propose a maximum value descent (MVD) algorithm to obtain the optimal solutions with low computational complexity. Moreover, after analysis, it is found that the MVD algorithm can be quickly executed by selecting the appropriate input. Detailed explanations are provided as follows.

5.1. Problem Formulation

In the context of negligible encoding and decoding delays, with the computation delays of workers being the dominant component, the delay for a worker to complete a single subtask, denoted as T can be represented by a shifted exponential distribution [4,7,11,12,40,41], whose cumulative distribution function (CDF) is given by:

$$\begin{array}{c}\hfill F_T\left(t\right)=\mathbb{P}[T\le t]=\left\{\begin{array}{cc}1-e^{-\mu (t-a)},\hfill & \mathrm{if}t\ge a,\hfill \\ 0,\hfill & \mathrm{otherwise},\hfill \end{array}\right.\end{array}$$

(31)

where $a>0$ is a parameter indicating the minimum processing time and $\mu >0$ is a parameter modeling the computing performance of workers. All N workers follow a uniform computation delay distribution defined in (31).

Recall that in the hierarchical structure, the completion of a particular set is dependent on the successful receiving of a sufficient number of results from its encoded subtasks. The overall completion of the entire task is achieved only when all r sets have been completed. Notably, $H_i$ is defined as the threshold number of successful results needed to ensure the completion of set i.

Following the discussion in Section 3 and assuming that privacy preservation is required which means $L>0$, the threshold for Case 1 of APCC can be expressed as $H_i=d(K_i+L-1)+1$ according to (11). For Case 2 of APCC, the threshold $H_i$ can be determined based on the desired approximation precision, with higher values of $H_i$ leading to more accurate approximations.

The completion time of sets is defined as $\mathit{t}\triangleq \{t_i,i\in [0:r-1]\}$, where $t_i$ denotes the time interval from the initial moment 0 of the entire task to the recovery moment of set i. The entire task is considered completed when all r sets have been recovered. Therefore, we denote the entire task completion delay as

$$T^{\left[e\right]}=\underset{i\in [0:r-1]}{max}{t}_i.$$

(32)

Note that while each worker executes the assigned subtasks in the order of set indices, the order in which these sets are recovered may not be the same. The completion time of sets is influenced not only by the set indices but also by the recovery thresholds $H_i$ determined by $K_i$.

Due to the randomness of delay, our objective is to minimize the entire task completion delay $T^{\left[e\right]}={max}_{i\in [0:r-1]}{t}_i$, upon which the probability of the master recovering desired results for all sets is higher than a given threshold ${\rho }_s$, as expressed by the following inequality:

$$\begin{array}{c}\hfill \mathbb{P}[R_0\left(t_0\right)\ge H_0,\dots ,R_{r-1}\left(t_{r-1}\right)\ge H_{r-1}]\ge {\rho }_s,\end{array}$$

(33)

where $R_i\left(t\right)$ is defined as the number of returned results for set i until time t.

However, to derive (33), we first need to obtain the distribution of the delay required to receive the last non-straggling result in each set and then derive their joint probability distribution, which is intractable, especially when considering the cancellation of completed sets. As a result, the problem with the constraint (33) is hard to solve.

In the following, we consider substituting (33) with an expectation constraint (34d) and formulate the problem as:

$$\begin{array}{cc}\hfill \mathcal{P}1-1:\underset{\left\{\mathit{K}\right\}}{min}& \underset{i\in [0:r-1]}{max}{t}_i\hfill \end{array}$$

(34a)

$$\begin{array}{cc}\hfill \mathbf{s}.\mathbf{t}.& \sum _{i=0}^{r-1}{K}_i=K,\hfill \end{array}$$

(34b)

$$\begin{array}{cc}\hfill & H_i\le N,\forall i\in [0,r-1]\hfill \end{array}$$

(34c)

$$\begin{array}{cc}\hfill & \mathbb{E}\left[R_i\left(t_i\right)\right]\ge H_i,\forall i\in [0,r-1]\hfill \end{array}$$

(34d)

$$\begin{array}{cc}\hfill & K_i,H_i\in {\mathbb{Z}}^{+},\forall i\in [0,r-1],\hfill \end{array}$$

(34e)

where $\mathit{K}\triangleq \left\{K_i\right|i\in [0:r-1]\}$ is the partitioning scheme.

Constraint (34a) corresponds to the hierarchical task partitioning, and (34c) indicates that the threshold for each set should be smaller than the number of workers. In constraint (34e), ${\mathbb{Z}}^{+}$ represents the set of positive integers. Constraint (34d) states that the master is expected to receive sufficient results of encoded subtasks from workers to recover ${f\left({\mathit{D}}_{i,j}\right)}_{j=0}^{K_i-1}$ in set i. Similar approximation approaches are also used in [4,12,40,41], and the performance gap can be bounded [12].

As previously shown, $H_i=d(K_i+L-1)+1$ for Case 1 of APCC. Additionally, the maximum of $t_i$ for all sets can be replaced with an optimization variable z by adding an extra constraint. Consequently, for Case 1 of APCC, $\mathcal{P}1-1$ can be equivalently written as:

$$\begin{array}{cc}\hfill \mathcal{P}1-2:\underset{\{\mathit{K},z\}}{min}z& \end{array}$$

(35a)

$$\begin{array}{cc}\hfill \mathbf{s}.\mathbf{t}.t_i-z& \le 0,\forall i\in [0,r-1],\hfill \end{array}$$

(35b)

$$\begin{array}{cc}\hfill d(K_i+L-1)+1-\mathbb{E}\left[R_i\left(t_i\right)\right]& \le 0,\forall i\in [0,r-1],\hfill \end{array}$$

(35c)

$$\begin{array}{cc}\hfill d(K_i+L-1)+1-N& \le 0,\forall i\in [0,r-1],\hfill \end{array}$$

(35d)

$$\begin{array}{cc}\hfill K_i\in & {\mathbb{Z}}^{+},\forall i\in [0,r-1],\hfill \end{array}$$

(35e)

$$\begin{array}{cc}\hfill & \mathrm{Constraint}\left(34\mathrm{b}\right).\hfill \end{array}$$

(35f)

For Case 2 of APCC, one only needs to adjust constraints (35c) and (35d) according to the relationship between $K_i$ and $H_i$, which does not affect the subsequent methods employed. Consequently, for the sake of convenience in expression, we will focus on Case 1 of APCC in the following parts of this section, without loss of generality.

5.2. APCC without Cancellation

If the cancellation of completed sets is not considered, we first denote the delay of one worker to continuously complete m subtasks as $T_m$, and derive its CDF from (31) as:

$$\begin{array}{c}\hfill \mathbb{P}[T_m\le t]=\left\{\begin{array}{cc}1-e^{-\mu ({\displaystyle \frac{t}{m}}-a)},\hfill & \mathrm{if}t\ge ma,\hfill \\ 0\hfill & \mathrm{otherwise}.\hfill \end{array}\right.\end{array}$$

(36)

Since computations on workers are independent, $\mathbb{E}\left[R_i\left(t_i\right)\right]$ can be written as:

$$\begin{array}{c}\hfill \mathbb{E}\left[R_i\left(t_i\right)\right]=\sum _{n=0}^{N-1}\mathbb{E}\left[{\mathbb{I}}_{\{T_{i+1}\le t_i\}}\right]=N·\mathbb{P}[T_{i+1}\le t_i],\end{array}$$

(37)

where ${\mathbb{I}}_{\left\{x\right\}}$ denotes the indicator function that equals 1 if event x is true and equals 0 otherwise. $\mathbb{P}[T_{i+1}\le t_i]$ is given by (36).

Substituting (37) into $\mathcal{P}1-2$, we find (35d) is covered by (35c) and obtain the following optimization problem:

$$\begin{array}{cc}\hfill \mathcal{P}2-1:& \underset{\{\mathit{K},z\}}{min}z\hfill \\ \hfill \mathbf{s}.\mathbf{t}.& d(K_i+L-1)+1-N[1-e^{-\mu ({\displaystyle \frac{t_i}{i+1}}-a)}]\le 0,\hfill \end{array}$$

(38a)

$$\begin{array}{cc}\hfill & \forall i\in [0,r-1],\hfill \end{array}$$

(38b)

$$\begin{array}{cc}\hfill & \mathrm{Constraints}\left(34\mathrm{b}\right),\left(35\mathrm{b}\right),\left(35\mathrm{e}\right).\hfill \end{array}$$

(38c)

As $\mathcal{P}2-1$ shows, it is a mixed integer non-linear programming (MINLP) problem, which is usually NP-hard. Although its optimal solution can be found by the Branch and Bound (B&B) algorithm [47], the computational complexity is up to $O\left({\left({\displaystyle \frac{N}{d}}\right)}^r\right)$, which means the B&B algorithm becomes extremely time-consuming when either N or r are large.

Accordingly, to efficiently obtain an optimal solution, we propose the maximum value descent (MVD) algorithm shown in Algorithm 2. The key idea of the MVD algorithm is to iteratively update the input solution $\mathit{K}=\{K_i,i\in [0:r-1]\}$ by adjusting $K_i$ for the set that corresponds to the maximum value descent of the objective function z. In the MVD algorithm, each do-while loop can be regarded as one update, and $K_j$ in Step 7 constantly approaches the optimal $K_j^{*}$. Once reduced in an update, $K_j$ will not increase because the objective function z must decrease in each update. When the updating process terminates, the optimal solution ${\mathit{K}}^{*}$ is exactly the obtained $\mathit{K}$ in the last update. Furthermore, the MVD algorithm has a computational complexity of $O\left({\displaystyle \frac{Nr}{d}}\right)$, as the number of do-while loops is determined by constraint (35d).

Furthermore, the MVD algorithm can be executed quickly by selecting a sufficiently good partitioning solution as input. It should be noted that after relaxation and cancellation of the integer constraint in (35e), $\mathcal{P}2-1$ can be transformed into a convex problem as follows:

$$\begin{array}{cc}\hfill \mathcal{P}2-2:& \underset{\{\mathit{K},z\}}{min}z\hfill \end{array}$$

(39a)

$$\begin{array}{cc}\hfill \mathbf{s}.\mathbf{t}.& \mathrm{Constraints}\left(38\mathrm{b}\right),\left(34\mathrm{b}\right),\left(35\mathrm{b}\right),\hfill \end{array}$$

(39b)

$$\begin{array}{cc}\hfill & K_i>0,\forall i\in [0,r-1].\hfill \end{array}$$

(39c)

and the optimal solution is given in Proposition 1 according to the Karush–Kuhn–Tucker (KKT) conditions.

Algorithm 2: MVD

Proposition 1. 

For given $(N,K,L,d,r,\mu ,a)$, the optimal solution ${\mathit{K}}^{\left[Prop1\right]}$ and corresponding delay ${\mathit{t}}^{\left[Prop1\right]}$ to $\mathcal{P}2-2$ are

$$\begin{array}{cc}\hfill & \sum _{i=0}^{r-1}{e}^{-\mu ({\displaystyle \frac{z^{*}}{i+1}}-a)}=r-{\displaystyle \frac{d(K+rL-r)+r}{N}},\hfill \\ \hfill & t_i^{\left[Prop1\right]}=z^{*},K_i^{\left[Prop1\right]}={\displaystyle \frac{N}{d}}[1-e^{-\mu ({\displaystyle \frac{z^{*}}{i+1}}-a)}]-{\displaystyle \frac{1}{d}}-L+1.\hfill \end{array}$$

(40)

Due to the convexity of $\mathcal{P}2-2$, the Euclidean distance between ${\mathit{K}}^{\left[Prop1\right]}$ and the optimal solution ${\mathit{K}}^{*}$ of $\mathcal{P}2-1$ is small. Therefore, it is recommended to use a rounded result of ${\mathit{K}}^{\left[Prop1\right]}$ as the input for the MVD algorithm.

5.3. APCC with Cancellation

If the cancellation of completed sets is considered, a worker may be canceled in a certain set but successfully return results in time for the subsequent sets. For example, worker n may be a straggler for set i but completes its assigned subtask and returns the result in time for the next set $(i+1)$ due to the cancellation. Such situations make it quite difficult to derive and analyze the expectation of $R_i\left(t\right)$ as in the previous Section 5.2, because the impact of the cancellation of the previous set on the delay of non-straggling workers in subsequent sets needs to be considered. Therefore, we provide the following alternative perspective to simplify this problem.

Note that if set i is the last completed one, the entire task is completed when the last needed result for this set is received. Thus, we define the delay of set i as $T_i^{\left[e\right]}$ and aim to minimize ${max}_{i\in [0:r-1]}\mathbb{E}\left[T_i^{\left[e\right]}\right]$. To derive $\mathbb{E}\left[T_i^{\left[e\right]}\right]$, consider that there are still $N-H_i+1=N-d(K_i+L-1)$ workers computing the last result for set i when other sets are finished. Once any one of these workers returns the first result, this set and the entire task will be completed. Accordingly, the CDF of $T_i^{\left[e\right]}$ can be written as follows:

$$\begin{array}{c}\hfill \mathbb{P}\left[T_i^{\left[e\right]}\le t\right]=1-{\left(1-\mathbb{P}\left[T_{i+1}\le t\right]\right)}^{N-d(K_i+L-1)}\\ \hfill =\left\{\begin{array}{cc}1-e^{-\mu (N-d(K_i+L-1))({\displaystyle \frac{t}{i+1}}-a)},\hfill & \mathrm{if}t\ge (i+1)a,\hfill \\ 0\hfill & \mathrm{otherwise},\hfill \end{array}\right.\end{array}$$

(41)

where $T_{i+1}$ is the delay needed to complete $(i+1)$ subtasks for one worker, shown previously in (36). Then we have

$$\begin{array}{c}\hfill \mathbb{E}\left[T_i^{\left[e\right]}\right]={\displaystyle \frac{i+1}{\mu [N-d(K_i+L-1)]}}+a(i+1).\end{array}$$

(42)

By further adding an extra optimization variable z to substitute ${max}_{i\in [0:r-1]}\mathbb{E}\left[T_i^{\left[e\right]}\right]$, the optimization problem can be formulated as:

$$\begin{array}{cc}\hfill \mathcal{P}3-1:\underset{\{\mathit{K},z\}}{min}& z\hfill \\ \hfill \mathbf{s}.\mathbf{t}.& {\displaystyle \frac{i+1}{\mu \left[N-d(K_i+L-1)\right]}}+a(i+1)-z\le 0,\hfill \end{array}$$

(43a)

$$\begin{array}{cc}\hfill & \forall i\in [0:r-1],\hfill \end{array}$$

(43b)

$$\begin{array}{cc}\hfill & \mathrm{Constraints}\left(34\mathrm{b}\right),\left(35\mathrm{d}\right),\left(35\mathrm{e}\right).\hfill \end{array}$$

(43c)

Note that $\mathcal{P}3-1$ is a MINLP problem similar to $\mathcal{P}2-1$ and has an $O\left({\left({\displaystyle \frac{N}{d}}\right)}^r\right)$ computational complexity to solve if using B&B algorithm. However, after relaxation and canceling the integer constraint in (35e), $\mathcal{P}3-1$ can also be transformed into a convex problem as:

$$\begin{array}{cc}\hfill \mathcal{P}3-2:& \underset{\{\mathit{K},z\}}{min}z\hfill \end{array}$$

(44a)

$$\begin{array}{cc}\hfill \mathbf{s}.\mathbf{t}.& \mathrm{Constraints}\left(43\mathrm{b}\right),\left(34\mathrm{b}\right),\left(35\mathrm{d}\right),\hfill \end{array}$$

(44b)

$$\begin{array}{cc}\hfill & K_i>0,\forall i\in [0,r-1],\hfill \end{array}$$

(44c)

and optimal solution is given in Proposition 2 according to the KKT conditions.

Proposition 2. 

For given $(N,K,L,d,r,\mu ,a)$, the closed-form expression of the optimal solution ${\mathit{K}}^{\left[Prop2\right]}$ to $\mathcal{P}3-2$ is

$$\begin{array}{cc}\hfill & \sum _{i=0}^{r-1}{\displaystyle \frac{i+1}{z^{*}-a(i+1)}}=\mu \left[rN-d(K+rL-r)\right],\hfill \end{array}$$

(45a)

$$\begin{array}{cc}\hfill & K_i^{\left[Prop2\right]}={\displaystyle \frac{N}{d}}-{\displaystyle \frac{i+1}{d\mu [z^{*}-a(i+1)]}}-L+1.\hfill \end{array}$$

(45b)

Consequently, the MVD algorithm is used again to solve $\mathcal{P}3-1$ with a computational complexity of $O\left({\displaystyle \frac{Nr}{d}}\right)$, and the rounded result of ${\mathit{K}}^{\left[Prop2\right]}$ is recommended to be used as the input.

6. Simulation Results

In this section, we leverage simulation results to evaluate the performance of APCC in terms of task completion delay and compare it with other state-of-the-art coded computing strategies, including LCC [15], LCC with multi-message communications (LCC-MMC) [35], and BACC [27]. Additionally, we analyze the impact of the number of partitioned sets r and the number of colluding workers L on the task completion delay of APCC.

In simulations, the entire task is given, leading to a constant computation load for the entire task. In this scenario, we aim to compare the entire task completion delay across various task divisions and coded computing strategies, illustrating the delay performance improvements introduced by APCC. We assume that the computation delay $T_0$ of a single worker to complete the entire task follows a shifted exponential distribution, which is modeled as:

$$\begin{array}{c}\hfill \mathbb{P}[T_0\le t]=\left\{\begin{array}{cc}1-e^{-{\mu }_0(t-a_0)},\hfill & \mathrm{if}t\ge a_0,\hfill \\ 0\hfill & \mathrm{otherwise},\hfill \end{array}\right.\end{array}$$

(46)

then the computation delay T of a single worker to complete one subtask follows:

$$\begin{array}{c}\hfill \mathbb{P}[T\le t]=\left\{\begin{array}{cc}1-e^{-{\mu }_0(Kt-a_0)},\hfill & \mathrm{if}t\ge {\displaystyle \frac{a_0}{K}},\hfill \\ 0\hfill & \mathrm{otherwise},\hfill \end{array}\right.\end{array}$$

(47)

where K denotes the task division number, which may vary depending on the chosen coded computing strategies. The parameter $a_0$ is set to $0.5$ s, and ${\mu }_0$ is set as ${\displaystyle \frac{1}{10a_0}}$. In APCC, ${\left\{K_i\right\}}_{i=0}^{r-1}$ corresponds to the number of subtasks in each set before encoding, and their values are obtained using the MVD algorithm. Then, $5\times {10}^4$ Monte Carlo realizations are run to obtain the average completion delay of the entire task, and the simulation codes are shared here (code link: https://github.com/Zemiser/APCC, accessed on 24 August 2024). Note that by comparing (47) with (31), we have $\mu =K{\mu }_0$ and $a={\displaystyle \frac{a_0}{K}}$, and can further derive the distribution of $T_m$ in (36).

The benchmarks involved in this section are as follows:

(1) APCC: APCC is our proposed coded computing strategy in this paper. It first divides the entire task into K subtasks and then partitions them into r sets with different sizes. The number of subtasks in set $i,i\in [0,r-1]$ is denoted as $K_i$, which satisfies ${\sum }_{i=0}^{r-1}{K}_i=K$. After that, each set is encoded into N subtasks assigned to the N workers. Consequently, each worker is assigned r subtasks. For Case 1 of APCC, the set i is recovered when the master has received $d(K_i+L-1)+1$ results, and the entire task is completed when all sets are recovered.

(2) LCC: LCC proposed in [15] divides the entire task into $K^{\prime }$ subtasks and then encodes them into N subtasks assigned to N workers. Each worker in LCC is assigned one subtask. Therefore, the entire task is completed when the master has received $d(K^{\prime }+L-1)+1$ results. $L=0$ means the absence of a requirement for privacy preservation. We assume that the number of workers N is greater than $d{K}^{\prime }-1$ to facilitate our analysis. Consequently, when $L=0$, the recovery threshold is defined as $d(K^{\prime }-1)+1$ instead of $N-⌊{\displaystyle \frac{N}{K^{\prime }}}⌋+1$ according to [15].

(3) LCC-MMC: MMC proposed in [35] is another approach to utilize the computing results of straggling workers except for the hierarchical structure. It also achieves a partial return of results from workers through a more granular task division. Specifically, LCC-MMC divides the entire task into $K^{LM}$ subtasks and then encodes them into $Nr$ subtasks. Each worker in LCC-MMC is assigned r subtasks and the entire task is completed when the master has received $d(K^{LM}-1)+1$ results. However, LCC-MMC cannot preserve the privacy of input data because multiple encoded data from the same encoding function are sent to a worker, which is different from the case of APCC where r subtasks assigned to the same worker are generated by r different encoding functions ${\left\{g_i\left(x\right)\right\}}_{i=0}^{r-1}$.

(4) BACC: The BACC strategy, as introduced in [27], offers approximated results with improved precision achievable by increasing the number of return results from workers. It shares a task division structure identical to LCC, partitioning the task into $K^{\prime }$ subtasks and then further encoding them into N subtasks. Each worker in BACC is assigned one such subtask.

To ensure fairness, all strategies employ an identical number of workers and distribute an equivalent computation load for a single worker. Assuming that the computation loads of the entire task are $O\left(\gamma \right)$, then each subtask $f\left({\mathit{D}}_k\right)$ in APCC has a computation load of $O\left({\displaystyle \frac{\gamma }{K}}\right)$, and the computation loads of each worker in APCC are $O\left({\displaystyle \frac{\gamma r}{K}}\right)$ because there are r partitioned sets. Similarly, we can derive that the computation loads of each worker in LCC, BACC and LCC-MMC are $O\left({\displaystyle \frac{\gamma }{K^{\prime }}}\right)$, $O\left({\displaystyle \frac{\gamma }{K^{\prime }}}\right)$ and $O\left({\displaystyle \frac{\gamma r}{K^{LM}}}\right)$, respectively. In order to ensure that each worker in these schemes performs an identical fraction of the entire task as APCC, we have

$$\begin{array}{c}\hfill K^{\prime }={\displaystyle \frac{K^{LM}}{r}}={\displaystyle \frac{K}{r}}.\end{array}$$

(48)

Due to the different applicability of various coded computing strategies, we will first conduct a comprehensive analysis and comparison of APCC alongside other strategies within the following three scenarios: (1) Accurate results with L colluding workers ($L>0$); (2) Accurate results without colluding workers ($L=0$); (3) Approximated results. Finally, we study the impact of the parameters r and L on the delay performance of APCC.

6.1. Accurate Results with L Colluding Workers ($L>0$)

In this scenario, we consider the following three benchmarks: LCC, APCC without cancellation, and APCC with cancellation. For fair comparison, the computation load of workers should be set the same, so we have $K^{\prime }={\displaystyle \frac{K}{r}}$.

As shown in Figure 5, the average completion delay of the entire task ${\left\{f\left({\mathit{D}}_k\right)\right\}}_{k=0}^{K-1}$ first decreases and then increases with the task division number K, indicating the existence of an optimal division that minimizes the delay. This trade-off arises from balancing the computation load of each worker and the minimum number of workers needed to recover ${\left\{f\left({\mathit{D}}_k\right)\right\}}_{k=0}^{K-1}$. On the one hand, as the division number decreases, the computation load of each subtask increases, which leads to longer computation delays for each worker due to the increased workload. Although the number of workers waiting for results decreases, the increase in load negates this advantage. On the other hand, while the division number approaches the maximum, as illustrated in the inequality (24), the number of workers that the master needs to wait for approaches N, making the straggling effect a bottleneck for performance and increasing the delay. The zigzag fluctuations in the curve are mainly due to the integer values of the partitioning numbers.

Note that the primary metric for evaluating different schemes in our study is the minimum task completion delay under different division numbers, as depicted in Figure 5. This is because the division number $K^{\prime }={\displaystyle \frac{K}{r}}$ corresponds to the division of computation function inputs, which is typically a high-dimensional matrix. As such, $K^{\prime }$ can be adjusted flexibly in most cases. Therefore, the minimum achieved task completion delay is the main focus of our analysis.

Figure 6 compares APCC and LCC in terms of the minimum task completion delay. In these benchmarks, ‘Brute-Force’ refers to a partitioning strategy derived from an exhaustive search across all possible values of $\left\{K_i\right\}$. Due to the highly complex traversal search, the brute-force results are only provided for scenarios with a smaller number of sets ($r=4$). Figure 6 illustrates that both APCC with and without cancellation yield sufficient reductions in task completion delay compared to LCC. For instance, when $N=100$, $L=10$, $d=2$, $r=16$, and the partitioning strategy obtained from the MVD algorithm is utilized, APCC with and without cancellation achieve delay reductions of $41.4\%$ and $47.5\%$, respectively, compared to LCC. Moreover, the comparison with the ‘Brute-Force’ benchmarks shows that the partitioning strategy $\left\{K_i\right\}$ obtained through the MVD algorithm is near-optimal.

6.2. Accurate Results without Colluding Workers ($L=0$)

In this scenario, we evaluate four benchmarks: LCC, LCC-MMC, and APCC with and without cancellation. Among these, only LCC does not consider partial results from straggling workers. Similar to Subsection IV.A, we set $K^{\prime }={\displaystyle \frac{K^{LM}}{r}}={\displaystyle \frac{K}{r}}$, with $K^{LM}$ representing the task division number for LCC-MMC.

In Figure 7, both LCC-MMC and APCC effectively reduce task completion delay compared to LCC. Specifically, when r is large enough, APCC with cancellation closely approaches the performance of LCC-MMC. This similarity arises because, in both APCC and LCC-MMC, the master utilizes nearly all computing results from workers when divided subtasks are sufficiently small. Figure 7 also illustrates that when privacy is not a concern, MMC is a viable method to reduce the delay in coded computing.

Compared to Figure 6, we observe that the absence of colluding workers limits the potential for delay optimization. For instance, with parameters $N=100$, $L=0$, $d=2$, and $r=16$, APCC with cancellation achieves only a $20.3\%$ delay reduction compared to LCC.

6.3. Approximated Results

In this subsection, we compare the task completion delay of BACC and case 2 of APCC, which can both provide approximated results with fewer workers than the recovery threshold. To ensure uniform worker computation load, we also set $K^{\prime }={\displaystyle \frac{K}{r}}$, as in our previous analysis. Furthermore, since BACC shares an identical task division structure with LCC, we employ a smaller recovery threshold of the same form as LCC to evaluate its delay performance. For instance, when the recovery threshold $d(K^{\prime }+L-1)+1$ exceeds N, a reduced uniform recovery threshold ${\displaystyle \frac{d}{2}}(K^{\prime }+L-1)+1$ below N can be employed for both BACC and APCC.

As shown in Figure 8, the hierarchical task partitioning and the cancellation of completed sets in APCC yield sufficient delay performance improvement. Compared to BACC, the proposed MVD algorithm for APCC achieves up to $42.9\%$ delay reduction. Note that in this scenario, both APCC and BACC can obtain approximated results with fewer returned results, while LCC for accurate computation fails to work when $K^{\prime }$ is larger than 20 in the two cases of Figure 8, as the recovery threshold of LCC needs to be larger than $d(K^{\prime }+L-1)+1$.

6.4. Impact of r and L on the Performance of APCC

The impact of the hierarchical partitioning number of sets r on the task completion delay of APCC is illustrated in Figure 9a. It is observed that a larger number of sets r results in a smaller computation delay, which is consistent with the results shown in previous figures. The reduction in delay can be attributed to the fact that a larger r implies a smaller computation load for each subtask in the hierarchical structure, and the difference in computation load between fast and slow workers can be described more precisely. Consequently, the proposed MVD algorithm can better utilize the computing results of straggling workers to reduce delay. Furthermore, Figure 9a indicates that the benefit of increasing r has a boundary effect, which corresponds to the upper bound of benefit brought by the granularity refinement of task divisions.

Recall that L denotes the maximum number of colluding workers that a coded computing scheme can tolerate. The value of L can serve as an indirect indicator of the level of privacy preservation offered by the scheme. Specifically, a larger value of L corresponds to more stringent privacy protection and a higher tolerance for colluders. It is demonstrated in Section 4.2 that complete data privacy can be achieved as long as the number of colluders remains below L.

Figure 9b illustrates the impact of the number of colluding workers L on the trade-off between delay and privacy preservation. It is worth noting that, for a fixed $K^{\prime }$, increasing the value of L leads to a larger recovery threshold H for the original subtasks, which results in a longer task completion delay. Moreover, as demonstrated in (24), choosing a larger value of L restricts the maximum number of task divisions. Consequently, the range of $K^{\prime }$ values corresponding to the plotted curves in Figure 9b varies with L.

7. Conclusions

In this paper, we have investigated a distributed computing system that consists of one master and multiple workers. We have first proposed an adaptive privacy-preserving coded computing (APCC) strategy, which is suitable for diverse task scenarios and computation functions. APCC adaptively provides accurate or approximated results with controllable error according to the form of computation functions, and the computation process remains numerically stable. We have rigorously proved the optimality of APCC in terms of encoding rate based on the optimal recovery threshold of LCC. The complete privacy preservation of input data has also been proved.

We have further provided a low-complexity maximum value descent algorithm to optimally solve the hierarchical task partitioning problem in APCC, with and without considering cancellation, aiming at minimizing task completion delay. The cancellation is our proposed operation aiming to further accelerate computation by timely canceling the completed tasks. Extensive simulations have demonstrated that APCC outperforms the state-of-the-art coded computing strategies by a range of $20.3\%$ to $42.9\%$ in terms of task completion delay.