Efficient Privacy-Preserving K-Means Clustering from Secret-Sharing-Based Secure Three-Party Computation

Abstract

Privacy-preserving machine learning has become an important study at present due to privacy policies. However, the efficiency gap between the plain-text algorithm and its privacy-preserving version still exists. In this paper, we focus on designing a novel secret-sharing-based K-means clustering algorithm. Particularly, we present an efficient privacy-preserving K-means clustering algorithm based on replicated secret sharing with honest-majority in the semi-honest model. More concretely, the clustering task is outsourced to three semi-honest computing servers. Theoretically, the proposed privacy-preserving scheme can be proven with full data privacy. Furthermore, the experimental results demonstrate that our proposed privacy version reaches the same accuracy as the plain-text one. Compared to the existing privacy-preserving scheme, our proposed protocol can achieve about 16.5×–25.2× faster computation and 63.8×–68.0× lower communication. Consequently, the proposed privacy-preserving scheme is suitable for secret-sharing-based secure outsourced computation.

1. Introduction

With the rapid development of machine learning, Machine Learning as a Service (MLaaS) has become a popular business. Nowadays, machine learning is also widely applied in different fields, such as finance, healthcare, image recognition, and so on. Major companies such as Microsoft, Google, Amazon, etc. are beginning to provide cloud-based MLaaS. In general, these services allow the machine learning algorithms to be updated and improved via input data from their users. In order to gain high-precision model, companies tend to come together and train a common model using their datasets.

However, with the improvement of awareness of privacy, the problems caused by data privacy leakage have become increasingly prominent. On the one hand, the user, who uses the MLaaS service, hopes that the service is conducted without revealing any information of their queries and prediction result. On the other hand, companies want to train a common model without sharing their dataset. Therefore, it is important to find a secure way for privacy-preserving machine learning (PPML) to proceed.

Privacy-preserving machine learning can be tracked back to privacy-preserving data mining, which was firstly introduced by Lindell and Pinkas [1]. Since then, more and more researchers have put focus on privacy-preserving machine learning.

The dataset can be divided into two main types: labeled data and unlabeled data. Generally speaking, the former usually uses supervised learning algorithms when training the model, while the latter unsupervised learning algorithm [2]. In recent years, most solutions of PPML only consider the supervised learning algorithm, and there is less consideration of the unsupervised learning algorithm.

As an unsupervised machine learning technique, similar input records are grouped into clusters while records belonging to different clusters should be maximally different [3]. In this work, we focus on clustering, which plays an extremely important role in data processing and analysis. The goal of clustering is to divide given unlabeled data into several disjoint subsets, such that each subset has similar properties to each other.

The K-means algorithm is one of the most well-known clustering algorithms. A privacy-preserving K-means clustering, which has full data privacy, allows the parties to cluster their combined datasets without revealing any other information except for the final centroid [3]. In other words, the information of intermediate centroids, cluster assignments, and cluster sizes should be protected in the protocol. Although there are several works for privacy-preserving K-means clustering at present, only few of them consider both full data privacy and efficiency. This raises the question:

Could we find a way to achieve both full data privacy for security and efficiency for practicability?

As we will show below, the answer is yes with our replicated secret-sharing-based K-means clustering protocol.

1.1. Related Work

In general, different algorithms have different privacy-preserving tools. For example, the chaotic system is a popular tool for image encryption [4]. Privacy-preserving K-means clustering falls into one of two categories: (i) homomorphic encryption-based and (ii) secure multiparty computation-based.

Homomorphic encryption (HE) was first proposed by Rivest et al. [5] in 1978. Homomorphic encryption is an encryption scheme where there exists a homomorphism relationship between operations on the plaintext and operations on the ciphertext, such that one can operation on the ciphertext can proceed without leaking any information of plaintext and it obtains the same effect as operation on plaintext after decrypting the result. HE is also widely applied in secure outsourced computation. HE can be divided into fully homomorphic encryption (FHE) and partially homomorphic encryption. FHE supports arbitrary computation on ciphertexts without any limitation. The first feasible FHE scheme was proposed by Gentry [6] in 2009, but it is inefficient. Instead of using inefficient FHE, many researchers adopt efficient partially homomorphic encryption, which only supports homomorphic addition or homomorphic multiplication. For example, RSA [5], Paillier [7], and ElGamal [8] are common partially homomorphic encryptions.

Generally speaking, an HE-based scheme provides full data privacy, as long as the underlying HE cryptosystem is secure. The first HE-based K-means clustering scheme was given by Vaidya and Clifton [9] in 2003, but it does not satisfy full data privacy. In 2007, Bunn and Ostrovsky [10] presented a two-party privacy-preserving K-means scheme based on additive homomorphic encryption that guarantees full data privacy in the semi-honest model. In 2015, Rao et al. [11] proposed a parallelable outsourced distributed clustering protocol based on Paillier homomorphic encryption in the federated cloud environment, but their work is inefficient due to its bit-array-based comparison; hence, Kim and Chang [12] improved it with a new secure comparison protocol. Jäschke and Armknecht [13] proposed K-means clustering based on FHE over the torus (TFHE) [14], which provides full privacy guarantees. Cai and Tang [15] proposed their K-means algorithm based on Liu’s homomorphic encryption [16], which has been proven insecure [17]. Unfortunately, all of these schemes do not scale for large datasets due to the heavily homomorphic operations.

Secure two-party computation (2PC) was first presented by Yao [18] in 1982 and extended to multiparty computation (MPC) by Goldreich et al. [19] in 1987. According to the parties number, we construct 2PC protocol with Yao’s garbled circuit (GC) [20], while MPC is done with secret sharing (SS). To further enhance efficiency, t-out-of-n threshold secret sharing has been applied in recent years. Moreover, the algebraic structure is another important optimized direction. For example, Shamir’s secret sharing (SSS) is a famous threshold secret sharing method [21], but it generally works for finite field, such as prime field ${\mathbb{Z}}_p$, which is inefficient compared with several protocols operating over the-power-of-two ring ${\mathbb{Z}}_{2^{\ell }}$ in PPML. This is because the latter takes full advantage of the underlying CPU architecture.

In this work, we only focus on SS-based K-means clustering. Doganay et al. [22] proposed distributed privacy preserving K-means clustering with additive secret sharing (ASS), but this work reveals the final cluster assignments to parties. Patel et al. [23,24] proposed their K-means algorithm under different security model. Upmanyu et al. [25] and Baby and Chandra [26] respectively presented a distributed threshold secret sharing scheme based on the Chinese remainder theorem (CRT-SS). However, none of them provide full privacy guarantees as shown in [3]. In 2020, Mohassel et al. [27] presented 2PC K-means clustering protocol with 2-out-of-2 additive secret sharing, and although it provides full data privacy, it is inefficient in terms of computation and communication overhead, because their work heavily relies on garbled circuit and oblivious transfer (OT). Therefore, this scheme is not practical for large-scale clustering tasks.

As shown above, most of the existing privacy-preserving K-means clustering protocols take no account of full data privacy. In addition, the gap of efficiency still exists compare with plaintext training. Algorithm inefficiency also limits its practicality, especially for large-scale training tasks. In this work, we want to construct privacy-preserving K-means clustering, which has full data privacy for security and high efficiency for practicality.

1.2. Our Contributions

In this work, we only focus on SS-based K-means clustering schemes. We provide the comparison with existing SS-based K-means clustering in Table 1. We propose an efficient three-party computation protocol for privacy-preserving K-means clustering. Concretely, our contributions are described as follows:

Table 1. The comparison of different SS-based K-means schemes. ASS: Additive secret sharing, CRT-SS: Chinese remainder theorem secret sharing, SSS: Shamir’s secret sharing, RSS: Replicated secret sharing. ZKP: Zero knowledge proof, GC: Garbled circuit. OT: Oblivious transfer. N/A: Undefined. L1: intermediate centroids, L2: intermediate cluster sizes, L3: other intermediate values (e.g., intermediate cluster assignments or distance comparison results). ✓: no leakage, ✗: leakage. FDP: Full data privacy.

Scheme	Security	Technology	Domain	L1	L2	L3	FDP
Doganay et al. [22]	Semi-honest	ASS	N/A	✓	✓	✗	✗
Upmanyu et al. [25]	Semi-honest	CRT-SS	${\mathbb{Z}}_p$	✓	✗	✗	✗
Patel et al. [23]	Semi-honest	SSS	${\mathbb{Z}}_p$	✗	✗	✓	✗
Patel et al. [24]	Malicious	SSS+ZKP	${\mathbb{Z}}_p$	✗	✗	✓	✗
Baby and Chandra [26]	N/A	CRT-SS	${\mathbb{Z}}_p$	✗	✗	✗	✗
Mohassel et al. [27]	Semi-honest	ASS+GC+OT	${\mathbb{Z}}_{2^{\ell }}$	✓	✓	✓	✓
This work	Semi-honest	RSS	${\mathbb{Z}}_{2^{\ell }}$	✓	✓	✓	✓

• Our protocol provides full privacy guarantees, which allows different computing parties to cluster the combined datasets without revealing any other information except the final centroids.
• Our protocol is based on replicated secret sharing (RSS), which is a 2-out-of-3 threshold secret sharing proposed by Araki et al. [28] and is suitable for constructing efficient protocol over ${\mathbb{Z}}_{2^{\ell }}$. Our protocol is secure against a single corrupt server under a semi-honest model. We analyze the security with universal composition framework [29].
• The experimental results demonstrate that our protocol reaches the same accuracy as the plaintext K-means clustering algorithm. With the fast network, our privacy-preserving scheme can deal with datasets of million points in an acceptable time.

1.3. Roadmap

The remaining sections are organized as follows. In Section 2, we give the definition of basic notation, threat model, and security assumption of secure computation, and plaintext algorithm related to K-means clustering. In Section 3, we give the cryptographic building blocks. In Section 4, we propose our efficient three-party protocol construction. We give detailed security analysis of our protocol in Section 5. Then, we report the experimental results of our construction in Section 6. Finally, we conclude this paper in Section 7.

2. Preliminaries

2.1. Basic Notation

We denote the party i by ${\mathcal{P}}_i$ for each $i\in \{1,2,3\}$. For simplicity, we define ${\mathcal{P}}_0={\mathcal{P}}_3$, and ${\mathcal{P}}_4={\mathcal{P}}_1$ in the context. $x{\in }_R\mathbb{F}$ is chosen uniformly at random from finite set $\mathbb{F}$. We write a bold letter $\mathit{v}$ to denote a d-dimension vector. The j-th component of vector $\mathit{v}$ is $v_j$. If x is a ℓ-bit number, then $x\left[i\right]$ is its i-th bit. Let $\kappa$ be the security parameter. We use $\left[n\right]$ to denote set $\{1,\cdots ,n\}$. Furthermore, we assume all float-point data are encoded as ℓ-bit fixed-point number with f-bit precision, where $f<\ell$.

2.2. Threat Model and Security Assumption

Our protocol follows a static and semi-honest model [30] under the honest-majority setting, i.e., the adversary $\mathcal{A}$ only corrupts a single and fixed party during protocol executing. In this setting, the corrupted party follows protocol honestly and wants to learn the input of other parties from received messages. Therefore, the semi-honest model is also called the passive model. Furthermore, we assume the parties communicate with other parties through a secure channel and the network is synchronized.

We prove security using a universally composable framework [29] in the ideal–real paradigm [30]. Let $\mathcal{F}$ be the ideal functionality executed by a trusted third party (TTP) in the ideal world, and $\prod$ be the real protocol executed by all parties in the real world. In the ideal world, there is a simulator $\mathsf{Sim}$ that plays as adversary $\mathcal{A}$. Let C be the set of corrupted parties and $x_i$ be ${\mathcal{P}}_i$’s input. We define the ideal interaction and the real interaction as follows:

• ${\mathrm{Ideal}}_{\mathcal{F},\mathsf{Sim}}(\kappa ,C;x_1,\cdots ,x_n)$: Compute $(y_1,\cdots ,y_n)\leftarrow \mathcal{F}(x_1,\cdots ,x_n)$;
  Output $\mathsf{Sim}(C,\{(x_i,y_i),i\in C\}),(y_1,\cdots ,y_n)$, where $y_i$ is ${\mathcal{P}}_i$’s output.
• ${\mathrm{Real}}_{\prod ,\mathcal{A}}(\kappa ,C;x_1,\cdots ,x_n)$: Run the protocol $\prod$;
  Output $\{{\mathrm{View}}_i,i\in C\},(y_1,\cdots ,y_n)$, where ${\mathrm{View}}_i$ is the final view of ${\mathcal{P}}_i$.

We say the protocol $\prod$ securely computes the functionality $\mathcal{F}$ in the semi-honest model, if the view of simulator in the ideal world is indistinguishable from the view of adversary in the real world. We refer the reader to [30] for more details.

2.3. The K-Means Clustering Algorithm

Given dataset $D=\{{\mathbf{P}}_1,\cdots ,{\mathbf{P}}_n\}$ with n data points, each point ${\mathbf{P}}_i$ is a d-dimension vector $(P_{i1},\cdots ,P_{id})$. We form $n\times d$ matrix $\mathbf{P}$. A standard K-means clustering algorithm includes the following steps [10,27]:

• Cluster centroids initialization: randomly choose K different points as initialized centroids ${\mathit{\varphi }}_1,\cdots ,{\mathit{\varphi }}_K$ for K groups, where ${\mathit{\varphi }}_k$ is d-dimension vector $({\varphi }_{k1},\cdots ,{\varphi }_{kd})$, $k\in \left[K\right]$.
• Repeat the following until the stopping criterion (Lloyd’s steps):

  (a)

  For $i\in \left[n\right],k\in \left[K\right]$, compute the Euclidean distance between point ${\mathbf{P}}_i$ and centroids ${\mathit{\varphi }}_k$ by

  $$X_{ik}=\sqrt{\sum _{j=1}^d{\left(P_{ij}-{\varphi }_{kj}\right)}^2}.$$

  (1)

  (b)

  Assign each data point ${\mathbf{P}}_i$ to the closest cluster $m_i$ for $i\in \left[n\right]$. This can be done by computing $k_i\leftarrow \mathsf{arg}\mathsf{min}\{X_{i1},\cdots ,X_{iK}\}$ firstly, and then generate a K-dimension one-hot vector ${\mathbf{c}}_i$ where ‘1’ indicates the $k_i$-th component of vector $(X_{i1},\cdots ,X_{iK})$. We form $K\times n$ matrix $\mathbf{C}$ such that the i-th column of $\mathbf{C}$ is the one-hot vector ${\mathbf{c}}_i$. Let ${\mathit{m}}_k$ be the k-th row of $\mathbf{C}$.

  (c)

  Recalculate the average of the points in each cluster. For each cluster $k\in \left[K\right]$, compute new cluster center with

  $${\mathit{\phi }}_k=\frac{{\mathit{m}}_k·\mathbf{P}}{D_k},$$

  (2)

  where $D_k={\sum }_{i=1}^n{m}_{ki}$ is the point number of k-th cluster.

  (d)

  Check the stopping criterion and update the new cluster center with the average. For each $k\in \left[K\right]$, compute the Euclidean distance between ${\mathit{\phi }}_k$ and ${\mathit{\varphi }}_k$ at first, and then the squared error can be computed by

  $$e=\sum _{k=1}^K{e}_k=\sum _{k=1}^K\sqrt{\sum _{j=1}^d{({\phi }_{kj}-{\varphi }_{kj})}^2}.$$

  (3)

  Given a small error $\epsilon$, if $e\ge \epsilon$, then update ${\mathit{\varphi }}_k$ with ${\mathit{\phi }}_k$. Otherwise, stop the criterion and output ${\mathit{\phi }}_k$.

3. Building Blocks

This section gives the building blocks for our privacy-preserving K-means clustering protocol.

3.1. Correlated Randomness

In order to generate randomness among parties without any interaction, similar to [28,31,32], correlated randomness is introduced to this work.

Let $F:{\mathbb{Z}}_2^{\kappa }\times {\mathbb{Z}}_2^{\kappa }\to {\mathbb{Z}}_{2^{\ell }}$ be a secure pseudo-random function (PRF). $\mathsf{count}$ is a counter maintained by the parties and updated after every PRF invocation. All parties run a one-time setup to establish PRF keys. The one-time setup can be done by letting each ${\mathcal{P}}_i$ choose a random key $k_i$, and then sending it to ${\mathcal{P}}_{i-1}$ for $i\in \left[3\right]$. Namely, each ${\mathcal{P}}_i$ has the random PRF keys $k_i$ and $k_{i-1}$ after the setup. In this way, the parties can generate the following correlated randomness locally:

• 3-out-of-3 randomness: ${\mathcal{P}}_i$ holds ${\alpha }_i=F_{k_i}\left(\mathsf{count}\right)-F_{k_{i-1}}\left(\mathsf{count}\right)$.
• 2-out-of-3 randomness: ${\mathcal{P}}_i$ holds $({\alpha }_i,{\alpha }_{i-1})=(F_{k_i}\left(\mathsf{count}\right),F_{k_{i-1}}\left(\mathsf{count}\right))$.

Note that 3-out-of-3 randomness has the property that ${\alpha }_1+{\alpha }_2+{\alpha }_3\equiv 0mod{2}^{\ell }$, which is known as zero sharing.

3.2. Replicated Secret Sharing

Replicated secret sharing (RSS) was first proposed by Ito et al. [33]. In CCS’16, Araki et al. [28] presented 2-out-of-3 replicated secret sharing scheme, which has high throughput and low latency. As a famous 3PC framework, ABY3 is also based on this variant over 2-power ring ${\mathbb{Z}}_{2^{\ell }}$ [31]. Our protocol also builds on ABY3. Let m be a general modulus, and we describe this replicated secret sharing as follows.

• $⟦x⟧\leftarrow \mathsf{share}\left(x\right)$: To share a secret $x\in {\mathbb{Z}}_m$, the dealer samples three random values $x_1,x_2,x_3{\in }_R{\mathbb{Z}}_m$ under the constraint that $x\equiv x_1+x_2+x_{3}modm$. For $i\in \left[3\right]$, ${\mathcal{P}}_i$ gets $(x_i,x_{i+1})$. We write $⟦x⟧:=(x_1,x_2,x_3)$.
• $x\leftarrow \mathsf{reconstruct}(⟦x⟧,\mathcal{P})$: To reveal $⟦x⟧$ to all parties, ${\mathcal{P}}_i$ sends $x_i$ to ${\mathcal{P}}_{i+1}$, then each party reconstructs x locally by computing $x_1,x_2,x_3{\in }_R{\mathbb{Z}}_m$. To reveal $⟦x⟧$ only to ${\mathcal{P}}_i$, ${\mathcal{P}}_{i-1}$ sends $x_{i-1}$ to ${\mathcal{P}}_{i-1}$ which reconstructs x locally.

In this work, m can be a different modulus. When $m=2^{\ell }$, we call that arithmetic share and denote it as $⟦x⟧$ or $⟦x{⟧}^{\mathsf{A}}$. When $m=2$, we call that boolean share and denote it as $⟦x{⟧}^{\mathsf{B}}$.

The linear operation between two shares can be computed locally because of the linearity property. This means that given public constant $a,b,c$ and two shares $⟦x⟧=(x_1,x_2,x_3),⟦y⟧=(y_1,y_2,y_3)$, $⟦ax\pm by\pm c⟧$ can be locally computed as $(a{x}_1\pm b{y}_1\pm c,a{x}_2\pm b{y}_2,a{x}_3\pm b{y}_3)$. In order to compute the shares of multiplication $⟦z⟧=⟦xy⟧$, the parties generate zero sharing locally at first, and then ${\mathcal{P}}_i$ locally computes 3-out-of-3 share $z_i=x_i{y}_i+x_{i+1}{y}_i+x_i{y}_{i+1}+{\alpha }_i$ for each $i\in \left[3\right]$. Finally, resharing is performed by the parties for 2-out-of-3 sharing semantics; this can be done by ${\mathcal{P}}_i$ that sends $z_i$ to ${\mathcal{P}}_{i+1}$. It is easy to see that each party only sends 1 ring element per multiplication. Compare that with ASS in the 3PC setting, wherein RSS reduces 50% communication overhead. In this context, we denote RSS multiplication protocol as ${\prod }_{\mathsf{Mul}}$.

If the dealer wants to share a random value $r{\in }_R{\mathbb{Z}}_{2^{\ell }}$ for j-th time to all parties, the PRF key from zero sharing can be used. In particular, ${\mathcal{P}}_i$ lets $r_i=F_{k_i}\left(j\right)$ and $r_{i-1}=F_{k_{i-1}}\left(j\right)$. If ${\mathcal{P}}_1$ wants to share his private input x to all parties, the parties first generate another zero sharing $({\beta }_1,{\beta }_2,{\beta }_3)$, then define the share of r as $⟦r⟧:=({\beta }_1+x,{\beta }_2,{\beta }_3)$, and ${\mathcal{P}}_i$ sends x to ${\mathcal{P}}_{i-1}$ in the end.

Note that the appearance of decimals in the computation is unavoidable in the computation, while secret sharing only works on the integer field. To represent a real number $\overline{x}\in \mathbb{R}$, we use a fixed-point representation with f-bit precision [34]. We scale $\overline{x}$ by a factor of $2^f$ and represent the rounded integer $x=\lfloor 2^f·\overline{x}\rfloor$ as a ℓ-bit signed integer over ${\mathbb{Z}}_{2^{\ell }}$. However, the multiplication result $⟦z^{\prime }⟧$ between two shares $⟦x⟧$ and $⟦y⟧$ will have $2f$-bit precision.

To reduce precision from $2f$ to f, Mohassel and Rindal [31] introduce two probability truncation techniques to truncate the f-bit of the result. In this work, we use the second probability truncation technique. First, the parties generate a random truncation pair $(⟦s^{\prime }⟧,⟦s⟧)$ in the preprocessing phase, where $s^{\prime }\in {\{0,1\}}^{\ell }$ with $2f$-bit precision, $s\in {\{0,1\}}^f$ with f-bit precision, such that $s=s^{\prime }/2^f$. In the online phase, the parties jointly compute $⟦z^{\prime }-s^{\prime }⟧$, then compute and open $(z^{\prime }-s^{\prime })$, which is followed by computing $⟦z⟧=⟦s⟧+(z^{\prime }-s^{\prime })/2^f$ locally. The truncation induces error is only $2^{-f}$.

3.3. Oblivious Selection Protocol

As an important part of our privacy-preserving K-means clustering protocol, we define the oblivious selection functionality ${\mathcal{F}}_{\mathsf{OS}}$, whose functionality takes the arithmetic shares $⟦x⟧$, $⟦y⟧$ and boolean share $⟦b{⟧}^{\mathsf{B}}$ as input, and returns $⟦x⟧$ if $b=0$, and $⟦y⟧$ otherwise. Note that ${\mathcal{F}}_{\mathsf{OS}}$ can be explained as $f(x,y,b)=(1-b)x+by=x+(y-x)b$. It seems that the parties only need to compute multiplication between $(y-x)$ and b once to implement ${\mathcal{F}}_{\mathsf{OS}}$. However, this is non-trivial since $⟦y-x{⟧}^{\mathsf{A}}$ is arithmetic share and $⟦b{⟧}^{\mathsf{B}}$ is boolean share, while the RSS multiplication only works for same shares.

A natural idea is to convert $⟦b{⟧}^{\mathsf{B}}$ to $⟦b{⟧}^{\mathsf{A}}$ by using bit injection protocol ${\prod }_{\mathsf{B}}\mathsf{2}\mathsf{A}$ from ABY3 [31], where three-party OT is required. Instead of using OT, we implement conversion with the Beaver trick [35]. Suppose that the parties access the precomputed conversion, and obtain precomputed random bit share $⟦c{⟧}^{\mathsf{B}}$ and $⟦c{⟧}^{\mathsf{A}}$ in the preprocessing phase. In the online phase, the parties compute and reconstruct the bit $e=b\oplus c$, followed by setting $⟦d{⟧}^{\mathsf{A}}=⟦1-c{⟧}^{\mathsf{A}}$ if $e=1$, and $⟦d{⟧}^{\mathsf{A}}=⟦c{⟧}^{\mathsf{A}}$ otherwise. Finally, the parties compute $⟦z{⟧}^{\mathsf{A}}=⟦(y-x)·d{⟧}^{\mathsf{A}}+⟦x{⟧}^{\mathsf{A}}$, where $⟦(y-x)·d{⟧}^{\mathsf{A}}$ can be computed by using RSS multiplication ${\prod }_{\mathsf{Mul}}$ between $⟦y-x{⟧}^{\mathsf{A}}$ and $⟦x{⟧}^{\mathsf{A}}$.

Since that random bit $⟦c{⟧}^{\mathsf{B}}$ and $⟦c{⟧}^{\mathsf{A}}$ are used as the one-time pad, the corrupted party can not learn any information about b. Even though the parties reveal the masked value e in the clear, our oblivious selection protocol is still secure. Looking ahead, the oblivious selection protocol is used to find the index of minimum component from vector and generate one-hot vector.

3.4. Secure Euclidean Squared Distance Protocol

From Section 2.3, it is important to compute the Euclidean distance between two points using Equation (1). However, the square root is unfriendly to construct the SS-based protocol since it is a nonlinear operation and has an expensive communication overhead. Note that $f\left(x\right)=\sqrt{x}$ is a monotonically increasing function, which can be replaced with $f\left(x\right)=x$. This would, however, not affect the results on clustering since the only thing we need here is the relationship of size between two values. In this way, we replace Equation (1) with the following Euclidean squared distance equation:

$$X_{ik}=\sum _{j=1}^d{\left(P_{ij}-{\varphi }_{kj}\right)}^2.$$

(4)

Similarly, Equation (3) can be replaced with the following Equation (5):

$$e=\sum _{k=1}^K{e}_k=\sum _{k=1}^K\sum _{j=1}^d{({\phi }_{kj}-{\varphi }_{kj})}^2.$$

(5)

Now, we focus on, given the share of vector $\mathit{x}=(x_1,\cdots ,x_d)$ and $\mathit{y}=(y_1,\cdots ,y_d)$, how to compute the share of Euclidean squared distance. We define this functionality as ${\mathcal{F}}_{\mathsf{ESD}}$. Observe that this can be done by first computing $\mathit{z}=\mathit{x}-\mathit{y}=(x_1-y_1,\cdots ,x_d-y_d)$, and then computing the inner product between $\mathit{z}$ and $\mathit{z}$. A naive way is for the parties to invoke RSS multiplication d times for d-dimension, consume d truncation pair to truncate the result, locally sum the result, and reshare. The communication is $O\left(d\right)$ ring elements, which is dependent on d, the size of vector.

In this work, we use the delay re-share technique [31] to reduce communication complexity for inner product, which is only communication $O\left(1\right)$ ring element and independent of d.

Let $(⟦s^{\prime }⟧,⟦s⟧)$ be the shares of truncation pair among the parties, where $s^{\prime }\in {\{0,1\}}^{\ell }$ with $2f$-bit precision, $s\in {\{0,1\}}^f$ with f-bit precision, such that $s=s^{\prime }/2^f$. The delay re-share technique can be explained as the following Equation (6):

$$⟦\mathbf{x}⟧·⟦\mathbf{y}⟧=\mathsf{reconstruct}((\sum _{j=1}^d⟦x_j⟧·⟦y_j⟧)+⟦s^{\prime }⟧)/2^f-⟦s⟧.$$

(6)

In a word, the parties first compute a 3-out-of-3 additive sharing of each $⟦x_i⟧⟦y_i⟧$ locally, then sum together, mask, truncate, and reshare the final result for 2-out-of-3 replicated sharing semantics. It is easy to see that this would only require communicate 1 ring element per party, which is independent of d. Furthermore, the truncation-induced error is only $2^{-f}$ with respect to the overall inner product.

The protocol for secure Euclidean squared distance is described in Figure 1.

Figure 1. Secure Euclidean squared distance protocol [31].

3.5. Secure Comparison Protocol

In order to obtain the relationship between two given values, we have to consider how to implement comparison when the values are shared. We define secure comparison functionality ${\mathcal{F}}_{\mathsf{LT}}$, which takes $⟦x⟧$ and $⟦y⟧$ as input, return boolean share $⟦b{⟧}^{\mathsf{B}}$, where bit $b=1$ if $x<y$, and $b=0$ otherwise.

Let $a=(x-y)$, then z can be computed by extracting the most significant bit (MSB) of a, i.e., $b=\mathsf{MSB}\left(a\right)$. Instead of using optimized parallel-prefix-adder-based bit extraction protocol from ABY3 [31], Wagh et al. [32] present a more efficient alternative method.

Recall that $⟦a⟧:=(a_1,a_2,a_3)$, and $a\equiv a_1+a_2+a_{3}mod{2}^{\ell }$, one has

$$b=\mathsf{MSB}\left(a\right)=\mathsf{MSB}\left(a_1\right)\oplus \mathsf{MSB}\left(a_2\right)\oplus \mathsf{MSB}\left(a_3\right)\oplus c,$$

(7)

where $c\in \{0,1\}$ is a carry bit from $(\ell -2)$-th index and can be computed by Equation (8).

$$c=\left\{\begin{array}{cc}1,\hfill & \mathrm{if}{2}^{\ell -1}\le a_1+a_2+a_3<2^{\ell },\hfill \\ 0,\hfill & \mathrm{Otherwise}.\hfill \end{array}\right.$$

(8)

From Equation (7), we observe that ${\mathcal{P}}_i$ can compute $\mathsf{MSB}\left(a_i\right)$ locally; thus, the main challenge here is how to compute c in a secure way. Note that Equation (8) is also equivalent to $c=\left((2a_1+2a_2+2a_3){\ge }_{?}{2}^{\ell }\right)$, which can be computed by wrap function. Wagh et al. [32] give us a solution for wrap function, denoted as ${\prod }_{\mathsf{WA}}$. We refer the reader to their work for correctness and security.

The secure comparison protocol is described in Figure 2. Furthermore, if one of the secrets is known to all parties, e.g., y, the share of y can be defined by letting $⟦y⟧:=(y+{\alpha }_1,{\alpha }_2,{\alpha }_3)$ without any interaction, where $({\alpha }_1,{\alpha }_2,{\alpha }_3)$ is zero sharing generated by PRF keys among the parties. Thus, the secure comparison protocol is also work. We denote this case as $⟦b{⟧}^{\mathsf{B}}\leftarrow {\mathcal{F}}_{\mathsf{LT}}(⟦x⟧,y)$.

Figure 2. Secure comparison protocol [32].

3.6. Secure Assignment Protocol

Recall that once the parties obtain the shares of the Euclidean squared distance between a point and all centorids, the following step is to assign this point to the closest cluster. This step can be abstracted as the question of, given K-dimension secret shared vector $⟦\mathit{v}⟧=(⟦v_1⟧,\cdots ,⟦v_K⟧)$, how to compute the secret shared one-hot vector $\mathit{e}$, where ‘1’ appears in the k-th component, and $k=\mathsf{arg}\mathsf{min}\{v_1,\cdots ,v_K\}$. We denote secure assignment functionality as ${\mathcal{F}}_{\mathsf{Assign}}$. The idea is straightforward. We implement secure assignment functionality with the following protocol ${\prod }_{\mathsf{Assign}}$ (see Figure 3).

Figure 3. Secure Assignment protocol.

3.7. Secure Division Protocol

As shown in Section 2.3, the parties need to recalculate the average vector of the points in each cluster in a secure way. Note that the average can be split by computing addition and division, where addition can be computed locally, hence the key point is to compute division. If the divisor is known to all parties, then division can be computed locally. However, observe that the divisor denotes the number of each cluster point and should be protected since full data privacy is required. Thus, the computation of division becomes difficult in our scenarios. We define secure division functionality ${\mathcal{F}}_{\mathsf{Div}}$ as follows: Given secret shared value $⟦a⟧$ and $⟦b⟧$ with $b\in {\mathbb{Z}}^{+}$, the parties compute the share $⟦c⟧$, such that $c=a/b$.

Instead of invoking division garbled circuit protocol, we implement division with numerical method. The numerical method is one of the most commonly used techniques for constructing SS-based secure protocol due to its efficiency. In this work, we implement division using Goldschmidt’s algorithm [36], which approximates the desired operation as a series of multiplication.

Let $w_0$ be an initial approximation of $1/b$, and ${ϵ}_0:=1-b·w_0$ be the relative error for the approximation $w_0$ such that $|{ϵ}_0|<1$. The Goldschmidt algorithm iteratively computes the following Equation (9):

$$c=\frac{a}{b}=a·\frac{1}{b}\approx a·w_0(1+{ϵ}_0)(1+{ϵ}_0^2)\cdots (1+{ϵ}_0^{2^{t-1}}),$$

(9)

where t is the number of iterations. When $t\to +\infty$, one has c converges to $a/b$ [36]. We set $t=2$, which is sufficient for a close approximation with our choice of fixed-point precision.

Catrina and Saxena [34] give us a good initial approximation of $w_0$ in the interval $[0.5,1)$, that is $w_0=2.9142-2b$. However, the major challenge of our work is that $b\in {\mathbb{Z}}^{+}$ does not belong to the interval $[0.5,1)$. In this work, we use the technique proposed by Wagh et al. [32]. The key insight here is that b is interpreted as a value with $(\alpha +1)$-bit fixed-point precision but not f-bit precision, where $\alpha \in \mathbb{Z}$ such that $2^{\alpha }\le b<2^{\alpha +1}$. Thus, one should first extract $\alpha$. The secure division protocol is described in Figure 4. From step 1 and step 2, we extract and reveal $\alpha$ to all parties, which only leaks the range of b and nothing else.

Figure 4. Secure division protocol [32].

4. Privacy-Preserving $\mathit{K}$-Means Clustering

We now give a formal description of our privacy-preserving K-means clustering protocol, following the basic building blocks outlined above.

4.1. Secret Distribution

Recall that all data are held by the data owners in the secure outsourced scenarios, thus secret distribution phase is completed by the data owner. This implies that all data are horizontal partitioned. As an optimized, instead of generating 2-out-of-3 replicated shares directly, the data owner generates 3-out-of-3 additive shares and then sends to three computing parties/servers, who reshare the shares and obtain valid 2-out-of-3 replicated shares. In this way, we reduce communication costs of the data owner by a half.

4.2. Cluster Initialization

As shown in Section 2.3, we need to initialize K centroids before the Lloyd’s steps. In this work, we assume that the data owner chooses K random points as the initialized centroids and secret share to three computing parties for simple. In this way, cluster initialization can be combined with secret distribution.

4.3. Lloyd’s Steps

4.3.1. Approximation of Euclidean Distance

Recall that we replace Euclidean distance with Euclidean squared distance, which is not affect the final result as we shown in Section 3.4. For $i\in \left[n\right]$ and $k\in \left[K\right]$, the parties first invoke ${\mathcal{F}}_{\mathsf{SED}}$ to compute the shares of Euclidean squared distance $X_{ik}$ between data point ${\mathbf{P}}_i$ and centroid ${\mathit{\varphi }}_k$, and then form $n\times K$ matrix $\mathbf{X}$.

4.3.2. Assigning Data Points to the Closest Cluster

For $i\in \left[n\right]$, we denote ${\mathbf{X}}_i$ as the i-th row vector of $\mathbf{X}$. In order to assign data point ${\mathbf{P}}_i$ to the closet cluster, the parties invoke our ${\mathcal{F}}_{\mathsf{Assign}}$ protocol as described in Section 3.6. The one-hot vector output $⟦{\mathbf{c}}_i⟧\leftarrow {\mathcal{F}}_{\mathsf{Assign}}(⟦{\mathbf{X}}_i⟧)$ indicates which cluster center this data point is assigned to. We form $K\times n$ cluster matrix $⟦\mathbf{C}⟧$, such that the i-th column of $\mathbf{C}$ is ${\mathbf{c}}_i$.

4.3.3. Recalculating Cluster Centers

Given cluster matrix $\mathbf{C}$, the parties need to recalculate each centroid. Observe that for each $k\in \left[K\right]$, the k-th cluster center has $D_k={\sum }_{i=1}^n{\mathbf{C}}_{ki}$ points exactly. Instead of computing ${\mathit{\phi }}_k=\frac{{\mathbf{C}}_k·\mathbf{P}}{D_k}$ separately, one can first compute $\mathbf{M}=\mathbf{C}·\mathbf{P}$, and then compute ${\mathit{\phi }}_k=\frac{{\mathbf{M}}_k}{D_k}$, where ${\mathbf{M}}_k$ is the k-th row of $\mathbf{M}$. Given secret shared matrix $⟦\mathbf{C}⟧$, the new centroid matrix $\mathit{\phi }$ can be computed by vectorized multiplication technique [31] and secure division protocol (as described in Section 3.7). Furthermore, the parties also compute the shares of Euclidean squared distance $e_k$ between ${\mathit{\phi }}_k$ and ${\mathit{\varphi }}_k$ for checking the stopping criterion.

4.3.4. Checking the Stopping Criterion and Updating Centroids

In order to check the stopping criterion, the parties first locally compute $e={\sum }_{k=1}^K{e}_k$, and then compare with a given small error $\epsilon$, stop the criterion if $e<\epsilon$, otherwise update $\mathit{\phi }$ and continue next round. This can be done by invoking secure comparison protocol ${\prod }_{\mathsf{LT}}$. The only reveal message is $b=\left(e{<}_{?}\epsilon \right)$, which does not affect full data privacy.

4.4. Main Construction

The secure K-means protocol is described in Figure 5. According to the definition of full data privacy, the information about the intermediate centroids, cluster assignments, and cluster sizes should be protected. From Figure 5, we can see that the only information we leak is the range of $D_k$ and nothing else. Therefore, our construction provides full data privacy.

Figure 5. Our Privacy-preserving K-means Protocol.

5. Security Analyses

Our protocol follows the universally composable framework [29] and provides security against a single corrupted party under the semi-honest model. The universally composable framework guarantees the security of arbitrary composition of different protocols. Therefore, we only need to prove the security of individual protocols.

Theorem 1.

${\prod }_{\mathsf{OS}},{\prod }_{\mathsf{SED}},{\prod }_{\mathsf{LT}},{\prod }_{\mathsf{Assign}},{\prod }_{\mathsf{Div}}$ securely realizes ${\mathcal{F}}_{\mathsf{OS}},{\mathcal{F}}_{\mathsf{SED}},{\mathcal{F}}_{\mathsf{LT}},{\mathcal{F}}_{\mathsf{Assign}},{\mathcal{F}}_{\mathsf{Div}}$, respectively, in the presence of one semi-honest corrupt party under the hybrid model.

Proof of Theorem 1.

We list the security of those protocols as follows:

Security for ${\prod }_{\mathsf{OS}}$: This can be reduced to the security of one-time pad and RSS multiplication ${\prod }_{\mathsf{Mul}}$.

Security for ${\prod }_{\mathsf{SED}}$: This protocol is based on the vectorized multiplication protocol [31], which has been proven secure under the semi-honest model.

Security for ${\prod }_{\mathsf{LT}}$: This can be reduced to the security of wrap function protocol, which has been proven secure in [32].

Security for ${\prod }_{\mathsf{Assign}}$: This can be reduced to the security of ${\prod }_{\mathsf{LT}}$ and ${\prod }_{\mathsf{OS}}$.

Security for ${\prod }_{\mathsf{Div}}$: This protocol has been proven secure in [32]. □

Theorem 2.

The protocol in Figure 5 securely computes the K-means clustering under the semi-honest model, given the ideal ${\mathcal{F}}_{\mathsf{SED}},{\mathcal{F}}_{\mathsf{Assign}},{\mathcal{F}}_{\mathsf{Mul}},{\mathcal{F}}_{\mathsf{LT}}$, and ${\mathcal{F}}_{\mathsf{Div}}$ functionalities, respectively.

Proof of Theorem 2.

We exhibit a simulator $\mathsf{Sim}$ for simulating a corrupt party ${\mathcal{P}}_1$. The simulator for ${\mathcal{P}}_2$ and ${\mathcal{P}}_3$ should be the same as ${\mathcal{P}}_1$.

$\mathsf{Sim}$ simulates the view of corrupt ${\mathcal{P}}_1$, which consists of his input/output and received messages, and proceeds as follows:

• Calls ${\mathcal{F}}_{\mathsf{SED}}$ simulator ${\mathsf{Sim}}_{{\mathcal{F}}_{\mathsf{SED}}}(⟦{\mathbf{P}}_i⟧,⟦{\mathit{\varphi }}_k⟧)$ to simulate step 1, then appends its output to the view;
• Calls ${\mathcal{F}}_{\mathsf{Assign}}$ simulator ${\mathsf{Sim}}_{{\mathcal{F}}_{\mathsf{Assign}}}(⟦{\mathbf{X}}_i⟧$ to simulate step 2, then appends its output to the view;
• Calls ${\mathcal{F}}_{\mathsf{Mul}}$ simulator ${\mathsf{Sim}}_{{\mathcal{F}}_{\mathsf{Mul}}}(⟦\mathbf{C}⟧,⟦\mathbf{P}⟧)$ to simulate step 3, then appends its output to the view;
• Calls ${\mathcal{F}}_{\mathsf{Div}}$ simulator ${\mathsf{Sim}}_{{\mathcal{F}}_{\mathsf{Div}}}(⟦{\mathbf{M}}_k⟧,⟦D_k⟧)$ to simulate step 3(b), then appends its output to the view.
• Calls ${\mathcal{F}}_{\mathsf{LT}}$ simulator ${\mathsf{Sim}}_{{\mathcal{F}}_{\mathsf{LT}}}(⟦e⟧,\epsilon )$ to simulate step 4(b), then appends its output to the view.

We argue the indistinguishability of the produced transcript from the execution of real world. At first, we formally show the simulation by proceeding the sequence of hybrid transcripts $T_0$, $T_1$, $T_2$, $T_3$, $T_4$, $T_5$, where $T_0$ is the real view of $\mathcal{A}$, and $T_5$ is the output of $\mathsf{Sim}$.

Hybrid 1. Let $T_1$ be the same as $T_0$, except the ${\mathcal{F}}_{\mathsf{SED}}$ execution is replaced with running the simulator ${\mathsf{Sim}}_{{\mathcal{F}}_{\mathsf{SED}}}(⟦{\mathbf{P}}_i⟧,⟦{\mathit{\varphi }}_k⟧)$. Because ${\prod }_{\mathsf{SED}}$ has been proven secure, thus ${\mathsf{Sim}}_{{\mathcal{F}}_{\mathsf{SED}}}$ is guaranteed to produce output indistinguishable from the execution of real world. Therefore, $T_1$ and $T_0$ are indistinguishable.

Hybrid 2. Let $T_2$ be the same as $T_1$, except the ${\mathcal{F}}_{\mathsf{Assign}}$ execution is replaced with running the simulator ${\mathsf{Sim}}_{{\mathcal{F}}_{\mathsf{Assign}}}(⟦{\mathbf{X}}_i⟧)$. This functionality outputs the share of the Euclidean squared distance between ${\mathbf{P}}_i$ and ${\mathit{\varphi }}_k$, which does not reveal any information about the result. Moreover, the output of ${\mathsf{Sim}}_{{\mathcal{F}}_{\mathsf{Assign}}}$ is indistinguishable from the execution of real world, thus $T_2$ and $T_1$ are indistinguishable.

Hybrid 3. Let $T_3$ be the same as $T_2$, except the ${\mathcal{F}}_{\mathsf{Mul}}$ execution is replaced with running the simulator ${\mathsf{Sim}}_{{\mathcal{F}}_{\mathsf{Mul}}}(⟦\mathbf{C}⟧,⟦\mathbf{P}⟧)$. Because ${\prod }_{\mathsf{Mul}}$ has been proven secure, ${\mathsf{Sim}}_{{\mathcal{F}}_{\mathsf{Mul}}}$ is guaranteed to produce output indistinguishable from the execution of the real world. Therefore, $T_3$ and $T_2$ are indistinguishable.

Hybrid 4. Let $T_4$ be the same as $T_3$, except the ${\mathcal{F}}_{\mathsf{Div}}$ execution is replaced with running the simulator ${\mathsf{Sim}}_{{\mathcal{F}}_{\mathsf{Div}}}(⟦{\mathbf{M}}_k⟧,⟦D_k⟧)$. This is because ${\prod }_{\mathsf{Div}}$ has been proven secure, and its output is the share, which is indistinguishable from the pseudo-randomness. In other words, the output of ${\mathsf{Sim}}_{{\mathcal{F}}_{\mathsf{Div}}}$ is indistinguishable from the execution of real world. Thus, $T_4$ and $T_3$ are indistinguishable.

Hybrid 5. Let $T_5$ be the same as $T_4$, except the ${\mathcal{F}}_{\mathsf{LT}}$ execution is replaced with running the simulator ${\mathsf{Sim}}_{{\mathcal{F}}_{\mathsf{LT}}}(⟦e⟧,\epsilon )$. The output is the share, which does not reveal any information about the data points. In addition, the output of ${\mathsf{Sim}}_{{\mathcal{F}}_{\mathsf{LT}}}$ is indistinguishable from the execution of real world. Thus $T_5$ and $T_4$ are indistinguishable.

In summary, $T_0$ and $T_5$ are indistinguishable, which is end of the proof. □

6. Experiments

6.1. Experimental Setup

We implement our privacy-preserving clustering protocol and report the experimental results in this section. All experiments are executed on Ubuntu 22.04 LTS with Intel(R) Xeon(R) Gold 5222 CPU @3.41GHz and 256 GB RAM. All parties run in the same network and the connection is simulated using the Linux tc command. The LAN setting has 0.2 ms round-trip latency and 5 Gbps network bandwidth, while the WAN setting has 20 ms round-trip latency and 400 Mbps network bandwidth. We implement our protocol with the C++ open source framework FALCON (https://github.com/snwagh/falcon-public (accessed on 18 June 2022)) [32].

In all experiments, we assume the orginal data has been normalized in the same level. For the public parameters, we set bit-length $\ell =64$, fixed-point precision $f=13$, and the number of iteration $t=2$. Table 2 summarizes the real datasets used in our experiments. Furthermore, we also compare with Mohassel et al. [27] in the self-generated dataset.

Table 2. Descriptions of the datasets we used in experiments, where n is the number of data points, K is the number of clusters, and d is the dimension. We also report the accuracy of different datasets, if the ground truth model of dataset exists.

Dataset	n	K	d	Accuracy
Iris	150	3	4	92.67%
arff	1000	4	2	98.20%
Self-generated	$\{10,000,100,000\}$	$\{2,5\}$	$\{5,10,15,20\}$	—

6.2. Accuracy

In order to evaluate the accuracy of clustering classification for the real dataset, we usually compare it to the ground truth model. We downloaded the ground truth model of Iris and arff from Github repository (https://github.com/deric/clustering-benchmark (accessed on 18 June 2022)). Note that the standard K-means clustering is sensitive to the initialized centroids; thus, we ran the algorithm many times with different initialized centroids and take the best result as the global optimal solution.

We used 2D dataset arff and 4D dataset Iris for evaluating accuracy and report the experimental results in Table 2. For a visual comparison, the experimental result of dataset arff is shown in Figure 6. Compared to the ground truth model of dataset arff, we reached 98.20% accuracy in our privacy-preserving model. For dataset Iris, we reached 92.67% accuracy in our privacy-preserving model. Both accuracy results are the same as the plaintext algorithm; hence, our privacy-preserving protocol is feasible.

Figure 6. Comparison of accuracy for ground truth, plaintext, and privacy-preserving model for 2D dataset arff. Our privacy-preserving model reaches the same accuracy as the plaintext model. The accuracy is 98.20% compared to the ground truth model.

Recall that our secure division protocol adopts the numerical method and the result of the secure multiplication protocol needs to be truncated; thus, the privacy-preserving result $\mathit{\phi }$ is only approximate to the plain result but not the exact result. In fact, our experiment shows that the relative error is about ${10}^{-2}$, which means this part has a negligible impact on model accuracy compare with the plaintext algorithm. In other words, our privacy-preserving protocol is feasible, even if $\mathit{\phi }$ is only approximate to the truth value. For the large-scale dataset, we argue that the relative error can be improved by taking bigger public parameters ℓ, f, and t. However, it will require more runtime and communication cost. There is a trade-off between runtime and accuracy of the division. We do not consider the accuracy of experiment in the following section.

6.3. Runtime and Communication

In this section, we focus on the total communication cost and runtime of our privacy-preserving protocol. We ran each experiment five times and computed the average of wall clock runtime as the reported runtime. We report the experimental results in Table 3. Note that iteration T depends on the initialized clustering centroids; hence, we set it to a fixed value in this experiment (say, $T=10$).

Table 3. The comparison of wall clock runtime and communication cost with different dimensions in the self-generated dataset both with LAN and WAN setting, where n is the number of data points, K is the number of clusters, d is the dimension, and the iteration $T=10$.

Parameters			Runtime		Comm. (MB)
n	K	d	LAN (s)	WAN (min)
10,000	2	5	63.8160	134.3737	37.1516
		10	63.8020	134.3832	37.2976
		20	63.6132	134.3962	37.5896
	5	5	160.9406	336.0150	134.2790
		10	161.2164	336.1254	134.6440
		20	161.3586	336.2652	135.3740
100,000	2	5	474.7150	1336.1333	370.1520
		10	473.9687	1336.1968	370.2980
		20	475.1037	1336.2415	370.5900

As shown in Table 3, the runtime and communication overhead are independent from the dimension d; this is because we enjoy the benefit from the vectorized multiplication and delay re-share technique [31].

Even though $n=100,000$, our scheme lasted less than 8 min and communicated less than 400 MB in total under the LAN setting. Although the overall communication cost is low, we observe that the runtime of our scheme is not good at the WAN setting. The reason for this is because WAN has low-bandwidth and high-latency, while the secret-sharing-based schemes usually have much communication rounds that is bad for this setting. Thus, we argue that our privacy-preserving scheme is practical when the network is fast, high-bandwidth, and low-latency. We estimate that our privacy-preserving protocol can be used to deal with datasets of million points in an acceptable time (for example, within 2 h for clustering one million points to 2 groups in the LAN setting).

6.4. Comparison with Mohassel et al. [27]

Recall that Mohassel et al. [27] also provides full data privacy guarantees (see Table 1); thus, we also compare to their work with the self-generated dataset in our experiment environment. We downloaded the code of Mohassel et al. [27] from their Github repository (https://github.com/osu-crypto/secure-kmean-clustering (accessed on 18 June 2022)). In order to save time, all experiments are only considered under the localhost setting, which has 0.027ms round-trip latency and 41.1Gbps network bandwidth. We ran each protocol five times and report the average of wall clock runtime and communication costs. Instead of running the code of Mohassel et al. [27] in every iteration, we measured its runtime for one round iteration and multiplied by the number of iterations T to save time.

Table 4 presents the computation cost and communication cost of our protocol compared with [27]. Our experimental results demonstrate that the computation costs of our protocol is about 16.5×–25.2× faster than [27], and the communication cost is about 63.8×–68.0× less than [27]. This is because their construction relies heavily on garbled circuit and oblivious transfer, while our scheme is only based on replicated secret sharing.

Table 4. Comparison with Mohassel et al. [27] in large-scale self-generated datasets under the localhost setting, where n is the number of data points, K is the number of clusters, and dimension $d=2$.

Parameters			Runtime (min)			Communication (MB)
n	K	T	[27]	This Work	Improved Factor	[27]	This Work	Improved Factor
${10}^4$	2	10	1.77	0.09	19.5×	2377	37	64.1×
		20	3.36	0.19	17.9×	4733	74	63.8×
	5	10	4.69	0.28	16.5×	9121	134	68.0×
		20	9.46	0.56	16.9×	18220	268	68.0×
${10}^5$	2	10	15.33	0.74	20.8×	23731	370	64.1×
		20	29.74	1.15	20.4×	47262	740	63.9×
	5	10	46.51	1.85	25.2×	91128	1339	68.0×
		20	91.61	3.65	25.1×	181867	2678	67.9×

7. Conclusions and Future Work

In this work, we presented an efficient RSS-based privacy-preserving K-means clustering scheme over ${\mathbb{Z}}_{2^{\ell }}$ under the semi-honest model. Our scheme provides full data privacy. The experiment report shows that our protocol is highly efficient and practical, as well as suitable for large-scale clustering tasks when the network is fast. Therefore, we argue that our scheme is suitable for secret-sharing-based secure outsourced computation.

The next direction of future work can extend our scheme to the malicious adversarial setting, which will be a non-trivial problem. This is because malicious adversary may not follow protocol specifying and deviate arbitrarily in any phase. For example, the adversary makes the corrupted party sends incorrect messages, such that it can break the correctness of protocol. Therefore, we should ensure that the sending message of the parties are correct. A promising direction for this case is to introduce the SPDZ protocol [37], where the correctness of the sending message can be protected by using message authentication codes (MACs).