Privacy-Preserving K-Nearest Neighbors Training over Blockchain-Based Encrypted Health Data

Abstract

Numerous works focus on the data privacy issue of the Internet of Things (IoT) when training a supervised Machine Learning (ML) classifier. Most of the existing solutions assume that the classifier’s training data can be obtained securely from different IoT data providers. The primary concern is data privacy when training a $K$-Nearest Neighbour ($K$-NN) classifier with IoT data from various entities. This paper proposes secure $K$-NN, which provides a privacy-preserving $K$-NN training over IoT data. It employs Blockchain technology with a partial homomorphic cryptosystem (PHC) known as Paillier in order to protect all participants (i.e., IoT data analyst C and IoT data provider P) data privacy. When C analyzes the IoT data of P, both participants’ privacy issue arises and requires a trusted third party. To protect each candidate’s privacy and remove the dependency on a third-party, we assemble secure building blocks in secure $K$-NN based on Blockchain technology. Firstly, a protected data-sharing platform is developed among various P, where encrypted IoT data is registered on a shared ledger. Secondly, the secure polynomial operation (SPO), secure biasing operations (SBO), and secure comparison (SC) are designed using the homomorphic property of Paillier. It shows that secure $K$-NN does not need any trusted third-party at the time of interaction, and rigorous security analysis demonstrates that secure $K$-NN protects sensitive data privacy for each P and C. The secure $K$-NN achieved $97.84\%$, $82.33\%$, and $76.33\%$ precisions on BCWD, HDD, and DD datasets. The performance of secure $K$-NN is precisely similar to the general $K$-NN and outperforms all the previous state of art methods.

1. Introduction

At present, smart cities include more innumerable superior IoT infrastructures [1] to manage their component efficiently [2]. A tremendous volume of information accumulated from numerous IoT devices stationed in different city areas, such as medical health, agriculture, transportation, and energy transmission [3]. A large volume of reforms prompted by ML technology were proposed to handle the issues emerging from processing obligations of IoT data [4]. $K$-means [5] and $K$-NN [6] are pre-eminent unsupervised and supervised learning models, respectively, that can effectively implement data classification amid all ML models [7]. Therefore, these ML models have been used in various specialties to answer real-world classification dilemmas in IoT-enabled smart health. Considering the synopsis of individual fitness and healthcare records observed by wearable IoT sensors, Refs. [8,9,10] could be fed to a $K$-NN classifier for fitness analysis. $K$-NN clustering classifier is also deployed in the field of network interference apprehension to recognize abnormalities from a group of traffic data derived from interactions among IoT devices [11].

The structure of supervised ML classifiers such as $K$-NN or Support Vector Machine (SVM) is denoted as the training stage, which trains a specific classifier to find underlying patterns with a defined output variable. The higher the significance of training samples increased, the more the performance of the ML classifier enhanced [12,13]. An individual such as a network provider or a hospital entity often owns the dataset required for training. It is generally bounded in courses of sample distinctiveness and amount. It is necessary to train the ML model classifiers with an efficient and effective mechanism using a unification of sample sets collected from various entities. Various entities are generally opposed to sharing their datasets for training as there are many safety concerns in terms of ownership, integrity, and data privacy:

• Most of the training phases manipulate intimate data samples such as medical data reported from clinical wearable IoT devices, resulting in the leakage of private or sensitive and confidential information at the time of training tasks.
• Latent invaders may cause unauthorized modification of data records by altering or tampering at the time of the data sharing process, resulting in an inaccurate classification of the ML model.
• The data provider may lose authority, and replication of the shared datasets may occur as datasets are available to the associates.

To secure the data privacy issue of individual data providers, most of the present solutions [14,15,16,17] focused on cryptography and differential privacy. Those solutions assumed that the data required for training could be obtained securely from various data providers to classify and analyze. Issues of ownership and data integrity were focused on trivially. However, solutions are invalid due to potential attacks, for most cases, in reality. This paper uses the Blockchain technology to build a reliable data-sharing terrace, which can cover the gap between realistic confinement and typical prediction. In general, a shared filing scheme intended to permit the distribution of tamper-proof records among various individuals is called a Blockchain [18]. Auditing is enabled on Blockchain for immutable records, which confirms the ownership of recorded data.

Some works are focused on privacy-preserving $K$-NN computation, search, query, and classification [19,20,21,22,23,24,25,26,27,28]. None of these works used Homomorphic encryption with Blockchain in order to secure sensitive information. To consolidate Blockchain into ML training method is laborious but encouraging. The first difficulty is to outline a suitable training data format convenient for adjusting on Blockchain to secure each data provider’s privacy. The second difficulty is to develop the training algorithm that establishes an accurate $K$-NN classifier using the Blockchain’s recorded data and secures sensitive information. Secure SVM [29] was proposed to address the problems mentioned earlier. In the proposed method, they employed a Blockchain-based, privacy-preserving training algorithm for the SVM using encrypted data of IoT devices from Smart Cities. A public-key cryptosystem is applied by Secure SVM to protect the privacy of the data, which are encrypted by the private keys of data providers. However, Secure SVM requires too many calculations, comparison, time, and space in order to analyze the health data. In most medical health research, $K$-NN outperformed SVM in terms of performance and time complexity [30,31,32].

To handle the above challenges, we propose secure $K$-NN, a privacy-preserving $K$-NN schema based on Blockchain and encrypted data of IoT devices. A public-key cryptosystem called Paillier has been employed to shield the IoT data privacy, which is encrypted by the own private key of the respected data providers. Paillier is an additive homomorphic cryptosystem (HC), which is very efficient in terms of time complexity for encryption and decryption than any other algorithm, such as Rabin, RSA, and Goldwasser-Micali [33]. Handling encrypted data could be a problem because of the tremendous amount of intercommunications. However, $K$-NN can handle these circumstances as there is no separate optimization algorithm. $K$-NN has essential procedures such as polynomial operations and comparison. We design secure building blocks SPO (addition and subtraction), SC, and SBO by using the homomorphic properties of Paillier for secure $K$-NN. With the above building blocks, all iterations of secure $K$-NN do not need any trusted third party at the time of interaction, which significantly lessens the risk of a data breach.

The main contribution of this paper is as follows.

• To establish protected and trustworthy IoT data sharing, Blockchain technology is employed. All the IoT data are encrypted locally by the own private key of the respected data provider. The encrypted data are recorded on a Blockchain by uniquely formatted transactions.
• We designed protected building blocks, such as SPO (addition, subtraction), SBO, and SC using the PHC, i.e., Paillier, and developed a secure $K$-NN training algorithm. There is no requirement for a trusted third-party.
• Rigorous analysis has been done to prove that the secure $K$-NN can protect data privacy at the time of training, achieve similar accuracy as general $K$-NN and outperform all the previous state of the art method.

The rest of the paper is articulated as follows. Related work and preliminaries are discussed in Section 2 and Section 3, respectively. The system overview is presented in Section 4. Section 5 summarized the proposed method. Analyzation of confidentiality issues and evaluation of the proposed scheme are explained in Section 6 and Section 7, respectively. Finally, in Section 8, the paper is concluded.

2. Related Work

Supervised learning contains two phases: the training phase, where the ML model learns from a given set of labeled specimens, and the classification phase, where labels are the result for a given sample with maximum possibility. Thus, current research on privacy-preserving ML can be divided into two sections, namely privacy-preserving ML training and privacy-preserving ML classification.

2.1. Privacy-Preserving ML Training

In most cases, multiple parties are involved when training an ML model, which results in the privacy issue of the IoT data. The main goal is to protect the data provider’s IoT data from being discovered by others at the time of training an ML model. During the last decade, numerous work have been done on this category [15,34,35,36,37,38,39,40] and our work focus here.

There are many methods used to secure data privacy in the publishing stage [41,42,43], but the most common approach is Differential privacy (DP) [15]. It assures the protection of published data by combining vigilantly computed distress to the fundamental data. The DP-based deep learning method was proposed by Abadi [15]. They developed a system to jointly train a neural network with preserving the sensitive information of their datasets. DP-based solutions can achieve more excellent computational performance. These solutions can execute calculations over plain text data. There are some limitations also: Firstly, due to perturbations, the quality and integrity of the training data are reduced significantly. Secondly, each training data’s sensitive information is publicly exhibited, so only disruptions are not enough to effectively secure data privacy. The privacy budget parameter is inversely proportional to the model accuracy but directly proportional to data privacy protection.

ML training achieves reliability and provides privacy guarantee on encrypted IoT data using homomorphic encryption, and it allows calculations on ciphertext and preserves the correctness of the data. In order to train various ML models, different protected methods based on HC have been proposed, such as SVM [29,34], Logistic Regression [35,36], Decision Tree [38], and Naive Bayes [37]. Secure protocols [34] have been developed for secure addition and subtraction depending on Paillier, which results in a Secure SVM training algorithm. Due to the computational limitation of Paillier, the authors developed an authorization server which worked as a trusted third-party.

PHC method can reach higher data privacy with more efficiency than the Fully homomorphic encryption (FHE) system. The PHC is much more practical than FHE for the addition and multiplication operations. Complicated calculations can be employed with a trusted third-party [34]. Without that, the model will be inaccurate due to the approximation of complex equations with an individual computational operation [39,40]. On the other hand, the calculations of FHE is costly in terms of time and space complexity. Therefore, existing uses of FHE is prohibitive from the scenario of encryption and forecast. As a result, they are unrealistic in terms of application.

2.2. Privacy-Preserving ML Classification

Usually, two different parties interact in a classification as a service scenario. One holds the data sample, and the other holds the ML model. It is not safe to reveal sensitive data to an unreliable ML model owner for a data owner who wants to know the classification result. On the other hand, the model owner may decline to share the classification result as the asset value is too high for the service provider.

Some existing solutions are [14,17,44,45,46] for developing an effective solution in order to secure the privacy of both parties. A method was proposed by Wang et al. [45] to classify encrypted images based on multi-layer learning. The authors used a public classifier but considered that the image data should be secure. A privacy-preserving nonlinear SVM method was proposed by Zhu et al. [46] for online-based medical prediagnosis. The authors can protect both individual information of the health record and the SVM model with their design. Rahulamathavan et al. [17] proposed a privacy-preserving SVM data classification system. It can securely classify multiclass datasets. In this schema, the client input data samples are anonymous to the server. At the same time, clients are also unaware of the server-side classifier during the classification process. A group of classification protocols was developed using the HC techniques for employing ML’s simple classifiers on encrypted data, such as Naive Bayes, hyperplane decision, and decision trees [14,44].

All the above studies employed standard ML classifiers and developed building blocks to assemble a privacy-preserving classification method. The calculations at the time of training an ML classifier are much complex compared to the classification phase. Those building blocks might be useless due to the complexity of the training algorithm.

2.3. The Novelty of This Paper

Earlier research on secure $K$-NN focused on any specific domain such as data confidentiality, secure query, secure search, secure computation, and secure classification [19,20,21,22,23,24,25,26,27,28]. None of them keep track of all the transactions, and most of them considered that the $K$-NN model is already trained. In this study, a partial cryptosystem known as Paillier is employed with Blockchain technology to handle the issues related to ownership, integrity, and data privacy at the time of training $K$-NN classifiers based on the data from various data providers. To be specific, all IoT data from individual data providers is encrypted using Paillier then registered on a distributed ledger. Any data analyst can obtain encrypted data by interacting with the respective data provider and train the $K$-NN classifier. The data analysts can never obtain the plaintext of IoT data on the Blockchain. The secure protocol of operation in $K$-NN is developed to conduct training tasks with encrypted data, i.e., SPO (addition/subtraction), SBO, and SC. A privacy-preserving $K$-NN training algorithm, secure $K$-NN, is proposed based on this building block, and there is no need for a trusted third party. The secure $K$-NN can train $K$-NN classifiers without the loss of accuracy as the training is based on Paillier.

Two well-known security definitions are used as security goals: secure two-party computation [47] and modular sequential composition [48]. The propose method illustrates that the individual data provided is inadequate to acquire any information about other data provider’s data. Simultaneously, the model parameters of data analysts are secure from the knowledge of any data providers during the training process.

3. Preliminaries

This section describes all notations, background ideas, and related technologies of this research.

3.1. Notation

A dataset D, which consists of m records, where $x_i$ and $y_i$ is the i-th record in D and $l_i$ is the label of the corresponding $x_i$ and $y_i$. Define d and $(c_{x_i},c_{y_i})$ as two relevant parameters of $K$-NN. In this paper, we use a PHC named Paillier as the cryptosystem, and let $\left[\right[m\left]\right]$ represent the encryption of message under Paillier. Notations are summarized in

Table 1. Notations.

Signs	Interpretations	Signs	Interpretations
D	dataset	d and $(c_{x_i},c_{y_i})$	model parameters
d	distance	$(c_{x_i},c_{y_i})$	initial centroid
$x_i,y_i$	$i^{th}$ record in dataset	t	threshold
$\varphi \left(N\right)$	euler phi-function	$l_i$	class label
m	size of the dataset D	$\left[\right[m\left]\right]$	the encryption of
A	Labeled data’s array	-	m under Paillier

.

3.2. Homomorphic Cryptosystem

Cryptosystems are mainly based on three algorithms: Generation of key (KeyGen), data encryption (Enc), and data decryption (Dec). In public-key cryptosystems, a pair of keys (PK; SK) is used, such as for encryption and decryption public key (PK) and private key (SK) are used respectively. A cryptosystem property, which can map the operations over ciphertext to the corresponding plaintext without being aware of the decryption key, is known as Homomorphic. Definition 1 describes the homomorphic property of the cryptosystem.

Definition 1.

(homomorphic [33]) A method of public-key encryption (Gen, Enc, Dec) can be homomorphic only if for all n and all (PK; SK) output by $Gen$ ($1^n$), it is possible to define groups $\mathbb{M},\mathbb{C}$ (depending on PK only) such that:

• $\mathbb{M}$ and $\mathbb{C}$ are the message space and all ciphertexts outcome respectively by $En{c}_{pk}$ are elements of $\mathbb{C}$.
• $De{c}_{sk}\left(o(c_1,c_2)\right)=\sigma (m_1,m_2)$ is held for any $m_1,m_2\in \mathbb{M}$, any $c_1$ output by $En{c}_{pk}\left(m_1\right)$, and any $c_2$ output by $En{c}_{pk}\left(m_2\right)$.

A partial homomorphic cryptosystem known as Paillier is being used in the proposed schema. It is a public key cryptography method with the partial homomorphic property as it allows only two operations, secure addition and subtraction. Let p and q are n-bit primes, $N=pq$. N and $(N,\varphi (N\left)\right)$ (Let, $N>1$ be an integer. Then $Z_N^{*}$ is an abelian group under multiplication modulo N. Define $\varphi \left(N\right)$$\underset{=}{def}$$\mid Z_N^{*}\mid$, the order of the group $Z_N^{*}$.) are the public key and private key, respectively. $c:=\left[\left[{(1+N)}^m{r}^{N}mod{N}^2\right]\right]$ is the encryption function in Paillier, where $m\in {\mathbb{Z}}_N$ and $m:=\left[[\frac{[c^{\varphi \left(N\right)}mod{N}^2-1]}{N}\times \varphi {\left(N\right)}^{-1}modN]\right]$ is the decryption function. More details about Paillier is explained in [32].

3.3. $K$-Nearest Neighbors ($K$-NN)

$K$-nearest neighbors ($K$-NN) [6] clustering is a type of supervised ML algorithm, used for classification and predictive regression problems. There is no specific training phase and uses whole training data [49] during classification because of that, it is called a lazy learning algorithm. It does not assume anything about the underlying data. Distance d needs to be calculated in order to find the designated centroid $(c_{x_j},c_{y_j})$. There are different methods to find the distance in $K$-NN algorithm, i.e., Euclidean distance $d_e$ (Equation (1)), Manhattan distance $d_m$ (Equation (2)), Cosine distance $d_c$ (Equation (3)), etc. methods. In this study, we will use Manhattan distance $d_m$. Let, $(x_1,y_1),(x_2,y_2)\dots ,(x_m,y_m)\in D$. Algorithm 1 illustrate the entire process.

$$d_e=\sqrt{{(c_{x_j}-x_i)}^2+{(c_{y_j}-y_i)}^2}$$

(1)

$$d_m=\left|(c_{x_j}-x_i)\right|+\left|(c_{y_j}-y_i)\right|$$

(2)

$$d_c=\frac{(c_{x_j}\times x_i)+(c_{y_j}\times y_i)}{\sqrt{x_i^2+c_{x_j}^2}\times \sqrt{y_i^2+c_{y_j}^2}}$$

(3)

Algorithm 1: Basic $K-$NN

3.4. Blockchain System

Blockchain is a public and shared ledger, consists of a list of blocks. It is developed in cryptocurrency systems for registering transactions such as Bitcoin. It ensures secure transactions among untrusted participants. Various Blockchain platforms are: Ethereum, HyperLedger, etc., have been employed in different real-life sectors. According to the access restriction of users in Blockchain, Blockchain platforms are classified into consortium Blockchains, private Blockchains, and public Blockchains.

There are various advantages of Blockchain:

• Decentralized: It is developed on a peer-to-peer network as a shared ledger, and there is no requirement of a trusted third-party.
• Tamper-proof: Consensus protocols are employed by Blockchain, such as Proof-of-Work (PoW). Thus, Data manipulation is impractical.
• Traceability: The rest participants can easily verify the transactions between two parties in a Blockchain system.

Despite having many advantages, Blockchain has the vulnerability of data privacy to skilled attackers. Initially, all transactions are registered as plain texts in blocks, which exposes the transaction’s vital information to other participants, and adversaries [50]. Therefore, privacy and security issues must be handled cautiously when using Blockchain in terms of data sharing platform.

4. Problem Description

This segment illustrates the issues of secure $K$-NN training across encrypted IoT data accumulated from various parties, which includes the system design, threat type, and design purposes.

4.1. System Design

A data flow IoT ecosystem is developed and shown in Figure 1, including IoT devices, data providers, the Blockchain platform, and data analysts.

• ZigBee, 3^rd generation (3G)/4^th generation (4G), and Wireless Fidelity (WiFi) are examples of the wired or wireless network through which IoT devices can sense and transmit valuable information, including medical data, smart cities, etc. In this study, due to the lack of computational capabilities, IoT devices will not participate in the data sharing and analysis processes.
• Data providers gather all the data from IoT devices within their range. All the data comprises sensitive information, so all the data are encrypted using partially homomorphic encryption by the data provider and registered in a Blockchain.
• To gather the encrypted IoT data from all data providers, the Blockchain-based IoT platform serves as a distributed database, where protocols are maintained, and all data are recorded in a shared ledger. The built-in consensus mechanism ensures the sharing of IoT data in a secure and tamper-proof way.
• IoT data analysts intend to get a rooted perspicacity within the data registered in the Blockchain-based platform by using the existing analyzing techniques. Data analysts will obtain encrypted data from corresponding data providers in order to train the $K-$NN classifiers.

4.2. Threat Type

Various latent threats exist over individual entities and at the time of their interactions, according to the system model description in Figure 1. This study focuses on the threats related to data privacy throughout the interaction between the data provider and data analysts. It is assumed that the data analyst is a curious but honest foe. To be more specific, the data analyst is honest for maintaining the protocol of predesigned ML training and curious regarding the contents of the data. Moreover, the data analyst strives to acquire further knowledge by analyzing the intermediate data at the time of computation on encrypted data.

The subsequent models of threat are considered based on the data analyst’s collected vital information with various attack inclinations mostly employed in the literature [51,52]. On the other hand, the data provider may also try to identify the data analyst’s model parameter from the intermediate data.

• Recognized Ciphertext Model. The data analyst can merely obtain the encrypted IoT data registered in the Blockchain Platform. The IoT data analysts can record intermediate outputs when training the secure algorithm, such as iteration steps.
• Recognized Background Model. The IoT data analyst expects to know more further details of shared data. However, from the shared ciphertext model, an IoT data analyst may gather more information by using her previous knowledge. To be more specific, the IoT data analyst can conspire with distinct IoT data providers to infer the sensitive information of other participants.

4.3. Design Purposes

Consider more than one IoT data provider and data analyst conspire to steal other participant’s privacy. Assume that all the participants as a curious-but-honest foe who executes protocol honestly but has an interest in other’s private information. Any number of participants may conspire with each other. The proposed method aims to shield the individual participant’s privacy and securely train the $K$-NN classifiers. The security goals are as follows:

• At the time of encountering curious-but-honest foe, the data analyst and individual data provider’s data are protected from disclosure.
• At the time of encountering more than one parties conspire with each other, the data analyst and individual data provider’s privacy also will be protected from disclosure.

5. The Construction of Secure $K$-NN

This section illustrates the system specifications of the proposed privacy-preserving $K$-NN training method over block-chain-based encrypted IoT data.

5.1. System Overview

For clearness, consider that a data analyst intents to train $K$-NN classifier based on the data gathered from various IoT data providers. Figure 2 illustrates the system overview, where the individual data provider preprocessed IoT data instances, encrypts them locally using their own private key, and register those encrypted data in the Blockchain-based distributed ledger. Present key supervision mechanisms [53,54,55] can be applied to handle the encryption abilities of data providers. The IoT data analyst can train a $K$-NN classifier by collecting the encrypted data registered in the public ledger and erect a protected algorithm with the building blocks of SPO, SBO, and SC. At the time of the training process, it is essential to interact between IoT data analyst and IoT data provider for reciprocating intermediate outcomes.

However, it is essential to mention that many comparison tasks are necessary to perform the training on an $K$-NN model. To accomplish the comparison task on encrypted data is extremely expensive, costly, and time-consuming. On the other hand, accurate intermediate data cannot be shared because the parameters of $K$-NN algorithm are easy to guess. There is a high possibility that in this situation, data analysts can be successful at guessing the original data. Therefore, to reduce the algorithm’s complexity, to make the method more realistic and protected from the privacy breach of both data providers and data analysts, we introduce a SBO. The data provider adds a small amount of bias ($\delta$) to secure the data’s privacy at the time of sharing the intermediate data. This small amount of discrimination does not cause any significant change in the classification process.

5.2. Encrypted Data Sharing via Blockchain

To aid model training, without the sacrifice of generality, consider that the same training task’s data instances have been locally preprocessed and designated with the corresponding feature vectors [16].

A unique transaction arrangement is defined in order to save the encrypted IoT data in the Blockchain. The proposed transaction structure primarily consists of two fields: input and output.

The input terminal comprises:

• The address of the data provider
• The encrypted version of data
• Name of the IoT device from where the data is generated

The corresponding output terminal holds:

• The address of the data analyst
• The encrypted version of data
• Name of the IoT device from where the data is generated

Hash value will be the addresses of the data provider and data analyst, and the encrypted data is determined from the homomorphic encryption, i.e., Paillier. Depending on the consideration that the length of the private key is 128 bytes, the length of the individual encrypted data instance is set to 128 bytes and stored in the Blockchain. The segment length of the IoT device type is 4 bytes.

The node serving as the data provider in the Blockchain network broadcasts it in a P2P system after assembling a new transaction, where the miner nodes will validate the correctness of the operation. A specific miner node can package the transaction in a new block and adding the block to the existing chain using current consensus algorithms, i.e., the PoW mechanism. Multiple transactions can be registered in a single block.

5.3. Building Blocks

Section 4 already specified that the goal is to secure the privacy of various IoT providers and design a privacy-preserving algorithm for training $K$-NN models over multiple private datasets afforded by diverse IoT providers.

5.3.1. $K$-NN

Several methods are available in order to calculate the distance, which is a model parameter of the $K$-NN. In this research, Manhattan distance $d_m$ (Equation (2)) is considered due to its simplicity in calculation. Algorithm 1 illustrate the entire process.

5.3.2. Secure Polynomial Operations (SPO)

In the proposed secure $K$-NN training schema, we develop secure polynomial addition and subtraction to securely train the $K$-NN model using the homomorphic property of Paillier’s. Reliable additions, secure subtractions and multiplication can be achieved straightforward. The homomorphic properties in Paillier can be defined as: $\left[[m_1+m_2]\right]=\left[\left[m_1\right]\right]\times \left[\left[m_2\right]\right]$$\left(mod{N}^2\right)$, and the homomorphic properties in case of subtraction can be described as:$\left[[m_1-m_2]\right]=\left[\left[m_1\right]\right]\times {\left[\left[m_2\right]\right]}^{-1}$$\left(mod{N}^2\right)$. $\left[\left[m^{-1}\right]\right]$ is known as the modular multiplicative inverse, which can calculate $\left[\left[m\right]\right]\times {\left[\left[m\right]\right]}^{-1}$$\left(mod{N}^2\right)$ $=1$ in Paillier. $\varphi \left(N\right)$ function can compute ${\left[\left[m\right]\right]}^{-1}$, ${\left[\left[m\right]\right]}^{-1}=$ ${\left[\left[m\right]\right]}^{\varphi \left(N\right)-1}$. Therefore, using ciphertext manipulation, the secure polynomial multiplication can be achieved, as shown in Equation (4).

$$\left[[a{m}_1+b{m}_2]\right]=\left[\left[m_1^a\right]\right]\times \left[\left[m_2^b\right]\right]\left(mod{N}^2\right)$$

(4)

Similarly, the secure polynomial division can be achieved, as shown in Equation (5).

$$\left[[m_1/a+m_2/b]\right]=\left[\left[m_1^{a^{-1}}\right]\right]\times \left[\left[m_2^{b^{-1}}\right]\right]\left(mod{N}^2\right)$$

(5)

However, this research need only secure polynomial addition and subtraction. Thus, the secure polynomial addition and subtraction are statistically indistinguishable, as Paillier is statistically indistinguishable [33].

5.3.3. Secure Biasing Operations (SBO)

In line no. 6 of Algorithm 2, data provider P calculate distance $\left[\left[d_{m_j}\right]\right]$ using SPO. Next step is to send the encrypted distance $\left[\left[d_{m_j}\right]\right]$ to the data analyst C. If the data provider P sent the encrypted distance $\left[\left[d_{m_j}\right]\right]$ to the data analyst C then C will decrypt it and try to guess the private data of the data provider. There is a high possibility of success as the data analyst C initiated the clustering point. Therefore, to protect the privacy of the data provider P’s data at the time of training the $K$-NN algorithm, we introduce the secure biasing operation (SBO). P will add a small amount of bias $\delta$ using SPO before sending the data to C in order to protect the data privacy. The bias $\delta$ will be unknown to C.

Algorithm 2: Secure Comparison

In this study, the range of bias will depend on the coefficient of variation ($CV$), where $CV$ = $\frac{StandardDeviation}{Mean}$ = $\sigma /\overline{x}$. Standard Deviation and Mean is computed using, $\sigma$ = $\sqrt{\frac{\sum x_i^2-\frac{{(\sum x_i)}^2}{n}}{n-1}}$ and $\overline{x}$= $\sum x_i/n$ respectively. Here n is the total number of data and $x_i$ stands for each data of the data-set. CV value greater than or equal to one, means that the data is scattered. CV value less than one, means that the is not scattered. Therefore, at the time of the experiment, various values for bias were chosen. The range was within $[1\le \delta \le 5]$, when $CV<1$ and $[-5\le \delta \le -1]$, when $CV\ge 1$. We found that bias range $[1\le \delta \le 3]$ and $[1\le \delta \le 3]$, gives proper classification results, when CV ≥ 1 and CV < 1, respectively. Therefore, If the coefficient of variation is greater than or equal to 1, $CV\ge 1$, than set the range of $\delta$ to [$-3\le \delta \le -1$] and On the other hand, if the coefficient of variation is less than 1, $CV<1$, than set the range of $\delta$ to [$1\le \delta \le 3$]. Note that the value of $\delta$ will never be equal to ($\delta ==0$).

5.3.4. Secure Comparison (SC)

The secure comparison in the proposed method is illustrated as the comparison between two encrypted numbers $\left[\left[m_1\right]\right]$ and $\left[\left[m_2\right]\right]$. For participants A and B engage in the secure comparison algorithm, neither parties can obtain real information. Our secure comparison algorithm protocol is exhibited in Algorithm 2, and the security proof is described in Section 6.

Proposition 1.

(Security of Secure Comparison Algorithm). Algorithm 2 is secure in the curious-but-honest model.

To develop SBO, we use the secure polynomial operation (mainly addition and subtraction) based on the homomorphic property of Paillier’s. A small amount of bias $\delta$ (where [$-3\le \delta \le -1$, $\delta \ne 0$, $1\le \delta \le 3$]) is encrypted and added with $d_{m_j}$ by data provider P using SPO (addition/subtraction). However, the data analyst C will never be able to extract the exact value of the bias $\delta$ value, or of the range, and this small amount of bias does not affect the classification task, which is shown in the performance evaluation Section 7. SBO ensures the privacy of the data provider P. If $\delta$ is positive, the definition will be:

$$\left[\left[m_1\right]\right]\times \left[\left[\delta \right]\right]\left(mod{N}^2\right)=\left[[m_1+\delta ]\right]$$

Again, when $\delta$ is negative, the definition will be:

$$\left[\left[m_1\right]\right]\times {\left[\left[\delta \right]\right]}^{-1}\left(mod{N}^2\right)=\left[[m_1-\delta ]\right]$$

5.4. Training Algorithm of Secure $K$-NN

For protected optimum design parameters, we outline a privacy-preserving $K$-NN training algorithm. Assume there is a single IoT data analyst C and n number of data providers P. Algorithm 3 specifies the training algorithm for secure $K$-NN. In Algorithm 3, the $K$-NN model parameters and sensitive data of IoT data providers are confidential. At the time of facing any collusion or curious-but-honest adversaries, individual members cannot infer any vital information of another member from the algorithm’s execution process’s intermediate outcomes. Section 6 illustrates the security proofs for Algorithm 3.

Proposition 2.

(Security of Privacy-Preserving $K-$NN Training Algorithm). Algorithm 3 is secure in the curious-but-honest model.

Algorithm 3: Secure $K-$NN Training Algorithm

6. Security Analysis

The security analysis manifests in this section under the identified ciphertext model and the public background model. Two security definitions were followed: secure two-party computation [47] and modular sequential composition [48]. A satisfied, protected two-party calculation protocol is safe in the face of curious-but-honest foes, and modular sequential composition implements a way to develop secret protocols in a modular way. Security proof of the proposed algorithm is described based on these two definitions.

6.1. Background of Security Proof

The notation in the article [14] was followed: Let, F is computed by a protocol $\pi$ and $F=$$(F_A,F_B)$ be a polynomial function; Input of A’s and B’s are a and b respectively using $\pi$, desire to calculate $F(a,b)$; A’s view is the tuple $vie{w}_A^{\pi }(\lambda ,a,b)$=$\lambda ;a;m_1,m_2,\dots ,m_n$ where $m_1,m_2,\dots ,m_n$ are the message received by A at the time of execution. B’s view is defined in a similar manner. $outpu{t}_A^{\pi }(a,b)$ and $outpu{t}_B^{\pi }(a,b)$ are the outputs of A and B respectively. $\pi$’s global outcome is $outpu{t}^{\pi }(a,b)$=$(outpu{t}_A^{\pi }(a,b),outpu{t}_B^{\pi }(a,b))$.

Definition 2.

(Secure Two-Party Computation [47]). If for all possible inputs $(a,b)$ and simulators $S_A$ and $S_B$ holds the following properties (≈ denotes computational indistinguishability against probabilistic polynomial-time adversaries with the negligible advantage in the security parameter λ.), only then a protocol π privately computes f with statistical security:

$$\{S_A,f_2(a,b)\}\approx \{vie{w}_A^{\pi }(a,b),outpu{t}^{\pi }(a,b)\}$$

$$\{f_1(a,b),S_B\}\approx \{outpu{t}^{\pi }(a,b),vie{w}_B^{\pi }(a,b)\}$$

The sequential modular composition’s fundamental idea is that: protocol $\pi$ is run to call an ideal functionality F by n participants, e.g., to calculate F privately, A and B send their inputs to a trusted third-party and receive the result. If the secure two-party computation is satisfied by the protocol $\pi$, and the same functionality as F can be achieved by protocol $\rho$ privately, then the protocol of $\rho$ in $\pi$ can replace the ideal protocol for the functionality F; Then, the latest protocol ${\pi }^{\rho }$ is protected and safe under the curious-but-honest model [14,48].

Theorem 1.

(Modular Sequential Composition [37]). Let the two-party probabilistic polynomial time functionalities be $F_1,F_2,\dots ,F_n$ and $F_1,F_2,\dots ,F_n$ is calculated by the protocols ${\rho }_1,{\rho }_2,\dots ,{\rho }_n$ in the presence of curious-but-honest adversaries. Let, two-party probabilistic polynomial time functionality be G and G is securely computed in the $F_1,F_2,\dots ,F_n$ by a protocol π - hybrid model in the presence of curious-but-honest adversaries. Then, ${\pi }^{{\rho }_1,{\rho }_2,⃯,{\rho }_n}$ securely calculate G in the presence of curious-but-honest adversaries.

6.2. Security Proof for Secure Comparison

Two entities are involved in Algorithm 2: P and C. The function is

$$F:F({\left[[m_1+\delta ]\right]}_C,{\left[[m_2+\delta ]\right]}_C,P{K}_C,S{K}_C,)=(\varphi ,(m_1\ge m_2))$$

Proof of Proposition 1.

The view of P is

$$vie{w}_P^{\pi }=({\left[[m_1+\delta ]\right]}_C,{\left[[m_2+\delta ]\right]}_C,P{K}_C)$$

Hence, the simulator:

$$S_P^{\pi }((m_1,m_2);F(m_1,m_2))=vie{w}_P^{\pi }({\left[\left[m_1\right]\right]}_C,{\left[\left[m_2\right]\right]}_C,{\left[\left[\delta \right]\right]}_C,P{K}_C)$$

where ${\left[\left[m_1\right]\right]}_C$ and ${\left[\left[m_2\right]\right]}_C$ are encrypted by $P{K}_C$ and the confidentiality of ${\left[\left[m_1\right]\right]}_C$ and ${\left[\left[m_2\right]\right]}_C$ are equivalent to the Paillier cryptosystem. Therefore, P cannot infer the value directly.

The view of C is

$$vie{w}_C^{\pi }=(\left(\left[[m_1+\delta ]\right]\right),\left(\left[[m_2+\delta ]\right]\right),P{K}_C,S{K}_C)$$

Then, $S_C^{\pi }$ runs as follows:

$$F((m_1+\delta ),(m_2+\delta ))=vie{w}_C^{\pi }((m_1+\delta ),(m_2+\delta ),P{K}_C,S{K}_C)$$

The bias $\delta$ is unknown to C, for that reason C would never be able to get the real $m_1$ and $m_2$ from $(m_1+\delta )$ and $(m_2+\delta )$. After comparison bewteen $(m_1+\delta )$ and $(m_2+\delta )$, C will return a flag with a value 0, if $(m_1+\delta )$ ≥ $(m_2+\delta )$, other wise value of flag will be 1. C is honest in following the method’s protocols. Therefore, C would never infer the value directly. □

6.3. Security Proof for Secure $K$-NN Training Algorithm

IoT data providers P and an IoT data analyst C, are the roles involved in Algorithm 2. Individual IoT data providers function in the same manner. Every one of the data providers will meet the security requirements if we can prove that one of them meets the security requirements. The function is:

$$F:F(D_{P_u},P{K}_C,S{K}_C)=(\varphi ,\left(\{(l_1,x_1,y_1),\dots ,(l_m,x_m,y_m)\}\right)).$$

Proof of Proposition 2.

Individual IoT data provider’s P view is

$$vie{w}_P^{\pi }=(D_{P_u},(\left[\left[c_{x_k}\right]\right],\left[\left[c_{y_k}\right]\right]),P{K}_C)$$

where $\left[\left[c_{x_k}\right]\right]$ and $\left[\left[c_{y_k}\right]\right]$ are encrypted by $P{K}_C$, the confidentiality of $\left[\left[c_{x_k}\right]\right]$ and $\left[\left[c_{y_k}\right]\right]$ will be equivalent to the cryptosystem Paillier. So none of the IoT data providers can infer the value directly.

The view of C is

$$vie{w}_C^{\pi }=((d_{m_j}+\delta ),(c_{x_k},c_{y_k}),P{K}_C,S{K}_C)$$

Now, the confidentiality of $(d_{m_j}+\delta )$ needs to be discussed, i.e., whether the IoT data analyst can predict the private data of individuals IoT data providers from the value. Clearly, the value is no-solution for the unknown $x_{i}and{y}_i$. The IoT data analyst may try to calculate unknown $x_{i}and{y}_i$ using the known distance $(d_{m_j}+\delta )$ and centroid $(c_{x_k},c_{y_k})$. It is not possible for IoT data analysts to identify the point of IoT data providers because the distance is added with bias value and the data analyst has no idea about the bias value or its range. Even with the brute force cracking, it is not possible to get the real value of dataset D. Consider that individual IoT data provider consists of a small dataset, which is $2-$dimensional, 100 instances, and each dimension is 32 bits (Typically, 4 bytes (32-bit) memory space is occupied by single-precision floating-point ). Under this situation, the probability of IoT data analyst successful guessing is $\frac{1}{2^{(n\times 6400)}}$, which is a negligible success probability [33]. □

We obtain the security of Algorithm 3 using modular sequential composition, as SPO, SBO, and SC are used in Algorithm 3, so it is secure in the curious-but-honest model.

7. Performance Evaluation

In this section, the performance of secure $K$-NN is evaluated based on efficiency and accuracy through extensive analysis using real-world dataset. Firstly, experiment settings are described, and the effectiveness and efficiency are demonstrated by the experimental results.

7.1. Experiment Setup

This segment discusses the testbed, dataset, and all other tasks for data preprocessing and experimental environment.

7.1.1. Testbed

In the proposed model, individual IoT data providers gather all the data in their domain from the IoT devices and perform data encryption. The experiments are executed on MacBook Pro equipped with an Intel Core i5 processor (2.5 GHz), memory (4 GB 1600 MHz DDR3), serving as IoT data providers and IoT data analysts simultaneously. The SPO, SBO, SC, and the Secure $K$-NN are implemented in the Platform: Google’s Colaboratory; Language: Python 3; Browser: Google Chrome.

7.1.2. Dataset

This study uses three real-world datasets, namely Breast Cancer Wisconsin Data Set (BCWD), Heart Disease Data Set (HDD), and Diabetes Data Set (DD) [56,57]. These datasets are publicly available from the UCI machine learning repository. The features of BCWD resembled a digitalized image of a breast mass and described characteristics of the cell nuclei present in the image. Each of the data instances is labeled as benign or malignant. The HDD and DD contain 13 and 9 numeric attributes, respectively. Instances are classified based on the types of heart diseases and diabetes symptoms.

Table 2. Statistics of Datasets.

Datasets	Instances Number	Attributes Number	Discrete Attributes	Numerical Attributes
BCWD	699	9	0	9
HDD	303	13	13	0
DD	768	9	0	9

represents the statistics of the dataset. We run 10-fold cross-validation to avoid overfitting or contingent results, and average results are recorded. 80% of the data has been selected for the model training and the remaining 20% for testing.

7.1.3. Float Format Conversion

The general K-NN training algorithms perform on both integer and floating-point numbers based on the data set. However, all the operations of cryptosystems are done on integers. Therefore for safety purposes, we should perform format conversion into an integer representation. Let, D is a binary floating-point number, which is represented as, $D={(-1)}^s\times M\times 2^E$, according to the global standard IEEE 754 [where the sign bit is s, significant number is M and exponential bit E]. A data analyst may perform this format conversion during the implementation of secure $K$-NN based on the dataset type.

7.1.4. Key Length Setting

The security of the public key cryptosystem is closely associated with the length, and a compact key may cause vulnerable encryption. A long key reduces the homomorphic operation’s efficiency, and a too-short key may cause the overflow of plaintext space during the homomorphic operations (i.e., the secure polynomial operation and secure biasing operation) on the ciphertext. Therefore, it is crucial to consider the length of the key in order to bypass the possibility of overflow. The Key of Paillier cryptosystem N is set to $1024$-bit in secure $K$-NN.

7.2. Evaluation Parameters

We use three commonly used criteria for evaluating ML classifiers (Accuracy (6), Precision (7), Recall (8)).

$$Accuracy=\frac{t_p+t_n}{t_p+t_n+f_p+f_n}$$

(6)

$$Precision=\frac{t_p}{t_p+f_p}$$

(7)

$$Recall=\frac{t_p}{t_p+f_n}$$

(8)

where $t_p$ is the number of relevant ( the positive class) that are labeled precisely, $f_p$ is the numbers of irrelevant (the negative class) that are labeled correctly, $f_n$ is the numbers of relevant that are mislabeled and $t_n$ is the number of irrelevant that are mislabeled in the test outcomes.

The general $K$-NN was implemented using raw Python language in order to demonstrate that secure $K$-NN does not lessen the accuracy upon preserving the individual IoT data provider’s privacy and securely training the classifier. The main focus is to train the classifier securely. For that reason, the train parameters are not adjusted, and default parameters are used. The results of precision and recall are summarized in

Table 3. Summary of performance.

Parameter	Model	Datasets
		BCWD	HDD	DD
Accuracy	SVM	$96.60\%$	$81.00\%$	$77.00\%$
	Secure SVM	$95.25\%$	$80.89\%$	$76.67\%$
	$K-$NN (t = 8)	$96.96\%$	$83.50\%$	$79.00\%$
	Secure $K-$NN (t = 8)	$97.80\%$	$82.33\%$	$78.00\%$
Precision	SVM	$96.16\%$	$81.79\%$	$75.00\%$
	Secure SVM	$96.02\%$	$81.25\%$	$74.80\%$
	$K-$NN (t = 8)	$96.54\%$	$83.85\%$	$77.00\%$
	Secure $K-$NN (t = 8)	$96.26\%$	$82.30\%$	$76.00\%$
Recall	SVM	$96.48\%$	$80.38\%$	$71.00\%$
	Secure SVM	$95.65\%$	$79.65\%$	$70.91\%$
	$K-$NN (t = 8)	$96.85\%$	$83.85\%$	$75.90\%$
	Secure $K-$NN (t = 8)	$96.67\%$	$82.66\%$	$75.10\%$

.

The performance of Secure $K$-NN is almost similar to standard $K$-NN and has better performance than SVM [29]. However, the data provider must be careful about the bias value because a larger bias may reduce the classifier’s performance. The proposed design shows better robustness on both (discrete attributes and numerical attributes) datasets.

7.3. Efficiency

7.3.1. Building Blocks Evaluation

Table 4. Performance of the building blocks in Secure $K$-NN against Secure SVM.

Dataset	Time	Secure SVM	Secure $\mathit{K}-$NN
BCWD	Total	3674 s	3357.2 s
	P	2789 s	2534 s
	C	1066 s	860 s
	$SPO$	3462 s	3113 s
HDD	Total	2735 s	2534 s
	P	1761 s	1520 s
	C	924 s	765 s
	$SPO$	2333 s	1922 s
DD	Total	3959 s	3709 s
	P	3199 s	2920 s
	C	1045 s	995 s
	$SPO$	3773 s	3527 s

illustrates the running time of the SPO with encrypted datasets on Algorithm 3. Table 4 also gives the time consumption of IoT data providers P and data analysis C, the total time consumption.

According to the performance results in Table 4, secure $K$-NN spend less than an hour with encrypted dataset BCWD, HDD, and DD at the time of training, which is an acceptable time consumption as a stand-alone algorithm. It is essential to mention that the general $K$-NN is comparatively slower, so it is better not to train a $K$-NN algorithm with a larger dataset at a time. We recommend that the reader convert the larger dataset into small portions and train the secure $K$-NN. In our implementation, we used multi-threading in Python to control the run time of a larger dataset.

In this experiment, various P is simulated linearly. Therefore the P time shown in Table 4 is the accumulation of time consumed by different P. In a real-world application, various P can run their algorithms parallelly so that the time consumption of P and the total time consumption can be reduced. We believe Algorithm 3 to be useful for real-world sensitive applications. Confronting various datasets such as BCWD, DD (numerical attributes), and HDD (discrete attributes), secure $K$-NN manifests satisfying robustness in time consumption.

7.3.2. Scalability Evaluation

Secure $K$-NN believes that various IoT data providers are engaging and contributing data. We distribute the dataset into various identical sections to mimic the situation of different IoT data providers. We observe the fluctuations in time consumption to evaluate the scalability of the proposed scheme when various data IoT providers strive for the calculation. Cases are simulated during the number of IoT data providers rises from 1 to 5. The outcomes are represented in Figure 3. The X-axis represents the number of IoT data providers associated with the calculation, and the Y-axis represents the time consumption.

Theoretically, The time consumption of secure $K$-NN is proportional to the amount of data and number of iterations in the comparison portion. If the total amount of data and data quality are fixed, the rise in the number of P will not affect the time consumption, and the time consumption of P or C remains the same at different numbers of P. There is a small vibration in the total time consumption when the number of P rises from 1 to 5 because the program’s run time gets disturbed as other host processes are used for the simulation.

8. Conclusions

This paper introduced a novel privacy-preserving $K$-NN training method called secure $K$-NN, which can handle data privacy and data integrity concerns. It employs Blockchain technology to train the algorithm in a multi-party scenario where the IoT data is received from various data providers. A partial homomorphic cryptosystem known as Paillier is applied to assemble an effective and reliable method. Efficiency and security of secure $K$-NN are demonstrated in this study. The proposed method achieves almost similar accuracy to general $K$-NN and outperforms the earlier state of the art. Future work includes developing a versatile structure that allows assembling a broad range of privacy-preserving ML training algorithms on a multi-party scenario with encrypted datasets.