Personalized Privacy Protection-Preserving Collaborative Filtering Algorithm for Recommendation Systems

Abstract

With the rapid development of ubiquitous data collection and data analysis, data privacy in a recommended system is facing more and more challenges. Differential privacy technology can provide strict privacy protection while reducing the risk of privacy leakage, but it also introduces unwanted noise, which makes the performance of the recommender system worsen. Among different users, the degree of their sensitivity to privacy is usually different. Thus, through considering the impact of users’ personalized requirements, the collaborative filtering algorithm can be designed to reduce the amount of unwanted noise. Taking the above assertions into account, we propose a collaborative filtering algorithm based on personalized privacy protection. First, it locally classifies ratings by privacy sensitivity on the user side, then utilizes the random flip mechanism to protect the privacy-sensitive ratings. Then, after the server catches the perturbed rating data, we reconstruct the joint item-item distribution through the Bayesian estimation method. Experimental results show that our proposed algorithm can significantly improve the recommendation performance of recommendation systems while protecting users’ privacy.

1. Introduction

Recommendation systems are widely used in real-life recommendation tasks as an effective method to solve the issue of information overload.

Collaborative filtering algorithm is the most classic and commonly used in the field of recommendation system. The basic idea of collaborative filtering algorithm is to recommend items to users based on their previous preferences and the choices of other users with similar interests, that is, based on users’ historical behavioral data to discover users’ preferences and to predict the items that users may like. It does not rely on any items’ additional information (e.g., their own characteristics) or any users’ additional information (e.g., age, gender, etc.). Currently, the most widely used collaborative filtering algorithm is the neighborhood-based method, which includes two main types of algorithms. One type is the user-based collaborative filtering algorithm (UserCF), which recommends products to users according to others has the similar interests as them, the other is the item-based collaborative filtering algorithm (ItemCF), which recommends items which are similar to their previously preferred items. Additionally, recommender systems usually need to collect users’ personal information and interaction records to train models and perform high-quality product recommendations [1,2]. Traditional recommender systems assume that platform parties and users are completely trustworthy with each other. However, the reality is often not as ideal as we think, there often exists the risk of privacy leakage in real scenarios, and this risk exists in multiple ways, including between users and platforms, between users and users, and between platforms and platforms. This leads to the risk of leakage of users’ private information. Collaborative filtering algorithms are widely applied in various fields; however, these algorithms extensively use users’ historical information in the process of preference calculation, which may lead to the risk of privacy leakage [3,4,5]. Therefore, privacy protection in recommendation algorithms is of great concern to users. To address the issue of privacy leakage during recommendation, many studies have tried to bring the differential privacy-preserving strategy into collaborative filtering algorithms [6]. Some typical algorithms include one designed by Zhu et al. [7], which is an item-based differential privacy recommendation algorithm and a user-based differential privacy recommendation algorithm, which can effectively solve the issue of privacy leakage of collaborative filtering algorithms, but due to its neglect of the sensitivity of user rating data, a large amount of noise data is introduced, resulting in unsatisfactory algorithm efficiency, and at the same time, the user-based strategy and the project-based strategy are independent to each other, so advantages of algorithms are not fully released. Wang et al. [8] combined the Bhattacharyya similarity with the K-medoids clustering to improve the accuracy of the similarity measure of the differential privacy-based collaborative filtering algorithms. Meanwhile, the differential privacy mechanism also introduces a lot of noise data that impairs the performance of the recommendation system. How to trade-off between algorithmic performance and the privacy security has become one of the hot spots in the field of differential privacy-preserving recommendation algorithms.

To better protect users’ privacy security, many scholars have tried to introduce the method of data reconstruction, which is an evaluation algorithm based on the probabilistic perturbation mechanism. Traditional evaluation algorithms can only reconstruct the distribution of a single vector; thus, Rade et al. [9] proposed a joint cardinality estimation method to reconstruct the joint distribution of multiple vectors. Then, Chen et al. [10] introduced the joint cardinality estimation method into the neighbor-based collaborative filtering algorithm, which effectively improved the performance of the recommendation system while protecting the security of users’ privacy. Guo et al. [11,12] proposed a joint frequency estimation algorithm based on the random response mechanism, and applied it to the neighbor-based collaborative filtering algorithm to protect the security of users’ privacy. However, the estimation results of these joint estimation algorithms mentioned above are sometimes negative, which can seriously affect the accuracy of the distribution of rating prediction. Furthermore, Ren et al. [13] proposed a distributed reconstruction method for the high-dimensional crowdsourced data based on the random response mechanism, which could greatly improve the release accuracy of the high-dimensional crowdsourced data. However, there exists great differences in structure between the crowdsourced data and the rating data, resulting in the reconstruction method based on high-dimensional crowdsourced data cannot be directly applied to the recommendation system. In addition, different users have different sensitivity to privacy, and ignoring differences between users not only does not meet the user’s personalized privacy requirements, but also introduces additional noise into the data, thereby reducing the performance of the recommendation system.

To address the issue of user privacy leakage, this paper proposes a collaborative filtering algorithm based on personalized privacy protection. Our main contributions include: (1) providing a personalized privacy sensitivity classification and coding method, which can effectively reduce the noise added by the privacy protection; and (2) propose a Bayesian joint distribution estimation method through creatively applying the Bayesian algorithm to the similarity calculation of recommendation systems.

2. Collaborative Filtering Algorithm Based on Personalized Privacy Protection

To better protect the security of users’ privacy, while reducing the impact of noise on the performances of the recommendation algorithms, this paper proposes a personalized collaborative filtering-based recommendation algorithm with the differential privacy protection, and the algorithmic architecture is shown in Figure 1.

Figure 1. Overview of Algorithmic architecture.

From Figure 1, each user locally classifies the privacy sensitivity of ratings and encodes it, then perturbs the ratings and sends the vector of perturbed ratings to the server. According to all the perturbed ratings, the server first utilizes the Bayesian estimation to reconstruct the joint distribution of items, then calculates the item-item similarity and sends the results to users. Finally, it locally completes the high-quality, personalized recommendation.

2.1. Evaluation and Encoding of the Privacy Sensitivity of User Ratings

For recommendation systems, malicious attackers often rely on users’ rating data to infer and obtain user privacy. Numerous studies [14,15,16] have proved that there exists a certain correlation between the size of the ratings and user privacy, and too high and too low ratings tend to be more favorable for malicious attackers to infer users’ private information, such as user preferences. On the other side, from the user’s perspective, both users’ rating habits and the degree of the sensitivity to privacy protection are different. Based on the above two factors, the classification rules of the privacy sensitivity towards users’ ratings are proposed as follows:

$$r_{ui}=\left\{\begin{array}{l} sensitive rating, \mathrm{when} r_{ui}\in \left[r_{min},\overline{r_u}-{\gamma }_u\right]\cup \left[\overline{r_u}+{\gamma }_u,r_{max}\right];\\ \mathrm{weakly}-sensitive rating, when r_{ui}\in \left(\overline{r_u}-{\gamma }_u,\overline{r_u}+{\gamma }_u\right);\end{array}\right.$$

(1)

where $r_{min}$ and $r_{max}$ represent the maximum and minimum values specified by the rating rules, respectively; $\overline{r_u}$ denotes the average score of user u; ${\gamma }_u$ is an adjustable variable to adjust the range of intervals classified.

Depending on the sensitivity type of ratings, the encoding function E is

$$E\left(r_{ui}\right)=\left\{\begin{array}{c}{b}_{ui}^{s+}, when r_{ui}\in \left(\overline{r_u}+{\gamma }_u,r_{max}\right);\\ b_{ui}^{ns}, when r_{ui}\in \left(\overline{r_u}-{\gamma }_u,\overline{r_u}+{\gamma }_u\right)\\ b_{ui}^{s-}, when r_{ui}\in \left(r_{min},\overline{r_u}-{\gamma }_u\right);\end{array}\right.;$$

(2)

where $b_{ui}^{s+}$, $b_{ui}^{ns}$, and $b_{ui}^{s-}$ are 1, 0, and −1, respectively, that is, the encoding function encodes the high-sensitivity score as 1, the low-sensitivity score as −1, and the weakly sensitive score as 0.

The rating of u constitutes the rating vector of $R_{u.}=\left\{r_{u1},r_{u2},r_{u3},\dots ,r_{un}\right\}$. According to the function E, $R_{u.}$ is encoded to be the sensitive rating vector $B_{u.}^s$ and weakly sensitive rating vector $B_{u.}^{ns}$

$$B_{u.}^s=\left\{b_{u{a}_1}^s, b_{u{a}_2}^s,b_{u{a}_3}^s,\dots ,b_{u{a}_x}^s\right\}$$

$$B_{u.}^{ns}=\left\{b_{u{c}_1}^{ns}, b_{u{c}_2}^{ns},b_{u{c}_3}^{ns},\dots ,b_{u{c}_{n-x}}^{ns}\right\}$$

where $a=\left\{a_1,a_2,a_3,\dots ,a_x\right\}$ represents the collection of items corresponding to the sensitive rating, and $c=\left\{c_1,c_2,c_3,\dots ,c_{n-x}\right\}$ represents the collection of items corresponding to the weakly sensitive rating. Additionally,$b_{u{a}_i}^s\in \left\{b_{u{a}_i}^{s+},b_{u{a}_i}^{s-}\right\}$.

2.2. Differential Privacy Protection for Rating Data

To protect the security of the encoded rating data, users need to locally perturb them. Based on the random flip mechanism, the perturbation function for the sensitive ratings is shown below:

$$b_{u{a}_i}^{\hat{s}}=\varnothing \left(b_{u{a}_i}^s\right)=\left\{\begin{array}{c}{b}_{u{a}_i}^s, if y>p;\\ b_{u{a}_i}^{\tilde{s}}, if y\le p;\end{array}\right.$$

(3)

where $p=\frac{1}{1+e^{\epsilon }}$, $q=1-p$, and $\mathsf{\epsilon }$ represents the privacy budget, respectively, and $\mathrm{y}$ represents one random number evenly distributed within [0, 1]. The variable $b_{u{a}_i}^{\tilde{s}}$ is the result of the inverse operation on $b_{u{a}_i}^s$. $b_{u{a}_i}^{\widehat{s}}$ is the output of ${\mathrm{b}}_{{\mathrm{ua}}_{\mathrm{i}}}^{\mathrm{s}}$ being perturbed. In the perturbation function of ∅, $b_{u{a}_i}^s$ flips with the probability of p and does not flip with the probability of q, i.e.,

$$pr\left(b_{u{a}_i}^{s+}\to b_{u{a}_i}^{s+}\right)=pr\left(b_{u{a}_i}^{s-}\to b_{u{a}_i}^{s-}\right)=q$$

$$pr\left(b_{u{a}_i}^{s-}\to b_{u{a}_i}^{s+}\right)=pr\left(b_{u{a}_i}^{s+}\to b_{u{a}_i}^{s-}\right)=p$$

Based on the encoding function $\mathrm{E}$ and the perturbation function $\varnothing$, Figure 2 shows the privacy protection operation for the local rating data. First, encode the rating vector $R_{u.}$ of user $u$ to be $B_{u.}^s$ and $B_{u.}^{ns}$ through the function $E$. Second, use the perturbation function $\varnothing$ to perturb each element within $B_{u.}^s$ to get the perturbed sensitive rating vector $\widehat{B_{u.}^s}$. Finally, stitch $\widehat{B_{u.}^s}$ and $B_{u.}^{ns}$ into $\widehat{B_{u.}}$, and send $\widehat{B_{u.}}$ to the server.

Figure 2. Privacy protection on rating data.

2.3. Item-Item Similarity

After the server receives the perturbed rating data $\widehat{B_{u.}}\left(u=1,2,\dots ,m\right)$ from each user, the perturbation matrix $\widehat{B}$ of ratings can be obtained as shown in Figure 3. $\widehat{B_{.\alpha }}$ and $\widehat{B_{.\beta }}$ represent the perturbation rating vectors for item $\mathsf{\alpha }$ and item $\beta$, respectively. According to the type of common ratings stemming from item $\mathsf{\alpha }$ and item $\beta$, divide the common ratings into the collection $O_{\alpha \beta }^s$ including all the sensitive rating pairs and the collection $O_{\alpha \beta }^{ns}$, including all the weakly sensitive rating pairs, where

$$O_{\alpha \beta }^s=\left\{\widehat{b_{.\alpha }^s},\widehat{b_{.\beta }^s}\right\}$$

$$O_{\alpha \beta }^{ns}=\left\{\left(\widehat{b_{.\alpha }^s},\widehat{b_{.\beta }^{ns}}\right),\left(\widehat{b_{.\alpha }^{ns}},\widehat{b_{.\beta }^s}\right),\left(\widehat{b_{.\alpha }^{ns}},\widehat{b_{.\beta }^{ns}}\right)\right\}$$

Figure 3. Perturbation matrix $\widehat{B}$.

Furthermore,$I{U}^s\left(\alpha ,\beta \right)$ and $I{U}^{ns}\left(\alpha ,\beta \right)$ represent the collections of users corresponding to $O_{\alpha \beta }^s$ and $O_{\alpha \beta }^{ns}$, respectively, then calculate the similarity of each rating pair, then obtain the item-item similarity between $\alpha$ and $\beta$ with the weighted sum, i.e.,

$$sim\left(\alpha ,\beta \right)=\lambda \times si{m}_1+\left(1-\lambda \right)\times si{m}_2$$

(4)

where $si{m}_1$ represents the similarity of the sensitive rating pair; $si{m}_2$ represents the similarity of the weakly sensitive rating pair; the parameter $\lambda$ denotes the weight coefficient.

2.3.1. Similarity of the Sensitive Rating Pair

To improve the performance of the recommendation system, we use the Bayesian algorithm to reconstruct the joint distribution of the sensitive rating pair within $\widehat{B}$. For $\widehat{B_{.\alpha }^s}$ and $\widehat{B_{.\beta }^s}$, $W$ and $\widehat{W}$ represent the collection of pre-perturbation sensitive rating pairs and the collection of post-perturbation sensitive rating pairs.

$$W=\left\{\left(b_{.\alpha }^{s-},b_{.\beta }^{s-}\right),\left(b_{.\alpha }^{s-},b_{.\beta }^{s+}\right),\left(b_{.\alpha }^{s+},b_{.\beta }^{s-}\right),\left(b_{.\alpha }^{s+},b_{.\beta }^{s+}\right)\right\}$$

$$\widehat{W}=\left\{\left(\widehat{b_{.\alpha }^{s-}},\widehat{b_{.\beta }^{s-}}\right),\left(\widehat{b_{.\alpha }^{s-}},\widehat{b_{.\beta }^{s+}}\right),\left(\widehat{b_{.\alpha }^{s+}},\widehat{b_{.\beta }^{s-}}\right),\left(\widehat{b_{.\alpha }^{s+}},\widehat{b_{.\beta }^{s+}}\right)\right\}$$

The joint distribution estimation algorithm based on Bayesian algorithm is shown as follows.

First, initialize the rating pairs in $W$ to be uniformly distributed, then calculate the prior probability through Bayesian method, i.e.,

$$P\left(\widehat{w}\in \widehat{W}|w\in W\right)={\displaystyle \prod }_{i=1}^{len\left(w\right)}{p}^{w\left[i\right]\oplus \widehat{w\left[i\right]}}\times q^{1-w\left[i\right]\oplus \widehat{w\left[i\right]}}$$

(5)

where $p$ and $q$ represent the random flip probability, respectively;$\mathrm{len}\left(\mathrm{w}\right)$ denotes the number of elements in the collection of $w$; $w\left[i\right]$, and $\widehat{w\left[i\right]}$ represent the i-th element in the collections of $\mathrm{w}$ and $\widehat{w}$, respectively.

Furthermore, iteratively update the value of the posterior probability estimation for $W$, i.e.,

$$P_{iter}\left(w|\left(\widehat{b_{u\alpha }^s},\widehat{b_{u\beta }^s}\right)\in \widehat{W}\right)=\frac{P_{iter}\left(w\right)\times P\left(\widehat{b_{u\alpha }^s},\widehat{b_{u\beta }^s}|w\right)}{{{\displaystyle \sum }}_{t\in W}{P}_{iter}\left(t\right)\times P\left(\widehat{b_{u\alpha }^s},\widehat{b_{u\beta }^s}|t\right)}$$

(6)

$$P_{iter+1}\left(w\in W\right)=\frac{{\sum }_{u\in I{U}^s\left(\alpha ,\beta \right)}{P}_{iter}\left(w|\left(\widehat{b_{u\alpha }^s},\widehat{b_{u\beta }^s}\right)\in \widehat{W}\right)}{N_{I{U}^s\left(\alpha ,\beta \right)}}$$

(7)

where $iter$ represents the times of iterations. In addition, the algorithm stops iterating under the following condition, i.e.,

$$\underset{W}{\max }{P}_{iter}\left(w\in W\right)-\underset{W}{\max }{P}_{iter-1}\left(w\in W\right)\le \delta$$

(8)

where $\delta$ is used to control the number of iterations. Finally, Algorithm 1 outputs the Bayesian estimation distribution $P\left(w\in W\right)$. According to $P\left(w\in W\right)$, the similarity of the sensitive rating pair can be obtained as follows, i.e.,

$$si{m}_1=P\left(w=\left(\widehat{b_{.\alpha }^{s-}},\widehat{b_{.\alpha }^{s-}}\right)\right)+P=\left(\left(\widehat{b_{.\alpha }^{s+}},\widehat{b_{.\alpha }^{s+}}\right)\right)$$

(9)

Algorithm 1: Joint distribution estimation algorithm based on Bayesian method

Input: p,q;$\widehat{B_{.\alpha }}$,$\widehat{B_{.\beta }}$;$\mathsf{\delta }$;$W$,$\widehat{W}$;$I{U}^s\left(\alpha ,\beta \right)$

Output: $P\left(w\in W\right)$

1. Initialize $P_0\left(w\in W\right)=0.25$;

2. for $\mathrm{w}$ in $W$

3.  for $\widehat{w}$ in $\widehat{W}$

4.    Calculate the prior probability $P\left(\widehat{w}|w\right)$ according to Formula 5;

5.  end for

6. end for

7. Initialize $iter=0$;//The number of iteration executions

8. while ($\underset{W}{\max }{P}_{iter}\left(w\in W\right)-\underset{W}{\max }{P}_{iter-1}\left(w\in W\right)>\delta$)

9.  for $u$ in $I{U}^s\left(\alpha ,\beta \right)$

10.    for $w$ in $W$

11.   Calculate the posteriori probability $P_{iter}\left(w|\left(\widehat{b_{u\alpha }^s},\widehat{b_{u\beta }^s}\right)\in \widehat{W}\right)$ according to Formula 6;

12.    end for

13.  end for

14.  Calculate $P_{iter+1}\left(w\in W\right)$ according to Formula 7;

15.  Update $iter=iter+1$;

16. return $P\left(w\in W\right)=P_{iter}\left(w\in W\right)$;//Return after the while loop ends

2.3.2. Similarity of the Weakly Sensitive Rating Pair

The weakly sensitive rating pair remains the same in similarity as the original rating pair, and the detailed theoretical analysis can be seen in Section 3.1. The similarity of the weakly sensitive rating pair can be calculated by:

$$si{m}_2=\frac{1}{N_{I{U}^{ns}\left(\alpha ,\beta \right)}}\times {\displaystyle \sum }_{u\in I{U}^{ns}\left(\alpha ,\beta \right)}\frac{2-\left|\widehat{b_{u\alpha }}-\widehat{b_{u\beta }}\right|}{2}$$

(10)

where $\left(\widehat{b_{u\alpha }},\widehat{b_{u\beta }}\right)\in O_{\alpha \beta }^{ns}$.

2.4. Local Top-N Recommendation

Users can obtain the collection $N\left(\alpha \right)$ of item $\alpha \prime s$ neighbors according to the similarity results in the server. The local prediction formula for the user $u$’s rating of the item α is shown below.

$$r\left(u,a\right)=\frac{{\sum }_{v\in N\left(\alpha \right)}sim\left(a,v\right)\times r_{uv}}{{\sum }_{v\in N\left(\alpha \right)}sim\left(a,v\right)}$$

(11)

where $r_{uv}$ denotes the original rating of item $v$ from user, $u$, and $v$ is an element of the collection of $N\left(\alpha \right)$. Finally, the top $N$ items with the highest predicted ratings are recommended to users.

3. Algorithmic Analysis

3.1. Analysis on Efficiency

In this paper, we mainly use the following two methods to ensure the validity of the rating data, reducing the impact stemming from the privacy noise on the recommendation process.

First, controlling the impact of privacy protection noise on recommendation results through classifying rating data according to its sensitivity. Specifically, for the weakly sensitive rating pair, its similarity is the same as the similarity when the rating data is not perturbed. This is because when encoding the rating data, the weakly sensitive rating is encoded as 0 and the sensitive score is ±1. The difference between the sensitive rating and the weakly sensitive rating, whether before or after the random flip operation, is always 1. Therefore, the sensitivity classification and the encoding strategy adopted in this paper can effectively reduce the influence of noise on the similarity calculation, thus substantially improving the accuracy of the recommendation system.

Second, for the sensitive scoring pair, we utilize the Bayesian algorithm to reconstruct its distribution. Given the distribution of the random flip probability and the distribution of the rating pairs having been randomly flipped, the distribution of the original rating pair can be estimated. The accuracy of the estimation result obtained by Algorithm 1 is affected by the initial value of $P_0$ and the iteration times of $\delta$. In this paper, initialize the original distribution be the uniform distribution, and through adjusting the times of iterations to ensure the accuracy of the estimation results. Reconstructing the distribution of the sensitive rating pair can reduce the impact of noisy data on the similarity calculation results of the sensitive rating pair, thereby further improving the performance of the recommendation system.

3.2. Analysis on Security

The proposed algorithm uses the random flip mechanism to perturb the encoded privacy-sensitive ratings and accomplish differential privacy protection for the sensitive rating. The proof that the proposed algorithm can satisfy the differential privacy security is as follows.

Assume $B_{u.}^s$ and $B_{u.}^{s^{\prime }}$ be two different collections of neighbor data for user, $u$, and there exists one different sensitive rating element between the two collections, i.e.,

$$B_{u.}^s=\left[b_{u{a}_1}^s, b_{u{a}_2}^s,b_{u{a}_3}^s,\dots ,b_{u{a}_i}^s,\dots ,b_{u{a}_x}^s\right]$$

$$B_{u.}^{s^{\prime }}=\left[b_{u{a}_1}^s, b_{u{a}_2}^s,b_{u{a}_3}^s,\dots ,b_{u{a}_i}^{s^{\prime }},\dots ,b_{u{a}_x}^s\right]$$

$\varnothing$ represents the random flip operation. Assume $X_u=\left[x_{u{a}_1},x_{u{a}_2},x_{u{a}_3},\dots ,x_{u{a}_x}\right]$ denote the random output of $\varnothing$. Then,

$$\frac{pr\left(\varnothing \left(B_{u.}^s=X_u\right)\right)}{pr\left(\varnothing \left(B_{u.}^s{}^{\prime }=X_u\right)\right)}=\frac{{\prod }_{t=1}^{x}pr\left(b_{u{a}_t}^s\to x_{u{a}_t}\right)}{{\prod }_{t=1}^{x}pr\left(b_{u{a}_t}^s{}^{\prime }\to x_{u{a}_t}\right)}=\frac{pr\left(b_{u{a}_t}^s\to x_{u{a}_t}\right)}{pr\left(b_{u{a}_t}^s{}^{\prime }\to x_{u{a}_t}\right)}\le \frac{q}{p}=e^{\epsilon }$$

Therefore, it can be proved that the proposed algorithm can satisfy the requirements of the ε-differential privacy.

4. Experimental Analysis

In this paper, the analysis of performance is carried out on the publicly available datasets MovieLens 1M and Yahoo Music, and the details of the two datasets are shown in Table 1. To better observe the experimental effect, MovieLens 1M and Yahoo Music are both divided into the training set and the test set in the ratio of 8:2.

Table 1. Basic information on datasets.

Dataset	Number of Users	Number of Items	Number of Ratings	Range of Ratings	Sparsity (%)
MovieLens 1M	6000	4000	1,000,000	{1,2,3,4,5}	95.83
Yahoo Music	8089	1000	270,121	{1,2,3,4,5}	96.66

Here, we use the mean absolute error (MAE) and the root mean square error (RMSE) as the evaluation indicators, and detailed definitions of MAE and RMSE are:

$$MAE=\frac{{\sum }_{u\in U,v\in V}\left|r_{uv}-r_{uv}^{\prime }\right|}{n}$$

(12)

$$\mathrm{RMSE}=\sqrt{\frac{{\sum }_{u\in U,v\in V}{\left(r_{uv}-r_{uv}^{\prime }\right)}^2}{N}}$$

(13)

where$r_{uv}$ and $r_{uv}^{\prime }$ represent the real rating and the predictive rating of user, $u$, on item, $v$, respectively. $U$ and $V$ represent the collection of users and the collection of items, respectively.

4.1. Algorithms for Comparison

To further illustrate the performance of the algorithm in this paper, the following mainstream algorithms are selected.

(1) IBCF-DS [1], which is a collaborative filtering algorithm based on item similarity. Similar to PPPCF, it also adopts the rating encoding method [12] to ensure the consistency of the comparison. However, IBCF-DS does not adopt any data privacy protection measures, so it can be used as a baseline for the performance comparison.

(2) PNCF [7], which is an item-based collaborative filtering algorithm. It brings the exponential mechanism and the Laplace mechanism to protect the privacy of data [13,14,15,16], and introduces the recommendation perception sensitivity and truncated similarity to improve the utility of the algorithm.

(3) DPLCF [10], which is a distributed recommendation algorithm. Similar to PPPCF, it uses the random flip mechanism to locally protect the privacy security of the implicit data on the user side, then uses the cardinality estimation mechanism [17,18] on the server side to reconstruct the joint cardinality between items and through reducing the Jekaard similarity error between projects to optimize the accuracy of recommendations.

(4) LDP item-base CF [11], which uses the UE encoding strategy to flip data with a certain probability to locally protect the privacy of data on the user side. On the server side, it uses the frequency estimation and the joint frequency estimation [19,20,21,22] to reconstruct data to reduce the error of privacy noise on similarity and ensure the accuracy of recommendations.

(5) Truncated PPPCE, which is constructed through removing the Bayesian estimation module from PPPCF, in order to illustrate the influence of the Bayesian estimation module on PPPCE.

4.2. Parameter Settings

In this paper, assign 0.05 to the iteration threshold $\delta$ of the Bayesian estimation, and assign 0.2 to the similarity adjustment parameter $\lambda$. However, assign 0.5 to the similarity adjustment parameter $\lambda$ in Equation 2 and Equation 3, and refer to the literature [11], assign 0.5 to the parameter of conditional probability similarity in the LDP item-base CF algorithm. To observe the algorithm performance under different privacy budgets, the range of the privacy budget is assigned to [0.1, 1] with the step size of 0.1. To observe the effect of the number of neighbors on the algorithm, the number $\mathrm{N}$ of neighbors of items is assigned to [20, 100] with the step size of 20.

4.3. Parameter Settings

4.3.1. Effect of N on Experimental Results

Assign 1 to the privacy budget $\epsilon$, and Table 2 shows the performance of each algorithm with different numbers of neighbors, i.e., N.

Table 2. Comparison of the algorithm performance with different values of N.

Dataset	Indicator	Algorithm	N = 20	N = 40	N = 60	N = 80	N = 100
MovieLens 1M	MAE	IBCF-DS	0.7219	0.7176	0.7169	0.7170	0.7171
		DPLCF	0.8912	0.8722	0.8718	0.8726	0.8694
		LDP item-base CF	0.8666	0.8484	0.8407	0.8361	0.8330
		PNCF	0.9600	0.9290	0.9110	0.8960	0.8870
		Truncated PPPCE	0.8582	0.8527	0.8537	0.8502	0.8498
		PPPCE	0.7911	0.7830	0.7783	0.7781	0.7798
	RMSE	IBCF-DS	0.9288	0.9220	0.9208	0.9207	0.9208
		DPLCF	1.1284	1.1050	1.1049	1.1050	0.1021
		LDP item-base CF	1.1217	1.0983	1.0888	1.0825	01.0784
		PNCF	1.2470	1.2040	1.1790	1.160	1.1480
		Truncated PPPCE	1.0944	1.0868	1.0883	1.0840	1.0837
		PPPCE	1.0071	0.9974	0.9220	0.9913	0.9932
Yahoo Music	MAE	IBCF-DS	0.9482	0.9484	0.9484	0.9485	0.9486
		DPLCF	1.0218	1.0243	1.0228	1.0235	1.0213
		LDP item-base CF	1.0461	1.0438	1.0427	1.0433	1.0414
		PNCF	1.0700	1.0620	1.0530	1.0389	1.0339
		Truncated PPPCE	1.0052	1.0038	1.0014	1.0037	1.0026
		PPPCE	0.9981	0.9980	0.9996	0.9989	0.9987
	RMSE	IBCF-DS	1.2464	1.2449	1.2447	1.2446	1.2445
		DPLCF	1.2842	1.2827	1.2813	1.2813	1.2790
		LDP item-base CF	1.3298	1.3257	1.3265	1.3260	1.3247
		PNCF	1.5120	1.4950	1.4790	1.4500	1.4290
		Truncated PPPCE	1.2830	1.2777	1.2756	1.2765	1.2751
		PPPCE	1.2727	1.2689	1.2713	1.2699	1.2685

Table 2 shows that with the increasing value of N, both MAE and RMSE of each algorithm continue to decrease, indicating that the increase in the number of neighbors will improve the accuracy of the recommendation system. PNCF performed poorly because it did not have the data reconstruction module. Among these differential privacy protection-based algorithms, PPPCE performed best in terms of MAE and RMSE, which were closest to the baseline algorithm IBCF-DS. Meanwhile, comparing with Truncated PPPCE, the average MAE of PPPCE is reduced by 8.2% (0.07) and the average RMSE of PPPCE is reduced by 8.3% (0.09) on the dataset of MovieLens 1M; the average MAE of PPPCE is reduced by 8.2% (0.07) and the average RMSE of PPPCE is reduced by 8.3% (0.09) on the dataset of Yahoo Music, indicating that the Bayesian estimation module has a good reconstruction effect on the perturbated data and can effectively improve the performance of the recommendation algorithm. Additionally, seen from Table 2, PPPCE outperforms DPLCF and LDP item-base CF on the two datasets, indicating that the Bayesian estimation method and the data sensitivity division method adopted in PPPCE perform better than the cardinality estimation method and frequency estimation method used in those two algorithms in terms of the data reconstruction.

4.3.2. Effect of $\mathsf{\epsilon }$ on Experimental Results

To ensure the effectiveness of privacy protection, we set the value of the privacy budget to be in the range of [0.1,1]. Figure 4 and Figure 5, respectively show the changes of MAE and RMSE on the dataset as the value of $\mathsf{\epsilon }$ changes. Due to the lack of the differential privacy protection mechanism, the curve of IBCF-DS remains unchanged.

Figure 4. MAE of each algorithm with different privacy budgets $\epsilon$. (a) Movielens 1M; (b) Yahoo Music.

Figure 5. Figure 5.RMSE of each algorithm with different privacy budgets $\epsilon$. (a) Movielens 1M; (b) Yahoo Music.

With the increase in ε of the two datasets, both MAE and RMSE show a downward trend. This is because as ε increases, the noise of data decreases and the availability of data increases. Seen from the above figures that PPPCF outperforms other algorithms except IBCF-DS on both datasets. First, each algorithm performs in terms of MAE on the dataset of MovieLens 1M, PPPCF is 9.2% (0.083) less than the Truncated PPPCE, 9.7% (0.089) less than DPLCF, 3.3% (0.028) less than LDP item-base CF, and 5.1% (0.044) less than PNCF. Then, each algorithm performs in terms of RMSE on the dataset of MovieLens 1M, PPPCF is 9.3% (0.106) less than the Truncated PPPCE, 10.6% (0.123) less than DPLCF, 4.2% (0.046) less than LDP item-base CF, and 5.7% (0.063) less than PNCF. Second, due to the data on the dataset of Yahoo Music is sparser, causing the effect of the Bayesian estimation mechanism, the cardinality estimation mechanism, and the frequency estimation mechanism is not obvious, but compared with PNCF without the data reconstruction module, these three data reconstruction algorithms can still bring obvious improvement in utility. Among the three algorithms with data reconstruction, the prediction error of PPPCF still performs best, indicating that PPPCF can indeed improve the accuracy of the recommendation algorithm.

4.3.3. Algorithmic Efficiency

PPPCF mainly consists of three operations: local data processing, similarity calculation, and rating prediction. The local data processing operation can be further divided into two parts, one is the classification and encoding of ratings, the other is the differential privacy protection for ratings. As the local operation needs to traverse all the user-rating vectors in the dataset, its time complexity is $O\left(n\right)$.

The item-item similarity calculation includes the similarity calculation of the sensitive rating pairs and the similarity calculation for the weakly sensitive rating pairs. Assuming that the number of sensitive rating pairs is $k$, according to the execution process of BBBCF, when calculating the similarity of an item-item pair, it should traverse the weakly sensitive rating pairs once and the sensitive rating pairs iter times, so the time complexity is $O\left(\left(\left(k-1\right)·iter\right)+m\right)$, and the time complexity of the similarity calculation on the server side is $O\left(\left(\left(k-1\right)·iter+m\right)·n^2\right)$. From Equation (11), the time complexity of the user’s local prediction for all items is$O\left(n\right)$. Choose the DPLCF and LDP item-based CF algorithms, which also apply the localized differential privacy frameworks, for the comparative analysis. According to the calculation process of these two algorithms, the time complexity of their local data processing is also $O\left(n\right)$, and the time complexity of the similarity calculation is $O\left(m·n^2\right)$. Additionally, since the rating prediction model used in the comparison algorithms is the same as PPPCF, so their time complexities of the rating prediction are $O\left(n\right)$. Compared with all the comparison algorithms, the time complexity of the local data processing section and the rating prediction section of PPPCF is the same as others, but in the similarity calculation section, the time complexity of PPPCF is higher due to the need to iteratively reconstruct the distribution of rating data. However, since $k\ll m$ and $iter\ll m$, it is still in the same range as all the comparison algorithms.

To further verify the results of the complexity analysis, taking the Movielens 1M and Yahoo Music datasets for example, assign $\epsilon$ and $N$ be 1, 100, then evaluate the time overhead of each algorithm on the two datasets, experimental results are shown in Table 3. Each algorithm spends about the same amount of time on local data processing operations, with the PPCF algorithm having the least time overhead. This is because the PPPCF algorithm classifies the data sensitivity before perturbing the data, which greatly reduces the amount of data that needs to be perturbed.

Table 3. Comparison of the time overhead.

Dataset	Runtime/s	DPLCF	LDP Item-Base CF	Truncated PPPCE	PPPCF
Yahoo Music	Local Data Processing	2.36	2.55	1.78	1.83
	Similarity Calculation	32.79	41.25	22.71	382.54
	Rating Prediction	53.47	52.80	47.59	57.39
MovieLens 1M	Local Data Processing	8.22	8.62	7.79	7.71
	Similarity Calculation	652.23	833.37	507.95	5497.68
	Rating Prediction	464.94	488.14	467.07	475.33

In the process of the similarity calculation, PPPCF is affected by $k$ (i.e., number of the sensitive rating pairs) and iter (i.e., times of iterations), so multiple iterations are required to ensure a lower prediction error, causing its time overhead for the similarity calculation is greater than the other comparison algorithms. Meanwhile, the benefit of high time overhead is that its prediction error is lower than others.

Additionally, PPPCF utilizes the Hamming distance to measure the similarity, and in the actual operation process, the Hamming distance between vectors can be obtained through XOR operation, which can improve the operation efficiency of PPPCF algorithm. This is confirmed by the fact that the Truncated PPPCE has a smaller time overhead in calculating similarity in Table 3. For the recommendation algorithm, the efficiency of the rating prediction calculation is particularly important, and the time taken to predict ratings for a single user on the Yahoo Music and Movielens 1M datasets is 6~7 ms and 78~81 ms, respectively, which can meet the actual needs.

In summary, although the PPPCF increases the time overhead caused by the similarity calculation on the server side, it does not affect the efficiency of the rating prediction, and the recommendation error of PPPCF is superior to other algorithms for comparison on datasets of different sparsity. This shows that under the interaction of the privacy sensitivity classification module and the Bayesian data reconstruction module, the similarity calculation is less affected by the perturbed data to achieve higher data availability and recommendation accuracy.

5. Conclusions

Based on the random flip mechanism, this paper proposes a collaborative filtering algorithm with differential privacy-preserving protection. In order to trade off the relationship between privacy protection and performance of recommendation, it classifies ratings into privacy-sensitive and weakly sensitive privacy ratings according to user’s personalized needs for privacy. Then, it brings a coding rule for ratings to effectively reduce unwanted noise resulting from privacy protection. Furthermore, it reconstructs the distribution of the perturbated data through the Bayesian joint estimation model. Experimental results on two public datasets verify that the proposed algorithm has better performance in trading off between the performance of recommendation system and the security of privacy, and it is of certain practical value. Due to the privacy protection reduces the recommendation efficiency of the system to a certain extent, in our future work, we will focus on the data pre-processing strategies, such as cleaning and classification of rating data to further reduce time consumption. Additionally, we will also consider doing some exploration in hybrid recommendation algorithms.