B-DP: Dynamic Collection and Publishing of Continuous Check-In Data with Best-Effort Differential Privacy

Abstract

Differential privacy (DP) has become a de facto standard to achieve data privacy. However, the utility of DP solutions with the premise of privacy priority is often unacceptable in real-world applications. In this paper, we propose the best-effort differential privacy (B-DP) to promise the preference for utility first and design two new metrics including the point belief degree and the regional average belief degree to evaluate its privacy from a new perspective of preference for privacy. Therein, the preference for privacy and utility is referred to as expected privacy protection (EPP) and expected data utility (EDU), respectively. We also investigate how to realize B-DP with an existing DP mechanism (KRR) and a newly constructed mechanism (EXP${}_Q$) in the dynamic check-in data collection and publishing. Extensive experiments on two real-world check-in datasets verify the effectiveness of the concept of B-DP. Our newly constructed EXP${}_Q$ can also satisfy a better B-DP than KRR to provide a good trade-off between privacy and utility.

1. Introduction

The explosive progress of mobile Internet and location technology, LBS (Location Based Service) applications, including Brightkite, Gowalla, Facebook and other social network platforms, generate a large number of check-in data every day. Check-in data generally include information such as time, locations, PoI (Points of Interest) attributes, mood and comments, and hence the check-in data has become a carrier of a user’s life trajectory and interest tendency [1,2,3,4]. However, a data analyst’s mining and analysis of the check-in data may directly or indirectly expose the sensitive information of a data provider [5,6,7,8,9]. There have been many privacy protection methods [10,11,12,13,14,15,16]. Some of them [10,11] rely on specific attack assumptions and background knowledge, and some methods [12,13,14,15] are based on differential privacy (DP) [17]. DP provides provable privacy protection, which is independent of the background knowledge and computational power of an attacker. The protection level of DP is evaluated by privacy budget [17]. When the privacy budget is relatively small, it has strong privacy protection, but the utility is often poor [17]. With the gradual integration of DP on practical applications, utility has become the bottleneck of its development and popularization.

In general, there is a contradiction between privacy and utility and it is necessary to be a trade-off [18,19]. In [19], the authors discussed a monotone trade-off in the semi-honest model. Therein, when the utility becomes worse, the privacy protection becomes stronger, and on the other hand, when the utility gets better, the privacy protection gets weaker. In many other DP theoretical studies, including strict $ϵ$-DP [17] and relaxed $(ϵ,\delta )$-DP [20], they often provide privacy priority and then make data more available or the best available, which is a kind of trade-off with satisfying utility as much as possible under the privacy guarantee. Unfortunately, the applications of DP in real-world do not seem to follow this principle completely. One of the best examples is the four applications in Apple’s MacOS Sierra (version 10.12), i.e., Emojis, New words, Deeplinks and Lookup Hints. When they collect the data, the privacy budget is set to only 1 or 2 per each datum, but the overall privacy budget for the four applications is as high as 16 per day [21]. Furthermore, Apple renews the available privacy budget every day, which would result in a potential privacy loss of 16 times the number of days that a user participated in DP data collection for the four applications [21]. It is far beyond the reasonable protection scope of DP [22].

Based on the above facts, when there exists a contradiction between privacy and utility, privacy is no longer a priority as suggested in the DP theoretical studies, but the most desirable way is to balance the preference for privacy and utility, where the preference for privacy and utility is referred to as expected privacy protection (EPP) and expected data utility (EDU), respectively. However, few researchers have proposed solutions to reasonably balance EPP and EDU except the authors in [23]. They proposed an adaptive DP and its mechanisms in a rational model, which can achieve a balance between the approximate EDU and the EPP by adding conditional filtering noise [23]. If the privacy protection intensity under the balance of the approximate EDU that is satisfied by the data analyst is not the expectation of the data provider, then it still cannot meet the EPP of the data provider. In addition, the absolute value range of the conditional filtering noise belongs to (0.5,1.5), which makes it easy to be attacked by background knowledge. Therefore, best-effort differential privacy (B-DP) is proposed to make the EDU satisfied first and then the EPP satisfied as much as possible in this paper. We face the following two basic challenges at least.

• If the EDU is to be satisfied first, then privacy protection may be no longer to be guaranteed by DP, how does it evaluate the guarantee degree of satisfying EPP as much as possible under B-DP?
• If there is a reasonable metric for the guarantee degree of satisfying EPP as much as possible under B-DP, does it exist an implementation mechanism (or algorithm) to realize B-DP?

With the challenges of B-DP above, this paper explores a typical application with dynamic collection and publishing of continuous check-in data, where the check-in scenario is a semi-honest model with an honest but curious data collector. Each check-in user visiting a POI generates a check-in state and perturbs his check-in state to a POI Center for his privacy protection, where the POI Center is a data collector. The frequency of check-in users are calculated by POI Center according to the received check-in states, which is approximately the check-in data distribution and used for publishing to data analysts. We assume that one check-in state is perturbed to only one check-in state and each publishing is required to satisfy the EDU first and then satisfy EPP as much as possible in the dynamic publishing, and moreover, the privacy to be protected is the check-in state of a user and the utility to be realized is the distribution of the check-in data with relative error as its metrics (see Section 4.1 for more details). In fact, since the relative error is used as a metric of the published distribution, it needs a distribution dependent privacy protection mechanism (or implementation) in order to satisfy the EPP as much as possible under the constraint of EDU. In addition, since each publishing is required to satisfy the EDU first and then satisfy EPP as much as possible in the dynamic publishing, it needs a algorithm to make the privacy protection under the constraint of EDU to be satisfied continuously as much as possible in the process of dynamic publishing. Therefore, this mechanism or algorithm will be proposed from a new perspective, which is different from the existing methods in literature.

1.1. Our Contributions

The main contributions of this paper are concluded as follows.

• A privacy protection concept of B-DP and two metrics of privacy guarantee degree are put forward. B-DP discussed in this paper is an expansion of the concept of DP, which can satisfy the EDU first and then provide the EPP as much as possible to be usefull for real-world applications. It uses two new metrics including the point belief degree (see Definition 4) and the regional average belief degree (see Definition 5) to quantify the degree of privacy protection for any expected privacy budget (see Section 4.2), rather than for DP itself by the privacy budget $ϵ$ to evaluate only one EPP with the expected privacy budget equal to $ϵ$. In addition, the regional average belief degree can be used as the average guarantee degree of the EPP in a region including multiple expected privacy budgets. To the best of our knowledge, it is a new discussion and definition of B-DP that is different from the existing literature, and it uses two new metrics to explore and analyze the performance of privacy from a new perspective of the preference for privacy.
• An EXP${}_Q$ mechanism is proposed (see Definition 10). The newly constructed EXP${}_Q$ mechanism can be used to the categorical data for privacy protection, which smartly alters the privacy budget based on its probability in the data distribution to make itself to realize a better B-DP compared to the existing KRR mechanism [24,25]. Thereby, it also verifies that B-DP can be better realized to provide a good trade-off between privacy and utility.
• The dynamic algorithm with the implementation algorithms of two perturbation mechanisms is proposed to realize the dynamic collection and publishing of continuous check-in data and meanwhile to satisfy B-DP. The two perturbation mechanisms include the newly constructed EXP${}_Q$ and a classical DP mechanism KRR [25,26] (a simple local differential privacy (LDP) mechanism). We take KRR as an example to show how to realize B-DP based on the existing DP mechanisms for the categorical data. Moreover, the number of domain values of both KRR and EXP${}_Q$ is more than 2 and both the randomized algorithms based on them only take one value as input and one value as output. In addition, the dynamic algorithm can also be used to other applications of social behavior except check-in data.

1.2. Outline

The remainder of this paper is organized as follows: Section 2 summarizes the related work on the trade-off methods, utility metrics of relative error and LDP mechanisms. Section 3 presents conceptual background of DP and details of KRR mechanism and utility metrics. Section 4 introduces the system model, the relevant definitions of B-DP, including two metrics of the guarantee degree, etc., and model symbolization of the check-in data. Section 5 introduces the design and implementation of B-DP mechanisms and Section 6 describes the design of B-DP mechanism algorithm in the dynamic collection and publishing. Section 7 provides the experimental evaluation of the dynamic collection and publishing algorithm based on both two B-DP mechanisms. Finally, we provide a discussion and conclusion in Section 8.

2. Related Work

DP has become a research hotspot in the field of privacy protection since Dwork [12] proposed it in 2006. The model of DP starts from the traditional centralization [15,18], gradually grows to be distributed [27], and develops to be localization [24,28] and even to be personalized localization [29] and so on. It is not only the evolution process of DP technique, but also the comprehensive embodiment of the gradual integration of DP technique with real-world applications. However, no matter how it evolves, the two themes running through DP are privacy and utility [18], which is also focused by this paper. Table 1 summarizes the mainly related work from the pespective of privacy and utility priority as well as their metrics, the used privacy mechanism and the focusing problem with EPP and EDU. It will be divided into three categories to show its details.

Table 1. Comparison of existing literature with the method proposed in this paper.

Selected Papers	Mechanism	Utility First	Privacy First	Privacy Metrics	Utility Metrics	EPP & EDU Involved
Katrina et al. [30]	Laplace	Yes	No	Central DP	Absolute error	No
Liu et al. [23]	Gauss with conditional filtering noise	Yes	No	Central DP	Relative error	Yes, but it may not provide EPP as much as possible
Maryam et al. [31]	Laplace	No	Yes	Central DP	Relative error	No
Xiao et al. [18]	Laplace	No	Yes	Central DP	Relative error	No
Kairouz et al. [25]	W-RR	No	Yes	LDP	KL divergence	No
Erlingsson et al. [28]	RAPPOR	No	Yes	LDP	Standard deviation	No
Bassily et al. [32]	S-Hist	No	Yes	LDP	Absolute error	No
Chen et al. [29]	PCEP	No	Yes	PLDP	KL divergence/relative error	No
Kairouz et al. [24,25]	KRR	No	Yes	LDP	KL divergence	No
Our paper	EXP${}_Q$	Yes	No	(Local) B-DP	Relative error	Yes, and it provides EPP as much as possible

• Trade-off model with utility first. The majority of DP research is based on the trade-off model with privacy first, while there are few relevant ones on the trade-off model with utility first. Therein, Katrina et al. [30] proposed a generalized “noise reduction” framework based on the modified “Above Threshold” algorithm [33] to minimize the empirical risk of privacy (ERM) on the premise of utility priority, but the scheme is only applicable to the framework that minimizes the empirical risk of privacy, where the privacy minimized may not be able to meet the EPP. Liu et al. proposed firstly that DP satisfies the monotonic trade-off between privacy and utility and its associated bounded monotone trade-off under the semi-honest model. They showed that there is no trade-off under the rational model, while unilateral trade-off could lead to utility disaster or privacy disaster [18,23,34]. They also presented an adaptive DP and its mechanisms under the rational model, which can realize the trade-off between approximately EDU and EPP by adding conditional filtering noise [23], but the mechanisms are probably not able to meet the expectation of data provider for privacy protection and are easily attacked by background knowledge because of the adding conditional filtering noise. Most importantly, the above two utility-first research [23,30] do not provide a quantitative metrics of the unmet privacy protection or the unmet degree of EPP, whereas this paper presents two detailed quantitative metrics including the point belief degree and the regional average belief degree to evaluate the privacy from a new perspective of preference for privacy.
• Utility metrics of relative error. Maryam et al. [31] presented DP in real-world applications, which discussed how to add Laplace [12] noise from a view of utility. They studied the relationship between the cumulative probability of noise and the privacy level in Laplace mechanism and combined with the relative error metrics to discuss how to use a DP mechanism reasonably without losing the established utility. However, the literature does not delve into the details that how the guarantee degree of privacy protection will be changed when utility is satisfied. Xiao et al. [18] presented a DP publishing algorithm on a batch query using resampling technique of correlation noise to reduce noise added and improve data utility. When the algorithm picks the priority items each time, it is based on the intermediate results with noise, and the intermediate results with noise are not enough to reflect the original order of data. In this way, there is a bias in adjusting the privacy budget allocation, which may cause the query items that should be optimized to be not optimized, thus affecting the utility of published data. However, the literature is a classical example of optimizing utility with privacy first, which runs counter to the theme of this paper. In addition, the above two schemes are essentially based on the central DP and use continuous Laplace mechanism, which are different from the LDP (discrete) data statistics and release required by the check-in application in this paper. Therefore, these schemes cannot be directly applied to the applications this paper considers.
• LDP mechanisms. In 1965, Warner first proposed the randomized response technique (W-RR) to collect statistical data on sensitive topics and keep the sensitive data of contributing individuals confidential [35]. Although W-RR can strictly satisfy $ϵ$-LDP [25] in one survey statistics, multiple collections on the same survey individuals will weaken the privacy protection intensity [12]. Therefore, Erlingsson et al. [28] used a double perturbation scheme combining permanent randomized response with instantaneous randomized response, namely, RAPPOR, to expand the application of W-RR, and it has been used by Google in Chrome browser to collect users’ behavior data. In addition, RAPPOR also uses Bloom Filter technology [36] as the encoding method, which maps the statistical attributes into a binary vector. Finally, the mapping relation and Lasso regression method [37] are combined to reconstruct the frequency statistics corresponding to the original attribute string. Due to the high communication cost of RAPPOR, Bassily et al. [32] proposed the S-Hist method. In the method, each user first encodes his attributes, then randomly selects one of the bits and uses the randomized response technique to perturb it, and finally sends the result of the perturbation to the data collector, so as to reduce the communication cost. Chen et al. [29] proposed a PCEP mechanism and designed a PLDP (personalized LDP) applied to spatial data with it, aiming to protect the users’ location information and count the number of users in the area. Therein, the privacy budget of the scheme is determined by the users’ personalization, and hence the utility depends on the users’ individual behavior settings. In addition, the mechanism combines the S-Hist [32] method and adopts the random projection technique [38]. Although it can greatly reduce the communication cost, it still has the problem of unstable query precision. Based on the check-in application with multiple check-in spots in this paper, the KRR mechanism [24,25] just easily fits this application with no prior data distribution knowledge, but it is not very good for B-DP. In addition, DP has already been studied in these applications, such as social networks [39,40], recommender systems [41], data publishing [42,43,44], deep learning [45], reinforcement learning [46] and federated learning [47].

3. Preliminaries

In this section, the key notations used in this paper are given in Table 2.

Table 2. Notations.

Symbol	Description
EDU	Expected data utility
EPP	Expected privacy protection
${\epsilon }_e$	Expected privacy budget
Region(${ϵ}_e$)	Expected privacy protection region around ${ϵ}_e$
$\eta$	Expected data utility
${ϵ}_{\eta }$	The privacy budget of a differential privacy mechanism that just meets the expected data utility $\eta$
$C_{{ϵ}_e}$	Point belief degree of ${ϵ}_e$
$C_{Region({ϵ}_e)}$	Regional average belief degree of Region(${ϵ}_e$)
$\mathbf{p}$	Original data distribution
$\tilde{\mathbf{p}}$	Perturbed data distribution
$\widehat{\mathbf{p}}$	Estimated data distribution
S	Check-in state space
$\mathbf{h}(\mathbf{S})$	Original check-in counts vector
$\tilde{\mathbf{h}}(\mathbf{S})$	Perturbed check-in counts vector
$\widehat{\mathbf{h}}(\mathbf{S})$	Estimated check-in counts vector
Q	Perturbation probability matrix
$q_{ij}$	The perturbation probability of the original check-in state $S_j$ to the check-instate $S_i$
KRR	k-ary randomized response mechanism
EXP${}_Q$	Perturbation mechanism
$\gamma$	Privacy setting parameter
${\gamma }_{\eta }$	Privacy setting parameter with satisfying the expected data utility $\eta$
${\kappa }_n$	The parameter of privacy protection intensity change point
w	Modified estimate parameter
Re${}_{thredthold}$	Update threshold parameter
$err(\mathbf{p},\widehat{\mathbf{p}})$	The maximum relative error between $\mathbf{p}$ and $\widehat{\mathbf{p}}$

3.1. Differential Privacy (DP)

Differential privacy (DP), broadly speaking, is a privacy protection technique that does not depend on an attacker’s background knowledge and computational power [17,20,48]. It can be generally divided into central DP and LDP depending on whether it is based on a trusted data collector [33]. The formal definitions of these two types of DP are given as follows.

Definition 1

(($ϵ,\delta$)-(Central) DP [17,20]). A randomized algorithm M and a set S of all possible outputs of M, for a given dataset D and any adjacent dataset $D^{\prime }$ that differ on at most one record, if M satisfies the following inequality, then it is said that M satisfies $(ϵ,\delta )$-(central) DP.

$$P[M(D)\in S]\le e^{ϵ}\times P[M(D^{\prime })\in S]+\delta ,$$

(1)

where $P[·]$ represents the risk of privacy disclosure and is controlled by the randomness of algorithm M, the parameter ϵ is called privacy budget that represents the level of privacy protection, and δ represents the probability of failure to satisfy ϵ-(central) DP. When $\delta =0$, M satisfies the ϵ-(central) DP.

Definition 2

($(ϵ,\delta )$-LDP [25,26]). A randomized algorithm $\mathcal{K}$, for a given dataset χ, any $x,x^{\prime }\in \chi$ and any $y\in Range(\mathcal{K})$, is said to satisfy $(ϵ,\delta )$-LDP if $\mathcal{K}$ satisfies

$$P[\mathcal{K}(x)=y]\le e^{ϵ}\times P[\mathcal{K}(x^{\prime })=y]+\delta ,$$

(2)

where $P[·],ϵ$ and δ have the similar meanings as above in Definition 1.

In the check-in application of this paper, the POI Center is an honest and curious data collector, even if the POI Center or other attackers can obtain the check-in state submitted by a user, they cannot conclusively infer the original check-in state of the user. If $\mathcal{K}$ can satisfy $(ϵ,\delta )$-LDP to protect the check-in state of the user, then it needs to meet the following definition.

Definition 3

(Check-in state of $(ϵ,\delta )$-LDP). A user u generates a check-in in a POI, whose check-in state variable is denoted as $s^u$ with $s^u\in \{S_1,S_2,\cdots ,S_n\}$. Assume that the original check-in state of u is $S_j$ or $S_{j^{\prime }}$ for $j,j^{\prime }\in [1,n]$. Moreover, $S_j$ and $S_{j^{\prime }}$ generate the same check-in state $S_i$ for $i\in [1,n]$ after being perturbed by a randomized algorithm $\mathcal{K}$, respectively, and the perturbed check-in state variable is ${\tilde{s}}^u$ with ${\tilde{s}}^u\in \{S_1,S_2,\cdots ,S_n\}$. If there exists an $ϵ\in R^{+}$ such that $\mathcal{K}$ satisfies the following constraints for $i,j,j^{\prime }\in [1,n]$,

$$P({\tilde{s}}^u=S_i|s^u=S_j)\le e^{ϵ}P({\tilde{s}}^u=S_i|s^u=S_{j^{\prime }})+\delta ,$$

(3)

where, $P({\tilde{s}}^u=S_i|s^u=S_j)$ and $P({\tilde{s}}^u=S_i|s^u=S_{j^{\prime }})$ are the perturbation probabilities of the original check-in states $S_j$ and $S_{j^{\prime }}$ to the check-in state $S_i$, respectively, then $\mathcal{K}$ will enable the check-in state to satisfy $(ϵ,\delta )$-LDP. When $\delta =0$, $\mathcal{K}$ satisfies ϵ-LDP.

Property 1

(Parallel composition [49]). Assume that randomized algorithms are ${\mathcal{K}}_1,{\mathcal{K}}_2,\cdots ,{\mathcal{K}}_n$ and their privacy budgets are ${ϵ}_1,{ϵ}_2,\cdots ,{ϵ}_n$, respectively, then for the disjoint datasets $D_1,D_2,\cdots ,$ $D_n$, an algorithm $\mathcal{K}({\mathcal{K}}_1(D_1),{\mathcal{K}}_2(D_2),\cdots ,{\mathcal{K}}_n(D_n))$ provides max$({ϵ}_i)$-(local) DP, and the level of privacy protection it provides depends on the largest privacy budget.

3.2. KRR Mechanism

KRR is a LDP mechanism [24,25], which satisfies the following probability distribution,

$$P(y|x)=\frac{1}{e^{ϵ}+k-1}\left\{\begin{array}{c}\hfill e^{ϵ},y=x\\ \hfill 1,y\ne x\end{array}\right.$$

(4)

where $x,y\in \chi$ and $|\chi |=k.$

KRR is a more general form of the randomized response mechanism of W-RR, that is, when $k=2$, KRR degenerates into W-RR.

3.3. Utility Metrics

In this paper, the worst relative error of POIs in check-in statistics will be used to measure the overall utility of the check-in application, where the calculation formula of the relative error of $PO{I}_i$ is as follows,

$$err(r_i,r_i^{\ast })=\frac{|r_i-r_i^{\ast }|}{max\{r_i,\varphi \}},$$

(5)

where $r_i^{\ast }$ is the estimated result of $PO{I}_i$ check-in statistics after LDP protection, $r_i$ is the real check-in result of the $PO{I}_i$, and the parameter $\varphi$ is a constant to avoid the situations that $r_i=0$ causes the denominator to be 0 or $r_i$ is too small [18,50,51]. For the convenience of analysis, this paper will use the relative root mean square error for the utility metrics approximately, the specific formula is as follows,

$$err(r_i,r_i^{\ast })=\frac{\sqrt{\xi (r_i,r_i^{\ast })}}{max\{r_i,\varphi \}},$$

(6)

where $\xi (r_i,r_i^{\ast })$ is the expectation of the mean square error between the real statistical result $r_i$ and the statistical estimate result $r_i^{\ast }$ after LDP protection, and the parameter $\varphi$ is defined as above. Then, the formula for calculating the maximum relative error of n POIs is as follows,

$$err(r,r^{\ast })=max(err(r_i,r_i^{\ast }))=max(\frac{\sqrt{\xi (r_i,r_i^{\ast })}}{max\{r_i,\varphi \}}).$$

(7)

As above, $r_i,r_i^{\ast }$ not only can represent data distribution, but also can represent frequency or counts.

4. Problem Formulations

4.1. System Model

As shown in Figure 1, there are three types of participants, namely, check-in users (data providers), POI Center (data collector), and data analysts (for example, POI managers) in the check-in model. Each check-in user visiting a POI generates a check-in state and sends it to POI Center through a terminal with the check-in APP, where each check-in state corresponds to a count and the check-in state belongs to catagory data. POI Center calculates the counts and frequency of check-in users’ visiting POIs according to the received check-in states, where frequency is approximately the check-in data distribution and used for publishing to data analysts. In addition, it is assumed that each check-in user is independent of each other and only one check-in state is submitted by a check-in user in one publishing. It is also assumed that the check-in scenario is a semi-honest model, in which the POI Center is an honest but curious data collector, and the check-in state of a user is sensitive. Hence, a user will adopt a perturbation mechanism (for example, LDP mechanism) to perturb his check-in state for his privacy protection, and then sends it to the POI Center. Therein, it is assumed that one check-in state is perturbed to only one check-in state.

Figure 1. POI check-in model. Therein, $S_i,S_j$ and $S_k$ represent check-in states. $\tilde{\mathbf{h}}(\mathbf{S})$ and $\tilde{\mathbf{p}}$ represent the check-in counts and the check-in frequency (data distribution) in perturbation phase, respectively, while $\widehat{\mathbf{h}}(\mathbf{S})$ and $\widehat{\mathbf{p}}$ represent the check-in counts and the check-in frequency (data distribution) in construction phase, respectively. K represents a perturbation mechanism. The more details can also be seen in Section 4.3.

In this paper, we focus on the dynamic collection and publishing of continuous check-in data with both privacy and utility requirements, where the privacy to be protected is the check-in state of a user and the utility to be realized is the distribution of the check-in data with relative error as its metrics. Therein, the privacy refers to EPP, which is the preference for privacy of a user, and the utility refers to EDU, which is the preference for utility of a data analyst. Moreover, each publishing is required to satisfy the EDU first and then satisfy EPP as much as possible in the dynamic publishing. Thereby, we adopt B-DP based on the LDP model including perturbation, aggregation, reconstruction and publishing, and we also need to have the process of initializing or updating the perturbation mechanism K at least to make every publishing to satisfy the EPP as much as possible under the EDU satisfied first in the dynamic publishing, as shown in Figure 1.

4.2. The Related Concepts of B-DP

In the concept of best-effort differential privacy (B-DP), there is an expected privacy protection (EPP) and an expected data utility (EDU), respectively. When the two cannot be satisfied simultaneously, the EDU should be satisfied first and the EPP should be satisfied as much as possible. Since the protection level of DP is evaluated by privacy budget [17], the preference for privacy also refers to the preference for the privacy budget in the B-DP. Hence, the EPP refers to a data provider’s preference for the privacy budget and we define this privacy budget as the expected privacy budget symbolized as ${ϵ}_e$. We use $Region({ϵ}_e)$ to symbolize the expected privacy protection region, which refers to a data provider’s preference for a region including multiple expected privacy budgets.

We use $\eta$ to symbolize the EDU. In this paper, the expectation of the maximum relative error of Formula (7) is used to measure data utility. When the expectation of the maximum relative error of Formula (7) is less than or equal to $\eta$, it means that the EDU is satisfied; when equal, it means that the EDU is just satisfied. The privacy budget of a DP mechanism that just satisfies the EDU $\eta$ is symbolized as ${ϵ}_{\eta }$.

Definition 4

($C_{{ϵ}_e}$-Point belief degree). It defines the guarantee degree of EPP under the expected privacy budget ${ϵ}_e$, which can be provided by the ${ϵ}_{\eta }$-DP mechanism, as the point belief degree, and the symbol is denoted as $C_{{ϵ}_e}$. Moreover, $C_{{ϵ}_e}={\sum }_{i=1}^n{\tilde{p}}_i\chi ({ϵ}_i,{ϵ}_e)$, where n represents the number of POIs in check-in application, ${\tilde{p}}_i$ represents the probability of $PO{I}_i$ perturbed by ${ϵ}_{\eta }$-DP mechanism, ${ϵ}_i$ represents the actual privacy budget of $PO{I}_i$, and $\chi ({ϵ}_i,{ϵ}_e)$ represents an indicator function for whether the EPP is satisfied, which is defined as follows,

$$\chi ({ϵ}_i,{ϵ}_e)=\left\{\begin{array}{c}0,{ϵ}_i>{ϵ}_e\hfill \\ 1,{ϵ}_i\le {ϵ}_e\hfill \end{array}\right..$$

(8)

Definition 5

($C_{Region({ϵ}_e)}$-Regional average belief degree). The average guarantee degree of the EPP under the expected privacy protection region $Region({ϵ}_e)$, which can be provided by the ${ϵ}_{\eta }$-DP mechanism, is defined as the regional average belief degree, and the symbol is denoted as $C_{Region({ϵ}_e)}$. When $Region({ϵ}_e)=\{{ϵ}_{e_1},{ϵ}_{e_2},\cdots ,{ϵ}_{e_K}\}$ and ${ϵ}_{e_1}<{ϵ}_{e_2}<\cdots <{ϵ}_{e_K}$ for $K\ge 2$, it defines

$$C_{Region({ϵ}_e)}=\frac{1}{{ϵ}_{e_K}-{ϵ}_{e_1}}\sum _{k=1}^{K-1}({ϵ}_{e_{k+1}}-{ϵ}_{e_k})C_{{ϵ}_{e_k}},$$

(9)

where $C_{{ϵ}_{e_k}}$ can refer the definition of point belief degree.

Definition 6

($({ϵ}_{\eta },C_{{ϵ}_e})$-B-DP). The DP mechanism that just satisfies the EDU η with the point belief degree $C_{{ϵ}_e}$ of the expected privacy budget ${ϵ}_e$ is defined as $({ϵ}_{\eta },C_{{ϵ}_e})$-B-DP. Therein, the $({ϵ}_{\eta },C_{{ϵ}_e})$-B-DP, where the point belief degree $C_{{ϵ}_e}$ is maximum, is defined as $({ϵ}_{\eta },C_{{ϵ}_e})$-Best-B-DP.

Definition 7

($({ϵ}_{\eta },C_{Region({ϵ}_e)})$-B-DP). The DP mechanism that just satisfies the EDU η with the regional average belief degree $C_{Region({ϵ}_e)}$ of the expected privacy protection region $Region({ϵ}_e)$ is defined as $({ϵ}_{\eta },C_{Region({ϵ}_e)})$-B-DP. Therein, the $({ϵ}_{\eta },$ $C_{Region({ϵ}_e)})$-B-DP, where the regional average belief degree $C_{Region({ϵ}_e)}$ is maximum, is defined as $({ϵ}_{\eta },C_{Region({ϵ}_e)})$-Best-B-DP.

Note that, generally, B-DP includes both central B-DP and local B-DP, which depends on whether it is based on a trusted data collector the same as the DP. This paper focuses on local B-DP.

4.3. Model Symbolization

Let $PO{I}_i$ with $i\in [1,n]$ represent n POIs in check-in scenario, and the check-in state space is $S=\{S_1,S_2,\cdots ,S_n\}$ where $S_i$ is the check-in state of $PO{I}_i$. Let $s^u,{\tilde{s}}^u,{\widehat{s}}^u\in S$ be variables of the original check-in state, the perturbed check-in state and the estimated check-in state of the user u, respectively. Let $\mathbf{p},\tilde{\mathbf{p}}$ and $\widehat{\mathbf{p}}$ be the probability distributions of the original check-ins, the perturbed check-ins and the estimated check-ins, respectively, where $\mathbf{p}={[p_1,p_2,\cdots ,p_n]}^T$, $\tilde{\mathbf{p}}={[{\tilde{p}}_1,{\tilde{p}}_2,\cdots ,{\tilde{p}}_n]}^T$ and $\widehat{\mathbf{p}}={[{\widehat{p}}_1,{\widehat{p}}_2,\cdots ,{\widehat{p}}_n]}^T$. Assume that it is the same probability distribution law for all the users, that is, $p_i=P(s^u=S_i)$, ${\tilde{p}}_i=P({\tilde{s}}^u=S_i)$ and ${\widehat{p}}_i=P({\widehat{s}}^u=S_i)$ for any $i\in [1,n]$ and u. $\mathbf{h}(\mathbf{S})={[h(S_1),h(S_2),\cdots ,h(S_n)]}^T$, $\tilde{\mathbf{h}}(\mathbf{S})={[\tilde{h}(S_1),\tilde{h}(S_2),\cdots ,\tilde{h}(S_n)]}^T$ and $\widehat{\mathbf{h}}(\mathbf{S})={[\widehat{h}(S_1),\widehat{h}(S_2),\cdots ,\widehat{h}(S_n)]}^T$ represent the original check-in counts vector, the perturbed check-in counts vector and the estimated check-in counts vector with $m\in N^{+}$ users, respectively.

Definition 8

(Random perturbation and perturbation probability matrix Q). The process for any user u to change check-in state from $S_j$ to $S_i$ with a certain perturbation probability is called random perturbation, and the perturbation probability is denoted as $q_{ij}=P({\tilde{s}}^u=S_i|s^u=S_j)$ with $S_i,S_j\in S$. The matrix composed of $q_{ij}$ for any $i,j\in [1,n]$ is called the perturbation probability matrix Q, where $Q={(q_{ij})}_{n\times n}$ and ${\sum }_{i=1}^n{q}_{ij}=1$ for any $j\in [1,n]$.

Therefore, the perturbed probability distribution $\tilde{\mathbf{p}}$, the original probability distribution $\mathbf{p}$ and the perturbation probability matrix Q have the following relationship

$$\begin{array}{cc}\hfill \tilde{\mathbf{p}}& =Q\mathbf{p}.\hfill \end{array}$$

(10)

From Equation (10), it can be seen that $\widetilde{p_i}={\sum }_{j=1}^n{q}_{ij}{p}_j$ for any $i\in [1,n]$. Obviously, $\widetilde{p_i}$ and $p_i$ are not always equal, and hence the result of the perturbation is biased. Assume that Q is always reversible and its inverse matrix is defined as $R=Q^{-1}={(r_{ij})}_{n\times n}$. Therefore, it can get the following theorem.

Theorem 1.

The check-in counts vector $\mathbf{h}(\mathbf{S})$ is perturbed by the perturbation probability matrix Q to obtain the perturbed check-in counts vector $\tilde{\mathbf{h}}(\mathbf{S})$ and then it is corrected by the inverse matrix R. The estimated check-in counts vector $\widehat{\mathbf{h}}(\mathbf{S})=R\tilde{\mathbf{h}}(\mathbf{S})$ satisfies $E[\widehat{\mathbf{h}}(\mathbf{S})]=\mathbf{h}(\mathbf{S})$.

Proof. 

Since $E[\widehat{\mathbf{h}}(\mathbf{S})]=E[R\tilde{\mathbf{h}}(\mathbf{S})]=E[RQ\mathbf{h}(\mathbf{S})]$, and $RQ=I$, it has $E[\widehat{\mathbf{h}}(\mathbf{S})]=E[\mathbf{h}(\mathbf{S})]=\mathbf{h}(\mathbf{S})$. Therefore, the result follows. □

Theorem 1 states that the estimated check-in counts vector $\widehat{\mathbf{h}}(\mathbf{S})$ obtained after the correction of the inverse matrix R is an unbiased estimate of the original check-in counts vector $\mathbf{h}(\mathbf{S})$.

Here, the relative root mean square error $err(h(S_i),\widehat{h}(S_i))$ of the original check-in counts $h(S_i)$ and the estimated check-in counts $\widehat{h}(S_i)$ for n POIs and m users on $PO{I}_i$ can be calculated as follows,

$$\begin{array}{c}\hfill err(h(S_i),\widehat{h}(S_i))=\frac{\sqrt{Var[\widehat{h}(S_i)]}}{E[\widehat{h}(S_i)]},\end{array}$$

(11)

where $\varphi =1$ and $Var[\widehat{h}(S_i)]$ can be calculated as follows,

$$\begin{array}{c}\hfill Var[\widehat{h}(S_i)]=\sum _{j=1}^n{r}_{ij}^2(\sum _{k=1}^n{q}_{jk}h(S_k))-h(S_i).\end{array}$$

(12)

Theorem 2.

The relative root mean square error between the original probability $p_i$ of $PO{I}_i$ check-ins and the estimated probability ${\widehat{p}}_i$ of $PO{I}_i$ check-ins is $err(p_i,{\widehat{p}}_i)=\frac{\sqrt{Var[\widehat{h}(S_i)]}}{E[\widehat{h}(S_i)]}$, where $i\in [1,n].$

Proof. 

The relative root mean square error between the original probability $p_i$ of $PO{I}_i$ check-ins and estimated probability ${\widehat{p}}_i$ of $PO{I}_i$ check-ins can be represented as follows,

$$\begin{array}{c}\hfill err(p_i,{\widehat{p}}_i)=\frac{\sqrt{E[{(p_i-{\widehat{p}}_i)}^2]}}{p_i}=err(h(S_i),\widehat{h}(S_i)).\end{array}$$

(13)

Therefore, $err(p_i,{\widehat{p}}_i)=\frac{\sqrt{Var[\widehat{h}(S_i)}]}{E[\widehat{h}(S_i)]}$, where $i\in [1,n].$ □

According to Formulas (11)–(13), it can be known $err(p_i,{\widehat{p}}_i)$, Q and $\mathbf{p}$ are related.

If $max(err(p_i,{\widehat{p}}_i))=\eta$ means that the EDU is just satisfied, then Q should satisfy the following constraints according to B-DP.

$$\begin{array}{cc}\hfill & max(err(p_i,{\widehat{p}}_i))\le \eta ,i\in [1,n],\hfill \\ \hfill & q_{ij}\le e^{{ϵ}_i}{q}_{i{j}^{\prime }},\forall i,j,j^{\prime }\in [1,n],{ϵ}_i\ge 0,\hfill \\ \hfill & \sum _{i=1}^n{q}_{ij}=1,\forall j\in [1,n].\hfill \end{array}$$

(14)

Assume that there is a randomized algorithm $\mathcal{K}$ with a perturbation probability matrix Q, which can provide the expected privacy budget ${ϵ}_e$ with the point belief degree $C_{{ϵ}_e}={\sum }_{i=1}^n{\tilde{p}}_i\chi ({ϵ}_i,{ϵ}_e)$. Then, if $\mathcal{K}$ wants to satisfy $({ϵ}_{\eta }=max({ϵ}_i),C_{{ϵ}_e})$-Best-B-DP, it should still need to maximize $C_{{ϵ}_e}$. Therefore, $\mathcal{K}$ should satisfy the following optimization problem.

$$\begin{array}{cc}\hfill & \underset{Q}{maximize}{C}_{{ϵ}_e}\hfill \\ \hfill & s.t.max(err(p_i,{\widehat{p}}_i))\le \eta ,i\in [1,n],\hfill \\ \hfill & q_{ij}\le e^{{ϵ}_i}{q}_{i{j}^{\prime }},\forall i,j,j^{\prime }\in [1,n],{ϵ}_i\ge 0,\hfill \\ \hfill & \sum _{i=1}^n{q}_{ij}=1,\forall j\in [1,n].\hfill \end{array}$$

(15)

Similarly, it is assumed that $\mathcal{K}$ can provide the expected privacy protection region $Region({ϵ}_e)=\{{ϵ}_{e_1},{ϵ}_{e_2},\cdots ,{ϵ}_{e_K}\}$ with the regional average belief degree $C_{Region({ϵ}_e)}=\frac{1}{{ϵ}_{e_K}-{ϵ}_{e_1}}{\sum }_{k=1}^{K-1}({ϵ}_{e_{k+1}}-{ϵ}_{e_k})C_{{ϵ}_{e_k}}$, where $C_{{ϵ}_{e_k}}$ is the point belief degree of the expected privacy budget ${ϵ}_{e_k}$. Then, if $\mathcal{K}$ wants to satisfy $({ϵ}_{\eta }=max({ϵ}_i),C_{Region({ϵ}_e)})$-Best-B-DP, it should still need to maximize $C_{Region({ϵ}_e)}$. Therefore, $\mathcal{K}$ should satisfy the following optimization problem.

$$\begin{array}{cc}\hfill & \underset{Q}{maximize}{C}_{Region({ϵ}_e)}\hfill \\ \hfill & s.t.max(err(p_i,{\widehat{p}}_i))\le \eta ,i\in [1,n],\hfill \\ \hfill & q_{ij}\le e^{{ϵ}_i}{q}_{i{j}^{\prime }},\forall i,j,j^{\prime }\in [1,n],{ϵ}_i\ge 0,\hfill \\ \hfill & \sum _{i=1}^n{q}_{ij}=1,\forall j\in [1,n].\hfill \end{array}$$

(16)

From the above optimization Equations (15) and (16), in each optimization problem, it can be concluded that the perturbation probability matrix Q contains $n^2$ unknown variables, $n^3$ inequality constraints, n equality constraints and one EDU $\eta$ constraint. Therefore, directly solving the optimization problem is a huge challenge, especially in the case of a large domain size n. Therefore, two simplified models are considered in this paper and we will present the details one by one in the following section.

5. Design and Implementation of B-DP Mechanism

This section includes the design of two B-DP mechanisms and their implementation algorithms. One is based on a classical LDP mechanism KRR, and the other is based on the newly constructed mechanism EXP${}_Q$ in this paper. The number of domain values of both two mechanisms is more than 2. Moreover, we combine three data distributions with the typical non-uniformity and two B-DP mechanisms to directly show and analyze the two metrics proposed in this paper, including the point belief degree and the regional average belief degree.

5.1. B-DP Mechanism Based on KRR

Without prior knowledge of the data distribution, we assume that it is uniform. Setting ${ϵ}_i={ϵ}_j={ϵ}_{\eta },$$q_{ii}=p,q_{ij}=q$ for $i,j\in [1,n],$$j\ne i$ and $q_{ii}\ge q_{ij}$, then it has $p=\frac{e^{{ϵ}_{\eta }}}{e^{{ϵ}_{\eta }}+n-1},q=\frac{1}{e^{{ϵ}_{\eta }}+n-1}$, and $err(\mathbf{p},\widehat{\mathbf{p}})=\eta$. Therefore, the privacy budget of KRR here is not arbitrary, which is constrained by the EDU of $err(\mathbf{p},\widehat{\mathbf{p}})=\eta$.

Definition 9

(${ϵ}_{\eta }$-KRR). KRR that just meets the EDU η is called ${ϵ}_{\eta }$-KRR.

Here, it can also derive the following theorem.

Theorem 3.

If there exists ${ϵ}_{\eta }$-KRR, then it satisfies $ϵ={ϵ}_{\eta }$-LDP. Moreover, the point belief degree of ${ϵ}_{\eta }$-KRR is

$$C_{{ϵ}_e}=\left\{\begin{array}{c}0,{ϵ}_e<{ϵ}_{\eta }\hfill \\ 1,{ϵ}_e\ge {ϵ}_{\eta }\hfill \end{array}.\right.$$

(17)

Proof. 

If there exists ${ϵ}_{\eta }$-KRR, then it has $p=\frac{e^{{ϵ}_{\eta }}}{e^{{ϵ}_{\eta }}+n-1},q=\frac{1}{e^{{ϵ}_{\eta }}+n-1}$. Since $ϵ=ln(\frac{p}{q})={ϵ}_{\eta }$, it satisfies $ϵ={ϵ}_{\eta }$-LDP. According to $C_{{ϵ}_e}={\sum }_{i=1}^n{\tilde{p}}_i\chi ({ϵ}_i,{ϵ}_e)$, when ${ϵ}_e<{ϵ}_{\eta }$, $\chi ({ϵ}_i,{ϵ}_e)=\chi ({ϵ}_{\eta },{ϵ}_e)=0$, it has $C_{{ϵ}_e}=0$; when ${ϵ}_e\ge {ϵ}_{\eta }$, $\chi ({ϵ}_i,{ϵ}_e)=\chi ({ϵ}_{\eta },{ϵ}_e)=1$, it has $C_{{ϵ}_e}=1$. Hence, the result follows. □

Thus, it is impossible for ${ϵ}_{\eta }$-KRR to provide the EPP ${ϵ}_e$, or provide the EPP ${ϵ}_e$ with a 100% satisfaction. Therefore, what KRR can achieve is two distinct jumps of EPP with or without guarantee, that is, it is not a good B-DP mechanism.

5.2. B-DP Mechanism Based on EXP${}_Q$

Since relative error is used as utility metrics about the check-in data distribution in this paper, and a privacy budget of DP usually determines absolute error which is the numerator of relative error, thus the privacy budget of every POI should vary with its probability in the distribution of check-ins, that is, the value of the privacy budget should be reduced when the corresponding probability in the data distribution becomes larger, and increased when the corresponding probability in the data distribution becomes smaller. In this way, the small amounts of check-ins can also satisfy the EDU, while the large amounts of check-ins can also satisfy the EPP in priority, so as to better realize B-DP. It defines the following perturbation mechanism EXP${}_Q$.

Definition 10

(Perturbation mechanism EXP${}_Q$). Given the data distribution $\mathbf{p}=[p_1,p_2,\cdots ,$$p_n{]}^T$, where $p_1\ge p_2\ge \cdots \ge p_n$. Call the randomized algorithm with Q as perturbation mechanism EXP${}_Q$ if Q satisfies $q_{ij}\propto e^{-\gamma u(j,i)}$, where $q_{ij}$ is the probability that the check-in state is perturbed from $S_j$ to $S_i$ for $i,j\in [1,n]$ and $\gamma \ge 0$ is the privacy setting parameter, where $u(j,i)$ satisfies Formulas (18)–(20), and ${\kappa }_n\in [0,n]$ is the parameter of privacy protection intensity change point.

$(i)$ When ${\kappa }_n=0$,

$$u(j,i)=\left\{\begin{array}{c}0,i=j\hfill \\ 1+p_{n-i+1},i\ne j\hfill \end{array}.\right.$$

(18)

$(ii)$ When ${\kappa }_n\in [1,n-1]$,

$$u(j,i)=\left\{\begin{array}{c}0,i=j\hfill \\ 1-p_i,i\ne j\mathit{and}i\le {\kappa }_n\hfill \\ 1+p_{n-i+{\kappa }_n+1},i\ne j\mathit{and}i>{\kappa }_n\hfill \end{array}.\right.$$

(19)

$(iii)$ When ${\kappa }_n=n$,

$$u(j,i)=\left\{\begin{array}{c}0,i=j\hfill \\ 1-p_i,i\ne j\hfill \end{array}.\right.$$

(20)

Theorem 4.

In perturbation mechanism EXP${}_Q$, its Q satisfies the following properties.

$(i)$ When ${\kappa }_n=0$ and the normalized factor perturbing from $PO{I}_j$ to a POI is ${\Omega }_j=1+{\sum }_{k=1,k\ne j}^n{e}^{-\gamma (1+p_{n-k+1})}$ for $i,j\in [1,n]$,

$$q_{ij}=\left\{\begin{array}{c}\frac{1}{{\Omega }_j},i=j\hfill \\ \frac{e^{-\gamma (1+p_{n-i+1})}}{{\Omega }_j},i\ne j\hfill \end{array}.\right.$$

(21)

$(ii)$ When ${\kappa }_n\in [1,n-1]$ and the normalized factor perturbing from $PO{I}_j$ to a POI is ${\Omega }_j=1+{\sum }_{k=1,k\ne j}^{{\kappa }_n}{e}^{-\gamma (1-p_k)}+{\sum }_{k={\kappa }_n+1,k\ne j}^n{e}^{-\gamma (1+p_{n-k+{\kappa }_n+1})}$ for $i,j\in [1,n]$,

$$q_{ij}=\left\{\begin{array}{c}\frac{1}{{\Omega }_j},i=j\hfill \\ \frac{e^{-\gamma (1-p_i)}}{{\Omega }_j},i\ne j\mathrm{and}i\le {\kappa }_n\hfill \\ \frac{e^{-\gamma (1+p_{n-i+{\kappa }_n+1})}}{{\Omega }_j},i\ne j\mathrm{and}i>{\kappa }_n\hfill \end{array}.\right.$$

(22)

$(iii)$ When ${\kappa }_n=n$ and the normalized factor perturbing from $PO{I}_j$ to a POI is ${\Omega }_j=1+{\sum }_{k=1,k\ne j}^n{e}^{-\gamma (1-p_k)}$ for $i,j\in [1,n]$,

$$q_{ij}=\left\{\begin{array}{c}\frac{1}{{\Omega }_j},i=j\hfill \\ \frac{e^{-\gamma (1-p_i)}}{{\Omega }_j},i\ne j\hfill \end{array}.\right.$$

(23)

Proof. 

According to Definition 10, it can be seen that $q_{ij}\propto e^{-\gamma u(j,i)}$ is the probability that the check-in state of $PO{I}_j$ is perturbed to that of $PO{I}_i$ for $i,j\in [1,n]$. Moreover, since ${\sum }_{k=1}^n{q}_{kj}=1$, it is easy to obtain the result of Theorem 4. □

Definition 11.

(${ϵ}_{\eta }$-EXP${}_Q$) EXP${}_Q$ that just satisfies the EPU η is called ${ϵ}_{\eta }$-EXP${}_Q$ where ${ϵ}_{\eta }=ma{x}_{1\le i\le n}({ϵ}_i)$ and ${ϵ}_i$ is the actual privacy budget for each $PO{I}_i$, that is, $e^{-{ϵ}_i}\le \frac{q_{ij}}{q_{i{j}^{\prime }}}\le e^{{ϵ}_i}$ for $j,j^{\prime }\in [1,n]$.

Theorem 5.

Based on the definition of ${ϵ}_{\eta }$-EXP${}_Q$,

(1) if there exists ${ϵ}_{\eta }$-EXP${}_Q$, then it satisfies $ϵ={ϵ}_{\eta }$-LDP;

(2) when ${\kappa }_n$ is fixed and the point belief degree of ${ϵ}_{\eta }$-EXP${}_Q$ is $C_{{ϵ}_e}={\sum }_{i=1}^n{\tilde{p}}_i\chi ({ϵ}_i,{ϵ}_e)$, where ${ϵ}_i$ is the actual privacy budget for each $PO{I}_i$, ${ϵ}_{\eta }$-EXP${}_Q$ is the approximately optimal $({ϵ}_{\eta },C_{{ϵ}_e})$-Best-B-DP, where the indicator function $\chi ({ϵ}_i,{ϵ}_e)$ is

$$\chi ({ϵ}_i,{ϵ}_e)=\left\{\begin{array}{c}0,{ϵ}_i>{ϵ}_e\hfill \\ 1,{ϵ}_i\le {ϵ}_e\hfill \end{array}\right.;$$

(24)

(3) if there exists ${ϵ}_{\eta }$-EXP${}_Q$ and its point belief degree is $C_{{ϵ}_e}$, then it satisfies $({ϵ}_e,1-C_{{ϵ}_e})$-LDP.

Proof. 

See Appendix A. □

5.3. Implementation of B-DP Machanism

For the check-ins scenario, two B-DP mechanisms based on KRR and EXP${}_Q$ are proposed and realized in this paper. KRR is one of the classical mechanisms of LDP, but it cannot well realize B-DP. EXP${}_Q$ is newly proposed in this paper, which can not only provide the protection of approximately optimal $({ϵ}_{\eta },C_{{ϵ}_e})$-Best-B-DP, but also provides the protection of relaxed $({ϵ}_e,1-C_{{ϵ}_e})$-LDP to satisfy the EDU. The pseudo codes of the two B-DP mechanisms are given in Algorithms 1 and 2, respectively.

Algorithm 1 B-DP machanism based on KRR.

Input: Probability distribution $\mathbf{p}={[p_1,p_2,\cdots ,p_n]}^T$, sample size m and expected data utility (EDU) $\eta$

Output: Privacy budget ${ϵ}_{\eta }$ and perturbation probability matrix Q

1: Initialize ${ϵ}_{\eta }>0$, iteration step size $\Delta {ϵ}_{\eta }>0$ and the worst utility $MaxRE=1$;  2: while$MaxRE>\eta$do  3:  Q is constructed by KRR with ${ϵ}_{\eta }$;  4:  According to the relative error formula, the current worst relative error CurrentMaxRE $=ma{x}_{i\in [1,n]}(\frac{\sqrt{{\sum }_{j=1}^n{r}_{ij}^2({\sum }_{k=1}^n{q}_{jk}{p}_k)-m{p}_i}}{m{p}_i})$ is obtained, where $R={(r_{i^{\prime }{j}^{\prime }})}_{n\times n}=Q^{-1},$$Q={(q_{i^{\prime }{j}^{\prime }})}_{n\times n}$, referred to Formulas (11)–(12) and Theorems 1 and 2 for details.  5:  if $CurrentMaxRE<MaxRE$ then  6:   $MaxRE=CurrentMaxRE$;  7:  end if  8:  if $MaxRE>\eta$ then  9:   ${ϵ}_{\eta }={ϵ}_{\eta }+\Delta {ϵ}_{\eta }$; 10:  end if 11: end while 12: return${ϵ}_{\eta }$ and Q

Algorithm 2 B-DP mechanism based on EXP${}_Q$.

Input: Probability distribution $\mathbf{p}={[p_1,p_2,\cdots ,p_n]}^T$, sample size m, expected data utility (EDU) $\eta$, expected privacy budget ${ϵ}_e$ (or expected privacy protection region $Region({ϵ}_e)=\{{ϵ}_{e_1},{ϵ}_{e_2},\cdots ,{ϵ}_{e_K}\}$ with ${ϵ}_{e_1}<{ϵ}_{e_2}<\cdots <{ϵ}_{e_K}$)

Output: Privacy setting parameter ${\gamma }_{\eta }$, the parameter of privacy protection intensity change point ${\kappa }_n$, perturbation probability matrix Q and actual privacy budget ${ϵ}_i$ of $PO{I}_i$ for $i\in [1,n]$

1: Initialize privacy setting parameter ${\gamma }_0>0$ and the iteration step size $\Delta {\gamma }_{\eta }>0$;  2: Initialize ${\kappa }_n=n$ and $tag=0$, where $tag$ is used to identify whether it exists a comparatively reasonable result or not;  3: while${\kappa }_n\ge 0$do  4:  Initialize ${\gamma }_{\eta }={\gamma }_0$, ${ϵ}_i=0$ for $i\in [1,n]$, the worst utility $MaxRE=1$ and $C_{{ϵ}_e}=0$ (here, the initialization of regional average belief degree $C_{Region({ϵ}_e)}$ is also uniformly recorded as $C_{{ϵ}_e}=0$);  5:  while $MaxRE>\eta$ do  6:   Q is constructed by EXP${}_Q$ with $\mathbf{p},\gamma ={\gamma }_{\eta }$ and ${\kappa }_n$, where the row represents the perturbed check-in state and the column represents the original check-in state;  7:   According to Q, use $ln\frac{max(q_{ij})}{min(q_{i{j}^{\prime }})}$ to update the value ${ϵ}_i$, where $Q={(q_{ij})}_{n\times n}$ and $i,j,j^{\prime }\in [1,n]$;  8:   According to the relative error formula, the current worst relative error is obtained $CurrentMaxRE=ma{x}_{i\in [1,n]}$$(\frac{\sqrt{{\sum }_{j=1}^n{r}_{ij}^2({\sum }_{k=1}^n{q}_{jk}{p}_k)-m{p}_i}}{m{p}_i})$, where $R={(r_{i^{\prime }{j}^{\prime }})}_{n\times n}=Q^{-1}$, referred to Formulas (11)–(12) and Theorems 1 and 2 for details;  9:   if $CurrentMaxRE<MaxRE$ then 10:    $MaxRE=CurrentMaxRE$; 11:   end if 12:   if $MaxRE>\eta$ then 13:    ${\gamma }_{\eta }={\gamma }_{\eta }+\Delta {\gamma }_{\eta }$; 14:   end if 15:  end while 16:  if ${ϵ}_i$ is not all zero for $i\in [1,n]$ then 17:   Calculate the current point belief degree value according to $C_{{ϵ}_e}^{\ast }={\sum }_{i=1}^n{\tilde{p}}_i\chi ({ϵ}_i,{ϵ}_e)$$={\sum }_{i=1}^n{\sum }_{j=1}^n$$q_{ij}{p}_j\chi ({ϵ}_i,{ϵ}_e)$, and set $C_{{ϵ}_e}^{♯}=C_{{ϵ}_e}^{\ast }$ (or, calculate the current regional average belief degree value according to $C_{Region({ϵ}_e)}^{\ast }=\frac{1}{{ϵ}_{e_K}-{ϵ}_{e_1}}{\sum }_{k=1}^{K-1}$$({ϵ}_{e_{k+1}}-{ϵ}_{e_k})C_{{ϵ}_{e_k}}$, where $C_{{ϵ}_{e_k}}={\sum }_{i=1}^n{\tilde{p}}_i\chi ({ϵ}_i,{ϵ}_{e_k})={\sum }_{i=1}^n{\sum }_{j=1}^n{q}_{ij}{p}_j\chi ({ϵ}_i,{ϵ}_{e_k})$, and set $C_{{ϵ}_e}^{♯}=C_{Region({ϵ}_e)}^{\ast }$); 18:   if $C_{{ϵ}_e}<C_{{ϵ}_e}^{♯}$ then 19:    Update $tag=1$, $C_{{ϵ}_e}=C_{{ϵ}_e}^{♯}$, ${\gamma }_{opt}={\gamma }_{\eta }$, ${\kappa }_{opt}={\kappa }_n$, $Q_{opt}=Q$ and ${ϵ}_i^{opt}={ϵ}_i(i\in [1,n])$; 20:   end if 21:  end if 22:  Update ${\kappa }_n={\kappa }_n-1$; 23: end while 24: if$tag=0$then 25:  Update ${\kappa }_n=0$, and update the value ${ϵ}_i$ according to Step 7; 26: else 27:  Record ${\gamma }_{\eta }={\gamma }_{opt}$, ${\kappa }_n={\kappa }_{opt}$, $Q=Q_{opt}$ and ${ϵ}_i={ϵ}_i^{opt}(i\in [1,n])$; 28: end if 29: return${\gamma }_{\eta }$, ${\kappa }_n$, Q and ${ϵ}_i$ for $i\in [1,n]$

5.4. Case Analysis of Point Belief Degree and Regional Average Beleif Degree

The above description theoretically analyzes two metrics, including the point belief degree and the regional average belief degree, on the two B-DP mechanisms based on KRR and EXP${}_Q$. In order to show the two metrics more clearly, the following of this section will use three data distributions with typical non-uniformity for analysis. For simplicity, in the following of this section, the KRR-based B-DP mechanism is represented by KRR and the EXP${}_Q$-based B-DP mechanism is represented by EXP${}_Q$, including the diagram descriptions.

(1) Three data distributions with typical non-uniformity.

The data distribution in this section is set as Pareto distribution, where the discrete case of Pareto distribution satisfies $p_j\propto \frac{1}{x_j^{\theta +1}}$ for $j\in [1,n],$$\theta >0$ and $x_j>0$. Three data distributions of $n=20$, $\theta =$ 1.55, 1.17 and 0.52 are shown in Figure 2 and are, respectively, denoted as $P1$, $P2$ and $P3$, where $x_j=x_1+(j-1)\Delta x,x_1=2,\Delta x=0.2$. Figure 2 shows both ordered and disordered cases of Pareto distribution, where the disordered case illustrates that the identification of scenic spots is independent of the order of probability. It also shows the corresponding Gini coefficient of $P1$, $P2$ and $P3$, which is calculated according to the method of Gini mean difference [52]. Gini coefficient is used to indicate the degree of unevenness of data distribution. There exists a quantitative relationship between Pareto distribution parameter and Gini coefficient in Table 3. As shown in Figure 2, the data distribution of $\theta =1.55$ is pretty uneven, and the data distribution of $\theta =1.17$ is relatively reasonably uneven, while the data distribution of $\theta =0.52$ is relatively even.

Figure 2. Pareto distribution.

Table 3. Gini coefficient VS. Pareto parameter of $\theta$.

Pareto Distribution	$\mathit{\theta }$	Gini Coefficient
P1	1.55	0.4471
P2	1.17	0.3884
P3	0.52	0.2784

(2) Point belief degree

In the point belief degree $C_{{ϵ}_e}$ in KRR, let ${ϵ}_e={ϵ}_{\eta }$ (${ϵ}_{\eta }$ is used as the expected privacy budget or used as a basis for division of the expected privacy protection region, just for better comparison between KRR and EXP${}_Q$) determined by ${ϵ}_{\eta }$-KRR (see definition of ${ϵ}_{\eta }$-KRR and Algorithm 1 for details), which equals the ${ϵ}_e$-coordinate of the jump point shown by the dotted line in Figure 3. It is also combined with the same ${ϵ}_e$ and $\eta$ to determine the perturbation probability and related parameters with EXP${}_Q$ (see Algorithm 2 for details). For example, when the EDU $\eta =0.1$, the point belief degree $C_{{ϵ}_e}$ of KRR and EXP${}_Q$ is shown in Figure 3.

Figure 3. Point belief degree $(C_{{ϵ}_e})$ with $\eta =0.1$.

From Figure 3, it can be seen that under the same EDU, if the expected privacy budget of any data provider is ${ϵ}_e\ge {ϵ}_{\eta }$, KRR can provide ${ϵ}_e$-DP with belief degree of 1. On the other hand, if the expected privacy budget of any data provider is ${ϵ}_e<{ϵ}_{\eta }$, its belief degree is 0. However, in EXP${}_Q$, if the expected privacy budget of any data provider is ${ϵ}_e\ge {ϵ}_{\eta }$, it indicates that it cannot satisfy the EPP when ${ϵ}_e$ is closer to ${ϵ}_{\eta }$, and when ${ϵ}_e$ is large enough, it can also provide ${ϵ}_e$-DP with belief degree of 1. Conversely, if the expected privacy budget of all data providers is ${ϵ}_e<{ϵ}_{\eta }$, it indicates that it can satisfy the EPP when ${ϵ}_e$ is closer to ${ϵ}_{\eta }$, and the degree of providing ${ϵ}_e$-DP is greater when ${ϵ}_e$ is closer to ${ϵ}_{\eta }$. Therefore, in the case of EDU first, EXP${}_Q$ can provide a privacy guarantee degree between 0 and 1 for the EPP, while KRR can only provide either 0 or 1. Moreover, EXP${}_Q$ can provide more privacy protection than KRR, especially when the EDU and the EPP are contradictory, and when the EPP of all data providers is not fully (partially) satisfied.

(3) Regional average belief degree

In the regional average belief degree $C_{Region({ϵ}_e)}$ in KRR, maximizing $C_{Region({ϵ}_e)}$ equals to maximize $C_{{ϵ}_e}$, and hence it is the same as Algorithm 1. According to the approximately optimal expected privacy budget ${ϵ}_{\eta }$ under satisfying the EDU $\eta$, the data provider’s expected privacy protection region $Region({ϵ}_e)$ can be roughly divided into three categories: $\{\forall {ϵ}_e\in Region({ϵ}_e)>{ϵ}_{\eta }\}$, $\{\forall {ϵ}_e\in Region({ϵ}_e)<{ϵ}_{\eta }\}$, and $\{\left\{{ϵ}_{\eta }\right\}\subset Region({ϵ}_e)\}$.

Similarly, EXP${}_Q$ can provide different levels of optimal privacy protection for the three categories of expected privacy protection region (see Algorithm 2 for details). Generally speaking, the regional average privacy protection degree of EXP${}_Q$ in region $\{\forall {ϵ}_e\in Region({ϵ}_e)>{ϵ}_{\eta }\}$ is less than or equal to that of KRR. However, the regional average privacy protection degree of EXP${}_Q$ in region $\{\forall {ϵ}_e\in Region({ϵ}_e)<{ϵ}_{\eta }\}$ is greater than or equal to that of KRR. For region $\{\left\{{ϵ}_{\eta }\right\}\subset Region({ϵ}_e)\}$, it may exist the situation where there is a contradiction between the EPP and the EDU. As shown in Figure 4, there is the regional average belief degree $C_{Region({ϵ}_e)}$ of both mechanisms with $Region({ϵ}_e)=[1,1.001,1.002,\cdots ,4]$ and the data distributions $P1$, $P2$ and $P3$, respectively, where ${ϵ}_{\eta }$ is determined by ${ϵ}_{\eta }$-KRR with $\eta =0.1$ (see the dotted line in Figure 4 where the value of ${ϵ}_e$ whose $C_{Region({ϵ}_e)}$ is the first non-zero value is equal to ${ϵ}_{\eta }$).

Figure 4. Regional average belief degree$(C_{Region({ϵ}_e)})$ with $\eta =0.1$.

As can be seen from Figure 4, under the same EDU and the same expected privacy protection region, EXP${}_Q$ is more capable of offering data providers with a certain degree of privacy protection than KRR.

6. B-DP Dynamic Collection and Publishing Algorithm Design

Algorithms 1 and 2 are implemented with KRR and EXP${}_Q$ under the known data distribution, and moreover, the point belief degree and the regional average belief degree under B-DP are analyzed. In real-world, there is often no prior data distribution at the beginning or accurate prior data distribution cannot be obtained. This means the implementation of two B-DP mechanisms of Algorithms 1 and 2 cannot be directly applied to the collection and publishing of continuous check-in data with relative error as utility metrics. Therefore, this paper designs an iterative update algorithm to adaptively update the data distribution in order to realize the two B-DP mechanisms, so as to adaptively realize B-DP dynamic collection and publishing of continuous check-in data. See pseudo codes of Algorithm 3 for more details. Therein, Algorithms 1 or 2 is a main part of Algorithm 3.

Algorithm 3 B-DP dynamic collection and publishing of check-in data algorithm—(KRR/EXP${}_Q$).

Initialization process: The data collector initializes the perturbation probability matrix Q, and the estimated data distribution ${\widehat{\mathbf{p}}}^{(0)}={[{\widehat{p}}_1^{(0)},{\widehat{p}}_2^{(0)},\cdots ,{\widehat{p}}_n^{(0)}]}^T$, and the perturbation probability matrix Q is broadcasted to the data provider. 1: For KRR, initialize $p_i=\frac{1}{n}$ for $i\in [1,n]$ and ${ϵ}_{\eta }=ln(\frac{1+(n-1)\sqrt{\frac{n-1}{m{\eta }^2+n-1}}}{1-\sqrt{\frac{n-1}{m{\eta }^2+n-1}}})$; For EXP${}_Q$, initialization $p_i=\frac{1}{n}$ for $i\in [1,n]$, ${\kappa }_n=0$ and $\gamma =\frac{n}{n+1}ln(\frac{1+(n-1)\sqrt{\frac{n-1}{m{\eta }^2+n-1}}}{1-\sqrt{\frac{n-1}{m{\eta }^2+n-1}}})$; 2: Q is constructed according to KRR/EXP${}_Q$; 3: Initialize ${\widehat{p}}_i^{(0)}=0$ for $i\in [1,n]$; 4: Q is broadcasted to the data provider. Perturbation process 1: The data provider uses Q to perturb the check-in data; 2: The perturbed check-in data are sent to the data collector. Statistics and update processes (1) Statistical process: including aggregation and reconstruction procedures 1: After collecting the check-in data of time slice T, the data collector carries out frequency statistics to get the perturbed data distribution $\tilde{\mathbf{p}}$. Assuming that the current time slice is the tth, it is recorded as ${\tilde{\mathbf{p}}}_t$; 2: From the inverse estimation formula ${\widehat{\mathbf{p}}}_t=Q^{-1}{\tilde{\mathbf{p}}}_t$, it obtains the estimated distribution ${\widehat{\mathbf{p}}}_t$; 3: Correcting the estimated data distribution ${\widehat{\mathbf{p}}}_t$, it gets the corrected estimate data distribution ${\widehat{\mathbf{p}}}^{(t)}$ of the tth time slice (to be released); if${\widehat{\mathbf{p}}}^{(t-1)}\ne \mathbf{0}$then  ${\widehat{\mathbf{p}}}^{(t)}=(1-w){\widehat{\mathbf{p}}}^{(t-1)}+w{\widehat{\mathbf{p}}}^{(t)}$, where $w\in (0,1)$ is the corrected estimate parameter of a positive real number; else  ${\widehat{\mathbf{p}}}^{(t)}={\widehat{\mathbf{p}}}_t$; end if (2) Update process 1: The data collector calculates maximum relative error $Re=max(|\frac{{\widehat{\mathbf{p}}}^{(t)}-{\widehat{\mathbf{p}}}^{(t-1)}}{{\widehat{\mathbf{p}}}^{(t-1)}}|)$ based on ${\widehat{\mathbf{p}}}^{(t)}$ and ${\widehat{\mathbf{p}}}^{(t-1)}$; 2: Initialize $R{e}_{thredthold}\in (0,1)$, which is a update threshold parameter of a positive real number; if$Re>R{e}_{thredthold}$then  2-1: Start a process of updating Q to get a new Q, as shown in Algorithm 1/Algorithm 2, where the input data distribution of Algorithm 1/Algorithm 2 is ${\widehat{\mathbf{p}}}^{(t)}$;  2-2: The data collector broadcasts the new Q to the data provider. end if

Since the original data distribution is assumed to be uniform during initialization, it is possible to calculate the privacy setting parameter ${ϵ}_{\eta }$ or $\gamma$ with a closed-form expression that satisfies the EDU $\eta$, as shown in the example with EXP${}_Q$ below. According to Corollary A1 of Appendix A, in the case of uniform data distribution, EXP${}_Q$ degenerates into KRR. Let ${\kappa }_n=0$, the probability $q_{ij}$ of Q be calculated as follows, where ${\gamma }_{\eta }$ is $\gamma$ that makes $max(err(p_i,{\widehat{p}}_i))=\eta$ true.

$$q_{ij}=\left\{\begin{array}{c}\frac{e^{{\gamma }_{\eta }(1+\frac{1}{n})}}{e^{{\gamma }_{\eta }(1+\frac{1}{n})}+n-1},i=j\hfill \\ \frac{1}{e^{{\gamma }_{\eta }(1+\frac{1}{n})}+n-1},i\ne j\hfill \end{array}.\right.$$

(25)

Let $p=\frac{e^{{\gamma }_{\eta }(1+\frac{1}{n})}}{e^{{\gamma }_{\eta }(1+\frac{1}{n})}+n-1},q=\frac{1}{e^{{\gamma }_{\eta }(1+\frac{1}{n})}+n-1}$, and the inverse matrix R of Q can be expressed as

$$r_{ij}=\left\{\begin{array}{c}\frac{1-q}{p-q},i=j\hfill \\ -\frac{q}{p-q},i\ne j\hfill \end{array}.\right.$$

(26)

Therefore, p, q and ${\gamma }_{\eta }$ that satisfy the EDU $\eta$ can be calculated. Since $p\ge q$, it has $q=-\frac{1}{n}\sqrt{\frac{n-1}{m{\eta }^2+n-1}}+\frac{1}{n}$ and $p=\frac{n-1}{n}\sqrt{\frac{n-1}{m{\eta }^2+n-1}}+\frac{1}{n}$. Hence, ${\gamma }_{\eta }=\frac{n}{n+1}ln(\frac{p}{q})$, and $\gamma ={\gamma }_{\eta }=\frac{n}{n+1}ln(\frac{1+(n-1)\sqrt{\frac{n-1}{m{\eta }^2+n-1}}}{1-\sqrt{\frac{n-1}{m{\eta }^2+n-1}}})$.

7. Experimental Evaluation of B-DP Dynamic Collection and Publishing Algorithm

In this paper, the check-in data uses relative error as its utility metrics and the implementation of the two B-DP mechanisms based on KRR and EXP${}_Q$ needs to rely on the data distribution. Therein, the number of domain values of both KRR and EXP${}_Q$ is more than 2, and moverover, both the randomized algorithms based on them only take one value as input and one value as output. Thereby, KRR and EXP${}_Q$ are fit for the check-in perturbation model we consider in this paper. In this section, we evaluate the performance of the dynamic algorithm based on the two B-DP mechanisms in terms of validity and robustness as well as privacy and utility. For simplicity, in the following of this section, we use KRR and EXP${}_Q$ to represent B-DP mechanism based on KRR and B-DP mechanism based on EXP${}_Q$ in the dynamic algorithm, respectively, including the diagram descriptions.

7.1. Experimental Settings

(1) Datasets

Two datasets with real-world data from location-based social networking platforms are used to verify the algorithms.

Brightkite [6]: It contains 4,491,143 check-ins over the period of April 2008–October 2010. In this paper, we used the check-ins of June 2008–September 2008, September 2009–December 2009 and January 2010—April 2010 from Brightkite to construct three types of check-in distributions with different uniformity degree according to the unified longitude and latitude division method, which are, respectively, abbreviated as B1, B2 and B3, and the number of regions is 12.

Gowalla [6]: It contains 6,442,890 check-ins over the period of Feburary 2009–October 2010. We used the check-ins of January 2010–April 2010 of Gowalla to construct three types of check-in distributions with different uniformity degree according to different partitioning methods of longitude and latitude, which are, respectively, abbreviated as G1, G2 and G3, and the number of regions is 25, respectively.

The average data distribution and the corresponding Gini coefficient of the data are shown in Figure 5 and Table 4, respectively. Therein, Gini coefficient is used to indicate the degree of unevenness of data distribution, which is calculated according to the method of Gini mean difference [52].

Figure 5. The average data distribution of Brightkite VS. Gowalla.

Table 4. Gini coefficient of data in Brightkite and Gowalla.

Datasets	Data Distributions	Gini Coefficient
Gowalla	G1	0.2357
	G2	0.3488
	G3	0.4465
Brightkite	B1	0.2849
	B2	0.3488
	B3	0.4628

Figure 5 and Table 4 both show that the daily check-in data in two datasets fluctuates greatly, meaning a high diveristy. We verify the effectiveness of our algorithms on these real-world datasets in our experiment.

(2) Utility/Privacy Metrics

Utility Metrics: The utility uses the maximum relative error as its metrics (see Section 3.3 for details). In this paper, it uses the mean and deviation of the maximum relative error to evaluate the same EDU between KRR and EXP${}_Q$ in the dynamic algorithm.

Privacy Metrics: The privacy uses two new metrics including the point belief degree and the regional average belief degree (see Defintions 4 and 5 for details). In this paper, it needs to compare the privacy gurantee degree about the expected privacy protection (EPP) under the same expected data utility (EDU) using these two privacy metrics between KRR and EXP${}_Q$ in the dynamic algorithm.

(3) Parameter Settings

We evaluate our solutions through experiments using two real-world datasets. The experiments are performed on an Intel Core CPU 2.50-GHz Windows 10 machine equipped with 8 GB of main memory by matlab. In the experiments, the total check-in amount of statistical validity is $m=100,000$. Three kinds of EDU are $\eta =0.1$, $0.08$ and $0.05$. The expected privacy protection region is $Region({ϵ}_e)=[1,1.001,1.002,\cdots ,4]$ or $Region({ϵ}_e)=[1,1.001,1.002,\cdots ,10]$. The modified estimate parameter w is set as Table 5. The update threshold parameter is $R{e}_{thredthold}=0.02$, and the remaining relevant parameters ${ϵ}_0$ and $\Delta {ϵ}_{\eta }$ are set to 0.5 and 0.005, respectively.

Table 5. All kinds of modified estimate parameter w used in the dynamic algorithm.

Mechanisms	Brightkite			Gowalla
	B1	B2	B3	G1	G2	G3
EXP${}_Q$	0.045	0.06	0.15	0.18	0.25	0.4
KRR	0.035	0.055	0.15	0.18	0.25	0.4

7.2. Validity and Robustness Evaluation

The performance of validity and robustness of the corresponding dynamic algorithm with KRR and EXP${}_Q$ is examined through the dynamic statistics process with two real-world datasets.

Figure 6 and Figure 7 show the mean values and deviations of the maximum relative error $err(\mathbf{p},\widehat{\mathbf{p}})$ under the three kinds of EDU ($\eta =0.1$, $0.08$ and $0.05$), which are shown by the statistics of the corresponding data subsets under B1, B2, B3, G1, G2 and G3 according to the frequency of once a day. Moreover, the frequency of each day is different and each result is repeated 10 times. In both Figure 6 and Figure 7, the horizontal axis of each graph represents the number of time slices in continuously, and the vertical axis represents the maximum relative error $err(\mathbf{p},\widehat{\mathbf{p}})=max(err(p_i,{\widehat{p}}_i))$ between the original data distribution and the estimated data distribution for any $i\in [1,n]$. As can be seen from the left small graphs of Figure 6 and Figure 7, the corresponding dynamic algorithm with KRR and EXP${}_Q$ can converge quickly and maintain the corresponding unified convergence stable state under different data distributions of B1, B2, B3, G1, G2 and G3. This verifies that the dynamic algorithm has a good validity and robustness.

Figure 6. The mean and deviation of $err(\mathbf{p},\widehat{\mathbf{p}})$ on B1, B2 and B3 subsets of Brightkite. Three $\eta$ settings of EDU, including $0.1$, $0.08$ and $0.05$, are compared. (a) B1; (b) B2; (c) B3.

Figure 7. The mean and deviation of $err(\mathbf{p},\widehat{\mathbf{p}})$ on G1, G2 and G3 subsets of Gowalla. Three $\eta$ settings of EDU, including $0.1$, $0.08$ and $0.05$, are compared. (a) G1; (b) G2; (c) G3.

7.3. Utility and Privacy Evaluation

The performance of utility and privacy of the corresponding dynamic algorithm with two B-DP mechanisms based on KRR and EXP${}_Q$ is also examined through the dynamic statistics process with two real-world datasets. As can also be seen from the right small graphs of Figure 6 and Figure 7, a part of the left small graphs of Figure 6 and Figure 7, it shows clearly that the dynamic algorithm can satisfy the utility even during the dynamic process.

In addition, Figure 8 and Figure 9 show the point belief degree and the regional average belief degree of each subset of two datasets under the three kinds of EDU ($\eta =0.1$, $0.08$ and $0.05$). In Figure 8 and Figure 9, the horizontal axis of each graph represents the EPP with different expected privacy buget ${ϵ}_e$ and the vertical axis represents the gurantee degree of the EPP satisfied. It shows that the gurantee degree of the EPP satisfied varies with the data distribution and EDU. For example, from the point belief degree of all small graphs in the left of Figure 8 and Figure 9, the gurantee degree of the EPP satisfied becomes better until its value up to 1 when the expected privacy buget becomes bigger, and the more evener distribution can support the EPP with the smaller ${ϵ}_e$ to provide a better privacy protection in the same EDU (as Figure 10 shown). The smaller value of EDU, i.e., the lower utility, can generally support the EPP with the smaller ${ϵ}_e$ to provide a better privacy protection in the same data distribution (as Figure 11 shown).

Figure 8. The belief degree on B1, B2 and B3 subsets of Brightkite, where the belief degree includes the point belief degree and the regional average belief degree. (a) B1; (b) B2; (c) B3.

Figure 9. The belief degree on G1, G2 and G3 subsets of Gowalla, where the belief degree includes the point belief degree and the regional average belief degree. (a) G1; (b) G2; (c) G3.

Figure 10. The minimum ${ϵ}_e$ with $C_{{ϵ}_e}>0$ based on the same EDU ($\eta$) and different data distributions, where $C_{{ϵ}_e}$ is the point belief degree on the EPP of ${ϵ}_e$, and moreover, $\eta =0.1$, $0.08$ and $0.05$ represent three kinds of EDU.

Figure 11. The minimum ${ϵ}_e$ with $C_{{ϵ}_e}>0$ based on the same data distribution and different EDU ($\eta$), where $C_{{ϵ}_e}$ is the point belief degree on the EPP of ${ϵ}_e$, and moreover, $\eta =0.1$, $0.08$ and $0.05$ represent three kinds of EDU.

In Figure 10, for EXP${}_Q$ on G1, G2 and G3 with the given EDU (such as $\eta =0.1$), it shows clearly that the minimum ${ϵ}_e$ with $C_{{ϵ}_e}>0$ is the smallest on G1 and the largest on G3. According to Table 4, G3 is pretty uneven, while G1 is relatively even. It is the same for EXP${}_Q$ on B1, B2 and B3, KRR on B1, B2 and B3 as well as on G1, G2 and G3. In Figure 11, for EXP${}_Q$ on G1, G2 and G3 with the given data distribution (such as G1), it also shows clearly that the minimum ${ϵ}_e$ with $C_{{ϵ}_e}>0$ is the smallest on the EDU with $\eta =0.1$ and the largest on the EDU with $\eta =0.05$. Similar trends can be observed for EXP${}_Q$ on B1, B2 and B3, KRR on B1, B2 and B3 as well as on G1, G2 and G3.

For the regional average belief degree, the similar results can be concluded from all small graphs in the right of Figure 8 and Figure 9. Moreover, in Figure 12, it shows the maximum difference of $C_{(Region({ϵ}_e))}$ in EXP${}_Q$ minus $C_{(Region({ϵ}_e))}$ in KRR with different $\eta$ on each subset. It shows that the more unevener the data distribution is, the more bigger the maximum difference is. It means that EXP${}_Q$ is more adapt to the unevener data distribution than KRR.

Figure 12. The maximum difference of $C_{(Region({ϵ}_e))}$ in EXP${}_Q$ minus $C_{(Region({ϵ}_e))}$ in KRR with different $\eta$, where $C_{(Region({ϵ}_e))}$ is the regional average belief degree on the region of $Region({ϵ}_e)=[1,1.001,1.002,\cdots ,4]$ or $Region({ϵ}_e)=[1,1.001,1.002,\cdots ,10]$, and $\eta =0.1$, $0.08$ and $0.05$ represents three kinds of EDU.

Furthermore, in order to be more objective evaluation of the privacy performance of KRR and EXP${}_Q$, it extends to use the privacy metrics of DP to compare the ${ϵ}_{\eta }$ on each subset shown in Table 6, where ${ϵ}_{\eta }$ refers to the privacy budget of a DP mechanism that just satisfies the EDU $\eta$. As can be seen from Table 6, except for $\eta =0.08$ and $\eta =0.1$ on B3, all the ${ϵ}_{\eta }$ of EXP${}_Q$ is a little greater than those of KRR, which means that EXP${}_Q$ provides a little worse DP than KRR. However, EXP${}_Q$ could provide better B-DP from a new perspective of preference for privacy and utility than KRR to provide a good trade-off between them.

Table 6. ${ϵ}_{\eta }$ on each subset, where $\eta =0.1$, $0.08$ and $0.05$ represent three kinds of EDU.

$\mathit{\eta }$	Mechanisms	Data Distributions
		B1	B2	B3	G1	G2	G3
0.1	EXP${}_Q$	1.738	1.874	6.627	2.928	4.429	5.774
	KRR	1.55	1.83	6.71	2.65	4.09	5.695
0.08	EXP${}_Q$	2.017	2.128	6.967	3.204	4.688	6.072
	KRR	1.91	2.01	7.065	2.945	4.505	5.955
0.05	EXP${}_Q$	2.568	2.751	7.995	3.858	5.451	7.041
	KRR	2.435	2.57	7.95	3.66	5.375	6.995

8. Discussions and Conclusions

This paper proposes a concept of best-effort differential privacy (B-DP) with the expected data utility (EDU) satisfied first and then with the expected privacy protection (EPP) satisfied as much as possible, and designs two new metrics including point belief degree and regional average belief degree to measure the guarantee degree of satisfying the EPP. Moreover, we also provide implementation algorithms, including the corresponding dynamic algorithm of two B-DP mechanisms based on KRR and a newly constructed mechanism EXP${}_Q$. Extensive experiments on two real-world check-in datasets verify the effectiveness of the concept of B-DP. It also verifies that the dynamic algorithm has a good validity and robustness, and can satisfy the utility even during the dynamic process. Besides, EXP${}_Q$ is more adapt to the unevener data distribution and satisfies a better B-DP than KRR to provide a good trade-off between privacy and utility.

Specifically, the point belief degree measures the guarantee degree of privacy protection for any one expected privacy budget, and the regional average belief degree measures the average guarantee degree of the EPP in a region including multiple expected privacy budgets. Compared with the ($ϵ,\delta$)-DP, the latter can measure only one EPP with the expected privacy budget equal to $ϵ$ and cannot directly measure the average guarantee degree of the EPP, that is, the ($ϵ,\delta$)-DP can only measure the guarantee degree of the EPP when ${ϵ}_e=ϵ$, i.e., $1-\delta$. In addition, many real-world applications can only provide an approximate value of ${ϵ}_e$ as their EPP, and hence a neighborhood interval with ${ϵ}_e$ can be regarded as their EPP. Therefore, the regional average belief degree introduced in this paper is very necessary.

Moreover, two B-DP mechanisms based on KRR and newly constructed EXP${}_Q$ in this paper are applied to the dynamic collection and publishing of check-in data with relative error as its utility metrics. Therein, KRR itself does not depend on the data distribution, but the dynamic collection and publishing algorithm with B-DP mechanism based on KRR needs to, where the privacy setting parameter has to be adjusted with the influence of data distribution to realize the utility guaranteed firstly in real time. In addition, EXP${}_Q$ itself is dependent on the data distribution to realize some of its outputs having strong privacy protection and some having weak privacy protection, which is different from KRR to provide consistent privacy protection intensity. Thus, the dynamic collection and publishing algorithm based on these two B-DP mechanisms needs to depend on the data distribution, and then it has to face the challenges of algorithm validity and robustness with unknown data distribution. Fortunately, the experimental results have already verified that the algorithm can solve both challenges and is promising for the typical application of check-in data.

Besides, if the scenic spots use EXP${}_Q$ for privacy protection, the data provider may be more inclined to visit these scenic spots with a large number of visitors, because the regions where these scenic spots are located may have a stronger privacy protection. Compared with the algorithms based on the existing DP mechanisms with consistent privacy protection intensity to realize B-DP, such as KRR in this paper, they maybe do not achieve the EPP at all, but the algorithm based on EXP${}_Q$ newly proposed in this paper can achieve the EPP partly at least, that is, EXP${}_Q$ can satisfy a better B-DP to provide a good trade-off between privacy and utility.

In a word, although the B-DP dynamic collection and publishing algorithm based on KRR or EXP${}_Q$ is not necessarily perfect, it fully proves the feasibility of the concept of B-DP in this paper. It is not only a great step forward for the basic theory of DP, but also provides two feasible solutions for the implementation of DP in practical applications. The two solutions take check-in data as an example, but are not limited to it. They can also be used to other category data for privacy protection where the perturbation model is one input and one output. In the future work, we will make a further discussion on other mechanisms with binary inputs in LDP, where the perturbation model can support one input is perpurbed to multiple outputs, such as RAPPOR, and design them to achieve better B-DP. Moreover, it is an interesting problem about correlated B-DP.