Improving Utility of Private Join Size Estimation via Shuffling

Abstract

Join size estimation plays a crucial role in query optimization, correlation computing, and dataset discovery. A recent study, LDPJoinSketch, has explored the application of local differential privacy (LDP) to protect the privacy of two data sources when estimating their join size. However, the utility of LDPJoinSketch remains unsatisfactory due to the significant noise introduced by perturbation under LDP. In contrast, the shuffle model of differential privacy (SDP) can offer higher utility than LDP, as it introduces randomness based on both shuffling and perturbation. Nevertheless, existing research on SDP primarily focuses on basic statistical tasks, such as frequency estimation and binary summation. There is a paucity of studies addressing queries that involve join aggregation of two private data sources. In this paper, we investigate the problem of private join size estimation in the context of the shuffle model. First, drawing inspiration from the success of sketches in summarizing data under LDP, we propose a sketch-based join size estimation algorithm, SDPJoinSketch, under SDP, which demonstrates greater utility than LDPJoinSketch. We present theoretical proofs of the privacy amplification and utility of our method. Second, we consider separating high- and low-frequency items to reduce the hash-collision error of the sketch and propose an enhanced method called SDPJoinSketch+. Unlike LDPJoinSketch, we utilize secure encryption techniques to preserve frequency properties rather than perturbing them, further enhancing utility. Extensive experiments on both real-world and synthetic datasets validate the superior utility of our methods.

1. Introduction

With the development of big data technology and the popularization of big data analysis, it has become ubiquitous to improve the quality of service by collecting and analyzing large amounts of user data. Join size estimation, also known as inner product estimation, represents a fundamental statistical problem with applications spanning various domains, including query optimization [1,2], similarity computation [3], and dataset discovery [4]. However, collecting transaction records directly from users may reveal sensitive information, thus raising privacy concerns.

In the context of private data collection and publishing, differential privacy (DP) [5] has been proposed to provide strong, mathematically based privacy-preserving guarantees and has so far been applied to numerous areas. Specifically, without the requirement of a trusted curator, the local differential privacy (LDP) [6] mitigates privacy concerns of users by locally sanitizing private data independently on their own devices before sending them to the server. Due to its practical setup and strong privacy protection, LDP has been deployed in some popular systems, such as Apple [7], Google [8,9], and Microsoft [10]. Most existing LDP methods focus on simple statistical data analysis tasks such as frequency estimation [11,12] and heavy hitters identification [13,14], but few address complex query tasks, such as aggregation with joins. A recent work [15] has investigated the problem of join size estimation under LDP. Nevertheless, LDP imposes a large amount of noise as each user executes the randomization independently.

To overcome challenges regarding the accuracy of LDP and the privacy risks of centralized DP, the shuffle model of differential privacy (SDP) [16,17] has emerged. The primary change in SDP is the introduction of a semi-trusted intermediary shuffler to permute user reports. Specifically, a typical SDP setting [17] involves three main participants: a large number of users on the client side, an intermediate semi-trusted shuffler, and an untrusted third-party aggregator on the server side. The workflow can be broken down into three steps: First, each user locally encodes and perturbs their sensitive data using LDP mechanisms, then sends the randomized report to the shuffler. Next, the shuffler permutes the sanitized reports in a uniformly random order and the shuffled set of reports is then submitted to the aggregator. Finally, the server-side aggregates the perturbed reports for further specific analysis. One significant advantage of SDP over LDP is its utility. High utility means that the data under privacy protection can still effectively support original analysis tasks, such as frequency or join size estimation, with minimal accuracy loss. The permutation operation performed by the shuffler breaks the linkage between users and their reports, which brings additional privacy benefits. Consequently, less noise is required on the client side to achieve the same level of privacy (also known as privacy amplification). Although SDP has been used for data privacy protection in decentralized settings, most existing works focus mainly on basic statistics, such as binary summation [18] and frequency estimation [19]. There are few works that provide solutions for more intricate analysis tasks, such as join aggregations, under SDP.

Recent work on join size estimation under LDP introduces LDPJoinSketch [15], which utilizes sketch techniques to address the challenge of a large data domain. It constructs sketches on the server side based on perturbed values collected from individual users and then estimates the results using these sketches. Additionally, a frequency-aware protocol is proposed to reduce hash-collision errors for utility improvement. While join size estimation under LDP has been well studied, handling join aggregations with sketches under SDP remains non-trivial and presents several challenges:

1. Analyze the privacy amplification of sketch-based join size estimation under SDP: Addressing the noise error under LDP, which stems from the large domain of the join attribute, often involves utilizing probabilistic data structures such as sketches or bloom filters to reduce the domain size. However, no existing research has explored sketch-based methods under SDP, where a privacy amplification effect is introduced by the shuffler. Traditionally, this effect is mainly influenced by the number of users and the data domain. But additional parameters related to sketch size and the sampling operation increase the complexity of the analysis. We investigate this problem and redesign the protocols for private join aggregations under SDP.

2. Reduce hash-collision errors under SDP: Hash-collision errors between high- and low-frequency items in sketch construction significantly affect utility. Specifically, to reduce hash-collision errors in sketches, an intuitive idea is to separate the storage of high- and low-frequency items. However, directly sending the true frequency properties to the server would result in privacy leakage to the semi-trusted shuffler. On the other hand, perturbing this information under LDP requires partitioning the privacy budget into two parts for users’ values and their frequency properties, respectively, potentially worsening estimation utility. LDPJoinSketch avoids splitting privacy budget but instead adopts different encoding schemes for user values with different frequency properties to achieve LDP; it requires additional user partitioning to separate items with different frequency properties and still introduces some irremovable noise on the server side, resulting in utility reduction. Therefore, we need a method that allows the server to obtain the true frequency property of each received item while ensuring that the information is not leaked to the shuffler during transmission and guarantees privacy.

To enhance the utility of sketch-based join size estimation on private values, we first propose SDPJoinSketch, which allows adding less noise on the client side while maintaining the same level of privacy protection as in LDP due to the additional randomness introduced by shuffling. We provide a formal proof of privacy amplification using the privacy blanket theory [20]. Furthermore, to address the issue of hash-collision errors, we propose an improved mechanism, SDPJoinSketch+, which enables each client to be aware of its own frequency properties and adopt secure encryption techniques to encode and transmit their true frequency properties rather than adding noise through LDP. This allows the server to obtain accurate information without leaking privacy to the shuffler.

Figure 1 depicts the workflow of our proposed method, which is structured into two phases. Note that we divide the users into two groups by sampling for the tasks of the two phases, rather than directly dividing the privacy budget as in most previous works [11,15,21], since a smaller privacy budget introduces a significant amount of noise being added, thus leading to unacceptable utility. Initially, the first phase focuses on identifying frequent items among the sampled users. Each user on the client side encodes (abbreviated as $E\left(\right)$) and perturbs (abbreviated as $P\left(\right)$) their private attribute value d and submits the sanitized result $P\left(E\right(d\left)\right)$ to the shuffler. The shuffler then randomly permutes the reports from n users to achieve anonymization and then sends the shuffled results to the server for aggregation. The server first constructs a sketch with the permuted results and then estimates the frequencies to obtain the frequent item set ($FI$) based on the sketch. The server broadcasts $FI$ to the remaining users. In the second phase, the outcome from the first phase is used to differentiate between high- and low-frequency items, improving the precision of the join size estimation. Based on $FI$, an indicator t is assigned to each user, denoting whether an item is high-frequency or low-frequency. Specifically, if the private value $d_i$ of user i is in $FI$, then $t_i=1$; otherwise, $t_i=0$. Similar to the first phase, each user encodes and perturbs d. Additionally, each user encrypts (abbreviated as $Enc\left(\right)$) the indicator t. Sanitized results $P\left(E\right(d\left)\right)$ and $Enc\left(t\right)$ for each user are transmitted to the shuffler for permutation. On the server side, it separates high-frequency and low-frequency items by the decryption (abbreviated as $Dec\left(\right)$) results $Dec\left(Enc\right(t\left)\right)$, and constructs sketches $MH$ and $ML$ to summarize the corresponding items.

Figure 1. Overview of SDPJoinSketch+ (semi-trusted shuffler between clients and server).

We contend that the shuffler enhances both phases of private join size estimation. In the first phase, the presence of the shuffler improves the utility of frequency estimation while maintaining the same level of privacy guarantees. In the second phase, the shuffler disrupts the linkage between each user and their corresponding report (including both the value and its frequency property) through permutation. As a result, there is no need to perturb both the value and its frequency property separately; we can simply send the encrypted value of the true frequency property. This approach avoids the misclassification of high- and low-frequency items caused by perturbing the frequency property.

Main contributions are summarized as follows:

• We design a sketch-based protocol, SDPJoinSketch, for join size estimation under SDP. We provide detailed proof of both the privacy amplification and utility of SDPJoinSketch.
• We present an improved algorithm called SDPJoinSketch+, which reduces hash-collision errors by leveraging the anonymity of SDP with secure encryption techniques.
• We conduct experiments demonstrating the utility improvements of our methods compared to state-of-the-art approaches.

Organization. The remainder of this paper is organized as follows. Section 2 introduces some related researches to this paper. Section 3 provides preliminary knowledge on join sketches and different models of DP. Section 4 shows our proposed sketch-based method for private join size estimation under SDP and its formal analysis of privacy amplification. We introduce an improved mechanism SDPJoinSketch+ which utilize the secure encryption technique to eliminate hash-collision errors in Section 5. Section 6 presents experimental results, and finally, in Section 7, we conclude the whole paper.

2. Related Work

The join size of two data streams, which represents the inner product of their frequency vectors, is a crucial metric in data stream analysis. However, due to the high time and space requirements, it is impractical and unnecessary to track the exact join size. Moreover, in scenarios involving privacy and large data domains, frequency estimation under local differential privacy tends to introduce substantial noise and is highly time-consuming. This section reviews solutions addressing the aforementioned issues.

2.1. Sketches for Join Size Estimation

As a class of streaming summaries, sketch techniques have undergone extensive development within the past few years. Sketch-based methods are among the most commonly used techniques for approximating join size [22]. The basic AGMS sketch was first proposed in  [23,24], which can be viewed as a random projection of data in the frequency domain on ±1 pseudo-random vectors. The Fast-AGMS sketch [25] preserves a matrix of basic AGMS counters to improve accuracy and efficiency simultaneously and achieves the best performance. To reduce hash-collision errors, the Red sketch [26] and the Skimmed sketch [27] propose to estimate the join size by separately estimating the inner product of high-frequency items and low-frequency items. JoinSketch [28] further improves the estimation accuracy, particularly in cases where the data is highly skewed. However, these studies do not address scenarios where the join attribute values involve user privacy. In the field of multi-party secure computation (MPC), Google’s Private-Join-and-Compute (PJC) [29] protocol combines private set intersection with homomorphic encryption to enable cross-organizational precise computation of join size without compromising privacy. However, it incurs high communication and computational overhead.

In recent years, several differentially private sketch techniques [7,30,31] have been introduced for conventional statistical tasks. More recently, the LDPJoinSketch [15] has been proposed to achieve high estimation accuracy while preserving individual privacy in an LDP setting for private join size estimation. But due to the nature of LDP, the level of noise required per user is extremely high, which limits its efficacy. Building on this study, we extend the task to the shuffler setting to tackle the utility issue.

2.2. Techniques and Applications Under SDP

To mitigate the high noise required in LDP, recent work has proposed SDP, where a semi-trusted shuffler is introduced to obscure individual private views within the crowd. The shuffling idea was originally presented in Prochlo [32]. The Encode, Shuffle, Analyze (ESA) model in Prochlo provides a general framework for incorporating a shuffling step in private protocols. Later work [17] demonstrated that the introduction of a secure shuffler amplifies the privacy guarantees. SDP was formally defined in [16] and it derives a similar conclusion for binary randomized response messages. Balle et al. [20] further improved the amplification results and introduced the idea of a privacy blanket whose main idea is to decompose the probability distribution of an LDP report into two distributions and mask the truthfully reported outputs. This technique provides the strongest amplification result and its proof method can be applied to other LDP protocols. We incorporate this idea into our protocol design and privacy amplification analysis.

Some recent works utilize SDP to achieve a better privacy–utility trade-off for specific analytical tasks. For frequency estimation tasks, Hist_KR [20] applies a basic random response mechanism within the ESA framework. It is the simplest and most direct method in frequency estimation. To further enhance utility, SLH [33] achieves an error that is independent of domain size, making it suitable for cases with large domains. Ghazi et al. [34] introduced a protocol in the SDP model for estimating the sum or mean of real- or integer-valued data, achieving error rates comparable to CDP. Wang et al. [35] proposed the SDPKV method, which integrates key-value perturbation with privacy budget amplification analysis under the SDP model, significantly improving the accuracy of key-value data collection. In addition, mechanisms [36,37] based on multi-message shuffle model achieve a smaller error compared to the previous methods. Nevertheless, existing research has not yet explored more complex statistical queries, such as join aggregations.

3. Preliminaries

We give the formal problem definition of sketch-based join size estimation. And we retrospectively examine the definition of differential privacy in the centralized, local, and shuffle models. We list commonly used notations of this paper in Table 1.

Table 1. The notations of symbols.

Notations	Description
n	The number of users
$d_i$	Private join attribute value of the ith user
D	The domain of attribute values and $\left|D\right|=N$
${ϵ}_l,{ϵ}_c$	Privacy budget of local and central model
$\delta$	The failure probability of differential privacy
M	The representation of sketch
$(k,m)$	The number of lines and columns of a sketch
$h_j$	Hash function of the jth line $h_j\left(x\right)\to [0,m-1]$
${\xi }_j$	Hash function of the jth line ${\xi }_j\left(x\right)\to \{-1,+1\}$
$\mathcal{S}$	The shuffling procedure
$\mathcal{R}$	The local randomizer
r	Sample rate for SDPJoinSketch+
$\theta$	High-frequency threshold
$\pi \left(i\right)$	A random permutation operation on index i

3.1. Fast-AGMS

We let F be a data stream with private join attribute values from n users. We use $d_i,i\in \left[n\right]$ to represent a data item in a data stream. We assume D is the domain of all items and $\left|D\right|=N$. For the data stream F, we define the frequency vector $f=(f_1,\dots ,f_t,\dots ,f_N)$ where $f_t$ represents the frequency of the tth item in domain D. Similarly, we can derive the frequency vector of another data stream G and $g=(g_1,\dots ,g_t,\dots ,g_N)$. The inner product of two data streams F and G is defined as $J=f\odot g={\sum }_{t=1}^N{f}_t·g_t$, which is equivalent to the join size.

The Fast-AGMS sketch is one of the typical sketches for join size estimation. A Fast-AGMS sketch [25] consists of an array of m counters. Besides the random function $\xi$ used in the basic AGMS sketch, another hash function h is introduced to map an item to a random counter. Based on the Fast-AGMS sketch, the join size estimation of F and G is the summation of the product of corresponding counters of the two Fast-AGMS sketches $M_F$ and $M_G$:

$$\widehat{J}=Est\left(J\right)=\sum _{x=1}^m{M}_F\left[x\right]\times M_G\left[x\right]$$

(1)

where m is the number of counters in a Fast-AGMS sketch.

The Fast-AGMS scheme is shown in Figure 2. Researches usually use the median estimation of multiple Fast-AGMS sketches to improve accuracy. The estimation can be written as

$$\widehat{J}=Est\left(J\right)=\underset{j\in \left[k\right]}{median}\{\sum _{x=1}^m{M}_F[j,x]\times M_G[j,x]\}$$

(2)

where $k,m$ are the number of lines and columns of a sketch. Intuitively, the estimation accuracy improves as the value of m increases due to lesser hash-collision probability.

Figure 2. Fast-AGMS sketch for join size estimation.

3.2. Centralized Differential Privacy (CDP)

Differential privacy (DP) [5] is a rigorous notion about an individual’s privacy. In the centralized model of DP (abbreviated as CDP), there is a trusted data curator obtaining raw data from all users and introducing proper mechanisms adding noises to the statistical results for preserving privacy.

Definition 1.

($(ϵ,\delta )$-differential privacy ($(ϵ,\delta )$-DP) [5]). A randomized algorithm $R:{\mathbb{X}}^n\to \mathbb{Y}$ satisfies $(ϵ,\delta )$-DP, where $ϵ,\delta \ge 0$, if and only if for any neighboring datasets $\mathcal{D}$ and ${\mathcal{D}}^{\prime }$, $\forall Z\subseteq \mathbb{Y}$:

$$Pr[R\left(\mathcal{D}\right)\in Z]\le e^{ϵ}·Pr[R\left({\mathcal{D}}^{\prime }\right)\in Z]+\delta$$

(3)

where the privacy budget ϵ indicates the degree of privacy protection. A smaller ϵ indicates a stronger privacy guarantee but lower data utility.

3.3. Local Differential Privacy (LDP)

In the local model of DP (abbreviated as LDP), each user independently perturbs their input by using a local randomizer.

Definition 2.

($ϵ$-local differential privacy ($ϵ$-LDP) [6]). A randomized algorithm R is ϵ-locally differentially private if for any pair of different inputs t and $t^{\prime }$, $\forall z\in \mathbb{Y}$:

$$Pr[R\left(t\right)=z]\le e^{ϵ}·Pr[R\left(t^{\prime }\right)=z]$$

(4)

The local model removes the requirement for a trusted curator, resulting in a higher level of noise per user to be tolerated to achieve the same privacy guarantee.

3.4. Shuffle Model of DP (SDP)

In the shuffle model of DP (abbreviated as SDP), a semi-trusted shuffler is deployed as an intermediary component between the clients and the server. The shuffler collects perturbed reports from multiple clients, applies a random permutation, and forwards the shuffled results to the server. We use $\mathcal{P}=(\mathcal{R},\mathcal{S},\mathcal{A})$ to denote the protocol of SDP. Here, $\mathcal{R}$ denotes the local randomizer with privacy budget ${ϵ}_l$, and $\mathcal{S}$ denotes the shuffling procedure. The privacy goal of the shuffle model is to ensure the shuffled messages $\mathcal{S}\circ \mathcal{R}\left(\mathcal{D}\right)=\mathcal{S}(\mathcal{R}\left(x_1\right),\dots ,\mathcal{R}\left(x_n\right))$ satisfy $({ϵ}_c,\delta )$-DP for all neighboring datasets, where ${ϵ}_c$ denotes the amplified privacy level.

Definition 3.

($(ϵ,\delta )$-DP in the shuffle model [16,17]). A protocol $\mathcal{P}=(\mathcal{R},\mathcal{S},\mathcal{A})$ is $(ϵ,\delta )$-DP if, for any neighboring datasets $\mathcal{D}$,${\mathcal{D}}^{\prime }$, the mechanism $\mathcal{M}=\mathcal{S}\circ \mathcal{R}$ satisfies $(ϵ,\delta )$-DP.

In SDP, the privacy of protocol $\mathcal{P}$ is a property of the entire set of n users’ reports, and the data collector only observes the shuffled results while obtaining no information about the linkage between the users and their reports.

Summary of CDP, LDP and SDP. These three DP models present a trade-off between privacy and utility. In terms of reliance on a trusted third party, LDP has the smallest trust assumption. And in terms of utility, CDP offers the highest result accuracy. Overall, the SDP falls between LDP and CDP in both aspects, striking a better balance between the privacy and utility.

Lemma 1.

From ${\mathsf{\Omega }}_{I_{\mathrm{p}}}(\mathcal{T},\mathcal{R})$, the probability of computing tag $t_i$’s $K_i$ or the current session key $K_{ji}$ maintained in $r_j$ is negligible.

Proof. 

The messages in ${\mathsf{\Omega }}_{I_{\mathrm{p}}}(\mathcal{T},\mathcal{R})$ that do not involve $K_i$ or $K_{ji}$ do not give any advantage in obtaining these keys. Here, $K_i$ is used in the form of $E.K_i(K_{ji}\left|\right|{\mathrm{ID}}_{\mathcal{T}}^i)$ and $K_{ji}$ is used in the form of $E.K_i(K_{ji}\left|\right|{\mathrm{ID}}_{\mathcal{T}}^i)$, $E.K_{ji}\left(\delta \right||{\mathrm{ID}}_{\mathcal{T}}^i)$, and $E.K_{ji}\left(\delta \right||K_{ji}^{\prime }\left|\right|$$E.K_i$$\left(K_{ji}^{\prime }\right|\left|{\mathrm{ID}}_{\mathcal{T}}^i\right))$. By Lemma 1, even if adversaries obtain many values of these forms, it is computationally infeasible to obtain $K_i$. As a result, when the length of $K_i$ and $K_{ji}$ is both $l_{\mathrm{key}}$, the probability of computing one of these keys is $1/2^{l_{\mathrm{key}}}$. If $l_{\mathrm{key}}$ is sufficiently long, this is negligible. Therefore, this lemma holds.    □

4. Sketch-Based Join Size Estimation Under SDP

In this section, we introduce a sketch-based join size estimation protocol under SDP (SDPJoinSketch) and give theoretical proof of the privacy amplification after shuffling.

4.1. SDPJoinSketch

We aim to design a protocol $P=(\mathcal{R},\mathcal{S},\mathcal{A})$ under SDP for join size estimation, which can achieve better utility than methods under LDP. The main idea of our proposed protocol is to utilize a semi-trusted shuffler to permute the sanitized reports, which consist of the indices and perturbed values of each individual from all users. This approach breaks the linkage between users and their reports, thus providing privacy benefits compared to solely using the LDP model. In other words, our protocol can provide the same level of privacy guarantee while requiring less noise addition than LDP. Consequently, the estimation utility on the server side is improved.

4.1.1. Client Side of SDPJoinSketch

The client side of SDPJoinSketch has two main tasks, which include encoding the private attribute values of users and then perturbing the encoded results with LDP mechanisms. Finally, for each user, it sends a triplet including a perturbed value and the corresponding indices of the counter in the sketch.

The pseudo-code of the local randomizer is illustrated in Algorithm 1. Given a private join attribute value $d_i\in \mathcal{D}$ from the ith user and the hash function pairs ${\{h_t,{\xi }_t\}}_{t\in \left[k\right]}$ for sketch construction, $h_t:N\to [0,m-1]$ and ${\xi }_t:N\to \{-1,+1\}$. The client side first encodes the value (Lines 3–5). The algorithm samples a line index j and a column index l uniformly at random from $\left[k\right]$ and $\left[m\right]$, respectively. A m-length zero vector v is initialized to record the hash result of value $d_i$, i.e., the $h_j\left(d_i\right)$ position of v is then encoded into ${\xi }_j\left(d_i\right)$. Inspired by the idea in [7], the algorithm leverages the Hadamard basis transform to decrease the communication cost that it generates for vector v, $\omega \leftarrow H_m\times v$, and only chooses the single bit $x=\omega \left[l\right]\in \{-{\xi }_j\left(d_i\right),{\xi }_j\left(d_i\right)\}$ to be further privatized. After encoding, the algorithm perturbs it with randomized response technique (Lines 6–11). Specifically, it first samples a value b from a Bernoulli distribution $Ber\left({\displaystyle \frac{2}{e^{{ϵ}_l}+1}}\right)$, where $b=1$ with probability ${\displaystyle \frac{2}{e^{{ϵ}_l}+1}}$ and $b=0$ with probability ${\displaystyle \frac{e^{{ϵ}_l}-1}{e^{{ϵ}_l}+1}}$. And if $b=0$, the encoding value remains unchanged, i.e., $y\leftarrow x$. Otherwise, if $b=1$, the algorithm uniformly at random chooses a value from $\{-1,+1\}$.

Algorithm 1 Local Randomizer $R_{{ϵ}_l,(k,m),n}$

Public Parameters: ${ϵ}_l,(k,m),n$

Input: Local data $d_i\in \mathcal{D}$ from i-th user, Hash function pairs ${\{h_t,{\xi }_t\}}_{t\in \left[k\right]}$

Output: indices $(j,l)$ and perturbed data $y_i\in \{-1,+1\}$

1: Sample $j,l$ uniformly at random from [k] and [m], respectively.

2: Initialize a vector $v\leftarrow {\left\{0\right\}}^{1\times m}$

3: $v\left[h_j\left(d_i\right)\right]\leftarrow {\xi }_j\left(d_i\right)$              ▹Encode (Lines 3–5)

4: $\omega \leftarrow H_m\times v$         ▹$H_m$ is a hadamard matrix of order m

5: $x\leftarrow \omega \left[l\right]$

6: Sample $b\leftarrow Ber\left({\displaystyle \frac{2}{e^{{ϵ}_l}+1}}\right)$            ▹ Perturb (Lines 6–11)

7: if  $b=0$ then

8:    $y\leftarrow x$

9: else

10:    $y\leftarrow$ Unif({−1,+1})

11: end if

12: return $(j,l),y$

Theorem 1.

The local randomizer $R_{{ϵ}_l,(k,m),n}$ of SDPJoinSketch satisfies ${ϵ}_l$-LDP.

Proof. 

Since the indices $\{(j_i,l_i),i\in \left[n\right]\}$ are sampled uniformly at random, we show for an arbitrary output $y\in \{-1,+1\}$ that the probability of observing the output is similar whether the user reported data d or $d^{\prime }$. We suppose the encodings of d and $d^{\prime }$ are v and $v^{\prime }$, respectively, and the differences between v and $v^{\prime }$ are on two bits, i.e., $v\left[h_j\left(d\right)\right]={\xi }_j\left(d\right)$ and $v^{\prime }\left[h_j\left(d^{\prime }\right)\right]={\xi }_j\left(d^{\prime }\right)$. We let J and L be the random variables selected uniformly from $\left[k\right]$ and $\left[m\right]$, respectively. And B is the random variable for the random bit b. Here, $Pr[B=0]={\displaystyle \frac{e^{{ϵ}_l}-1}{e^{{ϵ}_l}+1}}$, $Pr[B=1]={\displaystyle \frac{2}{1+e^{{ϵ}_l}}}$. We simplify $R_{{ϵ}_l,(k,m),n}$ as R, and therefore

$$\begin{array}{cc}\hfill & {\displaystyle \frac{Pr\left[R\right(d)=((j,l),y\left)\right]}{Pr[R\left(d^{\prime }\right)=((j,l),y)]}}\hfill \\ \hfill & ={\displaystyle \frac{Pr[(1-B)(H_m·v)\left[l\right]+B·Uni\left(2\right)=y|J=j,L=l]}{Pr[(1-B)(H_m·v^{\prime })\left[l\right]+B·Uni\left(2\right)=y|J=j,L=l]}}\hfill \\ \hfill & ={\displaystyle \frac{Pr[(1-B)·(H_m[l,h_j\left(d\right)]·{\xi }_j\left(d\right))+B·Uni\left(2\right)=y|J=j]}{Pr[(1-B)·(H_m[l,h_j\left(d^{\prime }\right)]·{\xi }_j\left(d^{\prime }\right))+B·Uni\left(2\right)=y|J=j]}}\hfill \end{array}$$

where we slightly abuse the notion and use Uni(2) for Unif({−1,+1}) and ${\xi }_j\left(d\right)$, ${\xi }_j\left(d^{\prime }\right)\in \{-1,1\}$, the samples from hadamard transform $(H_m[l,h_j\left(d\right)]·{\xi }_j\left(d\right))$, $(H_m[l,h_j\left(d^{\prime }\right)]·{\xi }_j\left(d^{\prime }\right))\in \{-1,1\}$. We assume the encodings of two different inputs d and $d^{\prime }$ are different, i.e., $(H_m[l,h_j\left(d\right)]·{\xi }_j\left(d\right))=1$ and $(H_m[l,h_j\left(d^{\prime }\right)]·{\xi }_j\left(d^{\prime }\right))=-1$. Without loss of generality, when $y=1$, the maximum probability for input d to obtain $y=1$ is ${\displaystyle \frac{e^{{ϵ}_l}-1}{e^{{ϵ}_l}+1}}+{\displaystyle \frac{1}{2}}·{\displaystyle \frac{2}{e^{{ϵ}_l}+1}}$ and for input $d^{\prime }$ to obtain $y=1$ is ${\displaystyle \frac{1}{2}}·{\displaystyle \frac{2}{e^{{ϵ}_l}+1}}$. We have

$${\displaystyle \frac{Pr\left[R\right(d)=((j,l),y\left)\right]}{Pr[R\left(d^{\prime }\right)=((j,l),y)]}}\le {\displaystyle \frac{\frac{e^{{ϵ}_l}-1}{e^{{ϵ}_l}+1}+\frac{1}{2}·\frac{2}{e^{{ϵ}_l}+1}}{\frac{1}{2}·\frac{2}{e^{{ϵ}_l}+1}}}=e^{{ϵ}_l}$$

(5)

Similarly, we can derive that $e^{-{ϵ}_l}\le {\displaystyle \frac{Pr\left[R\right(d)=((j,l),y\left)\right]}{Pr[R\left(d^{\prime }\right)=((j,l),y)]}}$. Thus, the local randomizer $R_{{ϵ}_l,(k,m),n}$ of SDPJoinSketch satisfies ${ϵ}_l$-LDP.    □

In summary, the client still satisfies LDP, reducing reliance on trusted curator and fostering greater trust in the model of SDPJoinSketch.

4.1.2. Intermediate Shuffler of SDPJoinSketch

In the shuffling process, we aim to shuffle the sanitized reports $Y={\{(j_i,l_i),y_i\}}_{i\in \left[n\right]}$ from all users in a random permutation $\pi$ and then output a shuffled set $Y^{\prime }={\{(j_{\pi \left(i\right)},l_{\pi \left(i\right)}),y_{\pi \left(i\right)}\}}_{i\in \left[n\right]}$, where $\pi \left(i\right)$ represents the shuffling result of index i, to achieve anonymity, which can result in a higher level of privacy protection on the server side. Particularly, we adopt the Fisher–Yates random permutation algorithm [38] for shuffling. The algorithm effectively permutes the inputs by iterating through the elements from the last to the first, swapping each element with another randomly selected element that comes before it. The shuffling process ensures that every permutation $\pi$ of the inputs is equally likely with probability,

$$Pr\left[\pi \right]={\displaystyle \frac{1}{{\prod }_{i=1}^{n}i}}={\displaystyle \frac{1}{n!}}$$

(6)

This result makes the shuffle unbiased, which is a essential property for the subsequent analysis of privacy amplification.

4.1.3. Server Side of SDPJoinSketch

In the context of private join size estimation, the server has two tasks. One is to collect shuffled reports from the shuffler and construct sketches. Another is to estimate join size using these sketches.

The pseudo-code of the analyzer is illustrated in Algorithm 2. The server side observes a multi-set generated from the shuffler and has no information about the linkage between each massage and the user. For the task of sketch aggregation (SKA), the server aims to construct an SDPJoinSketch with all randomized reports. First, an empty sketch M of size $k\times m$ is initialized (SKA, line 1). For each report $(j_{\pi \left(i\right)},l_{\pi \left(i\right)}),y_{\pi \left(i\right)})$ in $Y^{\prime }$, it multiplies $y_{\pi \left(i\right)}$ with k and ${\displaystyle \frac{e^{{ϵ}_l}+1}{e^{{ϵ}_l}-1}}$ for debiasing and then adds it to the sketch M at position $[j_{\pi \left(i\right)},l_{\pi \left(i\right)}]$ (SKA, Lines 2–4). After inserting all items into sketch, sketch M is calibrated by multiplying the hadamard matrix $H_m^T$ to recover information (SKA, Line 5). For the task of join size estimation (JSE), it takes two SDPJoinSketch $M_A$ and $M_B$ as input. It computes the inner product of each row from $M_A$ and $M_B$ (JSE, Lines 1–6) and takes the median of all k results as the final output (JSE, Lines 7–8). We analyze the utility of the estimator under SDP in Section 4.3. To clarify the protocols illustrated above, we provide a concrete example for further explanation.

Algorithm 2 Analyzer $A_{{ϵ}_l,(k,m),n}$

Public Parameters: ${ϵ}_l,(k,m),n$

⊳ Sketch Aggregation: (SKA)

Input: Multi-set $Y^{\prime }={\{(j_{\pi \left(i\right)},l_{\pi \left(i\right)}),y_{\pi \left(i\right)}\}}_{i\in \left[n\right]}$

Output: Private Sketch M

1: Initialize a sketch $M\leftarrow {\left\{0\right\}}^{k\times m}$

2: for  $i\in \left[n\right]$  do

3:       $M[j_{\pi \left(i\right)},l_{\pi \left(i\right)}]\leftarrow M[j_{\pi \left(i\right)},l_{\pi \left(i\right)}]+k·{\displaystyle \frac{e^{{ϵ}_l}+1}{e^{{ϵ}_l}-1}}·y_{\pi \left(i\right)}$

4: end for

5: $M\leftarrow M{H}_m^T$

6: return M

⊳ Join Size Estimation (JSE):

Input: Private Sketches $M_A$ and $M_B$ for user groups A and B

Output: Join Size Estimation $Est$

1: Initialize a vector $Es\leftarrow {\left\{0\right\}}^k$

2: for  $j\in \left[k\right]$  do

3:       for $x\in \left[m\right]$ do

4:             $Es\left[j\right]+=M_A[j,x]\times M_B[j,x]$

5:       end for

6: end for

7: $Est=\underset{j\in \left[k\right]}{median}\left\{Es\left[j\right]\right\}$

8: return  $Est$

Example 1.

We use an example in Figure 3 to show the workflow of SDPJoinSketch. Here, public sketch parameters are $k=3$, $m=4$. $d_i$ denotes the private value of the ith user. Suppose the indices are sampled randomly as $j=1$ and $l=2$, with the corresponding mapping results of the hash function pairs given by $h_j\left(d_i\right)=h_1\left(d_i\right)=2$ and ${\xi }_j\left(d_i\right)={\xi }_1\left(d_i\right)=-1$. ① Each client  encodes $d_i$ to a m-length one-hot vector $v=[0,0,-1,0]$ where $v\left[h_1\left(d_i\right)\right]={\xi }_1\left(d_i\right)$, and then transforms v to a single bit $x_i=1$ by multiplying it with a hadamard matrix $H_m$ and taking the value of l-$th$ bit. ② $x_i$ is then perturbed to $y_i$ based on the value of b, which is sampled from a Bernoulli distribution. Finally, the client sends $(j_i,l_i),y_i$ to the shuffler. ③ After the shuffler   receives the reports from all n users, the shuffler—functioning as an independent component between clients and the server—randomly permutes the data and sends the shuffled results to the server. Notice that at this point, the intrinsic association between each report and its respective users has been severed.  ④  The server   constructs a sketch M by adding each $y_{\pi \left(i\right)}$ multiplied with $H_m^T$ and a scale factor ${\displaystyle \frac{e^{{ϵ}_l}+1}{e^{{ϵ}_l}-1}}$ to the sketch at indices $(j_{\pi \left(i\right)},l_{\pi \left(i\right)})$ for debias.

Figure 3. Example of SDPJoinSketch (semi-trusted shuffler between clients and server).

We can learn from the example that after being permuted by the shuffler, the server loses the association between the user and the report, i.e., i to $\pi \left(i\right)$, since $\pi$ is a random shuffling operation. Therefore, the aggregated sketch on the server side achieves privacy amplification due to the addition of this extra randomness.

4.2. Privacy Amplification via Shuffling

As previously mentioned, the shuffler can break the linkage between the report and the user identification to further obtain some privacy benefits, since more randomness is introduced. Specifically, the server side observes a higher level of privacy protection (i.e., $({ϵ}_c,\delta )$-CDP) after shuffling compared with the situation where only the LDP model (i.e., ${ϵ}_l$-LDP) is implemented, as shown in Figure 4. So the privacy amplification can be presented as the change in ${ϵ}_l$ to ${ϵ}_c$, where ${ϵ}_c<{ϵ}_l$. Equivalently speaking, to measure the utility improvement rather than privacy amplification, we could fix the overall privacy guarantee on the server side ${ϵ}_c$ to derive the ${ϵ}_l$ needed on the client side. Therefore, with the same level of centralized privacy guarantee, less noise is required on the client side when shuffling is used, leading to higher utility. In this subsection, we provide the quantitative results of privacy amplification for SDPJoinSketch.

Figure 4. Framework of SDP (semi-trusted shuffler between clients and server).

In our SDPJoinSketch mechanism, we encode the value into binary form with private view domain $\mathcal{Z}=\{-1,1\}$. As shown in Figure 5, the final sketch aggregated on the server side consists of two parts based on the privacy blanket theory [20]. One is constructed by true reports (the blue sketch) while another is constructed by fake reports (the pink sketch) from uniformly random values. In the context of CDP, the private information of the target user can obscured by the sketch with independent random inputs (the pink sketch). To achieve differential privacy, the aggregated sketch changes by an appropriately bounded amount when computed on neighboring datasets where only a certain user’s input changes, and this user can response truthfully or randomly. Since the privacy argument does not rely on the anonymity of random fake inputs, we focus on the scenario where the target user uploads the true value. Suppose the number of users is n and the nth user responses truthfully, through Algorithm 1; about ${\displaystyle \frac{2(n-1)}{e^{ϵ}+1}}$ users respond uniform randomly. They contribute $B({\displaystyle \frac{2(n-1)}{e^{ϵ}+1}},0.5)$ binomial random noise to each frequency of $z\in \mathcal{Z}$, where the binomial random noise refers to the noise arising from the sum of independent randomized responses from multiple users, which follows a binomial distribution in aggregation. Based on the Binomial Mechanism proposed in [33], we present the formal privacy guarantee of SDPJoinSketch in Theorem 2.

Figure 5. Sketch decomposition under SDP.

Theorem 2.

Given the ${ϵ}_l-LDP$ local randomizer R for SDPJoinSketch, after shuffling, the mechanism $\mathcal{M}=\mathcal{S}\circ {\mathcal{R}}^n:\mathbb{X}\to {\mathbb{Y}}^n$ satisfies $({ϵ}_c,\delta )-CDP$, where

$${ϵ}_c=\sqrt{{\displaystyle \frac{14ln(2/\delta )(e^{{ϵ}_l}+1)}{n-1}}}$$

(7)

Proof. 

We assume D and $D^{\prime }$ are neighboring datasets and differ in the item of the nth user, i.e., $d_n\ne d_n^{\prime }$. From randomizer and shuffler, the outputs of all users are ${\{(j_i,l_i),y_i\}}_{i\in \left[n\right]}$ where $y_i\in \{-1,1\}$. We use E to present the encoding step of item $d_i$ and it is related to the hash function pair $(h_i,{\xi }_i)$ and randomness of hadamard transform (i.e., the selection of $(j,l)$, which equals to $k\times m$). To prove $\mathcal{M}$ satisfies $({ϵ}_c,\delta )-CDP$, it is suffice to prove that

$${\mathrm{Pr}}_{Z\sim \mathcal{M}\left(D\right)}[{\displaystyle \frac{\mathrm{Pr}\left[\mathcal{M}\right(D)=Z]}{\mathrm{Pr}[\mathcal{M}\left(D^{\prime }\right)=Z]}}\ge e^{{ϵ}_c}]\le \delta$$

(8)

where Z denotes the output of mechanism $\mathcal{M}$ on dataset D.

We divide the proof of Equation (8) into four steps. We assume that the attacker has maximum background knowledge, which means they know the true inputs of all $n-1$ users except the target user (the nth user for simplicity). We first start with the input of the nth user. To analyze whether the changes in the aggregated sketch can be bounded on two neighboring datasets differing on the inputs of the nth user for the requirement of differential privacy, we consider two cases based on previous discussion. Then, we decompose the aggregated sketch on the server side into three parts to facilitate the analysis of probabilities. Next, we compute the ratio of probabilities according to binomial mechanism and finally impose constraints on the result to derive upper bound of the amplified privacy budget.

Step 1. Analyze the above inequality in two cases based on the response of the nth user.

We assume that the encodings $E\left(d_n\right)=-1$ and $E\left(d_n^{\prime }\right)=1$. Case 1: The nth user in both D and $D^{\prime }$ submits a random value. Considering the case that the nth user in both D and $D^{\prime }$ submits a random value from $\{-1,1\}$ with probability ${\displaystyle \frac{2}{e^{{ϵ}_l}+1}}$, the privacy holds trivially since $\mathrm{Pr}[\mathcal{M}\left(D\right)=Z]=\mathrm{Pr}[\mathcal{M}\left(D^{\prime }\right)=Z]$. Case 2: The nth user in D and $D^{\prime }$ submits the true value, respectively. We focus on the case where user n responses truthfully with probability ${\displaystyle \frac{e^{{ϵ}_l}-1}{e^{{ϵ}_l}+1}}$. According to the shuffle algorithm, there are $n!$ different permutations $\pi$ in the Fisher–Yates shuffle of messages from n users. We can examine the probability after perturbing and shuffling on D:

$$\mathrm{Pr}\left[\mathcal{M}\right(D)=Z]=\sum _{\pi }{\displaystyle \frac{1}{n!}}\mathrm{Pr}[\mathcal{M}\left(D\right)=Z|\pi ]$$

(9)

Here, $\mathrm{Pr}\left[\mathcal{M}\right(D)=Z|\pi ]$ denotes the output probability under a specific permutation $\pi$.

Step 2. Expand the expression of output probability based on the reports from different kinds of users.

Inspired by the idea of privacy blanket [20], we can expand the probability into three parts, including the probability $P_T$ of reports from the users who respond truthfully among the first $n-1$ users, $P_R$ of reports from users who respond randomly among the first $n-1$ users, and $P_n$ of the report from user n. The probability $\mathrm{Pr}\left[\mathcal{M}\right(D)=Z]$ is expanded to

$$\mathrm{Pr}[\mathcal{M}\left(D\right)=Z]=\sum _{\pi }{\displaystyle \frac{1}{n!}}(P_T·P_R·P_n)$$

(10)

We let $\mathrm{Pr}\left[E_{\pi \left(i\right)}\right]$ represent the probability that user i chooses encoding indices $(j_{\pi \left(i\right)},l_{\pi \left(i\right)})$, where $j_{\pi \left(i\right)}\in \left[k\right]$ and $l_{\pi \left(i\right)}\in \left[m\right]$.Therefore,

$$\begin{array}{c}\hfill P_T=\prod _{i\in T}\mathrm{Pr}\left[E_{\pi \left(i\right)}\right]{\mathbb{I}}_{\{E_{\pi \left(i\right)}\left(d_i\right)=y_i\}}=\prod _{i\in T}{\displaystyle \frac{{\mathbb{I}}_{\{E_{\pi \left(i\right)}\left(d_i\right)=y_i\}}}{k·m}}\end{array}$$

(11)

$$\begin{array}{cc}\hfill P_R& =\prod _{i\in R}\mathrm{Pr}\left[E_{\pi \left(i\right)}\right]{\displaystyle \frac{1}{2}}=\prod _{i\in R}{\displaystyle \frac{1}{k·m}}·{\displaystyle \frac{1}{2}}\hfill \end{array}$$

(12)

where the indicator function ${\mathbb{I}}_{\{E_{\pi \left(i\right)}\left(d_i\right)=y_i\}}$ means that the report from the user in T matches the permuted report. If $(j_i,l_i),y_i$ equals to $(j_{\pi \left(i\right)},l_{\pi \left(i\right)}),y_{\pi \left(i\right)}$, then ${\mathbb{I}}_{\{E_{\pi \left(i\right)}\left(d_i\right)=y_i\}}=1$; otherwise, it is 0. Intuitively, the probabilities $P_T$ and $P_R$ are independent of $d_n$. And the probability for user n can be analyzed as follows:

$$\begin{array}{cc}\hfill P_n& =\mathrm{Pr}\left[E_{\pi \left(i\right)}\right]{\mathbb{I}}_{\{E_{\pi \left(n\right)}\left(d_n\right)=y_{\pi \left(n\right)}\}}={\displaystyle \frac{{\mathbb{I}}_{\{E_{\pi \left(n\right)}\left(d_n\right)=y_{\pi \left(n\right)}\}}}{k·m}}\hfill \end{array}$$

(13)

After we derive the output probabilities of different parts of users, we can combine them to determine the total output probability under permutation $\pi$.

$$\begin{array}{cc}\hfill & \mathrm{Pr}[\mathcal{M}\left(D\right)=Z]=\sum _{\pi }{\displaystyle \frac{1}{n!}}(\prod _{i\in T}{\displaystyle \frac{{\mathbb{I}}_{\{E_{\pi \left(i\right)}\left(d_i\right)=y_i\}}}{k·m}}·\prod _{i\in R}{\displaystyle \frac{1}{k·m}}{\displaystyle \frac{1}{2}}·{\displaystyle \frac{{\mathbb{I}}_{\{E_{\pi \left(n\right)}\left(d_n\right)=y_{\pi \left(n\right)}\}}}{k·m}})\hfill \end{array}$$

(14)

Step 3. Compute the ration of output probabilities on neighboring datasets with binomial random variables.

First, since $P_T·P_R$ is a constant independent of $d_n$ and $d_n^{\prime }$, we can simplify the fraction. Thus,

$$\begin{array}{cc}\hfill {\displaystyle \frac{\mathrm{Pr}\left[\mathcal{M}\right(D)=Z]}{\mathrm{Pr}[\mathcal{M}\left(D^{\prime }\right)=Z]}}& ={\displaystyle \frac{{\sum }_{\pi }{\mathbb{I}}_{\{E_{\pi \left(n\right)}\left(d_n\right)=y_{\pi \left(n\right)}\}}}{{\sum }_{\pi }{\mathbb{I}}_{\{E_{\pi \left(n\right)}\left(d_n^{\prime }\right)=y_{\pi \left(n\right)}\}}}}={\displaystyle \frac{n_{-}}{n_{+}}}\hfill \end{array}$$

(15)

where $n_{-}$ is the number of reports received by the server with value $-1$ except the truthful answers from the first $n-1$ users, i.e., $n_{-}=R_{y=-1}+{\mathbb{I}}_{E\left(d_n\right)=-1}$, which follows a binomial distribution $N_{-}\sim Bin(n-1,{\displaystyle \frac{1}{e^{{ϵ}_l}+1}})+1$. Similarly, $n_{+}$ follows a binomial distribution $N_{+}\sim Bin(n-1,{\displaystyle \frac{1}{e^{{ϵ}_l}+1}})$. Following this result, we simplify the left part of Equation (8):

$$\begin{array}{c}\hfill {\mathrm{Pr}}_{Z\sim \mathcal{M}\left(D\right)}[{\displaystyle \frac{\mathrm{Pr}\left[\mathcal{M}\right(D)=Z]}{\mathrm{Pr}[\mathcal{M}\left(D^{\prime }\right)=Z]}}\ge e^{{ϵ}_c}]=\mathrm{Pr}[{\displaystyle \frac{N_{-}}{N_{+}}}\ge e^{{ϵ}_c}]\end{array}$$

(16)

According to expectation of binomial distribution, we let $\sigma =\mathbb{E}\left[N_{+}\right]={\displaystyle \frac{n-1}{e^{ϵ}+1}}$. Based on Chebyshev’s inequality, ${\displaystyle \frac{N_{-}}{N_{+}}}\ge e^{{ϵ}_c}$ implies that either $N_{-}\ge \sigma e^{{ϵ}_c/2}$ or $N_{+}\le \sigma e^{-{ϵ}_c/2}$.

Step 4. Bound the ration of binomials with union bound and Chernoff bounds, thus deriving the upper bound of ${ϵ}_c$.

We first bound the probability of the ratio of binomials $N_{-}$ and $N_{+}$ with a union bound that $\mathrm{Pr}[{\displaystyle \frac{N_{-}}{N_{+}}}\ge e^{{ϵ}_c}]\le \mathrm{Pr}[N_{+}-\sigma \ge \sigma (e^{{ϵ}_c/2}-1-{\displaystyle \frac{1}{\sigma }})]+\mathrm{Pr}[N_{+}-\sigma \le \sigma (e^{-{ϵ}_c/2}-1)]$. Then, leveraging the Chernoff bound, we have

$$\mathrm{Pr}[N_{+}-\sigma \ge \sigma (e^{{ϵ}_c/2}-1-{\displaystyle \frac{1}{\sigma }})]\le e^{(-{\displaystyle \frac{\sigma }{3}}(e^{{ϵ}_c/2}-1-\frac{1}{\sigma }{)}^2)}$$

(17)

$$\mathrm{Pr}[N_{+}-\sigma \le \sigma (e^{-{ϵ}_c/2}-1)]\le e^{(-{\displaystyle \frac{\sigma }{2}}{(1-e^{-{ϵ}_c/2})}^2)}$$

(18)

Combining these results, the probability in Equation (16) can be bounded:

$$\mathrm{Pr}[{\displaystyle \frac{N_{-}}{N_{+}}}\ge e^{{ϵ}_c}]\le e^{(-{\displaystyle \frac{\sigma }{3}}(e^{{ϵ}_c/2}-1-\frac{1}{\sigma }{)}^2)}+e^{(-{\displaystyle \frac{\sigma }{2}}{(1-e^{-{ϵ}_c/2})}^2)}$$

(19)

Assuming ${ϵ}_c\le 1$, to ensure the overall result is less than $\delta$, each term on the right side of the inequality should be less than or equal to ${\displaystyle \frac{\delta }{2}}$, i.e., $e^{(-\frac{\sigma }{3}(e^{{ϵ}_c/2}-1-\frac{1}{\sigma }{)}^2)}\le {\displaystyle \frac{\delta }{2}}$ and $e^{(-\frac{\sigma }{2}(1-e^{-{ϵ}_c/2}{)}^2)}\le {\displaystyle \frac{\delta }{2}}$. Since $\sigma ={\displaystyle \frac{n-1}{e^{{ϵ}_l}+1}}$, we can derive that

$${ϵ}_c\le \sqrt{{\displaystyle \frac{14ln(2/\delta )(e^{{ϵ}_l}+1)}{n-1}}}$$

(20)

The probability ration is bounded, and the mechanism $\mathcal{M}$ satisfies $(\sqrt{{\displaystyle \frac{14ln(2/\delta )(e^{{ϵ}_l}+1)}{n-1}}},\delta )$-CDP.    □

In this subsection, we give the formal analysis of the privacy amplification of SDPJoinSketch that a lower privacy level ${ϵ}_l$ of LDP on the client side can result a higher privacy level ${ϵ}_c$ of CDP on the server side. Therefore, we can derive ${ϵ}_l$ from a fixed ${ϵ}_c$, i.e., ${ϵ}_l=ln[{\displaystyle \frac{{ϵ}_c^2(n-1)}{14ln(2/\delta )}}-1]$ and ${ϵ}_l>{ϵ}_c$.

4.3. Utility Analysis

Here, we analyze the utility of our proposed method. In particular, to analyze the accuracy of join size estimation, we measure the variance of the estimator, i.e.,

$$Est\left(\right|A\bowtie B\left|\right)=M_A·M_B=\underset{j\in \left[k\right]}{median}\sum _{x=1}^m\{M_A[j,x]·M_B[j,x]\},$$

(21)

where k and m are the number of lines and columns of a sketch.

Fixing the local ${ϵ}_l$, the variance has already been derived under LDP. We restate the result and extend it to SDP.

Lemma 2

([15]). Given the sketch size $(k,m)$ and the LDP parameter ${ϵ}_l$, the variance for an estimator $M_A\left[j\right]·M_B\left[j\right]$ under LDP is bounded up to ${\displaystyle \frac{2}{m}}(n_A+{\displaystyle \frac{k{\left(c_{{ϵ}_l}\right)}^2-1}{2}}{)}^2(n_B+{\displaystyle \frac{k{\left(c_{{ϵ}_l}\right)}^2-1}{2}}{)}^2$, where $c_{{ϵ}_l}={\displaystyle \frac{e^{{ϵ}_l}+1}{e^{{ϵ}_l}-1}}$, $n_A$ and $n_B$ represent the total number of users for attribute A and attribute B, respectively.

Based on the lemma, we derive the variance of SDPJoinSketch. To ensure the privacy level of the two join attribute values is the same on the server side, we fix the ${ϵ}_c$ and compute ${ϵ}_l^A$ and ${ϵ}_l^B$ on the client side, respectively.

Theorem 3.

Given centralized privacy budget ${ϵ}_c$ in the shuffler model, we denote ${ϵ}_l^A$ and ${ϵ}_l^B$ as the amplified result on the client side for SDPJoinSketches $M_A$ and $M_B$. Thus, the variance of shuffle model is bounded by ${\displaystyle \frac{2}{m}}(n_A+{\displaystyle \frac{k{\left(c_A\right)}^2-1}{2}}{)}^2(n_B+{\displaystyle \frac{k{(c_B)}^2-1}{2}}{)}^2$, where $c_A={\displaystyle \frac{{\left({ϵ}_c\right)}^2(n_A-1)}{{\left({ϵ}_c\right)}^2(n_A-1)-28ln(2/\delta )}}$, $c_B={\displaystyle \frac{{\left({ϵ}_c\right)}^2(n_B-1)}{{\left({ϵ}_c\right)}^2(n_B-1)-28ln(2/\delta )}}$.

Proof. 

From Theorem 2, we have $e^{{ϵ}_l}={\displaystyle \frac{{\left({ϵ}_c\right)}^2(n-1)}{14ln(2/\delta )}}-1$ and $c_{{ϵ}_l}={\displaystyle \frac{e^{{ϵ}_l}+1}{e^{{ϵ}_l}-1}}={\displaystyle \frac{{\left({ϵ}_c\right)}^2(n-1)}{{\left({ϵ}_c\right)}^2(n-1)-28ln(2/\delta )}}$. Since the privacy benefits are related to the number of users in data stream, the amplification results ${ϵ}_c$ for $M_A$ and $M_B$ are different. Plugging them in Lemma 2, the variance is ${\displaystyle \frac{2}{m}}(n_A+{\displaystyle \frac{k{\left(c_A\right)}^2-1}{2}}{)}^2(n_B+{\displaystyle \frac{k{\left(c_B\right)}^2-1}{2}}{)}^2$, where $c_A={\displaystyle \frac{{\left({ϵ}_c\right)}^2(n_A-1)}{{\left({ϵ}_c\right)}^2(n_A-1)-28ln(2/\delta )}}$, $c_B={\displaystyle \frac{{\left({ϵ}_c\right)}^2(n_B-1)}{{\left({ϵ}_c\right)}^2(n_B-1)-28ln(2/\delta )}}$.    □

Intuitively, SDPJoinSketch has a lower variance bound compared to the conventional LDP setting due to the larger privacy budget on the client side.

5. Improving Utility of SDPJoinSketch

In this section, we design protocols to improve utility of SDPJoinSketch by reducing hash-collision errors of the sketch under SDP.

5.1. Framework of SDPJoinSketch+

In the scenarios of real data, the estimation accuracy of SDPJoinSketch is usually poor because the real data often obey unbalanced distribution and are highly skewed. When constructing the sketch, the hash-collision errors are mainly introduced by the collision between high-frequency items and low-frequency items, since the probability of collisions between high-frequency items is so small that it can be ignored. Inspired by this conclusion, we can enable users to upload precise information about whether they possess high-frequency items. Nevertheless, two issues still need to be addressed. As we discussed earlier, the link between sanitized reports and individual users is broken, allowing us to preserve the true frequency property and the corresponding sanitized report to the server side. However, each client-side user has no idea whether their item is high-frequency or low-frequency. Furthermore, neither the true frequency property nor the true value can be disclosed to the semi-trusted shuffler.

To address the first issue, we propose a two-phase framework SDPJoinSketch+, which identifies high-frequency items in the first phase as a prior knowledge to help the frequency property identification in the second phase. In both of the phases, users’ true item values are protected through DP noise addition, which still satisfies the constraints of SDP. For the second issue, we leverage a secure encryption technique to achieve this goal. Specifically, in addition to transmitting the sanitized reports by SDP, users also encrypt and transmit the true frequency properties obtained from the first phase. There are two motivations. One is that partitioning the privacy budget for the protection of frequency properties introduces more noise into the item value perturbation, resulting in low utility. Another is that the advantage of the shuffle model has not been fully utilized, as we can allow the server side to obtain the true frequency property of each report while ensuring the privacy guarantee of user data, since the linkage between reports and users is disrupted by the intermediate shuffler.

The framework of SDPJoinSketch+ is shown in Figure 6. The main idea of this improved framework is that we identify the high-frequency item set $FI$ through a group of sampled users in the first phase by frequency estimation. And then the server broadcasts this set to remaining users so that each user can identify whether its data are high-frequency or low-frequency.

Figure 6. Framework of SDPJoinSketch+.

In terms of results, an indicator is introduced on the client side which we denote as $t_i$ where $t_i=1$ if the sensitive data of user i are in $FI$ and $t_i=0$ if not. Since the value of $t_i$ is also sensitive information, we need to handle it with privacy-preserving techniques. Instead of using differential privacy which causes utility reduction, we leverage secure encryption techniques, such as symmetric encryption, to protect $t_i$. Each user encrypts its $t_i$ to obtain $e_i=Ecrypt\left(t_i\right)$ and submits it to the shuffler with LDP reports, i.e., $\{e_i,(j_i,l_i),y_i\}$. After receiving reports from n users, the shuffler uniform-randomly permutes them and handles the set ${\{e_{\pi \left(i\right)},(j_{\pi \left(i\right)},l_{\pi \left(i\right)}),y_{\pi \left(i\right)}\}}_{i\in \left[n\right]}$ to the server. On the server side, it first decrypts the $e_{\pi \left(i\right)}$ of every report, i.e., $r_{\pi \left(i\right)}=Decrypt\left(e_{\pi \left(i\right)}\right)\in \{0,1\}$, thus acquiring the knowledge of whether this item is high-frequent but having no idea of which user it belongs to. Through this method, we can effectively separate high- and low-frequency items in the report while preserving privacy.

The pseudo-code is illustrated in Algorithm 3. In Stage 1, we aim to find the frequent item set $FI$ based on the sampled user subsets $A_S$ and $B_S$ for attributes A and B. The server constructs sketches $M{A}_S$, $M{B}_S$ using the perturbed and shuffled reports from two subsets (Line 1–2).

Then, we can obtain the frequency item set $FI$ (Line 3) through $\mathrm{FFI}$, whose pseudo-code is shown in Algorithm 4. The server side first estimates the frequency vector based on SDPJoinSketch ($\mathrm{FFI}$, Line 1–7) and then finds the high-frequency items whose frequency estimates are larger than the threshold ($\mathrm{FFI}$, Line 8–15). The server broadcasts this set to each user client. In Stage 2, the client side adds indicators t for each user to identify whether its item is high-frequency or not and encrypts the indicator (Line 1–5). It then perturbs data from remaining users and sends them to the shuffler (Line 6–7). After receiving the shuffled messages, the server first decrypts each $e_{\pi \left(i\right)}$ to $r_{\pi \left(i\right)}\in \{0,1\}$. So the shuffled messages can be divided into two parts, which are used for high-frequency and low-frequency sketch construction, respectively (Line 10–11). We use the $\mathrm{JSE}$ in Algorithm 2 to estimate the join size for low-frequency items $LEst$ and high-frequency items $HEst$ (Line 12–13). Finally, we compute the overall join size by scale the sum of $LEst$ and $HEst$ with factor ${\displaystyle \frac{\left|A\right|·\left|B\right|}{|A_R|·|B_R|}}$ (Line 14), where the notion $|·|$ denotes the total number of users in a subset.

Algorithm 3 SDPJoinSketch+

Public Parameters: $ϵ,(k,m),n$

⊳ Stage 1: Find frequent join values

Input: Sampled subsets $A_S,B_S$

Output: Frequent Item set $FI$

1: Clients: Perturb data from $A_S,B_S$ with Algorithm 1

2: Server: Construct SDPJoinSketch $M{A}_S$ and $M{B}_S$

3: Frequent item set $FI\leftarrow \mathrm{FFI}\left(M{A}_S\right)\cap \mathrm{FFI}\left(M{B}_S\right)$

4: return  $FI$

⊳ Stage 2: Improved join size estimation

Input: Remaining subsets $R:A_R$,$B_R$ and $FI$

Output: Join Size Estimation $JSest$

1: Clients: add indicator t for each user.

2: for $d_i\in A_R$ and $B_R$ do

3:       $t_i=1$ if $d_i\in FI$, else $t_i=0$

4:       $e_i\leftarrow Encrypt\left(t_i\right)$

5: end for

6: Perturb data from $A_R$ and $B_R$ with Algorithm 1

7: Return $Y={\{e_i,(j_i,l_i),y_i\}}_{i\in \left[\right|R\left|\right]}$

8: Shuffler:  $Y\to Y^{\prime }={\{e_{\pi \left(i\right)},(j_{\pi \left(i\right)},l_{\pi \left(i\right)}),y_{\pi \left(i\right)}\}}_{i\in \left[\right|R\left|\right]}$

9: Server: $r_{\pi \left(i\right)}\leftarrow Decrypt\left(e_{\pi \left(i\right)}\right)$

10: Construct sketches $M{A}_H$, $M{B}_H$ with $Y_{\{r_{\pi \left(i\right)}=1\}}^{\prime }$

11: Construct sketches $M{A}_L$, $M{B}_L$ with $Y_{\{r_{\pi \left(i\right)}=0\}}^{\prime }$

12: $LEst\leftarrow JSE(M{A}_L,M{B}_L)$ in Algorithm 2

13: $HEst\leftarrow JSE(M{A}_H,M{B}_H)$ in Algorithm 2

14: $JSest={\displaystyle \frac{\left|A\right|·\left|B\right|}{|A_R|·|B_R|}}(LEst+HEst)$

15: return  $JSest$

Algorithm 4 FFI: Find Frequent Items

Public Parameters: $ϵ,(k,m),n$, frequent threshold $\theta$

Input: SDPJoinSketch $M_S$

Output: Frequency item set $FI$

1: Initialize $\widetilde{f_S}={\left\{0\right\}}^{1\times \left|D\right|}$

2: for $d\in D$ do                    ▹ Estimating frequency

3:      for $d\in \left[k\right]$ do

4:             $\widetilde{f_S}\left[d\right]+=M_S[j,h_j\left(d\right)]·{\xi }_j\left(d\right)$

5:      end for

6:      $\widetilde{f_S}\left[d\right]=\widetilde{f_S}\left[d\right]/k$

7: end for

8: $T=\theta ·{\sum }_d^D\widetilde{f_S}\left[d\right]$             ▹ Threshold for high-frequency item

9: Initialize $FI=\{\varnothing \}$

10: for  $d\in D$  do

11:      if $\widetilde{f_S}\left[d\right]>T$ then

12:            Add d into $FI$

13:      end if

14: end for

15: return  $FI$

5.2. Privacy and Utility Analysis

User size n is a critical parameter in the shuffle model. Since in SDPJoinSketch+ we divide the users into two groups of different sizes for separate goals, the privacy amplification results of the two phases are different.

Theorem 4.

Given the sample rate r and local privacy budget ${ϵ}_1=\sqrt{{\displaystyle \frac{14ln(2/\delta )(e^{{ϵ}_l}+1)}{n·r-1}}}$, ${ϵ}_2=\sqrt{{\displaystyle \frac{14ln(2/\delta )(e^{{ϵ}_l}+1)}{n·(1-r)-1}}}$, Phase 1 of SDPJoinSketch+ satisfies $({ϵ}_1,\delta )-DP$ and Phase 2 of SDPJoinSketch+ satisfies $({ϵ}_2,\delta )-DP$, where r is the sampling rate.

Proof. 

Similar to the proof of Theorem 2. Note that the number of users is different in the two phases, thus leading to different privacy amplification results. □

The privacy result shows that when sampling rate r decreases, the first phase brings greater privacy benefits thus obtaining more accurate result of frequency estimation. On the other hand, the join size estimation based on remaining users in the second phase acquires worse utility. However, in SDPJoinSketch+, we only need the frequent items set rather than accurate frequencies. Even when the amplified ${ϵ}_l$ is small, we can use approximate frequency estimation to identify heavy hitters, especially in highly skewed data. Overall, the utility of SDPJoinSketch+ may become worse when the sampling rate increases, and our experimental results in Section 6.2 also confirm this conclusion.

For simplicity, we assume that frequent item set $FI$ obtained from Algorithm 4 is sufficiently accurate. In practice, this assumption has been validated through preliminary experiments: SDPJoinSketch+ achieves over $90\%$ accuracy in identifying the $FI$ during Phase 1, which is consistent with the experimental results presented in Section 6.2. Therefore, the final estimation result of SDPJoinSketch+ $JSest$ depends only on the sampling rate and estimation from high-frequency and low-frequency items in the second phase. We give the error bound as follows.

Theorem 5.

Given the sketch size parameters $(k,m)$, sampling rate r, and centralized privacy budget ${ϵ}_c$ in the shuffler model, we denote ${ϵ}_l^{A_R}$ and ${ϵ}_l^{B_R}$ as the local privacy budgets on the client side for the remaining users of data streams A and B, respectively, due to the privacy amplification effect. Assuming the proportion of users holding high-frequency items is μ, the variance for a join size estimation $JSest$ of SDPJoinSketch+ in Algorithm 3 is limited, i.e., $Var\left[JSest\right]\le {\displaystyle \frac{2}{m}}[\mu (1-r)n_A+{\displaystyle \frac{k{\left(c_{A_R}\right)}^2-1}{2}}{]}^2[\mu (1-r)n_B+{\displaystyle \frac{k{\left(c_{B_R}\right)}^2-1}{2}}{]}^2+{\displaystyle \frac{2}{m}}[(1-\mu )(1-r)n_A+{\displaystyle \frac{k{\left(c_{A_R}\right)}^2-1}{2}}{]}^2[(1-\mu )(1-r)n_B+{\displaystyle \frac{k{\left(c_{B_R}\right)}^2-1}{2}}{]}^2$, where $c_{A_R}={\displaystyle \frac{{\left({ϵ}_c\right)}^2((1-r)n_A-1)}{{\left({ϵ}_c\right)}^2((1-r)n_A-1)-28ln(2/\delta )}}$ and $c_{B_R}={\displaystyle \frac{{\left({ϵ}_c\right)}^2((1-r)n_B-1)}{{\left({ϵ}_c\right)}^2((1-r)n_B-1)-28ln(2/\delta )}}$.

Proof. 

Similar to the proof of Theorem 3. But there are two main differences. First, the local privacy budgets ${ϵ}_l$ vary with the change in user numbers by sampling due to the privacy amplification effect. Second, since the construction of sketches for high-frequency items $MH$ and low-frequency items $ML$ are independent, the variance of join size estimation results $Var\left[JSest\right]=Var[LEst+HEst]$ can be derived as the summation of $Var\left[LEst\right]=Var[M{L}_A\left[j\right]·M{L}_B\left[j\right]]$ and $Var\left[HEst\right]=Var[M{H}_A\left[j\right]·M{H}_B\left[j\right]]$, which can be easily calculated based on Theorem 3. □

The secure encryption techniques used in the second phase can prevent the privacy leakage from the shuffler, and the server can obtain accurate frequency information of the sanitized and anonymized reports. However, the frequency properties are not under the protection of differential privacy since the malicious users still can infer whether the item of an individual in the remaining users is frequent or infrequent by differential attacks. A simple way to address this issue is dividing the privacy budget and adding random noise into the frequency property indicators, but it significantly reduces utility. The experimental results of comparison between encryption and LDP perturbation in Section 6.2 validate this inference.

Moreover, our method can be easily extended to multi-way join scenarios. This is similar to LDPJoinSketch [15], with the difference being the need to analyze the privacy amplification effect when multiple parties are involved.

6. Experimental Evaluation

In this section, we mainly evaluate the utility of our proposed methods SDPJoinSketch and SDPJoinSketch+ for private join size estimation on both synthetic and real-world datasets.

6.1. Experiment Setup

All the experiments are implemented on a machine of 256 GB RAM running Ubuntu (20.04.1) with Python 3.9.

• Datasets: We use both synthetic and real-world datasets.

• Zipf datasets. We generate several datasets of size 1,000,000 following Zipf distribution, with skewness parameters $\alpha$ ranging from $1.1$ to $2.0$ (bigger $\alpha$ denotes higher skewness) and the data domain fixed at 500 for simplicity. The Zipf distribution reflects the skewed frequency patterns commonly found in real-world workloads.
• Gaussian dataset. We also generate a dataset of size 1,000,000 following Gaussian distribution, with a mean of 5,000 and a standard deviation of 50.
• Twitter ego-network dataset (https://snap.stanford.edu/data/ego-Twitter.html, accessed on 26 September 2025). Real-world dataset consists of data from 2,420,766 items across 77,072 domains from the Twitter app.
• Facebook ego-network dataset (https://snap.stanford.edu/data/ego-Facebook.html, accessed on 26 September 2025). Real-world dataset consists of data from 352,936 items across 4039 domains from the Facebook app.

• Competitors: To illustrate the effectiveness of our algorithm, we compare our SDPJoinSketch (SJS) and SDPJoinSketch+ (SJS+) with existing representative SDP, LDP, and DP methods.

SDP methods: Hist_KR [20], SFLH [33].

• Hist_KR. Applying basic randomized response mechanism on the ESA framework.
• SFLH. A Fast variant of SLH [33], which combines privacy amplification theory based on optimally local hashing.

LDP methods: KRR [11], FLH [12], LDPJoinSketch [15].

• KRR. K-ary randomized response that perturbs the join values and computes the join size with calibrated frequency vectors.
• FLH. The heuristic fast variant of Optimal Local Hashing (OLH), where multiple counters are used to improve efficiency.
• LDPJoinSketch (LJS). A sketch-based join size estimation method under LDP.

CDP methods: Laplace Mechanism [39].

• Laplace Mechanism (Lap). Ensuring privacy by adding noise drawn from a Laplace distribution to the original data.

Note that except LJS, all compared methods calculate the join size by the inner product of estimated frequency vectors. Intuitively, the SDP methods strike better balance between privacy and utility, which means they have better accuracy than LDP methods and more relaxed privacy assumption than CDP methods.

• Metrics: In the experiments, we use Absolute Error (AE) and Relative Error (RE) to measure data utility performance.

• Absolute Error (AE). ${\displaystyle \frac{1}{T}}\sum |\mathcal{J}-\widehat{\mathcal{J}}|$, $\mathcal{J}$ is the actual join size, $\widehat{\mathcal{J}}$ is the estimated result, and T is the testing rounds.
• Relative Error (RE). ${\displaystyle \frac{1}{T}}\sum |\mathcal{J}-\widehat{\mathcal{J}}|/\mathcal{J}$. The parameters are the same as those defined in AE.

• Methodology: We set $\delta ={10}^{-9}$ by default. The result for each experiment is averaged over 10 runs with different hash seeds. The threshold $\theta$ is set to 0.1 for Zipf datasets, 0.01 for the Gaussian dataset, and 0.001 for Twitter and Facebook datasets. The privacy budget $ϵ$ involved in the experiments represents the central guarantee.

6.2. Utility Comparison

We compare the performance of our algorithm SJS and SJS+ with several state-of-the-art methods. Particularly, we analyze the impact of different parameters on the accuracy of join size estimation on both synthetic and real-world datasets.

Impact of privacy budget $ϵ$. We vary the overall privacy guarantee $ϵ$ against the server from 0.1 to 1 and plot AE. The sketch parameters $(k,m)$ are set to $(9,256)$ and the sample rate is fixed at $0.1$. Figure 7 shows the results on both synthetic and real-world datasets. We observe that the accuracy of all methods improves as the value of $ϵ$ increases, and when $ϵ$ is small, our proposed methods SJS and SJS+ have better utility than SFLH and other methods under LDP. We ignore the result of Hist_KR on the Twitter dataset and Zipf datasets with $ϵ\le 0.3$ due to its unsatisfactory results with the privacy amplification effect when d is too large [20]. Hist_KR performs the best when $ϵ$ is large, but it incurs a significant communication overhead (shown in subsequent experimental results), making its implementation impractical. The experimental results for each method are the averages of 10 independent runs, with the errors relative to the mean being minimal (typically within a few percentage points), indicating that the final results exhibit high robustness.

Figure 7. Impact of the privacy budget $ϵ$ on the estimation accuracy of join size. (a) Zipf ($\alpha =1.1$); (b) Zipf ($\alpha =1.5$); (c) Zipf ($\alpha =2.0$); (d) Twitter; (e) Gaussian; (f) Facebook.

Communication cost. Figure 8 shows the communication costs of different SDP methods on Zipf ($\alpha =1.5$) and Twitter datasets. The sketch parameters for SJS and SJS+ are set to $k=9, m=256$, and the central privacy budget $ϵ=0.5$. We can observe that Hist_KR has the highest communication cost, especially on the Twitter dataset, whose data domain is extremely large. Both Hist_KR and SFLH have communication overhead that exhibits a logarithmic relationship with the size of data domain, i.e., $logN$, making it difficult for practical applications. Instead, the communication overhead of our SJS and SJS+ is independent of the domain and can be further reduced by optimizing the encryption technique in SJS+.

Figure 8. Communication cost on Zipf ($\alpha =1.5$) and Twitter datasets.

Impact of sampling rate r. We fix $ϵ=0.5$, $k=9$, $m=256$ and evaluate the performance of SJS+ by varying r from 0.1 to 0.5 on Zipf datasets. Figure 9 shows the results. Consistent with our analysis in Section 5.2, the estimation error of SJS+ increases with the increase in sampling rate.

Figure 9. Impact of sampling rate r on the estimation error of SJS+.

Impact of the data skewness. We fix $ϵ=0.5$, $k=9$, $m=256$. We evaluate the performance of different methods on Zipf datasets with the skewness parameter $\alpha$ varying from $1.1$ to $2.0$, where larger $\alpha$ indicates higher skewness. When the RE is below the grey dotted line, it indicates that the method is effective. As shown in Figure 10, SJS and SJS+ achieve better utility with different skewness levels because of the separation of high-frequency and low-frequency items for reducing hash collisions.

Figure 10. Impact of data skewness $\alpha$ on join size estimation accuracy.

Comparison between encryption and LDP perturbation for frequency property in SDPJoinSketch+. To verify the effectiveness and necessity of using secure encryption methods on frequency properties in SJS+, we compare it with traditional differential privacy methods that add noise to the properties. Note that the information uploaded by users in the second phase can be regarded as a two-dimensional value, i.e., the value to be inserted to the sketch $(j,l),y$ and the frequency property t. Therefore, the privacy budget needs to be divided to satisfy local differential privacy. Figure 11 shows the results of AE comparison between using encryption and directly adding LDP noise to the frequency properties on Zipf ($\alpha =1.5$), where we fix $k=9$, $m=256$. The decimal values in parentheses indicate the portion of the privacy budget allocated to the frequency properties. We can find that the encryption technique outperforms the LDP methods. And the more randomness added to the frequencies, the lower the estimation accuracy, which further demonstrates the necessity of preserving the true information of frequency properties.

Figure 11. Encryption vs. LDP perturbation for frequency property in SDPJoinSketch+.

Impact of sketch size $(k,m)$. We set $ϵ=0.5, r=0.1$ and evaluate the impact of different sketch sizes. We first fix $m=256$ and vary k in a range of $\{3, 6, 9, 12, 15\}$. When the AE is below the grey dotted line, it indicates that the method is effective. Figure 12 shows the impact of k on estimation accuracy. SJS and SJS+ also perform the best. The absolute error decreases as k increases, because more estimation results are used to calculate the median, which helps reduce the error. However, when k becomes too large, it may lead to the accumulation of systematic bias or an increase in the proportion of extreme values, thereby reducing the estimation accuracy. Second, we fix $k=9$ and vary m in a range of $\{64, 128, 256, 512, 1024\}$.

Figure 12. Impact of sketch size parameter k on the join size estimation accuracy. (a) k:Zipf ($\alpha$ = 1.1); (b) k:Zipf ($\alpha$ = 1.5); (c) k:Zipf ($\alpha$ = 2.0); (d) k:Twitter; (e) k:Gaussian; (f) k:Facebook.

Figure 13 shows the impact of m. The error of all methods decreases as m increases due to the lower hash-collision probability, but it may cause higher space and computation cost.

Figure 13. Impact of sketch size parameter m on the join size estimation accuracy. (a) m:Zipf ($\alpha$ = 1.1); (b) m:Zipf ($\alpha$ = 1.5); (c) m:Zipf ($\alpha$ = 2.0); (d) m:Twitter; (e) m:Gaussian; (f) m:Facebook.

Summaries of experimental results.

• SDPJoinSketch has better utility than methods under LDP for join size estimation.
• The enhanced mechanism SDPJoinSketch+ further improves utility by reducing hash-collision errors and has smaller communication overhead than conventional LDP protocols.
• Dealing with high-skewed data with small privacy budget, our proposed methods demonstrate better performance.

7. Conclusions

In this paper, we study the problem of join size estimation under the shuffle model of differential privacy. We propose a sketch-based method SDPJoinSketch and analyze its privacy amplification via shuffling. Furthermore, to reduce errors caused by hash-collision between high- and low-frequency items, we propose an improved method SDPJoinSketch+, which is a two-stage framework leveraging the advantages of the shuffle model to separate high-frequency and low-frequency items with secure encryption techniques during sketch construction, thus improving the utility of join size estimation. Experimental results show that our methods outperform state-of-the-art ones with respect to utility.

Future work: First, we consider techniques to improve the efficiency and handling more complex join operations such as self-joins and joins with predicates under SDP. Second, we consider further improving utility by optimizing the process of sketch construction, such as choosing an appropriate construction method to reduce unnecessary errors.