Kernel Density Estimation for Joint Scrambling in Sensitive Surveys

Abstract

Randomized response models aim to protect respondent privacy when sampling sensitive variables but consequently compromise estimator efficiency. We propose a new sampling method, titled joint scrambling, which preserves all true responses while protecting privacy by asking each respondent to jointly speak both their true response and multiple random responses in an arbitrary order. We give a kernel density estimator for the density function with asymptotically equivalent mean squared error for the optimal bandwidth yet greater generality than existing techniques for randomized response models. We also give consistent, unbiased estimators for a general class of estimands including the mean. For the cumulative distribution function, this estimator is more computationally efficient with asymptotically lower mean squared error than existing approaches. All results are verified via simulation and evaluated with respect to natural generalizations of existing privacy notions.

1. Introduction

The validity of sampling methodologies for random variables and corresponding estimators assumes respondent honesty. With sensitive variables, respondents naturally hesitate to divulge private and potentially compromising information. Thus, sensitive variable sampling methods exhibit a trade-off between privacy and efficiency in which schemes are engineered to protect respondent privacy without significantly compromising the performance of respective estimators. Most celebrated is the randomized response model, initially posited by Warner [1] for a binary random variable, Y. Warner’s randomization is achieved by asking the direct or indirect question with probability p or $1-p$, respectively, recording the corresponding response $Z=Y$ or $Z=1-Y$. Greenberg et al. [2] replaces this indirect question with an unrelated question $X\sim \mathrm{Bern}\left({\pi }_X\right)$, generally with known parameter ${\pi }_X$. In this manner, if a respondent answers “yes” to the question of interest, for $0<p<1$ the interviewer cannot tell with certainty if the respondent is admitting to the sensitive question or the unrelated/indirect question, and thus, respondent privacy is protected. Of course, as $|p-(1-p\left)\right|$ increases, privacy declines, as the interviewer can use the fact that one question is more likely than another to their predictive advantage. As $|p-1/2|$ decreases, by the privacy and efficiency trade-off, this additional protection comes at the cost of larger estimator variance.

Gupta et al. [3] aimed to compromise this trade-off via optionality, in which respondents use randomized response if and only if the question is sensitive for them, responding truthfully otherwise. Accordingly, the true response of a portion of respondents, that is, those who find the question non-sensitive, is preserved, decreasing the variance in comparison with the absence of optionality. Our proposed sampling method, joint scrambling, is motivated by a similar desire: to preserve all true responses of respondents without compromising privacy.

The aforementioned methods yield analogs for quantitative sensitive variables. Greenberg et al. [4] parallels the binary case, replacing the known parameter ${\pi }_X$ with the generally known distribution of X. For Warner, the concept of an indirect question becomes obselete, replacing Z with a linear scrambling $Z=Y+S$ where S is some independent noise drawn from a known distirbution [5]. In this vein, authors have considered alternative scrambling schemes, including multiplicative scrambling $Z=MY$ [6] and a general scrambling of the form $Z=MY+S$ where the multiplicative scrambling noise M is independent from the linear scrambling noise S [7]. Besides increasing variance, all scrambling mechanisms where $M,S$ are continuous random variables almost surely never provide a true respondent response, as this occurs solely when $M=1$ and $S=0$, which occurs with probability 0.

Summarizing the privacy protection of randomized response, Chapter 5 of Chaudhuri and Mukerjee [8] states that respondents, “must be convinced that their privacy is well guarded before they will be persuaded to make available damaging and incriminating documents”. However, beyond the mathematics of confidentiality protection, in practice, privacy protection is effective insofar as it is simple and explainable to the respondent.

We propose joint scrambling as an explainable sampling methodology for sensitive random variables that is well suited for kernel density estimation. Doing so, we also prove the asymptotic dominance of distribution function estimators to existing methods and give mean estimators asymptotically on par with the efficiency of [5] while still reaping privacy benefits.

2. Joint Scrambling

The motivation of joint scrambling is to maintain respondent privacy while preserving the true answers of all respondents. We do so by requesting each respondent to respond with not just their true answer but a list of answers including their true answer and r other responses that are randomly sampled from some known distribution.

Formally, we begin by fixing the sampling parameter $r\ge 0$. Our sampling method proceeds as follows. The k-th respondent is asked to privately record ${\overrightarrow{Y}}_k=(Y_k^1,\dots ,Y_k^{r+1})$ where

$$\begin{array}{cc}\hfill Y_k^1& =Y_k\hfill \\ \hfill Y_k^i& \stackrel{\mathrm{i}.\mathrm{i}.\mathrm{d}.}{\sim }S(2\le i\le r+1)\hfill \end{array}$$

such that $Y_k$ is their true response and $F_S$ is the known distribution of some randomization device. The respondent then speaks their private recordings in increasing order, which the interviewer records as ${\overrightarrow{Z}}_k=(Z_k^1,\dots ,Z_k^{r+1})$ where

$$Z_k^i=Y_k^{\left(i\right)}(1\le i\in r+1)$$

is the i-th order statistic of the k-th respondent’s recordings.

In this fashion, the respondent always reports their true response as $Y_k\in \{Z_k^1,\dots ,Z_k^{r+1}\}$, yet the interviewer is inhibited from deducing which of $\{Z_k^1,\dots ,Z_k^{r+1}\}$ is the true response. Joint scrambling refers to the technique of masking a signal by shuffling scrambling variables and reporting a joint distribution as opposed to perturbing the signal itself.

3. Distribution Estimators for Continuous Variables

We first aim to construct computationally efficient estimators with asymptotically equivalent or superior mean squared error. We begin with estimating quantitative distributions, specifically the probability density function $f_Y$ and the cumulative distribution function $F_Y$. For convenience, we use the following notation:

$$M_a^b\left(g\right)={\int }_{-\infty }^{\infty }{t}^a{g}^b\left(t\right)dt(g:\mathbb{R}\to \mathbb{R})$$

Existing work on such estimators for randomized response models derive from the following kernel density estimator of the reported responses, $Z_1,\dots ,Z_n$, with bandwidth h and known kernel $k:\mathbb{R}\to \mathbb{R}$.

$${\widehat{f}}_Z\left(z\right)={\displaystyle \frac{1}{nh}}\sum _{i=1}^{n}k\left({\displaystyle \frac{z-Z_i}{h}}\right)={\displaystyle \frac{1}{n}}\sum _{i=1}^n{k}_h\left(z-Z_i\right)$$

(1)

where $k_h\left(t\right)=k(t/h)/h$. By Wand and Jones [9], this estimator admits the following optimal bandwidth with respect to the asymptotic mean integrated squared error.

$$h_Z^{\ast }={\left\{{\displaystyle \frac{M_0^2\left(k\right)}{M_2^1{\left(k\right)}^2{M}_0^2\left(f_Y^{″}\right)}}\right\}}^{1/5}$$

(2)

3.1. Density Estimation

Current work only regards randomized response models using purely additive [10] or multiplicative scrambling [11] with a specific distribution, typically uniform. We begin by giving the following assumptions, as standard in the literature [9,10,11].

Assumption 1. 

Suppose $S\sim \mathrm{Unif}(0,T)$.

Assumption 2. 

Let $f_Y:\mathbb{R}\to \mathbb{R}$ be a density such that the second derivative $f_Y^{\prime \prime }$ is continuous, square-integrable, and monotone over $(-\infty ,-M)\cup (M,\infty )$ for some $M>0$. This substantiates the use of Taylor series expansion in this section’s proofs—also present in Appendix A. We assume that kernel $k:\mathbb{R}\to \mathbb{R}$ is a bounded probability density function, symmetric about the origin, with a finite fourth moment and bounded, integrable first derivative. Since $k\left(x\right)=k(-x)$ for $x\in \mathbb{R}$, we also have

$$M_a^1\left(k\right)=\left\{\begin{array}{cc}1\hfill & a=1\hfill \\ 0\hfill & a\equiv 1(mod2)\hfill \\ <\infty \hfill & a\le 4\hfill \end{array}\right.$$

We also naturally restrict h, specifically the sequence $h_n$, such that ${lim}_{n\to \infty }h=0$.

Assumption 3. 

We restrict $h=\omega \left(n^{-1/3}\right)$, meaning ${lim}_{n\to \infty }{n}^{3}h=\infty$.

Under Assumptions 1, 2, and 3, for the multiplicative scrambling $Z=S·Y$, [11] proposed the following density estimator with asymptotic mean integrated squared error:

$$\begin{array}{cc}\hfill {\widehat{f}}_{Y_{\mathrm{M}}}\left(y\right)& =-y{T}^2{\widehat{f}}_Z^{\prime }\left(yT\right)\hfill \\ \hfill \mathrm{AMSE}\left\{{\widehat{f}}_{Y_{\mathrm{M}}}\left(y\right)\right\}& =y^2{T}^4{f}_Z\left(yT\right)M_0^2\left(k^{\prime }\right){\left(n{h}^3\right)}^{-1}+{\displaystyle \frac{y^2{T}^4{\left\{f_Z^{\prime \prime \prime }\left(yT\right)M_2^1\left(k\right)\right\}}^2{h}^4}{4}}\hfill \end{array}$$

(3)

$$\begin{array}{cc}\hfill \mathrm{AMISE}\left({\widehat{f}}_{Y_{\mathrm{M}}}\right)& =T^4{M}_0^2\left(k^{\prime }\right)M_2^1\left(f_Z^{\ast }\right){\left(n{h}^3\right)}^{-1}+{\displaystyle \frac{y^2{T}^4{M}_2^1{\left(k\right)}^2{M}_0^2\left(f_Z^{\circ }\right)h^4}{4}}\hfill \\ \hfill \mathrm{MISE}\left\{{\widehat{f}}_{Y_{\mathrm{M}}}\left(y\right)\right\}& =\mathrm{AMISE}\left\{{\widehat{f}}_{Y_{\mathrm{M}}}\left(y\right)\right\}+o\{{\left(n{h}^3\right)}^{-1}+h^4\}\hfill \end{array}$$

(4)

where $f_Z^{\ast }=f_Z\left(yT\right)$ and $f_Z^{\circ }\left(t\right)=y{f}_Z^{\prime \prime \prime }\left(yT\right)$. Minimizing (4) with respect to h, [11] derives an asymptotic optimal bandwidth.

$$\begin{array}{cc}\hfill h_{\mathrm{M}}^{\ast }& ={\left\{{\displaystyle \frac{3M_0^2\left(k^{\prime }\right)M_2^1\left(f_Z^{\ast }\right)}{M_2^1{\left(k\right)}^2{M}_0^2\left(f_Z^{\circ }\right)}}\right\}}^{1/7}{n}^{-1/7}\hfill \end{array}$$

For clarity, Assumption 3 restricts the convergence rate of h with respect to n to permit computation of an optimal kernel bandwidth via minimization of asymptotic mean squared errors and mean integrated square errors. An exact mean integrated squared error may be derived in the manner of Section 2.6 in Wand and Jones [9], but it follows from Wand and Jones [9] and Mostafa and Ahmad [12] that this truncation of mean squared error to asymptotic mean squared error via Taylor expansion incurs an error of $o\{{\left(n{h}^3\right)}^{-1}+h^4\}$. Thus, asymptotically,

$$h_{\mathrm{M}}^{\ast }=\Theta \left(n^{-1/7}\right),\mathrm{MISE}\left\{{\widehat{f}}_{Y_{\mathrm{M}}}\left(y\right)\right\}=\Theta \left(n^{-4/7}\right)+o\left(n^{-4/7}\right)=\Theta \left(n^{-4/7}\right)$$

(5)

Assumption 4. 

We restrict $h=\omega \left(n^{-1}\right)$, meaning ${lim}_{n\to \infty }nh=\infty$.

Assumption 5. 

Suppose $Y\ge 0$ such that, under Assumption 1, this also implies $Z\ge 0$.

Under Assumptions 1, 2, 4, and 5, Shou and Gupta [10] derived similar results for the additive scrambling $Z=Y+S$.

$$\begin{array}{cc}\hfill {\widehat{f}}_{Y_{\mathrm{A}}}\left(y\right)=& {\displaystyle \frac{1}{T}}{\int }_y^{y+T}{\widehat{f}}_Z\left(z\right)dz\hfill \\ \hfill \mathrm{AMSE}\left({\widehat{f}}_{Y_{\mathrm{A}}}\right)=& {\displaystyle \frac{h^4{M}_2^1{\left(k\right)}^2}{4T^2}}{\int }_y^{y+T}{f}_Z^{\prime \prime }{\left(z\right)}^{2}dz+{\displaystyle \frac{M_0^2\left(k\right)}{nh{T}^2}}{\int }_y^{y+T}q\left(z\right)dz\hfill \\ \hfill & +{\displaystyle \frac{1}{T^2}}{\left\{{\int }_y^{y+T}{f}_Z\left(z\right)dz\right\}}^2+f_Y^2\left(y\right)-{\displaystyle \frac{2}{T}}{\int }_y^{y+T}{f}_Z\left(z\right)f_Y\left(y\right)dz\hfill \\ \hfill \mathrm{AMISE}\left\{{\widehat{f}}_{Y_{\mathrm{A}}}\left(y\right)\right\}=& {\displaystyle \frac{h^4{M}_2^1{\left(k\right)}^2}{4T^2}}{\int }_{-\infty }^{\infty }{\int }_y^{y+T}{f}_Z^{\prime \prime }{\left(z\right)}^{2}dzdy\hfill \\ \hfill & +{\displaystyle \frac{M_0^2\left(k\right)}{nh{T}^2}}{\int }_{-\infty }^{\infty }{\int }_y^{y+T}q\left(z\right)dzdy\hfill \\ \hfill & +{\displaystyle \frac{1}{T^2}}{\int }_{-\infty }^{\infty }{\left\{{\int }_y^{y+T}{f}_Z\left(z\right)dz\right\}}^{2}dy+M_0^2\left(f_Y\right)\hfill \\ \hfill & -{\displaystyle \frac{2}{T}}{\int }_{-\infty }^{\infty }{\int }_y^{y+T}{f}_Z\left(z\right)f_Y\left(y\right)dzdy\hfill \\ \hfill h_{\mathrm{A}}^{\ast }=& {\left\{{\displaystyle \frac{M_0^2\left(k\right){\int }_{-\infty }^{\infty }{\int }_y^{y+T}{f}_Z\left(z\right)dzdy}{M_2^1{\left(k\right)}^2{\int }_{-\infty }^{\infty }{\int }_y^{y+T}{f}_Z^{\prime \prime }{\left(z\right)}^{2}dzdy}}\right\}}^{1/5}{n}^{-1/5}\hfill \end{array}$$

(6)

Similar to the multiplicative case, it follows from Wand and Jones [9] and Mostafa and Ahmad [12] that truncating mean squared error to asymptotic mean squared error via Taylor approximation incurs an error of $o\{{\left(nh\right)}^{-1}+h^4\}$. Thus, asymptotically,

$$h_{\mathrm{A}}^{\ast }=\Theta \left(n^{-1/5}\right),\mathrm{MISE}\left\{{\widehat{f}}_{Y_{\mathrm{A}}}\left(y\right)\right\}=\Theta \left(n^{-4/5}\right)+o\left(n^{-4/5}\right)=\Theta \left(n^{-4/5}\right)$$

(7)

In the randomized response domain, additive scrambling is typically more attractive than the multiplicative alternative, but we also see that the mean integrated squared error of the estimator from the former (7) is asymptotically dominant over the latter (5) as $\Theta \left(n^{-4/5}\right)=o\{\Theta \left(n^{-4/7}\right)\}$.

Á la joint scrambling, our estimator modifies the kernel density estimate in (1) to estimate $f_Y$.

Theorem 1. 

Under solely Assumptions 2 and 4, we give the following consistent density estimator.

$$\begin{array}{cc}\hfill {\widehat{f}}_{Y_{\mathrm{JS}}}\left(y\right)& ={\displaystyle \frac{1}{n}}\sum _{i=1}^n\sum _{j=1}^{r+1}{k}_h\left(y-Z_i^j\right)-r(k_h\ast f_S)\left(y\right)\hfill \end{array}$$

(8)

where * is the convolution operator given by $(k_h\ast f_S)\left(y\right)={\int }_{-\infty }^{\infty }{k}_h(y-t)f_S\left(t\right)t$. We also derive the asymptotic mean square error, asymptotic mean integrated squared error, asymptotic optimal bandwidth, and corresponding mean integrated squared error for (8).

$$\begin{array}{cc}\hfill \mathrm{AMSE}\left\{{\widehat{f}}_{Y_{\mathrm{JS}}}\left(y\right)\right\}& ={\left(nh\right)}^{-1}{M}_0^2\left(k\right)\{f_Y\left(y\right)+r{f}_S\left(y\right)\}+{\displaystyle \frac{1}{4}}{h}^4{f}_Y^{\prime \prime }{\left(y\right)}^2{M}_2^1{\left(k\right)}^2\hfill \\ \hfill \mathrm{AMISE}\left\{{\widehat{f}}_{Y_{\mathrm{JS}}}\left(y\right)\right\}& ={\left(nh\right)}^{-1}(1+r)M_0^2\left(k\right)+{\displaystyle \frac{1}{4}}{h}^4{M}_2^1{\left(k\right)}^2{M}_0^2\left(f_Y^{\prime \prime }\right)\hfill \\ \hfill h_{\mathrm{JS}}^{\ast }& ={\left\{{\displaystyle \frac{(r+1)M_0^2\left(k\right)}{M_2^1{\left(k\right)}^2{M}_0^2\left(f_Y^{\prime \prime }\right)}}\right\}}^{1/5}{n}^{-1/5}\hfill \\ \hfill & ={(r+1)}^{1/5}{h}_Z^{\ast }\hfill \end{array}$$

(9)

$$\begin{array}{cc}\hfill \mathrm{MISE}\left\{{\widehat{f}}_{Y_{\mathrm{JS}}}\left(y\right)\right\}& =\mathrm{AMISE}\left\{{\widehat{f}}_{Y_{\mathrm{JS}}}\left(y\right)\right\}+o\{{\left(nh\right)}^{-1}+h^4\}\hfill \end{array}$$

(10)

Thus, asymptotically,

$$h_{\mathrm{JS}}^{\ast }=\Theta \left(r^{1/5}{n}^{-1/5}\right),\mathrm{MISE}\left\{{\widehat{f}}_{Y_{\mathrm{JS}}}\left(y\right)\right\}=O\left(r^{4/5}{n}^{-4/5}\right)=\Theta \left(n^{-4/5}\right)$$

(11)

Remark 1. 

Since, jointly, the $Z_i^j$ are not i.i.d. as in typical kernel density estimation, adapting the estimator of (1) to the joint scrambling setting as shown in the first term of (8) yields a bias that does not vanish as $n\to \infty$. The second term of (8) is an additive correction which mitigates this bias and permits convergence as described by (11). Further, Theorem 1 does not restrict the distribution of S or Y. While Ahmad [11] does generalize Assumption 1 to distributions of the form $F_S\left(s\right)\propto s^{\beta }$ for $0\le s\le T$, the removal of Assumptions 1 and 5 provide the statistician with greater scrambling generality than the randomized response variants (6) and (3).

We see that the joint scrambling estimator in (8) is asymptotically equivalent in mean integrated squared error to the additive scrambling estimator in (6) and asymptotically dominates the multiplicative scrambling estimator in (3).

Simulation Study

We use simulated data to evaluate the claimed scaling (11) of the joint scrambling density estimator’s mean integrated square error.

The tight matching in the left plot of Figure 1 corroborates the claim that $\mathrm{MISE}\left({\widehat{f}}_{Y_{\mathrm{JS}}}\right)=\Theta \left(n^{-4/5}\right)$ for constant r. While the right plot of Figure 1 does not match as well, we observe that the simulated scaling with respect to r for constant n is much slower than the proven bound $\Theta \left(r^{4/5}\right)$ (11). This means that while the bound applies to all cases, in certain settings, the error terms in the derivation of (11) de-constructively interfere, leading to better practical scaling.

We also argue that the joint scrambling estimator is better suited for practical application given its computational advantages. Computing the kernel density estimate for a given z requires $O\left(n\right)$ computation to evaluate the summation in (1). The additive scrambling estimator in (6) integrates $f_{Z_{\mathrm{KDE}}}\left(z\right)$ for $z\in [y,y+T]$. Since the integral in (6) does not have a closed form, it must be numerically approximated. Advancements in numerical integration techniques such as the Runge–Kutta family have replaced the naive Euler’s method with higher-order approximation, typically of order four or five [13]. But increasing approximation precision requires repeated evaluation of the expensive integrand, inversely affecting time complexity. Of course, the multiplicative scrambling estimator in (3) avoids this integration at the cost of lower statistical efficiency. Since $k_h$ is a known function, the derivative $k_h^{\prime }$ can typically be computed exactly, or via numerically differentiation, meaning $f_{Z_{\mathrm{KDE}}}^{\prime }\left(y\right)$ takes roughly $O\left(n\right)$ time. While the convolution correction term in (8) is an integral, since both $k_h$ and $f_S$ are known functions, this term can be computed exactly or numerically integrated, which contributes negligibly to asymptotic time complexity as evaluation of the integrand is $O\left(1\right)$. Then, computing the modified kernel density estimate summation in (8) is simply $O\left(n\right)$. Using Gauss–Konrod quadrature [14] for numerical integration and finite differencing for numerical differentiation, we demonstrate these computational considerations via simulation in

Table 1. Efficiency and runtime comparison of density estimators. n, number of respondents; $h^{\ast }$, optimal bandwidth; T, runtime in seconds; IAB, integrated absolute bias; IV, integrated variance; MISE; mean integrated squared error. We fix $Y\sim \mathcal{N}(0,1)$, $S\sim \mathrm{Unif}(0,T)$ with $T=2$, and $r=2$. We use the kernel $k\left(x\right)=f_K\left(x\right)$ where $K\sim \mathcal{N}(0,1/4)$. All values in columns IAB, IV, and MISE are approximated via Simpson’s rule on a mesh of 100 evenly spaced points $\in [-3,3]$ and have been multiplied by ${10}^2$. All values are averaged over ${10}^2$ trials.

Method	n	${\mathit{h}}^{\ast }$	T	IAB	IV	MISE
Joint Scrambling	${10}^2$	$1.051$	$0.106$	$12.23$	$0.599$	$0.930$
Shou and Gupta [10]	${10}^2$	$0.987$	$0.238$	$28.91$	$0.152$	$2.040$
Ahmad [11]	${10}^2$	$0.366$	$0.024$	$25.03$	$9.553$	$14.29$
Joint Scrambling	${10}^3$	$0.663$	$0.439$	$5.476$	$0.165$	$0.232$
Shou and Gupta [10]	${10}^3$	$0.623$	$3.049$	$26.04$	$0.017$	$1.549$
Ahmad [11]	${10}^3$	$0.263$	$0.221$	$22.82$	$1.336$	$6.190$
Joint Scrambling	${10}^4$	$0.418$	$3.468$	$1.754$	$0.026$	$0.034$
Shou and Gupta [10]	${10}^4$	$0.393$	$64.82$	$24.84$	$0.002$	$1.400$
Ahmad [11]	${10}^4$	$0.190$	$2.226$	$22.82$	$0.176$	$5.259$

.

Comparing estimator runtime performance, T, across each method in Table 1, we see that Shou and Gupta [10] has the largest runtime, followed by joint scrambling and Ahmad [11]. This is natural as we expect numerical integration to be a computational bottleneck, particularly when the integrand has non-constant time complexity as in the additive scrambling case. On the other hand, numerical differentiation and summation for a known kernel are less intensive.

For the integrated variance, $\mathrm{IV}$, we see that Ahmad [11] yields the worst result, followed by joint scrambling and then Shou and Gupta [10]. However, the integrated absolute bias, $\mathrm{IAB}$, is lowest for joint scrambling followed by varying results for Shou and Gupta [10] and Ahmad [11]. In this manner, we see that the joint scrambling estimator compensates for higher integrated variance with low integrated absolute bias. This results in the lowest mean integrated square error, $\mathrm{MISE}$, for joint scrambling followed by Shou and Gupta [10] and Ahmad [11].

Despite maintaining the same asymptotic mean integrated square error as existing techniques, we conclude that the joint scrambling density estimator provides greater scrambling generality and computational efficiency in addition to superior mean integrated square error in practice.

3.2. Cumulative Distribution Estimation

Prior work on cumulative distribution estimators, ${\widehat{F}}_Y$, for randomized response models is limited to a single estimator in the additive case when $Y\ge 0$ [10]. Generalizing this estimator to all Y, under Assumptions 1, 2, and 4,

$${\widehat{F}}_{Y_{\mathrm{A}}}\left(y\right)={\int }_{-\infty }^{y+T}{\widehat{f}}_Z\left(w\right)dw-{\displaystyle \frac{1}{T}}{\int }_y^{y+T}(z-y){\widehat{f}}_Z\left(z\right)dz$$

(12)

While Shou and Gupta [10] omitted further analysis of (12), in Appendix A.2, we derive the asymptotic behavior of the mean integrated squared error using the optimal bandwidth for the traditional kernel density estimator in (1) via Wand and Jones [9].

$$\mathrm{MISE}\left({\widehat{F}}_{Y_{\mathrm{A}}}\right)=\mathsf{\Omega }\left(n^{-4/5}\right)$$

(13)

Since $F_S$ is known, we modify the empirical cumulative distribution function to estimate $F_Y$ á la joint scrambling.

Theorem 2. 

We give the following consistent, unbiased estimator for the cumulative distribution function $F_Y$.

$${\widehat{F}}_{Y_{\mathrm{JS}}}\left(y\right)={\displaystyle \frac{1}{n}}\sum _{i=1}^n\sum _{j=1}^{r+1}\mathbb{I}(Z_i^j\le y)-r{F}_S\left(y\right)$$

(14)

We also derive the exact mean squared error for (14).

$$\mathrm{MSE}\left\{{\widehat{F}}_{Y_{JS}}\left(y\right)\right\}=\left[F_Y\left(y\right)\{1-F_Y\left(y\right)\}+r{F}_S\left(y\right)\{1-F_y\left(s\right)\}\right]n^{-1}$$

(15)

Corollary 1. 

Maximizing (15),

$$\begin{array}{cc}\hfill \underset{y\in \mathrm{im}\left(Y\right)}{max}\mathrm{MSE}\left\{{\widehat{F}}_{Y_{\mathrm{JS}}}\left(y\right)\right\}& ={\displaystyle \frac{(r+1)}{4n}}=\Theta \left(r{n}^{-1}\right)=\Theta \left(n^{-1}\right)\hfill \\ \hfill \mathrm{MISE}\left({\widehat{F}}_{Y_{JS}}\right)& =\mathsf{\Omega }\left(n^{-1}\right)\hfill \end{array}$$

It follows from Corollary 1 that the mean squared error in (15) asymptotically dominates that of Shou and Gupta [10]. This also exhibits the aforementioned privacy–efficiency trade-off as increasing r increases privacy but also increases mean squared error linearly. Of course, since r is a fixed design parameter, it does not effect the asymptotic properties of the estimator’s mean squared error as $\Theta \left(r{n}^{-1}\right)=\Theta \left(n^{-1}\right)$.

Simulation Study

We give Figure 2 as a verification that the sample mean squared error from simulation agrees with the theoretical result in (15).

Beyond statistical efficiency, the proposed estimator in (14) is less computationally expensive than existing work. The former simply requires an $O\left(n\right)$ summation of the reported responses and yields an exact estimate. However, the numerical integration of the kernel density estimate in (14) poses computational challenges in a similar manner to the density estimator of Shou and Gupta [10]. We demonstrate this via simulation in

Table 2. A comparison of cumulative distribution function estimators. n, number of respondents; $h^{\ast }$, optimal bandwidth; T, runtime in seconds; IAB, integrated absolute bias; IV, integrated variance; MISE; mean integrated squared error. We fix $Y\sim \mathcal{N}(0,1)$ and $S\sim \mathrm{Unif}(0,T)$ with $T=2$ and $r=2$. We use the kernel $k\left(x\right)=f_K\left(x\right)$ where $K\sim \mathcal{N}(0,1/4)$. All values in columns IAB, IV, and MISE are approximated via Simpson’s rule on a mesh of 100 evenly spaced points $\in [-3,3]$ and have been multiplied by ${10}^2$. All values are averaged over ${10}^2$ trials.

Method	n	${\mathit{h}}^{\ast }$	T	IAB	IV	MISE
Joint Scrambling	${10}^2$	$0.418$	$0.003$	$0.017$	$1.133$	$1.150$
Shou and Gupta [10]	${10}^2$	$0.987$	$1.107$	$1.815$	$0.350$	$2.165$
Joint Scrambling	${10}^3$	$0.418$	$0.002$	$0.002$	$0.118$	$0.120$
Shou and Gupta [10]	${10}^3$	$0.623$	$10.74$	$1.373$	$0.035$	$1.409$
Joint Scrambling	${10}^4$	$0.418$	$0.004$	$0.000$	$0.013$	$0.013$
Shou and Gupta [10]	${10}^4$	$0.393$	$108.6$	$1.188$	$0.004$	$1.193$

.

We observe from Table 2 that the comparison of joint scrambling’s cumulative distribution estimator to Shou and Gupta [10] is analogous to that of the density estimator. The joint scrambling estimator has significantly lower runtime, T, and mean integrated square error, $\mathrm{MISE}$, where the latter is achieved by compensating a higher integrated variance, $\mathrm{IV}$, for a low integrated absolute bias, $\mathrm{IAB}$. This advantage in mean integrated square error concurs with the asymptotic comparison.

We conclude that the joint scrambling cumulative distribution function estimator is asymptotically more efficient than (12) both statistically and computationally.

4. Mean Estimation

Randomized response models generally regard a mean estimand, $\mu =E\left(Y\right)$. Analogous to the construction of (14) from the empirical cumulative distribution function, we construct a mean estimator empirically, using the sample mean and ${\mu }_S=E\left(S\right)$.

Theorem 3. 

We give the following consistent, unbiased estimator of ${\mu }_Y$.

$${\widehat{\mu }}_{Y_{\mathrm{JS}}}={\displaystyle \frac{1}{n}}\sum _{i=1}^n\sum _{j=1}^{r+1}{Z}_i^j-rE\left(S\right)$$

(16)

We also give the variance for (16).

$$\mathrm{var}\left({\widehat{\mu }}_{S_{\mathrm{JS}}}\right)=\{\mathrm{var}\left(Y\right)+r\mathrm{var}\left(S\right)\}{n}^{-1}=\Theta \left(r{n}^{-1}\right)=\Theta \left(n^{-1}\right)$$

Comparison with Additive Scrambling

Given reported responses $Z_1,\dots ,Z_n$, Warner [5] gives the following mean estimator and variance for the additive scrambling model $Z=Y+S$:

$$\begin{array}{cc}\hfill {\widehat{\mu }}_Y& ={\displaystyle \frac{1}{n}}\sum _{i=1}^n{Z}_i-{\mu }_S\hfill \\ \hfill \mathrm{var}\left({\widehat{\mu }}_Y\right)& =\{\mathrm{var}\left(Y\right)+\mathrm{var}\left(S\right)\}{n}^{-1}\hfill \end{array}$$

Typically, the scrambling is centered such that $E\left(S\right)=0$, making ${\widehat{\mu }}_Y$ equivalent to the sample mean. For Warner’s additive model, it is known that the privacy/efficiency trade-off is solely parameterized by the distribution $F_S$. For joint scrambling and the corresponding mean estimator (16), the privacy/efficiency trade-off is parameterized by both r and $F_S$ where estimator variance is linear in r and $\mathrm{var}\left(S\right)$. Interestingly, for $r=1$, the variance of the joint scrambling and Warner’s additive mean estimators are equivalent.

$$\mathrm{var}\left({\widehat{\mu }}_{Y_{\mathrm{JS}}}\right)=\mathrm{var}\left({\widehat{\mu }}_Y\right)=\{\mathrm{var}\left(Y\right)+\mathrm{var}\left(S\right)\}{n}^{-1}$$

5. A General Class of Estimators

We unify our cumulative distribution function (14) and mean (16) estimator under a general class of estimators.

Theorem 4. 

Let g be some function on random variables and consider estimand $\theta =E\left[g\right(Y\left)\right]$. Then, the following is a consistent, unbiased estimator of $g\left(Y\right)$ with the corresponding variance:

$$\begin{array}{cc}\hfill {\widehat{\theta }}_{JS}& ={\displaystyle \frac{1}{n}}\sum _{i=1}^n\sum _{j=1}^{r+1}g\left(Z_j^i\right)-rE\left[g\left(S\right)\right]\hfill \\ \hfill \mathrm{var}\left({\widehat{\theta }}_{JS}\right)& =\left[\mathrm{var}\left\{g\right(Y\left)\right\}+r\mathrm{var}\left\{g\right(S\left)\right\}\right]n^{-1}\hfill \end{array}$$

Remark 2. 

Applying Theorem 4 to $g\left(Y\right)=\mathbb{I}(Y\le y)$ gives ${\widehat{F}}_{Y_{\mathrm{JS}}}\left(y\right)$ in (14). Taking $g\left(Y\right)=Y$ gives (16).

While the density estimator is not in this general class but rather uses the kernel density estimator of (1), we see that the convolution correction term of (8) is the density analogue of the $rE\left\{g\right(S\left)\right\}$ correction term in Theorem 4. Both terms seek to eliminate estimator bias originating from S. Theorem 4 allows our results to extend to mixed distributions and a large class of estimands, $E\left\{g\right(Y\left)\right\}$.

6. Privacy

Of course, the extent of privacy protection from joint scrambling depends on r and the scrambling distribution $F_S$. While many privacy notions exist including differential privacy [15], which has seen substantial recent interest, we opt for comparison via standard privacy benchmarks in survey sampling and defer further discussion to Section 8. Particularly, we extend the traditional definition of privacy $\nabla =E\left\{{(Z-Y)}^2\right\}$ from Yan et al. [16] to the joint response case by replacing Z with $\tilde{Y}=E(Y\mid \overrightarrow{Z})$, the expected true response given the respondent’s reported joint response.

In the one-dimensional case of $r=0$, $\tilde{Y}=E(Y\mid \overrightarrow{Z})=E(Y\mid Z)=Z$ agrees with the original construction. For joint scrambling, the joint response is $\overrightarrow{Z}=(Z^1,\dots ,Z^{r+1})$, random variables corresponding to the i-th number spoken to the interviewer. Ideally, with perfect scrambling $S\sim Y$, the interviewer can do no better than choosing a random element of $\{Z_k^1,\dots ,Z_k^{r+1}\}$, which correctly predicts $Y_k=Y_k^1$ with probability $1/(r+1)$.

Proposition 1. 

Ideal scrambling occurs when $S\sim Y$ and yields the following properties:

$$\tilde{Y}=E(Y\mid \overrightarrow{Z})={\displaystyle \frac{1}{r+1}}\sum _{j=1}^{r+1}{Z}^j,\nabla \equiv E\left\{{(\tilde{Y}-Y)}^2\right\}=\left(1-{\displaystyle \frac{1}{r+1}}\right)\mathrm{var}\left(Y\right)$$

We see that $\varphi \equiv \nabla /\mathrm{var}\left(Y\right)=1-1/(r+1)$ is independent of the variance in the underlying sensitive variable, Y, and is simply the failure probability of the prediction. For this reason, we call $\varphi$ the normalized privacy. Unfortunately, a general calculation of $\varphi$ for $S\nsim Y$ is difficult.

While joint scrambling protects privacy better than randomized response with respect to $\tilde{Y}=E(Y\mid \overrightarrow{Z})$, the expected response given the reported responses, it accomplishes this by leaking less sensitive information regarding Y. With one reported response in randomized response models, beyond $E(Y\mid Z)$, no further information can be conveyed. However, for joint response, reporting $Z^1,\dots ,Z^{r+1}$ conveys information about Y, specifically that $Z^1\le Y\le Z^{r+1}$ and, more specifically, $Y\in \{Z^1,\dots ,Z^{r+1}\}$. When $r=0$, joint scrambling becomes direct survey sampling with no privacy protection. For $r>0$, provided Y and S have the same support, the interviewer can never almost surely know Y. Whether the information that $Y\in \{Z^1,\dots ,Z^{r+1}\}$ is sensitive or not depends on realized responses $\{z^1,\dots ,z^{r+1}\}$ and the nature of the question. Suppose researchers poll students under the age of 21 regarding the number of alcoholic drinks they consume weekly. If $min\left(\{Z^1,\dots ,Z^{r+1}\}\right)>0$, even if Y is unknown to the interviewer, the positivity of the individual’s reported responses incriminates the underage student and inhibits privacy. If $min\left(\{Z^1,\dots ,Z^{r+1}\}\right)=0$, the extent to which privacy is inhibited depends on the distributions $F_S$ and $F_Y$. However, for alternative sensitive questions, such as annual income, $\mathbb{I}(Y>0)$ may not necessarily be sensitive.

Ultimately, the researcher must select privacy metrics, such as those proposed in this work, that encapsulate the privacy needs of the particular statistical question. The usage of joint scrambling versus any sensitive variable sampling method hinges on this discretion to gauge the privacy protection of the method, according to this metric, against the statistical and computational efficiency of the respective estimators.

7. Real Data Example

Given our own proposition of joint scrambling as a sensitive sampling technique, real survey data collected via joint scrambling is not publicly available. Consequently, we consider a hypothetical example involving real world data.

The National Survey on Drug Use and Health (NSDUH) from the Substance Abuse and Mental Health Services Administration (SAMHSA) is an annual survey considered to be the leading source of population-based statistical data on alcohol, drug use, and related health information. Data is collected via face-to-face interviews in people’s homes, though web-based interviews are also offered as of 2020 due to COVID-related changes to survey methodology.

While joint scrambling with a discrete scrambling S can be applied for discrete quantitative random variables, we desire a continuous random variable such that density estimation is well-posed. In particular, we choose the Body Mass Index (BMI), a health metric computed from one’s height and weight to estimate body fat. The 2023 NSDUH survey denotes the corresponding column “BMI2” with range [9.3, 68.6] and computes this via the standard formula from a respondent’s height and weight.

Suppose that SAMHSA were to return the NSDUH to fully in-person interviews. Since personal health data particularly related to one’s body fat may be sensitive when asked face-to-face, suppose SAMHSA further decides to use joint scrambling.

Using the 2023 NSDUH survey data, we simulate the effect of such a decision on the joint scrambling density estimator for various choices of bandwidth h and scrambling count r. For a choice of $(r,h)$, for each response in the real survey, we concatenate r independent draws from scrambling distribution $S\sim \mathrm{Unif}\left(\right[0.3, 68.6\left]\right)$ to the true response, order the concatenated responses for each respondent, and consider this to be the hypothetical survey data when collected with joint scrambling. This scrambling distribution is natural as it is simply uniform over the BMI range.

From the left plot of Figure 3, we see that as the number of scramblings, r, increases, our density estimates worsen from the most solid to the most dotted orange line. One nuance revealed by this is that the joint scrambling density estimator may be negative if the kernel summation term is smaller than the convolution correction term. We also see that the best considered density estimate, that is, for $r=2$ with the optimal bandwidth $h=h_{\mathrm{JS}}^{\ast }$, closely matches the empirical distribution.

From the right plot of Figure 3, we observe how this best density estimate changes as h deviates from the optimal bandwidth in either direction. Comparing the dotted lines to the solid in the right plot, we observe that the estimate does not change much in this case. However, other settings may be more sensitive to h.

From (9), we see that the optimal bandwidth choice depends on $f_Y^{\prime \prime }$, which is unknown. This is a nuance of kernel density estimation more broadly, and recent work in standard kernel density estimation settings has yielded a variety of bandwidth selection methods [17] and adaptive bandwidth approaches [18]. However, for the sake of comparison, we compute $h_{\mathrm{JS}}^{\ast }$ by first taking the true responses and performing standard kernel density estimation with a standard Gaussian kernel to approximate $f_Y$. Since this kernel is smooth, we can also approximate $f_Y^{\prime \prime }$ by instead summing the second derivative of the kernel. This allows us to derive $h_{\mathrm{JS}}^{\ast }$ via (9).

We believe that the intuition from the literature on bandwidth selection can be applied to joint scrambling, and the specifics of bandwidth selection under scrambling are of interest in future work. Fortunately, the bandwidth choice does not affect the actual data collection, meaning that multiple bandwidths can be experimented with in practice.

8. Discussion

The proposed joint scrambling scheme can be interpreted as a particular form of local differential privacy [19]. Since density estimation in differentially private mechanisms is an area of active research [20], the kernel density estimator in (8) is significant. Moreover, the simplicity and estimand diversity of joint scrambling is particularly compelling in the survey sampling setting.

With recent work on randomized response, model complexity has increased to account for optionality [3], mixed randomization, untruthfulness [21], measurement error [22], combinations of the above, and more. Yet, the effectiveness of privacy protection is mediated by respondent understanding of their privacy guarantees, which cruxes on model simplicity and explainability. The proposed method in Section 6 proceeds with limited respondent overhead, and we argue that, in tandem with the mathematical advantages, joint scrambling is a strong privacy alternative to randomized response in practice because it is interpretable.

Given the asymptotically superior mean squared errors of our cumulative distribution estimator, we are keen on expanding beyond the estimator class $E\left\{g\right(Y\left)\right\}$ of Theorem 4 to quantile estimands and others to fully develop the theory of joint scrambling. We are also interested in bandwidth selection techniques for our density estimator and hope this work particularly encourages research in density estimation for sensitive surveys.