Next Article in Journal
The Classical Origin of Spin: Vectors Versus Bivectors
Previous Article in Journal
On the Combination of the Laplace Transform and Integral Equation Method to Solve the 3D Parabolic Initial Boundary Value Problem
Previous Article in Special Issue
A Novel Poisson–Weibull Model for Stress–Strength Reliability Analysis in Industrial Systems: Bayesian and Classical Approaches
 
 
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

Privacy-Preserving Statistical Inference for Stochastic Frontier Analysis

1
College of Science, China University of Petroleum, Qingdao 266580, China
2
School of Economics and Management, China University of Petroleum, Qingdao 266580, China
*
Author to whom correspondence should be addressed.
Axioms 2025, 14(9), 667; https://doi.org/10.3390/axioms14090667
Submission received: 5 June 2025 / Revised: 20 August 2025 / Accepted: 27 August 2025 / Published: 29 August 2025

Abstract

We present the first differentially private framework for stochastic frontier analysis (SFA), addressing the challenge of non-convex objectives in privacy-preserving efficiency estimation. We construct a bounded parameter space to control gradient sensitivity and adapt the Frank–Wolfe algorithm with calibrated linear oracle noise to mitigate cumulative perturbation. Incorporating l 1 -regularization facilitates sparse and interpretable variable selection under strict ( ϵ , δ ) -differential privacy. Experiments demonstrate 15–35% MAE reduction under ϵ = 0.1 , along with strong scalability and estimation accuracy compared to prior DP methods for non-convex models.

1. Introduction

Stochastic frontier analysis (SFA) [1] is a widely used method for efficiency measurement that accounts for the difference between actual output and an efficient “frontier” by modeling technical inefficiency and random noise [2,3]. Traditionally, SFA employs parametric models [4,5] estimated via maximum likelihood or Bayesian methods [6]. Recent advances leverage machine learning to capture non-linear relationships, e.g., Kumbhakar [7] uses neural networks for flexible frontier estimation, Fan et al. [8] applies support vector regression to model production functions, and Tsionas [9] combines Bayesian deep learning with SFA for dynamic efficiency analysis.
However, modern SFA applications increasingly involve highly sensitive datasets—patient medical records, proprietary financial data, and confidential agricultural information—that pose severe privacy risks when conducting statistical inference or variable selection in high-dimensional settings [10]. This creates a fundamental challenge, i.e., how to perform accurate efficiency analysis while protecting individual privacy through rigorous mathematical processes.
Differential privacy (DP) [11] is the standard framework for rigorous privacy protection. While DP has been successfully applied to linear and logistic regression, as well as deep learning [12,13], its application to SFA is fundamentally challenging. The main difficulty lies in the non-convexity of the SFA log-likelihood and the complex sensitivity of its efficiency parameters, which violate the convexity assumptions required by most existing DP optimization methods.
For example, DP-stochastic gradient descent (DP-SGD) [13]—a widely adopted approach—adds Gaussian noise to clipped gradients at each iteration. While effective in convex or moderately non-convex problems, DP-SGD becomes unstable for SFA due to cumulative noise and gradient clipping interactions that undermine convergence. Other methods, such as output perturbation [14] or Frank–Wolfe-based mechanisms [15], introduce less noise but require substantial adaptation for non-convex objectives, which have not been developed for SFA.
Recent studies exploring DP alternatives, including one-step mechanisms [16,17] for convex empirical risk minimization and differentially private variance reduction methods [18], remain fundamentally limited by convexity assumptions that make them inapplicable to SFA. Non-gradient DP methods such as Bayesian DP [19] can theoretically handle non-linear efficiency parameters but are computationally prohibitive for large-scale applications. DP-LASSO methods for convex problems [20,21,22] introduce excessive noise when adapted to non-convex objectives, completely compromising accuracy and variable selection performance.
The Frank–Wolfe (FW) algorithm [23], originally developed for constrained convex optimization, offers theoretical promise due to its projection-free nature and ability to provide sparse solutions [24]. DP-FW methods enhance this framework by selectively adding calibrated noise to linear objectives [25,26,27]. Acharya et al. [28] introduce personalized DP for ridge regression, but their focus on l 2 -regularization does not address sparsity or non-convexity. However, these methods cannot be directly applied to SFA due to the non-convexity of its objective function; no existing work has analyzed the sensitivity properties of SFA’s efficiency parameters or developed appropriate noise calibration for the complex parameter interactions inherent in frontier models. Non-convex DP optimization methods, such as DP-projected gradient descent and DP-FW [29,30], provide frameworks for complex losses, but their application to SFA’s unique efficiency decomposition and sparse variable selection remains completely unaddressed.
The fundamental gap is that no existing method adequately addresses the intersection of non-convexity, efficiency parameter estimation, sparsity, and privacy, which defines the SFA privacy problem. Existing DP-SGD-based approaches fail due to instability and noise accumulation, while DP-Bayesian and convex DP-LASSO methods are either computationally intensive or fundamentally ill-suited for sparse variable selection under privacy constraints in non-convex problems. To address these critical limitations, we develop the first differentially private optimization framework specifically designed for stochastic frontier models. Our approach is a novel paradigm that combines a constrained Frank–Wolfe optimization framework with LASSO regularization to induce sparsity. This allows us to operate within a carefully constructed bounded convex parameter space that, for the first time, enables us to bound gradient sensitivity for SFA’s non-convex parameters. By adding calibrated Laplace noise at the linear oracle level, we fundamentally mitigate the noise accumulation that plagues gradient-based methods like DP-SGD. This provides rigorous ( ϵ , δ ) -differential privacy guarantees while preserving accuracy and interpretability in high dimensions—capabilities that existing methods cannot achieve.
The key innovations of this paper are as follows:
(1)
For non-convex stochastic frontier models, we establish a differentially private estimation framework by deriving bounded gradient sensitivity.
(2)
Based on this foundation, we design a constrained Frank–Wolfe optimization algorithm that enables efficient privacy-preserving estimation with calibrated noise.
(3)
For high-dimensional settings, we integrate l 1 -regularization into the framework to achieve sparse and interpretable variable selection under differential privacy.
The rest of the paper is organized as follows. Section 2 reviews the stochastic frontier model and l 1 -regularized variable selection, as well as introducing our bounded parameter space, gradient sensitivity bounds, noise calibration strategy, and a Frank–Wolfe algorithm with an ( ϵ , δ ) -privacy guarantee. Section 3 presents numerical results, including simulations and real-world validation on the California Housing and FADN datasets. Section 4 concludes the paper and outlines future research directions.

2. Methodology

This section develops our differentially private framework for SFA. It begins with a concise overview of the necessary background concepts before presenting our core methodological contributions—the design of a novel constrained parameter space, a formal sensitivity analysis, and a custom Frank–Wolfe algorithm with rigorous privacy guarantees.

2.1. Stochastic Frontier Model and Maximum Likelihood Estimation

The stochastic frontier model (SFM) assumes that the production process is influenced by both random noise and technical inefficiency. For N production units, the model can be expressed as follows:
Y = X β + ε , ε = ν u ,
where Y R N , X R N × p , and β R p are the production frontier parameters.
ν N ( 0 , σ ν 2 I N ) is the symmetric random error, capturing measurement errors and other random factors, where I N is the N-dimensional identity matrix. u N + ( 0 , σ u 2 I N ) is the one-sided error term, representing technical inefficiency, following a zero-truncated half-normal distribution.
We assume ν and u to be uncorrelated and independent of covariates X. The model’s parameter set is θ = { β , σ ν 2 , σ u 2 } . For estimation convenience, the total variance is defined as σ 2 = σ ν 2 + σ u 2 , while the proportion of inefficiency’s relative contribution is defined as λ 0 = σ u / σ ν .
Common approaches to estimating the stochastic frontier model include corrected least squares, the generalized method of moments, maximum likelihood estimation, and Bayesian methods. Among these, MLE stands out for its simplicity and efficiency. Below, we describe the MLE for the SFM, which forms the foundation for the variable selection techniques that are subsequently addressed.
The probability density function of the composite error term ϵ i = v i u i is obtained through convolution techniques. Given the distributional properties of v i and u i , the density of ϵ i incorporates the standard normal cumulative distribution function Ψ ( · ) . The log-likelihood function for the model is expressed as follows:
ln L ( λ , β , σ 2 | y , X ) = N 2 ln σ 2 + i = 1 N ln Ψ ϵ i λ σ 1 2 σ 2 i = 1 N ϵ i 2 ,
where ϵ i = y i x i β represents the residual for the i-th observation, and Ψ ( · ) denotes the cumulative distribution function of the standard normal distribution. MLE seeks to estimate the θ parameters by maximizing the log-likelihood function ln L ( λ , β , σ 2 | y , X ) .

2.2. Variable Selection via LASSO and Adaptive LASSO

In practical applications, when the number of covariates p is substantial compared to the sample size N (i.e., high-dimensional data scenarios), traditional MLE tends to cause overfitting and reduce model interpretability. Therefore, we introduce L1 regularization to identify the most explanatory features for output through variable selection, enhancing model robustness and practicality.
The penalized log-likelihood is as follows:
L 1 ( θ ; r ) = ln L ( λ 0 , β , σ 2 ) + r j = 1 p | β j | ,
where θ = { λ 0 , σ 2 , β } , and r controls the sparsity level. The estimator θ ˜ = arg min θ L 1 ( θ ; r ) is obtained via gradient-based optimization.
Using initial estimates β ˜ , adaptive weights are calculated as follows:
w ˜ j = 1 β ˜ j if β ˜ j 0 if β ˜ j = 0
(with the convention 0 · = 0 ).
The Adaptive Lasso Refinement is as follows:
L 2 ( θ ; r ) = ln L ( λ 0 , β , σ 2 ) + r j = 1 p w ˜ j | β j | .
This two-step approach enhances variable selection consistency while mitigating the overpenalization of significant coefficients.

2.3. Differential Privacy

We start by introducing the concept of differential privacy.
Definition 1
(Differential Privacy [11]). A randomized mechanism M : X Y is said to be (ϵ, δ)-differentially private for positive parameters ϵ, δ. If, for any pair of adjacent datasets X , X X differs by a single record, and for every measurable subset T Y under M ( · ) , the following holds:
P [ M ( X ) T ] e ϵ P [ M ( X ) T ] + δ ,
where the probability P arises solely from the randomness of the mechanism M.
In this definition, the datasets X and X are considered fixed, with the probability capturing the stochastic nature of M ( · ) . The privacy level against potential adversaries is determined by the parameters ϵ and δ . Smaller values of these parameters indicate stronger privacy protections.
A fundamental principle in differential privacy is sensitivity, which measures the maximum impact on an algorithm’s output resulting from the modification of a single record in the dataset.
Definition 2
(Sensitivity). For a deterministic, vector-valued function F ( · ) : X X R n , the l p sensitivity of F ( · ) is given as follows:
Δ p ( F ) : = sup X , X X F ( X ) F ( X ) p ,
where X and X differ by only one entry.
We proceed to describe the Laplace mechanism, which is a well-known technique that guarantees ( ϵ , δ )-differential privacy for functions exhibiting constrained l 1 sensitivity.
Lemma 1
(Laplace Mechanism [31]). For a function g : X R n with l 1 -sensitivity Δ 1 ( g ) , the mechanism M ( X ) = g ( X ) + Lap ( σ ) , where Lap ( σ ) Laplace ( 0 , σ ) , ensures ϵ-differential privacy, provided σ = Δ 1 ( g ) / ϵ .
Lemma 2
(Advanced Composition [32]). For a sequence of T mechanisms, each satisfying ϵ -differential privacy, their composition satisfies (ϵ, δ)-differential privacy, where the following is true:
ϵ = ϵ 2 T ln ( 1 / δ ) + T ϵ ( e ϵ 1 ) .

2.4. Constrained Parameter Space and Sensitivity Analysis

Our first methodological innovation is the formulation of a convex constrained parameter space for the SFM. This design plays a dual role—(i) it guarantees numerical stability during optimization by preventing extreme parameter realizations, and (ii) it ensures bounded gradient sensitivity, which is indispensable for the enforcement of differential privacy in a non-convex setting.
We define the optimization objective via the negative log-likelihood, as follows:
L ( θ ) = 1 N ln L ( λ 0 , β , σ 2 ) ,
where θ = { λ 0 , β , σ 2 } . By introducing parameter constraints, the feasible set is restricted to a convex region, thereby avoiding unstable or ill-posed solutions.
Definition 3
(Constrained Parameter Space). The feasible parameter set is defined as a convex polytope C R p + 2 , as follows:
C = ( β , σ 2 , λ 0 ) R p + 2 | β 1 B , σ 2 [ σ min 2 , σ max 2 ] , λ 0 [ 0 , λ max ] ,
where B ,   σ min 2 ,   σ max 2 ,   and   λ max > 0 are fixed constants.
These bounds simultaneously guarantee numerical stability and finite sensitivity. Unlike convex optimization problems, where the Lipschitz continuity of the gradient is sufficient for privacy guarantees, the non-convexity induced by the Φ ( · ) term in SFM complicates such analysis. Our approach circumvents this difficulty by focusing directly on bounding the sensitivity of the gradient.

Gradient Sensitivity Analysis

The sensitivity of the gradient is highly dependent on the bounds of λ 0 and σ 2 . From the derivations provided in Appendix A, the partial derivative with respect to λ 0 exhibits the following scaling:
O λ 0 σ min · M ( Z max ) ,
where Z max = ( 1 + B ) λ 0 σ min and M ( Z max ) = max 2 2 π , 2 Z max + 2 . This behavior implies that the gradient grows nearly linearly when λ 0 is small, but increases superlinearly as λ 0 becomes larger, primarily due to the Mills ratio.
Similarly, the partial derivative with respect to σ 2 satisfies the following:
O σ 2 + σ 4 ,
indicating that excessively small values of σ 2 cause gradient explosion. This effect not only destabilizes optimization but also amplifies the noise required by differentially private mechanisms.
Taken together, these analyses reveal a sharp trade-off, whereby smaller σ 2 and larger λ 0 improve model flexibility but substantially increase gradient sensitivity. Our convex constrained parameter design directly addresses this issue by imposing explicit bounds on both parameters. This constitutes the first methodological contribution of the paper, as it establishes a feasible optimization domain that ensures bounded sensitivity—a prerequisite for developing a differentially private stochastic frontier estimator.

2.5. Private SFM

This subsection presents our primary algorithmic contribution—a differentially private estimation framework for SFM, based on the Frank–Wolfe procedure within the previously defined constrained parameter space. The method leverages convex–analytic techniques to balance statistical efficiency with privacy guarantees.
The Frank–Wolfe algorithm [33] is a first-order method for constrained convex optimization. At each iteration, the algorithm linearizes the objective function around the current iterate and then updates the solution by moving toward the minimizer of the linearized problem over the convex domain. Formally, given the empirical loss function L defined in (9), the optimization problem can be written as follows:
min θ C L ( θ ) .
Due to the presence of the inefficiency term u i , the objective L ( θ ) is generally non-convex, as the cumulative distribution term Φ ( · ) introduces nonlinearities [2]. Nonetheless, the Frank–Wolfe procedure (Algorithm 1) remains a suitable vehicle for our private adaptation.
Algorithm 1 Frank–Wolfe Procedure
Input:  Feasible domain C R p + 2 , objective function L : C R , learning rate schedule { μ t }
  1: Initialize θ ( 1 ) C
  2: for  t = 1 to T 1  do
  3:     Solve linear subproblem: s ( t ) = arg min θ C L ( θ ( t ) ) , θ
  4:     Update iterate: θ ( t + 1 ) = ( 1 μ t ) θ ( t ) + μ t s ( t )
  5: end for
  6: return Final output θ ( T )
The convergence rate of Frank–Wolfe depends on the curvature of L . Intuitively, this curvature quantifies how much L deviates from its linear approximation over the domain C .
Definition 4
(Curvature Constant). The curvature constant of L over C is defined as follows:
Γ L = sup θ 1 , θ 2 C , γ ( 0 , 1 ) , θ 3 = θ 1 + γ ( θ 2 θ 1 ) 2 γ 2 L ( θ 3 ) L ( θ 1 ) L ( θ 1 ) , θ 3 θ 1 .
To safeguard individual-level data during SFM estimation, we embed differential privacy into the Frank–Wolfe framework by adapting the differentially private Frank–Wolfe (DP-FW) algorithm proposed by Bassily et al. [34]. In each iteration t, the gradient direction is selected using the exponential mechanism, with Laplace noise being calibrated to the l 1 sensitivity of the gradient. This ensures that private information about any single observation is obfuscated.
Although L ( θ ) is non-convex, DP-FW remains an effective heuristic, since privacy guarantees do not rely on convexity. Furthermore, the restricted parameter space C , being convex and polyhedral, enables efficient vertex-based updates.
The privacy budget depends critically on bounding the sensitivity of the gradient L ( θ ; D ) , where D = { ( x i , y i ) } i = 1 N denotes the dataset.
Theorem 1
(Gradient Sensitivity). The l 1 sensitivity of L ( θ ; D ) is as follows:
Δ = max θ C max D , D L ( θ ; D ) L ( θ ; D ) 1 2 G N ,
where G = sup θ C , d l ( θ ; d ) 1 < .
Proof. 
Let D and D be adjacent datasets differing in one record. Since the following is true:
L ( θ ; D ) = 1 N i = 1 N l ( θ ; d i ) ,
the difference between gradients under D and D reduces to the following:
L ( θ ; D ) L ( θ ; D ) = 1 N l ( θ ; d k ) l ( θ ; d k ) .
Thus,
L ( θ ; D ) L ( θ ; D ) 1 1 N l ( θ ; d k ) 1 + l ( θ ; d k ) 1 2 N sup θ C , d l ( θ ; d ) 1 .
With bounded covariates ( x i 1 ), bounded outputs ( | y i | 1 ), and constraints on C , a finite constant G exists such that sup θ , d l ( θ ; d ) 1 G . Hence, Δ 2 G / N .    □
Incorporating the sensitivity bound, we adapt the DP-FW algorithm to the SFM setting (Algorithm 2). At each iteration, noisy gradient scores are computed for all candidate vertices using the exponential mechanism, and the update is taken toward the vertex with the lowest perturbed score.
Algorithm 2 Differentially Private Frank–Wolfe for SFM.
Input:  Data D = { ( x i , y i ) } i = 1 N , loss L ( θ ; D ) , privacy budget ( ϵ , δ ) , feasible domain C = conv ( S ) , steps T
Output:  Private estimate θ priv
  1: Initialize θ ( 1 ) C
  2: for  t = 1 to T 1  do
  3:     for all  s S  do
  4:         Compute noisy score:
α s ( t ) s , L ( θ ( t ) ; D ) + Lap 2 G · C 1 · 8 T log ( 1 / δ ) N ϵ
  5:     Select vertex: s ( t ) = arg min s S α s ( t )
  6:     Update: θ ( t + 1 ) = ( 1 μ t ) θ ( t ) + μ t s ( t ) , with μ t = 2 t + 2
  7: return  θ priv = θ ( T )
Here, C 1 = max s S s 1 , and the noise calibration ensures ( ϵ , δ ) -differential privacy. This integration of Frank–Wolfe with differential privacy constitutes the second methodological contribution of this work.

2.6. Privacy Guarantee

This subsection establishes the formal privacy properties of our proposed algorithm, proving that it satisfies ( ϵ , δ ) -differential privacy. This constitutes a central theoretical contribution of the paper.
Theorem 2
(Laplace Noise Scale). In Algorithm 2, the required Laplace noise scale λ is given as follows:
λ = 2 G C 1 8 T log ( 1 / δ ) N ϵ ,
where C 1 = max s S s 1 .
Proof. 
Consider the score function α s = s , L ( θ ; D ) . Its sensitivity is as follows:
Δ α = max s S max D , D | s , L ( θ ; D ) L ( θ ; D ) | max s S s 1 · 2 G N = 2 G C 1 N .
According to the Laplace mechanism (Lemma 1), ensuring ϵ -differential privacy per iteration requires the following:
λ = Δ α ϵ = 2 G C 1 N ϵ .
To guarantee an overall ( ϵ , δ ) -differential privacy budget over T iterations, we invoke the advanced composition theorem (Lemma 2), setting the following:
ϵ = ϵ 8 T log ( 1 / δ ) .
Substituting yields the claimed result. □
We now establish the main theoretical guarantee for Algorithm 2.
Theorem 3.
Algorithm 2 satisfies ( ϵ , δ ) -differential privacy.
Proof. 
The proof builds on the analysis of the private Frank–Wolfe algorithm proposed by Talwar et al. [15]. At each iteration t, the algorithm selects a direction θ ˜ t using the exponential mechanism with the following score function:
α s = s , L ( θ t ; D ) + Lap ( λ ) ,
where λ = 2 G C 1 8 T log ( 1 / δ ) N ϵ .
We first verify per-iteration privacy. For adjacent datasets D and D , the sensitivity of the score function is as follows:
max D , D | s , L ( θ t ; D ) L ( θ t ; D ) | .
According to the Cauchy–Schwarz inequality in the l 1 -norm, the following is true:
| s , L ( θ t ; D ) L ( θ t ; D ) | s 1 · L ( θ t ; D ) L ( θ t ; D ) 1 .
From Theorem 1, the following holds true:
L ( θ t ; D ) L ( θ t ; D ) 1 2 G N ,
and since s C , we have s 1 C 1 . Thus, the overall sensitivity is bounded as follows:
max D , D | s , L ( θ t ; D ) L ( θ t ; D ) | 2 G C 1 N .
To ensure ϵ -differential privacy per iteration, the Laplace noise must satisfy the following:
λ sensitivity ϵ = 2 G C 1 N ϵ .
Setting ϵ = ϵ 8 T log ( 1 / δ ) gives the following:
λ 2 G C 1 8 T log ( 1 / δ ) N ϵ ,
matching the chosen λ in Algorithm 2.
Finally, applying the advanced composition theorem, the cumulative privacy loss after T iterations is as follows:
ϵ total = 2 T ln ( 1 / δ ) · ϵ + T · ϵ ( e ϵ 1 ) .
Substituting ϵ = ϵ 8 T log ( 1 / δ ) , we obtain the following:
2 T ln ( 1 / δ ) · ϵ = ϵ 2 .
For small ϵ , e ϵ 1 ϵ , implying the following:
T · ϵ ( e ϵ 1 ) ϵ 2 8 log ( 1 / δ ) .
Hence,
ϵ total ϵ 2 + ϵ 2 8 log ( 1 / δ ) ϵ ,
when δ is sufficiently small and ϵ is moderate. Therefore, Algorithm 2 satisfies ( ϵ , δ ) -differential privacy. Importantly, this guarantee holds irrespective of the potential non-convexity of L ( θ ) , as it relies solely on bounded sensitivity and the Laplace mechanism. □

3. Numerical Experiments

This section evaluates the proposed DP-FW framework through sensitivity analysis, simulation studies, and real-world data applications, contributing original empirical evidence to validate its effectiveness and robustness in SFA.

3.1. Gradient Sensitivity Analysis

To validate our theoretical gradient bounds and demonstrate the practical impact of parameter constraints, we conduct a comprehensive sensitivity analysis examining how λ 0 and σ 2 affect gradient magnitude and numerical stability.
Specifically, we fix B = 1 and p = 5 , while varying λ 0 [ 0.1 , 5.0 ] and σ 2 [ 0.01 , 2.0 ] . We first examine the influence curves of λ 0 and σ 2 on gradient components (Figure 1), and then provide a spatial heatmap visualization of sensitivity and stability across the parameter space (Figure 2; see Appendix Stability Metrics for stability score definitions).
Figure 1 illustrates that the gradient component associated with λ 0 grows exponentially when λ 0 > 0.45 , confirming the Mills ratio effect. For σ 2 , explosive sensitivity occurs near zero, with the gradient magnitude increasing by up to 4800 × when σ 2 < 0.1 . Overall, σ 2 dominates the gradient dynamics across most practical regimes.
The heatmaps in Figure 2 further highlight the interplay between λ 0 and σ 2 , visually confirming regions of instability. These results empirically validate our theoretical framework and provide practical guidance for parameter selection in differentially private implementations.

3.2. Simulation Study

We next investigate the statistical inference and variable selection capabilities of DP-FW under controlled simulation settings.
Our study addresses two fundamental questions—(1) How effectively does the proposed method estimate parameters across different sample sizes and privacy constraints? (2) Can the framework maintain variable selection accuracy in high-dimensional sparse settings while preserving privacy?
We benchmark DP-FW against maximum likelihood estimation (MLE), classical Frank–Wolfe (FW), stochastic gradient descent (SGD), and differentially private variants (DP-SGD and DP-Bayesian). The initial estimation focuses on DP-FW and DP-SGD to contrast Frank–Wolfe with gradient-based optimization, while DP-Bayesian is introduced later due to its computational cost and greater relevance in small-sample or high-dimensional settings. Evaluation metrics include Mean Squared Error (MSE), True Positive Rate (TPR), False Positive Rate (FPR), and Sum of Squared Errors (SSE).

3.2.1. Parameter Estimation

We adopt the following linear stochastic frontier model:
Y = X β + ϵ , ϵ = ν u ,
where ν N ( 0 , σ ν 2 I N ) represents symmetric noise, and u N + ( 0 , σ u 2 I N ) denotes non-negative inefficiency, both independent across N observations.
In low-dimensional settings, we fix s = 3 non-zero coefficients and p = 3 , with variance parameters σ ν 2 = 0.25 and σ u 2 = 0.09 , as well as true coefficients β = ( 1.0 , 2.0 , 3.0 ) . The sample size N is varied from 500 to 5000 in increments of 500, while privacy parameters are set to ϵ = 1 and δ = 10 5 . Hyperparameters are tuned via 5-fold cross-validation, with MSE as the criterion, using Optuna. Each experiment is repeated 100 times to mitigate randomness. For brevity, Table 1 reports representative results for N { 500 , 1500 , 3000 } .
Table 1 shows that while all methods accurately estimate the structural coefficients ( β ), MLE consistently achieves the lowest bias and variance, especially for the error parameters ( σ 2 and λ 0 ). Among the private methods, our proposed framework not only estimates β almost as well as the MLE baseline, but also decisively outperforms unconstrained private gradient methods in overall accuracy.
Sample Size and Privacy Effects
We further analyze sensitivity to data availability and privacy strength. Figure 3 reports MSE trends as N increases under ϵ = 1 . Figure 4 explores the privacy–utility trade-off by varying ϵ from 0.1 to 1 at fixed N = 5000 .
Figure 3 shows that increasing N consistently reduces MSE across all methods. While non-private estimators converge stably, private gradient-based methods exhibit volatility at moderate sample sizes. In contrast, DP-FW scales smoothly with N, confirming that bounded optimization effectively suppresses gradient sensitivity and facilitates the efficient use of additional data.
Figure 4 demonstrates the privacy–utility trade-off. As ϵ increases, all private methods benefit from reduced noise, but DP-FW shows a markedly smoother improvement curve. This stability arises from the controlled gradient domain, which ensures that injected perturbations scale predictably with privacy level. The result is better utility per unit of privacy budget—a critical advantage in practical applications.
Small-Sample Robustness
Small-sample scenarios are common in specialized efficiency analysis applications where data availability is inherently limited, posing a greater challenge for differentially private methods due to the amplified effect of injected noise.
To examine robustness, we conduct experiments with N { 50 , 100 , 150 } under ϵ = 1 , including DP-Bayesian methods for comparison. The results are summarized in Figure 5 and Table 2.
Table 2 and Figure 5 reveal several key insights. As anticipated, all methods show improved accuracy with increasing sample size. However, our proposed framework demonstrates superior robustness. Notably, at N = 100 , DP-FW achieves a good performance for structural parameters, which is comparable to the non-private SGD method.
This counterintuitive result—where private methods occasionally outperform non-private alternatives—arises from two complementary mechanisms. First, gradient descent methods exhibit a higher variance in small-sample settings, potentially leading to overfitting or unstable convergence paths. Second, bounded-domain optimization combined with calibrated perturbation provides implicit regularization effects that can improve generalization performance when data are limited.
Privacy–Utility Trade-Off
Building upon the previous analysis of sample size effects under a fixed privacy budget, we now examine how model performance changes with varying privacy parameters ϵ .
Figure 6 reports the MSE of differentially private estimators as the privacy budget ϵ varies. Unconstrained DP-SGD exhibits inflated MSE and instability under tight privacy constraints, reflecting the difficulty of controlling gradient sensitivity in high-dimensional settings. In contrast, both Bayesian approaches and DP-FW remain substantially more stable across privacy levels. Importantly, DP-FW consistently achieves a lower MSE than Bayesian alternatives, as well as displaying smoother improvements as ϵ increases.
Taken together, these results highlight how bounding the optimization domain translates into predictable sensitivity and improved variance control. Although differentially private estimators inevitably lose some efficiency relative to their non-private counterparts due to noise injection, DP-FW mitigates much of this loss by converting perturbation into a form of implicit regularization. The benefits are most pronounced in challenging regimes—small sample sizes, high-dimensional covariates, and stringent privacy budgets—where domain constraints and calibrated noise jointly stabilize estimation, narrow the gap to non-private baselines, and consistently outperform alternative private methods.

3.2.2. Variable Selection

This subsection evaluates the variable selection performance of our proposed DP-FW-Lasso in high-dimensional sparse settings, providing original evidence of its effectiveness under privacy constraints.
We consider a design with p = 500 predictors, of which only s = 6 coefficients are non-zero; these are randomly set to ± 1 . To promote sparsity, we incorporate LASSO regularization into both the non-private Frank–Wolfe estimator and its private counterpart (DP-FW-Lasso). This integration is natural, as Frank–Wolfe directly accommodates l 1 -type constraints. As a non-private benchmark, we include Adaptive LASSO (ALasso) with MLE, which is well known for its variable selection consistency. We deliberately exclude a private version of ALasso, as its reliance on accurate initial estimates and iterative reweighting amplifies the effects of privacy-induced noise, reducing stability and complicating privacy budget allocation. In contrast, standard LASSO is more amenable to privacy-preserving optimization, ensuring both computational tractability and stability.
Hyperparameters were tuned using the Bayesian Information Criterion (BIC) and 5-fold cross-validation. Performance was assessed using TPR, FPR, SSE, and MSE. Table 3 reports the results for N = 5000 and ϵ = 1 , with graphical comparisons provided in Figure 7.
Table 3 shows that all methods recover the true variables (TPR = 1.00) but differ in false positive control and estimation accuracy. Among non-private estimators, FW-LASSO produces the most parsimonious models. Under privacy constraints, DP-FW-Lasso attains the lowest FPR and estimation error, outperforming DP-SGD-Lasso and DP-Bayesian while avoiding their scalability and stability issues.
Figure 7 further shows that DP-FW-Lasso better preserves sparsity and interpretability than other DP methods. These findings confirm the design objectives of our framework—by leveraging a convex constraint space, controlling gradient sensitivity, and integrating LASSO regularization, DP-FW-Lasso achieves a robust and superior balance between privacy, sparsity, and estimation accuracy in high-dimensional stochastic frontier models.

3.3. Real Data Analysis

We evaluate our differentially private stochastic frontier inference framework using two complementary datasets. The California Housing dataset serves as a methodological validation platform to assess algorithm performance in high-dimensional sparse regression under privacy constraints. In contrast, the Farm Accountancy Data Network (FADN) dataset demonstrates a genuine application of SFA for measuring farm production efficiency.
The California Housing dataset (https://www.dcc.fc.up.pt/~ltorgo/Regression/cal_housing.html, accessed on 1 March 2025) comprises 20,640 observations with 8 features from the 1990 U.S. census. We expand these through nonlinear transformations, interactions, and spatial indicators to exceed 100 dimensions, creating a challenging setting for differentially private inference. The FADN dataset (http://ec.europa.eu/agriculture/rica/, accessed on 1 March 2025) tracks farm income and productivity across EU member states. Using a 2013 cross-section, we constructed a dataset with 51 features after handling missing values, outliers, and feature engineering. The model regresses total output on total input, agricultural land, and labor, with pronounced multicollinearity posing a rigorous test for inference methods.
We compare the following six estimators:
  • MLE-ALasso: Non-private maximum likelihood estimation with adaptive LASSO.
  • FW-Lasso: Non-private Frank–Wolfe estimator with l 1 constraint.
  • SGD-Lasso: Stochastic gradient descent with l 1 penalty—non-private.
  • DP-SGD-Lasso: Standard differentially private SGD with l 1 penalty.
  • DP-FW-Lasso: Proposed differentially private Frank–Wolfe estimator with l 1 constraint.
  • DP-Bayesian: Bayesian differentially private approach via posterior sampling.
All private methods ensure ( ϵ , δ ) -differential privacy with δ = 10 5 . Each model is evaluated over five independent trials to assess variance, and prediction accuracy is measured via MAE and RMSE.
On the California Housing dataset, we assess methods under ϵ { 0.1 , 0.5 , 1.0 } . Figure 8 shows that DP-FW-Lasso consistently outperforms other private methods, with its advantage growing under stricter privacy constraints where gradient-based approaches exhibit pronounced instability. Bayesian methods remain competitive at moderate privacy levels but are limited in high-dimensional scenarios due to scalability and computational demands. These results motivate the selection of ϵ = 0.1 for subsequent real-world evaluation.
Under the strictest privacy constraint ( ϵ = 0.1 ), we apply all methods to the FADN dataset. Figure 9 and Table 4 summarize the MAE and RMSE results. DP-FW-Lasso achieves a 35% lower MAE than DP-SGD-Lasso and a 12% improvement over DP-Bayesian. These gains reflect the superior handling of the privacy–utility trade-off, whereby Bayesian approaches are hindered by computational cost and limited scalability, while gradient descent suffers from optimization instability exacerbated by tight privacy constraints.
Privacy-preserving methods inherently incur higher errors than non-private baselines, as noise injection for ( ϵ , δ ) -differential privacy increases estimation uncertainty in non-convex SFM. Frank–Wolfe’s slightly higher errors relative to maximum likelihood reflect its focus on sparsity and variable selection over pure likelihood maximization, along with extra variance from mini-batch optimization.
Overall, across both datasets and privacy regimes, DP-FW-Lasso demonstrates consistent robustness, outperforming alternative private approaches in high-dimensional and structured econometric settings, while providing more scalable inference than computationally intensive Bayesian methods. These findings underscore the practical advantages of our framework for deploying differentially private stochastic frontier models in real-world applications.

4. Summary

This paper develops a novel framework for differentially private stochastic frontier analysis that enables reliable efficiency estimation under strict privacy protection. By bounding gradient sensitivity in non-convex frontier models and embedding these constraints into a Frank–Wolfe optimization scheme, we provide the first tractable approach for rigorous privacy-preserving efficiency analysis. The integration with l 1 regularization further allows consistent variable selection in high-dimensional settings.
Empirical studies confirm the advantages of the proposed framework. Compared with existing private methods, it achieves 15–35% less prediction errors, maintains robustness under small samples and strong privacy constraints, and perfectly recovers sparsity in high-dimensional cases. Application to agricultural data further demonstrates that the method preserves both accuracy and interpretability, making it suitable for practical economic efficiency evaluation under privacy restrictions.
While achieving substantial advances, our framework relies on convex relaxations and bounded parameter domains, which may limit applicability to highly flexible frontier specifications. The current cross-sectional focus also constrains application to dynamic efficiency tracking. Future research directions include extending the approach to panel data settings, developing adaptive noise calibration mechanisms for improved privacy–utility trade-offs, and exploring hybrid Bayesian–private inference for richer uncertainty quantification.

Author Contributions

Y.S.: study conception and design, development of methodology; M.Q.: data analysis, interpretation, and manuscript preparation and editing. X.W.: development of methodology. All authors have read and agreed to the published version of the manuscript.

Funding

This research is supported by the National Key Research and Development Program of China (2021YFA1000102), the Natural Science Foundation (NSF) project of Shandong Province of China (ZR2024MA074), the Ministry of education of Humanities and Social Science project (24YJA910003), and the Fundamental Research Funds for the Central Universities (No. 23CX03012A).

Data Availability Statement

This study has associated data that are deposited in data repositories.

Conflicts of Interest

The authors declare no conflicts of interest.

Appendix A. Derivation of Gradient Bound G

This appendix provides a detailed derivation of the gradient bound G, which is essential for the sensitivity analysis in Theorem 1. The derivation relies on the following constraints on the parameter space and data:
  • Bounded covariates: x i 1 ;
  • Bounded outputs: | y i | 1 ;
  • Constrained parameter space: β 1 B , λ 0 [ 0 , λ max ] , σ 2 [ σ min 2 , σ max 2 ] with σ min 2 > 0 .
The log-likelihood for a single observation is as follows:
l ( θ ; d i ) = 1 2 ln σ 2 + ln Φ ( y i x i β ) λ 0 σ ( y i x i β ) 2 2 σ 2 ,
where σ = σ 2 .
The gradient of the log-likelihood is as follows:
l β = λ 0 σ · ϕ ( z ) Φ ( z ) x i + ( y i x i β ) σ 2 x i , l λ 0 = ϕ ( z ) Φ ( z ) · ( y i x i β ) σ , l σ 2 = 1 2 σ 2 ϕ ( z ) Φ ( z ) · z 2 σ 2 + ( y i x i β ) 2 2 ( σ 2 ) 2 ,
where z = ( y i x i β ) λ 0 σ and ϕ ( z ) is the standard normal probability density function.
The l 1 -norm of the gradient is given as follows:
l ( θ ; d i ) 1 = l β 1 + l λ 0 + l σ 2 .

Appendix A.1. Bounding the Components

Let ε i = y i x i β be the residual term. Given the data constraints x i 1 and | y i | 1 , and the parameter constraint β 1 B , the residual is bounded as follows:
| ε i | = | y i x i β | | y i | + | x i β | 1 + x i β 1 1 + B .
Thus, the argument z is bounded by | z | = | ε i | λ 0 σ ( 1 + B ) λ max σ min : = Z max .
A crucial component is the inverse Mills ratio, ϕ ( z ) Φ ( z ) . We establish its boundedness via the following lemma.
Lemma A1
(Mills Ratio Bound). For z [ Z max , Z max ] , there exists a finite constant M > 0 such that the following is true:
ϕ ( z ) Φ ( z ) M .
Proof. 
We consider the following two cases:
  • Case 1:  z 0 . Since Φ ( z ) 1 2 and ϕ ( z ) 1 2 π , we have the following:
    ϕ ( z ) Φ ( z ) 1 / 2 π 1 / 2 = 2 2 π .
  • Case 2:  z < 0 . Using Mill’s inequality, for t > 0 , we have 1 Φ ( t ) ϕ ( t ) t + t 2 + 4 . For z = t with t > 0 , we have Φ ( z ) = 1 Φ ( t ) , so the following holds true:
    ϕ ( z ) Φ ( z ) = ϕ ( t ) 1 Φ ( t ) ϕ ( t ) ϕ ( t ) / ( t + t 2 + 4 ) = t + t 2 + 4 2 t + 2 = 2 | z | + 2 .
Since | z | Z max , we have ϕ ( z ) Φ ( z ) 2 Z max + 2 . Therefore, a finite upper bound for the inverse Mills ratio is given by M = max 2 2 π , 2 Z max + 2 . □

Appendix A.2. Final Bound

Using the bounds derived above, we can now bound each gradient component. Given x i 1 , we have x i 1 p .
  • Gradient with respect to β :
    l β 1 λ 0 σ · ϕ ( z ) Φ ( z ) x i 1 + | ε i | σ 2 x i 1 λ max σ min · M · p + 1 + B σ min 2 · p = p λ max M σ min + 1 + B σ min 2 .
  • Gradient with respect to λ 0 :
    l λ 0 ϕ ( z ) Φ ( z ) · | ε i | σ M · 1 + B σ min .
  • Gradient with respect to σ 2 :
    l σ 2 1 2 σ 2 + ϕ ( z ) Φ ( z ) · | z | 2 σ 2 + ε i 2 2 ( σ 2 ) 2 1 2 σ min 2 + M · Z max 2 σ min 2 + ( 1 + B ) 2 2 σ min 4 = 1 2 σ min 2 + M · ( 1 + B ) λ max 2 σ min 3 + ( 1 + B ) 2 2 σ min 4 .
Combining all components, the final upper bound G on the gradient’s l 1 -norm is as follows:
G = p λ max M σ min + 1 + B σ min 2 + M · 1 + B σ min + 1 2 σ min 2 + M · ( 1 + B ) λ max 2 σ min 3 + ( 1 + B ) 2 2 σ min 4 ,
where M = max 2 2 π , 2 Z max + 2 and Z max = ( 1 + B ) λ max σ min . Since all parameters are bounded within the constrained space, we have l ( θ ; d i ) 1 G < , ensuring a well-defined sensitivity bound for differential privacy.

Appendix A.3. Parameter Impact Analysis on Gradient Sensitivity

The gradient sensitivity is particularly influenced by the parameters λ 0 and σ 2 . A quantitative analysis reveals that λ 0 impacts sensitivity through two primary mechanisms—the inverse Mills ratio and a direct multiplicative effect. Numerical analysis demonstrates that as λ 0 increases from 0.5 to 2.0, the gradient bound G increases superlinearly. The dominant contribution shifts from the σ 2 component to the β component as λ 0 increases, with a critical threshold near λ 0 1.5 where numerical instability becomes prominent.
The parameter σ 2 exhibits a more dramatic influence, primarily due to an inverse power–law relationship and a cascade effect. The gradient bound contains terms that grow with the inverse fourth power of σ , and small values of σ 2 amplify both the Mills ratio and direct terms. The interaction between λ 0 and σ 2 creates a multiplicative amplification effect.

Appendix A.4. Numerical Stability Analysis

Stability Metrics

To quantify numerical stability, we define a score based on the parameters’ proximity to regions of high sensitivity.
Definition A1
(Numerical Stability Score). The stability score is defined as follows:
S ( λ 0 , σ min ) = 1 1 + Z max · min 1 , σ min 2 ( 1 + B ) 2 .

References

  1. Kumbhakar, S.C.; Lovell, C.K. Stochastic Frontier Analysis; Cambridge University Press: Cambridge, UK, 2003. [Google Scholar]
  2. Aigner, D.; Lovell, C.K.; Schmidt, P. Formulation and estimation of stochastic frontier production function models. J. Econom. 1977, 6, 21–37. [Google Scholar] [CrossRef]
  3. Meeusen, W.; van Den Broeck, J. Efficiency estimation from Cobb-Douglas production functions with composed error. Int. Econ. Rev. 1977, 18, 435–444. [Google Scholar] [CrossRef]
  4. Goldberger, A.S. The interpretation and estimation of Cobb-Douglas functions. Econom. J. Econom. Soc. 1968, 36, 464–472. [Google Scholar] [CrossRef]
  5. Kim, H.Y. The translog production function and variable returns to scale. Rev. Econ. Stat. 1992, 74, 546–552. [Google Scholar] [CrossRef]
  6. Kumbhakar, S.C. Estimation of profit functions when profit is not maximum. Am. J. Agric. Econ. 2001, 83, 1–19. [Google Scholar] [CrossRef]
  7. Kumbhakar, S.C. Efficiency measurement with multiple outputs and multiple inputs. J. Product. Anal. 1996, 7, 225–255. [Google Scholar] [CrossRef]
  8. Fan, G.F.; Peng, L.L.; Hong, W.C.; Sun, F. Electric load forecasting by the SVR model with differential empirical mode decomposition and auto regression. Neurocomputing 2016, 173, 958–970. [Google Scholar] [CrossRef]
  9. Tsionas, M.G. A coherent approach to Bayesian data envelopment analysis. Eur. J. Oper. Res. 2020, 281, 439–448. [Google Scholar] [CrossRef]
  10. Tibshirani, R. Regression shrinkage and selection via the lasso. J. R. Stat. Soc. Ser. B Stat. Methodol. 1996, 58, 267–288. [Google Scholar] [CrossRef]
  11. Dwork, C.; McSherry, F.; Nissim, K.; Smith, A. Calibrating noise to sensitivity in private data analysis. In Proceedings of the Theory of Cryptography: Third Theory of Cryptography Conference, TCC 2006, New York, NY, USA, 4–7 March 2006; Proceedings 3. Springer: Berlin/Heidelberg, Germany, 2006; pp. 265–284. [Google Scholar]
  12. Chaudhuri, K.; Monteleoni, C.; Sarwate, A.D. Differentially private empirical risk minimization. J. Mach. Learn. Res. 2011, 12, 1069–1109. [Google Scholar] [PubMed]
  13. Abadi, M.; Chu, A.; Goodfellow, I.; McMahan, H.B.; Mironov, I.; Talwar, K.; Zhang, L. Deep learning with differential privacy. In Proceedings of the 2016 ACM SIGSAC Conference on Computer and Communications Security, Vienna, Austria, 24–28 October 2016; pp. 308–318. [Google Scholar]
  14. Chaudhuri, K.; Sarwate, A.; Sinha, K. Near-optimal differentially private principal components. In Proceedings of the Neural Information Processing Systems Conference, Lake Tahoe, NV, USA, 3–6 December 2012. [Google Scholar]
  15. Talwar, K.; Guha Thakurta, A.; Zhang, L. Nearly optimal private lasso. In Proceedings of the NIPS’15: Neural Information Processing Systems, Montreal, QC, Canada, 7–12 December 2015. [Google Scholar]
  16. Bassily, R.; Smith, A. Local, private, efficient protocols for succinct histograms. In Proceedings of the Forty-Seventh Annual ACM Symposium on Theory of Computing, Portland, OR, USA, 14–17 June 2015; pp. 127–135. [Google Scholar]
  17. Dwork, C.; Feldman, V. Privacy-preserving prediction. In Proceedings of the Conference on Learning Theory, PMLR, Stockholm, Sweden, 6–9 July 2018; pp. 1693–1702. [Google Scholar]
  18. Zhang, Z.; Hu, R. Byzantine-robust federated learning with variance reduction and differential privacy. In Proceedings of the 2023 IEEE Conference on Communications and Network Security (CNS), Orlando, FL, USA, 2–5 October 2023; pp. 1–9. [Google Scholar]
  19. Triastcyn, A.; Faltings, B. Bayesian differential privacy for machine learning. In Proceedings of the International Conference on Machine Learning, PMLR, Virtual Event, 13–18 July 2020; pp. 9583–9592. [Google Scholar]
  20. Smith, A. Efficient, differentially private point estimators. arXiv 2008, arXiv:0809.4794. [Google Scholar] [CrossRef]
  21. Bassily, R.; Feldman, V.; Talwar, K.; Guha Thakurta, A. Private stochastic convex optimization with optimal rates. In Proceedings of the 33rd International Conference on Neural Information Processing Systems, Vancouver, BC, Canada, 8–14 December 2019. [Google Scholar]
  22. Wang, Y.X. Revisiting differentially private linear regression: Optimal and adaptive prediction & estimation in unbounded domain. arXiv 2018, arXiv:1803.02596. [Google Scholar] [CrossRef]
  23. Jaggi, M. Revisiting Frank-Wolfe: Projection-free sparse convex optimization. In Proceedings of the International Conference on Machine Learning, PMLR, Atlanta, GA, USA, 16–21 June 2013; pp. 427–435. [Google Scholar]
  24. Jain, P.; Thakurta, A. Differentially private learning with kernels. In Proceedings of the International Conference on Machine Learning, PMLR, Atlanta, GA, USA, 16–21 June 2013; pp. 118–126. [Google Scholar]
  25. Talwar, K.; Thakurta, A.; Zhang, L. Private empirical risk minimization beyond the worst case: The effect of the constraint set geometry. arXiv 2014, arXiv:1411.5417. [Google Scholar]
  26. Raff, E.; Khanna, A.; Lu, F. Scaling up differentially private lasso regularized logistic regression via faster frank-wolfe iterations. Adv. Neural Inf. Process. Syst. 2023, 36, 36349–36363. [Google Scholar]
  27. Khanna, A.; Lu, F.; Raff, E.; Testa, B. Differentially private logistic regression with sparse solutions. In Proceedings of the 16th ACM Workshop on Artificial Intelligence and Security, Copenhagen, Denmark, 30 November 2023; pp. 1–9. [Google Scholar]
  28. Acharya, K.; Boenisch, F.; Naidu, R.; Ziani, J. Personalized differential privacy for ridge regression. arXiv 2024, arXiv:2401.17127. [Google Scholar] [CrossRef]
  29. Wang, D.; Xu, J. Differentially private empirical risk minimization with smooth non-convex loss functions: A non-stationary view. In Proceedings of the AAAI Conference on Artificial Intelligence, Honolulu, HI, USA, 27 January–1 February 2019; Volume 33, pp. 1182–1189. [Google Scholar]
  30. Cai, Z.; Li, S.; Xia, X.; Zhang, L. Private estimation and inference in high-dimensional regression with fdr control. arXiv 2023, arXiv:2310.16260. [Google Scholar] [CrossRef]
  31. Dwork, C.; Roth, A. The algorithmic foundations of differential privacy. Found. Trends® Theor. Comput. Sci. 2014, 9, 211–407. [Google Scholar] [CrossRef]
  32. Dwork, C.; Rothblum, G.N.; Vadhan, S. Boosting and differential privacy. In Proceedings of the 2010 IEEE 51st Annual Symposium on Foundations of Computer Science, Las Vegas, NV, USA, 23–26 October 2010; pp. 51–60. [Google Scholar]
  33. Frank, M.; Wolfe, P. An algorithm for quadratic programming. Nav. Res. Logist. Q. 1956, 3, 95–110. [Google Scholar] [CrossRef]
  34. Bassily, R.; Smith, A.; Thakurta, A. Private empirical risk minimization: Efficient algorithms and tight error bounds. In Proceedings of the 2014 IEEE 55th Annual Symposium on Foundations of Computer Science, Philadelphia, PA, USA, 18–21 October 2014; pp. 464–473. [Google Scholar]
Figure 1. Gradient-bound sensitivity with respect to λ 0 and σ 2 . (a) Overall gradient bound vs. parameter values. (b) Gradient components with respect to β , λ 0 , and σ 2 .
Figure 1. Gradient-bound sensitivity with respect to λ 0 and σ 2 . (a) Overall gradient bound vs. parameter values. (b) Gradient components with respect to β , λ 0 , and σ 2 .
Axioms 14 00667 g001
Figure 2. Parameter space visualization of gradient sensitivity and numerical stability. (a) Gradient-bound surface. (b) Stability score surface across ( λ 0 , σ 2 ) .
Figure 2. Parameter space visualization of gradient sensitivity and numerical stability. (a) Gradient-bound surface. (b) Stability score surface across ( λ 0 , σ 2 ) .
Axioms 14 00667 g002
Figure 3. Impact of data size on MSE at ϵ = 1 . (a) All methods, including private and non-private ones. (b) Stochastic gradient-based methods: SGD and DP-SGD. (c) Stable methods.
Figure 3. Impact of data size on MSE at ϵ = 1 . (a) All methods, including private and non-private ones. (b) Stochastic gradient-based methods: SGD and DP-SGD. (c) Stable methods.
Axioms 14 00667 g003
Figure 4. Privacy–utility trade-off with varying ϵ at N = 5000 . (a) All methods, including private and non-private ones. (b) Stochastic gradient-based methods: SGD and DP-SGD. (c) Stable methods.
Figure 4. Privacy–utility trade-off with varying ϵ at N = 5000 . (a) All methods, including private and non-private ones. (b) Stochastic gradient-based methods: SGD and DP-SGD. (c) Stable methods.
Axioms 14 00667 g004
Figure 5. Comparative performance of different methods in small-sample regimes for ϵ = 1 .
Figure 5. Comparative performance of different methods in small-sample regimes for ϵ = 1 .
Axioms 14 00667 g005
Figure 6. Privacy–utility trade-off with varying ϵ at N = 3000 . Comparison among DP methods.
Figure 6. Privacy–utility trade-off with varying ϵ at N = 3000 . Comparison among DP methods.
Axioms 14 00667 g006
Figure 7. Variable selection performance.
Figure 7. Variable selection performance.
Axioms 14 00667 g007
Figure 8. Prediction MAE under different privacy levels ( ϵ ) on the California Housing dataset.
Figure 8. Prediction MAE under different privacy levels ( ϵ ) on the California Housing dataset.
Axioms 14 00667 g008
Figure 9. Prediction MAE under ϵ = 0.1 on the FADN dataset across all methods.
Figure 9. Prediction MAE under ϵ = 0.1 on the FADN dataset across all methods.
Axioms 14 00667 g009
Table 1. Comparison of parameter estimates and standard deviations across methods ( ϵ = 1 ).
Table 1. Comparison of parameter estimates and standard deviations across methods ( ϵ = 1 ).
NParam. (True)Method
MLE FW DP-FW SGD DP-SGD
Est SD Est SD Est SD Est SD Est SD
500 β 1 (1.0)0.9800.0570.9850.0610.9850.0601.0060.0630.9480.072
β 2 (2.0)2.0110.0832.0890.0912.0640.0871.9100.0922.0390.105
β 3 (3.0)3.0160.1043.1480.1133.1130.1083.1200.1152.9800.130
σ 2 (0.34)0.3340.0330.3810.0380.4490.0410.3300.0340.4600.045
λ 0 (0.6)0.4250.0980.7320.1050.6810.1020.4300.1000.7500.110
MSE0.3260.4090.3990.4600.570
1500 β 1 (1.0)1.0780.0321.0120.0351.0250.0340.9160.0351.0150.041
β 2 (2.0)2.0130.0472.0440.0512.0430.0502.0480.0521.9880.060
β 3 (3.0)2.9970.0642.9680.0672.9630.0662.9750.0702.9920.079
σ 2 (0.34)0.3510.0200.4110.0210.4200.0230.3480.0200.4300.025
λ 0 (0.6)0.7460.0631.0020.0721.0690.0690.7400.0651.0800.075
MSE0.3470.3480.3500.3100.348
3000 β 1 (1.0)0.9700.0221.0490.0241.0160.0241.0120.0241.0670.028
β 2 (2.0)1.9930.0331.9590.0351.9650.0351.9920.0362.0150.042
β 3 (3.0)3.0000.0422.9330.0452.9440.0432.9910.0462.9940.052
σ 2 (0.34)0.3260.0130.4030.0150.3670.0140.3270.0140.3800.016
λ 0 (0.6)0.5760.0420.9150.0480.8190.0460.5800.0420.8500.050
MSE0.3190.3440.3380.3680.418
Table 2. Parameter estimates and standard deviations in small-sample regimes ( ϵ = 1 ).
Table 2. Parameter estimates and standard deviations in small-sample regimes ( ϵ = 1 ).
NParam. (True)Method
MLE FW SGD DP-FW DP-SGD DP-Bayesian
Est SD Est SD Est SD Est SD Est SD Est SD
50 β 1 (1.0)0.9520.1851.0890.2031.1240.2181.0780.1960.8470.2451.0150.208
β 2 (2.0)2.0340.2682.1560.2952.2450.3182.1420.2851.7850.3522.0890.301
β 3 (3.0)3.0680.3423.2340.3783.1850.3953.1560.3682.8920.4453.2010.389
σ 2 (0.34)0.3480.1080.4250.1250.3980.1320.4650.1180.5120.1480.4780.128
λ 0 (0.6)0.3850.2850.8920.3250.4250.3180.7350.2950.9650.3850.8250.342
MSE0.7280.7810.8420.9161.1000.950
100 β 1 (1.0)0.9760.1281.0450.1421.0890.1561.0320.1380.9150.1751.0680.148
β 2 (2.0)2.0180.1852.0980.2052.1340.2252.0760.1981.8920.2482.1250.215
β 3 (3.0)3.0420.2383.1560.2653.0980.2853.0890.2582.9450.3183.1420.275
σ 2 (0.34)0.3420.0750.3980.0880.3850.0950.4250.0850.4850.1080.4450.092
λ 0 (0.6)0.4250.1980.7850.2250.4580.2180.6950.2050.8250.2680.7580.238
MSE0.4820.5310.6610.6120.7620.695
150 β 1 (1.0)0.9850.1051.0280.1181.0560.1281.0150.1150.9480.1421.0420.122
β 2 (2.0)2.0120.1522.0680.1682.0850.1852.0450.1651.9250.2052.0780.175
β 3 (3.0)3.0250.1953.0980.2183.0650.2353.0580.2122.9850.2583.0890.225
σ 2 (0.34)0.3380.0620.3850.0720.3750.0780.4050.0700.4550.0880.4250.075
λ 0 (0.6)0.4850.1620.7250.1850.5120.1780.6680.1680.7850.2150.7150.192
MSE0.3780.4210.4350.4720.6280.512
Table 3. Variable selection performance metrics.
Table 3. Variable selection performance metrics.
MetricMLE-ALassoFW-LassoDP-FW-LassoSGD-LassoDP-SGD-LassoDP-Bayesian
TPR1.001.001.001.001.001.00
FPR0.220.170.52 *0.490.730.56
SSE0.020.210.47 *0.340.510.50
MSE0.300.360.49 *0.330.540.52
Note: within DP methods, values marked with * indicate the best performance.
Table 4. Comparison of MAE and RMSE (Mean and Std) across different methods.
Table 4. Comparison of MAE and RMSE (Mean and Std) across different methods.
MethodMAERMSE
Mean Std Mean Std
MLE-ALasso0.5000.0010.6880.001
FW-Lasso0.5390.0130.6950.010
SGD-Lasso0.5460.0190.7080.05
DP-SGD-Lasso0.9760.0181.3170.015
DP-Bayesian0.7200.001 *0.8820.001 *
DP-FW-Lasso0.634 *0.0150.728 *0.013
Note: Within DP methods, values marked with * indicate the best performance.
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content.

Share and Cite

MDPI and ACS Style

Quan, M.; Song, Y.; Wang, X. Privacy-Preserving Statistical Inference for Stochastic Frontier Analysis. Axioms 2025, 14, 667. https://doi.org/10.3390/axioms14090667

AMA Style

Quan M, Song Y, Wang X. Privacy-Preserving Statistical Inference for Stochastic Frontier Analysis. Axioms. 2025; 14(9):667. https://doi.org/10.3390/axioms14090667

Chicago/Turabian Style

Quan, Mengxiang, Yunquan Song, and Xinmin Wang. 2025. "Privacy-Preserving Statistical Inference for Stochastic Frontier Analysis" Axioms 14, no. 9: 667. https://doi.org/10.3390/axioms14090667

APA Style

Quan, M., Song, Y., & Wang, X. (2025). Privacy-Preserving Statistical Inference for Stochastic Frontier Analysis. Axioms, 14(9), 667. https://doi.org/10.3390/axioms14090667

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop