Distributed Estimation for ℓ0-Constrained Quantile Regression Using Iterative Hard Thresholding
Abstract
:1. Introduction
Literature Review on Distributed Estimation and Our Contribution
How could distributed estimation methods with -constraints be designed to achieve convergence rates comparable to centralized estimators in sparse quantile regression models?
2. Background and Methodology
2.1. Quantile Regression with Constraint
2.2. Distributed Estimation
Algorithm 1 distributed estimation for quantile regression using IHT |
|
3. Main Results
- (A1)
- Define and (it can be shown that actually contains the second-order partially derivatives of and is thus referred to as the population Hessian matrix). We assume that is bounded from above by a constant and is bounded from below by a constant , for all -sparse unit vectors (that is, ).
- (A2)
- Components of are sub-Gaussian random variables in the sense that for any and some positive constants .
4. Simulations
4.1. Convergence Illustration
4.2. Variable Identification Performance
5. Conclusions
Author Contributions
Funding
Institutional Review Board Statement
Informed Consent Statement
Data Availability Statement
Conflicts of Interest
Appendix A. Proof of Theorem 1
References
- James, G.; Witten, D.; Hastie, T.; Tibshirani, R. An Introduction to Statistical Learning: With Applications in R; Springer: New York, NY, USA, 2013. [Google Scholar]
- Tibshirani, R. The lasso method for variable selection in the Cox model. Stat. Med. 1997, 16, 385–395. [Google Scholar] [CrossRef]
- Shen, X.; Pan, W.; Zhu, Y. Likelihood-based selection and sharp parameter estimation. J. Am. Stat. Assoc. 2012, 107, 223–232. [Google Scholar] [CrossRef] [PubMed]
- Jain, P.; Tewari, A.; Kar, P. On iterative hard thresholding methods for high-dimensional M-estimation. Adv. Neural Inf. Process. Syst. 2014, 27. [Google Scholar]
- Wang, Y.; Lu, W.; Lian, H. Best subset selection for high-dimensional non-smooth models using iterative hard thresholding. Inf. Sci. 2023, 625, 36–48. [Google Scholar] [CrossRef]
- Koenker, R.; Bassett, G., Jr. Regression quantiles. Econom. J. Econom. Soc. 1978, 1, 33–50. [Google Scholar] [CrossRef]
- Koenker, R. Quantile Regression; Econometric Society Monographs; Cambridge University Press: Cambridge, UK, 2005; p. xv. 349p. [Google Scholar]
- Portnoy, S.; Koenker, R. The Gaussian hare and the Laplacian tortoise: Computability of squared-error versus absolute-error estimators. Stat. Sci. 1997, 12, 279–300. [Google Scholar] [CrossRef]
- Koenker, R.; Hallock, K.F. Quantile regression. J. Econ. Perspect. 2001, 15, 143–156. [Google Scholar] [CrossRef]
- Koenker, R.; Geling, O. Reappraising medfly longevity: A quantile regression survival analysis. J. Am. Stat. Assoc. 2001, 96, 458–468. [Google Scholar] [CrossRef]
- Cade, B.S.; Noon, B.R. A gentle introduction to quantile regression for ecologists. Front. Ecol. Environ. 2003, 1, 412–420. [Google Scholar] [CrossRef]
- Rosenblatt, J.D.; Nadler, B. On the optimality of averaging in distributed statistical learning. Inf. Inference 2016, 5, 379–404. [Google Scholar] [CrossRef]
- Lin, S.; Guo, X.; Zhou, D. Distributed learning with regularized least squares. J. Mach. Learn. Res. 2017, 18, 1–31. [Google Scholar]
- Lian, H.; Fan, Z. Divide-and-conquer for debiased l1-norm support vector machine in ultra-high dimensions. J. Mach. Learn. Res. 2018, 18, 1–26. [Google Scholar]
- Huang, C.; Huo, X. A distributed one-step estimator. Math. Program. 2019, 174, 41–76. [Google Scholar] [CrossRef]
- Shamir, O.; Srebro, N.; Zhang, T. Communication-efficient distributed optimization using an approximate Newton-type method. In Proceedings of the 31st International Conference on Machine Learning, Beijing, China, 21–26 June 2014; Volume 32, pp. 1000–1008. [Google Scholar]
- Jordan, M.I.; Lee, J.D.; Yang, Y. Communication-efficient distributed statistical inference. J. Am. Stat. Assoc. 2018, 114, 668–681. [Google Scholar] [CrossRef]
- Zou, H.; Yuan, M. Composite quantile regression and the oracle model selection theory. Ann. Stat. 2008, 36, 1108–1126. [Google Scholar] [CrossRef]
- Wang, H.J.; Li, D. Estimation of extreme conditional quantiles through power transformation. J. Am. Stat. Assoc. 2013, 108, 1062–1074. [Google Scholar] [CrossRef]
- Shi, W.; Ling, Q.; Wu, G.; Yin, W. Extra: An exact first-order algorithm for decentralized consensus optimization. Siam J. Optim. 2015, 25, 944–966. [Google Scholar] [CrossRef]
- Ling, Q.; Shi, W.; Wu, G.; Ribeiro, A. DLM: Decentralized linearized alternating direction method of multipliers. IEEE Trans. Signal Process. 2015, 63, 4051–4064. [Google Scholar] [CrossRef]
- Ma, B.; Feng, Y.; Chen, G.; Li, C.; Xia, Y. Federated adaptive reweighting for medical image classification. Pattern Recognit. 2023, 144, 109880. [Google Scholar] [CrossRef]
N | d | m | * | |||||||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
PE | EE | PE | EE | PE | EE | PE | EE | |||||||
12,000 | 100 | 10 | ||||||||||||
100 | ||||||||||||||
1500 | 10 | |||||||||||||
100 |
N | s | m | * | |||||||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
MCR (%) | PE | EE | MCR (%) | PE | EE | MCR (%) | PE | EE | ||||||
5000 | 30 | 10 1 | ||||||||||||
20 2 | ||||||||||||||
10 | 10 3 | |||||||||||||
20 4 |
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content. |
© 2025 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).
Share and Cite
Zhao, Z.; Lian, H. Distributed Estimation for ℓ0-Constrained Quantile Regression Using Iterative Hard Thresholding. Mathematics 2025, 13, 669. https://doi.org/10.3390/math13040669
Zhao Z, Lian H. Distributed Estimation for ℓ0-Constrained Quantile Regression Using Iterative Hard Thresholding. Mathematics. 2025; 13(4):669. https://doi.org/10.3390/math13040669
Chicago/Turabian StyleZhao, Zhihe, and Heng Lian. 2025. "Distributed Estimation for ℓ0-Constrained Quantile Regression Using Iterative Hard Thresholding" Mathematics 13, no. 4: 669. https://doi.org/10.3390/math13040669
APA StyleZhao, Z., & Lian, H. (2025). Distributed Estimation for ℓ0-Constrained Quantile Regression Using Iterative Hard Thresholding. Mathematics, 13(4), 669. https://doi.org/10.3390/math13040669