Consistent Estimators of the Population Covariance Matrix and Its Reparameterizations
Abstract
:1. Introduction
2. Preliminary Notations
3. Optimal Estimators of When It Is Reparameterized
3.1. The Stein Phenomenon
3.2. The Optimal Properties of the MLE
3.3. The Best Orthogonally Equivariant Estimator
4. High-Dimensional Case
4.1. The Marenko–Pastur Equation
4.2. The Consistent Estimators of Population Eigenvalues
4.3. The Consistent Estimator of the Population Covariance Matrix
5. The Decomposite -Test When the Dimension Is Large
6. General Remarks When
6.1. When , Both n and p Are Fixed
6.2. When , Both n and p Are Large, So That
6.3. Use in High-Dimensional Low-Sample-Size (HDLSS) Categorical Data Models
Author Contributions
Funding
Data Availability Statement
Acknowledgments
Conflicts of Interest
References
- Pourahmadi, M. High-Dimensional Covariance Estimation; Wiley: New York, NY, USA, 2013. [Google Scholar]
- Zagidullina, A. High-Dimensional Covariance Matrix Estimation: An Introduction to Random Matrix Theory; SpringerBriefs in Applied Statistics and Econometrics; Springer: Cham, Switzerland, 2021. [Google Scholar]
- Stein, C. Estimation of a covariance matrix. In Proceedings of the Rietz lecture, 39th Annual Meeting IMS, Atalanta, GA, USA, 1975. [Google Scholar]
- Stein, C. Lectures on the theory of estimation of many parameters. J. Math. Sci. 1986, 43, 1373–1403. [Google Scholar] [CrossRef]
- Ledoit, O.; Wolf, M. Nonlinear shrinkage estimation of large-dimensional covariance matrices. Ann. Statist. 2012, 40, 1024–1060. [Google Scholar] [CrossRef]
- Ledoit, O.; Wolf, M. Optimal estimation of a large-dimensional covariance matrix under Stein’s loss. Bernoulli 2018, 24, 3791–3832. [Google Scholar] [CrossRef]
- Ledoit, O.; Wolf, M. Analytical nonlinear shrinkage of large-dimensional covariance matrices. Ann. Statist. 2020, 48, 3043–3065. [Google Scholar] [CrossRef]
- Ledoit, O.; Wolf, M. Shrinkage estimation of large covariance matrices: Keep it simple, statistician? J. Multivar. Anal. 2021, 186, 104796. [Google Scholar] [CrossRef]
- Rajaratnam, B.; Vincenzi, D. A theoretical study of Stein’s covariance estimator. Biometrika 2016, 103, 653–666. [Google Scholar] [CrossRef]
- Tsai, M.-T.; Tsai, C.-H. On the orthogonally equivariant estimators of a covariance matrix. arXiv 2024, arXiv:2405.06877. [Google Scholar]
- Silverstein, J.W. Strong convergence of the empirical distribution of eigenvalues of large dimensional random matrices. J. Multivar. Anal. 1995, 55, 331–339. [Google Scholar] [CrossRef]
- Anderson, T.W. An Introduction to Multivariate Statistical Analysis, 3rd ed.Weily: New York, NY, USA, 2003. [Google Scholar]
- Stein, C. Inadmissibility of the usual estimator of the mean of a multivariate normal distribution. Proc. Third Berkeley Symp. Math. Statist. Probab. 1956, 1, 197–206. [Google Scholar]
- James, W.; Stein, C. Estimation with quadratic loss. Proc. Fourth Berkeley Symp. Math. Statist. Probab. 1961, 1, 361–379. [Google Scholar]
- Tsai, M.-T. On the maximum likelihood estimator of a covariance matrix. Math. Method. Statist. 2018, 27, 71–82. [Google Scholar] [CrossRef]
- von Neumann, J. Some matrix-inequalities and metrization of matric-space. Tomsk. Univ. Rev. 1937, 1, 286–300. [Google Scholar]
- Ledoit, O.; Péché, S. Eigenvectors of some large sample covariance matrix ensembles. Probab. Theory Relat. Fields. 2011, 151, 233–264. [Google Scholar] [CrossRef]
- Marčenko, V.A.; Pastur, L.A. Distribution of eigenvalues for some sets of random matrices. Sb. Math. 1967, 1, 457–483. [Google Scholar]
- Choi, S.I.; Silverstein, J.W. Analysis of the limiting spectral distribution of large dimensional random matrices. J. Multivar. Anal. 1995, 54, 295–309. [Google Scholar]
- Bickle, p.J.; Levina, E. Regularized estimation of large covariance matrices. Ann. Statist. 2008, 36, 199–227. [Google Scholar] [CrossRef]
- Fan, J.; Fan, Y.; Lv, J. High dimensional covariance matrices using a factor model. J. Econom. 2008, 147, 186–197. [Google Scholar] [CrossRef]
- Bai, Z.D.; Miao, B.Q.; Pan, G.H. On asymptotics of eigenvectors of large sample covariance matrix. Ann. Probab. 2007, 35, 1532–1572. [Google Scholar] [CrossRef]
- Johnstone, I.M.; Paul, D. PCA in high dimensions: An orientation. Proc. IEEE 2018, 106, 1277–1292. [Google Scholar] [CrossRef]
- Johnstone, I.M.; Lu, A.Y. On consistency and sparsity for principal components analysis in high dimensions. J. Amer. Statist. Assoc. 2009, 104, 682–693. [Google Scholar] [CrossRef]
- Uhlig, H. On singular Wishart and singular multivariate Beta distributions. Ann. Statist. 1994, 22, 395–405. [Google Scholar] [CrossRef]
- Gini, C.W. Variabilita e Mutabilita, Studi Economico-Giuridici della R; Universita de Cagliary: Cagliari, Italy, 1912; Volume 2, pp. 3–159. [Google Scholar]
- Simpson, E.H. The measurement of diversity. Nature 1949, 163, 688. [Google Scholar] [CrossRef]
- Shannon, C.E. A mathematical theory of communication. Bell System Techni. J. 1948, 27, 379–423, 623–656. [Google Scholar] [CrossRef]
- Chakraborty, R.; Rao, C.R. Measurement of genetic variation for evolutionary studies. In Handbook of Statistics Vol. 8: Statistical Methods in Biological and Medical Sciences; Rao, C.R., Chakraborty, R., Eds.; Elsevier: Amsterdam, The Netherlands, 1991; pp. 271–316. [Google Scholar]
- Hoeffding, W. A class of statistics with asymptotically normal distribution. Ann. Math. Statist. 1948, 19, 293–325. [Google Scholar] [CrossRef]
- Sen, P.K. Some invariance principles relating to Jackknifing and their role in sequential analysis. Ann. Statist. 1977, 5, 315–329. [Google Scholar] [CrossRef]
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content. |
© 2025 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).
Share and Cite
Tsai, C.-H.; Tsai, M.-T. Consistent Estimators of the Population Covariance Matrix and Its Reparameterizations. Mathematics 2025, 13, 191. https://doi.org/10.3390/math13020191
Tsai C-H, Tsai M-T. Consistent Estimators of the Population Covariance Matrix and Its Reparameterizations. Mathematics. 2025; 13(2):191. https://doi.org/10.3390/math13020191
Chicago/Turabian StyleTsai, Chia-Hsuan, and Ming-Tien Tsai. 2025. "Consistent Estimators of the Population Covariance Matrix and Its Reparameterizations" Mathematics 13, no. 2: 191. https://doi.org/10.3390/math13020191
APA StyleTsai, C.-H., & Tsai, M.-T. (2025). Consistent Estimators of the Population Covariance Matrix and Its Reparameterizations. Mathematics, 13(2), 191. https://doi.org/10.3390/math13020191