Bootstrapping Not Independent and Not Identically Distributed Data
Abstract
1. Introduction and Motivation
1.1. State of the Art
1.2. Structure of the Paper
2. Bootstrap Methods for NINID Data
2.1. Independent Bootstrap
Procedure 1 Independent bootstrap for the sample mean. |
Input: Data consisting of n IID vectors of observations . |
Output: Empirical bootstrap distribution of , i.e., the empirical distribution, where the probability mass concentrates at each of . |
|
2.2. Moving Block Bootstrap
Procedure 2 Moving block bootstrap for the sample mean. |
Input: Data consisting of n NINID vectors of observations and . |
Output: Empirical bootstrap distribution of sample mean , i.e., the empirical distribution, where the probability mass concentrates at each of . |
|
2.3. Blocksize
3. Types of Bootstrap Convergences
3.1. Properties of the Bootstrap Convergences
- (i)
- (ii)
- For each subsequence such that
- (iii)
- For each subsequence there exists a subsequence such that conditional on converges in distribution in probability to the distributional limit of as .
- (i)
- (ii)
- (iii)
- ;
- (iv)
- ;
- (v)
- ;
- (vi)
- , provided that and are invertible;
- (vii)
- , provided that and are invertible.
- (viii)
- (ix)
- (x)
- ;
- (xi)
- ;
- (xii)
- ;
- (xiii)
- , provided that and are invertible;
- (xiv)
- , provided that and are invertible.
3.2. Weak Dependence
- (i)
- ,
- (ii)
- .
4. Bootstrap Laws of Large Numbers
4.1. Bootstrap Weak LLN for Independent Data
4.2. Bootstrap Weak LLNs for NINID
5. Bootstrap Central Limit Theorems
5.1. Bootstrap CLT for Independent Data
- (i)
- ,
- (ii)
- is defined for all and non-decreasing on .
5.2. Bootstrap CLTs for NINID
6. Real Data Analyses
6.1. Psychometry
6.2. Insurance
7. Conclusions and Discussion
Author Contributions
Funding
Data Availability Statement
Acknowledgments
Conflicts of Interest
Abbreviations
IID | Independent and identically distributed |
NINID | Not independent and not identically distributed |
LLN | Law of large numbers |
SLLN | Strong law of large numbers |
WLLN | Weak law of large numbers |
BWLLN | Bootstrap weak law of large numbers |
CLT | Central limit theorem |
BCLT | Bootstrap central limit theorem |
Appendix A. Proofs
- (i)
- (ii)
- (iii)
References
- Efron, B. Bootstrap methods: Another look at the jackknife. Ann. Stat. 1979, 7, 1–26. [Google Scholar]
- Bickel, P.J.; Freedman, D.A. Some asymptotic theory for the bootstrap. Ann. Stat. 1981, 9, 1196–1217. [Google Scholar] [CrossRef]
- Efron, B.; Tibshirani, R. An Introduction to the Bootstrap; Chapman and Hall: London, UK, 1990. [Google Scholar]
- Efron, B.; Hastie, T. Computer Age Statistical Inference, Student ed.; Cambridge University Press: Cambridge, UK, 2013. [Google Scholar]
- Hall, P. The Bootstrap and Edgeworth Expansion; Springer: New York, NY, USA, 2013. [Google Scholar]
- Lahiri, S.N. Resampling Methods for Dependent Data; Springer: New York, NY, USA, 2003. [Google Scholar]
- Politis, D.N.; Romano, J.P.; Wolf, M. Subsampling; Springer: New York, NY, USA, 1999. [Google Scholar]
- Künsch, H.R. The jacknife and the bootstrap for general stationary observations. Ann. Stat. 1989, 17, 1217–1241. [Google Scholar] [CrossRef]
- Liu, R.Y.; Singh, K. Moving blocks jackknife and bootstrap capture weak dependence. In Proceedings of the Exploring the Limits of Bootstrap; Lapage, R., Billard, L., Eds.; Wiley: New York, NY, USA, 1992; pp. 225–248. [Google Scholar]
- Lahiri, S.N. Non-strong mixing autoregressive processes. Stat. Probabil. Lett. 1991, 11, 335–341. [Google Scholar]
- Politis, D.N.; Romano, J.P. A general resampling scheme for triangular arrays of α-mixing random variables with application to the problem of spectral density estimation. Ann. Stat. 1992, 20, 1985–2007. [Google Scholar] [CrossRef]
- Fitzenberger, B. The moving block bootstrap and robust inference for linear least squares and quantile regression. J. Econometr. 1998, 82, 235–287. [Google Scholar] [CrossRef]
- Maciak, M.; Peštová, B.; Pešta, M. Structural breaks in dependent, heteroscedastic, and extremal panel data. Kybernetika 2019, 54, 1106–1121. [Google Scholar] [CrossRef]
- Pesarin, F.; Salmaso, L. Permutation Tests for Complex Data: Theory, Applications and Software; Wiley: New York, NY, USA, 2010. [Google Scholar]
- Pešta, M. Total least squares and bootstrapping with application in calibration. Statistics 2013, 47, 966–991. [Google Scholar] [CrossRef]
- Pešta, M. Block bootstrap for dependent errors-in-variables. Commun. Stat. A-Theory 2017, 46, 1871–1897. [Google Scholar] [CrossRef]
- Hall, P.; Horowitz, J.L.; Jing, B.Y. On blocking rules for the bootstrap with dependent data. Biometrika 1995, 82, 561–574. [Google Scholar]
- Politis, D.N.; White, H. Automatic block-length selection for the dependent bootstrap. Econometr. Rev. 2004, 23, 53–70. [Google Scholar] [CrossRef]
- Lahiri, S.; Furukawa, K.; Lee, Y.D. A nonparametric plug-in rule for selecting optimal block lengths for block bootstrap methods. Stat. Methodol. 2007, 4, 292–321. [Google Scholar] [CrossRef]
- Kirch, C. Resampling Methods for the Change Analysis of Dependent Data. Ph.D. Thesis, University of Cologne, Cologne, Germany, 2006. [Google Scholar]
- Peštová, B.; Pešta, M. Abrupt change in mean using block bootstrap and avoiding variance estimation. Comput. Stat. 2018, 33, 413–441. [Google Scholar]
- Billingsley, P. Convergence of Probability Measures, 2nd ed.; John Wiley & Sons: New York, NY, USA, 1999. [Google Scholar]
- Bradley, R.C. Basic properties of strong mixing conditions. A survey and some open questions. Probab. Surv. 2005, 2, 107–144. [Google Scholar]
- Gijbels, I.; Omelka, M.; Pešta, M.; Veraverbeke, N. Score tests for covariate effects in conditional copulas. J. Multivar. Anal. 2017, 159, 111–133. [Google Scholar]
- Rosenblatt, M. A central limit theorem and a strong mixing condition. Proc. Natl. Acad. Sci. USA 1956, 42, 43–47. [Google Scholar] [CrossRef]
- Lin, Z.; Lu, C. Limit Theory for Mixing Dependent Random Variables; Springer: New York, NY, USA, 1997. [Google Scholar]
- Ibragimov, I.A. Some limit theorems for stochastic processes stationary in the strict sense. Dokl. Akad. Nauk. SSSR 1959, 125, 711–714. (In Russian) [Google Scholar]
- Pešta, M. Asymptotics for weakly dependent errors-in-variables. Kybernetika 2013, 49, 692–704. [Google Scholar]
- Anderson, T.W. An Introduction to Multivariate Statistical Analysis; John Wiley & Sons: New York, NY, USA, 1958. [Google Scholar]
- Billingsley, P. Convergence of Probability Measures, 1st ed.; John Wiley & Sons: New York, NY, USA, 1968. [Google Scholar]
- Doob, J.L. Stochastic Processes; John Wiley & Sons: New York, NY, USA, 1953. [Google Scholar]
- Ibragimov, I.A.; Linnik, Y.V. Independent and Stationary Sequences of Random Variables; Wolters-Noordhoff: Amsterdam, The Netherlands, 1971. [Google Scholar]
- Rosenblatt, M. Markov Processes: Structure and Asymptotic Behavior; Springer: Berlin/Heidelberg, Germany, 1971. [Google Scholar]
- Antoch, J.; Hušková, M.; Prášková, Z. Effect of dependence on statistics for determination of change. J. Stat. Plan. Infer. 1997, 60, 291–310. [Google Scholar] [CrossRef]
- Andrews, D.W.K. Heteroskedasticity and autocorrelation consistent covariance matrix estimation. Econometrica 1991, 59, 817–858. [Google Scholar] [CrossRef]
- Belyaev, Y.K. Bootstrap, Resampling and Mallows Metric. Lecture Notes 1; Institute of Mathematical Statistics, UmeåUniversity: Umeå, Sweden, 1995. [Google Scholar]
- Singh, K. On the asymptotic accuracy of Efron’s bootstrap. Ann. Stat. 1981, 9, 1187–1195. [Google Scholar] [CrossRef]
- Peštová, B.; Pešta, M. Testing structural changes in panel data with small fixed panel size and bootstrap. Metrika 2015, 78, 665–689. [Google Scholar]
- Peštová, B.; Pešta, M. Erratum to: Testing structural changes in panel data with small fixed panel size and bootstrap. Metrika 2016, 79, 237–238. [Google Scholar] [CrossRef]
- Peštová, B.; Pešta, M. Change point estimation in panel data without boundary issue. Risks 2017, 5, 7. [Google Scholar] [CrossRef]
- Culpepper, S.A.; Balamuta, J.J. Inferring latent structure in polytomous data with a higher-order diagnostic model. Multivar. Behav. Res. 2021, 1–19. [Google Scholar] [CrossRef]
- Maciak, M.; Mizera, I.; Pešta, M. Functional profile techniques for claims reserving. ASTIN Bull. 2022, 52, 449–482. [Google Scholar]
- Gerthofer, M.; Pešta, M. Stochastic claims reserving in insurance using random effects. Prague Econ. Pap. 2017, 26, 542–560. [Google Scholar] [CrossRef]
- McNeil, A.J. Estimating the tails of loss severity distributions using extreme value theory. ASTIN Bull. 1997, 27, 117–137. [Google Scholar] [CrossRef]
- Pešta, M. Changepoint in error-prone relations. Mathematics 2021, 9, 89. [Google Scholar] [CrossRef]
- Pešta, M.; Wendler, M. Nuisance-parameter-free changepoint detection in non-stationary series. Test 2020, 29, 379–408. [Google Scholar]
- Maciak, M.; Pešta, M.; Peštová, B. Changepoint in dependent and non-stationary panels. Stat. Pap. 2020, 61, 1385–1407. [Google Scholar] [CrossRef]
- Pešta, M.; Peštová, B.; Maciak, M. Changepoint estimation for dependent and non-stationary panels. Appl. Math.-Czech. 2020, 65, 299–310. [Google Scholar] [CrossRef]
- Pešta, M.; Hudecová, Š. Asymptotic consistency and inconsistency of the chain ladder. Insur. Math. Econ. 2012, 51, 472–479. [Google Scholar] [CrossRef]
- Hudecová, Š.; Pešta, M. Modeling dependencies in claims reserving with GEE. Insur. Math. Econ. 2013, 53, 786–794. [Google Scholar]
- Pešta, M.; Okhrin, O. Conditional least squares and copulae in claims reserving for a single line of business. Insur. Math. Econ. 2014, 56, 28–37. [Google Scholar] [CrossRef]
- Maciak, M.; Okhrin, O.; Pešta, M. Infinitely stochastic micro reserving. Insur. Math. Econ. 2021, 100, 30–58. [Google Scholar]
- Belyaev, Y.K.; Sjöstedt-de Luna, S. Weakly approaching sequences of random distributions. J. Appl. Probab. 2000, 37, 807–822. [Google Scholar] [CrossRef][Green Version]
- Zagdaǹski, A. On the construction and properties of bootstrap-t prediction intervals for stationary time series. Probab. Math. Statist. 2005, 25, 133–153. [Google Scholar]
- van der Vaart, A.W. Asymptotic Statistics; Cambridge University Press: New York, NY, USA, 1998. [Google Scholar]
- Chen, X.; Wu, Y. Strong law for mixing sequence. Acta Math. Appl. Sin. 1989, 5, 367–371. [Google Scholar]
- Xuejun, W.; Shuhe, H.; Yan, S.; Wenzhi, Y. Moment inequalities for φ-mixing sequences and its applications. J. Inequal. Appl. 2009, 2009, 12. [Google Scholar] [CrossRef][Green Version]
- Herrndorf, N. A functional central limit theorem for strongly mixing sequence of random variables. Probab. Theory Relat. Fields 1985, 69, 541–550. [Google Scholar] [CrossRef]
- Utev, S.A. The central limit theorem for φ-mixing arrays of random variables. Theory Probab. Appl. 1990, 35, 131–139. [Google Scholar] [CrossRef]
- Katz, M.L. Note on the Berry–Esseen theorem. Ann. Math. Stat. 1963, 34, 1107–1108. [Google Scholar]
Empirical Quantity | Independent Bootstrap | Moving Block Bootstrap |
---|---|---|
th percentile | 0.2647 | 0.3125 |
First quartile | 0.3824 | 0.4062 |
Median | 0.4412 | 0.4375 |
Third quartile | 0.5000 | 0.4999 |
97.5th percentile | 0.6176 | 0.5938 |
Empirical Quantity | Independent Bootstrap | Moving Block Bootstrap |
---|---|---|
2.5th percentile | 27.94 | 29.39 |
First quartile | 30.28 | 31.20 |
Median | 31.64 | 32.20 |
Third quartile | 33.08 | 33.29 |
97.5th percentile | 35.90 | 35.38 |
Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations. |
© 2022 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).
Share and Cite
Hrba, M.; Maciak, M.; Peštová, B.; Pešta, M. Bootstrapping Not Independent and Not Identically Distributed Data. Mathematics 2022, 10, 4671. https://doi.org/10.3390/math10244671
Hrba M, Maciak M, Peštová B, Pešta M. Bootstrapping Not Independent and Not Identically Distributed Data. Mathematics. 2022; 10(24):4671. https://doi.org/10.3390/math10244671
Chicago/Turabian StyleHrba, Martin, Matúš Maciak, Barbora Peštová, and Michal Pešta. 2022. "Bootstrapping Not Independent and Not Identically Distributed Data" Mathematics 10, no. 24: 4671. https://doi.org/10.3390/math10244671
APA StyleHrba, M., Maciak, M., Peštová, B., & Pešta, M. (2022). Bootstrapping Not Independent and Not Identically Distributed Data. Mathematics, 10(24), 4671. https://doi.org/10.3390/math10244671