Analytical Approximation of the Jackknife Linking Error in Item Response Models Utilizing a Taylor Expansion of the Log-Likelihood Function
Abstract
:1. Introduction
2. Analytical Approximation of the Jackknife Linking Error
2.1. Use of the Approximation in Scaling
2.2. Implementation Details for Computing Derivatives of the Log-Likelihood Function
2.3. Bias Correction due to Sampling Error
2.4. Jackknife Linking Error Based on Testlets
3. Simulation Study
3.1. Method
3.2. Results
On the Bias in the Estimated Standard Deviation
4. Discussion
Funding
Institutional Review Board Statement
Informed Consent Statement
Data Availability Statement
Conflicts of Interest
Abbreviations
1PL | one-parameter logistic |
2PL | two-parameter logistic |
DIF | differential item functioning |
IRT | item response theory |
JK | jackknife |
LE | linking error |
PIRLS | progress in international reading literacy study |
PISA | programme for international student assessment |
References
- Bock, R.D.; Moustaki, I. Item response theory in a general framework. In Handbook of Statistics, Volume 26: Psychometrics; Rao, C.R., Sinharay, S., Eds.; Elsevier: Amsterdam, The Netherlands, 2007; pp. 469–513. [Google Scholar] [CrossRef]
- van der Linden, W.J.; Hambleton, R.K. (Eds.) Handbook of Modern Item Response Theory; Springer: New York, NY, USA, 1997. [Google Scholar] [CrossRef]
- van der Linden, W.J. Unidimensional logistic response models. In Handbook of Item Response Theory, Volume 1: Models; van der Linden, W.J., Ed.; CRC Press: Boca Raton, FL, USA, 2016; pp. 11–30. [Google Scholar] [CrossRef]
- Rutkowski, L.; von Davier, M.; Rutkowski, D. (Eds.) A Handbook of International Large-Scale Assessment: Background, Technical Issues, and Methods of Data Analysis; Chapman Hall/CRC Press: London, UK, 2013. [Google Scholar] [CrossRef]
- OECD. PISA 2018. Technical Report; OECD: Paris, France, 2020. [Google Scholar]
- Foy, P.; Yin, L. Scaling the PIRLS 2016 achievement data. In Methods and Procedures in PIRLS 2016; Martin, M.O., Mullis, I.V., Hooper, M., Eds.; IEA: Boston College: Chestnut Hill, MA, USA, 2017. [Google Scholar]
- Yen, W.M.; Fitzpatrick, A.R. Item response theory. In Educational Measurement; Brennan, R.L., Ed.; Praeger Publishers: Westport, CT, USA, 2006; pp. 111–154. [Google Scholar]
- Rasch, G. Probabilistic Models for Some Intelligence and Attainment Tests; Danish Institute for Educational Research: Copenhagen, Denmark, 1960. [Google Scholar]
- Birnbaum, A. Some latent trait models and their use in inferring an examinee’s ability. In Statistical Theories of Mental Test Scores; Lord, F.M., Novick, M.R., Eds.; MIT Press: Reading, MA, USA, 1968; pp. 397–479. [Google Scholar]
- Aitkin, M. Expectation maximization algorithm and extensions. In Handbook of Item Response Theory, Volume 2: Statistical Tools; van der Linden, W.J., Ed.; CRC Press: Boca Raton, FL, USA, 2016; pp. 217–236. [Google Scholar] [CrossRef]
- Bock, R.D.; Aitkin, M. Marginal maximum likelihood estimation of item parameters: Application of an EM algorithm. Psychometrika 1981, 46, 443–459. [Google Scholar] [CrossRef]
- Penfield, R.D.; Camilli, G. Differential item functioning and item bias. In Handbook of Statistics, Volume 26: Psychometrics; Rao, C.R., Sinharay, S., Eds.; Elsevier: Amsterdam, The Netherlands, 2007; pp. 125–167. [Google Scholar] [CrossRef]
- Joo, S.; Ali, U.; Robin, F.; Shin, H.J. Impact of differential item functioning on group score reporting in the context of large-scale assessments. Large-Scale Assess. Educ. 2022, 10, 18. [Google Scholar] [CrossRef]
- Robitzsch, A.; Lüdtke, O. A review of different scaling approaches under full invariance, partial invariance, and noninvariance for cross-sectional country comparisons in large-scale assessments. Psychol. Test Assess. Model. 2020, 62, 233–279. [Google Scholar]
- Battauz, M. Multiple equating of separate IRT calibrations. Psychometrika 2017, 82, 610–636. [Google Scholar] [CrossRef]
- Monseur, C.; Berezner, A. The computation of equating errors in international surveys in education. J. Appl. Meas. 2007, 8, 323–335. [Google Scholar]
- OECD. PISA 2012. Technical Report; OECD: Paris, France, 2014. [Google Scholar]
- Robitzsch, A.; Lüdtke, O. Linking errors in international large-scale assessments: Calculation of standard errors for trend estimation. Assess. Educ. 2019, 26, 444–465. [Google Scholar] [CrossRef]
- Robitzsch, A. Robust and nonrobust linking of two groups for the Rasch model with balanced and unbalanced random DIF: A comparative simulation study and the simultaneous assessment of standard errors and linking errors with resampling techniques. Symmetry 2021, 13, 2198. [Google Scholar] [CrossRef]
- Wu, M. Measurement, sampling, and equating errors in large-scale assessments. Educ. Meas. 2010, 29, 15–27. [Google Scholar] [CrossRef]
- Efron, B.; Tibshirani, R.J. An Introduction to the Bootstrap; CRC Press: Boca Raton, FL, USA, 1994. [Google Scholar] [CrossRef]
- Kolenikov, S. Resampling variance estimation for complex survey data. Stata J. 2010, 10, 165–199. [Google Scholar] [CrossRef] [Green Version]
- Yuan, K.H.; Cheng, Y.; Patton, J. Information matrices and standard errors for MLEs of item parameters in IRT. Psychometrika 2014, 79, 232–254. [Google Scholar] [CrossRef]
- Chakraborty, S. Generating discrete analogues of continuous probability distributions—A survey of methods and constructions. J. Stat. Distrib. Appl. 2015, 2, 6. [Google Scholar] [CrossRef] [Green Version]
- Sireci, S.G.; Thissen, D.; Wainer, H. On the reliability of testlet-based tests. J. Educ. Meas. 1991, 28, 237–247. [Google Scholar] [CrossRef]
- Wainer, H.; Bradlow, E.T.; Wang, X. Testlet Response Theory and Its Applications; Cambridge University Press: Cambridge, UK, 2007. [Google Scholar] [CrossRef]
- Monseur, C.; Sibberns, H.; Hastedt, D. Linking errors in trend estimation for international surveys in education. IERI Monogr. Ser. 2008, 1, 113–122. [Google Scholar]
- Caflisch, R.E. Monte Carlo and quasi-Monte Carlo methods. Acta Numer. 1998, 7, 1–49. [Google Scholar] [CrossRef] [Green Version]
- Robitzsch, A. About the equivalence of the latent D-scoring model and the two-parameter logistic item response model. Mathematics 2021, 9, 1465. [Google Scholar] [CrossRef]
- R Core Team. R: A Language and Environment for Statistical Computing; R Core Team: Vienna, Austria, 2022; Available online: https://www.R-project.org/ (accessed on 11 January 2022).
- Robitzsch, A.; Kiefer, T.; Wu, M. TAM: Test Analysis Modules. 2022. R Package Version 4.1-4. Available online: https://CRAN.R-project.org/package=TAM (accessed on 28 August 2022).
- Robitzsch, A. A comparison of linking methods for two groups for the two-parameter logistic item response model in the presence and absence of random differential item functioning. Foundations 2021, 1, 116–144. [Google Scholar] [CrossRef]
- Robitzsch, A.; Lüdtke, O. Mean comparisons of many groups in the presence of DIF: An evaluation of linking and concurrent scaling approaches. J. Educ. Behav. Stat. 2022, 47, 36–68. [Google Scholar] [CrossRef]
- Muthén, B. A general structural equation model with dichotomous, ordered categorical, and continuous latent variable indicators. Psychometrika 1984, 49, 115–132. [Google Scholar] [CrossRef] [Green Version]
- Ip, E.H. Empirically indistinguishable multidimensional IRT and locally dependent unidimensional item response models. Br. J. Math. Stat. Psychol. 2010, 63, 395–416. [Google Scholar] [CrossRef]
- Giordano, R.; Stephenson, W.; Liu, R.; Jordan, M.I.; Broderick, T. A higher-order swiss army infinitesimal jackknife. arXiv 2019, arXiv:1806.00550v5. [Google Scholar] [CrossRef]
- Jaeckel, L.A. The Infinitesimal Jackknife; Bell Telephone Laboratories: Washington, WA, USA, 1972. [Google Scholar]
- Jennrich, R.I. Nonparametric estimation of standard errors in covariance analysis using the infinitesimal jackknife. Psychometrika 2008, 73, 579–594. [Google Scholar] [CrossRef]
- Kolen, M.J.; Brennan, R.L. Test Equating, Scaling, and Linking; Springer: New York, NY, USA, 2014. [Google Scholar] [CrossRef]
- González, J.; Wiberg, M. Applying Test Equating Methods. Using R; Springer: New York, NY, USA, 2017. [Google Scholar] [CrossRef]
- Andersson, B. Asymptotic variance of linking coefficient estimators for polytomous IRT models. Appl. Psychol. Meas. 2018, 42, 192–205. [Google Scholar] [CrossRef] [PubMed]
- Battauz, M. Factors affecting the variability of IRT equating coefficients. Stat. Neerl. 2015, 69, 85–101. [Google Scholar] [CrossRef]
- Ogasawara, H. Standard errors of item response theory equating/linking by response function methods. Appl. Psychol. Meas. 2001, 25, 53–67. [Google Scholar] [CrossRef]
- Brennan, R.L. Generalizabilty Theory; Springer: New York, NY, USA, 2001. [Google Scholar] [CrossRef]
- Husek, T.R.; Sirotnik, K. Item Sampling in Educational Research; CSEIP Occasional Report No. 2; University of California: Los Angeles, CA, USA, 1967. [Google Scholar]
- Wu, H.; Browne, M.W. Quantifying adventitious error in a covariance structure as a random effect. Psychometrika 2015, 80, 571–600. [Google Scholar] [CrossRef] [PubMed]
Mean | SD | COV95 | |||||||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|---|
Bias | SD | EXP | JK | AN | JK | AN | AAD | JK | AN | JK | AN | ||
Linking error of estimated mean | |||||||||||||
0.25 | 10 | 0.000 | 0.079 | 0.079 | 0.079 | 0.070 | 0.990 | 0.875 | 0.0091 | 0.0202 | 0.0177 | 91.6 | 88.6 |
0.25 | 20 | 0.003 | 0.056 | 0.056 | 0.056 | 0.053 | 1.000 | 0.945 | 0.0031 | 0.0096 | 0.0091 | 93.2 | 91.7 |
0.25 | 30 | 0.000 | 0.045 | 0.046 | 0.046 | 0.044 | 1.007 | 0.970 | 0.0017 | 0.0065 | 0.0062 | 94.3 | 93.3 |
0.25 | 40 | 0.003 | 0.039 | 0.040 | 0.040 | 0.039 | 1.013 | 0.986 | 0.0011 | 0.0047 | 0.0045 | 96.3 | 95.7 |
0.5 | 10 | 0.003 | 0.161 | 0.158 | 0.153 | 0.135 | 0.949 | 0.837 | 0.0181 | 0.0397 | 0.0354 | 91.6 | 87.9 |
0.5 | 20 | 0.002 | 0.111 | 0.112 | 0.111 | 0.104 | 1.000 | 0.942 | 0.0065 | 0.0185 | 0.0177 | 93.5 | 91.8 |
0.5 | 30 | 0.004 | 0.090 | 0.091 | 0.091 | 0.087 | 1.011 | 0.972 | 0.0036 | 0.0130 | 0.0127 | 94.5 | 93.5 |
0.5 | 40 | 0.008 | 0.080 | 0.079 | 0.078 | 0.076 | 0.978 | 0.948 | 0.0025 | 0.0097 | 0.0095 | 93.2 | 92.3 |
Linking error of estimated standard deviation | |||||||||||||
0.25 | 10 | −0.007 | 0.039 | — | 0.041 | 0.033 | 1.051 | 0.866 | 0.0073 | 0.0111 | 0.0091 | 95.2 | 92.3 |
0.25 | 20 | −0.008 | 0.023 | — | 0.023 | 0.021 | 1.009 | 0.914 | 0.0022 | 0.0042 | 0.0038 | 94.7 | 93.0 |
0.25 | 30 | −0.008 | 0.019 | — | 0.017 | 0.016 | 0.901 | 0.848 | 0.0011 | 0.0026 | 0.0024 | 92.4 | 91.7 |
0.25 | 40 | −0.008 | 0.016 | — | 0.014 | 0.014 | 0.898 | 0.856 | 0.0007 | 0.0019 | 0.0017 | 90.9 | 90.8 |
0.5 | 10 | −0.030 | 0.078 | — | 0.077 | 0.062 | 0.985 | 0.796 | 0.0155 | 0.0206 | 0.0166 | 94.9 | 90.8 |
0.5 | 20 | −0.029 | 0.053 | — | 0.043 | 0.040 | 0.827 | 0.752 | 0.0047 | 0.0079 | 0.0068 | 89.3 | 87.5 |
0.5 | 30 | −0.027 | 0.042 | — | 0.033 | 0.031 | 0.774 | 0.727 | 0.0026 | 0.0048 | 0.0044 | 85.9 | 84.7 |
0.5 | 40 | −0.027 | 0.037 | — | 0.027 | 0.026 | 0.723 | 0.690 | 0.0019 | 0.0034 | 0.0033 | 85.0 | 83.0 |
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content. |
© 2023 by the author. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).
Share and Cite
Robitzsch, A. Analytical Approximation of the Jackknife Linking Error in Item Response Models Utilizing a Taylor Expansion of the Log-Likelihood Function. AppliedMath 2023, 3, 49-59. https://doi.org/10.3390/appliedmath3010004
Robitzsch A. Analytical Approximation of the Jackknife Linking Error in Item Response Models Utilizing a Taylor Expansion of the Log-Likelihood Function. AppliedMath. 2023; 3(1):49-59. https://doi.org/10.3390/appliedmath3010004
Chicago/Turabian StyleRobitzsch, Alexander. 2023. "Analytical Approximation of the Jackknife Linking Error in Item Response Models Utilizing a Taylor Expansion of the Log-Likelihood Function" AppliedMath 3, no. 1: 49-59. https://doi.org/10.3390/appliedmath3010004
APA StyleRobitzsch, A. (2023). Analytical Approximation of the Jackknife Linking Error in Item Response Models Utilizing a Taylor Expansion of the Log-Likelihood Function. AppliedMath, 3(1), 49-59. https://doi.org/10.3390/appliedmath3010004