Regularized Mislevy-Wu Model for Handling Nonignorable Missing Item Responses

Abstract

Missing item responses are frequently found in educational large-scale assessment studies. In this article, the Mislevy-Wu item response model is applied for handling nonignorable missing item responses. This model allows that the missingness of an item depends on the item itself and a further latent variable. However, with low to moderate amounts of missing item responses, model parameters for the missingness mechanism are difficult to estimate. Hence, regularized estimation using a fused ridge penalty is applied to the Mislevy-Wu model to stabilize estimation. The fused ridge penalty function is separately defined for multiple-choice and constructed response items because previous research indicated that the missingness mechanisms strongly differed for the two item types. In a simulation study, it turned out that regularized estimation improves the stability of item parameter estimation. The method is also illustrated using international data from the progress in international reading literacy study (PIRLS) 2011 data.

1. Introduction

In educational large-scale assessment (LSA) studies [1,2], such as the progress in international reading literacy study (PIRLS; [3]), the trends in international mathematics and science study (TIMSS; [4]), or the programme for international student assessment (PISA; [5]), students’ abilities are assessed using cognitive test items. Often, however, students do not respond to specific items leading to missing item responses [6]. It is not obvious how item nonresponse  [7] should be treated in the computation of values of abilities (i.e., values of the latent trait) in item response theory (IRT) models  [8,9,10] that are used as scaling models.

Researchers frequently argue for applying complex IRT models that model missing item responses in order to avoid biased item parameters [6,11]. If students omit items, the most obvious option would be treating them as either wrong or missing, which effectively means removing them from the estimation. In the latter case, missing item responses are simply ignored. Slightly more complex treatments assume that missing item responses can be ignored when conditioning on further latent variables (i.e., latent ignorability; [12]). However, it has been shown that these kinds of models do not adequately fit typical LSA datasets [13]. Recently, the Mislevy-Wu (MW) model received some attention  [7,13,14,15,16] that relaxes the strict assumption that missingness on item responses should either be treated as wrong or latent ignorable. However, the MW model tends to produce unstable parameter estimates if the missingness parameters are estimated item-specific. To circumvent this issue, this paper proposes a regularized estimation approach to the MW model to stabilize parameter estimation.

The decision of how to score missing item responses in LSA studies is a delicate one. On the one hand, students might omit item responses because of a lack of motivation. On the other hand, students could simply not know the correct answer and therefore do not deliver an item response. Even if students have only low motivation to respond to an item in LSA studies, it could be generally questioned whether item omissions should not be scored as wrong. Treating item omission as wrong might induce a strategy for students to only respond to those items that they do know with sufficient confidence. Introducing different item selection (or solution) strategies would undoubtedly impact the interpretation of results in an LSA study. Hence, one can conclude that the decision on how to score item responses is not (mainly) a statistical one [17]. Nevertheless, we use the regularized MW model in this paper to explore potential causes of missing item responses and the consequences of different missing data treatments.

The rest of the article is structured as follows. The regularized MW model is introduced in Section 2. Section 3 presents results from a simulation study that investigates the performance of regularized estimation of the MW model. In Section 4, an empirical example involving PIRLS 2011 data is provided. Finally, the article closes with a discussion in Section 5.

2. Mislevy-Wu Model

In this section, we review missing data terminology and introduce the regularized MW model for handling nonignorable missing item responses.

A vector of item responses for person p is denoted by ${\mathit{X}}_p$. In the presence of missing values, we decompose ${\mathit{X}}_p$ into ${\mathit{X}}_p=({\mathit{X}}_{\mathrm{obs},p},{\mathit{X}}_{\mathrm{mis},p})$, where ${\mathit{X}}_{\mathrm{obs},p}$ denotes the observed and ${\mathit{X}}_{\mathrm{mis},p}$ the missing item responses. Let ${\mathit{R}}_p$ denote the vector of response indicators whose values are 1 if an item is observed and 0 if it is missing. We can factorize the joint distribution of ${\mathit{X}}_p$ and ${\mathit{R}}_p$ as

$$\mathrm{P}({\mathit{R}}_p,{\mathit{X}}_{\mathrm{obs},p},{\mathit{X}}_{\mathrm{mis},p})=\mathrm{P}\left({\mathit{R}}_p\right|{\mathit{X}}_{\mathrm{obs},p},{\mathit{X}}_{\mathrm{mis},p})\mathrm{P}({\mathit{X}}_{\mathrm{obs},p},{\mathit{X}}_{\mathrm{mis},p})$$

(1)

Missing data literature distinguishes different missingness mechanisms regarding the assumptions of the conditional distribution $\mathrm{P}\left({\mathit{R}}_p\right|{\mathit{X}}_{\mathrm{obs},p},{\mathit{X}}_{\mathrm{mis},p})$ (see  [18,19]). The most important distinction is between missing at random (MAR;  [20]) and missing not at random (MNAR). MAR holds if

$$\mathrm{P}\left({\mathit{R}}_p\right|{\mathit{X}}_{\mathrm{obs},p},{\mathit{X}}_{\mathrm{mis},p})=\mathrm{P}\left({\mathit{R}}_p\right|{\mathit{X}}_{\mathrm{obs},p}).$$

(2)

If (2) is violated, the missing data are MNAR. Based on the MAR assumption in (2), we can integrate out the missing data ${\mathit{X}}_{\mathrm{mis},p}$ and obtain

$$\int \mathrm{P}\left({\mathit{R}}_p\right|{\mathit{X}}_{\mathrm{obs},p},{\mathit{X}}_{\mathrm{mis},p})\mathrm{P}({\mathit{X}}_{\mathrm{obs},p},{\mathit{X}}_{\mathrm{mis},p})\mathrm{d}{\mathit{X}}_{\mathrm{mis},p}=\mathrm{P}\left({\mathit{R}}_p\right|{\mathit{X}}_{\mathrm{obs},p})\int \mathrm{P}({\mathit{X}}_{\mathrm{obs},p},{\mathit{X}}_{\mathrm{mis},p})\mathrm{d}{\mathit{X}}_{\mathrm{mis},p}$$

(3)

The crucial point is that the factor $\mathrm{P}\left({\mathit{R}}_p\right|{\mathit{X}}_{\mathrm{obs},p})$ does not depend on missing data ${\mathit{X}}_{\mathrm{mis},p}$, which is the reason why likelihood-based inference can rely on the observed data by parametrizing the distribution $\int \mathrm{P}({\mathit{X}}_{\mathrm{obs},p},{\mathit{X}}_{\mathrm{mis},p})\mathrm{d}{\mathit{X}}_{\mathrm{mis},p}$. If the model parameters of the two factors are distinct  [18], missing data are denoted as ignorable. Hence, we also label the MAR assumption (2) as manifest ignorability (MI).

Latent ignorability (LI;  [21,22,23,24,25,26,27]) is one of the weakest nonignorable missingness mechanisms. LI weakens the assumption of ignorability for MAR data. In this case, the existence of a latent variable ${\mathit{\eta }}_p$ is assumed. The dimension of ${\mathit{\eta }}_p$ is typically much lower than the dimension of ${\mathit{X}}_p$. LI is formally defined as (see  [27])

$$\mathrm{P}\left({\mathit{R}}_p\right|{\mathit{X}}_{\mathrm{obs},p},{\mathit{X}}_{\mathrm{mis},p},{\mathit{\eta }}_p)=\mathrm{P}\left({\mathit{R}}_p\right|{\mathit{X}}_{\mathrm{obs},p},{\mathit{\eta }}_p).$$

(4)

That is, the probability of missing item responses depends on observed item responses and the latent variable ${\mathit{\eta }}_p$, but not the unknown missing item responses ${\mathit{X}}_{\mathrm{mis},p}$ itself. By integrating out ${\mathit{X}}_{\mathrm{mis},p}$, we obtain

$$\int \mathrm{P}\left({\mathit{R}}_p\right|{\mathit{X}}_{\mathrm{obs},p},{\mathit{X}}_{\mathrm{mis},p},{\mathit{\eta }}_p)\mathrm{P}({\mathit{X}}_{\mathrm{obs},p},{\mathit{X}}_{\mathrm{mis},p}|{\mathit{\eta }}_p)\mathrm{d}{\mathit{X}}_{\mathrm{mis},p}\mathrm{d}{\mathit{\eta }}_p=\int \mathrm{P}\left({\mathit{R}}_p\right|{\mathit{X}}_{\mathrm{obs},p},{\mathit{\eta }}_p)\mathrm{P}({\mathit{X}}_{\mathrm{obs},p},{\mathit{X}}_{\mathrm{mis},p}|{\mathit{\eta }}_p)\mathrm{d}{\mathit{X}}_{\mathrm{mis},p}\mathrm{d}{\mathit{\eta }}_p$$

(5)

Specification (4) is also known as a shared-parameter model [28,29]. In most applications, conditional independence of item responses $X_{pi}$ and response indicators $R_{pi}$ conditional on ${\mathit{\eta }}_p$ is assumed [27]. In this case, Equation (5) simplifies to

$$\int \mathrm{P}\left({\mathit{R}}_p\right|{\mathit{X}}_{\mathrm{obs},p},{\mathit{X}}_{\mathrm{mis},p},{\mathit{\eta }}_p)\mathrm{P}({\mathit{X}}_{\mathrm{obs},p},{\mathit{X}}_{\mathrm{mis},p}|{\mathit{\eta }}_p)\mathrm{d}{\mathit{X}}_{\mathrm{mis},p}\mathrm{d}{\mathit{\eta }}_p=\int \prod _{i=1}^I\left[\mathrm{P}(R_{pi}=r_{pi}|{\mathit{\eta }}_p)\mathrm{P}{(X_{pi}=x_{pi}|{\mathit{\eta }}_p)}^{r_{pi}}\right]\mathrm{d}{\mathit{\eta }}_p.$$

(6)

In the rest of this paper, it is assumed that the latent variable ${\mathit{\eta }}_p$ consists of a latent ability ${\theta }_p$ and a latent response propensity ${\xi }_p$. The latent response propensity ${\xi }_p$ is a unidimensional latent variable that represents the dimensional structure of the response indicators ${\mathit{R}}_p$.

The IRT model of interest follows a two-parameter logistic (2PL) model [30]:

$$\mathrm{P}(X_{pi}=1|{\theta }_p)=\Psi \left(a_i({\theta }_p-b_i)\right),$$

(7)

where $\Psi$ is the logistic link function, $a_i$ represents item discriminations, and $b_i$ represents item difficulties.

Regularized Mislevy-Wu Model

For allowing nonignorable missing item responses, the conditional distribution $\mathrm{P}(R_{pi}=1|X_{pi}=x,{\theta }_p,{\xi }_p)$ must be specified. The conditional probability of a missing item response in the MW model  [13,14,16,31,32,33] is defined as

$$\begin{array}{c}\hfill \mathrm{P}(R_{pi}=1,X_{pi}=x|{\theta }_p,{\xi }_p)=\Psi ({\xi }_p-{\beta }_i-{\rho }_{i}x)\mathrm{for}x=0,1.\end{array}$$

(8)

The total probability of a missing item response is given by

$$\begin{array}{c}\hfill \mathrm{P}(R_{pi}=0|{\theta }_p,{\xi }_p)=\mathrm{P}(R_{pi}=1,X_{pi}=\mathrm{NA}|{\theta }_p,{\xi }_p)=\sum _{x=0}^1\mathrm{P}(R_{pi}=1,X_{pi}=x|{\theta }_p,{\xi }_p).\end{array}$$

(9)

By combining (8) and (9), we get

$$\mathrm{P}(X_{pi}=x,R_{pi}=r|{\theta }_p,{\xi }_p)=\left\{\begin{array}{cc}\left[1-\Psi \left(a_i({\theta }_p-b_i)\right)\right]\Psi ({\xi }_p-{\beta }_i)\hfill & \mathrm{if}x=0\mathrm{and}r=1,\hfill \\ \Psi \left(a_i({\theta }_p-b_i)\right)\Psi ({\xi }_p-{\beta }_i-{\rho }_i)\hfill & \mathrm{if}x=1\mathrm{and}r=1,\hfill \\ \Psi (a_i({\theta }_p-b_i)\Psi ({\xi }_p-{\beta }_i-{\rho }_i)+\left[1-\Psi \left(a_i({\theta }_p-b_i)\right)\right]\Psi ({\xi }_p-{\beta }_i)\hfill & \mathrm{if}x=\mathrm{NA}\mathrm{and}r=0.\hfill \end{array}\right.$$

(10)

Note that the model defined in Equation (10) can be interpreted as an IRT model for a variable $U_{pi}$ that has three categories: Category 0 (observed incorrect): $X_{pi}=0$, $R_{pi}=1$, Category 1 (observed correct): $X_{pi}=1$, $R_{pi}=1$, and Category 2 (missing item response): $X_{pi}=\mathrm{NA}$, $R_{pi}=0$ (see  [34,35]). The marginal distribution $\mathrm{P}(X_{pi}=x|{\theta }_p,{\xi }_p)$ in (10) follows the 2PL model (7). The conditional probabilities for response indicators $R_{pi}$ are modeled with parameters ${\beta }_i$ and ${\delta }_i$. The parameter ${\beta }_i$ parametrizes the item-specific proportion of missing item responses, while the parameter ${\delta }_i$ quantifies the dependence of the responding to item i conditional on the true but possibly unobserved item response $X_{pi}$. It has been pointed out in  [13,33] that the MW model (10) contains the special cases of treating missing item responses as latent ignorable and as wrong as two extreme cases. Moreover, the simulation studies in  [13,14] demonstrated that a common $\delta$ parameter that is constant across items could be consistently estimated.

Figure 1 graphically displays the MW model. Note the dependency of response indicators $R_i$ from items $X_i$.

In this article, a bivariate normal distribution for $({\theta }_p,{\xi }_p)$ is assumed, where $\mathrm{SD}\left({\theta }_p\right)$ is fixed to one, and $\mathrm{SD}\left({\xi }_p\right)$, as well as $\mathrm{Cor}({\theta }_p,{\xi }_p)$, are estimated (see [36,37] for more complex distributions). LI in general and the model of Holman and Glas [12] in particular is obtained in the MW model by fixing all ${\delta }_i$ parameters equal to zero. If one fixes all ${\delta }_i$ parameters in the MW model at a sufficiently small (negative) value, such as $-9.99$, students with missing item responses are scored as incorrect. Moreover, if one fixes all ${\delta }_i$ parameters to zero and sets the correlation of $\theta$ and $\xi$ to zero, MI (i.e., MAR) is obtained. Hence, LI can be tested against MI. Moreover, the MW model is more general than the LI model because the latter model only models the dependence of missingness on item i from $\xi$, but not the item itself.

The MW model can be estimated with maximum likelihood (ML). By denoting all item parameters by $\mathit{\gamma }=({\mathit{\gamma }}_1,\dots ,{\mathit{\gamma }}_I)$ and distribution parameters by $\mathit{\alpha }$, the log-likelihood function is given by

$$l(\mathit{\gamma },\mathit{\alpha };\mathit{X},\mathit{R})=\sum _{p=1}^{N}log{\int }_{-\infty }^{\infty }\prod _{i=1}^I\left[\mathrm{P}(X_{pi}=x_{pi},R_{pi}=r_{pi}|\theta ,\xi ;{\mathit{\gamma }}_i)\right]f(\theta ,\xi ;\mathit{\alpha })\mathrm{d}\theta \mathrm{d}\xi ,$$

(11)

where $\mathit{X}={\left(x_{pi}\right)}_{pi}$ and $\mathit{R}={\left(r_{pi}\right)}_{pi}$ denote the datasets of item responses and response indicators. The item-specific parameters are given as ${\mathit{\gamma }}_i=(a_i,b_i,{\beta }_i,{\delta }_i)$. The log-likelihood function can be numerically maximized to obtain item parameter estimates $\widehat{\mathit{\gamma }}$ and distribution parameters $\widehat{\mathit{\alpha }}$. In IRT software, the expectation-maximization algorithm is frequently utilized  [38,39].

In our experience, estimating item-specific ${\delta }_i$ parameters in the MW model can become quite unstable. Moreover, it has been shown that average ${\delta }_i$ parameters typically strongly differ between constructed response (CR) and multiple-choice (MC) items, because the omission of CR items is more associated with the true but not fully observed item response, while omissions of MC items are only weakly associated with true item responses  [13]. For stabilizing the estimation of ${\delta }_i$ parameters in ML, we propose to employ regularized ML estimation with fused ridge-type penalty functions  [40].

Let ${\mathcal{I}}_{\mathrm{MC}}\subset \mathcal{I}$ and ${\mathcal{I}}_{\mathrm{CR}}\subset \mathcal{I}$ be distinct integer sets of multiple-choice and constructed response items, respectively, where $\mathcal{I}=\{1,\dots ,I\}$. The fused ridge penalty function $\mathcal{P}$ for the MW model is defined by

$$\mathcal{P}(\mathit{\gamma };\lambda )=\lambda \left[\sum _{i,j\in {\mathcal{I}}_{\mathrm{CR}}}{({\delta }_i-{\delta }_j)}^2+\sum _{i,j\in {\mathcal{I}}_{\mathrm{MC}}}{({\delta }_i-{\delta }_j)}^2\right],$$

(12)

where $\lambda$ is a fixed regularization parameter. In regularized ML estimation, one maximizes the penalized log-likelihood function $l_{pen}$ defined by

$$\begin{array}{c}\hfill l_{pen}(\mathit{\gamma },\mathit{\alpha };\lambda ,\mathit{X},\mathit{R})\end{array}=l(\mathit{\gamma },\mathit{\alpha };\mathit{X},\mathit{R})-\mathcal{P}(\gamma ;\lambda ).$$

(13)

Using the penalty function in (12) implies normal priors for ${\delta }_i$ with means for CR and MC items, respectively, and a common variance [40]. Importantly, by only considering differences in pairs of item parameters ${\delta }_i$, the item-type specific means of ${\delta }_i$ are not explicitly estimated. The MW model (10) applied with regularized ML estimation using the fitting function (13) is also called the regularized MW model.

The maximization of $l_{pen}$ involves the unknown regularization parameter $\lambda$. The k-fold cross-validation approach is used for obtaining the optimal regularization parameter ${\lambda }_{\mathrm{opt}}$. The dataset is divided into k groups, and the parameters of the regularized MW model are estimated on $k-1$ folds leaving one fold out to evaluate the cross-validation error. This is performed by leaving one fold out in turn and for each value of the regularization parameter $\lambda$. In this article, the error was evaluated using the negative log-likelihood function value [40]. The cross-validation error is calculated as ${\sum }_{h=1}^k\begin{array}{c}\hfill l({\widehat{\mathit{\gamma }}}_{(-h)},{\widehat{\mathit{\alpha }}}_{(-h)};{\mathit{X}}_h,{\mathit{R}}_h)\end{array}$, where ${\widehat{\mathit{\gamma }}}_{(-u)}$ and ${\widehat{\mathit{\alpha }}}_{(-h)}$ are the vector of item parameter and distribution estimates obtained by excluding the hth group of data. Moreover, ${\mathit{X}}_h$ and ${\mathit{R}}_h$ denote the datasets of item responses and response indicators in the hth part of the data, respectively. In cross-validation, the log-likelihood function is predicted on the part of the data that has not been used for parameter estimation [41]. The smallest cross-validated log-likelihood value determines the optimal regularization parameter ${\lambda }_{\mathrm{opt}}$. In practice, $k=5$ or $k=10$ is frequently chosen.

3. Simulation Study

3.1. Method

In this simulation study that studies the performance of the regularized MW model, we fixed the number of items to $I=20$ and fixed item parameters $a_i$, $b_i$ and ${\delta }_i$ throughout all replications. To mimic real-data situations, we assumed that the first ten items C01,... , C10 were CR, while the last ten items M11, ..., M20 were MC. On average, the missing proportion of CR items was larger than for MC items. The ${\delta }_i$ parameters were varied according to two data-generating models DGM1 and DGM2. In DGM1, the missing proportions were 0.112 for MC items and 0.153 for CR items. In DGM2, a higher missing proportion of 0.341 for CR items was assumed while retaining the missing proportion for MC items at 0.112. The item parameters used in the simulation study can be found in

Table A1. Simulation Study: Data-generating item parameters of the Mislevy-Wu model.

Item	Type	${\mathit{a}}_{\mathit{i}}$	${\mathit{b}}_{\mathit{i}}$	${\mathit{\beta }}_{\mathit{i}}$		${\mathit{\delta }}_{\mathit{i}}$
				DGM1	DGM2
C01	CR	$\phantom{-}$1.7	$\phantom{-}$1.4	−1.7	$\phantom{-}$0.3	−2.0
C02	CR	$\phantom{-}$1.2	$\phantom{-}$0.4	−2.7	−0.7	−1.7
C03	CR	$\phantom{-}$0.5	$\phantom{-}$1.3	−2.2	−0.2	−3.6
C04	CR	$\phantom{-}$2.2	−0.6	−1.4	$\phantom{-}$0.6	−2.9
C05	CR	$\phantom{-}$2.7	−0.3	−1.3	$\phantom{-}$0.7	−3.1
C06	CR	$\phantom{-}$2.8	−0.1	−1.2	$\phantom{-}$0.8	−3.8
C07	CR	$\phantom{-}$1.3	−1.4	−2.5	−0.5	−4.8
C08	CR	$\phantom{-}$1.3	−1.5	−1.8	$\phantom{-}$0.2	−2.0
C09	CR	$\phantom{-}$1.1	−0.4	−2.5	−0.5	−1.3
C10	CR	$\phantom{-}$0.5	$\phantom{-}$0.8	−2.4	−0.4	−0.6
M11	MC	$\phantom{-}$0.9	−1.3	−3.2	−3.2	$\phantom{-}$ 0.5
M12	MC	$\phantom{-}$1.0	−0.4	−3.4	−3.4	−0.8
M13	MC	$\phantom{-}$0.7	−0.8	−3.6	−3.6	−0.3
M14	MC	$\phantom{-}$1.2	−0.6	−3.7	−3.7	−0.2
M15	MC	$\phantom{-}$1.1	−0.8	−2.8	−2.8	$\phantom{-}$ 0.4
M16	MC	$\phantom{-}$1.2	−1.6	−2.8	−2.8	−0.3
M17	MC	$\phantom{-}$1.8	−1.7	−2.8	−2.8	−1.1
M18	MC	$\phantom{-}$1.0	−1.1	−3.4	−3.4	−1.0
M19	MC	$\phantom{-}$1.0	−0.8	−2.8	−2.8	−0.5
M20	MC	$\phantom{-}$1.5	−2.0	−1.8	−1.8	−0.9

Note. DGM = data-generating model; Type = item format; CR = constructed response item; MC = multiple-choice item.

in Appendix A (see also the directory "Simulation Study" https://osf.io/5pd28 (accessed on 21 June 2023)).

We chose sample sizes $N=1000$ and $N=2500$. We did not opt for smaller sample sizes because we think that estimating an IRT model for response indicators requires sufficiently large sample sizes. Hence, the MW model is more suitable for LSA studies than for small-scale studies.

A bivariate normal distribution was simulated for the ability variable $\theta$ and the response propensity $\xi$. The standard deviation of $\theta$ was set to 1, while the standard deviation of $\xi$ was fixed at 2. Moreover, the correlation of $\theta$ and $\xi$ was fixed at 0.5 when simulating the data.

The regularized MW model was estimated for a fixed sequence of values for the regularization parameter $\lambda$. A grid of 21 regularization parameters was chosen: 0.000010, 0.000018, 0.000034, 0.000062, 0.000113, 0.000207, 0.000379, 0.000695, 0.001274, 0.002336, 0.004281, 0.007848, 0.014384, 0.026367, 0.048329, 0.088587, 0.162378, 0.297635, 0.545560, 1.0, and 10,000. Values between 0.000010 and 1.0 were equidistantly chosen on a logarithmic scale. In k-fold cross-validation, $k=5$ folds were used. In the MW model, the estimated distribution parameters $\mathit{\alpha }$ consisted of the variance of $\xi$ and the covariance of $\theta$ and $\xi$.

In total, 2500 replications were conducted in each simulation condition. We assessed the performance of parameter estimates by bias and root mean square error (RMSE). To provide simple summary statistics, we averaged absolute biases and RMSE values across items for the same item parameter groups (i.e., the a, b, $\beta$, and $\delta$ parameters).

The statistical software R [42] was employed for all parts of the simulation. The estimation of the regularized MW model was carried out using the sirt::xxirt() function in the sirt package [43]. Replication material can be found in the directory "Simulation Study" at https://osf.io/5pd28 (accessed on 21 June 2023).

3.2. Results

In Figure 2, the average absolute bias (red dashed line) and the average RMSE (solid black line) for item parameter groups $\delta$, $\beta$, a, and b are displayed for the DGM2. It can be seen that for $N=1000$, the minimum average RMSE for $\delta$ parameters is obtained for a $\lambda$ value that is substantially larger than the optimal regularization parameter ${\lambda }_{\mathrm{opt}}$ obtained with k-fold cross-validated log-likelihood estimation. Interestingly, biases in all parameters became relevant for sufficiently large regularization parameters. Hence, the search for an optimal $\lambda$ parameter regarding RMSE reflects a bias-variance tradeoff. However, it should be emphasized that for a broad range of sufficiently small $\lambda$ values, the bias and RMSE for item discriminations $a_i$, and item difficulties $b_i$ were almost unaffected by the choice of $\lambda$.

In

Table 1. Simulation Study: Average absolute bias (Bias) and average absolute RMSE of estimated item parameters of the regularized Mislevy-Wu model as a function of sample size N and different choices of the regularization parameter $\lambda$ for two data-generating models (DGM) DGM1 and DGM2.

DGM	Par	N	Bias for $\mathit{\lambda }=$				RMSE for $\mathit{\lambda }=$
			${\mathbf{10}}^{-\mathbf{5}}$	${\mathit{\lambda }}_{\mathrm{opt}}$	$\mathbf{0.0263665}$	${\mathbf{10}}^{\mathbf{5}}$	${\mathbf{10}}^{-\mathbf{5}}$	${\mathit{\lambda }}_{\mathrm{opt}}$	$\mathbf{0.0263665}$	${\mathbf{10}}^{\mathbf{5}}$
DGM1	${\delta }_i$	1000	0.123	0.076	0.304	0.754	0.915	0.845	0.638	0.875
		2500	0.067	0.045	0.190	0.757	0.563	0.540	0.451	0.812
	${\beta }_i$	1000	0.063	0.068	0.092	0.221	0.313	0.309	0.285	0.329
		2500	0.028	0.031	0.052	0.221	0.189	0.189	0.186	0.277
	$a_i$	1000	0.008	0.008	0.011	0.028	0.154	0.154	0.154	0.156
		2500	0.002	0.002	0.005	0.028	0.096	0.096	0.096	0.102
	$b_i$	1000	0.016	0.018	0.028	0.064	0.146	0.146	0.144	0.154
		2500	0.008	0.010	0.017	0.062	0.089	0.090	0.090	0.109
DGM2	${\delta }_i$	1000	0.063	0.063	0.184	0.750	0.687	0.631	0.518	0.896
		2500	0.018	0.028	0.090	0.750	0.382	0.374	0.349	0.822
	${\beta }_i$	1000	0.038	0.052	0.080	0.266	0.311	0.299	0.280	0.376
		2500	0.017	0.023	0.039	0.267	0.188	0.187	0.183	0.323
	$a_i$	1000	0.012	0.012	0.015	0.043	0.172	0.172	0.171	0.178
		2500	0.004	0.004	0.006	0.041	0.105	0.105	0.105	0.118
	$b_i$	1000	0.013	0.020	0.032	0.108	0.165	0.164	0.161	0.201
		2500	0.005	0.008	0.015	0.105	0.100	0.100	0.099	0.155

Note. Par = item parameter group; ${\lambda }_{\mathrm{opt}}$ = optimal regularization parameter selected with cross-validated log-likelihood.

, average absolute bias and average RMSE are displayed for the optimal regularization parameter ${\lambda }_{\mathrm{opt}}$, and fixed regularization parameters ${10}^{-5}$, $0.0263665$, and ${10}^5$. It can be seen that $\lambda =0.0263665$ strongly outperformed the other $\lambda$ choices in terms of RMSE for the ${\delta }_i$ parameters. However, a nonnegligible bias in ${\delta }_i$ and ${\beta }_i$ parameters was introduced by using this regularization parameter. Nevertheless, inducing too much regularization could stabilize estimated item parameters for the response indicators, while the target parameters $a_i$ and $b_i$ were almost unaffected by the choice of $\lambda$. Hence, one could generally conclude that the MW model should be utilized to estimate the missing response mechanism flexibly. The regularization technique is only applied for stabilizing parameter estimates without introducing relevant bias in target item parameter estimates.

4. Empirical Example

4.1. Method

In the following analysis, item responses of booklet 13 in PIRLS 2011 (i.e., the PIRLS Reader) consisting of 35 items (20 CR items and 15 MC items with four response alternatives) were used. For this booklet, item responses of 968 Austrian (AUT), 809 German (DEU), 901 French (FRA), and 802 Dutch (NLD) students were available. The resulting dataset is used for illustrative purposes in this section. For ease of presentation, all polytomous items were dichotomized, where only the highest scores were recoded as correct. The dataset has been made available as data.pirlsmissing in the R [42] package sirt  [43].

Descriptive analyses showed that the average proportion of missing item responses varied considerably between items and countries (AUT: 0.112, DEU: 0.079, FRA: 0.136, NLD: 0.027). For MC items, the average rate of missing item responses was 0.023 ($SD=0.016$). For CR items, the average rate of missing item responses was substantially larger ($M=0.141$, $SD=0.070$).

We estimated the nonregularized MW model with freely estimated ${\delta }_i$ parameters and compared this model to constrained alternatives. In the LI model [12], all ${\delta }_i$ parameters were fixed to zero. In the WR model, all missing item responses are treated as incorrect, which was implemented by fixing all ${\delta }_i$ parameters to $-9.99$ setting the response probabilities effectively to zero for students who do not know the item. Finally, in the MI model, we fixed all ${\delta }_i$ parameters to zero and fixed the correlation of $\theta$ and $\xi$ to zero. Model comparisons were conducted based on the Akaike information criterion (AIC) and the Bayesian information criterion (BIC).

The MW model was also estimated using regularized estimation. The optimal regularization parameter ${\lambda }_{\mathrm{opt}}$ was selected by minimizing the negative cross-validated log-likelihood value. The sequence of the regularization parameter $\lambda$ was selected between ${10}^{-8}$ and 10,000, equidistantly spaced on a logarithmic scale.

The R [42] package sirt [43] using the sirt::xxirt() was employed for fitting the IRT models. Replication material can be found in the directory Empirical Example at https://osf.io/5pd28 (accessed on 21 June 2023).

4.2. Results

In

Table 2. PIRLS Reader 2011: Model comparisons.

Model	#npars	AIC	BIC	$\mathit{SD}\left(\mathit{\xi }\right)$	$\mathit{Cor}(\mathit{\theta },\mathit{\xi })$	${\mathit{\delta }}_{\mathit{i}}$
MI	106	162,192	162,844	2.41	0 ${}^{‡}$ $\phantom{00}$	$\phantom{-}$0 ${}^{‡}$$\phantom{99}$
LI	107	161,796	162,454	2.38	0.41 $\phantom{{}^{‡}}$	$\phantom{-}$0 ${}^{‡}$$\phantom{99}$
WR	107	161,414	162,073	2.29	0.41 $\phantom{{}^{‡}}$	−9.99 ${}^{‡}$
MW	142	161,086	161,960	2.34	0.41 $\phantom{{}^{‡}}$	est

Note. #npars = number of estimated model parameters; MI = manifest ignorabiity; LI = latent ignorability; WR = treating missing item responses as wrong (i.e., 0); MW = Mislevy-Wu model; ‡ = fixed model parameter; est =  estimated model parameters; Entries with the least AIC or BIC are printed in bold font, respectively.

, model comparisons of the four nonregularized models are displayed. The most general MW model turned out to be the best-fitting model in terms of AIC and BIC. In line with [13], the WR model (i.e., treating missing item responses as incorrect) outperformed the LI model (i.e., treating missing item responses as missing). The standard deviation of $\xi$ slightly varies across models, being smallest when treating missing item responses as wrong in model MW. Also note that the correlation of $\theta$ and $\xi$ was practically identical for the models LI, WR, and MW.

In

Table 3. PIRLS Reader 2011: Estimated item parameters of the regularized Mislevy-Wu model.

					${\mathit{a}}_{\mathit{i}}$				${\mathit{b}}_{\mathit{i}}$				${\mathit{\beta }}_{\mathit{i}}$	${\mathit{\delta }}_{\mathit{i}}$
Item	Type	Freq0	Freq1	FreqNA	MI	LI	WR	MW	MI	LI	WR	MW	MW	MW
R31G02C	CR	$\phantom{-}$0.28	$\phantom{-}$0.64	$\phantom{-}$0.08	$\phantom{-}$0.86	$\phantom{-}$0.85	$\phantom{-}$0.89	$\phantom{-}$0.91	−1.04	−1.04	−0.81	−0.80	−3.03	−4.45
R31G04C	CR	$\phantom{-}$0.58	$\phantom{-}$0.21	$\phantom{-}$0.20	$\phantom{-}$1.05	$\phantom{-}$1.04	$\phantom{-}$1.06	$\phantom{-}$1.05	$\phantom{-}$1.23	$\phantom{-}$1.25	$\phantom{-}$1.43	$\phantom{-}$1.36	−2.34	−1.40
R31G08CZ	CR	$\phantom{-}$0.41	$\phantom{-}$0.40	$\phantom{-}$0.19	$\phantom{-}$1.27	$\phantom{-}$1.26	$\phantom{-}$1.34	$\phantom{-}$1.23	$\phantom{-}$0.18	$\phantom{-}$0.20	$\phantom{-}$0.48	$\phantom{-}$0.30	−2.01	−0.90
R31G08CA	CR	$\phantom{-}$0.40	$\phantom{-}$0.37	$\phantom{-}$0.23	$\phantom{-}$1.76	$\phantom{-}$1.74	$\phantom{-}$1.82	$\phantom{-}$1.74	$\phantom{-}$1.32	$\phantom{-}$1.35	$\phantom{-}$1.44	$\phantom{-}$1.39	−1.99	−0.76
R31G08CB	CR	$\phantom{-}$0.61	$\phantom{-}$0.14	$\phantom{-}$0.25	$\phantom{-}$1.45	$\phantom{-}$1.44	$\phantom{-}$1.51	$\phantom{-}$1.41	$\phantom{-}$0.09	$\phantom{-}$0.11	$\phantom{-}$0.31	$\phantom{-}$0.17	−2.41	−0.79
R31G10C	CR	$\phantom{-}$0.48	$\phantom{-}$0.40	$\phantom{-}$0.13	$\phantom{-}$1.13	$\phantom{-}$1.13	$\phantom{-}$1.20	$\phantom{-}$1.17	$\phantom{-}$0.28	$\phantom{-}$0.29	$\phantom{-}$0.40	$\phantom{-}$0.35	−3.08	−1.42
R31G12C	CR	$\phantom{-}$0.51	$\phantom{-}$0.29	$\phantom{-}$0.20	$\phantom{-}$0.49	$\phantom{-}$0.48	$\phantom{-}$0.63	$\phantom{-}$0.49	$\phantom{-}$1.24	$\phantom{-}$1.26	$\phantom{-}$1.47	$\phantom{-}$1.25	−2.53	−0.04
R31G13CZ	CR	$\phantom{-}$0.17	$\phantom{-}$0.66	$\phantom{-}$0.17	$\phantom{-}$2.43	$\phantom{-}$2.46	$\phantom{-}$3.31	$\phantom{-}$3.18	−0.55	−0.53	−0.27	−0.33	−1.55	−2.81
R31G13CA	CR	$\phantom{-}$0.23	$\phantom{-}$0.58	$\phantom{-}$0.20	$\phantom{-}$2.11	$\phantom{-}$2.12	$\phantom{-}$2.85	$\phantom{-}$2.72	−0.36	−0.34	−0.11	−0.14	−1.35	−3.40
R31G13CB	CR	$\phantom{-}$0.26	$\phantom{-}$0.52	$\phantom{-}$0.22	$\phantom{-}$2.19	$\phantom{-}$2.19	$\phantom{-}$2.75	$\phantom{-}$2.68	−0.12	−0.10	$\phantom{-}$0.07	$\phantom{-}$0.05	−1.54	−3.21
R31G13CC	CR	$\phantom{-}$0.32	$\phantom{-}$0.46	$\phantom{-}$0.22	$\phantom{-}$3.58	$\phantom{-}$3.67	$\phantom{-}$4.88	$\phantom{-}$5.07	−0.76	−0.74	−0.51	−0.57	−1.69	−2.69
R31P02C	CR	$\phantom{-}$0.23	$\phantom{-}$0.73	$\phantom{-}$0.04	$\phantom{-}$0.81	$\phantom{-}$0.81	$\phantom{-}$0.79	$\phantom{-}$0.81	−1.61	−1.62	−1.52	−1.48	−3.77	−3.88
R31P03C	CR	$\phantom{-}$0.16	$\phantom{-}$0.79	$\phantom{-}$0.06	$\phantom{-}$1.26	$\phantom{-}$1.25	$\phantom{-}$1.24	$\phantom{-}$1.29	−1.57	−1.57	−1.42	−1.38	−2.88	−4.36
R31P05C	CR	$\phantom{-}$0.45	$\phantom{-}$0.48	$\phantom{-}$0.08	$\phantom{-}$1.08	$\phantom{-}$1.09	$\phantom{-}$1.07	$\phantom{-}$1.02	−0.05	−0.05	$\phantom{-}$0.04	−0.11	−4.51	$\phantom{-}$0.59
R31P06C	CR	$\phantom{-}$0.19	$\phantom{-}$0.76	$\phantom{-}$0.04	$\phantom{-}$1.40	$\phantom{-}$1.39	$\phantom{-}$1.34	$\phantom{-}$1.37	−1.28	−1.28	−1.23	−1.24	−3.85	−2.43
R31P07C	CR	$\phantom{-}$0.19	$\phantom{-}$0.74	$\phantom{-}$0.07	$\phantom{-}$1.74	$\phantom{-}$1.74	$\phantom{-}$1.67	$\phantom{-}$1.74	−1.08	−1.08	−0.98	−0.98	−2.90	−3.12
R31P09C	CR	$\phantom{-}$0.14	$\phantom{-}$0.80	$\phantom{-}$0.06	$\phantom{-}$1.25	$\phantom{-}$1.26	$\phantom{-}$1.37	$\phantom{-}$1.33	−1.71	−1.68	−1.40	−1.55	−3.40	−1.83
R31P14C	CR	$\phantom{-}$0.33	$\phantom{-}$0.54	$\phantom{-}$0.13	$\phantom{-}$1.06	$\phantom{-}$1.06	$\phantom{-}$1.15	$\phantom{-}$1.07	−0.48	−0.47	−0.24	−0.39	−2.92	−1.22
R31P15C	CR	$\phantom{-}$0.51	$\phantom{-}$0.36	$\phantom{-}$0.13	$\phantom{-}$0.52	$\phantom{-}$0.53	$\phantom{-}$0.63	$\phantom{-}$0.53	$\phantom{-}$0.74	$\phantom{-}$0.75	$\phantom{-}$0.92	$\phantom{-}$0.81	−3.19	−0.55
R31P16C	CR	$\phantom{-}$0.49	$\phantom{-}$0.38	$\phantom{-}$0.13	$\phantom{-}$0.76	$\phantom{-}$0.75	$\phantom{-}$0.86	$\phantom{-}$0.80	$\phantom{-}$0.48	$\phantom{-}$0.49	$\phantom{-}$0.62	$\phantom{-}$0.57	−3.03	−1.48
R31G01M	MC	$\phantom{-}$0.18	$\phantom{-}$0.81	$\phantom{-}$0.01	$\phantom{-}$1.11	$\phantom{-}$1.15	$\phantom{-}$1.13	$\phantom{-}$1.15	−1.66	−1.63	−1.66	−1.68	−10.51$\phantom{-}$	$\phantom{-}$4.20
R31G03M	MC	$\phantom{-}$0.26	$\phantom{-}$0.73	$\phantom{-}$0.01	$\phantom{-}$1.19	$\phantom{-}$1.18	$\phantom{-}$1.04	$\phantom{-}$1.10	−1.11	−1.11	−1.24	−1.23	−7.52	$\phantom{-}$1.24
R31G05M	MC	$\phantom{-}$0.42	$\phantom{-}$0.56	$\phantom{-}$0.02	$\phantom{-}$0.84	$\phantom{-}$0.85	$\phantom{-}$0.81	$\phantom{-}$0.80	−0.41	−0.40	−0.42	−0.50	−7.56	$\phantom{-}$2.18
R31G06M	MC	$\phantom{-}$0.27	$\phantom{-}$0.72	$\phantom{-}$0.01	$\phantom{-}$0.98	$\phantom{-}$0.97	$\phantom{-}$0.84	$\phantom{-}$0.90	−1.20	−1.22	−1.38	−1.30	−5.72	−2.94
R31G07M	MC	$\phantom{-}$0.40	$\phantom{-}$0.58	$\phantom{-}$0.02	$\phantom{-}$1.04	$\phantom{-}$1.04	$\phantom{-}$0.95	$\phantom{-}$0.99	−0.41	−0.41	−0.45	−0.45	−5.64	−0.82
R31G09M	MC	$\phantom{-}$0.38	$\phantom{-}$0.60	$\phantom{-}$0.02	$\phantom{-}$0.69	$\phantom{-}$0.69	$\phantom{-}$0.65	$\phantom{-}$0.65	−0.71	−0.71	−0.74	−0.78	−6.06	$\phantom{-}$0.08
R31G11M	MC	$\phantom{-}$0.36	$\phantom{-}$0.62	$\phantom{-}$0.02	$\phantom{-}$1.25	$\phantom{-}$1.26	$\phantom{-}$1.23	$\phantom{-}$1.24	−0.54	−0.54	−0.56	−0.58	−5.81	$\phantom{-}$0.03
R31G14M	MC	$\phantom{-}$0.33	$\phantom{-}$0.60	$\phantom{-}$0.07	$\phantom{-}$1.16	$\phantom{-}$1.17	$\phantom{-}$1.07	$\phantom{-}$1.09	−0.65	−0.64	−0.53	−0.79	−5.41	$\phantom{-}$1.68
R31P01M	MC	$\phantom{-}$0.25	$\phantom{-}$0.74	$\phantom{-}$0.01	$\phantom{-}$1.06	$\phantom{-}$1.06	$\phantom{-}$0.97	$\phantom{-}$0.99	−1.24	−1.24	−1.33	−1.37	−9.84	$\phantom{-}$3.88
R31P04M	MC	$\phantom{-}$0.53	$\phantom{-}$0.46	$\phantom{-}$0.01	$\phantom{-}$0.76	$\phantom{-}$0.77	$\phantom{-}$0.75	$\phantom{-}$0.74	$\phantom{-}$0.19	$\phantom{-}$0.19	$\phantom{-}$0.17	$\phantom{-}$0.12	−8.56	$\phantom{-}$3.12
R31P08M	MC	$\phantom{-}$0.19	$\phantom{-}$0.79	$\phantom{-}$0.02	$\phantom{-}$1.19	$\phantom{-}$1.21	$\phantom{-}$1.13	$\phantom{-}$1.16	−1.52	−1.51	−1.53	−1.58	−5.75	−0.01
R31P10M	MC	$\phantom{-}$0.11	$\phantom{-}$0.87	$\phantom{-}$0.03	$\phantom{-}$1.88	$\phantom{-}$1.89	$\phantom{-}$1.74	$\phantom{-}$1.81	−1.66	−1.65	−1.65	−1.69	−4.80	−1.29
R31P11M	MC	$\phantom{-}$0.27	$\phantom{-}$0.70	$\phantom{-}$0.03	$\phantom{-}$1.07	$\phantom{-}$1.07	$\phantom{-}$1.04	$\phantom{-}$1.04	−1.05	−1.05	−1.05	−1.09	−5.18	−0.78
R31P12M	MC	$\phantom{-}$0.33	$\phantom{-}$0.64	$\phantom{-}$0.03	$\phantom{-}$1.02	$\phantom{-}$1.03	$\phantom{-}$1.02	$\phantom{-}$1.00	−0.77	−0.76	−0.75	−0.80	−5.33	−0.32
R31P13M	MC	$\phantom{-}$0.09	$\phantom{-}$0.88	$\phantom{-}$0.03	$\phantom{-}$1.59	$\phantom{-}$1.60	$\phantom{-}$1.49	$\phantom{-}$1.54	−1.97	−1.96	−1.90	−1.99	−4.53	−1.28

Note. Type  = item format; CR = constructed response item; MC = multiple-choice item; MI = manifest ignorabiity; LI = latent ignorability; WR = treating missing item responses as wrong (i.e., 0); MW = Mislevy-Wu model.

, estimated item parameters of the regularized Mislevy-Wu model are displayed. Notably, the average $\delta$ parameters for CR items ($M=-2.01,Med=-1.65,$ $SD=1.44$) were lower than MC items ($M=0.60,Med=0.03,SD=2.06$). Moreover, the average ${\beta }_i$ parameter was larger for CR items ($M=-2.70,Med=-2.89,SD=0.85$) than for MC items ($M=-6.55,Med=-5.75,SD=1.84$), reflecting that the missing proportions for CR items were larger than for MC items.

It is also noteworthy that item discriminations $a_i$ hardly varied between the MI and LI model (model MI for CR items: $M=1.41,Med=1.25,SD=0.74$; model LI for CR items: $M=1.41,Med=1.25,SD=0.75$). However, item discriminations $a_i$ were larger for models WR ($M=1.62,Med=1.29,SD=1.06$) and MW ($M=1.58,Med=1.26,$ $SD=1.09$). Similarly, item difficulties $b_i$ did not show practical differences between MI and LI models (model MI for CR items: $M=-0.25,Med=-0.24,SD=0.96$; model LI for CR items: $M=-0.24,Med=-0.22,SD=0.97$). In line with expectations, the MW model (CR items: $M=-0.14,Med=-0.12,SD=0.93$) and the WR model resulted in larger item difficulties (CR items: $M=-0.07,Med=-0.03,SD=0.97$). The pattern was similar for MC items but less pronounced because the missing proportion rates were smaller for MC items compared to CR items.

In Figure 3, the negative cross-validated log-likelihood value is displayed as a function of the regularization parameter $\lambda$. For sufficiently small $\lambda$ values, there is almost no difference in cross-validated log-likelihood values. The optimal regularization parameter was estimated as ${\lambda }_{\mathrm{opt}}=0.0004342$.

In Figure 4, estimated ${\delta }_i$ item parameters are displayed as a function of the regularization parameter $\lambda$ for CR and MC items, respectively. With increasing $\lambda$ parameters, item parameters are fused to item-format-specific parameters. The fused values were ${\delta }_i=-3.18$ for CR items and ${\delta }_i=-0.79$ for MC items. This result indicated that missing item responses for CR items are more likely associated with a wrong item response than for MC items. Notably, the fused ${\delta }_i$ parameters were both negative.

In Figure 5, estimated ${\beta }_i$ item parameters are displayed as a function of the regularization parameter $\lambda$ for CR and MC items, respectively. The regularization of the ${\delta }_i$ also affected the estimated ${\beta }_i$ parameters, particularly for MC items.

Finally, Figure 6 and Figure 7 display the $a_i$ and $b_i$ parameters as a function of the regularization parameter $\lambda$. The target item parameters are hardly affected for small values of the regularization parameter $\lambda$, but show some variation for $\lambda$ parameters larger than ${10}^{-2}$.

5. Discussion

In this article, we proposed a regularization estimation approach to the Mislevy-Wu model. This approach allows sufficiently complex missingness mechanisms as well as estimation in moderate sample sizes such as $N=1000$. Interestingly, the most stable item parameter estimates in terms of RMSE were obtained for values of the regularization parameters that were larger than the one obtained by k-fold cross-validation based on the log-likelihood function value.

To further stabilize estimation, the fused ridge penalty function could also involve the ${\beta }_i$ parameters because they are also difficult to estimate for items with low missing proportion rates or in moderate sample sizes.

It has been shown in the PIRLS 2011 application that the Mislevy-Wu model outperformed all other estimation approaches. Omissions on constructed response items were strongly associated with true item responses. This implies that students who do not know an item likely do not respond to it  [33]. In contrast, multiple-choice items were only weakly associated with true but non-fully observed item responses. Given these findings, it seems plausible in large-scale assessment studies to score omitted constructed response items as wrong while treating multiple-choice as fractionally correct in a pseudo-likelihood estimation approach [44]. In the latter case, a multiple-choice item with $K_i$ answer alternative is scored with $1/K_i$.

It could be generally argued that constructed response items are omitted more to a lack of knowledge than multiple-choice items. In this sense, as argued by an anonymous reviewer, omissions on constructed response items are likely missing not at random data. In contrast, multiple-choice items could be regarded as missing at random data. In practice, the tendency to omit items can be associated with person traits [45].

The Mislevy-Wu model can be easily extended to item response models for polytomous items. For dichotomous items, the dependence of response indicators $R_i$ from true item responses $X_i$ is modeled by the item parameter ${\delta }_i$. For polytomous items scored between 0 and $K_i$, $K_i$ parameters ${\delta }_{i,k}$ ($k=1,\dots ,K_i$) that differentially weigh the impact of item category k on the response indicator can be identified from the data.

The Mislevy-Wu model can be extended to include covariates for predicting the latent ability ${\theta }_p$ and the latent response propensity ${\xi }_p$ [46]. In a latent regression model  [47,48], the estimated item parameters could be fixed, and IRT packages such as TAM  [49] could be utilized for estimation. Such an approach could also be applied in providing plausible values  [50] in LSA studies as realizations of the latent ability ${\theta }_p$ that can be used for secondary analysis. In this sense, the Mislevy-Wu model can be implemented in operational practice when scaling item responses in LSA studies such as PISA, PIRLS, or TIMSS [51].

Missing item responses are typically classified into omitted and not-reached item responses [52]. In this article, we only investigated omitted item responses within a test. For speeded tests, it might be preferable not to score not-reached item responses as wrong. However, large-scale assessment studies like PIRLS are not strongly speeded such that there is only a low prevalence of not-reached items.

The Mislevy-Wu model follows a model-based strategy in which the missingness mechanism for the response indicators is simultaneously modeled with the item response model (e.g., 2PL model) of interest. It might be beneficial to weaken the assumption of a unidimensional ability variable $\theta$ and unidimensional response propensity variable $\xi$ and to estimate multidimensional variables with an exploratory loading structure  [35] in an imputation model. In this case, the imputation model is more complex than the intentionally misspecified analysis model  [17,53]. Certainly, such an estimation approach would need even larger sample sizes, and regularized estimation could also be applied to the exploratory loading structure.

Although modeling missingness mechanisms in educational studies now receive wide attention, only in rare cases, the dependence of item omissions from the item itself is considered a viable alternative (e.g., see [6]). This is unfortunate because we empirically demonstrated that there are several studies in which treating constructed response items as wrong [13] instead of latent ignorable (i.e., as missing; [6]) resulted in superior model fit. The Mislevy-Wu model contains these two extreme scoring treatments as particular constrained models and also parameterizes processes that are a mixture of both. Hence, if missing item responses should be modeled in large-scale assessment studies, there is no excuse for neglecting the Mislevy-Wu model from the preferred psychometrician’s toolkit.