Linking Error in the 2PL Model

Abstract

The two-parameter logistic (2PL) item response model is likely the most frequently applied item response model for analyzing dichotomous data. Linking errors quantify the variability in means or standard deviations due to the choice of items. Previous research presented analytical work for linking errors in the one-parameter logistic model. In this article, we present linking errors for the 2PL model using the general theory of M-estimation. Linking errors are derived in the case of log-mean-mean linking for linking two groups. The performance of the newly proposed formulas is evaluated in a simulation study. Furthermore, the linking error estimation in the 2PL model is also treated in more complex settings, such as chain linking, trend estimation, fixed item parameter calibration, and concurrent calibration.

1. Introduction

Item response theory (IRT) models [1,2] are an important class of multivariate statistical models for analyzing dichotomous random variables used to model testing data from educational or psychological applications. Of particular relevance is the application of item response models in educational large-scale assessments [3], such as the programme for international student assessment (PISA; [4]) study.

In this article, we only investigate unidimensional IRT models. Let $\mathit{X}=(X_1,\dots ,X_I)$ be the vector of I dichotomous random variables $X_i\in \{0,1\}$ (also referred to as items). A unidimensional item response model [5] is a statistical model for the probability distribution $\mathrm{P}(\mathit{X}=\mathit{x})$ for $\mathit{x}=(x_1,\dots ,x_I)\in {\{0,1\}}^I$, where

$${\displaystyle \mathrm{P}(\mathit{X}=\mathit{x};\mathit{\delta },\mathit{\gamma })={\int }_{-\infty }^{\infty }\prod _{i=1}^I\left[{P_i(\theta ;{\mathbf{\gamma }}_i)}^{x_i}{\left(1-P_i(\theta ;{\mathit{\gamma }}_i)\right)}^{1-x_i}\right]\varphi (\theta ;\mu ,\sigma )\mathrm{d}\theta ,}$$

(1)

where $\varphi$ is the density of the normal distribution with mean $\mu$ and standard deviation $\sigma$. The vector $\mathit{\delta }=(\mu ,\sigma )$ contains the distribution parameters. The vector $\mathbf{\gamma }=({\mathbf{\gamma }}_1,\dots ,{\mathbf{\gamma }}_I)$ contains all estimated item parameters of item response functions $P_i(\theta ;{\mathbf{\gamma }}_i)=\mathrm{P}(X_i=1|\theta )$.

The one-parameter logistic (1PL) model (also referred to as the Rasch model; [6]) employs the item response function $P_i(\theta )=\Psi (\theta -b_i)$, where $\Psi$ denotes the logistic distribution function, and $b_i$ is the item difficulty of item i. In this case, ${\mathbf{\gamma }}_i=(b_i)$. The two-parameter logistic (2PL) model [7] additionally includes the item discrimination $a_i$ (i.e., ${\mathbf{\gamma }}_i=(a_i,b_i)$), and the item response function is given by $P_i(\theta )=\Psi \left(a_i(\theta -b_i)\right)$.

Note that distribution parameters $\mathit{\delta }$ and item parameters $\mathbf{\gamma }$ cannot be simultaneously identified. In applications such as PISA in which a country mean $\mu$ and a country standard deviation $\sigma$, item parameters ${\mathbf{\gamma }}_i$ are often fixed at values ${\mathbf{\gamma }}_i^{*}$ that are used for all countries. In this situation, $\mu$ and $\sigma$ can be identified. If sample data ${\mathit{X}}_1,\dots ,{\mathit{X}}_N$ for N persons are available, unknown model parameters in (1) can be estimated by (marginal) maximum likelihood (ML) using an expectation maximization algorithm [8,9].

In practice, data-generating item parameters ${\mathbf{\gamma }}_i$ differ from assumed fixed item parameters ${\mathbf{\gamma }}_i^{*}$. This property is also referred to as differential item functioning (DIF; [10,11]). DIF effects ${\mathit{e}}_i$ are defined as deviations ${\mathit{e}}_i={\mathbf{\gamma }}_i-{\mathbf{\gamma }}_i^{*}$. The occurrence of DIF causes additional variability in the estimated (country) mean $\mu$ and standard deviation $\sigma$ [12,13]. The estimated distribution parameters depend on the choice of selected items, even in infinite sample sizes of persons. This variability is quantified in the linking error [14,15,16,17,18,19].

There exist simple formulas for linking errors based on variance components for the 1PL model [15,17]. For more complex models, resampling techniques [20,21] such as jackknife [15,17] or the (balanced) half sampling [18] of items can be employed. In this article, we provide closed formulas for the linking error for the 2PL model in various applications based on the M-estimation theory. The proposed formulas have the advantage of avoiding computationally more demanding resampling approaches for computing linking errors.

The rest of this article is structured as follows. The foundation of the M-estimation theory for the application of the computation of the linking error in the 2PL model is presented in Section 2. The specialization of M-estimation to log-mean-mean linking is treated in Section 3. The performance of the newly proposed linking errors is investigated in a simulation study in Section 4. Section 5 provides a further analytical treatment of the linking error in the 2PL model to more complex applications. Finally, this article closes with a discussion in Section 6.

2. Linking Error and M-Estimation

In this section, we discuss the computation of the linking error in the 2PL model for two groups. We do this in a general setting of M-estimation theory [22,23,24] because our treatment will apply to many of the recently discussed linking methods. However, we focus on log-mean-mean linking in this article as an important example in Section 3 and Section 4.

Assume that the 2PL model holds in two groups $g=1,2$ or two time points. The goal is to determine the mean $\mu$ and the standard deviation $\sigma$ of the second group, while the first group is assumed to have a mean of 0 and a standard deviation of 1. The DIF effects $f_i$ and $e_i$ for logarithmized item discriminations and item difficulties follow

$$\begin{array}{ccc}log{a}_{i2}& =& log{a}_{i1}+f_i\hfill \\ b_{i2}& =& b_{i1}+e_i\hfill \end{array}.$$

(2)

It is assumed that $f_i$ and $e_i$ are independently and identically distributed with zero means and variances ${\tau }_a^2$, ${\tau }_b^2$, and the covariance is defined as $\mathrm{Cov}(e_i,f_i)={\tau }_{ab}$.

In the first step of the linking approach, the 2PL model is separately estimated in each of the two groups. Because the ability $\theta$ for the first group has zero mean and a standard deviation of 1, the identified item parameters ${\widehat{a}}_{i1}$ and ${\widehat{b}}_{i1}$ equal the data-generating item parameters $a_{i1}$ and $b_{i1}$, respectively, ($i=1,\dots ,I$). In the second group, we fix the mean to 0 and the standard deviation to 1 and obtain identified parameters

$${\widehat{a}}_{i2}=\sigma a_{i2}\mathrm{and}{\widehat{b}}_{i2}={\sigma }^{-1}\left(b_{i2}-\mu \right).$$

(3)

In the second step of the linking approach, identified item parameters $\{({\widehat{a}}_{i1},{\widehat{b}}_{i1})\}$ and $\{({\widehat{a}}_{i2},{\widehat{b}}_{i2})\}$ are used in determining the mean $\mu$ and the standard deviation $\sigma$ in the second group. Note that we assume that identified item parameters are known. Hence, we implicitly have infinite sample sizes of persons. In practice, we estimate item parameters from finite sample sizes. Appropriate adjustments are discussed in Section 5.8.

A general estimating equation of the type

$$\sum _{i=1}^I\mathit{g}(\delta ;{\widehat{a}}_{i1},{\widehat{b}}_{i1},{\widehat{a}}_{i2},{\widehat{b}}_{i2})=\sum _{i=1}^I\mathit{g}(\delta ;a_{i1},b_{i1},f_i,e_i)=\mathbf{0}$$

(4)

is employed for determining the parameter $\mathit{\delta }=(\sigma ,\mu )$ or $\mathit{\delta }=(s,\mu )$ with $s=log\sigma$ as the distribution parameters of interest. M-estimation theory provides the asymptotic variance in an estimate $\widehat{\mathit{\delta }}$. Because linking errors refer to the uncertainty regarding item choice, this asymptotic variance can be used to compute the linking error.

In the two-group 2PL case, we have two unknown distribution parameters $\sigma$ (or $s=log\sigma$) and $\mu$. Hence, $\mathit{g}=(g_1,g_2)$ involves two equations for two unknowns that must be solved. The two M-estimation equations of the linking approaches can be generally written as

$$\begin{array}{ccc}{\displaystyle \sum _{i=1}^I{g}_1(\delta ;a_{i1},b_{i1},f_i,e_i)}\hfill & =& 0\\ {\displaystyle \sum _{i=1}^I{g}_2(\delta ;a_{i1},b_{i1},f_i,e_i)}\hfill & =& 0\end{array}$$

(5)

M-estimation theory provides the asymptotic variance (for $I\to \infty$) for the estimate $\widehat{\mathit{\delta }}$ with the sandwich formula [22]

$${\mathit{V}}_I(\widehat{\mathit{\delta }})={\mathit{A}}_I{(\widehat{\mathit{\delta }})}^{-1}{\mathit{B}}_I(\widehat{\mathit{\delta }}){\mathit{A}}_I{(\widehat{\mathit{\delta }})}^{-1}.$$

(6)

The matrix ${\mathit{A}}_I$ is denoted as the bread matrix, while the matrix ${\mathit{B}}_I$ is referred to as the meat matrix. The latter matrix is given by

$${\mathit{B}}_I(\mathit{\delta })=\sum _{i=1}^I\mathrm{Var}\left(\mathit{g}(\mathit{\delta };a_{i1},b_{i1},f_i,e_i)\mathit{g}{(\mathit{\delta };a_{i1},b_{i1},f_i,e_i)}^{\top }\right),$$

(7)

where $\mathrm{Var}$ denotes a covariance matrix. The bread matrix ${\mathit{A}}_I$ is given as

$${\mathit{A}}_I(\mathit{\delta })=\sum _{i=1}^I\mathrm{E}\left(\frac{\partial \mathit{g}}{\partial \mathit{\delta }}(\mathit{\delta };a_{i1},b_{i1},f_i,e_i)\right).$$

(8)

The two matrices in Equations (7) and (8) require the computation of expected values and variances of random variables. If these were unavailable or the quantities could not be algebraically determined, sample-based versions of the bread and the meat matrix are frequently used [23]. The meat matrix ${\mathit{B}}_I$ can be estimated based on sample data using (7)

$${\widehat{\mathit{B}}}_I(\widehat{\mathit{\delta }})=\sum _{i=1}^I\mathit{g}(\widehat{\mathit{\delta }};{\widehat{a}}_{i1},{\widehat{b}}_{i1},{\widehat{a}}_{i2},{\widehat{b}}_{i2})\mathit{g}{(\widehat{\mathit{\delta }};{\widehat{a}}_{i1},{\widehat{b}}_{i1},{\widehat{a}}_{i2},{\widehat{b}}_{i2})}^{\top }.$$

(9)

An empirical version of ${\mathit{A}}_I$ is given by

$${\widehat{\mathit{A}}}_I(\widehat{\mathit{\delta }})=\sum _{i=1}^I\frac{\partial \mathit{g}}{\partial \mathit{\delta }}(\widehat{\mathit{\delta }};{\widehat{a}}_{i1},{\widehat{b}}_{i1},{\widehat{a}}_{i2},{\widehat{b}}_{i2}).$$

(10)

If ${\mathit{A}}_I(\widehat{\mathit{\delta }})$ and ${\mathit{B}}_I(\widehat{\mathit{\delta }})$ are used for computing the variance matrix for $\widehat{\mathit{\delta }}$ (i.e., ${\mathit{V}}_{I,\mathrm{ESW}}(\widehat{\mathit{\delta }})$), the estimate is denoted as the expected sandwich (ESW) estimate. The observed sandwich (OSW) estimate is obtained by using ${\widehat{\mathit{A}}}_I(\widehat{\mathit{\delta }})$ and ${\widehat{\mathit{B}}}_I(\widehat{\mathit{\delta }})$ in the formula of the variance matrix (i.e., ${\mathit{V}}_{I,\mathrm{OSW}}(\widehat{\mathit{\delta }})$). Finally, a bias-corrected observed sandwich (BOSW) is obtained by using ${\mathit{V}}_{I,\mathrm{BOSW}}(\widehat{\mathit{\delta }})=I/(I-1){\mathit{V}}_{I,\mathrm{OSW}}(\widehat{\mathit{\delta }})$ (see [25,26,27]).

We want to emphasize that M-estimation theory is not restricted to applications of linking approaches for two groups and two distribution parameters. The parameter $\mathit{\delta }$ can be of any finite dimensionality and could, for example, involve $2(G-1)$ unknown parameters for linking G groups.

M-estimation theory was applied in the investigation of DIF and linking in [28,29,30]. The simultaneous treatment of standard errors and linking errors in IRT models relying on M-estimation was presented in [18,31] (see also [32]).

3. Linking Error of Log-Mean-Mean Linking

We now apply the M-estimation of computing linking errors to log-mean-mean linking [33,34] for linking two groups in the 2PL model. The logarithm of the standard deviation $s=log\sigma$ is estimated by

$$\widehat{s}=\frac{1}{I}\sum _{i=1}^I\left(log{\widehat{a}}_{i2}-log{\widehat{a}}_{i1}\right).$$

(11)

It can be shown that $\widehat{s}$ is an unbiased and consistent estimate for s [34]. Moreover, $\widehat{\sigma }$ is obtained by computing $\widehat{\sigma }=exp(\widehat{s})$.

An estimate of the group mean $\mu$ is obtained using

$$\widehat{\mu }=\frac{1}{I}\sum _{i=1}^I\left(exp(\widehat{s}){\widehat{b}}_{i2}-{\widehat{b}}_{i1}\right).$$

(12)

We can reformulate (11) and (12) as M-estimators

$$\sum _{i=1}^I{g}_1(\delta ;a_{i1},b_{i1},f_i,e_i)=\sum _{i=1}^I\left(log{\widehat{a}}_{i2}-log{\widehat{a}}_{i1}-s\right)=0\mathrm{and}$$

(13)

$$\sum _{i=1}^I{g}_2(\delta ;a_{i1},b_{i1},f_i,e_i)=\sum _{i=1}^I\left(exp(s){\widehat{b}}_{i2}-{\widehat{b}}_{i1}+\mu \right)=0$$

(14)

using $\mathit{g}=(g_1,g_2)$. Now, we determine the linking errors for $\widehat{\sigma }$ and $\widehat{\mu }$ in log-mean-mean linking using the sandwich formula (6) of M-estimation. First, we compute the variance matrix of $\mathit{g}$. We obtain using ${\widehat{a}}_{i2}=a_{i1}exp(s)exp(f_i)$ and ${\widehat{b}}_{i2}={\sigma }^{-1}(b_{i1}-\mu )$

$$\sum _{i=1}^I\mathrm{Var}\left(g_1(\delta ;a_{i1},b_{i1},f_i,e_i)\right)=\sum _{i=1}^I\mathrm{Var}\left(log{\widehat{a}}_{i2}-log{\widehat{a}}_{i1}-s\right)=\sum _{i=1}^I\mathrm{Var}\left(f_i\right)=I{\tau }_a^2,$$

(15)

$$\sum _{i=1}^I\mathrm{Cov}\left(g_1(\delta ;a_{i1},b_{i1},f_i,e_i),g_2(\delta ;a_{i1},b_{i1},f_i,e_i)\right)=\sum _{i=1}^I\mathrm{Cov}\left(f_i,e_i\right)=I{\tau }_{ab},\mathrm{and}$$

(16)

$$\sum _{i=1}^I\mathrm{Var}\left(g_2(\delta ;a_{i1},b_{i1},f_i,e_i)\right)=\sum _{i=1}^I\mathrm{Var}\left(e_i\right)=I{\tau }_b^2.$$

(17)

Hence, it follows that

$${\mathit{B}}_I(\widehat{\mathit{\delta }})=I\left(\begin{array}{cc}{\tau }_a^2& {\tau }_{ab}\\ {\tau }_{ab}& {\tau }_b^2\end{array}\right)$$

(18)

Now, we compute derivatives of $\mathit{g}$ with respect to s and $\mu$ and obtain the bread matrix as

$${\mathit{A}}_I(\widehat{\mathit{\delta }})=\left(\begin{array}{cc}{\displaystyle \frac{\partial g_1}{\partial s}}& {\displaystyle \frac{\partial g_1}{\partial \mu }}\\ {\displaystyle \frac{\partial g_2}{\partial s}}& {\displaystyle \frac{\partial g_2}{\partial \mu }}\end{array}\right)=I\left(\begin{array}{cc}-1& 0\\ -(\mu -{\overline{b}}_{•1})& 1\end{array}\right)$$

(19)

The inverse of the bread matrix can be determined as

$${\mathit{A}}_I{(\widehat{\mathit{\delta }})}^{-1}=-I^{-1}\left(\begin{array}{cc}1& 0\\ \mu -{\overline{b}}_{•1}& -1\end{array}\right)$$

(20)

Formulas (18) and (19) involve unknown parameters ${\tau }_a$, ${\tau }_b$, ${\tau }_{ab}$, and $\mu$ that must be estimated. The quantities $f_i$ and $e_i$ can be replaced with their sample estimates to estimate unknown variances and covariances in the ${\mathit{A}}_I$ and ${\mathit{B}}_I$ matrices. We obtain the expected sandwich estimator (ESW) ${\mathit{V}}_{I,\mathrm{ESW}}$ using Equation (6) by

$${\mathit{A}}_{I,\mathrm{ESW}}(\widehat{\mathit{\delta }})=I\left(\begin{array}{cc}-1& 0\\ -(\widehat{\mu }-{\overline{b}}_{•1})& 1\end{array}\right)\mathrm{and}{\mathit{B}}_{I,\mathrm{ESW}}(\widehat{\mathit{\delta }})=I\left(\begin{array}{cc}{\widehat{\tau }}_a^2& {\widehat{\tau }}_{ab}\\ {\widehat{\tau }}_{ab}& {\widehat{\tau }}_b^2\end{array}\right).$$

(21)

The observed sandwich (OSW) estimate ${\mathit{V}}_{I,\mathrm{OSW}}$ uses empirical moments in the estimation. First, the bread matrix is estimated by

$${\mathit{A}}_{I,\mathrm{OSW}}(\widehat{\mathit{\delta }})=\left(\begin{array}{cc}-I& {\displaystyle 0}\\ {\displaystyle exp(s)\sum _{i=1}^I{\widehat{b}}_{i2}}& {\displaystyle I}\end{array}\right).$$

(22)

The meat matrix is estimated by

$${\mathit{B}}_{I,\mathrm{OSW}}(\widehat{\mathit{\delta }})=\left(\begin{array}{cc}{\displaystyle \sum _{i=1}^I{\left(log{\widehat{a}}_{i2}-log{\widehat{a}}_{i1}-\widehat{s}\right)}^2}& {\displaystyle \sum _{i=1}^I(log{\widehat{a}}_{i2}-log{\widehat{a}}_{i1}-\widehat{s})(exp(\widehat{s}){\widehat{b}}_{i2}-{\widehat{b}}_{i1}+\widehat{\mu })}\\ {\displaystyle \sum _{i=1}^I(log{\widehat{a}}_{i2}-log{\widehat{a}}_{i1}-\widehat{s})(exp(\widehat{s}){\widehat{b}}_{i2}-{\widehat{b}}_{i1}+\widehat{\mu })}& {\displaystyle \sum _{i=1}^I{(exp(\widehat{s}){\widehat{b}}_{i2}-{\widehat{b}}_{i1}+\widehat{\mu })}^2}\end{array}\right)$$

(23)

Finally, we use a bias-corrected variant of the observed sandwich estimator [27] as

$${\mathit{V}}_{I,\mathrm{BOSW}}=\frac{I}{I-1}{\mathit{V}}_{I,\mathrm{OSW}}.$$

(24)

Linking errors can be obtained as square roots of the diagonal elements of the variance matrices $\mathit{V}$. The linking error for $\widehat{s}$ based on the ESW estimate is given by

$$\mathrm{LE}(\widehat{s})=\frac{{\widehat{\tau }}_a}{\sqrt{I}}.$$

(25)

By utilizing the delta method, we can obtain the linking error for $\widehat{\sigma }=exp(\widehat{s})$ as

$$\mathrm{LE}(\widehat{\sigma })=\widehat{\sigma }\frac{{\widehat{\tau }}_a}{\sqrt{I}}.$$

(26)

Finally, the linking error for $\widehat{\mu }$ can be estimated by

$$\mathrm{LE}(\widehat{\mu })=\sqrt{\frac{{\widehat{\tau }}_b^2+{(\widehat{\mu }-{\overline{b}}_{•1})}^2{\widehat{\tau }}_a^2-2(\widehat{\mu }-{\overline{b}}_{•1}){\widehat{\tau }}_{ab}}{I}}.$$

(27)

In the absence of a nonuniform DIF, we have ${\widehat{\tau }}_a^2={\widehat{\tau }}_{ab}=0$, and the linking error for the 1PL model is obtained from (27)

$$\mathrm{LE}(\widehat{\mu })=\frac{{\widehat{\tau }}_b}{\sqrt{I}}.$$

(28)

Interestingly, the presence of a nonuniform DIF (${\tau }_a^2>0$) introduces additional uncertainty in computing the group mean. However, there is only an effect of a nonuniform DIF if the average item difficulty does not match the group mean (i.e., $\widehat{\mu }-{\overline{b}}_{•1}\ne 0$). In typical applications, the third term $2(\widehat{\mu }-{\overline{b}}_{•1}){\widehat{\tau }}_{ab}$ in (27) will be much more important than the second term ${(\widehat{\mu }-{\overline{b}}_{•1})}^2{\widehat{\tau }}_a^2$. Hence, a nonuniform DIF particularly plays an important role if uniform and nonuniform DIF effects are strongly correlated.

4. Simulation Study

4.1. Method

In this simulation study, we investigate the performance of different linking error estimates for the log-mean-mean linking approach in the 2PL model. In particular, we compare the jackknife linking error (JK) with linking errors obtained by the empirical sandwich (ESW), observed sandwich (OSW), and the bias-corrected observed sandwich (BOSW) estimates. The formulas for the sandwich estimates are presented in Section 3. The jackknife linking error estimate is computed by repeating the linking procedure when omitting the ith item for $i=1,\dots ,I$. Let $u=\mu$ or $u=\sigma$ be the distribution parameter of interest and $\widehat{u}$ be the corresponding estimate. Let ${\widehat{u}}_{(-i)}$ be the estimated parameter if item i was removed from the linking procedure. Then, the jackknife linking error is defined as (see [15])

$$\mathrm{LE}(\widehat{u})=\sqrt{\frac{I-1}{I}\sum _{i=1}^I{({\widehat{u}}_{(-i)}-\widehat{u})}^2}$$

(29)

For identification, the first group had a zero mean and a standard deviation of the ability variable $\theta$. For the second group, we defined $\mu =-0.2$ and $\sigma =0.9$ in the simulation. Item parameters for 10 items are presented in

Table 1. Simulation Study: Used item parameters of the 2PL model.

Item	${\mathit{a}}_{\mathit{i}}$	${\mathit{b}}_{\mathit{i}}$
1	0.73	−1.31
2	1.25	$\phantom{-}$1.44
3	1.20	−1.20
4	1.47	$\phantom{-}$0.10
5	0.97	$\phantom{-}$0.10
6	1.38	−0.74
7	1.05	$\phantom{-}$1.48
8	1.14	−0.61
9	1.15	$\phantom{-}$0.82
10	0.67	−0.07

Note. a_i = item discrimination; b_i = item difficulty.

. In the simulation, we used $I=10,20,40$, or 80 items. For item numbers as multiples of 10, we duplicated the item parameters of the 10 items presented in Table 1 accordingly. The standard deviation of uniform DIF effects $e_i$ was chosen as ${\tau }_b=0.25$ or 0.50. The standard deviation of nonuniform DIF effects $f_i$ was chosen as ${\tau }_a=0.01$ or 0.25. The first condition mimics the case of the practical absence of nonuniform DIF effects. The correlation of DIF effects between $e_i$ and $f_i$ was set at 0.3 in all simulation conditions (i.e., ${\tau }_{ab}=0.3·{\tau }_a{\tau }_b$). Finally, we chose three types of distributions for DIF effects $(f_i,e_i)$. We specified them as a bivariate normal copula model and chose different marginal distributions. First, we chose the normal distribution (i.e., denoted as “Normal”) as a marginal distribution appropriately scaled by ${\tau }_a$ and ${\tau }_b$. Second, we chose a scaled t distribution with four degrees of freedom (i.e., denoted as “$t_4$”) with an appropriate scaling factor to match the desired standard deviation of DIF effects. Third, we use the distribution function F of a normal mixture model (i.e., denoted as “Normal Mixture”) of the type

$$F=(1-\epsilon )\mathrm{N}(0,{\tau }^2)+\epsilon \mathrm{N}(0,k{\tau }^2),$$

(30)

where $k=3$ and $\epsilon =0.05$. This distribution can be interpreted as a contaminated distribution that includes a few outlying DIF effects in $\mathrm{N}(0,k{\tau }^2)$ with proportion $\epsilon$. Such a distribution is often employed in robust statistics [35]. For a prespecified DIF effect ${\tau }_b$, we obtain from (30) the determining equation $\tau ={\left((1-\epsilon )+\epsilon k^2\right)}^{-1/2}{\tau }_a$.

To disentangle standard errors due to the sampling of persons from linking errors due to item choice, we assumed no sampling error for identified parameters $\{({\widehat{a}}_{i1},{\widehat{b}}_{i1})\}$ and $\{({\widehat{a}}_{i2},{\widehat{b}}_{i2})\}$. That is, identified item parameters for the second group only vary across replications in the simulation study because different DIF effects $f_i$ and $e_i$ were simulated in each replication. It seems reasonable in the simulation study in the comparison of the different M-estimation approaches with a jackknife to exclude the effects of sampling errors because these are just another source of uncertainty in distribution parameter estimates.

In each of the $4(I)\times 2({\tau }_b)\times 2({\tau }_a)\times 3(\mathrm{distribution})=48$ cells of the simulation, 40,000 replications were conducted. We assessed coverage rates at the 95% confidence level based on the normal distribution for distribution parameter estimates. The linking error computation for the estimated standard deviation $\widehat{\sigma }=exp(\widehat{s})$ utilized the delta method.

The R software [36] was used for simulation and analysis. We used the qmixnorm function from the R package KScorrect [37] for determining quantiles in the data simulation of DIF effects. Because analytical solutions are not available to compute a quantile function for the normal mixture model, the qmixnorm function approximates the quantile function using a spline function calculated from cumulative density functions for the specified mixture distribution [37]. Quantiles for probabilities near zero or one are approximated by taking a randomly generated sample.

4.2. Results

In

Table 2. Simulation Study: Coverage rates for estimated mean $\widehat{\mu }$ as a function of the standard deviation of DIF effects for a (${\tau }_a$) and b (${\tau }_b$), number of items (I), and the type of distribution for DIF effects.

			Normal				${\mathit{t}}_4$				Normal Mixture
${\mathit{\tau }}_{\mathit{a}}$	${\mathit{\tau }}_{\mathit{b}}$	I	JK	ESW	OSW	BOSW	JK	ESW	OSW	BOSW	JK	ESW	OSW	BOSW
0.01	0.25	10	92.2	92.2	90.8	92.1	92.9	92.9	91.5	92.9	92.7	92.7	91.3	92.7
		20	93.8	93.8	93.2	93.8	94.5	94.5	93.8	94.5	94.4	94.4	93.8	94.4
		40	94.6	94.6	94.3	94.6	94.9	94.9	94.5	94.8	94.9	94.9	94.6	94.9
		80	95.2	95.2	95.0	95.1	95.2	95.2	95.0	95.1	95.2	95.2	95.1	95.2
0.01	0.50	10	92.5	92.5	91.2	92.5	92.9	92.9	91.5	92.9	93.1	93.1	91.7	93.0
		20	94.1	94.1	93.5	94.1	94.6	94.6	94.0	94.6	94.4	94.4	93.8	94.4
		40	94.7	94.7	94.4	94.7	95.0	95.0	94.7	95.0	94.8	94.8	94.5	94.8
		80	95.1	95.1	94.9	95.0	95.3	95.3	95.1	95.3	95.4	95.4	95.2	95.3
0.25	0.25	10	93.9	93.8	91.1	92.6	94.6	94.4	92.0	93.4	94.4	94.3	91.8	93.1
		20	94.8	94.8	93.1	93.8	94.9	94.9	93.2	93.9	95.1	95.1	93.4	94.0
		40	95.1	95.1	93.7	94.0	95.5	95.5	94.0	94.4	95.2	95.2	93.6	94.0
		80	95.2	95.2	93.9	94.0	95.4	95.4	94.2	94.4	95.5	95.5	94.3	94.4
0.25	0.50	10	93.0	92.9	90.9	92.3	93.7	93.6	91.6	93.0	93.5	93.5	91.3	92.7
		20	94.3	94.3	93.1	93.8	94.4	94.4	93.2	93.8	94.5	94.5	93.3	94.0
		40	95.0	95.0	94.2	94.6	95.1	95.1	94.3	94.7	95.1	95.1	94.3	94.6
		80	95.2	95.2	94.6	94.8	95.2	95.1	94.6	94.7	95.1	95.1	94.4	94.6

Note. Normal = DIF effects normally distributed; t₄ = DIF effects distributed according to scaled t₄ distribution; Normal Mixture = DIF effects distributed according to contaminated mixture model; JK = linking error (LE) estimated by jackknife; ESW = LE estimated by expected sandwich estimator; OSW = LE estimated by observed sandwich estimator; BOSW = LE estimated by bias-corrected observed sandwich estimator; coverage rates smaller than 92.5% or larger than 97.5% are printed in bold.

, the coverage rates for the estimated mean $\widehat{\mu }$ are presented as a function of the standard deviation of the DIF effects, the number of items, and the type of distribution for the DIF effects. It turned out that there were no substantial differences in the performance of the different linking error methods with respect to the distribution types of the DIF effects. The jackknife and the ESW estimates were very similar. The OSW estimate did not reach the desired coverage rates in a short test (i.e., $I=10$) but improved in longer tests. Moreover, the BOSW slightly improved the OSW estimate but still was inferior to the ESW estimate.

In

Table 3. Simulation Study: Coverage rates for estimated standard deviation $\widehat{\sigma }$ as a function of the standard deviation of DIF effects for a (${\tau }_a$) and b (${\tau }_b$), number of items (I), and the type of distribution for DIF effects.

			Normal				${\mathit{t}}_4$				Normal Mixture
${\mathit{\tau }}_{\mathit{a}}$	${\mathit{\tau }}_{\mathit{b}}$	I	JK	ESW	OSW	BOSW	JK	ESW	OSW	BOSW	JK	ESW	OSW	BOSW
0.01	0.25	10	92.6	92.6	86.1	87.3	92.9	92.9	84.9	86.2	92.9	92.9	85.3	86.7
		20	93.8	93.8	83.0	83.8	94.5	94.5	82.1	82.8	94.4	94.4	82.5	83.3
		40	94.6	94.6	80.2	80.6	95.0	95.0	79.1	79.5	94.9	94.9	79.2	79.5
		80	95.2	95.2	77.9	78.1	95.1	95.1	76.2	76.4	95.0	95.0	76.3	76.6
0.01	0.50	10	92.1	92.2	92.3	93.0	93.2	93.2	91.8	92.6	93.0	93.0	92.2	92.9
		20	94.1	94.1	91.3	91.7	94.4	94.4	90.7	91.2	94.2	94.2	90.8	91.3
		40	94.5	94.5	91.3	91.6	94.8	94.8	90.6	90.8	94.9	94.9	90.7	90.9
		80	95.1	95.1	93.7	93.7	95.0	95.0	92.5	92.6	95.3	95.3	92.9	93.0
0.25	0.25	10	92.3	92.3	89.8	91.3	93.1	93.1	90.5	92.0	92.6	92.6	90.0	91.4
		20	93.8	93.8	92.4	93.0	94.5	94.5	93.0	93.6	94.2	94.2	92.7	93.3
		40	94.7	94.7	93.6	93.9	95.1	95.1	93.9	94.3	94.7	94.7	93.7	94.0
		80	94.9	94.9	94.0	94.2	95.2	95.2	94.3	94.5	95.2	95.2	94.3	94.5
0.25	0.50	10	92.3	92.3	87.7	89.3	93.0	92.9	88.2	89.7	92.7	92.6	88.0	89.5
		20	94.1	94.1	91.2	91.8	94.3	94.3	91.2	91.9	94.1	94.1	90.9	91.6
		40	94.8	94.8	92.7	93.0	94.9	94.9	92.7	93.0	94.9	94.9	92.7	93.1
		80	95.1	95.1	93.3	93.4	95.4	95.4	93.5	93.7	95.1	95.1	93.3	93.4

Note. Normal = DIF effects normally distributed; t₄ = DIF effects distributed according to scaled t₄ distribution; Normal Mixture = DIF effects distributed according to contaminated mixture model; JK = linking error (LE) estimated by jackknife; ESW = LE estimated by expected sandwich estimator; OSW = LE estimated by observed sandwich estimator; BOSW = LE estimated by bias-corrected observed sandwich estimator; coverage rates smaller than 92.5% or larger than 97.5% are printed in bold.

, the coverage rates for the estimated standard deviation $\widehat{\sigma }$ are presented as a function of the standard deviation of the DIF effects, the number of items, and the type of distribution for the DIF effects. The OSW and BOSW had particular issues in the coverage rates with very small nonuniform DIF effects. These issues also remained in the longer tests. However, the estimated standard deviations of the nonuniform DIF effects ${\widehat{\tau }}_a$ turned out to be unbiased and can be detectable in such situations to indicate that linking errors for $\widehat{\sigma }$ would be tiny in this situation. Hence, the practical absence of nonuniform DIF effects using the simulation condition ${\tau }_a=0.01$ might not be very realistic, and future studies could investigate the performance using ${\tau }_a=0.10$.

Overall, the findings of this simulation study indicate that the sandwich estimates (in particular the ESW) are as effective as the jackknife estimates for linking errors.

5. Further Applications of the Linking Error in the 2PL Model

In this section, several applications of the linking error computations in the 2PL model are presented. In Section 5.1, the M-estimation theory is applied to linking approaches other than the log-mean-mean linking. Section 5.2 discusses the computation of the linking errors if the items are nested within testlets. The linking errors for chain linking and trend estimation are discussed in Section 5.3 and Section 5.4, respectively. In Section 5.5, the linking error under a fixed item parameter calibration is derived. Section 5.6 presents the linking error in the 2PL model for a concurrent calibration. Section 5.7 investigates the computation of the linking errors of derived parameters. Finally, Section 5.8 focuses on the computation of the total error and sampling error corrections in the linking error estimation.

5.1. Different Linking Methods

We now illustrate how the sandwich estimates in the M-estimation theory from Section 2 can be used for other linking approaches.

5.1.1. Robust Log-Mean-Mean Linking

Log-mean-mean linking involves two steps that compute the mean for determining the logarithm of the standard deviation $s=log\sigma$ and the mean $\mu$. A few outlying items might introduce bias in the estimated distribution parameters [18]; robust estimators for the location measures can be preferred. In this case, the estimating functions for $\mathit{\delta }=(s,\mu )$ are given by

$$\sum _{i=1}^I{g}_1(\delta ;a_{i1},b_{i1},f_i,e_i)=\sum _{i=1}^I\rho \left(log{\widehat{a}}_{i2}-log{\widehat{a}}_{i1}-s\right)=0\mathrm{and}$$

(31)

$$\sum _{i=1}^I{g}_2(\delta ;a_{i1},b_{i1},f_i,e_i)=\sum _{i=1}^I\rho \left(exp(s){\widehat{b}}_{i2}-{\widehat{b}}_{i1}+\mu \right)=0$$

(32)

using a robust function that fulfills the property $\rho (x)/\left|x\right|\to \infty$ for $\left|x\right|\to \infty$ (see [29,30]). For example, if the median was used, $\rho$ would be the sign function $\rho (x)={\mathbf{1}}_{\{x>0\}}-{\mathbf{1}}_{\{x<0\}}$. A differentiable approximation of this function is given by $\tilde{\rho }(x)=\frac{\mathrm{d}}{\mathrm{d}x}{(x^2+\epsilon )}^{p/2}$ for $p=1$ and $\epsilon =0.01$. The observed sandwich formula can be easily applied to obtain linking errors for a wide class of robust linking approaches.

5.1.2. Haebara Linking

Haebara (HAE) linking [38] aligns item response functions instead of directly aligning item parameters. The linking function in HAE linking is given as

$$H(\mu ,\sigma )=\sum _{i=1}^I\int {\left[\Psi \left({\widehat{a}}_{i1}(\theta -{\widehat{b}}_{i1})\right)-\Psi \left({\sigma }^{-1}{\widehat{a}}_{i2}(\theta -\sigma {\widehat{b}}_{i2}-{\mu }_2)\right)\right]}^2\omega (\theta )\mathrm{d}\theta ,$$

(33)

where $\omega$ is a weighting function. By defining the difference

$$h(\theta ,\mu ,\sigma ;{\widehat{a}}_{i1},{\widehat{b}}_{i1},{\widehat{a}}_{i2},{\widehat{b}}_{i2})=\Psi \left({\widehat{a}}_{i1}(\theta -{\widehat{b}}_{i1})\right)-\Psi \left({\sigma }^{-1}{\widehat{a}}_{i2}(\theta -\sigma {\widehat{b}}_{i2}-{\mu }_2)\right),$$

(34)

we can rewrite (33) as

$$H(\mu ,\sigma )=\sum _{i=1}^I\int {\left[h(\theta ,\mu ,\sigma ;{\widehat{a}}_{i1},{\widehat{b}}_{i1},{\widehat{a}}_{i2},{\widehat{b}}_{i2})\right]}^2\omega (\theta )\mathrm{d}\theta .$$

(35)

The estimating equations for $\sigma$ and $\mu$ can be determined by

$$\sum _{i=1}^I{g}_1(\delta ;a_{i1},b_{i1},f_i,e_i)=\sum _{i=1}^I\int h(\theta ,\mu ,\sigma ;{\widehat{a}}_{i1},{\widehat{b}}_{i1},{\widehat{a}}_{i2},{\widehat{b}}_{i2})\left[\frac{\partial }{\partial \sigma }h(\theta ,\mu ,\sigma ;{\widehat{a}}_{i1},{\widehat{b}}_{i1},{\widehat{a}}_{i2},{\widehat{b}}_{i2})\right]\omega (\theta )\mathrm{d}\theta =0\mathrm{and}$$

(36)

$$\sum _{i=1}^I{g}_2(\delta ;a_{i1},b_{i1},f_i,e_i)=\sum _{i=1}^I\int h(\theta ,\mu ,\sigma ;{\widehat{a}}_{i1},{\widehat{b}}_{i1},{\widehat{a}}_{i2},{\widehat{b}}_{i2})\left[\frac{\partial }{\partial \mu }h(\theta ,\mu ,\sigma ;{\widehat{a}}_{i1},{\widehat{b}}_{i1},{\widehat{a}}_{i2},{\widehat{b}}_{i2})\right]\omega (\theta )\mathrm{d}\theta =0.$$

(37)

Again, linking errors can be easily obtained using the observed sandwich (OSW) formula.

5.2. Linking Error with Testlets

In educational large-scale assessment studies such as PISA, several items frequently share a common item stimulus. In this situation, items are nested within testlets [39,40,41]. It was demonstrated that DIF effects are also pronounced at the testlet level and not only at the item level [17,42]. This additional source of uncertainty should be included in the computation of linking errors to avoid negatively biased linking error estimates.

We illustrate how to apply the theory of M-estimation in the case of testlets. The linking error for $\widehat{\sigma }$ and $\widehat{\mu }$ in log-mean-mean linking is derived. Let there be H testlets, and there are $I_h\ge 1$ items within each testlet h. It holds that ${\sum }_{h=1}^H{I}_h=I$. The data-generating model for DIF effects in the 2PL model in Equation (2) is adapted to include DIF effects at the item and the testlet level. DIF effects for logarithmized item discriminations now include two terms referring to testlets (i.e., h) and items within testlets (i.e., item i nested within testlet h). We assume

$$\begin{array}{ccc}log{a}_{ih2}& =& log{a}_{ih1}+f_h+f_{ih}\hfill \\ b_{ih2}& =& b_{ih1}+e_h+e_{ih}\hfill \end{array}$$

(38)

Note that item parameters now possess an item index i and a testlet index h in (38). Again, it is assumed that the DIF effects are independently distributed across items, and we define ${\tau }_{a,\mathrm{testlet}}^2=\mathrm{Var}(f_h)$, ${\tau }_{a,\mathrm{item}}^2=\mathrm{Var}(f_{ih})$, ${\tau }_{b,\mathrm{testlet}}^2=\mathrm{Var}(e_h)$, ${\tau }_{b,\mathrm{item}}^2=\mathrm{Var}(e_{ih})$, ${\tau }_{ab,\mathrm{testlet}}=\mathrm{Cov}(e_h,f_h)$, and ${\tau }_{ab,\mathrm{item}}=\mathrm{Cov}(e_{ih},f_{ih})$.

The estimating equations in log-mean-mean linking are the same. However, they must include the testlet structure of items. The estimating Equations (13) and (14) are rearranged as

$$\sum _{h=1}^H\sum _{i=1}^{I_h}{g}_1(\delta ;a_{ih1},b_{ih1},f_h,f_{ih},e_h,e_{ih})=\sum _{h=1}^H\sum _{i=1}^{I_h}\left(log{\widehat{a}}_{ih2}-log{\widehat{a}}_{ih1}-s\right)=0\mathrm{and}$$

(39)

$$\sum _{h=1}^H\sum _{i=1}^{I_h}{g}_2(\delta ;a_{ih1},b_{ih1},f_h,f_{ih},e_h,e_{ih})=\sum _{h=1}^H\sum _{i=1}^{I_h}\left(exp(s){\widehat{b}}_{ih2}-{\widehat{b}}_{ih1}+\mu \right)=0.$$

(40)

The essential change in the computation of the sandwich variance in the testlet case is that the variance matrix ${\mathit{B}}_I$ (i.e., the meat matrix) requires the computation of the variance that is carried out at each testlet h instead of each individual item i (see [27]). To indicate the dependency from the testlet structure, it is more appropriate to label the meat matrix ${\mathit{B}}_H$ because testlets are independent units, not items. The entry involving the variance in DIF effects in item discriminations in the meat matrix can be computed as

$$\sum _{h=1}^H\mathrm{Var}\left(\sum _{i=1}^{I_h}{g}_1(\delta ;a_{i1},b_{i1},f_h,f_{ih},e_h,e_{ih})\right)=\sum _{h=1}^H\mathrm{Var}\left(\sum _{i=1}^{I_h}\left(log{\widehat{a}}_{i2}-log{\widehat{a}}_{i1}-s\right)\right)=\left(\sum _{h=1}^H{I}_h^2\right){\tau }_{a,\mathrm{testlet}}^2+I{\tau }_{a,\mathrm{item}}^2.$$

(41)

The other variance and the covariance can be derived similarly. Consequently, the meat matrix is given by

$${\mathit{B}}_H(\widehat{\mathit{\delta }})=\left(\sum _{h=1}^H{I}_h^2\right)\left(\begin{array}{cc}{\tau }_{a,\mathrm{testlet}}^2& {\tau }_{ab,\mathrm{testlet}}\\ {\tau }_{ab,\mathrm{testlet}}& {\tau }_{b,\mathrm{testlet}}^2\end{array}\right)+I\left(\begin{array}{cc}{\tau }_{a,\mathrm{item}}^2& {\tau }_{ab,\mathrm{item}}\\ {\tau }_{ab,\mathrm{item}}& {\tau }_{b,\mathrm{item}}^2\end{array}\right).$$

(42)

The bread matrix ${\mathit{A}}_H$ only involves the expected value of the derivatives of the estimating equations. Hence, this matrix remains unchanged in a testlet structure [27].

The unknown variance and covariance components in (42) can be replaced by sample estimates. Then, an expected sandwich variance estimate can be obtained. A sample estimate of the meat matrix can be obtained by replacing the population variance in (41) with an empirical variance. Like in the case of independent items, the observed sandwich variance estimate ${\widehat{\mathit{V}}}_H$ can be modified to obtain a bias-corrected variant. In the testlet case, one should use correction factor $H/(H-1)$ instead of $I/(I-1)$.

5.3. Linking Error in Chain Linking

In this subsection, we discuss the computation of the linking error in chain linking [33,43,44] in log-mean-mean linking. Figure 1 illustrates the test design in the chain linking. The items are administered at three time points, T1, T2, and T3 (or in three groups). The goal is to determine the distribution parameters at T3. The distribution parameters of the ability variable $\theta$ at T3 can be compared with those at T1 by carrying out the linking step T1↔T2 and T2↔T3. The set of all items is denoted as $\mathcal{J}=\{1,\dots ,I\}$ and is partitioned into three distinct sets, ${\mathcal{J}}_0$, ${\mathcal{J}}_1$, and ${\mathcal{J}}_2$. The set ${\mathcal{J}}_0$ contains items that were administered at all three time points. Items in ${\mathcal{J}}_1$ and ${\mathcal{J}}_2$ are administered at T1 and T2 and T2 and T3, respectively. We fix the distribution parameters at the first time point T1 to a mean of 0 and a standard deviation of 1. The mean of $\theta$ at T2 is denoted by ${\mu }_1$ and the standard deviation by ${\sigma }_1$. The mean of $\theta$ at T3 is denoted by ${\mu }_2$ and the standard deviation by ${\sigma }_2$.

We assume that the 2PL model holds at the three time points. For longitudinal data, DIF is referred to as item parameter drift (IPD; [45,46]). The data-generating model involves IPD effects $f_{it}$ for item discriminations and $e_{it}$ for item intercepts ($t=1,2$).

$$\begin{array}{ccc}log{a}_{i2}& =& log{a}_{i1}+f_{i1}\hfill \\ b_{i2}& =& b_{i1}+e_{i1}\hfill \\ log{a}_{i3}& =& log{a}_{i2}+f_{i2}\hfill \\ b_{i3}& =& b_{i2}+e_{i2}\hfill \end{array}$$

(43)

All IPD effects are allowed to be correlated within each item i but are uncorrelated across items.

In chain linking, the 2PL model is separately estimated for the three time points. The identified item parameters are given as ${\widehat{a}}_{i1}=a_{i1}$, ${\widehat{b}}_{i1}=b_{i1}$ and

$$\begin{array}{ccc}{\widehat{a}}_{i2}& =& a_{i2}{\sigma }_1\hfill \\ {\widehat{b}}_{i2}& =& {\sigma }_1^{-1}(b_{i2}-{\mu }_1)\hfill \\ {\widehat{a}}_{i3}& =& a_{i3}{\sigma }_2\hfill \\ {\widehat{b}}_{i3}& =& {\sigma }_2^{-1}(b_{i3}-{\mu }_2)\hfill \end{array}$$

(44)

In the first linking step T1↔T2, the mean ${\mu }_1$ and the standard deviation ${\sigma }_1=exp(s_1)$ are determined in log-mean-mean linking. In the second linking step T1↔T2, linking constants $s_2$ and $m_2$ are derived that refer to the linear transformation $\theta \mapsto exp(s_2)\theta +m_2$.

In the chain linking approach, the unknown parameters are collected in the vector $\mathit{\delta }=({\mu }_1,s_1,m_2,s_2)$. The parameters of interest $({\mu }_2,{\sigma }_2)$ are computed as a derived parameter given by

$${\mu }_2=exp(s_2){\mu }_1+m_2\mathrm{and}{\sigma }_2=exp(s_2){\sigma }_1.$$

(45)

Let $I_k=\left|{\mathcal{J}}_k\right|$ be the number of items in the sets ${\mathcal{J}}_k$ for $k=0,1,2$. We define ${\kappa }_0=I_0/I$ and ${\kappa }_k=I_k/I$ for $k=1,2$. The two successive linking steps are formulated as a joint linking problem involving four estimating equations

$$\left(\begin{array}{c}{\displaystyle \sum _{i=1}^I{g}_1(\mathit{\delta };{\mathit{X}}_i)}\\ {\displaystyle \sum _{i=1}^I{g}_2(\mathit{\delta };{\mathit{X}}_i)}\\ {\displaystyle \sum _{i=1}^I{g}_3(\mathit{\delta };{\mathit{X}}_i)}\\ {\displaystyle \sum _{i=1}^I{g}_4(\mathit{\delta };{\mathit{X}}_i)}\end{array}\right)=\left(\begin{array}{c}{\displaystyle \sum _{i=1}^I{d}_{i1}(log{\widehat{a}}_{i2}-log{\widehat{a}}_{i1}-s_1)}\hfill \\ {\displaystyle \sum _{i=1}^I{d}_{i1}(exp(s_1){\widehat{b}}_{i2}-{\widehat{b}}_{i1}+{\mu }_1)}\hfill \\ {\displaystyle \sum _{i=1}^I{d}_{i2}(log{\widehat{a}}_{i3}-log{\widehat{a}}_{i2}-s_2)}\hfill \\ {\displaystyle \sum _{i=1}^I{d}_{i2}(exp(s_2){\widehat{b}}_{i3}-{\widehat{b}}_{i2}+m_2)}\hfill \end{array}\right)=\left(\begin{array}{c}0\\ 0\\ 0\\ 0\end{array}\right),$$

(46)

where ${\mathit{X}}_i$ includes all identified item parameters and design variables for item i. The first two estimating equations in (46) refer to log-mean-mean linking of the step T1↔T2, while the last two refer to step T2↔T3. In Equation (46), dummy indicators $d_{ik}$ are used that take the value of one if item i is contained in ${\mathcal{J}}_0$ or ${\mathcal{J}}_k$ for $k=1,2$. Note that ${\sum }_{i=1}^I{d}_{ik}=I_k$ for $k=1,2$ and ${\sum }_{i=1}^I{d}_{i1}{d}_{i2}=I_0$.

We simplify the terms in (46) to

$$g_1(\mathit{\delta };{\mathit{X}}_i)=d_{i1}\left\{log{\widehat{a}}_{i2}-log{\widehat{a}}_{i1}-s_1\right\}=d_{i1}{f}_{i1}$$

(47)

$$g_2(\mathit{\delta };{\mathit{X}}_i)=d_{i1}\left\{exp(s_1){\widehat{b}}_{i2}-{\widehat{b}}_{i1}+{\mu }_1\right\}=d_{i1}{e}_{i1}$$

(48)

$$g_3(\mathit{\delta };{\mathit{X}}_i)=d_{i2}\left\{log{\widehat{a}}_{i3}-log{\widehat{a}}_{i2}-s_2\right\}=d_{i2}{f}_{i2}$$

(49)

$$g_4(\mathit{\delta };{\mathit{X}}_i)=d_{i2}\left\{exp(s_2){\widehat{b}}_{i3}-{\widehat{b}}_{i2}+m_2\right\}=d_{i2}{\sigma }_1^{-1}{e}_{i2}$$

(50)

We can now compute the meat matrix ${\mathit{B}}_I$

$${\mathit{B}}_I(\mathit{\delta })=I\left(\begin{array}{cccc}{\kappa }_1{\tau }_{f_1}^2& {\kappa }_1{\tau }_{e_1{f}_1}& {\kappa }_0{\tau }_{f_1{f}_2}& {\kappa }_0{\sigma }_1^{-1}{\tau }_{f_1{e}_2}\\ {\kappa }_1{\tau }_{e_1{f}_1}& {\kappa }_1{\tau }_{e_1}^2& {\kappa }_0{\tau }_{e_1{f}_2}& {\kappa }_0{\sigma }_1^{-1}{\tau }_{e_1{e}_2}\\ {\kappa }_0{\tau }_{f_1{f}_2}& {\kappa }_0{\tau }_{e_1{f}_2}& {\kappa }_2{\tau }_{f_2}^2& {\kappa }_2{\sigma }_1^{-1}{\tau }_{e_2{f}_2}\\ {\kappa }_0{\sigma }_1^{-1}{\tau }_{f_1{e}_2}& {\kappa }_0{\sigma }_1^{-1}{\tau }_{e_1{e}_2}& {\kappa }_2{\sigma }_1^{-1}{\tau }_{e_2{f}_2}& {\kappa }_2{\sigma }_1^{-2}{\tau }_{e_2}^2\end{array}\right).$$

(51)

Moreover, we can determine the expected value of the bread matrix $\mathrm{E}({\mathit{A}}_I(\mathit{\delta }))$ as

$${\mathit{A}}_I(\mathit{\delta })=\left(\begin{array}{cccc}-I{\kappa }_1& 0& 0& 0\\ I{\kappa }_1{D}_1& I{\kappa }_1& 0& 0\\ 0& 0& -I{\kappa }_2& 0\\ 0& 0& I{\kappa }_2{D}_2& I{\kappa }_2\end{array}\right),\mathrm{where}$$

(52)

$$\begin{array}{ccc}{D}_1& =& {\displaystyle I^{-1}{\kappa }_1^{-1}\sum _{i=1}^I{d}_{i1}(b_{i1}-{\mu }_1)}\hfill \\ D_2& =& {\displaystyle I^{-1}{\kappa }_2^{-1}\sum _{i=1}^I{d}_{i2}(b_{i1}-{\mu }_2)}\hfill \end{array}.$$

(53)

The inverse of ${\mathit{A}}_I$ can be computed as

$${\left[{\mathit{A}}_I(\mathit{\delta })\right]}^{-1}=\left(\begin{array}{cccc}-I^{-1}{\kappa }_1^{-1}& 0& 0& 0\\ {\displaystyle I^{-1}{\kappa }_1^{-1}{D}_1}& I^{-1}{\kappa }_1^{-1}& 0& 0\\ 0& 0& -I^{-1}{\kappa }_2^{-1}& 0\\ 0& 0& {\displaystyle I^{-1}{\kappa }_2^{-1}{D}_2}& I^{-1}{\kappa }_2^{-1}\end{array}\right).$$

(54)

Using the matrices ${\mathit{B}}_I(\mathit{\delta })$ and ${\left[{\mathit{A}}_I(\mathit{\delta })\right]}^{-1}$, the variance matrix ${\mathit{V}}_I(\mathit{\delta })$ of $\widehat{\mathit{\delta }}$ can be computed using the sandwich formula (6).

We now explicitly derive linking errors for ${\widehat{\sigma }}_2$ and ${\widehat{\mu }}_2$. First, the standard deviation ${\sigma }_2$ at T3 is given as ${\sigma }_2=h(\mathit{\delta })=exp(s_1)exp(s_2)$. We derive the variance by applying the delta method to the nonlinear transformation h and using the variance matrix ${\mathit{V}}_I$. The first-order partial derivatives of h are given by

$${\mathit{u}}^{\top }=\left(\frac{\partial {\sigma }_2}{\partial \mathit{\delta }}\right)=\left(\begin{array}{cccc}exp(s_1)exp(s_2)& 0& exp(s_1)exp(s_2)& 0\end{array}\right).$$

(55)

Hence, the linking error of the estimated standard deviation ${\widehat{\sigma }}_2$ can be determined as

$$\mathrm{LE}({\widehat{\sigma }}_2)=\sqrt{{\mathit{u}}^{\top }{\mathit{V}}_I(\widehat{\mathit{\delta }})\mathit{u}}.$$

(56)

The algebraic derivation for the linking error formula was somewhat more intricate, which is why the R package rSymPy package [47] as a wrapper to the SymPy computer algebra system [48] was used. The square of the linking error for ${\widehat{\sigma }}_2$ (i.e., the quantity $\mathrm{LE}{({\widehat{\sigma }}_2)}^2$) is computed using (56)

$$\begin{array}{ccc}\mathrm{LE}{({\widehat{\sigma }}_2)}^2& =& {\displaystyle \frac{{\tau }_{f_1}^2}{I}\left[\frac{{\sigma }_1^2{\sigma }_2^2}{{\kappa }_1}\right]+\frac{{\tau }_{f_2}^2}{I}\left[\frac{{\sigma }_1^2{\sigma }_2^2}{{\kappa }_2}\right]-\frac{{\tau }_{e_1{f}_2}}{I}\left[\frac{D_1{\kappa }_0{\sigma }_1^2{\sigma }_2^2}{{\kappa }_1{\kappa }_2}\right]}\hfill \\ & & {\displaystyle -\frac{{\tau }_{e_2{f}_2}}{I}\left[\frac{D_2{\sigma }_1{\sigma }_2^2}{{\kappa }_2}\right]-\frac{{\tau }_{f_1{e}_2}}{I}\left[\frac{D_2{\kappa }_0{\sigma }_1{\sigma }_2^2}{{\kappa }_1{\kappa }_2}\right]+\frac{{\tau }_{f_1{f}_2}}{I}\left[\frac{2{\kappa }_0{\sigma }_1^2{\sigma }_2^2}{{\kappa }_1{\kappa }_2}\right].}\hfill \end{array}$$

(57)

The linking error for the estimated mean ${\widehat{\mu }}_2$ can be determined as a derived parameter using the transformation ${\mu }_2=h(\mathit{\delta })=exp(s_2){\mu }_1+m_2$. The first-order partial derivatives of h are given by

$${\mathit{u}}^{\top }=\left(\frac{\partial {\mu }_2}{\partial \mathit{\delta }}\right)=\left(\begin{array}{cccc}0& exp(s_2)& exp(s_2){\mu }_1& 1\end{array}\right).$$

(58)

The linking error for ${\widehat{\mu }}_2$ can be computed by using (56)

$$\begin{array}{ccc}\mathrm{LE}{({\widehat{\mu }}_2)}^2& =& {\displaystyle \frac{{\tau }_{f_2}^2}{I}\left[\frac{{\mu }_1^2{\sigma }_2^2-D_2{\mu }_1{\sigma }_2}{{\kappa }_2}\right]+\frac{{\tau }_{e_1}^2}{I}\left[\frac{{\sigma }_2^2}{{\kappa }_1}\right]+\frac{{\tau }_{e_2}^2}{I}\left[\frac{1+D_2{\mu }_1{\sigma }_2}{{\kappa }_2{\sigma }_1^2}\right]}\hfill \\ & & {\displaystyle +\frac{{\tau }_{e_1{f}_2}}{I}\left[\frac{D_2{\kappa }_0{\sigma }_2-2{\kappa }_0{\mu }_1{\sigma }_2^2}{{\kappa }_1{\kappa }_2}\right]+\frac{{\tau }_{e_2{f}_2}}{I}\left[\frac{D_2-2{\mu }_1{\sigma }_2+{\mu }_1{\sigma }_2{D}_2^2-D_2{\mu }_1^2{\sigma }_2^2}{{\kappa }_2{\sigma }_1})\right]}\hfill \\ & & {\displaystyle +\frac{{\tau }_{f_1{e}_2}}{I}\left[\frac{D_1{\kappa }_0{\sigma }_2+D_1{D}_2{\kappa }_0{\mu }_1{\sigma }_2^2}{{\kappa }_1{\kappa }_2{\sigma }_1}\right]-\frac{{\tau }_{f_1{f}_2}}{I}\left[D_1\frac{{\kappa }_0{\mu }_1{\sigma }_2^2}{{\kappa }_1{\kappa }_2}\right].}\hfill \end{array}$$

(59)

In chain linking in the 1PL model, the linking error for ${\widehat{\mu }}_2$ substantially simplifies by setting ${\sigma }_1={\sigma }_2=1$, $D_2=0$ (see also [49])

$$\mathrm{LE}{({\widehat{\mu }}_2)}^2=\frac{{\tau }_{e_1}^2}{I}\left[\frac{1}{{\kappa }_1}\right]+\frac{{\tau }_{e_2}^2}{I}\left[\frac{1}{{\kappa }_2}\right].$$

(60)

Note that all DIF effects referring to item discriminations vanish in this case.

5.4. Linking Error for Trend Estimates in Educational Large-Scale Assessment Studies

We now turn to the important application of linking errors for trend estimates [13,15,50,51] in educational LSA studies such as PISA [16,52]. The main goal is to compute a linking error for a trend estimate in country means or country standard deviations between two successive assessments. Again, we rely on the 2PL model and use log-mean-mean linking for the derivation of the linking errors. Previous research derived closed formulas for linking errors in the 1PL model [17]. It was stated in official PISA publications that there does not exist a simple generalization to the 2PL model [52]. However, this section provides a closed formula for the 2PL model.

Figure 2 illustrates the problem of trend estimation in LSA studies. The label “NAT” refers to a nation (i.e., a country) c. The label “INT” refers to the international metric that is defined as a pooled sample comprising all students of participating countries in the LSA study. A trend estimate for a country between two LSA assessments can involve means and standard deviations at T1 and T2 (i.e., it compares NAT2 and NAT1). The first linking step NAT1↔INT1 maps country-specific results onto an international metric at T1. This step allows a cross-sectional comparison of countries on an international reference metric. The second linking step INT1↔INT2 links results of two LSA studies at T1 and T2 at the international metric. The third linking step NAT2↔INT2 maps country-specific results to the international metric at T2.

The set of administered items typically differs across assessments [53]. There are $I_0$ link items that are administered at both time points. A set of $I_1$ unique items is only administered at T1, while a set of $I_2$ unique items is only administered at T2. For identification reasons, we assume that the $\theta$ ability variable has zero mean and a standard deviation of one at T1. Then, we can identify the mean ${\mu }_{c1}$ and the standard deviation ${\sigma }_{c1}$ of country c at T1. Furthermore, we assume that the mean at the international metric at T2 is ${\mu }_0$ and the standard deviation is ${\sigma }_0$. We can also identify the mean ${\mu }_{c2}$ and the standard deviation ${\sigma }_{c2}$ of country c at T2.

The first linking step NAT1↔INT1 in log-mean-mean linking estimates ${\mu }_{c1}$ and $s_{c1}=log{\sigma }_{c2}$. The second linking step INT1↔INT2 estimates ${\mu }_0$ and $s_0=log{\sigma }_0$. The third linking step NAT2↔INT2 estimates linking constants $m_{c2}$ and $s_{c2}$ to put the results of country c at the international metric at T2.

The country mean and standard deviation of country c at T2 are derived functions of the estimated linking parameters

$${\mu }_{c2}=exp(s_0){\mu }_0+m_{c2}\mathrm{and}{\sigma }_{c2}=exp(s_{c2})exp(s_0)$$

(61)

The linking constants for the link INT2↔NAT2 are recomputed using (61) as

$$m_{c2}={\mu }_{c2}-exp(s_0){\mu }_0\mathrm{and}{s}_{c2}=log{\sigma }_{c2}-s_0$$

(62)

The main idea is to apply the M-estimation theory and the sandwich formula for deriving linking errors for trend estimates. The three-step linking procedure can also be written as a simultaneous estimation problem involving six estimating equations for the vector of unknown linking parameters $\mathit{\delta }=(s_{c1},{\mu }_{c1},s_0,m_0,s_{c2},m_{c2})$. The trend estimate in means is given as

$$\Delta {\mu }_c={\mu }_{c2}-{\mu }_{c1}=exp(s_0){\mu }_0+m_{c2}-{\mu }_{c1}.$$

(63)

The trend estimate in standard deviations is given as

$$\Delta {\sigma }_c={\sigma }_2-{\sigma }_1=exp(s_0+s_{c2})-exp(s_{c1}).$$

(64)

The source of linking errors in trend estimates is the presence of DIF effects. We now present a data-generating model for DIF effects in item parameters in the 2PL model. Let $a_{ict}$ and $b_{ict}$ for $t=1,2$ be item discriminations and item difficulties for country c. These item parameters are referred to as national item parameters [17]. International item parameters that result from item response models at the international metric that involves students from all countries are denoted by ${\mathsf{\alpha }}_{it}$ and ${\mathsf{\beta }}_{it}$. We use the same random effects model as in [17] for item difficulties

$$\begin{array}{ccc}{b}_{ict}& =& b_i+e_{it}+e_{ic}+e_{ict}\hfill \\ {\mathsf{\beta }}_{ict}& =& b_i+e_{it}\hfill \end{array}$$

(65)

The variance component $\mathrm{Var}(e_{it})={\tau }_{b,\mathrm{IPD}}^2$ refers to item parameter drift (IPD; [54]). The variance $\mathrm{Var}(e_{ic})={\tau }_{b,\mathrm{DIF}}^2$ is referred to as cross-sectional country DIF, and the variance $\mathrm{Var}(e_{ict})={\tau }_{b,\mathrm{DIF}\times \mathrm{IPD}}^2$ refers to time-point-specific country DIF. All DIF effects are uncorrelated with each other.

We now extend the random effects model in [17] to item discriminations

$$\begin{array}{ccc}log{a}_{ict}& =& log{a}_i+f_{it}+f_{ic}+f_{ict}\hfill \\ log{\mathsf{\alpha }}_{ict}& =& log{a}_i+f_{it}\hfill \end{array}$$

(66)

All DIF effects for logarithmized item discriminations are uncorrelated, but DIF effects e and f can be correlated for the same type of heterogeneity (i.e., $\mathrm{Cov}(e_{it},f_{it})={\tau }_{ab,\mathrm{IPD}}$, $\mathrm{Cov}(e_{ic},f_{ic})={\tau }_{ab,\mathrm{DIF}}$, and $\mathrm{Cov}(e_{ict},f_{ict})={\tau }_{ab,\mathrm{DIF}\times \mathrm{IPD}}$).

Identified national item parameters are given as ${\widehat{a}}_{ict}=a_{ict}{\sigma }_{ct}$ and ${\widehat{b}}_{ict}={\sigma }_{ct}^{-1}(b_{ict}-{\mu }_{ct})$ for $t=1,2$. Moreover, for international item parameters, it holds that ${\widehat{\mathsf{\alpha }}}_{i1}={\mathsf{\alpha }}_{i1}$ and ${\widehat{\mathsf{\beta }}}_{i1}={\mathsf{\beta }}_{i1}$ at T1, and ${\widehat{\mathsf{\alpha }}}_{i2}={\mathsf{\alpha }}_{i2}{\sigma }_1$ and ${\widehat{\mathsf{\beta }}}_{i2}={\sigma }_0^{-1}({\mathsf{\beta }}_{i2}-{\mu }_0)$.

We now apply the M-estimation theory to the estimation of linking errors. The three log-mean-mean linking steps can be formalized as the following six estimating equations:

$$\left(\begin{array}{c}{\displaystyle \sum _{i=1}^I{g}_1(\mathit{\delta };{\mathit{X}}_i)}\\ {\displaystyle \sum _{i=1}^I{g}_2(\mathit{\delta };{\mathit{X}}_i)}\\ {\displaystyle \sum _{i=1}^I{g}_3(\mathit{\delta };{\mathit{X}}_i)}\\ {\displaystyle \sum _{i=1}^I{g}_4(\mathit{\delta };{\mathit{X}}_i)}\\ {\displaystyle \sum _{i=1}^I{g}_5(\mathit{\delta };{\mathit{X}}_i)}\\ {\displaystyle \sum _{i=1}^I{g}_6(\mathit{\delta };{\mathit{X}}_i)}\end{array}\right)=\left(\begin{array}{c}{\displaystyle \sum _{i=1}^I{d}_{i1}(log{\widehat{a}}_{ic1}-log{\widehat{\mathsf{\alpha }}}_{i1}-s_1)}\hfill \\ {\displaystyle \sum _{i=1}^I{d}_{i1}(exp(s_1){\widehat{b}}_{ic1}-{\widehat{\mathsf{\beta }}}_{i1}+{\mu }_1)}\hfill \\ {\displaystyle \sum _{i=1}^I{d}_{i0}(log{\widehat{\mathsf{\alpha }}}_{i2}-log{\widehat{\mathsf{\alpha }}}_{i1}-s_0)}\hfill \\ {\displaystyle \sum _{i=1}^I{d}_{i0}(exp(s_0){\widehat{\mathsf{\beta }}}_{i2}-\widehat{\mathsf{\beta }}{b}_{i1}+{\mu }_0)}\hfill \\ {\displaystyle \sum _{i=1}^I{d}_{i2}(log{\widehat{a}}_{ic2}-log{\widehat{\mathsf{\alpha }}}_{i2}-s_2)}\hfill \\ {\displaystyle \sum _{i=1}^I{d}_{i2}(exp(s_2){\widehat{b}}_{ic2}-{\widehat{\mathsf{\beta }}}_{i2}+m_2)}\hfill \end{array}\right)=\left(\begin{array}{c}0\\ 0\\ 0\\ 0\\ 0\\ 0\end{array}\right)$$

(67)

The estimating equations in (67) define an estimate of the parameter of interest $\mathit{\delta }=(s_{c1},{\mu }_{c1},s_0,m_0,s_{c2},m_{c2})$. We now use the sandwich formula to derive the asymptotic variance of $\widehat{\mathit{\delta }}$ by means of the sandwich Formula (6).

The entries of the meat matrix ${\mathit{B}}_I(\mathit{\delta })$ in the sandwich formula are denoted by

$${\mathit{B}}_I(\mathit{\delta })=\left(\begin{array}{cccccc}{B}_{11}& B_{21}& 0& 0& B_{51}& B_{61}\\ B_{21}& B_{22}& 0& 0& B_{52}& B_{62}\\ 0& 0& B_{33}& B_{43}& 0& 0\\ 0& 0& B_{43}& B_{44}& 0& 0\\ B_{51}& B_{52}& 0& 0& B_{55}& B_{65}\\ B_{61}& B_{62}& 0& 0& B_{65}& B_{66}\end{array}\right).$$

(68)

The non-zero entries in ${\mathit{B}}_I(\mathit{\delta })$ are given as

$$\begin{array}{ccc}\hfill B_{11}& =& I{\kappa }_1({\tau }_{a,\mathrm{DIF}}^2+{\tau }_{a,\mathrm{DIF}\times \mathrm{IPD}}^2)\hfill \\ \hfill B_{21}& =& I{\kappa }_1({\tau }_{ab,\mathrm{DIF}}+{\tau }_{ab,\mathrm{DIF}\times \mathrm{IPD}})\hfill \\ \hfill B_{22}& =& I{\kappa }_1({\tau }_{b,\mathrm{DIF}}^2+{\tau }_{b,\mathrm{DIF}\times \mathrm{IPD}}^2)\hfill \\ \hfill B_{33}& =& 2I{\kappa }_0{\tau }_{a,\mathrm{IPD}}^2\hfill \\ \hfill B_{43}& =& 2I{\kappa }_0{\tau }_{ab,\mathrm{IPD}}^2\hfill \\ \hfill B_{44}& =& 2I{\kappa }_0{\tau }_{b,\mathrm{IPD}}^2\hfill \\ \hfill B_{51}& =& I{\kappa }_0{\tau }_{a,\mathrm{DIF}}^2\hfill \\ \hfill B_{52}& =& I{\kappa }_0{\tau }_{ab,\mathrm{DIF}}\hfill \\ \hfill B_{61}& =& I{\kappa }_{0}exp{(s_0)}^{-1}{\tau }_{ab,\mathrm{DIF}}\hfill \\ \hfill B_{62}& =& I{\kappa }_{0}exp{(s_0)}^{-1}{\tau }_{b,\mathrm{DIF}}^2\hfill \\ \hfill B_{55}& =& I{\kappa }_2({\tau }_{a,\mathrm{DIF}}^2+{\tau }_{a,\mathrm{DIF}\times \mathrm{IPD}}^2)\hfill \\ \hfill B_{65}& =& I{\kappa }_{2}exp{(s_0)}^{-1}({\tau }_{ab,\mathrm{DIF}}+{\tau }_{ab,\mathrm{DIF}\times \mathrm{IPD}})\hfill \\ \hfill B_{66}& =& I{\kappa }_{2}exp{(s_0)}^{-2}({\tau }_{b,\mathrm{DIF}}^2+{\tau }_{b,\mathrm{DIF}\times \mathrm{IPD}}^2)\hfill \end{array}$$

(69)

Moreover, we can determine the bread matrix ${\mathit{A}}_I(\mathit{\delta })$ as

$${\mathit{A}}_I(\mathit{\delta })=\left(\begin{array}{cccccc}-I{\kappa }_1& 0& 0& 0& 0& 0\\ {\displaystyle I{\kappa }_1{D}_1}& I{\kappa }_1& 0& 0& 0& 0\\ 0& 0& -I{\kappa }_0& 0& 0& 0\\ 0& 0& I{\kappa }_0{D}_0& I{\kappa }_0& 0& 0\\ 0& 0& 0& 0& -I{\kappa }_2& 0\\ 0& 0& 0& 0& I{\kappa }_2{D}_2& I{\kappa }_2\end{array}\right),\mathrm{where}$$

(70)

$$\begin{array}{ccc}{D}_1& =& {\displaystyle I^{-1}{\kappa }_1^{-1}\sum _{i=1}^I{d}_{i1}(b_i-{\mu }_1)}\\ D_0& =& {\displaystyle I^{-1}{\kappa }_0^{-1}\sum _{i=1}^I{d}_{i0}(b_i-{\mu }_0)}\\ D_2& =& {\displaystyle I^{-1}{\kappa }_2\sum _{i=1}^I{d}_{i2}(b_i-m_2)}\end{array}$$

(71)

The inverse of ${\mathit{A}}_I(\mathit{\delta })$ can be computed as

$${({\mathit{A}}_I(\mathit{\delta }))}^{-1}=\left(\begin{array}{cccccc}-I^{-1}{\kappa }_1^{-1}& 0& 0& 0& 0& 0\\ {\displaystyle I^{-1}{\kappa }_1^{-1}{D}_1}& I^{-1}{\kappa }_1^{-1}& 0& 0& 0& 0\\ 0& 0& -I^{-1}{\kappa }_0^{-1}& 0& 0& 0\\ 0& 0& I^{-1}{\kappa }_0^{-1}{D}_0& I^{-1}{\kappa }_0^{-1}& 0& 0\\ 0& 0& 0& 0& -I^{-1}{\kappa }_2^{-1}& 0\\ 0& 0& 0& 0& I^{-1}{\kappa }_2^{-1}{D}_2& I^{-1}{\kappa }_2^{-1}\end{array}\right)$$

(72)

Using (69) and (72), the variance matrix ${\mathit{V}}_I$ can be computed.

We now derive the linking error of the trend estimate in standard deviations that is a nonlinear function of $\mathit{\delta }=(s_{c1},{\mu }_{c1},s_0,m_0,s_{c2},m_{c2})$

$$\Delta {\sigma }_c=h(\mathit{\delta })={\sigma }_2-{\sigma }_1=exp(s_0+s_{c2})-exp(s_{c1}).$$

(73)

The first-order partial derivatives of h are given by

$$\left(\frac{\partial h}{\partial \mathit{\delta }}\right)=\left(\begin{array}{cccccc}-exp(s_{c1})& 0& exp(s_0)exp(s_{c2})& 0& exp(s_0)exp(s_{c2})& 0\end{array}\right).$$

(74)

Using (56), we can determine the square of the linking error $\mathrm{LE}{(\Delta {\sigma }_c)}^2$ by using computer algebra software [47,48] as

$$\begin{array}{ccc}\mathrm{LE}{(\Delta {\sigma }_c)}^2& =& {\displaystyle \frac{{\tau }_{a,\mathrm{DIF}}^2}{I}\left[\frac{{\sigma }_1^2}{{\kappa }_1}+\frac{{\sigma }_2^2}{{\kappa }_2}-\frac{2{\sigma }_1{\sigma }_2{\kappa }_0}{{\kappa }_1{\kappa }_2}\right]+\frac{{\tau }_{a,\mathrm{IPD}}^2}{I}\left[\frac{2{\sigma }_2^2}{{\kappa }_0}\right]+\frac{{\tau }_{a,\mathrm{DIF}\times \mathrm{IPD}}^2}{I}\left[\frac{{\sigma }_1^2}{{\kappa }_1}+\frac{{\sigma }_2^2}{{\kappa }_2}\right]}\hfill \\ & & {\displaystyle +\frac{{\tau }_{ab,\mathrm{DIF}}}{I}\left[\frac{D_2{\sigma }_1{\sigma }_2{\kappa }_0}{{\kappa }_1{\kappa }_2{\sigma }_0}+\frac{D_1{\sigma }_1{\sigma }_2{\kappa }_0}{{\kappa }_1{\kappa }_2}-\frac{D_1{\sigma }_1^2}{{\kappa }_1}-\frac{D_2{\sigma }_2^2}{{\kappa }_2{\sigma }_0}\right]-\frac{{\tau }_{ab,\mathrm{IPD}}}{I}\left[\frac{2D_0{\sigma }_2^2}{{\kappa }_0}\right]}\hfill \\ & & {\displaystyle -\frac{{\tau }_{ab,\mathrm{DIF}\times \mathrm{IPD}}}{I}\left[\frac{D_1{\sigma }_1^2}{{\kappa }_1}+\frac{D_2{\sigma }_2^2}{{\kappa }_2{\sigma }_0}\right].}\hfill \end{array}$$

(75)

We now compute the linking error for the trend estimate in the mean in the 2PL model. The country mean difference can be computed as

$$\Delta {\mu }_c=h(\mathit{\delta })={\mu }_{c2}-{\mu }_{c1}=exp(s_0){\mu }_0+m_{c2}-{\mu }_{c1}.$$

(76)

The first-order partial derivatives are given as

$$\left(\frac{\partial h}{\partial \mathit{\delta }}\right)=\left(\begin{array}{cccccc}0& -1& exp(s_0){\mu }_0& exp(s_0)& 0& 1\end{array}\right).$$

(77)

The linking error for the trend estimate in means is given by

$$\begin{array}{ccc}\mathrm{LE}{(\Delta {\mu }_c)}^2& =& {\displaystyle \frac{{\tau }_{b,\mathrm{DIF}}^2}{I}\left[\frac{1}{{\kappa }_1}+\frac{1}{{\kappa }_2{\sigma }_0^2}-\frac{2{\kappa }_0}{{\sigma }_0{\kappa }_1{\kappa }_2}\right]+\frac{{\tau }_{b,\mathrm{IPD}}^2}{I}\left[\frac{2{\sigma }_0^2(1+D_0{\mu }_0)}{{\kappa }_0}\right]+\frac{{\tau }_{b,\mathrm{DIF}\times \mathrm{IPD}}^2}{I}\left[\frac{1}{{\kappa }_1}+\frac{1}{{\sigma }_0^2{\kappa }_2}\right]}\hfill \\ & & {\displaystyle +\frac{{\tau }_{ab,\mathrm{DIF}}}{I}\left[\frac{D_1}{{\kappa }_1}+\frac{D_2}{{\sigma }_0{\kappa }_2}-\frac{D_2{\kappa }_0}{{\kappa }_1{\kappa }_2}-\frac{D_1{\kappa }_0}{{\sigma }_0{\kappa }_1{\kappa }_2}\right]+\frac{{\tau }_{a,\mathrm{IPD}}^2}{I}\left[\frac{2{\sigma }_0^2{\mu }_0({\mu }_0-D_0)}{{\kappa }_0}\right]}\hfill \\ & & {\displaystyle +\frac{{\tau }_{ab,\mathrm{IPD}}}{I}\left[\frac{2D_0{\sigma }_0^2+2{\mu }_0{D}_0^2{\sigma }_0^2-2D_0{\sigma }_0^2{\mu }_0^2-4{\mu }_0{\sigma }_0^2}{{\kappa }_0}\right]+\frac{{\tau }_{ab,\mathrm{DIF}\times \mathrm{IPD}}}{I}\left[\frac{D_1}{{\kappa }_1}+\frac{D_2}{{\sigma }_0{\kappa }_2}\right].}\hfill \end{array}$$

(78)

The linking error formula for the 1PL model derived in [17] can be obtained from the terms in (78) that involve the variance components ${\tau }_{b,\mathrm{DIF}}^2$, ${\tau }_{b,\mathrm{IPD}}^2$, and ${\tau }_{b,\mathrm{DIF}\times \mathrm{IPD}}^2$ by setting ${\sigma }_0^2=1$ and $D_0=0$:

$$\mathrm{LE}{(\Delta {\mu }_c)}^2=\frac{{\tau }_{b,\mathrm{DIF}}^2}{I}\left[\frac{1}{{\kappa }_1}+\frac{1}{{\kappa }_2}-\frac{2{\kappa }_0}{{\kappa }_1{\kappa }_2}\right]+\frac{{\tau }_{b,\mathrm{IPD}}^2}{I}\left[\frac{2}{{\kappa }_0}\right]+\frac{{\tau }_{b,\mathrm{DIF}\times \mathrm{IPD}}^2}{I}\left[\frac{1}{{\kappa }_1}+\frac{1}{{\kappa }_2}\right]$$

(79)

All other components in (78) vanish in the case of the 1PL model.

5.5. Linking Error in Fixed Item Parameter Calibration

In this subsection, linking errors for the estimated mean and the estimated standard deviations under fixed item parameter calibration (FIPC) are derived. It is assumed that one uses fixed item parameters $a_{i1}$ and $b_{i1}$, but the true item parameters $a_{i2}$ and $b_{i2}$ have DIF effects and follow the data-generating model (2).

The FIPC is typically applied using marginal maximum likelihood estimation [55]. However, we derive the linking error for a diagonally weighted least squares (DWLS) estimation that approximates the former estimation method [56]. The DWLS minimizes the weighted sums of the differences between the estimated and model-implied item thresholds as well as the tetrachoric correlations. For the simplicity of exposition in this subsection, we assume that there are only DIF effects in item difficulties (i.e., uniform DIF). Hence, we can assume that the standard deviation $\sigma$ can be estimated without bias.

The item-specific weights in the DWLS estimation are precision weights $\omega$ that are defined as the inverse of the variance in thresholds $\tau$. When omitting a factor that involves the sample size, precision weights $\omega$ are determined by

$$\omega =\omega (\tau )=\frac{\varphi {(\tau )}^2}{\Phi (\tau )\left(1-\Phi (\tau )\right)}.$$

(80)

Equation (80) is displayed in Figure 3. It can be seen that extreme thresholds are downweighted.

The optimization function for $\mu$ in DWLS is given by

$$H(\mu )=\sum _{i=1}^I{\omega }_i{\left({\widehat{\tau }}_{i2}-{\tau }_{i2}(\mu )\right)}^2,$$

(81)

where ${\widehat{\tau }}_{i2}$ is the identified threshold of item i and ${\tau }_{i2}$ is the model-implied threshold. Now, define ${\alpha }_i=a_i\sigma {\left(a_i^2{\sigma }^2+\mathcal{L}\right)}^{-1/2}$, where $\mathcal{L}$ is the logistic variance that is the byproduct of using the logistic instead of the probit link function in the 2PL model. Then, (81) can be rewritten as

$$H(\mu )=\sum _{i=1}^I{\omega }_i{\alpha }_i{\left({\tilde{b}}_{i2}-(\mu -b_{i1})\right)}^2=\sum _{i=1}^I{w}_i{\left({\widehat{b}}_{i2}-(\mu -b_{i1})\right)}^2,$$

(82)

where $w_i=C{\omega }_i{\alpha }_i$ (C is an appropriate scaling constant) and ${\widehat{b}}_{i2}={\alpha }_i^{-1}{\widehat{\tau }}_{i2}$. Note that ${\tilde{b}}_{i2}={\mu }_0-b_{i2}={\mu }_0-b_{i1}-e_i$ for a data-generating true mean ${\mu }_0$.

The minimization of (82) leads to a weighted mean

$$\widehat{\mu }=\frac{{\displaystyle \sum _{i=1}^I{w}_i({\tilde{b}}_{i2}-b_{i1})}}{W}={\mu }_0-\frac{{\displaystyle \sum _{i=1}^I{w}_i{e}_i}}{W},$$

(83)

where $W={\sum }_{i=1}^I{w}_i$, and ${\mu }_0$ in (83) denotes the true mean.

Hence, one can determine the linking error as the variance in the second term on the right-hand side in (83). We then obtain the linking error in fixed item parameter calibration

$$\mathrm{LE}(\widehat{\mu })=\frac{\sqrt{{\displaystyle \sum _{i=1}^I{w}_i^2}}}{W}{\tau }_b.$$

(84)

By the Cauchy–Schwarz inequality, it holds that

$$W=\sum _{i=1}^I{w}_i\le \sqrt{I}·\sqrt{\sum _{i=1}^I{w}_i^2}.$$

(85)

Hence, we get from (84) by using (85)

$$\mathrm{LE}(\widehat{\mu })\ge \frac{1}{\sqrt{I}}{\tau }_b.$$

(86)

Hence, the linking error has a lower bound in which all item-specific weights were set equal to one. This case corresponds to the linking error in the Rasch model obtained for mean-mean linking [15,17]

$$\mathrm{LE}(\widehat{\mu })=\frac{{\tau }_b}{\sqrt{I}}.$$

(87)

This linking error is also obtained in FIPC of the 1PL model using unweighted least squares (ULS) estimation.

Interestingly, the finding in (86) illustrates that using incorrect item parameters in the FIPC in the presence of DIF effects results in a precision loss. Hence, the maximum likelihood estimation can only achieve the most efficient estimates under correctly specified models. Because it is almost always expected that there are some intentionally unmodeled DIF effects in real datasets, the dominance of the maximum likelihood estimation in LSA practice can be questioned.

5.6. Linking Error in Concurrent Calibration

In this subsection, we derive the linking error for the estimation of the 2PL model in two groups. The mean and the standard deviation of the ability variable $\theta$ are fixed to 0 and 1, respectively. The mean $\mu$ and the standard deviation $\sigma$ of the second group are estimated.

Like in Section 5.5, we assume that there are only DIF effects $e_i$ in item difficulties and no DIF effects in item discriminations. We assume that $\sigma$ can be estimated without bias and derive the linking error under concurrent calibration using DWLS. DIF effects $e_i$ with zero means follow $b_{i2}=b_{i1}+e_i$. The identified parameters are given by ${\widehat{b}}_{i1}=b_{i1}$ and ${\widehat{b}}_{i2}=b_{i2}-\mu$. The estimation of the vector of common item difficulties $\mathit{b}=(b_1,\dots ,b_I)$ and the mean $\mu$ is conducted using a weighted square loss of differences in estimated and model-implied item thresholds. By simplifying the setting while assuming known item discriminations and standard deviation, the optimization function is given by

$$H(\mu ,\mathit{b})=\sum _{i=1}^I{w}_{i1}{({\widehat{b}}_{i1}-b_i)}^2+\sum _{i=1}^I{w}_{i2}{({\widehat{b}}_{i2}+\mu -b_i)}^2,$$

(88)

where weights $w_{ig}$ are allowed to be group specific ($g=1,2$). The estimating equations for $\mu$ and $\mathit{b}$ are given as

$$w_{i1}({\widehat{b}}_{i1}-{\widehat{b}}_i)+w_{i2}({\widehat{b}}_{i2}+\widehat{\mu }-{\widehat{b}}_i)=0\mathrm{and}$$

(89)

$$\sum _{i=1}^I{w}_{i2}({\widehat{b}}_{i2}+\widehat{\mu }-{\widehat{b}}_i)=0.$$

(90)

By inserting (89) into (90), we obtain

$$\sum _{i=1}^I{w}_{i2}{\widehat{b}}_{i2}+\sum _{i=1}^I{w}_{i2}\widehat{\mu }-\sum _{i=1}^I{w}_{i2}\frac{w_{i1}{\widehat{b}}_{i1}+w_{i2}{\widehat{b}}_{i2}+w_{i2}\widehat{\mu }}{w_{i1}+w_{i2}}=0.$$

(91)

Equation (91) can be further simplified to

$$\sum _{i=1}^I\frac{w_{i1}{w}_{i2}}{w_{i1}+w_{i2}}{\widehat{b}}_{i2}+\sum _{i=1}^I\frac{w_{i1}{w}_{i2}}{w_{i1}+w_{i2}}\widehat{\mu }-\sum _{i=1}^I\frac{w_{i1}{w}_{i2}}{w_{i1}+w_{i2}}{\widehat{b}}_{i1}=0.$$

(92)

By defining $w_i^{*}={(w_{i1}+w_{i2})}^{-1}{w}_{i1}{w}_{i2}$, we obtain from (92)

$$\widehat{\mu }=-\frac{{\displaystyle \sum _{i=1}^I{w}_i^{*}({\widehat{b}}_{i2}-{\widehat{b}}_{i1})}}{{\displaystyle \sum _{i=1}^I{w}_i^{*}}}=\mu -\frac{{\displaystyle \sum _{i=1}^I{w}_i^{*}{e}_i}}{{\displaystyle \sum _{i=1}^I{w}_i^{*}}}.$$

(93)

Hence, we obtain the linking error for $\widehat{\mu }$

$$\mathrm{LE}(\widehat{\mu })=\frac{\sqrt{{\displaystyle \sum _{i=1}^I{(w_i^{*})}^2}}}{W^{*}}{\tau }_b,$$

(94)

where $W^{*}={\sum }_{i=1}^I{w}_i^{*}$. Again, one can conclude $\mathrm{LE}(\widehat{\mu })\ge {\tau }_b/\sqrt{I}$. Hence, using the linking error ${\tau }_b/\sqrt{I}$ that does not take item-specific weights into account provides a lower bound of the true linking error.

If the weights $w_{i1}$ and $w_{i2}$ would be equal across both groups, we obtain the same linking error like under fixed item parameter calibration.

5.7. Linking Error for Derived Parameters

In previous sections of this paper, we derived the linking error for $\mu$ and $\sigma$ for different applications of the 2PL model. Sometimes, other distribution parameters might be estimated. In this subsection, we assume that the ability variable $\theta$ is approximately normally distributed and derive the linking error for nonlinear functions h of $\mu$ and $\sigma$. Let $\nu =h(\mathit{\delta })=h(\mu ,\sigma )$ be a nonlinear function of the mean $\mu$ and the standard deviation $\sigma$. Let $\mathit{V}$ denote the variance matrix of $\mathit{\delta }$ regarding item choice (i.e., quantifying linking errors). The linking error estimate $\widehat{\nu }=h(\widehat{\mu },\widehat{\sigma })$ is given as

$$\mathrm{LE}(\widehat{\nu })=\sqrt{{\mathit{u}}^{\top }\mathit{V}\mathit{u}},\mathrm{where}{\mathit{u}}^{\top }=\left(\begin{array}{cc}{\displaystyle \frac{\partial h}{\partial \mu }}& {\displaystyle \frac{\partial h}{\partial \sigma }}\end{array}\right)\mathrm{and}\mathit{V}=\left(\begin{array}{cc}{v}_{\mu }^2& v_{\mu \sigma }\\ v_{\mu \sigma }& v_{\sigma }^2\end{array}\right).$$

(95)

We now illustrate the computation in two examples.

5.7.1. Proportions

First, we are interested in computing the probability p

$$p=h(\mu ,\sigma )=\mathrm{P}(\theta \in [c_1,c_2])=\Phi \left({\sigma }^{-1}(c_2-\mu )\right)-\Phi \left({\sigma }^{-1}(c_1-\mu )\right).$$

(96)

The partial derivatives of h are given as

$$\frac{\partial h}{\partial \mu }=-{\sigma }^{-1}\left[\varphi \left({\sigma }^{-1}(c_2-\mu )\right)-\varphi \left({\sigma }^{-1}(c_1-\mu )\right)\right]=-{\sigma }^{-1}d(\mu ,\sigma ,c_1,c_2)\mathrm{and}$$

(97)

$$\frac{\partial h}{\partial \sigma }=-{\sigma }^{-2}\left[\varphi \left({\sigma }^{-1}(c_2-\mu )\right)-\varphi \left({\sigma }^{-1}(c_1-\mu )\right)\right]=-{\sigma }^{-2}d(\mu ,\sigma ,c_1,c_2),$$

(98)

where $d(\mu ,\sigma ,c_1,c_2)=\varphi \left({\sigma }^{-1}(c_2-\mu )\right)-\varphi \left({\sigma }^{-1}(c_1-\mu )\right)$ Hence, we can determine the linking error of $\widehat{p}=g(\widehat{\mu },\widehat{\sigma })$ using (95) as

$$\mathrm{LE}(\widehat{p})={\widehat{\sigma }}^{-1}d(\widehat{\mu },\widehat{\sigma },c_1,c_2)\sqrt{{\widehat{v}}_{\mu }^2+{\widehat{\sigma }}^{-2}{\widehat{v}}_{\sigma }^2+2{\widehat{\sigma }}^{-1}{\widehat{v}}_{\mu \sigma }},$$

(99)

where ${\widehat{v}}_{\mu }^2$, ${\widehat{v}}_{\sigma }^2$, and ${\widehat{v}}_{\mu \sigma }$ are estimated linking variances and covariances.

5.7.2. Percentiles

Second, the linking error of the pth percentile $Q_p$ should be computed. The pth percentile is defined by

$$\mathrm{P}(\theta \le Q_p)=\Phi \left({\sigma }^{-1}(Q_p-\mu )\right)=p.$$

(100)

Hence, we can solve (100) for $Q_p$ and obtain

$$Q_p=h(\mu ,\sigma )=\mu +\sigma z_p,$$

(101)

where $z_p={\Phi }^{-1}(p)$ is the pth percentile for the standard normal distribution.

Now, we can compute the linking error of ${\widehat{Q}}_p$. We obtain

$$\frac{\partial h}{\partial \mu }=1\mathrm{and}\frac{\partial h}{\partial \sigma }=z_p.$$

(102)

The linking error of the percentile $Q_p$ using (95) is given by

$$\mathrm{LE}({\widehat{Q}}_p)=\sqrt{{\widehat{v}}_{\mu }^2+z_p^2{\widehat{v}}_{\sigma }^2+2z_p{\widehat{v}}_{\mu \sigma }}.$$

(103)

It can be seen that for more extreme percentiles, the absolute value of $z_p$ gets larger and the linking error for $\sigma$ (i.e., ${\widehat{v}}_{\sigma }$ in (103)) becomes more relevant.

5.8. Computation of Total Error and Sampling Error Correction for Linking Error Estimates

In practical applications, the sampling error $\mathrm{SE}$ due to the sampling of persons must also be taken into account to quantify the uncertainty in the $\sigma$ and $\mu$ estimates. The total error ($\mathrm{TE}$) is given by (see [18,19])

$$\mathrm{TE}=\sqrt{{\mathrm{SE}}^2+{\mathrm{LE}}^2}.$$

(104)

A critical issue might be that estimated linking errors also include a portion of variability that can be attributed to the sampling error of persons. We illustrate an analytical bias correction method for the case of log-mean-mean linking. The issue in the sandwich estimate is the variance matrix (i.e., the meat matrix) ${\mathit{B}}_I$ (see (15) for the computation). The identified item parameters also include a sampling error variance that can be estimated by fitting an item response model. Let $v_{ia}^{*}$ be the variance of $log{\widehat{a}}_{i2}-log{\widehat{a}}_{i1}$ due to person sampling (i.e., $v_{ia}^{*}=\mathrm{Var}(log{\widehat{a}}_{i2}-log{\widehat{a}}_{i1})$. Then, we can determine a corresponding entry in the meat matrix ${\mathit{B}}_I$ by modifying (15) into

$$\sum _{i=1}^I\mathrm{Var}\left(g_1(\delta ;a_{i1},b_{i1},f_i,e_i)\right)=\sum _{i=1}^I\mathrm{Var}\left(log{\widehat{a}}_{i2}-log{\widehat{a}}_{i1}-s\right)=\sum _{i=1}^I\left(\mathrm{Var}\left(f_i\right)+v_{ia}^{*}\right)=I{\tau }_a^2+\sum _{i=1}^I{v}_{ia}^{*}.$$

(105)

In the derivation of (105), we relied on the property that item parameters of different items are approximately uncorrelated in sufficiently long tests [57]. The other entries in ${\mathit{B}}_I$ can be similarly determined. By defining $v_{ib}^{*}=\mathrm{Var}({\widehat{b}}_{i2}-{\widehat{b}}_{i1})$ and $v_{iab}^{*}=\mathrm{Cov}(log{\widehat{a}}_{i2}-log{\widehat{a}}_{i1},{\widehat{b}}_{i2}-{\widehat{b}}_{i1})$, Equations (16) and (17) can be modified to

$$\sum _{i=1}^I\mathrm{Cov}\left(g_1(\delta ;a_{i1},b_{i1},f_i,e_i),g_2(\delta ;a_{i1},b_{i1},f_i,e_i)\right)=I{\tau }_{ab}+\sum _{i=1}^I{v}_{iab}^{*},\mathrm{and}$$

(106)

$$\sum _{i=1}^I\mathrm{Var}\left(g_2(\delta ;a_{i1},b_{i1},f_i,e_i)\right)=I{\tau }_b^2+\sum _{i=1}^I{v}_{ib}^{*}.$$

(107)

Hence, the originally obtained meat matrix ${\mathit{B}}_I$ can be removed from sampling error contributions by defining a bias-corrected estimate

$${\mathit{B}}_I^{*}(\widehat{\mathit{\delta }})={\mathit{B}}_I(\widehat{\mathit{\delta }})-{\mathit{V}}^{*},\mathrm{where}{\mathit{V}}^{*}=\left(\begin{array}{cc}{\displaystyle \sum _{i=1}^I{v}_{ia}^{*}}& {\displaystyle \sum _{i=1}^I{v}_{iab}^{*}}\\ {\displaystyle \sum _{i=1}^I{v}_{iab}^{*}}& {\displaystyle \sum _{i=1}^I{v}_{ib}^{*}}\end{array}\right).$$

(108)

This approach was implemented in the simplified setting in the 1PL model [17].

We now show how to generalize the bias-corrected estimate of the meat matrix ${\mathit{B}}_I$. We can rewrite the estimate from Equation (9) as

$${\widehat{\mathit{B}}}_I(\widehat{\mathit{\delta }})=\sum _{i=1}^I\mathit{g}(\widehat{\mathit{\delta }};{\widehat{\mathbf{\gamma }}}_i)\mathit{g}{(\widehat{\mathit{\delta }};{\widehat{\mathbf{\gamma }}}_i)}^{\top },$$

(109)

where ${\widehat{\mathbf{\gamma }}}_i=({\widehat{a}}_{i1},{\widehat{b}}_{i1},{\widehat{a}}_{i2},{\widehat{b}}_{i2})$. The estimate ${\widehat{\mathbf{\gamma }}}_i$ includes the sampling error, and the sampling variance is denoted as

$${\mathit{V}}_i=\mathrm{Var}({\widehat{\mathbf{\gamma }}}_i)=\mathrm{E}\left[({\widehat{\mathbf{\gamma }}}_i-{\mathbf{\gamma }}_i){({\widehat{\mathbf{\gamma }}}_i-{\mathbf{\gamma }}_i)}^{\top }\right].$$

(110)

The estimating function $\mathit{g}$ in (109) can be viewed as a function $\mathit{g}=\mathit{g}(\mathit{\delta },\mathbf{\gamma })$. Denote by ${\mathit{g}}_{\mathbf{\gamma }}=(\partial \mathit{g})/(\partial \mathbf{\gamma })$ the matrix of partial derivatives with respect to $\mathbf{\gamma }$. We can now apply a Taylor expansion and obtain

$$\mathit{g}(\widehat{\mathit{\delta }},{\widehat{\mathbf{\gamma }}}_i)=\mathit{g}(\widehat{\mathit{\delta }},{\mathbf{\gamma }}_i)+{\mathit{g}}_{\mathbf{\gamma }}(\widehat{\mathit{\delta }},{\mathbf{\gamma }}_i)({\widehat{\mathbf{\gamma }}}_i-{\mathbf{\gamma }}_i).$$

(111)

Using $\mathrm{E}({\widehat{\mathbf{\gamma }}}_i)=\mathbf{\gamma }$, we can obtain a bias-corrected estimate of the bread matrix as

$${\mathit{B}}_I^{*}(\widehat{\mathit{\delta }})={\widehat{\mathit{B}}}_I(\widehat{\mathit{\delta }})-\sum _{i=1}^I{\mathit{g}}_{\mathbf{\gamma }}(\widehat{\mathit{\delta }},{\mathbf{\gamma }}_i){\mathit{V}}_i{\mathit{g}}_{\mathbf{\gamma }}{(\widehat{\mathit{\delta }},{\mathbf{\gamma }}_i)}^{\top }.$$

(112)

6. Discussion

In this article, we have shown that the sandwich formula from the M-estimation theory can be successfully employed for computing the linking error in the 2PL model in a variety of situations. It was shown in a simulation study for the log-mean-mean linking of two groups that the expected sandwich estimate of the linking error produced satisfactory coverage rates. Interestingly, it had a comparable performance to the jackknife linking error in the 2PL model.

As with any simulation study, some limitations of our study can be stated. More comprehensive studies could involve a different range of standard deviations of the DIF effects, different test lengths, or other linking approaches. It might be interesting to compare the performance of the M-estimation approach with the jackknife linking error for the more complex linking problems of chain linking and trend estimation.

In the simulation study, we only considered uncertainty in distribution parameters due to DIF effects (i.e., linking errors). In practical applications, there will also be a sampling of persons, and the simultaneous assessment of sampling errors and linking errors would be an exciting extension of this study.

In most of the applications involving instruments with cognitive and noncognitive items, linking errors are not reported even if linking approaches were utilized [43,58,59]. There might be two reasons why this is the case. First, simulation studies typically presuppose that the IRT model perfectly fits the data. That is, the DIF is absent, and the IRT model is correctly specified. Second, items could be treated as fixed, and the model misspecification is taken for granted but is not included in the uncertainty quantification of the linking approach. In our view, linking errors provide additional information about the impact of heterogeneous item functioning on parameters of interest and should, therefore, (almost always) be additionally reported. In general, we do not think that the presence of DIF or IPD threatens the validity of group differences or trend estimates.

It should be emphasized that our proposed linking error for the trend estimates in the 2PL model differs from a newly proposed linking error estimate since PISA 2015 [4]. The latter relies on a recalibration approach of the item response data from the first time point [34,60]. The new PISA linking error rather assesses the extent of the assumed noninvariant item parameters across the two PISA studies instead of quantifying the variability due to heterogeneous item functioning.

The derived linking errors rely on the assumption that item parameters are identified. In LSA studies such as PISA, balanced incomplete block designs are employed in which only a subset of items is administered to each student [61]. If item parameters can be identified in such test designs, linking error formulas would not change because country means and standard deviations are based on all the items administered in the test, irrespective of the proportion of items administered to each student.

As argued by an anonymous reviewer, the computation of linking errors relies on the assumption of a representative item sample from an item population. Notably, the identification of item parameters in the joint maximum likelihood estimation also requires asymptotic regimes regarding the sample size and the number of items [62,63,64] but does not require that items are a random sample from an item population. However, the main difference in the computation of linking errors is that linking errors reflect the variability in the distribution parameter estimates due to the DIF. Infinite item samples are not required for model identification; they are only used as a statistical tool for justifying the statistical inference with respect to modeled or unmodeled DIF.

Finally, we assumed that DIF effects had zero means throughout this paper. However, this assumption is not essential in deriving linking errors because the M-estimation theory does not require that estimated parameters converge to true parameters. The M-estimation theory will nevertheless provide the asymptotic variance in a potentially biased estimator. However, DIF effects could also result in a mean that is different from zero. For example, Ref. [28] assumes that the median of the DIF effects equals zero. In this case, log-mean-mean linking in the 2PL model could be replaced by an alternative robust linking method in which the mean is substituted with the median. Notably, the choice of an adequate estimating function to provide unbiased distribution parameters depending on the distribution of DIF effects is a different topic though [18].