Comparing Robust Linking and Regularized Estimation for Linking Two Groups in the 1PL and 2PL Models in the Presence of Sparse Uniform Differential Item Functioning

Abstract

In the social sciences, the performance of two groups is frequently compared based on a cognitive test involving binary items. Item response models are often utilized for comparing the two groups. However, the presence of differential item functioning (DIF) can impact group comparisons. In order to avoid the biased estimation of groups, appropriate statistical methods for handling differential item functioning are required. This article compares the performance-regularized estimation and several robust linking approaches in three simulation studies that address the one-parameter logistic (1PL) and two-parameter logistic (2PL) models, respectively. It turned out that robust linking approaches are at least as effective as the regularized estimation approach in most of the conditions in the simulation studies.

1. Introduction

Item response theory (IRT) models [1,2] are an important class of multivariate statistical models for analyzing dichotomous random variables used to model testing data from social science applications. Of vital importance is the application of item response models in educational large-scale assessment (LSA; [3,4]), such as the programme for international student assessment (PISA; [5]) study.

In this article, we only investigate unidimensional IRT models. Let $\mathit{X}=(X_1,\dots ,X_I)$ be the vector of I dichotomous random variables $X_i\in \{0,1\}$ (also referred to as test items or items). A unidimensional item response model [6] is a statistical model for the probability distribution $\mathrm{P}(\mathit{X}=\mathit{x})$ for the vector $\mathit{x}=(x_1,\dots ,x_I)\in {\{0,1\}}^I$, where

$${\displaystyle \mathrm{P}(\mathit{X}=\mathit{x};\mathbf{\delta },\mathbf{\gamma })={\int }_{-\infty }^{\infty }\prod _{i=1}^I\left[{P_i(\theta ;{\mathbf{\gamma }}_i)}^{x_i}{\left(1-P_i(\theta ;{\mathbf{\gamma }}_i)\right)}^{1-x_i}\right]\varphi (\theta ;\mu ,\sigma )\mathrm{d}\theta ,}$$

(1)

where $\varphi$ denotes the density of the normal distribution with mean $\mu$ and standard deviation $\sigma$. The vector $\mathbf{\delta }=(\mu ,\sigma )$ contains the distribution parameters of the ability variable $\theta$. The vector $\mathbf{\gamma }=({\mathbf{\gamma }}_1,\dots ,{\mathbf{\gamma }}_I)$ contains all estimated item parameters of item response functions $P_i(\theta ;{\mathbf{\gamma }}_i)=\mathrm{P}\left(X_i=1\right|\theta )$.

Different IRT models emerge by choosing particular item response functions $P_i$ in (1). The one-parameter logistic (1PL) model (also referred to as the Rasch model; [7]) employs the item response function $P_i\left(\theta \right)=\mathrm{\Psi }(\theta -b_i)$, where $\mathrm{\Psi }$ denotes the logistic distribution function, and $b_i$ is the item difficulty of item i (i.e., ${\mathbf{\gamma }}_i=\left(b_i\right)$). The two-parameter logistic (2PL) model [8] additionally includes the item discrimination $a_i$ (i.e., ${\mathbf{\gamma }}_i=(a_i,b_i)$), and the item response function is given by $P_i\left(\theta \right)=\mathrm{\Psi }\left(a_i(\theta -b_i)\right)$.

Note that distribution parameters $\mathbf{\delta }$ and item parameters $\mathbf{\gamma }$ cannot be simultaneously identified. For example, in the 2PL model, the mean $\mu$ and the standard deviation $\sigma$ must be fixed to 0 or 1, respectively, if all item discriminations $a_i$ and item difficulties $b_i$ were estimated. As an alternative, one could fix at least (or all) item parameters to predefined values to enable the estimation of distribution parameters.

In this article, we investigate the estimation of the 1PL and 2PL models in the presence of two groups. In this case, we are primarily interested in comparing the mean ${\mu }_2$ of the second group with the mean ${\mu }_1$ of the first group. In order to enable group comparisons, assumptions on item parameters must be imposed in the 1PL and 2PL models. First, one must fix ${\mu }_1$ to zero. Second, some assumptions on item parameters must be made to enable the identification of the distribution parameters of the second group. A typical assumption is the measurement invariance assumption [9,10], which says that item parameters do not differ across the two groups. However, in practical applications, some item parameters will almost always differ across groups. This situation is also referred to as differential item functioning (DIF; [11,12]). In this article, we assume that there could be DIF in item difficulties in the 1PL and 2PL models (i.e., uniform DIF; [12]). The classes of techniques of robust linking and regularized estimation are compared through three simulation studies regarding the performance of parameter recovery of the group comparison.

The rest of the article is structured as follows. Different statistical methods for group comparisons in the 1PL and the 2PL models in the presence of uniform DIF are discussed in Section 2. Section 3 presents results from Simulation Study 1, which investigates the group comparison in the 1PL model in the presence of DIF. Section 4 more thoroughly investigates the choice of tuning parameters for regularized estimation in a subset of conditions of Simulation Study 1 in the Focused Simulation Study 1A. Section 5 presents results from Simulation Study 2, which investigates the group comparison in the 2PL model in the presence of uniform DIF effects. Finally, the article closes with a discussion in Section 6.

2. Two-Group Comparison under Sparse DIF

We now present IRT estimation in two groups $g=1,2$. Let ${\mathit{X}}_{pg}=(X_{pg1},\dots ,X_{pgI})$ of person $p=1,\dots ,N_g$ in group $g=1,2$. We now define the log-likelihood function for data ${\mathcal{D}}_g=({\mathit{X}}_{1g},\dots ,{\mathit{X}}_{N_{g}g})$ in group g ($g=1,2$) as

$$l({\mu }_g,{\sigma }_g,{\mathit{a}}_g,{\mathit{b}}_g;{\mathcal{D}}_g)=\sum _{p=1}^{N_g}log\left[\int \prod _{i=1}^I{P}_i(x_{pgi},\theta ;a_{ig},b_{ig})\varphi (\theta ;{\mu }_g,{\sigma }_g)\mathrm{d}\theta \right],$$

(2)

where $P_i(1,\theta ;a_i,b_i)=\mathrm{\Psi }\left(a_i(\theta -b_i)\right)$ and $P_i(0,\theta ;a_i,b_i)=1-P_i(1,\theta ;a_i,b_i)$ is the 2PL model, and the vectors of item parameters are defined as ${\mathit{a}}_g=(a_{1g},\dots ,a_{Ig})$ and ${\mathit{b}}_g=(b_{1g},\dots ,b_{Ig})$.

For reasons of identification, we fix the mean ${\mu }_1$ in the first group to zero and the standard deviation ${\sigma }_1$ to 1. We assume that the 2PL (or the 1PL model as a restricted version) holds in both groups, and there is uniform DIF in item difficulties (i.e., equal item discriminations $a_{i1}=a_{i2}$ in the two groups are assumed). That is, we model the difference in item difficulties as

$$b_{i2}=b_{i1}+e_i.$$

(3)

We assume that DIF effects are fixed (see [13] for a random DIF perspective). Throughout this paper, we impose a sparsity assumption on DIF effects $e_i$. In this case, the majority of items have DIF effects of zero, while only a few DIF effects differ from zero [14,15]. The situation is known as partial invariance [16,17,18,19]. Hence, DIF effects can be regarded as outliers that might bias the estimation of group mean differences [14,20,21,22,23,24].

In the next subsections, we discuss alternative methods for two-group comparisons in the 1PL and 2PL models in the presence of uniform DIF. In Section 2.1, concurrent calibration relying on invariant item parameters is discussed. Section 2.2 investigates regularization approaches to handling DIF in two-group comparisons. Robust linking approaches are treated in Section 2.3. Finally, relationships between regularization and robust linking are highlighted in Section 2.4.

2.1. Concurrent Calibration

Concurrent calibration jointly estimates common (group-invariant) item parameters and distribution parameters in one estimation run. This property is particularly convenient for practitioners because no additional steps or postprocessing of model results is required. We now present concurrent calibration separately for the 1PL and 2PL models.

2.1.1. 1PL Model

We now estimate the distribution parameters of the second group (i.e., ${\mu }_2$ and ${\sigma }_2$) with a concurrent calibration approach in the 1PL model. In the 1PL model, item discriminations are set to one and indicate this using the notation ${\mathit{a}}_g={\displaystyle 1}$ of a vector of ones. The distribution parameters and the vector of common item difficulties are estimated by minimizing the negative of the log-likelihood function (see (2))

$$({\widehat{\mu }}_2,{\widehat{\sigma }}_2,{\widehat{\sigma }}_1,\widehat{\mathit{b}})=\underset{({\mu }_2,{\sigma }_2,{\sigma }_1,\mathit{b})}{arg\; min}\left\{-l(0,{\sigma }_1,{\displaystyle 1},\mathit{b};{\mathcal{D}}_1)-l({\mu }_2,{\sigma }_2,{\displaystyle 1},\mathit{b};{\mathcal{D}}_2)\right\}.$$

(4)

Note that the minimization in (4) assumes invariant item difficulties $\mathit{b}$ across the two groups. In the presence of uniform DIF, the log-likelihood function in (4) is misspecified. However, under certain conditions, it is possible that group differences could nevertheless be unbiasedly estimated [25,26].

2.1.2. 2PL Model

We now weaken the assumption of equal item discriminations in the 2PL model. Common item discriminations $\mathit{a}$ and item difficulties $\mathit{b}$ are estimated by minimizing the estimation function

$$({\widehat{\mu }}_2,{\widehat{\sigma }}_2,\widehat{\mathit{a}},\widehat{\mathit{b}})=\underset{({\mu }_2,{\sigma }_2,\mathit{a},\mathit{b})}{arg\; min}\left\{-l(0,1,\mathit{a},\mathit{b};{\mathcal{D}}_1)-l({\mu }_2,{\sigma }_2,\mathit{a},\mathit{b};{\mathcal{D}}_2)\right\}.$$

(5)

Like in the 1PL model, the log-likelihood function in (5) is misspecified in the presence of uniform DIF. Nevertheless, it is interesting to empirically investigate situations in which misspecified concurrent calibration can provide approximately unbiased results.

2.2. Regularization Approaches

In practical applications, all items could be prone to DIF effects. However, modeling all DIF effects without any constraints leads to an unidentified IRT model. To circumvent this issue, regularization techniques have been proposed in statistics to estimate nonidentified models under sparsity assumptions [27,28]. The main idea of using regularization techniques for multiple-group IRT estimation is that by adding an appropriate penalty term to the negative log-likelihood function, some simplified structure on DIF effects is imposed. It can be shown that under sparse DIF effects, regularized estimation provides unbiased group means [20]. Regularized estimation recently became popular in psychometrics, such as item response modeling [29,30], structural equation modeling [31,32,33], structured latent class analysis [34,35,36], and mixture models [37,38]. The investigation of regularization approaches of known demographic groups, such as gender or language groups, is an important topic in educational measurement. Moreover, regularization techniques were recently discussed for manifest DIF detection in the 1PL and 2PL models [19,20,39,40,41,42,43,44].

For a scalar parameter x, lasso penalty is a popular penalty function used in regularization [28], and it is defined as

$${\mathcal{P}}_{\mathrm{Lasso}}(x,\lambda )=\lambda \left|x\right|,$$

(6)

where $\lambda$ is a non-negative regularization parameter that controls the extent of regularization. It is known that the lasso penalty induces bias in estimated parameters. To circumvent this issue, the smoothly clipped absolute deviation (SCAD; [45]) penalty has been proposed. It is defined by

$${\mathcal{P}}_{\mathrm{SCAD}}(x,\lambda ,a)=\left\{\begin{array}{cc}\lambda \left|x\right|\hfill & \mathrm{if}\left|x\right|<\lambda \hfill \\ -(x^2-{2a\lambda \left|x\right|}^2+{\lambda }^2{)\left(2(a-1)\right)}^{-1}\hfill & \mathrm{if}\lambda \le \left|x\right|\le a\lambda \hfill \\ (a+1){\lambda }^2\hfill & \mathrm{if}\left|x\right|>a\lambda \hfill \end{array}\right.$$

(7)

with $a>2$. In many studies, the recommended value of $a=3.7$ (see [45]) has been adopted (e.g., [27,36,38,46,47,48]). However, other studies considered the simultaneous selection of both tuning parameters $\lambda$ and a [31,49,50,51].

Figure 1 displays the SCAD penalty function for different values of a with a fixed $\lambda$ value of 0.2. The SCAD penalty retains the penalization rate and the induced bias of the lasso for model parameters close to zero, but continuously relaxes the rate of penalization as the absolute value of the model parameters increases. Note that ${\mathcal{P}}_{\mathrm{SCAD}}$ has the property of the lasso penalty around zero, but has zero derivatives for x values that strongly differ from zero. In contrast, the derivative of the lasso penalty is 1 or $-1$ for positive and negative x values, respectively.

We now present regularization estimation under uniform DIF for the 1PL and the 2PL model.

2.2.1. 1PL Model

In regularization, we specify an overidentified IRT model but enable the identification of distribution parameters of the second group (i.e., ${\mu }_2$ and ${\sigma }_2$) by adding a penalty function to the negative log-likelihood function. Compared with (4), we additionally introduce the vector of DIF effects $\mathit{e}=(e_1,\dots ,e_I)$ and consider the minimization problem

$$({\widehat{\mu }}_2,{\widehat{\sigma }}_2,{\widehat{\sigma }}_1,\widehat{\mathit{b}},\widehat{\mathit{e}})=\underset{({\mu }_2,{\sigma }_2,{\sigma }_1,\mathit{b},\mathit{e})}{arg\; min}\left\{-l(0,{\sigma }_1,{\displaystyle 1},\mathit{b};{\mathcal{D}}_1)-l({\mu }_2,{\sigma }_2,{\displaystyle 1},\mathit{b}+\mathit{e};{\mathcal{D}}_2)+N^{*}\sum _{i=1}^I{\mathcal{P}}_{\mathrm{SCAD}}(e_i,\lambda ,a)\right\},$$

(8)

where $N^{*}=(N_1+N_2)/2$ includes the sample sizes in the penalty term.

In practice, the minimization of (8) for a fixed value of $\lambda$ results in a subset of DIF effects that are different from zero, where the rest of the DIF effects has been set to zero. In essence, the group difference is only based on those items whose DIF effects $e_i$ are estimated equal to zero.

Typically, the regularization parameter $\lambda$ is an unknown nuisance parameter in (8) that must also be estimated. In practice, the minimization of (8) is carried out on a discrete grid of $\lambda$ values, and the optimal regularization parameter ${\lambda }_{\mathrm{opt}}$ is selected that minimizes the Akaike information criterion (AIC) or the Bayesian information criterion (BIC).

The regularized estimation problem (8) can be minimized using marginal maximum likelihood estimation and the expectation maximization (EM) algorithm [30,36,46]. The EM algorithm alternates between the E-step and the M-step. The E-step computation is identical to the estimation in nonregularized item response models. In the M-step, the minimizing of the regularized negative log-likelihood function is carried out using expected counts that are computed in the expected log-likelihood function. The difference between regularized estimation and ordinary maximum likelihood estimation is that the optimization function becomes nondifferentiable, because the SCAD penalty is nondifferentiable. The optimization of nondifferentiable optimization can be performed using gradient descent [28] approaches or by substituting the nondifferentiable optimization functions with differentiable approximating functions [36,52,53,54]. In our experience, the latter approach performs quite satisfactorily in applications.

2.2.2. 2PL Model

We now turn to regularized estimation in the 2PL model. Like in the 1PL model, the overidentified vector of the DIF effects $\mathit{e}$ is introduced. In addition, the vector of common item discriminations $\mathit{a}$ is estimated. The following estimation function is minimized for determining group differences

$$({\widehat{\mu }}_2,{\widehat{\sigma }}_2,\widehat{\mathit{a}},\widehat{\mathit{b}},\widehat{\mathit{e}})=\underset{({\mu }_2,{\sigma }_2,\mathit{a},\mathit{b},\mathit{e})}{arg\; min}\left\{-l(0,1,\mathit{a},\mathit{b};{\mathcal{D}}_1)-l({\mu }_2,{\sigma }_2,\mathit{a},\mathit{b}+\mathit{e};{\mathcal{D}}_2)+N^{*}\sum _{i=1}^I{\mathcal{P}}_{\mathrm{SCAD}}(e_i,\lambda ,a)\right\}.$$

(9)

In principle, regularized estimation in the 2PL model using common item discriminations is not different from regularized estimation in the 1PL model. The principle of regularization can be extended to modeling DIF effects in item discriminations by introducing additional penalty terms for these DIF effects [44].

2.3. Robust Linking Approaches

Linking methods are typically two-step methods that separately estimate IRT models, such as the 1PL or 2PL models in each of the groups, and compute group distribution parameters (i.e., group means and standard deviations) in a second step [25,55,56,57]. We replace the separate estimation with a simultaneous estimation in a two-group IRT model. We differ from this usual setup in this article for two reasons. First, we want to highlight the notational similarity to regularized estimation. Second, we also want to conduct linking in the 2PL model under the assumption of common item discriminations if there is a strong belief that most of the DIF effects can be attributed to uniform instead of nonuniform DIF effects. Robust linking as a concept refers to the property that group differences can be estimated without bias (or only with small bias) despite the presence of (uniform) DIF [25,58].

2.3.1. 1PL Model

We now present robust linking approaches in the 1PL model. In the first step, we consider the minimization problem

$$({\widehat{\sigma }}_2,{\widehat{\sigma }}_1,\widehat{\mathit{b}},\widehat{\mathit{e}})=\underset{({\sigma }_2,{\sigma }_1,\mathit{b},\mathit{e})}{arg\; min}\left\{-l(0,{\sigma }_1,{\displaystyle 1},\mathit{b};{\mathcal{D}}_1)-l(0,{\sigma }_2,{\displaystyle 1},\mathit{b}+\mathit{e};{\mathcal{D}}_2)\right\}.$$

(10)

Note that (10) differs from regularized estimation (8) in two aspects. First, it does not involve a penalty function. Second, it does not provide an estimate of the group mean ${\mu }_2$ of the second group. We want to point out that the estimates from the minimization (10) are equivalent to the separate estimation of the 1PL model with item difficulties $\widehat{\mathit{b}}$ in the first group and difficulties $\widehat{\mathit{b}}+\widehat{\mathit{e}}$ in the second group.

Robust (and nonrobust) linking methods employ estimated DIF effects $\widehat{\mathit{e}}=({\widehat{e}}_1,\dots ,{\widehat{e}}_I)$ to determine a group mean estimate ${\widehat{\mu }}_2$.

Robust Linking Using the $L_p$ Loss Function

The first robust linking method uses the $L_p$ loss function $\rho \left(x\right)={\left|x\right|}^p$ for $p>0$ (see [25,59,60]) and determines the group mean estimate as

$${\widehat{\mu }}_2=\underset{{\mu }_2}{arg\; min}\left\{\sum _{i=1}^I\rho ({\widehat{e}}_i+{\mu }_2)\right\}.$$

(11)

Mean–mean linking (MM; [55,61]) results with $p=2$. The case $p=1$ corresponds to median linking [20] and is expected to be more robust than $p=2$. Finally, the case $p=0.5$ corresponds to the loss function used in invariance alignment [59,62,63]. In optimization, one can replace the nondifferentiable function $\rho$ with $\tilde{\rho }\left(x\right)={(x^2+\epsilon )}^{p/2}$ using a sufficiently small $\epsilon >0$, such as $\epsilon =0.01$ [59].

Robust Linking Using the MAD Statistic

Researchers Matthias von Davier and Bezirhan proposed robust outlier detection of items with large DIF effects and removed them from linking [64] (see also [61,65,66]). For each item i, a z statistic is defined by

$$z_i=\frac{{\widehat{e}}_i-\mathrm{Mdn}\left(\widehat{\mathit{e}}\right)}{\mathrm{MAD}\left(\widehat{\mathit{e}}\right)},$$

(12)

where $\mathrm{Mdn}\left(\widehat{\mathit{e}}\right)$ denotes the median of the vector $\widehat{\mathit{e}}$ of estimated DIF effects, and MAD is the (scaled) median absolute deviation of the DIF effects. An item is declared an outlier (because of possessing a large DIF effect) if $|z_i|$ exceeds the cutoff of 2.7 [64]. The group mean difference is defined as

$${\widehat{\mu }}_2=-\frac{{\displaystyle \sum _{i=1}^I{w}_i{\widehat{e}}_i}}{{\displaystyle \sum _{i=1}^I{w}_i}}\mathrm{with}{w}_i={{\displaystyle 1}}_{\left\{\right|z_i|\le 2.7\}}.$$

(13)

Obviously, items with large DIF effects are removed from the computation of the group mean in (13).

2.3.2. 2PL Model

Linking based on the 2PL model can be applied in two variants. The first approach relies on a first simultaneous estimation step in which common item discriminations $\mathit{a}$ are estimated. It is well known that the 2PL model gets unstable with small samples, which motivates the estimation of a simplified model with joint item discriminations. In the first step of the linking approach, we determine item parameters and the standard deviation in the second group by

$$({\widehat{\sigma }}_2,\widehat{\mathit{a}},\widehat{\mathit{b}},\widehat{\mathit{e}})=\underset{({\sigma }_2,\mathit{a},\mathit{b},\mathit{e})}{arg\; min}\left\{-l(0,1,\mathit{a},\mathit{b};{\mathcal{D}}_1)-l(0,{\sigma }_2,\mathit{a},\mathit{b}+\mathit{e};{\mathcal{D}}_2)\right\}.$$

(14)

Alternatively, separate estimation of the 2PL model in the two groups can be conducted, which results in estimated item parameters ${\widehat{\mathit{a}}}_1$ and ${\widehat{\mathit{b}}}_1$ and ${\widehat{\mathit{a}}}_2$ and ${\widehat{\mathit{b}}}_2$, respectively.

Robust Linking Using $L_p$ Loss Function or MAD Statistic

The estimation of ${\mu }_2$ based on the $L_p$ loss function and outlier removal can also be based on the estimated vector $\widehat{\mathit{e}}$. No change in formulas in (11), (12), and (13) is required.

Joint Haberman Linking Using Common Item Discriminations

We now show how to perform Haberman linking [67,68] if the first linking step is carried out using common item discriminations (see minimization problem (14). The more general $L_p$ loss function is again used for Haberman linking [60]. In this case, common item difficulties $\tilde{\mathit{b}}=({\tilde{b}}_1,\dots ,{\tilde{b}}_I)$ are estimated by minimizing

$$({\widehat{\mu }}_2,\widehat{\tilde{\mathit{b}}})=\underset{({\mu }_2,\tilde{\mathit{b}})}{arg\; min}\left\{\sum _{i=1}^I\rho ({\widehat{b}}_i-{\tilde{b}}_i)+\sum _{i=1}^I\rho ({\widehat{b}}_i+{\widehat{e}}_i-{\tilde{b}}_i+{\mu }_2)\right\}.$$

(15)

Haberman Linking Based on Separate Calibration

We also compare the performance of joint Haberman linking using common item discriminations with the ordinary Haberman linking that is based on separation calibration [60,68]. In the first step of Haberman linking, common logarithmized item discriminations $\mathbf{\alpha }=({\alpha }_1,\dots ,{\alpha }_I)$, and the logarithmized standard deviation $s_2$ of the second group is determined as the minimizer of

$$({\widehat{s}}_2,\widehat{\mathbf{\alpha }})=\underset{(s_2,\mathbf{\alpha })}{arg\; min}\left\{\sum _{i=1}^I\rho (log{\widehat{a}}_{i1}-{\alpha }_i)+\sum _{i=1}^I\rho (log{\widehat{a}}_{i2}-{\alpha }_i-s_2)\right\}.$$

(16)

Note that the standard deviation of the second group is given by ${\sigma }_2=exp\left(s_2\right)$.

In the second step of Haberman linking, the group means ${\mu }_2$ of the second group are estimated along with common item difficulties ${\mathit{b}}_i$

$$({\widehat{\mu }}_2,\widehat{\mathit{b}})=\underset{({\mu }_2,\mathit{b})}{arg\; min}\left\{\sum _{i=1}^I\rho ({\widehat{b}}_{i1}-b_i)+\sum _{i=1}^I\rho ({\widehat{\sigma }}_2{\widehat{b}}_{i2}-b_i+{\mu }_2)\right\}.$$

(17)

2.4. On the Relation of Robust Linking and Regularized Estimation

It has been argued that robust linking yields very similar results to regularization approaches for linking two groups [20]. We now sketch a heuristic proof of why this is the case. In the regularization approach, one uses the SCAD penalty, which behaves similarly to the lasso penalty ${\mathcal{P}}_{\mathrm{Lasso}}\left(x\right)=\lambda \left|x\right|$ for x values close to zero. We now use the notation of $\rho$ for the penalty function in regularized estimation to indicate that a general $L_p$ loss function can be used in regularized estimation. The optimization function in regularized estimation in the 1PL model is then given by

$$({\widehat{\mu }}_2,{\widehat{\sigma }}_2,{\widehat{\sigma }}_1,\widehat{\mathit{b}},\widehat{\mathit{e}})=\underset{({\mu }_2,{\sigma }_2,{\sigma }_1,\mathit{b},\mathit{e})}{arg\; min}\left\{-l(0,{\sigma }_1,{\displaystyle 1},\mathit{b};{\mathcal{D}}_1)-l({\mu }_2,{\sigma }_2,{\displaystyle 1},\mathit{b}+\mathit{e};{\mathcal{D}}_2)+N^{*}\lambda \sum _{i=1}^I\rho \left(e_i\right)\right\}.$$

(18)

We demonstrated that robust linking relies on the first minimization step

$$({\widehat{\sigma }}_2,{\widehat{\sigma }}_1,\widehat{\mathit{b}},\widehat{\mathit{e}})=\underset{({\sigma }_2,{\sigma }_1,\mathit{b},\mathit{e})}{arg\; min}\left\{-l(0,{\sigma }_1,{\displaystyle 1},\mathit{b};{\mathcal{D}}_1)-l(0,{\sigma }_2,{\displaystyle 1},\mathit{b}+\mathit{e};{\mathcal{D}}_2)\right\}.$$

(19)

Importantly, the log-likelihood in (19) does not change under reparametrization by including the redundant group mean ${\mu }_2$. We obtain

$$\left\{-l(0,{\sigma }_1,{\displaystyle 1},\mathit{b};{\mathcal{D}}_1)-l(0,{\sigma }_2,{\displaystyle 1},\mathit{b}+\mathit{e};{\mathcal{D}}_2)\right\}=\left\{-l(0,{\sigma }_1,{\displaystyle 1},\mathit{b};{\mathcal{D}}_1)-l({\mu }_2,{\sigma }_2,{\displaystyle 1},\mathit{b}+\tilde{\mathit{e}};{\mathcal{D}}_2)\right\}$$

(20)

for any ${\mu }_2$ and $\tilde{\mathit{e}}={\mu }_2+\mathit{e}$. Hence, the overidentified right-hand side of (20) can be made identifiable by adding a penalty function to the log-likelihood function. A condition for unbiased estimation of the group mean ${\mu }_2$ is to define DIF effects $e_i$ in such a way that ${\sum }_{i=1}^I\rho \left(e_i\right)=0$ is minimal [20]. This condition can also encode sparsity assumptions on DIF effects using the $L_p$ loss function and $p\to 0$. Importantly, robust linking can be defined as

$${\widehat{\mu }}_2=\underset{{\mu }_2}{arg\; min}\left\{\sum _{i=1}^I\rho ({\widehat{e}}_i+{\widehat{\mu }}_2)\right\},$$

(21)

which demonstrates the practical equivalence of robust linking and regularized estimation in sufficiently large samples.

3. Simulation Study 1: DIF Effects in the 1PL Model

In this Simulation Study 1, we compare robust linking and regularized estimation in the 1PL model in the presence of DIF effects. We consider the case of two groups.

3.1. Method

In this simulation study, we fixed the number of items to 20 and fixed item difficulties throughout all replications. Item parameters and DIF effects can be found at https://osf.io/tma3f/ (accessed on 8 December 2022). Item difficulties $b_i$ (see column “b”) ranged between $-1.88$ and $1.61$, with a mean of 0.00 and a standard deviation of 1.20. Items with DIF effects $e_i$ (column “e”) can also be found at https://osf.io/tma3f/ (accessed on 8 December 2022). The DIF effects $e_i$ had values $\{-1,0,1\}$. Only 4 out of 20 items were simulated to have DIF effects different from zero.

Item response data were generated according to the 1PL model using item difficulties $b_i$ in the first group and $b_i+\delta e_i$ in the second group. The DIF effect size $\delta$ was either 0.5 (i.e., small DIF effects) or 1.0 (i.e., large DIF effects). Furthermore, DIF was chosen to be balanced or unbalanced. In the balanced DIF condition, two items had DIF effects $-\delta$, and two items had DIF effects $\delta$. In the unbalanced DIF condition, all items had DIF effects $\delta$.

The distribution of the ability variable $\theta$ was assumed as standard normal (i.e., $\theta \sim \mathrm{N}(0,1)$) in the first group and had a mean of ${\mu }_2=0.3$ and a standard deviation ${\sigma }_2=1.2$ in the second group. Finally, we varied sample sizes per group N as 500, 1000, 2500, and 5000.

The different linking approaches presented for the 1PL model in Section 2 were evaluated in Simulation Study 1. In the scaling models, we used rectangular integration on a discrete quadrature of 41 equidistant $\theta$ points on $[-8.0,8.0]$. In concurrent calibration, we assumed invariant item parameters across groups. Regularized estimation was carried out using the SCAD penalty, and the regularized model was estimated at a grid of regularization parameters between 1.0 and 0.005 (see replication material on https://osf.io/tma3f/(accessed on 8 December 2022) for specification details). In this simulation study, the tuning parameter a was fixed to 3.7. We chose estimates of the regularization approach using the optimal regularization ${\lambda }_{\mathrm{opt}}$ based on AIC, BIC, and fixed $\lambda$ values of 0.05, 0.10, and 0.15 (see [38,69] for a similar approach). Moreover, powers $p=2$ ($L_2$; mean–mean linking), $p=1$ ($L_1$; median–median linking), and $p=0.5$ ($L_{0.5}$; invariance alignment) were used in the robust linking approach that utilizes the $L_p$ loss function. Finally, we determined the group mean of the second group on the outlier removal approach using the MAD statistic and a cutoff value of 2.7.

In total, 3000 replications were conducted in each simulation condition. We assessed the bias and root mean square error (RMSE) of the estimated group mean ${\widehat{\mu }}_2$. In each of the R replications in a simulation condition, the group mean ${\widehat{\mu }}_{2r}$ ($r=1,\dots ,R$) was estimated. The bias was estimated by

$$\mathrm{Bias}=\frac{1}{R}\sum _{r=1}^R({\widehat{\mu }}_{2r}-{\mu }_2).$$

(22)

The RMSE was estimated by

$$\mathrm{RMSE}=\frac{1}{R}\sum _{r=1}^R{({\widehat{\mu }}_{2r}-{\mu }_2)}^2.$$

(23)

To ease the comparability of the RMSE between different methods across sample sizes, we used a relative RMSE in which we divided the RMSE of a particular method by the RMSE of the best-performing method in a simulation condition. Hence, a relative RMSE of 100 is the reference value for the best-performing method.

The statistical software R [70] was employed for all parts of the simulation. Concurrent calibration in a multiple-group IRT model was estimated using the TAM package [71]. Regularized estimation was carried out using the xxirt function in the sirt package [72]. Replication material can be found at https://osf.io/tma3f/ (accessed on 8 December 2022).

3.2. Results

In

Table 1. Simulation Study 1: Bias of estimated group means for balanced and unbalanced DIF effects as a function of the size of DIF effects $\delta$ and sample size N.

			Choice of $\mathit{\lambda }$
$\mathit{\delta }$	N	MAD	AIC	BIC	0.05	0.10	0.15	${\mathit{L}}_{0.5}$	${\mathit{L}}_1$	${\mathit{L}}_2$	CC
		Balanced DIF
0.5	500	0.00	0.00	0.00	0.00	0.00	0.00	0.00	0.00	0.00	0.00
	1000	0.00	0.00	0.00	0.00	0.00	0.00	0.00	0.00	0.00	0.00
	2500	0.00	0.00	0.00	0.00	0.00	0.00	0.00	0.00	0.00	0.00
	5000	0.00	0.00	0.00	0.00	0.00	−0.01	0.00	0.00	0.00	0.00
1.0	500	0.00	0.00	0.00	0.00	0.00	0.00	0.00	0.00	0.00	0.00
	1000	0.00	0.00	0.00	0.00	0.00	0.00	0.00	0.00	0.00	0.00
	2500	0.00	0.00	0.00	0.00	0.00	0.00	0.00	0.00	0.00	0.00
	5000	0.00	0.00	0.00	0.00	0.00	0.00	0.00	0.00	0.00	0.00
		Unbalanced DIF
0.5	500	−0.06	−0.02	−0.03	−0.04	−0.02	−0.06	−0.03	−0.05	−0.10	−0.09
	1000	−0.03	−0.02	−0.01	−0.01	−0.01	−0.08	−0.02	−0.04	−0.10	−0.10
	2500	0.00	−0.02	−0.01	−0.01	−0.01	−0.09	−0.01	−0.02	−0.10	−0.10
	5000	0.00	−0.01	−0.01	−0.01	−0.01	−0.09	−0.01	−0.02	−0.10	−0.10
1.0	500	−0.01	−0.02	0.00	−0.05	0.00	0.00	−0.02	−0.05	−0.20	−0.18
	1000	0.00	−0.02	0.00	−0.02	0.00	0.00	−0.01	−0.04	−0.20	−0.18
	2500	0.00	−0.01	0.00	−0.02	0.00	0.00	−0.01	−0.02	−0.20	−0.18
	5000	0.00	−0.01	0.00	−0.03	0.00	0.00	−0.01	−0.02	−0.20	−0.18

Note. MAD = robust linking using the MAD statistic; Choice of $\lambda$ = method or value for determining the regularization parameter $\lambda$; L_p = linking employing the L_p loss function with p = 0.5, 1.0, or 2.0; CC = concurrent calibration assuming invariant item parameters; Absolute biases larger than 0.03 are printed in bold.

, the bias of the estimated group mean ${\widehat{\mu }}_2$ as a function of the size of the DIF effect $\delta$ and the sample size N is presented. It turned out that all methods performed well in the case of balanced DIF.

In the case of unbalanced DIF, nonrobust linking methods such as concurrent calibration (CC) and mean–mean linking ($L_2$) were substantially biased. Interestingly, there was also a bias for median–median linking ($L_1$) with unbalanced DIF. However, the bias decreased in larger samples. This can be expected because median–median linking (i.e., $L_1$ linking) recovers the true group difference in infinite sample sizes. In finite samples, estimated DIF effects with a true value of 0 will differ from 0, but have an expected mean of 0. As a consequence, the median of all estimated DIF effects (DIF items and non-DIF items) will be negative in the case of unbalanced DIF, because estimated DIF effects for DIF items are positive, resulting in a negative bias of the estimated group mean of the second group.

The robust $L_{0.5}$ linking approach performed well regarding the bias, particularly in larger samples. Overall, the estimated group means tended to be less biased when the BIC instead of the AIC was used in regularized estimation. However, the bias for robust linking and regularization approaches under unbalanced DIF turned out to be larger in conditions with small DIF effects of $\delta =0.5$ compared with large DIF effects of $\delta =1.0$.

Table 2. Simulation Study 1: Relative root mean sqaure error (RMSE) of estimated group means for balanced and unbalanced DIF effects as a function of the size of DIF effects $\delta$ and sample size N.

			Choice of $\mathit{\lambda }$
$\mathit{\delta }$	N	MAD	AIC	BIC	0.05	0.10	0.15	${\mathit{L}}_{0.5}$	${\mathit{L}}_1$	${\mathit{L}}_2$	CC
		Balanced DIF
0.5	500	111	115	111	120	110	111	122	110	101	100
	1000	111	112	106	109	108	118	115	108	101	100
	2500	104	133	103	122	118	135	114	108	101	100
	5000	103	147	129	139	126	153	111	107	100	100
1.0	500	107	111	105	115	106	104	118	109	102	100
	1000	104	110	103	108	103	103	115	108	102	100
	2500	105	113	104	112	103	103	114	109	102	100
	5000	104	111	103	126	103	103	112	108	102	100
		Unbalanced DIF
0.5	500	120	108	104	120	100	117	113	110	142	138
	1000	124	126	100	122	108	164	117	119	196	187
	2500	102	203	185	194	161	258	114	120	288	274
	5000	100	249	240	243	217	374	114	124	408	383
1.0	500	108	109	100	141	101	100	122	121	270	247
	1000	100	113	100	125	100	100	117	122	370	336
	2500	101	113	101	244	103	100	113	121	572	519
	5000	100	115	100	352	103	100	111	124	808	730

Note. MAD = robust linking using the MAD statistic; Choice of $\lambda$ = method or value for determining the regularization parameter $\lambda$; L_p = linking employing the L_p loss function with p = 0.5, 1.0, or 2.0; CC = concurrent calibration assuming invariant item parameters; Relative RMSE values larger than 125 are printed in bold.

presents the relative RMSE as a function of the size of DIF effects $\delta$ and the sample size N. It can be seen that CC is the frontrunner in terms of RMSE in the condition of unbalanced DIF. Notably, regularized estimation was particularly inefficient in larger samples of $N=2500$ or $N=5000$ in the condition of a small DIF effect. Furthermore, the robust linking approach based on the MAD statistic that performs outlier removal showed less variability than linking based on the $L_{0.5}$ invariance alignment loss function.

The situation changed in the conditions of unbalanced DIF. As expected, nonrobust linking approaches CC and $L_2$ had large RMSE values. In the case of small DIF effects (i.e., $\delta =0.5$), the performance of the methods depended on the sample size. For moderate sample sizes, $N=500$ and $N=1000$, regularized estimation based on the BIC was satisfactory, while the MAD approach would be preferred in larger sample sizes $N=2500$ and $N=5000$. Nevertheless, the $L_{0.5}$ robust linking approach was quite competitive across all sample sizes for small DIF effects. For large unbalanced DIF effects (i.e., $\delta =1.0$), regularized estimation based on BIC and robust linking using the MAD statistic outperformed other methods. Interestingly, regularized estimation using fixed $\lambda$ values of 0.10 or 0.15 also yielded satisfactory group mean estimates. Notably, the invariance alignment $L_{0.5}$ approach had some efficiency loss but might still be considered a viable alternative to the MAD linking and regularization approach.

4. Focused Simulation Study 1A: Optimal Choice of two Tuning Parameters for the SCAD Penalty

In this Focused Simulation Study 1A, we additionally investigated the impact of different choices of the second tuning parameter a for the SCAD penalty.

4.1. Method

This focused simulation study only considered the impact of the tuning parameter a for the SCAD penalty in regularized estimation. We only investigated selected conditions of Simulation Study 1. We focused on unbalanced DIF and chose sample sizes $N=500$, 1000, and 2000. Regularized estimation with the SCAD penalty was carried out using different a values of 2.2, 2.5, 3, 3.7, 4.5, 6, and 9. The same grid of $\lambda$ values as in Simulation Study 1 was employed (see Section 3.1). For each fixed a value, the optimal $\lambda$ value was determined by means AIC or BIC. In addition, we also determined estimated group means by choosing the optimal pair $({\lambda }_{\mathrm{opt}},a_{\mathrm{opt}})$ for AIC and BIC.

The performance of the different analytical choices was evaluated by using a relative RMSE. The relative RMSE was obtained by dividing the RMSE of a method by the RMSE of the best-performing method. The best-performing method could either be a regularized estimation with a particular choice of $\lambda$ and a or a regularized estimation with an optimal choice of $\lambda$ or a using AIC or BIC.

4.2. Results

Table 3. Focused Simulation Study 1A: Relative root mean square error (RMSE) of estimated group means for unbalanced DIF effects as a function of the size of DIF effects $\delta$ and sample size N for different values a of the SCAD penalty.

		Best		Choice of $\mathit{\lambda }$ Based on AIC with $\mathit{a}=$								Choice of $\mathit{\lambda }$ Based on BIC with $\mathit{a}=$
$\mathit{\delta }$	N	a	$\mathit{\lambda }$	2.2	2.5	3	3.7	4.5	6	9	${\mathit{a}}_{\mathbf{opt}}$	2.2	2.5	3	3.7	4.5	6	9	${\mathit{a}}_{\mathbf{opt}}$
0.5	500	9	0.04	110.8	111.2	110.9	111.8	110.2	110.3	109.6	108.0	103.4	103.4	103.3	103.4	103.4	103.5	104.0	104.0
	1000	3.7	BIC	128.7	130.1	129.5	127.7	129.4	126.1	122.7	121.4	100.2	100.0	100.1	100.0	100.2	100.1	100.4	100.2
	2500	2.2	0.19	139.2	138.6	137.7	136.3	138.1	137.8	132.3	129.4	126.0	126.6	125.2	122.7	125.1	124.6	117.8	111.3
1	500	3.7	0.13	111.4	110.4	110.5	108.7	108.7	107.9	109.9	108.6	102.5	102.4	102.5	100.3	100.2	100.2	100.4	100.6
	1000	3.7	0.13	113.1	112.5	112.5	112.9	112.1	111.2	108.9	110.3	100.6	100.6	100.7	100.6	100.7	100.6	100.9	100.8
	2500	9	0.08	126.9	119.8	120.4	119.5	120.4	114.4	116.4	122.8	111.3	103.4	103.4	105.8	103.4	103.4	103.3	111.2

Note. Best = pair of ($a$, $\lambda$) values in all estimated models that resulted in the least relative RMSE value of 100; Choice of $\lambda$ = method or value for determining the regularization parameter $\lambda$; and a_opt = choice of optimal a parameter based on AIC or BIC with corresponding optimal $\lambda$ parameter.

presents the relative RMSE of estimated group means for different choices of a and $\lambda$. Like in Simulation Study 1, the RMSE was smaller for methods that relied on BIC than on AIC. For methods based on AIC, it turned out that using the optimal a parameter across a range of a values resulted in the least RMSE for $N=500$ and $N=1000$. However, this was not the case for $N=2500$. Across different conditions, the choice $a=3.7$ did not result in estimated group means with the least RMSE values. This finding also occurred for regularized estimation based on BIC. However, differences between different choices of a values turned out to be smaller. For $N=500$ or $N=1000$, the choice of a for the SCAD penalty does not seem to matter. In contrast, using the optimal a value resulted in a substantial RMSE decrease for small DIF effects (i.e., $\delta =0.5$). On the other hand, using the optimal a value resulted in an RMSE increase for $N=2500$ in the presence of large DIF effects (i.e., $\delta =1$) compared with methods that use a fixed a value of the SCAD penalty. Interestingly, the least RMSE values could also be obtained with proper choices of fixed values of a and $\lambda$. In particular, for a large sample size of $N=2500$, relying on a fixed instead of an optimal $\lambda$ value can substantially decrease the RMSE.

As a conclusion of this focused simulation study, one could state that the choice of a for the SCAD penalty can have some impact. However, it is more important whether regularized estimation is carried out using AIC or BIC.

5. Simulation Study 2: Uniform DIF Effects in the 2PL Model

In this Simulation Study 2, robust linking and regularization estimation is compared in the 2PL IRT model in the presence of uniform DIF effects.

5.1. Method

The simulation design of Simulation Study 2 closely follows the design of Simulation Study 1 described in Section 3.1. We now only emphasize the differences between this study and the first one.

Notably, item response data were generated based on the 2PL model using the same invariant item discriminations across the two groups (see https://osf.io/tma3f/ (accessed on 8 December 2022)) for data-generating item parameters; see column “a” for item discriminations. Like in Simulation Study 1, item difficulties $b_i$ ranged between $-1.88$ and $1.61$ with a mean of 0.00 and a standard deviation of 1.20. Item discriminations $a_i$ ranged between $0.58$ and $1.57$ with a mean of 1.00 and a standard deviation of 0.30. Item difficulties and item discriminations were essentially uncorrelated ($r=-0.02$). DIF effects only occurred in item difficulties (i.e., uniform DIF). Like in Simulation Study 1, we set ${\mu }_2=0.3$ and ${\sigma }_2=1.2$ as data-generating distribution parameters in the second group. Moreover, the number of items (i.e., $I=20$) was also fixed throughout all conditions. DIF effects were either small ($\delta =0.5$) or large ($\delta =1.0$) and either balanced or unbalanced. Finally, we also simulated four different sample sizes N of 500, 1000, 2500, and 5000.

For regularized estimation, the same grid of $\lambda$ values like in Simulation Study 1 and Focused Simulation Study 1A was utilized. In this simulation study, the tuning parameter a was fixed to 3.7.

In the linking approaches, we either relied on a simultaneous first estimation step assuming common item discriminations or separate estimation assuming groupwise item discriminations. In contrast to Simulation Study 1, we also applied joint Haberman linking (JHL) to common item discriminations and ordinary Haberman linking (HL) to groupwise estimated item discriminations with powers $p=2$, 1, and 0.5.

In total, 3000 replications were simulated in each condition. R software [70] was used throughout the whole simulation. The R packages TAM [71] and sirt [72] were employed for estimating the IRT models. Replication material can again be found at https://osf.io/tma3f/ (accessed on 8 December 2022).

5.2. Results

Table 4. Simulation Study 2: Bias of estimated group means for balanced and unbalanced DIF effects as a function of the size of DIF effects $\delta$ and sample size N.

			Choice of $\mathit{\lambda }$					JHL with $\mathit{p}=$			HL with $\mathit{p}=$
$\mathit{\delta }$	N	MAD	AIC	BIC	0.05	0.10	0.15	0.5	1	2	0.5	1	2	${\mathit{L}}_{0.5}$	${\mathit{L}}_1$	${\mathit{L}}_2$	CC
		Balanced DIF
0.5	500	−0.01	−0.01	−0.01	−0.02	−0.01	−0.04	−0.01	−0.01	−0.01	0.00	0.00	0.01	−0.01	−0.01	−0.01	−0.04
	1000	0.01	0.00	0.01	0.00	0.01	−0.03	0.00	0.00	0.01	0.00	0.00	0.00	0.00	0.00	0.01	−0.04
	2500	−0.01	−0.01	−0.01	−0.01	0.00	−0.05	−0.01	−0.01	−0.01	0.00	0.00	0.00	−0.01	−0.01	−0.01	−0.04
	5000	0.00	0.00	0.01	0.01	0.01	−0.04	0.00	0.00	0.00	0.00	0.00	0.00	0.00	0.00	0.00	−0.04
1.0	500	−0.01	−0.02	−0.01	−0.02	−0.01	−0.01	−0.02	−0.02	−0.01	0.00	0.00	0.01	−0.02	−0.02	−0.01	−0.06
	1000	0.00	0.00	0.00	0.00	0.00	0.00	0.00	0.00	0.00	0.00	0.00	0.00	0.00	0.00	0.00	−0.06
	2500	−0.01	−0.02	−0.01	−0.01	−0.01	−0.01	−0.02	−0.01	−0.01	0.00	0.00	0.00	−0.02	−0.01	−0.01	−0.06
	5000	0.00	0.00	0.00	0.00	0.00	0.00	0.00	0.00	0.00	0.00	0.00	0.00	0.00	0.00	0.00	−0.06
		Unbalanced DIF
0.5	500	−0.07	−0.03	−0.04	−0.04	−0.03	−0.07	−0.04	−0.06	−0.10	−0.04	−0.06	−0.10	−0.04	−0.05	−0.10	−0.10
	1000	−0.03	−0.01	0.00	−0.01	−0.01	−0.07	−0.02	−0.04	−0.10	−0.03	−0.05	−0.10	−0.02	−0.03	−0.10	−0.10
	2500	−0.01	−0.03	−0.03	−0.03	−0.02	−0.10	−0.02	−0.04	−0.10	−0.02	−0.04	−0.10	−0.01	−0.03	−0.10	−0.10
	5000	0.01	−0.01	0.00	−0.01	0.00	−0.09	0.00	−0.02	−0.10	−0.01	−0.03	−0.10	0.00	−0.01	−0.10	−0.10
1.0	500	−0.03	−0.03	−0.02	−0.06	−0.02	−0.02	−0.03	−0.07	−0.21	−0.03	−0.07	−0.20	−0.04	−0.06	−0.21	−0.17
	1000	0.00	−0.01	0.00	−0.02	0.00	0.00	−0.01	−0.04	−0.20	−0.02	−0.05	−0.20	−0.01	−0.03	−0.20	−0.17
	2500	−0.01	−0.02	−0.02	−0.05	−0.02	−0.02	−0.02	−0.04	−0.21	−0.01	−0.04	−0.20	−0.02	−0.03	−0.21	−0.17
	5000	0.00	0.00	0.00	−0.02	0.00	0.00	0.00	−0.02	−0.20	−0.01	−0.03	−0.20	0.00	−0.01	−0.20	−0.17

Note. MAD = robust linking using the MAD statistic; Choice of $\lambda$ = method or valuze for determining the regularization parameter $\lambda$; JHL = joint Haberman linking using joint item discriminations; HL = Haberman linking using group-specific item discriminations; L_p = linking employing the unweighted L_p loss function with p = 0.5, 1.0, or 2.0 using joint item discriminations; CC = concurrent calibration assuming invariant item parameters; Absolute biases larger than 0.03 are printed in bold.

presents the bias for the estimated group mean ${\widehat{\mu }}_2$ as a function of the size of DIF effects $\delta$ and the sample size N. It turned out that all methods except concurrent calibration (CC) resulted in unbiased estimates of group means. Consistent with other studies, CC resulted in slightly biased estimates in the case of the 2PL model, even in the situation of balanced DIF effects. The reason might be that the presence of DIF negatively impacted the estimation of common item discriminations and the standard deviation of the second group.

In the case of unbalanced DIF, robust linking approaches based on $L_{0.5}$ or MAD, as well as the regularization approach based on BIC, performed satisfactorily in terms of bias. Joint Haberman linking (JHL) and Haberman linking based on separate calibration (HL) resulted in approximately unbiased estimates with $p=0.5$.

Table 5. Simulation Study 2: Relative root mean square error (RMSE) of estimated group means for balanced and unbalanced DIF effects as a function of the size of DIF effects $\delta$ and sample size N.

			Choice of $\mathit{\lambda }$					JHL with $\mathit{p}=$			HL with $\mathit{p}=$
$\mathit{\delta }$	N	MAD	AIC	BIC	0.05	0.10	0.15	0.5	1	2	0.5	1	2	${\mathit{L}}_{0.5}$	${\mathit{L}}_1$	${\mathit{L}}_2$	CC
		Balanced DIF
0.5	500	108	119	109	130	111	120	112	103	100	127	115	125	115	105	100	113
	1000	109	113	106	111	106	127	107	102	100	120	112	118	112	104	100	121
	2500	104	185	117	183	139	178	105	103	100	113	110	116	111	105	100	152
	5000	103	120	112	115	104	209	103	102	100	110	109	113	110	104	100	194
1.0	500	105	109	101	117	101	100	109	103	100	127	117	127	113	105	100	127
	1000	104	108	102	107	102	101	109	103	100	124	116	123	112	106	100	149
	2500	105	113	100	127	100	100	106	104	102	114	111	116	112	107	102	192
	5000	103	108	100	108	100	100	102	101	100	113	113	118	110	104	100	258
		Unbalanced DIF
0.5	500	118	118	101	130	100	121	103	107	140	119	118	148	109	106	140	135
	1000	126	136	100	133	113	163	107	118	190	126	137	201	116	115	190	192
	2500	105	293	272	299	212	269	107	134	288	118	146	284	113	123	288	276
	5000	102	276	279	279	253	356	100	123	375	115	156	391	109	110	375	384
1.0	500	109	115	100	146	107	105	110	125	265	128	141	271	120	123	265	231
	1000	100	114	105	131	105	105	105	122	359	124	145	366	113	118	359	315
	2500	101	179	169	308	163	171	108	144	536	113	146	529	112	131	536	459
	5000	103	117	100	345	100	100	104	135	776	116	161	778	112	118	776	677

Note. MAD = robust linking using the MAD statistic; Choice of $\lambda$ = method or valuze for determining the regularization parameter $\lambda$; JHL = joint Haberman linking using joint item discriminations; HL = Haberman linking using group-specific item discriminations; L^p = linking employing the unweighted L^p loss function with p = 0.5, 1.0, or 2.0 using joint item discriminations; CC = concurrent calibration assuming invariant item parameters; Relative RMSE values larger than 125 are printed in bold.

presents the relative RMSE of the estimated group mean as a function of the size of the DIF effect $\delta$ and the sample size N. In the case of unbalanced DIF, mean–mean linking $L_2$ and JHL performed the best across all conditions. It should be emphasized that the RMSE of CC became unacceptable in larger sample sizes due to bias. Moreover, a robust linking approach with powers $p=0.5$ or $p=1$ resulted in some efficiency losses.

The case of unbalanced DIF in the 2PL model was similar to the 1PL model presented in Simulation Study 1 in Section 3.2. For small DIF effects (i.e., $\delta =0.5$), regularized estimation using BIC was the frontrunner for moderate sample sizes $N=500$ and $N=1000$, while robust linking based on the MAD statistic performed well in larger sample sizes $N=2500$ and $N=5000$. For large DIF effects (i.e., $\delta =1.0$), robust linking using the MAD statistic performed satisfactorily. Moreover, JHL with $p=0.5$ and the robust $L_{0.5}$ linking also resulted in group mean estimates of acceptable variability.

6. Discussion

In this article, we investigated the performance of robust linking and regularized estimation in the presence of sparse uniform differential item functioning by means of three simulation studies. It turned out that robust linking approaches were competitive or outperformed regularized estimation in most conditions in the simulation studies. In particular, regularized estimation was not quite successful in parameter recovery in conditions with small DIF effects. It was also found that robust linking based on outlier removal using the MAD statistic [64] was often superior to robust linking using the $L_p$ loss function with $p=0.5$, which corresponds to invariance alignment. Overall, one could generally conclude that there is probably no need for using the computationally much more demanding regularization approaches instead of employing robust linking.

As in any simulation study, our findings are limited to the studied conditions. First, we only considered a fixed test of 20 items. Additional studies could also involve a larger number of items or balanced incomplete block designs for item response data [73]. Second, only 4 out of 20 items (i.e., 20% of the items) showed uniform DIF effects. Other research indicated that higher DIF rates would also be possible in robust linking, as long as the threshold of 50% of DIF items is not exceeded [21,24]. Third, we simulated uniform DIF under a sparsity condition. The 16 non-DIF items had DIF effects of exactly 0. Future research could also assume that there would also be small DIF effects that could add up to zero [26,74]. Fourth, future research could also investigate the case of nonuniform DIF in item discriminations. It could be that larger sample sizes would be required for robust linking in this situation, and regularized estimation might be advantageous.

In this article, we assumed that DIF effects potentially bias group mean differences. Hence, DIF effects should be essentially removed from group comparisons. However, it could be argued that eliminating items from comparisons poses a threat to validity [74,75,76,77], and statistical criteria should not determine which items enter group comparisons [78].