Comparing Different Specifications of Mean–Geometric Mean Linking

Abstract

Mean–geometric mean (MGM) linking compares group differences on a latent variable $\theta$ within the two-parameter logistic (2PL) item response theory model. This article investigates three specifications of MGM linking that differ in the weighting of item difficulty differences: unweighted (UW), discrimination-weighted (DW), and precision-weighted (PW). These methods are evaluated under conditions where random DIF effects are present in either item difficulties or item intercepts. The three estimators are analyzed both analytically and through a simulation study. The PW method outperforms the other two only in the absence of random DIF or in small samples when DIF is present. In larger samples, the UW method performs best when random DIF with homogeneous variances affects item difficulties, while the DW method achieves superior performance when such DIF is present in item intercepts. The analytical results and simulation findings consistently show that the PW method introduces bias in the estimated group mean when random DIF is present. Given that the effectiveness of MGM methods depends on the type of random DIF, the distribution of DIF effects was further examined using PISA 2006 reading data. The model comparisons indicate that random DIF with homogeneous variances in item intercepts provides a better fit than random DIF in item difficulties in the PISA 2006 reading dataset.

1. Introduction

Item response theory (IRT) models [1,2,3] provide a statistical framework for modeling multivariate discrete outcomes. This work specifically addresses binary item responses and explores methods for comparing two populations using linking techniques. Consider a response vector $\mathit{X}=(X_1,\dots ,X_I)$, where each variable $X_i\in \{0,1\}$ represents a dichotomously scored item. A unidimensional IRT model [4] specifies the joint probability distribution $\mathsf{P}(\mathit{X}=\mathit{x})$ for response patterns $\mathit{x}=(x_1,\dots ,x_I)\in {\{0,1\}}^I$ through a parametric formulation:

$${\displaystyle \mathsf{P}(\mathit{X}=\mathit{x};\mathbf{\delta },\mathit{\gamma })=\int \prod _{i=1}^I\left[{P_i(\theta ;{\mathit{\gamma }}_i)}^{x_i}{\left(1-P_i(\theta ;{\mathit{\gamma }}_i)\right)}^{1-x_i}\right]f(\theta ;\mu ,\sigma )\mathrm{d}\theta ,}$$

(1)

where f denotes the normal density function, parameterized by the mean $\mu$ and the standard deviation (SD) $\sigma$. The distribution parameters $\mu$ and $\sigma$ of the latent variable $\theta$, often referred to as a trait or ability variable, are collected in the vector $\mathbf{\delta }=(\mu ,\sigma )$. The vector $\mathit{\gamma }=({\mathit{\gamma }}_1,\dots ,{\mathit{\gamma }}_I)$ collects the item parameters for the item response functions (IRFs) $P_i(\theta ;{\mathit{\gamma }}_i)=\mathsf{P}(X_i=1|\theta )$ for $i=1,\dots ,I$. The IRF of the two-parameter logistic (2PL) model [5] is defined by

$$P_i(\theta ;{\mathit{\gamma }}_i)=\mathsf{\Psi }\left(a_i(\theta -b_i)\right),$$

(2)

where $a_i$ and $b_i$ represent the item discrimination and the item difficulty $b_i$, respectively. The function $\mathsf{\Psi }\left(x\right)={(1+exp(-x))}^{-1}$ corresponds to the standard logistic distribution function. In this formulation, the item parameter vector is ${\mathit{\gamma }}_i=(a_i,b_i)$. Alternatively, the 2PL model can be reparametrized by replacing the difficulty parameter $b_i$ with the intercept ${\nu }_i$, resulting in

$$P_i(\theta ;{\mathit{\gamma }}_i)=\mathsf{\Psi }\left(a_i\theta +{\nu }_i\right).$$

(3)

The two 2PL parameterizations are related by the identity ${\nu }_i=-a_i{b}_i$ (see [6,7]). In this parametrization, the item parameter vector is ${\mathit{\gamma }}_i=(a_i,{\nu }_i)$.

Given a sample of N individuals with independent and identically distributed response vectors ${\mathit{x}}_1,\dots ,{\mathit{x}}_N$ drawn from the distribution of $\mathit{X}$, the parameters of the IRT model specified in (1) can be consistently estimated through marginal maximum likelihood (MML) methods [8,9].

IRT models are widely employed to compare the test performance of two groups by assessing differences in the parameters of the latent variable $\theta$, as defined in the IRT framework of (1). This article specifically examines linking methods [10] based on the 2PL model.

In the first step of the linking approach, the 2PL model is estimated separately for each group, allowing for the presence of differential item functioning (DIF), where item behavior may vary across groups [11,12,13]. More specifically, item parameters are permitted to differ between groups, indicating that the groups may respond differently to an item even after accounting for overall differences in the $\theta$ variable. In the second step, differences in item parameters are used to estimate group differences in $\theta$ through a linking procedure [10,14,15].

This article evaluates the performance of mean–geometric mean (MGM; [7,10,16,17,18,19]) linking in the presence of DIF [13] in either item difficulties or item intercepts. The standard MGM method is based on the mean difference of log-transformed item discriminations for estimating the group SD and the mean difference of untransformed item difficulties for estimating the group mean. This study considers three specifications of MGM linking under random DIF [20,21,22] in item difficulties or intercepts. Prior research has shown that random DIF contributes to increased variance in the estimated linking parameters [23,24]. This concept is also referred to as a linking error in educational large-scale assessment studies [23,25,26,27,28,29,30,31,32].

The three MGM specifications considered here differ in the weighting of item difficulty differences when computing the group mean. To the best of the authors’ knowledge, the performance of these MGM variants under random DIF has not yet been systematically examined. The performance of the MGM estimators is assessed analytically and through a simulation study.

The remainder of the article is organized as follows. Section 2 reviews the MGM specifications. Section 3 presents the results from a simulation study comparing their performance. Section 4 provides an empirical illustration using PISA data. Section 5 concludes with a discussion.

2. Mean–Geometric Mean Linking

2.1. Identified Item Parameters in Separate Scaling

The various specifications of the MGM method are based on item parameters from the 2PL model, estimated separately for each group. The following describes the identification of item parameters under the assumption that no DIF is present in item discriminations $a_i$ or item difficulties $b_i$. In both groups, the latent variable $\theta$ is standardized by fixing its mean and SD to 0 and 1, respectively, allowing all item parameters to be estimated within each group. In the first group, the identified item parameters are given by ${\widehat{a}}_{i1}=a_i$, ${\widehat{b}}_{i1}=b_i$, and ${\widehat{\nu }}_{i1}={\nu }_i=-a_i{b}_i$, where $a_i$ and $b_i$ represent the invariant item parameters in the 2PL model across groups.

In the second group, the latent variable $\theta$ is assumed to have a mean $\mu$ and SD $\sigma$. By fixing the $\theta$ mean to 0 and SD to 1, the identified item parameters from the separate 2PL model estimation in this group are given by

$${\widehat{a}}_{i2}=\sigma a_i\mathrm{or}{\widehat{\alpha }}_{i2}=log{\widehat{a}}_{i2}=log\sigma +log{a}_i,$$

(4)

$${\widehat{b}}_{i2}={\sigma }^{-1}(b_i-\mu )\mathrm{and}$$

(5)

$${\widehat{\nu }}_{i2}=a_i\mu +{\nu }_i.$$

(6)

The MGM method aims to recover the parameters $\mu$ and $\sigma$ using the group-specific item parameters ${\widehat{a}}_{ig}$ and ${\widehat{b}}_{ig}$ ($i=1,\dots ,I$, $g=1,2$), obtained from separate estimations under the 2PL model.

2.2. Weighted Means

The different specifications of MGM linking for estimating $\mu$ are essentially different weighted means of item difficulty differences. The following briefly reviews the statistical properties of such weighted means. Let $Y_i$ ($i=1,\dots ,I$) denote normally distributed observations with mean $\mu$ and variances ${\sigma }_i^2$. A weighted mean with fixed weights $w_i$ is defined as

$${\overline{Y}}_w={\displaystyle \frac{{\displaystyle \frac{1}{I}\sum _{i=1}^I{w}_i{Y}_i}}{{\displaystyle \frac{1}{I}\sum _{i=1}^I{w}_i}}}.$$

(7)

Note that the multiplication factor $1/I$ in (7) could be omitted; however, it is included to maintain consistency with later expressions in the various specifications of MGM linking. The expected value of ${\overline{Y}}_w$ is $\mu$, and its variance is given by

$$\mathsf{Var}\left({\overline{Y}}_w\right)={\displaystyle \frac{{\displaystyle \sum _{i=1}^I{w}_i^2{\sigma }_i^2}}{{\displaystyle {\left(\sum _{i=1}^I{w}_i\right)}^2}}}.$$

(8)

The minimal variance in (8) is attained when the observations $Y_i$ are weighted by their precisions $1/{\sigma }_i^2$ (i.e., the inverses of their variances) (see [33]). A weighted mean using these weights is commonly referred to as a precision-weighted mean.

2.3. Random DIF in Item Difficulties or Item Intercepts

The occurrence of random DIF [20,34] can be characterized by whether DIF manifests in item difficulties [34] or item intercepts [35]. Assume that DIF arises only as deviations in item parameters for the second group relative to the first group. Further, assume invariant item parameters. Let $e_i$ and ${ϵ}_i$ denote random DIF in item difficulties and item intercepts with zero means, respectively, under the assumptions

$$b_{i2}=b_i+e_i\mathrm{and}{\nu }_{i2}={\nu }_i+{ϵ}_i.$$

(9)

The two DIF effects are related by

$${ϵ}_i=-a_i{e}_i\mathrm{or}{e}_i=-{\displaystyle \frac{{ϵ}_i}{a_i}}.$$

(10)

If the DIF effects $e_i$ in item difficulties have variances ${\tau }_i^2$, corresponding to random DIF, the variances of the DIF effects ${ϵ}_i$ in item intercepts are given by

$$\mathsf{Var}\left({ϵ}_i\right)=a_i^2\mathsf{Var}\left(e_i\right).$$

(11)

If the random DIF effects $e_i$ have homogeneous variances ${\tau }^2$, it follows from (11) that the corresponding DIF effects ${ϵ}_i$ in item intercepts exhibit heterogeneous variances $a_i^2{\tau }^2$. Conversely, if the DIF effects ${ϵ}_i$ in item intercepts have homogeneous variances ${\tau }^2$, then the DIF effects $e_i$ in item difficulties have heterogeneous variances ${\tau }^2/a_i^2$.

The formulation of random DIF in terms of item difficulties or item intercepts is statistically equivalent when allowing for heterogeneous variances ${\tau }_i^2$. However, empirical analyses may test whether random DIF with homogeneous variance is more plausible in item difficulties or in item intercepts. As will be shown later, the performance of the different MGM specifications depends on whether random DIF with homogeneous variances occurs in item difficulties or in item intercepts.

2.4. Estimation of $\sigma$ in MGM Linking

In MGM linking, the SD $\sigma$ is estimated using the means of log-transformed item discriminations. Specifically, the estimate $\widehat{\sigma }$ is computed as (see [10,16])

$$\widehat{\sigma }=exp\left({\displaystyle \frac{1}{I}}\sum _{i=1}^{I}log{\widehat{a}}_{i2}-{\displaystyle \frac{1}{I}}\sum _{i=1}^{I}log{\widehat{a}}_{i1}\right).$$

(12)

Since averages on the logarithmic scale are used, this method is referred to as log-mean-mean linking.

2.5. Estimation of $\mu$ in MGM Linking

The estimation of $\mu$ is now addressed. Three variants of weighted means of item difficulties are considered to derive the estimate $\widehat{\mu }$. In the formal treatment, assume that random DIF occurs in item difficulties, with $\mathsf{E}\left(e_i\right)=0$ and $\mathsf{Var}\left(e_i\right)={\tau }_i^2$. If random DIF in item difficulties exhibits homogeneous variance, then ${\tau }_i^2={\tau }^2$. Alternatively, if random DIF with homogeneous variances occurs in item intercepts, then ${\tau }_i^2={\tau }^2/a_i^2$.

In addition to random DIF, sampling errors affect the item parameter estimates. Let $u_{ig}$ denote the sampling error in the estimated item difficulty ($i=1,\dots ,I$; $g=1,2$). The estimated item difficulty in the first group is then given by

$${\widehat{b}}_{i1}=b_i+u_{i1}\mathrm{with}\mathsf{E}\left(u_{i1}\right)=0\mathrm{and}\mathsf{Var}\left(u_{i1}\right)\simeq {\displaystyle \frac{C_0+C_1{b}_i^2}{N{a}_i^2}},$$

(13)

where $C_0$ and $C_1$ are constants that depend on the dataset. The variance expression in (13) is supported by empirical evidence from simulation studies [9,36].

The estimated item difficulty in the second group satisfies

$${\widehat{b}}_{i2}={\displaystyle \frac{1}{\sigma }}(b_i+e_i-\mu )+u_{i2}\mathrm{with}\mathsf{E}\left(u_{i2}\right)=0\mathrm{and}\mathsf{Var}\left(u_{i2}\right)\simeq {\displaystyle \frac{{\displaystyle C_0+C_1\frac{1}{{\sigma }^2}{(b_i+e_i-\mu )}^2}}{N{a}_i^2{\sigma }^2}}.$$

(14)

Here, $u_{i2}$ represents the sampling error, while $e_i$ denotes the random DIF effect. For a sufficiently large number of items, the estimated item difficulties can be treated as approximately independent across items [36].

2.5.1. Unweighted MGM Linking (UW)

The original variant of MGM linking for estimating $\mu$ is based on the difference in item difficulties, defined as

$$\widehat{\mu }=-\left({\displaystyle \frac{1}{I}}\sum _{i=1}^I\widehat{\sigma }{\widehat{b}}_{i2}-{\displaystyle \frac{1}{I}}\sum _{i=1}^I{\widehat{b}}_{i1}\right)=-{\displaystyle \frac{1}{I}}\sum _{i=1}^I\left(\widehat{\sigma }{\widehat{b}}_{i2}-{\widehat{b}}_{i1}\right).$$

(15)

This estimator uses the previously calculated SD $\widehat{\sigma }$ from (12) and applies equal weights to the differences $\widehat{\sigma }{\widehat{b}}_{i2}-{\widehat{b}}_{i1}$ in item difficulties. For this reason, the estimator in (15) is referred to as unweighted MGM linking (UW).

To examine the statistical properties of $\widehat{\mu }$, the expression in (15) is rewritten using (13) and (14) as

$$\widehat{\mu }={\displaystyle \frac{\widehat{\sigma }}{\sigma }}\mu -{\displaystyle \frac{\widehat{\sigma }-\sigma }{\sigma }}·{\displaystyle \frac{1}{I}}\sum _{i=1}^I{b}_i-{\displaystyle \frac{\widehat{\sigma }}{\sigma }}·{\displaystyle \frac{1}{I}}\sum _{i=1}^I{e}_i-{\displaystyle \frac{1}{I}}\sum _{i=1}^I\left(\widehat{\sigma }{u}_{i2}-u_{i1}\right).$$

(16)

Since $\mathsf{E}\left(\widehat{\sigma }\right)\to \sigma$ as $I\to \infty$, it follows that $\mathsf{E}\left(\widehat{\mu }\right)\to \mu$ under the assumptions $\mathsf{E}\left(e_i\right)=\mathsf{E}\left(u_{ig}\right)=0$. Thus, a simplified form of $\widehat{\mu }$ is given by (16) as

$$\widehat{\mu }=\mu -{\displaystyle \frac{1}{I}}\sum _{i=1}^I{e}_i-{\displaystyle \frac{1}{I}}\sum _{i=1}^I\left(\widehat{\sigma }{u}_{i2}-u_{i1}\right).$$

(17)

To derive the variance of $\widehat{\mu }$, assume that $\mathsf{Var}\left(\widehat{\sigma }\right)\simeq 0$. Then, from (17),

$$\mathsf{Var}\left(\widehat{\mu }\right)={\displaystyle \frac{1}{I^2}}\sum _{i=1}^I{\tau }_i^2+{\displaystyle \frac{1}{I^2}}\sum _{i=1}^I\left[{\sigma }^2\mathsf{Var}\left(u_{i2}\right)+\mathsf{Var}\left(u_{i1}\right)\right].$$

(18)

Equation (18) shows that the variance of $\widehat{\mu }$ consists of two components: the variance due to random DIF effects $e_i$ and the variance from sampling errors $u_{ig}$. Using (13) and (14), the expression can be further rephrased as

$$\mathsf{Var}\left(\widehat{\mu }\right)={\displaystyle \frac{1}{I^2}}\sum _{i=1}^I{\tau }_i^2+{\displaystyle \frac{1}{N{I}^2}}\sum _{i=1}^I{\displaystyle \frac{{\displaystyle C_0({\sigma }^2+1)+C_1\left[{(b_i+e_i-\mu )}^2+b_i^2\right]}}{a_i^2}}.$$

(19)

As the sample size increases, the contribution of the sampling error variance diminishes. However, the variance component due to random DIF remains nonzero even in the limit of infinite sample size.

2.5.2. Discrimination-Weighted MGM Linking (DW)

An alternative MGM linking estimate for $\mu$ relies on the identification of Equation (6). Following the rationale used in the invariance alignment [37,38] method, the absence of DIF effects yields the identity

$${\widehat{\nu }}_{i2}-{\widehat{\nu }}_{i1}-{\displaystyle \frac{{\widehat{a}}_{i2}}{\sigma }}\mu =0.$$

(20)

This identity motivates the estimation of $\mu$ as the minimizer of

$$H\left(\mu \right)=\sum _{i=1}^I{\left({\widehat{\nu }}_{i2}-{\widehat{\nu }}_{i1}-{\displaystyle \frac{{\widehat{a}}_{i2}}{\widehat{\sigma }}}\mu \right)}^2,$$

(21)

which leads to the estimator

$$\widehat{\mu }=\widehat{\sigma }{\displaystyle \frac{{\displaystyle \frac{1}{I}\sum _{i=1}^I\left({\widehat{\nu }}_{i2}-{\widehat{\nu }}_{i1}\right){\widehat{a}}_{i2}}}{{\displaystyle \frac{1}{I}\sum _{i=1}^I{\widehat{a}}_{i2}^2}}}.$$

(22)

The estimator $\widehat{\mu }$ in (22) can be further rewritten as (see [19])

$$\widehat{\mu }=\widehat{\sigma }{\displaystyle \frac{{\displaystyle \frac{1}{I}\sum _{i=1}^I\left({\widehat{a}}_{i2}{\widehat{b}}_{i2}-{\widehat{a}}_{i1}{\widehat{b}}_{i1}\right){\widehat{a}}_{i2}}}{{\displaystyle \frac{1}{I}\sum _{i=1}^I{\widehat{a}}_{i2}^2}}}.$$

(23)

To analyze the statistical properties of $\widehat{\mu }$ as defined in (23), simplifying assumptions are applied: $\mathsf{Var}\left(\widehat{\sigma }\right)\simeq 0$, ${\widehat{a}}_{i1}\simeq a_i$, and ${\widehat{a}}_{i2}\simeq \sigma a_i$. Under these assumptions, the estimator simplifies to

$$\widehat{\mu }={\displaystyle \frac{{\displaystyle \sum _{i=1}^I{a}_i^2\left(\sigma {\widehat{b}}_{i2}-{\widehat{b}}_{i1}\right)}}{{\displaystyle \sum _{i=1}^I{a}_i^2}}}.$$

(24)

This expression reveals that $\widehat{\mu }$ is a weighted average of item difficulty differences, where the weights are proportional to the squared item discriminations. Consequently, the estimator in (24) is referred to as discrimination-based MGM linking (DW).

The estimator $\widehat{\mu }$ in (24) can be expressed in terms of the DIF effects $e_i$ and the sampling errors $u_{ig}$ as

$$\widehat{\mu }=\mu +{\displaystyle \frac{{\displaystyle \sum _{i=1}^I{a}_i^2{e}_i}}{{\displaystyle \sum _{i=1}^I{a}_i^2}}}+{\displaystyle \frac{{\displaystyle \sum _{i=1}^I{a}_i^2\left(\sigma u_{i2}-u_{i1}\right)}}{{\displaystyle \sum _{i=1}^I{a}_i^2}}}.$$

(25)

As with the UW estimator, this formulation yields an asymptotically unbiased estimate of $\mu$, i.e., $\mathsf{E}\left(\widehat{\mu }\right)\to \mu$ as $I\to \infty$. The variance of $\widehat{\mu }$ is given by

$$\mathsf{Var}\left(\widehat{\mu }\right)={\displaystyle \frac{{\displaystyle \sum _{i=1}^I{a}_i^4{\tau }_i^2}}{{\displaystyle {\left(\sum _{i=1}^I{a}_i^2\right)}^2}}}+{\displaystyle \frac{{\displaystyle \sum _{i=1}^I{a}_i^4\left[{\sigma }^2\mathsf{Var}\left(u_{i2}\right)+\mathsf{Var}\left(u_{i1}\right)\right]}}{{\displaystyle {\left(\sum _{i=1}^I{a}_i^2\right)}^2}}},$$

(26)

which can be further simplified using the expressions for $\mathsf{Var}\left(u_{ig}\right)$ from (19) as

$$\mathsf{Var}\left(\widehat{\mu }\right)={\displaystyle \frac{{\displaystyle \sum _{i=1}^I{a}_i^4{\tau }_i^2}}{{\displaystyle {\left(\sum _{i=1}^I{a}_i^2\right)}^2}}}+{\displaystyle \frac{1}{N}}·{\displaystyle \frac{{\displaystyle \sum _{i=1}^I{a}_i^2\left\{{\displaystyle C_0({\sigma }^2+1)+C_1\left[{(b_i+e_i-\mu )}^2+b_i^2\right]}\right\}}}{{\displaystyle {\left(\sum _{i=1}^I{a}_i^2\right)}^2}}}.$$

(27)

The left-hand variance component in (27) indicates that an optimal estimate of $\mu$ is obtained when the DIF effects $e_i$ satisfy $\mathsf{Var}\left(e_i\right)=a_i^{-2}{\tau }^2$, corresponding to random DIF with homogeneous variances in item intercepts. In this case, the weighting by item discriminations enhances precision, as the sampling variance of estimated item difficulties is also proportional to $a_i^{-2}$. However, if random DIF with homogeneous variance occurs in item difficulties rather than in item intercepts, the discrimination-based weighting in DW may result in a higher variance compared to the equal weighting used in UW, particularly in large samples where the contribution from sampling error becomes negligible.

2.5.3. Precision-Weighted MGM Linking (PW)

The UW linking method assigns equal weights to the item difficulty differences $\widehat{\sigma }{\widehat{b}}_{i2}-{\widehat{b}}_{i1}$. As an alternative, these differences can be weighted by their precisions, that is, the inverse of their sampling variances [7,39]. This approach yields the estimator

$$\widehat{\mu }=-{\displaystyle \frac{{\displaystyle \sum _{i=1}^I{\omega }_i\left(\sigma {\widehat{b}}_{i2}-{\widehat{b}}_{i1}\right)}}{{\displaystyle \sum _{i=1}^I{\omega }_i}}},$$

(28)

where the precision weights ${\omega }_i$ must be estimated. The variances $\mathsf{Var}\left({\widehat{b}}_{i1}\right)$ and $\mathsf{Var}\left({\widehat{b}}_{i2}\right)$ are obtained from the observed information matrix in the group-wise scaling models. Based on these variances, the weights ${\omega }_i$ are defined as

$${\omega }_i={\left[{\widehat{\sigma }}^2\mathsf{Var}\left({\widehat{b}}_{i2}\right)+\mathsf{Var}\left({\widehat{b}}_{i1}\right)\right]}^{-1}.$$

(29)

Using (13) and (14), the precision weights can be approximately determined as

$${\omega }_i={\left[{\widehat{\sigma }}^2\mathsf{Var}\left(u_{i2}\right)+\mathsf{Var}\left(u_{i1}\right)\right]}^{-1}={\displaystyle \frac{N{a}_i^2}{{\displaystyle C_0({\sigma }^2+1)+C_1\left[{(b_i+e_i-\mu )}^2+b_i^2\right]}}}.$$

(30)

Importantly, (30) highlights that the estimated precision weights ${\omega }_i$ depend on the random DIF effects $e_i$. For small values of $e_i$, a linear Taylor approximation of (30) yields

$${\omega }_i\simeq {\omega }_{i0}-{\omega }_{i1}{e}_i(b_i-\mu ),\mathrm{where}$$

(31)

$${\omega }_{i0}=N{a}_i^2{h}_i,{\omega }_{i1}=2C_{1}N{a}_i^2{h}_i^2,\mathrm{and}{h}_i={\left\{{\displaystyle C_0({\sigma }^2+1)+C_1\left[{(b_i-\mu )}^2+b_i^2\right]}\right\}}^{-1}.$$

(32)

Note that ${\omega }_{i0}$ and ${\omega }_{i1}$ are independent of the random DIF effect $e_i$.

The estimator $\widehat{\mu }$ in (28) can be rephrased as

$$\widehat{\mu }\simeq \mu -{\displaystyle \frac{{\displaystyle \sum _{i=1}^I\left[{\omega }_{i0}-{\omega }_{i1}{e}_i(b_i-\mu )\right]e_i}}{{\displaystyle \sum _{i=1}^I\left({\omega }_{i0}-{\omega }_{i1}{e}_i\right)}}}-{\displaystyle \frac{{\displaystyle \sum _{i=1}^I\left[{\omega }_{i0}-{\omega }_{i1}{e}_i(b_i-\mu )\right]\left(u_{i2}-u_{i1}\right)}}{{\displaystyle \sum _{i=1}^I\left({\omega }_{i0}-{\omega }_{i1}{e}_i\right)}}}.$$

(33)

Assuming independence between $e_i$ and $u_{ig}$, the expectation of $\widehat{\mu }$ as $I\to \infty$ is given by

$$\mathsf{E}\left(\widehat{\mu }\right)\to \mu -{\displaystyle \frac{{\displaystyle \sum _{i=1}^I{\omega }_{i1}{\tau }_i^2(\mu -b_i)}}{{\displaystyle \sum _{i=1}^I{\omega }_{i0}}}}\mathrm{for}I\to \infty .$$

(34)

According to (34), the PW linking method may produce a negatively biased estimate of $\mu$ when $\mu$ is, on average, greater than $b_i$. However, in the absence of random DIF, the PW method does not exhibit bias in the estimation of $\mu$.

The variance of the PW estimate can be derived analogously to that of the UW and DW estimators, although it offers limited additional insight. By construction, the PW linking method yields the smallest variance in the absence of DIF, as it employs optimal precision weights.

3. Simulation Study

In this Simulation Study, the performances of the three MGM linking specifications (i.e., UW, DW, and PW) outlined in Section 2.5 are compared.

3.1. Method

The data-generating model was based on the 2PL model applied to two groups. For the first group, the latent variable $\theta$ followed a standard normal distribution with a fixed mean of 0 and SD of 1. For the second group, $\theta$ was also normally distributed with a fixed mean $\mu =0.3$ and SD $\sigma =1.2$, which was consistent across all simulation conditions.

The simulation study used $I=20$ or $I=40$ items. Group-specific item parameters $a_{ig}$ and $b_{ig}$ for each item $i=1,\dots ,I$ and for groups $g=1,2$ were derived from fixed base parameters and newly simulated random DIF effects in each replication. The item parameters were constructed using 10 base items. These base items were duplicated twice in the 20-item condition and four times in the 40-item condition. For the 10 base items, the base item discriminations $a_{i0}$ were set to 0.6 for the first five items and 1.2 for the remaining five. Base item difficulties were assigned values of $-1.4$, $-0.7$, $0.0$, $0.7$, and $1.4$ for the first five items, with the same sequence repeated for the remaining items. The complete set of item parameters is available at https://osf.io/xa4qz (accessed on 3 May 2025).

For the first group, item discriminations and item difficulties were set to the base item parameters. In the second group, DIF effects with a homogeneous variance were introduced either in item difficulties or item intercepts. A normally distributed random DIF effect with DIF SD $\tau$ was added to the corresponding item difficulty or item intercept. The DIF SD $\tau$ was chosen as 0, 0.25, or 0.5. Combined with the type of DIF effects (i.e., in item difficulties $b_i$ or item intercepts ${\nu }_i$), five different DIF conditions (i.e., no DIF, $\tau =0.25$ and DIF in $b_i$, $\tau =0.5$ and DIF in $b_i$, $\tau =0.25$ and DIF in ${\nu }_i$, and $\tau =0.5$ and DIF in ${\nu }_i$) were simulated. Item discriminations in the second group were kept identical to the base values, ensuring no DIF in discrimination parameters.

Per-group sample sizes of $N=500$, 1000, 2000, and infinity (denoted as Inf) were selected to represent typical ranges encountered in medium- to large-scale testing scenarios involving the 2PL model [40]. For infinite sample sizes, no item responses were simulated. However, the item parameters used in MGM linking still included the random DIF effects in this case.

In each of the 4 (sample size N) × 2 (number of items I) × 5 (random DIF conditions) $=40$ simulation conditions, 7500 replications were conducted. The three MGM specifications—UW, DW, and PW—were applied to the simulated datasets. The bias, SD, and root mean square error (RMSE) of the estimated mean $\widehat{\mu }$ were computed. The relative RMSE of the $\widehat{\mu }$ estimator was defined as the RMSE of a given method divided by the RMSE of the UW method, which served as the reference.

All analyses in this simulation study were performed using R (Version 4.4.1; [41]). The 2PL model was fitted using the sirt::xxirt() function from the R package sirt (Version 4.2-114; [42]). Dedicated functions were developed to estimate the different MGM models. Replication materials for this study can be accessed at https://osf.io/xa4qz (accessed on 3 May 2025).

3.2. Results

Table 1 presents the bias of the estimated group mean $\widehat{\mu }$ as a function of the number of items I and the sample size N. In the absence of DIF ($\tau =0$), all three MGM methods yielded unbiased estimates. When DIF was present in either item difficulties $b_i$ or item intercepts ${\nu }_i$, the UW and DW methods continued to produce unbiased estimates. Consistent with the analytical findings in Section 2.5.3, the PW method exhibited bias under these conditions. The magnitude of the bias increased with larger DIF SD $\tau$. Notably, the bias of the PW method did not diminish with increasing sample size N.

Table 1. Simulation Study: Bias of estimated mean $\widehat{\mu }$ as a function of the DIF SD $\tau$, the type of DIF effects, the number of items I, and sample size N.

		$\mathit{\tau }=0$				$\mathit{\tau }=0.25$, DIF in ${\mathit{b}}_{\mathit{i}}$				$\mathit{\tau }=0.5$, DIF in ${\mathit{b}}_{\mathit{i}}$				$\mathit{\tau }=0.25$, DIF in ${\mathit{\nu }}_{\mathit{i}}$				$\mathit{\tau }=0.5$, DIF in ${\mathit{\nu }}_{\mathit{i}}$
$\mathit{I}$	$\mathit{N}$	UW	DW	PW		UW	DW	PW		UW	DW	PW		UW	DW	PW		UW	DW	PW
20	500	0.003	−0.003	−0.007		0.002	−0.003	−0.014		0.009	0.001	−0.026		0.004	−0.002	−0.013		0.005	0.000	−0.029
	1000	0.002	−0.002	−0.004		0.002	−0.002	−0.011		0.001	−0.003	−0.029		0.003	−0.001	−0.011		0.000	−0.005	−0.033
	2000	0.001	0.000	−0.001		0.000	−0.001	−0.009		0.000	0.000	−0.027		−0.001	0.000	−0.011		−0.001	0.000	−0.029
	Inf	0.000	0.000	—		0.000	0.000	—		0.000	0.000	—		0.000	−0.001	—		−0.003	−0.001	—
40	500	0.005	0.003	−0.002		0.004	0.003	−0.008		0.004	0.006	−0.024		0.005	0.004	−0.008		0.005	0.009	−0.022
	1000	0.003	−0.001	−0.002		0.001	−0.002	−0.010		0.004	0.001	−0.025		0.001	−0.001	−0.011		−0.001	−0.004	−0.031
	2000	−0.003	0.001	−0.002		−0.003	0.001	−0.009		−0.004	0.001	−0.027		−0.002	0.002	−0.008		−0.004	0.002	−0.027
	Inf	0.000	0.000	—		0.001	0.001	—		0.000	0.001	—		−0.001	−0.001	—		0.002	0.001	—

Note. SD = standard deviation; Inf = infinite sample size; UW = unweighted mean–geometric mean linking; DW = discrimination-weighted mean–geometric mean linking; PW = precision-weighted mean–geometric mean linking; — = linking method not applied; Values of absolute bias larger than 0.010 are printed in bold font.

Table 2 reports the SD of the estimated group mean $\widehat{\mu }$ as a function of the number of items I and the sample size N. As expected, the SD decreased with increasing sample size and increased with higher DIF SD $\tau$. The SD also declined with a larger number of items. In the no DIF condition ($\tau =0$), the PW method produced estimates with the lowest SD, followed by the DW and UW methods. When random DIF was present in item difficulties $b_i$, PW resulted in the smallest SD for smaller sample sizes, whereas UW became more efficient than DW and PW as the sample size increased. A comparable pattern emerged for DIF in item intercepts ${\nu }_i$, with the distinction that DW instead of UW yielded the smallest SD in larger samples.

Table 2. Simulation Study: Standard deviation (SD) of estimated mean $\widehat{\mu }$ as a function of the DIF SD $\tau$, the type of DIF effects, the number of items I, and sample size N.

		$\mathit{\tau }=0$				$\mathit{\tau }=0.25$, DIF in ${\mathit{b}}_{\mathit{i}}$				$\mathit{\tau }=0.5$, DIF in ${\mathit{b}}_{\mathit{i}}$				$\mathit{\tau }=0.25$, DIF in ${\mathit{\nu }}_{\mathit{i}}$				$\mathit{\tau }=0.5$, DIF in ${\mathit{\nu }}_{\mathit{i}}$
$\mathit{I}$	$\mathit{N}$	UW	DW	PW		UW	DW	PW		UW	DW	PW		UW	DW	PW		UW	DW	PW
20	500	0.095	0.085	0.082		0.110	0.107	0.103		0.150	0.160	0.149		0.120	0.102	0.101		0.179	0.147	0.142
	1000	0.066	0.058	0.058		0.087	0.088	0.087		0.131	0.145	0.136		0.099	0.084	0.085		0.166	0.134	0.130
	2000	0.046	0.042	0.042		0.074	0.079	0.078		0.122	0.139	0.131		0.087	0.073	0.075		0.156	0.127	0.125
	Inf	0.000	0.000	—		0.056	0.066	—		0.112	0.130	—		0.072	0.058	—		0.147	0.118	—
40	500	0.083	0.079	0.077		0.090	0.091	0.087		0.116	0.124	0.114		0.099	0.089	0.087		0.135	0.117	0.109
	1000	0.058	0.054	0.054		0.071	0.071	0.069		0.098	0.106	0.099		0.078	0.069	0.069		0.120	0.100	0.097
	2000	0.040	0.038	0.038		0.056	0.060	0.059		0.089	0.103	0.095		0.065	0.057	0.057		0.111	0.092	0.089
	Inf	0.000	0.000	—		0.039	0.046	—		0.079	0.092	—		0.052	0.042	—		0.105	0.084	—

Note. Inf = infinite sample size; UW = unweighted mean–geometric mean linking; DW = discrimination-weighted mean–geometric mean linking; PW = precision-weighted mean–geometric mean linking; — = linking method not applied.

Table 3 presents the relative RMSE of the estimated group mean $\widehat{\mu }$ as a function of the number of items I and the sample size N. The PW method exhibited the lowest RMSE in the no DIF condition and in DIF conditions with small sample sizes. In large samples, the UW method yielded the smallest RMSE when DIF was present in item difficulties $b_i$. In contrast, in conditions with DIF in item intercepts ${\nu }_i$, the DW and PW methods outperformed UW, with DW showing a slight efficiency advantage over PW at larger sample sizes.

Table 3. Simulation Study: Relative root mean square error (RMSE) of estimated mean $\widehat{\mu }$ as a function of the DIF SD $\tau$, the type of DIF effects, the number of items I, and sample size N.

		$\mathit{\tau }=0$				$\mathit{\tau }=0.25$, DIF in ${\mathit{b}}_{\mathit{i}}$				$\mathit{\tau }=0.5$, DIF in ${\mathit{b}}_{\mathit{i}}$				$\mathit{\tau }=0.25$, DIF in ${\mathit{\nu }}_{\mathit{i}}$				$\mathit{\tau }=0.5$, DIF in ${\mathit{\nu }}_{\mathit{i}}$
$\mathit{I}$	$\mathit{N}$	UW	DW	PW		UW	DW	PW		UW	DW	PW		UW	DW	PW		UW	DW	PW
20	500	100	89.0	86.8		100	97.0	94.7		100	106.1	100.7		100	85.0	85.1		100	82.5	80.8
	1000	100	87.8	87.5		100	101.0	99.8		100	111.0	106.4		100	84.2	85.9		100	80.8	81.1
	2000	100	91.9	91.4		100	107.0	106.3		100	114.2	109.4		100	83.9	87.1		100	81.4	82.1
	Inf	—	—	—		100	116.7	—		100	116.1	—		100	80.7	—		100	79.8	—
40	500	100	95.3	92.9		100	100.3	96.3		100	106.7	99.9		100	90.2	87.8		100	86.2	82.2
	1000	100	93.0	93.1		100	100.0	98.8		100	109.0	104.3		100	87.7	88.9		100	83.4	84.3
	2000	100	96.0	94.5		100	107.6	106.1		100	115.2	111.4		100	86.7	87.7		100	83.2	84.0
	Inf	—	—	—		100	116.2	—		100	116.4	—		100	79.5	—		100	79.9	—

Note. SD = standard deviation; Inf = infinite sample size; — = linking method not applied; UW = unweighted mean–geometric mean linking; DW = discrimination-weighted mean–geometric mean linking; PW = precision-weighted mean–geometric mean linking; The UW method was used as the reference method to compute the relative RMSE.

Overall, the results of this Simulation Study showed that the performance of the UW, DW, and PW methods depended on the type of simulated DIF effects. The PW linking method yielded unbiased and efficient estimates only in the absence of random DIF. In DIF conditions with item difficulties affected, the UW method outperformed DW. Conversely, when DIF was simulated in item intercepts, the DW method was superior to UW.

4. Empirical Example: PISA 2006 Reading

The Simulation Study presented in Section 3 demonstrates that the performance of the three MGM methods depends on the presence and nature of random DIF in the data. To investigate whether random DIF occurs in item intercepts or item difficulties, the PISA 2006 dataset [43] for the reading domain was analyzed. This dataset includes participants from 26 selected countries (see Appendix A) that participated in the PISA 2006 study. The full PISA 2006 dataset is publicly accessible at https://www.oecd.org/en/data/datasets/pisa-2006-database.html as of (accessed on 3 May 2025).

Items in the reading domain were administered to a subset of students participating in the PISA 2006 study. The analysis included students who had been administered at least one item from the respective cognitive domain. In total, the analysis included 110,236 students, with sample sizes per country ranging from 2010 to 12,142 ($M=4239.8$, $SD=3046.7$).

A few of the 28 reading items were originally scored polytomous but were recoded into dichotomous scores for simplicity in this empirical example, with only the highest category considered correct. The other items were handled as dichotomous, consistent with the original treatment in PISA.

Student (sampling) weights were applied in all analyses. To guarantee equal influence from each country, weights within each country were normalized to sum to 5000. It should be noted that the choice of 5000 is arbitrary; any constant value would serve equally well to balance contributions across countries.

In the first step, international item parameters were estimated by fitting the 2PL model to the weighted, combined dataset for each domain. These item parameters, along with other relevant information, are presented in Table 4. The average item discrimination was 1.402, suggesting a relatively well-discriminating test, while the average item difficulty was $-0.163$, indicating that the items were slightly easier relative to the ability of students in the total population.

Table 4. Empirical Example, PISA 2006 Reading: International item parameters and descriptive statistics of DIF effects for all 28 items.

				DIF Effects
Item	#CNT	${\mathit{a}}_{\mathit{i}}$	${\mathit{b}}_{\mathit{i}}$	${\widehat{\mathit{\tau }}}_{\mathrm{obs}}$	${\widehat{\mathit{\tau }}}_{\mathrm{bc}}$	Min	Max	p(SW)
R055Q01	26	1.395	−1.486	0.218	0.210	−0.447	0.654	0.124
R055Q02	26	1.379	0.043	0.214	0.207	−0.394	0.411	0.522
R055Q03	26	1.620	−0.335	0.279	0.272	−0.445	0.496	0.095
R055Q05	26	2.117	−0.777	0.188	0.182	−0.353	0.644	0.002
R067Q01	26	1.228	−2.069	0.350	0.339	−0.664	1.022	0.033
R067Q04	26	0.832	0.723	0.710	0.694	−2.041	0.976	0.017
R067Q05	26	1.088	−0.307	0.526	0.513	−1.258	1.164	0.508
R102Q04A	25	1.460	0.669	0.383	0.373	−0.604	0.783	0.266
R102Q05	26	1.330	0.244	0.298	0.290	−0.611	0.435	0.073
R102Q07	24	1.418	−1.493	0.427	0.416	−0.680	0.821	0.083
R104Q01	26	1.628	−1.321	0.185	0.178	−0.267	0.445	0.122
R104Q02	26	0.583	1.337	0.685	0.664	−0.873	2.194	0.008
R104Q05	26	1.206	2.974	0.449	0.428	−0.674	1.066	0.530
R111Q01	26	1.365	−0.604	0.259	0.251	−0.400	0.558	0.366
R111Q02B	26	1.044	1.917	0.500	0.486	−0.858	1.027	0.738
R111Q06B	26	1.589	0.542	0.224	0.217	−0.589	0.307	0.124
R219Q01E	26	1.633	−0.250	0.295	0.287	−1.042	0.541	0.006
R219Q01T	26	1.861	−0.667	0.242	0.235	−0.522	0.478	0.986
R219Q02	26	1.534	−1.179	0.229	0.221	−0.423	0.346	0.451
R220Q01	26	1.762	0.308	0.211	0.205	−0.317	0.460	0.377
R220Q02B	25	1.521	−0.376	0.159	0.152	−0.221	0.338	0.143
R220Q04	26	1.302	−0.312	0.320	0.312	−0.546	0.373	0.003
R220Q05	26	1.977	−1.145	0.165	0.157	−0.370	0.286	0.297
R220Q06	26	1.167	−0.675	0.393	0.383	−0.500	0.688	0.014
R227Q01	26	0.778	−0.151	0.671	0.655	−1.550	1.275	0.827
R227Q02T	26	0.994	0.792	0.629	0.614	−0.995	1.437	0.738
R227Q03	26	1.665	−0.183	0.235	0.227	−0.650	0.484	0.557
R227Q06	26	1.766	−0.777	0.225	0.218	−0.314	0.555	0.021

Note. #CNT = number of countries per item; $a_i$ = item discrimination in the 2PL model; $b_i$ = item difficulty in the 2PL model; ${\widehat{\tau }}_{\mathrm{obs}}$ = observed SD of DIF effects; ${\widehat{\tau }}_{\mathrm{bc}}$ = bias-corrected estimate of SD of DIF effects; Min = smallest DIF effect per item across countries; Max = largest DIF effect per item across countries; p(SW) = p-value of Shapiro–Wilk test for normality of DIF effects.

In the second step, country means and country SDs were computed using the fixed international item parameters presented in Table 4. The means and SDs for the 26 countries, based on the original logit scale of the 2PL model, are reported in Table 5. The country means had an average of $M=0.000$ (with $SD=0.228$), while the country SDs had an average of $M=0.973$ (with $SD=0.078$).

Table 5. Empirical Example, PISA 2006 Reading: Descriptive statistics for countries and estimated SD of DIF effects.

CNT	N	I	M	SD	${\widehat{\mathit{\tau }}}_{\mathbf{obs}}$	${\widehat{\mathit{\tau }}}_{\mathbf{bc}}$	${\widehat{\mathit{\tau }}}_{\mathit{\nu },\mathbf{bc}}$	$\mathbf{\Delta }\mathbf{LL}$
AUS	7562	28	0.170	0.960	0.249	0.246	0.350	−1.46
AUT	2646	27	−0.037	1.033	0.272	0.265	0.310	3.86
BEL	4840	28	0.059	1.071	0.266	0.255	0.307	3.21
CAN	12,142	28	0.276	0.934	0.283	0.279	0.359	1.34
CHE	6578	28	0.023	0.958	0.327	0.320	0.377	3.87
CZE	3246	28	−0.168	1.130	0.335	0.326	0.393	3.25
DEU	2701	28	−0.039	1.140	0.522	0.500	0.445	11.56
DNK	2431	27	0.001	0.891	0.398	0.394	0.447	4.63
ESP	10,506	28	−0.351	0.815	0.413	0.408	0.479	3.90
EST	2630	28	−0.007	0.838	0.344	0.339	0.432	1.66
FIN	2536	28	0.516	0.854	0.330	0.326	0.378	4.26
FRA	2524	28	−0.010	0.984	0.332	0.320	0.405	1.88
GBR	7061	28	−0.016	0.985	0.340	0.336	0.447	0.48
GRC	2606	28	−0.431	0.952	0.490	0.479	0.510	6.62
HUN	2399	28	−0.148	0.918	0.320	0.306	0.371	3.23
IRL	2468	28	0.184	0.946	0.275	0.269	0.343	1.60
ISL	2010	28	−0.069	0.915	0.326	0.320	0.405	1.87
ITA	11,629	28	−0.285	0.984	0.350	0.340	0.422	2.55
JPN	3203	28	0.028	1.034	0.438	0.435	0.598	−0.51
KOR	2790	27	0.561	0.959	0.589	0.576	0.628	5.71
LUX	2443	27	−0.180	1.012	0.333	0.315	0.358	4.65
NLD	2666	28	0.092	1.017	0.429	0.425	0.516	2.98
NOR	2504	28	−0.107	1.018	0.453	0.439	0.461	7.10
POL	2968	28	0.068	1.000	0.306	0.302	0.411	−0.21
PRT	2773	28	−0.242	0.955	0.534	0.529	0.542	7.76
SWE	2374	28	0.107	1.004	0.288	0.283	0.327	4.39

Note. CNT = country label (see Appendix A); N = sample size per country; I = number of items per country; M = country mean; SD = country SD; ${\widehat{\tau }}_{\mathrm{obs}}$ = observed SD of DIF effects in item difficulties $b_i$; ${\widehat{\tau }}_{\mathrm{bc}}$ = bias-corrected SD estimate of DIF effects in item difficulties $b_i$; ${\widehat{\tau }}_{\nu ,\mathrm{bc}}$ = bias-corrected SD estimate of DIF effects in item intercepts ${\nu }_i$; $\mathsf{\Delta }LL$ = difference in log-likelihood values for models with DIF effects in item intercepts ${\nu }_i$ or item difficulties $b_i$. Positive $\mathsf{\Delta }LL$ values indicate a better model fit for DIF effects in item intercepts.

In the third step, DIF effects $e_i$ were determined for each country. The country mean and country SD were fixed at $\widehat{\mu }$ and $\widehat{\sigma }$, as obtained from the second step, while the international item parameters $a_i$ and $b_i$ were used. Specifically, the IRT model

$$\mathsf{P}(X_i=1|\theta )=\mathsf{\Psi }\left(a_i(\theta -b_i-e_i)\right)\mathrm{with}\mathsf{N}(\widehat{\mu },{\widehat{\sigma }}^2)$$

(35)

was applied in each country, where $\mathsf{N}$ denotes the normal distribution. It is important to note that in (35), only DIF effects and their sampling variances were computed.

Using the data on estimated DIF effects, the distribution of DIF effects within each country was examined. As an initial descriptive step, the empirical SD of DIF effects $e_i$ (i.e., ${\widehat{\tau }}_{\mathrm{obs}}$) was calculated and is reported in Table 5. The average DIF SD was $M=0.367$ with $SD=0.091$, indicating considerable heterogeneity in DIF effects across countries.

To account for the contribution of sampling variance in the observed DIF effect estimates ${\widehat{e}}_i$, maximum likelihood estimation was applied to the following model for DIF effects:

$${\widehat{e}}_i\sim \mathsf{N}(\kappa ,{\tau }^2+v_i^2)\mathrm{for}i=1,\dots ,I.$$

(36)

Here, the parameters $\kappa$ and the DIF SD $\tau$ were estimated, and $v_i^2$ denotes the estimated sampling variance of ${\widehat{e}}_i$. Model (36) corresponds to a random-effects meta-analysis model with known error variances and was estimated using the stats::optim() function in R (Version: 4.4.1; [41]). The resulting $\tau$ estimates, referred to as ${\widehat{\tau }}_{\mathrm{bc}}$ (i.e., bias-corrected estimates), are also reported in Table 5 and were, as expected, slightly lower than the empirical values: $M=0.359$, $SD=0.089$. Note that model (36) assumes random DIF with homogeneous variances for DIF in item difficulties.

The model for DIF effects in (36) is contrasted with the alternative model

$${\widehat{e}}_i\sim \mathsf{N}\left(\kappa ,{\displaystyle \frac{{\tau }^2}{a_i^2}}+v_i^2\right)\mathrm{for}i=1,\dots ,I,$$

(37)

which represents DIF in item intercepts under the assumption of a homogeneous variance. Again, this model was fitted using the stats::optim() function in R [41]. The corresponding $\tau$ estimates, denoted as ${\widehat{\tau }}_{\nu ,\mathrm{bc}}$, are reported in Table 5. On average, ${\widehat{\tau }}_{\nu ,\mathrm{bc}}$ was slightly larger than ${\widehat{\tau }}_{\mathrm{bc}}$, with $M=0.424$ and $SD=0.083$. This result aligns with expectations, given that the average item discrimination clearly exceeded 1.

Because the competing models (36) and (37) were based on the same data (i.e., estimated DIF effects ${\widehat{e}}_i$) and involved the same number of estimated parameters, their log-likelihood values can be directly compared to assess whether the assumption of DIF in item difficulties or item intercepts is more appropriate. The corresponding difference in log-likelihood, denoted as $\mathsf{\Delta }LL$, is reported in Table 5. Notably, the model assuming DIF in item intercepts provided a better fit in 23 out of 26 countries.

Table 4 also includes DIF SD estimates, ${\widehat{\tau }}_{\mathrm{obs}}$ and ${\widehat{\tau }}_{\mathrm{bc}}$, computed for individual items across countries to examine whether certain items were more susceptible to country-level DIF than others. Substantial variability in ${\widehat{\tau }}_{\mathrm{bc}}$ estimates across items was observed, with $M=0.335$ and $SD=0.165$. To evaluate the plausibility of the normality assumption for DIF effects, the Shapiro–Wilk test for normality was applied. The corresponding p values are listed as p(SW) in Table 4. A total of 8 out of 28 items showed statistically significant deviations from normality. Additionally, Figure 1 presents histograms of estimated DIF effects for nine selected items, along with the estimated DIF SD $\tau$ and the Shapiro–Wilk p value. While outliers in DIF effects were evident for items R104Q02 and R219Q01E, the overall pattern suggested unsystematic variation in DIF effects, supporting the plausibility of the normal distribution assumption for random DIF. In contrast, the commonly assumed partial invariance structure—where only a subset of items exhibit large absolute DIF effects while the majority show small or no DIF effects [44,45]—was clearly not supported by the data.

Figure 1. Empirical Example, PISA 2006 Reading: Histograms of estimated DIF effects for nine selected items (R102Q07, R104Q01, R104Q02, R104Q05, R111Q01, R111Q02B, R111Q06B, R219Q01E, and R219Q01T), along with estimated DIF SD $\tau$ and Shapiro–Wilk test for normality (p(SW)). DIF effects of −0.4 and 0.4 are displayed in a red vertical dashed line.

5. Discussion

This article examines three specifications of MGM linking that differ in the computation of the group mean $\widehat{\mu }$. The UW method assigns equal weight to all item difficulty differences; the DW method weights these differences by the squared item discrimination; and the PW method applies precision weights that account for the sampling error in item difficulty differences. The relative performance of the three methods depends on the data-generating model. When no random DIF is present, the PW method consistently outperforms the other two. In the presence of random DIF, the estimated group mean is influenced by both DIF and sampling error. Thus, the effectiveness of each method depends on the relative contribution of these two sources of variance.

When random DIF with homogeneous variances affects item difficulties, the UW method outperforms both DW and PW under large sample conditions. Conversely, if random DIF with homogeneous variances influences item intercepts, the DW method yields superior results among the three approaches.

Because the estimated precision weights in PW linking reflect the presence of DIF effects, a bias arises due to the covariance between these weights and the item difficulty differences since DIF affects both quantities. Therefore, if random DIF is suspected, the PW method is not recommended.

Given that the performance of the UW and DW methods depends on whether random DIF affects item difficulties or item intercepts, the PISA 2006 reading dataset was analyzed to investigate which assumption is more tenable. Model fit comparisons indicated that DIF effects were more prevalent in item intercepts than in item difficulties for the majority of countries. This empirical evidence suggests that the DW method may be preferable when selecting an MGM specification in applied settings. Furthermore, as DIF in item intercepts appears to be the more tenable assumption in practice, future simulation studies may benefit from focusing on DIF effects in item intercepts rather than in item difficulties. Nevertheless, future research should examine whether this specific result from the PISA 2006 reading dataset generalizes to other empirical contexts.

However, weighting items based on item discrimination may not accurately reflect group differences, as the group difference should ideally assign equal weight to all items to preserve the intended test composition [46,47]. Nonetheless, this critique may not fully hold, as item weighting is already introduced when selecting the 2PL model—incorporating item discrimination—over the Rasch model [48], which applies equal weighting in the IRT framework.

Future research could examine the comparison between the UW and PW methods within the Rasch model. In this setting, MGM linking is replaced by mean–mean linking, as only the group means are aligned, and the group SDs are freely estimated. As noted by an anonymous reviewer, the Rasch model may exhibit special measurement properties compared to the 2PL model [49,50,51] (but see [52,53,54]), which can lead to its preference among practitioners [55,56,57,58]. Several established tools for detecting DIF have been developed within the Rasch framework [59,60]. The relative efficiency of the UW and PW methods depends on the relative magnitude of random DIF SD and sampling error. Importantly, the PW method is also expected to introduce bias in the estimated group mean under the Rasch model when random DIF effects are present.