Standard Error Estimation in Invariance Alignment

Abstract

The invariance alignment (IA) method enables group comparisons in factor models involving either continuous or discrete items. This article evaluates the performance of the commonly used delta method for standard error estimation against alternative bootstrap confidence interval (CI) approaches for IA using the $L_{0.5}$ and $L_0$ loss functions. For IA applied to continuous items, both the delta method and all bootstrap methods yielded acceptable coverage rates. In contrast, for dichotomous items, only bias-corrected bootstrap CIs provided reliable statistical inference in moderate to large sample sizes. In small sample sizes with dichotomous items, none of the individual methods performed consistently well. However, a newly proposed average bootstrap CI approach—based on averaging the lower and upper CI limits from two bootstrap methods—achieved acceptable coverage rates.

1. Introduction

When comparing multiple groups in confirmatory factor analysis (CFA; [1]) or item response theory (IRT; [2,3]) models with respect to a factor variable, identification assumptions are required to separate group differences in item parameters from differences in factor variables [4]. A common assumption is that item parameters in CFA or IRT models are assumed to be equal across groups. This property is referred to as measurement invariance [5,6], which is widely discussed in the social sciences [6,7]. In IRT, violations of this assumption are typically addressed as differential item functioning (DIF; [8,9]). The assumption of equal item parameters across groups is prevalent in the social sciences for two main reasons. First, there is ambiguity in how to impose identification constraints that define group differences in CFA or IRT models. For example, one might assume that the deviations of group-specific item intercepts from the overall item intercept average to zero—similar to the constraint in linear regression that residuals sum to zero. However, any weighted sum of deviations that equals zero could serve as a valid identification constraint. Second, it is often believed that the latent variable can only be meaningfully compared across groups if items operate equivalently across those groups ([10,11,12,13]; but see [4,14,15]).

To handle violations of measurement invariance, the invariance alignment (IA) method [16,17,18,19], also known as alignment optimization [20,21], has been proposed for comparing groups in unidimensional CFA or IRT models. IA seeks to identify a solution in which the majority of item parameters remain (approximately) invariant, allowing for small deviations. This enhances the robustness of group comparisons when invariance is only partially met, that is, in cases where only a limited number of item parameters differ across groups. For this reason, IA is considered advantageous over the strict invariance assumption by many researchers, as it relaxes that assumption [20,21,22]. The IA method has gained wide application in social science research using questionnaire data [23,24,25,26,27,28,29,30,31,32].

Despite its widespread empirical use, statistical inference for IA has primarily been investigated for continuous items, with only limited research for dichotomous or polytomous items. In particular, comparisons of the widely used delta method against alternative bootstrap methods for confidence interval estimation remain scarce. Moreover, there is a lack of comprehensive evaluation of statistical inference for IA using the recently proposed $L_0$ loss function [33], which serves as an alternative to the more commonly used $L_{0.5}$ loss function.

The adequacy of statistical inference is typically evaluated through simulation studies, focusing on coverage rates or standard error ratios. The following discussion is limited to IA simulation studies using maximum likelihood estimation. Asparouhov and Muthén [16] examined coverage rates for IA with continuous items and observed inflated coverage for group means—i.e., overcoverage—while coverage for group standard deviations was acceptable. They also reported that standard error ratios exceeded 1 for these parameters, indicating overestimated standard errors and explaining the overcoverage. In contrast, Asparouhov and Muthén [17,34] and Wen and Hu [35] did not observe inflated coverage for either continuous or dichotomous items. Flake and McCoach [36] found undercoverage of factor means when IA was applied to polytomous items. Finch [37] reported acceptable coverage for factor means in the case of dichotomous items. Robitzsch [38] observed slightly elevated coverage rates—above the nominal 95% level—for IA applied to continuous items.

This article compares the delta method for standard error estimation in IA with different bootstrap methods and examines whether the results differ between continuous and dichotomous items.

The remainder of the article is structured as follows. Section 2 provides an overview of the IA method. Section 3 introduces alternative methods for standard error and confidence interval estimation in IA. Results from a simulation study involving continuous and dichotomous items are presented in Section 4 and Section 5, respectively. The article concludes with a discussion in Section 6.

2. Invariance Alignment

This section reviews the IA method for unidimensional factor models involving continuous and dichotomous items. Let ${\mathit{X}}_g=(X_{1g},\dots ,X_{Ig})$ denote the vector of $I>1$ items in group $g=1,\dots ,G$. The item vector ${\mathit{X}}_g$ is linked to a normally distributed factor variable ${\theta }_g$, commonly referred to as a latent variable in the social sciences. It is assumed that ${\theta }_g$ follows a normal distribution with mean ${\mu }_g$ and standard deviation (SD) ${\sigma }_g$ in group g. For identification purposes, the constraints ${\mu }_1=0$ and ${\sigma }_1=1$ are imposed in the first group.

In unidimensional factor models, the items $X_{ig}$ are assumed to be conditionally independent given the latent variable ${\theta }_g$. This conditional independence is formally expressed as

$$\mathsf{P}(X_{1g},\dots ,X_{Ig}|{\theta }_g)=\prod _{i=1}^I\mathsf{P}\left(X_{ig}\right|{\theta }_g).$$

(1)

Equation (1) indicates that conditional on the latent variable ${\theta }_g$, the items are statistically independent. Consequently, all associations among the items are accounted for by this unidimensional latent variable.

The unidimensional factor model for continuous items in CFA is defined as

$$X_{ig}={\nu }_{ig}+{\lambda }_{ig}{\theta }_g+{\xi }_{ig}\mathrm{with}{\theta }_g\sim \mathsf{N}({\mu }_g,{\sigma }_g^2)\mathrm{and}{\xi }_{ig}\sim \mathsf{N}(0,{\omega }_{ig}),$$

(2)

where ${\nu }_{ig}$ denotes the item intercept and ${\lambda }_{ig}$ represents item discrimination. The residual terms ${\xi }_{ig}$ are normally distributed, uncorrelated with the factor variable ${\theta }_g$, and have variance ${\omega }_{ig}$. The parameters of the unidimensional factor model in (2) can be estimated using maximum likelihood [1,39].

When the model (2) is estimated separately for each group $g=1,\dots ,G$, identification constraints ${\mu }_g=0$ and ${\sigma }_g=1$ must again be imposed. Under these constraints, the identified item parameters ${\widehat{\lambda }}_{ig}$ and ${\widehat{\nu }}_{ig}$ are given by

$${\widehat{\lambda }}_{ig}={\lambda }_{ig}{\sigma }_g\mathrm{and}{\widehat{\nu }}_{ig}={\nu }_{ig}+{\lambda }_{ig}{\mu }_g={\nu }_{ig}+{\displaystyle \frac{{\widehat{\lambda }}_{ig}}{{\sigma }_g}}{\mu }_g.$$

(3)

Consider now the case of dichotomous items, ${\mathit{X}}_g\in {\{0,1\}}^I$. In this context, the unidimensional factor model is commonly referred to as an IRT model. The function $P_{ig}\left({\theta }_g\right)=\mathsf{P}(X_{ig}=1\mid {\theta }_g)$ for $i=1,\dots ,I$ in group $g=1,\dots ,G$ is known as the item response function (IRF). The IRF of the two-parameter logistic (2PL) model [40], which is employed here as the unidimensional factor model for dichotomous items, is defined as

$$P_{ig}\left({\theta }_g\right)=\Psi \left({\lambda }_{ig}{\theta }_g+{\nu }_{ig}\right)\mathrm{with}{\theta }_g\sim \mathsf{N}({\mu }_g,{\sigma }_g^2),$$

(4)

where ${\lambda }_{ig}$ and ${\nu }_{ig}$ denote the item discrimination and item intercept, respectively, and $\Psi \left(x\right)={(1+exp(-x))}^{-1}$ is the logistic distribution function. The parameters of the 2PL model can be consistently estimated using marginal maximum likelihood (MML) estimation [41,42].

When model (4) is estimated separately for each group g, the identification constraints ${\mu }_g=0$ and ${\sigma }_g=1$ must be imposed. Under these constraints, the same transformation formulas as in (3) apply for the identified parameters ${\widehat{\lambda }}_{ig}$ and ${\widehat{\nu }}_{ig}$.

The optimization function used in the IA method proposed by Asparouhov and Muthén [16,18] is described below. The IA method has been discussed in the literature for continuous items [16], dichotomous items [18], and polytomous items [43].

The IA method estimates group means $\mathit{\mu }=({\mu }_2,\dots ,{\mu }_G)$ and group SDs $\mathit{\sigma }=({\sigma }_2,\dots ,{\sigma }_G)$ by minimizing a discrepancy function based on transformed item parameter differences across group pairs. For identification, define ${\mu }_1=0$ and ${\sigma }_1=1$. Let $\mathit{\delta }=(\mathit{\mu },\mathit{\sigma })$ denote the vector of distribution parameters of interest. The estimate $\widehat{\mathit{\delta }}$ is obtained by minimizing

$$H\left(\mathit{\delta }\right)=\sum _{i=1}^I\sum _{g\ne h}\rho \left({\displaystyle \frac{{\widehat{\lambda }}_{ig}}{{\sigma }_g}}-{\displaystyle \frac{{\widehat{\lambda }}_{ih}}{{\sigma }_h}}\right)+\sum _{i=1}^I\sum _{g\ne h}\rho \left({\widehat{\nu }}_{ig}-{\widehat{\nu }}_{ih}-{\displaystyle \frac{{\widehat{\lambda }}_{ig}}{{\widehat{\sigma }}_g}}{\mu }_g+{\displaystyle \frac{{\widehat{\lambda }}_{ih}}{{\widehat{\sigma }}_h}}{\mu }_h\right),$$

(5)

where $\rho$ denotes a robust loss function. The original IA method also incorporates weights for group pairs $(g,h)$ to account for varying sample sizes in the objective function [16]. These weights are omitted here, as they are unnecessary when sample sizes are equal across groups.

A robust loss function $\rho$ is employed in the minimization problem (5) to account for the assumption that only a subset of item parameters deviates across groups. This reflects a sparse pattern of deviations from full measurement invariance. Accordingly, the IA method is particularly well suited to situations involving partial invariance. In its original formulation, the IA method uses a differentiable approximation of the nondifferentiable $L_p$ loss function $x\mapsto {\left|x\right|}^p$ (see [44,45,46]), defined as (see [16])

$$\rho \left(x\right)={\left({\left|x\right|}^2+\epsilon \right)}^{p/2},$$

(6)

where $\epsilon >0$ is a tuning parameter that governs the approximation error [16,18]. In the default setting of the commercially available Mplus software (Version: 8.11) by Muthén and Asparouhov [47], the power is $p=0.5$, which is also adopted in this paper. Empirical studies suggest that a value of $\epsilon =0.001$ is effective [29,33], although Mplus applies a default of $\epsilon =0.01$.

As an alternative to the $L_{0.5}$ loss function, a differentiable approximation of the $L_0$ loss function $x\mapsto {\mathbf{1}}_{\{x\ne 0\}}$—the indicator function that returns one if the argument differs from zero—has been employed within the IA framework [38]; see also [48]. The corresponding loss function is defined as

$$\rho \left(x\right)={\displaystyle \frac{x^2}{x^2+\epsilon }},$$

(7)

where $\epsilon >0$ is a tuning parameter that controls the approximation error of $\rho$ relative to the nondifferentiable $L_0$ loss function. Empirical findings indicate that setting $\epsilon =0.01$ yields satisfactory performance in practical applications [33,49].

The $L_0$ loss function typically reduces bias relative to the $L_{0.5}$ loss function; however, this reduction in bias is accompanied by increased variance. The choice between the two loss functions depends on the bias-variance trade-off in a given context, as the root mean squared error (RMSE) may favor either one depending on the relative magnitudes of bias and variance [33].

3. Standard Error and Confidence Interval Estimation in Invariance Alignment

This section presents alternative methods for computing standard errors and confidence intervals (CIs) for IA. While the delta method discussed in Section 3.1 relies on asymptotic theory, the bootstrap methods outlined in Section 3.2 leverage computational resources to compute a CI.

Let $\mathit{\delta }=(\mathit{\mu },\mathit{\sigma })$ represent the vector of distribution parameters, with estimates denoted as $\widehat{\mathit{\delta }}=(\widehat{\mathit{\mu }},\widehat{\mathit{\sigma }})$. The vector $\widehat{\mathit{\gamma }}$ contains the estimated item parameters, including item discriminations ${\widehat{\lambda }}_{ig}$ and item intercepts ${\nu }_{ig}$. The corresponding estimated variance matrix is denoted by ${\mathit{V}}_{\widehat{\mathit{\gamma }}}$, which is typically derived from statistical software that estimates the unidimensional factor models for continuous or dichotomous items separately in each of the G groups.

3.1. Delta Method

The standard error of $\widehat{\mathit{\delta }}$ and the corresponding confidence intervals (CIs) for its components are computed using the delta method (DM; see [16,17,50,51,52,53,54,55]). In IA, the estimate $\widehat{\mathit{\delta }}=(\widehat{\mu },\widehat{\sigma })$ satisfies the estimating equation

$$h_{\mathit{\delta }}(\mathit{\delta },\widehat{\mathit{\gamma }})=\mathbf{0},$$

(8)

where $h_{\mathit{\delta }}$ is the partial derivative of H with respect to $\mathit{\delta }$ in the IA minimization problem (5). The DM approach applies a Taylor expansion of $h_{\mathit{\delta }}$ around the population values $(\mathit{\delta },\mathit{\gamma })$, yielding

$$h_{\mathit{\delta }}(\widehat{\mathit{\delta }},\widehat{\mathit{\gamma }})=h_{\mathit{\delta }}(\mathit{\delta },\mathit{\gamma })+h_{\mathit{\delta }\mathit{\delta }}(\mathit{\delta },\mathit{\gamma })(\widehat{\mathit{\delta }}-\mathit{\delta })+h_{\mathit{\delta }\mathit{\gamma }}(\mathit{\delta },\mathit{\gamma })(\widehat{\mathit{\gamma }}-\mathit{\gamma }).$$

(9)

Here, $h_{\mathit{\delta }\mathit{\delta }}$ and $h_{\mathit{\delta }\mathit{\gamma }}$ represent the Jacobians of $h_{\mathit{\delta }}$ with respect to $\mathit{\delta }$ and $\mathit{\gamma }$, respectively. From (8) and the identity $h_{\mathit{\delta }}(\mathit{\delta },\mathit{\gamma })=\mathbf{0}$, expression (9) simplifies to

$$\widehat{\mathit{\delta }}-\mathit{\delta }=h_{\mathit{\delta }\mathit{\delta }}{(\mathit{\delta },\mathit{\gamma })}^{-1}{h}_{\mathit{\delta }\mathit{\gamma }}(\mathit{\delta },\mathit{\gamma })(\widehat{\mathit{\gamma }}-\mathit{\gamma }).$$

(10)

Letting $\mathit{A}=h_{\mathit{\delta }\mathit{\delta }}{(\mathit{\delta },\mathit{\gamma })}^{-1}{h}_{\mathit{\delta }\mathit{\gamma }}(\mathit{\delta },\mathit{\gamma })$, the variance of $\widehat{\mathit{\delta }}$ becomes

$$\mathsf{Var}\left(\widehat{\mathit{\delta }}\right)=\mathit{A}{\mathit{V}}_{\widehat{\mathit{\gamma }}}{\mathit{A}}^{\top }.$$

(11)

An estimate of $\mathit{A}$ is given by

$$\widehat{\mathit{A}}=h_{\mathit{\delta }\mathit{\delta }}{(\widehat{\mathit{\delta }},\widehat{\mathit{\gamma }})}^{-1}{h}_{\mathit{\delta }\mathit{\gamma }}(\widehat{\mathit{\delta }},\widehat{\mathit{\gamma }}),$$

(12)

resulting in the estimated variance matrix

$${\mathit{V}}_{\widehat{\mathit{\delta }}}=\widehat{\mathit{A}}{\mathit{V}}_{\widehat{\mathit{\gamma }}}{\widehat{\mathit{A}}}^{\top }.$$

(13)

Standard errors for the entries in $\widehat{\mathit{\delta }}$ are computed as the square roots of the diagonal elements in ${\mathit{V}}_{\widehat{\mathit{\delta }}}$. Confidence intervals can then be constructed assuming normality and utilizing these computed standard errors.

3.2. Bootstrap Methods

This section discusses the application of parametric bootstrap methods to compute confidence intervals for the parameter estimate $\widehat{\mathit{\delta }}$ in IA. The estimate $\widehat{\mathit{\delta }}$ satisfies the estimating equation $h_{\mathit{\delta }}(\mathit{\delta },\widehat{\mathit{\gamma }})=\mathbf{0}$. The item parameters $\widehat{\mathit{\delta }}$ have a variance matrix ${\mathit{V}}_{\widehat{\mathit{\gamma }}}$, which is estimated separately for each group. The parametric bootstrap method resamples item parameters based on ${\mathit{V}}_{\widehat{\mathit{\gamma }}}$, generating a distribution of IA parameter estimates $\widehat{\mathit{\delta }}$.

To obtain bootstrap samples $b=1,\dots ,B$, draw ${\widehat{\mathit{\gamma }}}^{\left(b\right)}$ from a multivariate normal distribution with mean vector $\widehat{\mathit{\gamma }}$ and variance matrix ${\mathit{V}}_{\widehat{\mathit{\gamma }}}$. For each bth bootstrap sample, the estimate ${\widehat{\mathit{\delta }}}^{\left(b\right)}$ satisfies the estimating equation $h_{\mathit{\delta }}(\mathit{\delta },{\widehat{\mathit{\gamma }}}^{\left(b\right)})=\mathbf{0}$. Standard bootstrap techniques can then be applied to the resulting estimates ${\widehat{\mathit{\gamma }}}^{\left(b\right)}$ for $b=1,\dots ,B$ (see [56]).

Let $\widehat{\beta }$ represent an entry of $\widehat{\mathit{\delta }}$, corresponding to estimated group means or group SDs, and let ${\widehat{\beta }}^{\left(b\right)}$ denote the corresponding parameter estimate in the bth bootstrap sample. Define ${\widehat{G}}_{\widehat{\beta },\mathrm{boot}}$ as the empirical distribution function of $\widehat{\beta }$, derived from the parametric bootstrap using B samples. The associated inverse distribution function (i.e., the quantile or percentile function) is denoted as ${{\widehat{G}}_{\widehat{\beta },\mathrm{boot}}}^{-1}$.

The following outlines alternative methods for CI estimation at a confidence level $1-\alpha$ (e.g., $1-\alpha =0.95$), based on the bootstrap procedures described in [56,57].

The normal distribution bootstrap (BNO) CI assumes a normal distribution for the parameter estimate $\widehat{\beta }$. Let ${\Phi }^{-1}$ denote the inverse distribution function (i.e., the quantile function) of the standard normal distribution. The BNO confidence interval is given by

$$CI=\left[\widehat{\beta }-{\Phi }^{-1}(1-\alpha /2)s_{\widehat{\beta }},\widehat{\beta }+{\Phi }^{-1}(1-\alpha /2)s_{\widehat{\beta }}\right],$$

(14)

where $s_{\widehat{\beta }}$ denotes the empirical standard deviation of the bootstrap estimates ${\widehat{\beta }}^{\left(b\right)}$. For a confidence level of $1-\alpha =0.95$, the quantile in (14) is ${\Phi }^{-1}(1-\alpha /2)=1.96$.

The percentile bootstrap (BPE) CI is based on the quantiles (i.e., percentiles) of the empirical distribution of the bootstrap parameter estimates $\widehat{\beta }$. The CI is defined as

$$CI=\left[{{\widehat{G}}_{\widehat{\beta },\mathrm{boot}}}^{-1}(\alpha /2),{{\widehat{G}}_{\widehat{\beta },\mathrm{boot}}}^{-1}(1-\alpha /2)\right].$$

(15)

An advantage of the BPE method is its applicability to cases where the distribution of $\widehat{\beta }$ is asymmetric.

The bias-corrected bootstrap (BBC) CI accounts for potential bias in the bootstrap samples and accommodates asymmetric distributions of the linking parameter estimate $\widehat{\beta }$. It is defined as

$$CI=\left[{{\widehat{G}}_{\widehat{\beta },\mathrm{boot}}}^{-1}\left(\Phi \left\{2{\mathcal{B}}_{\widehat{\beta }}-{\Phi }^{-1}(1-\alpha /2)\right\}\right),{{\widehat{G}}_{\widehat{\beta },\mathrm{boot}}}^{-1}\left(\Phi \left\{2{\mathcal{B}}_{\widehat{\beta }}+{\Phi }^{-1}(1-\alpha /2)\right\}\right)\right],$$

(16)

where

$${\mathcal{B}}_{\widehat{\beta }}={\Phi }^{-1}\left({\displaystyle \frac{1}{B}}\sum _{b=1}^B\mathbf{1}({\widehat{\beta }}^{\left(b\right)}<\widehat{\beta })\right)$$

(17)

is a bias correction term and $\mathbf{1}$ denotes the indicator function.

If two confidence intervals (CIs), $C{I}_j=[l_j,u_j]$ for $j=1,2$, are available, an average bootstrap (BAV) CI can be computed as

$$CI=\left[{\displaystyle \frac{l_1+l_2}{2}},{\displaystyle \frac{u_1+u_2}{2}}\right].$$

(18)

The rationale for using BAV is that if $C{I}_1$ results in overcoverage and $C{I}_2$ in undercoverage, averaging the lower and upper bounds of the two CIs yields a coverage rate that is between the original CIs.

3.3. Comparison

The delta method and parametric bootstrap methods discussed in Section 3.1 and Section 3.2, respectively, rely on the asymptotic normality of the item parameter estimates. These estimates are treated as input to the IA procedure, which is a nonlinear function of the item parameters. The delta method applies a linear Taylor approximation to compute confidence intervals for the IA estimates, whereas the bootstrap method does not rely on this approximation and instead constructs confidence intervals using computational resources. Although the bootstrap methods may be more accurate due to fewer distributional assumptions, they are computationally more demanding. In contrast, the delta method is more efficient but may be less accurate because of the linear approximation involved in the parameter transformation.

4. Simulation Study 1: Continuous Items

In this Simulation Study 1, the performance of confidence intervals (CIs) is compared for IA using the $L_{0.5}$ and $L_0$ loss functions with continuous items.

4.1. Method

The data-generating model for this study was adapted from Asparouhov and Muthén [16]. Continuous item responses were simulated using a unidimensional CFA model with $G=3$ groups. The normally distributed factor variable was measured by $I=5$ normally distributed items.

In the simulation, the group means ${\mu }_g$ of the factor variable ${\theta }_g$ were set to 0, 0.3, and 1.0, while the group SDs ${\sigma }_g$ were set to 1, 1.5, and 1.2, respectively. Item responses included DIF in item discriminations and item intercepts. Each group had one out of five items with noninvariant item intercept ${\nu }_{ig}$ and one noninvariant item discrimination ${\lambda }_{ig}$. For all groups, invariant item discriminations and residual variances of the indicator variables were set to ${\lambda }_{ig}=1$ ($i=1,\dots ,5$), and invariant item intercepts were set to ${\nu }_{ig}=0$. The noninvariant item parameters in the first group were ${\nu }_{51}=0.5$ and ${\lambda }_{13}=1.4$. The noninvariant item parameters in the second group were ${\nu }_{12}=-0.5$ and ${\lambda }_{52}=0.5$. The noninvariant item parameters in the third group were ${\nu }_{23}=0.5$ and ${\lambda }_{43}=0.3$. The complete set of item parameters is also available at https://osf.io/8hfjc (accessed on 5 May 2025).

Sample sizes per group were selected as $N=250$, 500, 1000, and 2000 to encompass a broad range of sample sizes, reflecting the variety of potential applications for IA alignment.

In each of the 4 simulation conditions (with sample size N), 5000 replications were conducted. IA was applied for $p=0.5$ with $\epsilon =0.001$ (i.e., $L_{0.5}$ loss function) and for $p=0$ with $\epsilon =0.01$ (i.e., $L_0$ loss function). For both IA specifications, CIs at a confidence level of $1-\alpha =0.95$ were computed using five methods: DM, BNO, BPE, BBC, and BAV (see Section 3.2). The BAV method averaged the CI limits from BNO and BBC. A total of $B=1000$ bootstrap samples were used for the parametric bootstrap approaches. Bias and root mean squared error (RMSE) were evaluated for the estimated group means ${\widehat{\mu }}_g$ and SDs ${\widehat{\sigma }}_g$. Coverage rates were also assessed for the two IA methods, which were crossed with the five CI estimation approaches. For each IA method, a pseudo-true parameter was defined as the average parameter estimate across replications within a given simulation condition, isolating coverage performance from parameter bias. The coverage rate was defined as the proportion of replications in which the CI contained the pseudo-true parameter. Coverage rates between 91% and 98% were considered acceptable [58]. A coverage rate below 91% indicated undercoverage, while a rate above 98% signified overcoverage.

All analyses for this simulation study were conducted using the statistical software R (Version 4.4.1; [59]). The unidimensional CFA model was fitted using the R function sirt::invariance_alignment_cfa_config() from the sirt package (Version 4.2-114; [60]). This function is a wrapper for the lavaan package (Version 0.6-19; [61,62]). IA estimation, including point estimates and standard errors based on DM, was performed using the R function sirt::invariance.alignment() from the sirt package. Custom functions were developed by the author to implement the parametric bootstrap approach. Replication materials for this simulation study are available at https://osf.io/8hfjc (accessed on 5 May 2025).

4.2. Results

Table 1. Simulation Study 1, continuous items: Bias and root mean square error for estimated group means and group SDs as a function of sample size N.

		${\mathit{\mu }}_2$			${\mathit{\mu }}_3$			${\mathit{\sigma }}_2$			${\mathit{\sigma }}_3$
Crit	$N$	$p=0.5$	$p=0$		$p=0.5$	$p=0$		$p=0.5$	$p=0$		$p=0.5$	$p=0$
Bias	250	−0.042	−0.007		0.019	0.022		−0.016	0.005		−0.009	0.005
	500	−0.027	0.000		0.007	0.006		−0.013	0.002		−0.007	0.004
	1000	−0.020	−0.003		0.004	0.003		−0.009	0.001		−0.007	0.001
	2000	−0.012	0.000		0.003	0.002		−0.007	0.001		−0.005	0.000
RMSE	250	0.140	0.138		0.151	0.172		0.128	0.135		0.111	0.116
	500	0.092	0.088		0.101	0.105		0.084	0.085		0.073	0.074
	1000	0.066	0.063		0.068	0.068		0.057	0.056		0.051	0.050
	2000	0.046	0.044		0.046	0.046		0.039	0.038		0.035	0.034

Note. Crit = criterion (bias or RMSE); p = used power in the loss function $\rho$ in invariance alignment; Values of absolute bias larger than 0.010 are printed in bold font.

presents the bias and RMSE for the estimated group means and group SDs as a function of sample size for IA specified with the loss functions using powers $p=0.5$ and $p=0$. Notably, IA with $p=0.5$ exhibited a larger bias than IA with $p=0$. However, as the sample size increased, the bias diminished. Despite this, IA using the $L_0$ loss function outperformed the $L_p$ loss in terms of RMSE across most simulation conditions.

Table 2. Simulation Study 1, continuous items: Coverage rates for estimated group means and group SDs as a function of sample size N.

		$\mathit{p}=0.5$						$\mathit{p}=0$
Par	$\mathit{N}$	DM	BNO	BPE	BBC	BAV		DM	BNO	BPE	BBC	BAV
${\mu }_2$	250	96.4	96.6	96.2	93.2	95.4		96.0	98.0	97.3	94.2	96.5
	500	96.9	96.7	96.2	95.1	96.2		96.6	97.3	97.0	95.7	96.8
	1000	96.0	95.8	95.4	94.1	95.1		95.7	96.2	95.8	95.4	96.0
	2000	96.0	95.8	95.5	95.0	95.6		95.5	95.8	95.5	95.4	95.6
${\mu }_3$	250	96.4	96.5	97.1	94.3	95.7		94.3	97.2	98.1	94.7	96.4
	500	97.0	97.1	96.8	95.6	96.5		96.2	98.3	97.2	96.5	97.7
	1000	96.4	96.6	96.6	95.2	95.9		96.3	97.5	96.7	96.2	97.0
	2000	96.1	96.0	96.0	95.4	95.7		95.4	96.1	95.7	95.8	96.0
${\sigma }_2$	250	96.8	96.0	96.9	93.8	95.3		95.9	97.3	97.8	95.8	96.9
	500	96.9	96.5	97.0	95.2	96.0		96.1	97.5	97.6	96.4	97.1
	1000	96.4	96.3	96.4	94.9	95.8		95.9	96.5	96.5	96.1	96.4
	2000	96.2	96.3	96.3	95.7	96.2		95.6	96.2	95.9	95.7	96.1
${\sigma }_3$	250	97.0	96.5	97.4	94.1	95.9		96.1	97.6	98.2	95.5	97.0
	500	96.7	96.5	97.2	95.0	95.9		96.4	97.5	97.8	96.5	97.0
	1000	96.3	96.1	96.4	94.9	95.6		95.8	96.7	96.7	96.0	96.4
	2000	96.0	96.1	96.2	95.5	95.8		95.7	96.1	96.2	96.0	96.1

Note. Par = parameter; p = used power in the loss function $\rho$ in invariance alignment; DM = delta method; BNO = bootstrap CI based on normal distribution; BPE = percentile bootstrap CI; BBC = bias-corrected bootstrap CI; BAV = average of CI limits of BNO and BBC; Coverage rates smaller than 91.0 or larger than 98.0 are printed in bold font.

shows the coverage rates for the estimated means and group SDs as a function of sample size. In this simulation study with continuous items, all five CI estimation methods resulted in acceptable coverage rates. However, the bootstrap methods BNO and BPE exhibited slightly inflated coverage rates. For BPE, two conditions with the smallest sample size ($N=250$) resulted in overcoverage. The DM method produced coverage rates slightly above the target value of 95.0, but this was not considered problematic.

5. Simulation Study 2: Dichotomous Items

This Simulation Study 2 evaluates the performance of CIs for IA based on the $L_{0.5}$ and $L_0$ loss functions in the context of dichotomous items.

5.1. Method

As in Simulation Study 1 presented in Section 4, this study considered $G=3$ groups. In contrast to the previous study, the present design involved dichotomous items simulated under the 2PL model, rather than continuous items. The number of items was set to either $I=15$ or $I=30$.

In the simulation, the group means ${\mu }_g$ of the factor variable ${\theta }_g$ were set to 0, 0.3, and 0.8, and the group SDs ${\sigma }_g$ were specified as 1, 1.4, and 1.2, respectively. Item discriminations ${\lambda }_{ig}$ were held invariant across groups, whereas item intercepts ${\nu }_{ig}$ exhibited DIF with an absolute effect of 0.5. The specification of item parameters is described for the case of $I=15$ items; for $I=30$ items, the parameters were duplicated. The item discriminations for the 15 items were set to 0.78, 0.74, 1.27, 0.82, 1.40, 0.85, 0.84, 1.30, 1.34, 0.75, 1.16, 1.15, 1.10, 0.70, and 0.80, yielding an average discrimination of $M=1.00$ with $SD=0.25$. The invariant item intercepts were 1.45, −0.31, −1.09, −1.22, −1.64, 0.05, −0.49, 1.67, −0.07, −0.48, 0.01, −1.79, 0.36, −1.08, and 0.14, resulting in $M=-0.30$ and $SD=0.97$. DIF in item intercepts was introduced for three items per group. In the first group, Items 1–3 had intercepts 1.95, −0.81, and −0.59. In the second group, Items 4–6 had intercepts −0.72, −1.14, and 0.55. In the third group, Items 7–9 had intercepts −0.99, 1.17, and −0.57. The full set of item parameters is available at https://osf.io/8hfjc (accessed on 5 May 2025).

In this simulation, sample sizes per group were set to $N=500$, 1000, 2000, and 4000, reflecting the increased sample size requirements typically associated with estimating the 2PL model compared to the unidimensional CFA model for continuous items.

In each of the 4 (sample size N) × 2 (number of items I) $=8$ simulation conditions, 5000 replications were performed. The same analysis models—IA with $L_{0.5}$ and $L_0$ loss functions and CI estimation methods—as in Simulation Study 1 were applied. The parametric bootstrap procedure again used $B=1000$ bootstrap samples.

This simulation study was also conducted using the statistical software R (Version 4.4.1; [59]). The unidimensional 2PL model was fitted using the R function sirt::xxirt() from the sirt package (Version 4.2-114; [60]). IA estimation was performed using the sirt::invariance.alignment() function from the same package. The custom functions used to implement the parametric bootstrap approach were identical to those employed in Simulation Study 1. Replication materials for this simulation study are available at https://osf.io/8hfjc (accessed on 5 May 2025).

5.2. Results

Table 3. Simulation Study 2, dichotomous items: Bias and root mean square error for estimated group means and group SDs as a function of the number of items I sample size N.

			${\mathit{\mu }}_2$			${\mathit{\mu }}_3$			${\mathit{\sigma }}_2$			${\mathit{\sigma }}_3$
Crit	$I$	$N$	$p=0.5$	$p=0$		$p=0.5$	$p=0$		$p=0.5$	$p=0$		$p=0.5$	$p=0$
Bias	15	500	0.040	0.031		−0.099	−0.068		0.061	0.069		0.053	0.059
		1000	0.025	0.011		−0.064	−0.018		0.028	0.032		0.027	0.029
		2000	0.014	0.003		−0.035	0.000		0.015	0.016		0.013	$\phantom{-}$0.014
		4000	0.008	0.001		−0.023	0.000		0.007	0.007		0.006	0.006
	30	500	0.037	0.023		−0.100	−0.055		0.047	0.053		0.043	0.047
		1000	0.022	0.006		−0.061	−0.009		0.022	0.023		0.020	0.021
		2000	0.015	0.004		−0.035	0.000		0.009	0.009		0.009	0.010
		4000	0.008	0.001		−0.022	0.000		0.004	0.004		0.005	0.005
RMSE	15	500	0.124	0.146		0.160	0.180		0.139	0.170		0.121	0.149
		1000	0.084	0.091		0.110	0.114		0.088	0.104		0.076	0.091
		2000	0.056	0.055		0.068	0.063		0.057	0.063		0.050	0.056
		4000	0.037	0.036		0.045	0.038		0.038	0.039		0.034	0.035
	30	500	0.109	0.123		0.145	0.149		0.105	0.128		0.093	0.113
		1000	0.072	0.073		0.094	0.081		0.067	0.075		0.058	0.066
		2000	0.049	0.047		0.058	0.049		0.044	0.046		0.039	0.041
		4000	0.033	0.032		0.039	0.032		0.030	0.030		0.026	0.026

Note. Crit = criterion (bias or RMSE); p = used power in the loss function $\rho$ in invariance alignment; Values of absolute bias larger than 0.010 are printed in bold font.

reports the bias and RMSE for the estimated group means and group SDs as a function of sample size, based on IA specified with the loss functions using powers $p=0.5$ and $p=0$. For the group means, IA with the $L_0$ loss function exhibited smaller bias than the specification with $p=0.5$; however, this advantage did not extend to the estimation of group SDs. As observed in the continuous item setting, the bias associated with IA decreased as sample size increased.

Table 4. Simulation Study 2, dichotomous items: Coverage rates for estimated group means and group SDs as a function of the number of items I sample size N.

			$\mathit{p}=0.5$						$\mathit{p}=0$
Par	$I$	$N$	DM	BNO	BPE	BBC	BAV		DM	BNO	BPE	BBC	BAV
${\mu }_2$	15	500	99.3	97.9	98.9	92.0	95.4		98.2	98.8	99.6	91.3	96.0
		1000	99.2	97.9	98.5	93.6	96.0		98.5	99.3	99.6	95.3	98.2
		2000	98.8	97.9	97.9	95.1	96.9		98.3	98.9	99.0	96.7	98.2
		4000	98.1	97.1	97.4	95.3	96.6		97.7	97.8	97.9	96.7	97.6
	30	500	99.2	96.8	97.9	92.1	94.8		98.6	98.4	99.4	92.0	96.2
		1000	99.3	97.3	97.5	93.7	95.9		98.9	99.0	99.2	96.6	98.4
		2000	98.6	97.0	97.0	94.8	96.4		98.6	98.5	98.5	97.0	97.9
		4000	97.7	96.7	96.8	95.4	96.0		97.4	97.1	97.3	96.2	96.8
${\mu }_3$	15	500	99.3	97.6	99.4	88.3	94.0		96.8	98.6	99.9	86.3	93.5
		1000	99.2	97.3	98.8	89.3	94.5		97.3	98.3	99.8	90.8	95.8
		2000	99.4	98.6	97.7	93.6	97.2		99.1	99.5	99.5	96.5	98.9
		4000	99.1	97.9	98.0	94.1	96.6		99.0	98.9	99.3	96.9	98.2
	30	500	99.6	96.5	98.3	89.5	93.8		98.0	97.9	99.6	87.2	94.0
		1000	99.6	97.6	97.4	90.1	95.2		99.1	99.2	99.4	94.7	98.1
		2000	99.3	98.0	96.3	92.9	96.6		99.2	99.2	99.3	97.1	98.4
		4000	98.5	97.1	96.4	93.8	96.0		97.8	97.6	98.1	96.6	97.3
${\sigma }_2$	15	500	99.8	98.8	99.0	89.3	95.7		99.0	99.8	100.0	89.7	97.3
		1000	99.8	98.5	99.0	91.2	96.1		99.5	99.5	99.9	92.7	97.7
		2000	99.6	98.2	98.9	93.3	96.3		99.0	99.6	99.8	95.4	98.3
		4000	99.2	97.5	98.4	94.2	96.2		98.7	98.9	99.4	96.7	98.2
	30	500	99.8	97.9	97.4	90.9	95.3		99.3	99.2	99.9	90.6	96.7
		1000	99.7	97.6	98.0	92.4	95.9		99.5	99.3	99.8	94.7	97.9
		2000	99.3	97.3	97.6	93.8	95.7		99.1	98.9	99.2	96.1	97.9
		4000	98.6	96.7	97.1	94.8	96.0		97.9	97.8	98.0	96.5	97.3
${\sigma }_3$	15	500	99.8	98.7	98.9	89.7	95.9		99.1	99.5	99.9	90.1	97.2
		1000	99.7	98.4	99.2	91.8	96.1		99.2	99.4	99.9	92.4	97.4
		2000	99.7	98.3	98.8	93.5	96.5		99.5	99.6	99.9	95.9	98.5
		4000	99.2	97.6	98.6	94.3	96.0		98.8	99.0	99.6	96.6	97.9
	30	500	99.9	97.9	97.4	90.3	95.5		99.6	99.5	99.8	90.5	96.9
		1000	99.8	97.8	98.0	92.4	96.0		99.6	99.4	99.8	94.3	97.8
		2000	99.5	97.4	97.9	93.3	96.1		99.2	99.1	99.5	96.2	97.9
		4000	98.6	96.8	97.1	94.2	95.9		98.2	98.1	98.3	96.2	97.4

Note. Par = parameter; p = used power in the loss function $\rho$ in invariance alignment; DM = delta method; BNO = bootstrap CI based on normal distribution; BPE = percentile bootstrap CI; BBC = bias-corrected bootstrap CI; BAV = average of CI limits of BNO and BBC; Coverage rates smaller than 91.0 or larger than 98.0 are printed in bold font.

reports the coverage rates for estimated group means and group SDs. In contrast to the results obtained with continuous items, the study involving dichotomous items revealed that the DM method led to markedly inflated coverage rates (i.e., overcoverage) for both the $L_{0.5}$ and $L_0$ loss functions. The bootstrap methods BNO and BPE, which rely on normality assumptions or empirical percentiles, respectively, performed slightly better for IA with $p=0.5$ than for $p=0$. However, several simulation conditions still yielded overcoverage for these CI methods. The BBC method produced acceptable coverage rates for sample sizes of at least $N=1000$, but showed undercoverage for $N=500$. In the latter condition, the BAV method, which averages the CI limits of the BNO and BBC methods, demonstrated improved performance in terms of coverage. While BAV consistently resulted in acceptable coverage for IA with $p=0.5$, it occasionally produced overcoverage for IA with $p=0$, particularly at larger sample sizes.

6. Discussion

This study compared the commonly used delta method (DM) for standard error estimation in IA with alternative bootstrap-based approaches for computing CIs. For IA applied to continuous items, both the $L_{0.5}$ and $L_0$ loss functions yielded reliable statistical inference, with all CI methods producing acceptable coverage rates. In contrast, for dichotomous items, the DM method resulted in pronounced overcoverage, and bootstrap CIs based on the normal distribution (BNO) or empirical percentiles (BPE) also showed a tendency toward overcoverage. In such cases, bias-corrected bootstrap (BBC) CIs should be preferred for moderate to large sample sizes, whereas in smaller samples, a bootstrap CI method that averages the CI limits from BNO and BBC (BAV) demonstrated better coverage performance. Use of the BBC bootstrap is recommended for practitioners of the IA method, at least in the case of dichotomous items.

The finding that standard error assessment for IA using the delta method leads to inflated coverage rates for dichotomous items has not been clearly addressed in the literature. Further research is needed to understand why the results for dichotomous items differ substantially from those for continuous items. One possible explanation is that the normality assumption for estimated item parameters may be better satisfied with continuous items than with dichotomous ones. In addition, item parameter estimates for dichotomous items tend to have much larger standard errors. The linear Taylor approximation used in the delta method may be more appropriate when standard errors are small, and the normality assumption is more closely met.

Consistent with previous studies, IA using the $L_0$ loss function—rather than the commonly applied $L_{0.5}$ loss function—produced less biased and more precise estimates [29,33,38]. Although the Mplus software [47] also allows for the specification of the $L_{0.25}$ loss function, this alternative is similarly outperformed by the $L_0$ loss function. Therefore, greater use of the $L_0$ loss function is recommended in empirical research.

Future research could investigate whether the validity of statistical inference for IA with dichotomous items depends on the degree of invariance violations or the magnitude of DIF effects. Additionally, it would be informative to examine whether similar patterns of overcoverage occur when IA is applied to polytomous item responses [36,43].