Modeling Model Misspecification in Structural Equation Models

Abstract

Structural equation models constrain mean vectors and covariance matrices and are frequently applied in the social sciences. Frequently, the structural equation model is misspecified to some extent. In many cases, researchers nevertheless intend to work with a misspecified target model of interest. In this article, a simultaneous statistical inference for sampling errors and model misspecification errors is discussed. A modified formula for the variance matrix of the parameter estimate is obtained by imposing a stochastic model for model errors and applying M-estimation theory. The presence of model errors is quantified in increased standard errors in parameter estimates. The proposed inference is illustrated with several analytical examples and an empirical application.

1. Introduction

Confirmatory factor analysis (CFA) and structural equation models (SEM) are statistical approaches to analyzing multivariate data in the social sciences [1,2,3,4,5,6,7]. These models relate a multivariate vector $\mathit{X}=(X_1,\dots ,X_I)$ of observed (i.e., manifest) I variables (also referred to as indicators or items) to a vector of latent variables (i.e., factors) $\mathit{\eta }$. SEMs impose constraints on the mean vector $\mathit{\mu }$ and the covariance matrix $\mathbf{\Sigma }$ of the random variable $\mathit{X}$ as a function of an unknown parameter vector $\mathit{\theta }$. In particular, the mean vector is represented as $\mathit{\mu }\left(\mathit{\theta }\right)$, and the covariance matrix is represented by $\mathbf{\Sigma }\left(\mathit{\theta }\right)$.

SEM, and CFA as a particular case, define a measurement model that relates the observed variables $\mathit{X}$ to latent variables $\mathit{\eta }$

$$\mathit{X}=\mathit{\nu }+\mathbf{\Lambda }\mathit{\eta }+\mathit{ϵ}.$$

(1)

In addition, we denote the covariance matrix $\mathsf{Var}\left(\mathit{ϵ}\right)=\mathbf{\Psi }$, and $\mathit{\eta }$ and $\mathit{ϵ}$ are multivariate normally distributed random vectors. The random vectors $\mathit{\eta }$ and $\mathit{ϵ}$ are assumed to be uncorrelated. In CFA, these vectors follow a multivariate normal (MVN) distribution as $\mathit{\eta }\sim \mathsf{MVN}(\mathit{\alpha },\mathbf{\Phi })$ and $\mathit{ϵ}\sim \mathsf{MVN}(0,\mathbf{\Psi })$. Hence, we can write the mean and the covariance matrix in CFA as

$$\mathit{\mu }\left(\mathit{\theta }\right)=\mathit{\nu }+\mathbf{\Lambda }\mathit{\alpha }\mathrm{and}\mathbf{\Sigma }\left(\mathit{\theta }\right)=\mathbf{\Lambda }\mathbf{\Phi }{\mathbf{\Lambda }}^{\top }+\mathbf{\Psi }.$$

(2)

In SEM, relationships among the latent variables can be specified as regression models or path models

$$\mathit{\eta }=\mathbf{B}\mathit{\eta }+\mathit{\xi }\mathrm{with}\mathsf{E}\left(\mathit{\xi }\right)=\mathit{\alpha }\mathrm{and}\mathsf{Var}\left(\mathit{\xi }\right)=\mathbf{\Phi },$$

(3)

where $\mathit{B}$ denotes a matrix of regression coefficients. Hence, the mean vector and the covariance matrix are represented in SEM as

$$\mathit{\mu }\left(\mathit{\theta }\right)=\mathit{\nu }+\mathbf{\Lambda }{(\mathit{I}-\mathit{B})}^{-1}\mathit{\alpha } \mathrm{and} \mathbf{\Sigma }\left(\mathit{\theta }\right)=\mathbf{\Lambda }{(\mathit{I}-\mathit{B})}^{-1}\mathbf{\Phi }{\left[{(\mathit{I}-\mathit{B})}^{-1}\right]}^{\top }{\mathbf{\Lambda }}^{\top }+\mathbf{\Psi },$$

(4)

where $\mathit{I}$ denotes the identity matrix.

In practice, SEM parsimoniously parametrizes the mean vector and the covariance matrix using a parameter vector $\mathit{\theta }$ as a statistical summary. Such restrictions are unlikely to hold in practice, and model assumptions in SEM are only an approximation of a true data-generating model. In SEM, model deviations (e.g., model errors, model misspecification) in covariances emerge as a difference between a population covariance matrix $\mathbf{\Sigma }$ and a model-implied covariance matrix $\mathbf{\Sigma }\left(\mathit{\theta }\right)$ (see Refs. [8,9]). Furthermore, there can be differences in the population mean vector $\mathit{\mu }$ and the model-implied mean vector $\mathit{\mu }\left(\mathit{\theta }\right)$.

This article addresses how to include model misspecification in statistical inference for parameter estimates. Wu and Browne [9,10] proposed an estimation approach that simultaneously models sampling errors and model errors. They do so by modifying the estimation function in the SEM and estimating the model with the maximum likelihood approach. Uanhoro [11] builds on the approach of Wu and Browne [9] but employs Bayesian (i.e., Markov chain Monte Carlo) estimation. Both approaches have in common that the presence of model errors is quantified in increased standard errors in parameter estimates. In this article, the estimation function in the SEM remains unchanged. We derive a simultaneous statistical inference regarding sampling and model errors based on M-estimation theory [12]. As a consequence, this article only addresses an alternative method of estimating standard errors in SEMs. The estimates of model parameters in the SEM are left unchanged.

The remainder of the article is organized as follows. Different estimation methods and standard error estimates with respect to sampling error are reviewed in Section 2. Section 3 presents a stochastic model for model errors and derives the extended variance formula that simultaneously addresses sampling errors and model errors. In Section 4, three analytical illustrative examples show how model errors are reflected in the variance of parameter estimates. Section 5 presents a numerical example using a survey dataset in which the proposed approach is applied. Finally, the article closes with a discussion in Section 6.

2. Estimating Structural Equation Models

In this section, we review different estimation methods for multiple-group SEMs. Note that some identification constraints at the population level must be imposed to estimate the SEM [2,13,14]. When modeling multivariate normally distributed data without missing data, the empirical mean vector $\overline{x}$ and the empirical covariance matrix $\mathit{S}$ are sufficient statistics for an unknown mean vector $\mathit{\mu }$ and covariance matrix $\mathbf{\Sigma }$. Hence, the statistics $\overline{x}$ and $\mathit{S}$ are also sufficient for the parameter vector $\mathit{\theta }$ of the SEM.

Assume that there are G groups with sample sizes $N_g$ and empirical means ${\overline{\mathit{x}}}_g$ and covariance matrices ${\mathit{S}}_g$ for groups $g=1,\dots ,G$. The population mean vectors are denoted by ${\mathit{\mu }}_g$, and the population covariance matrices are denoted by ${\mathbf{\Sigma }}_g$ ($g=1,\dots ,G$). The model-implied mean vectors are denoted by ${\mathit{\mu }}_g\left(\mathit{\theta }\right)$ and the model-implied covariance matrix by ${\mathbf{\Sigma }}_g\left(\mathit{\theta }\right)$. The parameter vector $\mathit{\theta }$ can have common parameters across groups and parameters that are group-specific. For example, in a CFA, equal factor loadings and item intercepts across groups are imposed (i.e., measurement invariance holds; [15,16]) by assuming the same loading matrix $\mathbf{\Lambda }$ and the same intercept vector $\mathit{\nu }$ across groups, while mean vectors and covariance matrices are allowed to differ across groups.

The maximum likelihood (ML) function for the parameter $\mathit{\theta }$ in the SEM is given by the following (see Refs. [2,4]):

$$F_{\mathrm{ML}}(\mathit{\theta };\left\{{\overline{x}}_g\right\},\left\{{\mathit{S}}_g\right\})=-\sum _{g=1}^G\frac{N_g}{2}\left(-Ilog\left(2\pi \right)+\log |{\mathbf{\Sigma }}_g\left(\mathit{\theta }\right)|+\mathrm{tr}({\mathit{S}}_g{\mathbf{\Sigma }}_g{\left(\mathit{\theta }\right)}^{-1})+{({\overline{x}}_g-{\mathit{\mu }}_g\left(\mathit{\theta }\right))}^{\top }{\mathbf{\Sigma }}_g{\left(\mathit{\theta }\right)}^{-1}({\overline{x}}_g-{\mathit{\mu }}_g\left(\mathit{\theta }\right))\right),$$

(5)

  where $\left\{{\overline{x}}_g\right\}$ and $\left\{{\mathit{S}}_g\right\}$ denote the sets of the empirical mean vectors and empirical covariance matrices for groups $g=1,\dots ,G$, respectively. In practice, the model-implied covariance matrix can be misspecified [17,18,19,20], and $\mathit{\theta }$ is a pseudo-true parameter defined as the maximizer of the fitting function $F_{\mathrm{ML}}$ in (5). Importantly, $\mathit{\theta }$ does not refer to a parameter of the data-generating model in this case, but it should be interpreted as a summary of the data that is of central interest to the researcher.

A more general class of fitting functions in SEMs is weighted least squares (WLS) estimation [3,4,21]. The parameter vector $\mathit{\theta }$ is determined as the minimizer of

$$F_{\mathrm{WLS}}(\mathit{\theta };\left\{{\overline{x}}_g\right\},\left\{{\mathit{S}}_g\right\})=\sum _{g=1}^G{({\overline{x}}_g-{\mathit{\mu }}_g\left(\mathit{\theta }\right))}^{\top }{\mathit{W}}_{1g}({\overline{x}}_g-{\mathit{\mu }}_g\left(\mathit{\theta }\right))+\sum _{g=1}^G{(s_g-{\mathbf{\sigma }}_g\left(\mathit{\theta }\right))}^{\top }{\mathit{W}}_{2g}(s_g-{\mathbf{\sigma }}_g\left(\mathit{\theta }\right)),$$

(6)

  where matrices $\mathbf{\Sigma }$ and $\mathit{S}$ have been replaced by vectors $\mathbf{\sigma }$ and $s$ that collect all nonduplicated elements of the matrices in vectors. Formally, we denote the vech operator for this transformation that defines ${\mathbf{\sigma }}_g=\mathrm{vech}\left({\mathbf{\Sigma }}_g\right)$ and $s_g=\mathrm{vech}\left({\mathit{S}}_g\right)$. The weight matrices ${\mathit{W}}_{1g}$ and ${\mathit{W}}_{2g}$ ($g=1,\dots ,G$) can also depend on parameters that must be estimated prior to solving the estimation problem (6). Diagonally weighted least squares (DWLS) estimation results by choosing diagonal weight matrices ${\mathit{W}}_{g1}$ and ${\mathit{W}}_{g2}$. In this case, the fitting function can be written as

$$F_{\mathrm{DWLS}}(\mathit{\theta };\left\{{\overline{\mathit{x}}}_g\right\},\left\{{\mathit{S}}_g\right\})=\sum _{g=1}^G\sum _{i=1}^I{w}_{1gi}{({\overline{x}}_{gi}-{\mu }_{gi}\left(\mathit{\theta }\right))}^2+\sum _{g=1}^G\sum _{i=1}^I\sum _{j=i}^I{w}_{2gij}{(s_{gij}-{\sigma }_{gij}\left(\mathit{\theta }\right))}^2,$$

(7)

  where $w_{1gi}$ and $w_{2gij}$ are appropriate elements in ${\mathit{W}}_{1g}$ and ${\mathit{W}}_{2g}$, respectively. Unweighted least squares (ULS) estimation is obtained by setting all weights $w_{1gi}$ and $w_{2gij}$ equal to one.

Interestingly, the minimization of $F_{\mathrm{DWLS}}$ in (6) with respect to the parameter $\mathit{\theta }$ can be viewed as a nonlinear least squares estimation problem with sufficient statistics $\left\{{\overline{\mathit{x}}}_g\right\}$ and $\left\{{\mathit{S}}_g\right\}$ as input data [22]. It has been shown that ML estimation can be approximately written as DWLS estimation [23] with particular weight matrices. The weights are approximately determined by $w_{1gi}=1/u_{gi}^2$ and $w_{2gij}=1/\left(u_{gi}^2{u}_{gj}^2\right)$, where $u_{gi}^2$ are sample unique standardized variances with $u_{gi}^2={\psi }_{gii}/{\sigma }_{gii}$ (see Ref. [23]).

The fitting functions can be slightly more generally formulated as a sum of group-specific fitting functions

$$F(\mathit{\theta },\widehat{\mathit{\xi }})=\sum _{g=1}^G{F}_g(\mathit{\theta },{\widehat{\mathit{\xi }}}_g),$$

(8)

where ${\widehat{\mathit{\xi }}}_g=({\overline{x}}_g,s_g)$ denote the vectors of group-specific sufficient statistics ($g=1,\dots ,G$). The parameter estimate $\widehat{\mathit{\theta }}$ is obtained as the root of the partial derivative of F with respect to $\mathit{\theta }$ defined in (8):

$$F_{\mathit{\theta }}(\mathit{\theta },\widehat{\mathit{\xi }})=\sum _{g=1}^G{F}_{g,\mathit{\theta }}(\mathit{\theta },{\widehat{\mathit{\xi }}}_g)=\mathbf{0},$$

(9)

where $F_{\mathit{\theta }}$ and $F_{g,\mathit{\theta }}$ denote the partial derivatives of F and $F_g$ with respect to $\mathit{\theta }$, respectively. The parameter estimate $\widehat{\mathit{\theta }}$ is a nonlinear function of the input vector of sufficient statistics $\widehat{\mathit{\xi }}$. Hence, the distribution of $\widehat{\mathit{\theta }}$ can be expressed as a function of the distribution of $\widehat{\mathit{\xi }}$ by applying the multivariate delta method [17,24] (see also [25]).

The asymptotic distribution of the vector of sufficient statistics ${\widehat{\mathit{\xi }}}_g$ is given as

$${\widehat{\mathit{\xi }}}_g-\mathit{\xi }=\left(\begin{array}{c}{\overline{\mathit{x}}}_g\\ {\mathit{s}}_g\end{array}\right)-\left(\begin{array}{c}{\mathit{\mu }}_g\\ {\mathbf{\sigma }}_g\end{array}\right)\sim \mathsf{MVN}(\mathbf{0},{\mathit{V}}_g).$$

(10)

The covariance matrix ${\mathit{V}}_g$ is determined by

$${\mathit{V}}_g=N_g^{-1}\left(\begin{array}{cc}{\mathbf{\Sigma }}_g& \mathbf{0}\\ \mathbf{0}& \mathit{K}({\mathbf{\Sigma }}_g\otimes {\mathbf{\Sigma }}_g){\mathit{K}}^{\top }\end{array}\right),$$

(11)

where ⊗ denotes the Kronecker product and $\mathit{K}$ is a matrix containing entries 0, 0.5, and 1 such that ${\mathbf{\sigma }}_g=\mathrm{vech}\left({\mathbf{\Sigma }}_g\right)=\mathit{K}\mathrm{vec}\left({\mathbf{\Sigma }}_g\right)$, where the $\mathrm{vec}$ operator stacks all elements of a matrix into a vector. The covariance matrix ${\mathit{V}}_g$ in (11) can be estimated by substituting the population covariance matrix ${\mathbf{\Sigma }}_g$ with the empirical covariance matrix ${\mathit{S}}_g$:

$${\widehat{\mathit{V}}}_g=N_g^{-1}\left(\begin{array}{cc}{\mathit{S}}_g& \mathbf{0}\\ \mathbf{0}& \mathit{K}({\mathit{S}}_g\otimes {\mathit{S}}_g){\mathit{K}}^{\top }\end{array}\right).$$

(12)

The covariance matrix $\mathit{V}=\mathsf{Var}\left(\widehat{\mathit{\xi }}\right)$ is given as a block-diagonal matrix of covariance matrices ${\mathit{V}}_g$ for $g=1,\dots ,G$:

$$\mathit{V}=\left(\begin{array}{cccc}{\mathit{V}}_1& \mathbf{0}& \cdots & \mathbf{0}\\ \mathbf{0}& {\mathit{V}}_2& & \mathbf{0}\\ ⋮& & \ddots & ⋮\\ \mathbf{0}& \mathbf{0}& \cdots & {\mathit{V}}_G\end{array}\right).$$

(13)

A corresponding estimate $\widehat{\mathit{V}}$ is obtained by using estimates ${\widehat{\mathit{V}}}_g$ ($g=1,\dots ,G$) for group-specific covariance matrices.

Assume that the population sufficient statistics are denoted by ${\mathit{\xi }}_0$ and there exists a pseudo-true parameter ${\mathit{\theta }}_0$ such that $F_{\mathit{\theta }}(\mathit{\theta },{\mathit{\xi }}_0)=\mathbf{0}$. Hence, we can write ${\mathit{\xi }}_0=\mathit{\xi }\left({\mathit{\theta }}_0\right)$. Note again that the parameter ${\mathit{\xi }}_0$ does not refer to a data-generating parameter, but it is defined as a summary of the data by choosing a particular function F. Different pseudo-true parameters ${\mathit{\theta }}_0$ will be obtained for different choices of fitting functions F in misspecified SEMs; specifically, ${\mathit{\theta }}_0$ is a function of ${\mathit{\xi }}_0$ and F (i.e., ${\mathit{\theta }}_0=g({\mathit{\xi }}_0,F)$ for some function g).

We now derive the covariance matrix of $\widehat{\mathit{\theta }}$ by utilizing a Taylor expansion of $F_{\mathit{\theta }}$ around $({\mathit{\theta }}_0,{\mathit{\xi }}_0)$. Denote by $F_{\mathit{\theta }\mathit{\theta }}$ and $F_{\mathit{\theta }\mathit{\xi }}$ the matrices of second-order partial derivatives of $F_{\mathit{\theta }}$ with respect to $\mathit{\theta }$ and $\mathit{\xi }$, respectively. Then, we obtain

$$F_{\mathit{\theta }}(\widehat{\mathit{\theta }},\widehat{\mathit{\xi }})=F_{\mathit{\theta }}({\mathit{\theta }}_0,{\mathit{\xi }}_0)+F_{\mathit{\theta }\mathit{\theta }}({\mathit{\theta }}_0,{\mathit{\xi }}_0)(\widehat{\mathit{\theta }}-{\mathit{\theta }}_0)+F_{\mathit{\theta }\mathit{\xi }}({\mathit{\theta }}_0,{\mathit{\xi }}_0)(\widehat{\mathit{\xi }}-{\mathit{\xi }}_0).$$

(14)

As the parameter estimate $\widehat{\mathit{\theta }}$ is a nonlinear function of $\widehat{\mathit{\xi }}$, the Taylor expansion (14) provides the approximation

$$\widehat{\mathit{\theta }}-{\mathit{\theta }}_0=-F_{\mathit{\theta }\mathit{\theta }}{({\mathit{\theta }}_0,{\mathit{\xi }}_0)}^{-1}{F}_{\mathit{\theta }\mathit{\xi }}({\mathit{\theta }}_0,{\mathit{\xi }}_0)(\widehat{\mathit{\xi }}-{\mathit{\xi }}_0).$$

(15)

By defining $\mathit{A}=F_{\mathit{\theta }\mathit{\theta }}({\mathit{\theta }}_0,{\mathit{\xi }}_0)$ and $\mathit{B}=F_{\mathit{\theta }\mathit{\xi }}({\mathit{\theta }}_0,{\mathit{\xi }}_0)$, we obtain the multivariate delta formula [17,26]

$$\mathsf{Var}(\widehat{\mathit{\theta }})={\mathit{A}}^{-1}\mathit{B}\mathit{V}{\mathit{B}}^{\top }{\left({\mathit{A}}^{-1}\right)}^{\top }.$$

(16)

The matrices $\mathit{A}$ and $\mathit{B}$ can be estimated by $\widehat{\mathit{A}}=F_{\mathit{\theta }\mathit{\theta }}(\widehat{\mathit{\theta }},\widehat{\mathit{\xi }})$ and $\widehat{\mathit{B}}=F_{\mathit{\theta }\mathit{\xi }}(\widehat{\mathit{\theta }},\widehat{\mathit{\xi }})$. The estimated covariance matrix $\mathsf{Var}\left(\widehat{\mathit{\theta }}\right)$ in (16) can be used for statistical inference, such as the computation of standard errors or the application of Wald tests.

The standard error (SE) of the lth entry ${\widehat{\theta }}_l$ in $\widehat{\mathit{\theta }}$ is given by

$$\mathrm{SE}\left({\widehat{\theta }}_l\right)=\sqrt{{\left(\mathsf{Var}\left(\widehat{\mathit{\theta }}\right)\right)}_{ll}}=\sqrt{{\left({\mathit{A}}^{-1}\mathit{B}\mathit{V}{\mathit{B}}^{\top }{\left({\mathit{A}}^{-1}\right)}^{\top }\right)}_{ll}}.$$

(17)

In this section, we computed the covariance matrix of the parameter estimate $\widehat{\mathit{\xi }}$ with respect to sampling errors. In particular, we assumed a sampling scheme of identically and independently distributed observations that led to variability in sufficient statistics $\widehat{\mathit{\xi }}$, which, in turn, resulted in variability in estimated model parameters $\widehat{\mathit{\theta }}$ across repeated sampling. In the next section, we additionally address the extent of model misspecification errors in the covariance matrix. The presence of model misspecification should be quantified in increased standard errors. We do so by imposing a stochastic model on model specification errors.

3. Modeling Model Misspecification

In this section, we impose a stochastic model on model misspecification in the SEM. At the population level, the population mean vector $\mathit{\mu }$ can differ from the model-implied mean vector $\mathit{\mu }\left({\mathit{\theta }}_0\right)$, and the (vectorized) population covariance matrix $\mathbf{\sigma }$ can differ from the model-implied covariance matrix $\mathbf{\sigma }\left({\mathit{\theta }}_0\right)$. As in Section 2, we define the vector $\mathit{\xi }=(\mathit{\mu },\mathbf{\sigma })$, where $\mathit{\xi }$ contains all group-specific means and covariances.

3.1. Stochastic Model for Model Misspecification

Assume that there exists a ${\mathit{\theta }}_0$ such that

$$\mathit{\xi }=\mathit{\xi }\left({\mathit{\theta }}_0\right)+\mathit{e},$$

(18)

where $\mathit{e}$ constitutes the model specification error. The vector $\mathit{e}$ contains deviations (i.e., model misspecification) in all means and pairwise covariances in all groups. Formally, we define ${\mathit{e}}_{\mathit{\mu }}={\left(e_{\mathit{\mu },g,i}\right)}_{g=1,\dots ,G;i=1,\dots ,I}$, ${\mathit{e}}_{\mathbf{\sigma }}={\left(e_{\mathbf{\sigma },g,i}\right)}_{g=1,\dots ,G;i,j=1,\dots ,I\mathrm{for}i<j}$ and $\mathit{e}=({\mathit{e}}_{\mathit{\mu },g},{\mathit{e}}_{\mathbf{\sigma },g})$. Assume that $\mathsf{E}\left(\mathit{e}\right)=0$.

To model misspecification in the mean structure, assume that $e_{\mathit{\mu },g,i}$ are normally distributed variables with zero mean and variance ${\tau }_{\mathit{\mu }}$ (i.e., $\mathsf{E}\left(e_{\mathit{\mu },g,i}\right)=0$ and $\mathsf{Var}\left(e_{\mathit{\mu },g,i}\right)={\tau }_{\mathit{\mu }}$). All variables $e_{\mathit{\mu },g,i}$ contained in the vector ${\mathit{e}}_{\mathit{\mu }}$ are independently and identically distributed.

To model misspecification in the covariance structure, we assume an effect decomposition of the error in the modeled covariance of items $i\ne j$ in group g as

$$e_{\mathbf{\sigma },g,ij}=u_{g,i}+u_{g,j}+v_{g,ij},$$

(19)

where $u_{g,i}$ and $v_{g,ij}$ are uncorrelated random effects for all i and all pairs $i\ne j$, respectively. The model (19) is a cross-classified two-level model [27]. The stochastic model in (19) fundamentally differs from the approach in [11] that assumes independent $e_{\mathbf{\sigma },g,ij}$ effects (i.e., there are no item effects $u_{g,i}$ for model errors in covariances). Because an item appears in several item pairs referring to different covariances, we find the inclusion of item effects $u_{g,i}$ in (19) more plausible. The appearance of $u_{g,i}$ and $u_{g,j}$ in (19) might be motivated by the fact that (intentionally) misspecified factor loadings of item i enter all residuals $e_{\mathbf{\sigma },g,ij}$ with $j\ne i$ (see Appendix A). For multidimensional factor models, the stochastic model (19) might be made more general (see (A7) in Appendix A). Note that we set $e_{\mathbf{\sigma },g,ii}=0$ as in [11]. Thus, diagonal entries in the covariance matrix are assumed to be correctly specified at the population level. We can compute for $i\ne j$ and $k\ne h$

$$\mathsf{E}\left(e_{\mathbf{\sigma },g,ij}{e}_{\mathbf{\sigma },g,kh}\right)=\left\{\begin{array}{cc}0& \mathrm{if}\mathrm{card}\left(\{i,j\}\cap \{k,h\}\right)=0\\ {\tau }_{\mathbf{\sigma },2}& \mathrm{if}\mathrm{card}\left(\{i,j\}\cap \{k,h\}\right)=1\\ 2{\tau }_{\mathbf{\sigma },2}+{\tau }_{\mathbf{\sigma },1}& \mathrm{if}\mathrm{card}\left(\{i,j\}\cap \{k,h\}\right)=2\end{array}\right.,$$

(20)

where $\mathrm{card}\left(A\right)$ denotes the cardinality of a set A. The condition $\mathrm{if}\mathrm{card}\left(\{i,j\}\cap \{k,h\}\right)=2$ means that $i=k$ and $j=h$; that is, $\mathsf{E}\left(e_{\mathbf{\sigma },g,ij}^2\right)=2{\tau }_{\mathbf{\sigma },2}+{\tau }_{\mathbf{\sigma },1}$.

3.2. Estimating the Variance Components in the Stochastic Model

The model errors $e_{\mathit{\mu },g,i}$ and $e_{\mathbf{\sigma },g,ij}$ are not directly observable. For statistical inference regarding the stochastic model for model misspecification, the variance components in (20) must be estimated. Instead of computing $\mathit{e}=\mathit{\xi }-\mathit{\xi }\left({\mathit{\theta }}_0\right)$, we compute empirical residuals $\widehat{\mathit{e}}$ that are defined as $\widehat{\mathit{e}}=\widehat{\mathit{\xi }}-\mathit{\xi }\left(\widehat{\mathit{\theta }}\right)$. Note that these residuals are included in the standard output of widespread SEM software [28,29,30].

One can compute quantities ${\widehat{e}}_{\mathit{\mu },g,i}^2$ and ${\widehat{e}}_{\mathbf{\sigma },g,ij}{\widehat{e}}_{\mathbf{\sigma },g,kh}$ for $(i,j,k,h)$ as estimates of ${\mathit{e}}_{\mathit{\mu },g,i}$ and $e_{\mathbf{\sigma },g,ij}{e}_{\mathbf{\sigma },g,hk}$ and equate them with expected values. This approach is referred to as the method of moments [31]. Define the vector $\mathbf{\tau }=({\tau }_{\mathit{\mu }},{\tau }_{\mathbf{\sigma },2},{\tau }_{\mathbf{\sigma },1})$. The vector of empirical variances and covariances that contain the product quantities is denoted by $\widehat{\mathbf{z}}$. According to (20), the expected values of cross-products are linear in $\mathbf{\tau }$. Hence, the method of moments maps the empirical (co)variances defined in $\widehat{\mathbf{z}}$ to the vector of unknown variance components $\mathbf{\tau }$ using the linear model

$$\widehat{\mathbf{z}}=\mathit{H}\mathbf{\tau }+\mathit{\epsilon },$$

(21)

where $\mathit{H}$ is an appropriate known design matrix that contains entries 0, 1, or 2. The linear model (21) can be solved by

$$\widehat{\mathbf{\tau }}=\tilde{\mathit{H}}\widehat{\mathbf{z}}\mathrm{with}\tilde{\mathit{H}}={\left({\mathit{H}}^{\top }\mathit{H}\right)}^{-1}\mathit{H}.$$

(22)

Negatively estimated variances can be set to zero.

In the case of our defined variance component model for the mean and the covariance structure, simple formulas for the variance estimates can be derived. The variance ${\tau }_{\mathit{\mu }}$ can be estimated by

$${\widehat{\tau }}_{\mathit{\mu }}=\frac{1}{GI}\sum _{g=1}^G\sum _{i=1}^I{\widehat{e}}_{\mathit{\mu },g,i}^2.$$

(23)

Let ${\mathcal{M}}_a$ ($a=0,1,2$) denote the set of cross-products of residuals ${\widehat{e}}_{\mathbf{\sigma },g,ij}$ and ${\widehat{e}}_{\mathbf{\sigma },g,kh}$ with $\mathrm{card}\left(\{i,j\}\cap \{k,h\}\right)=a$. We define ${\mathcal{Y}}_a$ as the average of the products from the set ${\mathcal{M}}_a$. Then, we use the estimate ${\widehat{\tau }}_{\mathbf{\sigma },2}=max(0,{\mathcal{Y}}_1)$. Finally, we compute ${\widehat{\tau }}_{\mathbf{\sigma },1}=max(0,{\mathcal{Y}}_2-2{\widehat{\tau }}_{\mathbf{\sigma },2})$.

However, $\widehat{\mathbf{z}}$ is affected by sampling error. We can write $\widehat{\mathit{e}}-\mathit{e}\sim \mathsf{MVN}(\mathbf{0},{\mathit{V}}_2)$. The vector $\widehat{\mathbf{z}}$ contains products of normally distributed variables. Hence, the bias $\mathcal{B}$ in $\widehat{\mathbf{z}}$ due to sampling errors can be estimated by computing

$$\mathsf{E}\left(\widehat{\mathbf{z}}\right)=\mathcal{B}+\mathbf{z}$$

(24)

Then, we obtain from (22) and (24)

$$\widehat{\mathbf{\tau }}=\tilde{\mathit{H}}\left(\widehat{\mathbf{z}}-\mathcal{B}\right)$$

(25)

The bias $\mathcal{B}$ can also be determined by resampling techniques. The parameter estimates ${\widehat{\tau }}_{\mathit{\mu }}$, ${\widehat{\tau }}_{\mathbf{\sigma },2}$, and ${\widehat{\tau }}_{\mathbf{\sigma },1}$ can be repeatedly computed from bootstrap samples of subjects. Then, a bootstrap bias of variance components can be determined [32]. As a result, bias-corrected variance component estimates can be computed. Again, negatively estimated variances are set to zero.

3.3. Error in Model Parameters Due to Model Misspecification

Now, the variance component $\mathbf{\tau }$ referring to model misspecification has been determined. In the next step, we compute the variance in the SEM parameter estimate $\widehat{\mathit{\theta }}$ due to model misspecification error. As in Section 2, we apply a Taylor expansion around $({\mathit{\theta }}_0,{\mathit{\xi }}_0)$ with $\mathit{\xi }=\mathit{\xi }\left({\mathit{\theta }}_0\right)$ and obtain

$$F_{\mathit{\theta }}(\widehat{\mathit{\theta }},\mathit{\xi })=F_{\mathit{\theta }}({\mathit{\theta }}_0,{\mathit{\xi }}_0)+F_{\mathit{\theta }\mathit{\theta }}({\mathit{\theta }}_0,{\mathit{\xi }}_0)(\widehat{\mathit{\theta }}-{\mathit{\theta }}_0)+\sum _{g=1}^G{F}_{g,\mathit{\theta }\mathit{\xi }}({\mathit{\theta }}_0,{\mathit{\xi }}_{g,0})({\mathit{\xi }}_g-{\mathit{\xi }}_{g,0})$$

(26)

Using again the abbreviation $\mathit{A}=F_{\mathit{\theta }\mathit{\theta }}({\mathit{\theta }}_0,{\mathit{\xi }}_0)$, we obtain, by solving for $\widehat{\mathit{\theta }}$ in (26),

$$\widehat{\mathit{\theta }}-{\mathit{\theta }}_0={\mathit{A}}^{-1}\sum _{g=1}^G{F}_{g,\mathit{\theta }\mathit{\xi }}({\mathit{\theta }}_0,{\mathit{\xi }}_{g,0})({\mathit{\xi }}_g-{\mathit{\xi }}_{g,0})$$

(27)

We now simplify (27) regarding the distributional assumptions of model misspecification. Denote by ${\mathcal{M}}_{g,i}$ and ${\mathcal{C}}_{g,ij}$ the corresponding second-order derivatives with respect to appropriate entries in ${\mathit{\xi }}_g$ in the function $F_{g,\mathit{\theta },\mathit{\xi }}$. Moreover, we set ${\mathcal{C}}_{g,ii}=0$ and note that ${\mathcal{C}}_{g,ij}={\mathcal{C}}_{g,ji}$. We then obtain, for the variance contribution of group g,

$$\begin{array}{ccc}\hfill F_{g,\mathit{\theta }\mathit{\xi }}({\mathit{\theta }}_0,{\mathit{\xi }}_{g,0})({\mathit{\xi }}_g-{\mathit{\xi }}_{g,0})& =& {\displaystyle \sum _{i=1}^I{\mathcal{M}}_{g,i}{e}_{\mathit{\mu },g,i}+\sum _{i=1}^{I-1}\sum _{j=i+1}^I{\mathcal{C}}_{g,ij}{e}_{\mathbf{\sigma },g,ij}}\hfill \\ & =& {\displaystyle \sum _{i=1}^I{\mathcal{M}}_{g,i}{e}_{\mathit{\mu },g,i}+\sum _{i=1}^{I-1}\sum _{j=i+1}^I{\mathcal{C}}_{g,ij}(u_{g,i}+u_{g,j}+v_{g,ij})}\hfill \\ & =& {\displaystyle \sum _{i=1}^I{\mathcal{M}}_{g,i}{e}_{\mathit{\mu },g,i}+\sum _{i=1}^I{u}_{g,i}\sum _{j=1}^I{\mathcal{C}}_{g,ij}+\sum _{i=1}^{I-1}\sum _{j=i+1}^I{\mathcal{C}}_{g,ij}{v}_{g,ij}}\hfill \end{array}$$

(28)

We can now derive the variance ${\mathcal{V}}_g=\mathsf{Var}\left(F_{g,\mathit{\theta }\mathit{\xi }}({\mathit{\theta }}_0,{\mathit{\xi }}_{g,0})({\mathit{\xi }}_g-{\mathit{\xi }}_{g,0})\right)$:

$${\displaystyle {\mathcal{V}}_g={\tau }_{\mathit{\mu }}\left(\sum _{i=1}^I{\mathcal{M}}_{g,i}{\mathcal{M}}_{g,i}^{\top }\right)+{\tau }_{\mathbf{\sigma },1}\left(\sum _{i=1}^I\left(\sum _{j=1}^I{\mathcal{C}}_{g,ij}\right){\left(\sum _{j=1}^I{\mathcal{C}}_{g,ij}\right)}^{\top }\right)+{\tau }_{\mathbf{\sigma },2}\left(\sum _{i=1}^{I-1}\sum _{j=i+1}^I{\mathcal{C}}_{g,ij}{\mathcal{C}}_{g,ij}^{\top }\right).}$$

(29)

By using the abbreviation $\mathcal{V}={\sum }_{g=1}^G{\mathcal{V}}_g$, we obtain by using (27)

$$\mathsf{Var}\left(\widehat{\mathit{\theta }}\right)={\mathit{A}}^{-1}\mathcal{V}{\left({\mathit{A}}^{-1}\right)}^{\top }.$$

(30)

The misspecification error (ME) for th lth entry ${\widehat{\theta }}_l$ in $\widehat{\mathit{\theta }}$ is given by

$$\mathrm{ME}\left({\widehat{\theta }}_l\right)=\sqrt{{\left(\mathsf{Var}\left(\widehat{\mathit{\theta }}\right)\right)}_{ll}}=\sqrt{{\left({\mathit{A}}^{-1}\mathcal{V}{\left({\mathit{A}}^{-1}\right)}^{\top }\right)}_{ll}}.$$

(31)

As an alternative to the proposed analytical solution in this subsection, the uncertainty in parameter estimates due to model misspecification can be assessed by parametric bootstrapping. This procedure is based on the stochastic model (18). If variance components for the stochastic model for residuals $\mathit{e}$ in (18) are estimated, a random draw of new residuals ${\mathit{e}}^{*}$ can be conducted for each bootstrap sample, which subsequently provides a draw from the vector of sufficient statistics ${\mathit{\xi }}^{*}=\mathit{\xi }\left(\widehat{\mathit{\theta }}\right)+{\mathit{e}}^{*}$. A parameter estimate $\widehat{\mathit{\theta }}$ for each bootstrap sample is obtained by solving $F_{\mathit{\theta }}(\mathit{\theta },{\mathit{\xi }}^{*})=\mathbf{0}$. By drawing a large number of bootstrap samples, the distribution of $\widehat{\mathit{\theta }}$ with respect to misspecification error can be determined (see [33,34] for a similar approach).

3.4. Computing the Total Error

The variance in (30) is due to the imposed stochastic model on model errors. In addition, there exists a sampling error in the parameter estimate $\widehat{\mathit{\theta }}$ that has been derived in Section 2. By adding the variance matrices computed in (16) (i.e., sampling error) and (30) (i.e., model error), we finally obtain the total variance matrix:

$$\mathsf{Var}\left(\widehat{\mathit{\theta }}\right)={\mathit{A}}^{-1}\left(\mathit{B}\mathit{V}{\mathit{B}}^{\top }+\mathcal{V}\right){\left({\mathit{A}}^{-1}\right)}^{\top }.$$

(32)

Hence, the total error (TE) of the lth entry ${\widehat{\theta }}_l$ in $\widehat{\mathit{\theta }}$ is determined by

$$\mathrm{TE}\left({\widehat{\theta }}_l\right)=\sqrt{\mathrm{SE}{\left({\widehat{\theta }}_l\right)}^2+\mathrm{ME}{\left({\widehat{\theta }}_l\right)}^2}.$$

(33)

To summarize, the steps described in the previous and this section resulted in the variance formula (32) that integrates sampling error and model error in a simultaneous inference without changing the estimation equation. The steps described here should be sufficient for the practical implementation of the proposed standard errors.

In the next section, we present illustrative examples of the computation of the model error component for the parameter estimate $\widehat{\mathit{\theta }}$.

4. Analytical Illustrative Examples

In this section, three illustrative examples are presented in which the variance due to model errors is quantified. In the examples, we assume infinite sample sizes. Specifically, sampling errors are ignored. Furthermore, we only consider ULS estimation. However, we believe that despite the simplified assumptions, the properties of the modeled specification error can be grasped more easily.

4.1. Example 1: Misspecified Error Structure in Unidimensional Factor Analysis

In Example 1, we consider a unidimensional CFA in a single group. We assume equal loadings of one and estimate the factor variance $\varphi$. The residual variances are allowed to vary across items. For I items $X_1,\dots ,X_I$, the covariance is defined as ${\sigma }_{ij}=\mathsf{Cov}(X_i,X_j)$. The assumed stochastic model for model errors is defined as

$${\sigma }_{ij}=\varphi +e_{\mathbf{\sigma },ij},$$

(34)

where the error is decomposed into (see (19))

$$e_{\mathbf{\sigma },ij}=u_i+u_j+v_{ij}$$

(35)

with $\mathsf{E}\left(u_i\right)=\mathsf{E}\left(v_{ij}\right)=0$, $\mathsf{Var}\left(u_i\right)={\tau }_{\mathbf{\sigma },2}$, and $\mathsf{Var}\left(v_{ij}\right)={\tau }_{\mathbf{\sigma },1}$.

The estimating equation for $\varphi$ in ULS estimation is given by

$$F_{\mathit{\theta }}(\mathit{\theta };\mathit{\xi })=\sum _{i=1}^{I-1}\sum _{j=i+1}^I({\sigma }_{ij}-\varphi )=0.$$

(36)

Hence, the second-order derivative is given by

$$F_{\mathit{\theta }\mathit{\theta }}(\mathit{\theta };\mathit{\xi })=-\frac{I(I-1)}{2}.$$

(37)

By inserting the data-generating model for model errors in (36), we obtain

$$0=-\frac{I(I-1)}{2}(\widehat{\varphi }-\varphi )+\sum _{i=1}^{I-1}\sum _{j=i+1}^I{e}_{\mathbf{\sigma },ij}$$

(38)

Solving this equation with respect to $\widehat{\varphi }$, we arrive at

$$\widehat{\varphi }=\varphi +\frac{2}{I}\sum _{i=1}^I{u}_i+\frac{2}{I(I-1)}\sum _{i=1}^{I-1}\sum _{j=i+1}^I{v}_{ij}.$$

(39)

Hence, the variance in $\widehat{\varphi }$ due to model errors can be calculated as

$$\mathsf{Var}\left(\widehat{\varphi }\right)=\frac{4}{I}{\tau }_{\mathbf{\sigma },2}+\frac{2}{I(I-1)}{\tau }_{\mathbf{\sigma },1}.$$

(40)

4.2. Example 2: Misspecified Error Structure in Confirmatory Factor Analysis

In Example 2, we consider a two-dimensional CFA. The items $X_1,\dots ,X_{I_1}$ load on the first factor, while items $X_{I_1+1},\dots ,X_{I_1+I_2}$ load on the second factor. There are no modeled cross-loadings in the analysis model. However, by imposing a stochastic model for model errors, the effect of unmodeled cross-loadings is quantified in increased standard errors in the model parameter estimates.

The data-generating model for covariances ${\sigma }_{ij}=\mathsf{Cov}(X_i,X_j)$ for $i\ne j$ is given by

$${\sigma }_{ij}=\left\{\begin{array}{cc}{\varphi }_{11}+e_{\mathbf{\sigma },ij}& \mathrm{if}i\le I_1\mathrm{and}j\le I_1\hfill \\ {\varphi }_{12}+e_{\mathbf{\sigma },ij}& \mathrm{if}i\le I_1\mathrm{and}j\ge I_1+1\hfill \\ {\varphi }_{22}+e_{\mathbf{\sigma },ij}& \mathrm{if}i\ge I_1+1\mathrm{and}j\ge I_1+1\hfill \end{array}\right.$$

(41)

As in Example 1, we impose the stochastic model for misspecified covariances as

$$e_{\mathbf{\sigma },ij}=u_i+u_j+v_{ij}.$$

(42)

The estimating equation for the factor covariance ${\varphi }_{12}$ based on ULS is given by (see [8])

$$F_{\mathit{\theta }}(\mathit{\theta };\mathit{\xi })=\sum _{i=1}^{I_1}\sum _{j=I_1+1}^{I_2}({\sigma }_{ij}-{\varphi }_{12})=0.$$

(43)

Hence, we obtain for the estimate ${\widehat{\varphi }}_{12}$

$${\widehat{\varphi }}_{12}={\varphi }_{12}+\frac{1}{I_1{I}_2}\sum _{i=1}^{I_1}\sum _{j=I_1+1}^{I_2}{e}_{\mathbf{\sigma },ij}.$$

(44)

Therefore, we obtain the variance for ${\widehat{\varphi }}_{12}$ due to model misspecification

$$\mathsf{Var}\left({\widehat{\varphi }}_{12}\right)=\left(\frac{1}{I_1}+\frac{1}{I_2}\right){\tau }_{\mathbf{\sigma },2}+\frac{1}{I_1{I}_2}{\tau }_{\mathbf{\sigma },1}.$$

(45)

Interestingly, the variance in ${\widehat{\varphi }}_{12}$ is determined by the smaller number of items per factor (i.e., $min(I_1,I_2)$) if there exists variance in the random item factor (i.e., ${\tau }_{\mathbf{\sigma },2}>0$).

4.3. Example 3: Measurement Noninvariance in Multiple-Group SEM

Finally, in Example 3, we quantify the extent of measurement noninvariance in increased standard errors in parameter estimates. It has been argued that misspecified SEM can result if violations of measurement invariance are intentionally ignored in model estimation [35,36,37,38].

We assume a multiple-group unidimensional CFA. We assume model deviations in the mean structure of group g that refer to measurement noninvariance

$${\mu }_{g,i}={\nu }_i+{\lambda }_i{\alpha }_g+e_{\mathit{\mu },g,i}.$$

(46)

The estimating equation for the group mean ${\alpha }_g$ for group g is given by

$$F_{\mathit{\theta }}(\mathit{\theta };\mathit{\xi })=\sum _{i=1}^I({\mu }_{g,i}-{\nu }_i-{\lambda }_i{\alpha }_g)=0,$$

(47)

where parameters ${\nu }_i$ and ${\lambda }_i$ are assumed to be already estimated for $i=1,\dots ,I$ for ease of presentation. Then, we obtain the estimate ${\widehat{\alpha }}_g$ as

$${\widehat{\alpha }}_g=\frac{{\displaystyle \sum _{i=1}^I({\mu }_{g,i}-{\nu }_i)}}{{\displaystyle \sum _{i=1}^I{\lambda }_i}}={\alpha }_g+\frac{{\displaystyle \sum _{i=1}^I{e}_{\mathit{\mu },g,i}}}{{\displaystyle \sum _{i=1}^I{\lambda }_i}}.$$

(48)

Therefore, the variance of ${\widehat{\alpha }}_g$ can be computed as

$$\mathsf{Var}\left({\widehat{\alpha }}_g\right)=\frac{I}{{\displaystyle {\left(\sum _{i=1}^I{\lambda }_i\right)}^2}}{\tau }_{\mathit{\mu }}$$

(49)

If all loadings are set to one (i.e., ${\lambda }_i=1$ for all $i=1,\dots ,I$), we obtain from (49)

$$\mathsf{Var}\left({\widehat{\alpha }}_g\right)=\frac{1}{I}{\tau }_{\mathit{\mu }}$$

(50)

The quantity in (50) corresponds to the well-known linking error of the one-parameter logistic item response model [39,40,41,42]. Hence, the quantification of model misspecification can be seen as an alternative to assessing uncertainty in model parameters regarding the selected items. To some extent, one can argue that modeling model misspecification is conceptually equivalent to assessing linking errors, although linking errors are mainly considered in multiple-group settings.

5. Numerical Illustrative Example: ESS 2005 Data

5.1. Method

In this empirical example, we use a dataset that was also analyzed in [25,43,44,45]. The data came from the European Social Survey (ESS) conducted in the year 2005 (ESS 2005) that included subjects from 26 countries. The latent factor variable of tradition and conformity was assessed by four items presented in portrait format, where the scale of the items is such that a high value represents a low level of tradition conformity. The wording of the four items was as follows (see [45]): “It is important for him to be humble and modest. He tries not to draw attention to himself”. (item TR9); “Tradition is important to him. He tries to follow the customs handed down by his religion or family” (item TR20); “He believes that people should do what they’re told. He thinks people should follow rules at all times, even when no one is watching” (item CO7); and “It is important for him to always behave properly. He wants to avoid doing anything people would say is wrong” (item CO16). The full dataset used in [45] was downloaded from https://www.statmodel.com/Alignment.shtml (accessed on 9 May 2023).

In this application, we used ten selected countries C01, C05, C08, C10, C13, C15, C16, C17, C21, C25 using the country labels from [25]. This resulted in a subsample of $N=\mathrm{19,916}$ persons. The sample sizes per country ranged between 1450 and 2622, with an average of 1991.6 ($SD=375.4$). We only included participants in the sample that had no missing values on all four items.

A multiple-group one-dimensional factor model with 10 groups (i.e., 10 countries) was specified, assuming invariant item intercepts ${\nu }_i$ and factor loadings ${\lambda }_i$ ($i=1,\dots ,4$) across countries. The residual variances were allowed to vary across countries. The CFA model was identified by fixing the factor mean of the first group to 0 and the factor variance in the first group to 1.

To compute standard errors with respect to the sampling of persons, nonparametric bootstrapping of persons was conducted. In total, $R=100$ bootstrap samples were drawn. We used the nonparametric bootstrap samples to obtain bias-corrected estimates of the variance components for the stochastic model for misspecification errors (see Section 3.2). Misspecification error in parameter estimates was determined by parametric bootstrapping (see Section 3.3) using 200 bootstrap samples. The total error for all parameter estimates that comprised standard error and misspecification error was computed using Equation (33).

Because jackknifing items were suggested to investigate the stability in parameter estimates due to changes in the model [40,46,47], we computed the misspecification error with a jackknife-based variability measure. The dataset included four items such that a jackknife sample of items included three items. An alternative misspecification error based on jackknife (JKME) was obtained by applying the jackknife standard error formula [32].

The obtained country means and country standard deviations of the factor variable were linearly transformed for all different estimators such that the total population comprising all persons from all 10 countries had a mean of 0 and a standard deviation of 1. Hence, the factor variable was standardized in the total population that comprised all 10 countries. The multiple-group SEM was estimated with ULS.

The analysis was conducted using the sirt:::mgsem() function from the R [48] package sirt [49]. The dataset used in this analysis can be found at https://osf.io/hj3k9/ (accessed on 9 May 2023).

5.2. Results

The variance ${\tau }_{\mathit{\mu }}$ for residuals in the mean structure was estimated as 0.0332 with the raw estimation method. The bias-corrected estimate used for subsequent statistical inference was slightly smaller, with 0.0328. The estimate of ${\tau }_{\mathbf{\sigma },2}$ was negative and set to 0. The raw estimate of ${\tau }_{\mathbf{\sigma },1}$ was 0.0050, while the bias-corrected estimate was 0.0042. From these results, it can be concluded that misspecification was more severe in the mean structure than in the covariance structure.

In

Table 1. Empirical example: estimated model parameters with their standard errors, misspecification errors, and total errors.

Par	Est	SE	ME	JKME	TE
${\nu }_1$	3.070	0.021	0.097	-	0.100
${\nu }_2$	2.698	0.017	0.094	-	0.095
${\nu }_3$	2.602	0.022	0.126	-	0.127
${\nu }_4$	2.678	0.019	0.101	-	0.103
${\lambda }_1$	0.591	0.021	0.029	-	0.036
${\lambda }_2$	0.567	0.021	0.029	-	0.036
${\lambda }_3$	0.685	0.024	0.032	-	0.040
${\lambda }_4$	0.556	0.018	0.029	-	0.034
${\alpha }_1$	0 ${}^{‡}$	-	-	-	-
${\alpha }_2$	0.062	0.046	0.226	0.329	0.231
${\alpha }_3$	0.186	0.061	0.187	0.154	0.197
${\alpha }_4$	0.326	0.052	0.205	0.339	0.212
${\alpha }_5$	−0.421	0.031	0.220	0.206	0.223
${\alpha }_6$	−0.109	0.055	0.214	0.188	0.221
${\alpha }_7$	−0.012	0.049	0.215	0.141	0.220
${\alpha }_8$	−0.504	0.042	0.239	0.236	0.243
${\alpha }_9$	0.232	0.041	0.206	0.174	0.210
${\alpha }_{10}$	−0.544	0.050	0.213	0.211	0.218
${\varphi }_1$	1 ${}^{‡}$	-	-	-	-
${\varphi }_2$	1.329	0.141	0.135	0.333	0.195
${\varphi }_3$	1.132	0.097	0.118	0.183	0.153
${\varphi }_4$	1.534	0.149	0.152	0.190	0.213
${\varphi }_5$	1.363	0.100	0.123	0.307	0.158
${\varphi }_6$	1.423	0.100	0.131	0.477	0.164
${\varphi }_7$	2.164	0.172	0.154	0.434	0.231
${\varphi }_8$	1.370	0.116	0.135	0.284	0.178
${\varphi }_9$	1.142	0.086	0.105	0.308	0.136
${\varphi }_{10}$	1.148	0.092	0.116	0.420	0.148

Note. Par = model parameter; Est = parameter estimate; SE = standard error; ME = misspecification error; JKME = misspecification error estimated by jackknifing items; TE = total error based on (33); ${\nu }_i$ = item intercept; ${\lambda }_i$ = factor loading ($i=1,\dots ,4$); ${\alpha }_g$ = factor mean; ${\varphi }_g$ = factor variance ($g=1,\dots ,10$); ${}^{‡}$ = factor mean ${\alpha }_1$ and factor variance ${\varphi }_1$ for first country were fixed to 0 and 1, respectively.

, parameter estimates and their standard errors, misspecification errors, and total errors are displayed. The misspecification errors (ME; estimated by parametric bootstrapping) were substantially larger than the standard error. The error ratio (ER) defined as the quotient of ME and SE was on average 4.69 ($SD=1.15$) and provided evidence that inferences regarding the factor means are much more affected by the choice of items than the sampling of persons. For factor means, the jackknife misspecification error (JKME) had an average of 0.220 and was very similar to the average of 0.214 of the ME values. Notably, the total error (TE) was mainly determined by the ME. Total errors were on average 480% larger than standard errors.

The situation was quite different for factor variances. The error ratio had an average of 1.14, indicating that sampling and modeling errors had a similar impact regarding the uncertainty in factor variances. Total errors were on average 52% larger than standard errors. Notably, the JKME ($M=0.326$) was much larger than the ME ($M=0.130$). We suppose that the standard error computation for jackknife does not reflect the stochastic model for residuals in covariances, which might explain this large difference.

In

Table 2. Empricial example: estimated factor means and factor variances after population standardization (i.e., mean of 0 and standard deviation of 1 in the total population) with their standard errors, misspecification errors, and total errors.

Country	Est	SE	ME	JKME	TE
	Factor Means
1	0.065	0.024	0.124	0.102	0.127
2	0.117	0.023	0.134	0.166	0.136
3	0.220	0.030	0.113	0.074	0.117
4	0.336	0.029	0.116	0.156	0.119
5	−0.285	0.025	0.106	0.215	0.109
6	−0.026	0.031	0.118	0.209	0.122
7	0.056	0.030	0.110	0.026	0.114
8	−0.354	0.025	0.127	0.135	0.130
9	0.258	0.022	0.135	0.136	0.137
10	−0.387	0.023	0.130	0.108	0.132
	Factor Variances
1	0.831	0.027	0.032	0.089	0.042
2	0.958	0.024	0.024	0.034	0.034
3	0.884	0.022	0.027	0.044	0.035
4	1.029	0.024	0.027	0.106	0.036
5	0.970	0.019	0.023	0.033	0.030
6	0.991	0.027	0.028	0.072	0.039
7	1.223	0.019	0.023	0.015	0.029
8	0.973	0.029	0.027	0.049	0.039
9	0.888	0.017	0.030	0.046	0.035
10	0.890	0.020	0.031	0.081	0.037

Note. Est = parameter estimate; SE = standard error; ME = misspecification error; JKME = misspecification error estimated by jackknifing items; TE = total error based on (33).

, factor means and factor variances and their error estimates are presented after standardizing the factor variable in the total population comprising all 10 countries. The ME estimates for factor means had an average of 0.121 ($SD=0.010$). While the estimates based on jackknifing items (i.e., JKME) had a similar average of 0.133, the variability across countries ($SD=0.058$) was much larger. This fact reflects the possibility that the extent of model misspecification error is allowed to vary across countries but is assumed as homogeneous across countries when estimating the ME. Moreover, note that the ME estimates for factor means in Table 2 when using the population standardization were much smaller than the factor means in the estimated model that used the first country as the reference (see Table 1). Setting scaling issues aside, this finding can be simply explained by the fact that is represented in the parameter estimate. When using population standardization, a country is compared with an average across countries. In the analysis model that uses a reference country, a respective country is compared with a reference country. In the latter approach, uncertainty within averages across countries is minor so that only the ME for one country is taken into account. However, in the former approach, the ME for a country reflects the misspecification of the corresponding country and the reference country because a comparison is conducted. Hence, the difference is in full alignment with what can be expected when using different identification constraints when estimating factor means and factor variances. In line with the results from Table 1, jackknife-based estimates of ME (i.e., JKME) were substantially larger than the ME estimates.

6. Discussion

In this article, we present a simultaneous statistical inference regarding sampling errors and model errors in single-group and multiple-group SEMs. Our framework closely follows that of Wu and Browne [9] but differs in the fact that we use the same estimation function (e.g., maximum likelihood or diagonally weighted least squares).

The procedure can be summarized as follows. Let $\widehat{\mathit{\xi }}$ contain estimated (group) means and (group) covariances. An SEM estimates a parameter $\widehat{\mathit{\theta }}$ that summarizes the mean and covariance structure. Thus, we expect that the model-implied means and covariances approximate or predict the observed means and covariances somehow. We can write $\widehat{\mathit{\xi }}\approx \mathit{\xi }\left(\widehat{\mathit{\theta }}\right)$. In samples, $\widehat{\mathit{\theta }}$ is an estimate of the population means and covariances $\mathit{\xi }$, and we can write $\widehat{\mathit{\xi }}=\mathit{\xi }+\mathit{\epsilon }$ with a sampling error $\mathit{\epsilon }$. Typically, the SEM will be misspecified. In other words, there exists some parameter ${\mathit{\theta }}_0$ that fits the population means and covariances best with respect to a chosen fitting function F. Model specification error exists if there exists a vector of residuals $\mathit{e}$ such that $\mathit{\xi }=\mathit{\xi }\left({\mathit{\theta }}_0\right)+\mathit{e}$. The vector $\mathit{e}$ is also referred to as a model error. Hence, we observe that there simultaneously appear sampling errors $\mathit{\epsilon }$ and model errors $\mathit{e}$, and the estimated means and covariances are represented as $\widehat{\mathit{\xi }}=\mathit{\xi }\left({\mathit{\theta }}_0\right)+\mathit{e}+\mathit{\epsilon }$. In this paper, a stochastic model is imposed on model errors $\mathit{e}$ that allows statistical inference for $\widehat{\mathit{\theta }}$, which is a function of the parameter ${\mathit{\theta }}_0$ and the two errors $\mathit{e}$ and $\mathit{\epsilon }$. Ordinary statistical inference only reflects sampling error $\mathit{\epsilon }$ in standard errors in parameter estimates, while the proposed method additionally includes model errors in the standard errors.

Although the illustrations in Section 4 and the empirical example in Section 5 utilized ULS estimation, the derivations apply to any differentiable fitting function for SEM, such as the ML fitting function. In our stochastic model for the modeling of misspecification, we assume that there is no residual error in the diagonal matrix of residuals. Specifically, at the population level, the model-implied variances and the total variances of all items coincide. This is likely fulfilled with ULS estimation if residual variances have group-specific estimates resulting in zero residuals. However, even if the residual variances are group-specific, residuals for variances are typically different from zero in ML estimation. Hence, we suppose that the stochastic model for misspecification must be slightly adapted.

In this article, analytical illustrations and a numerical example are provided. In future studies, it would be interesting to investigate the performance of our approach in simulation studies. Nevertheless, we believe that the proposed method has clear asymptotic foundations. We suspect that the number of items by the number of groups is critical for the reliable estimation of the variance components of the stochastic model for model misspecification.

Because our approach only relies on modeling misspecification in the mean and the covariance structure, it can be directly applied to SEMs with ordinal data by substituting the mean vector with a vector of thresholds and the covariance matrix with a polychoric correlation matrix. In this case, a closer correspondence to linking errors that are mainly discussed in item response models can be investigated.

We would like to note that the simultaneous assessment of sampling errors and model specification errors has similarities to generalizability theory [50,51,52], domain sampling theory [53,54,55], or linking errors [56,57]. Notably, resampling techniques regarding items could also approximate statistical inference with respect to model misspecification [57].

In our approach, we opt for a particular stochastic model to model specification errors. There is always ambiguity in choosing such a stochastic model. For example, model errors in the same item or same item pair might be correlated across groups. Such an extension can also be addressed in our estimation approach with slight changes in the variance formula.

Our derivations show that the misspecification error reduces if the number of items increases. This result is a consequence of assuming independently distributed model errors across items. Sometimes, it might be more plausible to use a two-level model to model misspecification in the mean structure such that items are nested within item groups (or item clusters). In this case, model residuals in the mean structure might be positively correlated within an item group.

In general, one could argue that the stochastic model for model misspecification is of no relevance because it does not refer to an actual sampling model of items or the effects of model discrepancies in data generation (or data collection). We do not believe that this would be a viable objection. A statistical model is always a model in which an investigator defines randomness by the means of random variables, which must not have any connection to a sampling procedure. The model residuals are merely modeled by a random variable and the variability is quantified by a single variance for the mean structure and two variances for the covariance structure. Independence assumptions of residuals should be compared with random sampling assumptions across persons. In a concrete sample, there is no test to determine whether the independence assumption across persons is fulfilled. It is simply a definition that can be useful in applications for statistical inference. The same holds true for model residuals. They are simply assumed to be independent according to a stochastic model. This assumption cannot (be fully) tested.

If model misspecification is present, one can speculate as to whether ML estimation should be the preferred estimation method. ML achieves the most efficient parameter estimation if the analysis model is correctly specified. However, ML can produce more variable estimates for DWLS or ULS estimation in the presence of model misspecification. Hence, choosing between ML and DWLS (or ULS) is a decision regarding whether input data should be more reflected regarding sampling errors (i.e., preferring ML) or model residuals (i.e., preferring DWLS or ULS). We tend to prefer DWLS in many, if not almost all, applications because correct model specification is generally not guaranteed.

There can always be arguments that researchers should not interpret parameter estimates from misspecified models [58]. However, we disagree with such a view. With a data-driven modification of a target model of interest, the meaning of the primary model parameters changes. Hence, researchers implicitly change the meaning of the latent variables and their relationships in an SEM. We do not see why statistics (or psychometrics as a special branch of it) should redefine the target estimand of interest in a data-driven way. In contrast, researchers intentionally use models because they describe some phenomenon of interest. In our view, model misspecification should not lead to model modification but should be reflected as a type of error that can be reported. We believe that including model misspecification as an increase in standard errors in parameter estimates is a viable concept to quantify model errors. We hope that it can be applied to standard research practices that utilize SEMs.