Misspecification in Generalized Linear Mixed Models and Its Impact on the Statistical Wald Test

Featured Application

This manuscript is part of ongoing research on the strengths and limitations of Wald statistical tests taking into account the incorrect specification of distribution of random effects in generalized linear mixed models.

Abstract

Generalized linear mixed models are commonly used in repeated measurement studies and account for the dependence between observations obtained from the same experimental unit. The designs of repeated measurements in which each experimental unit (e.g., subject) is proven in more than one experimental condition are widespread in psychology, neuroscience, medicine, social sciences and agricultural research. Estimation in generalized linear mixed models is often based on the maximum likelihood theory, which assumes that the assumptions about the underlying probability model are correct. These assumptions include the specification of the distribution of random effects. This research study aimed to identify the impact of the incorrect specification of this distribution on the probability of a type I error and the statistical power of the Wald test. This was achieved through a simulation study where different distributions were considered for random effects in generalized linear mixed models with Poisson and negative binomial responses. Evidence of the impact of the incorrect specification was presented in distributions for random effects different from the normal ones. Lognormal was used for random intercepts and bivariate exponential and Tukey for random intercepts and slopes. Lognormal has positive asymmetry and high kurtosis. Exponential has moderate asymmetry and kurtosis, and Tukey has moderate asymmetry and high kurtosis.

1. Introduction

Self-regressive correlation structures and missing data do not favour the use of the ordinary linear model ANOVA for repeated measurements. Two procedures proposed to address these problems involve the use of generalized linear mixed models (GLMMs) or solving generalized estimating equations (GEEs) [1].

GLMMs expand ordinary regression by allowing non-normal responses and including random effects [2]. An example of this is found in [3], where a GLMM was considered to analyze count data, assuming a Poisson response variable. Furthermore, in GLMMs, it is common to assume a parametric distribution for random effects (e.g., normal, gamma), generally for computational or conventional considerations [4]. However, there is often little information about the form of the joint distribution of random effects. Thus, distribution for these unobserved random effects cannot be directly evaluated [5]. For this reason, a genuine concern in the use of GLMMs is the incorrect specification of the model for random effects [6].

Since it is impossible to observe random effects (latent amounts), the distribution assumption for them cannot be directly evaluated and, according to [7], there is not a consensus on the impact of the random-effects distribution being incorrectly specified. Some authors have addressed the impact of incorrect specification of random-effects distribution in GLMMs [4,8,9,10,11]. Other types of incorrect specification are also possible, including incorrect specification of the link function, or not considering the presence of overdispersion, among others.

Many authors have voiced concern about tests to detect incorrect specifications. For example, ref. [6] proposed a two-stage diagnostic method to detect incorrect specifications of the random-effects model in a GLMM; ref. [12] recommended two diagnostic tests based on the information matrix of the model; ref. [13] stated that, in general, if the model selection for random effects in a GLMM is incorrectly specified, then parameter estimation and inference procedures such as the Wald test may also be affected. Therefore, assessing the results is always recommended.

Wald Z, χ² and t and F tests for GLMMs evaluate a null non-effect hypothesis by adjusting the parameter estimates or parameter combinations by their estimated standard errors and comparing the resulting test statistic with the zero [14]. Several studies refer to the use of the Wald test for the assessment of statistical significance. For example, [15,16,17,18] used such a test in the context of modelling non-linear mixed effects and evaluating their type I errors and statistical power.

In basic research, analysts often emphasize avoiding type I errors more than avoiding type II errors [19]. On the other hand, the statistical power analysis (i.e., the probability that a test will reject the null hypothesis when the null hypothesis is false, or alternatively, 1 − P(type II error)), has gained wide acceptance among scientists over the last thirty years. The number of searches on the Thomson Reuters science website for ‘sampl * and power analysis’ increased from 115 (1996–2000) to 214 (2001–2005 period) and 265 (2006–2010) [20].

Several studies have assessed type I errors and statistical power related to the inference of the fixed parameters. For example, ref. [21] extensively studied how the incorrect specification of the size of the model’s cluster affects inference in joint modelling, using the Wald test and type I error, as well as the power associated with it. Refs. [4,22] addressed the impact of incorrect specification of random-effects distribution on type I errors and the power of the Wald test for the mean structure in GLMMs.

Wald Z and χ² tests are only suitable for GLMMs without overdispersion, while Wald t and F tests account for uncertainty in overdispersion estimations. This uncertainty depends on the number of degrees of freedom of the residual, which can be very difficult to calculate because the actual number of parameters used by a random effect ranges between 1 (i.e., a single standard deviation parameter) and N − 1 (i.e., a parameter for each additional level of random effect) [14]. Although a comprehensive performance evaluation for small samples of the asymptotic Wald-type test aimed at evaluating fixed effects on the mixed model has not been reported, there is evidence indicating that a normal or chi-square approximation is unreliable [23].

One strategy that has been suggested for the improvement of Wald-type tests involves the replacement of the asymptotic approximation based on normal and chi-square distributions with approximations based on t and F distributions. Several effective methods have been proposed to define the degrees of freedom in the denominator used in the t and F approximations [23]. The degrees of freedom for random effects, required for Wald t or F tests, must range between 1 and N − 1 (where N is the number of levels of random effects). Software packages vary significantly in their approach to calculating degrees of freedom. The most straightforward approach (the default value in SAS) uses the minimum number of degrees of freedom provided by the random effects that affect the term that is being tested. Satterthwaite and Kenward–Roger (KR) approximations use more complicated rules to approach the degrees of freedom and adjust standard errors. KR, only available in SAS, generally works best (at least for mixed linear models). The Satterthwaite approximation is available in the PROC MIXED of SAS [14].

Another approach is to use the test based on the Wald-type test with the sandwich method. The sandwich procedure for estimating the covariance matrix is valid even if the model is specified incorrectly, provided that the structure of the mean of ${\mathrm{y}}_{\mathrm{i}}$ has been specified correctly. This estimation procedure is therefore referred to as robust estimation [23].

An alternative to the Wald test for fixed effects is the likelihood ratio test (LRT). The likelihood ratio test compares the logarithm of the likelihood of two models where one is nested into the other [24]. The Wald test is less computationally intensive since it can be performed on any analysis without the need to repeat the analysis for the null model. It may bring advantages for some strategies of covariance model building [15]. Ref. [23] proposed a Bartlett-type correction for the likelihood ratio test. The essence of the Bartlett correction is to increase the LRT statistic by a scale factor, resulting in a statistic that has moments closest to a χ². Additionally, ref. [25] developed modified versions of the likelihood ratio test to infer fixed effects in linear mixed models. Namely, they derived a Bartlett correction into a test of this kind and a test based on a modified profile likelihood function.

According to [26], Wald tests and likelihood ratio (LR) tests, along with Score (another test for evaluating hypotheses), are known for having incorrect type I errors; for this reason, they considered an extension of the results that Rothenberg (1984) achieved for general consistent estimators and tried to derive their own Bartlett-type corrections. In their research, they studied the behaviour of type I errors and the power of the three classical tests mentioned above, as well as that of the adjusted tests with Bartlett-type corrections through a Monte Carlo simulation.

The simulation study presented here allowed us to focus on the possibilities and limitations of Wald tests applied to GLMMs. This manuscript is part of ongoing research on the strengths and limitations of Wald statistical tests taking into account the incorrect specification of distribution of random effects in these models.

1.1. Generalized Linear Mixed Models

When dealing with non-Gaussian data with multiple sources of variation, a commonly used subject-specific model is the generalized linear mixed model [10]. Let ${\mathrm{y}}_{\mathrm{ij}}$ be the response of the ith subject at the point of time j. Conditional to a vector of random individual effects ${\mathrm{b}}_{\mathrm{i}}$, all the response variables ${\mathrm{y}}_{\mathrm{ij}}$ are independent and have density functions belonging to the exponential family:

$$\mathrm{f}\left({\mathrm{y}}_{\mathrm{ij}};{\mathsf{\theta }}_{\mathrm{ij}},\mathsf{\varphi }\right)=\exp \left[{\mathsf{\varphi }}^{-1}\left\{{\mathrm{y}}_{\mathrm{ij}}{\mathsf{\theta }}_{\mathrm{ij}}-\mathsf{\psi }\left({\mathsf{\theta }}_{\mathrm{ij}}\right)\right\}+\mathrm{c}\left({\mathrm{y}}_{\mathrm{ij}},\mathsf{\varphi }\right)\right]$$

(1)

where ${\mathsf{\theta }}_{\mathrm{ij}}=\mathsf{\eta }\left({{\mathrm{x}}^{\prime }}_{\mathrm{ij}}\mathsf{\beta }+{{\mathrm{z}}^{\prime }}_{\mathrm{ij}}{\mathrm{b}}_{\mathrm{i}}\right)$, $\mathsf{\eta }\left(·\right)$ is a known inverse link function, ${\mathsf{\beta }}_0$ is an intercept, ${\mathrm{x}}_{\mathrm{ij}}$ and ${\mathrm{z}}_{\mathrm{ij}}$ are covariate vectors, $\mathsf{\beta }$ is a vector of fixed unknown regression coefficients, $\mathsf{\varphi }$ is a scale parameter, and $\mathrm{c}\left(·\right)$ is a function that depends only on ${\mathrm{y}}_{\mathrm{ij}}$ and $\mathsf{\varphi }$. In addition, $\mathsf{\psi }\left(·\right)$ is a function that satisfies $\mathrm{E}\left({\mathrm{y}}_{\mathrm{ij}}\right)={\mathsf{\psi }}^{\prime }\left({\mathsf{\theta }}_{\mathrm{ij}}\right)$ and $\mathrm{Var}\left({\mathrm{y}}_{\mathrm{ij}}\right)=\mathsf{\varphi }{\mathsf{\psi }}^{″}\left({\mathsf{\theta }}_{\mathrm{ij}}\right)$. Subject-specific effects ${\mathrm{b}}_{\mathrm{i}}$ are generally assumed to be normally distributed with mean 0 and a variance–covariance matrix $\mathrm{D}$ [4].

The probability distributions that belong to the exponential family include normal distributions, gamma, binomial, Poisson and negative binomial, among others. The last two are used for count data and are the two distributions that are used in this article to conduct the simulation study. The Poisson distribution (2) is the cornerstone for modelling these data [27]. This is a parameter model and its conditional mean is supposed to be equal to the conditional variance [28].

$$\mathrm{f}\left(\left.\mathrm{y}\right|\mathsf{\lambda }\right)=\frac{{\mathrm{e}}^{-\mathsf{\lambda }}{\mathsf{\lambda }}^{\mathrm{y}}}{\mathrm{y}!}, \mathrm{y}=0,1,2,\dots$$

(2)

where $\mathrm{E}\left(\mathrm{Y}\right)=\mathsf{\lambda }$ and $\mathrm{Var}\left(\mathrm{Y}\right)=\mathsf{\lambda }$, with $\mathsf{\lambda }>0$.

In many real-life applications, the simple Poisson model cannot describe the data, because, generally speaking, the variance of the sample is higher than the mean obtained by the simple Poisson model [23]. For count data with extra Poisson variation (count data with overdispersion and extra zeros), several modifications or extensions to the Poisson model have been proposed, such as the negative binomial distribution (NB) (3).

$$\mathrm{f}\left(\mathrm{y}|\mathsf{\mu },\mathsf{\alpha }\right)=\left(\begin{array}{c}\mathrm{y}+\frac{1}{\mathsf{\alpha }}-1\\ \frac{1}{\mathsf{\alpha }}-1\end{array}\right){\left(\frac{1}{1+\mathsf{\alpha }\mathsf{\mu }}\right)}^{\frac{1}{\mathsf{\alpha }}}{\left(\frac{\mathsf{\alpha }\mathsf{\mu }}{1+\mathsf{\alpha }\mathsf{\mu }}\right)}^{\mathrm{y}}\mathrm{y}=0,1,2,\dots$$

(3)

where $\mathrm{E}\left(\mathrm{Y}\right)=\mathsf{\mu }$ and $\mathrm{Var}\left(\mathrm{Y}\right)=\mathsf{\mu }+{\mathsf{\alpha }\mathsf{\mu }}^2$, with $\mathsf{\mu }>0$ and $\mathsf{\alpha }>0$.

NB generalizes the Poisson model through the relaxation of the Poisson assumption of equidispersion (allowing the variance to exceed the mean). This is achieved through the addition of another source of variability, the dispersion parameter, k, beyond which the Poisson process is derived. The added dispersion parameter allows the variance to exceed the mean, and therefore, it allows the NB distribution to account for the overdispersion. The dispersion is expressed as parameter k in some formulations. When this dispersion parameter becomes zero, the NB model is reduced to the Poisson model [24].

1.2. Wald Test

In many situations, data analysts use statistical tests to assess whether or not a drug has a significant influence. Despite the fact that consistency has been studied in the literature to some extent, it seems that there is not a lot of research on the behaviour of test statistics [10]. Therefore, one of the aims of this research consisted in studying the impact of the incorrect specification of the distribution of random effects on the type I error and on the power of the statistical Wald test in generalized linear mixed models, with Poisson and negative binomial responses and a random intercept and a random intercept and slope.

The statistical Wald test, which is used to prove a hypothesis of type ${\mathrm{H}}_0:\mathsf{\beta }=0$ vs. ${\mathrm{H}}_1:\mathsf{\beta }\ne 0$, is established as follows [20]:

$$\mathrm{Z}=\frac{\widehat{\mathsf{\beta }}-\mathsf{\beta }}{\widehat{\mathrm{SE}}\left(\widehat{\mathsf{\beta }}\right)}$$

(4)

where $\widehat{\mathsf{\beta }}$ is the maximum likelihood estimation of the parameter, β is the value of the real parameter, and $\widehat{\mathrm{SE}}$ is the standard error of $\widehat{\mathsf{\beta }}$. The statistical Wald test has an approximate standard normal distribution [25].

2. Materials and Methods

A simulation study was conducted to identify the impact of incorrectly specifying the distribution of random effects on type I errors and the power in the Wald test for generalized linear mixed models. In the first part, mixed generalized linear models were considered with a random intercept and Poisson and NB response variables. In the second part, mixed generalized linear models were considered with a random intercept and slope and Poisson and NB response variables.

2.1. Poisson and Negative Binomial Models with Random Intercept

For the simulation study of generalized linear mixed models with a Poisson or NB response and random intercept, the same structure featured in the article by [26] was used, which focused on studying the impact on the probability of a type I error when ignoring overdispersion in longitudinal configurations, for which the authors generated Poisson and NB responses with the mean ${\mathsf{\mu }}_{\mathrm{ij}}=\exp \left({\mathsf{\beta }}_0+{\mathrm{b}}_{\mathrm{i}}+{\mathsf{\beta }}_1{\mathrm{t}}_{\mathrm{ij}}+{\mathsf{\beta }}_2{\mathrm{z}}_{\mathrm{i}}+{\mathsf{\beta }}_3{\mathrm{t}}_{\mathrm{ij}}{\mathrm{z}}_{\mathrm{i}}\right)$ where $\mathrm{i}=1,2,\dots ,\mathrm{m}$ and ${\mathrm{t}}_{\mathrm{ij}}=1,2,\dots ,{\mathrm{n}}_{\mathrm{i}}$ denoting the subject and the time of measurement, respectively. In addition, ${\mathrm{b}}_{\mathrm{i}}~\mathrm{N}\left(0,{\mathsf{\sigma }}_{\mathrm{b}}^2\right)$, and ${\mathrm{z}}_{\mathrm{i}}$ is an indicator variable of group treatment taking values of 0/1.

Equation (5) shows the model that was used for the Poisson, and Equation (6) shows the one used for NB:

$$\begin{gathered} {\mathrm{y}}_{\mathrm{ij}}|{\mathrm{b}}_{\mathrm{i} }\widetilde{\mathrm{ind}. } \mathrm{Poisson} \\ \log \left({\mathsf{\mu }}_{\mathrm{ij}}\right)={\mathsf{\beta }}_0+{\mathrm{b}}_{\mathrm{i}}+{\mathsf{\beta }}_1{\mathrm{t}}_{\mathrm{ij}}+{\mathsf{\beta }}_2{\mathrm{z}}_{\mathrm{i}}+{\mathsf{\beta }}_3{\mathrm{t}}_{\mathrm{ij}}{\mathrm{z}}_{\mathrm{i}} \\ {\mathrm{b}}_{\mathrm{i} }\widetilde{\mathrm{ind}. }{ \mathrm{G}}_{\mathrm{T}} \end{gathered}$$

(5)

$$\begin{gathered} {\mathrm{y}}_{\mathrm{ij}}|{\mathrm{b}}_{\mathrm{i} }\widetilde{\mathrm{ind}. } \mathrm{BN} {\mathsf{\mu }}_{\mathrm{ij}},\mathsf{\alpha }=0.5 \\ \log \left({\mathsf{\mu }}_{\mathrm{ij}}\right)={\mathsf{\beta }}_0+{\mathrm{b}}_{\mathrm{i}}+{\mathsf{\beta }}_1{\mathrm{t}}_{\mathrm{ij}}+{\mathsf{\beta }}_2{\mathrm{z}}_{\mathrm{i}}+{\mathsf{\beta }}_3{\mathrm{t}}_{\mathrm{ij}}{\mathrm{z}}_{\mathrm{i}} \\ {\mathrm{b}}_{\mathrm{i} }\widetilde{\mathrm{ind}. }{ \mathrm{G}}_{\mathrm{T}} \end{gathered}$$

(6)

The ${\mathrm{b}}_{\mathrm{i}}$ were generated based on 4 different distributions ${\mathrm{G}}_{\mathrm{T}}$, normal, mixture of 2 normal distributions, uniform and lognormal [4,27,28], with media 0 and 4 variance values: ${\mathsf{\sigma }}_{\mathrm{b}}^2$= 1, 2, 4 and 8 (Figure 1).

To assess the impact of incorrectly specifying the distribution of the random intercept on the type I error and the power of the Wald test, ${\mathsf{\beta }}_2$ was used as the parameter of interest. This parameter assumed the values of ${\mathsf{\beta }}_2=0, 0.5, 1.0, 1.5$ for the Poisson response case and values of ${\mathsf{\beta }}_2=0, 1.5, 2.0, 2.5$ for the NB case (

Table 1. Hypothesis of interest and ${\mathsf{\beta }}_2$ values used for the simulations of a GLMM with random intercept.

Hypothesis	Poisson	BN
${\mathbf{H}}_0:{\mathsf{\beta }}_2=0$ vs. ${\mathbf{H}}_1:{\mathsf{\beta }}_2\ne 0$	${\mathsf{\beta }}_2=0,0.5,1,1.5$	${\mathsf{\beta }}_2=0,1.5,2,2.5$

Source: authors.

). For the other parameters, the values used in the simulation were fixed as ${\mathsf{\beta }}_0=-2,{ \mathsf{\beta }}_1=-0.5$ and ${\mathsf{\beta }}_3=1.0$. In practice, small samples for cluster are often available; for this reason, in this study four different sample sizes, ${\mathrm{n}}_{\mathrm{i}}=5, 10, 15, 20$, were considered. The number of conglomerates was fixed at $\mathrm{m}=100$ conglomerates.

For each scenario given by the combination of ${\mathrm{n}}_{\mathrm{i}}$, ${\mathsf{\beta }}_2$, and the true distribution of ${\mathrm{b}}_{\mathrm{i}}$ and ${\mathsf{\sigma }}_{\mathrm{b}}^2$, 1000 sets of data were simulated with the structure (5) and 1000 with the structure (6) (e.g., 1000 sets of data with ${\mathrm{n}}_{\mathrm{i}}$= 5, ${\mathsf{\beta }}_2$= 1.5, true distribution = uniform, ${\mathsf{\sigma }}_{\mathrm{b}}^2$= 2 and structure (5)), and subsequently, the GLMM was adjusted with the function glmer for the Poisson case and with the function glmer.nb for the NB case. These functions can be found in the R package lme4. The ratio of times that an effect of the parameter of interest other than zero could be detected was calculated, i.e., the times that ${\mathrm{H}}_0:{\mathsf{\beta }}_2=0$ was rejected with a significance level of 5%. When there is no parameter effect (${\mathsf{\beta }}_2=0$), this ratio corresponds to a type I error, and for other values of ${\mathsf{\beta }}_2$, this ratio represents the power of the test being studied [22].

2.2. Poisson and Negative Binomial Models with Random Intercept and Slope

A simulation study was conducted to evaluate the impact of the incorrect specification of the distribution of random effects for the case of generalized linear mixed models with the Poisson response and a random intercept and slope. The following structure, taken from [29], was considered:

$$\begin{gathered} {\mathrm{y}}_{\mathrm{ij}}|{\mathrm{b}}_{\mathrm{i} }\widetilde{\mathrm{ind}. } \mathrm{Poisson} \left({\mathsf{\mu }}_{\mathrm{ij}}\right), \\ \log \left({\mathsf{\mu }}_{\mathrm{ij}}\right)={\mathsf{\beta }}_0+{\mathrm{b}}_{0\mathrm{i}}+({\mathsf{\beta }}_1+{\mathrm{b}}_{1\mathrm{i}}){\mathrm{x}}_{1\mathrm{ij}}+{\mathsf{\beta }}_2{\mathrm{x}}_{2\mathrm{ij}}+{\mathsf{\beta }}_3{\mathrm{x}}_{3\mathrm{i}} \\ {\mathrm{b}}_{\mathrm{i} }\widetilde{\mathrm{ind}. }{ \mathrm{G}}_{\mathrm{T}} \end{gathered}$$

(7)

where $\mathrm{i}=1,2,\dots ,\mathrm{m}, \mathrm{j}=1,2,\dots ,{\mathrm{n}}_{\mathrm{i}}$.

Based on the structure proposed by [29], NB response variables were also simulated with the following structure:

$$\begin{gathered} {\mathrm{y}}_{\mathrm{ij}}|{\mathrm{b}}_{\mathrm{i} }\widetilde{\mathrm{ind}. } \mathrm{BN} ({\mathsf{\mu }}_{\mathrm{ij}},\mathsf{\alpha }=0.5), \\ \log \left({\mathsf{\mu }}_{\mathrm{ij}}\right)={\mathsf{\beta }}_0+{\mathrm{b}}_{0\mathrm{i}}+({\mathsf{\beta }}_1+{\mathrm{b}}_{1\mathrm{i}}){\mathrm{x}}_{1\mathrm{ij}}+{\mathsf{\beta }}_2{\mathrm{x}}_{2\mathrm{ij}}+{\mathsf{\beta }}_3{\mathrm{x}}_{3\mathrm{i}} \\ {\mathrm{b}}_{\mathrm{i} }\widetilde{\mathrm{ind}. }{ \mathrm{G}}_{\mathrm{T}} \end{gathered}$$

(8)

with ${\mathrm{x}}_{1\mathrm{ij}}$, which has equally spaced values between −1 and 1; ${\mathrm{x}}_{2\mathrm{ij}}$, a covariate within conglomerates with values ${\mathrm{x}}_{2\mathrm{i}}={\left(-0.5, 1.0, 0,-1.0, 0.5\right)}^{\mathrm{T}}$; and a binary covariate among conglomerates, ${\mathrm{x}}_{3\mathrm{i}}$, which was established to take the value of 0 for half of the conglomerates and 1 for the rest. The three covariates are mutually orthogonal [29]. In addition, four sample sizes, ${\mathrm{n}}_{\mathrm{i}}=5, 10, 15, 20$, were considered, as well as $\mathrm{m}=100$ conglomerates.

The random intercept and slope, both for the Poisson and for the NB ${\mathrm{b}}_{\mathrm{i}}={\left({\mathrm{b}}_{0\mathrm{i}},{\mathrm{b}}_{1\mathrm{i}}\right)}^{\mathrm{T}}$, were generated based on 4 different distributions ${\mathrm{G}}_{\mathrm{T}}$ with ${\mathsf{\mu }}_{{\mathrm{b}}_{0\mathrm{i}}}={\mathsf{\mu }}_{{\mathrm{b}}_{1\mathrm{i}}}=0$, 4 variance values, ${\mathsf{\sigma }}_{{\mathrm{b}}_{0\mathrm{i}}}^2={ \mathsf{\sigma }}_{{\mathrm{b}}_{1\mathrm{i}}}^2=1,2,4,8$, and a correlation of 0.5 between the intercept and the random slope:

• ${\mathrm{b}}_{\mathrm{i}}~$bivariate normal.
• ${\mathrm{b}}_{\mathrm{i}}~$bivariate t with 3 degrees of freedom.
• ${\mathrm{b}}_{\mathrm{i}}~$exponential.
• ${\mathrm{b}}_{\mathrm{i}}~$bivariate Tukey $\mathrm{g}-\mathrm{h}$, where parameter $\mathrm{g}$ controls the amount and direction of asymmetry, while parameter $\mathrm{h}$ controls the amount of elongation (kurtosis) of the bivariate Tukey distribution [30].

Figure 2 shows the contours for the four distributions used to simulate the random intercept and slope.

A total of 1000 repetitions were generated for each combination of ${\mathrm{n}}_{\mathrm{i}}$, ${\mathsf{\beta }}_{\mathrm{k}}$ with $\mathrm{k}=1,2$, the true distribution of ${\mathrm{b}}_{\mathrm{i}}$, ${\mathsf{\sigma }}_{{\mathrm{b}}_{0\mathrm{i}}}^2$ and ${\mathsf{\sigma }}_{{\mathrm{b}}_{1\mathrm{i}}}^2$ with structures (7) and (8) adjusting the model with the function glmer for the Poisson case and glmer.nb for the NB case, both of which were incorporated into the R package lme4.

As for generalized linear mixed models with a random intercept, the interest was focused on determining the ratio of times that an effect of the parameter of interest was detected; in this case, there are two: ${\mathsf{\beta }}_1$ and ${\mathsf{\beta }}_2$; i.e., where ${\mathrm{H}}_0:{\mathsf{\beta }}_{\mathrm{k}}=0$ was rejected with a significance level of 5%, with this ratio corresponding to a type I error if ${\mathsf{\beta }}_{\mathrm{k}}=0$ and for other values of the parameter, this ratio represents the power of the analysis test [22], with $\mathrm{k}=1,2$. The values used for the parameters ${\mathsf{\beta }}_1$ and ${\mathsf{\beta }}_2$ are shown in

Table 2. Hypothesis of interest and values of the parameters used in the simulations of a GLMM with random intercept and slope.

Cases	Hypothesis	Poisson	BN
Case 1	${\mathrm{H}}_0:{\mathsf{\beta }}_1=0$ vs. ${\mathrm{H}}_1:{\mathsf{\beta }}_1\ne 0$	${\mathsf{\beta }}_2$ fixed at 1.0 ${\mathsf{\beta }}_1=0, 0.3, 0.6, 0.9$	${\mathsf{\beta }}_2$ fixed at 1.0 ${\mathsf{\beta }}_1=0, 0.7, 1.4, 2.1$
Case 2	${\mathrm{H}}_0:{\mathsf{\beta }}_2=0$ vs. ${\mathrm{H}}_1:{\mathsf{\beta }}_2\ne 0$	${\mathsf{\beta }}_1$ fixed at 1.0 ${\mathsf{\beta }}_2=0, 0.04, 0.08, 0.12$	${\mathsf{\beta }}_1$ fixed at 1.0 ${\mathsf{\beta }}_2=0, 0.2, 0.4, 0.6$

Source: authors.

. For the other parameters, the values of ${\mathsf{\beta }}_0=-2.5$ and ${\mathsf{\beta }}_3=1.0$ were used.

3. Results

We studied the impact of the incorrect specification of the distribution of random effects on type I errors and the power of the Wald test in generalized linear mixed models with a random intercept and with a random intercept and slope. Below are the results for the first case.

3.1. Models with Random Intercept

Figure 3 and Figure 4 show the results for type I errors and the power of the statistical Wald test when testing ${\mathrm{H}}_0:{\mathsf{\beta }}_2=0$ vs. ${\mathrm{H}}_1:{\mathsf{\beta }}_2\ne 0$ in generalized linear mixed models with Poisson and NB response variables, respectively.

In the first column of Figure 3, we can observe the rejection rates for the null hypothesis ${\mathrm{H}}_0:{\mathsf{\beta }}_2=0$ when in fact, ${\mathsf{\beta }}_2=0$. In this case, we expect rejection rates around the significance level of 5%. For the random intercept with lognormal distribution (dash–dot line), we note that rejection rates are the greatest, thus indicating an impact of the incorrect specification of the distribution of the random intercept.

Columns 2, 3 and 4 from Figure 3 correspond to the rejection rates of the hypothesis ${\mathrm{H}}_0:{\mathsf{\beta }}_2=0$ for ${\mathsf{\beta }}_2=0.5, 1.0, 1.5$, respectively. Here, it is expected that the rejection rates go up to 100% as both the value of ${\mathsf{\beta }}_2$ and the value of ${\mathrm{n}}_{\mathrm{i}}$ increase. The values of 0.5 and 1.5 for the parameter ${\mathsf{\beta }}_2$ in the simulation study were selected intentionally because with them we obtained rejection curves that show a gradual increase. From Figure 3, it can be seen that for all cases, except when the true distribution is lognormal, the highest rejection rates correspond to the normal distribution. Moreover, for the uniform distribution, represented by the dotted line, there are rates below the normal distribution, which also indicates an impact of the incorrect specification of the distribution of the random intercept.

Figure 4 exhibits a behaviour similar to that found in Figure 3, as it displays an impact of incorrectly specifying the distribution of the random intercept by assuming it is normal when it actually comes from a lognormal distribution. Similar to the results for a Poisson GLMM, the impact is greater as the variance values increase (higher rejection rates appear, i.e., greater probability of type I error).

3.2. Models with Random Intercept and Slope

The results of type I errors and the power for the statistical Wald test for the hypothesis tests set out in Table 2 in the case of models with Poisson or NB response and random intercept and slope are presented in Figure 5, Figure 6, Figure 7 and Figure 8.

Figure 5 shows the type I error and the power of the Wald test for the hypothesis test ${\mathrm{H}}_0:{\mathsf{\beta }}_1=0$ vs. ${\mathrm{H}}_1:{\mathsf{\beta }}_1\ne 0$ in a Poisson GLMM, for which four values, ${\mathsf{\beta }}_1=0, 0.3, 0.6, 0.9$, and four different sample sizes, ${\mathrm{n}}_{\mathrm{i}}=5, 10, 15, 20$, were considered. The lowest rates of rejection of the null hypothesis (this being true), which are shown in Column 1, correspond to the normal distribution, as was expected. For ${\mathsf{\sigma }}_{{\mathrm{b}}_{0\mathrm{i}}}^2={ \mathsf{\sigma }}_{{\mathrm{b}}_{1\mathrm{i}}}^2=$ 2 and 8, the highest rates correspond to the bivariate Tukey distribution (dash–dot line), showing an impact of incorrectly specifying the distribution of random effects, as they have been assumed to be normal instead of Tukey. In terms of the rejection rates when ${\mathsf{\beta }}_1\ne 0$, represented in Columns 2, 3 and 4, it is also possible to see an impact of this specification, as rates lower than the normal distribution were obtained for the bivariate exponential distribution (dotted line).

The results of the type I error and the power of the Wald test when proving hypothesis ${\mathrm{H}}_0:{\mathsf{\beta }}_2=0$ vs. ${\mathrm{H}}_1:{\mathsf{\beta }}_2\ne 0$ in a Poisson GLMM with four different values for ${\mathsf{\beta }}_2=0, 0.04, 0.08, 0.12$ and four sizes of sample ${\mathrm{n}}_{\mathrm{i}}=5, 10, 15, 20$ are presented in Figure 6. Opposite to what was found when proving hypothesis ${\mathrm{H}}_0:{\mathsf{\beta }}_1=0$ vs. ${\mathrm{H}}_1:{\mathsf{\beta }}_1\ne 0$, incorrectly specifying the distribution of the random intercept and slope does not seem to have an impact, since there are no differences in the rejection rates when ${\mathsf{\beta }}_2=0$ upon comparing the four true distributions for random effects (Column 1). With regard to the rejection rates of the hypothesis when ${\mathsf{\beta }}_2\ne 0$ (Columns 2, 3 and 4), no rates below the normal distribution are observed. We can thus conclude that there is no evidence of an impact of incorrectly specifying the distribution of random effects for this case.

Figure 7 and Figure 8, on the other hand, show the rejection rates of the null hypotheses ${\mathrm{H}}_0:{\mathsf{\beta }}_1=0$ vs. ${\mathrm{H}}_0:{\mathsf{\beta }}_2=0$, respectively, and the results of the power of the statistical Wald test assuming values for ${\mathsf{\beta }}_1=0, 0.7, 1.4, 2.1$ and ${\mathsf{\beta }}_2=0$, 0.2, 0.4, 0.6 in an NB GLMM with a random intercept and slope.

Figure 7, similar to what was found for the Poisson GLMM with a random intercept and slope, shows evidence that there is indeed an impact of incorrectly specifying the distribution of random effects, since in Column 1, where the rejection rates of hypothesis ${\mathrm{H}}_0:{\mathsf{\beta }}_1=0$ vs. ${\mathrm{H}}_1:{\mathsf{\beta }}_1\ne 0$ when ${\mathsf{\beta }}_1=0$ (type I error), the highest rates correspond to the exponential and Tukey bivariate distributions, which becomes much more evident as the value of the variance increases and when it has a value of ${\mathsf{\sigma }}_{{\mathrm{b}}_{0\mathrm{i}}}^2={ \mathsf{\sigma }}_{{\mathrm{b}}_{1\mathrm{i}}}^2=8$. Additionally, when Columns 2, 3 and 4 are observed, where the power of the Wald test is shown when rejecting the hypotheses of interest (this being false), it can be seen how for some values of ${\mathsf{\beta }}_1$ and ${\mathsf{\sigma }}_{{\mathrm{b}}_{0\mathrm{i}}}^2={ \mathsf{\sigma }}_{{\mathrm{b}}_{1\mathrm{i}}}^2$, the rates are lower when the true distribution of the effects is the bivariate exponential, if compared with the results obtained for the bivariate normal distribution, thus indicating an impact of the incorrect specification of the distribution of the random effects.

Finally, Figure 8 shows the rejection rates of the hypothesis test ${\mathrm{H}}_0:{\mathsf{\beta }}_2=0$ vs. ${\mathrm{H}}_1:{\mathsf{\beta }}_2\ne 0$ in an NB GLMM with a random intercept and slope, for which it seems that all curves in Column 1 are above 10%, which suggests evidence of the impact of incorrectly specifying the distribution of random effects, especially with greater variances. In the second and third columns, the power seems to be too low, but this is because the intentional choice of the values for the parameters show rejection curves with a gradual increase, as is shown in all columns of the Figure.

4. Conclusions

In this article, the aim was to identify whether there is actually an impact of the incorrect specification of the distribution of random effects on the inferential procedures of Wald-type hypothesis tests. Data sets were simulated with Poisson or NB response variables, which had generalized linear mixed models adjusted, taking into consideration, in some cases, a random intercept, and in others, a random intercept and slope.

The simulation study presented here allows us to conclude that the impact of the incorrect specification appears to depend on the complexity of the structure of the random effects, the variance of the distribution of the underlying random effects and the parameters of interest. Additionally, the results indicate an impact of incorrect specification in the assumed distributions for random effects, which are quite different from the well-known characteristics of normal distribution: lognormal (positive asymmetry and high kurtosis) for random intercepts and bivariate exponential (moderate asymmetry and kurtosis) and bivariate Tukey (moderate asymmetry and high kurtosis) for random intercepts and slopes.

For both cases—Poisson and BN with a random intercept—an impact was found from incorrectly specifying the distribution of said intercept, as greater rejection rates were obtained for hypothesis ${\mathrm{H}}_0:{\mathsf{\beta }}_2=0$ vs. ${\mathrm{H}}_1:{\mathsf{\beta }}_2\ne 0$, where ${\mathsf{\beta }}_2=0$, i.e., higher probabilities of making a type I error when the true distribution of the intercept is lognormal, and also, the impact was greatest as the variance values increased for the random intercept. Moreover, evidence was found of said impact with the lowest rejection rates when the true distribution was uniform when ${\mathsf{\beta }}_2\ne 0$; i.e., with lower rates for the test power.

In the cases where generalized linear mixed models were adjusted with a Poisson or NB response and a random intercept and slope, and where the interest was focused on the Wald-type hypothesis tests for determining an effect of the parameter ${\mathsf{\beta }}_1$ or the parameter ${\mathsf{\beta }}_2$, an impact was indeed found arising from the incorrect specification of the random effects when proving hypothesis ${\mathrm{H}}_0:{\mathsf{\beta }}_1=0$ vs. ${\mathrm{H}}_1:{\mathsf{\beta }}_1\ne 0$, for the adjustment of both the Poisson and the NB models. This impact was observed with the highest rejection rates when the null hypothesis was true; i.e., the highest probability of making a type I error was found when the real distribution of the random effects was bivariate Tukey, and with the lowest rejection rates when the null hypothesis was false, being related to the statistical power when the true distribution was a bivariate exponential. When testing hypothesis ${\mathrm{H}}_0:{\mathsf{\beta }}_2=0$ vs. ${\mathrm{H}}_1:{\mathsf{\beta }}_2\ne 0$, no evidence was found for the incorrect specification of the distribution of the random effects.

The results presented herein contribute to understanding the implications of an incorrect specification of the distribution of random effects in GLMMs. However, this contribution is limited by the computational behaviour of considering only two functions in the simulation study (glmer, glmer.nb), which are found in the lme4 package of software R. Considerations and comparisons with other packages and software would be ideal; but this would imply a higher computational cost, which is beyond the scope and resources of this research.

5. Future Work

Although this study addressed the problem of the incorrect specification of the distribution of random effects, authors such as [31] identify other aspects of incorrect specification, such as the fact that the distribution of random effects may depend on a covariate or that this may depend on the sample size of the conglomerate. Therefore, for future work, it would be interesting to study the impact on the estimates of the parameters in generalized linear mixed models in these types of specifications, as well as those related to the incorrect specification of the link function or not taking into account the presence of overdispersion, among others.

Another aspect to consider in future works is the performance of the 100(1 − α)% confidence intervals, based on the Wald method, as studied in [2], which found in its simulation study that, in general, coverage rates are associated with the 95% value, with α = 0.05, except for highly skewed configurations. Therefore, a simulation study of this type would be expected to find high probability rates of a type I error with low coverage rates for the different parameters studied because both work with the exact estimates (β and SE(β)).

Finally, as a product of this work, as the distribution of random effects is unknown, a study that identifies the minimum ${\mathrm{n}}_{\mathrm{i}}$ needed to minimize the impact on type I error and statistical power would be desirable.