Nonnegative Estimation of Variance Components for a Nested Three-Way Random Model

Abstract

A nonnegative variance estimation procedure is suggested for an unbalanced data where two factors are nested in another. Since the involved factors are all random, the approach is based on a nested three-way random model. The proposed method for the estimation of variance components is compared with Henderson’s Method I and III in view of the same estimation procedure based on the method of moments. Although both the Henderson’s Method I and III are known to be useful for the estimation of variance components for balanced or unbalanced data, they often yield negative values as variance estimates whereas the estimates by the suggested method are never be negative. Hence, it points out what makes this happen and discusses how to fix the problem. The proposed method shows how to define sums of squares and the orthogonal coefficient matrices that are necessary for the evaluation of expectations. All the matrices of the quadratic forms for computing sums of squares are symmetric and idempotent. It also reveals the individual coefficient of each variance component does not change from equation to equation.

1. Introduction

The estimation of variance components in random models is extremely important in scientific research area for the evaluation of random effects. There are lots of literature on interesting topics concerning random components such as [1,2,3,4]. Substantial attention has been given for decades on fixing the problem of negative values which can not be admissible as estimates of variance components. Much literature has concentrated on the perplexing issue but the problem has not been resolved yet. Instead, several strategies of action are presented when it happens. Some reasonable comments are given by Thompson [5,6,7], Nelder [8], and Searl and Fawcett [9]. In recent research there are some improvements on nonnegative estimates from the study of variance estimation of random effects such as Choi [10,11]. The purpose of this study is to fix the problem of getting negative estimates of variance components in a three-way random model. Most often negative estimates come out of the analysis of unbalanced data. So, the nonnegative estimation is developed for an unbalanced data under a nested three-way random model. When a nesting occurs in the treatment structure nested effects are involved in the model. Related topics are shown in the literature such as Milliken and Johnson [12], Montgomery [13], Searle [14] Khan et al. [15] Sharma [16] and Ferreira et al. [17]. Although either Henderson’s Method I or III [18] as a method of moments is mainly used for the estimation of variance components, there seems to be no strategy when negative estimates happen. The discussion of nonnegative estimation procedure for the nested random model has been made a lot in literature but the solution for that issue still remains unsolved. Therefore, it is worthy to develop a method for getting nonnegative solution in a different way from Henderson’s Method I or III, and compare the pros and cons. Since the nesting occurs in the treatment structure it should be noticed that there is only one size of experimental unit available in data analysis. Otherwise, there would be one more size of experimental unit when the nesting occurs in the design structure. This paper concerns that how to get sums of squares for nested main effects and nested interaction effects when two random factors are nested in another, and deals with the problem how to fix if the estimates of the variance components are negative.

2. Nested and Crossed Random Effects Model

Suppose that data are from an experimental design where the three factors in the treatment structure are random, the levels of B are crossed with the levels of C, and the levels of two crossed factors B and C are nested within the levels of A, all in a completely randomized design structure. So, the data can be displayed on a three-way table. Let’s consider a model describing the data. For the model description, let $y_{ijkl}$ denote the observation of a unit treated with the ith level of A, jth level of B, and kth level of C. The model for the assumed experimental situation is

$$\begin{array}{cc}\hfill y_{ijkl}& =\mu +{\alpha }_i+{\beta }_{j\left(i\right)}+{\gamma }_{k\left(i\right)}+{\delta }_{jk\left(i\right)}+{ϵ}_{ijkl},\hfill \\ \hfill & i=1,2,\dots ,a;j=1,2,\dots ,b;k=1,2,\dots ,c;l=1,2,\dots ,n_{ijk},\hfill \end{array}$$

(1)

where $\mu$ is an overall mean, ${\alpha }_i$ is the effect of the ith level of random factor A, ${\beta }_{j\left(i\right)}$ is the effect of the jth level of random factor B nested in ith level of A, ${\gamma }_{k\left(i\right)}$ is the effect of the kth level of random factor C nested in ith level of A, ${\delta }_{jk\left(i\right)}$ is the interaction effect of the jth level of factor B and the kth level of factor C nested in ith level of A, and ${ϵ}_{l\left(ijk\right)}$ is the random error term. In the random model, we assume that the ${\alpha }_i$’s, ${\beta }_{j\left(i\right)}$’s, ${\gamma }_{k\left(i\right)}$’s and ${\delta }_{jk\left(i\right)}$’s are random with zero means and variances ${\sigma }_{\alpha }^2$, ${\sigma }_{\beta }^2$ , ${\sigma }_{\gamma }^2$ and ${\sigma }_{\beta \gamma }^2$ respectively. ${ϵ}_{ijkl}$’s are random errors assumed to have mean zero and variance ${\sigma }_{ϵ}^2$. All of the random effects in the model are assumed to be independent of each other.

The matrix form of the model can be expressed as

$$\begin{array}{c}\hfill \mathit{y}=\mathit{j}\mu +{\mathit{X}}_{\alpha }\mathit{\alpha }+{\mathit{X}}_{\beta }\mathit{\beta }+{\mathit{X}}_{\gamma }\mathit{\gamma }+{\mathit{X}}_{\delta }\mathit{\delta }+\mathit{ϵ},\end{array}$$

(2)

where $\mathit{y}$ denotes the $n\times 1$ vector of observations ($n={\sum }_{i=1}^a{\sum }_{j=1}^b{\sum }_{k=1}^c{n}_{ijk}$); $\mathit{j}$ denotes $n\times 1$ vector of ones; ${\mathit{X}}_{\alpha }$ is the $n\times a$ coefficients matrix of $\mathit{\alpha }$; $\mathit{\alpha }$ denotes the $a\times 1$ random vector and is assumed to be distributed as $N(\mathit{0},{\sigma }_{\alpha }^2{\mathit{I}}_a)$; ${\mathit{X}}_{\beta }$ denotes the $n\times b$ coefficients matrix of $\mathit{\beta }$; $\mathit{\beta }$ denotes the $\mathit{b}\times 1$ random vector and is assumed to be distributed as $N(\mathit{0},{\sigma }_{\beta }^2{\mathit{I}}_b)$; ${\mathit{X}}_{\gamma }$ denotes the $n\times c$ coefficients matrix of $\gamma$; $\gamma$ denotes the $c\times 1$ random vector and is assumed to be distributed as $N(\mathit{0},{\sigma }_{\gamma }^2{\mathit{I}}_c)$; ${\mathit{X}}_{\delta }$ denotes the $n\times bc$ coefficients matrix of $\delta$; $\delta$ denotes the $bc\times 1$ random vector and is assumed to be distributed as $N(\mathit{0},{\sigma }_{\delta }^2{\mathit{I}}_{bc})$; $ϵ$ is the usual $n\times 1$ random error vector and is assumed to be distributed as $N(\mathit{0},{\sigma }_{ϵ}^2{\mathit{I}}_n)$.

The parameters of the model are $\mu$, ${\sigma }_{\alpha }^2$, ${\sigma }_{\beta }^2$, ${\sigma }_{\gamma }^2$, ${\sigma }_{\delta }^2$ and ${\sigma }_{ϵ}^2$. There are many ways to estimate variance components from unbalanced data due to the way of fitting rather different from balanced data.

3. Sums of Squares and Orthogonal Coefficient Matrices

A method can be derived from the the matrix form of the model (2) for the nonnegative estimation of variance components. This method can be available for finding a set of sums of squares or quadratic forms in the observations that can be used for estimating variance components and getting a set of orthogonal coefficient matrices of random effects that is necessary for the expectations of mean squares. The procedure to have necessary sums of squares for estimating variance components is partitioning the vector space of observations into two subspaces, i.e., estimation space and error space, by the proper projections. The definition of projection, it’s related theorems and ideas are well discussed in Graybill [19], and Johnson and Wichern [20]. The number of sums of squares is required as many as the number of variance components in the model. A sum of squares is obtained in a quadratic form in the observations; the quadratic form is based on the concepts of symmetry; that is, the matrix of the quadratic form is symmetric and idempotent.

For the model (2), we let $\mathit{y}$ as

$$\begin{array}{c}\hfill \mathit{y}=\mathit{X}\mathit{\theta }+\mathit{ϵ},\end{array}$$

(3)

where $\mathit{X}=(\mathit{j},{\mathit{X}}_{\alpha },{\mathit{X}}_{\beta },{\mathit{X}}_{\gamma },{\mathit{X}}_{\delta })$ and $\mathit{\theta }={(\mu ,\mathit{\alpha },\mathit{\beta },\mathit{\gamma },\mathit{\delta })}^{\prime }$. The vector space of $\mathit{y}$ can be divided into two orthogonal vector subspaces by projecting $\mathit{y}$ onto the column space generated by $\mathit{X}$. Let the vector subspace generated by the model matrix $\mathit{X}$ of the model (3) be ${\mathit{V}}_s$. Then, the projection of $\mathit{y}$ onto the vector subspace ${\mathit{V}}_s$ is defined as $\mathit{X}{\mathit{X}}^{-}\mathit{y}$ where ${\mathit{X}}^{-}$ denotes the Moore-Penrose generalized inverse. A sum of squares can be obtained as a quadratic form in the observations such as ${\mathit{y}}^{\prime }(\mathit{I}-\mathit{X}{\mathit{X}}^{-})\mathit{y}$ whereas the coefficient matrix of the error vector $ϵ$ can be found as $\mathit{I}-\mathit{X}{\mathit{X}}^{-}$ from the residual vector. Let ${\mathit{P}}_{ϵ}$ denote the coefficient matrix of the error vector $ϵ$. Then, we get one pair of information about the residual sum of squares and the coefficient matrix from fitting the model (3). They are defined as

$$\begin{array}{cc}\hfill {\mathit{Q}}_e& ={\mathit{y}}^{\prime }(\mathit{I}-\mathit{X}{\mathit{X}}^{-})\mathit{y}\hfill \\ \hfill {\mathit{P}}_{ϵ}& =\mathit{I}-\mathit{X}{\mathit{X}}^{-}.\hfill \end{array}$$

(4)

The second model to be fitted is

$$\begin{array}{c}\hfill \mathit{y}={\mathit{X}}_c{\mathbf{\theta }}_c+{\mathbf{ϵ}}_c,\end{array}$$

(5)

where ${\mathit{X}}_c=(\mathit{j},{\mathit{X}}_{\alpha },{\mathit{X}}_{\beta },{\mathit{X}}_{\gamma })$ and ${\mathbf{\theta }}_c={(\mu ,\mathit{\alpha },\mathit{\beta },\mathit{\gamma })}^{\prime }$. Let ${\mathit{V}}_c$ be the vector subspace generated by the model matrix ${\mathit{X}}_c$ of the model (5). Then, ${\mathit{X}}_c{\mathit{X}}_c^{-}\mathit{y}$ is the projection of $\mathit{y}$ onto the vector subspace,${\mathit{V}}_c$. Denote the second sum of squares and the coefficient matrix of $\mathbf{\delta }$ by ${\mathit{Q}}_c$ and ${\mathit{P}}_{\delta }$ respectively. They are defined as

$$\begin{array}{cc}\hfill {\mathit{Q}}_c& ={\mathit{y}}^{\prime }(\mathit{I}-{\mathit{X}}_c{\mathit{X}}_c^{-})\mathit{y}\hfill \\ \hfill {\mathit{P}}_{\delta }& =(\mathit{I}-{\mathit{X}}_c{\mathit{X}}_c^{-}){\mathit{X}}_{\delta },\hfill \end{array}$$

(6)

where ${\mathit{P}}_{\delta }$ is the coefficient matrix of $\mathbf{\delta }$ and also orthogonal to the coefficient matrix of the random error vector. Since ${\mathit{Q}}_c$ is the squared distance of the residual vector it has the information on two variance components ${\sigma }_{\delta }^2$ and ${\sigma }_{ϵ}^2$. The model to be fitted as the third is

$$\begin{array}{c}\hfill \mathit{y}={\mathit{X}}_b{\mathbf{\theta }}_b+{\mathbf{ϵ}}_b,\end{array}$$

(7)

where ${\mathit{X}}_b=(\mathit{j},{\mathit{X}}_{\alpha },{\mathit{X}}_{\beta })$ and ${\mathbf{\theta }}_b={(\mu ,\mathit{\alpha },\mathit{\beta })}^{\prime }$. Let ${\mathit{V}}_b$ be the vector subspace generated by ${\mathit{X}}_b$ of the model (7). Then, ${\mathit{X}}_b{\mathit{X}}_b^{-}\mathit{y}$ is the projection of $\mathit{y}$ onto the vector subspace,${\mathit{V}}_b$. We define ${\mathit{Q}}_b$ and ${\mathit{P}}_{\gamma }$ as follows:

$$\begin{array}{cc}\hfill {\mathit{Q}}_b& ={\mathit{y}}^{\prime }(\mathit{I}-{\mathit{X}}_b{\mathit{X}}_b^{-})\mathit{y}\hfill \\ \hfill {\mathit{P}}_{\gamma }& =(\mathit{I}-{\mathit{X}}_b{\mathit{X}}_b^{-}){\mathit{X}}_{\gamma },\hfill \end{array}$$

(8)

where ${\mathit{Q}}_b$ is the sum of squares represented in the quadratic form as ${\mathit{y}}^{\prime }(\mathit{I}-{\mathit{X}}_b{\mathit{X}}_b^{-})\mathit{y}$ which has the information on three variance components; i.e., ${\sigma }_{\gamma }^2$, ${\sigma }_{\delta }^2$ and ${\sigma }_{ϵ}^2$. The coefficient matrix, ${\mathit{P}}_{\gamma }$, of the random vector $\mathbf{\gamma }$ can be driven from the residual vector of the model (7). Since there are 5 variance components in the model (3) we need two more sums of squares and two more corresponding coefficient matrices as well. The fourth model to be fitted is

$$\begin{array}{c}\hfill \mathit{y}={\mathit{X}}_a{\mathbf{\theta }}_a+{\mathbf{ϵ}}_a,\end{array}$$

(9)

where ${\mathit{X}}_a=(\mathit{j},{\mathit{X}}_{\alpha })$ and ${\mathbf{\theta }}_a={(\mu ,\mathit{\alpha })}^{\prime }$. Let ${\mathit{V}}_a$ be the vector subspace generated by ${\mathit{X}}_a$ of the model (9). Then, ${\mathit{X}}_a{\mathit{X}}_a^{-}\mathit{y}$ is the projection of $\mathit{y}$ onto the vector subspace,${\mathit{V}}_a$. Denote the sums of squares by ${\mathit{Q}}_a$ and the derived coefficient matrix by ${\mathit{P}}_{\beta }$. Then, they are defined as

$$\begin{array}{cc}\hfill {\mathit{Q}}_a& ={\mathit{y}}^{\prime }(\mathit{I}-{\mathit{X}}_a{\mathit{X}}_a^{-})\mathit{y}\hfill \\ \hfill {\mathit{P}}_{\beta }& =(\mathit{I}-{\mathit{X}}_a{\mathit{X}}_a^{-}){\mathit{X}}_{\beta },\hfill \end{array}$$

(10)

where ${\mathit{Q}}_a$ is the sum of squares represented in the quadratic form as ${\mathit{y}}^{\prime }(\mathit{I}-{\mathit{X}}_a{\mathit{X}}_a^{-})\mathit{y}$ which has the information on four variance components; i.e., ${\sigma }_{\beta }^2$ , ${\sigma }_{\gamma }^2$, ${\sigma }_{\beta \gamma }^2$ and ${\sigma }_{ϵ}^2$. The fourth coefficient matrix ${\mathit{P}}_{\beta }$ of $\mathit{\beta }$ can be derived from the residual vector after fitting the model (9). The final model to be fitted is

$$\begin{array}{c}\hfill \mathit{y}={\mathit{X}}_j{\theta }_j+{\mathbf{ϵ}}_j,\end{array}$$

(11)

where ${\mathit{X}}_j=\left(\mathit{j}\right)$ and ${\theta }_j=\mu$. Let ${\mathit{V}}_j$ be the vector subspace generated by ${\mathit{X}}_j$ of the model (11). Then, ${\mathit{X}}_j{\mathit{X}}_j^{-}\mathit{y}$ is the projection of $\mathit{y}$ onto the vector subspace,${\mathit{V}}_j$. Denote the sums of squares by ${\mathit{Q}}_j$ and the derived coefficient matrix by ${\mathit{P}}_{\mu }$. Then, they are defined as

$$\begin{array}{cc}\hfill {\mathit{Q}}_j& ={\mathit{y}}^{\prime }(\mathit{I}-{\mathit{X}}_j{\mathit{X}}_j^{-})\mathit{y}\hfill \\ \hfill {\mathit{P}}_{\alpha }& =(\mathit{I}-{\mathit{X}}_j{\mathit{X}}_j^{-}){\mathit{X}}_{\alpha },\hfill \end{array}$$

(12)

where ${\mathit{Q}}_j$ is the sum of squares represented in the quadratic form as ${\mathit{y}}^{\prime }(\mathit{I}-{\mathit{X}}_j{\mathit{X}}_j^{-})\mathit{y}$ which has the information on five variance components; i.e., ${\sigma }_{\alpha }^2$, ${\sigma }_{\beta }^2$, ${\sigma }_{\gamma }^2$, ${\sigma }_{\beta \gamma }^2$ and ${\sigma }_{ϵ}^2$. The fifth coefficient matrix ${\mathit{P}}_{\alpha }$ of $\mathit{\alpha }$ can be defined from the residual vector after fitting the model (11).

Since all the necessary coefficient matrices of random effects vectors and the sums of squares have been defined through the model fitting procedures, expectations of sums of squares should be studied for the estimation of variance as a next step.

4. Expectations of Sums of Squares

When $\mathit{y}$ is represented by the sum of all the orthogonal projections from the foregoing model fittings, it can be defined as

$$\begin{array}{c}\hfill \mathit{y}={\mathit{X}}_j{\mathit{X}}_j^{-}\mathit{y}+{\mathit{P}}_{\alpha }{\mathit{P}}_{\alpha }^{-}\mathit{y}+{\mathit{P}}_{\beta }{\mathit{P}}_{\beta }^{-}\mathit{y}+{\mathit{P}}_{\gamma }{\mathit{P}}_{\gamma }^{-}\mathit{y}+{\mathit{P}}_{\delta }{\mathit{P}}_{\delta }^{-}\mathit{y}+{\mathit{P}}_{ϵ}{\mathit{P}}_{ϵ}^{-}\mathit{y}.\end{array}$$

(13)

So, the covariance matrix of $\mathit{y}$ can be found as

$$\begin{array}{cc}\hfill \mathrm{Var}\left(\mathit{y}\right)& ={\sigma }_{\alpha }^2{\mathit{P}}_{\alpha }{\mathit{P}}_{\alpha }^{\prime }+{\sigma }_{\beta }^2{\mathit{P}}_{\beta }{\mathit{P}}_{\beta }^{\prime }+{\sigma }_{\delta }^2{\mathit{P}}_{\delta }{\mathit{P}}_{\delta }^{\prime }+{\sigma }_{\gamma }^2{\mathit{P}}_{\gamma }{\mathit{P}}_{\gamma }^{\prime }+{\sigma }_{ϵ}^2{\mathit{P}}_{ϵ}{\mathit{P}}_{ϵ}^{\prime }\hfill \\ \hfill & =\mathsf{\Sigma }.\hfill \end{array}$$

(14)

Let ${\mathit{y}}^{\prime }\mathit{Q}\mathit{y}$ be a quadratic form in the observations of $\mathit{y}$. The associated matrix $\mathit{Q}$ with the quadratic form is symmetric and idempotent. Then,

$$\begin{array}{cc}\hfill E\left({\mathit{y}}^{\prime }\mathit{Q}\mathit{y}\right)& =\mathrm{tr}\left(\mathit{Q}\mathsf{\Sigma }\right)+{\mu }^2{\mathit{j}}^{\prime }\mathit{Q}\mathit{j}\hfill \\ \hfill & =\mathrm{tr}\left(\mathit{Q}\mathsf{\Sigma }\right),\hfill \end{array}$$

(15)

where tr($\mathit{W}$) is the sum of the diagonal elements of the square matrix $\mathit{W}$. Since all the matrices in quadratic forms are defined such as $\mathit{Q}\mathit{j}=\mathit{0}$, the expectations of the sums of squares do not depend on $\mu$. The expectation of the quadratic form ${\mathit{y}}^{\prime }\mathit{Q}\mathit{y}$ is

$$\begin{array}{cc}\hfill E\left({\mathit{y}}^{\prime }\mathit{Q}\mathit{y}\right)& =\mathrm{tr}\left(\mathit{Q}\left({\sigma }_{\alpha }^2{\mathit{P}}_{\alpha }{\mathit{P}}_{\alpha }^{\prime }+{\sigma }_{\beta }^2{\mathit{P}}_{\delta }{\mathit{P}}_{\beta }^{\prime }+{\sigma }_{\gamma }^2{\mathit{P}}_{\gamma }{\mathit{P}}_{\gamma }^{\prime }+{\sigma }_{\delta }^2{\mathit{P}}_{\delta }{\mathit{P}}_{\delta }^{\prime }+{\sigma }_{ϵ}^2{\mathit{P}}_{ϵ}{\mathit{P}}_{ϵ}^{\prime }\right)\right)\hfill \\ \hfill & ={\sigma }_{\alpha }^2\mathrm{tr}\left({\mathit{P}}_{\alpha }^{\prime }\mathit{Q}{\mathit{P}}_{\alpha }\right)+{\sigma }_{\beta }^2\mathrm{tr}\left({\mathit{P}}_{\beta }^{\prime }\mathit{Q}{\mathit{P}}_{\beta }\right)+{\sigma }_{\gamma }^2\mathrm{tr}\left({\mathit{P}}_{\gamma }^{\prime }\mathit{Q}{\mathit{P}}_{\gamma }\right)+{\sigma }_{\delta }^2\mathrm{tr}\left({\mathit{P}}_{\delta }^{\prime }\mathit{Q}{\mathit{P}}_{\delta }\right)+{\sigma }_{ϵ}^2\mathrm{tr}\left({\mathit{P}}_{ϵ}^{\prime }\mathit{Q}{\mathit{P}}_{ϵ}\right))\hfill \\ \hfill & =c_{q\alpha }{\sigma }_{\alpha }^2+c_{q\beta }{\sigma }_{\beta }^2+c_{q\gamma }{\sigma }_{\gamma }^2+c_{q\delta }{\sigma }_{\delta }^2+c_{qϵ}{\sigma }_{ϵ}^2,\hfill \end{array}$$

(16)

where c’s are constants denoting the traces. Hartley’s synthesis [21] can be used for the calculation of the constants. Since there are five sums of squares in the quadratic form of the observations, expectations of the quadratic forms are given as follows.

$$\begin{array}{cc}\hfill E\left({\mathit{Q}}_e\right)& =\mathrm{tr}\left((\mathit{I}-\mathit{X}{\mathit{X}}^{-})\mathsf{\Sigma }\right)\hfill \\ \hfill & ={\sigma }_{\alpha }^2{c}_{e\alpha }+{\sigma }_{\beta }^2{c}_{e\beta }+{\sigma }_{\gamma }^2{c}_{e\gamma }+{\sigma }_{\delta }^2{c}_{e\delta }+{\sigma }_{ϵ}^2{c}_{eϵ},\hfill \\ \hfill E\left({\mathit{Q}}_c\right)& =\mathrm{tr}\left((\mathit{I}-{\mathit{X}}_c{\mathit{X}}_c^{-})\mathsf{\Sigma }\right)\hfill \\ \hfill & ={\sigma }_{\alpha }^2{c}_{c\alpha }+{\sigma }_{\beta }^2{c}_{c\beta }+{\sigma }_{\gamma }^2{c}_{c\gamma }+{\sigma }_{\delta }^2{c}_{c\delta }+{\sigma }_{ϵ}^2{c}_{cϵ},\hfill \\ \hfill E\left({\mathit{Q}}_b\right)& =\mathrm{tr}\left((\mathit{I}-{\mathit{X}}_b{\mathit{X}}_b^{-})\mathsf{\Sigma }\right)\hfill \\ \hfill & ={\sigma }_{\alpha }^2{c}_{b\alpha }+{\sigma }_{\beta }^2{c}_{b\beta }+{\sigma }_{\gamma }^2{c}_{b\gamma }+{\sigma }_{\delta }^2{c}_{b\delta }+{\sigma }_{ϵ}^2{c}_{bϵ},\hfill \\ \hfill E\left({\mathit{Q}}_a\right)& =\mathrm{tr}\left((\mathit{I}-{\mathit{X}}_a{\mathit{X}}_a^{-})\mathsf{\Sigma }\right)\hfill \\ \hfill & ={\sigma }_{\alpha }^2{c}_{a\alpha }+{\sigma }_{\beta }^2{c}_{a\beta }+{\sigma }_{\gamma }^2{c}_{a\gamma }+{\sigma }_{\delta }^2{c}_{a\delta }+{\sigma }_{ϵ}^2{c}_{aϵ},\hfill \\ \hfill E\left({\mathit{Q}}_j\right)& =\mathrm{tr}\left((\mathit{I}-{\mathit{X}}_j{\mathit{X}}_j^{-})\mathsf{\Sigma }\right)\hfill \\ \hfill & ={\sigma }_{\alpha }^2{c}_{j\alpha }+{\sigma }_{\beta }^2{c}_{j\beta }+{\sigma }_{\gamma }^2{c}_{j\gamma }+{\sigma }_{\delta }^2{c}_{j\delta }+{\sigma }_{ϵ}^2{c}_{jϵ}.\hfill \\ \hfill \end{array}$$

5. A Set of Linear Equations in Variance Components

Equating the sums of squares to their corresponding expected values to obtain variance component estimators yields the equations that are linear in variance components of the model. The set of equations can be summarized as follows.

$$\begin{array}{cc}\hfill {\mathit{Q}}_e& ={\sigma }_{\alpha }^2{c}_{e\alpha }+{\sigma }_{\beta }^2{c}_{e\beta }+{\sigma }_{\gamma }^2{c}_{e\gamma }+{\sigma }_{\delta }^2{c}_{e\delta }+{\sigma }_{ϵ}^2{c}_{eϵ},\hfill \\ \hfill {\mathit{Q}}_c& ={\sigma }_{\alpha }^2{c}_{c\alpha }+{\sigma }_{\beta }^2{c}_{c\beta }+{\sigma }_{\gamma }^2{c}_{c\gamma }+{\sigma }_{\delta }^2{c}_{c\delta }+{\sigma }_{ϵ}^2{c}_{cϵ},\hfill \\ \hfill {\mathit{Q}}_b& ={\sigma }_{\alpha }^2{c}_{b\alpha }+{\sigma }_{\beta }^2{c}_{b\beta }+{\sigma }_{\gamma }^2{c}_{b\gamma }+{\sigma }_{\delta }^2{c}_{b\delta }+{\sigma }_{ϵ}^2{c}_{bϵ},\hfill \\ \hfill {\mathit{Q}}_a& ={\sigma }_{\alpha }^2{c}_{a\alpha }+{\sigma }_{\beta }^2{c}_{a\beta }+{\sigma }_{\gamma }^2{c}_{a\gamma }+{\sigma }_{\delta }^2{c}_{a\delta }+{\sigma }_{ϵ}^2{c}_{aϵ},\hfill \\ \hfill {\mathit{Q}}_j& ={\sigma }_{\alpha }^2{c}_{j\alpha }+{\sigma }_{\beta }^2{c}_{j\beta }+{\sigma }_{\gamma }^2{c}_{j\gamma }+{\sigma }_{\delta }^2{c}_{j\delta }+{\sigma }_{ϵ}^2{c}_{jϵ},\hfill \\ \hfill \end{array}$$

(17)

When the set of linear equations are arranged in matrix form, it can be displayed as

$$\begin{array}{cc}\hfill \left(\begin{array}{c}{\mathit{Q}}_e\\ {\mathit{Q}}_c\\ {\mathit{Q}}_b\\ {\mathit{Q}}_a\\ {\mathit{Q}}_j\end{array}\right)& =\left(\begin{array}{ccccc}{c}_{e\alpha }& c_{e\beta }& c_{e\gamma }& c_{e\delta }& c_{eϵ}\\ c_{c\alpha }& c_{c\beta }& c_{c\gamma }& c_{c\delta }& c_{cϵ}\\ c_{b\alpha }& c_{b\beta }& c_{b\gamma }& c_{b\delta }& c_{bϵ}\\ c_{a\alpha }& c_{a\beta }& c_{a\gamma }& c_{a\delta }& c_{aϵ}\\ c_{j\alpha }& c_{j\beta }& c_{j\gamma }& c_{j\delta }& c_{jϵ}\end{array}\right)\left(\begin{array}{c}{\sigma }_{\alpha }^2\\ {\sigma }_{\beta }^2\\ {\sigma }_{\gamma }^2\\ {\sigma }_{\delta }^2\\ {\sigma }_{ϵ}^2\end{array}\right)\mathrm{or}\mathit{Q}=\mathit{C}{\mathbf{\sigma }}^2,\hfill \end{array}$$

(18)

where $\mathit{Q}$ denotes ${({\mathit{Q}}_e,{\mathit{Q}}_c,{\mathit{Q}}_b,{\mathit{Q}}_a,{\mathit{Q}}_j)}^{\prime }$, $\mathit{C}$ is the matrix of the coefficients, and ${\mathbf{\sigma }}^2={({\sigma }_{\alpha }^2,{\sigma }_{\beta }^2,{\sigma }_{\gamma }^2,{\sigma }_{\delta }^2,{\sigma }_{ϵ}^2)}^{\prime }$ in (17). Since the system of equations in (18) can be inconsistent in general, the normal equations are used for the best approximate solution to the system. They are:

$$\begin{array}{cc}\hfill {\mathit{C}}^{\prime }\mathit{C}\widehat{{\sigma }^2}& ={\mathit{C}}^{\prime }\mathit{Q}.\hfill \end{array}$$

(19)

From the normal equations, we get ${\hat{\sigma }}^2={\left({\mathit{C}}^{\prime }\mathit{C}\right)}^{-1}\left({\mathit{C}}^{\prime }\mathit{Q}\right)$.

6. Comparison of Three Sets of Variance Component Estimates

Variance components can never be negative by definition in any experimental situations. However, some experimental designs with random effects can yield negative values. A real data set is studied for verifying the nonnegative estimation procedure. As for the comparison of estimates of variance components, Milliken and Johnson’s data [12] are analyzed under an assumed model by 3 different methods; i.e., the new method, the analysis of variance method or Henderson’s Method I, and the fitting constants method or Henderson’s Method III. The data are for the study of the efficiency of workers in assembly lines at several plants. Three plants, four assembly sites within each plant, and three workers at each plant were randomly selected. Each worker was to work five times at each assembly site in his or her plant. The efficiency scores are recorded. Since there are unequal number of observations on each treatment combination, the data set is unbalanced. The total number of observations in the data are 118. The model used to describe the data is

$$\begin{array}{cc}\hfill y_{ijkl}& =\mu +p_i+w_{j\left(i\right)}+s_{k\left(i\right)}+{ws}_{jk\left(i\right)}+{ϵ}_{ijkl},\hfill \end{array}$$

(20)

where $p_i$ is the ith plant effect, $w_{j\left(i\right)}$ is the jth worker effect within plant i, $s_{k\left(i\right)}$ is the site effect within plant i, $w{s}_{jk\left(i\right)}$ is the interaction effect of worker and site in plant i, and ${ϵ}_{ijkl}$ is the error term. $p_i$, $w_{j\left(i\right)}$, $s_{k\left(i\right)}$, $w{s}_{jk\left(i\right)}$ and ${ϵ}_{ijkl}$ are assumed to follow $N(0,{\sigma }_p^2)$, $N(0,{\sigma }_w^2)$, $N(0,{\sigma }_s^2)$, $N(0,{\sigma }_{ws}^2)$, and $N(0,{\sigma }_{ϵ}^2)$ respectively. For the estimation of the variance components in the model (20), we apply the new method described as before. Then, the set of linear equations are

$$\begin{array}{cc}\hfill {\mathit{Q}}_e& =82\widehat{{\sigma }_{ϵ}^2},\hfill \\ \hfill {\mathit{Q}}_s& =51.42503\widehat{{\sigma }_{\delta }^2}+82\widehat{{\sigma }_{ϵ}^2},\hfill \\ \hfill {\mathit{Q}}_w& =82.73543\widehat{{\sigma }_{\gamma }^2}+51.42503\widehat{{\sigma }_{\delta }^2}+82\widehat{{\sigma }_{ϵ}^2},\hfill \\ \hfill {\mathit{Q}}_p& =78.21936\widehat{{\sigma }_{\beta }^2}+82.74543\widehat{{\sigma }_{\gamma }^2}+51.42503\widehat{{\sigma }_{\delta }^2}+82\widehat{{\sigma }_{ϵ}^2},\hfill \\ \hfill {\mathit{Q}}_j& =77.88136\widehat{{\sigma }_{\alpha }^2}+78.21936\widehat{{\sigma }_{\beta }^2}+82.73543\widehat{{\sigma }_{\gamma }^2}+51.42503\widehat{{\sigma }_{\delta }^2}+82\widehat{{\sigma }_{ϵ}^2},\hfill \\ \hfill \end{array}$$

(21)

where ${\mathit{Q}}_e=408.617$, ${\mathit{Q}}_s=2329.906$, ${\mathit{Q}}_w=3086.348$, ${\mathit{Q}}_p=5828.000$, and ${\mathit{Q}}_j=\mathrm{10,455.518}$. From the normal equations, we get $\widehat{{\mathbf{\sigma }}^2}={(\widehat{{\sigma }_p^2},\widehat{{\sigma }_w^2},\widehat{{\sigma }_s^2},\widehat{{\sigma }_{ws}^2},\widehat{{\sigma }_{ϵ}^2})}^{\prime }$, where $\widehat{{\sigma }_p^2}=59.417532$, $\widehat{{\sigma }_w^2}=35.050804$, $\widehat{{\sigma }_s^2}=9.142909$, $\widehat{{\sigma }_{ws}^2}=37.360959$, $\widehat{{\sigma }_{ϵ}^2}=4.983134$. Three different sets of estimates are displayed in

Table 1. Estimates of Variance Components.

Variance Component	New Method	Analysis of Variance Method	Henderson’s Method III
${\sigma }_p^2$	59.417532	49.727005	48.806265
${\sigma }_w^2$	35.050804	23.329973	24.254899
${\sigma }_s^2$	9.142909	−8.593521	−4.877959
${\sigma }_{ws}^2$	37.360959	39.332304	35.616742
${\sigma }_{ϵ}^2$	4.983134	4.983134	4.983134

. The new method yields nonnegative estimates while the other two methods, Henderson’s I and III, do not.

7. Discussion

Compared to the previous studies on the three-way random models with nested effects in literature, the proposed method for the estimation of variance components under the nested random model has several findings that are very significant and worthy to note. The first thing is it always provides nonnegative estimates regardless of whether the data are balanced or not. Secondly, it is notable the method is fitting the sub-models in the sequential fashion by excluding parameters one by one from the the whole model in order to get the sums of squares and corresponding orthogonal coefficient matrices of the random effects vectors. Thirdly, it reveals that the individual coefficient of each variance component does not change within the set of linear equations in variance components unlike the other methods. This implies that Satterthwait’s approximation [22] is not necessary for testing the hypothesis of a variance component. That the individual coefficient of each variance component has the same value is attributed to the use of the orthogonal coefficient matrix derived from the model fitting. Defining orthogonal coefficients matrices of random effects vectors are somewhat complicated depending on the model structure. Also, the coefficients matrices may be different depending on the model fitting procedure even for the same model. However, the estimates are the same for all the different fitting procedures. Since there are many various type of data, techniques for deriving the orthogonal coefficients matrices should be studied in view of model structure.

8. Conclusions

The suggested method can be applied to get both sums of squares and orthogonal coefficient matrices due to the source of variation in any type of three-way random models with nested random effects. The things obtained through the stepwise procedure can be used to get nonnegative estimates of variance components.The nonnegative estimation procedure can increase the precision of inference on variance components and hence it can be used as a proper tool for analyzing data analysis. The suggested method can be seen as an extension of the projection method for a two-way random model in Choi [10] or the projection method III for mixed-effects models in Choi [11]. The reason why we are employing the projection method III here is that the expectation of a sum of square can be expressed as the familiar form seen in expected mean square column in the analysis of variance table. However, the use of any one of three methods can yield same nonnegative estimates for variance components irrespective of the methods. The new method as an extension of the projection method yields nonnegative estimates for variance components of random main effects, nested random interaction effects and random error when there exists nested relation among factors in three-way random model. This paper delineates the procedure how to define orthogonal coefficients for unbalanced data in three-way classification with two crossed factors nested in another.