Asymptotic Results for Multinomial Models

Abstract

In this work, we derived new asymptotic results for multinomial models. To obtain these results, we started by studying limit distributions in models with a compact parameter space. This restriction holds since the key parameter whose components are the probabilities of the possible outcomes have non-negative components that add up to 1. Based on these results, we obtained confidence ellipsoids and simultaneous confidence intervals for models with normal limit distributions. We then studied the covariance matrices of the limit normal distributions for the multinomial models. This was a transition between the previous general results and on the inference for multinomial models in which we considered the chi-square tests, confidence regions and non-linear statistics—namely log-linear models with two numerical applications to those models. Namely, our approach overcame the hierarchical restrictions assumed to analyse the multidimensional contingency table.

1. Introduction

In several fields of study such as health, business, social sciences and education, the outcomes of variables are mainly discrete, i.e., the variables only take finite or countable numbers. A discrete variable whose outcome only takes finite numbers is called a categorical variable [1]. A categorical variable consists of a set of categories that are non-overlapping [2] and the outcome could be binary (dichotomous), i.e., with just two possible levels, such as “present” or “absent” as a desired condition, or polytomous, i.e., with more than two levels, as is the case of the “Likert” scale [3]. There are two common types of polytomous variables, as can be seen in [4], which are the ordinal and nominal scale of measurement. Categorical variables, such as one’s eye colour, ethnicity and affiliations, the categories of which cannot be ordered in any way, are nominal, while categories such as the level of resistance to a drug of a patient, the level of education and economic status exhibit a natural order and are thus ordinal.

In a study in which all the observed variables are categorical, the most common way of representing the data is in a contingency table, which is a cross-tabulation of the variables [5,6]. When there are m-variables, the contingency table is an m-dimensional table—also known as a multidimensional table when the attributes are more than two. The information of a contingency table is mainly summarized through appropriate measures such as measures of association or models. Association measures, although easy in their computation and interpretation, lead to a great loss of information, as can be seen in [7]. Models are preferred in the case where a more sensitive analysis is required. A model is a “theory” or a conceptual framework about observations, and the parameters in the model represent the “effects” that particular variables or combinations of variables have in determining the values taken by the observations.

The easiest and most common model for a contingency table is the log-linear model [8]. It is constructed by taking the natural logarithms of the cell probabilities by the analogy of the analysis of variance (ANOVA) models, as can be seen in [9,10,11]. Classical log-linear models are sometimes regarded in the framework of the generalized linear model (GLM). They are also important in connection with contingency matrices, as can be seen in [12]. Contemporary problems in categorical data analysis with extremely high-dimensional data with demanding computational procedures require the development of complex models. Much work has been done on the modelling of categorical data, as can be seen in [7,13]. For example, in [14], the author used regression models for modelling categorical data. In our work, we derived new asymptotic results that will enable us to obtain confidence ellipsoids and simultaneous confidence intervals, respectively, for the vector of probabilities and its components, which will enable us to overcome some inference limitations of the existing procedures.

Inferential statistical analysis requires assumptions about the probability distribution of the response variable. For categorical data, the main distribution is the multinomial distribution. Most of the time, categorical data result from n-independent and identical trials with each trial having two or more possible outcomes. When the n is identical and independent trials have the same category probabilities, then the distribution of counts in the various categories is the multinomial distribution. The binomial distribution is a special case of the multinomial distribution with just two possible outcomes for each trial. Usually, the parameters of the multinomial distribution are not known and these parameters are often estimated from the sample data by several estimation methods such as the maximum likelihood estimation (MLE), as can be seen, for instance, in [15], the minimum discrimination information (MDI) [16], weighted least squares (WLS) [17] and Bayesian estimation (BA) [18]. In a previous study [19], we wanted to minimize the average cost so we used statistical decision theory (SDT) since there were only a finite number of possible choices. We point out that we achieved consistency since the probability of selecting the choice with the least average cost tends towards 1 and when the sample size tends towards infinity.

If we have n realizations of an experiment with m possible results with probabilities $p_1,\cdots ,p_m$, we have the probability mass function, as can be seen in [20,21]:

$$Pr\left[\bigcap _{l=1}^m\left(N_l=n_l\right)\right]={\displaystyle \frac{n!}{{\prod }_{l=1}^m{n}_l!}}{p}_l^{n_l}$$

(1)

for the vector $\mathit{N}=\left(N_1,\cdots ,N_m\right)$ of the times we obtain the different results. This probability mass function corresponds to the singular multinomial distribution $\mathit{M}\left(·|n,\mathit{p}\right)$. We name as multinomial the models for describing these sets of independent realizations of experiments with a finite number of results:

For the vector with $\mathit{p}=\left(p_1,\cdots ,p_m\right)$ of probabilities, we have the vector of estimators:

$$\tilde{\mathit{p}}=\left({\tilde{p}}_1,\cdots ,{\tilde{p}}_m\right)$$

(2)

with:

$${\tilde{p}}_l={\displaystyle \frac{n_l}{n}},l=1,\cdots ,m$$

(3)

Moreover, as can be seen in [22], as $n\to \infty$:

$$\sqrt{n}\left({\tilde{\mathit{p}}}_m-\mathit{p}\right)\sim \mathcal{N}\left(\mathbf{0},\mathit{U}\left(\mathit{p}\right)\right)$$

(4)

where ∼ indicates the limit distribution, in this case $\mathcal{N}\left(\mathbf{0},\mathit{U}\left(\mathit{p}\right)\right)$, the normal distribution with the null mean vector and covariance matrix:

$$\mathit{U}\left(\mathit{p}\right)=\mathcal{D}\left(\mathit{p}\right)-\mathit{p}{\mathit{p}}^t,$$

(5)

where $\mathcal{D}\left(\mathit{p}\right)$ is the diagonal matrix with principal elements $p_1,\cdots ,p_m$. This result will play an important role in the asymptotic treatment of the multinomial models which is this paper’s goal.

To carry out that asymptotic treatment, we start by obtaining a convenient version of the continuous mapping theorem [23] in the next section on limit distributions. Then, we obtained confidence regions in Section 3, namely the confidence ellipsoids and simultaneous confidence intervals. Then, in Section 4, we studied the algebraic structure of the limit covariance matrix, $\mathit{U}\left(\mathit{p}\right)$.

In Section 5, we obtained chi-square tests for hypotheses on outcome probabilities and confidence ellipsoids and simultaneous confidence intervals for them. We also considered log-linear models for which we presented a numerical application. We pointed out that our approach to these models overcame the hierarchical restriction used to analyse multidimensional contingency tables.

Our use of both the classical and the new version of the parametrized continuous mapping theorem (PCMT) enabled us to carry out statistical inference for multinomial models. This inference was similar to ANOVA and related techniques but F-tests were replaced by chi-square tests which is highly convenient since now we have an infinity of degrees of freedom for the error.

Finally, we stress the close relationship between our ANOVA-like inference using the chi-square test and the usual treatment of fixed effect models. We point out that the F-test in that treatment had interesting invariance properties that expressed the symmetry of those models, especially since those models are associated with orthogonal partitions or sub-spaces which are invariant for rotation.

2. Limit Distributions

Let $\mathcal{C}$ be the class of continuous functions. If $\mathit{l}(·)\in \mathcal{C}$, and the distribution $F_{{\mathit{Y}}_n}$, of ${\mathit{Y}}_n$ converges to $F_{\mathit{Y}}\in \mathcal{C}$ (that is $F_{{\mathit{Y}}_n}\left({\mathit{y}}_{·}\right)⇝F_{\mathit{Y}}\left({\mathit{y}}_{·}\right)$, whenever ${\mathit{y}}_{·}$ is a continuity point of $F_{\mathit{Y}}(·)$), we have, as can be seen in [23,24,25,26]:

$$F_{\mathit{l}\left({\mathit{Y}}_n\right)}\underset{n\to \infty }{\to }{F}_{\mathit{l}\left(\mathit{Y}\right)}$$

(6)

as follows from the continuous mapping theorem.

If $\mathit{z}$ is obtained superposing sub-vectors $\mathit{u}$ and $\mathit{v}$, we put $\mathit{z}={\left[\mathit{u}\mathit{v}\right]}^t$. Then, if ${\mathit{\theta }}_n\stackrel{p}{\to }\mathit{\theta }$, with $\mathit{\theta }$ belonging to a compact set $\mathcal{D}$, and $F_{{\mathit{Y}}_n}\stackrel{}{\to }{F}_{\mathit{Y}}$, putting ${\dot{\mathit{Y}}}_n={\left[{\mathit{Y}}_n^t{\mathit{\theta }}_n^t\right]}^t$, ${\mathit{Y}}_n^{+}={\left[{\mathit{Y}}_n^t{\mathit{\theta }}^t\right]}^t$ and $\dot{\mathit{Y}}={\left[{\mathit{Y}}^t{\mathit{\theta }}^t\right]}^t$ to show that:

$$F_{\dot{\mathit{l}}\left({\dot{\mathit{Y}}}_n\right)}⟶F_{\dot{\mathit{l}}\left(\dot{\mathit{Y}}\right)}\left(·|\mathit{\theta }\right),$$

(7)

it is only needed to show that, as can be seen in [27]:

$$sup\left\{|F_{\dot{\mathit{l}}\left(\dot{{\mathit{Y}}_n}\right)}-F_{\dot{\mathit{l}}\left({\mathit{Y}}_n^{+}\right)}|\right\}\underset{n\to \infty }{\to }0$$

(8)

since $F_{\dot{\mathit{l}}\left({\mathit{Y}}_n^{+}\right)}(·)=F_{\dot{\mathit{l}}\left(\dot{\mathit{Y}}\right)}(·|\mathit{\theta })$

With ${\xi }_n(.)$ and $\xi (.)$, the probability measures associated with $F_{{\mathit{Y}}_n}$ and $F_{\mathit{Y}}$ and representing the Cartesian product by ×, whatever $\epsilon >0$, there exists a parallelepiped:

$$H\left(\epsilon \right)={\times }_{i=1}^m\left[a\left(\epsilon \right);b\left(\epsilon \right)\right]$$

(9)

with ${\mathit{Y}}_n,\mathit{Y}\in R^n$, such that $\xi \left(H\right(\epsilon \left)\right)\ge 1-\epsilon$. Since $F_{{\mathit{Y}}_n}⟶F_{\mathit{Y}}$, we have ${\xi }_n\left(H\left(\epsilon \right)\right)\underset{n\to \infty }{\to }\xi \left(H\left(\epsilon \right)\right)$ and so there will be $n\left(\epsilon \right)$ such that, for $n>n\left(\epsilon \right)$:

$${\xi }_n\left(H\left(\epsilon \right)\right)>1-2\epsilon .$$

(10)

Now:

$$\dot{H}\left(\epsilon \right)=H\left(\epsilon \right)\times \mathcal{D}$$

(11)

will also be a compact. Thus, if $\dot{\mathit{l}}(·)\in \mathcal{C}$, it is restriction to $\dot{H}\left(\epsilon \right)$ will be uniformly continuous. So, whatever $\delta >0$, there exists ${\delta }^{\prime }\left(\delta \right)>0$, such that, if $\dot{\mathit{Y}},{\dot{\mathit{Y}}}^{\prime }\in \dot{H}\left(\epsilon \right)$ and $\parallel \dot{\mathit{Y}}-{\dot{\mathit{Y}}}^{\prime }\parallel \le {\delta }^{\prime }\left(\delta \right)$, $\parallel \dot{\mathit{l}}\left(\dot{\mathit{Y}}\right)-\dot{\mathit{l}}\left({\dot{\mathit{Y}}}^{\prime }\right)\parallel <\delta$, where $\parallel ·\parallel$ indicates the Euclidean norm of a vector.

Let ${\dot{E}}_n\left(\delta \right)$ and ${\dot{E}}_n\left(\epsilon \right)$ be events that occur when $\parallel \dot{\mathit{l}}\left(\dot{\mathit{Y}}\right)-\dot{\mathit{l}}\left({\dot{\mathit{Y}}}^{\prime }\right)\parallel <\delta$ and when ${\mathit{Y}}_n\in H\left(\epsilon \right)$, respectively. We now establish:

Lemma 1.

$Pr\left({\dot{E}}_n\left(\delta \right)\right)\underset{n\to \infty }{\to }1$.

Proof. 

Since the restriction on $\dot{\mathit{l}}(·)$ to $\dot{H}\left(\epsilon \right)$ is uniformly continuous:

$$Pr\left[{\dot{E}}_n\left(\delta \right)|{\dot{E}}_n\left(\epsilon \right)\right]\underset{n\to \infty }{\to }1.$$

Thus, we only have to point out that $\epsilon$ is arbitrary and that:

$$Pr\left[{\dot{E}}_n\left(\delta \right)|{\dot{E}}_n\left(\epsilon \right)\right]-Pr\left[{\dot{E}}_n\left(\delta \right)\right]\le 1-Pr\left[{\dot{E}}_n\left(\epsilon \right)\right]$$

(12)

so that:

$$Pr\left[{\dot{E}}_n\left(\delta \right)\right]\ge Pr\left[{\dot{E}}_n\left(\delta \right)|{\dot{E}}_n\left(\epsilon \right)\right]-\left(1-Pr\left[{\dot{E}}_n\left(\epsilon \right)\right]\right)$$

(13)

where $\underset{n\to \infty }{lim}\left(1-Pr\left[{\dot{E}}_n\left(\epsilon \right)\right]\right)\le \epsilon$, whatever $\epsilon >0$, to establish the thesis. □

Since:

$$Pr\left({\dot{E}}_n\left(\delta \right)\right)\underset{n\to \infty }{\to }1,\forall \delta >0,$$

we have:

$$\left\{\begin{array}{c}Pr\left[\left(\dot{\mathit{l}}\left({\mathit{Y}}_n^{+}\right)\le \mathfrak{z}\pm \epsilon {\mathbf{1}}_m\right)|{\dot{E}}_n\left(\delta \right)\right]\underset{n\to \infty }{\to }Pr\left[\dot{\mathit{l}}\left({\mathit{Y}}_n^{+}\right)\le (\mathfrak{z}\pm \epsilon {\mathbf{1}}_m)\right]=F_{\dot{\mathit{l}}\left({\mathit{Y}}_n^{+}\right)}\left(\mathfrak{z}\pm \epsilon {\mathbf{1}}_m\right)\hfill \\ Pr\left[\left(\dot{\mathit{l}}\left({\dot{\mathit{Y}}}_n\right)\le \mathfrak{z}\pm \epsilon {\mathbf{1}}_m\right)|{\dot{E}}_n\left(\delta \right)\right]\underset{n\to \infty }{\to }Pr\left[\dot{\mathit{l}}\left({\dot{\mathit{Y}}}_n\right)\le (\mathfrak{z}\pm \epsilon {\mathbf{1}}_m)\right]=F_{\dot{\mathit{l}}\left({\dot{\mathit{Y}}}_n\right)}\left(\mathfrak{z}\pm \epsilon {\mathbf{1}}_m\right)\end{array}\right.$$

(14)

thus, also whatever $\delta >0$, there exists $\ddot{n}\left(\epsilon \right)$ such that, for $n\ge \ddot{n}\left(\epsilon \right)$, we have:

$$F_{\dot{\mathit{l}}\left({\mathit{Y}}_n^{+}\right)}\left(\mathfrak{z}\right)=F_{\dot{\mathit{l}}\left({\mathit{Y}}_n\right)}\left(\mathfrak{z}|\mathit{\theta }\right)\underset{n\to \infty }{\to }{F}_{\dot{\mathit{l}}\left(\mathit{Y}\right)}\left(\mathfrak{z}|\mathit{\theta }\right)$$

(15)

as well as:

$$F_{\dot{\mathit{l}}\left({\mathit{Y}}_n^{+}\right)}\left(\mathfrak{z}\right)\underset{n\to \infty }{\to }{F}_{\dot{\mathit{l}}\left(\mathit{Y}\right)}\left(\mathfrak{z}|\mathit{\theta }\right)$$

(16)

whenever $\mathfrak{z}$ is a continuity point of $F_{\dot{\mathit{l}}\left(\mathit{Y}\right)}$. Thus, we have the parametrized continuous mapping theorem (PCMT).

If $\dot{\mathit{l}}(·)\in \mathcal{C},F_{{\mathit{Y}}_n}\stackrel{}{\to }{F}_{\mathit{Y}}\in \mathcal{C},$ and ${\mathit{\theta }}_n\underset{n\to \infty }{\overset{p}{\to }}\mathit{\theta }$, with $\mathit{\theta }\in \mathcal{D}$, a compact set:

$$F_{\dot{\mathit{l}}\left({\dot{\mathit{Y}}}_n\right)}\left(·\right)\stackrel{}{\to }{F}_{\mathit{l}\left(\mathit{Y}\right)}\left(|\mathit{\theta }\right)$$

(17)

Remark 1

(Corollary to PCMT ).

a 

With $\mathit{\theta }$ a parameter both for $F_{\mathit{Y}}$ and $F_{\mathit{l}\left(\mathit{Y}\right)}$;

b 

when $F_{\mathit{Y}}$ is $\mathcal{N}\left(\mathbf{0},\mathit{V}\left(\mathit{\theta }\right)\right)$ and ${\mathit{\theta }}_n\stackrel{p}{\to }\mathit{\theta }$, we have:

$$F_{\dot{\mathit{l}}\left({\dot{\mathit{Y}}}_n\right)}\stackrel{}{\to }\mathcal{N}\left(\mathbf{0},\mathcal{G}\mathit{V}\left(\mathit{\theta }\right){\mathcal{G}}^t\right)$$

when $\dot{\mathit{l}}\left({\dot{\mathit{Y}}}_n\right)={\left[{\left(\mathcal{G}{\mathit{Y}}_n\right)}^t,{\mathit{\theta }}_n^t\right]}^t$ with $\mathcal{G}$ a matrix and $\mathit{V}\left(\mathit{\theta }\right)$ the covariance matrix of the parameter, $\mathit{\theta }$.

This remark will be renamed as a corollary of PCMT (CPCMT), as can be seen in [6].

We now consider the case of a sequence of random vectors with the same limit distribution. The sequence $\left\{{\mathit{X}}_n\right\}$ of random vectors is the mean stable when all its vectors have the same mean vector $\mathit{\mu }$.

Between the mean stable sequences, we may establish an equivalence relation writing $\left\{{\mathit{X}}_n\right\}\gamma \left\{{\mathit{Y}}_n\right\}$ if and only if:

$$S_n=sup\left\{\parallel {\mathit{Y}}_n-{\mathit{X}}_n\parallel \right\}\underset{n\to 0}{\overset{s}{\to }}0$$

(18)

where $\stackrel{s}{\to }$ means stochastic convergence, i.e., convergence in probability, as can be seen in [27,28]. We now establish:

Proposition 1.

If ${\mathit{S}}_n\stackrel{s}{\to }0$ and $F_{{\mathit{X}}_n}\to F^{\circ }\in \mathcal{C}\mathit{,}\mathit{then}{F}_{{\mathit{Y}}_n}\to F^{\circ }$, whenever $\left\{{\mathit{X}}_n\right\}\gamma \left\{{\mathit{Y}}_n\right\}$.

Proof. 

Let the vectors in $\left\{{\mathit{X}}_n\right\}$ and $\left\{{\mathit{Y}}_n\right\}$ have m components. Given $\mathit{x}\in R^m$, we consider the events $A_{n,1}\left(\epsilon \right)=\left\{{\mathit{X}}_n\le \mathit{x}-\epsilon {\mathbf{1}}_m\right\},A_{n,2}\left(\epsilon \right)=\left\{{\mathit{Y}}_n\le \mathit{x}\right\},A_{n,3}\left(\epsilon \right)=\left\{{\mathit{X}}_n\le \epsilon +{\mathbf{1}}_m\right\}$ and $B_n\left(\epsilon \right)=\left\{S_n\le \epsilon \right\}$.

Taking ${\dot{A}}_{n,i}\left(\epsilon \right)=A_{n,i}\left(\epsilon \right)\bigcap B_n\left(\epsilon \right)$, and ⇒ to indicate the implication, we have ${\dot{A}}_{n,1}\left(\epsilon \right)\Rightarrow {\dot{A}}_{n,2}\left(\epsilon \right)\Rightarrow {\dot{A}}_{n,3}\left(\epsilon \right)$. Thus, with $q_{n,i}\left(\epsilon \right)=Pr\left(A_{n,1}\left(\epsilon \right)\right)$ and ${\dot{q}}_{n,i}\left(\epsilon \right)=Pr\left({\dot{A}}_{n,1}\left(\epsilon \right)\right),i=1,2,3$, we have ${\dot{q}}_{n,1}\le {\dot{q}}_{n,2}\le {\dot{q}}_{n,3}$.

Moreover, since $S_n\underset{n\to \infty }{\overset{s}{\to }}0$, whatever $\delta >0$, there exist $n(\epsilon ,\delta )$ such that, for $n>n(\epsilon ,\delta ),Pr\left(B_n\left(\epsilon \right)\right)>1-\delta$, so ${\dot{q}}_{n,i}\left(\epsilon \right)\le q_{n,i}\left(\epsilon \right)\le {\dot{q}}_{n,i}\left(\epsilon \right)+\delta$, and, since $\delta$ is arbitrary:

$$\underset{n\to \infty }{lim}{\dot{q}}_{n,i}\left(\epsilon \right)=\underset{n\to \infty }{lim}{q}_{n,i}\left(\epsilon \right),i=1,2,3$$

(19)

so:

$$\underset{n\to \infty }{lim}{q}_{n,1}\left(\epsilon \right)\le \underset{n\to \infty }{lim}{q}_{n,2}\left(\epsilon \right)\le \underset{n\to \infty }{lim}{q}_{n,3}\left(\epsilon \right)$$

(20)

since $F^{\circ }\in \mathcal{C}$, we have $\underset{n\to \infty }{lim}{q}_{n,1}\left(\epsilon \right)=F^{\circ }\left(\mathit{x}-\epsilon {\mathbf{1}}_n\right)$ and $\underset{n\to \infty }{lim}{q}_{n,3}\left(\epsilon \right)=F^{\circ }\left(\mathit{x}+\epsilon {\mathbf{1}}_n\right)$, so:

$$F^{\circ }\left(\mathit{x}-\epsilon {\mathbf{1}}_n\right)\le \underset{n\to \infty }{lim}{F}_{{\mathit{Y}}_n}\left(\mathit{x}\right)\le F^{\circ }\left(\mathit{x}+\epsilon {\mathbf{1}}_n\right)$$

(21)

and given $\epsilon$ is arbitrary and $\mathit{x}$ may be whatever, from $F^{\circ }\in \mathcal{C}$, we obtain:

$$F_{{\mathit{Y}}_n}\left(\mathit{x}\right)\underset{n\to \infty }{\to }{F}^{\circ }\left(\mathit{x}\right)$$

(22)

which completes the proof. □

We then consider normal limit distributions, starting with:

Proposition 2.

Given $\mathcal{G}(·)=\left\{g_1(·),\cdots ,g_w(·)\right\}$ such that its component functions have gradients ${\mathit{g}}_1(·),\cdots ,{\mathit{g}}_w(·)$, Hessian matrices ${\underline{g}}_1(·),\cdots ,{\underline{g}}_w(·)$, and continuous second-order partial derivatives. Whatever the mean stable sequence $\left\{{\mathit{Z}}_n\right\}$ with the invariant mean vector $\mathit{\mu }$, taking ${\mathit{Y}}_n=\mathcal{G}\left({\mathit{Z}}_n\right)$ and ${\mathit{X}}_n=\mathcal{G}\left({\mathit{Z}}_{\mathit{n}}\right)$ with $\mathcal{G}(·)={\left[{\mathit{g}}_1(·),\cdots ,{\mathit{g}}_w(·)\right]}^t$, we have $\left\{{\mathit{X}}_n\right\}\gamma \left\{{\mathit{Y}}_n\right\}$, whenever $\sqrt{n}\left({\mathit{Z}}_n-{\mathit{\mu }}_n\right)$ converges in distribution to $F^{\circ }\in \mathcal{C}$.

Proof. 

We have $g_j\left({\mathit{Z}}_n\right)-g_j\left(\mathit{\mu }\right)={\mathit{g}}_j{\left(\mathit{\mu }\right)}^t\left({\mathit{Z}}_n-\mathit{\mu }\right)+{\displaystyle \frac{1}{2}}{\left({\mathit{Z}}_n-\mathit{\mu }\right)}^t{\underline{\mathit{g}}}_j\left({\mathit{\theta }}_{n,j}\right)\left({\mathit{Z}}_n-\mathit{\mu }\right)$ with $\left({\mathit{\theta }}_{n,j}\right)$ between ${\mathit{Z}}_n$ and $\mathit{\mu }$. Since ${\mathit{Z}}_n\stackrel{s}{\to }\mathit{\mu },$ we also have ${\mathit{\theta }}_{n,j}\stackrel{s}{\to }\mathit{\mu }$. Then, with ${\mathsf{\Theta }}_{\epsilon }\left(\mu \right)$ the radius $\epsilon$ sphere with centre $\mathit{\mu },Pr\left({\mathit{Z}}_n,{\mathit{\theta }}_{n,j}\epsilon {\mathsf{\Theta }}_{\epsilon }\left(\mathit{\mu }\right)\right)\stackrel{}{\to }1,j=1,\cdots ,m$. Now $\parallel {\underline{\mathit{g}}}_j\left(\mathfrak{z}\right)\parallel$ is a continuous function of $\mathfrak{z}$ so it will have a maximum $u_{j,\epsilon }$ in ${\mathsf{\Theta }}_{\epsilon }\left(\mathit{\mu }\right)$ that will exceed the supremum of the spectral radius of ${\underline{\mathit{g}}}_j\left(\mathit{\mu }\right)$ in ${\mathsf{\Theta }}_{\epsilon }\left(\mathit{\mu }\right)$, so:

$$\parallel {\displaystyle \frac{1}{2}}{\left({\mathit{Z}}_n-\mathit{\mu }\right)}^t{\underline{\mathit{g}}}_j\left({\mathit{\theta }}_{n,j}\right)\left({\mathit{Z}}_n-\mathit{\mu }\right)\parallel \le u_{j,\epsilon }{\parallel {\mathit{Z}}_n-\mathit{\mu }\parallel }^2,j=1,\cdots ,m$$

(23)

thus:

$$\left(g_j\left({\mathit{Z}}_n\right)-g_j\left({\mathit{\mu }}_n\right)\right)-{\mathit{g}}_j{\left(\mathit{\mu }\right)}^t\left({\mathit{Z}}_n-\mathit{\mu }\right)\stackrel{s}{\to }0,j=1,\cdots ,m$$

(24)

and so:

$$\parallel \left(\mathcal{G}\left({\mathit{Z}}_n\right)-\mathcal{G}\left(\mathit{\mu }\right)\right)-\mathcal{G}{\left(\mathit{\mu }\right)}^t\left({\mathit{Z}}_n-\mathit{\mu }\right)\parallel \stackrel{s}{\to }0$$

(25)

and the thesis follows from Proposition 1. □

Corollary 3.

If $\sqrt{n}\left({\mathit{Z}}_n-\mathit{\mu }\right)\sim \mathcal{N}\left(\mathbf{0},\mathit{V}\right)$, under the hypothesis of Propositions 1 and 2, $\sqrt{n}\left[\mathcal{G}\left({\mathit{Z}}_n\right)-\mathcal{G}\left(\mathit{\mu }\right)\right]\sim \mathcal{N}\left[\mathbf{0},\mathcal{G}\left(\mathit{\mu }\right)\mathit{V}\mathcal{G}{\left(\mathit{\mu }\right)}^t\right]$.

Proof. 

The thesis follows from Propositions 1 and 2 since the continuous mapping theorem, as can be seen in [23,24], implies that the limit distribution of $\sqrt{n}\mathcal{G}\left(\mathit{\mu }\right)\left({\mathit{Z}}_n-\mathit{\mu }\right)$ is $\mathcal{N}\left(\mathbf{0},\mathcal{G}\left(\mathit{\mu }\right)\mathit{V}\mathcal{G}{\left(\mathit{\mu }\right)}^t\right)$. □

3. Confidence Ellipsoids

We start by establishing the following.

Proposition 4.

If $\mathit{Y}$—not necessarily normal—has a covariance matrix $\mathit{C}$, with a range space $\mathsf{\Omega }=R\left(\mathit{C}\right)$, and the mean vector $\mathit{\mu }$, then:

$$Pr\left(\mathit{Y}-\mathit{\mu }\in \mathsf{\Omega }\right)=1$$

(26)

Proof. 

Let ${\mathit{\alpha }}_1,\cdots ,{\mathit{\alpha }}_m$ constitute an orthonormal basis for the orthogonal complement, ${\mathsf{\Omega }}^{\perp }$, of $\mathsf{\Omega }$. Then, ${\mathit{\alpha }}_j^t\left(\mathit{Y}-\mathit{\mu }\right)$ will have a null mean value and variance. Thus, according to the Bienaymé–Tchebycheff inequality:

$$Pr\left({\mathit{\alpha }}_j^t\left(\mathit{Y}-\mathit{\mu }\right)=0\right)=1,j=1,\cdots ,m.$$

Therefore, we obtain, with $\mathit{A}$, the matrix with row vectors ${\mathit{\alpha }}_1,\cdots ,{\mathit{\alpha }}_m$:

$$\begin{array}{ccc}\hfill Pr\left(\mathit{Y}-\mathit{\mu }\in \mathsf{\Omega }\right)& =& Pr\left[\mathit{A}\left(\mathit{Y}-\mathit{\mu }\right)=\mathbf{0}\right]\hfill \\ & =& Pr\left[\bigcap _{j=1}^m\left({\mathit{\alpha }}_j^t\left(\mathit{Y}-\mathit{\mu }\right)=0\right)\right]\hfill \\ & =& m-(m-1)\hfill \\ & =& 1,\hfill \end{array}$$

as follows from the Boole generalized inequalities, and so the thesis is established. □

We then have:

Lemma 2.

Given $\mathit{B}$ is a positive semi-definite [definite] matrix with positive eigenvalues ${\epsilon }_1,\cdots ,and {\epsilon }_h$ corresponding to eigenvectors ${\mathit{\alpha }}_1,\cdots ,{\mathit{\alpha }}_m$, we have $\mathit{B}={\mathit{A}}^t\mathcal{D}\mathit{A}$ and ${\mathit{B}}^{+}={\mathit{A}}^t{\mathcal{D}}^{-1}\mathit{A}$ with + indicating the Moore–Penrose inverse, $\mathcal{D}$ the diagonal matrix with principal elements ${\epsilon }_1,\cdots ,{\epsilon }_h$ and ${\mathit{A}}^t=\left[{\mathit{\alpha }}_1,\cdots ,{\mathit{\alpha }}_h\right]$.

Proof. 

It is easy to show that ${\mathit{BB}}^{+}$ and ${\mathit{B}}^{+}\mathit{B}$ are asymmetrical and that ${\mathit{BB}}^{+}\mathit{B}=\mathit{B}$ and ${\mathit{B}}^{+}{\mathit{BB}}^{+}={\mathit{B}}^{+}$ establish the thesis. □

We can now establish the following.

Proposition 5.

If $\mathit{Y}\sim \mathcal{N}\left(\mathit{\mu },{\sigma }^2\mathit{B}\right)$, with ${\mathit{B}}^{+}$ the Moore–Penrose inverse of B:

$${\left(\mathit{Y}-\mathit{\mu }\right)}^t{\mathit{B}}^{+}\left(\mathit{Y}-\mathit{\mu }\right)\sim {\sigma }^2{\chi }_h^2$$

where ${\chi }_h^2$ is a central chi-square distribution with $h=rank\left(\mathit{B}\right)$, degrees of freedom and ${\sigma }^2$ is the variance of $\mathit{Y}$.

Proof. 

As stated in Lemma 2, we have:

$${\left(\mathit{Y}-\mathit{\mu }\right)}^t{\mathit{B}}^{+}\left(\mathit{Y}-\mathit{\mu }\right)={\ddot{\mathit{Y}}}^t\ddot{\mathit{Y}}\sim {\sigma }^2{\chi }_h^2$$

with $\ddot{\mathit{Y}}={\mathcal{D}}^{-\frac{1}{2}}\mathit{A}\left(\mathit{Y}-\mathit{\mu }\right)$ where ${\mathcal{D}}^{-\frac{1}{2}}$ is the diagonal matrix with principal elements ${\epsilon }_1^{-1/2},\cdots ,{\epsilon }_h^{-1/2}$. We now only have to point out that $\ddot{\mathit{Y}}\sim \mathcal{N}\left(\mathbf{0},{\sigma }^2{\mathit{I}}_h\right)$ to establish the thesis, where ${\mathit{I}}_h$ is the identity matrix. □

We now consider confidence ellipsoids and simultaneous confidence intervals. Ellipsoids and their support planes are presented in [29]; the affine point of $\mathit{x}$ belongs to the ellipsoid:

$$\xi \left(\mathit{\mu },\mathit{B},r\right)=\left\{\mathit{x}:{\left(\mathit{x}-\mathit{\mu }\right)}^t{\mathit{B}}^{+}\left(\mathit{x}-\mathit{\mu }\right)\le r\right\}$$

(27)

if and only if:

$$\bigcap _{\mathit{v}}\left(|{\mathit{v}}^t\mathit{\mu }-{\mathit{v}}^t\mathit{x}|\le \sqrt{r\left({\mathit{v}}^t\mathit{B}\mathit{v}\right)}\right),$$

(28)

where $\bigcap _{\mathit{v}}$ indicates that all possible vectors $\mathit{v}$ are considered. We now establish:

Proposition 6.

If $\mathit{Y}\sim \mathcal{N}\left(\mathit{\mu },\mathit{B}\right):$

$$Pr\left[\bigcap _{\mathit{v}}\left(|{\mathit{v}}^t\mathit{\mu }-{\mathit{v}}^t\mathit{y}|\le \sqrt{x_{h,1-q}\left({\mathit{v}}^{t}B\mathit{v}\right)}\right)\right]=1-q,$$

(29)

with $x_{h,1-q}$, the (1-q)-th quantile of ${\chi }_h^2$ (the central chi-square with h degrees of freedom), when $rank\left(\mathit{B}\right)=h$.

Proof. 

The proof for the case $\mathit{Y}\sim \mathcal{N}\left(\mathit{\mu },\mathit{B}\right)$ directly follows from the previous considerations. Thus, we only have to point out that $\mathit{x}\in \xi \left(\mathit{\mu },\mathit{B},r\right)$ is equivalent to, as can be seen in [29]:

$$\bigcap _{\mathit{v}}\left(|{\mathit{v}}^t\mathit{\mu }-{\mathit{v}}^t\mathit{x}|\le \sqrt{r\left({\mathit{v}}^{t}B\mathit{v}\right)}\right).$$

(30)

 □

Since, as we saw ${\left(\mathit{Y}-\mathit{\mu }\right)}^t{\mathit{B}}^{+}\left(\mathit{Y}-\mathit{\mu }\right)\sim {\sigma }^2{\chi }_h^2$ when $\mathit{Y}\sim \mathcal{N}\left(\mathit{\mu },{\sigma }^2\mathit{B}\right)$, we have ${\left(\mathit{p}-\tilde{\mathit{p}}\right)}^t{\mathit{U}}^{+}\left(\mathit{p}\right)\left(\mathit{p}-\tilde{\mathit{p}}\right)\sim {\chi }_{m-1}^2$, because, as we shall see in the next section, $rank\left(\mathit{U}\left(p\right)\right)=m-1$.

In the next section, we will obtain results on $\left(\mathit{U}\left(\mathit{p}\right)\right)$ that will be used to obtain chi-square confidence regions for $\mathit{p}$ and through duality, test hypothesis on $\mathit{p}$.

4. Covariance Matrices

As we saw, for $\tilde{\mathit{p}}$, the limit covariance matrix of $\sqrt{n}\left({\tilde{\mathit{p}}}_m-\mathit{p}\right)$ is:

$$\mathit{U}\left(\mathit{p}\right)=\mathcal{D}\left(\mathit{p}\right)-\mathit{p}{\mathit{p}}^t$$

(31)

where $\mathit{p}=\left(p_1,\cdots ,p_m\right),p_j>0,j=1,\cdots ,m$ and ${\displaystyle \sum _{j=1}^m}{p}_j=1$. For the rank of the covariance matrix, we have:

$$\begin{array}{ccc}\hfill rank\left(\mathit{U}\left(\mathit{p}\right)\right)& =& rank\left({\mathit{D}\left(\mathit{p}\right)-\mathit{pp}}^t\right)\ge m-1\hfill \end{array}$$

since $rank\left(\mathcal{D}\left(\mathit{p}\right)\right)=m$ and $rank\left(\mathit{p}{\mathit{p}}^t\right)=1$, as follows from, as can be seen in [30], page 46, that $\left|rank\right(A)-rank(B\left)\right|\le rank(A+B)$. In addition to this, $\mathit{U}\left(\mathit{p}\right)\mathbf{1}=\mathbf{0}$ so $rank\left(\mathit{U}\left(\mathit{p}\right)\right)\le m-1$. Thus, $rank\left(\mathit{U}\left(\mathit{p}\right)\right)=m-1$:

Matrix $\mathit{U}\left(\mathit{p}\right)$ is a covariance matrix which, as can be seen in, [30], is positive semi-definite. There is therefore an orthogonal matrix $\mathit{P}\left(\mathit{p}\right)$ and a diagonal matrix $\mathcal{D}\left(\mathit{v}\right)$ whose principal elements are the eigenvalues $v_1,\cdots ,v_m$ of $\mathit{U}\left(\mathit{p}\right)$ such that:

$$\mathit{U}\left(\mathit{p}\right)=\mathit{P}\left(\mathit{p}\right)\mathcal{D}\left(\mathit{v}\right)\mathit{P}{\left(\mathit{p}\right)}^t$$

(32)

Since $rank\left(\mathit{U}\right(\mathit{p}\left)\right)=m-1$, we may order its eigenvalues to have $v_j>0,j=1,\cdots ,m-1$, and $v_m=0$. With $\mathcal{D}{\left(\mathit{v}\right)}^{\frac{1}{2}}$, the diagonal matrix with principal elements, $v_1^{1/2},\cdots ,v_m^{1/2}$ and:

$$\mathit{U}{\left(\mathit{p}\right)}^{\frac{1}{2}}=\mathit{P}\left(\mathit{p}\right)\mathcal{D}{\left(\mathit{v}\right)}^{\frac{1}{2}}{\mathit{P}\left(\mathit{p}\right)}^t$$

(33)

we will have:

$$\mathit{U}\left(\mathit{p}\right)={\mathit{U}\left(\mathit{p}\right)}^{\frac{1}{2}}{\mathit{U}\left(\mathit{p}\right)}^{\frac{1}{2}}.$$

(34)

We now establish:

Lemma 3.

If the $m\times m$ matrices ${\mathit{M}}_1\cdots {\mathit{M}}_w$ are such that ${\mathit{M}}_j^t{\mathit{M}}_{j^{\prime }}={\mathbf{0}}_{m\times m}$, when $j\ne j^{\prime }$, we have $rank\left({\displaystyle \sum _{j=1}^w}{\mathit{M}}_j\right)={\displaystyle \sum _{j=1}^w}rank\left({\mathit{M}}_j\right)$.

Proof. 

With $g_j=rank\left({\mathit{M}}_j\right)$ and ${\mathit{M}}_j=\left[{\mathit{m}}_{j,1},\cdots ,{\mathit{m}}_{j,m}\right],j=1,\cdots ,w$, there will be $g_j$ linearly independent column vectors $\left\{{\dot{\mathit{m}}}_{j,l};l\in {\mathcal{D}}_j\right\}$ of ${\mathit{M}}_j,j=1,\cdots ,w$. The vectors in $\bigcup _{j=1}^w\left\{{\dot{\mathit{m}}}_{j,l};l\in {\mathcal{D}}_j\right\}$ will be linearly independent, since, when $j\ne j^{\prime }$, they are orthogonal. Thus, $rank\left([{\mathit{M}}_1\cdots {\mathit{M}}_w]\right)\ge {\displaystyle \sum _{j=1}^w}{g}_j={\displaystyle \sum _{j=1}^w}rank\left({\mathit{M}}_j\right)$. Moreover, if we join another column vector of $\left[{\mathit{M}}_1\cdots {\mathit{M}}_w\right]$, say ${\mathit{m}}_{j^{\prime },l^{\prime }}$, to the set $\bigcup _{j=1}^w\left\{{\mathit{m}}_{j,l};l\in {\mathcal{D}}_j\right\}$, it will linearly depend on the $\left\{{\mathit{m}}_{j^{\prime },l^{\prime }}^{\prime };l^{\prime }\in {\mathcal{C}}_{j^{\prime }}\right\}$. Thus, the vectors in the extended set will not be linearly independent. Thus, $rank\left([{\mathit{M}}_1\cdots {\mathit{M}}_w]\right)={\displaystyle \sum _{j=1}^w}{g}_j={\displaystyle \sum _{j=1}^w}rank\left({\mathit{M}}_j\right)$. □

Consider that ${\displaystyle \sum _{j=1}^w}{\mathit{M}}_j=\left[{\mathit{M}}_1\cdots {\mathit{M}}_w\right]\left({\mathbf{1}}_w⨂{\mathit{I}}_m\right)$, with ⨂ indicating the Kronecker matrix product, as can be seen in [31], with $rank\left({\mathbf{1}}_w⨂{\mathit{I}}_m\right)=rank\left({\mathbf{1}}_m\right)rank\left({\mathit{I}}_m\right)=m$. Thus, as can be seen in [30]:

$$\begin{array}{ccc}\hfill rank\left({\displaystyle \sum _{j=1}^w}{\mathit{M}}_j\right)& \ge & rank\left([{\mathit{M}}_1\cdots {\mathit{M}}_w]\right)+rank\left({\mathbf{1}}_w⨂{\mathit{I}}_m\right)-m\hfill \\ & =& rank\left([{\mathit{M}}_1,\cdots ,{\mathit{M}}_w]\right)\hfill \\ & =& {\displaystyle \sum _{j=1}^w}rank\left({\mathit{M}}_j\right).\hfill \end{array}$$

and:

$$rank\left({\displaystyle \sum _{j=1}^w}{M}_j\right)\le rank\left([M_1,\cdots ,M_w]\right)={\displaystyle \sum _{j=1}^w}rank\left(M_j\right)$$

(35)

so:

$$rank\left({\displaystyle \sum _{j=1}^w}{M}_j\right)={\displaystyle \sum _{j=1}^w}rank\left(M_j\right),$$

(36)

as we wished to establish.

Let ${\mathit{Q}}_1,\cdots ,{\mathit{Q}}_w$ now be pairwise orthogonal orthogonal projection matrices (POOPM) with ${\mathit{Q}}_1={\displaystyle \frac{1}{m}}{\mathbf{1}}_m{\mathbf{1}}_n^t$. Now, $\mathit{U}\left(\mathit{p}\right)$ is $m\times m$ with rank $m-1$. Thus, its nullity space, $N\left(\mathit{p}\right)$ will have a dimension 1 and since ${\alpha }_1={\displaystyle \frac{1}{\sqrt{m}}}{\mathbf{1}}_m\in N\left(\mathit{p}\right),{\mathit{\alpha }}_1{\mathit{\alpha }}_1^t$ will be the orthogonal projection matrix on $N\left(\mathit{p}\right)$, since $\mathit{U}\left(\mathit{p}\right)$ is symmetrical so its range space $R\left(\mathit{p}\right)$ will be the orthogonal complement $N{\left(\mathit{p}\right)}^{\perp }$ of $N\left(\mathit{p}\right)$. The orthogonal projection matrix $\mathit{T}\left(\mathit{p}\right)$ on $\mathit{R}\left(\mathit{p}\right)$ will then be:

$$\mathit{T}\left(\mathit{p}\right)={\mathit{I}}_m-{\mathit{Q}}_1.$$

(37)

Thus, if ${\displaystyle \sum _{j=1}^w}{\mathit{Q}}_j={\mathit{I}}_m$, we will have $\mathit{T}\left(\mathit{p}\right)={\displaystyle \sum _{j=2}^w}{\mathit{Q}}_j$ as well as

$$\mathit{U}\left(\mathit{p}\right)=\mathit{T}\left(\mathit{p}\right)\mathit{U}\left(\mathit{p}\right)={\displaystyle \sum _{j=2}^w}{\mathit{Q}}_j\mathit{U}\left(\mathit{p}\right)$$

(38)

and, according to Lemma 3:

$$m-1=rank\left(\mathit{U}\left(\mathit{p}\right)\right)={\displaystyle \sum _{j=1}^w}rank\left({\mathit{Q}}_j\mathit{U}\left(\mathit{p}\right)\right).$$

(39)

Now:

$$rank\left({\mathit{Q}}_j\mathit{U}\left(\mathit{p}\right)\right)\le rank\left({\mathit{Q}}_j\right),j=2,\cdots ,w$$

and $m-1={\displaystyle \sum _{j=1}^w}rank\left({\mathit{Q}}_j\right)$. Thus, we must have:

$$rank\left({\mathit{Q}}_j\mathit{U}\left(\mathit{p}\right)\right)=rank\left({\mathit{Q}}_j\right),j=2,\cdots ,w$$

(40)

We now highlight that $\mathit{U}\left(\mathit{p}\right)$ and ${\mathit{U}\left(\mathit{p}\right)}^{\frac{1}{2}}$ have the same eigenvectors associated with positive eigenvalues. These eigenvectors constitute an orthonormal basis for $R\left(\mathit{U}\left(\mathit{p}\right)\right)=R\left({\mathit{U}\left(\mathit{p}\right)}^{\frac{1}{2}}\right)$. Thus:

$$\mathit{T}\left(\mathit{p}\right){\mathit{U}\left(\mathit{p}\right)}^{\frac{1}{2}}={\mathit{U}\left(\mathit{p}\right)}^{\frac{1}{2}}$$

(41)

and, reasoning as above, we obtain:

$$rank\left({\mathit{Q}}_j{\mathit{U}\left(\mathit{p}\right)}^{\frac{1}{2}}\right)=rank\left({\mathit{Q}}_j\right),j=2,\cdots ,w$$

(42)

Matrices ${\mathit{Q}}_1,\cdots ,{\mathit{Q}}_w$ naturally appear, as can be seen in [32], when there are factors that cross or groups of nested factors that cross. The sums of squares of effects and interactions of these factors are the $\parallel {\mathit{A}}_j{\mathit{Y}\parallel }^2$, $j=2,...,w$, and $\parallel {\mathit{A}}_1{\mathit{Y}\parallel }^2$ can be associated with the general mean.

5. Inference

5.1. Chi-Square Tests

According to the PCMT, when:

$$H_0\left(\mathit{A}\right):\mathit{Ap}=\mathbf{0}$$

(43)

holds, the limit distribution of:

$$L_n\left(\mathit{A}\right)=n{\left(\mathit{A}{\tilde{\mathit{p}}}_n\right)}^t{\left(\mathit{A}\mathit{U}\left({\tilde{\mathit{p}}}_n\right){\mathit{A}}^t\right)}^{+}\left(\mathit{A}{\tilde{\mathit{p}}}_n\right)$$

(44)

will be that of ${\chi }_r^2$ with $r=rank\left(\mathit{AU}\left({\tilde{\mathit{p}}}_n\right){\mathit{A}}^t\right)$, since when $H_0\left(\mathit{A}\right)$ holds:

$$\sqrt{n}\mathit{A}{\tilde{\mathit{p}}}_n\sim \mathcal{N}\left(\mathbf{0},\mathit{AU}\left({\mathit{p}}_n\right){\mathit{A}}^t\right)$$

(45)

and we also have ${\tilde{\mathit{p}}}_n\stackrel{p}{\to }\mathit{p}$. We thus have for $H_0\left(\mathit{A}\right)$ a q limit level test with statistic $L_n\left(\mathit{A}\right)$ and a critical value $x_{r,1-q}$, the $(1-q)-th$ is a quantile of ${\chi }_r^2$.

Moreover, under any alternative to $H_0\left(\mathit{A}\right)$:

$$H_1\left(\mathit{A}\right):\mathit{Ap}=\mathit{q}$$

(46)

we have, whatever $K>0$:

$$Pr\left(L_n\left(\mathit{A}\right)>K\right)\underset{n\to \infty }{\to }1$$

(47)

so the chi-square tests will be strongly consistent.

Let us assume that the probabilities $p_1,\cdots ,p_m$ correspond to the treatments of a fixed effects model in which d factors, with $h_1,\cdots ,h_d$ levels, cross (for instance, probabilities of cures for different treatments). We then have:

$$m=\prod _{l=1}^d{h}_l$$

(48)

which, as can be seen in [32], tests the effects and interactions of the factors. These effects and interactions correspond to subsets of $\overline{d}=\left\{1,\cdots ,d\right\}$. Thus, the null set, $\varphi$, will correspond to the general mean value, if the set has one element, it will be associated with the effects of the factor whose index belongs to the set. Otherwise, if the set has more than one element, it will be associated with the interaction between the levels of the factors with those indices. The sets can be ordered by the indexes:

$$j\left(\phi \right)=1+{\displaystyle \sum _{l\in \phi }}{2}^{l-1}$$

(49)

Putting ${\phi }_j$ to indicate the $j-th$ set, $j=1,\cdots ,2^d$, we have, as can be seen in [30], the matrices:

$$\mathit{A}\left({\phi }_j\right)=\underset{j=1}{\overset{d}{⨂}}{\mathit{A}}_l\left({\phi }_j\right),j=1,\cdots ,2^d$$

(50)

where:

$$\left\{\begin{array}{c}{\mathit{A}}_l\left({\phi }_j\right)={\displaystyle \frac{1}{\sqrt{h_l}}}{\mathbf{1}}_{h_l}^t,l\notin {\phi }_{j}j=1,\cdots ,2^d\hfill \\ {\mathit{A}}_l\left({\phi }_j\right)={\mathit{T}}_{h_l},l\in {\phi }_{j}j=1,\cdots ,2^d,\hfill \end{array}\right.$$

(51)

we then have, with ${\mathit{A}}_j=\mathit{A}\left({\phi }_j\right),j=1,\cdots ,2^d$:

$$g_j=rank\left({\mathit{A}}_j\right)=\prod _{j={\phi }_j}\left(h_j-1\right)j=1,\cdots ,2^d$$

(52)

Thus, for testing the:

$$H_{0,j}=H_0\left({\mathit{A}}_j\right):j=1,\cdots ,2^d$$

(53)

we have the statistic $L_n\left({\mathit{A}}_j\right)$, with $g_j$ degrees of freedom, $j=1,\cdots ,2^d$.

Another interesting case is that of cross-nesting factors. The factors in the $h$-th group have $a_{h,1},\cdots ,a_{h,f_h}$ levels, $h=1,\cdots ,d$. There are then $b_{h,v}=\prod _{v^{\prime }=1}^v{a}_{h,v^{\prime }},v=1,\cdots ,f_h$, combinations of levels of the first v factors in group h, and we also put $b_{h,0}=1,h=1,\cdots ,d$. Each of the combinations contains $c_{h,v}=\prod _{v^{\prime }=v+1}^{f_h}{a}_{h,v^{\prime }},0\le vs.<f_h$, or $c_{h,f_h}=1$, for combinations of levels of the remaining factors, and we have the matrices:

$$\left\{\begin{array}{c}{\mathit{A}}_{h,0}={\displaystyle \frac{1}{\sqrt{c_{h,0}}}}{\mathbf{1}}_{c_{h,0}}^t,h=1,\cdots ,d\hfill \\ {\mathit{A}}_{h,j}=I_{b_h,j-1}⨂T_{a_{h,j}}⨂{\displaystyle \frac{1}{\sqrt{c_{h,j}}}}{\mathbf{1}}_{c_{h,j}}^{t}j=1,\cdots ,f_h,h=1,\cdots ,d,\hfill \end{array}\right.$$

(54)

where ${\mathit{T}}_{\mathit{r}}$ is obtained by deleting the first row equal to ${\displaystyle \frac{1}{\sqrt{r}}}{\mathbf{1}}_r^t$ of a $r\times r$ orthogonal matrix and ⨂ indicates the Kronecker matrix product. These matrices have ranks:

$$\left\{\begin{array}{c}{g}_{h,0}=1,h=1,\cdots ,d\hfill \\ g_{h,j}=b_{h,j}-b_{h,j-1}h=1,\cdots ,d,\hfill \end{array}\right.$$

(55)

where $b_{h,0}=1,h=1,\cdots ,d$.

The effects and interactions in this cross-nesting are associated with the vectors $\mathit{j}=\left(j_1,\cdots ,j_d\right)$ with $j_h=0,\cdots ,f_h,h=1,\cdots ,d$. So $\mathit{j}$ is associated with the matrices:

$${\mathit{A}}_{\mathit{j}}=\underset{h=1}{\overset{d}{⨂}}{\mathit{A}}_{h,j_h};\mathit{j}\in \Gamma$$

(56)

with ranks:

$$g_{\mathit{j}}=\prod _{h=1}^d{g}_{h,j_h};\mathit{j}\in \Gamma$$

(57)

and where $\Gamma =\left\{\mathit{j};0\le j_h\le f_h,h=1,\cdots ,d\right\}$.

To test the hypothesis:

$$H_{0,\mathit{j}}=H_{0,\mathit{j}}\left({\mathit{A}}_{\mathit{j}}\right);\mathit{j}\in \Gamma$$

(58)

we have the statistic $L_n\left({\mathit{A}}_{\mathit{j}}\right)$ with $g_{\mathit{j}}$ degrees of freedom, $\mathit{j}\in \Gamma$.

5.2. Confidence Regions

According to the continuity of the Moore–Penrose inverses, as can be seen in [30], pp. 221–224, we have:

$$\mathit{U}{\left({\tilde{\mathit{p}}}_n\right)}^{+}\underset{n\to \infty }{\to }\mathit{U}{\left(\mathit{p}\right)}^{+},$$

so the PCMT gives:

$$n{\left(\mathit{p}-{\tilde{\mathit{p}}}_n\right)}^t\mathit{U}{\left({\tilde{\mathit{p}}}_n\right)}^{+}\left(\mathit{p}-{\tilde{\mathit{p}}}_n\right)\sim {\chi }_{m-1}^2$$

(59)

as well as:

$$n{\left({\mathit{A}}_{\mathit{j}}\mathit{p}-{\mathit{A}}_{\mathit{j}}{\tilde{\mathit{p}}}_n\right)}^t{\left({\mathit{A}}_{\mathit{j}}\mathit{U}\left({\tilde{\mathit{p}}}_n\right){\mathit{A}}_{\mathit{j}}^t\right)}^{+}\left({\mathit{A}}_{\mathit{j}}\mathit{p}-{\mathit{A}}_{\mathit{j}}{\tilde{\mathit{p}}}_n\right)\sim {\chi }_{g_{\mathit{j}}}^2,\mathit{j}\in \Gamma$$

(60)

for models with factors crossing and cross-nesting.

Thus:

$$\left\{\begin{array}{c}Pr\left[n{\left(\mathit{p}-{\tilde{\mathit{p}}}_n\right)}^t\mathit{U}{\left({\tilde{\mathit{p}}}_n\right)}^{+}\left(\mathit{p}-{\tilde{\mathit{p}}}_n\right)\le x_{m-1,1-q}\right]\underset{n\to \infty }{\to }1-q\hfill \\ \\ Pr\left[n{\left({\mathit{A}}_j\mathit{p}-{\mathit{A}}_j{\tilde{\mathit{p}}}_n\right)}^t{\left({\mathit{A}}_j\mathit{U}\left({\tilde{\mathit{p}}}_n\right){\mathit{A}}_j^t\right)}^{+}\left({\mathit{A}}_j\mathit{p}-{\mathit{A}}_j{\tilde{\mathit{p}}}_n\right)\le x_{r\left({\mathit{A}}_j\right),1-q}\right]\underset{n\to \infty }{\to }1-q,j=2,\cdots ,w\hfill \\ \\ Pr\left[n{\left({\mathit{A}}_{\mathit{j}}\mathit{p}-{\mathit{A}}_{\mathit{j}}{\tilde{\mathit{p}}}_n\right)}^t{\left({\mathit{A}}_{\mathit{j}}\mathit{U}\left({\tilde{\mathit{p}}}_n\right){\mathit{A}}_{\mathit{j}}^t\right)}^{+}\left({\mathit{A}}_{\mathit{j}}\mathit{p}-{\mathit{A}}_{\mathit{j}}{\tilde{\mathit{p}}}_n\right)\le x_{r\left({\mathit{A}}_{\mathit{j}}\right),1-q}\right]\underset{n\to \infty }{\to }1-q,\mathit{j}\in \Gamma \hfill \end{array}\right.$$

(61)

We can now apply Proposition 6 to obtain:

$$\left\{\begin{array}{c}Pr\left[\bigcap _{\mathit{v}}\left(|{\mathit{v}}^t\mathit{p}-{\mathit{v}}^t{\tilde{\mathit{p}}}_n|\le \sqrt{x_{n-1,1-q}\left({\mathit{v}}^t\mathit{U}\left({\tilde{\mathit{p}}}_n\right)\mathit{v}\right)}\right)\right]\underset{n\to \infty }{\to }1-q\hfill \\ \\ Pr\left[\bigcap _{\mathit{v}}\left(|{\mathit{v}}^t{\mathit{A}}_j\mathit{p}-{\mathit{v}}^t{\mathit{A}}_j{\tilde{\mathit{p}}}_n|\le \sqrt{x_{g_j,1-q}\left({\mathit{v}}^t{\mathit{A}}_j\mathit{U}\left({\tilde{\mathit{p}}}_n\right){\mathit{A}}_j^t\mathit{v}\right)}\right)\right]\underset{n\to \infty }{\to }1-q,j=2,\cdots ,w\hfill \\ \\ Pr\left[\bigcap _{\mathit{v}}\left(|{\mathit{v}}^t{\mathit{A}}_{\mathit{j}}\mathit{p}-{\mathit{v}}^t{\mathit{A}}_{\mathit{j}}{\tilde{\mathit{p}}}_n|\le \sqrt{x_{g_{\mathit{j}},1-q}\left({\mathit{v}}^t{\mathit{A}}_{\mathit{j}}\mathit{U}\left({\tilde{\mathit{p}}}_n\right){\mathit{A}}_{\mathit{j}}^t\mathit{v}\right)}\right)\right]\underset{n\to \infty }{\to }1-q,\mathit{j}\in \Gamma \hfill \end{array}\right.$$

(62)

5.3. Non-Linear Statistics

We assume that the component functions $g_l(·),l=1,\cdots ,w$ of $G(·)$ have continuous partial derivatives of the second order to apply Proposition 2 and its corollary in showing that:

$$\sqrt{n}\left(G\left({\tilde{\mathit{p}}}_n\right)-G\left(\mathit{p}\right)\right)\sim \mathcal{N}\left(\mathbf{0},\mathit{G}\left(\mathit{p}\right)\mathit{U}\left(\mathit{p}\right){\mathit{G}\left(\mathit{p}\right)}^t\right).$$

(63)

An interesting application of this result is that to log-linear models, as can be seen in [12], in which we use:

$$\left\{\begin{array}{c}G\left({\tilde{\mathit{p}}}_n\right)=\mathit{log}\left({\tilde{\mathit{p}}}_n\right)=\left(log{\tilde{p}}_{n,1},\cdots ,log{\tilde{p}}_{n,m}\right)\hfill \\ G\left(\mathit{p}\right)=\mathit{log}\left(\mathit{p}\right)=\left(log{p}_1,\cdots ,log{p}_m\right)\hfill \end{array}\right.$$

(64)

We now have:

$$\mathit{G}\left(\mathit{p}\right)=\mathcal{D}\left({\displaystyle \frac{1}{p_1}},\cdots ,{\displaystyle \frac{1}{p_m}}\right)=\mathcal{D}{\left(\mathit{p}\right)}^{-1},$$

(65)

so:

$$\mathit{G}\left(\mathit{p}\right)\mathit{U}\left(\mathit{p}\right)\mathit{G}{\left(\mathit{p}\right)}^t=\mathcal{D}{\left(\mathit{p}\right)}^{-1}-{\mathbf{1}}_m{\mathbf{1}}_m^t=\mathit{W}\left(\mathit{p}\right),$$

(66)

since $\mathcal{D}{\left(\mathit{p}\right)}^{-1}\mathit{p}=\mathbf{1}$.

Moreover, since $\mathcal{D}{\left(\mathit{p}\right)}^{-1}$ is invertible, taking:

$${\mathit{U}}^0\left(\mathit{p}\right)=\mathcal{D}{\left(\mathit{p}\right)}^{-1}\mathit{U}\left(\mathit{p}\right)\mathcal{D}{\left(\mathit{p}\right)}^{-1}$$

(67)

we have:

$$rank\left({\mathit{U}}^0\left(\mathit{p}\right)\right)=rank\left(\mathit{U}\left(\mathit{p}\right)\right)=m-1.$$

(68)

Thus:

$${\mathit{U}}^0\left(\mathit{p}\right)\left({\displaystyle \frac{1}{\parallel \mathit{p}\parallel }}\mathit{p}\right)=\mathbf{0}$$

(69)

so ${\displaystyle \frac{1}{\parallel \mathit{p}\parallel }}\mathit{p}$ belonging to the nullity space $N^0\left(\mathit{p}\right)$ of ${\mathit{U}}^0\left(\mathit{p}\right)$ constitutes an orthonormal basis for that space. Since ${\mathit{U}}^0\left(\mathit{p}\right)$ is symmetrical, its range space $R^0\left(\mathit{p}\right)$ will be $N^0{\left(\mathit{p}\right)}^{\perp }$, and the orthogonal projection matrix on $R^0\left(\mathit{p}\right)$ will be:

$${\mathit{T}}^0\left(\mathit{p}\right)={\mathit{I}}_m-{\displaystyle \frac{1}{\parallel \mathit{p}\parallel }}\mathit{p}{\mathit{p}}^t$$

(70)

Let ${\mathit{A}}^0$ have row vectors that constitute an orthonormal basis for a sub-space ${\nabla }^0$. Putting:

$$\left\{\begin{array}{c}{\mathit{l}}^0\left(\mathit{p}\right)=\mathit{log}\left(\mathit{p}\right)\hfill \\ {\mathit{l}}^0\left({\tilde{\mathit{p}}}_n\right)=\mathit{log}\left({\tilde{\mathit{p}}}_n\right)\hfill \end{array}\right.$$

(71)

We have, according to the PCMT:

$$\sqrt{n}\left[\mathit{A}{\mathit{l}}^0\left({\tilde{\mathit{p}}}_n\right)-\mathit{Al}\left(\mathit{p}\right)\right]\sim \mathcal{N}\left(\mathbf{0},{\mathit{AU}}^0\left(\mathit{p}\right){\mathit{A}}^t\right).$$

Thus, to test:

$$H_0\left(\mathit{A}\right):\mathit{A}{\mathit{l}}^0\left(\mathit{p}\right)=\mathbf{0}$$

(72)

we have the limit q level chi-square test with the statistic $n{\mathit{l}}^0{\left({\tilde{\mathit{p}}}_n\right)}^t{\left({\mathit{AU}}^0\left(\mathit{p}\right){\mathit{A}}^t\right)}^{+}{\mathit{l}}^0\left({\tilde{\mathit{p}}}_n\right)$ and the critical value $x_{r^0\left(\mathit{A}\right),1-q}$ with $r^0\left(\mathit{A}\right)=rank\left({\mathit{AU}}^0\left(\mathit{p}\right){\mathit{A}}^t\right)$. These tests will be strongly consistent as follows from ${\mathit{l}}^0\left({\tilde{\mathit{p}}}_n\right)\underset{n\to \infty }{\overset{p}{\to }}{\mathit{l}}^0\left(\mathit{p}\right)$.

Moreover, we have the limit level $1-q$ confidence ellipsoids given by

$$\left\{\begin{array}{c}Pr\left[{\left({\mathit{l}}^0\left(\mathit{p}\right)-{\mathit{l}}^0\left({\tilde{\mathit{p}}}_n\right)\right)}^t{\mathit{U}}^0{\left({\tilde{\mathit{p}}}_n\right)}^{+}\left({\mathit{l}}^0\left(\mathit{p}\right)-{\mathit{l}}^0\left({\tilde{\mathit{p}}}_n\right)\right)\le x_{m-1,1-q}\right]\underset{n\to \infty }{\to }1-q\hfill \\ Pr\left[{\left({\mathit{Al}}^0\left(\mathit{p}\right)-{\mathit{Al}}^0\left({\tilde{\mathit{p}}}_n\right)\right)}^t{\mathit{AU}}^0{\left({\tilde{\mathit{p}}}_n{\mathit{A}}^t\right)}^{+}\left({\mathit{Al}}^0\left(\mathit{p}\right)-\mathit{A}{\mathit{l}}^0\left({\tilde{\mathit{p}}}_n\right)\right)\le x_{r^0\left(\mathit{A}\right),1-q}\right]\underset{n\to \infty }{\to }1-q\hfill \end{array}\right.$$

(73)

We can now apply Proposition 6 to obtain:

$$\left\{\begin{array}{c}Pr\left[\bigcap _{\mathit{v}}\left(|{\mathit{v}}^t{\mathit{l}}^0\left(\mathit{p}\right)-{\mathit{v}}^t{\mathit{l}}^0\left({\tilde{\mathit{p}}}_n\right)|\le \sqrt{x_{m-1,1-q}\left({\mathit{v}}^t{\mathit{U}}^0\left({\tilde{\mathit{p}}}_n\right)\mathit{v}\right)}\right)\right]\underset{n\to \infty }{\to }1-q\hfill \\ Pr\left[\bigcap _{\mathit{v}}\left(|{\mathit{v}}^t{\mathit{Al}}^0\left(\mathit{p}\right)-{\mathit{v}}^t{\mathit{Al}}^0\left({\tilde{\mathit{p}}}_n\right)|\le \sqrt{x_{r^0\left(\mathit{A}\right),1-q}\left({\mathit{v}}^t{\mathit{AU}}^0\left({\tilde{\mathit{p}}}_n\right){\mathit{A}}^t\mathit{v}\right)}\right)\right]\underset{n\to \infty }{\to }1-q,j=2,\cdots ,w\hfill \end{array}\right.$$

(74)

5.4. Numerical Example

In this section, we apply our results, for non-linear statistic, to a dataset on coronary heart disease analysed by [12,33], with the log-linear model, a non-linear statistic. In all, a total of 1330 sick patients were categorized with respect to three variables, namely blood pressure, serum cholesterol and whether they had a coronary heart disease or not. The Blood pressure, which was the first variable had four categorical levels. The second variable, which was, Serum cholesterol, also had four categorical levels while the third variable, which indicated the presence of coronary heart disease, had two levels. So in all, the data had 32 classes with a total of 1330 sick patients. Kindly refer to Section 5.6 of [12] for details on the variables and their levels as well as a cross-classification of the frequencies for each category.

To proceed with the analysis by the application of our method, as can be seen in Equation (63), we firstly estimate the probabilities of each of the 32 classes. In order to apply the non-linear statistics, we calculated the logarithms of those estimates. As in Equation (63), the composite functions of the estimates were normally distributed with a null mean vector and a certain covariance matrix. We proceeded by evaluating our covariance matrix—firstly by determining the Jacobian matrix of the gradient of our composite function, as in Equation (65) and by Equation (66), and then determining the covariance matrix for our composite function.

To test the hypothesis of the absence of effects and interactions, we started by obtaining matrices $\mathit{A}$ using orthogonal matrices.

The first orthogonal matrix we considered is:

$${\mathit{P}}_2=\left[\begin{array}{cc}{\displaystyle \frac{1}{\sqrt{2}}}& {\displaystyle \frac{1}{\sqrt{2}}}\\ {\displaystyle \frac{1}{\sqrt{2}}}& -{\displaystyle \frac{1}{\sqrt{2}}}\end{array}\right]$$

(75)

Since we have 32 classes, we build the $\mathit{P}$ matrices up to ${\mathit{P}}_{32}$ with Kronecker matrix products:

$$\begin{array}{ccc}\hfill {\mathit{P}}_4& =& {\mathit{P}}_2⨂{\mathit{P}}_2\hfill \\ \hfill {\mathit{P}}_8& =& {\mathit{P}}_2⨂{\mathit{P}}_4\hfill \\ \hfill {\mathit{P}}_{16}& =& {\mathit{P}}_2⨂{\mathit{P}}_8\hfill \\ \hfill {\mathit{P}}_{32}& =& {\mathit{P}}_2⨂{\mathit{P}}_{16},\mathrm{where}⨂\mathrm{is}\mathrm{the}\mathrm{Kronecker}\mathrm{product}.\hfill \end{array}$$

After getting the orthogonal matrix ${\mathit{P}}_{32}$, we determined our $\mathit{A}$ matrices. Since we have three factors, we have eight $\mathit{A}$ matrices defined as follows.

Let the set containing the factors indexes be $\phi =\left\{1,2,3\right\}$, then the subsets of $\phi$ and the corresponding factor effects and interactions are:

$$\begin{array}{ccc}\hfill j=1,\phi =⌀& :& \mathrm{overall}\mathrm{or}\mathrm{general}\mathrm{mean}\mathrm{effect}\hfill \\ \hfill j=2,\phi =\left\{1\right\}& :& \mathrm{effects}\mathrm{of}\mathrm{the}\mathrm{first}\mathrm{factor}\hfill \\ \hfill j=3,\phi =\left\{2\right\}& :& \mathrm{effects}\mathrm{of}\mathrm{the}\mathrm{second}\mathrm{factor}\hfill \\ \hfill j=4,\phi =\left\{1,2\right\}& :& \mathrm{interactions}\mathrm{between}\mathrm{the}\mathrm{first}\mathrm{and}\sec \mathrm{ond}\mathrm{factors}\hfill \\ \hfill j=5,\phi =\left\{3\right\}& :& \mathrm{effects}\mathrm{of}\mathrm{the}\mathrm{third}\mathrm{factor}\hfill \\ \hfill j=6,\phi =\left\{1,3\right\}& :& \mathrm{interactions}\mathrm{between}\mathrm{the}\mathrm{first}\mathrm{and}\mathrm{third}\mathrm{factors}\hfill \\ \hfill j=7,\phi =\left\{2,3\right\}& :& \mathrm{interactions}\mathrm{between}\mathrm{the}\sec \mathrm{ond}\mathrm{and}\mathrm{third}\mathrm{factors}\hfill \\ \hfill j=8,\phi =\left\{1,2,3\right\}& :& \mathrm{interactions}\mathrm{between}\mathrm{all}\mathrm{factors}\hfill \end{array}$$

Now, the ${\mathit{A}}_j,j=1,\cdots ,8$ are obtained from ${\mathit{P}}_{32}$ as follows: ${\mathit{A}}_1$ is the first row; ${\mathit{A}}_2$ is the second to fourth rows; ${\mathit{A}}_3$ is the fifth, ninth and thirteenth rows; ${\mathit{A}}_4$ is the sixth, seventh, eighth, tenth, eleventh, twelfth, fourteenth, fifteenth and sixteenth rows. Similarly, ${\mathit{A}}_5$ is the seventeenth row; ${\mathit{A}}_6$ is the eighteenth, nineteenth and twentieth rows; ${\mathit{A}}_7$ is the twenty-first, twenty-fifth and twenty-ninth rows; and ${\mathit{A}}_8$ is the twenty-second, twenty-third, twenty-forth, twenty-sixth, twenty-seventh, twenty-eight, thirtieth, thirty-first and thirty-second rows.

Now, according to Equation (44), we have the covariance matrices:

$${\mathit{V}}_j\left(\mathit{p}\right)={\mathit{A}}_j^t\mathit{W}\left(\mathit{p}\right){\mathit{A}}_j,j=1,\cdots ,8,$$

(76)

where $\mathit{W}\left(\mathit{p}\right)$ is defined in Equation (66), for the:

$${\mathit{Z}}_j\left(\mathit{p}\right)={\mathit{A}}_{j}G\left(\mathit{p}\right),j=1,\cdots ,8,$$

(77)

where $G\left(\mathit{p}\right)$ is given by Equation (64). The sum of squares $\left(\mathit{SS}\right)$ for the statistic ${\mathit{Z}}_j\left(\mathit{p}\right),j=1,\cdots ,8$ are given by

$${\mathit{SS}}_j=n{\mathit{Z}}_j{\left(\mathit{p}\right)}^t{\mathit{V}}_j{\left(\mathit{p}\right)}^{+}{\mathit{Z}}_j\left(\mathit{p}\right),j=1,\cdots ,8$$

(78)

where n is the total number of our observations, and + indicates the Moore–Penrose inverses of the ${\mathit{V}}_j\left(\mathit{p}\right),j=1,\cdots ,8$. The ${\mathit{SS}}_j,j=1,\cdots ,8$ are chi-squares with degrees of freedom the $rank\left({\mathit{A}}_j\right),j=1,\cdots ,8$.

Table 1. ANOVA-like table for the coronary heart disease data.

Sources of Variation	Degree of Freedom	Sum of Squares
General mean, $\mu$	1	4752.504 ***
1	3	30.289 ***
2	3	20.915 ***
1 × 2	9	5.623
3	1	365.871 ***
1 × 3	3	24.204 ***
2 × 3	3	22.284 ***
1 × 2 × 3	9	4.568

*** indicates “significance” at $0.005$ levels. Source: author’s own calculations.

is an ANOVA-like table that presents the results of our analysis of the coronary heart disease data. This gives the sources of variation of the general mean, the main effects and the interaction effects. It also presents the degrees of freedom and the sum of squares for these effects. The significance level of these effects are indicated in *. From Table 1, we see that the general mean is highly significantly different from 0. Moreover, the factors blood pressure, serum cholesterol and coronary heart disease as well as both interactions between the first and second factors with the third factor were highly significant. The interactions in which the first and second factors did partake were not significant. We pointed out that we were able to consider the interaction between the three factors by using our approach. In the classical analysis of the data, as can be seen in [12] (Section 5.6), this would not be possible.

Again, we applied our method to analyse the General Social Survey (GSS 2008). The General Social Survey (GSS) is a survey which conducts basic scientific research on the structure and development of American society with a data-collection program designed to both monitor societal change within the United States and to compare the United States with other nations [34]. We considered three categorical variables, “Education”, “Political party affiliation” and “Gender”. The “Education” variable had 5 categorical levels, while “Political party affiliation” and “Gender” has 7 and 2 categorical levels respectively, as analyzed by [7] in Section 3.2.4.

Just as in Table 1 and

Table 2. ANOVA-like table for the GSS 2008 data.

Sources of Variation	Degree of Freedom	Sum of Squares
General mean, $\mu$	1	49,387.730 ***
1	4	860.829 ***
2	6	84.692 ***
1 × 2	24	89.636 ***
3	1	$1.943$
1 × 3	4	$1.624$
2 × 3	6	17.466 **
1 × 2 × 3	24	27.902 ***

*** and ** indicate “significance” at $0.005$ and $0.01$ levels respectively. Source: author’s own calculations.

, an ANOVA-like table, presents the results of our analysis. It gives the sources of variation of the general mean, the main effects and the interaction effects for our GSS (2008) data. It also presents the degrees of freedom and the sum of square for these effects. The significance level of these effects is indicated in *. From the table, both factors 1 and 2 have significant effects as well as a significant interaction. The interactions between factors 2 and 3 is also significant. Factor 3 had neither significant effects nor interactions with the exception of its interaction with factor 2.

We can thus conclude that the core of significance was in factors 1 and 2 (education and political party affiliation) and that both genders behave similarly.

Comparing Procedures

Our procedure is based on the asymptotic distribution given by

$$\sqrt{n}\left({\tilde{\mathit{p}}}_n-\mathit{p}\right)\sim \mathcal{N}\left(\mathbf{0},U\left(\mathit{p}\right)\right)$$

(79)

and by

$$\sqrt{n}\left(\mathit{l}\left({\tilde{\mathit{p}}}_n\right)-\mathit{l}\left(\tilde{\mathit{p}}\right)\right)\sim \mathcal{N}\left(\mathbf{0},J\left(\mathit{p}\right)U\left(\mathit{p}\right)J{\left(\mathit{p}\right)}^t\right)$$

(80)

where $J(·)$ is the Jacobian matrix of the gradients of $\mathit{l}(·)$ (the row vectors of $J(·)$ are the gradients of the components of $\mathit{l}(·)$):

This enabled us to, given an orthogonal partition:

$$R^m={⊞}_{j=1}^d{\nabla }_j,$$

(81)

where m is the number of probabilities, test the hypotheses:

$$H_{0,j}:\mathit{l}\left(\mathit{p}\right)\in {\nabla }_j^{\perp }.$$

(82)

In our numerical example, we have:

$$\begin{array}{c}\nabla (⌀):\mathrm{general}\mathrm{mean}\hfill \\ \nabla \left(\right\{1\left\}\right):\mathrm{null}\mathrm{effect}\mathrm{of}\mathrm{the}\mathrm{first}\mathrm{factor}\hfill \\ \nabla \left(\right\{2\left\}\right):\mathrm{null}\mathrm{effect}\mathrm{of}\mathrm{the}\mathrm{second}\mathrm{factor}\hfill \\ \nabla \left(\right\{3\left\}\right):\mathrm{null}\mathrm{effect}\mathrm{of}\mathrm{the}\mathrm{third}\mathrm{factor}\hfill \\ \nabla \left(\right\{1,2\left\}\right):\mathrm{null}\mathrm{interaction}\mathrm{of}\mathrm{the}\mathrm{first}\mathrm{and}\mathrm{second}\mathrm{factors}\hfill \\ \nabla \left(\right\{1,3\left\}\right):\mathrm{null}\mathrm{interaction}\mathrm{of}\mathrm{the}\mathrm{first}\mathrm{and}\mathrm{third}\mathrm{factors}\hfill \\ \nabla \left(\right\{2,3\left\}\right):\mathrm{null}\mathrm{interaction}\mathrm{of}\mathrm{the}\mathrm{second}\mathrm{and}\mathrm{third}\mathrm{factors}\hfill \\ \nabla \left(\right\{1,2,3\left\}\right):\mathrm{null}\mathrm{interaction}\mathrm{of}\mathrm{the}\mathrm{three}\mathrm{factors}\hfill \end{array}$$

(83)

This way, we overcame the requirement of using hierarchical models.

6. Conclusions

Multinomial models having their limit distribution given by $\sqrt{n}\left({\tilde{\mathit{p}}}_n-\mathit{p}\right)\sim \mathcal{N}\left(\mathbf{0},\mathit{U}\left(\mathit{p}\right)\right)$ with $rank\left(\mathit{U}\left(\mathit{p}\right)\right)=m-1$ and $\mathit{p}$ belonging to a compact set led us to establish the parametrized continuous mapping theorem (PCMT) to carry out inference for them. Namely, we obtained the confidence ellipsoids and confidence intervals. Moreover, we obtained chi-square tests for the hypothesis:

$$H_0\left(\mathit{A}\right):\mathit{Ap}=\mathbf{0}$$

which led to ANOVA-like inference for both multinomial and log-linear models. We pointed out that now hierarchical assumptions made on log-linear models are no longer necessary and all effects and interactions can be tested without any restrictions. Actually, this was used in the two numerical examples we presented. In addition to this, the replacement of F-tests by chi-square ones increases the power of our inference by replacing a finite number of degrees of freedom for the error by an infinity of them.