Asymptotic Theory for a Parameter Dimension-Split Estimation in Time Series Analysis for Multinomial Data

Abstract

The parameter space in a regression model for multinomial time series data contains the regression parameters those explain the effects of the time dependent covariates, and the dynamic dependence or category transition parameters those explain the influence of the past responses on the multinomial response at a given time. The estimation of the regression parameters can be negatively affected when higher dimension of the parameter space is considered specially for the transition parameters. In this paper we propose a parameter dimension-split approach where a conditional generalized quasi-likelihood (CGQL) estimating function is first developed for the dynamic dependence parameters in terms of unknown regression parameters which is exploited in the next step to develop an observed information matrix based maximum likelihood (ML) estimating equation for the main regression parameters. More specifically, this split approach helps to write the actual joint likelihood function of regression and dynamic dependence parameters as a likelihood function of regression parameters only by replacing the dynamic dependence parameters with their CGQL estimates obtained in the first step. As the time series length is generally large in practice, we have made sure that the proposed CGQL and ML estimators are asymptotically reliable, that is consistent for the respective parameters.

1. Introduction

Binary time series analysis is an important research topic in economics and statistics, among other areas. For example, the economic status (profit/loss) of a pharmaceutical industry may be recorded over the years along with certain exogenous explanatory covariates, such as type of industry, yearly advertising cost, and other research and development expenditures. It is likely that the binary profit status of an industry in a given year is correlated with the status of profits from previous years. It is of interest to know both (i) the effects of the time-dependent covariates, and (ii) the dynamic relationship among the responses over the years. Because obtaining the binary status may depend on certain latent variables, the binary time series data have been modeled and analyzed in a variety of ways over the last four decades, primarily in the fields of statistics and econometrics. Readers may refer to existing studies ([1,2,3,4,5,6,7,8,9,10,11,12]) on various binary time series models. More specifically, most of the models considered in these studies have either binary probit or logit forms. By the same token, the multinomial time series analysis is also an important research topic. In this case, as an extension to the binary response variable, one deals with a categorical response variable with more than two categories. For example, it may be more appropriate to classify the profit status of the industries into several categories, such as heavy loss, moderate loss, no loss, moderate profit, or healthy profit, and then examine the effects of exogenous covariates on such categorical responses collected over a long period of time. The correlations among categories over time are also of interest. This type of multinomial time series data has been analyzed mainly in statistics literature by a few authors such as refs. [13,14,15,16,17,18]. As far as the dynamic relationship is concerned, studies by refs. [16,18], for example, have considered a multinomial dynamic logit (MDL) model as a generalization of the binary logit time series model.

As far as the BDL (binary dynamic logits) model is concerned, it is constructed as follows. Let $\{y_t^{*},t=1,\dots ,T\}$ be a sequence of a latent variable, and a binary response $y_t$ is observed at time t using the relationship

$$y_t=\left\{\begin{array}{cc}1\hfill & \mathrm{if} y_t^{*}>0\hfill \\ 0\hfill & \mathrm{otherwise}.\hfill \end{array}\right.$$

(1)

Also, let $x_t={(x_{t1},\dots ,x_{t\mathsf{\ell }},\dots ,x_{t,p+1})}^{\prime }$ denote the $(p+1)$-dimensional exogenous explanatory covariate vector and $\beta ={({\beta }_0,{\beta }_1,\dots ,{\beta }_p)}^{\prime }$ denote the effect of $x_t$ on the binary response $y_t.$ Next, suppose that the latent variable $y_1^{*}$ in (1) follows a logistic distribution $f_L\left(y_1^{*}\right)$ ([19]) with mean $g_1^{*}=x_1^{\prime }\beta$ and variance ${\displaystyle \frac{{\pi }^2}{3}}$, whereas for $t=2,\dots ,T,y_t^{*}$ follows the same logistic distribution $f_L(·)$ but with mean $g_t^{*}=x_t^{\prime }\beta +\gamma y_{t-1}$ and variance ${\displaystyle \frac{{\pi }^2}{3}},$$\gamma$ being a lag 1 dynamic dependence parameter. It then follows from (1) that

$$\begin{array}{ccc}\hfill Pr(y_1=1)& =& {\int }_{-\infty }^{x_1^{\prime }\beta }{f}_L\left(y_1^{*}\right)d{y}_1^{*}=\exp \left(x_1^{\prime }\beta \right)/[1+\exp \left(x_1^{\prime }\beta \right)]\hfill \\ Pr(y_t=1|y_{t-1})\hfill & =& {\int }_{-\infty }^{x_t^{\prime }\beta +\gamma y_{t-1}}{f}_L\left(y_t^{*}\right)d{y}_t^{*}, \mathrm{for} t=2,\dots ,T\hfill \\ & =& \exp (x_t^{\prime }\beta +\gamma y_{t-1})/[1+(x_t^{\prime }\beta +\gamma y_{t-1})],\hfill \end{array}$$

(2)

(see also [6], p. 422). To understand the recursive mean $E\left[Y_t\right],$ variance $\mathrm{var}\left[Y_t\right],$ and lag ℓ correlations between $Y_t$ and $Y_{t-\mathsf{\ell }},$ produced by this BDL model (2), it is of interest to estimate $\beta$ and $\gamma .$ Note that there is no range restriction for these parameters.

The above mentioned MDL model is a generalization of the BDL model given by (2). For a discussion on this generalization, see, for example, the studies by refs. [16,17,18,20]. More specifically, the MDL (multinomial dynamic logits) model is constructed as follows: Let ${\mathbf{y}}_{\mathbf{t}}={(y_{t1},\dots ,y_{tc},\dots ,y_{t,C-1})}^{\prime }$ denote the $(C-1)$-dimensional multinomial response variable and for $c=1,\dots ,C-1,$

$${\mathbf{y}}_t^{\left(c\right)}={(y_{t1}^{\left(c\right)},\dots ,y_{tc}^{\left(c\right)},\dots ,y_{t,C-1}^{\left(c\right)})}^{\prime }={({01}_{c-1}^{\prime },1,{01}_{C-1-c}^{\prime })}^{\prime }\equiv {\delta }_{tc}$$

(3)

indicates that the multinomial response recorded at time t belongs to the cth category. For $c=C,$ one writes $y_t^{\left(C\right)}={\delta }_{tC}={01}_{C-1}.$ Here, and also in (3), for a scalar constant $c_0$, we use $c_0{1}_C$ for simplicity to represent $c_0\otimes 1_c,$ with ⊗ being the well-known Kronecker or direct product. This notation will also be used throughout the rest of the paper when needed. At any time point $t,$ let ${\mathbf{fi}}_c={({\beta }_{c0},{\beta }_{c1},\dots ,{\beta }_{cp})}^{\prime }$ denote the effect of $x_t$ on $y_t^{\left(c\right)}$ for $c=1,\dots ,C-1.$ Also, let ${\pi }_{\left(1\right)c}$ denote the marginal multinomial probability at time $t=1$ for the observation ${\mathbf{y}}_t$ to be in the cth category; and for $t=2,\dots ,T,$ let the transitional probability from the gth $(g=1,\dots ,C)$ category at time $t-1$ to the cth category at time $t,$ be denoted by ${\eta }_{t|t-1}^{\left(c\right)}\left(g\right).$ As an extension of the BDL model (2), one may then write the MDL model as

$$\begin{array}{ccc}\hfill P[{\mathbf{y}}_1={\mathbf{y}}_1^{\left(c\right)}]={\pi }_{\left(1\right)c}& =& \left\{\begin{array}{cc}{\displaystyle \frac{\exp \left(x_1^{\prime }{\mathbf{fi}}_c\right)}{1+{\sum }_{g=1}^{C-1}\exp \left(x_1^{\prime }{\mathbf{fi}}_g\right)}}\hfill & \mathrm{for} c=1,\dots ,C-1\hfill \\ {\displaystyle \frac{1}{1+{\sum }_{g=1}^{C-1}\exp \left(x_1^{\prime }{\mathbf{fi}}_g\right)}}\hfill & \mathrm{for} c=C\hfill \end{array}\right.\hfill \\ P\left({\mathbf{Y}}_t={\mathbf{y}}_t^{\left(c\right)}|{\mathbf{Y}}_{t-1}={\mathbf{y}}_{t-1}^{\left(g\right)}\right)\hfill & =& {\eta }_{t|t-1}^{\left(c\right)}\left(g\right) \mathrm{for} t=2,\dots ,T\hfill \\ & =& \left\{\begin{array}{cc}{\displaystyle \frac{\exp \left[x_t^{\prime }{\beta }_c+{\gamma }_c^{\prime }{\mathbf{y}}_{t-1}^{\left(g\right)}\right]}{1+{\sum }_{v=1}^{C-1}\exp \left[x_t^{\prime }{\beta }_v+{\gamma }_v^{\prime }{\mathbf{y}}_{t-1}^{\left(g\right)}\right]}},\hfill & \mathrm{for} c=1,\dots ,C-1\hfill \\ {\displaystyle \frac{1}{1+{\sum }_{v=1}^{C-1}\exp \left[x_t^{\prime }{\beta }_v+{\gamma }_v^{\prime }{\mathbf{y}}_{t-1}^{\left(g\right)}\right]}},\hfill & \mathrm{for} c=C,\hfill \end{array}\right.\hfill \end{array}$$

(4)

where ${\gamma }_c={({\gamma }_{c1},\dots ,{\gamma }_{cv},\dots ,{\gamma }_{c,C-1})}^{\prime }$ denotes the dynamic dependence parameters.

Note that for further notational convenience, one may re-express the transitional probabilities in (4) as

$$\begin{array}{c}{\eta }_{t|t-1}^{\left(c\right)}\left(g\right)=\left\{\begin{array}{cc}{\displaystyle \frac{\exp \left[x_t^{\prime }{\beta }_c+{\gamma }_c^{\prime }{\delta }_{(t-1)g}\right]}{1+{\sum }_{v=1}^{C-1}\exp \left[x_t^{\prime }{\beta }_v+{\gamma }_v^{\prime }{\delta }_{(t-1)g}\right]}},\hfill & \mathrm{for} c=1,\dots ,C-1\hfill \\ {\displaystyle \frac{1}{1+{\sum }_{v=1}^{C-1}\exp \left[x_t^{\prime }{\beta }_v+{\gamma }_v^{\prime }{\delta }_{(t-1)g}\right]}},\hfill & \mathrm{for} c=C,\hfill \end{array}\right.\hfill \end{array}$$

(5)

where for $t=2,\dots ,T,$ ${\delta }_{(t-1)g}$ through (1) has the formula

$${\delta }_{(t-1)g}=\left\{\begin{array}{cc}{[{01}_{g-1}^{\prime },1,{01}_{C-1-g}^{\prime }]}^{\prime }\hfill & \mathrm{for} g=1,\dots ,C-1\hfill \\ {01}_{C-1}\hfill & \mathrm{for} g=C.\hfill \end{array}\right.$$

Note that in (5), the category g occurred at time $t-1.$ Thus the category g depends on time $t-1,$ and ${\delta }_{(t-1)g}\equiv {\delta }_{g_{t-1}}.$ However, for simplicity, we use g for $g_{t-1}.$ For convenience, we use ${\beta }^{*}={({\beta }_1^{\prime },\dots ,{\beta }_c^{\prime },\dots ,{\beta }_{C-1}^{\prime })}^{\prime }:(p+1)(C-1)\times 1$ to represent all regression effects involved in the marginal and conditional probabilities given in (4) and (5). Similarly, we use ${\gamma }^{*}={({\gamma }_1^{\prime },\dots ,{\gamma }_c^{\prime },\dots ,{\gamma }_{C-1}^{\prime })}^{\prime }:{(C-1)}^2\times 1$ to represent all dynamic dependence parameters involved in the transitional probabilities (4) or (5).

We remark that as we explain below or more specifically in Section 2, the estimation of the aforementioned regression and dynamic dependence parameters through existing joint likelihood approach (for both parameters) may be negatively affected when the dimension of these parameter vectors is large. As a remedy in this paper we provide a dimension-split approach where a new likelihood function is constructed only for the main regression parameters by replacing the large dimensional dynamic dependence parameters in the joint likelihood function with their conditional estimates those will be obtained first using a conditional generalized quasi-likelihood (CGQL) approach conditional on unknown main regression parameters.

Turning back to the estimation importance, notice that the unconditional means $E\left[{\mathbf{Y}}_t\right],$ variances $\mathrm{var}\left[{\mathbf{Y}}_t\right],$ and pair-wise correlations $\mathrm{corr}[{\mathbf{Y}}_u,{\mathbf{Y}}_t],u<t,$ computed by exploiting the model (4) will involve ${\beta }^{*}$ and ${\gamma }^{*}.$ Thus, to understand these basic properties of the multinomial time series, it is of importance to obtain consistent estimates for ${\beta }^{*}$ and ${\gamma }^{*},$ at least asymptotically, that is, when $T\to \infty .$ As far as the formulas are concerned, the means as the functions of ${\beta }^{*}$ and ${\gamma }^{*}$ have the recursive relationship given by

$$\begin{array}{ccc}& & E\left[{\mathbf{Y}}_t\right]={\eta }_{\left(t\right|t-1)}\left(C\right)+\left[{\eta }_{\left(t\right|t-1),M}-{\eta }_{\left(t\right|t-1)}\left(C\right)1_{C-1}^{\prime }\right]E\left[{\mathbf{Y}}_{t-1}\right]\hfill \\ & =& {\tilde{\pi }}_{\left(t\right)}({\beta }^{*},{\gamma }^{*})={({\tilde{\pi }}_{\left(t\right)1},\dots ,{\tilde{\pi }}_{\left(t\right)c},\dots ,{\tilde{\pi }}_{\left(t\right)(C-1)})}^{\prime }, \mathrm{for} t=2,\dots ,T-1,\hfill \end{array}$$

(6)

where the expectation at the initial time $t=1$ has the formula

$$\begin{array}{ccc}E\left[{\mathbf{Y}}_1\right]\hfill & =& {[{\pi }_{\left(1\right)1},\dots ,{\pi }_{\left(1\right)c},\dots ,{\pi }_{\left(1\right)(C-1)}]}^{\prime }\hfill \\ & =& {\pi }_{\left(1\right)}\left({\beta }^{*}\right),\hfill \end{array}$$

where ${\pi }_{\left(1\right)c}$ is given by (4) for all $c=1,\dots ,C.$ In (6), ${\eta }_{\left(t\right|t-1)}\left(C\right)$ is the $(C-1)$-dimensional vector of conditional probabilities, given by

$${\eta }_{\left(t\right|t-1)}\left(C\right)={[{\eta }_{t|t-1}^{\left(1\right)}\left(C\right),\dots ,{\eta }_{t|t-1}^{\left(c\right)}\left(C\right)\dots ,{\eta }_{t|t-1}^{(C-1)}\left(C\right)]}^{\prime },$$

(7)

and ${\eta }_{\left(t\right|t-1),M}$ is the $(C-1)\times (C-1)$ matrix of conditional probabilities given by

$${\eta }_{\left(t\right|t-1),M}=\left({\eta }_{t|t-1}^{\left(c\right)}\left(g\right)\right)$$

(8)

where ${\eta }_{t|t-1}^{\left(c\right)}\left(g\right)$ is the $(c,g)$-th element of the matrix for $c=1,\dots ,C-1;$ $g=1,\dots ,C-1.$ Furthermore, similar to (6), the variances and covariances also have the recursive relationships and they are given by

$$\begin{array}{ccc}\mathrm{var}\left[{\mathbf{Y}}_t\right]\hfill & =& \mathrm{diag}[{\tilde{\pi }}_{\left(t\right)1}\left({\beta }^{*}\right),\dots ,{\tilde{\pi }}_{\left(t\right)c}({\beta }^{*},{\gamma }^{*}),\dots ,{\tilde{\pi }}_{\left(t\right)(C-1)}({\beta }^{*},{\gamma }^{*})]\hfill \\ & -& {\tilde{\pi }}_{\left(t\right)}({\beta }^{*},{\gamma }^{*}){\tilde{\pi }}_{\left(t\right)}^{\prime }({\beta }^{*},{\gamma }^{*})\hfill \\ & =& \left(\mathrm{cov}({\mathbf{Y}}_{tc},{\mathbf{Y}}_{tk})\right)=\left({\tilde{\sigma }}_{\left(tt\right)ck}({\beta }^{*},{\gamma }^{*})\right), c,k=1,\dots ,C-1\hfill \\ & =& {\tilde{\Sigma }}_{\left(tt\right)}({\beta }^{*},{\gamma }^{*}), \mathrm{for} t=1,\dots ,T\hfill \end{array}$$

(9)

$$\begin{array}{ccc}\mathrm{cov}[{\mathbf{Y}}_u,{\mathbf{Y}}_t]\hfill & =& {\Pi }_{s=u+1}^t\left[{\eta }_{\left(s\right|s-1),M}-{\eta }_{\left(s\right|s-1)}\left(C\right)1_{C-1}^{\prime }\right]\mathrm{var}\left[{\mathbf{Y}}_u\right], \mathrm{for} u<t,t=2,\dots ,T\hfill \\ & =& \left(\mathrm{cov}({\mathbf{Y}}_{uc},{\mathbf{Y}}_{tk})\right)=\left({\tilde{\sigma }}_{\left(ut\right)ck}({\beta }^{*},{\gamma }^{*})\right), c,k=1,\dots ,C-1\hfill \\ & =& {\tilde{\Sigma }}_{\left(ut\right)}({\beta }^{*},{\gamma }^{*}).\hfill \end{array}$$

(10)

As far as the inference is concerned, some authors such as [18] used a partial likelihood approach for the estimation of the regression and the dynamic dependence parameters $({\beta }^{*},{\gamma }^{*})$ under the assumption that the observable covariates $\{x_t, t=1,\dots ,T\}$ are random, and perhaps dependent on lagged values of the response variable. This approach is equivalent to the so-called conditional likelihood approach where the likelihood is obtained by conditioning the history of both covariates and responses. Consequently, one may easily obtain the information matrix conditional on the whole process history ([16], Equation (17), p. 364.) Notice, however, that the information matrix conditional on the entire history is not the same as the information matrix conditional on the covariate process; but, in many cases, it is the same as the observed information matrix, for example, when the covariate distribution does not involve lagged values of the response variable. Among others, the study by ref. [18] (Equations (17)–(19)) used a joint likelihood approach (see also [20]) to estimate the parameters ${\beta }^{*}$ and ${\gamma }^{*},$ where the likelihood estimating equation was solved by using the Fisher information matrix based on Newton’s iterative procedure. These authors found through an intensive simulation study that with a minimal dimension of the parameter space, both approaches work quite well even with relatively short-length series; however, when higher dimensions of the parameter space is considered, it was found that, although both methods require longer series to achieve acceptable levels of accuracy, the approach based on the observed information matrix to compute the standard errors of the parameter estimators performs relatively worse than the approach based on the Fisher information matrix. But obtaining the Fisher information matrix can be computationally involved.

Note that because both the observed and Fisher information matrix-based estimating equations produce similarly efficient estimates for the parameters when the dimension of the parameter space is minimal, and also because the computation of the Fisher information can be complex, the main objective of this paper is to use an observed information matrix-based estimation approach with minimal dimensions for the parameter space. More specifically, we first develop a consistent estimating function for the dynamic dependence parameter ${\gamma }^{*}$ as a function of the unknown regression effects ${\beta }^{*}.$ We denote this function as ${\widehat{\gamma }}^{*}\left({\beta }^{*}\right).$ We then estimate the regression parameters ${\beta }^{*}$ by exploiting the likelihood function $L({\beta }^{*},{\widehat{\gamma }}^{*}\left({\beta }^{*}\right)),$ say, instead of the joint likelihood function $L({\beta }^{*},{\gamma }^{*}).$ Thus, in this approach, the estimation of ${\beta }^{*}$ will not depend on the dimension of the dynamic dependence parameters ${\gamma }^{*}.$ This reduced dimension-based estimation approach using the observed information matrix is further elucidated in Section 2. The asymptotic theory for this dimension-reduction approach is discussed in Section 3. Finally, concluding remarks are given in Section 4.

2. Estimation of Parameters: A Dimension-Reduction Approach

Because ${\beta }^{*}={({\beta }_1^{\prime },\dots ,{\beta }_{C-1}^{\prime })}^{\prime }:(C-1)(p+1)\times 1,$ and ${\gamma }^{*}={({\gamma }_1^{\prime },\dots ,{\gamma }_{C-1}^{\prime })}^{\prime }:{(C-1)}^2\times 1,$ the dimension of the parameter space depends on both p and $C.$ Customarily, the joint estimation of ${\beta }^{*}$ and ${\gamma }^{*},$ that is, $\theta ={({{\beta }^{*}}^{\prime },{{\gamma }^{*}}^{\prime })}^{\prime }$, is performed by maximizing the likelihood function with regard to $\theta ,$ where the likelihood function under the model (4) has the form

$$\begin{array}{ccc}L\left(\theta \right)=L({\beta }^{*},{\gamma }^{*})\hfill & =& {\Pi }_{c=1}^C{\left[{\pi }_{\left(1\right)c}({\beta }^{*};x_1)\right]}^{y_{1c}}\hfill \\ & \times & {\Pi }_{t=2}^T{\Pi }_{c=1}^C{\Pi }_{g=1}^C{\left[{\eta }_{t|t-1}^{\left(c\right|g)}({\beta }^{*},{\gamma }^{*};x_t)\right]}^{y_{tc}},\hfill \end{array}$$

(11)

where ${\pi }_{\left(1\right)c}\left({\beta }^{*}\right)$ and ${\eta }_{t|t-1}^{\left(c\right|g)}({\beta }^{*},{\gamma }^{*})$ are marginal (at $t=1$) and transitional probabilities, respectively. The solution of the log likelihood estimating equation, namely ${\displaystyle \frac{\partial LogL\left(\theta \right)}{\partial \theta }}=0$, may be obtained by solving the Hessian or Fisher information matrix-based iterative equations. More specifically, the Hessian matrix-based iterative equation has the form

$$\widehat{\theta }(r+1)=\widehat{\theta }\left(r\right)+{\left[{\left\{{\displaystyle \frac{{\partial }^{2}LogL\left(\theta \right)}{\partial \theta \partial {\theta }^{\prime }}}\right\}}^{-1}{\displaystyle \frac{\partial LogL\left(\theta \right)}{\partial \theta }}\right]}_{|\theta =\widehat{\theta }\left(r\right)}$$

(12)

(see [16]), and the Fisher information matrix-based equation has the form

$$\widehat{\theta }(r+1)=\widehat{\theta }\left(r\right)+{\left[{\left\{E{\displaystyle \frac{{\partial }^{2}LogL\left(\theta \right)}{\partial \theta \partial {\theta }^{\prime }}}\right\}}^{-1}{\displaystyle \frac{\partial LogL\left(\theta \right)}{\partial \theta }}\right]}_{|\theta =\widehat{\theta }\left(r\right)}$$

(13)

(see [18]). Under the assumption that the Hessian matrix is positive semi-definite, some authors, such as [14] (Equations (4.1) and (4.4)), studied the asymptotic properties (as $T\to \infty$) of the likelihood estimator obtained from (12). When the dimension of the parameter ($\theta$) space is large, however, the authors of [18] found that this type of Hessian matrix-based likelihood estimate of $\theta$ performs relatively worse for moderately large $T,$ as compared to the Fisher information matrix-based estimate (13). But, obtaining the Fisher information matrix is algebraically involved. When the dimension of the parameter space was small, both Hessian and Fisher information matrix-based estimates were found to work almost the same. For this reason, in this paper, we consider a dimension-splitting or reduction approach for the parameter space, so that the Hessian matrix-based likelihood estimates can be used even when T is not infinitely large. More specifically, we develop a conditional generalized quasi-likelihood (CGQL) estimating function for ${\gamma }^{*}$ as a function of unknown ${\beta }^{*}.$ This estimating function may be denoted by ${\widehat{\gamma }}^{*}\left({\beta }^{*}\right).$ We then use this estimating function and construct a modified likelihood function for ${\beta }^{*}$ as $L({\beta }^{*},{\widehat{\gamma }}^{*}\left({\beta }^{*}\right)),$ so that the estimation of ${\beta }^{*}$ does not depend on the dimension of ${\gamma }^{*}.$ Also, this approach will provide a different asymptotic theory than the one in [14] for the estimator of the main regression parameter ${\beta }^{*}.$ The CGQL cum-modified maximum likelihood estimation (MMLE) approach is provided in Section 2.1 and Section 2.2 below, and the asymptotic theory for the estimates is given in Section 3.

2.1. CGQL Estimating Function for Dynamic Dependence Parameters $\left({\gamma }^{*}\right)$ as a Function of Unknown ${\beta }^{*}$

Notice from (4) that conditional on ${\mathbf{y}}_{t-1},$ the response vector ${\mathbf{y}}_t={(y_{t1},\dots ,y_{tc},\dots ,y_{t,C-1})}^{\prime }$ follows a multinomial distribution with $C-1$-dimensional mean vector

$$\begin{array}{c}E\left[{\mathbf{Y}}_t\right|{\mathbf{y}}_{t-1}]={\eta }_{t|t-1}({\beta }^{*},{\gamma }^{*}),\hfill \end{array}$$

(14)

and $(C-1)\times (C-1)$ covariance matrix

$$\begin{array}{ccc}\mathrm{cov}\left[{\mathbf{Y}}_t\right|{\mathbf{y}}_{t-1}]\hfill & =& D_{{\eta }_{t|t-1}}-{\eta }_{t|t-1}({\beta }^{*},{\gamma }^{*}){\eta }_{t|t-1}^{\prime }({\beta }^{*},{\gamma }^{*})\hfill \\ & =& {\Sigma }_{t|t-1}({\beta }^{*},{\gamma }^{*}), \left(say\right),\hfill \end{array}$$

(15)

where

$${\eta }_{t|t-1}({\beta }^{*},{\gamma }^{*})={[{\eta }_{t|t-1}^{\left(1\right)}({\beta }^{*},{\gamma }^{*}),\dots ,{\eta }_{t|t-1}^{\left(c\right)}({\beta }^{*},{\gamma }^{*}),\dots ,{\eta }_{t|t-1}^{(C-1)}({\beta }^{*},{\gamma }^{*})]}^{\prime }:(C-1)\times 1,$$

with ${\eta }_{t|t-1}^{\left(c\right)}({\beta }^{*},{\gamma }^{*})=\exp \left[x_t^{\prime }{\beta }_c+{\gamma }_c^{\prime }{\mathbf{y}}_{t-1}\right]/\left[1+{\sum }_{v=1}^{C-1}\exp \left[x_t^{\prime }{\beta }_v+{\gamma }_v^{\prime }{\mathbf{y}}_{t-1}\right]\right],$ for $c=1,\dots ,C-1.$ The diagonal matrix in (15) has the form

$$D_{{\eta }_{t|t-1}}=\mathrm{diag}[{\eta }_{t|t-1}^{\left(1\right)}({\beta }^{*},{\gamma }^{*}),\dots ,{\eta }_{t|t-1}^{\left(c\right)}({\beta }^{*},{\gamma }^{*}),\dots ,{\eta }_{t|t-1}^{(C-1)}({\beta }^{*},{\gamma }^{*})].$$

In notation, for $t=2,\dots ,T,$ we write this multinomial distribution of ${\mathbf{Y}}_t$ as

$${\mathbf{Y}}_t|{\mathbf{y}}_{t-1}\sim Mult\left({\eta }_{t|t-1}({\beta }^{*},{\gamma }^{*}),{\Sigma }_{t|t-1}({\beta }^{*},{\gamma }^{*})\right).$$

(16)

Now, to develop a CGQL estimating function ${\widehat{\gamma }}^{*}\left({\beta }^{*}\right)$ for ${\gamma }^{*},$ as a generalization of the QL approach of [21], we follow the GQL approach in [22] and exploit the conditional mean vector ${\eta }_{t|t-1}({\beta }^{*},{\gamma }^{*})$ and the conditional covariance matrix ${\Sigma }_{t|t-1}({\beta }^{*},{\gamma }^{*}).$ For this purpose, suppose that the following two assumptions hold.

Assumption 1.

Consider the conditional probability function

$${\eta }_{t|t-1}^{\left(c\right)}({\beta }^{*},{\gamma }^{*})=\exp \left[x_t^{\prime }{\beta }_c+{\gamma }_c^{\prime }{\mathbf{y}}_{t-1}\right]/\left[1+{\displaystyle \sum _{v=1}^{C-1}}\exp \left[x_t^{\prime }{\beta }_v+{\gamma }_v^{\prime }{\mathbf{y}}_{t-1}\right]\right]$$

defined in (15) for a categorical observation to be in c-th category at time t which weights ${\gamma }_c={({\gamma }_{c1},\dots ,{\gamma }_{c(C-1)})}^{\prime }$ with all possible ${\gamma }_h={({\gamma }_{h1},\dots ,{\gamma }_{h(C-1)})}^{\prime }, h=1,\dots ,C-1$ that could occur at time $(t-1).$ We assume that this weighted probability function is continuous, that is, ${\displaystyle \frac{\partial {\eta }_{t|t-1}^{\left(c\right)}({\beta }^{*},{\gamma }^{*})}{\partial {\gamma }_h}}$ exists for all $c,h=1,\dots ,C-1.$

Assumption 2.

The second-order derivative matrix ${\displaystyle \frac{\partial {\eta }_{t|t-1}^{\prime }({\beta }^{*},{\gamma }^{*})}{\partial {\gamma }^{*}}}{\displaystyle \frac{\partial {\eta }_{t|t-1}({\beta }^{*},{\gamma }^{*})}{\partial {{\gamma }^{*}}^{\prime }}}$ is bounded and positive definite.

Proposition 1.

When the above two assumptions hold, the CGQL estimator ${\widehat{\gamma }}_{CGQL}^{*}\left({\beta }^{*}\right)$ for ${\gamma }^{*}$ may be obtained by using the iterative equation

$$\begin{array}{ccc}& & {\widehat{\gamma }}_{CGQL}^{*}\left({\beta }^{*}\right)(r+1)={\widehat{\gamma }}_{CGQL}^{*}\left({\beta }^{*}\right)\left(r\right)\hfill \\ & +& \left[{\left\{{\displaystyle \sum _{t=2}^T}{\displaystyle \frac{\partial {\eta }_{t|t-1}^{\prime }({\beta }^{*},{\gamma }^{*})}{\partial {\gamma }^{*}}}{{\Sigma }^{-1}}_{t|t-1}({\beta }^{*},{\gamma }^{*}){\displaystyle \frac{\partial {\eta }_{t|t-1}({\beta }^{*},{\gamma }^{*})}{\partial {{\gamma }^{*}}^{\prime }}}\right\}}^{-1}\right.\hfill \\ & \times & {\left.\left\{{\displaystyle \sum _{t=2}^T}{\displaystyle \frac{\partial {\eta }_{t|t-1}^{\prime }({\beta }^{*},{\gamma }^{*})}{\partial {\gamma }^{*}}}{{\Sigma }^{-1}}_{t|t-1}({\beta }^{*},{\gamma }^{*})[{\mathbf{y}}_t-{\eta }_{t|t-1}({\beta }^{*},{\gamma }^{*})]\right\}\right]}_{|{\gamma }^{*}={\widehat{\gamma }}_{CGQL}^{*}\left(r\right)},\hfill \end{array}$$

(17)

Proof of Proposition 1.

This proposition follows from the fact that under the model (4), one may write the GQL estimating equation for ${\gamma }^{*}$ as

$${\displaystyle \sum _{t=2}^T}{\displaystyle \frac{\partial {\eta }_{t|t-1}^{\prime }({\beta }^{*},{\gamma }^{*})}{\partial {\gamma }^{*}}}{{\Sigma }^{-1}}_{t|t-1}({\beta }^{*},{\gamma }^{*})[{\mathbf{y}}_t-{\eta }_{t|t-1}({\beta }^{*},{\gamma }^{*})]=0$$

(18)

 □

Lemma 1.

In Assumption 2, the ${(C-1)}^2\times (C-1)$ derivative matrix ${\displaystyle \frac{\partial {\eta }_{t|t-1}^{\prime }({\beta }^{*},{\gamma }^{*})}{\partial {\gamma }^{*}}}$ has the computational formula

$$\begin{array}{ccc}& & {\displaystyle \frac{\partial {\eta }_{t|t-1}^{\prime }({\beta }^{*},{\gamma }^{*})}{\partial {\gamma }^{*}}}=[{\displaystyle \frac{\partial {\eta }_{t|t-1}^{\left(1\right)}({\beta }^{*},{\gamma }^{*})}{\partial {\gamma }^{*}}},\dots ,{\displaystyle \frac{\partial {\eta }_{t|t-1}^{\left(c\right)}({\beta }^{*},{\gamma }^{*})}{\partial {\gamma }^{*}}},\dots ,{\displaystyle \frac{\left(\begin{array}{c}{\eta }_{t|t-1}^{(C-1)}({\beta }^{*},{\gamma }^{*})\end{array}\right)}{\partial {\gamma }^{*}}}]\hfill \\ & =& \left(\begin{array}{ccc}{\eta }_{t|t-1}^{\left(1\right)}({\delta }_{(t-1)1}-{\eta }_{t|t-1})& \dots & {\eta }_{t|t-1}^{(C-1)}({\delta }_{(t-1)(C-1)}-{\eta }_{t|t-1})\end{array}\right)\otimes {\mathbf{y}}_{t-1}\hfill \\ & =& {\eta }_{\left(t\right|t-1),M}^{*}({\beta }^{*},{\gamma }^{*})\otimes {\mathbf{y}}_{t-1},\hfill \end{array}$$

(19)

where ${\eta }_{t|t-1}^{*}({\beta }^{*},{\gamma }^{*})$ denotes the matrix constructed by using the $(C-1)$-dimensional column vectors $\left[{\eta }_{t|t-1}^{\left(c\right)}({\delta }_{(t-1)c}-{\eta }_{t|t-1})\right]$ for all $c=1,\dots ,C-1.$

Proof. 

Because ${\eta }_{t|t-1}^{\left(c\right)}({\beta }^{*},{\gamma }^{*})=\exp \left[x_t^{\prime }{\beta }_c+{\gamma }_c^{\prime }{\mathbf{y}}_{t-1}\right]/\left[1+{\sum }_{v=1}^{C-1}\exp \left[x_t^{\prime }{\beta }_v+{\gamma }_v^{\prime }{\mathbf{y}}_{t-1}\right]\right],$ for $c=1,\dots ,C-1,$ by (5), it then follows that

$$\begin{array}{ccc}{\displaystyle \frac{\partial {\eta }_{t|t-1}^{\left(c\right)}}{\partial {\gamma }_h}}\hfill & =& \left\{\begin{array}{cc}{y}_{t-1}{\eta }_{t|t-1}^{\left(c\right)}[1-{\eta }_{t|t-1}^{\left(c\right)}]\hfill & \mathrm{for} h=c;h,c=1,\dots ,C-1\hfill \\ -y_{t-1}{\eta }_{t|t-1}^{\left(c\right)}{\eta }_{t|t-1}^{\left(h\right)}\hfill & \mathrm{for} h\ne c;h,c=1,\dots ,C-1.\hfill \end{array}\right.\hfill \end{array}$$

(20)

Next, because ${\gamma }^{*}={({\gamma }_1^{\prime },\dots ,{\gamma }_h^{\prime },\dots ,{\gamma }_{C-1}^{\prime })}^{\prime },$ one obtains

$$\begin{array}{ccc}{\displaystyle \frac{\partial {\eta }_{t|t-1}^{\left(c\right)}}{\partial {\gamma }^{*}}}\hfill & =& \left(\begin{array}{c}-{\eta }_{t|t-1}^{\left(1\right)}{\eta }_{t|t-1}^{\left(c\right)}\\ ⋮\\ {\eta }_{t|t-1}^{\left(c\right)}[1-{\eta }_{t|t-1}^{\left(c\right)}]\\ ⋮\\ -{\eta }_{t|t-1}^{(C-1)}{\eta }_{t|t-1}^{\left(c\right)}\end{array}\right)\otimes {\mathbf{y}}_{t-1}:(C-1)(C-1)\times 1\hfill \\ & =& \left[{\eta }_{t|t-1}^{\left(c\right)}({\delta }_{(t-1)c}-{\eta }_{t|t-1})\right]\otimes {\mathbf{y}}_{t-1},\hfill \end{array}$$

(21)

with

$${\delta }_{(t-1)c}=\left\{\begin{array}{cc}{[{01}_{c-1}^{\prime },1,{01}_{C-1-c}^{\prime }]}^{\prime }\hfill & \mathrm{for} c=1,\dots ,C-1\hfill \\ {01}_{C-1}\hfill & \mathrm{for} c=C.\hfill \end{array}\right.$$

The lemma, i.e., Equation (19), then follows from (21). □

Note that for the asymptotic studies to be discussed in Section 3, it is convenient to use (19) in (17) and re-express the iterative Equation (17) for ${\gamma }^{*}$ as

$$\begin{array}{ccc}& & {\widehat{\gamma }}_{CGQL}^{*}\left({\beta }^{*}\right)(r+1)={\widehat{\gamma }}_{CGQL}^{*}\left({\beta }^{*}\right)\left(r\right)\hfill \\ & +& \left[{\left\{{\displaystyle \sum _{t=2}^T}\left[({\eta }_{\left(t\right|t-1),M}^{*}({\beta }^{*},{\gamma }^{*})\otimes {\mathbf{y}}_{t-1}){{\Sigma }^{-1}}_{t|t-1}({\beta }^{*},{\gamma }^{*}){({\eta }_{\left(t\right|t-1),M}^{*}({\beta }^{*},{\gamma }^{*})\otimes {\mathbf{y}}_{t-1})}^{\prime }\right]\right\}}^{-1}\right.\hfill \\ & \times & {\left.\left\{{\displaystyle \sum _{t=2}^T}({\eta }_{\left(t\right|t-1),M}^{*}({\beta }^{*},{\gamma }^{*})\otimes {\mathbf{y}}_{t-1}){{\Sigma }^{-1}}_{t|t-1}({\beta }^{*},{\gamma }^{*})[{\mathbf{y}}_t-{\eta }_{t|t-1}({\beta }^{*},{\gamma }^{*})]\right\}\right]}_{|\gamma =\widehat{\gamma }\left(r\right)}.\hfill \end{array}$$

(22)

Let ${\widehat{\gamma }}_{CGQL}^{*}\left({\beta }^{*}\right)$ denote the moment estimating function of ${\gamma }^{*}$ obtained via (22).

2.2. Modified Maximum Likelihood (MML) Estimation for ${\beta }^{*}$ Using Observed Information

Notice that because ${\gamma }^{*}$ can be estimated by ${\widehat{\gamma }}_{CGQL}^{*}\left({\beta }^{*}\right)$ using the estimating function given in (22), one is not concerned about the dimension of ${\gamma }^{*}$ for ${\beta }^{*}$ estimation. Thus in a reduced dimension setup, we may estimate ${\beta }^{*}$ by exploiting the modified likelihood function for ${\beta }^{*}$, which is obtained as follows by replacing ${\gamma }^{*}$ with ${\widehat{\gamma }}_{CGQL}^{*}\left({\beta }^{*}\right)$ in the joint likelihood function $L({\beta }^{*},{\gamma }^{*}),$ for ${\beta }^{*}$ and ${\gamma }^{*}.$ More specifically, the modified likelihood function for ${\beta }^{*},$ by (4), may be written as

$$\begin{array}{ccc}L({\beta }^{*},{\widehat{\gamma }}_{CGQL}^{*}\left({\beta }^{*}\right))\hfill & =& {\Pi }_{c=1}^C{\left[{\pi }_{\left(1\right)c}({\beta }^{*};x_1)\right]}^{y_{1c}}\hfill \\ & \times & {\Pi }_{t=2}^T{\Pi }_{c=1}^C{\Pi }_{g=1}^C{\left[{\tilde{\eta }}_{t|t-1}^{\left(c\right|g)}({\beta }^{*},{\widehat{\gamma }}_{CGQL}^{*}\left({\beta }^{*}\right);x_t)\right]}^{y_{tc}},\hfill \end{array}$$

(23)

where ${\tilde{\eta }}_{t|t-1}^{\left(c\right|g)}(·)$ denotes the partially estimated dynamic probability function obtained from the true dynamic probability function ${\eta }_{t|t-1}^{\left(c\right|g)}(·)$ defined in (4), by replacing ${\gamma }^{*}$ with ${\widehat{\gamma }}_{CGQL}^{*}\left({\beta }^{*}\right).$ Thus,

$$\begin{array}{ccc}& & {\tilde{\eta }}_{t|t-1}^{\left(c\right|g)}({\beta }^{*},{\widehat{\gamma }}_{CGQL}^{*}\left({\beta }^{*}\right);x_t)\hfill \\ & =& \left\{\begin{array}{cc}{\displaystyle \frac{\exp \left[x_t^{\prime }{\beta }_c+{\widehat{\gamma }}_{c,CGQL}^{\prime }\left({\beta }^{*}\right){\delta }_{(t-1)g}\right]}{1+{\sum }_{v=1}^{C-1}\exp \left[x_t^{\prime }{\beta }_v+{\widehat{\gamma }}_{v,CGQL}^{\prime }\left({\beta }^{*}\right){\delta }_{(t-1)g}\right]}},\hfill & \mathrm{for} c=1,\dots ,C-1\hfill \\ {\displaystyle \frac{1}{1+{\sum }_{v=1}^{C-1}\exp \left[x_t^{\prime }{\beta }_v+{\widehat{\gamma }}_{v,CGQL}^{\prime }\left({\beta }^{*}\right){\delta }_{(t-1)g_j}\right]}},\hfill & \mathrm{for} c=C.\hfill \end{array}\right.\hfill \end{array}$$

(24)

We remark that as ${\widehat{\gamma }}_{CGQL}^{*}\left({\beta }^{*}\right)$ from (22) has an implicit functional form, the construction of the likelihood estimating equation for ${\beta }^{*}$ encounters a computational problem because of the difficulty in obtaining ${\displaystyle \frac{\partial {\widehat{\gamma }}_{CGQL}^{*}\left({\beta }^{*}\right)}{\partial {\beta }^{*}}}$ from an implicit function. However, the likelihood estimating equation for ${\beta }^{*}$ involving ${\displaystyle \frac{\partial {\widehat{\gamma }}_{CGQL}^{*}\left({\beta }^{*}\right)}{\partial {\beta }^{*}}}$ may be computed as follows:

Likelihood estimating equation for ${\mathit{\beta }}^{*}$: We follow (23) and write this estimating equation for ${\beta }^{*}$ as

$$\begin{array}{ccc}& & {\displaystyle \frac{\partial Log L({\beta }^{*},{\widehat{\gamma }}_{CGQL}^{*}\left({\beta }^{*}\right))}{\partial {\beta }^{*}}}={\displaystyle \sum _{c=1}^C}{\displaystyle \frac{y_{1c}}{{\pi }_{\left(1\right)c}\left({\beta }^{*}\right)}}{\displaystyle \frac{\partial {\pi }_{\left(1\right)c}\left({\beta }^{*}\right)}{\partial {\beta }^{*}}}\hfill \\ & +& {\displaystyle \sum _{t=2}^T}{\displaystyle \sum _{g=1}^C}{\displaystyle \sum _{c=1}^C}\left[{\displaystyle \frac{y_{tc}}{{\tilde{\eta }}_{t|t-1}^{\left(c\right|g)}({\beta }^{*},{\widehat{\gamma }}_{CGQL}^{*}\left({\beta }^{*}\right))}}{\displaystyle \frac{\partial {\tilde{\eta }}_{t|t-1}^{\left(c\right|g)}({\beta }^{*},{\widehat{\gamma }}_{CGQL}^{*}\left({\beta }^{*}\right))}{\partial {\beta }^{*}}}\right]=0,\hfill \end{array}$$

(25)

where ${\pi }_{\left(1\right)c}\left({\beta }^{*}\right)$ and ${\tilde{\eta }}_{t|t-1}^{\left(c\right|g)}({\beta }^{*},{\widehat{\gamma }}_{CGQL}^{*}\left({\beta }^{*}\right))$ are given by (4) and (23), respectively. Their derivatives with respect to ${\beta }^{*}$ needed for (24) are given in the following two Lemmas.

Lemma 2.

Computation of ${\displaystyle \frac{\partial {\pi }_{\left(1\right)c}\left({\beta }^{*}\right)}{\partial {\beta }^{*}}}.$ This derivative has the formula

$${\displaystyle \frac{\partial {\pi }_{\left(1\right)c}\left({\beta }^{*}\right)}{\partial {\beta }^{*}}}=\left[{\pi }_{\left(1\right)c}\left({\beta }^{*}\right)({\delta }_{\left(1\right)c}-{\pi }_{\left(1\right)}\left({\beta }^{*}\right))\right]\otimes x_1:(C-1)(p+1)\times 1,$$

(26)

where

$${\delta }_{\left(1\right)c}=\left\{\begin{array}{cc}{[{01}_{c-1}^{\prime },1,{01}_{C-1-c}^{\prime }]}^{\prime }\hfill & \mathit{for} c=1,\dots ,C-1\hfill \\ {01}_{C-1}\hfill & \mathit{for} c=C.\hfill \end{array}\right.$$

Proof. 

The proof is obvious because ${\pi }_{\left(1\right)}\left({\beta }^{*}\right)={({\pi }_{\left(1\right)1}\left({\beta }^{*}\right),\dots ,{\pi }_{\left(1\right)1c}\left({\beta }^{*}\right),\dots ,{\pi }_{\left(1\right)(C-1)}\left({\beta }^{*}\right))}^{\prime },$ with

$${\pi }_{\left(1\right)c}\left({\beta }^{*}\right)=\left\{\begin{array}{cc}{\displaystyle \frac{\exp \left(x_1^{\prime }{\beta }_c\right)}{1+{\sum }_{g=1}^{C-1}\exp \left(x_1^{\prime }{\beta }_g\right)}}\hfill & \mathrm{for} c=1,\dots ,C-1;\hfill \\ {\displaystyle \frac{1}{1+{\sum }_{g=1}^{C-1}\exp \left(x_1^{\prime }{\beta }_g\right)}}\hfill & \mathrm{for} c=C,\hfill \end{array}\right.$$

by (4). □

Lemma 3.

Computation of ${\displaystyle \frac{\partial {\tilde{\eta }}_{t|t-1}^{\left(c\right|g)}({\beta }^{*},{\widehat{\gamma }}_{CGQL}^{*}\left({\beta }^{*}\right))}{\partial {\beta }^{*}}}:(p+1)(C-1)\times 1.$ The computation of this derivative matrix requires the formula for the derivative matrix ${\displaystyle \frac{\partial \left[{\widehat{\gamma }}_{c,CGQL}^{\prime }\left({\beta }^{*}\right)\right]}{\partial {\beta }^{*}}}:(p+1)(C-1)\times (C-1),$ for all $c=1,\dots ,C-1,$ which can be derived from the formula for the derivative matrix ${\displaystyle \frac{\partial \left[{\widehat{\gamma }}_{CGQL}^{{*}^{\prime }}\left({\beta }^{*}\right)\right]}{\partial \beta }}:(p+1)(C-1)\times {(C-1)}^2$ as follows:

$$\begin{array}{ccc}& & {\displaystyle \frac{\partial \left[{\widehat{\gamma }}_{CGQL}^{{*}^{\prime }}\left({\beta }^{*}\right)\right]}{\partial {\beta }^{*}}}\hfill \\ & =& -\left[\left\{{\displaystyle \sum _{t=2}^T}({\eta }_{\left(t\right|t-1),M}^{*}({\beta }^{*},{\gamma }^{*})\otimes x_t){{\Sigma }^{-1}}_{t|t-1}({\beta }^{*},{\gamma }^{*})({\eta }_{\left(t\right|t-1),M}^{*}({\beta }^{*},{\gamma }^{*})\otimes {\mathbf{y}}_{t-1}^{\prime })\right\}\right.\hfill \\ & \times & \left.{\left\{{\displaystyle \sum _{t=2}^T}\left[({\eta }_{\left(t\right|t-1),M}^{*}({\beta }^{*},{\gamma }^{*})\otimes {\mathbf{y}}_{t-1}){{\Sigma }^{-1}}_{t|t-1}({\beta }^{*},{\gamma }^{*}){({\eta }_{\left(t\right|t-1),M}^{*}({\beta }^{*},{\gamma }^{*})\otimes {\mathbf{y}}_{t-1})}^{\prime }\right]\right\}}^{-1}\right]\hfill \\ & =& [{\displaystyle \frac{\partial \left[{\widehat{\gamma }}_{1,CGQL}^{\prime }\left({\beta }^{*}\right)\right]}{\partial {\beta }^{*}}},\dots ,{\displaystyle \frac{\partial \left[{\widehat{\gamma }}_{c,CGQL}^{\prime }\left({\beta }^{*}\right)\right]}{\partial {\beta }^{*}}},\hfill \\ & & \dots ,{\displaystyle \frac{\partial \left[{\widehat{\gamma }}_{(C-1),CGQL}^{\prime }\left({\beta }^{*}\right)\right]}{\partial {\beta }^{*}}}{]:(p+1)(C-1)\times (C-1)}^2.\hfill \end{array}$$

(27)

Proof. 

Because the CGQL estimating function for ${\gamma }^{*}$, that is, ${\widehat{\gamma }}_{CGQL}^{*}\left({\beta }^{*}\right),$ is obtained from (22) at its final iteration stage, the estimating function has the form

$$\begin{array}{ccc}& & {\widehat{\gamma }}_{CGQL}^{*}\left({\beta }^{*}\right)={\gamma }^{*}\hfill \\ & +& \left[{\left\{{\displaystyle \sum _{t=2}^T}\left[({\eta }_{\left(t\right|t-1),M}^{*}({\beta }^{*},{\gamma }^{*})\otimes {\mathbf{y}}_{t-1}){{\Sigma }^{-1}}_{t|t-1}({\beta }^{*},{\gamma }^{*}){({\eta }_{\left(t\right|t-1),M}^{*}({\beta }^{*},{\gamma }^{*})\otimes {\mathbf{y}}_{t-1})}^{\prime }\right]\right\}}^{-1}\right.\hfill \\ & \times & \left.\left\{{\displaystyle \sum _{t=2}^T}({\eta }_{\left(t\right|t-1),M}^{*}({\beta }^{*},{\gamma }^{*})\otimes {\mathbf{y}}_{t-1}){{\Sigma }^{-1}}_{t|t-1}({\beta }^{*},{\gamma }^{*})[{\mathbf{y}}_t-{\eta }_{t|t-1}({\beta }^{*},{\gamma }^{*})]\right\}\right].\hfill \end{array}$$

(28)

The lemma now follows, first because the ${\beta }^{*}$ involved in the first derivative as well as in the inverse covariance matrix ${{\Sigma }^{-1}}_{t|t-1}({\beta }^{*},{\gamma }^{*})$ is treated to be known from the previous, i.e., from the second-last iteration, and next because by a similar operation as (19), the derivative of the second term in (28) follows from the formula

$$\begin{array}{ccc}& & {\displaystyle \frac{\partial {\eta }_{t|t-1}^{\prime }({\beta }^{*},{\gamma }^{*})}{\partial {\beta }^{*}}}=[{\displaystyle \frac{\partial {\eta }_{t|t-1}^{\left(1\right)}({\beta }^{*},{\gamma }^{*})}{\partial {\beta }^{*}}},\dots ,{\displaystyle \frac{\partial {\eta }_{t|t-1}^{\left(c\right)}({\beta }^{*},{\gamma }^{*})}{\partial {\beta }^{*}}},\dots ,{\displaystyle \frac{\partial {\eta }_{t|t-1}^{(C-1)}({\beta }^{*},{\gamma }^{*})}{\partial {\beta }^{*}}}]\hfill \\ & =& \left(\begin{array}{ccc}{\eta }_{t|t-1}^{\left(1\right)}({\delta }_{(t-1)1}-{\eta }_{t|t-1})& \dots & {\eta }_{t|t-1}^{(C-1)}({\delta }_{(t-1)(C-1)}-{\eta }_{t|t-1})\end{array}\right)\otimes x_t\hfill \\ & =& {\eta }_{\left(t\right|t-1),M}^{*}({\beta }^{*},{\gamma }^{*})\otimes x_t,(C-1)(p+1)\times (C-1).\hfill \end{array}$$

(29)

 □

Lemma 4.

Computation of ${\displaystyle \frac{\partial {\tilde{\eta }}_{t|t-1}^{\left(c\right|g)}({\beta }^{*},{\widehat{\gamma }}_{CGQL}^{*}\left({\beta }^{*}\right))}{\partial {\beta }^{*}}}:(p+1)(C-1)\times 1$ (continued). This derivative has the formula given by

$$\begin{array}{ccc}{\displaystyle \frac{\partial {\tilde{\eta }}_{t|t-1}^{\left(c\right|g)}({\beta }^{*},{\widehat{\gamma }}_{CGQL}^{*}\left({\beta }^{*}\right))}{\partial {\beta }^{*}}}\hfill & =& {\tilde{\eta }}_{t|t-1}^{\left(c\right|g)}({\beta }^{*},{\widehat{\gamma }}_{CGQL}^{*}\left({\beta }^{*}\right))\left[\left({\delta }_{\left(t\right)c}\otimes x_t\right)+{\displaystyle \frac{\partial \left[{\widehat{\gamma }}_{c,CGQL}^{\prime }\left({\beta }^{*}\right)\right]}{\partial {\beta }^{*}}}{\mathbf{y}}_{t-1}^{\left(g\right)}\right]\hfill \\ & -& {\tilde{\eta }}_{t|t-1}^{\left(c\right|g)}({\beta }^{*},{\widehat{\gamma }}_{CGQL}^{*}\left({\beta }^{*}\right)){\displaystyle \sum _{\nu }^{C-1}}{\tilde{\eta }}_{t|t-1}^{\left(\nu \right|g)}({\beta }^{*},{\widehat{\gamma }}_{CGQL}^{*}\left({\beta }^{*}\right))\hfill \\ & \times & \left[\left({\delta }_{\left(t\right)\nu }\otimes x_t\right)+{\displaystyle \frac{\partial \left[{\widehat{\gamma }}_{\nu ,CGQL}^{\prime }\left({\beta }^{*}\right)\right]}{\partial {\beta }^{*}}}{\mathbf{y}}_{t-1}^{\left(g\right)}\right],\hfill \end{array}$$

(30)

where, for example, ${\displaystyle \frac{\partial \left[{\widehat{\gamma }}_{c,CGQL}^{\prime }\left({\beta }^{*}\right)\right]}{\partial {\beta }^{*}}}$ is the $(p+1)(C-1)\times (C-1)$-dimensional cth component matrix in (27) for all $c=1,\dots ,C-1,$ and ${\displaystyle \frac{\partial \left[{\widehat{\gamma }}_{C,CGQL}^{\prime }\left({\beta }^{*}\right)\right]}{\partial {\beta }^{*}}}=0,$ ${\delta }_{\left(t\right)C}={01}_{C-1}$ without any loss of generality.

Proof. 

Re-express the formula for ${\tilde{\eta }}_{t|t-1}^{\left(c\right|g)}({\beta }^{*},{\widehat{\gamma }}_{CGQL}^{*}\left({\beta }^{*}\right);x_t)$ in (24) as

$$\begin{array}{ccc}& & {\tilde{\eta }}_{t|t-1}^{\left(c\right|g)}({\beta }^{*},{\widehat{\gamma }}_{CGQL}^{*}\left({\beta }^{*}\right);x_t)\hfill \\ & =& \left\{\begin{array}{cc}{\displaystyle \frac{N_c}{D}},\hfill & \mathrm{for} c=1,\dots ,C-1\hfill \\ {\displaystyle \frac{1}{D}},\hfill & \mathrm{for} c=C,\hfill \end{array}\right.\hfill \end{array}$$

(31)

where $N_c=\exp \left[x_t^{\prime }{\beta }_c+{\widehat{\gamma }}_{c,CGQL}^{\prime }\left({\beta }^{*}\right){\mathbf{y}}_{t-1}^{\left(g\right)}\right],$ and $D=1+{\sum }_{\nu =1}^{C-1}\exp \left[x_t^{\prime }{\beta }_{\nu }+{\widehat{\gamma }}_{\nu ,CGQL}^{\prime }\left({\beta }^{*}\right){\mathbf{y}}_{t-1}^{\left(g\right)}\right].$ It then follows that

$$\begin{array}{ccc}& & {\displaystyle \frac{\partial {\tilde{\eta }}_{t|t-1}^{\left(c\right|g)}({\beta }^{*},{\widehat{\gamma }}_{CGQL}^{*}\left({\beta }^{*}\right))}{\partial {\beta }^{*}}}={\tilde{\eta }}_{t|t-1}^{\left(c\right|g)}({\beta }^{*},{\widehat{\gamma }}_{CGQL}^{*}\left({\beta }^{*}\right))\left[{\displaystyle \frac{\partial }{\partial {\beta }^{*}}}\left\{x_t^{\prime }{\beta }_c+{\widehat{\gamma }}_{c,CGQL}^{\prime }\left({\beta }^{*}\right){\mathbf{y}}_{t-1}^{\left(g\right)}\right\}\right.\hfill \\ & & -\left.{\displaystyle \frac{1}{D}}{\displaystyle \sum _{\nu =1}^{C-1}}\left[\exp \left(x_t^{\prime }{\beta }_{\nu }+{\widehat{\gamma }}_{\nu ,CGQL}^{\prime }\left({\beta }^{*}\right){\mathbf{y}}_{t-1}^{\left(g\right)}\right){\displaystyle \frac{\partial }{\partial {\beta }^{*}}}\left\{x_t^{\prime }{\beta }_{\nu }+{\widehat{\gamma }}_{\nu ,CGQL}^{\prime }\left({\beta }^{*}\right){\mathbf{y}}_{t-1}^{\left(g\right)}\right\}\right]\right].\hfill \end{array}$$

(32)

The formula in (30) follows from (32) because

$${\tilde{\eta }}_{t|t-1}^{\left(\nu \right|g)}({\beta }^{*},{\widehat{\gamma }}_{CGQL}^{*}\left({\beta }^{*}\right))={\displaystyle \frac{1}{D}}\exp \left(x_t^{\prime }{\beta }_{\nu }+{\widehat{\gamma }}_{\nu ,CGQL}^{\prime }\left({\beta }^{*}\right){\mathbf{y}}_{t-1}^{\left(g\right)}\right), \mathrm{and} {\displaystyle \frac{\partial }{\partial {\beta }^{*}}}\left\{x_t^{\prime }{\beta }_{\nu }\right\}=\left({\delta }_{\left(t\right)\nu }\otimes x_t\right).$$

 □

Simplified likelihood estimating equation for ${\mathit{\beta }}^{*}$: Notice that by using the derivative formulas from (26) and (32), one may reduce the the likelihood estimating Equation (25) as

$$\begin{array}{ccc}& & {\displaystyle \frac{\partial Log L({\beta }^{*},{\widehat{\gamma }}_{CGQL}^{*}\left({\beta }^{*}\right))}{\partial {\beta }^{*}}}={\displaystyle \sum _{c=1}^C}{y}_{1c}\left[{\delta }_{\left(1\right)c}-{\pi }_{\left(1\right)}\left({\beta }^{*}\right)\right]\otimes x_1\hfill \\ & +& {\displaystyle \sum _{t=2}^T}{\displaystyle \sum _{g=1}^C}{\displaystyle \sum _{c=1}^C}{y}_{tc}\left[\left\{\left({\delta }_{\left(t\right)c}\otimes x_t\right)+{\displaystyle \frac{\partial \left[{\widehat{\gamma }}_{c,CGQL}^{\prime }\left({\beta }^{*}\right)\right]}{\partial {\beta }^{*}}}{\mathbf{y}}_{t-1}^{\left(g\right)}\right\}\right.\hfill \\ & -& \left.{\displaystyle \sum _{\nu }^{C-1}}{\tilde{\eta }}_{t|t-1}^{\left(\nu \right|g)}({\beta }^{*},{\widehat{\gamma }}_{CGQL}^{*}\left({\beta }^{*}\right))\left\{\left({\delta }_{\left(t\right)\nu }\otimes x_t\right)+{\displaystyle \frac{\partial \left[{\widehat{\gamma }}_{\nu ,CGQL}^{\prime }\left({\beta }^{*}\right)\right]}{\partial {\beta }^{*}}}{\mathbf{y}}_{t-1}^{\left(g\right)}\right\}\right]=0,\hfill \end{array}$$

(33)

which is easily computable as the formulas for ${\displaystyle \frac{\partial \left[{\widehat{\gamma }}_{c,CGQL}^{\prime }\left({\beta }^{*}\right)\right]}{\partial {\beta }^{*}}}$ for all $c=1,\dots ,C-1$ involved in this reduced form are available from (27). For $c=C$ one uses ${\displaystyle \frac{\partial \left[{\widehat{\gamma }}_{C,CGQL}^{\prime }\left({\beta }^{*}\right)\right]}{\partial {\beta }^{*}}}=0,$ and ${\delta }_{\left(t\right)C}={01}_{C-1}.$

Lemma 5.

The likelihood Equation (33) for ${\beta }^{*}$ may be obtained by using the iterative equation

$$\begin{array}{ccc}& & {\widehat{\beta }}^{*}(r+1)={\widehat{\beta }}^{*}\left(r\right)\hfill \\ & +& {\left[{\left\{{\displaystyle \frac{{\partial }^{2}Log L({\beta }^{*},{\widehat{\gamma }}_{CGQL}^{*}\left({\beta }^{*}\right))}{\partial {\beta }^{*}\partial {{\beta }^{*}}^{\prime }}}\right\}}^{-1}{\displaystyle \frac{\partial Log L({\beta }^{*},{\widehat{\gamma }}_{CGQL}^{*}\left({\beta }^{*}\right))}{\partial {\beta }^{*}}}\right]}_{{\beta }^{*}={\widehat{\beta }}^{*}\left(r\right)},\hfill \end{array}$$

(34)

where, under the assumption that ${\beta }^{*}$ involved in the derivative ${\displaystyle \frac{\partial {\widehat{\gamma }}_{c,CGQL}^{\prime }\left({\beta }^{*}\right)}{\partial {\beta }^{*}}}$ in (30) or (33) for $c=1,\dots ,C-1,$ is known from the previous iteration, the second-order derivative in (34), by (33), has the formula

$$\begin{array}{ccc}& & {\displaystyle \frac{{\partial }^{2}Log L({\beta }^{*},{\widehat{\gamma }}_{CGQL}^{*}\left({\beta }^{*}\right))}{\partial {\beta }^{*}\partial {{\beta }^{*}}^{\prime }}}\hfill \\ & =& -{\displaystyle \sum _{c=1}^C}{y}_{1c}\left[{\displaystyle \frac{\partial {\pi }_{\left(1\right)}\left({\beta }^{*}\right)}{\partial {{\beta }^{*}}^{\prime }}}\right]\otimes x_1\hfill \\ & -& {\displaystyle \sum _{t=2}^T}{\displaystyle \sum _{g=1}^C}{\displaystyle \sum _{c=1}^C}{y}_{tc}\left[{\displaystyle \sum _{\nu }^{C-1}}\left\{\left({\delta }_{\left(t\right)\nu }\otimes x_t\right)+{\displaystyle \frac{\partial \left[{\widehat{\gamma }}_{\nu ,CGQL}^{\prime }\left({\beta }^{*}\right)\right]}{\partial {\beta }^{*}}}{\mathbf{y}}_{t-1}^{\left(g\right)}\right\}{\displaystyle \frac{\partial {\tilde{\eta }}_{t|t-1}^{\left(\nu \right|g)}({\beta }^{*},{\widehat{\gamma }}_{CGQL}^{*}\left({\beta }^{*}\right))}{\partial {{\beta }^{*}}^{\prime }}}\right],\hfill \end{array}$$

(35)

where ${\pi }_{\left(1\right)}\left({\beta }^{*}\right)={[{\pi }_{\left(i\right)1},\dots ,{\pi }_{\left(i\right)c},\dots ,{\pi }_{\left(i\right)(C-1)}]}^{\prime }$ and

$${\displaystyle \frac{\partial {\pi }_{\left(1\right)c}\left({\beta }^{*}\right)}{\partial {\beta }^{*}}}=\left[{\pi }_{\left(1\right)c}\left({\beta }^{*}\right)({\delta }_{\left(1\right)c}-{\pi }_{\left(1\right)}\left({\beta }^{*}\right))\right]\otimes x_1:(C-1)(p+1)\times 1,$$

by Lemma 2; and ${\displaystyle \frac{\partial {\tilde{\eta }}_{t|t-1}^{\left(\nu \right|g)}({\beta }^{*},{\widehat{\gamma }}_{CGQL}^{*}\left({\beta }^{*}\right))}{\partial {{\beta }^{*}}^{\prime }}}$ has the same formula as in (30).

3. Asymptotics

3.1. Consistency of ${\widehat{\gamma }}_{CGQL}^{*}\left({\beta }^{*}\right)$ for Dynamic Dependence Parameter ${\gamma }^{*}$

For this purpose, we will derive the asymptotic (as $T\to \infty$) distribution of ${\widehat{\gamma }}_{CGQL}^{*}\left({\beta }^{*}\right)$ (solution of moment Equation (22)) as in Theorem 1 given below. Recall that the CGQL estimating function for $\gamma ,$ i.e., the left-hand side of (18) [see also (19)] has the formula

$$\begin{array}{ccc}& & f_T({\beta }^{*},{\gamma }^{*})={\displaystyle \sum _{t=2}^T}{\displaystyle \frac{\partial {\eta }_{t|t-1}^{\prime }({\beta }^{*},{\gamma }^{*})}{\partial {\gamma }^{*}}}{{\Sigma }^{-1}}_{t|t-1}({\beta }^{*},{\gamma }^{*})[{\mathbf{y}}_t-{\eta }_{t|t-1}({\beta }^{*},{\gamma }^{*})]\hfill \\ & =& {\displaystyle \sum _{t=2}^T}[{\eta }_{\left(t\right|t-1),M}^{*}({\beta }^{*},{\gamma }^{*})\otimes {\mathbf{y}}_{t-1}]{{\Sigma }^{-1}}_{t|t-1}({\beta }^{*},{\gamma }^{*})[{\mathbf{y}}_t-{\eta }_{t|t-1}({\beta }^{*},{\gamma }^{*})]\hfill \end{array}$$

(36)

so that ${\widehat{\gamma }}_{CGQL}^{*}\left({\beta }^{*}\right)$ satisfies

$$\begin{array}{ccc}& & f_T({\beta }^{*},{\widehat{\gamma }}_{CGQL}^{*}\left({\beta }^{*}\right))={\displaystyle \sum _{t=2}^T}[{\tilde{\eta }}_{\left(t\right|t-1),M}^{*}({\beta }^{*},{\widehat{\gamma }}_{CGQL}^{*}\left({\beta }^{*}\right))\otimes {\mathbf{y}}_{t-1}]\hfill \\ & \times & {{\tilde{\Sigma }}^{-1}}_{t|t-1}({\beta }^{*},{\widehat{\gamma }}_{CGQL}^{*}\left({\beta }^{*}\right))[{\mathbf{y}}_t-{\tilde{\eta }}_{t|t-1}({\beta }^{*},{\widehat{\gamma }}_{CGQL}^{*}\left({\beta }^{*}\right))]=0.\hfill \end{array}$$

(37)

Notice that conditional on the past history, ${\mathbf{y}}_t$ in (36) for $t=2,\dots ,T,$ depends on only the lag 1 response ${\mathbf{y}}_{t-1}.$ Thus, conditional on ${\mathbf{y}}_{t-1}$, all ${\mathbf{y}}_t$ may be treated to be independent. Consequently, $f_T({\beta }^{*},{\gamma }^{*})$ in (36) is, conditionally, a sum of $T-1$ independent quantities. Furthermore, as shown in (16),

$${\mathbf{Y}}_t|{\mathbf{y}}_{t-1}\sim Mult\left({\eta }_{t|t-1}({\beta }^{*},{\gamma }^{*}),{\Sigma }_{t|t-1}(\beta ,\gamma )\right).$$

Next, for true ${\gamma }^{*},$ using (36), we write

$$\begin{array}{ccc}{\overline{f}}_T({\beta }^{*},{\gamma }^{*})\hfill & =& {\displaystyle \frac{1}{T-1}}{\displaystyle \sum _{t=2}^T}{f}_t({\beta }^{*},{\gamma }^{*})\hfill \\ & =& {\displaystyle \frac{1}{T-1}}{\displaystyle \sum _{t=2}^T}[{\eta }_{\left(t\right|t-1),M}^{*}({\beta }^{*},{\gamma }^{*})\otimes {\mathbf{y}}_{t-1}]{{\Sigma }^{-1}}_{t|t-1}({\beta }^{*},{\gamma }^{*})[y_t-{\eta }_{t|t-1}({\beta }^{*},{\gamma }^{*})].\hfill \end{array}$$

(38)

Because ${\mathbf{Y}}_t|{\mathbf{y}}_{t-1}$ has the aforementioned multinomial distribution (see also (16)), it then follows that

$$\begin{array}{ccc}E\left[{\overline{f}}_T({\beta }^{*},{\gamma }^{*})\right]\hfill & =& 0\hfill \\ \mathrm{cov}\left[{\overline{f}}_T({\beta }^{*},{\gamma }^{*})\right]\hfill & =& {\displaystyle \frac{1}{{(T-1)}^2}}{\displaystyle \sum _{t=2}^T}\left[{\eta }_{\left(t\right|t-1),M}^{*}({\beta }^{*},{\gamma }^{*})\otimes {\mathbf{y}}_{t-1}\right]\hfill \\ & \times & {{\Sigma }^{-1}}_{t|t-1}({\beta }^{*},{\gamma }^{*}){\left[{\eta }_{\left(t\right|t-1),M}^{*}({\beta }^{*},{\gamma }^{*})\otimes {\mathbf{y}}_{t-1}\right]}^{\prime }\hfill \end{array}$$

(39)

$$\begin{array}{ccc}& =& {\displaystyle \frac{1}{{(T-1)}^2}}{V}_T^{*}({\beta }^{*},{\gamma }^{*}), \left(\mathrm{say}\right).\hfill \end{array}$$

(40)

We now derive the asymptotic distribution of ${\widehat{\gamma }}_{CGQL}^{*}\left({\beta }^{*}\right)$ as in the following theorem.

Theorem 1.

We assume that $f_t(·)$ in (38) satisfy the Lindeberg condition, that is,

$$\underset{T\to \infty }{\lim }{V^{*}}_T^{-1}{\displaystyle \sum _{t=2}^T}\sum _{\left(f_t^{\prime }{V^{*}}_T^{-1}{f}_t\right)>ϵ}{f}_t{f}_t^{\prime }g\left(f_t\right)=0$$

(41)

for all $ϵ>0,g(·)$ being the probability distribution of $f_t(·).$ Then

$$\begin{array}{ccc}& & \underset{T\to \infty }{\lim }{\widehat{\gamma }}_{CGQL}^{*}\left({\beta }^{*}\right)\hfill \\ & \to & N\left({\gamma }^{*},\left\{E_{Y_{t-1}}{\displaystyle \sum _{t=2}^T}\left[({\eta }_{\left(t\right|t-1),M}^{*}({\beta }^{*},{\gamma }^{*})\otimes {\mathbf{y}}_{t-1}){{\Sigma }^{-1}}_{t|t-1}({\beta }^{*},{\gamma }^{*})\right.\right.\right.\hfill \\ & \times & {\left.\left.{({\eta }_{\left(t\right|t-1),M}^{*}({\beta }^{*},{\gamma }^{*})\otimes {\mathbf{y}}_{t-1})}^{\prime }\right]\right\}}^{-1}{E}_{{\mathbf{Y}}_{t-1}}\left[V_T^{*}({\beta }^{*},{\gamma }^{*})\right]\hfill \\ & \times & \left.{\left\{E_{{\mathbf{Y}}_{t-1}}{\displaystyle \sum _{t=2}^T}\left[({\eta }_{\left(t\right|t-1),M}^{*}({\beta }^{*},{\gamma }^{*})\otimes {\mathbf{y}}_{t-1}){{\Sigma }^{-1}}_{t|t-1}({\beta }^{*},{\gamma }^{*}){({\eta }_{\left(t\right|t-1),M}^{*}({\beta }^{*},{\gamma }^{*})\otimes {\mathbf{y}}_{t-1})}^{\prime }\right]\right\}}^{-1}\right).\hfill \end{array}$$

(42)

Proof. 

Let

$$\begin{array}{ccc}{Z}_T\hfill & =& {\left[\mathrm{cov}\left[{\overline{f}}_T({\beta }^{*},{\gamma }^{*})\right]\right]}^{-{\displaystyle \frac{1}{2}}}{\overline{f}}_T({\beta }^{*},{\gamma }^{*})\hfill \\ & =& (T-1){\left[V_T^{*}({\beta }^{*},{\gamma }^{*})\right]}^{-{\displaystyle \frac{1}{2}}}{\overline{f}}_T({\beta }^{*},{\gamma }^{*}),\hfill \end{array}$$

(43)

where ${\overline{f}}_T({\beta }^{*},{\gamma }^{*})={\displaystyle \frac{1}{T-1}}{\sum }_{t=2}^T{f}_t({\beta }^{*},{\gamma }^{*})$ as in (38). Here $\left\{f_t(·)\right\}$s are not identically distributed because, by (16),

$${\mathbf{Y}}_t|{\mathbf{y}}_{t-1}\sim \mathrm{Mult}\left({\eta }_{t|t-1}({\beta }^{*},{\gamma }^{*}),{\Sigma }_{t|t-1}({\beta }^{*},{\gamma }^{*})\right),$$

(44)

where the mean vectors and covariance matrices are different at different time points $t.$ However, because $\left\{f_t(·)\right\}$s satisfy the Lindeberg condition (41), it then follows from the Lindeberg–Feller central limit theorem ([6], Theorem 3.3.6, [23], Theorem 2.2) that $Z_T$ in (43) has the following limiting distribution:

$$\underset{T\to \infty }{\lim }{Z}_T\to N(0,I_{{(C-1)}^2}).$$

(45)

Next, because ${\widehat{\gamma }}_{CGQL}^{*}\left({\beta }^{*}\right)$ obtained by (22) is a solution of (18), it satisfies (37), i.e.,

$$\begin{array}{ccc}& & {\displaystyle \sum _{t=2}^T}{f}_t({\beta }^{*},{\widehat{\gamma }}_{CGQL}^{*}\left({\beta }^{*}\right))\hfill \\ & =& {\displaystyle \sum _{t=2}^T}[{\tilde{\eta }}_{\left(t\right|t-1),M}^{*}({\beta }^{*},{\widehat{\gamma }}_{CGQL}^{*}\left({\beta }^{*}\right))\otimes {\mathbf{y}}_{t-1}]\hfill \\ & \times & {{\tilde{\Sigma }}^{-1}}_{t|t-1}({\beta }^{*},{\widehat{\gamma }}_{CGQL}^{*}\left({\beta }^{*}\right))[{\mathbf{y}}_t-{\tilde{\eta }}_{t|t-1}({\beta }^{*},{\widehat{\gamma }}_{CGQL}^{*}\left({\beta }^{*}\right))]=0,\hfill \end{array}$$

(46)

which, by first-order Taylor’s series expansion, produces

$${\displaystyle \sum _{t=2}^T}{f}_t({\beta }^{*},{\gamma }^{*})+({\widehat{\gamma }}_{CGQL}^{*}\left({\beta }^{*}\right)-{\gamma }^{*}){\displaystyle \sum _{t=2}^T}{\displaystyle \frac{\partial f_t({\beta }^{*},{\gamma }^{*})}{\partial {{\gamma }^{*}}^{\prime }}}\simeq 0.$$

(47)

Thus,

$$\begin{array}{ccc}& & [{\widehat{\gamma }}_{CGQL}^{*}\left({\beta }^{*}\right)-{\gamma }^{*}]\simeq -{\left[{\displaystyle \sum _{t=2}^T}{\displaystyle \frac{\partial f_t({\beta }^{*},{\gamma }^{*})}{\partial {{\gamma }^{*}}^{\prime }}}\right]}^{-1}{\displaystyle \sum _{t=2}^T}{f}_t({\beta }^{*},{\gamma }^{*})\hfill \\ & \to & -\left\{-E_{{\mathbf{Y}}_{t-1}}{\displaystyle \sum _{t=2}^T}\left[({\eta }_{\left(t\right|t-1),M}^{*}({\beta }^{*},{\gamma }^{*})\otimes {\mathbf{y}}_{t-1}){{\Sigma }^{-1}}_{t|t-1}({\beta }^{*},{\gamma }^{*})\right.\right.\hfill \\ & \times & {\left.\left.{({\eta }_{\left(t\right|t-1),M}^{*}({\beta }^{*},{\gamma }^{*})\otimes {\mathbf{y}}_{t-1})}^{\prime }\right]\right\}}^{-1}{E}_{{\mathbf{Y}}_{t-1}}{\displaystyle \sum _{t=2}^T}{f}_t({\beta }^{*},{\gamma }^{*})\hfill \\ & =& {\left\{E_{{\mathbf{Y}}_{t-1}}{\displaystyle \sum _{t=2}^T}\left[({\eta }_{\left(t\right|t-1),M}^{*}({\beta }^{*},{\gamma }^{*})\otimes {\mathbf{y}}_{t-1}){{\Sigma }^{-1}}_{t|t-1}({\beta }^{*},{\gamma }^{*}){({\eta }_{\left(t\right|t-1),M}^{*}({\beta }^{*},{\gamma }^{*})\otimes {\mathbf{y}}_{t-1})}^{\prime }\right]\right\}}^{-1}\hfill \\ & \times & E_{{\mathbf{Y}}_{t-1}}\left[{\left[V_T^{*}({\beta }^{*},{\gamma }^{*})\right]}^{{\displaystyle \frac{1}{2}}}{\left[V_T^{*}({\beta }^{*},{\gamma }^{*})\right]}^{-{\displaystyle \frac{1}{2}}}(T-1){\overline{f}}_T({\beta }^{*},{\gamma }^{*})\right]\hfill \\ & =& {\left\{E_{{\mathbf{Y}}_{t-1}}{\displaystyle \sum _{t=2}^T}\left[({\eta }_{\left(t\right|t-1),M}^{*}({\beta }^{*},{\gamma }^{*})\otimes {\mathbf{y}}_{t-1}){{\Sigma }^{-1}}_{t|t-1}({\beta }^{*},{\gamma }^{*}){({\eta }_{\left(t\right|t-1),M}^{*}({\beta }^{*},{\gamma }^{*})\otimes {\mathbf{y}}_{t-1})}^{\prime }\right]\right\}}^{-1}\hfill \\ & \times & {\left[E_{{\mathbf{Y}}_{t-1}}{V}_T^{*}({\beta }^{*},{\gamma }^{*})\right]}^{{\displaystyle \frac{1}{2}}}{Z}_T\hfill \\ & =& {\left[E_{{\mathbf{Y}}_{t-1}}{V}_T^{*}({\beta }^{*},{\gamma }^{*})\right]}^{-{\displaystyle \frac{1}{2}}}{Z}_T\hfill \end{array}$$

(48)

by (40) and (43). The theorem, i.e., (42), follows from (48) because the limiting distribution $\left(q\right(·\left)\right)$ of $Z_T$ is normal, that is, ${\lim }_{T\to \infty }q\left(Z_T\right)\to N(0,I_{{(C-1)}^2})$ by (45). Furthermore, because the quantity in the right-hand side of (48) can be re-expressed by (40) as

$$E_{{\mathbf{Y}}_{t-1}}{\left[{(T-1)}^2\mathrm{cov}\left({\overline{f}}_T({\beta }^{*},{\gamma }^{*})\right|{\mathbf{y}}_{t-1})\right]}^{-{\displaystyle \frac{1}{2}}}{Z}_T,$$

it then follows under a mild regularity condition (i.e., by assuming that the covariance $E_{{\mathbf{Y}}_{t-1}}\mathrm{cov}\left({\overline{f}}_T({\beta }^{*},{\gamma }^{*})\right|{\mathbf{y}}_{t-1})$ conditional on the past history is finite) that

$$\underset{T\to \infty }{\lim }[{\widehat{\gamma }}_{CGQL}^{*}\left({\beta }^{*}\right)-{\gamma }^{*}]\to 0,$$

showing that ${\widehat{\gamma }}_{CGQL}^{*}\left({\beta }^{*}\right)$ is a mean squared error consistent function for ${\gamma }^{*},$ for any ${\beta }^{*}.$ □

We remark that a Lindeberg’s condition similar to (41) was examined earlier in [14]. For details on this, we refer to ([14], Assumption N(iii), p. 89) for Lindeberg’s condition, and ([14], second paragraph under proof of Theorem 1, p. 93) for its proof.

3.2. Consistency of MML Estimtor ${\widehat{\beta }}^{*}$ for ${\beta }^{*}$

Let $h({\beta }^{*},{\widehat{\gamma }}_{CGQL}^{*}\left({\beta }^{*}\right))$ denote the likelihood estimating function, i.e., the left-hand side of the likelihood estimating Equation (25) [see also (33)] for ${\beta }^{*}.$ We further express this function as

$$h({\beta }^{*},{\widehat{\gamma }}_{CGQL}^{*}\left({\beta }^{*}\right))=\left[h_1\left({\beta }^{*}\right)+h_{T,2}({\beta }^{*},{\widehat{\gamma }}_{CGQL}^{*}\left({\beta }^{*}\right))\right],$$

(49)

where

$$\begin{array}{ccc}\hfill h_1\left({\beta }^{*}\right)& =& {\displaystyle \sum _{c=1}^C}{\displaystyle \frac{y_{1c}}{{\pi }_{\left(1\right)c}\left({\beta }^{*}\right)}}{\displaystyle \frac{\partial {\pi }_{\left(1\right)c}\left({\beta }^{*}\right)}{\partial {\beta }^{*}}}, \mathrm{and}\hfill \\ h_{T,2}({\beta }^{*},{\widehat{\gamma }}_{CGQL}^{*}\left({\beta }^{*}\right))\hfill & =& {\displaystyle \sum _{t=2}^T}{\displaystyle \sum _{g=1}^C}{\displaystyle \sum _{c=1}^C}\left[{\displaystyle \frac{y_{tc}}{{\tilde{\eta }}_{t|t-1}^{\left(c\right|g)}({\beta }^{*},{\widehat{\gamma }}_{CGQL}^{*}\left({\beta }^{*}\right))}}{\displaystyle \frac{\partial {\tilde{\eta }}_{t|t-1}^{\left(c\right|g)}({\beta }^{*},{\widehat{\gamma }}_{CGQL}^{*}\left({\beta }^{*}\right))}{\partial {\beta }^{*}}}\right].\hfill \end{array}$$

(50)

The results from the following Lemma 6 will be used in Theorem 2 below to derive the asymptotic as a ($T\to \infty$) distribution of ${\widehat{\beta }}^{*}$ (solution of likelihood estimating Equation (25)).

Lemma 6.

Let ${\overline{h}}_T({\beta }^{*},{\widehat{\gamma }}_{CGQL}^{*}\left({\beta }^{*}\right))={\displaystyle \frac{1}{T}}h({\beta }^{*},{\widehat{\gamma }}_{CGQL}^{*}\left({\beta }^{*}\right)).$ This $(p+1)$-dimensional mean vector function has the expectation and the $(p+1)\times (p+1)$ conditional (on lag 1 response) covariance matrix as

$$\begin{array}{ccc}& & E\left[{\overline{h}}_T({\beta }^{*},{\widehat{\gamma }}_{CGQL}^{*}\left({\beta }^{*}\right))\right]={\displaystyle \frac{1}{T}}E\left[h({\beta }^{*},{\widehat{\gamma }}_{CGQL}^{*}\left({\beta }^{*}\right))\right]=0, and\hfill \\ & & \mathit{cov}\left({\overline{h}}_T({\beta }^{*},{\widehat{\gamma }}_{CGQL}^{*}\left({\beta }^{*}\right))\right)\hfill \\ & =& {\displaystyle \frac{1}{T^2}}\left[\left\{{\displaystyle \sum _{c=1}^C}{\displaystyle \frac{1-{\pi }_{\left(1\right)c}\left({\beta }^{*}\right)}{{\pi }_{\left(1\right)c}\left({\beta }^{*}\right)}}{\displaystyle \frac{\partial {\pi }_{\left(1\right)c}\left({\beta }^{*}\right)}{\partial {\beta }^{*}}}{\displaystyle \frac{\partial {\pi }_{\left(1\right)c}\left({\beta }^{*}\right)}{\partial {{\beta }^{*}}^{\prime }}}-{\displaystyle \sum _{u\ne c}^C}{\displaystyle \frac{\partial {\pi }_{\left(1\right)c}\left({\beta }^{*}\right)}{\partial {\beta }^{*}}}{\displaystyle \frac{\partial {\pi }_{\left(1\right)u}\left({\beta }^{*}\right)}{\partial {{\beta }^{*}}^{\prime }}}\right\}\right.\hfill \\ & +& {\displaystyle \sum _{t=2}^T}{\displaystyle \sum _{g=1}^C}\left\{\left({\displaystyle \sum _{c=1}^C}{\displaystyle \frac{1-{\tilde{\eta }}_{t|t-1}^{\left(c\right|g)}({\beta }^{*},{\widehat{\gamma }}_{CGQL}^{*}\left({\beta }^{*}\right))}{{\tilde{\eta }}_{t|t-1}^{\left(c\right|g_j)}({\beta }^{*},{\widehat{\gamma }}_{CGQL}^{*}\left({\beta }^{*}\right))}}{\displaystyle \frac{\partial {\tilde{\eta }}_{t|t-1}^{\left(c\right|g)}({\beta }^{*},{\widehat{\gamma }}_{CGQL}^{*}\left({\beta }^{*}\right))}{\partial {\beta }^{*}}}{\displaystyle \frac{\partial {\tilde{\eta }}_{t|t-1}^{\left(c\right|g)}({\beta }^{*},{\widehat{\gamma }}_{CGQL}^{*}\left({\beta }^{*}\right))}{\partial {{\beta }^{*}}^{\prime }}}\right)\right.\hfill \\ & -& \left.\left.\left({\displaystyle \sum _{c\ne u}^C}{\displaystyle \frac{\partial {\tilde{\eta }}_{t|t-1}^{\left(c\right|g)}({\beta }^{*},{\widehat{\gamma }}_{CGQL}^{*}\left({\beta }^{*}\right))}{\partial {\beta }^{*}}}{\displaystyle \frac{\partial {\tilde{\eta }}_{t|t-1}^{\left(u\right|g)}({\beta }^{*},{\widehat{\gamma }}_{CGQL}^{*}\left({\beta }^{*}\right))}{\partial {{\beta }^{*}}^{\prime }}}\right)\right\}\right],\hfill \end{array}$$

(51)

respectively.

Proof. 

Because ${\pi }_{\left(1\right)C}\left({\beta }^{*}\right)=1-{\sum }_{c=1}^{C-1}{\pi }_{\left(1\right)c}\left({\beta }^{*}\right)$ and ${\tilde{\eta }}_{t|t-1}^{\left(C\right|g)}({\beta }^{*},{\widehat{\gamma }}_{CGQL}^{*}\left({\beta }^{*}\right))=1-{\sum }_{c=1}^{C-1}{\tilde{\eta }}_{t|t-1}^{\left(c\right|g)}({\beta }^{*},{\widehat{\gamma }}_{CGQL}^{*}\left({\beta }^{*}\right)),$ it follows from (50) that

$$\begin{array}{ccc}E\left[h_1\left({\beta }^{*}\right)\right]\hfill & =& {\displaystyle \sum _{c=1}^C}{\displaystyle \frac{E\left(Y_{1c}\right)}{{\pi }_{\left(1\right)c}\left({\beta }^{*}\right)}}{\displaystyle \frac{\partial {\pi }_{\left(1\right)c}\left({\beta }^{*}\right)}{\partial {\beta }^{*}}}\hfill \\ & =& {\displaystyle \sum _{c=1}^C}{\displaystyle \frac{\partial {\pi }_{\left(1\right)c}\left({\beta }^{*}\right)}{\partial {\beta }^{*}}}=0 \mathrm{and}\hfill \end{array}$$

(52)

$$\begin{array}{ccc}E\left[h_{T,2}({\beta }^{*},{\widehat{\gamma }}_{CGQL}^{*}\left({\beta }^{*}\right))\right]\hfill & =& {\displaystyle \sum _{t=2}^T}{\displaystyle \sum _{g=1}^C}{\displaystyle \sum _{c=1}^C}\left[{\displaystyle \frac{E\left(Y_{tc}\right|y_{(t-1),g})}{{\tilde{\eta }}_{t|t-1}^{\left(c\right|g)}({\beta }^{*},{\widehat{\gamma }}_{CGQL}^{*}\left({\beta }^{*}\right))}}{\displaystyle \frac{\partial {\tilde{\eta }}_{t|t-1}^{\left(c\right|g)}({\beta }^{*},{\widehat{\gamma }}_{CGQL}^{*}\left({\beta }^{*}\right))}{\partial {\beta }^{*}}}\right]\hfill \\ & =& {\displaystyle \sum _{t=2}^T}{\displaystyle \sum _{g=1}^C}{\displaystyle \sum _{c=1}^C}\left[{\displaystyle \frac{\partial {\tilde{\eta }}_{t|t-1}^{\left(c\right|g)}({\beta }^{*},{\widehat{\gamma }}_{CGQL}^{*}\left({\beta }^{*}\right))}{\partial {\beta }^{*}}}\right]=0,\hfill \end{array}$$

(53)

Next because

$$h({\beta }^{*},{\widehat{\gamma }}_{CGQL}^{*}\left({\beta }^{*}\right))=\left[h_1\left({\beta }^{*}\right)+h_{T,2}({\beta }^{*},{\widehat{\gamma }}_{CGQL}^{*}\left({\beta }^{*}\right))\right]$$

as stated in (49), one writes

$$\begin{array}{ccc}E\left[{\overline{h}}_T({\beta }^{*},{\widehat{\gamma }}_{CGQL}^{*}\left({\beta }^{*}\right))\right]\hfill & =& E\left[{\displaystyle \frac{1}{T}}h({\beta }^{*},{\widehat{\gamma }}_{CGQL}^{*}\left({\beta }^{*}\right))\right]\hfill \\ & =& {\displaystyle \frac{1}{T}}E\left[h_1\left({\beta }^{*}\right)\right]+{\displaystyle \frac{1}{T}}E\left[h_{T,2}({\beta }^{*},{\widehat{\gamma }}_{CGQL}^{*}\left({\beta }^{*}\right))\right].\hfill \end{array}$$

Hence by applying (52) and (53) we obtain

$$E\left[{\overline{h}}_T({\beta }^{*},{\widehat{\gamma }}_{CGQL}^{*}\left({\beta }^{*}\right))\right]=0$$

as stated in the lemma.

Next we compute the $(p+1)\times (p+1)$ conditional (on the history up to time $t-1$ ($H_{t-1}$)) covariance matrix of the mean vector function ${\overline{h}}_T({\beta }^{*},{\widehat{\gamma }}_{CGQL}^{*}\left({\beta }^{*}\right))$ as

$$\begin{array}{ccc}& & \mathrm{cov}\left({\overline{h}}_T({\beta }^{*},{\widehat{\gamma }}_{CGQL}^{*}\left({\beta }^{*}\right))\right)={\displaystyle \frac{1}{T^2}}\left[\mathrm{cov}\left(h_1\left({\beta }^{*}\right)\right)+\mathrm{cov}\left(h_{T,2}({\beta }^{*},{\widehat{\gamma }}_{CGQL}^{*}\left({\beta }^{*}\right))\right)\right]\hfill \\ & =& {\displaystyle \frac{1}{T^2}}\left[\left\{{\displaystyle \sum _{c=1}^C}{\displaystyle \frac{\mathrm{var}\left(Y_{1c}\right)}{{\pi }_{\left(1\right)c}^2\left({\beta }^{*}\right)}}{\displaystyle \frac{\partial {\pi }_{\left(1\right)c}\left({\beta }^{*}\right)}{\partial {\beta }^{*}}}{\displaystyle \frac{\partial {\pi }_{\left(1\right)c}\left({\beta }^{*}\right)}{\partial {{\beta }^{*}}^{\prime }}}+{\displaystyle \sum _{u\ne c}^C}{\displaystyle \frac{\mathrm{cov}(Y_{1c},Y_{1u})}{{\pi }_{\left(1\right)c}\left({\beta }^{*}\right){\pi }_{\left(1\right)u}\left({\beta }^{*}\right)}}{\displaystyle \frac{\partial {\pi }_{\left(1\right)c}\left({\beta }^{*}\right)}{\partial {\beta }^{*}}}{\displaystyle \frac{\partial {\pi }_{\left(1\right)u}\left({\beta }^{*}\right)}{\partial {{\beta }^{*}}^{\prime }}}\right\}\right.\hfill \\ & +& {\displaystyle \sum _{t=2}^T}{\displaystyle \sum _{g=1}^C}\left\{{\displaystyle \sum _{c=1}^C}{\displaystyle \frac{\mathrm{var}\left(Y_{tc}\right|H_{t-1})}{{\left[{\tilde{\eta }}_{t|t-1}^{\left(c\right|g)}({\beta }^{*},{\widehat{\gamma }}_{CGQL}^{*}\left({\beta }^{*}\right))\right]}^2}}{\displaystyle \frac{\partial {\tilde{\eta }}_{t|t-1}^{\left(c\right|g)}({\beta }^{*},{\widehat{\gamma }}_{CGQL}^{*}\left({\beta }^{*}\right))}{\partial {\beta }^{*}}}{\displaystyle \frac{\partial {\tilde{\eta }}_{t|t-1}^{\left(c\right|g)}({\beta }^{*},{\widehat{\gamma }}_{CGQL}^{*}\left({\beta }^{*}\right))}{\partial {{\beta }^{*}}^{\prime }}}\right\}\hfill \\ & +& \left.\left\{{\displaystyle \sum _{c\ne u}^C}{\displaystyle \frac{\mathrm{cov}\left((Y_{tc},Y_{tu})\right|H_{t-1})}{\left[{\tilde{\eta }}_{t|t-1}^{\left(c\right|g)}({\beta }^{*},{\widehat{\gamma }}_{CGQL}^{*}\left({\beta }^{*}\right))\right]\left[{\tilde{\eta }}_{t|t-1}^{\left(u\right|g)}({\beta }^{*},{\widehat{\gamma }}_{CGQL}^{*}\left({\beta }^{*}\right))\right]}}{\displaystyle \frac{\partial {\tilde{\eta }}_{t|t-1}^{\left(c\right|g)}({\beta }^{*},{\widehat{\gamma }}_{CGQL}^{*}\left({\beta }^{*}\right))}{\partial {\beta }^{*}}}{\displaystyle \frac{\partial {\tilde{\eta }}_{t|t-1}^{\left(u\right|g)}({\beta }^{*},{\widehat{\gamma }}_{CGQL}^{*}\left({\beta }^{*}\right))}{\partial {{\beta }^{*}}^{\prime }}}\right\}\right],\hfill \end{array}$$

(54)

yielding the second part of the lemma because

$$\begin{array}{ccc}\hfill \mathrm{var}\left(Y_{1c}\right|H_{t-1})& =& {\pi }_{\left(1\right)c}\left({\beta }^{*}\right)[1-{\pi }_{\left(1\right)c}\left({\beta }^{*}\right)]\hfill \\ \hfill \mathrm{cov}(Y_{1c},Y_{1u})& =& -{\pi }_{\left(1\right)c}\left({\beta }^{*}\right){\pi }_{\left(1\right)u}\left({\beta }^{*}\right)\hfill \\ \hfill \mathrm{var}\left(Y_{tc}\right|H_{t-1})& =& {\tilde{\eta }}_{t|t-1}^{\left(c\right|g)}({\beta }^{*},{\widehat{\gamma }}_{CGQL}^{*}\left({\beta }^{*}\right))[1-{\tilde{\eta }}_{t|t-1}^{\left(c\right|g)}({\beta }^{*},{\widehat{\gamma }}_{CGQL}^{*}\left({\beta }^{*}\right))], t=2,\dots ,T\hfill \\ \hfill \mathrm{cov}\left((Y_{tc},Y_{tu})\right|H_{t-1})& =& -{\tilde{\eta }}_{t|t-1}^{\left(c\right|g)}({\beta }^{*},{\widehat{\gamma }}_{CGQL}^{*}\left({\beta }^{*}\right)){\tilde{\eta }}_{t|t-1}^{\left(u\right|g)}({\beta }^{*},{\widehat{\gamma }}_{CGQL}^{*}\left({\beta }^{*}\right)), t=2,\dots ,T.\hfill \end{array}$$

 □

Lemma 7.

The second-order derivative matrix ${\displaystyle \frac{{\partial }^2{\pi }_{\left(1\right)c}\left({\beta }^{*}\right)}{\partial {\beta }^{*}\partial {{\beta }^{*}}^{\prime }}}$ has the formula

$$\begin{array}{ccc}& & {\displaystyle \frac{{\partial }^2{\pi }_{\left(1\right)c}\left({\beta }^{*}\right)}{\partial {\beta }^{*}\partial {{\beta }^{*}}^{\prime }}}\hfill \\ & =& \left(\begin{array}{c}\left[-{\pi }_{\left(1\right)1}{\pi }_{\left(1\right)c}\left\{{\left({\delta }_{\left(1\right)c}+{\delta }_{\left(1\right)1}-2{\pi }_{\left(1\right)}\right)}^{\prime }\otimes x_1^{\prime }\right\}\right]\otimes x_1\\ \left[-{\pi }_{\left(1\right)2}{\pi }_{\left(1\right)c}\left\{{\left({\delta }_{\left(1\right)c}+{\delta }_{\left(1\right)2}-2{\pi }_{\left(1\right)}\right)}^{\prime }\otimes x_1^{\prime }\right\}\right]\otimes x_1\\ ⋮\\ \left[{\pi }_{\left(1\right)c}(1-2{\pi }_{\left(1\right)c})\left\{{\left({\delta }_{\left(1\right)c}-{\pi }_{\left(1\right)}\right)}^{\prime }\otimes x_1^{\prime }\right\}\right]\otimes x_1\\ ⋮\\ \left[-{\pi }_{\left(1\right)(C-1)}{\pi }_{\left(1\right)c}\left\{{\left({\delta }_{\left(1\right)c}+{\delta }_{\left(1\right)(C-1)}-2{\pi }_{\left(1\right)}\right)}^{\prime }\otimes x_1^{\prime }\right\}\right]\otimes x_1\end{array}\right),\hfill \end{array}$$

(55)

with ${\pi }_{\left(1\right)}={[{\pi }_{\left(1\right)1},\dots ,{\pi }_{\left(1\right)c},\dots ,{\pi }_{\left(1\right)(C-1)}]}^{\prime };$ and the second-order derivative matrix ${\displaystyle \frac{{\partial }^2{\tilde{\eta }}_{t|t-1}^{\left(c\right|g)}({\beta }^{*},{\widehat{\gamma }}_{CGQL}^{*}\left({\beta }^{*}\right))}{\partial {\beta }^{*}\partial {{\beta }^{*}}^{\prime }}}$ has an approximate formula

$$\begin{array}{ccc}& & {\displaystyle \frac{{\partial }^2{\tilde{\eta }}_{t|t-1}^{\left(c\right|g)}({\beta }^{*},{\widehat{\gamma }}_{CGQL}^{*}\left({\beta }^{*}\right))}{\partial {\beta }^{*}\partial {{\beta }^{*}}^{\prime }}}\hfill \\ & \approx & \left[\left({\delta }_{\left(t\right)c}\otimes x_t\right)+{\displaystyle \frac{\partial \left[{\widehat{\gamma }}_{c,CGQL}^{\prime }\left({\beta }^{*}\right)\right]}{\partial {\beta }^{*}}}{y}_{t-1}^{\left(g\right)}\right]{\displaystyle \frac{\partial {\tilde{\eta }}_{t|t-1}^{\left(c\right|g)}({\beta }^{*},{\widehat{\gamma }}_{CGQL}^{*}\left(\beta \right))}{\partial {{\beta }^{*}}^{\prime }}}\hfill \\ & -& {\displaystyle \sum _{\nu }^{C-1}}\left[{\tilde{\eta }}_{t|t-1}^{\left(\nu \right|g)}({\beta }^{*},{\widehat{\gamma }}_{CGQL}^{*}\left({\beta }^{*}\right))\left\{\left({\delta }_{\left(t\right)\nu }\otimes x_t\right)+{\displaystyle \frac{\partial \left[{\widehat{\gamma }}_{\nu ,CGQL}^{\prime }\left({\beta }^{*}\right)\right]}{\partial {\beta }^{*}}}{y}_{t-1}^{\left(g\right)}\right\}\right]{\displaystyle \frac{\partial {\tilde{\eta }}_{t|t-1}^{\left(c\right|g)}({\beta }^{*},{\widehat{\gamma }}_{CGQL}^{*}\left({\beta }^{*}\right))}{\partial {{\beta }^{*}}^{\prime }}}\hfill \\ & -& {\tilde{\eta }}_{t|t-1}^{\left(c\right|g)}({\beta }^{*},{\widehat{\gamma }}_{CGQL}^{*}\left({\beta }^{*}\right)){\displaystyle \sum _{\nu }^{C-1}}\left[\left\{\left({\delta }_{\left(t\right)\nu }\otimes x_t\right)+{\displaystyle \frac{\partial \left[{\widehat{\gamma }}_{\nu ,CGQL}^{\prime }\left({\beta }^{*}\right)\right]}{\partial {\beta }^{*}}}{y}_{t-1}^{\left(g\right)}\right\}\right.\hfill \\ & \times & \left.{\displaystyle \frac{\partial {\tilde{\eta }}_{t|t-1}^{\left(\nu \right|g)}({\beta }^{*},{\widehat{\gamma }}_{CGQL}^{*}\left({\beta }^{*}\right))}{\partial {{\beta }^{*}}^{\prime }}}\right].\hfill \end{array}$$

(56)

Proof. 

To derive the formula in (55), we first recall from Lemma 2 or more specifically from (26) that

$${\displaystyle \frac{\partial {\pi }_{\left(1\right)c}}{\partial {\beta }^{*}}}=\left[{\pi }_{\left(1\right)c}({\delta }_{\left(1\right)c}-{\pi }_{\left(1\right)})\right]\otimes x_1:(C-1)(p+1)\times 1.$$

A further derivative then produces

$$\begin{array}{ccc}& & {\displaystyle \frac{{\partial }^2{\pi }_{\left(1\right)c}\left({\beta }^{*}\right)}{\partial {\beta }^{*}\partial {{\beta }^{*}}^{\prime }}}={\displaystyle \frac{\partial }{\partial {{\beta }^{*}}^{\prime }}}\left[\left\{{\pi }_{\left(1\right)c}({\delta }_{\left(1\right)c}-{\pi }_{\left(1\right)})\right\}\otimes x_1\right]\hfill \\ & =& {\displaystyle \frac{\partial }{\partial {{\beta }^{*}}^{\prime }}}\left[\left(\begin{array}{c}-{\pi }_{\left(1\right)1}{\pi }_{\left(1\right)c}\\ ⋮\\ {\pi }_{\left(1\right)c}[1-{\pi }_{\left(1\right)c}]\\ ⋮\\ -{\pi }_{\left(1\right)(C-1)}{\pi }_{\left(1\right)c}\end{array}\right)\otimes x_1\right],\hfill \end{array}$$

(57)

which yields the formula in (55) after some algebraic calculations.

Computing the exact derivative matrix ${\displaystyle \frac{{\partial }^2{\tilde{\eta }}_{t|t-1}^{\left(c\right|g)}({\beta }^{*},{\widehat{\gamma }}_{CGQL}^{*}\left({\beta }^{*}\right))}{\partial {\beta }^{*}\partial {{\beta }^{*}}^{\prime }}}$ is algebraically complicated. The approximate Formula (56) follows from (30) under the assumption that ${\beta }^{*}$ involved in the derivative ${\displaystyle \frac{\partial {\widehat{\gamma }}_{c,CGQL}^{\prime }\left({\beta }^{*}\right)}{\partial {\beta }^{*}}}$ in (30) for $c=1,\dots ,C-1$ is known from a previous iteration.

We now provide the asymptotic distribution of ${\widehat{\beta }}^{*}$ (solution of (25) or (33)) as in the following theorem. □

Theorem 2.

Denote the covariance matrix of ${\overline{h}}_T\left({\beta }^{*}\right)$ computed in Lemma 6 by $\mathit{cov}\left({\overline{h}}_T\left({\beta }^{*}\right)\right)={\displaystyle \frac{1}{T^2}}{P}_T\left({\beta }^{*}\right).$ Next, assume that $(h_1\left({\beta }^{*}\right)+h_{T,2}\left({\beta }^{*}\right))$ in (49) satisfies the Lindeberg condition, that is,

$$\underset{T\to \infty }{\lim }{P}_T^{-1}\sum _{\left\{{(h_1+h_{T,2})}^{\prime }{P}_T^{-1}(h_1+h_{T,2})\right\}>ϵ}\left\{(h_1+h_{T,2}){(h_1+h_{T,2})}^{\prime }g(h_1+h_{T,2})\right\}=0$$

(58)

for all $ϵ>0,g(·)$ being the probability distribution of $(h_1+h_{T,2}).$ Then, the limiting distribution of ${\widehat{\beta }}^{*}$ (say, $q({\widehat{\beta }}^{*}$)) is normal, and is given by

$$\begin{array}{ccc}\underset{T\to \infty }{\lim }q{\left(\widehat{\beta }\right)}^{*}\hfill & \to & N\left({\beta }^{*},{\left[E_y{\displaystyle \frac{\partial (h_1+h_{T,2})}{\partial {{\beta }^{*}}^{\prime }}}\right]}^{-1}{P}_T\left({\beta }^{*}\right)\right.\hfill \\ & \times & \left.{\left[E_y{\displaystyle \frac{\partial (h_1+h_{T,2})}{\partial {{\beta }^{*}}^{\prime }}}\right]}^{-1}\right).\hfill \end{array}$$

(59)

Proof. 

The proof of this theorem is similar to that of Theorem 1. The difference lies between the forms of the functions $f_t(·)$ in Theorem 1 and $(h_1\left({\beta }^{*}\right)+h_{T,2}\left({\beta }^{*}\right)$ in the present theorem. Thus, by a similar justification as in (48), under some mild regularity condition, it follows that

$$\underset{T\to \infty }{\lim }{\widehat{\beta }}^{*}\to {\beta }^{*},$$

showing that ${\widehat{\beta }}^{*}$ is consistent for ${\beta }^{*}.$ Details are omitted.

Note that to compute the expected function in (59), that is, $E_y{\displaystyle \frac{\partial (h_1+h_{T,2})}{\partial {{\beta }^{*}}^{\prime }}},$ one may first assume that ${\beta }^{*}$ involved in the first-order derivatives in (50) is known from the previous iteration, and then an approximate second-order derivative as

$$\begin{array}{ccc}{\displaystyle \frac{\partial h_1\left({\beta }^{*}\right)}{\partial {{\beta }^{*}}^{\prime }}}\hfill & =& -{\displaystyle \sum _{c=1}^C}{\displaystyle \frac{y_{1c}}{{\pi }_{\left(1\right)c}^2\left({\beta }^{*}\right)}}{\displaystyle \frac{\partial {\pi }_{\left(1\right)c}\left({\beta }^{*}\right)}{\partial {\beta }^{*}}}{\displaystyle \frac{\partial {\pi }_{\left(1\right)c}\left({\beta }^{*}\right)}{\partial {{\beta }^{*}}^{\prime }}}, \mathrm{and}\hfill \\ {\displaystyle \frac{\partial h_{T,2}\left({\beta }^{*}\right)}{\partial {{\beta }^{*}}^{\prime }}}\hfill & =& -{\displaystyle \sum _{t=2}^T}{\displaystyle \sum _{g=1}^C}{\displaystyle \sum _{c=1}^C}\left[{\displaystyle \frac{y_{tc}}{{\left[{\tilde{\eta }}_{t|t-1}^{\left(c\right|g)}({\beta }^{*},{\widehat{\gamma }}_{CGQL}^{*}\left({\beta }^{*}\right))\right]}^2}}{\displaystyle \frac{\partial {\tilde{\eta }}_{t|t-1}^{\left(c\right|g)}({\beta }^{*},{\widehat{\gamma }}_{CGQL}^{*}\left({\beta }^{*}\right))}{\partial {\beta }^{*}}}{\displaystyle \frac{\partial {\tilde{\eta }}_{t|t-1}^{\left(c\right|g)}\left({\beta }^{*}{\widehat{\gamma }}_{CGQL}^{*}\left({\beta }^{*}\right)\right)}{\partial {{\beta }^{*}}^{\prime }}}\right],\hfill \end{array}$$

(60)

yielding the expectation as

$$\begin{array}{ccc}& & E_y{\displaystyle \frac{\partial (h_1+h_2)}{\partial {{\beta }^{*}}^{\prime }}}=-\left[{\displaystyle \sum _{c=1}^C}{\displaystyle \frac{1}{{\pi }_{\left(1\right)c}\left({\beta }^{*}\right)}}{\displaystyle \frac{\partial {\pi }_{\left(1\right)c}\left({\beta }^{*}\right)}{\partial {\beta }^{*}}}{\displaystyle \frac{\partial {\pi }_{\left(1\right)c}\left({\beta }^{*}\right)}{\partial {{\beta }^{*}}^{\prime }}}\right.\hfill \\ & +& \left.{\displaystyle \sum _{t=2}^T}{\displaystyle \sum _{g=1}^C}{\displaystyle \sum _{c=1}^C}\left\{{\displaystyle \frac{1}{{\tilde{\eta }}_{t|t-1}^{\left(c\right|g)}({\beta }^{*},{\widehat{\gamma }}_{CGQL}^{*}\left({\beta }^{*}\right))}}{\displaystyle \frac{\partial {\tilde{\eta }}_{t|t-1}^{\left(c\right|g)}({\beta }^{*},{\widehat{\gamma }}_{CGQL}^{*}\left({\beta }^{*}\right))}{\partial {\beta }^{*}}}{\displaystyle \frac{\partial {\tilde{\eta }}_{t|t-1}^{\left(c\right|g)}({\beta }^{*},{\widehat{\gamma }}_{CGQL}^{*}\left({\beta }^{*}\right))}{\partial {{\beta }^{*}}^{\prime }}}\right\}\right].\hfill \end{array}$$

(61)

 □

4. Concluding Remarks

There has been a considerable number of studies on the inferences for multinomial distributions in a panel data setup, where a small number of repeated multinomial responses are collected from a large number of independent individuals. The dimension reduction of parameter space is also considered. We, however, did not include such panel data methods for discussion in this paper as the inferences are quite different under the panel data and time series setups, because in a time series setup a large number of repeated multinomial responses are taken from one unit or individual only. Returning to the time series setup, we have discussed in the paper following [18] (see also [16,17]) that there may be negative inference effects when joint estimation is performed for a large multinomial parameter space involving regression and dynamic dependence parameters. As a possible remedy, we have offered a parameter split or dimension-reduction estimation approach. The asymptotic properties, such as the consistency of the estimators of the dynamic dependence function (for any unknown regression parameters) and then for the main regression parameters, are studied in detail.