Some Estimation Methods for a Random Coefficient in the Gegenbauer Autoregressive Moving-Average Model

Abstract

The Gegenbauer autoregressive moving-average (GARMA) model is pivotal for addressing non-additivity, non-normality, and heteroscedasticity in real-world time-series data. While primarily recognized for its efficacy in various domains, including the health sector for forecasting COVID-19 cases, this study aims to assess its performance using yearly sunspot data. We evaluate the GARMA model’s goodness of fit and parameter estimation specifically within the domain of sunspots. To achieve this, we introduce the random coefficient generalized autoregressive moving-average (RCGARMA) model and develop methodologies utilizing conditional least squares (CLS) and conditional weighted least squares (CWLS) estimators. Employing the ratio of mean squared errors (RMSE) criterion, we compare the efficiency of these methods using simulation data. Notably, our findings highlight the superiority of the conditional weighted least squares method over the conditional least squares method. Finally, we provide an illustrative application using two real data examples, emphasizing the significance of the GARMA model in sunspot research.

1. Introduction

Fitting a Gegenbauer autoregressive moving-average (GARMA) model to non-Gaussian data becomes intricate when real-world time series exhibit high anomalies, such as non-additivity and heteroscedasticity. Addressing this behavior has garnered significant interest, leading to numerous studies over the past few decades. For example, Albarracin et al. [1] analyzed the structure of GARMA models in practical applications, Huntet al. [2] proposed an R (Version R-4.4.0) package called ’garma’ to fit and forecast GARMA models, and Darmawan et al. [3] used a GARMA model to forecast COVID-19 data in Indonesia.

The conditional heteroscedastic autoregressive moving-average (CHARMA) model is commonly employed to capture unobserved heterogeneity characteristics in real-world data [4]. Another closely related model is the random coefficient autoregressive (RCA) model, introduced by Nicholls and Quinn [5], with recent investigations into its properties conducted by Appadoo et al [6].

The GARMA model has recently emerged as a suitable framework to identify and handle such features in real-world data under specific parameter values [7]. An inference for the estimators of the GARMA model was given by Beaumont and Smallwood [8], and an efficient estimation approach for the regression parameters of the generalized autoregressive moving-average model was provided by Hossain et al. [9].

In this context, this study proposes a random coefficient approach, namely the random coefficient Gegenbauer autoregressive moving-average (RCGARMA) model, to capture unobserved heterogeneity. The RCGARMA model extends the GARMA model by introducing an additional source of random variation to the standard coefficient model. While the GARMA model has been extensively analyzed with non-random coefficients (see [10,11,12], among others), the RCGARMA model provides flexibility in modeling unobserved heterogeneity in profit structures of dependent data, alongside long short-term dependence and seasonal fluctuations at different frequencies. Analyzing the statistical properties and estimation of this model is crucial for its application to real-world data.

To begin, we introduce the fundamentals of the random GARMA model, along with notations and commonly used assumptions. Throughout this paper, we focus on random coefficients in the GARMA model, akin to random-effect regression models, where the fixed coefficients of the GARMA model are randomly perturbed (see, for example, [13,14], and other relevant literature).

We consider a scenario in which we observe a sequence of random variables $X_t$, generated by the following recursive model [15]:

$$X_t=\sum _{n=1}^{+\infty }\mathrm{\Theta }\left(n\right)X_{t-n}+{\epsilon }_t,\mathrm{t}\in \mathbb{Z}$$

(1)

where ${\epsilon }_t$ is the noise, which is assumed to have zero mean and variance ${\sigma }_{\epsilon }^2$, and $\mathrm{\Theta }\left(n\right)$ is determined by the following assumptions:

$$\mathrm{\Theta }\left(n\right)=\varphi C_{n-1}^{(-d)}\left(\eta \right)-C_n^{(-d)}\left(\eta \right)$$

(2)

where $C_n^{-d}\left(\eta \right)={\displaystyle \sum _{k=0}^{\left[\frac{n}{2}\right]}}{(-1)}^k\frac{\mathrm{\Gamma }(n-k-d)}{\mathrm{\Gamma }(-d)k!(n-2k)!}{\left(2\eta \right)}^{n-2k}$ are the Gegenbauer polynomial coefficients, defined in terms of their generating function (see, for instance, Magnus et al. [16] and Rainville and Earl [17]), as follows:

$${(1-2\eta L+L^2)}^d=\sum _{n=0}^{\infty }{C}_n^{-d}\left(\eta \right)L^n$$

where $\mathrm{\Gamma }(.)$ denotes the Gamma function and $\left[\frac{n}{2}\right]$ stands for the integer part of $\frac{n}{2}$ [2].

Therefore, in this study, we extend the GARMA models to RCGARMA models, as discussed above, by including dynamic random coefficient effects. We assume that the row vector of coefficients, $\mathrm{\Theta }\left(n\right)$ in Equation (2), that gives the impact of the time-varying variables $\left(X_{t-n}\right)$ on $X_t$ in Equation (1), initially presumed to be fixed and time-invariant parameters, does change over time. In this scenario, the parameter vector can be partitioned as $\varphi +{\varphi }_t$, where $\varphi$ is constant reflecting the fixed coefficients, and ${\varphi }_t$ denotes the sequence of random coefficients related to the nuisance parameters that can be omitted from the model. To identify this model, we assume that $E\left({\varphi }_t\right)=0$ and $E\left({\varphi }_t^2\right)={\sigma }_{{\varphi }_t}^2$. It is also assumed that ${\varphi }_t$ is uncorrelated with ${\epsilon }_t$, and that $cov({\varphi }_t,{\epsilon }_s)=0$ for all t and s. We use $\mathrm{\Theta }(n,t)=(\varphi +{\varphi }_t)C_{n-1}^{(-d)}\left(\eta \right)-C_n^{(-d)}\left(\eta \right)$ to represent the set of all model coefficients, then the general model (Equation (1)) becomes:

$$X_t=\sum _{n=1}^{+\infty }\mathrm{\Theta }(n,t)X_{t-n}+{\epsilon }_t,\mathrm{t}\in \mathbb{Z}$$

(3)

The concept of fixed and random coefficient time-series models has been applied for testing the presence of random coefficients in autoregressive models [18], and for handling the possible nonlinear features of real-world data [19]. Another relevant reference is the study by Mundlak [20], which introduced the dynamic random effects model for panel data analysis. Mundlak’s work highlights the importance of incorporating time-varying random effects into panel data.

In this context, it can be implicitly assumed that truncating the right-hand side of Equation (3) at lag m is valid for n = 0, 1, …, m. Under this assumption, the RCGARMA model should be accurately formulated as follows:

$$X_{t,m}={\mathrm{\Theta }}_t^{\prime }X(t-1)+{\epsilon }_t,\mathrm{t}\in \mathbb{Z}$$

(4)

where the sequence $\left\{X_{t,m}\right\}$ represents the truncated Gegenbauer process, with its behavior contingent on the selected finite truncation lag order denoted by $m=1,2,3,$ …$,M<\infty$. This concept draws parallels to the MA approximation presented by Dissanayake et al. [21], along with comprehensive literature reviews on diverse issues in long-memory time series, encompassing Gegenbauer processes and their associated properties. ${\mathrm{\Theta }}_t=(\mathrm{\Theta }(1,t),\mathrm{\Theta }(2,t),$…${,\mathrm{\Theta }(m,t))}^{\prime }$ is an $(m\times 1)$ vector of random coefficients. $X(t-1)=(X_{t-1,m},$…$,X_{t-m,m}{)}^{\prime }$ is an $(m\times 1)$ vector of past observations.

Concerning the estimation of the unknown parameters of interest, we define a vector denoted as $(\mathrm{\Theta },V_{\mathrm{\Theta }},{\sigma }_{\mathrm{\Theta }\epsilon },{\sigma }_{\epsilon }^2)$. Under the assumption that the random sequences ${\mathrm{\Theta }}_t$ are allowed to exhibit correlations with the error process ${\epsilon }_t$, we represent

$$E\left(\begin{array}{c}{\mathrm{\Theta }}_t\\ {\epsilon }_t\end{array}\right)=\left(\begin{array}{c}\mathrm{\Theta }\\ 0\end{array}\right), \mathit{Var}\left(\begin{array}{c}{\mathrm{\Theta }}_t\\ {\epsilon }_t\end{array}\right)=\left(\begin{array}{cc}{V}_{\mathrm{\Theta }}& {\sigma }_{\mathrm{\Theta }\epsilon }\\ {\sigma }_{\mathrm{\Theta }\epsilon }^{\prime }& {\sigma }_{\epsilon }^2\end{array}\right), \mathrm{\Theta }=({\mathrm{\Theta }}_1, {\mathrm{\Theta }}_2, \dots , {\mathrm{\Theta }}_m)’,$$

where $V_{\mathrm{\Theta }}$ is the $(m\times m)$ matrix representing the variance of ${\mathrm{\Theta }}_t$, ${\sigma }_{\mathrm{\Theta }\epsilon }$ is the $(m\times 1)$ vector representing the covariance between ${\mathrm{\Theta }}_t$ and ${\epsilon }_t$, and ${\sigma }_{\epsilon }^2$ is the variance of the error process ${\epsilon }_t$.

The ordinary least squares (OLS) method is commonly used in this case. This method aims to estimate the parameters by minimizing the sum of squared differences between the observed and predicted values, and it assumes the independence, homoskedasticity, and normality of error distribution.

However, the assumptions of our model do not align with those of ordinary least squares estimation. While OLS assumes independence and homoskedasticity of errors, our model may exhibit heteroskedasticity and correlation structures due to the presence of random effects. Additionally, the errors in our model are not strictly bound to a normal distribution.

To address these issues, we employ an estimation procedure using conditional least squares (CLS) and weighted least squares (WLS) estimators. CLS adjusts for heteroskedasticity by incorporating the conditional variance structure into the estimation process, while WLS provides more robust estimates compared to OLS by assigning weights to observations based on their variances. This methodology is based on the work proposed by Hwang and Basawa [22], which has demonstrated favorable performance for generalized random coefficient autoregressive processes. Additionally, the studies by Nicholls and Quinn [5] and Hwang and Basawa [23] are also relevant in this context.

By implementing these procedures, one can obtain significant estimates for random effects. This paper begins by assessing the parameters using the conditional least squares estimation method in Section 2. Section 3 introduces an alternative estimator based on the weighted conditional least squares estimation method. Following this, Section 4 and Section 5 compare the performance of these methods using simulation data and real-world data, respectively.

2. Conditional Least Squares Estimation Method

The conditional least squares estimation method is a flexible technique commonly used for estimation. It offers valuable characteristics like consistency and asymptotic normality under specific conditions. This approach is valuable because it helps ensure that the estimated coefficients approach the true values as more data are obtained, which is known as consistency. Furthermore, the asymptotic normality property indicates that as the sample size increases, the distribution of the estimators approximates a normal distribution. By minimizing the sum of squared deviations, we are essentially finding the best-fitting line or curve that represents the relationship between variables in the data, making it a widely used and reliable technique in statistical analysis.

The utilization of conditional least squares estimation in the context of GARMA models with random effects provides a robust approach to estimating model parameters while accounting for the inherent uncertainties and complexities introduced by random effects. Conditional least squares estimation in random effects GARMA models stands out for its capability to handle issues stemming from unobserved heterogeneity and time-varying dynamics [24]. This approach enables researchers to discern the influence of random effects on model parameters, thereby enhancing the interpretability and robustness of the estimated coefficients [25].

Moreover, the interpretability of the results obtained through CLS estimation in GARMA models with random effects is augmented by the incorporation of a conditional framework. By conditioning the estimation on information present at each time point, researchers can acquire a deeper understanding of the temporal progression of model parameters and their individual impacts on the observed data [26].

In this section, we use the conditional least squares method to estimate the unknown parameters in the RCGARMA model. Viewing it as a regression model with the predictor variable $X_{t-n}$ and response variable $X_t$, the least squares estimation involves minimizing the sum of squares of the differences. The initial step involves estimating the vector mean parameters $\mathrm{\Theta }=E\left(\mathrm{\Theta }\right)$ of the regression function (see Equation (4)).

2.1. Estimation of Parameters $\mathrm{\Theta }$

In this section, we discuss the estimation of $\mathrm{\Theta }$ through the utilization of the conditional least squares estimation procedure. The estimator, denoted as $\widehat{\mathrm{\Theta }}$, represents the optimal selection of values for our parameters and is calculated using a sample ($X_{-m+1},$…$,X_0,X_1,$…$.X_n$). When performing conditional least squares estimation, the objective is to minimize the following conditional sum of squares:

$$Q=\sum _{t=1}^n{[X_t-E(X_{t,m}/X(t-1))]}^2$$

with respect to the vector $\mathrm{\Theta }=({\mathrm{\Theta }}_1,$…$,{\mathrm{\Theta }}_m{)}^{\prime }$. This is achieved by solving the system $dQ/d\mathrm{\Theta }=0$.

Replacing $E(X_{t,m}/X(t-1))$ in the last equation with ${\mathrm{\Theta }}^{\prime }X(t-1)$ yields the following result:

$$\stackrel{\widehat{\mathrm{\Theta }}}{(m\times 1)}={\left(\sum _{t=1}^{n}X(t-1)X^{\prime }(t-1)\right)}^{-1}\left(\sum _{t=1}^n{X}_{t,m}X(t-1)\right)$$

(5)

The derivation of the asymptotic properties of our estimator $\widehat{\mathrm{\Theta }}$ relies on the following standard conditions:

(C.0)

The square matrix $E\left(X^{\prime }(t-1)X(t-1)\right)$ must have full rank.

(C.1)

The stationary distribution of $\left\{X_{t,m}\right\}$ must have a fourth-order moment, meaning that $E\left|X^4\right|<\infty$.

In time-series analysis, condition (C.0) is crucial for the estimation process. It requires the square matrix $E\left(X^{\prime }(t-1)X(t-1)\right)$ to have full rank. This condition essentially ensures that the information provided by the data is sufficient and not redundant, allowing for accurate estimation of the parameters. When this condition is met, it facilitates reliable inference and prediction based on the time-series data. The condition in the time series, identified as (C.1), states that the stationary distribution of $\left\{X_{t,m}\right\}$ needs to exhibit a fourth-order moment, indicating that $E\left|X^4\right|<\infty$. Now, assuming that these two conditions are verified, we have the following theorem, which describes the limit distribution of the estimator of $\mathrm{\Theta }$.

Theorem 1. 

Let $\widehat{\mathrm{\Theta }}$ be the estimator of Θ. Under (C.0) and (C.1), we have:

$$\sqrt{n}(\widehat{\mathrm{\Theta }}-\mathrm{\Theta })\stackrel{d}{\to }{\mathbb{N}}_m(0,{\mathrm{\Gamma }}^{-1}W{\mathrm{\Gamma }}^{-1}),\hspace{1em}as\hspace{1em}n⟶+\infty$$

where

$$\stackrel{\mathrm{\Gamma }}{(m\times m)}=Cov(X_1,X_2,\dots ,X_m)$$

and

$$\stackrel{W}{(m\times m)}=E\left({\beta }_t\left(X(t-1)X^{\prime }(t-1)\right)\right)$$

with

$${\beta }_t=Var(X_{t,m}/X(t-1))=X^{\prime }(t-1)V_{\mathrm{\Theta }}X(t-1)+2X(t-1){\sigma }_{\mathrm{\Theta }\epsilon }+{\sigma }_{\epsilon }^2$$

(6)

Proof. 

To prove the result of this theorem, let

$$\begin{array}{cc}\hfill \widehat{\mathrm{\Theta }}-\mathrm{\Theta }& ={\left(\sum _{t=1}^{n}X(t-1)X^{\prime }(t-1)\right)}^{-1}\left(\sum _{t=1}^n{X}_{t,m}X(t-1)\right)-\mathrm{\Theta }\hfill \\ & ={\left(\sum _{t=1}^{n}X(t-1)X^{\prime }(t-1)\right)}^{-1}(\sum _{t=1}^n{X}_{t,m}X(t-1)-\sum _{t=1}^{n}X(t-1)X^{\prime }(t-1)\mathrm{\Theta })\hfill \\ & ={\left(\sum _{t=1}^{n}X(t-1)X^{\prime }(t-1)\right)}^{-1}\left(\sum _{t=1}^n(X_{t,m}-{\mathrm{\Theta }}^{\prime }X(t-1))X(t-1)\right)\hfill \end{array}$$

Let ${\stackrel{\sim }{\mathrm{\Theta }}}_t=(\stackrel{\sim }{\mathrm{\Theta }}(t,1),$…$,\stackrel{\sim }{\mathrm{\Theta }}(t,m))$, where $\stackrel{\sim }{\mathrm{\Theta }}(t,j)=\mathrm{\Theta }(t,j)-\mathrm{\Theta }\left(j\right)$, j = 1, …, m. Consider

$$\begin{array}{cc}\hfill n^{-\frac{1}{2}}\sum _{t=1}^n(X_{t,m}-{\mathrm{\Theta }}^{\prime }X(t-1))X(t-1)& =n^{-\frac{1}{2}}\sum _{t=1}^n({\epsilon }_t+{\stackrel{\sim }{\mathrm{\Theta }}}_{t}X(t-1))X(t-1)\hfill \\ & =n^{-\frac{1}{2}}\sum _{t=1}^n{G}_t\hfill \end{array}$$

where

$$G_t=({\epsilon }_t+{\stackrel{\sim }{\mathrm{\Theta }}}_{t}X(t-1))X(t-1)$$

Given that $G_t$ is a stationary ergodic zero-mean martingale difference for all t = 1, …, m, we apply the central limit theorem for stationary ergodic martingale differences proposed by  Billingsley [27].

Thus, we find that

$$n^{-1}\sum _{t=1}^n{G}_t\stackrel{d}{\to }{\mathbb{N}}_m(0,W)$$

(7)

Furthermore, by the ergodic theorem, we have

$$n^{-1}\sum _{t=1}^{n}X(t-1)X^{\prime }(t-1)\stackrel{p}{\to }\mathrm{\Gamma }$$

(8)

Finally, from (7) and (8), we find that

$$\sqrt{n}(\widehat{\mathrm{\Theta }}-\mathrm{\Theta })\stackrel{d}{\to }{\mathbb{N}}_m(0,{\mathrm{\Gamma }}^{-1}W{\mathrm{\Gamma }}^{-1})$$

   □

Therefore, the result of Theorem 1 is proven.

In the following section, we calculate the estimators for the variance component parameters of the RCGARMA model described earlier. These parameters define the variance of random errors, the covariances between errors and random effects, and the variance matrix parameters of random effects in the model. The covariance matrix is typically expressed as a mysterious linear combination of known cofactor matrices, as shown below.

2.2. Covariance Parameter Estimators

Let $\upsilon$ be the unknown parameter vector containing all model parameters representing the variance components. We partition the parameter vector into three sub-vectors: ${\sigma }_{\epsilon }^2$ includes all parameters related to the time-invariant variance of errors, ${\sigma }_{\mathrm{\Theta }\epsilon }$ includes all covariance parameters in the RCGARMA process, and $V_{ij}$ includes parameters for the matrix elements of the variance of random effects in the RCGARMA process. Specifically, it can be noted as:

$$\underset{k\times 1}{\upsilon }={({\sigma }_{\epsilon }^2,{\sigma }_{\mathrm{\Theta }\epsilon },V_{11},V_{22},\dots ,V_{mm},V_{12},V_{23},\dots ,V_{m-1m},\dots ,V_{1m})}^{\prime }$$

where $l=\frac{1}{2}(m+1)(m+2)$ and $V_{ij}$ denotes the (i,j)th element of the variance matrix of $\mathrm{\Theta }$, $V_{\mathrm{\Theta }}$.

To estimate the variance component parameters in the RCGARMA model using the least squares estimator, assume the available dataset is $(X_{t-m},$…$,X_0,X_1,$…$,X_n)$, and note that $R_t\left(\mathrm{\Theta }\right)$ and ${\beta }_t$ in Equation (6) are, respectively, given as follows:

$$R_t\left(\mathrm{\Theta }\right)=X_{t,m}-E(X_{t,m}/X(t-1))\mathrm{and}{\beta }_t\left(\upsilon \right)=E(R_t{\left(\mathrm{\Theta }\right)}^2/X(t-1)).$$

The conditional least squares estimator is used to estimate the parameters $\upsilon$. This method allows researchers to decompose the total variability observed in the data into different components, including the variance contributed by random model effects. A study by Ngatchou-Wandji [28] applied conditional least squares to estimate the parameters in a class of heteroscedastic time-series models, demonstrating its consistency and its asymptotic normality.

By utilizing conditional least squares to estimate variance components in the RCGARMA model (e.g., GARMA models with random effects), the estimator, denoted as $\widehat{\upsilon }$, is acquired by minimizing the sum of squares represented by ${\sum }_{t=1}^n{(R_t^2\left(\mathrm{\Theta }\right)-{\beta }_t\left(\upsilon \right))}^2$. This process allows us to derive the conditional least squares estimator $\widehat{\upsilon }=\widehat{\upsilon }\left(\widehat{\mathrm{\Theta }}\right)$, where $\widehat{\mathrm{\Theta }}$ is given by Equation (5) for $\upsilon$. The estimator is obtained as a solution to the equation represented by:

$$\sum _{t=1}^n(R_t^2\left(\mathrm{\Theta }\right)-{\beta }_t\left(\upsilon \right))\frac{\partial {\beta }_t\left(\upsilon \right)}{\partial \upsilon }=0$$

The derivation of the asymptotic properties of our estimator $\widehat{\mathrm{\Theta }}$ relies on the following standard conditions: (C.0), as described above, and (C.2), which states that the stationary distribution of $\left\{X_{t,m}\right\}$ must have an eight-order moment, meaning that $E\left|X^8\right|<\infty$.

Then, it can be shown that $\sqrt{n}(\widehat{\upsilon }-\upsilon )$ can converge in distribution to a multinormal matrix, as given in the theorem below.

Theorem 2. 

Under (C.0) and (C.2), we have:

$$\sqrt{n}(\widehat{\upsilon }-\upsilon )\stackrel{d}{\to }{\mathbb{N}}_m(0,L^{-1}\Sigma L^{-1}), as n⟶+\infty$$

where

$$\stackrel{L}{(k\times k)}=E\left(\left(\frac{\partial {\beta }_t\left(\upsilon \right)}{\partial \upsilon }\right){\left(\frac{\partial {\beta }_t\left(\upsilon \right)}{\partial \upsilon }\right)}^{\prime }\right)$$

and

$$\stackrel{\Sigma }{(k\times k)}=E\left({(R_t^2\left(\mathrm{\Theta }\right)-{\beta }_t\left(\upsilon \right))}^2\left(\frac{\partial {\beta }_t\left(\upsilon \right)}{\partial \upsilon }\right){\left(\frac{\partial {\beta }_t\left(\upsilon \right)}{\partial \upsilon }\right)}^{\prime }\right)$$

Proof. 

See [22].    □

3. Weighted Conditional Least Squares Estimation Method

The weighted conditional least squares estimation method for generalized autoregressive moving-average (GARMA) models with random effects is a sophisticated statistical technique used to improve efficiency in the presence of heteroskedasticity. The traditional ordinary least squares estimation used above may not be the optimal choice when dealing with data exhibiting varying error variances, as it does not take this heteroskedasticity into account. In such cases, a more suitable approach involves employing a conditional weighted least squares estimator for the parameter of interest, denoted as $\mathrm{\Theta }$. By incorporating weighted factors into the estimation process, this method assigns different weights to observations based on the variability of their error terms.

This allows for a more precise estimation of the parameters in the presence of heteroscedasticity in GARMA models, ultimately leading to more reliable statistical inference. This approach is particularly crucial when faced with heteroskedasticity in the data, as mentioned in $Var(X_{t,m}/X(t-1))$ in Equation (3).

The recognition of this heteroskedasticity highlights the need for more advanced estimation techniques to ensure the accuracy and reliability of the statistical analysis. This section concentrates on implementing this approach. We seek to enhance efficiency by employing a conditional weighted least squares estimator for $\mathrm{\Theta }$.

Assuming that the nuisance parameter $\upsilon$ is known, the weighted conditional least squares estimator ${\widehat{\mathrm{\Theta }}}_W\left(\upsilon \right)$ of $\mathrm{\Theta }$ is obtained by minimizing:

$$\sum _{t=1}^n\left(\frac{{(X_{t,m}-E(X_{t,m}/X(t-1)))}^2}{Var(X_{t,m}/X(t-1))}\right)$$

Since $E(X_{t,m}/X(t-1))={\mathrm{\Theta }}^{\prime }X(t-1)$ and $Var(X_{t,m}/X(t-1))={\beta }_t\left(\upsilon \right)$, the estimator ${\widehat{\mathrm{\Theta }}}_W\left(\beta \right)$ is given by:

$${\widehat{\mathrm{\Theta }}}_W\left(\upsilon \right)={\left(\sum _{t=1}^n\left(\frac{X(t-1)X^{\prime }(t-1)}{{\beta }_t\left(\upsilon \right)}\right)\right)}^{-1}\left(\sum _{t=1}^n\left(\frac{X(t-1)X_{t,m}}{{\beta }_t\left(\upsilon \right)}\right)\right)$$

Consider the following conditions:

(C.3)

$E\left(\frac{X\left(0\right)X^{\prime }\left(0\right)}{{\beta }_1\left(\upsilon \right)}\right)<\infty$.

(C.4)

The differentiability of $\frac{1}{{\beta }_t(.)}$ at $\widehat{\upsilon }$ is established as follows:

There exists a linear map L such that:

$$\frac{1}{{\beta }_t\left(\upsilon \right)}-\frac{1}{{\beta }_t\left(\widehat{\upsilon }\right)}=L(\upsilon -\widehat{\upsilon })+o_P\left(|\upsilon -\widehat{\upsilon }|\right)$$

Then, we show that $\sqrt{n}({\widehat{\mathrm{\Theta }}}_W\left(\upsilon \right)-\mathrm{\Theta })$ converges in distribution in the theorem below.

Theorem 3. 

Under (C.0), (C.1), and (C.3), we have:

$$\sqrt{n}({\widehat{\mathrm{\Theta }}}_W\left(\upsilon \right)-\mathrm{\Theta })\stackrel{d}{\to }{\mathbb{N}}_m(0,S)$$

where

$$S={\left(E\left({\displaystyle \frac{X\left(0\right)X^{\prime }\left(0\right)}{{\beta }_1\left(\upsilon \right)}}\right)\right)}^{-1}$$

Proof. 

Note that

$${\mathrm{\Theta }}_W\left(\upsilon \right)-\mathrm{\Theta }={\left(\sum _{t=1}^n\frac{X(t-1)X^{\prime }(t-1)}{{\beta }_t\left(\upsilon \right)}\right)}^{-1}\left(\sum _{t=1}^n{\displaystyle \frac{(X_{t,m}-{\mathrm{\Theta }}^{\prime }X(t-1))X(t-1)}{{\beta }_t\left(\upsilon \right)}}\right)$$

(9)

Using the ergodic theorem, we find that the first factor of (9) converges strongly:

$${\displaystyle \frac{1}{n}}\sum _{t=1}^n\frac{X(t-1)X^{\prime }(t-1)}{{\beta }_t\left(\upsilon \right)}\underset{n\to +\infty }{\stackrel{a.s}{⟶}}E\left({\displaystyle \frac{X\left(0\right)X^{\prime }\left(0\right)}{{\beta }_1\left(\upsilon \right)}}\right)$$

(10)

The next step is to check the convergence of the second factor of (9):

Let $Z_t={\displaystyle \frac{(X_{t,m}-{\mathrm{\Theta }}^{\prime }X(t-1))X(t-1)}{{\beta }_t\left(\upsilon \right)}}$. We have $E(Z_t/X(t-1))=0$, and under (C.3), $E\left(Z_1^2\right)<\infty$.

So, via the martingale central limit theorem of  Billingsley [27]:

$${\displaystyle \frac{1}{\sqrt{n}}}\sum _{t=1}^n{Z}_t\sim {\mathbb{N}}_m(0,E\left(Z_1^2\right))$$

(11)

Finally, from Equations (10) and (11), we have:

$$\sqrt{n}({\widehat{\mathrm{\Theta }}}_W\left(\upsilon \right)-\mathrm{\Theta })\stackrel{d}{\to }{\mathbb{N}}_m(0,E{\left(Z_1^2\right)}^{-1})$$

   □

When $\beta$ is unknown, we replace it in ${\widehat{\mathrm{\Theta }}}_W$ with $\widehat{\upsilon }$. Then, we denote ${\widehat{\mathrm{\Theta }}}_W={\widehat{\mathrm{\Theta }}}_W\left(\widehat{\upsilon }\right)$. Herein, we give the limit distribution of ${\widehat{\mathrm{\Theta }}}_W$.

Theorem 4. 

Under (C.0) and (C.2)–(C.4), we have:

$$\sqrt{n}({\widehat{\mathrm{\Theta }}}_W-\mathrm{\Theta })\stackrel{d}{\to }{\mathbb{N}}_m(0,S),\hspace{1em}as\hspace{1em}n⟶+\infty$$

Proof. 

Note that

$${\widehat{\mathrm{\Theta }}}_W-\mathrm{\Theta }={\left(\frac{1}{n}\sum _{t=1}^n\frac{X(t-1)X^{\prime }(t-1)}{{\beta }_t\left(\widehat{\upsilon }\right)}\right)}^{-1}\left(\frac{1}{\sqrt{n}}\sum _{t=1}^n{\displaystyle \frac{{\epsilon }_{t}X(t-1)}{{\beta }_t\left(\widehat{\upsilon }\right)}}\right)$$

and

$${\widehat{\mathrm{\Theta }}}_W\left(\upsilon \right)-{\widehat{\mathrm{\Theta }}}_W={\widehat{\mathrm{\Theta }}}_W\left(\upsilon \right)-\mathrm{\Theta }-({\widehat{\mathrm{\Theta }}}_W-\mathrm{\Theta })=A^{-1}B-{\widehat{A}}^{-1}\widehat{B}$$

where

• $\widehat{A}=\frac{1}{n}{\sum }_{t=1}^n\frac{X(t-1)X^{\prime }(t-1)}{{\beta }_t(\widehat{\upsilon )}}$, $\widehat{B}=\frac{1}{\sqrt{n}}{\sum }_{t=1}^n{\displaystyle \frac{{\epsilon }_{t}X(t-1)}{{\beta }_t\left(\widehat{\upsilon }\right)}}$
• $A=\frac{1}{n}{\sum }_{t=1}^n\frac{X(t-1)X^{\prime }(t-1)}{{\beta }_t\left(\upsilon \right)}$ and $B=\frac{1}{\sqrt{n}}{\sum }_{t=1}^n{\displaystyle \frac{{\epsilon }_{t}X(t-1)}{{\beta }_t\left(\upsilon \right)}}$

First, we show that $A^{-1}-{\widehat{A}}^{-1}\stackrel{P}{\to }0$:

We have:

$$A-\widehat{A}=\frac{1}{n}\sum _{t=1}^{n}X(t-1)X^{\prime }(t-1)({\displaystyle \frac{1}{{\beta }_t\left(\upsilon \right)}}-{\displaystyle \frac{1}{{\beta }_t\left(\widehat{\upsilon }\right)}})$$

Using Theorem 2 and under (C.4), we find that:

$${\displaystyle \frac{1}{{\beta }_t\left(\upsilon \right)}}-{\displaystyle \frac{1}{{\beta }_t\left(\widehat{\upsilon }\right)}}\stackrel{P}{\to }0$$

Then,

$$A-\widehat{A}\stackrel{P}{\to }0$$

Finally,

$$A^{-1}-{\widehat{A}}^{-1}\stackrel{P}{\to }0$$

(12)

Next, we show that $B-\widehat{B}\stackrel{P}{\to }0$.

We have

$$A-\widehat{A}=\frac{1}{n}\sum _{t=1}^n{\epsilon }_{t}X(t-1)({\displaystyle \frac{1}{{\beta }_t\left(\upsilon \right)}}-{\displaystyle \frac{1}{{\beta }_t\left(\widehat{\upsilon }\right)}})$$

Under (C.4) and Theorem 2, we find that:

$$B-\widehat{B}\stackrel{P}{\to }0$$

(13)

Using Equations (12) and (13) and the equality $\widehat{A}\widehat{B}-AB=\widehat{A}(\widehat{B}-B)+(\widehat{A}-A)B$, we find that:

$${\widehat{\mathrm{\Theta }}}_W\left(\upsilon \right)-{\widehat{\mathrm{\Theta }}}_W\stackrel{P}{\to }0$$

Finally, from Slutsky’s theorem [29], we prove that:

$$\sqrt{n}({\widehat{\mathrm{\Theta }}}_W-\mathrm{\Theta })\stackrel{d}{\to }{\mathbb{N}}_m(0,S),\hspace{1em}\mathrm{as}\hspace{1em}n⟶+\infty$$

   □

4. Comparison of Methods by Simulation

A simulation was designed to compare the performance of the two estimators—the conditional least squares estimator and the weighted conditional least squares estimator—where we applied a GARMA model with random effects (RCGARMA) and investigated the behavior of the proposed approximate ratio for these two realizations using the following expression:

$$RatMSE={\displaystyle \frac{E\left({(\widehat{\mathrm{\Theta }}-\mathrm{\Theta })}^{\prime }(\widehat{\mathrm{\Theta }}-\mathrm{\Theta })\right)}{E\left({({\widehat{\mathrm{\Theta }}}_W-\mathrm{\Theta })}^{\prime }({\widehat{\mathrm{\Theta }}}_W-\mathrm{\Theta })\right)}}$$

where $\widehat{\mathrm{\Theta }}$ and ${\widehat{\mathrm{\Theta }}}_W$ are, respectively, the conditional least squares estimator and the weighted conditional least squares estimator. R (Version R-4.4.0) programs were used to generate the data from the GARMA(1,0) model, with the random coefficients (we utilized the code provided in Appendix A) defined as follows:

$$X_{t,m}={\mathrm{\Theta }}_t^{\prime }X(t-1)+{\epsilon }_t,t=1,2, \dots , \mathrm{n}$$

where $\eta =1$ and $d=-0.3$; ${\epsilon }_t\sim \mathcal{N}(0,1)$; $\mathrm{\Theta }(k,t)=(\varphi +{\varphi }_t)C_{k-1}^{\left(0.3\right)}(-1)-C_k^{\left(0.3\right)}(-1)$, with $\varphi =0.2$ and ${\varphi }_t\sim \mathcal{N}(0,{\sigma }_{\varphi }^2)$; $\mathrm{\Theta }=E\left({\mathrm{\Theta }}_t\right)=({\mathrm{\Theta }}_1,{\mathrm{\Theta }}_2,$…$,{\mathrm{\Theta }}_m{)}^{\prime }$, with ${\mathrm{\Theta }}_k=\varphi C_{k-1}^{\left(0.3\right)}(-1)-C_k^{\left(0.3\right)}(-1)$; $V_{\varphi }=\left(\begin{array}{cccc}{V}_{11}& V_{12}& \cdots & V_{1m}\\ ⋮& ⋮& \ddots & ⋮\\ V_{m1}& V_{m2}& \cdots & V_{mm}\end{array}\right)$, with $V_{jk}=C_j^{\left(0.3\right)}(-1)C_k^{\left(0.3\right)}(-1){\sigma }_{\varphi }^2;$ $\mathrm{and} \mathit{Cov}({\mathrm{\Theta }}_t, {\epsilon }_t)=({\sigma }_{\mathrm{\Theta }\epsilon ,1}, {\sigma }_{\mathrm{\Theta }\epsilon ,2},\dots ,{\sigma }_{\mathrm{\Theta }\epsilon ,m})’$, with ${\sigma }_{\mathrm{\Theta }\epsilon ,k}=C_k^{\left(0.3\right)}(-1){\sigma }_{\varphi }^{\epsilon }$.

In this study, we performed simulations and created essential tables and graphs. To accomplish this, realizations were generated with sample sizes of $n=100,200,500,1000$ for various values of ${\sigma }_{\varphi }^2$ (ranging from 0.1 to 0.5) and ${\sigma }_{\varphi \epsilon }$ (ranging from −0.8 to 0.8). Additionally, m was set equal to $n-1$, and 100 replications were conducted. Replication proved somewhat challenging due to the complexity of the techniques involved. The models contained numerous parameters, making convergence difficult to achieve.

Each section of

Table 1. The results for the sample estimates of the ratio of mean squared errors of the estimators $\widehat{\mathrm{\Theta }}$ and ${\widehat{\mathrm{\Theta }}}_W$ for sample sizes $n=100$, $n=200$, $n=500$, and $n=1000$ (out of 100 replications).

n		${\mathit{\sigma }}_{\mathit{\varphi }}^2$	0.1	0.2	0.3	0.4	0.5
	${\mathit{\sigma }}_{\mathit{\varphi }\mathit{\epsilon }}$
100	−0.8		1.879	2.324349	2.742248	3.038494	3.402452
	−0.3		1.414706	2.302044	2.512617	2.814392	3.03135
	0		1.49661	1.922926	2.354512	2.836419	2.913413
	0.3		2.1155329	2.222519	2.663681	2.88795	3.376377
	0.8		2.017635	2.663554	3.169506	3.473052	3.324323
200	−0.8		1.545888	2.401473	2.674726	3.342016	3.415802
	−0.3		1.464456	2.220699	2.494791	3.179188	3.527889
	0		1.452352	2.230288	2.41021	2.804334	3.226655
	0.3		2.2502556	2.5625583	2.7882553	3.119562	3.522819
	0.8		1.908849	2.265391	3.579418	3.583716	3.399324
500	−0.8		1.333323	1.978135	2.830228	3.764042	4.11792
	−0.3		1.337001	1.72787	2.005013	2.695551	3.56467
	0		1.31299	2.408952	2.433346	2.752368	2.729764
	0.3		1.491299	2.010183	2.7554335	3.145245	3.101716
	0.8		1.603849	2.190652	3.287553	3.6035565	3.811159
1000	−0.8		1.398848	1.836679	3.176915	3.82384	3.91652
	−0.3		1.537031	1.82593	2.15439	2.62439	3.25078
	0		1.363849	1.906971	2.24641	2.71785	2.643287
	0.3		1.439188	1.643234	1.95662	2.341102	2.399511
	0.8		1.585528	2.134298	2.33947	2.893886	2.628264

presents the ratio of mean squared errors for different combinations of ${\sigma }_{\varphi }^2$ and ${\sigma }_{\varphi \epsilon }$ with the same length of series n. These findings are also depicted in the four panels in Figure 1.

The ratio of mean squared errors serves as a comparative measure, where values exceeding 1 (>1) indicate that the second estimator performs better than the first. This comparison enables an assessment of the relative performance between the two estimators.

When analyzing the results presented in Table 1 and visualized in Figure 1, it is evident that all ratios of mean squared errors surpassed the threshold of 1. This observation underscores the superior efficiency of the ${\widehat{\mathrm{\Theta }}}_W$ estimator over $\widehat{\mathrm{\Theta }}$ across various parameter combinations.

These findings reinforce the validity and robustness of the simulation study, providing empirical evidence in support of the enhanced performance of the ${\widehat{\mathrm{\Theta }}}_W$ estimator in comparison to $\widehat{\mathrm{\Theta }}$.

5. Real-World Data Examples

In this section, we apply the conditional least squares and weighted least squares methods to analyze the RCGARMA model, as discussed in previous sections. To illustrate our theoretical results, we utilize two real datasets obtained from R packages. Our initial focus is on estimating the GARMA (1,0) model with a random coefficient, expressed in matrix form, as follows:

$$X_{t,m}={\mathrm{\Theta }}_t^{\prime }X(t-1)+{\epsilon }_t,t=1,2,\dots , \mathrm{n}$$

To assess the performance of our methods, we execute the model 100 times and compute the ratio of mean squared errors for each example, as described below.

Example 1 

(“Yearly Sunspot” Data). In this example, we consider the "Yearly Sunspot" dataset, which is available in R using the command data (sunspot.year), containing observations of the yearly sunspot number time series observed from 1700 to 1988. This dataset is often used to demonstrate time-series analysis techniques (for more details, see [30]).

The ratio of mean squared errors is calculated based on the quality of a predictor, as follows:

$$MSE\left(\widehat{X}\right)/MSE\left({\widehat{X}}_W\right)=1.484773$$

where $\widehat{X}$ and ${\widehat{X}}_W$ are, respectively, the output variable $X_{t,m}$, calculated by the estimators $\widehat{\mathrm{\Theta }}$ and ${\widehat{\mathrm{\Theta }}}_W$.

Example 2 

(“NileMin” Data). The “NileMin” dataset contains historical observations of yearly minimal water levels of the Nile River from 622 to 1281, measured at the Roda gauge near Cairo. These data are used to study the variability of river flow and its impact on water resource management (more descriptions and details can be found in [31]).

In the case of the “NileMin” dataset, the ratio of mean squared errors is computed based on the quality of a predictor, as follows:

$$MSE\left(\widehat{X}\right)/MSE\left({\widehat{X}}_W\right)=1.624286$$

The results clearly indicate that the forecasted values obtained from the conditional weighted least squares method closely align with the observed datasets, indicating that this method surpasses the conditional least squares method. This close correspondence between the predicted and actual values underscores the effectiveness of the conditional weighted least squares approach in capturing the underlying patterns and dynamics inherent in yearly sunspot data and yearly minimal water levels in the Nile River data. Importantly, our analysis reveals that the performance of the conditional weighted least squares method surpasses that of the conditional least squares method. This indicates that the former method offers superior predictive capabilities, resulting in more accurate and reliable forecasts of yearly sunspot activity and yearly minimal water levels. These findings hold significant implications for the field of environmental research, as they highlight the potential of the conditional weighted least squares method to enhance our comprehension and predictive abilities in this domain.

6. Concluding Remarks

This paper introduces alternative methods for estimating the random parameters of the GARMA model, namely the conditional least squares estimation method and the weighted conditional least squares method. Both of these estimators demonstrate improved performance. To compare their effectiveness, we conducted a simulation study and examined two real-world data examples. Based on the results, we concluded that the weighted least squares estimator outperforms the conditional least squares estimator.