Comparison of Linear and Beta Autoregressive Models in Forecasting Nonstationary Percentage Time Series

Abstract

Positive percentage time series are present in many empirical applications; they take values in the continuous interval (0,1) and are often modeled with linear dynamic models. Risks of biased predictions (outside the admissible range) and problems of heteroskedasticity in the presence of asymmetric distributions are ignored by practitioners. Alternative models are proposed in the statistical literature; the most suitable is the dynamic beta regression which belongs to generalized linear models (GLM) and uses the logit transformation as a link function. However, owing to the Jensen inequality, this approach may also not be optimal in prediction; thus, the aim of the present paper is the in-depth forecasting comparison of linear and beta autoregressions. Simulation experiments and applications to nonstationary time series (the US unemployment rate and BR hydroelectric energy) are carried out. Rolling regression for time-varying parameters is applied to both linear and beta models, and a prediction criterion for the joint selection of model order and sample size is defined.

1. Introduction

In time series analysis, one may often encounter positive percentage data, i.e., data that fall in the open interval (0,1). Examples include the unemployment rate in economics [1], plant utilization in engineering [2], drought index in ecology [3], crime rates in sociology [4] and infection spread in epidemiology [5]. Despite the fact that these data violate the assumption of Gaussianity and often have strong asymmetric distributions, they are usually fitted with linear dynamic models; e.g., [6]. This approach involves the risk of bias in the forecasts (if they are outside the admissible range) and bias in the tests of causality with other series. Indeed, as the response variable approaches the boundaries, its variance shrinks and the implied models become heteroskedastic. This, in turn, involves inefficient parameter estimates with biased standard errors, which affect confidence intervals and test statistics.

To avoid these problems, the statistical literature has proposed various solutions; the simplest is to transform the data with log-based functions to have unbounded codomains, and then fit the resulting series with linear models [7]. More technical approaches intervene on the structure of the systems, by controlling the parameter values with constrained least squares (LS) [8]; or by modeling the response variable with bounded support densities (mainly of the beta family) and various models [9], and parameter estimation is performed using maximum likelihood (ML) [10]. This approach involves iterative algorithms, which easily converge if the likelihood is unimodal; however, their iterative nature hinders the application of recursive methods for nonstationary data (e.g., the Kalman filter) [11].

For stationary time series, beta regression is the best-known method [9] and has been extended to dynamic models by [12,13]. It belongs to generalized linear models (GLMs), which adopt link functions to connect the response variable to the regression function. The preferred link of the beta regression is the logit one, typically adopted in binary response models; its usage in the percentage data can be seen as the attempt to project the response variable on the real line $\pm \infty$ and/or the regression function in the interval (0,1). However, data transformations involve issues of coefficient interpretation [14] and loss of optimality in forecasting; in fact, owing to the Jensen inequality, the expectation of a transformation is different from the transformation of the expectation.

Given this scenario, in this paper, we will compare the forecasting performance of standard linear models and beta dynamic regression, applied to two monthly series: the unemployment rate in the US [1,6] and the hydroelectric energy storage in BR [13]. In the statistical literature there are no such comparisons, and our evaluation is carried out in-depth, on wide out-of-sample periods and long-range prediction intervals. The analysis is restricted to autoregressive (AR) models in view of their long-range forecasting ability; we find that there are not significant differences between linear and beta AR models, unless the distribution function is strongly skewed. Perhaps a hybrid (intermediate) solution, based on the linear modeling of logit-transformed data, may be a suitable compromise to avoid reciprocal risks.

Real-world time series, particularly those related to social phenomena, often exhibit various nonstationary components such as trends, stochastic cycles, and sudden jumps. These features cause irregular dynamic structures in the models (with sparse coefficients at peculiar lags) and the time-variability of parameters. The paper faces these issues with adaptive solutions, such as stepwise identification and rolling regression, which can also be applied to the ML estimator of the beta model. In this case, focusing on pure AR models is computationally useful, and a joint selection strategy based on forecast statistics can be defined for the model order (p) and the sample size (n) of rolling estimators.

Specific nonstationary extensions of the beta approach consider the dispersion parameter as a function of the regressors [15]; or the use of generalized logit-normal (GLN) distribution for data with variable upper bounds [16]. However, these extensions introduce a significant degree of nonlinearity and parametrization in the models, with consequent problems of identifiability, initial values, and the convergence of the ML estimators. Furthermore, their forecasting performance should then be compared with that of nonlinear time series models (such as bilinear, threshold, state-dependent, ARCH, etc.), a goal that is outside the scope of this paper.

The plan of this work is as follows: Section 1 is completed with the introduction of the first case study; Section 2 presents the basic models and estimators and the adaptive forecasting approach; Section 3 applies the methods to two real datasets; Section 4 summarizes the major results and provides conclusions; Appendix A is an appendix with computational aspects.

Preview of the Case Study

The unemployment rate (the proportion of unemployed individuals in the labor force) is a key indicator of both social well-being and a country’s economic activity. As such, it is the focus of many studies [7] and a primary target of economic policies. For the USA, it is measured by the Bureau of Labor Statistics [17] and, at the monthly level, it is available for the period since 1948. The series from January 1948 to April 2025 is displayed in Figure 1a, showing a significant nonstationarity both in level and covariance; particularly impressive is the jump in April 2019, caused by the COVID-19 pandemic [5].

Figure 1c shows the histogram of frequencies; the normality assumption is rejected by major tests (Kolmogorov–Smirnov, Jarque–Bera, etc.) with p-values < 0.001. Instead, fitting the data with a beta density provides coefficients ${\widehat{\alpha }}_y$ = 11.04, ${\widehat{\beta }}_y$ = 183.35, which are 99% statistically significant. Figure 1b shows the residuals obtained by fitting $Y_t$ with an AR(12) model; a big outlier is present at t = 868 (April 2020), and it is also followed by six minor outliers. Figure 1d shows the histogram of residuals ${\widehat{e}}_t$; despite its relative symmetry, the hypothesis of normality is rejected with $p<$ 0.001. Its positive transformation ${\tilde{e}}_t=[{\widehat{e}}_t-\min \left({\widehat{e}}_t\right)]\in [0,1)$ can still be fitted with a beta density with ${\tilde{\alpha }}_e$ = 1.41 and ${\tilde{\beta }}_e$ = 73.9, significant at 95%.

The subset model identified by sequentially dropping the 99% nonsignificant terms is $Y_t=0.0009+1.106Y_{t-1}-0.121Y_{t-6}+e_t$; the AR(1) component is quite natural, while the AR(6) may be due to the duration of short-term working contracts. The series $Y_t$ is monthly but does not contain seasonality; the lag 6 is peculiar and useful for medium-term predictions. This subset model is hardly identifiable with classical methods based on the inspection of ACR functions or the minimization of information criteria (IC), which require regular AR structures.

2. Statistical Methods

2.1. Linear Models

Time series data ${\left\{Y_t\right\}}_{t=1}^T$, with serial correlation, are usually modeled with autoregressive (AR) models of the type

$$\begin{array}{cc}\hfill Y_t={\alpha }_0+{\varphi }_1{Y}_{t-1}+\cdots +{\varphi }_p{Y}_{t-p}& +e_t,e_t\sim \mathrm{IID}(0,{\sigma }_e^2),\hfill \end{array}$$

(1a)

$$\begin{array}{cc}\hfill \mathrm{linear}:\left(1-{\varphi }_1\mathcal{L}-\cdots -{\varphi }_p{\mathcal{L}}^p\right)Y_t& =e_t,t=1,2\dots T,\hfill \end{array}$$

(1b)

the latter is the polynomial representation, where ${\mathcal{L}}^k{Y}_t=Y_{t-k}$ is the lag operator. The order p depends on the serial correlation; ${\alpha }_0$ is the drift and may be a function of time as $\alpha \left(t\right)$; the residuals $e_t$ are independent and identically distributed (IID). The mathematical foundation of the model (1) is constituted by difference equations and the physical concepts of memory and past–present causality. Its stability requires that the roots of the equation ${\varphi }_p\left(\mathcal{L}\right)=0$ lie outside the unit circle; a condition that is always satisfied for processes bounded in (0,1).

The extension to moving average (MA) and mixed (ARMA) models attempts to replace a polynomial ${\varphi }_p\left(\mathcal{L}\right)$ with a large order p, with a low-order rational polynomial $1/{\theta }_q\left(\mathcal{L}\right)$ with $q\ll p$; see [18] p. 53. However, the resulting MA(q) model

$$Y_t={\alpha }_0+{\theta }_1{e}_{t-1}+\dots +{\theta }_q{e}_{t\cdots q}+e_t,$$

has a limited forecasting horizon $h\le q$, as for $h>q$ its forecasts are constant; see [18] p. 144. In this paper, we are interested in long-range predictions (with h = 12) and adaptive estimation; hence, we focus on the linear model (1). Furthermore, Section 1 has shown that the subset AR modeling offers a more flexible perspective.

Under the assumption of normal residuals, $f\left(e_t\right)\propto \exp (-0.5e_t^2/{\sigma }_e^2)$, and the ML estimation of the model (1) coincides with the LS one for large T. Indeed, rewrite the model (1) in multiple regression form as

$$Y_t& ={\mathit{\beta }}^{\prime }{x}_t+e_t,t=p+1,\dots ,T,$$

(2a)

$${\mathit{\beta }}^{\prime }& =\left[{\alpha }_0,{\varphi }_1\dots {\varphi }_p\right],x_t^{\prime }=\left[1,Y_{t-1}\dots Y_{t-p}\right],$$

(2b)

the vector of regressors $x_t$ includes lagged terms, but may also contain trend, exogenous, and dummy variables. The maximization of the Gaussian likelihood $L_T\left(\mathit{\beta }\right)$ is then equivalent to the minimization of the quadratic loss $Q_T\left(\mathit{\beta }\right)={\sum }_t{e}_t^2$, and the LS estimator becomes

$$\begin{array}{cc}\hfill {\widehat{\mathit{\beta }}}_T& ={\left(\sum _{t=p+1}^T{x}_t{x}_t^{\prime }\right)}^{-1}\sum _{t=p+1}^T{x}_t{Y}_t,\hfill \end{array}$$

(3a)

$$\begin{array}{cc}\hfill {\widehat{\sigma }}_e^2& ={\displaystyle \frac{1}{T-2p-1}}\sum _{t=p+1}^T{\left(Y_t-{\widehat{\mathit{\beta }}}_T^{\prime }{x}_t\right)}^2\hfill \end{array}$$

(3b)

The estimator (3a) converges in probability to $\mathit{\beta }$, for any condition of stability of the model (1) (see [19]); however, under stationarity, one has the general result

$$\sqrt{T}\left({\widehat{\mathit{\beta }}}_T-\mathit{\beta }\right)\to \mathrm{N}\left[0,\mathrm{E}{\left(x_t{x}_t^{\prime }\right)}^{-1}{\sigma }_e^2\right].$$

As regards the inference, one may use the estimated dispersion matrices

$$\begin{array}{cc}\hfill {\widehat{V}}_{{\widehat{\mathit{\beta }}}_T}& ={\left(\sum _{t=p+1}^T{x}_t{x}_t^{\prime }\right)}^{-1}{\widehat{\sigma }}_e^2,\hfill \end{array}$$

(4a)

$$\begin{array}{cc}\hfill {\widehat{W}}_{{\widehat{\mathit{\beta }}}_T}& ={\left(\sum _{t=p+1}^T{x}_t{x}_t^{\prime }\right)}^{-1}\left(\sum _{t=p+1}^T{x}_t{x}_t^{\prime }{\widehat{e}}_t^2\right){\left(\sum _{t=p+1}^T{x}_t{x}_t^{\prime }\right)}^{-1},\hfill \end{array}$$

(4b)

where the latter is the so-called sandwich estimator. It is consistent under heteroskedastic residuals ${\widehat{e}}_t$, although care must be taken in model selection; see [20]. An identification method, suitable for subset models (i.e., with sparse coefficients at peculiar lags), is the backward stepwise regression, which is based on the significance of the estimates ${\widehat{\beta }}_k$. It starts from a large order p and selects the regressors as

$$Y_{t-k}:|{\widehat{z}}_k|>{\mathrm{z}}_{\alpha /2}\mathrm{with}{\widehat{z}}_k={\widehat{\beta }}_k/\sqrt{w_{kk}},k=1,2\dots p,$$

(5)

where ${\mathrm{z}}_{\alpha /2}$ are the tail values of the standard normal and $w_{kk}$ are the diagonal entries of the dispersions (4). The method works well even with regular and seasonal models, and uses of dispersion (4b) in the presence of heteroskedastic residuals.

2.2. Beta Models

When the process $Y_t$ is bounded in (0,1) and stationary: $f\left(Y_t\right)=f\left(y\right)$, it may be modeled as a beta distribution: $f\left(y\right)\propto y^{\alpha -1}{(1-y)}^{\beta -1}$; this function is driven by two coefficients $\alpha ,\beta >0$ and is symmetric for $\alpha =\beta$. The reparameterizations $\mu =\alpha /(\alpha +\beta )$ and $\delta =\alpha +\beta$ are directly interpretable as mean value E$\left(y\right)=\mu$ and concentration index V$\left(y\right)=\mu (1-\mu )/(1+\delta )$. In this context, the beta modeling of $Y_t$ may be developed with the GLM framework, where the regression function is connected to the conditional mean ${\mu }_t=\mathrm{E}\left(Y_t\right|Y_{t-k})$ by a proper linkage [9]. Considering the model (2a), a suitable link between ${\mu }_t$ and ${\mathit{\beta }}^{\prime }{x}_t$ is the logit transformation $\mathsf{\ell }\left(y\right)=\log \left[y/(1-y)\right]$

$$\begin{array}{cc}\hfill \mathrm{E}\left(Y_t|Y_{t-k}\right)& ={\mu }_t={\mathsf{\ell }}^{-1}\left({\mathit{\beta }}^{\prime }{x}_t\right),\hfill \end{array}$$

(6a)

$$\begin{array}{cc}\hfill \mathsf{\ell }\left({\mu }_t\right)& =\log \left[{\mu }_t/(1-{\mu }_t)\right]={\mathit{\beta }}^{\prime }{x}_t,\hfill \end{array}$$

(6b)

$$\begin{array}{cc}\hfill {\mu }_t& =1/[1+1/\exp \left({\mathit{\beta }}^{\prime }{x}_t)\right].\hfill \end{array}$$

(6c)

This solution can be viewed as an attempt to transform the one-step-ahead predictor ${\mathsf{\ell }}^{-1}\left({\mathit{\beta }}^{\prime }{x}_t\right)$ in percentage, according to the constraint ${\mu }_t\in (0,1)$. Conversely, it transforms the process $Y_t\in (0,1)$ into $Z_t=\mathsf{\ell }\left(Y_t\right)\in \Re$, on the real line, so that the unbounded LS estimator (3) can be applied.

In stationary AR models, the predictor naturally lies ${\mu }_t\in (0,1)$; however, the presence of deterministic components $\alpha \left(t\right)$ or exogenous variables $X_t$ in the model (1) makes the inverse logit transformation necessary. On the other hand, the cost of $Z_t=\mathsf{\ell }\left(Y_t\right)$ is the change of scale of the regression coefficients, with consequent difficulties of interpretation, such as trend extraction and tests of causality. As in the GLM approach, the beta AR model can be defined as follows

$$\begin{array}{cc}\hfill Z_t& =\mathsf{\ell }\left(Y_t\right)=\log \left[Y_t/\left(1-Y_t\right)\right]\in \Re ,\hfill \end{array}$$

(7a)

$$\begin{array}{cc}\hfill \mathrm{hybrid}:Z_t& ={\alpha }_0^{*}+{\varphi }_1^{*}{Z}_{t-1}+\cdots +{\varphi }_p^{*}{Z}_{t-p}+v_t,v_t\sim \mathrm{IID}(0,{\sigma }_v^2)\hfill \end{array}$$

(7b)

$$\begin{array}{cc}\hfill \mathrm{beta}:Y_t& ={\mathsf{\ell }}^{-1}\left({\mathit{\beta }}^{\prime }{x}_t\right)+u_t,u_t\in (-1,+1)\mathrm{IID},\hfill \end{array}$$

(7c)

where Equation (7b) is the hybrid solution (intermediate between linear and beta) and in Equation (7c) the vector $x_t$ may contain the entries $Z_{t-k}$. Notice that the two error terms in (7b,c) have a substantially different nature; while $\left\{v_t\right\}$ is generally unbounded, $\left\{u_t\right\}$ must lie in the interval $(-1,+1)$. This feature will be shown in detail in the next simulation experiments.

The consequences of the full beta approach are important because the ML method has no alternatives. With the reparameterization, the stationary beta density becomes $f\left(y\right)\propto y^{\mu \delta -1}{\left(1-y\right)}^{(1-\mu )\delta -1}$; hence, the likelihood function of the model (7c) results in

$$\begin{array}{cc}\hfill L_T\left(\mathit{\beta },\delta |{\left\{Y_t\right\}}_1^T\right)& =\prod _{t=p+1}^T{\displaystyle \frac{\mathrm{\Gamma }\left(\delta \right)}{\mathrm{\Gamma }\left({\mu }_t\delta \right)\mathrm{\Gamma }\left[(1-{\mu }_t)\delta \right]}}{Y}_t^{({\mu }_t\delta -1)}{\left(1-Y_t\right)}^{((1-{\mu }_t)\delta -1)},\hfill \end{array}$$

(8a)

$$\begin{array}{cc}\hfill \mathrm{with}{\mu }_t& ={\mathsf{\ell }}^{-1}\left({\mathit{\beta }}^{\prime }{x}_t\right)=1/\left[1+1/\exp \left({\mathit{\beta }}^{\prime }{x}_t\right)\right]\in (0,1),\hfill \end{array}$$

(8b)

where $\mathrm{\Gamma }\left(z\right)={\int }_0^{\infty }(u^{z-1}/e^u)du$ is the gamma function, which normalizes the beta density. Notice also the importance of the inverse logit transformation in bounding the regression function in (0,1); see [9,12].

The maximization of the likelihood (8) proceeds in the usual way, by applying logarithms, computing derivatives, and solving the normal equations iteratively. Letting the parameters be ${\mathit{\delta }}^{\prime }=[{\mathit{\beta }}^{\prime },\delta ]$, the Newton–Raphson algorithm is

$$\begin{array}{cc}\hfill {\mathit{h}}_T\left(\mathit{\delta }\right)& =\partial \log L_T\left(\mathit{\delta }\right)/\partial \mathit{\delta },\hfill \end{array}$$

(9a)

$$\begin{array}{cc}\hfill {\mathit{H}}_T\left(\mathit{\delta }\right)& =\partial {\mathit{h}}_T\left(\mathit{\delta }\right)/\partial {\mathit{\delta }}^{\prime },\hfill \end{array}$$

(9b)

$$\begin{array}{cc}\hfill {\widehat{\mathit{\delta }}}_T^{(k+1)}& ={\widehat{\mathit{\delta }}}_T^{\left(k\right)}-{\mathit{H}}_T^{-1}\left({\widehat{\mathit{\delta }}}_T^{\left(k\right)}\right){\mathit{h}}_T\left({\widehat{\mathit{\delta }}}_T^{\left(k\right)}\right),\hfill \end{array}$$

(9c)

where $k>0$ is the index of iterations and the initial value ${\widehat{\mathit{\delta }}}_T^{\left(0\right)}$ may be provided by the LS estimate of model (7b). Under the stationarity of $Y_t$, the ML estimator (9c) has statistical properties similar to (3); see [12] and [18] p. 226

$$\sqrt{T}\left({\widehat{\mathit{\delta }}}_T-\mathit{\delta }\right)\to \mathrm{N}\left[\mathbf{0},-\mathrm{E}{\left[{\partial }^2\log L\left(\mathit{\delta }\right|Y_t)/\partial \mathit{\delta }\partial {\mathit{\delta }}^{\prime }\right]}^{-1}\right],$$

where, for convenience, we have set $k=T$. Hence, the dispersion matrix of (9c) is provided by the negative inverse of the second derivatives (Hessian) in (9b)

$${\widehat{\mathit{V}}}_{{\widehat{\mathit{\delta }}}_T^{\left(k\right)}}=-{\mathit{H}}_T^{-1}\left({\widehat{\mathit{\delta }}}_T^{\left(k\right)}\right),$$

this can be computed numerically through finite differences (see the Appendix A).

2.3. Forecasting

As regards the out-of-sample prediction, the AR modeling allows the application of the chain rule of forecasting and the inverse transform

$$\begin{array}{cc}\hfill {\widehat{Y}}_{T+h}& =\mathrm{E}\left(Y_{T+h}|Y_{T-k},k\ge 0\right),h=1,2\dots H,\hfill \end{array}$$

(10a)

$$\begin{array}{cc}\hfill \mathrm{linear}:{\widehat{Y}}_{T+h}& ={\alpha }_0+{\varphi }_1{\widehat{Y}}_{T+h-1}+\cdots +{\varphi }_p{Y}_{T+h-p},h<p,\hfill \end{array}$$

(10b)

$$\begin{array}{cc}\hfill \mathrm{beta}:{\tilde{Y}}_{T+h}& ={\mathsf{\ell }}^{-1}\left({\alpha }_0^{*}+{\varphi }_1^{*}{\widehat{Z}}_{T+h-1}+\cdots +{\varphi }_p^{*}{Z}_{T+h-p}\right),h<p,\hfill \end{array}$$

(10c)

where, in the latter, one may use either the LS estimates of ${\varphi }_k^{*}$ applied to Equation (7b) or their ML refinement obtained from Equation (9c).

While the linear predictor (10b) may be biased when ${\widehat{Y}}_{T+h}\notin (0,1)$, the beta predictor (10c) may not be optimal with respect to the general definition (10a). Indeed, by extending the Jensen inequality $\mathrm{E}\left[g\left(y\right)\right]\ne g\left[\mathrm{E}\left(y\right)\right]$ to the conditional expectation, from Equation (10a,b,c), one has the inequality

$$\begin{array}{cc}\hfill {\tilde{Y}}_{T+h}& ={\mathsf{\ell }}^{-1}\left({\widehat{Z}}_{T+h}\right)={\mathsf{\ell }}^{-1}\left[\mathrm{E}\left(Z_{T+h}|Z_{T-k}\right)\right]\ne \hfill \\ \hfill & \ne \mathrm{E}\left[{\mathsf{\ell }}^{-1}\left(Z_{T+h}|Z_{T-k}\right)\right]=\mathrm{E}\left(Y_{T+h}|Y_{T-k}\right)={\widehat{Y}}_{T+h},\hfill \end{array}$$

In general, with the logit transformation, there must be care both in interpreting the estimated coefficients and in using the forecasts of the original series. This is a common problem for all GLM models.

Given the uncertainty about the optimality of predictors (10b,c), an empirical way to evaluate the models is to make out-of-sample forecasts over a sufficiently long period, e.g., $T/3$. This means estimating the model on the first set of data $t_0=T·2/3$, computing the empirical forecasts for $t>t_0$, and then evaluating the mean absolute percentage errors (MAPE) statistics

$${\mathrm{MAPE}}_N\left(h\right)={\displaystyle \frac{1}{N}}\sum _{t=t_0+1}^{T-H}\left|{\displaystyle \frac{{\widehat{Y}}_{t+h}-Y_{t+h}}{Y_{t+h}}}\right|,h=1,2\dots H,$$

(11)

where $N=T-H-t_0$; the best model is that with lower ${\sum }_{h=1}^H{\mathrm{MAPE}}_N\left(h\right)$.

2.4. Non-Stationarity

The bounded process $Y_t\in (0,1)$ is, by definition, stable (non-divergent); however, its models may exhibit local non-stationarity in the form of time-varying parameters $\left\{{\mathit{\beta }}_t\right\}$. In real phenomena, such variability may be caused by omitted explanatory variables $\left\{X_t\right\}$ and/or by hidden nonlinear dynamics; a general assumption is that parameters vary in a stochastic manner as ${\mathit{\beta }}_t={\mathit{\beta }}_{t-1}+{\mathit{a}}_t\sim \mathrm{IN}\left(\mathbf{0},\mathit{I}{\sigma }_a^2\right)$. In the linear model (1), they can be estimated recursively with a simplified version of the Kalman filter (KF), see [11] and [18] p. 496.

$$\begin{array}{cc}\hfill {\widehat{\mathit{\beta }}}_t& ={\widehat{\mathit{\beta }}}_{t-1}+{\mathit{V}}_t{x}_t\left(Y_t-{\widehat{\mathit{\beta }}}_{t-1}^{\prime }{x}_t\right),{\widehat{\mathit{\beta }}}_0=\mathbf{0},\hfill \\ \hfill {\mathit{V}}_t& ={\mathit{V}}_{t-1}-{\displaystyle \frac{{\mathit{V}}_{t-1}{x}_t{x}_t^{\prime }{\mathit{V}}_{t-1}}{{\sigma }_e^2+x_t^{\prime }{\mathit{V}}_{t-1}{x}_t}}+\mathit{I}{\sigma }_a^2,{\mathit{V}}_0=\mathit{I}{\sigma }_a^2,\hfill \end{array}$$

where ${\mathit{V}}_t$ is the dispersion matrix, that depends on ${\sigma }_e^2,{\sigma }_a^2$.

Although simplified, such an algorithm is difficult to apply to the beta model (7c) and its ML estimator (9). A simple nonparametric alternative to KF is the rolling regression, which uses the moving sub-samples of size $n<T$; for the ordinary LS, its formula is given by Equation (3a,b) with the sums replaced by ${\sum }_{s=t-n+1}^t(·)$, providing ${\widehat{\mathit{\beta }}}_{n,t}$. For the beta models, the rolling estimator can be expressed as local ML

$${\widehat{\mathit{\delta }}}_{n,t}=\arg \max L_n\left(\mathit{\beta },\delta |{\left\{Y_s\right\}}_{t-n+1}^t\right),$$

(12)

where $L_n(·)$ is defined as in Equation (8a,b). The solver (9) involves iterations at each t, but Equation (12) may use the initial values ${\widehat{\mathit{\delta }}}_{n,t}^{\left(0\right)}={\widehat{\mathit{\delta }}}_{n,t-1}^{\left(k\right)}$. In this framework, the window-width n tunes the degree of adaptation and may be selected to minimize the prediction errors of (10), including its MAPE statistics, as can be seen in Equation (14).

Notice that the solution (12) includes the precision parameter $\delta$ of the beta distribution; its time-variability, say ${\delta }_t$, has important consequences on the behavior of the conditional variance of the process, given by

$${\sigma }_t^2=\mathrm{V}\left(Y_t|Y_{t-k},k>0\right)={\displaystyle \frac{{\mu }_t(1-{\mu }_t)}{(1+{\delta }_t)}},$$

This expression shows that ${\sigma }_t^2$ could potentially be stabilized by ${\delta }_t$, reducing the heteroskedasticity of the model. In general, the ultimate purpose of time-varying parameters is to obtain stationary (homoskedastic) residuals. The approach followed in this paper with Equation (12) is entirely non-parametric, and n could also be selected to achieve the desired variability of ${\sigma }_t^2$.

In the literature on beta regression, the attempt to stabilize ${\sigma }_u^2$ is pursued with parametric solutions. Following [15], one may assume that the precision parameter has a regression structure of the type ${\delta }_t=\exp \left({\mathit{\alpha }}^{\prime }{x}_t\right)$, where $x_t={[1,Y_{t-1}\dots ]}^{\prime }$. However, this introduces into the beta system a considerable degree of nonlinearity and parameterization, which is not easy to manage and evaluate. In linear models, a similar topic is treated with the ARCH (autoregressive conditional heteroskedastic) representation of the residual variance ${\sigma }_t^2={\mathit{\alpha }}^{\prime }{e}_t^2$, where $e_t^2={[1,e_{t-1}^2\dots ]}^{\prime }$, as can be seen in [18] p. 362. Both solutions allow us to predict the series volatility, but their effect on the point forecasts (10) is uncertain; in any case, the aim and scope of the present paper are not concerned with the nonlinear time series models.

2.5. Sequential Design

Previous forecast statistics and rolling estimates can be combined into a joint sequential procedure: once the estimate ${\widehat{\mathit{\beta }}}_{n,t}$ is computed at time $t>n$, it can immediately be used to calculate the adaptive forecasts ${\widehat{Y}}_{n,t+h}$, for $1\le h\le H$, and their prediction errors ${\widehat{e}}_{n,t+h}$. The MAPE statistics (11) then average these local errors over $t>t_0$; specifically, given the out-of-sample period $t_0\le t\le T$, the joint sequential procedure for the linear AR model is given by

$$\begin{array}{cc}\hfill {\widehat{\mathit{\beta }}}_{n,t}& ={\left(\sum _{s=t-n+1}^t{x}_s{x}_s^{\prime }\right)}^{-1}\sum _{s=t-n+1}^t{x}_s{Y}_s,t=t_0\dots T-H,\hfill \end{array}$$

(13a)

$$\begin{array}{cc}\hfill {\widehat{Y}}_{n,t+h}& ={\widehat{\alpha }}_{0,t}+{\widehat{\varphi }}_{1,t}{\widehat{Y}}_{n,t+h-1}+\cdots +{\widehat{\varphi }}_{p,t}{Y}_{t+h-p},\mathrm{if}h<p,\hfill \end{array}$$

(13b)

$$\begin{array}{cc}\hfill {\mathrm{MAPE}}_N\left(h\right|n)& ={\displaystyle \frac{1}{N}}\sum _{t=t_0+1}^{T-H}|\left(Y_{t+h}-{\widehat{Y}}_{n,t+h}\right)/Y_{t+h}|,h=1,2\dots H,\hfill \end{array}$$

(13c)

where $N=T-H-t_0$. This approach can also be applied to the Formulas (10c)–(12) of the beta AR model; Figure 2 shows the scheme of the procedure (13).

Given the time-variability of parameters, it is possible that certain $\left\{{\varphi }_{k,t}\right\}$ are not significant on $t<t_0$, but may be useful in forecasting. The model selection procedure (5) is based on fitting in-sample data, while prediction is concerned with the out-of-sample period $t>t_0$. This leads to consider the full model (1) in forecasting and select its order p, together with the rolling window-width n, on the basis of the average MAPE statistic (13c); namely

$$\left(\widehat{p},\widehat{n}\right)=\arg \min {\displaystyle \frac{1}{H}}\sum _{h=1}^H{\mathrm{MAPE}}_N\left(h|n,p\right),$$

(14)

where each MAPE(h) could also be weighted by $1/h$ in the sum (14). Notice that the parameters ${\varphi }_k$, used in forecasting, continue to be estimated on past data, and the solution (14) can also be applied to beta models.

2.6. Simulation Studies

We conclude the section with two simulation experiments on AR processes having beta distributions, to evaluate the statistical performance (bias and efficiency) of the estimators (3) and (9). The data are generated from random sequences $u_t\sim \mathrm{IB}(\alpha ,\beta )$ centered on 0; in order to bound the realizations of $Y_t\in (0,1)$, the variance of $u_t$ is kept low (i.e., $\alpha ,\beta$ large), and drift and mean coefficients are included in the models. Specifically, the simulated processes are

$$\begin{array}{cc}\hfill Y_t& =0.25+0.5Y_{t-1}+e_t,e_t=(u_t-{\overline{u}}_t),u_t\sim \mathrm{IB}(5,30),\hfill \\ \hfill y_t& =1.5y_{t-1}-0.6y_{t-2}+e_t,e_t=(u_t-{\overline{u}}_t),u_t\sim \mathrm{IB}(150,50),\hfill \end{array}$$

(15a)

$$\begin{array}{cc}\hfill Y_t& =y_t+0.5,\hfill \end{array}$$

(15b)

with starting values $Y_0$ = 0.25 and $y_0$ = 0. Figure 3 shows a realization of process (15a) with $T=300$; the series $Y_t$ has a beta distribution in Figure 3c with estimated parameters ${\widehat{\alpha }}_y$ = 26.7 and ${\widehat{\beta }}_y$ = 27.1. Instead, the process (15b) is nearly unstable, with a greater signal-to-noise ratio ${\sigma }_y/{\sigma }_e$.

Table 1. Performance of ML (9) and LS (3) estimators applied to the models (15a,b); Bias, RMSE, mean p-value of the Jarque–Bera normality test are computed over 500 replications of length T = 300.

(15a)			${\alpha }_0$			${\varphi }_1$
Model	Method	Bias	RMSE	N-test	Bias	RMSE	N-test
Linear (1a)	LS (3)	0.0025	0.0255	0.133	−0.0050	0.0511	0.1350
Hybrid (7b)	LS (3)	−0.2494	0.2495	0.001	−0.0074	0.0512	0.1307
Beta (7c)	ML (9)	−0.2494	0.2495	0.001	−0.0141	0.0521	0.1546
(15b)			${\varphi }_1$			${\varphi }_2$
Model	Method	Bias	RMSE	N-test	Bias	RMSE	N-test
Linear (1a)	LS (3)	0.0468	0.0669	0.0284	0.0487	0.0687	0.0326
Hybrid (7b)	LS (3)	−0.0061	0.0495	0.0010	−0.0009	0.0487	0.0010
Beta (7c)	ML (9)	−0.0101	0.0493	0.0102	0.0012	0.0472	0.0936

reports the basic statistics of fitting $N=500$ realizations of the models (15a,b) with LS and ML estimators applied to the series $Y_t,Z_t=\mathsf{\ell }\left(Y_t\right)$. The results show the better performance of the LS method in (15a); in particular, for the drift coefficient ${\alpha }_0$, which is affected by the logit transformation and cannot be converted into the original scale of (15a) using ${\mathsf{\ell }}^{-1}\left({\tilde{\alpha }}_0\right)$. Instead, when only AR parameters are involved in (15b), the ML estimator is the best, with the hybrid close.

3. Applications to Real Data

3.1. US Monthly Unemployment Rate

In this section, we conduct an in-depth evaluation of the forecasting performance of linear and beta models using real data. The series in Figure 1 seems affected by various nonstationary components (in level, autocovariance, heteroskedasticity, etc.), but the most dangerous issue for model inference is the presence of a big jump in 2020. Sudden changes in $Y_t$ produce outliers in the residuals, which, in turn, make parameter estimates and forecasts biased and inefficient. A good method to contrast these effects is the introduction of pulse (dummy) or step variables as $X_t=\dots 0,0,1,1,1\dots$ in correspondence with the anomalous residuals (see [18], Chapter 13). However, this solution is not applicable in forecasting, as the jumps are unpredictable and the effects of $X_t$ can be estimated only after their occurrence.

1. Model Identification.

To show the extent of the problem, we fit linear and beta AR(12) models to the whole series in Figure 1a and drop non-significant coefficients ${\widehat{\varphi }}_k$ stepwise, as in Equation (5); surprisingly, both models reduce to AR(1)

$$\begin{array}{cc}\hfill Y_t& =\underset{\left(2.71\right)}{0.0017}+\underset{\left(94.7\right)}{0.970}{Y}_{t-1}+e_t,{\widehat{\sigma }}_e=0.0041,H_e=\underset{\left[0.50\right]}{0.447},\hfill \end{array}$$

(16a)

$$\begin{array}{cc}\hfill Y_t& ={\mathsf{\ell }}^{-1}\left(\underset{(-3.69)}{-0.0686}+\underset{\left(145.7\right)}{0.975}{Z}_{t-1}\right)+u_t,\widehat{\delta }=\underset{\left(21.4\right)}{4652.8},H_u=\underset{\left[0.31\right]}{1.048},\hfill \end{array}$$

(16b)

where the values in parentheses are t-type statistics and $\delta$ is the precision parameter of the beta model. $H_e$ is Engle’s test for heteroskedasticity; the p-values in square brackets result in the hypothesis of homoskedastic residuals being accepted.

Now, the introduction of nine dummy variables, in correspondence with major outlying residuals (see [18], p. 482) increases the parametric complexity of Equation (16a,b), leading to subset AR(10) models. In general, dummy variables tend to self-feed (as dropping a major outlier often yields another minor one), and have uncertain effects on the model structure. Since sudden jumps are unpredictable, and we are interested in comparing the forecasting performance of the models, we will avoid using dummies and look for other solutions.

Figure 4 shows the cleaning effect of nine dummy variables on the residuals; since they are placed at the beginning and at the end of the series, we decide to restrict the analysis to the central part of $Y_t$, namely from 1949.01 to 2019.12, a total of T = 852 observations. On this interval, the linear and beta AR models, identified with the stepwise method (5), retain the same structure:

$$\begin{array}{cc}\hfill Y_t& =\underset{\left(3.62\right)}{0.0008}+\underset{\left(83.6\right)}{1.109}{Y}_{t-1}-\underset{(-9.2)}{0.124}{Y}_{t-6}+e_t,{\widehat{\sigma }}_e=0.0018,{\tilde{\sigma }}_e=0.0016,\hfill \end{array}$$

(17a)

$$\begin{array}{cc}\hfill Y_t& ={\mathsf{\ell }}^{-1}\left(\underset{(-3.57)}{-0.040}+\underset{\left(107.1\right)}{1.101}{Z}_{t-1}-\underset{(-11.2)}{0.115}{Z}_{t-6}\right)+u_t,\widehat{\delta }=\underset{\left(20.5\right)}{16455},\hfill \end{array}$$

(17b)

where ${\tilde{\sigma }}_e$ is the robust, mean absolute deviation (MAD), estimate of the standard deviation, the fact that ${\widehat{\sigma }}_e\approx {\tilde{\sigma }}_e$ in (17a) confirms the absence of outliers during the period 1949–2019. The subset nature of models (17) is highlighted by the peculiar term $Y_{t-6}$ (non-seasonal); this lag is difficult to detect with classical methods (correlograms and information criteria) used for regular models, as those in [18].

2. Forecasting Evaluation.

Given the nonstationarity of the data, the design of the experiments for comparing the forecasts of the models (17) involve the rolling regression outlined in Equations (12)–(14); in practice:

(1)

The span H of the predictors (10) is set to one year: $h=1,2\dots 12$;

(2)

The out-of-sample forecasting period starts at $t_0+1$ = 517 (1991.01);

(3)

Models are estimated on $t\le t_0$, with sample sizes $n=50,100\dots 500$;

(4)

At each $t=t_0+1\dots T-H$, the parameters and forecasts are recomputed;

(5)

Prediction statistics are the mean absolute percentage errors (MAPE) (13c).

The results of this procedure are displayed in graphical form in Figure 5; Panel (a) shows that, with n = 500, the three models (linear, beta and hybrid) have a similar performance with a small predominance of the hybrid one (the linear modeling of the logit $Z_t=\mathsf{\ell }\left(Y_t\right))$. Panel (b) shows the effect of the estimation window n on the annual average of MAPE statistics; in this case, the difference among the three models seems greater, but the common best window is n = 400. Panel (c) shows the time-path of the parameters ${\alpha }_0,{\varphi }_1,{\varphi }_6$ for n = 50 in the forecasting period, and Panel (d) shows the last forecast at $T-12$ (both for the hybrid model).

The main conclusion of Figure 5 is that the hybrid solution (linear AR modeling of the logit-transformed data, with inverse transformation in forecasting) is slightly better than the others. The distortion induced by the Jensen inequality does not appear, and the best window-width for the rolling regression is relatively large, i.e., n = 400, about 33 years.

3. Joint Selection of p,n.

Following Equation (14), we evaluate the joint effect of the model order p and sample size n on the MAPE(h) statistics of regular AR(p) models, in the same way as Figure 5b. The rationale of using all coefficients ${\left\{{\varphi }_k\right\}}_1^p$ is that some ${\varphi }_{kt}$ may locally be significant in the out-of-sample period $t>t_0$ and may be useful in forecasting; similarly, the selection of n should be based on prediction errors, rather than in-sample residuals. The results for H = 1, 12 are displayed in Figure 6 for the hybrid model; the optimal values during the period 1991–2019 are $\widehat{p}$ = 8, $\widehat{n}$ = 400. This means that parsimonious AR models ($p\le 3$) are not suitable for both short- (h = 1) and long-range forecasts of the US unemployment series.

3.2. BR Hydroelectric Energy Storage

The second application deals with a dataset investigated with beta dynamic models by [13], but not compared with the corresponding linear models. The series regards the proportion of hydroelectric energy stored in South Brazil, i.e., the megawatts that can be generated from a stored volume of water, as a proportion of the total capacity. The monthly series covers the period 2000.07 to 2018.04 (a total of T = 214, see Figure 7a) and is provided by the Brazilian Electric Energy Agency [21]. The authors [13] compare several beta ARMA models in forecasting the last six values, including a beta-AR(3) estimated on the first $t_0$ = 150 observations.

Table 2. Estimates of AR(3) models on the first n = 150 data of Figure 7a; standard errors are in parentheses and p-values in square brackets. $H_e$ is the Engle’s homoskedastic test.

Model	Method	${\mathit{\alpha }}_0$	${\mathit{\varphi }}_1$	${\mathit{\varphi }}_2$	${\mathit{\varphi }}_3$	$\mathit{\delta },{\mathit{\sigma }}_{\mathit{e}}$	${\mathit{R}}^2$	${\mathit{H}}_{\mathit{e}}$
Beta	([13], Table 9)	.	$\underset{\left(0.033\right)}{0.949}$	$\underset{\left(0.051\right)}{-0.338}$	$\underset{\left(0.042\right)}{0.087}$	.	.	.
Beta	ML (9)	$\underset{\left(0.067\right)}{0.211}$	$\underset{\left(0.079\right)}{0.967}$	$\underset{\left(0.103\right)}{-0.372}$	$\underset{\left(0.070\right)}{0.118}$	$\underset{\left(1.63\right)}{14.11}$	$\underset{\left(\mathrm{pseudo}\right)}{0.654}$	$\underset{\left[0.465\right]}{0.534}$
Linear	LS (3)	$\underset{\left(0.037\right)}{0.156}$	$\underset{\left(0.080\right)}{1.146}$	$\underset{\left(0.116\right)}{-0.439}$	$\underset{\left(0.081\right)}{0.068}$	0.104	0.722	$\underset{\left[0.512\right]}{0.429}$

confirms their parameter estimates in ([13], Table 9), except for standard errors, which indicate an AR(2) (consistent with the correlograms in ([13], Figure 1)).

The authors [13] show that their beta AR(3) outperforms the beta ARMA(2,1) in forecasting the last six data, as can be seen in ([13], Table 10); this may be due to the short prediction range of the MA components [18]. We now evaluate the in-depth forecasting performance of AR(2) models with the procedure (13), by setting $t_0$ = 151, p = 2, n = 50, 100, 150, h = 6; the results are reported in Figure 7.

We considered AR(2) models because our estimates of the coefficient ${\varphi }_3$ in Table 2 are not significant, and their MAPE statistics are smaller than those of the AR(3). Furthermore, we assessed that AR(3) predictions of the linear model go beyond the bound 1 at some points and need to be censored. Although the hydroelectric series is nearly stationary, the results of Figure 7 agree with those of Figure 5; in particular, there are not significant differences between the three models, and the best sample size n for the rolling estimations is not small.

Finally, to check nonstationarity, Figure 8 displays the rolling ML estimates of Equation (11) with n = 50 of the parameters of the beta AR(2) model. A certain time-variability is present, particularly in the precision parameter ${\widehat{\delta }}_t$ in Panel (b); notice also that the variances of ${\widehat{u}}_t=(Y_t-{\widehat{\mu }}_t)$, obtained with the descriptive statistic $s_u^2$ and the analytical formula ${\widehat{\sigma }}_t^2={\widehat{\mu }}_t(1-{\widehat{\mu }}_t)/(1+{\widehat{\delta }}_t)$, have the same pattern.

4. Conclusions

This paper has compared linear and beta autoregressive models in forecasting percentage time series in (0,1). Such series are often present in the real world, and in theory, should be treated with generalized linear models and maximum likelihood estimators. Instead, practitioners often fit them with classical linear models and ordinary least squares; the major risk is the bias of out-of-range forecasts and the inefficiency due to the heteroskedasticity in the presence of asymmetric distributions. On the other hand, the full beta modeling, by relying on the logit transformation of data, provides parameter estimates that are difficult to interpret, and the inverse transformation for obtaining forecasts in (0,1) may yield bias.

In the applications, we have seen that statistical differences, in fitting and forecasting between linear and beta AR models are not significant. This may mean that the bias of linear models (forecasts outside the range) and the bias induced by the inverse transformation (via the Jensen inequality) are similar. Now, the first bias can be easily detected (and corrected), and we checked that it is sporadically present only in the second application (where the series is concentrated near the upper border). For nonstationary time series, the best solution turned out to be the hybrid one (the linear modeling of logit-transformed data), because the use of logit transformation may mitigate the inefficiency due to heteroskedasticity. Instead, for stationary time series with strongly asymmetric distribution, the full beta modeling is preferable. Owing to the change of scale produced by the logit, a problem of interpretation of the coefficients of exogenous components remains.

The evaluation approach of this paper is based on MAPE statistics computed on long-range predictions and wide out-of-sample periods. This approach has been combined with the rolling estimation (also applied to the ML of beta models), leading to a predictive selection strategy of the model order p and the sample size n of the estimator; see Equation (14) and Figure 6. Notice that prediction errors ${\widehat{e}}_t\left(h\right)$ are more challenging than the usual in-sample residuals and may be defined on multiple horizons h. The restriction of the analysis to pure AR models was motivated by their long-range forecasting ability and by their simple regression structure, which allows the direct application of complex statistical techniques.

Specifically, the maintenance of the linear structure (including the hybrid solution) is a necessary condition for extending the modeling toward adaptive methods, such as Kalman filters, M-type estimators, nonparametric smoothers, etc. [11]. These solutions are necessary in the presence of nonlinear dynamics, time-varying parameters, jumps in the series, and multiple outliers; instead, adaptive methods can hardly be implementable within the iterative ML framework (9) of the full beta approach. These are directions for further research, together with the comparison of the forecasts of nonlinear beta models [15,16] with those of the nonlinear models of econometrics (e.g., bilinear, threshold, state-space, ARCH, etc.).