The Use of Minimization Solvers for Optimizing Time-Varying Autoregressive Models and Their Applications in Finance

Abstract

Time series data are fundamental for analyzing temporal dynamics and patterns, enabling researchers and practitioners to model, forecast, and support decision-making across a wide range of domains, such as finance, climate science, environmental studies, and signal processing. In the context of high-dimensional time series, the Vector Autoregressive model (VAR) is widely used, wherein each variable is modeled as a linear combination of lagged values of all variables in the system. However, the traditional VAR framework relies on the assumption of stationarity, which states that the autoregressive coefficients remain constant over time. Unfortunately, this assumption often fails in practice, especially in systems subject to structural breaks or evolving temporal dynamics. The Time-Varying Vector Autoregressive (TV-VAR) model has been developed to address this limitation, allowing model parameters to vary over time and thereby offering greater flexibility in capturing non-stationary behavior. In this study, we propose an enhanced modeling approach for the TV-VAR framework by incorporating minimization solvers in generalized additive models and one-sided kernel smoothing techniques. The effectiveness of the proposed methodology is assessed using simulations based on non-homogeneous Markov chains, accompanied by a detailed discussion of its advantages and limitations. Finally, we illustrate the practical utility of our approach using an application to real-world financial data.

1. Introduction

The autoregressive (AR) model, as a versatile tool for analyzing and forecasting time series data, plays an important role in economics, weather forecasting, signal processing, and healthcare fields. In the application of the autoregressive model, the stationarity of time series data, in which the mean, variance, and autocovariance for the given processes remain constant over time, is one of the key assumptions. Unfortunately, in real-world scenarios, most signals and processes exhibit non-stationary behavior, which makes them unsuitable for AR modeling and can lead to systematic bias or inconsistent results [1]. Beyond signal processing, traditional risk models often rely on assumptions of stationarity and constant interdependencies, which are frequently violated during periods of market stress or structural shifts. In such turbulent conditions, asset correlations tend to rise sharply, a phenomenon known as correlation breakdown, resulting in a substantial underestimation of portfolio risk when using static or constant-parameter models. Thus, to effectively characterize such processes, it is essential to account for their non-stationary nature using either parametric or non-parametric models. Following this thought, one well-established approach in this context is the time-varying autoregressive (TV-AR) model, which is described in detail in [2]. In general, the framework of the TV-AR model with r time lags can be expressed as:

$$y^{\left(t\right)}=c_t+\sum _{k=1}^r{\alpha }_{k,t}{y}^{(t-k)}+\varphi \left(t\right),$$

where $y^{\left(t\right)}$ is time-varying intercept, ${\alpha }_{k,t}$ are time-varying parameters, and perturbation $\varphi \left(t\right)$ is zero-mean stationary process with $\mathbb{E}\left[\varphi {\left(n\right)}^2\right]={\sigma }^2$, and $\mathbb{E}\left[\varphi \right(n\left)\varphi \right(m\left)\right]=0$ for $n\ne m$. In contrast to the AR model, the TV-AR model has a significant advantage in flexibility and adaptability due to its time-dependent parameters, making it more suitable for capturing non-stationary behavior and structural changes in time series data.

Next, in scenarios involving the investigation of complex internal relationships, where the internal relationship is non-negligible, the Vector Autoregressive (VAR) approach is employed as an efficient high-dimensional analytical method. However, as in the case of the AR model, the stationarity assumption is often violated in multivariate settings; therefore, the Time-Varying Vector Autoregressive (TV-VAR) model is employed as a multivariate framework to capture this temporal dynamics and interdependencies among multiple variables [3]. Typically, the TV-VAR model with r time lags can be expressed as:

$$y^{\left(t\right)}=C_t+\sum _{k=1}^r{B}_{k,t}{y}^{(t-k)}+\Phi \left(t\right),$$

where vector time series data $y^{\left(t\right)}\in {\mathbb{R}}^p$, time-varying intercepts $C_t\in {\mathbb{R}}^p$, time-varying matrix $B\in {\mathbb{R}}^{p\times p}$, and $\Phi \left(t\right)$ are independent samples drawn from a multivariate zero-mean stationary process with covariance matrix $\Sigma$. Although TV-VAR models are widely used to capture non-stationary dynamics in multivariate time series, existing estimation approaches, such as local kernel smoothing and Time-Varying Parameter Bayesian VAR (TVP-BVAR), often lack explicit mechanisms for enforcing model stability. In particular, traditional methods do not directly control the stability of the time-varying process, despite its inherent tendency toward explosiveness in the absence of appropriate constraints. While TVP-BVAR is capable of estimating time-varying coefficients, its high computational cost and the complexity of modeling evolving parameters limit its practical applicability, particularly in high-dimensional or large-scale settings. In contrast, the generalized additive framework (GAM) and one-sided kernel smoothing (OKS) based frameworks adopted in this study offer more tractable and scalable alternatives for estimating time-varying dynamics with greater computational efficiency. Hence, in this study, we focus on modeling the TV-VAR process by incorporating the GAM- and OKS-based techniques, as proposed by [4]. We aim to enhance these methods by making the following contributions:

• We first derive recursive formulas to compute the expectation and covariance matrix of the TV-VAR process with r lags. These statistical properties serve as the foundation for reformulating the TV-VAR model as a constrained optimization problem. To ensure stable and reliable estimation of the time-varying coefficients, we conduct a detailed analysis of the underlying statistical structure and implement the solution using a numerical optimization framework in Python 3.9.
• In applying the generalized additive framework and kernel smoothing techniques, we further reformulate the TV-VAR optimization problem in a way that allows it to be solved using a variety of optimization methods.
• We present the performance results of our GAM-based and OKS-based methods using simulations of non-homogeneous Markov chains and discuss their respective strengths and limitations.
• Ultimately, our research suggests that non-stationary processes are better captured by a TV-VAR structure than traditional VAR models. Within both the GAM and OKS frameworks, gradient-based optimization algorithms incorporating stability constraints yield high performance in estimating time-varying coefficients.

The structure of the paper is as follows: Section 3 presents the recursive formulas for the mean and covariance, along with the corresponding optimization problems for three types of processes. Section 4 introduces the reformulated optimization framework, incorporating the generalized additive model and one-sided kernel smoothing techniques. Section 5 provides simulation results for four different scenarios of non-homogeneous Markov chains. Finally, Section 6 demonstrates the effectiveness of our approach using a finance-related dataset.

2. Literature Review

The Time-Varying Autoregressive model, as an extension of the AR model, offers non-negligible advantages in coefficient agility, making it highly suitable for non-stationary or structurally changing environments. Currently, TV-AR is widely used in psychology, for example, according to the research by [5,6], TV-AR shows a significant advantage in the study of psychology changing dynamics, especially for the emotion dynamics detection. In the research conducted by [7], the model was used to investigate potential changes of intra-individual dynamics in the perception of situations and emotions of individuals varying in personality traits. In addition to psychology, TV-AR is also widely used in fields such as finance and economics. For example, according to the research by [8], TV-AR can be used to examine how monetary policy jointly affects asset prices and the real economy in the United States. In the study conducted by [9], the TV-AR model was employed to quantify the economic and policy influences, capturing both regime-dependent and time-varying effects. In a study by [10], a TV-AR model was developed to analyze the effects of oil revenue on economic growth, providing empirical support for the resource curse hypothesis in Nigeria. Another area where TV-AR models are widely applied is engineering, including domains such as aerospace, mechanical systems, transportation, environmental monitoring, and infrastructure management. For example, in the research conducted by [11], a TV-AR model was developed to identify modal in structural health monitoring, and [12] used TV-AR to analyze the global or volcano seismological signals.

Compared to the AR model, when interdependencies among multiple variables are significant, the VAR model is a more effective approach for capturing autoregressive dynamics. For example, in the research conducted by [13], VAR was used to analyze the interrelationships between price spreads and the effects of wind forecast and demand forecast errors, and other exogenous variables. In the research conducted by [14], a Bayesian variant based VAR models was developed to forecast the international set of macroeconomic and financial variables. In the study conducted by [15], VAR was used to detect and analyze the relationships between two subgroups in eating behaviour, depression, anxiety, and eating control. However, as with the TV-AR model, incorporating time-varying parameters becomes essential when the assumption of stationarity is violated; hence, to accommodate both non-stationarity and inter-variable dynamics, the TV-VAR model provides a crucial extension. In practical applications, TV-AR and TV-VAR often share methodological similarities; however, TV-VAR places greater emphasis on modeling the dynamic interactions among variables. For example, in the field of psychology, the study by [16] demonstrated that time-varying parameters provide a robust framework for analyzing co-occurrence, synchrony, and the directionality of lagged relationships in real-world psychological data. Similarly, ref. [17] employed a TV-VAR model to examine the dynamics of automatically tracked head movements in mothers and infants, revealing temporal variations both within and across episodes of the Still-Face Paradigm. Furthermore, the research by [18] applied the TV-VAR model to capture complex multivariate interactions among psychological variables.

The TV-VAR model also plays a crucial role in the fields of economics and marketing. For instance, [19] employed a TV-VAR model to analyze the Japanese economy and monetary policy, uncovering their evolving structure over the period from 1981 to 2008. Similarly, ref. [20] used a TV-VAR framework to investigate the dynamic relationship between West Texas Intermediate crude oil and the U.S. S& P 500 stock market. In the context of stock market forecasting, ref. [21] demonstrated that the TV-VAR model holds significant potential for enhancing portfolio allocation strategies. Similarly, in the study by [22], the TV-VAR model was employed to analyze four types of oil price fluctuations: oil supply shocks, global demand shocks, domestic demand shocks, and oil-specific demand shocks. Additionally, ref. [23] applied this model to examine the dynamic interactions among economic policy uncertainty, investor sentiment, and financial stability. Further, ref. [24] used the TV-VAR model to explore return spillovers among major financial markets, including equity indices, exchange rates, Brent crude oil prices in the Asia-Pacific region, and the NASDAQ index. In addition to economic analysis, the TV-VAR model has also been widely applied in quantitative assessments of policy impacts, including analyses of monetary policy shocks [25], the effects of geopolitical risks and trade policy uncertainty [26], and international spillovers of US monetary policy [27].

3. Proposed Non-Stationary Process

In this section, the notation for the general stationary VAR model is first introduced, encompassing three different types of processes and their stochastic behavior. Subsequently, in the next section, the Generalized Additive Modeling (GAM-based) approach and the one-sided Kernel-smoothing (OKS-based) method are employed to estimate the parameters in the time-varying VAR models.

3.1. Time-Varying Autoregressive Model

In the TV-AR model with r lags, each observation at time t is predicted using the previous r observations, with time-varying coefficients that adjust to capture evolving dynamics. Overall, the TV-AR model is defined as:

$$y^{\left(t\right)}=c_t+\sum _{k=1}^r{\alpha }_{k,t}{y}^{(t-k)}+\varphi \left(t\right),$$

(1)

where $y^{\left(t\right)}$ is reference time series data, $c_t$ represents the time-varying intercept, ${\alpha }_{k,t}$ are time-varying parameters, and perturbation $\varphi \left(t\right)$ is a zero-mean stationary process with $\mathbb{E}\left[\varphi {\left(n\right)}^2\right]={\sigma }^2$, and $\mathbb{E}\left[\varphi \right(n\left)\varphi \right(m\left)\right]=0$ for $n\ne m$. To estimate the mean value at time t, first define the mean value of TV-AR(r) at time t is ${\mu }_t=\mathbb{E}\left[y_t\right]$, and assume the initial mean values ${\mu }_1,\dots ,{\mu }_r$ are given, then use the mean recursive function to calculate ${\mu }_t$:

$$\begin{array}{cc}\hfill {\mu }_t& =\mathbb{E}\left[c_t+\sum _{k=1}^r{\alpha }_{k,t}{y}^{(t-k)}+\varphi \left(t\right)\right]=c_t+\sum _{k=1}^r{\alpha }_{k,t}{\mu }_{t-k}.\hfill \end{array}$$

(2)

Next, assume the covariance between time t and time $t-i$ is ${\gamma }_{t,t-i}=\mathbb{E}\left[y^{\left(t\right)}{y}^{(t-i)}\right]-\mathbb{E}\left[y^{\left(t\right)}\right]\mathbb{E}\left[y^{(t-i)}\right]$, so that:

$$\begin{array}{cc}\hfill {\gamma }_{t,t-i}& =\mathbb{E}\left[\left(c_t+\sum _{m=1}^r{\alpha }_{m,t}{y}^{(t-m)}+\varphi \left(t\right)\right)\left(c_{t-i}+\sum _{n=1}^r{\alpha }_{n,t-i}{y}^{(t-i-n)}+\varphi (t-i)\right)\right]\hfill \\ \hfill & -\left(c_t+\sum _{m=1}^r{\alpha }_{m,t}{\mu }_{t-m}\right)\left(c_{t-i}+\sum _{n=1}^r{\alpha }_{n,t-i}{\mu }_{t-i-n}\right)\hfill \\ \hfill & =\sum _{m=1}^r\sum _{n=1}^r{\alpha }_{m,t}{\alpha }_{n,t-i}\left[\mathbb{E}\left[y^{(t-m)}{y}^{(t-i-n)}\right]-{\mu }_{t-m}{\mu }_{t-i-n}\right]+\sum _{n=1}^r{\alpha }_{n,t-i}\mathbb{E}\left[y^{(t-i-n)}\varphi \left(t\right)\right]\hfill \\ \hfill & +\sum _{m=1}^r{\alpha }_{m,t}\mathbb{E}\left[y^{(t-m)}\varphi (t-i)\right]+\mathbb{E}\left[\varphi \left(t\right)\varphi (t-i)\right]\hfill \\ \hfill & =\sum _{m=1}^r\sum _{n=1}^r{\alpha }_{m,t}{\alpha }_{n,t-i}{\gamma }_{t-m,t-i-n}+\sum _{m=1}^r{\alpha }_{m,t}\mathbb{E}\left[y^{(t-m)}\varphi (t-i)\right]+\mathbb{E}\left[\varphi \left(t\right)\varphi (t-i)\right].\hfill \end{array}$$

In terms of $\mathbb{E}\left[y^{(t-m)}\varphi (t-i)\right]$, first assume $m=i$, so that $\mathbb{E}\left[y^{(t-i)}\varphi (t-i)\right]=\mathbb{E}\left[\varphi (t-i)\varphi (t-i)\right]={\sigma }^2$. Next, for $m>i$, these terms can be expressed as the following recursive function:

$$\begin{array}{cc}\hfill \mathbb{E}\left[y^{(t-m)}\varphi (t-i)\right]& =\mathbb{E}\left[\left(c_{t-m}+\sum _{s=1}^r{\alpha }_{s,t-m}{y}^{(t-m-s)}+\varphi (t-m)\right)\varphi (t-i)\right]\hfill \\ \hfill & =\sum _{s=1}^r{\alpha }_{s,t-m}\mathbb{E}\left[y^{(t-m-s)}\varphi (t-i)\right].\hfill \end{array}$$

Hence, $\mathbb{E}\left[y^{(t-m)}\varphi (t-i)\right]$ can be computed recursively, and the covariance recursive function can be written as follows:

$$\begin{array}{c}\hfill {\gamma }_{t,t-i}=\left\{\begin{array}{cc}\sum _{m,n=1}^r{\alpha }_{m,t}{\alpha }_{n,t-i}{\gamma }_{t-m,t-i-n}+\sum _{m=1}^r{\alpha }_{m,t}\mathbb{E}\left[y^{(t-m)}\varphi (t-i)\right]\hfill & i>0;\hfill \\ \sum _{m,n=1}^r{\alpha }_{m,t}{\alpha }_{n,t}{\gamma }_{t-m,t-n}+{\sigma }^2\hfill & i=0.\hfill \end{array}\right.\end{array}$$

(3)

Take TV-AR(1) as an example, the expectation recursive function is given by:

$${\mu }_t=c_t+{\alpha }_t{\mu }_{t-1}.$$

The covariance recursive function is:

$$\begin{array}{c}\hfill {\gamma }_{t,t-i}=\left\{\begin{array}{cc}{\alpha }_t{\alpha }_{t-i}{\gamma }_{t-1,t-i-1}+\prod _{k=t-i}^t{\alpha }_k{\sigma }^2\hfill & i>0;\hfill \\ {\alpha }_t^2{\gamma }_{t-1,t-1}+{\sigma }^2\hfill & i=0.\hfill \end{array}\right.\end{array}$$

To estimate the time-varying parameters in (1), it is essential to assume smoothness along the time axis, which defines $g_t={[g_0\left(t\right),g_1\left(t\right),\dots ,g_r\left(t\right)]}^T\in {\mathbb{R}}^{(r+1)}$ with the smoothness functions $g_i\left(t\right)$ in time t. Next, define $z_t={[1,y^{(t-1)},\dots ,y^{(t-r)}]}^T\in {\mathbb{R}}^{(r+1)}$, and the recursive formulation implies that, for each iterations, if ${\sum }_{s=1}^r\left|{\alpha }_{s,p}\right|\le 1$, then $\mathbb{E}\left[y^{(t-m)}\varphi (t-i)\right]\le {\sigma }^2$, As implied by the covariance recursion function, if the time-varying coefficient exceeds 1, as time t increase, the sequence becomes unstable and may diverge to infinity. This indicates that even a small initial error can propagate and amplify over time, resulting in substantial inaccuracies in future estimates. While such behavior is theoretically permissible in non-stationary processes, it introduces heightened sensitivity to initial conditions or early observations, which can substantially increase the risk of overfitting. Hence, to prevent covariance from exploding and to estimate the time-varying parameters $c_t,{\alpha }_{1,t}\dots ,{\alpha }_{r,t}$, the TV-AR model can be reformulated and solved as the optimization problem (4).

$$\begin{array}{cc}\hfill & \underset{g_t}{\mathbf{Minimize}:}\sum _{t=r}^T{(y^{\left(t\right)}-g_t^T{z}^t)}^2;\hfill \\ \hfill & \mathbf{Subject}\mathbf{to}:h|g_t|\le 1;\hfill \end{array}$$

(4)

where $h=\left[\begin{array}{cccc}0& 1& \dots & 1\end{array}\right]\in {\mathbb{R}}^{(r+1)}$.

3.2. Vector Autoregressive Model

The VAR, one of the most fundamental multivariate time series analysis models, assumes that an p-dimensional time series vector $y_t\in {\mathbb{R}}^{n\times p}$ can be expressed as:

$$y^{\left(t\right)}=C+\sum _{i=1}^r{B}_i{y}^{(t-i)}+{\epsilon }_t,$$

(5)

where $C\in {\mathbb{R}}^p$ is a constant intercept vector, $B_i\in {\mathbb{R}}^{p\times p}$ are the autoregressive coefficient matrices, and ${\epsilon }_t\sim N(0,\Phi )$ represents zero-mean multivariate Gaussian noise with covariance matrix $\Phi$. Compared to the TV-AR model, the stationary assumption is essential for VAR; thus, to compute its mean, the process is assumed to be weakly stationary, with a constant mean $\mu =\mathbb{E}\left[y_t\right]$, and time-invariant covariances. Finally, the expectation is:

$$\begin{array}{cc}\hfill \mu & =C+\sum _{i=1}^r{B}_i\mathbb{E}\left[y^{(t-i)}\right]=C+\mu \sum _{i=1}^r{B}_i=C{\left(I-\sum _{i=1}^r{B}_i\right)}^{-1}.\hfill \end{array}$$

(6)

Next, assume the covariance between time t and time $t-h$ is ${\Gamma }_h=\mathbb{E}\left[y^{\left(t\right)}{\left(y^{(t-h)}\right)}^T\right]-\mu {\mu }^T\in {\mathbb{R}}^{p\times p}$, and the initial covariance functions ${\Gamma }_i$ where $1\le i\le r$ are given. Then:

$$\begin{array}{cc}\hfill {\Gamma }_h& =\mathbb{E}\left[\left(C+\sum _{k=1}^r{B}_k{y}^{(t-k)}+{\epsilon }_t\right){\left(y^{(t-h)}\right)}^T\right]-\mu {\mu }^T\hfill \\ \hfill & =\sum _{k=1}^r{B}_k\mathbf{Cov}\left(y^{(t-k)},y^{(t-h)}\right)+\mathbb{E}\left[{\epsilon }_t{\left(y^{(t-h)}\right)}^T\right].\hfill \end{array}$$

By the definition of weak stationarity that $\mathbf{Cov}\left(y^{(t-k)},y^{(t-h)}\right)={\Gamma }_{h-k}$, and ${\Gamma }_{-h}={\Gamma }_h^T$. Then, the covariance recursive function is:

$$\begin{array}{c}\hfill {\Gamma }_h=\left\{\begin{array}{cc}\sum _{k=1}^r{B}_k{\Gamma }_{h-k}\hfill & h>0;\hfill \\ \sum _{m,n=1}^r{B}_m{\Gamma }_{m-n}{B}_n^T+\Phi \hfill & h=0.\hfill \end{array}\right.\end{array}$$

(7)

Finally, to estimate the stationary parameters in the VAR model (5), it is challenging to estimate the full coefficient matrix directly; therefore, a row-by-row estimation strategy is adopted. First, define $z_t={[1,y_1^{(t-1)},y_2^{(t-1)},\dots ,y_p^{(t-r)}]}^T\in {\mathbb{R}}^{(rp+1)}$, ${\beta }_i={[C_i,B_{i,1,1},\dots ,B_{i,p,r}]}^T\in {\mathbb{R}}^{(rp+1)}$, thus, the $i_{th}$ row of stationary parameter can be estimated by solving optimization problem (8).

$$\underset{{\beta }_i}{\mathbf{Minimize}:}\sum _{t=r}^T{(y_i^{\left(t\right)}-{\beta }_i^T{z}_t)}^2.$$

(8)

To solve this optimization problem, the simplest approach is to take the derivative of the objective function with respect to ${\beta }_i$ and set it equal to zero. Hence, the solution for ${\beta }_i$ is then given by:

$${\beta }_i=\sum _{t=r}^T{y}_i^{\left(t\right)}{\left(z_t{z}_t^T\right)}^{-1}{z}_t.$$

3.3. Time-Varying Vector Autoregressive Models

The TV-VAR model, a non-stationary multivariate extension of VAR, allows the model parameters to vary over time, enabling dynamic adaptation to evolving relationships. In this framework, analogous to the TV-AR case, it is critical to assume smoothness along the time axis, as the process models a p-dimensional time series vector $y_t\in {\mathbb{R}}^{n\times p}$ that evolves smoothly over time and can be expressed as:

$$y^{\left(t\right)}=C_t+\sum _{k=1}^r{B}_{k,t}{y}^{(t-k)}+{\epsilon }_t,$$

(9)

where time-varying intercepts $C_t\in {\mathbb{R}}^p$, time-varying matrix $B\in {\mathbb{R}}^{p\times p}$, and $\Phi \left(t\right)$ are independent samples drawn from a multivariate zero-mean stationary process with covariance matrix $\Sigma$. Unlike VAR, weak stationarity does not hold in TV-VAR.in Thus, to calculate its mean value, first assume ${\mu }_t\in {\mathbb{R}}^p$ and the initial mean vectors ${\mu }_1,\dots ,{\mu }_r$ are given, then the mean recursive function is given:

$${\mu }_t=\mathbb{E}\left[C_t+\sum _{k=1}^r{B}_{k,t}{y}^{(t-k)}+{\epsilon }_t\right]=C_t+\sum _{k=1}^r{B}_{k,t}{\mu }_{t-k}.$$

(10)

Next, to calculate the covariance function between time t and time $t-i$ is ${\Gamma }_{t,t-i}\in {\mathbb{R}}^{p\times p}$, assume the initial covariance functions ${\Gamma }_{r-i,r-j}$ where $1\le i,j\le r$ are given. Then:

$$\begin{array}{cc}\hfill {\Gamma }_{t,t-i}& =\mathbb{E}\left[y^{\left(t\right)}{\left(y^{(t-i)}\right)}^T\right]-\mathbb{E}\left[y^{\left(t\right)}\right]\mathbb{E}\left[y^{(t-i)}\right]\hfill \\ \hfill & =\sum _{m=1}^r\sum _{n=1}^r{B}_{m,t}{\Gamma }_{t-m,t-i-n}{B}_{n,t-i-n}^T+\sum _{m=1}^r{B}_{m,t}\mathbb{E}\left[y^{(t-m)}{\epsilon }_{t-i}^T\right]+\mathbb{E}\left[{\epsilon }_t{\epsilon }_{t-i}\right].\hfill \end{array}$$

Similar to TV-AR, $\mathbb{E}\left[y^{(t-m)}{\epsilon }_{t-i}^T\right]$ can also be calculated recursively. Hence, the covariance recursive function is:

$$\begin{array}{c}\hfill {\Gamma }_{t,t-i}=\left\{\begin{array}{cc}\sum _{m=1}^r\sum _{n=1}^r{B}_{m,t}{\Gamma }_{t-m,t-i-n}{B}_{n,t-i-n}^T+\sum _{k=1}^r{B}_{k,t-m}\mathbb{E}\left[y^{(t-m-k)}{\epsilon }_{t-i}^T\right]\hfill & i>0;\hfill \\ \sum _{m=1}^r\sum _{n=1}^r{B}_{m,t}{\Gamma }_{t-m,t-i-n}{B}_{n,t-n}^T+\Phi \hfill & i=0.\hfill \end{array}\right.\end{array}$$

(11)

Next, to prevent the explosion of the covariance matrix, similar to the issue encountered in the TV-AR case, one of the most direct ways is to impose the constant constraint ${\sum }_{k=1}^r{\sum }_{n=1}^p\left|B_{k,t,n,m}\right|\le 1$ for each column m; in other words, the sum of the absolute values of each column across all time-varying parameter matrices is bounded by 1. Ultimately, estimating the time-varying coefficient matrix requires integrating the estimation approaches used in both the TV-AR and VAR models. First, define $z_t={[1,y_1^{(t-1)},y_2^{(t-1)},\dots ,y_p^{(t-r)}]}^T\in {\mathbb{R}}^{(rp+1)}$, ${\beta }_{i,t}={[C_{i,t},B_{i,t,1,1},\dots ,B_{i,t,p,r}]}^T\in {\mathbb{R}}^{(rp+1)}$. Thus, the $i_{th}$ row of time-varying parameter at time t, ${\beta }_{i,t}$, is the solution of the optimization problem (12) under the sum constant constraint.

$$\begin{array}{cc}\hfill & \underset{{\beta }_{i,t}}{\mathbf{Minimize}:}\sum _{t=r}^T{(y_i^{\left(t\right)}-{\beta }_{i,t}^T{z}_t)}^2;\hfill \\ \hfill & \mathbf{Subject}\mathbf{to}:h|{\beta }_{i,t}|\le 1,\hfill \end{array}$$

(12)

where $h={[0,1\dots ,1]}^T$. Finally, to solve the TV-VAR optimization problem (12), we introduce two approaches: one based on the Generalized Additive Model and the other on kernel smoothing techniques. These methods will be detailed in the following section.

4. Estimating Time-Varying Vector Autoregressive Model

In this section, we describe the estimation of the TV-VAR model using the GAM framework (e.g., as introduced in [28]), which accommodates non-linear relationships among variables. In addition to the GAM, we also consider a kernel smoothing approach, which estimates the time-varying coefficient matrices directly through kernel-weighted averaging, rather than relying on basis functions. The following subsections outline the training steps and overall workflow for implementing TV-VAR estimation using both the GAM and kernel smoothing methods.

4.1. Generalized Additive Model Workflow

The GAM is often introduced as a flexible extension of linear models, where smooth functions, referred to as smoothers, are added to capture non-linear patterns in the data. The core assumption of this approach is that the effect of each predictor on the response can be modeled by a smooth, non-parametric function rather than a fixed linear coefficient. This allows the model to capture complex relationships without requiring the analyst to predefine a specific functional form. The general structure of a GAM with n basis functions can be expressed as:

$$g\left(t\right)=\sum _{i=1}^n{\zeta }_i{f}_i\left(t\right),$$

where $f_i$ represents a smooth basis function and ${\zeta }_i$ represents the corresponding linear coefficient in the GAM framework. In this research, to estimate the time-varying coefficient matrices in a TV-VAR model with r lags, we assume the existence of r non-zero smooth functions $g_1\left(t\right)$ to $g_r\left(t\right)$, each capturing the temporal variation of the corresponding lagged effect. Thus, the time-varying matrix $B_{k,t}$ and intercept vector $C_t$ under the GAM framework can be expressed as follows:

$$\begin{array}{cc}\hfill B_{k,t}& =\left[\begin{array}{cccc}& B_{t,k,1,1}& \dots & B_{t,k,1,p}\\ & \dots & \dots & \dots \\ & B_{t,k,p,1}& \dots & B_{t,k,p,p}\end{array}\right]=\left[\begin{array}{cccc}& g_{k,1,1}\left(t\right)& \dots & g_{k,1,p}\left(t\right)\\ & \dots & \dots & \dots \\ & g_{k,p,1}\left(t\right)& \dots & g_{k,p,p}\left(t\right)\end{array}\right]\in {\mathbb{R}}^{p\times \left(np\right)};\hfill \\ \hfill C_t& =\sum _{i=1}^n{\zeta }_i{f}_i\left(t\right).\hfill \end{array}$$

(13)

Next, to simplify the notation in time-varying matrix estimation, we make the following assumptions:

$i_{th}$ time value vector:

$$y_i=\left[y_i^{\left(m\right)},\dots ,y_i^{\left(r\right)}\right]\in {\mathbb{R}}^{m-r};$$

Time varying matrix:

$$A={\left[a_1,\dots ,a_{m-r}\right]}^T\in {\mathbb{R}}^{(m-r)\times \left(n\right(pr+1\left)\right)},$$

where:

$$a_i=\left[{\zeta }_{0,1}f\left(i\right),\dots ,{\zeta }_{0,n}f\left(i\right),{\zeta }_{1,1}f(i-1)y_1^{(i-1)},\dots ,{\zeta }_{r,n}f(i-r)y_p^{(i-r)}\right];$$

and coefficient bound vector:

$$h=\left[0,\dots ,0,1,\dots ,1\right]\in {\mathbb{R}}^{n(pr+1)}.$$

Thus, the GAM-based TV-VAR coefficients can be calculated by solving the following optimization problem:

$$\begin{array}{cc}\hfill & B_i=\underset{x,h^{T}x\le 1}{\mathbf{argmin}:}\left|\right|y_i-Ax{\left|\right|}_2^2.\hfill \end{array}$$

(14)

Finally, overparameterization is crucial when selecting basis functions, as it enables benign overfitting that enhances model flexibility without harming generalization. However, excessive flexibility can lead to malignant overfitting and unstable predictions [29,30]. Thus, a central challenge lies in choosing the appropriate type and number of basis functions. Common spline bases, such as cubic splines, B-splines, P-splines, and thin plate splines, offer distinct advantages for modeling shapes or surfaces with varying smoothness and flexibility requirements, spanning a continuum from moderately flexible (e.g., cubic and B-splines) to highly adaptive (e.g., P-splines and thin plate splines). However, the performance of spline-based models is highly sensitive to the number and placement of knots; suboptimal configurations can result in overfitting or underfitting. To mitigate these issues, data-driven approaches, such as adaptive splines and penalized splines, introduce mechanisms for automatic knot placement and smoothing parameter estimation, thereby enhancing model flexibility and reducing the need for manual tuning. However, these approaches often require the high cost of computational resources, particularly in high dimensions. Hence, beyond spline-based formulations, alternative basis functions, including exponential, linear, and Gaussian kernels, can be employed depending on the structural assumptions of the target function and the intended trade-offs between interpretability, smoothness, and computational efficiency. Therefore, in this study, we adopt splines alongside other selected basis functions, including Gaussian and tanh functions (a subset of the basis functions used in this study is shown in Figure 1).

In the end, in the TV-VAR(r) model, after selecting the appropriate basis functions, a variety of strategies are available to solve the optimization problem (14) (The workflow of TV-VAR(1) is illustrated in Figure 2). One approach is to apply unconstrained regression methods such as ordinary linear regression, Lasso, or Ridge regression, though each comes with its limitations. For example, ordinary linear regression does not effectively enforce constraints, which can lead to an explosion in the covariance matrix and increasingly unstable predictions over time. In contrast, Lasso and Ridge regression address this issue by introducing a regularization term controlled by a tuning parameter $\lambda$, which governs the strength of the penalty and can be effectively selected via cross-validation [31].

Unfortunately, the traditional regression methods often fail to enforce constraints effectively; therefore, gradient-based optimization algorithms might be a more suitable alternative. Currently, several methods are available, including naive gradient descent, conjugate gradient, and the Broyden–Fletcher–Goldfarb–Shanno algorithm, all of which can handle both constrained and unconstrained optimization. However, for those algorithms, constrained nonlinear optimization problems can sometimes be time-consuming and might cause malignant overfitting. Hence, to address such linear constraints in optimization problem (14), in this research, we employ a Lagrangian gradient-based solver (from the Python SciPy package, named ’trust-constr’ model).

4.2. Kernel Smoothing Method Based Workflow

Kernel smoothing (KS), a non-parametric method for estimating unknown functions, is commonly used to evaluate time-varying parameters by fitting and combining a sequence of local models across different time points in the series. Similar to GAM, KS also assumes that the effects of predictor variables can be smoothly captured by non-parametric functions. In the KS model, the bandwidth parameter b plays a critical role, which should be small enough to capture the time-varying structure of the underlying model. However, if b is too small, the algorithm may become overly sensitive to local fluctuations, leading to overfitting. Typically, the KS weight at time $t^{*}$ is given by:

$$K_b(t,t^{*})={\displaystyle \frac{1}{\sqrt{2\pi b^2}}}{e}^{-{\displaystyle \frac{{(t-t^{*})}^2}{2b^{*}}}},$$

where t denotes a time point in the series, $t^{*}$ is the target time at which the time-varying parameter matrix is to be estimated, and b is the bandwidth parameter that controls the degree of smoothness in the estimation. Next, to estimate the time-varying coefficient matrix in the TV-VAR(r) model using the KS approach, the weighted optimization problem is used in place of the original optimization problem (12). However, in the standard KS model, where the time typically ranges from r to the final time T, when estimating parameters for times between $t^{*}+1$ and T, the model may inadvertently incorporate information from the “future” observations, data points occurring after the target time, potentially introducing look-ahead bias in certain applications. Therefore, this research applied the one-sided kernel smooth (OKS) method, which forces the final time T to be $t^{*}$, and its optimization problem is given as (15).

$$\begin{array}{cc}\hfill & \underset{{\beta }_{i,t^{*}}}{\mathbf{minimize}:}\sum _{t=r}^{t^{*}}K(t,t^{*}){(y_i^{\left(t\right)}-{\beta }_{i,t^{*}}^T{z}_t)}^2;\hfill \\ \hfill & \mathbf{Subject}\mathbf{to}:h^T\left|{\beta }_{i,t^{*}}\right|\le 1.\hfill \end{array}$$

(15)

Traditionally, the minimization problem in Equation (15) admits a solution; however, this solution does not necessarily satisfy the required constraints. Therefore, to effectively solve the optimization problem, we reformulate it into an equivalent problem that can be solved numerically, similar to the approach used in the GAM case. Before proceeding with the numerical solver, to clarify the notation, here are some necessary assumptions:

Time value $i_{th}$ vector:

$$y_i={\left[K(t^{*},t^{*})y_i^{\left(t^{*}\right)},\dots ,K(r,t^{*})y_i^{\left(r\right)}\right]}^T\in {\mathbb{R}}^{t^{*}-r};$$

Time varying matrix:

$$A={\left[a_1,\dots ,a_{t^{*}-r}\right]}^T\in {\mathbb{R}}^{(t^{*}-r)\times (rp+1)},$$

where:

$$a_i=\left[\sqrt{K(r+i-1,t^{*})},\sqrt{K(r+i-1,t^{*})}{y}_1^{(t^{*}-1)},\dots ,\sqrt{K(r+i-1,t^{*})}{y}_p^{(t^{*}-r)}\right];$$

and coefficient bound vector:

$$h=[0,1,\dots ,1]\in {\mathbb{R}}^{pr+1.}$$

Thus, the OKS-based TV-VAR coefficients at time $t^{*}$ can be calculated by (16), and the workflow of this method is shown in Figure 3.

$$\begin{array}{c}\hfill B_i=\underset{x,h^{T}x\le 1}{\mathbf{argmin}:}\left|\right|y_i-Ax{\left|\right|}_2^2.\end{array}$$

(16)

Finally, the critical step is selecting the bandwidth b. As noted earlier, increasing the bandwidth b incorporates more data into the estimation around a given point, but if b is too large, it may lead to underfitting by oversmoothing the data and obscuring local patterns. In contrast, a bandwidth that is too small may result in overfitting by capturing noise instead of the underlying structure. In the study by [32], several strategies were proposed to select an appropriate bandwidth. Among these, cross-validation was identified as the most suitable for our context and is therefore employed in this research to determine the optimal bandwidth. In the end, similar to the GAM framework, the optimization problem is solved using a gradient-based method, specifically, the Lagrange gradient descent implemented with the publicly available SciPy package.

5. Evaluating Performance via Simulation

In this section, we present our simulation studies based on non-homogeneous Markov chains to evaluate the performance of the proposed methods in practical scenarios. The non-homogeneous Markov chain is a stochastic process in which the transition probabilities vary over time, in contrast to a homogeneous Markov chain, where the transition probabilities remain fixed and independent of the time step. In real-world applications, this type of Markov chain is widely used for mathematical modeling, simulation, and analysis of complex systems with uncertain or evolving dynamics, particularly in network analysis across various fields, including social sciences, environmental sciences, bioinformatics, and finance [33,34,35]. Specifically, this research investigates two approaches to modeling non-homogeneous Markov chains: the first employs a transition matrix that evolves smoothly and continuously over time without an intercept term, while the second introduces an additional time-varying constant to capture subtle dynamic variations. Accordingly, the primary process for generating the second-order time-varying test data is defined as follows:

$$y^{\left(t\right)}=C_t+f_0(A_0,t)y^{(t-1)}+f_1(A_1,t-1)y^{(t-2)}+W_{\epsilon }{\epsilon }_t,$$

where $C_t$ denotes a time-varying intercept, $A_0\left(t\right)$ and $A_1\left(t\right)\in {\mathbb{R}}^{p\times p}$ are time-varying coefficient matrices, $\epsilon \sim N(0,\Sigma )$ is a Gaussian noise vector with zero mean and covariance $\Sigma \in {\mathbb{R}}^{p\times p}$, and $W_{\epsilon }$ is a scalar controlling the noise magnitude. Subsequently, to evaluate the performance of our TV-VAR–based methods, the mean absolute error (MAE), as defined in Equation (17), is computed over a forecast horizon of length H. Initially, two additional evaluation metrics, Mean Absolute Percentage Error (MAPE) and Root Mean Squared Error (RMSE), were considered for performance assessment. However, the presence of values approaching zero in some test cases led to numerical instability in MAPE, thereby undermining its reliability. Furthermore, the differences between RMSE and MAE were consistently minimal, suggesting that RMSE provided limited additional analytical value beyond what was captured by MAE. Therefore, this study focuses exclusively on MAE as the primary evaluation metric.

$$\begin{array}{c}\hfill \mathbf{MAE}(\widehat{y},y)={\displaystyle \frac{1}{n}}\sum _{i=1}^n|{\widehat{y}}_i-y_i|.\end{array}$$

(17)

For the GAM-based approach, the time-varying coefficient matrix, constructed using basis functions, was estimated from past data and then used to predict the values for the next H time steps, with the corresponding prediction error calculated. In contrast, the OKS method assumed a fixed coefficient matrix over the forecast horizon, and predictions were made under this assumption, followed by evaluation of the resulting prediction error. All simulations in this study were conducted using Python, with the source code and related datasets available as detailed in the Data Availability Statement.

5.1. Simulation Preparation

For the case of a transition matrix that changes smoothly over time without time-varying intercepts, we consider the following specific scenarios:

• Scenario 1: consider a zero-intercept time-series data with dimension $p=2$, where the initial value is set to $y_0={[0,0]}^T$, a small noise scaling parameter which fixed to $W_{\epsilon }=1/10$, and the Gaussian noise covariance matrix $\Sigma =\mathbf{diag}\left(\overrightarrow{1}\right)\in {\mathbb{R}}^{2\times 2}$. The time-varying coefficient matrix is initialized as a uniform random matrix, normalized row-wise to form a row-stochastic matrix, and updated using the following functions:

  $$\begin{array}{c}\hfill f_0(A,t)\leftarrow {\displaystyle \frac{1}{2}}\left[\begin{array}{cc}\mathbf{min}\{a_{0,0}^{(t-1)}+1-t,1\},& 1-a_{0,0}^{\left(t\right)}\\ \mathbf{min}\{a_{1,0}^{(t-1)}+1-t,1\},& 1-a_{1,0}^{\left(t\right)}\end{array}\right],\\ \hfill f_1(A,t)\leftarrow {\displaystyle \frac{1}{2}}\left[\begin{array}{cc}\mathbf{max}\{a_{0,0}^{(t-1)}-1+t,0\},& 1-a_{0,0}^{\left(t\right)}\\ \mathbf{min}\{a_{1,0}^{(t-1)}+1-t,1\},& 1-a_{1,0}^{\left(t\right)}\end{array}\right];\end{array}$$
• Using the same initial value, noise scaling parameter, and strategy for generating the initial time-varying coefficient matrix as in Scenario 1, but with an increased dimension $p=3$, the Gaussian noise covariance matrix $\Sigma =\mathbf{diag}\left(\overrightarrow{1}\right)\in {\mathbb{R}}^{3\times 3}$, and the new time-varying coefficient matrix update function, which is defined as follows:

  $$\begin{array}{cc}\hfill & f_0(A,t)\leftarrow {\displaystyle \frac{1}{2}}\left[\begin{array}{ccc}\mathbf{max}\{a_{0,0}^{(t-1)}-1+t,0\},& \mathbf{min}\{a_{0,1}^{(t-1)}+(1-t)/2,0\},& 1-a_{0,0}^{\left(t\right)}-a_{0,1}^{\left(t\right)}\\ \mathbf{min}\{a_{1,0}^{(t-1)}+(1-t)/2,0\},& \mathbf{max}\{a_{1,1}^{(t-1)}-1+t,0\},& 1-a_{1,0}^{\left(t\right)}-a_{1,1}^{\left(t\right)}\\ \mathbf{max}\{a_{2,0}^{(t-1)}-1+t,0\},& \mathbf{min}\{a_{2,1}^{(t-1)}+(1-t)/3,0\},& 1-a_{2,0}^{\left(t\right)}-a_{2,1}^{\left(t\right)}\end{array}\right],\hfill \\ \hfill & f_1(A,t)\leftarrow {\displaystyle \frac{1}{2}}\left[\begin{array}{ccc}\mathbf{max}\{a_{0,0}^{(t-1)}-t,0\},& \mathbf{min}\{a_{0,1}^{(t-1)}+t/2,0\},& 1-a_{0,0}^{\left(t\right)}-a_{0,1}^{\left(t\right)}\\ \mathbf{min}\{a_{1,0}^{(t-1)}+t/2,0\},& \mathbf{max}\{a_{1,1}^{(t-1)}-t,0\},& 1-a_{1,0}^{\left(t\right)}-a_{1,1}^{\left(t\right)}\\ \mathbf{max}\{a_{2,0}^{(t-1)}-t,0\},& \mathbf{min}\{a_{2,1}^{(t-1)}+t/3,0\},& 1-a_{2,0}^{\left(t\right)}-a_{2,1}^{\left(t\right)}\end{array}\right].\hfill \end{array}$$

Next, a small time-varying intercept $C_t=t/20$ is introduced in Scenarios 1 and 2 to enhance the dynamic behavior and evaluate the model performance. In addition to the prediction analysis, to better analyze the performance of the TV-VAR model in visualization, we apply the GAM in multiple cases of Scenarios 1 and 3 to reconstruct the whole process, as the stationarity and local stationarity assumptions of VAR and OKS make them unsuitable for this type of visualization. Specifically, we consider a time series consisting of 200 data points, with the first 180 used for training and the remaining 20 reserved for prediction. For visualization analysis, after estimating the time-varying coefficient matrices using GAM, we feed the first r lag values into the model and let it recursively estimate the remaining $200-r$ data points. The initial time-varying coefficient matrix is defined below:

$$\begin{array}{c}\hfill A_0=\left[\begin{array}{cc}0.475,& 0.525\\ 0.145,& 0.875\end{array}\right],A_1=\left[\begin{array}{cc}0.450,& 0.550\\ 0.255,& 0.745\end{array}\right].\end{array}$$

5.2. Simulation Results

For each scenario, the predictions are made two steps ahead using 100 randomly generated time series samples, each of length 200, with time t normalized from 0 to 1, and the performance is evaluated using MAE across these steps. In this study, we implement the examples using GAM, OKS, traditional Vector Autoregressive (VAR) models (estimated via matrix-based updates), and L-VAR (with coefficients estimated through Lasso regression) with lags $r=1$ and $r=2$. The results for the two scenarios without an intercept are shown in

Table 1. The summary statistics of absolute errors across 100 random samples in Scenario 1.

Method	H	MAE	SV	Min	Max
GAM(1)	1	$8.499\times {10}^{-2}$	$3.897\times {10}^{-3}$	$1.056\times {10}^{-3}$	$3.002\times {10}^{-1}$
GAM(2)	1	$1.053\times {10}^{-1}$	$5.690\times {10}^{-3}$	$6.055\times {10}^{-4}$	$3.457\times {10}^{-1}$
GAM(1)	2	$8.536\times {10}^{-2}$	$3.828\times {10}^{-3}$	$1.092\times {10}^{-3}$	$3.302\times {10}^{-1}$
GAM(2)	2	$1.022\times {10}^{-1}$	$5.045\times {10}^{-3}$	$2.272\times {10}^{-4}$	$3.401\times {10}^{-1}$
OKS(1)	1	$9.022\times {10}^{-2}$	$3.923\times {10}^{-3}$	$1.140\times {10}^{-4}$	$3.138\times {10}^{-1}$
OKS(2)	1	$1.002\times {10}^{-1}$	$5.475\times {10}^{-3}$	$1.025\times {10}^{-3}$	$3.356\times {10}^{-1}$
OKS(1)	2	$8.957\times {10}^{-2}$	$3.932\times {10}^{-3}$	$1.721\times {10}^{-3}$	$3.137\times {10}^{-1}$
OKS(2)	2	$9.799\times {10}^{-2}$	$5.196\times {10}^{-3}$	$2.063\times {10}^{-3}$	$3.943\times {10}^{-1}$
VAR(1)	1	$1.071\times {10}^{-1}$	$6.552\times {10}^{-3}$	$2.551\times {10}^{-4}$	$3.729\times {10}^{-1}$
VAR(2)	1	$1.221\times {10}^{-1}$	$7.057\times {10}^{-3}$	$1.494\times {10}^{-4}$	$4.303\times {10}^{-1}$
VAR(1)	2	$1.101\times {10}^{-1}$	$6.624\times {10}^{-3}$	$4.366\times {10}^{-4}$	$4.074\times {10}^{-1}$
VAR(2)	2	$1.218\times {10}^{-1}$	$7.981\times {10}^{-3}$	$2.111\times {10}^{-3}$	$4.845\times {10}^{-1}$
L-VAR(1)	1	$1.030\times {10}^{-1}$	$6.165\times {10}^{-3}$	$1.713\times {10}^{-3}$	$3.389\times {10}^{-1}$
L-VAR(2)	1	$1.227\times {10}^{-1}$	$1.147\times {10}^{-2}$	$5.122\times {10}^{-4}$	$5.689\times {10}^{-1}$
L-VAR(1)	2	$1.120\times {10}^{-1}$	$7.190\times {10}^{-3}$	$3.882\times {10}^{-4}$	$4.726\times {10}^{-1}$
L-VAR(2)	2	$1.321\times {10}^{-1}$	$1.116\times {10}^{-2}$	$3.779\times {10}^{-4}$	$6.606\times {10}^{-1}$

and

Table 2. The summary statistics of absolute errors across 100 random samples in Scenario 2.

Method	H	MAE	SV	Min	Max
GAM(1)	1	$9.685\times {10}^{-2}$	$5.471\times {10}^{-3}$	$8.732\times {10}^{-4}$	$3.540\times {10}^{-1}$
GAM(2)	1	$1.067\times {10}^{-1}$	$5.198\times {10}^{-3}$	$1.591\times {10}^{-3}$	$4.162\times {10}^{-1}$
GAM(1)	2	$8.766\times {10}^{-2}$	$4.799\times {10}^{-3}$	$6.468\times {10}^{-5}$	$3.679\times {10}^{-1}$
GAM(2)	2	$9.398\times {10}^{-2}$	$4.931\times {10}^{-3}$	$5.201\times {10}^{-4}$	$3.328\times {10}^{-1}$
OKS(1)	1	$1.001\times {10}^{-1}$	$6.051\times {10}^{-3}$	$3.322\times {10}^{-4}$	$3.598\times {10}^{-1}$
OKS(2)	1	$1.033\times {10}^{-1}$	$5.552\times {10}^{-3}$	$1.457\times {10}^{-4}$	$3.600\times {10}^{-1}$
OKS(1)	2	$9.903\times {10}^{-2}$	$6.705\times {10}^{-3}$	$2.344\times {10}^{-4}$	$4.733\times {10}^{-1}$
OKS(2)	2	$1.077\times {10}^{-1}$	$5.976\times {10}^{-3}$	$3.031\times {10}^{-4}$	$3.851\times {10}^{-1}$
VAR(1)	1	$1.149\times {10}^{-1}$	$8.411\times {10}^{-3}$	$1.707\times {10}^{-3}$	$6.388\times {10}^{-1}$
VAR(2)	1	$1.240\times {10}^{-1}$	$8.426\times {10}^{-3}$	$2.506\times {10}^{-4}$	$4.782\times {10}^{-1}$
VAR(1)	2	$9.043\times {10}^{-2}$	$5.266\times {10}^{-3}$	$1.103\times {10}^{-4}$	$3.929\times {10}^{-1}$
VAR(2)	2	$1.091\times {10}^{-1}$	$5.890\times {10}^{-3}$	$7.651\times {10}^{-5}$	$3.778\times {10}^{-1}$
L-VAR(1)	1	$1.216\times {10}^{-1}$	$8.860\times {10}^{-3}$	$1.176\times {10}^{-3}$	$4.664\times {10}^{-1}$
L-VAR(2)	1	$1.577\times {10}^{-1}$	$2.785\times {10}^{-2}$	$1.538\times {10}^{-5}$	$1.317$
L-VAR(1)	2	$1.297\times {10}^{-1}$	$1.060\times {10}^{-2}$	$7.170\times {10}^{-4}$	$5.000\times {10}^{-1}$
L-VAR(2)	2	$1.648\times {10}^{-1}$	$3.918\times {10}^{-2}$	$4.257\times {10}^{-4}$	$1.604$

, while the results for the scenarios, including MAE, mean, sample variance (SV), and minimum (Min) and maximum (Max) values, with time-varying intercepts are presented in

Table 3. The summary statistics of absolute errors across 100 random samples in Scenario 3.

Method	H	MAE	SV	Min	Max
GAM(1)	1	$9.690\times {10}^{-2}$	$6.179\times {10}^{-3}$	$1.807\times {10}^{-4}$	$4.450\times {10}^{-1}$
GAM(2)	1	$9.687\times {10}^{-2}$	$5.538\times {10}^{-3}$	$1.142\times {10}^{-3}$	$3.745\times {10}^{-1}$
GAM(1)	2	$1.053\times {10}^{-1}$	$6.149\times {10}^{-3}$	$9.526\times {10}^{-4}$	$4.096\times {10}^{-1}$
GAM(2)	2	$9.646\times {10}^{-2}$	$5.265\times {10}^{-3}$	$7.879\times {10}^{-4}$	$3.612\times {10}^{-1}$
OKS(1)	1	$1.043\times {10}^{-1}$	$7.294\times {10}^{-3}$	$1.452\times {10}^{-3}$	$4.336\times {10}^{-1}$
OKS(2)	1	$1.158\times {10}^{-1}$	$6.752\times {10}^{-3}$	$1.672\times {10}^{-3}$	$4.136\times {10}^{-1}$
OKS(1)	2	$1.017\times {10}^{-1}$	$5.822\times {10}^{-3}$	$3.147\times {10}^{-3}$	$3.562\times {10}^{-1}$
OKS(2)	2	$1.064\times {10}^{-1}$	$6.591\times {10}^{-3}$	$3.352\times {10}^{-4}$	$3.244\times {10}^{-1}$
VAR(1)	1	$1.421\times {10}^{-1}$	$1.147\times {10}^{-2}$	$1.660\times {10}^{-3}$	$5.029\times {10}^{-1}$
VAR(2)	1	$1.950\times {10}^{-1}$	$1.299\times {10}^{-2}$	$4.270\times {10}^{-4}$	$4.774\times {10}^{-1}$
VAR(1)	2	$1.303\times {10}^{-1}$	$8.040\times {10}^{-3}$	$2.782\times {10}^{-4}$	$4.306\times {10}^{-1}$
VAR(2)	2	$1.713\times {10}^{-1}$	$1.051\times {10}^{-2}$	$3.078\times {10}^{-3}$	$4.372\times {10}^{-1}$
L-VAR(1)	1	$1.389\times {10}^{-1}$	$1.085\times {10}^{-2}$	$2.825\times {10}^{-4}$	$5.330\times {10}^{-1}$
L-VAR(2)	1	$3.376\times {10}^{-1}$	$1.371\times {10}^{-1}$	$7.087\times {10}^{-4}$	$2.122$
L-VAR(1)	2	$1.990\times {10}^{-1}$	$1.535\times {10}^{-2}$	$1.556\times {10}^{-3}$	$4.938\times {10}^{-1}$
L-VAR(2)	2	$3.771\times {10}^{-1}$	$1.665\times {10}^{-1}$	$1.296\times {10}^{-4}$	$2.113$

and

Table 4. The summary statistics of absolute errors across 100 random samples in Scenario 4.

Method	H	MAE	SV	Min	Max
GAM(1)	1	$1.073\times {10}^{-1}$	$5.848\times {10}^{-3}$	$1.557\times {10}^{-4}$	$4.182\times {10}^{-1}$
GAM(2)	1	$1.122\times {10}^{-1}$	$7.580\times {10}^{-3}$	$5.888\times {10}^{-5}$	$4.330\times {10}^{-1}$
GAM(1)	2	$9.938\times {10}^{-2}$	$5.378\times {10}^{-3}$	$7.262\times {10}^{-4}$	$3.778\times {10}^{-1}$
GAM(2)	2	$1.040\times {10}^{-1}$	$6.911\times {10}^{-3}$	$1.882\times {10}^{-4}$	$5.597\times {10}^{-1}$
OKS(1)	1	$1.322\times {10}^{-1}$	$1.251\times {10}^{-2}$	$8.679\times {10}^{-4}$	$7.965\times {10}^{-1}$
OKS(2)	1	$1.363\times {10}^{-1}$	$1.627\times {10}^{-2}$	$1.381\times {10}^{-4}$	$1.084$
OKS(1)	2	$1.388\times {10}^{-1}$	$2.157\times {10}^{-2}$	$3.240\times {10}^{-4}$	$1.272$
OKS(2)	2	$1.499\times {10}^{-1}$	$3.640\times {10}^{-2}$	$5.670\times {10}^{-4}$	$1.888$
VAR(1)	1	$1.655\times {10}^{-1}$	$1.632\times {10}^{-2}$	$1.351\times {10}^{-3}$	$7.625\times {10}^{-1}$
VAR(2)	1	$1.829\times {10}^{-1}$	$1.782\times {10}^{-2}$	$1.415\times {10}^{-4}$	$5.897\times {10}^{-1}$
VAR(1)	2	$1.076\times {10}^{-1}$	$6.766\times {10}^{-3}$	$1.152\times {10}^{-3}$	$4.453\times {10}^{-1}$
VAR(2)	2	$1.227\times {10}^{-1}$	$8.897\times {10}^{-3}$	$9.184\times {10}^{-5}$	$4.009\times {10}^{-1}$
L-VAR(1)	1	$1.667\times {10}^{-1}$	$1.759\times {10}^{-2}$	$8.022\times {10}^{-4}$	$6.426\times {10}^{-1}$
L-VAR(2)	1	$6.904\times {10}^{-1}$	$4.975\times {10}^{-1}$	$1.444\times {10}^{-3}$	$4.091$
L-VAR(1)	2	$1.779\times {10}^{-1}$	$1.867\times {10}^{-2}$	$4.661\times {10}^{-4}$	$7.056\times {10}^{-1}$
L-VAR(2)	2	$7.190\times {10}^{-1}$	$5.882\times {10}^{-1}$	$1.092\times {10}^{-3}$	$3.715$

. Next, the reconstructed results of the GAM-based method are shown in Figure 4. Finally, to evaluate the processing time of the GAM- and OKS-based methods across different time series dimensions, we ran each method 50 times per dimension under Scenario 1. The average processing time is presented in

Table 5. Computational time for the GAM- and OKS-based methods across different dimensions p.

Method	p	Time Cost (Seconds)	p	Time Cost (Seconds)	p	Time Cost (Seconds)	p	Time Cost (Seconds)
GAM(1)	2	2.51	3	3.88	4	5.56	5	6.94
GAM(1)	6	10.82	7	14.98	8	21.78	9	71.35
GAM(2)	2	3.10	3	4.75	4	11.76	5	47.61
GAM(2)	6	63.23	7	90.57	8	110.39	9	144.46
OKS(1)	2	27.42	3	28.26	4	31.88	5	36.55
OKS(1)	6	44.42	7	51.20	8	59.64	9	64.62
OKS(2)	2	15.65	3	23.66	4	25.95	5	33.92
OKS(2)	6	45.34	7	56.07	8	72.78	9	78.57

.

5.3. Discussion

The simulation results above compare the prediction performance of GAM, OKS, and VAR in modeling non-homogeneous Markov chains, both with and without an intercept. As expected, the MAE generally increases as the forecast horizon H extends from 1 to 2. In scenarios with stronger time-varying dynamics, both GAM and OKS outperform VAR and L-VAR, which aligns with our hypothesis, given that VAR and L-VAR assume stationarity in the whole process. However, OKS performs worse than GAM because it assumes that the estimated time-varying coefficient matrix remains fixed after the prediction point, which limits its adaptability, especially in situations where the coefficient matrix changes dramatically over time. Next, the performance of the GAM(2) model appears less effective in the zero-intercept setting compared to the non-zero intercept case. This is because, in processes with non-zero intercepts, the influence of time-varying effects tends to diminish in significance, thereby allowing even a stationary VAR model to achieve satisfactory performance. Furthermore, when reconstructing the entire time series, neither VAR nor OKS is suitable due to their reliance on stationarity assumptions; VAR assumes global stationarity, while OKS relies on local stationarity. As a result, they are only capable of forecasting or simulating specific time points, rather than capturing the full dynamic process. Finally, to evaluate robustness with respect to input length, we assessed the performance of both the GAM- and OKS-based models across time series ranging from 50 to 200 observations, and found their performance to be largely insensitive to series length. This insensitivity is likely due to the short forecasting horizon, a fixed two-step-ahead prediction, which limits temporal dependency and thereby reduces the influence of the overall input length on predictive accuracy.

Unfortunately, both our GAM-based and OKS-based methods require significantly more coefficients compared to traditional VAR, leading to increased computational complexity and higher time costs, which can limit overall performance. For example, consider a vector time series with dimension $p=3$, and assume five different basis functions. In the case of a second-order model, GAM(2) requires estimating $N_{GAM}=5\times 2\times 3^2+5=95$ coefficients, similarly, OKS(2) requires $N_{OKS}=5\times 2\times 3^2+1=91$ coefficients. Next, the time complexity of the proposed approach largely depends on the choice of optimization algorithm and the computational hardware used. In this study, we employed the ’trust-constr’ algorithm from the SciPy package, which utilizes Lagrangian gradient descent to handle constrained optimization. While switching to a faster algorithm, such as ’L-BFGS-B’, may reduce computation time, it generally lacks support for constraints, potentially compromising solution feasibility. Specifically, all experiments were conducted on a system equipped with an NVIDIA RTX 3060 GPU. Under this setup, training the GAM-based method with a lag order of one required approximately 2 to 71 s, depending on the time series dimension. When the lag order increased to two, the per-sample training time rose to approximately 3 to 144 s, reflecting the increased model complexity. Additionally, the OKS-based method exhibited a distinct trend. Although significantly slower than GAM at lower dimensions, due to the additional computation required for cross-validation, it became substantially more efficient at higher dimensions. In contrast, although VAR requires matrix inversion operations, it remains significantly faster than both GAM and OKS in practice. Overall, increasing the number of lags enables the model to capture longer-term dependencies and reveal stronger connections within the time series, potentially improving forecast performance, but it also significantly increases parameter complexity and the risk of overfitting. Therefore, effectively applying TV-VAR models requires a careful balance between model complexity and the number of lags to capture temporal dependencies without overfitting.

6. Estimating Time-Varying VAR Model on Finance-Related Datasets

In the real-world application, we consider the time series data in the finance domain, incorporating some key market indicators, including the S&P 500 index (as a broad measure of U.S. equity market performance), the VIX index (reflecting market volatility and investor sentiment), and U.S. Treasury securities with a one-month constant maturity (serving as a proxy for short-term interest rates and risk-free returns). In this research, all financial data were obtained from publicly available sources, including the Federal Reserve Economic Data at https://fred.stlouisfed.org/ (accessed on 1 May 2025) and the Chicago Board Options Exchange at https://www.cboe.com/ (accessed on 1 May 2025). These indices are widely used in financial modeling and risk assessment due to their relevance in capturing market dynamics, investor behavior, and macroeconomic conditions.

6.1. Data Preparation

Before modeling, several preprocessing steps are essential. For the S&P 500 index, prior research has traditionally focused on various forms of returns, such as simple and logarithmic returns, based on the assumption of stationarity. While this return-based approach facilitates certain statistical modeling techniques, it may overlook important information embedded in the actual index levels, making such models less effective in accurately predicting future price levels. Hence, to address this limitation, this study instead focuses on modeling the index itself rather than its derived returns. Next, occasional breakpoints in the one-month constant maturity series, caused by sudden shifts in market conditions or expectations, such as spikes in volatility, changes in monetary policy, or other technical factors, are filled using the value from the preceding day. The data used in this study were collected for the period spanning 4 January 2016 to 31 December 2024.

6.2. Results

To evaluate the performance of our methods on real-world financial data, we use the period from 4 January 2016 to 28 June 2024, to train the TV-VAR model under both the GAM and OKS frameworks with a lag order of $r=2$. After that, we perform one-step-ahead forecasting and compare the predicted values with the actual observations, denoted as $y_{true}$, which is then fed back into the model to retrain both GAM and OKS for the next prediction step. The complete prediction results, including the mean absolute error, are presented in Figure 5 and

Table 6. The summary statistics of absolute errors in the real-world financial data.

Method	Index	Mean	SV	Min	Max
GAM(2)	VIX	$2.482$	$5.933$	$1.795\times {10}^{-3}$	$54.997$
GAM(2)	S&P500	$34.352$	$36.924$	$5.469\times {10}^{-1}$	$268.401$
GAM(2)	Securities	$1.138\times {10}^{-1}$	$9.629\times {10}^{-2}$	$2.988\times {10}^{-3}$	$6.323\times {10}^{-1}$
OKS(2)	VIX	$2.599$	$5.911$	$1.068\times {10}^{-4}$	$58.564$
OKS(2)	S&P500	$43.628$	$43.485$	$1.572\times {10}^{-1}$	$268.967$
OKS(2)	Securities	$8.516\times {10}^{-2}$	$1.185\times {10}^{-1}$	$7.050\times {10}^{-5}$	$8.489\times {10}^{-1}$

.

The results show that both GAM and OKS perform well, achieving low MAEs across all three indices, although GAM exhibits slightly more fluctuation than OKS in predicting U.S. Treasury securities. Additionally, the findings suggest that when the VIX is high, U.S. Treasury securities tend to decline the following day. This aligns with expectations, as high market volatility often drives investors toward safe-haven assets, such as Treasury bonds, which in turn affects yields. Conversely, when the VIX decreases, investors tend to reallocate capital into equities such as the S&P 500, leading to reduced demand for Treasuries. Finally, the relationship between the VIX and the S&P 500 also holds: when the VIX is elevated, the S&P 500 typically declines or exhibits only modest gains, consistent with established market behavior.

7. Conclusions

Compared to stationarity, non-stationarity—where a time series’ statistical properties change over time—better aligns with real-world dynamics in fields such as finance, climate science, healthcare, and signal processing. Failing to account for it may lead to poor forecasts, especially in long-term predictions. The autoregressive model, a foundational tool in time series analysis, plays a crucial role in modeling time-dependent data, and its extension, the time-varying vector autoregressive (TV-VAR) model, broadens this framework to effectively handle both non-stationary dynamics and high-dimensional settings. Recent research on TV-VAR models has highlighted the effectiveness of the generalized additive model (GAM) framework and kernel smoothing (KS) methods in capturing dynamic relationships. However, few studies explicitly consider the case of TV-VAR with r lags. To model this process efficiently, we reformulate it as a series of optimization problems under appropriate constraints, which preserves the covariance structure between predictions and historical data without overfitting it over time. The methods presented in this research model TV-VAR time series data using optimization solvers, and simulation results demonstrate a clear advantage in prediction accuracy, especially under strong time variation. Furthermore, compared to the stationary VAR model and the OKS method, the GAM-based model not only delivers significantly better predictive performance but also enables reconstruction of the entire training process from the initial r lags, providing enhanced interpretability and flexibility. Compared to traditional hedging strategies, which assume stationarity, the TV-VAR framework offers significant advantages by capturing time-varying relationships and enabling the implementation of adaptive hedge ratios. In other words, when the model detects increased portfolio sensitivity to shocks or rising market volatility, it can trigger more proactive hedging with appropriate instruments, an adaptability that is especially crucial for managing tail risk, where sudden market shifts require swift and responsive adjustments.

As with other studies, our approaches—both GAM-based and OKS-based—have certain limitations. The primary drawback is the large number of parameters that must be estimated, especially in models with high dimensionality and many lags, making the model’s performance highly dependent on the efficiency of the optimization solver and susceptible to accuracy loss in the presence of high data noise. Hence, future research could explore two potential directions: first, reconstructing the GAM-based model using neural networks, particularly physics-informed neural networks, by integrating the time-varying framework in TV-VAR with physical constraints to enhance model structure and interpretability; second, refining the bandwidth selection for the OKS method, as the current cross-validation approach is time-consuming, and developing a more efficient method could significantly enhance its effectiveness. Therefore, our future work will involve reformulating the TV-VAR model with physics-based constraints and integrating it with deep learning techniques.