KOSLM: A Kalman-Optimal Hybrid State-Space Memory Network for Long-Term Time Series Forecasting

Abstract

Long-term time series forecasting (LTSF) remains challenging, as models must capture long-range dependencies and remain robust to noise accumulation. Traditional recurrent models, such as Recurrent Neural Networks (RNNs) and Long Short-Term Memory (LSTM), often suffer from instability and information degradation over extended horizons. The state-of-the-art method xLSTMTime improves memory retention through exponential gating and enhanced memory-transition rules, but it still lacks principled guidance. To address these issues, we propose the Kalman-Optimal Selective Long-Term Memory (KOSLM) model, which embeds a Kalman-optimal selective mechanism driven by the innovation signal within a structured state-space reformulation of LSTM. KOSLM dynamically regulates information propagation and forgetting to minimize state estimation uncertainty, providing both theoretical interpretability and practical efficiency. Extensive experiments across energy, finance, traffic, healthcare, and meteorology datasets show that KOSLM reduces mean squared error (MSE) by 14.3–38.9% compared with state-of-the-art methods, with larger gains at longer horizons. The model is lightweight, scalable, and achieves up to 2.5× speedup over Mamba-2. Beyond benchmarks, KOSLM is further validated on real-world Secondary Surveillance Radar (SSR) tracking under noisy and irregular sampling, demonstrating robust and generalizable long-term forecasting performance.

1. Introduction

Long-term time series forecasting (LTSF) aims to predict future values over extended horizons based on historical observations. Accurate long-range forecasts are critical for applications such as energy scheduling, climate modeling, financial planning, traffic management, and healthcare resource allocation. LTSF presents unique challenges: capturing long-range dependencies, mitigating error accumulation, and adapting to non-stationary temporal dynamics [1].

Traditional recurrent architectures, including Recurrent Neural Networks (RNNs) [2] and Long Short-Term Memory (LSTM) networks [3,4], often struggle with long sequences due to vanishing or exploding gradients [5]. While LSTM gates alleviate some of these issues, their heuristic design lacks explicit structural constraints derived from a principled optimality criterion, potentially leading to suboptimal memory retention over extended horizons. The recently proposed xLSTMTime further enhances long-range modeling by introducing the sLSTM and mLSTM components. The sLSTM employs scalar memory with exponential gating to manage long-term dependencies, while the mLSTM component uses matrix memory and a covariance update rule to enhance storage capacity and relevant information retrieval capabilities. Despite these improvements, xLSTMTime remains constrained by heuristic recurrent design.

The Kalman filter (KF) [6] provides optimal state estimation under Gaussian noise [7]. Recent studies have explored integrating KF with LSTM to improve the accuracy of time-series forecasting. Representative approaches include the following: (i) Deep Kalman Filters [8], which parameterize the state-transition and observation functions of KF using LSTM; (ii) KalmanNet [9], which employs an LSTM to learn residual corrections to the Kalman gain under a known KF model; and (iii) uncertainty-aware LSTM–KF hybrids [10], which estimate the covariance or uncertainty structures of KF through recurrent dynamics. However, these approaches generally maintain a loose coupling between LSTM and KF—they do not embed Kalman-inspired feedback directly into the internal gating dynamics of LSTM.

Recently, selective state space models (SSMs) [11,12] have demonstrated efficient sequence modeling with linear-time complexity. By dynamically modulating SSM’s parameters based on the input, they can filter task-irrelevant patterns while retaining critical long-term information. This selective modulation property motivates revisiting LSTM gates from a state-space perspective, into which Kalman-inspired feedback can be injected, thereby endowing the gating mechanism with Kalman-optimal structural constraints.

In this work, we propose the Kalman-Optimal Selective Long-Term Memory (KOSLM) model, which establishes a context-aware feedback pathway that optimally balances memory retention and information updating, providing both theoretical interpretability and practical efficiency.

Our main contributions are as follows:

• State-space reformulation of LSTM: We formalize LSTM networks as input- and state-dependent SSMs, where each gate dynamically parameterizes the state-transition and input matrices. This framework provides a principled explanation of LSTM’s long-term memory behavior.
• Kalman-optimal selective gating: Inspired by the Kalman filter and selective SSMs, we introduce a Kalman-optimal selective mechanism in which the state-transition and input matrices are linearly modulated by a Kalman gain learned from the innovation term, establishing a feedback pathway that minimizes state estimation uncertainty.
• Applications to real-world forecasting: KOSLM consistently outperforms state-of-the-art baselines across LTSF benchmarks in energy, finance, traffic, healthcare, and meteorology, achieving 14.3–38.9% lower mean squared error (MSE) and a maximum reduction in mean absolute error (MAE) of 25.2%; it also delivers up to 2.5× faster inference than Mamba-2. In real-world Secondary Surveillance Radar (SSR) tracking under noisy and irregular sampling, KOSLM demonstrates strong robustness and generalization.

By bridging heuristic LSTM gating with principled Kalman-optimal estimation, KOSLM provides a robust, interpretable, and scalable framework for long-term sequence modeling, offering both methodological novelty and practical forecasting utility.

2. Background and Theory

2.1. LSTM Networks

RNNs model sequential data through recursive temporal computation. However, traditional RNNs often suffer from vanishing and exploding gradients when capturing long-term dependencies. The LSTM network [3] addresses this issue by introducing gating mechanisms that regulate the flow of information over time. The network structure of the LSTM neuron is shown in Figure 1.

Each LSTM unit maintains a cell state $C_t$ that carries long-term information and a hidden state $H_t$ that provides short-term representations. The cell state is updated according to

$$C_t=F_t\odot C_{t-1}+I_t\odot {\tilde{C}}_t,$$

(1)

where $F_t$, $I_t$, and $O_t$ denote the forget, input, and output gates, respectively. These gates are nonlinear functions of the current input $x_t$ and the previous hidden state $H_{t-1}$. The final output is obtained as follows:

$$H_t=O_t\odot tanh\left(C_t\right).$$

(2)

This gating mechanism allows the model to selectively retain or discard information, mitigating gradient degradation. However, the gates are learned heuristically through data-driven optimization rather than derived from an explicit structural optimality constraint, making the LSTM sensitive to noise and unstable for long-term dependencies.

2.2. State Space Models

2.2.1. Selective State Space Models

Selective State Space Models (S6) are a recent class of sequence models for deep learning that are broadly related to RNNs and classical SSMs. They are inspired by a particular system (Equation (3)) that maps a one-dimensional function or sequence $x_t\in \mathbb{R}\to y_t\in \mathbb{R}$ through an implicit latent state $h_t\in {\mathbb{R}}^N$:

$$\begin{array}{cc}\hfill h_t& =A{h}_{t-1}+B{x}_t,\hfill \end{array}$$

(3a)

$$\begin{array}{cc}\hfill y_t& =M{h}_t.\hfill \end{array}$$

(3b)

where A is the state transition matrix, defining how the past latent state $h_{t-1}$ influences the current state; B is the input matrix, determining how $x_t$ modulates $h_t$; and M maps the implicit latent state $h_t$ to the output.

These models integrate the SSM described above into deep learning frameworks and introduce input-dependent selective mechanisms (see Appendix A for a detailed discussion), achieving Transformer-level modeling capability with linear computational complexity.

The theoretical connections among LSTM, KF, and SSM provide the foundation for constructing a unified Kalman-optimal selective memory framework.

2.2.2. Kalman Filter

The KF [6] is a classical instance of the state-space model, providing the minimum mean-square-error (MMSE) estimate of hidden system states under noisy observations. The system dynamics are expressed as

$$\begin{array}{cc}\hfill h_t& =A{h}_{t-1}+w_t,w_t\sim \mathcal{N}(0,Q_t),\hfill \end{array}$$

(4a)

$$\begin{array}{cc}\hfill z_t& =M{h}_t+v_t,v_t\sim \mathcal{N}(0,R_t),\hfill \end{array}$$

(4b)

where M is the same as the M in the SSM introduced above, but in the context of the KF, it is commonly referred to as the observation matrix; $w_t$ and $v_t$ denote process and observation noise with covariances $Q_t$ and $R_t$, respectively. Note that the observation $z_t$ can be regarded as the input $x_t$ in the SSMs.

At each time step, the KF performs two operations: prediction and update. The prediction step estimates the prior state ${\widehat{h}}_t^{-}$ and prior error covariance ${\widehat{P}}_t^{-}$ based on the posterior state ${\widehat{h}}_{t-1}$ and posterior error covariance ${\widehat{P}}_{t-1}$ from the previous time step:

$${\widehat{h}}_t^{-}=A{\widehat{h}}_{t-1},$$

(5)

$${\widehat{P}}_t^{-}=A{\widehat{P}}_{t-1}{A}^T+Q_t$$

(6)

The update step then refines this prediction using the observation $z_t$:

$$K_t={\widehat{P}}_t^{-}{M}^T{(M{\widehat{P}}_t^{-}{M}^T+R_t)}^{-1}$$

(7)

$${\widehat{h}}_t={\widehat{h}}_t^{-}+K_t(z_t-M{\widehat{h}}_t^{-})$$

(8)

$${\widehat{P}}_t=(I-K_{t}M){\widehat{P}}_t^{-}$$

(9)

Here, $K_t$ is the Kalman gain, which minimizes the posterior error covariance ${\widehat{P}}_t$. It determines how much new information (the innovation term $z_t-M{\widehat{h}}_t^{-}$) should be incorporated into the updated state (${\widehat{h}}_t$ is the posterior state). This principle of optimal selective information integration forms the theoretical foundation for the innovation-driven gating design proposed later. A detailed justification for interpreting the Kalman gain as a prototype of dynamic selectivity is provided in Appendix B.1.

3. Proposed Method

3.1. Reformulating LSTM as a State-Space Model

Following the LSTM formulation in Equation (1), the LSTM cell can be equivalently reformulated as a time-varying SSM, where the cell state $C_t$ evolves under nonlinear, input- and state-dependent dynamics:

$$C_t=A_t{C}_{t-1}+B_t{z}_t,$$

(10)

Here, $z_t$ serves as the observation in the KF (corresponding to the input $x_t$ in the SSM and LSTM), and the matrices $A_t$ and $B_t$ are determined by the forget gate $F_t$ and input gate $I_t$, respectively. A detailed derivation of this LSTM-to-SSM reconstruction, including the mapping of gating mechanisms to state-space parameters, is provided in Appendix B.2.

3.2. Kalman-Optimal Selectivity via Innovation-Driven Gain

We introduce the innovation term from the KF:

$${\mathit{Innov}}_t=z_t-M_t{A}_{t-1}{C}_{t-1},$$

(11)

which measures the discrepancy between the observation input $z_t$ and the predicted state based on the previous cell state $C_{t-1}$, serving as a real-time correction signal between model prediction and actual measurement.

In classical KF, this innovation drives the computation of the Kalman gain $K_t$ (Equation (7)), which regulates the incorporation of new information and the retention of prior state during the update step. In KOSLM, rather than explicitly solving the Riccati recursion [13], we learn a functional mapping:

$$K_t=\varphi ({\mathit{Innov}}_t;{\theta }_{\varphi }),$$

(12)

where $\varphi (·)$ is a lightweight multi-layer perceptron (MLP) with sigmoid activation, with its width and depth controlled by two hyperparameters, $\mathrm{MLP\_d}$ and $\mathrm{MLP\_layers}$ (e.g., $\mathrm{MLP\_d}=64$, $\mathrm{MLP\_layers}=2$), and parameterized by ${\theta }_{\varphi }$. The sigmoid activation ensures that the estimated gain $K_t\in (0,1)$, maintaining physical interpretability as an adaptive weighting coefficient and preventing numerical instability.

The learned gain dynamically regulates the trade-off between prior memory and new information, yielding a learnable yet principled mechanism for Kalman-optimal selectivity. The bounded output range of $K_t$ further stabilizes the state-space update and constrains divergence during long-horizon inference. Appendix B.3 provides the theoretical derivation demonstrating that the innovation term serves as a sufficient statistic for learning the Kalman gain. Appendix B.4 further presents controlled experiments confirming that the learned gain accurately approximates the oracle Kalman gain across various $(Q,R)$ regimes (

Table A1. Quantitative Results of Kalman Gain Learning. Supervised-$\varphi$ achieves orders-of-magnitude lower gain MSE and consistently better state estimation RMSE across all noise regimes compared to end-to-end learning, highlighting the critical role of explicitly modeling the innovation term in recovering accurate Kalman gain dynamics.

(Q, R)	Supervised-$\mathit{\varphi }$		End-to-End
	MSE	RMSE	MSE	RMSE
(5, 0.1)	$1.3\times {10}^{-6}$	0.317	$3.5\times {10}^{-3}$	0.368
(5, 0.5)	$2.9\times {10}^{-5}$	0.684	$1.8\times {10}^{-3}$	0.694
(10, 0.1)	$1.2\times {10}^{-6}$	0.216	$1.3\times {10}^{-3}$	0.341
(100, 0.5)	$2.2\times {10}^{-7}$	0.310	$6.8\times {10}^{-4}$	0.747

). Controlled ablation studies (see Section 4.2.2 and

Table 1. Capacity ablation (corrected). Performance of KOSLM with varying $\varphi$ network capacities on four long-term forecasting datasets ($L=720$). Averaged over five runs with standard deviations. Lower values (indicated by the down arrow “↓”) correspond to better performance.

Dataset	Variant	#Params	MSE ↓	MAE ↓	Δ MSE vs. SmallMLP
ETTm1	Linear	0.183M	0.480 ± 0.012	0.355 ± 0.008	+46.3%
	SmallMLP	0.255M	0.328 ± 0.010	0.282 ± 0.007	—
	MediumMLP	0.389M	0.335 ± 0.011	0.284 ± 0.001	+2.13%
	HighCap	0.898M	0.342 ± 0.015	0.288 ± 0.010	+4.27%
ETTh1	Linear	0.183M	0.440 ± 0.010	0.330 ± 0.004	+30.95%
	SmallMLP	0.255M	0.336 ± 0.008	0.291 ± 0.006	—
	MediumMLP	0.389M	0.341 ± 0.010	0.294 ± 0.002	+1.49%
	HighCap	0.898M	0.355 ± 0.012	0.302 ± 0.008	+5.65%
Traffic	Linear	0.183M	0.290 ± 0.011	0.232 ± 0.006	+9.02%
	SmallMLP	0.255M	0.266 ± 0.010	0.212 ± 0.007	—
	MediumMLP	0.389M	0.270 ± 0.012	0.216 ± 0.002	+1.50%
	HighCap	0.898M	0.279 ± 0.014	0.222 ± 0.005	+4.89%
Exchange	Linear	0.183M	0.272 ± 0.010	0.305 ± 0.005	+4.62%
	SmallMLP	0.255M	0.260 ± 0.009	0.291 ± 0.009	—
	MediumMLP	0.389M	0.264 ± 0.010	0.294 ± 0.007	+1.54%
	HighCap	0.898M	0.285 ± 0.012	0.310 ± 0.009	+9.62%

) show that increasing the hidden layer size or depth yields only marginal improvements in predictive performance, indicating that KOSLM’s performance is largely insensitive to the specific configuration of $\varphi$. This confirms that the performance gains primarily stem from the Kalman-optimal structural design rather than the network capacity. These results jointly establish the theoretical and empirical foundation for embedding Kalman-optimal selectivity into deep learning models.

We then define the state-space evolution as follows:

$$A_t=(I-K_t{M}_t)A,B_t=K_t,$$

(13)

where A is the base state transition matrix, serving as a learnable parameter of the model; $A_t$ and $B_t$ are dynamically modulated by the Kalman gain $K_t$.

3.3. Structural Overview of KOSLM

The KOSLM cell preserves the computational efficiency of a standard LSTM while embedding a Kalman-inspired feedback loop. The KOSLM network architecture is illustrated in Figure 2; at each timestep, the operations are as follows:

• Initialization: The learnable matrix A is initialized following S4D-Lin and for the real case is S4D-Real [14], which is based on the HIPPO theory [15]. These define the n-th element of A as $-{\displaystyle \frac{1}{2}}+ni$ and $-(n+1)$ respectively;
• Compute $z_t$ and $M_t$ via the candidate gate ${\tilde{C}}_t$ and output gate $O_t$;
• Compute the innovation: ${\mathit{Innov}}_t=z_t-M_{t}A{C}_{t-1}$;
• Estimate the Kalman gain: $K_t=\varphi \left({\mathit{Innov}}_t\right)$;
• Update state transition matrices: $A_t=(I-K_t{M}_t)A$, $B_t=K_t$;
• Propagate the hidden state: $C_t=A_t{C}_{t-1}+B_t{z}_t$;
• Compute the output hidden representation: $H_t=M_t{C}_t$.

To further clarify the conceptual and structural differences between KOSLM and existing Kalman-based neural architectures,

Table 2. Core structural comparison between KalmanNet and KOSLM.

Aspect	KalmanNet	KOSLM (Ours)
Core idea	Strictly follows the classical KF architecture (Section 2.2.2); the neural network learns to correct the Kalman gain $K_t^{\mathrm{KF}}$ under partially known linear dynamics.	Reinterprets LSTM gating as a Kalman-optimal state estimation problem; state estimation does not strictly adhere to the KF equations, but directly learns $K_t$ from the innovation term.
System dynamics	Assumes fixed parameters $(A,M,Q,R)$; suitable for systems with known or partially known dynamics.	Learns $(A_t,B_t,M_t)$ from data; A is a learnable parameter matrix; fully adaptive to nonlinear and nonstationary environments.
Form of gain	$K_t=K_t^{\mathrm{KF}}+f(s_t;{\theta }_f)$—learns a residual correction to the classical Kalman gain.	$K_t=\varphi ({\mathit{Innov}}_t;{\theta }_{\varphi })$—directly learns the gain function from the innovation.
Gain network input	$s_t=[{\widehat{h}}_{t|t-1},z_t-M{\widehat{h}}_{t|t-1}]$—uses both predicted state and innovation as inputs.	${\mathrm{Innov}}_t=z_t-M_t{A}_{t-1}{C}_{t-1}$—relies solely on the innovation signal for gain computation (see Appendices Appendix B.3 and Appendix B.4, which prove its sufficiency).
Output role	Outputs a residual correction $\Delta K_t$ added to $K_t^{\mathrm{KF}}$.	Outputs $K_t$ and integrates gain estimation into the state transition: $A_t=(I-K_t{M}_t)A$, $B_t=K_t$, forming a unified recurrent–estimation pathway.
System dependency	Requires partial knowledge of $(A,H,Q,R)$ for baseline Kalman computation.	Fully data-driven; no analytical Kalman gain or explicit system parameters required.
Theoretical interpretation	Neural residual learning to compensate model mismatch.	Innovation-driven dynamic selectivity that enforces Kalman-optimal information update behavior.

summarizes a direct comparison with the KalmanNet [9].

3.4. Theoretical Interpretation

Under linear–Gaussian assumptions and with a sufficiently expressive mapping $\varphi$, the learned gain $K_t$ converges to the oracle Kalman solution. Consequently, KOSLM inherits the nonlinear expressive power of LSTM while achieving the minimum-variance estimation property of the Kalman filter in its linear regime. This leads to improved stability and robustness, particularly in long-horizon or noisy sequence modeling.

3.5. Practical Advantages

KOSLM offers several practical benefits:

• Robustness: The feedback structure mitigates error accumulation and improves performance under noise or distributional shifts.
• Efficiency: With only 0.24 M parameters, KOSLM achieves up to 2.5× faster inference than Mamba-2, while maintaining competitive accuracy.
• Versatility: The model generalizes across diverse domains, from energy demand forecasting to radar-based trajectory tracking.

4. Results

This section provides a comprehensive evaluation of the proposed KOSLM model through large-scale long-term forecasting benchmarks (Section 4.1), component ablation studies (Section 4.2), efficiency assessments (Section 4.3), and a real-world radar trajectory tracking case study (Section 4.4). The experimental analyses collectively aim to validate both the predictive accuracy and robustness of KOSLM across diverse and noisy temporal conditions.

4.1. Main Experiments on Benchmark Datasets

To examine the capability of KOSLM in modeling long-range dependencies and maintaining stability over extended forecasting horizons, we conduct systematic experiments on nine widely used real-world datasets covering domains such as traffic flow, electrical load, exchange rate, meteorology, and epidemiology. The datasets vary in frequency, dimensionality, and temporal regularity, providing a comprehensive benchmark for assessing model generalization.

4.1.1. Dataset Details

We summarize the datasets used in this study as follows. Weather [16] contains 21 meteorological variables (e.g., temperature and humidity) recorded every 10 min throughout 2020. ETT (Electricity Transformer Temperature) [17] includes four subsets: two hourly-level datasets (ETTh1, ETTh2) and two 15-minute-level datasets (ETTm1, ETTm2). Electricity [18], derived from the UCI Machine Learning Repository, records hourly power consumption (kWh) of 321 clients from 2012 to 2014. Exchange [19] comprises daily exchange rates among eight countries. Traffic [20] consists of hourly road occupancy rates measured by 862 sensors on San Francisco Bay Area freeways from January 2015 to December 2016. Illness (ILI) dataset [21] tracks the weekly number of influenza-like illness patients in the United States.

Table 3. Details of benchmark datasets used in our experiments.

Dataset	Frequency	# Features	Time Steps	Time Span
ETTh1	1 h	7	17,420	2016–2017
ETTh2	1 h	7	17,420	2017–2018
ETTm1	15 min	7	69,680	2016–2017
ETTm2	15 min	7	69,680	2017–2018
Exchange	1 day	8	7588	1990–2010
Weather	10 min	21	52,696	2020
Electricity	1 h	321	26,304	2012–2014
ILI	7 days	7	966	2002–2020
Traffic	1 h	862	17,544	2015–2016

summarizes the statistical properties of all nine benchmark datasets. All datasets are divided into training, validation, and test subsets with a ratio of 7:1:2.

4.1.2. Implementation Details

All models are trained using the Adam optimizer without weight decay. The learning rate is selected from $[1\times {10}^{-3},1\times {10}^{-2}]$ via grid search. Batch size is set to 32 by default, adjustable up to 256 depending on GPU memory. Training is performed for 15 epochs, and the checkpoint with the lowest validation loss is used for testing. Experiments are repeated five times, and average results are reported (RMS values are provided in the Supplementary Materials). All models adopt a 2-layer architecture with hidden dimension 64. We use PyTorch’s default weight initialization, and no additional regularization (dropout or gradient clipping) is applied. All experiments are implemented in PyTorch 2.1.0 with Python 3.11 on NVIDIA RTX 4090 GPUs. Random seeds for Python, NumPy, and PyTorch are fixed to ensure reproducibility.

The gain function $\varphi$ is implemented as a one-layer MLP with hidden dimension 64 and sigmoid activation, followed by a linear projection to the gain matrix $K_t$. The same $\varphi$ network is shared across all timesteps to ensure parameter consistency.

4.1.3. Experimental Setup of Main Experiments

All experiments follow the evaluation protocol established in xLSTMTime [22], adopting prediction horizons of $T\in \{96,192,336,720\}$ for standard datasets and $T\in \{24,36,48,60\}$ for the weekly-sampled ILI dataset. We compare the proposed KOSLM with nine recent state-of-the-art baselines that represent diverse architectural paradigms, spanning state-space, recurrent, attention-based, and linear modeling frameworks:

• SSM-based: FiLM [23], S-Mamba [24];
• LSTM-based: xLSTMTime [22];
• Transformer-based: FEDformer [25], iTransformer [26], Crossformer [27];
• MLP/TCN-based: DLinear [28], PatchTST [29], TimeMixer [30].

This comprehensive selection enables a fair and systematic comparison across diverse sequence modeling paradigms.

4.1.4. Overall Performance

Table 4. Long-term forecasting: Multivariate long-term forecasting results on the Traffic, Electricity, Exchange, Weather, and ILI datasets. The prediction length is set to $T\in \{96,192,336,720\}$ for all datasets except ILI, which uses $O\in \{24,36,48,60\}$ due to its weekly resolution. The best results are shown in bold red, and the second-best are underlined purple. All results are averaged over 5 runs. Lower is better.

Models		KOSLM		xLSTMTime		FiLM		iTransformer		FEDFormer		S-Mamba		Crossformer		DLinear		PatchTST		TimeMixer
Metric		MSE	MAE	MSE	MAE	MSE	MAE	MSE	MAE	MSE	MAE	MSE	MAE	MSE	MAE	MSE	MAE	MSE	MAE	MSE	MAE
Traffic	96	0.290	0.230	0.358	0.242	0.416	0.294	0.395	0.268	0.587	0.366	0.382	0.261	0.522	0.290	0.650	0.396	0.462	0.295	0.462	0.285
	192	0.260	0.210	0.378	0.253	0.408	0.288	0.417	0.276	0.604	0.373	0.396	0.267	0.530	0.293	0.598	0.370	0.466	0.296	0.473	0.296
	336	0.227	0.179	0.392	0.261	0.425	0.298	0.433	0.283	0.621	0.383	0.417	0.276	0.558	0.305	0.605	0.373	0.482	0.304	0.498	0.297
	720	0.290	0.232	0.434	0.287	0.520	0.353	0.467	0.302	0.626	0.382	0.460	0.300	0.589	0.328	0.645	0.394	0.514	0.322	0.506	0.313
+31.96%	Avg	0.266	0.212	0.391	0.261	0.442	0.308	0.428	0.282	0.610	0.376	0.434	0.287	0.550	0.304	0.625	0.383	0.481	0.304	0.485	0.297
Electricity	96	0.136	0.184	0.128	0.221	0.154	0.267	0.148	0.240	0.193	0.308	0.139	0.235	0.219	0.314	0.197	0.282	0.181	0.270	0.153	0.247
	192	0.131	0.184	0.150	0.243	0.164	0.258	0.162	0.253	0.201	0.315	0.159	0.255	0.231	0.322	0.196	0.285	0.188	0.274	0.166	0.256
	336	0.120	0.177	0.166	0.259	0.188	0.283	0.178	0.269	0.214	0.329	0.176	0.272	0.246	0.337	0.209	0.301	0.204	0.293	0.185	0.277
	720	0.158	0.205	0.185	0.276	0.236	0.332	0.225	0.317	0.246	0.355	0.204	0.298	0.280	0.363	0.245	0.333	0.246	0.324	0.225	0.310
+13.37%	Avg	0.136	0.187	0.157	0.250	0.186	0.285	0.178	0.270	0.214	0.327	0.170	0.265	0.244	0.334	0.212	0.300	0.205	0.290	0.182	0.272
Exchange	96	0.135	0.212	–	–	0.086	0.204	0.086	0.206	0.148	0.278	0.086	0.207	0.256	0.367	0.088	0.218	0.088	0.205	0.095	0.207
	192	0.279	0.292	–	–	0.188	0.292	0.177	0.299	0.271	0.315	0.182	0.304	0.470	0.509	0.176	0.315	0.176	0.299	0.151	0.293
	336	0.341	0.339	–	–	0.356	0.433	0.331	0.417	0.460	0.427	0.332	0.418	1.268	0.883	0.313	0.427	0.301	0.397	0.264	0.361
	720	0.282	0.322	–	–	0.727	0.669	0.847	0.691	1.195	0.695	0.867	0.703	1.767	1.068	0.839	0.695	0.901	0.714	0.586	0.602
+5.47%	Avg	0.259	0.291	–	–	0.339	0.400	0.360	0.403	0.519	0.429	0.367	0.408	0.940	0.707	0.354	0.414	0.367	0.404	0.274	0.365
Weather	96	0.144	0.171	0.144	0.187	0.199	0.262	0.174	0.214	0.217	0.296	0.165	0.210	0.158	0.230	0.196	0.255	0.177	0.218	0.163	0.209
	192	0.224	0.236	0.192	0.236	0.228	0.288	0.221	0.254	0.276	0.336	0.214	0.252	0.206	0.277	0.237	0.296	0.225	0.259	0.208	0.250
	336	0.249	0.248	0.237	0.272	0.267	0.323	0.278	0.296	0.339	0.380	0.274	0.297	0.272	0.335	0.283	0.335	0.278	0.297	0.251	0.287
	720	0.169	0.175	0.313	0.326	0.319	0.361	0.358	0.347	0.403	0.428	0.350	0.345	0.398	0.418	0.345	0.381	0.354	0.348	0.339	0.341
+11.26%	Avg	0.197	0.208	0.222	0.255	0.253	0.309	0.258	0.278	0.309	0.360	0.251	0.276	0.259	0.315	0.265	0.317	0.259	0.281	0.240	0.271
ILI	24	1.160	0.507	1.514	0.694	1.970	0.875	3.154	1.235	3.228	1.260	–	–	3.041	1.186	2.215	1.081	1.319	0.754	1.453	0.827
	36	1.262	0.561	1.519	0.722	1.982	0.859	2.544	1.083	2.679	1.150	–	–	3.406	1.232	1.963	0.963	1.579	0.870	1.627	0.903
	48	1.098	0.545	1.500	0.725	1.868	0.896	2.489	1.112	2.622	1.080	–	–	3.459	1.221	2.130	1.024	1.553	0.815	1.644	0.914
	60	1.118	0.583	1.418	0.715	2.057	0.929	2.675	1.034	2.857	1.078	–	–	3.640	1.305	2.368	1.096	1.470	0.788	1.633	0.908
+22.04%	Avg	1.160	0.549	1.488	0.714	1.969	0.890	2.715	1.116	2.847	1.170	–	–	3.387	1.236	2.169	1.041	1.480	0.807	1.589	0.888

and 

Table 5. Long-term forecasting: Forecasting results on the ETT datasets with prediction lengths $T\in \{96,192,336,720\}$. The best results are shown in bold red, and the second-best are underlined purple. All results are averaged over 5 runs. Lower is better.

Models		KOSLM		xLSTMTime		FiLM		iTransformer		FEDFormer		S-Mamba		Crossformer		DLinear		PatchTST		TimeMixer
Metric		MSE	MAE	MSE	MAE	MSE	MAE	MSE	MAE	MSE	MAE	MSE	MAE	MSE	MAE	MSE	MAE	MSE	MAE	MSE	MAE
ETTm1	96	0.204	0.221	0.286	0.335	–	–	0.334	0.368	0.379	0.419	0.333	0.368	0.404	0.426	0.345	0.372	0.329	0.367	0.320	0.357
	192	0.233	0.237	0.329	0.361	–	–	0.377	0.391	0.426	0.441	0.376	0.390	0.450	0.451	0.380	0.389	0.367	0.385	0.361	0.381
	336	0.414	0.339	0.358	0.379	–	–	0.426	0.420	0.445	0.459	0.408	0.413	0.532	0.515	0.413	0.413	0.399	0.410	0.390	0.404
	720	0.481	0.368	0.416	0.411	–	–	0.491	0.459	0.543	0.490	0.475	0.448	0.666	0.589	0.474	0.453	0.454	0.439	0.458	0.441
+4.03%	Avg	0.333	0.291	0.347	0.372	–	–	0.407	0.410	0.448	0.452	0.398	0.405	0.513	0.496	0.403	0.407	0.387	0.400	0.382	0.395
ETTm2	96	0.100	0.232	0.164	0.250	0.165	0.256	0.180	0.264	0.203	0.287	0.179	0.263	0.287	0.366	0.193	0.292	0.175	0.259	0.175	0.258
	192	0.132	0.274	0.218	0.288	0.222	0.296	0.250	0.309	0.269	0.328	0.250	0.309	0.414	0.492	0.284	0.362	0.241	0.302	0.237	0.299
	336	0.244	0.382	0.271	0.322	0.277	0.333	0.311	0.348	0.325	0.366	0.312	0.349	0.597	0.542	0.369	0.427	0.305	0.343	0.298	0.340
	720	0.268	0.408	0.361	0.380	0.371	0.389	0.412	0.407	0.421	0.415	0.411	0.406	1.730	1.042	0.554	0.522	0.402	0.400	0.275	0.323
+24.39%	Avg	0.186	0.324	0.254	0.310	0.259	0.319	0.288	0.332	0.305	0.349	0.288	0.332	0.757	0.610	0.350	0.401	0.281	0.326	0.246	0.306
ETTh1	96	0.298	0.277	0.368	0.395	–	–	0.386	0.405	0.376	0.419	0.386	0.405	0.423	0.448	0.386	0.400	0.414	0.419	0.375	0.400
	192	0.337	0.306	0.401	0.416	–	–	0.441	0.436	0.420	0.448	0.443	0.437	0.471	0.474	0.437	0.432	0.460	0.445	0.479	0.421
	336	0.395	0.334	0.422	0.437	–	–	0.487	0.458	0.459	0.465	0.489	0.468	0.570	0.546	0.481	0.459	0.501	0.466	0.484	0.458
	720	0.471	0.364	0.441	0.465	–	–	0.503	0.491	0.506	0.507	0.502	0.489	0.653	0.621	0.519	0.516	0.500	0.488	0.498	0.482
+8.09%	Avg	0.375	0.320	0.408	0.428	–	–	0.454	0.447	0.440	0.460	0.455	0.450	0.529	0.522	0.456	0.452	0.469	0.454	0.459	0.440
ETTh2	96	0.194	0.338	0.273	0.333	–	–	0.297	0.349	0.358	0.397	0.296	0.348	0.745	0.584	0.333	0.387	0.302	0.348	0.289	0.341
	192	0.238	0.384	0.340	0.378	–	–	0.380	0.400	0.429	0.439	0.376	0.396	0.877	0.656	0.477	0.476	0.388	0.400	0.372	0.392
	336	0.258	0.394	0.373	0.403	–	–	0.428	0.432	0.496	0.487	0.424	0.431	1.043	0.731	0.594	0.541	0.426	0.433	0.386	0.414
	720	0.314	0.432	0.398	0.430	–	–	0.427	0.445	0.463	0.474	0.426	0.444	1.104	0.763	0.831	0.657	0.431	0.446	0.412	0.434
+27.46%	Avg	0.251	0.387	0.346	0.386	–	–	0.383	0.407	0.437	0.449	0.381	0.405	0.942	0.684	0.559	0.515	0.387	0.407	0.364	0.395

report the long-term multivariate forecasting results on nine real-world datasets, evaluated by MSE and MAE. Across nearly all datasets and prediction horizons, the proposed KOSLM achieves the best or near-best performance, highlighting its superior generalization and robustness under diverse temporal dynamics.

• Consistent superiority across domains: KOSLM outperforms all competing baselines, particularly under complex and noisy datasets. In terms of average MSE and MAE reductions, KOSLM achieves relative improvements of 31.96% and 18.77% on Traffic, 13.37% and 25.20% on Electricity, 5.47% and 20.27% on Exchange, 11.26% and 18.43% on Weather, and 22.04% and 23.09% on ILI. Moreover, on the four ETT benchmarks, KOSLM achieves MSE improvements of up to 27.46% and MAE improvements of up to 25.2%, demonstrating strong adaptability to varying periodic and nonstationary patterns. These consistent gains verify that the Kalman-inspired selective updating mechanism effectively filters noise and dynamically adjusts to regime shifts, ensuring stable forecasting accuracy over long horizons.
• Stable error distribution and reduced variance: The MSE–MAE gap of KOSLM remains narrower than that of other baselines, implying reduced large deviations and more concentrated prediction errors. This indicates more stable error behavior, which is crucial for long-horizon forecasting where cumulative drift often occurs. The innovation-driven Kalman gain estimation provides adaptive correction at each timestep, ensuring smooth and consistent prediction trajectories under uncertain dynamics.
• Strong scalability and generalization: KOSLM achieves leading performance not only on large-scale datasets (Traffic, Electricity) but also on small, noisy datasets (ILI), confirming robust generalization across different temporal resolutions and noise levels. Its consistent advantage over Transformer-based (e.g., iTransformer, FEDFormer, PatchTST), recurrent (e.g., xLSTMTime), and state-space models (e.g., S-Mamba, FiLM) demonstrates that the proposed Kalman-optimal selective mechanism provides an effective inductive bias for modeling long-term dependencies.
• Advantage over LSTM-based architectures: Compared with advanced LSTM-based models such as xLSTMTime, KOSLM achieves consistently better results across nearly all datasets and horizons. This verifies that replacing heuristic gates with Kalman-optimal selective gating enhances memory retention and update stability. While xLSTMTime alleviates gradient decay via hierarchical memory, KOSLM further refines state updates through innovation-driven gain estimation, thereby achieving a more principled and stable information flow.

To further demonstrate the long-horizon stability of the proposed model, we compare KOSLM with the recent S-Mamba [24], a state-of-the-art state space model that represents the latest advancement in efficient sequence modeling. Figure 3 presents the forecasting trajectories at the long horizon ($T=720$) across five representative datasets. KOSLM maintains accurate trend alignment and amplitude consistency with the ground truth, showing particularly superior convergence behavior on smoother and more stationary datasets such as Weather and Exchange, where the Kalman gain adaptation stabilizes long-term predictions. In contrast, S-Mamba exhibits slight temporal lag and amplitude attenuation under extended forecasting conditions. These results visually confirm the advantage of the proposed Kalman-based feedback selectivity in preserving long-term temporal fidelity.

4.2. Ablation Study

To further analyze the effectiveness of each proposed component, we conduct ablation experiments on four widely used long-term time-series forecasting datasets (ETTm1, ETTh1, Traffic, and Exchange) with a prediction length of $L=720$, following the data preprocessing described in Section 4.1.1. All model variants are trained and tested following the default implementation settings described in Section 4.1.2, and the results are averaged over five runs to ensure statistical reliability.

4.2.1. Structural Ablation

We first evaluate the contribution of the Kalman-based structure by comparing three variants: (i) Full (KOSLM): the complete model where $K_t=\varphi \left(\mathrm{Innov}\right)$ and $A_t=(I-K_t{M}_t)A,B_t=K_t$; (ii) No-Gain: $\varphi$ receives the innovation but directly outputs $(A_t,B_t)$, removing the explicit computation of the Kalman gain $K_t$, i.e., the pathway $\mathrm{Innov}\to K_t\to (A_t,B_t)$ is replaced by $\mathrm{Innov}\to (A_t,B_t)$; (iii) No-Innov: the innovation statistic is removed. $\varphi$ receives the standard network input (e.g., $[x_t;H_{t-1}]$), while $K_t$ is still computed and mapped to $(A_t,B_t)$ via the Kalman form, i.e., $[x_t;H_{t-1}]\to K_t\to (A_t,B_t)$.

As shown in

Table 6. Structural ablation. Comparison of KOSLM and its structural variants on four long-term forecasting datasets ($L=720$). Removing either the Kalman gain path or the innovation input leads to consistent performance degradation. All results are averaged over five runs with mean ± standard deviation. Lower values (indicated by the down arrow “↓”) correspond to better performance.

Dataset	Model	MSE ↓	MAE ↓	Δ MSE vs. Full
ETTm1	Full	0.498 ± 0.011	0.416 ± 0.007	—
	No-Gain	0.662 ± 0.017	0.553 ± 0.014	+32.9%
	No-Innov	0.583 ± 0.012	0.487 ± 0.008	+17.1%
ETTh1	Full	0.510 ± 0.012	0.378 ± 0.010	—
	No-Gain	0.605 ± 0.022	0.448 ± 0.015	+18.6%
	No-Innov	0.554 ± 0.016	0.411 ± 0.010	+8.6%
Traffic	Full	0.321 ± 0.008	0.195 ± 0.016	—
	No-Gain	0.428 ± 0.011	0.260 ± 0.008	+33.4%
	No-Innov	0.375 ± 0.009	0.228 ± 0.007	+16.9%
Exchange	Full	0.266 ± 0.014	0.341 ± 0.023	—
	No-Gain	0.299 ± 0.018	0.383 ± 0.023	+12.3%
	No-Innov	0.294 ± 0.015	0.377 ± 0.019	+10.6%

, removing either the gain computation or the innovation input consistently leads to higher errors across all datasets. For instance, on ETTm1, the MSE increases by 32.9% when the gain path is removed, confirming that both the innovation statistic and Kalman gain are essential for stable long-horizon prediction. This finding aligns with the Kalman filtering principle that the innovation serves as a sufficient statistic for state correction.

4.2.2. Capacity Ablation

To verify that the performance gains of KOSLM mainly stem from the Kalman-optimal structural design rather than the network capacity, we conduct a capacity ablation study. We evaluate the $\varphi$ network under four progressively larger configurations: (i) Linear: A single linear layer with input dimension equal to the number of input features and output dimension 64, without any activation; (ii) SmallMLP: A two-layer MLP with 128 hidden units and sigmoid activation; (iii) MediumMLP: A three-layer MLP with 256 hidden units per layer and sigmoid activation; (iv) HighCap: A four-layer MLP with 512 hidden units per layer and sigmoid activation.

All models are trained under identical settings with five independent random seeds, and the results are reported as mean ± standard deviation. This allows us to evaluate both performance trends and statistical reliability. From Table 1, it is evident that increasing the $\varphi$ network capacity beyond SmallMLP does not consistently improve MSE or MAE; in some cases, the error even increases slightly. The small MediumMLP improvements (ETTm1) or slight deterioration (ETTh1, Traffic, Exchange) indicate that performance gains are dominated by the Kalman-optimal structural design rather than network depth or over-parameterization. This confirms the effectiveness of the innovation-driven gain estimation in KOSLM for stable long-horizon prediction.

4.3. Efficiency Benchmark

To evaluate the computational efficiency of KOSLM, we benchmark it against representative sequence modeling baselines, including Transformer, Mamba-2, and LSTM, across four key metrics: runtime scalability, end-to-end throughput, memory footprint, and parameter count. All experiments are conducted using PyTorch 2.1.0 with torch.compile optimization on a single NVIDIA RTX 4090 GPU.

4.3.1. Runtime Scalability

We assess runtime performance on both a controlled synthetic setup and the ETTm1 dataset. As illustrated in Figure 4, KOSLM exhibits near-linear growth in inference time with increasing sequence length L. For sequences longer than $L=2\mathrm{k}$, KOSLM surpasses optimized Transformer implementations (FlashAttention [31]) and achieves up to $2.5\times$ faster execution than the fused-kernel version of Mamba-2 [24] at $L>4\mathrm{k}$. Compared with a standard PyTorch LSTM, KOSLM achieves approximately $1.3$–$1.9\times$ speedup across the tested sequence lengths, demonstrating robust scalability for long sequences.

4.3.2. Throughput Analysis

Figure 5 reports the end-to-end inference throughput measured in million tokens/s (M tokens/s). KOSLM maintains consistently high throughput across all tested sequence lengths, achieving up to $1.9\times$ higher throughput than LSTM and $2.7\times$ higher than Mamba-2 in the 4K–8K regime. The performance remains stable for even longer sequences, highlighting the model’s suitability for extended temporal dependencies.

4.3.3. Memory Footprint

Table 7. Memory footprint under different batch sizes, input length = 2048. KOSLM exhibits lower memory usage than Transformer and Mamba-2, while remaining comparable to LSTM despite its smaller size.

Batch Size	Transformer (GB)	Mamba-2 (GB)	LSTM (GB)	KOSLM (GB)
1	0.223	0.085	0.081	0.061
2	0.363	0.158	0.109	0.104
4	0.640	0.290	0.166	0.188
8	1.180	0.561	0.283	0.344
16	2.256	1.103	0.518	0.668
32	4.408	2.188	0.987	1.317
64	8.712	4.357	1.926	2.615
128	17.000	8.696	3.803	5.211

reports GPU memory usage during training for varying batch sizes with input length 2048 and models containing approximately 0.24 M parameters. KOSLM demonstrates favorable memory efficiency relative to similarly sized Mamba-2 implementations and remains comparable to a standard LSTM with roughly ten times more parameters. This efficiency results from KOSLM’s compact Kalman-optimal selective mechanism, which dynamically modulates state transitions with minimal parameter overhead.

4.3.4. Model Size

Table 8. Model parameter sizes. KOSLM remains lightweight while providing competitive long-horizon forecasting performance.

Model	Parameters (M)
LSTM	2.17
Transformer	5.33
Mamba-2	0.21
FiLM	1.50
KOSLM	0.24

summarizes the parameter counts. KOSLM contains only 0.24 M parameters, roughly 4.5% of the Transformer and 11% of the LSTM baselines, while achieving comparable or superior forecasting performance. The compact design emphasizes KOSLM’s suitability for deployment in resource-constrained environments.

This efficiency stems directly from the Kalman-optimal selective mechanism, which adaptively regulates the state update through innovation-driven gain modulation. Such a formulation not only reduces redundant computation but also provides a principled pathway for scaling to long sequences.

4.4. Real-World Application: Secondary Surveillance Radar (SSR) Target Trajectory Tracking

To further evaluate the robustness and practical generalizability of KOSLM under real-world noisy conditions, we conducted a real-world experiment on SSR target trajectory tracking. SSR is a ground-based air traffic surveillance system where aircraft respond to radar interrogations with encoded transponder signals, generating sparse range–azimuth sequences known as raw SSR plots. These observation sequences present three major challenges for sequential modeling:

• High stochastic noise: Measurement noise leads to random fluctuations in the estimated positions;
• Irregular sampling: Aircraft maneuvers and radar scan intervals result in uneven temporal spacing;
• Correlated anomalies: Spurious echoes or missing detections introduce discontinuities in the trajectories.

These characteristics make SSR a natural yet challenging testbed for assessing model stability, noise resilience, and temporal consistency. KOSLM addresses these challenges by integrating context-aware selective state updates with learnable dynamics under the Kalman optimality principle, enabling robust filtering, smoothing, and extrapolation in partially observed environments.

4.4.1. Experimental Setup

To achieve a balance between realism and experimental controllability, we adopted a semi-physical simulation approach. Training data were derived from Automatic Dependent Surveillance–Broadcast (ADS-B) flight tracks obtained from the OpenSky Network [32]. Controlled Gaussian noise with an SNR of 33 dB was added to emulate SSR observation uncertainty. For evaluation, we deployed KOSLM on real SSR radar data collected from an operational ground-based system (peak transmit power 6 kW), capturing eight live air-traffic targets under operational conditions. Detailed data acquisition and preprocessing procedures are provided in Appendix C.

Unlike the main benchmarking experiments on standardized long-term time series forecasting (LTSF) datasets, this case study aims to demonstrate practical applicability and operational robustness. Due to irregular sampling, sparse observations, and high stochastic noise, conventional quantitative metrics such as MSE or MAE are not meaningful. Therefore, we focus on qualitative trajectory visualizations to illustrate model performance in realistic conditions.

4.4.2. Results and Analysis

Figure 6 presents representative trajectories of eight air-traffic targets, comparing classical KF algorithm, Transformer, Mamba, and KOSLM. The classical KF algorithm exhibits locally inaccurate and jagged trajectories due to its fixed linear dynamics, which cannot adapt to irregular sampling or abrupt maneuvers. The Transformer captures general trends but produces fragmented and temporally inconsistent tracks under sparse and noisy observations. Mamba improves noise robustness but still demonstrates local instability during complex maneuvers.

In contrast, KOSLM generates smooth, coherent, and temporally consistent trajectories that closely follow the true flight paths, highlighting its ability to handle high stochastic noise, irregular sampling, and correlated anomalies. This robustness stems from the innovation-driven Kalman gain and context-aware selective state updates, which allow KOSLM to adapt dynamically to non-stationary motion patterns.

5. Conclusions

This study addressed a fundamental limitation of recurrent architectures such as LSTM, whose heuristically designed gates lack structural constraints for optimality, leading to instability and information decay over long sequences. To overcome this issue, we proposed the Kalman-Optimal Selective Long-Term Memory Network (KOSLM), which reconstrues LSTM as a nonlinear, input- and state-dependent state-space model, and integrates an innovation-driven Kalman-optimal gain path for principled information selection. This formulation unifies LSTM gating, selective state-space modeling, and Kalman filtering into a single theoretically grounded recurrent framework. We note that the proposed KOSLM is lightweight, containing only 0.24 M parameters—significantly smaller than typical LSTM or Transformer baselines—while still achieving competitive or superior performance. This compact design ensures efficient implementation on standard GPU hardware without imposing constraints on memory or computation.

Extensive experiments demonstrate that KOSLM achieves state-of-the-art performance on long-term forecasting benchmarks, while ablation studies confirm the essential role of the innovation statistic and Kalman-form gain in stabilizing long-horizon modeling. Moreover, validation on real-world SSR trajectory tracking highlights its robustness under noisy and non-stationary conditions.

In summary, embedding Kalman-optimal principles into deep recurrent networks provides both theoretical insights and practical benefits for robust long-term sequence modeling. Future work will focus on extending KOSLM beyond Gaussian assumptions and applying it to multimodal and cross-domain time-series scenarios.