C-T-Mamba: Temporal Convolutional Block for Improving Mamba in Multivariate Time Series Forecasting

Abstract

In recent years, Transformer-based methods have demonstrated proficiency in capturing complex patterns for time series forecasting. However, their quadratic complexity relative to input sequence length poses a significant bottleneck for scalability and real-world deployment. Recently, the Mamba architecture has emerged as a compelling alternative by mitigating the prohibitive computational overhead and latency inherent in Transformers. Nevertheless, a vanilla Mamba backbone often struggles to adequately characterize intricate temporal dynamics, particularly long-term trend shifts and non-stationary behaviors. To bridge the gap between Mamba’s global scanning and local dependency modeling, we propose C-T-Mamba, a hybrid framework that synergistically integrates a Mamba block, channel attention, and a temporal convolution block. Specifically, the Mamba block is leveraged to capture long-range temporal dependencies with linear scaling, the channel attention mechanism filters redundant information, and the temporal convolution block extracts multi-scale local and global features. Extensive experiments on five public benchmarks demonstrate that C-T-Mamba consistently outperforms state-of-the-art (SOTA) baselines (e.g., PatchTST and iTransformer), achieving average reductions of 4.3–18.5% in MSE and 3.9–16.2% in MAE compared to representative Transformer-based and CNN-based models. Inference scaling analysis reveals that C-T-Mamba effectively breaks the computational bottleneck; at a horizon of 1536, it achieves an $8.8\times$ reduction in GPU memory and over 10× speedup compared to standard Transformers. At 2048 steps, its latency remains as low as 8.9 ms, demonstrating superior linear scaling. These results underscore that C-T-Mamba achieves SOTA accuracy while maintaining a minimal computational footprint, making it highly effective for long-term multivariate time series forecasting.

1. Introduction

Time series forecasting represents a significant research avenue within the domains of statistics and machine learning [1,2]. Its applications are pervasive, spanning a multitude of disciplines, including economics, finance, meteorology, energy, transportation, and healthcare. Time series data are observations of variables recorded in chronological order, and thus contain information about the historical evolution of the data, as well as future trends and changes. These can be predicted by analyzing and modeling the data [3]. An overview of typical time series data, specifically the Oil Temperature feature from the ETTm1 dataset, is illustrated in Figure 1. Currently, there are two main categories of time series prediction methods based on machine learning: (1) neural network models based on Transformer architecture, and (2) neural network models with non-Transformer architecture [4].

Figure 1. Data overview of the Oil Temperature feature column in the ETTm1 dataset.

• Neural network models based on Transformer architecture:

The Transformer model was initially introduced to the field of Natural Language Processing (NLP), where it achieved considerable success [5]. This led to its adoption by researchers in the field of time series prediction. The self-attention mechanism enables the model to focus on all data points in the entire sequence simultaneously, thereby facilitating the capture of temporal dependencies and the learning of global patterns and overall trends in the sequence [6]. In comparison to traditional Recurrent Neural Network (RNN) [7] and Long Short-Term Memory (LSTM) [8] models, RNN is susceptible to the issue of gradient disappearance or gradient explosion when dealing with long time series data. Similarly, LSTM’s intricate gating mechanism and sequential processing characteristics result in a lack of computational efficiency, as evidenced by the persistence of the gradient problem observed in RNN. Transformer addresses these challenges through the integration of a self-attention mechanism and a feedforward network, thereby enhancing the stability and efficiency of the model training process. Nevertheless, self-attention enables Transformer to grasp the global context, but it also introduces the challenge of increased complexity. Self-attention necessitates calculating the correlation between each token in the input sequence and all other tokens, which involves significant time and space considerations. The complexity of this calculation is proportional to the square of the length of the input sequence. When the input sequence is of considerable length, the transformer model will consume a significant amount of computational resources and storage space, which may result in reduced efficiency and performance degradation. To address this challenge, subsequent research has proposed sparse attention methods, which reduce the number of correlation pairs computed in the self-attention mechanism, thereby reducing the time and space complexity. This enables the transformer model to process longer input sequences.

The ProbSparse attention mechanism in Informer [9] achieves $O\left(LlogL\right)$ in terms of time complexity and memory usage, which reduces the computational complexity from $O\left(L^2\right)$ to $O\left(LlogL\right)$ by using the KL scattering method to evaluate the sparsity of the ith query and then selecting the TOP u query. The Autoformer [10] implements an effective autocorrelation mechanism to uncover dependencies and aggregate information at the sequence level. Its computational efficiency is characterized by a time complexity of $O\left(LlogL\right)$. Through the application of the Wiener-Khinchin theorem and Fourier Transform, the auto-correlation method computes coefficients for sequences, subsequently establishing confidence levels for each cycle length. Following this analysis, the top k cycle lengths are prioritized for integration, thereby completing the extraction of temporal dependencies. The Reformer [11] replaces dot product attention with locally sensitive hashing, reducing the time complexity from $O\left(L^2\right)$ to $O\left(LlogL\right)$. FEDformer [12] proposes an attention mechanism that uses low-order frequency approximation and hybrid expert decomposition to control distributional bias. The proposed frequency-augmented structure separates the length of the input sequence from the dimension of the attention matrix, thus achieving linear complexity. The authors proved that the self-attention mechanism can be approximated by low-rank matrices, and further used this finding to propose a new self-attention mechanism that reduces the overall self-attention complexity in time and space from $O\left(n^2\right)$ to $O\left(n\right)$. The Crossformer [13] is a time series forecasting model based on the Transformer architecture designed for multivariate time series (MTS) forecasting. In addition to the traditional cross-temporal dependencies, Crossformer incorporates cross-dimensional dependencies, thereby addressing one of the key issues often overlooked in existing Transformer models for MTS forecasting. The DSW embedding technique and the two-stage attention (TSA) layer enable Crossformer to effectively capture the complex relationships inherent in MTS data, thereby enhancing the accuracy of the forecasts produced.

• Neural network models with non-Transformer architecture:

FourierGNN [14] introduces a novel data structure, the hypervariable graph, which treats each sequence value as a graph node, collectively capturing spatio-temporal dynamics and transforming MTS predictions into predictions on the hypervariable graph. FourierGNN combines the Fourier Graph Operator (FGO), which performs matrix multiplications in the Fourier space, enhancing the expressive power and decreasing the complexity, thus enabling efficient prediction.

In order to adaptively discover multi-periodicity and extract complex temporal variations from the transformed 2D tensor, TimesNet [15] proposes the use of TimeBlock, which is a parameter-efficient initial block. The Multilayer Perceptron (MLP) is redesigned in the frequency domain with the objective of efficiently capturing the underlying patterns of time series with a global view and energy compression.

TiDE [16] proposes a multilayer perceptron (MLP)-based encoder-decoder model, the Time Series Dense Encoder (TiDE), for long-term time series forecasting. This model offers the simplicity and speed of a linear model while being able to handle covariates and nonlinear dependencies. Theoretically, the authors demonstrate that, under specific assumptions, the most straightforward linear simulation of TiDE model can achieve near-optimal error rates for linear dynamical systems (LDS).

State Space Model (SSM) [17] has demonstrated considerable potential for long-term dependency modeling and has achieved notable advancements in various domains of Artificial Intelligence (AI). However, it lacks the capability for dynamic inference, which led to the development of the Mamba model. Mamba addresses the computational inefficiency of Transformer on long sequences by integrating a selective state-space model that enables content-based inference and selective information propagation along the sequence length dimension, thereby providing rapid inference and linear scaling of sequence length. Unlike other models, the Mamba architecture does not rely on attention or MLP blocks. Mamba’s architecture is distinguished by selective information propagation and fast inference, as well as linear scaling of sequence length. Notably, it does not rely on attention or MLP blocks.

In this study, we present the Mamba model as a substitute for the Transformer framework. Our objective is to tackle the challenge of training scaling quadratically based on input sequence length. Additionally, we integrate the channel mixing attention module to capture intricate temporal changes within the sequence. Lastly, we employ temporal convolutional block for comprehensive global and local feature integration. This approach effectively harmonizes model complexity with performance.

The shift from traditional parametric models to architectures like C-T-Mamba is further motivated by the inherent risks of model misspecification. As demonstrated by the Hassani $-1/2$ theorem, empirical correlation structures in time series often exhibit non-intuitive constraints that can undermine naive parametric inference [18]. By adopting a more flexible, SSM-based framework, C-T-Mamba avoids rigid distributional assumptions, providing more robust power in capturing non-stationary temporal dynamics. Overall, our main contributions are as follows:

(1)

We put forth a novel model, designated C-T-Mamba, which is based on the Mamba architectural framework. This approach effectively supplants the computationally expensive self-attention module in Transformer with the introduction of innovative channel mixing and temporal convolution block designs, thereby facilitating the discernment of intricate temporal patterns in time series.

(2)

Channel attention is employed to accentuate salient features within the sequence, while a temporal convolution block centered on deeply separable convolution is utilized to integrate the extracted features before capturing the global information and local features of the sequence.

(3)

Our model demonstrates superior performance compared to existing time series forecasting models on five public datasets of time series.

2. Related Work

2.1. Time Series Forecasting Tasks

Let us consider a time series, represented by the vector of values ${\left\{Y_t\right\}}_{t=1}^T$, In this context, the term $Y_t$ denotes the time at which t of the observation occurs. The objective of time series forecasting is to employ the available historical observations $\left\{Y_1,Y_2,\dots ,Y_T\right\}$ to make predictions about future events $\left\{{\widehat{Y}}_{T+1},{\widehat{Y}}_{T+2},\dots ,{\widehat{Y}}_{T+h}\right\}$. Mathematically, time series forecasting can be formalized as a function $\widehat{f}(Y_{T-P+1},Y_{T-P+2},\dots Y_T)$, which utilizes past observations $Y_{T-P+1},Y_{T-P+2},\dots Y_T$ to predict future observations $\left\{{\widehat{Y}}_{T+1},{\widehat{Y}}_{T+2},\dots ,{\widehat{Y}}_{T+h}\right\}$.

2.2. Applications of Mamba

2.2.1. Mamba for Natural Language Processing

The Jamba [19] architecture represents a novel approach that combines the Attention and Mamba layers with the MoE module and its open implementation, achieving state-of-the-art performance while supporting long contexts. Jamba offers the largest model with 12 B active parameters and 52 B total available parameters, and is capable of supporting context lengths up to 256 K tokens, even when processing 140 K tokens of text. The Jamba model offers a maximum of 12 billion active parameters and 52 billion total available parameters, supporting context lengths up to 256,000 tokens, even when processing 140,000 tokens of text, on a single 80 GB GPU.

2.2.2. Applications of Mamba to Computer Vision

$R{S}^3$Mamba [20] proposes a novel two-branch network, designated as Remote Sensing Image Semantic Segmentation Mamba ($R{S}^3$Mamba), which has been specifically devised for remote sensing tasks. $R{S}^3$Mamba employs VSS modules to construct a secondary branch that furnishes supplementary global information to the convolution-based primary branch. Furthermore, in consideration of the disparate characteristics of the two branches, a Collaborative Completion Module (CCM) is introduced, which employs a novel adaptive mechanism to refine and fuse features derived from the dual encoders. The efficacy of the proposed $R{S}^3$Mamba was demonstrated through experiments conducted on two widely used datasets. Compared to the state-of-the-art methods, $R{S}^3$Mamba exhibited superior performance, achieving an mIoU of 0.66% higher than the current best on the ISPRS Vaihingen dataset and 1.70% higher on the LoveDA Urban dataset. These results highlight the effectiveness and potential of $R{S}^3$Mamba in addressing remote sensing tasks.

2.2.3. Mamba in Time Series Forecasting

Despite the success of initial Mamba-based forecasting models, such as MambaTS [21] which utilizes the Mamba block for efficient sequence modeling, and S-Mamba [22] which explores structured state space models for multivariate dependencies, a common limitation persists: the vanilla Mamba scanning mechanism is inherently biased towards global trends and may lack the granular sensitivity required for multi-scale local patterns.

C-T-Mamba departs from these approaches by introducing a hierarchical feature enrichment pipeline. Unlike TimeMachine [23] which focuses on multi-scale Mamba integration, our framework employs a Temporal Convolutional Block (TCB) acting as a local feature refiner prior to global reasoning. This allows C-T-Mamba to decouple local fluctuation extraction from global dependency modeling, addressing the ’local blindness’ inherent in pure SSM architectures. By synergistically combining channel attention with TCB, we ensure that the subsequent Mamba block operates on filtered, high-fidelity temporal semantics.

2.2.4. Mamba Applications in Other Areas

In the Mamba-in-Mamba [24] study, a novel and pioneering approach to HSI classification utilizing the newly developed Mamba architecture is introduced. Specifically, a centralized Mamba-Cross-Scan (MCS) for multidirectional feature aggregation is implemented, which, in conjunction with the Mamba sequence model, results in a more lightweight and streamlined framework for visual cognition. To further enhance this architecture, a T-Mamba encoder is designed that combines bidirectional Gaussian Decay Mask (GDM) and Semantic Token Learner (STL) to optimize feature quality and support downsampling-driven semantic markup representation. Concurrently, a multi-hole Semantic Token Fuser (STF) efficiently fuses the learned abstract representation with the original sequence features. Furthermore, a Weighted MCS Fusion (WMF) module and a multi-scale loss function are integrated to optimize the decoding efficiency.

U-Mamba [25] has developed a hybrid convolutional neural network (CNN) and self-supervised learning (SSL) module that integrates the local feature extraction capabilities of convolutional layers with the long-range dependency modeling capabilities of SSL. Furthermore, U-Mamba has a self-configuration mechanism that enables automatic adaptation to diverse datasets without human intervention. Extensive experiments were conducted on four distinct tasks, including 3D abdominal organ segmentation in CT and MR images, instrumentation segmentation in endoscopic images, and cell segmentation in microscopic images. The results demonstrate that U-Mamba outperforms leading CNN- and Transformer-based segmentation networks in all tasks.

3. Preliminaries

3.1. State Space Model

The typical approach to modeling the state of a system over time is through the use of differential equations. The objective is to identify a function that can describe the state of the system at any given time step, given the initial state of the system. This is achieved through the use of state spaces. The state space model permits the mapping of the input signal x of t to the output signal y of t via the state representation h of t.

As shown below,

$$h^{\prime }\left(t\right)=Ah\left(t\right)+Bx\left(t\right)$$

(1)

$$y\left(t\right)=Ch\left(t\right)$$

(2)

$h^{\prime }\left(t\right)$ means the derivative of h with respect to time, $h\left(t\right)$ describes the state of the system at each time step t, $y\left(t\right)$ is the output of the system.

As previously stated, the state space model is linear and time-invariant. This is due to the fact that the parameter matrices A, B, and C remain constant over time. As illustrated, these matrices are not dependent on the time at which they are evaluated. Each time step is consistently fixed.

In order to ascertain the output y of the model at time t, it is necessary to determine the solution $h\left(t\right)$ to the differential Equation (2). This entails identifying the $h\left(t\right)$ function that describes the state of the system for all time steps. However, solving this differential equation can be challenging, particularly given that continuous signals are typically not amenable to analysis. The input X is typically not continuous due to the inherent discrete nature of computers and digital devices. One approach to address this is to discretize the system, enabling the computation of an approximate solution to $h\left(t\right)$ in a discrete manner. This section will illustrate how the system can be discretized to compute the system output.

3.2. Discretization

The conversion of a state space model from a continuous process to a discrete process is a common undertaking in the fields of control theory and signal processing. This is due to the fact that digital computers and microcontrollers are typically only capable of handling discrete-time signals. In order to implement Kalman filters on digital devices, it is necessary to discretize the state space model in continuous time. The continuous parameters A and B, as defined in Equations (1) and (2), are discretized using the zero-order hold (ZOH) rule in the state-space model

$$\overline{A}=exp(△A)$$

(3)

$$\overline{B}={(▵A)}^{-1}\left(exp(▵A-I)\right)·▵B$$

(4)

The discrete effect is shown in Figure 2:

Figure 2. Discretization of the state space model.

3.3. Skip Connection

$$h^{\prime }\left(t\right)=Ah\left(t\right)+Bx\left(t\right)$$

(5)

$$y\left(t\right)=Ch\left(t\right)+Dx\left(t\right)$$

(6)

To illustrate, consider the scenario where we have an input X, which represents the input to the system. Let us assume that we have this X, which is sent to a black box, which we will refer to as the state-space model. This model will continuously loop through the state computation and will produce the output Y. However, we can see that the $Dx$ of t essentially means that we The input is accepted and then jumped through the state-space model, where it is multiplied by a number D. This number is then sent directly to the output, negating the need to model D in order to model the state space. This is because D does not depend on the state system of the state space at any time step. Consequently, it can be represented as a skip connection [26]. Figure 3 provides a straightforward graphical illustration of the aforementioned text, which delineates the skip connections. Figure 4 presents a comprehensive account of the skip connections within the context of the state space model.

Figure 3. Skip connections.

Figure 4. The detailed process of Skip connections for the state space model.

4. Methodology

The structure of C-T-Mamba is depicted in Figure 5, composed of four primary layers. Beginning with the initial layer, a linear labeling layer encodes and labels time series inputs, embedding them into a specified space. The second layer features a single Mamba block designed to capture intricate temporal dependencies within the variables. Following this, the third layer incorporates a channel attention mechanism to compress temporal features, emphasizing critical aspects. The fourth layer integrates a temporal convolutional block comprising depth-separable convolutions and an MLP layer. Here, the depth-separable convolution captures global receptive fields and local positional characteristics, while the MLP models future time series via a linear detector head. Ultimately, a projection layer maps these temporal features to generate predictions. Detailed descriptions of each component will be elaborated in the subsequent section.

Figure 5. General Architecture Diagram of the C-T-Mamba Model.

4.1. Linear Tokenization Layer

In processing the time-series data, we employed a methodology analogous to that utilized by iTransformer. The initial step is to tag the sequence data in order to guarantee the precision and standardization of the data. This step is of great consequence, as it directly affects the quality of the final output of the model. To achieve this goal, we employ a simple yet effective linear transformation layer. Here, the linear layer is used as a transformation tool to convert the raw time series data into a normalized form. This has the advantage of unifying the dimensions of the data, thus making it easier to compare and analyze data between different points in time.

$$U=Linear\left(Batch\left(U_{in}\right)\right)$$

(7)

In this context, U represents the output of the layer, $Batch$ represents the Batch Normalization (BatchNorm).

4.2. Mamba

4.2.1. Selective Processing of Information

In Equations (5) and (6), the parameters A, B, C, and D are identical for each input, and thus cannot be altered for each specific input. All different state-space model laws target reasoning for different inputs. Mamba’s solution is to account for a simple selection mechanism that allows the model to selectively process information in order to focus on or ignore particular inputs [21].

In particular, the authors of Mamba allow the B matrix, the C matrix, and $\Delta$ to be functions of the inputs, thereby enabling the model to adaptively adjust its behavior in accordance with the content of the inputs. The dimensions of the B matrix, which affects the inputs, and the C matrix, which affects the state, are modified from $(D,N)$ to $(B,L,N)$. This transformation corresponds to the batch size, sequence length, and hidden state size, respectively. Furthermore, the size of $\Delta$ has been modified from D to $(B,L,D)$, indicating that a distinct $\Delta$ is associated with each token in a batch (totaling $BxL$). Notably, the B matrix, C matrix, and $\Delta$ of each position are not identical, resulting in a unique B and C matrices for each input token. This approach effectively addresses the challenge of content awareness.

4.2.2. Parallel Scan

As mentioned earlier, since the matrices A B C are now dynamic, it is not possible to compute them using a convolutional representation (CNNs require a fixed kernel), so we have to use a cyclic representation, thus losing the parallel training capability provided by the convolution. To achieve parallelization, we investigate methodologies for computing outputs using a loop [27].

Each state, say $H_1$ times $\overline{A}$, plus the sum of the current input $x_1$ times $\overline{B}$, is called a scanning operation, and can be easily computed using a for loop, however, parallelization is not possible in this state (since each current state can only be computed if the previous state has been acquired).

The beneficial aspect of Mamba is that it facilitates eventual parallelization through the parallel scanning algorithm. This algorithm postulates that the sequence in which operations are performed is independent of the associated mathematics. Consequently, computations can be segmented and iteratively combined, as exemplified by the dynamic matrix B and the parallel scanning algorithm (see Figure 6), which collectively form the selective scanning algorithm. In the context of parallel computing, the variable t in the time complexity equation $O(n/t)$ typically represents the number of processors or computing units employed to complete the task.

Figure 6. Visualrepresentation of scanning operationl.

4.3. Channel Attention

While the Mamba block excels at temporal scanning, it often overlooks cross-variable dependencies in multivariate series. The Channel Attention module is strategically placed before the Mamba block to perform ‘feature screening,’ prioritizing informative variables and suppressing noise, which provides a more coherent input for global temporal modeling.

The channel attention [28,29] mechanism enhances the model’s performance by prioritizing the crucial channels within the features when processing time series data. Global pooling and average pooling operations effectively compress the features in the time dimension, distilling the information present across the entire time series. The two-layer convolutional network is capable of modeling the relationship between channels and capturing the interdependence between different channels. The outcomes of the average pooling and maximum pooling operations are aggregated after traversing the convolutional layers, respectively, to derive the integrated attention weights, as illustrated in the structural diagram in Figure 7. Ultimately, the weights are subjected to an element-level multiplication operation with the original input to obtain the weighted output [30]. This step effectively weights the features at each time step, thus emphasizing the crucial channel features.

Figure 7. Visualization of Parallel Scan.

4.4. Temporal Convolution Block

In the field of time series forecasting, most deep learning models rely on attention mechanisms to model global relationships. In contrast, our C-T-Mamba introduces a Temporal Convolutional Block, utilizing depthwise separable convolution [31] as its core module to extract local features while maintaining computational efficiency. Unlike standard convolutions, depthwise separable convolution decomposes the operation into depthwise and pointwise convolutions [32], significantly reducing parameter count and computational overhead (Figure 8). This efficiency is particularly advantageous for processing large-scale, long-term time series data in real-time scenarios. Furthermore, the integration of Multi-Layer Perceptrons (MLPs) [33] allows the model to capture complex nonlinear patterns. While convolutional layers focus on localized trends and periodicity, the fully connected structure of the MLP facilitates the learning of global features and the capture of inherent long-range dependencies.

Figure 8. Structural diagram of channel attention.

The choice of Depthwise Separable Convolution (DWConv) within the TCB is motivated by its ability to disentangle spatial (channel) and temporal correlations. Unlike standard TCNs, DWConv significantly reduces the parameter count while maintaining the receptive field size, preventing overfitting in complex multivariate scenarios. This architectural choice ensures that the local pattern extraction does not compromise the linear complexity of the overall Mamba-based framework.

4.5. Design Rationale

The synergistic integration of Channel Attention, TCB, and the Mamba block provides a transparent hierarchical feature extraction pipeline. First, the Channel Attention module acts as a “feature screener”, prioritizing informative variables while suppressing cross-channel noise. Subsequently, the TCB—utilizing multi-scale depthwise separable convolutions—extracts granular local temporal semantics that are often overlooked by pure State Space Models. Finally, the Mamba block performs long-range dependency modeling on these refined features.

Crucially, this non-parametric collaborative design significantly mitigates the risk of model misspecification. As motivated by the Hassani $-1/2$ theorem in Section 1, traditional parametric approaches are often vulnerable to rigid distributional assumptions [18]. By combining dynamic scanning with local convolutional refinement, C-T-Mamba avoids such restrictive assumptions, ensuring robust modeling of complex, non-stationary temporal dynamics while maintaining linear computational efficiency.

5. Experiments

5.1. Multivariate Long-Term Forecasting

Datset: The datasets are as follows: The present study employs five widely utilized multivariate time series datasets for the purpose of forecasting. The detailed statistics of the dataset sizes are provided in the following section. (1) ETTm(1,2) contains time series of non-stationary factors related to oil volume and electrical loads, as observed in electric transformers from July 2016 to July 2018. (2) Traffic is a collection of 48 months (2015–2016) of hourly data from the California Department of Transportation. The data set describes the occupancy of roadways (measured on a scale of 0 to 1) as observed by sensors deployed on freeways in the San Francisco Bay Area. (3) Exchange. The Exchange dataset compiles daily exchange rates for eight countries from 1990 to 2016. (4) The ECL dataset contains the hourly electricity consumption of 321 customers from 2012 to 2014.

We selected these five datasets (ETT, Traffic, Exchange, and ECL) because they represent diverse real-world scenarios with distinct temporal characteristics: ETT reflects highly periodic electrical load patterns; Traffic and ECL capture human activity dynamics; while Exchange represents financial fluctuations with relatively weak seasonality. This diversity enables a comprehensive evaluation of the model’s generalization capability across both stationary and non-stationary time series. Table 1 describes in detail the number of variables, sampling frequency, and total observations for these data sets.

Table 1. Statistical information on the five data sets.

Datasets	ETTm1	ETTm2	Traffic	Exchange	ECL
Variable Number	7	7	862	8	321
Sampling Frequency	15 min	15 min	1 h	1 Day	1 h
Total Observations	69,680	69,680	17,544	7588	26,304

All datasets are split into training, validation, and test sets with a ratio of 7:2:1. To prevent data leakage, we utilized a chronological split without shuffling, ensuring the model is always tested on “future” data relative to the training set.

Baselines: A comparative analysis of the models was conducted, in which our model was evaluated in conjunction with five representative and state-of-the-art (SOTA) predictive models. These included:Transformer-based methods, namely iTransformer, PatchTST, and Autoformer;Linear-based methods, specifically TiDE; and Time-convolutional network-based methods. A concise overview of these models is provided below.

• The iTransformer [34] reverses the responsibilities of the attention mechanism and the feedforward network. Specifically, the time points of individual series are embedded in variable tokens, which are utilized by the attention mechanism to capture multivariate correlations. Concurrently, a feedforward network is applied to each variable token to learn the nonlinear representation.
• PatchTST [35] presents an efficient design of a transformer-based time series prediction task model, introducing two key components: patch and channel-independent structures. It captures local semantic information and benefits from a longer backtracking window than previous work. The model has been shown to outperform other baselines in supervised learning, and also demonstrates its power in self-supervised representation learning and migration learning.
• The Autoformer [10] is a decomposition architecture that progressively aggregates the long-term trend component from intermediate forecasts by embedding series decomposition blocks as internal operators. Furthermore, an efficient autocorrelation mechanism is designed to perform dependency discovery and information aggregation at the series level, which contrasts with the previous auto-attention family. The autoformer naturally achieves $O\left(LlogL\right)$ complexity and produces consistent state-of-the-art performance across a wide range of realistic datasets.
• TiDE [16] presents a simple MLP-based encoder-decoder model that exhibits performance comparable to or superior to existing neural network baselines on widely utilized long-term prediction benchmarks. Concurrently, TiDE operates at a processing speed that is 5–10 times faster than the most efficient Transformer-based baselines. The study posits that, at least with respect to these long-term prediction benchmarks, the identification of periodicity and trend patterns may not necessitate self-referential processes.
• The TimesNet [15] model serves as a foundational framework for time series analysis, driven by multi-periodicity. Its modular architecture enables the elucidation of complex temporal variations, while parameter-efficient starting blocks facilitate the capture of intra-periodic and inter-periodic variations in two-dimensional space. The versatility and performance of TimesNet have been demonstrated in five mainstream analysis tasks through experimental investigation.

Parameter setting: The following section outlines the parameter settings utilized in the experiment. All experiments were conducted on a NVIDIA RTX 3090 (24 GB) (NVIDIA, Santa Clara, CA, USA), with a single Mamba block and a single temporal convolution block. The initial learning rate was adjusted to $\left\{0.005,0.0005,0.001\right\}$. The early stopping strategy was set to three, the batch size was set to either 16 or 32, the number of train epochs was set to 10, and the number of neurons in the fully connected layer of the model was adjusted to 128, 256, or 512.

Metric: We use two evaluation metrics, including

$$MSE={\displaystyle \frac{1}{n}}\sum _{i=1}^n{(y-\widehat{y})}^2$$

(8)

$$MAE={\displaystyle \frac{1}{n}}\sum _{i=1}^n\left|y-\widehat{y}\right|$$

(9)

5.2. Overall Performance

Table 2 depicts the experimental outcomes of the six models on five datasets. The optimal results for MSE are indicated in black bold, while the optimal results for MAE are indicated in red bold. As evidenced by the data in the table, the last row of our model exhibits the The highest number of wins indicates that the generalization ability of our model is superior to that of the other five models. Our model outperforms the best PatchTST based on the Transformer architecture and PatchTST, as well as TimesNet based on CNN architecture. This demonstrates that our model’s performance is superior.

Table 2. Multivariate forecasting results with prediction lengths $S\in \{96,192,336,720\}$. We fix the lookback length T = 96. All the results are averaged from all prediction lengths. The smaller the MSE, MAE, the more accurate the prediction results. Detailed results with standard deviations are provided in Appendix A.2.

Model		C-T-Mamba		iTransformer		PatchTST		Autoformer		TiDE		TimesNet
Metric		MSE	MAE	MSE	MAE	MSE	MAE	MSE	MAE	MSE	MAE	MSE	MAE
ETTm1	96	0.329	0.364	0.334	0.368	0.329	0.367	0.505	0.475	0.364	0.387	0.338	0.375
	192	0.375	0.391	0.377	0.391	0.367	0.385	0.553	0.496	0.398	0.404	0.374	0.387
	336	0.404	0.410	0.426	0.420	0.399	0.410	0.621	0.537	0.428	0.425	0.410	0.411
	720	0.472	0.448	0.491	0.459	0.454	0.439	0.671	0.561	0.487	0.461	0.478	0.450
ETTm2	96	0.178	0.261	0.180	0.264	0.175	0.259	0.255	0.339	0.207	0.305	0.187	0.267
	192	0.247	0.307	0.250	0.309	0.241	0.302	0.281	0.340	0.290	0.364	0.249	0.309
	336	0.310	0.347	0.311	0.348	0.305	0.343	0.339	0.372	0.377	0.422	0.321	0.351
	720	0.411	0.403	0.412	0.407	0.402	0.400	0.433	0.432	0.558	0.524	0.408	0.403
traffic	96	0.390	0.262	0.395	0.268	0.462	0.295	0.613	0.388	0.805	0.493	0.593	0.321
	192	0.410	0.279	0.417	0.276	0.466	0.296	0.616	0.382	0.756	0.474	0.617	0.336
	336	0.428	0.279	0.433	0.283	0.482	0.304	0.622	0.337	0.762	0.477	0.629	0.336
	720	0.462	0.298	0.467	0.302	0.514	0.322	0.660	0.408	0.719	0.449	0.640	0.350
Exchange	96	0.086	0.207	0.085	0.206	0.088	0.205	0.197	0.323	0.094	0.218	0.107	0.234
	192	0.178	0.300	0.177	0.299	0.176	0.299	0.300	0.369	0.184	0.307	0.226	0.344
	336	0.330	0.416	0.331	0.417	0.301	0.397	0.509	0.524	0.349	0.431	0.367	0.448
	720	0.862	0.701	0.847	0.691	0.901	0.714	1.447	0.941	0.852	0.698	0.964	0.746
ECL	96	0.140	0.235	0.148	0.240	0.181	0.270	0.201	0.317	0.237	0.329	0.168	0.272
	192	0.159	0.253	0.162	0.253	0.188	0.274	0.222	0.334	0.236	0.330	0.184	0.289
	336	0.176	0.272	0.178	0.269	0.204	0.293	0.231	0.338	0.249	0.344	0.198	0.300
	720	0.205	0.297	0.225	0.317	0.246	0.324	0.254	0.361	0.284	0.373	0.220	0.320
count		21		2		19		0		0		0

The MSE and MAE of iTransformer on ETTm1 and ETTm2 on these two datasets demonstrate superior performance within certain prediction windows (e.g., 192 and 336), yet in general, they are not as effective. These results are comparable to those of C-T-Mamba and PatchTST, which may be attributed to the use of linear layers for data embedding in iTransformer. While this approach is relatively straightforward, it may not be sufficient for capturing the temporal features present in complex datasets. Indeed, the ETT dataset is more complex than the other three datasets, as it combines short-term cyclical patterns, long-term cyclical patterns, long-term trends, and numerous irregular patterns. This aligns with the findings of our analysis. iTransformer The full-attention mechanism of iTransformer is capable of capturing long- and short-term dependencies. However, its computational complexity is considerable, and it may encounter performance bottlenecks when dealing with large-scale data.

The proposed model demonstrates inferior performance compared to PatchTST in the two datasets, ETTm1 and ETTm2. In fact, the proposed model exhibits only marginally superior performance, likely due to the reduced number of variables, the intricate temporal features, and the fusion of multiple temporal features in the dataset. Additionally, the channel attention mechanism is unable to effectively extract temporal dependencies from the complex features, or the extracted temporal dependencies are insufficient, resulting in the observed ineffectiveness.

Furthermore, to assess the predictive capacity of C-T-Mamba in a more intuitive manner, we present a visual comparison of the prediction outcomes of C-T-Mamba and the remaining five benchmarks on our ECL dataset, as illustrated in Figure 9. Specifically, a variable was randomly selected and its backward sequence was input. The true sequence is represented by the blue line, and the model’s prediction is represented by the red line in the following Figure 9. It is evident that the prediction results of C-T-Mamba are in close proximity to the actual values and superior to the other five baseline models on the ECL dataset.

Figure 9. Visualization of 96 outcomes on ECL dataset predicted by All models. (a) C-T-Mamba; (b) iTranformer; (c) PatchTST; (d) Autoformer; (e) TiDE; (f) TimesNet.

5.3. Ablation Experiment

To assess the effectiveness of the constituent elements of C-T-Mamba, we conducted an ablation study, whereby channel attention and temporal convolution block were removed. C-Mamba comprises solely a Mamba block and a channel attention component, while T-Mamba incorporates only a Mamba block and a temporal convolution block component. In contrast, Mamba is devoid of both channel attention and temporal convolution block, relying solely on a unidirectional Mamba block.The detailed ablation experimental results on the ETTm1, ETTm2, and ECL datasets are shown in Table 3.

Table 3. Complete results of ETTm1, ETTm2, and ECL ablation studies. We fixed the backtracking window $L=96$ and made predictions for $T=\{96,192,336,720\}$. Bolded fonts with colors indicate the best results.

Model		C-Mamba		T-Mamba		Mamba		C-T-Mamba
Metric		MSE	MAE	MSE	MAE	MSE	MAE	MSE	MAE
ETTm1	96	0.328	0.363	0.330	0.366	0.327	0.363	0.329	0.364
	192	0.374	0.389	0.378	0.393	0.374	0.390	0.375	0.391
	336	0.407	0.412	0.409	0.414	0.405	0.410	0.404	0.410
	720	0.471	0.448	0.475	0.451	0.471	0.448	0.472	0.448
ETTm2	96	0.178	0.262	0.181	0.263	0.179	0.261	0.178	0.261
	192	0.249	0.309	0.249	0.310	0.252	0.311	0.247	0.307
	336	0.312	0.349	0.312	0.349	0.312	0.349	0.310	0.347
	720	0.410	0.403	0.410	0.405	0.408	0.403	0.411	0.403
ECL	96	0.144	0.239	0.142	0.237	0.144	0.240	0.140	0.235
	192	0.160	0.254	0.159	0.254	0.162	0.256	0.159	0.253
	336	0.176	0.271	0.177	0.273	0.177	0.271	0.176	0.272
	720	0.207	0.301	0.219	0.307	0.208	0.300	0.205	0.297
count		9		1		10		17

5.3.1. Look-Back Window

To investigate whether model performance scales consistently with the expansion of the look-back window, we conducted experiments on the Traffic and ECL datasets. By fixing the label and prediction lengths while incrementally increasing the input sequence length ($L_{seq}$), we tested the hypothesis that a broader receptive field enhances the capture of long-range temporal dependencies.The experimental results, illustrated in the accompanying Figure 10, reveal a counterintuitive trend: the predictive accuracy of the Informer, Reformer, and Transformer models fails to improve—and in some cases degrades—as the retrospective window extends. This phenomenon may be attributed to the inherent limitations of the self-attention mechanism; specifically, as the sequence length increases, the attention maps often become increasingly sparse or “diluted”. This causes the model to over-index on local fluctuations while losing the ability to extract coherent global patterns.In contrast, our proposed model maintains superior performance across varying window lengths (see Figure 10). By integrating depthwise separable convolutions with Multilayer Perceptrons (MLPs) within the convolutional block, our architecture effectively balances the extraction of fine-grained local features with the integration of global temporal contexts. This structural synergy appears to be a primary factor in overcoming the sequence-length bottleneck observed in pure attention-based architectures.

Figure 10. Prediction performance for both Traffic and ECL datasets with backtracking length L 96, 192, 336, 720 and prediction length T = 96 for both datasets.

5.3.2. Operational Efficiency Analysis

A comprehensive comparison of GPU memory utilization and inference runtime across temporal feature extraction components provides clear quantitative evidence of our model’s computational advantages. As illustrated in Figure 11, both the MambaBlock and its auxiliary modules—Channel Attention and the Temporal Convolution Block—demonstrate near-linear scaling behavior with respect to sequence length. This trend contrasts sharply with the AutoCorrelation and FullAttention mechanisms, whose resource demands increase superlinearly.

Figure 11. Efficiency analysis of various components for extracting temporal features.

Empirically, GPU memory usage for MambaBlock grows modestly from 0.04 GB at L = 48 to 0.88 GB at L = 1536, whereas AutoCorrelation rises to 1.24 GB under the same conditions, implying an approximate 29% reduction in peak memory cost. Similarly, inference latency increases linearly from 0.52 ms to 8.9 ms for MambaBlock, compared to 12.1 ms for AutoCorrelation, confirming a lower empirical complexity than the theoretical $O(LlogL)$ bound associated with correlation-based attention.

Figure 11 further indicates that Channel Attention achieves the best memory–latency trade-off, reflecting its lightweight gating mechanism that amplifies channel selectivity with minimal overhead. In contrast, the FullAttention and ProbAttention variants exhibit substantially higher runtime slopes, consistent with quadratic dependency on sequence length.

These findings jointly demonstrate that our proposed C–T–Mamba architecture achieves superior operational efficiency, balancing long-range temporal dependency modeling with reduced computational cost. The empirical runtime slope closely approximates linear growth, validating the theoretical design goal of sub–$O(LlogL)$ complexity. Consequently, C–T–Mamba presents a scalable and resource-efficient framework for high-dimensional time series forecasting tasks.

6. Conclusions

In this paper, we introduce C-T-Mamba, a novel long-term time series forecasting model built upon the Mamba framework. C-T-Mamba leverages a hybrid architecture incorporating channel mixing and temporal convolutional blocks. This innovative design enables the model to discern extended temporal trends while mitigating redundancy, thereby effectively extracting pivotal local and global insights to enhance forecasting accuracy. Validation of our experimental findings across five publicly available datasets underscores the model’s robust forecasting performance and computational efficiency, rendering it a promising solution across diverse forecasting domains. The exploration of channel independence and mixing in temporal series represents a burgeoning area of interest among researchers. Our utilization of channel mixing for temporal feature extraction in this study sets a foundation for future investigations into the efficacy of channel independence in time series forecasting, alongside further exploration of the Mamba model’s potential in this domain.

Discussions and Limitations

Although C-T-Mamba achieves superior efficiency and accuracy, it has limitations. First, while TCB captures local semantics, the optimal kernel size may vary across different data frequencies. Second, our study primarily focuses on five public benchmarks; its robustness in extremely noisy or irregular sampling scenarios (e.g., medical streaming data) requires further validation. In future work, we aim to explore adaptive kernel mechanisms and extend the framework to real-time online forecasting tasks.

Appendix A.1. Hyperparameters

Data Preprocessing and Splitting. To ensure a rigorous and fair evaluation, all datasets used in our experiments were processed following the standard protocol in long-term time series forecasting (LTSF). Specifically:

Random Seeds: To ensure the reliability and reproducibility of our results, all experiments were conducted using 5 independent runs with different random seeds: $\{2021,2022,2023,2024,2025\}$.The final performance metrics (MSE and MAE) are reported as the average values across these runs.

Data Splitting: For the ETT datasets (ETTm1, ETTm2), we followed the official split (12:4:4 months for training, validation, and testing). For other custom datasets (Electricity, Traffic, Weather), we adopted a chronological split ratio of 7:2:1, as implemented in our class.

Normalization: We applied Z-score standardization to each variable independently. The mean and standard deviation were computed solely from the training set to prevent any data leakage from the validation or testing phases.

Model Architecture and Hyperparameters. C-T-Mamba was implemented using PyTorch 2.0. The core architecture comprises a Mamba backbone, a Channel Attention module, and a Temporal Convolutional Block (TCB). The hyperparameter configurations are detailed below:

Sequence Lengths: For all experiments, the lookback window size L was fixed at 96, while the prediction horizons T were set to $\{96,192,336,720\}$. Module Specifics:

Mamba Block: The state dimension $d_{state}$ was dynamically adjusted based on dataset complexity. As shown in our training scripts, we utilized $d_{state}=2$ for smaller datasets (ETT, Weather) to prevent overfitting, and $d_{state}=16$ for high-dimensional datasets (Electricity, Traffic) to capture intricate cross-dimension dependencies.

TCB: The Temporal Convolutional Block utilized depthwise separable convolutions with a fixed kernel size of 3, stride of 1, and ELU activation.

Optimization Strategy: We used the Adam optimizer with a batch size of 16 or 32. The learning rate was tuned via grid search within the range $[3\times {10}^{-5},2\times {10}^{-3}]$. Specifically: For ETTm1/m2, a lower learning rate of $5\times {10}^{-5}$ was used.For Traffic/Electricity, the learning rate was set between $5\times {10}^{-4}$ and $1\times {10}^{-3}$ to accommodate the larger feature space.

Training Details: All models were trained for 5 to 10 epochs (e.g., 5 epochs for Electricity and Traffic) with an early stopping mechanism (patience of 3) to ensure convergence.

Hardware and Software Environment. All experiments were conducted on a workstation equipped with NVIDIA RTX 3090 (24GB VRAM) GPUs. The software environment included Ubuntu 20.04, Python 3.8, and CUDA 11.8. To ensure the reliability of our results, all metrics reported (MSE and MAE) are averaged over 5 independent runs with different random seeds, as detailed in the statistical analysis in Appendix A.2.

Appendix A.2. Statistical Stability Analysis

Table A1. Detailed Forecasting Results (Mean ± Std) for C-T-Mamba.

Dataset		ETTm1	ETTm2	Traffic	Exchange	ECL
Horizon
96	MSE	$0.329\pm 0.003$	$0.178\pm 0.001$	$0.390\pm 0.004$	$0.086\pm 0.002$	$0.140\pm 0.002$
	MAE	$0.364\pm 0.002$	$0.261\pm 0.001$	$0.262\pm 0.002$	$0.207\pm 0.002$	$0.235\pm 0.002$
192	MSE	$0.375\pm 0.003$	$0.247\pm 0.002$	$0.410\pm 0.004$	$0.178\pm 0.004$	$0.159\pm 0.002$
	MAE	$0.391\pm 0.002$	$0.307\pm 0.002$	$0.279\pm 0.002$	$0.300\pm 0.003$	$0.253\pm 0.001$
336	MSE	$0.404\pm 0.004$	$0.310\pm 0.004$	$0.428\pm 0.005$	$0.330\pm 0.006$	$0.176\pm 0.003$
	MAE	$0.410\pm 0.003$	$0.347\pm 0.002$	$0.279\pm 0.003$	$0.416\pm 0.005$	$0.272\pm 0.002$
720	MSE	$0.472\pm 0.006$	$0.411\pm 0.005$	$0.462\pm 0.007$	$0.862\pm 0.012$	$0.205\pm 0.004$
	MAE	$0.448\pm 0.004$	$0.403\pm 0.004$	$0.298\pm 0.004$	$0.701\pm 0.008$	$0.297\pm 0.003$

Table A1. Detailed Forecasting Results (Mean ± Std) for C-T-Mamba.

Dataset		ETTm1	ETTm2	Traffic	Exchange	ECL
Horizon
96	MSE	$0.329\pm 0.003$	$0.178\pm 0.001$	$0.390\pm 0.004$	$0.086\pm 0.002$	$0.140\pm 0.002$
	MAE	$0.364\pm 0.002$	$0.261\pm 0.001$	$0.262\pm 0.002$	$0.207\pm 0.002$	$0.235\pm 0.002$
192	MSE	$0.375\pm 0.003$	$0.247\pm 0.002$	$0.410\pm 0.004$	$0.178\pm 0.004$	$0.159\pm 0.002$
	MAE	$0.391\pm 0.002$	$0.307\pm 0.002$	$0.279\pm 0.002$	$0.300\pm 0.003$	$0.253\pm 0.001$
336	MSE	$0.404\pm 0.004$	$0.310\pm 0.004$	$0.428\pm 0.005$	$0.330\pm 0.006$	$0.176\pm 0.003$
	MAE	$0.410\pm 0.003$	$0.347\pm 0.002$	$0.279\pm 0.003$	$0.416\pm 0.005$	$0.272\pm 0.002$
720	MSE	$0.472\pm 0.006$	$0.411\pm 0.005$	$0.462\pm 0.007$	$0.862\pm 0.012$	$0.205\pm 0.004$
	MAE	$0.448\pm 0.004$	$0.403\pm 0.004$	$0.298\pm 0.004$	$0.701\pm 0.008$	$0.297\pm 0.003$