STAE-BiSSSM: A Traffic Flow Forecasting Model with High Parameter Effectiveness

Abstract

Traffic flow forecasting plays a significant role in intelligent transportation systems (ITSs) and is instructive for traffic planning, management and control. Increasingly complex traffic conditions pose further challenges to the traffic flow forecasting. While improving the accuracy of model forecasting, the parameter effectiveness of the model is also an issue that cannot be ignored. In addition, existing traffic prediction models have failed to organically integrate data with well-designed model architectures. Therefore, to address the above two issues, we propose the STAE-BiSSSM model as a solution. STAE-BiSSSM consists of Spatio-Temporal Adaptive Embedding (STAE) and Bidirectional Selective State Space Model (BiSSSM), where STAE aims to process features to obtain richer spatio-temporal feature representations. BiSSSM is a novel structural design serving as an alternative to Transformer, capable of extracting patterns of traffic flow changes from both the forward and backward directions of time series with much fewer parameters. Comparative tests between baseline models and STAE-BiSSSM on five real-world datasets illustrates the advance performance of STAE-BiSSSM. This is especially so on METRLA and PeMSBAY datasets, compared with the SOTA model STAEformer. In the short-term forecasting task (horizon: 15 min), MAE, RMSE and MAPE of STAE-BiSSSM decrease by 1.89%/13.74%, 3.72%/16.19% and 1.46%/17.39%, respectively. In the long-term forecasting task (horizon: 60 min), MAE, RMSE and MAPE of STAE-BiSSSM decrease by 3.59%/13.83%, 7.26%/16.36% and 2.16%/15.65%, respectively.

1. Introduction

With the continuous development of urbanization, the number of vehicles has increased gradually [1], which has brought great pressure on the transportation system. The reduced efficiency of the transportation system directly leads to a series of phenomena, such as increased cost of transportation travel, frequent traffic congestion and a significant increase in the rate of traffic accidents and indirectly leads to a series of problems, such as energy waste, environmental pollution and economic losses. Geographic information science (GIS)-enabled intelligent transportation systems (ITSs), integrated with efficient and intelligent road health monitoring functions [2], are emerging as an effective solution to improve the traffic environment [3]. ITS can anticipate future traffic states by collecting and processing diverse historical data through smart infrastructure and advanced algorithms [4]. As a vital metric to measure traffic states, traffic flow reflects the traffic pressure on roads. Advanced path planning and traffic intervention based on traffic flow forecasting are efficient solutions to alleviate traffic congestion. Therefore, our proposed model STAE-BiSSSM is of practical significance.

In general, traffic forecasting models can be divided into three main categories including parametric techniques based models, machine learning techniques-based models and deep learning techniques-based models [5]. The representative models of parametric techniques based models include integrated autoregressive moving average (ARIMA)-based models [6,7,8] and kalman filters-based models [9,10,11], etc. The representative models of machine learning techniques-based models include support vector machine (SVM)-based models [12,13,14] and k-nearest neighbor (KNN)-based models [15], etc. However, parametric techniques-based models require prior knowledge and the construction of mathematical models in advance. Machine learning-based models lack the ability to process large-scale data. In recent years, the application of deep neural network in traffic forecasting has become a hot topic. This is because deep neural networks are characterized by high accuracy, strong nonlinear relationship processing ability, strong generalization ability, strong ability to deal with large-scale data, strong real-time forecasting and response and strong adaptability [16]. These deep learning models contain nonlinear autoregression with external input neural network (NARX), deep belief networks (DBNs), recurrent neural networks (RNNs), convolution neural networks (CNNs), graph neural networks (GNNs) and attention mechanisms [17]. Ref. [18] is a success application case for NARX for traffic flow forecasting combining data denoising. Ref. [19] is a representative example for DBN applied in traffic flow forecasting, which learns the features with limited prior knowledge. Subsequently, DBN have been overtaken by newer, more modern architectures, such as RNNs and CNNs. RNNs were first used to extract temporal features in traffic flow time series. Jordan network, a kind of RNN, was applied in [20,21]. The success of LSTM in NLP extends its application in traffic forecasting areas. For example, refs. [22,23,24,25] applied the LSTM and GRU for short-term traffic flow forecasting. Since traffic flow also have spatial properties, CNNs can be used to model the spatial relationship between roads. For example, models such as refs. [26,27,28,29,30,31,32,33] are structured by LSTM and CNN; GNN’s efficient processing of graph data makes it naturally suited to handle traffic flow forecasting tasks. For example, STGCN [34] is structured by CNN and GCN; models such as ref. [35,36,37,38,39], DCRNN [40] and TGCN [41] are characterized by novel graph convolution. Models such as AGCRN [42], GWNet [43], GTS [44], MTGNN [45] and Refs. [46,47] are able to learn graph structures to model more accurate, fine-grained spatial relationships between roads. Attention mechanism-based models become popular for their outstanding performance. For example, GMAN [48], PDFormer [49], AutoFormer [50], TrafficFormer [51] and MVSTT [52] are the representative models achieving the advanced performance. While more sophisticated and improved models are being designed to improve forecasting performance, attention is returning to the traffic data itself. For example, Shao et al. proposed STID [53] model which uses the one-hot encoding to integrate the spatial and temporal feature information into input data, thus enhances sample distinguishability. Liu et al proposed STAEformer [54] model which introduces a spatial–temporal adaptive embedding that enhances the capabilities of vanilla transformers for traffic flow forecasting.

Existing traffic flow forecasting models have not yet effectively addressed two key issues.

1.

Currently, mainstream traffic forecasting models are Transformer-based models. Quadratic computational complexity of attention mechanism results in a large number of parameters for this class of models, leading to high hardware requirements and high training cost, making model difficult deployment. Therefore, finding models with fewer parameters and excellent performance to replace Transformer has become a trend, such as RetNet [55], RWKV [56] and Mamba [57].

2.

There exists a disconnect in the optimization dimensions of current research efforts—some studies focus solely on improving the model’s network structure to enhance its fitting capability, while others concentrate exclusively on mining the features of traffic flow data. These two aspects fail to achieve organic integration, making it difficult to fully exert the synergistic effect of “structure adapting to data and data feeding back to structure”.

To address above issues, we propose the STAE-BiSSSM model, combining STAE and BiSSSM. BiSSSM is a newly designed model, and its core advantage lies in its ability to effectively capture the bidirectional dependencies of traffic flow from both the forward and backward direction of time series, thereby fully learning and modeling the dynamic evolution patterns of traffic flow. In addition, STAE enriches the representation of traffic flow feature, which significantly enhances the learning outcomes of BiSSSM. Compared with mainstream Transformer-based models, STAE-BiSSSM not only maintains performance advantages but also significantly reduces the number of model parameters, thus featuring higher efficiency. The main contributions of this paper are summarized as follows:

• We designed a novel bidirectional selective state space model (BiSSSM) which is able to capture the bidirectional dependency of the traffic flow from both forward and backward direction of time series and better learn the dynamic change patterns of traffic flow.
• We combined spatio-temporal adaptive embedding (STAE) and BiSSSM as a new model, STAE-BiSSSM, where STAE enriches the representation of traffic flow feature, thus enhancing the learning outcomes of BiSSSM.
• STAE-BiSSSM shows the outstanding long- and short-term forecasting ability across five authoritative real-world datasets. In particular, on datasets METRLA and PeMSBAY, STAE-BiSSSM outperforms the SOTA model STAEformer.
• Compared with STAEformer, STAE-BiSSSM has higher parameter effectiveness. While possessing superior performance, the number of trainable parameters of STAE-BiSSSM is much less than STAEformer.

The rest of this paper is organized as follows. Section 2 introduces the prerequisite knowledge and details of the STAE-BiSSSM model. In Section 3, we verify the effectiveness of the proposed model through a series of experiments. Finally, we conclude this paper in Section 4.

2. Methodology

2.1. Problem Definition

In this paper, we use the STAE-BiSSSM model to predict traffic flow over a certain time horizon based on historical traffic flow. Given traffic flow time series with l steps, we use $X_{t-l+1:t}$ to infer the future $l^{\prime }$ steps by training model $F(·)$ with parameters $\theta$, which can be formulated as:

$$\left[X_{t+1},X_{t+2},\dots ,X_{t+l^{\prime }}\right]=F\left(\left[X_{t-l+1},X_{t-l+2},\dots ,X_t\right];\theta \right)$$

(1)

where traffic flow step is $X_i\in R^{n\times d_{raw}}$ and n is the number of spatial nodes. $d_{raw}$ is the dimension of input feature.

2.2. Embedding Layer

The embedding layer contains three types of embedding which are feature embedding, periodicity embedding, and spatio-temporal adaptive embedding. In order to further enrich the feature representation of traffic flow while preserving the native information of the data. We utilize the fully connected layer to obtain feature embedding $E_{fea}\in R^{l\times n\times d_{fea}}$:

$$E_{fea}=FC\left(X_{t-l+1:t}\right)$$

(2)

where $d_{fea}$ is the dimension of feature embedding and $FC(·)$ indicates a fully connected layer. Since traffic is time-varying, labeling timestamps is critical. We denote the learnable timestamp-of-day embedding dict as $Dic{t}_{tod}\in R^{n_{tod}\times d_{tod}}$, and the learnable day-of-week embedding dict as $Dic{t}_{dow}\in R^{7\times d_{dow}}$, where $n_{tod}$ is the number of timestamps in one day and is set as 288 in our case, $d_{tod}$ is the dimension of timestamp-of-day embedding and $d_{dow}$ is the dimension of day-of-week embedding. The timestamp-of-day embedding of timestamp i in one day is $E_{tod}^i\in R^{d_{tod}\times 1}$, and the day-of-week embedding of timestamp i in one week is $E_{dow}^i\in R^{d_{dow}\times 1}$. By concatenating and broadcasting them, we get the periodicity embedding $E_{per}\in R^{l\times n\times (d_{tow}+d_{dow})}$ for the traffic flow time series. Furthermore, periodicity is not the only factor affecting temporal relationship. From a localized perspective, temporal relationship is decided by chronological order in the traffic time series. From a global perspective, temporal relationships between different roads may be affected by spatial arrangement and the time series from different roads tend to have different temporal patterns. In order to dispense with the high computational costs pre-defined or dynamic adjacency matrix and better model local and global spatio-temporal relationships, we use a spatio-temporal adaptive embedding $E_{adp}\in R^{l\times n\times d_{adp}}$ to capture the complicated spatio-temporal relationship efficiently, where $d_{adp}$ is the dimension of spatio-temporal adaptive embedding. In particular, $E_{adp}$ is shared across different traffic time series. By concatenating the above three embeddings, we obtain the final spatio-temporal representation of feature $F_{in}\in R^{l\times n\times d_{in}}$ as follows:

$$F_{in}=E_{fea}\left|\right|E_{per}\left|\right|E_{adp}$$

(3)

where $d_{in}$ is the dimension for STAE-BiSSSM inputs which equals $d_{fea}+(d_{tow}+d_{dow})+d_{adp}$. The structure of the embedding layer is shown as Figure 1b.

2.3. Selective State Space Model (SSSM)

The state space model(SSM) is the basic structure of Selective State Space Model (SSSM) which can construct long distance time dependence. The inference process of SSM [58] for time series modeling is shown as Algorithm 1:

Algorithm 1 Inference Process of SSM for time series modeling.

Inputs $u:[b,l,d_{in}]$ Parameter Matrices Initialization:      $A,B,C:[d_{in},d_{hid}]$,   $\Delta :\left[d_{in}\right]$,   $h\left(0\right):[b,d_{in},d_{hid}]$ start:      Discretization:         $\overline{A}=exp(einsum(“d_{in},d_{in}{d}_{hid}\to d_{in}{d}_{hid}”,\Delta ,A)$         $\overline{B}=einsum(“d_{in},d_{in}{d}_{hid}\to d_{in}{d}_{hid}”,\Delta ,B)$         $\overline{B}u=einsum(“d_{in}{d}_{hid},bl{d}_{in}\to bl{d}_{in}{d}_{hid}”,\overline{B},u)$         $\overline{A},\overline{B}:[d_{in},d_{hid}]$   $\overline{B}u:[b,l,d_{in},d_{hid}]$      Hidden State Iteration:         for $k=0:l-1$ do:               $\overline{A}h\left(k\right)=einsum(“d_{in}{d}_{hid},b{d}_{in}{d}_{hid}\to b{d}_{in}{d}_{hid}”,\overline{A},h\left(k\right))$               $h(k+1)=\overline{A}h\left(k\right)+\overline{B}u\left(k\right)$         $h=\left[h\right(1),h(2),...,h(l\left)\right]$         $y=einsum(‘‘d_{in}{d}_{hid},bl{d}_{in}{d}_{hid}\to bl{d}_{in}’’,C,h)$         $h\left(k\right),\overline{A}h\left(k\right),\overline{B}u\left(k\right):[b,d_{in},d_{hid}]$,   $h:[b,l,d_{in},d_{hid}]$ end: Outputs $y:[b,l,d_{in}]$

Where $A,\overline{A}$ are the system matrix and discrete system matrix; $B,\overline{B}$ are the input matrix and discrete input matrix; C is the output matirx; $\Delta$ is the discrete parameter and $h\left(k\right)$ is the hidden state. b is the batch size; l is the length of time series; $d_{in}$ is the feature dimension of one time step in time series and $d_{hid}$ is the dimension of hidden state. $einsum(·)$ indicates Einstein summation convention.

However, the parameter matrices of the conventional SSM is static, which means parameter matrices are not capable of adjusting to the inputs. In order to make the parameter matrix sensitive to the inputs for the purpose of selective retention of critical information, SSSM introduces the selective mechanism to the SSM to build relationship between parameter matrices and inputs. This relationship is defined as a linear projection from inputs u to $\Delta$, B and C which can be formulated as follows:

$$\begin{array}{cc}\hfill {\delta }_{b\times l\times d_{mid}}& =Linea{r}_{d_{in}\to d_{mid}}\left(u\right)=u_{b\times l\times d_{in}}{W}_{d_{in}\times d_{mid}}+bia{s}_{b\times l\times d_{mid}}\hfill \end{array}$$

(4)

$$\begin{array}{cc}\hfill {\Delta }_{b\times l\times d_{in}}& =Softplus\left(Linea{r}_{d_{mid}\to d_{in}}\left(\delta \right)\right)=Softplus({\delta }_{b\times l\times d_{mid}}{W}_{d_{mid}\times d_{in}}+bia{s}_{b\times l\times d_{in}})\hfill \end{array}$$

(5)

$$\begin{array}{cc}\hfill B_{b\times l\times d_{hid}}& =Linea{r}_{d_{in}\to d_{hid}}\left(u\right)=u_{b\times l\times d_{in}}{W}_{d_{in}\times d_{hid}}+bia{s}_{b\times l\times d_{hid}}\hfill \end{array}$$

(6)

$$\begin{array}{cc}\hfill C_{b\times l\times d_{hid}}& =Linea{r}_{d_{in}\to d_{hid}}\left(u\right)=u_{b\times l\times d_{in}}{W}_{d_{in}\times d_{hid}}+bia{s}_{b\times l\times d_{hid}}\hfill \end{array}$$

(7)

where $\delta$ is the middle rank state and $d_{mid}<d_{in}/2$ is the dimension of $\delta$. $Softplus(·)$ indicates the activation function $f\left(x\right)=log(1+e^x)$. The introduction of $\delta$ aims to decrease the parameters number of projection and prevents model overfitting. The inference process of Selective SSM [57] for time series modeling is shown as Algorithm 2:

Algorithm 2 Inference Process of Selective SSM (SSSM) for time series modeling.

Inputs $u:[b,l,d_{in}]$ Parameter Matrices Initialization:      $A:[d_{in},d_{hid}]$,   $h\left(0\right):[b,d_{in},d_{hid}]$ start:      Linear Projection:         $\delta =Linea{r}_{d_{in}\to d_{mid}}\left(u\right)$,      $\delta :[b,l,d_{mid}]$         $\Delta =Linea{r}_{d_{mid}\to d_{in}}\left(\delta \right)$,      $\Delta :[b,l,d_{in}]$         $B=Linea{r}_{d_{in}\to d_{hid}}\left(u\right)$,      $B:[b,l,d_{hid}]$         $C=Linea{r}_{d_{in}\to d_{hid}}\left(u\right)$,      $C:[b,l,d_{hid}]$      Discretization:         $\overline{A}=exp\left(einsum(‘‘bl{d}_{in},d_{in}{d}_{hid}\to bl{d}_{in}{d}_{hid}’’,\Delta ,A)\right)$         $\overline{B}=einsum(“bl{d}_{in},bl{d}_{hid}\to bl{d}_{in}{d}_{hid}”,\Delta ,B)$         $\overline{B}u=einsum(“bl{d}_{in}{d}_{hid},bl{d}_{in}\to bl{d}_{in}{d}_{hid}”,\overline{B},u)$         $\overline{A},\overline{B},\overline{B}u:[b,l,d_{in},d_{hid}]$      Hidden State Iteration:         for $k=0:l-1$ do:               $\overline{A}\left(k\right)h\left(k\right)=einsum(“b{d}_{in}{d}_{hid},b{d}_{in}{d}_{hid}\to b{d}_{in}{d}_{hid}”,\overline{A}\left(k\right),h\left(k\right))$               $h(k+1)=\overline{A}\left(k\right)h\left(k\right)+\overline{B}u\left(k\right)$         $h=\left[h\right(1),h(2),...,h(l\left)\right]$         $y\left(k\right)=einsum(“bl{d}_{hid},bl{d}_{in}{d}_{hid}\to bl{d}_{in}”,C,h)$         $h\left(k\right),\overline{A}\left(k\right),\overline{A}\left(k\right)h\left(k\right),\overline{B}u\left(k\right):[b,d_{in},d_{hid}]$,   $h:[b,l,d_{in},d_{hid}]$ end Outputs $y:[b,l,d_{in}]$

Where the “for” loop can be replaced by a parallel scanning algorithm that accelerates the inference speed of SSSM. The structure of SSSM is shown as Figure 1d.

2.4. STAE-Bidirectional SSSM (STAE-BiSSSM)

Traffic flow has bidirectional dependence, which means that the current traffic flow state in time series is both influenced by the historical flow states and implicitly constrained by potential future flow states. For example, The forward dependence of traffic flow is reflected in the fact that the upward trend of morning peak flow is influenced by the previous hour’s flow states, and the backward dependence of traffic flow is reflected in the fact that the congestion dissipation process in the evening peak is related to the flow states in the next 30 min. Therefore, we propose a STAE-BiSSSM for traffic flow forecasting, where STAE is used for enriching the representation of traffic flow feature, contributing to the model learning; BiSSSM, including forward SSSM and backward SSSM, is used for extracting bidirectional dependence in traffic flow series, contributing to increase the forecasting accuracy of the model. The inference process of STAE-BiSSSM is shown in Algorithm 3. An overall structure of STAE-BiSSSM is shown as Figure 1a and the structure of BiSSSM is shown as Figure 1c.

Algorithm 3 Inference Process of STAE-Bidirectional SSSM (STAE-BiSSSM) for traffic flow forecasting (Ours).

Inputs $u:[b,l,n,d_{raw}]$ start:      $u^{\prime }=transpose(u,shape=(b,n,l,d_{raw}))$      $u^{\prime \prime }=reshape(u^{\prime },shape=(b\ast n,l,d_{raw}))$      Embedding Layer:         $F_{in}=embeddin{g}_{d_{raw}\to d_{in}}\left(u^{\prime \prime }\right)$,   $F_{in}:[b\ast n,l,d_{in}]$      BiSSSM:         Convolution Layer:             $Con{v}_{out}=conv1D(F_{in};d_{conv})$,   $Con{v}_{out}:[b\ast n,l,d_{in}]$             $Con{v^{\prime }}_{out}=silu\left(Con{v}_{out}\right)$,   $Con{v^{\prime }}_{out}:[b\ast n,l,d_{in}]$         Position Switching:             $BSSS{M}_{in}=positionswitchin{g}_{i\iff l-1-i}\left(Con{v^{\prime }}_{out}\right)$,   $BSSS{M}_{in}:[b\ast n,l,d_{in}]$         Forward SSSM (FSSSM):             $FSSS{M}_{out}=FSSSM\left(Con{v^{\prime }}_{out}\right)$,   $FSSS{M}_{out}:[b\ast n,l,d_{in}]$         Backward SSSM (BSSSM):             $BSSS{M}_{out}=BSSSM\left(BSSS{M}_{in}\right)$,   $BSSS{M}_{out}:[b\ast n,l,d_{in}]$         Concatenation:             $BiSSS{M}_{out}=FSSS{M}_{out}\left|\right|BSSS{M}_{out}$,   $BiSSS{M}_{out}:[b\ast n,l,2d_{in}]$         Reduced Dimensional Projection:             $BiSSS{M^{\prime }}_{out}=Linea{r}_{2d_{in}\to d_{in}}\left(BiSSS{M}_{out}\right)$,   $BiSSS{M^{\prime }}_{out}:[b\ast n,l,d_{in}]$      Residual Connection:         $BiSSS{M^{″}}_{out}=BiSSS{M^{\prime }}_{out}\ast silu\left(F_{in}\right)+F_{in}$,   $BiSSS{M^{″}}_{out}:[b\ast n,l,d_{in}]$      Regression Layer:         $Re{g}_{in}=RMSnorm\left(BiSSS{M^{\prime }}_{out}\right)$,   $Re{g}_{in}:[b\ast n,l,d_{in}]$         $Re{g^{\prime }}_{in}=reshape(Re{g}_{in},shape=(b\ast n,l\ast d_{in}))$         $Re{g}_{out}=Linea{r}_{l\ast d_{in}\to l^{\prime }\ast d_{out}}\left(Re{g^{\prime }}_{in}\right)$,   $Re{g}_{out}:[b\ast n,l^{\prime }\ast d_{out}]$      $y=reshape(Re{g}_{out},shape=(b,n,l^{\prime },d_{out}))$      $y^{\prime }=transpose(y,shape=(b,l^{\prime },n,d_{out}))$ end Outputs $y^{\prime }:[b,l^{\prime },n,d_{out}]$

Where $transpose(·)$ indicates the tensor transpose; $reshape(·)$ indicates the tensor reshaping; $shape(·)$ indicates the shape of target tensor; $silu(·)$ indicates the activation function $f\left(x\right)=x·{\displaystyle \frac{1}{1+e^{-x}}}$; $conv1D$ indicates one-dimensional convolution; $RMSnorm(·)$ indicates the Root Mean Square Layer Normalization [59]; $d_{conv}$ is the kernel size of one-dimensional convolution and $d_{out}$ is the output dimension of STAE-BiSSSM.

3. Experimental Study

In this section, we briefly introduce the datasets and baseline models used in the experiments in Section 3.1. Metrics for evaluating the performance of the model are given in Section 3.2. Implementation details of the experiments are introduced in Section 3.3. The results of the experiments and the real case forecasting are shown in Section 3.4. The results of the ablation study for STAE-BiSSSM are shown in Section 3.5. The parameter effectiveness analysis is included in Section 3.6.

3.1. Description of Experimental Datasets and Baseline Models

3.1.1. Experimental Datasets

In the experiments, we apply five widely used authoritative datasets including PeMSD4, PeMSD7, PeMSD8, METRLA and PeMSBAY to evaluate the forecasting performance of STAE-BiSSSM. Details of five datasets are shown in

Table 1. Details of traffic flow datasets in experiments.

Dataset	Number of Sensors	Time Steps	Sample Interval	Time Range	Location
PeMSD4	307	16,992	5 min	01/2018–02/2018	California District 4
PeMSD7	883	28,224	5 min	05/2017–08/2017	California District 7
PeMSD8	170	17,856	5 min	07/2016–08/2016	California District 8
METRLA	207	34,272	5 min	03/2012–06/2012	Los Angeles
PeMSBAY	325	52,116	5 min	01/2017–05/2017	San Francisco Bay Area

.

3.1.2. Baseline Models

In this section, we introduce the baseline models included in the comparative tests based on their characteristics.

1.

Historical Inertia (HI) [60]: HI serves as the conventional benchmark, reflecting standard industry practices.

2.

STGCN [34]: STGCN proposes a spatio-temporal convolution block (ST-Conv Block) that consists of a spatial graph convolutional layer (GCN) and a temporally gated convolutional layer (Gated CNN) to realize the joint extraction of spatio-temporal features.

3.

DCRNN [40]: DCRNN proposes diffusion convolution to model the spatial dependence of directed graphs by bidirectional stochastic wandering (forward diffusion and backward diffusion).

4.

AGCRN [42]: AGCRN dynamically generates the adjacency matrix through learnable node embedding.

5.

GWNet [43]: GWNet introduces an adaptive graph modeling approach to discover the implicit path dependence in traffic flow, and dilated causal convolution (DCC) is used to extract the long time series dependency.

6.

GTS [44]: GTS develops a forecasting approach for multiple interrelated time series, learning a graph structure in conjunction with a GNN, which addresses the shortcomings of earlier methods.

7.

MTGNN [45]: MTGNN generates an asymmetric neighborhood matrix by learning the node embedding matrix to capture unidirectional causality in multidimensional time series.

8.

STNorm [61]: STNorm achieves multimodal feature fusion through temporal embedding, spatial embedding and feature embedding, which enhances model learning.

9.

GMAN [48]: GMAN enhances the model’s ability to resolve complex spatio-temporal patterns by parallelizing multiple attention modules (spatial attention, temporal attention and transformational attention) to capture different dimensional dependencies in traffic data separately.

10.

PDFormer [49]: PDFormer introduces a Delayed Feature Transformation (DFT) module based on attention mechanism. The DFT Module can better model information delay in real traffic situations, thus improving the forecasting accuracy of the model.

11.

STID [53]: STID proposes a spatio-temporal unique heat coding method, which enriches the feature representation and makes features discriminative. Thanks to spatio-temporal unique heat coding, it greatly improves the performance of fully connected networks on traffic flow forecasting.

12.

STAEformer [54]: STAEformer proposes a spatio-temporal adaptive embedding to enhance feature representation and distinguishability and thus makes the vanilla Transformer SOTA for traffic flow forecasting.

3.2. Metrics of Model Evaluation

To evaluate the predictive performance of the model, we use the following three metrics to assess the difference between the real traffic flow of the road V and the predicted traffic flow of the road $\widehat{V}$:

1.

Mean Average Error (MAE):

$$MAE={\displaystyle \frac{1}{n}}\sum _{i=1}^n\left|V_i-{\widehat{V}}_i\right|$$

(8)

2.

Root Mean Square Error (RMSE):

$$RMSE=\sqrt{{\displaystyle \frac{1}{n}}\sum _{i=1}^n{(V_i-{\widehat{V}}_i)}^2}$$

(9)

3.

Mean Absolute Percentage Error (MAPE):

$$MAPE={\displaystyle \frac{1}{n}}\sum _{i=1}^n\left|{\displaystyle \frac{V_i-{\widehat{V}}_i}{V_i}}\right|\times 100\%$$

(10)

MAE, RMSE and MAPE are the metrics used to measure the forecasting error; the smaller the value of three metrics, the better the forecasting performance of the model.

3.3. Experiments Implementation

We implement the model with the PyTorch toolkit on a Linux server with one Nvidia A100 GPU. For the datasets settings, PeMSD4, PeMSD7 and PeMSD8 are divided into training, validation and test sets in a ratio of 6:2:2. The time steps of METRLA and PeMSBAY datasets are much longer than PeMSD4, PeMSD7 and PeMSD8 datasets, which means that more changes in traffic flow were contained in METR-LA and PeMS-BAY datasets. In order to ensure sufficient samples for training the models to fully learn the dynamic patterns of traffic flow data and sufficient data to test the performance of the model, METRLA and PeMSBAY are divided in a ratio of 7:1:2 instead of 6:2:2. Before feeding the training data in training sets into the baseline models and STAE-BiSSSM for training, we performed standard normalization on the training data. The expression of standard normalization is shown as follows:

$$z={\displaystyle \frac{x-\mu }{\sigma }}$$

(11)

where x is the training data, and $\mu$ and $\sigma$ are the mean value of the data and the standard deviation of the data, respectively.

Model parameters are crucial to the model performance. A description and details of model parameters are shown in

Table 2. Description and details of model parameters.

Hyperparameter	Description	Dimension Size
$d_{fea}$	Embedding dimension of feature embedding	24
$d_{tod}$	Embedding dimension of time of day embedding	24
$d_{dow}$	Embedding dimension of day-of-week embedding	24
$d_{adp}$	Embedding dimension of spatial–temporal adaptive embedding	80
$d_{conv}$	Size of convolution kernel	5
$d_{hid}$	Dimension of hidden state	64
$d_{mid}$	Dimension of middle-rank state	16

. For PeMSD4, PeMSD7 and PeMSD8 Datasets, we set the input length and output length to be 1 h $l=l^{\prime }=12$. For METRLA and PeMSBAY datasets, we set the input length to be 1 h $l=12$ and output length to be 15 min, 30 min and 60 min $l^{\prime }=3,6,12$, respectively. The batch size is 16, and Adam is chosen as the optimizer with the learning rate decaying from 0.001. An early-stop mechanism is applied if the validation error converges within 20 continuous steps for PeMSD4, PeMSD7 and PeMSBAY, and 30 continuous steps for PeMSD8 and METRLA. For the settings of loss function in the experiments, we apply the Huber loss as the loss function for PeMSD4, PeMSD7 and PeMSD8 datasets, which is shown as follows:

$$L_{\delta }\left(y,\widehat{y}\right)=\left\{\begin{array}{cc}{\displaystyle \frac{1}{2}}{\left(y-\widehat{y}\right)}^2& |y-\widehat{y}|\le \delta \\ \delta |y-\widehat{y}|-{\displaystyle \frac{1}{2}}{\delta }^2& |y-\widehat{y}|>\delta \end{array}\right.$$

(12)

where $\delta$ is the hyperparameter which determines the emphasis of Huber loss on MSE and MAE. In our case, $\delta$ is set to be 1. For the PeMSD4, PeMSD7 and PeMSD8 datasets with a large amount of noise, Huber loss is more robust for the outlier values while ensuring relatively fast convergence. For METRLA and PeMSBAY datasets, we use the masked MAE as the loss function.

3.4. Experiment Results

In order to illustrate the advancement of STAE-BiSSSM, we conducted full, persuasive comparative experiments. Section 3.4.1 illustrates the advantages of STAE-BiSSSM compared with other baseline models according to the metrics. Section 3.4.2 shows the impressive performance of STAE-BiSSSM through real case forecasting.

3.4.1. Metrics Analysis

Table 3. Predictive metrics of STAE-BiSSSM and other baseline models on PeMSD4, PeMSD7 and PeMSD8 datasets.

Dataset		PeMSD4			PeMSD7			PeMSD8
	Metric	MAE	RMSE	MAPE	MAE	RMSE	MAPE	MAE	RMSE	MAPE
Model
HI		42.35	61.66	29.92%	49.03	71.18	22.75%	36.66	50.45	21.63%
GWNet		18.53	29.92	12.89%	20.47	33.47	8.61%	14.40	23.39	9.21%
DCRNN		19.63	31.26	13.59%	21.16	34.14	9.02%	15.22	24.17	10.21%
AGCRN		19.38	31.25	13.40%	20.57	34.40	8.74%	15.32	24.41	10.03%
STGCN		19.57	31.38	13.44%	21.74	35.27	9.24%	16.08	25.39	10.60%
GTS		20.96	32.95	14.66%	22.15	35.10	9.38%	16.49	26.08	10.54%
MTGNN		19.17	31.70	13.37%	20.89	34.06	9.00%	15.18	24.24	10.20%
STNorm		18.96	30.98	12.69%	20.50	34.66	8.75%	15.41	24.77	9.76%
GMAN		19.14	31.60	13.19%	20.97	34.10	9.05%	15.31	24.92	10.13%
STAE-BiSSSM (Ours)		18.46	30.16	12.24%	19.91	33.16	8.43%	13.84	23.23	9.24%

shows results of comparative tests between HI, STGCN, DCRNN, AGCRN, GWNet, GTS, MTGNN, STNorm, GMAN and STAE-BiSSSM on test sets of PeMSD4, PeMSD7 and PeMSD8 datasets. Results of comparative tests between the above baseline models, SOTA models (PDFormer, STID and STAEformer) and STAE-BiSSSM on test sets of METRLA and PeMSBAY datasets are shown in

Table 4. Predictive metrics of STAE-BiSSSM and other baseline models on METRLA.

Dataset		METRLA
Horizon		15 min			30 min			60 min
	Metric	MAE	RMSE	MAPE	MAE	RMSE	MAPE	MAE	RMSE	MAPE
Model
HI		6.80	14.21	16.72%	6.80	14.21	16.72%	6.80	14.20	10.15%
GWNet		2.69	5.15	6.99%	3.08	6.20	8.47%	3.51	7.28	9.96%
DCRNN		2.67	5.16	6.86%	3.12	6.27	8.42%	3.54	7.47	10.32%
AGCRN		2.85	5.53	7.63%	3.20	6.52	9.00%	3.59	7.45	10.47%
STGCN		2.75	5.29	7.10%	3.15	6.35	8.62%	3.60	7.43	10.35%
GTS		2.75	5.27	7.12%	3.14	6.33	8.62%	3.59	7.44	10.25%
MTGNN		2.69	5.16	6.89%	3.05	6.13	8.16%	3.47	7.21	9.70%
STNorm		2.81	5.57	7.40%	3.18	6.59	8.47%	3.57	7.51	10.24%
GMAN		2.80	5.55	7.41%	3.12	6.49	8.73%	3.44	7.35	10.07%
PDFormer		2.83	5.45	7.77%	3.20	6.46	9.19%	3.62	7.47	10.91%
STID		2.82	5.53	7.75%	3.19	6.57	9.39%	3.55	7.55	10.95%
STAEformer		2.65	5.11	6.85%	2.97	6.00	8.13%	3.34	7.02	9.70%
STAE-BiSSSM (Ours)		2.60	4.92	6.75%	2.86	5.59	7.88%	3.22	6.51	9.49%

and

Table 5. Predictive metrics of STAE-BiSSSM and other baseline models on PeMSBAY.

Dataset		PeMSBAY
Horizon		15 min			30 min			60 min
	Metric	MAE	RMSE	MAPE	MAE	RMSE	MAPE	MAE	RMSE	MAPE
Model
HI		3.06	7.05	6.85%	3.06	7.04	6.84%	3.05	7.03	6.83%
GWNet		1.30	2.73	2.71%	1.63	3.73	3.73%	1.99	4.60	4.71%
DCRNN		1.31	2.76	2.73%	1.65	3.75	3.71%	1.97	4.60	4.68%
AGCRN		1.35	2.88	2.91%	1.67	3.82	3.81%	1.94	4.50	4.55%
STGCN		1.36	2.88	2.86%	1.70	3.84	3.79%	2.02	4.63	4.72%
GTS		1.37	2.92	2.85%	1.72	3.86	3.88%	2.06	4.60	4.88%
MTGNN		1.33	2.80	2.81%	1.66	3.77	3.75%	1.95	4.50	4.62%
STNorm		1.33	2.82	2.76%	1.65	3.77	3.66%	1.92	4.45	4.46%
GMAN		1.35	2.90	2.87%	1.65	3.82	3.74%	1.92	4.49	4.52%
PDFormer		1.32	2.83	2.78%	1.64	3.79	3.71%	1.91	4.43	4.51%
STID		1.31	2.79	2.78%	1.64	3.73	3.73%	1.91	4.42	4.55%
STAEformer		1.31	2.78	2.76%	1.62	3.68	3.62%	1.88	4.34	4.41%
STAE-BiSSSM (Ours)		1.13	2.33	2.28%	1.36	2.96	2.93%	1.62	3.63	3.72%

. In each table, the metrics with underlines are the optimal ones.

As shown in Table 3, STAE-BiSSSM achieves the best performance compared to other baseline models on PeMSD4, PeMSD7 and PeMSD8 datasets. Compared with the most advanced model GWNet, MAE and MAPE of STAE-BiSSSM decrease by 0.38% and 5.04%, respectively on PeMSD4 datasets; MAE, RMSE and MAPE decrease by 2.74%, 0.93% and 2.09% on PEMSD7 datasets and MAE and RMSE of STAE-BiSSSM decrease by 3.89% and 0.68% on PEMSD8 datasets. As shown in Table 4 and Table 5, STAE-BiSSSM achieves the best performance compared to other baseline models on METRLA and PeMSBAY datasets under different forecasting horizon. In the short-term forecasting task (horizon: 15 min), compared with the SOTA model STAEformer, MAE, RMSE and MAPE of STAE-BiSSSM decrease by 1.89%/13.74%, 3.72%/16.19% and 1.46%/17.39% on METRLA and PEMSBAY datasets. In the long-term forecasting task (horizon: 60 min), compared with the STAEformer, MAE, RMSE and MAPE of STAE-BiSSSM decrease by 3.59%/13.83%, 7.26%/16.36% and 2.16%/15.65% on METRLA and PEMSBAY datasets. In a nutshell, STAE-BiSSSM not only performs well on the short-term forecasting task, but also performs even better on the long-term forecasting task.

3.4.2. Real Case Analysis

As shown in Figure 2, the forecasting curve of the STAE-BiSSSM can perfectly fit the ground truth under long-term forecasting tasks (Horizon: 12) on PeMSD4 datasets. As shown in Figure 3, Figure 4 and Figure 5, the forecasting curves of the STAE-BiSSSM can reflect the trend of the ground truth under the long-term forecasting task (Horizon: 12) well on PeMSD7, PeMSD8 and METRLA datasets with noisy data. Figure 6 shows the forecasting curves of STAE-BiSSSM under the short-term and long-term forecasting task on PeMSBAY datasets. For short-term forecasting (Horizon: 3), the forecasting curve of STAE-BiSSSM can fit the ground truth well. For long-term forecasting (Horizon: 12), the forecasting curve of STAE-BiSSSM can also reflect the trend of the ground truth well. In conclusion, STAE-BiSSSM has the ability to perform real-world traffic flow forecasting under different forecasting tasks.

3.5. Ablation Study

In order to illustrate the validity of critical components of STAE-BiSSSM, we introduce an ablation study including two variants of our model as follows:

• FSSSM: It removes the spatio-temporal adaptive embedding $E_{adp}$ and backward SSSM in STAE-BiSSSM.
• STAE-FSSSM: It removes the backward SSSM in STAE-BiSSSM.

The results of the ablation study on PeMSD4, METRLA and PeMSBAY datasets under forecasting horizon 12 are shown in

Table 6. Ablation study on PeMSD4, METRLA and PeMSBAY datasets.

Dataset		Model	FSSSM	STAE-FSSSM	STAE-BiSSSM
	Metric
PeMSD4	MAE		21.85	18.54	18.46
	RMSE		35.11	30.20	30.16
	MAPE		14.82%	12.51%	12.24%
METRLA	MAE		3.92	3.24	3.22
	RMSE		7.84	6.53	6.51
	MAPE		11.94%	9.82%	9.49%
PeMSBAY	MAE		2.03	1.64	1.62
	RMSE		4.43	3.65	3.63
	MAPE		4.82%	3.88%	3.72%

. Compared with FSSSM, MAE, RMSE and MAPE of the STAE-FSSSM decrease by 15.15%, 13.98% and 15.59% on PeMSD4 datasets; 17.35%, 16.71% and 17.76% on METRLA datasets; and 19.21%, 17.61% and 19.50% on PeMSBAY datasets, respectively. It can be seen that $E_{adp}$ can enrich the representation of the feature, thus contributing the model learning. Compared with STAE-FSSSM, MAE, RMSE and MAPE of the STAE-BiSSSM decrease by 0.43%, 0.13% and 2.16% on PeMSD4 datasets; 0.62%, 0.31% and 3.36% on METRLA datasets; and 1.22%, 0.55% and 4.12% on PeMSBAY datasets. It can seen that bidirectional feature extraction of time series can be better mined to find out the dynamic change pattern of traffic flow. Therefore, each critical component of STAE-BiSSSM is necessary.

3.6. Parameter Effectiveness Analysis

Table 7. Total trainable parameters of STAEformer and STAE-BiSSSM on METRLA and PEMSBAY datasets under different forecasting horizon.

Datasets		Models	STAEformer	STAE-BiSSSM
	Horizon
METRLA	3 (15 min)		1,242,555	327,195
	6 (30 min)		1,248,030	332,670
	9 (60 min)		1,258,980	343,620
PeMSBAY	3 (15 min)		1,355,835	440,475
	6 (30 min)		1,361,310	445,950
	9 (60 min)		1,372,260	456,900

and Figure 7 show the total trainable parameters of STAEformer and STAE-BiSSSM on METRLA and PeMSBAY datasets under different forecasting horizon. Figure 8 and Figure 9 show a comparison of three performance metrics between STAEformer and STAE-BiSSSM under different forecasting horizon. For METRLA datasets, the number of STAE-BiSSSM trainable parameters is about 26–27% of the number of STAEformer trainable parameters. For PEMSBAY datasets, the number of STAE-BiSSSM trainable parameters is about 32–33% of the number of STAEformer trainable parameters. However, the predictive performance of the STAE-BiSSSM model outperforms STAEformer on both datasets which suggests that the parameter effectiveness of STAE-BiSSSM is superior to that of STAEformer. This provides further evidence that the BiSSSM structure is more effective than the attention mechanism, and that fewer parameters can be used to retain critical information in the time series. At the same time, a smaller number of trainable parameters means that the model is cheaper to train and less demanding on hardware, which facilitates model deployment and application.

4. Conclusions

In this paper, we focus on both new structural design and the nature of the data itself. Therefore, a novel BiSSSM is proposed based on an emerging efficient model selective SSM to extract the bidirectional dependency of traffic flow time series, and spatio-temporal adaptive embedding is used for enriching the representation of features. We combine BiSSSM and STAE to obtain STAE-BiSSM, incorporating the advantages of both for better traffic flow forecasting. Comparative tests across five authoritative real-world datasets demonstrate that the STAE-BiSSSM is capable of both long- and short-term forecasting and achieves advanced performance. An ablation study illustrates that each critical component of STAE-BiSSSM is necessary. Parameter effectiveness analysis of STAE-BiSSSM suggests that STAE-BiSSSM can also achieve excellent forecasting results with a small number of parameters. In a nutshell, STAE-BiSSSM shows a promising direction for addressing the challenges in traffic forecasting. However, one a limitation is that we have not incorporated the interactions and connections between traffic flow and other traffic states into the forecasting of traffic flow. Joint traffic flow forecasting using multiple traffic features is a potential future research direction.