What Is the Best Model for Highway Traffic Flow Prediction? A Large-Scale Test for Empirical Data

Abstract

Traffic flow prediction is an important and fundamental task for the operation of Intelligent Transportation Systems. In recent years, most studies on traffic prediction have focused on two-dimensional network traffic flow prediction, while there is still no clear consensus on the study of one-dimensional highway traffic flow prediction, for instance, regarding which model is the most appropriate. To address this gap, we conducted a systematic comparative evaluation of 27 models across five classes, including Statistical models, Machine Learning, Artificial Neural Networks, Deep Neural Networks, and Graph Neural Networks, based on five representative highway traffic datasets. To ensure fairness, evaluations were performed on raw data without signal decomposition or auxiliary modules. Surprisingly, the experimental results reveal that complex deep learning models do not demonstrate advantages in terms of conventional metrics. Instead, simple models, particularly Historical Averaging and tree-based Machine Learning models, exhibit superior performance in most scenarios. And then, we study the underlying reasons for this phenomenon from various perspectives, including the complexity of prediction tasks, the tabular data characteristics, the spectral bias of Neural Networks, and theoretical error bounds. Furthermore, we also analyze why these findings were overlooked in the previous literature, attributing the oversight to the predominant focus on signal decomposition preprocessing, inconsistent prediction settings, and the lack of comprehensive benchmarking. Supported by rich data and extensive information, this work offers valuable references and practical implications for researchers in highway traffic flow prediction. It further advocates that in the era of pursuing sophisticated models, scenario-specific analysis and appropriate simple models still deserve more attention.

1. Introduction

Traffic congestion has become a serious problem in modern cities due to the fast growth of urban areas and the increasing number of cars. This issue causes economic losses and pollution, and it also wastes people’s time. To solve this problem, Intelligent Transportation Systems (ITSs) are widely used to manage road networks better. In ITSs, traffic flow prediction is a very important part. Accurate prediction results can help traffic managers control signals and guide vehicles effectively. Therefore, how to predict traffic flow accurately has always been a key question.

Among the different types of traffic prediction, studies on network-level traffic flow using 2D data usually attract more attention from researchers, while the number of studies on highway traffic flow using 1D data is relatively small. This is because network-level traffic prediction involves higher spatiotemporal complexity and often requires sophisticated models to achieve satisfactory performance. Although the prediction for a single location may seem relatively straightforward, we argue that such neglect is inappropriate, and there remains considerable scope for further explorations. After all, highways are critical components of modern transportation networks, supporting long-distance travel and freight transportation.

Although numerous prediction models have been proposed by previous researchers, there has been no clear answer to some questions so far: what constitutes the optimal highway traffic flow prediction model? Among the many existing models with distinct characteristics, how should we make a selection? Is there a fair evaluation system that can serve as a benchmark for judgment? In fact, this aspect has rarely been addressed in previous studies, as the core topic of many recent papers is framed around the narrative of “This paper proposes a novel model that outperforms other baseline models.” However, the proposed model is often improved by some signal decomposition method, while the differences in the performance of the models are usually blurred.

Thus, the main research object of this paper is the comparison between some typical models for highway traffic flow predictions. We want to find the best models for this job by testing several datasets with various sample numbers. Based on highway traffic flow data collected across different time periods and locations, we constructed five representative highway traffic flow prediction datasets and evaluated 27 relevant models. After a large-scale test, our findings reveal a somewhat surprising result: when assessed against the three conventional metrics (MAE, MAPE, and RMSE), complex Neural Networks do not demonstrate any notable advantages. Instead, several simple models, such as classical Machine Learning models and even Historical Averaging (HA), exhibit better performance. This phenomenon is more or less unexpected, and we will elaborate on it further in subsequent sections of this paper.

The rest of this paper is organized as follows. Section 2 gives the literature review about highway traffic flow prediction methods. Section 3 introduces the five public datasets we collected. Section 4 describes the five types of prediction models we compared and the basic rules for fair evaluation. Section 5 presents experimental results on all the datasets and compares the metrics of up to 27 models. Finally, Section 6 gives further discussions about some important topics, while Section 7 concludes the paper.

2. Literature Review

2.1. Single Models

In the early stage, researchers mainly relied on Statistical models for predicting highway traffic flow. For example, ARIMA and SARIMA are the most classic ones; Hamed et al. [1] found that ARIMA models are efficient for short-term urban traffic prediction. Williams and Hoel [2] successfully applied SARIMA to traffic flow and demonstrated its superiority in capturing seasonal patterns compared to simple baselines. Williams et al. [3] also compared SARIMA with Exponential Smoothing and found that modeling seasonality is crucial for reducing prediction errors. However, Statistical models often fail to capture non-linear patterns in complex environments.

To solve this, Machine Learning (ML) models were introduced. Smith and Demetsky [4] compared four different approaches and highlighted that non-parametric regression could better handle the stochastic nature of traffic flow. Davis and Nihan [5] applied K-Nearest Neighbors (KNN) to search for similar historical traffic states, proving that simple pattern matching is robust for short-term forecasting. Similarly, Wu et al. [6] validated that Support Vector Regression (SVR) had better generalization ability than linear baselines, as it can map data into high-dimensional space to find optimal boundaries.

In recent years, tree-based ensemble methods have attracted people’s attention due to their efficiency on tabular data. Random Forest (RF) [7] constructs parallel trees to reduce prediction variance, which is proven to be very robust against noise. XGBoost [8] and LightGBM [9] use gradient boosting techniques to learn from previous mistakes step by step, significantly improving accuracy and training speed. Many comparative studies have proved their superiority in the transportation field. For instance, Lam [10] compared XGB with Long Short-Term Memory (LSTM) and found that XGB achieved not only higher accuracy but also much faster training efficiency on medium-sized datasets. Afandizadeh et al. [11] also reported that RF outperformed LSTM when using calendar-based features, suggesting that tree models are better at handling discrete tabular features.

With the development of AI, deep learning (DL) models have become mainstream. LSTM is widely used to capture temporal dependencies; Ma et al. [12] proved that LSTM could effectively overcome the forgetting problem of traditional Recurrent Neural Networks (RNNs) and capture long-term traffic trends. Convolutional Neural Networks (CNNs) are also applied to extract spatial patterns; for example, Ma et al. [13] treated traffic data as images, demonstrating that CNNs can capture local spatial correlations that 1D models ignore.

More recently, some Deep Neural Network (DNN) models like Deep Belief Networks (DBNs) [14] have been used for unsupervised feature learning, and Temporal Convolutional Networks (TCNs) [15] have been proven to handle long sequences more efficiently than RNNs. Transformers [16], with their self-attention mechanism, have achieved excellent results by capturing global dependencies. And then, several studies have focused on comparing these deep models; for example, Katambire et al. [17] compared ARIMA with LSTM and concluded that LSTM handles complex junction traffic better. Jang and Chen [18] analyzed the differences between LSTM and GRU, finding that GRU achieves similar accuracy with fewer parameters. In addition to single models, some researchers tried to improve performance by combining models or using optimization algorithms. Chahal et al. [19] combined ARIMA and LSTM to utilize the advantages of both linear and non-linear modeling. Vo et al. [20] used the Sparrow Search Algorithm to optimize a Bi-GRU network, proving that hyperparameter tuning is key to model performance. Shafiei et al. [21] also proposed a hybrid framework based on the empirical data of traffic fundamental diagrams, showing that incorporating physical knowledge can enhance deep learning predictions.

Note that network traffic flow prediction (using 2D data) is different from the highway traffic flow prediction (using 1D data) studied in this paper. For the former study, complex Graph Neural Network (GNN) models are often used, including STGCN, GWNet and AFDGCN, etc. Actually, we find these models also could be used for our task, as long as we adjust the format of the data input. We think how these models perform in highway traffic flow prediction is also worthy of analysis.

From the above review, it can be seen that different model families serve different purposes and involve different trade-offs. Statistical models are suitable when interpretability, stability, and low computational burden are emphasized. ML models are often advantageous when the prediction task can be represented effectively as a tabular regression problem with a limited data scale. DNN models are designed to learn complex temporal patterns automatically, but they usually require more training effort. GNN models are particularly meaningful when multiple detectors and explicit spatial interactions are available, whereas their applicability to single-location highway prediction is less straightforward. In a word, whether the performance of these models in highway traffic flow prediction is satisfied or not needs to be further verified.

2.2. Signal Decomposition Methods

For highway traffic flow prediction, another important research direction is about the signal decomposition method. Researchers can decompose the complex, non-linear traffic data into simpler subsequences, predict them separately, and then sum them up. In the early stages, Wavelet Transform was the standard tool. It predicts time series by decomposing the original non-stationary data into a low-frequency trend and several high-frequency details, then forecasting each component individually and combining them to get the final result. Jiang and Adeli [22] proposed a dynamic wavelet Neural Network to decompose traffic data into frequency components, which helped the model distinguish between signal and noise. Xie and Zhang [23] also found that wavelet decomposition effectively filtered out high frequency noise, making the Neural Network training more stable. Later, Seasonal-Trend Decomposition (STL) [24] was applied. Yuan et al. [25] utilized STL to separate seasonality from the trend, demonstrating that this preprocessing step significantly reduces the difficulty of predicting periodic traffic flow. Similarly, Facebook Prophet [26] performs forecasting by fitting an additive Bayesian regression model that decomposes historical data into distinct components, including a flexible non-linear trend, periodic seasonal patterns, and the impacts of holidays and special events. It independently estimates each component then sums them to generate robust predictions.

Recently, more advanced decomposition algorithms like Empirical Mode Decomposition (EMD) and Variational Mode Decomposition (VMD) have become popular. These are often combined with complex deep learning models. For EMD-based methods, Wang et al. [27] used Complete Ensemble Empirical Mode Decomposition with Adaptive Noise (CEEMDAN) combined with LSSVM, showing that adaptive decomposition reduces the non-linearity of the input data. Wu et al. [28] and Zhao et al. [29] also proposed hybrid frameworks based on CEEMDAN, validating that decomposing volatile traffic data into stable intrinsic mode functions leads to higher prediction accuracy. Li et al. [30] proposed a highly complex model named PLO-ICEEMDAN-KAN, integrating noise adaptive decomposition with the novel Kolmogorov–Arnold Networks. They claimed that this combination allows for capturing traffic details that traditional networks miss.

And for VMD-based methods, Liu et al. [31] combined VMD with Bi-LSTM and GMDH networks. They demonstrated that VMD is superior to EMD in avoiding mode mixing issues. Zhao et al. [32] proposed a VMD and IDBO-LSTM model, claiming that optimizing the parameters of VMD is crucial for extracting effective modal components. Some researchers even went further to use Secondary Decomposition. For example, Bing et al. [33] proposed a method that decomposes the data twice before feeding it into a CNN–Transformer model, arguing that a single round of decomposition is insufficient for highly complex traffic flows.

Finally, although these decomposition-based methods often report high accuracy, they rely heavily on the preprocessing strategy rather than the prediction model itself. This can easily lead to misunderstandings among researchers and make it impossible to accurately measure the effect of the model itself. Thus, in this paper, we do not consider the use of decomposition-based methods.

3. Data

In order to get convincible results, we collected five typical public datasets of traffic volume, as shown in Table 1. The criteria of chosen datasets are as follows:

Table 1. The basic information of the five datasets.

Dataset	Country	Interval (Minute)	Period of Known Data	Sequence Length of Known Data	Period of Predicted Data	Sequence Length of Predicted Data	Proportion
1	UK	15	1 January 2022–28 January 2022	2688	29 January 2022–31 January 2022	288	0.11
2	USA	60	2 October 2012–31 March 2013	4335	1 April 2013–30 April 2013	720	0.17
3	USA	5	1 April 2020–26 May 2020	8640	27 May 2020–31 May 2020	1440	0.17
4	USA	60	1 August 2016–30 June 2018	16,776	1 July 2018–30 September 2018	2208	0.13
5	UK	15	1 January 2010–31 December 2010	35,040	1 January 2011–15 March 2011	7104	0.20

(1)

The dataset is complete, and no missing data or obvious errors could be found. Synthetic data is not used in this paper.

(2)

The data was collected in different countries, different locations and different time periods, and that with different time intervals was given priority. In this way, the datasets we selected could be quite representative.

(3)

There are no data points where the traffic volume is zero. This is important for calculating the results of MAPE.

(4)

The influence of COVID-19 on travel behavior is excluded. For example, when considering the range of Dataset 3, the data in March of 2020 are not considered: in this month, the daily volume had decreased somewhat compared with February and April.

(5)

The proportions in Table 1 are calculated by the volume of predicted data divided by that of known data. It is easy to understand that when the proportion is larger, the difficulty of prediction increases and the indicators become worse. Therefore, in this paper we consider the proportions within a small range (between 0.1 and 0.2).

(6)

In some datasets, some other auxiliary information could be found, e.g., the weather condition. Since the task studied in this paper is the evaluations of typical traffic flow prediction models, this information is not considered.

For data preprocessing, Min-Max normalization is applied before training, so that the input values are scaled into the range of [0, 1]. After prediction, the normalized outputs are transformed back to the original scale by inverse normalization, and all metrics are calculated on the restored values. Next, among the known data, 70% are used for training and the remaining 30% are used for validation. All the experiments were implemented in Python 3.13 and PyTorch 2.10 and were conducted on a computing platform equipped with an NVIDIA (Santa Clara, CA, USA) GeForce RTX 4090 GPU, an and Intel (Santa Clara, CA, USA) Core i7-12700H CPU, and 32 GB RAM.

Here we show the results of predicted data in Figure 1, since we can clearly observe the characteristics of the targets to be predicted by all subsequent models, which can help to qualitatively establish a relationship between the prediction results and model performance. To get a clear presentation, for Datasets 2, 4 and 5, we only show the data of one week. Generally speaking, all the curves are qualitatively similar: the volume always reaches the peak in the day and touches the bottom in the night.

Figure 1. The predicted data used in this paper: (a) Dataset 1; (b) first week of Dataset 2; (c) Dataset 3; (d) first week of Dataset 4; (e) first week of Dataset 5.

4. Models

In this paper, we consider five classes of possible models for predictions and comparisons:

(1)

Statistical models. They are all classical models, including Historical Averaging (HA), the Exponential Smoothing Method (ESM) and SARIMA (Seasonal AutoRegressive Integrated Moving Average). Since the periodicity in the traffic flow data is clear, it is not a good choice to use the original ARIMA model for this job.

(2)

Machine Learning (ML) models, including eight typical ones: K-Nearest Neighbors (KNN), Support Vector Machine (SVM), Decision Tree (DT), Adaptive Boosting (ADB), Gradient Boosting Decision Tree (GBDT), Random Forest (RF), eXtreme Gradient Boosting (XGB) and Light Gradient Boosting (LGB). They are very popular in many different fields.

(3)

Artificial Neural Network (ANN) models: They are simple networks with fewer layers, including Multi-Layer Perceptron (MLP), Convolutional Neural Network (CNN), Gated Recurrent Unit (GRU), Long Short-Term Memory (LSTM), Bi-LSTM and CNN-LSTM models. Here, two types of CNNs are considered: one-dimensional and two-dimensional ones (CNN-1D and CNN-2D).

(4)

Deep Neural Network (DNN) models: They have been proposed by some previous researchers, and we did not adjust their original network structures. They include Deep Belief Network (DBN), Temporal Convolutional Network (TCN) and Transformer.

(5)

Graph Neural Network (GNN) models: They are usually used for more complex tasks, e.g., the two-dimensional predictions for network traffic. Nevertheless, we find they also could be used for one-dimensional predictions, as long as they are adapted using a single-node graph formulation, in which the graph is reduced to one node with a self-loop connection. In other words, inter-node spatial message passing has been removed. For example, in a typical GNN model, when the historical flow input is [B, C, N, T] and the temporal feature input is [B, T, N, D], if N = 1, the dynamically generated adjacency matrix degenerates to a 1 × 1 self-connection. In this paper, some classical and open-sourced GNN models are processed as above, including DDGCRN, STIDGCN, STGCN, ASTGCN, GWNET, MA2GCN and AFDGCN.

Notably, large language models (LLMs) are applicable to this task. In our recent experiments, we also found that they can sometimes produce satisfactory outputs for highway traffic flow prediction. However, their performance is often unstable, which arises from the high randomness inherent in LLM outputs. For instance, it may automatically select ARIMA in the first run, LSTM in the second run, and a weighted version of HA in the third run. Thus, we do not consider it in this paper.

In order to get fair results, we think something about the model evaluations should be noted:

(1)

No extra preprocessing method is used. As mentioned in Section 2.2, many recent studies concentrate on the valid preprocessing method when predicting traffic flow, especially the signal decomposition method, including STL and CEEMDAN. As shown in Section 3, the differences between weekdays and weekends could be observed in all the datasets. Thus, we believe the use of the signal decomposition method could improve the final prediction performance. However, since the main focus of this paper is the performance of the predictive model itself, for the sake of fairness, we do not consider any preprocessing step and only use the raw data in the following evaluations.

(2)

Only two input windows are considered: one day and one week. After testing the performance of various input windows, we find that the results are rather complex. It is very difficult to identify a universally optimal value, as it usually varies across different datasets and models. Thus, we decided to only use two typical input windows in this paper, since they are more interpretable: one day and one week represent natural and representative periods for traffic flow, making them more intuitive than other arbitrary window lengths.

(3)

For time series predictions, both single-step and multi-step methods are possible. When considering two inputs mentioned in (2), there are four possible combinations: single-step and one day input (SD), single-step and one week input (SW), multi-step and one day input (MD), and multi-step and one week input (MW). Here “single-step” settings refer to an iterated single-step strategy: the model is trained to predict one step ahead, and it is then recursively rolled forward to generate a long-horizon forecast. In this process, the model’s previous predictions are repeatedly fed back as inputs for subsequent steps. On the contrary, the “multi-step” settings denote two direct settings with one-day and one-week input windows, where the model outputs multiple future steps simultaneously. In short, we find these four settings affect the results differently, which will be discussed later.

(4)

Only a typical/default set of hyperparameters is used. On one hand, for some simpler models, we find the change in many hyperparameters only leads to minor differences. Here we take XGB as an example. Usually, researchers consider XGB as a simple model. However, XGB has around 30 core parameters, while the total number of configurable parameters listed in its official documentation exceeds 70. In other words, it is nearly impossible to perform a really rigorous parameter tuning for XGB. Nevertheless, we find most parameters are not influential. For example, when other parameters are fixed, the influence of two main parameters (n_estimators and colsample_bytree) in XGB could be shown in the sensitivity analysis (see Figure 2). Both of them remain nearly stable during a large interval. Thus, we think using their default values is possible for follow-up investigations, and a similar situation could also be observed in other simpler models.

Figure 2. A typical example of sensitivity analysis: influence of some parameters of XGB for Dataset 2 under SW: (a) n_estimators; (b) colsample_bytree.

On the other hand, for some more complex models (e.g., GNN), the adjustment of hyperparameters is very complex, or even not introduced in the original papers. Therefore, we could only use the default values of the original models.

5. Results

For all the prediction results shown in this paper, some precautions need to be explained:

(1)

In this paper, we consider MAPE as the main indicator. Some other typical indicators, including MAE and RMSE, represent absolute errors that are directly related to the magnitude of traffic flow. Judging from their values alone, it is impossible to determine whether the error is large or small. In contrast, MAPE is a relative error metric that clearly reflects the scale of the error, making it suitable for comparing differences across different datasets.

(2)

For a better presentation effect, we only show the figures about MAPE values in the main text. Five colors are used to indicate five different classes of models. And the details of all the metrics can be found in Appendix A.

(3)

For unsatisfied and unreasonable results, when the MAPE is even larger than 80%, we just show “Diverged” in the tables of Appendix A. In addition, all the “Diverged” results in Figure 3, Figure 4, Figure 5, Figure 6 and Figure 7 are marked in gray.

Figure 3. MAPE comparison across 27 models for Dataset 1: (a) SD; (b) SW; (c) MD; (d) MW.

Figure 4. MAPE comparison across 27 models for Dataset 2: (a) SD; (b) SW; (c) MD; (d) MW.

Figure 5. MAPE comparison across 27 models for Dataset 3: (a) SD; (b) SW; (c) MD; (d) MW.

Figure 6. MAPE comparison across 27 models for Dataset 4: (a) SD; (b) SW; (c) MD; (d) MW.

Figure 7. MAPE comparison across 27 models for Dataset 5: (a) SD; (b) SW; (c) MD; (d) MW.

(4)

For Statistical models, four methods are not available, and we can only get one result. Thus, we only present their results in the figures of SD for comparison.

(5)

For simplicity, more details about the ANN models (especially the influence of the numbers of layers) are shown in Appendix B rather than the main text.

Next, we show the evaluation results of Dataset 1 in Figure 3. Figure 3a,b correspond to SD and SW, while Figure 3c,d correspond to MD and MW. All the results are the average values after 10 runs. We can see the following:

(1)

The most prominent feature is that many Neural Network models, including ANN, DNN and GNN, do not perform well. The differences between them are also not clear. In other words, the advantages of Neural Networks cannot be revealed in a one-dimensional prediction task, while the disadvantages could be amplified, e.g., poor generalization performance.

(2)

On the contrary, the best results could be obtained by ML models, e.g., some tree-based models (RF, XGB and LGB) and KNN. Their best MAPE values can reach below 20%. At the same time, their speeds are much faster than those Neural Networks. In particular, the results of HA are satisfied and are even better than many models. While this seems like a counterintuitive conclusion, it is also understandable: for relatively simple tasks, simple models are sometimes better, as long as they can capture the repeat of historical patterns.

(3)

For time series predictions, we often think RNN models are good choices. However, the performance of RNN models in Figure 3 is not satisfied, including GRU, LSTM, bi-LSTM and CNN-LSTM. The best MAPE values of these models are all between 30 and 40%, which is worse than MLP and CNN models. In addition, for the single-step predictions (including SD and SW), the output results of many RNN models have become a straight line.

(4)

The quantitative differences between four methods are clear. For all the models, the best MAPE values do not appear in the situation under SD. But it is hard to summarize in one or two sentences which one among SW, MD and MW is the best. It looks like some good results are obtained with SW, including the best ones of tree-based models (RF, XGB and LGB) and KNN. A possible explanation is that for Dataset 1, only three days need to be predicted, which makes the multi-step predictions less prominent.

(5)

Note that the results of “Diverged” are observed in some models, which are mainly under SD and SW. In other words, no matter whether using ML or NN models, for long-horizon forecasting, multi-step predictions are usually better than single-step ones. A possible reason is that SD and SW generate long-horizon forecasts by recursively feeding previous predictions back into the model. As the rollout proceeds, small one-step errors accumulate over time and may eventually lead to severe deviations.

Next, we check the results in larger datasets. Generally speaking, we think the results in Datasets 2~5 are qualitatively similar to that in Dataset 1. For example, the performance of tree models in Machine Learning is usually better than various Neural Networks; many models face the result of “Diverged” under the condition of SD, while the advantages of MD and MW are further confirmed. Therefore, in the following discussions we only concentrate on the main differences between them.

For Dataset 2, in addition to the five models that perform well in Dataset 1, several other models also performed well, including SVM and two GNN models. This might be because the prediction difficulty of Dataset 2 is lower than that of Dataset 1. For example, the best MAPE for Dataset 1 is about 17%, while for Dataset 2 it reaches about 11% in Figure 4. In other words, it is not the case that the more time steps need to be predicted, the greater the prediction difficulty becomes.

For Dataset 3, the metrics of all the models become even better. Some models that previously performed poorly even reach the “excellent level”, including GBDT and Bi-LSTM. And many models easily reach the “good level” when their MAPE becomes lower than 20%, including SARIMA and ESM.

For Dataset 4, many results are similar to those of Dataset 2 and 3, especially the ones under MD and MW. The main differences are that (1) only the MAPE of HA and ML models are less than 10% in Figure 6 and (2) many models under SD and SW face the results of “Diverged”. This is most likely because when the prediction horizon is longer, the error accumulation of recursive single-step prediction will become more severe, and inevitably more errors will arise in the end.

It is interesting that the prediction difficulties (which could be also presented by the best MAPE values) of Datasets 2, 3, and 4 are lower than that of Dataset 1, although their characteristics are quite different. To explain them collectively, we find that on the one hand, if the data interval is large (e.g., that of Dataset 2 and 4 is 1 h), it will automatically eliminate some accidental and often hard-to-calculate irregular fluctuations, which reduces the prediction difficulty. On the other hand, if the number of time periods is small (e.g., Dataset 3 only requires predicting 5 days), the prediction difficulty is also relatively low, and this is more conducive to the application of classic Statistical models.

Finally, for Dataset 5, we are surprised to find that the performance of all models dropped off a cliff in Figure 7, which is completely different from what we observed in Dataset 4. Except HA and a GNN model (STIDGCN), all the ML models cannot reach the “good level”. As we all know, deep learning models should benefit from larger datasets. However, as time periods increase, this prediction problem becomes “long-term prediction”, and its difficulty is bound to increase, which inevitably influences the performance of most models. In addition, the time interval in Dataset 5 is 15 min, which makes the prediction task more difficult, as discussed above.

Here we also show some statistics about the stability of these metrics. We show the situations of Dataset 3 under MW as an example in Table 2. Here the standard deviations of Statistical and ML models are very small (even <0.01%), while those of Neural Networks are a little larger. In a word, except for two GNN models, the standard deviations of the remaining models do not exceed 2%, which demonstrates the stability of the MAPE values.

Table 2. A typical example of the standard deviations of MAPE on Dataset 3 under MW.

Model	Average	Standard Deviation	Model	Average	Standard Deviation
HA	9.68%	<0.01%	Bi-LSTM	10.91%	1.50%
SARIMA	13.38%	<0.01%	CNN	11.29%	0.77%
ESM	17.57%	<0.01%	CNN-LSTM	13.93%	1.88%
KNN	10.56%	<0.01%	DBN	10.12%	0.32%
SVM	22.38%	<0.01%	TCN	13.22%	0.65%
ADB	12.07%	<0.01%	Transformer	10.64%	0.60%
DT	11.71%	<0.01%	STIDGCN	10.08%	1.17%
GBDT	9.67%	<0.01%	DDGCRN	10.65%	0.34%
RF	9.15%	<0.01%	STGCN	12.99%	0.68%
XGB	9.81%	<0.01%	ASTGCN	15.38%	1.28%
LGB	9.35%	<0.01%	GWNET	13.47%	0.56%
MLP	12.54%	0.75%	MA2GCN	30.22%	2.34%
GRU	11.97%	1.05%	AFDGCN	13.41%	2.26%
LSTM	14.85%	1.53%

At the end of this section, we discuss the details of the “Diverged” results in Figure 8. Generally speaking, there are two typical situations for these unsatisfied outcomes. Since all the results in our figures are the average values after 10 runs, we can make sure that for the specific model and conditions, such a situation occurs stably in nearly every run and is not a random occurrence:

Figure 8. Two typical situations for the results of “Diverged”. (a) DBN on Dataset 2 under SD. (b) LSTM on Dataset 5 under SW.

(1)

As shown in Figure 8a, sometimes the predicted results become nearly “a straight line”. This phenomenon mainly appears in the SD and SW settings, where a one-step-ahead model is recursively used to generate forecasts. As the recursive process continues, the model repeatedly takes its own previous predictions as inputs for the following steps, so small one-step errors accumulate over time. As a result, the model gradually loses its ability to reproduce large fluctuations. To reduce the overall error, the predicted values eventually become overly smooth and approach a straight line.

(2)

As shown in Figure 8b, even though the model can catch the general pattern of peaks and bottoms, the built-up errors over time cause a severe “timing shift”. For instance, a predicted traffic peak might be pushed to a time when there is actually a bottom. This total mismatch between peaks and bottoms makes error metrics like MAE and MAPE skyrocket.

In addition, we also add some typical results of the sensitivity analysis about the input window lengths in Figure 9. Firstly, we consider simpler models on small datasets (Dataset 1). The first situation can be seen in Figure 9a, when the choice of input window length has no significant impact on MAPE. And the second situation is shown in Figure 9b, when the consideration of one week is much better than that of one day, i.e., the model can capture the weekly periodic pattern of highway traffic flow. Next, for more complex models on large datasets (Dataset 4), the results shown in Figure 9c,d exhibit similar trends.

Figure 9. The sensitivity analysis about the input window lengths. (a) GRU for Dataset 1 and multi-step prediction; (b) LGB for Dataset 1 and single-step prediction; (c) STIDGCN for Dataset 4 and single-step prediction; (d) DDGCRN for Dataset 4 and multi-step prediction.

Note that although these results are obtained from different models and different prediction methods, they share a clear common feature: when the input window length exceeds one week, the performance cannot be further improved, which is mainly attributed to the intrinsic characteristics of highway traffic flow data. In other words, the optimal value selected from the one-day and one-week prediction results is (or very close to) the best value across all input window lengths, which is suitable for comparison and ranking.

6. Discussion

In this section, we will discuss some important topics.

(1)

The overall evaluation results of all the models.

In Table 3, we evaluate all the models by calculating the sum of their performance across the five datasets. The criteria for evaluating these models on certain datasets are based on the best MAPE among the four methods:

Table 3. The overall evaluation results of all the models.

Rank	Type	Model	D1	D2	D3	D4	D5	Overall
1	Statistical	HA	2	2	1	1	2	8
2	ML	RF	2	2	1	1	3	9
3	ML	XGB	2	2	1	1	3	9
4	ML	LGB	2	2	1	1	3	9
5	ML	KNN	2	2	2	1	3	10
6	GNN	STIDGCN	3	2	1	2	2	10
7	ML	SVM	3	2	2	1	4	12
8	ML	GBDT	3	3	1	2	3	12
9	ML	DT	3	3	2	2	3	13
10	ANN	MLP	3	3	2	2	3	13
11	ANN	Bi-LSTM	3	3	1	2	4	13
12	ANN	CNN	3	3	2	2	3	13
13	ANN	CNN-LSTM	3	3	2	2	3	13
14	DNN	Transformer	3	3	2	2	3	13
15	GNN	STGCN	3	2	2	2	4	13
16	Statistical	SARIMA	3	3	2	3	3	14
17	ANN	GRU	3	3	2	2	4	14
18	ANN	LSTM	3	3	2	2	4	14
19	DNN	DBN	3	3	2	2	4	14
20	DNN	TCN	3	3	2	2	4	14
21	GNN	GWNET	3	3	2	2	4	14
22	GNN	AFDGCN	3	3	2	2	4	14
23	ML	ADB	3	3	2	3	4	15
24	GNN	ASTGCN	3	3	2	3	4	15
25	GNN	DDGCRN	4	4	2	2	4	16
26	GNN	MA2GCN	4	3	3	3	4	17
27	Statistical	ESM	4	4	2	4	4	18

Excellent (value = 1), when it is smaller than 10%.

Good (value = 2), when it is between 10% and 20%.

Average (value = 3), when it is between 20% and 50%.

Deficient (value = 4), when it is larger than 50%.

In other words, the smaller the number here, the better the performance of the model. Therefore, the main conclusion of this paper is somewhat counterintuitive: for current datasets, simpler models (HA and some tree-based ML models) perform better for highway traffic flow predictions. Although it may seem somewhat counterintuitive, given that these datasets exhibit distinct characteristics with significant differences in collection time, environment, sampling interval, and the lengths used for analysis and prediction, the fact that such conclusions hold across all datasets warrants in-depth consideration.

(2)

Why do the HA and ML models perform better, whereas the DNN and ANN models show inferior performance?

After conducting multi-party research, we think the possible main reasons are as follows:

▪

The complexity of prediction tasks. Manibardo et al. [34] mentioned that for certain simple tasks, such as single road prediction, the input data contains limited information. Therefore, there is no fundamental need for deep learning to extract complex features. On the contrary, shallow models are sufficient to capture all effective information, resulting in the advantages of relatively simple methods like ML models.

▪

The nature of data structure. Grinsztajn et al. [35] mentioned that tree-based models outperform deep learning on medium-sized tabular data. This is because tabular data contains many uninformative features and possesses non-rotationally invariant properties. Neural Networks, which usually have rotation invariance, are suitable for isotropic data like images but struggle with tabular data. Additionally, the target functions of tabular data are usually irregular. On the contrary, tree models can easily fit this irregularity through piece-wise constant functions, whereas Neural Networks struggle due to their bias toward smooth solutions. Thus, since highway traffic flow data also belongs to tabular data, the results are understandable.

▪

The theoretical bound of datasets. The study of Li et al. [36] shows that for any given dataset, there exists a theoretical error bound. It is determined by the inherent characteristics of the data (i.e., entropy) rather than any model. Therefore, when the performance of the model already approaches this entropy level, any performance improvement of Model A over Model B may merely reflect random fluctuations in results. At this point, it is of no practical significance to adopt a model with a more complex structure.

▪

The spectral bias of deep learning. Rahaman et al. [37] proposed via Fourier analysis that Neural Networks may possess a “spectral bias,” prioritizing low-frequency components while learning high-frequency components much slower. Traffic flow data typically contains non-smooth, high-frequency abrupt changes (e.g., congestions during daytime). Due to the spectral bias, deep learning models tend to output overly smooth predictions, failing to capture extreme points.

▪

The difficulty of parameter optimization. The nature of the last reason may be different from that of the previous ones. As we mentioned in Section 4, we only use the default parameters for all the DNN and GNN models. Theoretically speaking, it is possible that the performance of these models could be improved if we can find the best parameters. However, such a process is very difficult for ordinary users, since most papers about these models do not specifically describe their hyperparameter settings and parameter tuning processes. Thus, we can only use the currently obtained results for evaluations, and we also acknowledge that the conclusions of this paper may be incomplete.

(3)

Why are the findings in this paper not revealed in the previous related studies?

For the task of time series prediction, there are too many papers with various model structures. But most of them focus on various fields, e.g., weather conditions, electric load, the stock market, etc. In this part, “the previous related studies” correspond to the papers only concentrating on traffic flow datasets. We find the possible explanations are as follows:

▪

The wide use of signal decomposition methods. It is easy to understand that feature engineering is very important for a better model output when predicting traffic flow. As discussed in Section 2, many recent papers about such a prediction concentrate on the use and improvement of signal decomposition methods [24,25,27,28,29,30,31,32,33]. From the perspective of the research itself, this is entirely correct. However, this approach causes most researchers to overlook the performance of the prediction models themselves and fail to carefully consider which model is more suitable for such tasks.

▪

The confusion of prediction conditions. As we discussed in Section 5, different prediction conditions (SD/SW/MD/MW) will lead to completely different results. However, these conditions are often mixed. As mentioned by Elsayed et al. [38], some papers force baselines to use iterated multi-step forecasting, which in our setting corresponds to the recursive single-step schemes SD/SW and inevitably suffers from error accumulation. At the same time, the proposed models often utilize direct multi-step forecasting, which in our setting corresponds to MD/MW. Such an unfair comparison is not always easy for readers to recognize.

▪

The insufficiently systematic investigation. Actually, in some related papers, similar conclusions have been made. For example, Uzel [39] showed in a case study that LSTM performs worse than traditional ARIMA. This is because models like LSTM are high-variance ones, requiring large amounts of data to constrain their parameter space. Katambire et al. [17] also got similar conclusions when comparing ARIMA and LSTM for the case of Muhima Junction. After comparing LSTM with XGB and RF, Lam [10] found that the performance of ML models is better. In addition, when considering the fundamental diagrams constructed by highway traffic flow data, Shafiei et al. [21] found that the metrics of ML models and Neural Networks are nearly the same. Nevertheless, in these papers only two or three typical models are compared. Without a systematic comparison, it would also be very difficult for these authors to directly conclude that “a simple model performs better in this context”.

(4)

Other unexpected findings.

Although we have analyzed why Neural Networks do not perform well in our task, there are some unexpected findings, e.g., the good performance of STIDGCN. Since GNN models are not mainly designed for one-dimensional traffic flow predictions, it is not strange that most such models do not perform well in our tests. However, as shown in Table 3, the metrics of STIDGCN are very good, similar to many ML models. Considering this is the output of the default hyperparameters, it is indeed unexpected. This may be attributed to the structural distinction between STIDGCN and most GNN models. Inspired by the multi-level wavelet decomposition network, STIDGCN downsamples the input sequence according to the parity of indices and adopts an interactive learning strategy among the segmented subsequences. This integrated structure greatly enhances the model’s capability to capture long-term temporal dependencies, thus leading to its superior performance in the prediction task in this paper.

(5)

Limitations of this study

Although this paper provides a fair and relatively large-scale benchmark for highway traffic flow prediction, several limitations should still be acknowledged:

Constraints of one-dimensional time series modeling. The findings of this paper are mainly intended for raw one-dimensional highway traffic flow prediction under consistent benchmark settings. Therefore, for more complex forecasting scenarios involving spatial dependence, external variables, or heterogeneous data sources, further investigation is still needed.

Lack of theoretical derivation. The main goal of this paper is to provide a fair and unified benchmark under consistent experimental settings. Thus, the main contribution of this study is not a fully theoretical investigation of these models. In the future, a more thorough theoretical study would be highly valuable.

Lack of statistically rigorous conclusions. Although we have made lots of comparisons, the current results only show the quantitative differences between these models on certain datasets. It does not mean there exists any statistically significant differences, and the current conclusions may not necessarily be generalized to other datasets. To arrive at a statistically more rigorous conclusion, a more comprehensive investigation into more datasets may be needed in the future.

7. Conclusions

In this paper we focus on the evaluation of model performance for highway traffic flow prediction, which is based on 1D time series data from five representative highway traffic datasets. A total of 27 models across five categories are considered, including Statistical models, Machine Learning models, Artificial Neural Networks, Deep Neural Networks, and Graph Neural Networks. To ensure fairness, evaluations are performed on raw data without signal decomposition or auxiliary modules. After many tests, a surprising phenomenon for all the datasets is that complex deep learning models do not demonstrate significant advantages in terms of conventional metrics. Instead, simple models, particularly Historical Averaging and tree-based Machine Learning models (e.g., Random Forest, XGBoost), exhibit superior performance in most scenarios. Since these datasets have distinct characteristics that differ significantly in terms of collection time, environment, sampling interval, and the lengths used for analysis and prediction, the fact that such conclusions hold across all datasets warrants in-depth consideration.

For such a situation, we have to discuss some important topics, including “Why do the HA and ML models perform better, whereas the DNN and ANN models show inferior performance” and “Why are the findings in this paper not revealed in the previous related studies”? The main causes of the previous question include the complexity of prediction tasks, the nature of data structure, the theoretical bound of datasets, the spectral bias of deep learning and the difficulty of parameter optimization for some Neural Networks. For the latter question, the possible reasons include the wide use of signal decomposition methods, the confusion of prediction conditions, and the insufficiently systematic investigation in the previous papers.

In a word, we think this study is valuable and contributes to research in this field. With the advancement of modern technology, we have become increasingly inclined to employ complex models for various tasks, including traffic flow prediction. Many researchers prefer to use more sophisticated Neural Networks or even large models with hundreds of billions of parameters. However, our study indicates that such blind pursuit of model complexity is inappropriate. At least in certain specific fields (e.g., highway traffic flow prediction), simpler models may yield better performance—not only saving computational resources but also achieving higher accuracy. Therefore, even in an era of rapid development of large models, we should not overlook the effectiveness of early, classic models. Rather, we should ground our choices in empirical traffic evidence and tailor models to specific traffic tasks.

Despite the extensive experiments and findings presented in this paper, there are still several important directions for future research. On one hand, future work should expand from single-variable prediction to multi-source data fusion. In this study, we only utilized historical traffic volume data to verify the performance of all models. However, in real-world scenarios, traffic flow is highly sensitive to external factors such as weather conditions, traffic accidents, holidays, and the distribution of points of interest (POIs). Future research could explore how to effectively integrate these heterogeneous data sources into the prediction framework. On the other hand, an efficient method for hyperparameter tuning is needed. As mentioned in Section 4, for DNN and GNN models, we only consider the default hyperparameters, which are mainly due to the difficulty of properly adjusting them. If we can find a fast method to calibrate these hyperparameters, the results and conclusions of this paper may change. However, how to achieve this goal still requires great effort in future research.