A Multi-Model Dynamic Selection Framework Using Deep Contextual Bandits for Urban Traffic Flow Prediction in Large-Scale Road Networks

Abstract

To address the challenge of model selection in large-scale traffic flow prediction tasks, this paper proposes a dynamic multi-model selection framework based on Deep Contextual Bandits (DCB). Centered on the optimal combination of sub-models, the framework leverages contextual information of road segments to select dynamically among candidate predictors, achieving more efficient and accurate traffic flow prediction. Several mechanisms are introduced to improve strategy learning and convergence, including a baseline network, experience replay, double-model estimation, and prioritized experience sampling. A clustering-based strategy is further designed to reduce the search space and enhance the generalization and transferability. Experiments on real-world traffic datasets demonstrate that the proposed framework significantly outperforms traditional static fusion methods, reinforcement learning (RL) baselines, and mainstream spatiotemporal prediction models. In particular, the framework yields a 1.0% improvement in R² and a 3.2% reduction in MAE compared to state-of-the-art baselines, while reducing inference time by 43.1%. Moreover, the proposed framework shows strong capability in adaptive model selection under varying contexts, with ablation studies confirming the effectiveness of its key components.

1. Introduction

Traffic flow prediction in urban road networks is a prerequisite for developing strategies to alleviate congestion and enhance traffic capacity [1,2,3]. In large-scale urban networks, traffic flow across different road segments is influenced by various heterogeneous factors, including road network topology, spatiotemporal dependencies, and unexpected events such as traffic accidents or severe weather conditions [4,5,6,7]. For instance, traffic in central urban areas is typically shaped by the coupling effects of abrupt traffic surges during peak hours and complex topological structures [8]. Quantitatively, a recent empirical study on urban road networks revealed significant spatial heterogeneity, with the coefficient of variation (CoV) of traffic conditions ranging widely from 0.102 to 0.57 across different zones during peak periods [9]. In contrast, suburban traffic is more susceptible to fluctuations due to weather changes or sparse vehicle flow [10]. This heterogeneity suggests that a single predictive model, constrained by limited generalization capabilities, is often insufficient to meet the accuracy and stability requirements for traffic flow forecasting across diverse road segments in large-scale networks. Consequently, it becomes a meaningful research challenge to develop a dynamic model selection framework that can accurately account for the diverse combination effects of these influencing factors when selecting appropriate prediction models for each segment [11].

Classical traffic flow prediction models for urban networks generally fall into three categories. The first comprises traditional statistical models such as ARIMA, which are typically limited in capturing complex nonlinear relationships and are highly sensitive to noise in large-scale datasets [12]. The second category includes machine learning model such as support vector regression (SVR) and random forests (RFs). These models rely heavily on handcrafted feature engineering, making them impractical for deployment across extensive and heterogeneous networks [13]. The third category, which has gained popularity recently, consists of deep learning models such as long short-term memory (LSTM) networks and graph convolutional networks (GCNs). While these models can automatically extract spatial and temporal features, research shows that their single-architecture design still lacks the flexibility to dynamically adapt to the varying characteristics of different road segments in large-scale networks [14,15]. Classical single-model approaches suffer from limited generalization ability and tend to produce high global prediction errors when applied to complex, heterogeneous urban traffic systems.

Many researchers have attempted to capture the spatiotemporal heterogeneity of traffic flow through clustering methods to address the limitations of classical single-prediction models, namely, their weak generalization capabilities and unstable prediction accuracy. By categorizing different types of traffic patterns, they aim to assign distinct prediction models accordingly, thereby improving efficiency and accuracy. For instance, Ryu et al. [16] employed the clustering of large applications based on the randomized sampling (CLARANS) algorithm to cluster spatiotemporal traffic patterns, leveraging the classification of historical data to enhance prediction performance. Huang et al. [17] proposed an enhanced entropy regularization k-means (ERKM) algorithm, which improves clustering quality by maximizing inter-cluster distances based on the K-means framework. The resulting clusters were then integrated into an attention-based convolutional LSTM architecture to construct a multimodal prediction model. Similarly, Deng et al. [18] introduced the cluster clustering-based modular integrated DNN (CC-MIDNN) model in the aviation domain, using K-means clustering to decompose flight tasks into parallel sub-networks. Their approach significantly improved prediction stability by incorporating Bayesian optimization to reduce network complexity.

While such methods enhance the generalization ability of predictive models to a certain extent via clustering-based dimensionality reduction, they are still static multi-model selection frameworks that follow a “cluster-then-predict” paradigm. This makes them poorly suited to dynamically adapting to stochastic variations in traffic flow across large-scale urban road networks. For example, using fixed clustering centers or predefined dependencies fails to capture real-time changes in road segment characteristics. Sudden congestion propagation or unexpected incidents can also break these predefined dependencies, leading to increased prediction errors. Furthermore, these approaches often overlook subtle and dynamic interactions between road segments, such as time-varying influences from neighboring intersections or evolving traffic demand. As a result, their robustness deteriorates in noisy or volatile environments.

Nevertheless, ensemble prediction offers valuable insights for developing a dynamic multi-model selection framework. Specifically, by selecting the most suitable prediction model from a predefined pool of effective models for each road segment, it is possible to improve the stability and accuracy of traffic flow prediction across large-scale, heterogeneous road networks. In recent years, Reinforcement Learning (RL) has emerged as a promising approach to improve the generalization ability of predictive models further. RL methods operate under a two-stage “training–inference” paradigm, where the computationally intensive optimization is performed during the training phase. In contrast, the inference phase requires only lightweight online decision-making. This characteristic makes RL particularly suitable for the real-time prediction demands of large-scale urban road networks. However, current RL-based approaches predominantly focus on optimizing a single prediction model. To our knowledge, limited methodological exploration exists on applying reinforcement learning for adaptive model selection in heterogeneous road environments with dynamic segment characteristics.

Based on the above background, this paper proposes a multi-model dynamic selection framework built upon the Deep Contextual Bandit (DCB). The selection of DCB as the foundation is motivated by three key advantageous properties that align with the requirements of large-scale, heterogeneous traffic prediction:

(1)

Sample Efficiency: Unlike Reinforcement Learning (RL), which requires complex state transition learning, DCB focuses on maximizing immediate rewards based on the current context. This formulation is theoretically advantageous for traffic prediction, where the optimal model depends primarily on the present pattern rather than future state sequences.

(2)

Inference Efficiency and Interpretability: In contrast to soft-fusion ensembles (e.g., Mixture-of-Experts), DCB performs a deterministic selection of a single optimal sub-model. This reduces computational overhead during inference and provides interpretability by explicitly identifying the preferred model type for specific traffic conditions.

(3)

Handling Heterogeneity: DCB leverages deep neural networks to learn high-dimensional representations from traffic and topological features, enabling effective adaptation to spatiotemporal heterogeneity across thousands of road segments—a capability often limited in traditional bandit or static clustering approaches.

Building upon these strengths, the main contributions of this study are as follows.

(1)

A DCB-based dynamic model selection framework. This study is the first to introduce the DCB framework into the domain of large-scale urban road network traffic flow prediction, developing a fully end-to-end multi-model dynamic selection system. Leveraging reinforcement learning mechanisms, the framework dynamically adapts to evolving traffic patterns (e.g., sudden congestion, demand shifts) and topological changes using real-time contextual features of road segments—such as historical traffic flow and topological attributes. It dynamically selects the optimal prediction sub-model for each segment, significantly reducing the prediction error caused by applying a single model to heterogeneous road networks and enhancing overall prediction stability. Furthermore, the framework exhibits strong scalability, making it applicable to urban road networks of varying sizes and structures.

(2)

Reward-driven design with R² metric. We propose using the Coefficient of Determination (R²) as the reward signal to quantify sub-model prediction accuracy. Due to its standardized nature, R² effectively measures the proportion of data variance explained by the model, offering a more robust and consistent evaluation metric. Unlike traditional error-based indicators (e.g., RMSE, MAE), R² is less sensitive to traffic volume disparities or noise across road segments and more stable in the presence of outliers, thus improving comparability and reliability in model selection [19].

(3)

A hybrid exploration strategy. We design a hybrid exploration mechanism that combines Priority Sampling with ε-greedy exploration. Priority Sampling dynamically adjusts learning focus based on the magnitude of performance differences (e.g., R² disparities) among sub-models across road segments, thereby accelerating policy optimization in critical regions. Meanwhile, ε-greedy maintains a level of stochastic exploration at the local segment level to preserve diversity. This strategy effectively balances global feature learning with local heterogeneity adaptation, improving robustness to noise in large-scale networks.

(4)

Efficient and stable training optimization techniques. To improve training efficiency and stability, we integrate several key strategies into the DCB model: Experience Replay enhances training stability by reusing historical interaction data; A baseline-based reward formulation reduces variance caused by differences in the inherent prediction difficulty among road segments. A Double Network architecture mitigates the common problem of Q-value overestimation, leading to more stable policy learning.

Experimental results based on data from 4156 urban road segments demonstrate that the proposed DCB framework significantly outperforms traditional clustering-based methods and state-of-the-art single-model approaches in terms of prediction accuracy and robustness to noise. With its fast response and adaptive capabilities, DCB shows strong potential as a scalable tool for real-time urban traffic management, particularly in complex and interference-prone scenarios.

The remainder of this paper is structured as follows. Section 2 reviews related work on road segment traffic flow prediction and contextual bandit learning. Section 3 details the design and algorithmic implementation of the DCB framework. Section 4 presents experimental results and comparative analyses. Section 5 summarizes the findings and outlines directions for future work.

2. Literature Review

This section reviews the relevant literature to contextualize our work. First, we summarize the evolution of traffic flow prediction methodologies, highlighting the shift from single-model approaches to more complex deep learning architectures that attempt to handle spatiotemporal heterogeneity (Section 2.1). Next, we review the application of Reinforcement Learning (RL) in transportation, specifically focusing on Deep Q-Networks (DQN) and key optimization mechanisms—such as experience replay—that inspire our model design (Section 2.2). Finally, we discuss the principles of Deep Contextual Bandits (DCB) and explain how we modify these techniques to construct a dynamic model selection framework suitable for large-scale road networks (Section 2.3).

2.1. Deep Learning for Traffic Prediction

Early road segment traffic flow prediction approaches primarily relied on traditional statistical models (e.g., ARIMA) and classical machine learning methods. However, their limitations in capturing complex spatiotemporal dependencies have driven a shift toward deep learning. Convolutional Neural Networks (CNNs) were among the first deep learning models applied to capture local spatial features. For instance, Song et al. [20] demonstrated through comparative experiments that CNNs reduced prediction errors by 6% compared to multilayer perceptrons (MLPs) in short-term speed forecasting. Liu et al. [21] further enhanced area-level traffic flow prediction by incorporating residual units, thereby improving the model’s generalization ability. Subsequently, Recurrent Neural Networks (RNNs) and their variants—such as LSTM and GRU—gained popularity for their strength in modeling temporal sequences. Bharti et al. [14] introduced a PSO-Bi-LSTM hybrid model, which combines Particle Swarm Optimization (PSO) for global parameter tuning with bidirectional LSTM to capture temporal dependencies in both directions, significantly improving convergence speed and predictive accuracy.

However, conventional CNNs and RNNs struggle to model the non-Euclidean structure of road networks effectively. This limitation spurred the development of Graph Convolutional Networks (GCNs). Chen et al. [22] proposed an enhanced dynamic graph neural network (GNN) that integrates Spatial–Temporal dependency modeling with data interpolation, significantly improving the robustness of network-wide traffic flow prediction. Building upon this, Zheng et al. [23] developed the VDGCNeT model, which innovatively combines virtual dynamic graphs with Transformer architectures to capture dynamic latent relationships between road segments. Graph convolutions are then used to refine spatially fused features, leading to improved predictive performance.

Hybrid architectures and attention mechanisms have become prominent research areas as model complexity increases. Petrović et al. [24] addressed the issue of data sparsity in large-scale road networks by proposing the GCRFBC-GCRF hybrid model, which explicitly models spatial dependencies and outperforms traditional methods in traffic state estimation. Wang et al. [25] introduced PFNet, a cascaded progressive attention framework that balances computational efficiency and predictive performance in large-scale forecasting tasks. Xing et al. [26] proposed the SLTTCN model, which employs spatial linear transformers and bidirectional temporal convolutional networks for efficient large-scale traffic flow prediction.

These advancements illustrate a growing focus on spatiotemporal fusion and self-supervised learning within traffic flow prediction research. While traditional neural network models exhibit strengths in specific prediction scenarios, their performance tends to be unstable in complex and diverse settings. This highlights the urgent need for a dynamic model selection framework that can deliver consistently high accuracy across various large-scale road network conditions.

2.2. Reinforcement Learning in Transportation

The introduction of the Deep Q-learning Network (DQN) marked a significant breakthrough in reinforcement learning, enabling deep reinforcement learning for a wide range of complex decision-making tasks. The core idea of DQN is to approximate the action-value function (Q-function) using a deep neural network (DNN), which predicts the expected cumulative reward $Q\left(s,a\right)$ of taking action $a$ given the current state $s$. Unlike traditional reinforcement learning methods that rely on discrete state-action spaces, DQN leverages the nonlinear representation capabilities of neural networks to directly process high-dimensional, continuous inputs (e.g., images or time-series data), thereby substantially broadening the scope of reinforcement learning applications.

The success of DQN is largely attributed to two key innovations: experience replay and the target network. Experience replay stores agent-environment transitions (i.e., states, actions, rewards, next states) in a replay buffer for training. During training, mini-batches are randomly sampled from this buffer, which breaks the temporal correlation between samples and effectively mitigates issues such as policy oscillation and overfitting. The target network, which shares the same architecture as the main Q-network but is updated at a slower rate, is used to compute the target Q-values $Q^{\prime }\left(s^{\prime },a^{\prime }\right)$. This reduces the interference of rapidly changing parameters on target estimation, improving training stability.

Many improvements to DQN have been proposed to enhance performance further. For example, Prioritized Experience Replay (PER) [27] adjusts the sampling probability of experiences based on the temporal difference (TD) error, allowing the model to prioritize learning from high-value samples and accelerate convergence. Double DQN [28] addresses the problem of overestimated Q-values by decoupling action selection and evaluation. These improvements have been successfully applied across various domains, including robot path planning, multi-agent coordination, and traffic flow prediction. For instance, Chen et al. [29] utilized experience replay to improve decision efficiency and robustness in multi-aircraft conflict resolution. In robot navigation, Li et al. [30] adopted PER to quantify the learning potential of experiences using TD-error, further integrating importance sampling to reduce estimation bias. In traffic flow prediction, Nie et al. [31] proposed a hybrid model combining DQN with Generative Adversarial Networks (GANs) to handle traffic big data under COVID-19 conditions. Ullah et al. [32] developed a GCN–Double DQN-based framework to address the dynamic and complex nature of task offloading in vehicular networks.

However, existing DQN-based methods in traffic prediction primarily focus on optimizing a single model—such as improving network architectures or fine-tuning parameters. To the best of our knowledge, limited research has systematically investigated the application of reinforcement learning for dynamic model selection in heterogeneous road segments.

2.3. DCB-Based Dynamic Model Selection

To meet the requirements of traffic flow prediction in large-scale road networks, this study incorporates modular ideas from the Deep Q-learning Network (DQN) to modify the Deep Contextual Bandit (DCB) appropriately, aiming to enhance prediction accuracy and efficiency.

Traditional Multi-Armed Bandit (MAB) algorithms achieve optimal decision-making in dynamic environments by balancing exploration and exploitation. However, conventional MAB approaches struggle to effectively process complex contextual information in high-dimensional state spaces, which limits their applicability in real-world scenarios [33]. To overcome this limitation, the Deep Contextual Bandit (DCB) was introduced. Its core idea is to leverage Deep Neural Networks (DNNs) to automatically learn contextual feature representations and integrate dynamic exploration strategies, such as Upper Confidence Bound (UCB), Thompson Sampling, and probabilistic matching to optimize action selection and efficiently estimate expected rewards [34]. Compared with traditional MAB, DCB significantly improves decision-making efficiency in high-dimensional and nonlinear environments, making it a powerful tool for optimizing large-scale dynamic systems.

Importantly, unlike Reinforcement Learning (RL), which requires learning a state transition model and optimizing for cumulative future rewards, the contextual bandit framework only considers the immediate reward given the current context. This matches our problem setting where the optimal model for a road segment depends primarily on its current traffic pattern, and the selection at one time step does not directly affect future states. This makes DCB more sample-efficient and stable for large-scale, heterogeneous prediction tasks compared to general RL methods.

As shown in Figure 1, the input to the DCB is the contextual state $s\in S$, and the output is the expected reward for each action $a\in A$. Unlike the linear reward models used in traditional MAB, DCB models the reward function using a deep neural network, as shown in Equation (1).

$$R\left(s,a,\theta \right)=f_{\theta }\left(s,a\right),$$

(1)

where $\theta$ represents the network parameters and $f_{\theta }$ denotes the DNN. This nonlinear modeling capability allows DCB to capture complex context–action interactions, thereby enhancing its responsiveness to varying traffic environments.

Figure 1. Network architecture of the DCB.

At the heart of DCB lies its exploration strategy, which aims to balance exploration and exploitation. Unlike the simplistic strategies used in traditional MAB, DCB typically adopts uncertainty-aware exploration mechanisms:

• Upper Confidence Bound (UCB): Estimates an upper confidence bound for the expected reward of each action and selects the action with the highest bound. This approach encourages the exploration of actions with high potential but uncertain reward estimates.
• Thompson Sampling: Maintains a posterior distribution over the reward function. At each step, it samples a parameter instance from this distribution and selects the action with the highest expected reward under that sample. This naturally balances exploration and exploitation.
• Other Probabilistic Matching Strategies: These sample actions are based on the probability that each action is currently optimal, according to the estimated reward distribution.

Compared to random exploration strategies, these uncertainty-driven approaches exploit the learned feature representations and uncertainty information from the deep network, enabling more efficient exploration in high-dimensional contextual spaces. While the standard ε-greedy strategy [35] is simple and widely used, its random nature often leads to inefficiency in the DCB framework. It is typically replaced or augmented by a more intelligent strategy based on uncertainty.

In summary, the traditional ε-greedy strategy exhibits poor scalability for large-scale road networks due to inefficient exploration. Although UCB and Thompson Sampling [36] improve the intelligence of exploration, they still face challenges regarding computational complexity and implementation in high-dimensional traffic state spaces. To address this, we propose a hybrid exploration strategy that combines Priority Sampling with ε-greedy, aligning with hierarchical exploration. Priority Sampling is employed at the macro level to identify which road segments to explore. Moreover, a small amount of ε-greedy is retained at the micro level to ensure that potentially high-performing models are not overlooked.

By dynamically allocating learning resources based on quantified differences in model performance, this approach is better suited for learning the diverse characteristics of heterogeneous road segments. It offers a novel and efficient exploration paradigm for large-scale traffic flow prediction.

To bridge the research gap in dynamic model selection and leverage these advantages, the DCB framework proposed in this paper draws inspiration from DQN’s experience replay and dual-network mechanisms, while introducing novel enhancements. Specifically, we redefine the action space to represent the model selection problem and design a context-aware dynamic decision mechanism. This enables adaptive selection of the optimal prediction model for each road segment based on its real-time traffic flow characteristics, thereby significantly improving the accuracy and robustness of traffic prediction across large-scale road networks.

3. Model

3.1. Deep Contextual Bandit (DCB) Model

3.1.1. Model Design Motivation and Overall Framework

The core idea behind the Deep Contextual Bandit (DCB) model is to utilize deep neural networks to learn a mapping between the contextual traffic flow characteristics of a given road segment and the predictive performance of various forecasting models. Based on this learned relationship, the DCB framework dynamically selects the most suitable prediction sub-model for each road segment. As illustrated in Figure 2, the input to the DCB model consists of contextual features of the target road segment, which capture its temporal and spatial traffic patterns. The model estimates the expected predictive performance (i.e., expected reward) of each candidate sub-model under the given contextual conditions. The sub-model with the highest expected reward is then selected to generate the final traffic flow prediction for that segment. This adaptive model selection mechanism enables the DCB framework to effectively tailor its predictions to diverse and dynamic traffic scenarios across a large-scale road network, thereby significantly enhancing both accuracy and robustness compared to static, single-model approaches. This capability to handle contextual heterogeneity aligns with the broader need in intelligent transportation systems for models that maintain consistent performance under varying conditions—a challenge also highlighted in related tasks such as vision-based crash detection across day-night scenes [37].

Figure 2. Framework diagram for model selection.

From a theoretical perspective, the contextual bandit framework provides regret guarantees under stationary environments, ensuring that the learning process converges to the optimal model selection policy. In non-stationary traffic conditions—such as those caused by events, holidays, or long-term trends—DCB’s exploration mechanisms (e.g., the hybrid ε-greedy strategy with Priority Sampling) enable continuous adaptation. Although establishing formal regret bounds is more challenging for non-stationary bandits, our empirical design incorporates experience replay and periodic retraining to mitigate performance degradation under gradual distribution shifts. Consequently, DCB is well-suited for real-world traffic networks characterized by both short-term fluctuations and long-term evolution.

Fundamentally, DCB employs a deterministic ‘hard’ selection strategy, unlike the soft weighted fusion in Mixture-of-Experts (MoE). This approach not only reduces inference latency but also enhances interpretability by explicitly identifying the single most effective model for a given traffic context.

3.1.2. State-Action-Reward Ternary Modeling

To implement the model selection mechanism of the DCB framework, this paper formalizes the traffic flow prediction task as a contextual bandit problem, which is defined by a core ternary structure consisting of state, action, and reward. In each decision-making round, the model receives a context vector, which encodes the current environmental state (e.g., traffic flow patterns of the target road segment). Based on this context, the model selects and executes an action, corresponding to choosing one of the available prediction sub-models. The system then provides a reward value based on the predictive performance of the selected model, which is used to update the decision policy.

The key components of the contextual bandit formulation are illustrated in Figure 3. In this framework, the State $s$ represents the environmental context which serves as the input for the decision policy to determine the Action ($a$). Crucially, the reward $r$ is modeled as a function of both the state $s$ and action $a$ ($r\sim \mathcal{D}(s,a)$). This formulation explicitly captures that a sub-model’s efficacy is intrinsically tied to the prevailing traffic conditions (state). The agent utilizes this state-dependent reward feedback to optimize its policy.

Figure 3. State–action–revenue triad structure.

• State (state,s)

The state represents the context of traffic features used to describe the current road section. In this paper, 16-dimensional features are extracted, including the following three categories.

• Autocorrelation Coefficient

To portray the time-dependence of the traffic flow, this paper calculates the Autocorrelation Coefficient of the road section under different time lags (lag). Let the time series of traffic flow be $x_t$, and the autocorrelation coefficient of its lag is defined in Equation (2).

$${\rho }_l={\displaystyle \frac{{\sum }_{t=1}^{T-l}\left(x_t-\overline{x}\right)\left(x_{t+l}-\overline{x}\right)}{{\sum }_{t=1}^T{\left(x_t-\overline{x}\right)}^2}},$$

(2)

where $\overline{x}$ is the sample mean. The lagged value $l\in \left\{1,2,3,4,8,12,96,672\right\}$ is selected.

• Statistical distribution characteristics

Skewness $Skew$, kurtosis $Kurt$, mean $\mu$, standard $\sigma$ of traffic flow; these indicators can assist the model to determine the volatility of traffic flow and abnormal behavior, as shown in Equations (3)–(6).

$$\mu ={\displaystyle \frac{1}{T}}{\sum }_{t=1}^T{x}_t,$$

(3)

$$\sigma =\sqrt{{\displaystyle \frac{1}{T}}{\sum }_{t=1}^T{\left(x_t-\mu \right)}^2},$$

(4)

$$Skew\left(x\right)={\displaystyle \frac{1}{T}}{\sum }_{t=1}^T{\left({\displaystyle \frac{x_t-\mu }{\sigma }}\right)}^3,$$

(5)

$$Kurt\left(x\right)={\displaystyle \frac{1}{T}}{\sum }_{t=1}^T{\left({\displaystyle \frac{x_t-\mu }{\sigma }}\right)}^4,$$

(6)

where $x_t$ is the observed traffic volume at time step $t$; $T$ is the total length of the time series; $\mu$ is the mean of the traffic flow; and $\sigma$ is the standard deviation of the traffic flow.

• Topological Features

These include Degree Centrality $C_D$, Clustering Coefficient $C$, Betweenness Centrality $C_B$, and Eigenvector Centrality $C_E$. The formulas are Equations (7)–(10).

$$C_D\left(i\right)={\displaystyle \frac{k_i}{N-1}}$$

(7)

$$C\left(i\right)={\displaystyle \frac{2E_i}{k_i\left(k_i-1\right)}}$$

(8)

$$C_B\left(i\right)=\sum _{s\ne i\ne t}{\displaystyle \frac{{\sigma }_{st}\left(i\right)}{{\sigma }_{st}}},$$

(9)

$$C_E\left(i\right)={\displaystyle \frac{1}{\lambda }}\sum _{jϵN\left(i\right)}{A}_{ij}{C}_E\left(j\right),$$

(10)

where $N$ is the total number of nodes in the network; $k_i$ is the degree of a node $i$, i.e., the number of edges connected to it; $E_i$ represents the number of actual edges between the neighboring nodes of node $i$; ${\sigma }_{st}$ is the total number of shortest paths between node $s$ and node $t$; ${\sigma }_{st}\left(i\right)$ is the number of those shortest paths that pass through node $i$; $A_{ij}$ is the element of the adjacency matrix, if node $i$ is connected to node $j$, then $A_{ij}=1$, otherwise it is 0; $\lambda$ is the eigenvalue corresponding to the largest eigenvalue; $C_E\left(i\right)$ is the eigenvector centrality score of node $i$.

By integrating these three dimensions—temporal dependencies, statistical distribution patterns, and topological structures—we construct a comprehensive 16–dimensional feature vector that fully characterizes the traffic context of each road segment. This multi-view representation enables the agent to distinguish between different traffic scenarios effectively. The detailed composition and categorization of these contextual features are summarized in Table 1.

Table 1. Summary of the 16 Contextual Features.

Category	Feature Name	Description/Parameters	Count
Temporal Features	Autocorrelation Coefficients	Calculated at different time lags $l\in \left\{1,2,3,4,8,12,96,672\right\}$ to capture short-term and long-term temporal dependencies.	8
Traffic State Features	Statistical Distribution Metrics	Includes Mean ($\mu$), Standard Deviation ($\sigma$), Skewness ($Skew$), and Kurtosis ($Kurt$) of the historical traffic flow sequence.	4
Network Topology Features	Graph Structural Indicators	Includes Degree Centrality ($C_D$), Clustering Coefficient ($C$), Betweenness Centrality ($C_B$), and Eigenvector Centrality ($C_E$) to describe the node’s importance in the road network.	4
Total			16

2.

Action (action, a)

In the DCB framework proposed in this paper, the action set consists of multiple candidate prediction models, denoted as $\{M_1,M_2,...,M_N\}$. At each decision step, DCB evaluates the prediction performance (i.e., action value) of each candidate model on the current road segment based on its contextual features, and selects the model with the highest action value as the prediction model for that segment.

3.

Reward (reward, r)

In traffic flow prediction tasks, commonly used error metrics such as Root Mean Square Error (RMSE) and Mean Absolute Error (MAE) have certain limitations. First, these metrics lack standardized reward signals, which reduces their comparability across different scenarios. For instance, the absolute values of RMSE and MAE tend to fluctuate significantly during peak periods, holidays, or under weather-related disturbances, making it challenging to assess model performance over time consistently. Second, traditional metrics, especially RMSE, are highly sensitive to noise, and in the presence of unexpected events, significant error points can be disproportionately amplified, thereby distorting the overall evaluation. While RMSE and MAE can offer insights into localized prediction errors, they often fail to effectively capture cyclical patterns, trends, and structural characteristics inherent in traffic flow data.

In contrast, when used as the reward signal, the coefficient of determination (R²) effectively addresses these issues. It provides a standardized gain signal by measuring how well the model explains the overall variance in the data, enabling more consistent model evaluation across diverse scenarios. R² also reflects trends and cyclical variations in traffic flow, making it better suited for complex and dynamic environments.

Therefore, this paper adopts R² as the reward metric in the DCB framework, and it is defined in Equation (11).

$$R^2=1-{\displaystyle \frac{SSE}{SST}}=1-{\displaystyle \frac{{\sum }_{t=1}^T{\left(x_t-{\widehat{x}}_t\right)}^2}{{\sum }_{t=1}^T{\left(x_t-\overline{x}\right)}^2}},$$

(11)

where SSE is the sum of the squares of the gap (error) between the model’s predictions and the actual values, representing the degree of model error, and SST is the sum of the squares of the degree of volatility of the actual values themselves, representing the degree of complexity of the data itself. ${\widehat{x}}_t$ is the predicted value.

3.2. Improvement Strategies

To enhance the stability, robustness, and learning efficiency of the Deep Contextual Bandit (DCB) model in traffic flow prediction tasks, this paper proposes several key optimization strategies, including the experience replay buffer, relative gain design, prioritized sampling mechanism, and dual-network architecture. The following sections elaborate on each strategy in detail.

3.2.1. Experience Replay Buffer

Traditional contextual bandit algorithms typically update model parameters based only on the state–action–reward (gain) triplet obtained during a single round of interaction with the environment, after which the data from that round are discarded. This approach severely limits the reuse of historical experience, resulting in low sample utilization. As shown in Figure 4, to fully exploit the potential value of past experiences, this paper introduces an Experience Replay Buffer (denoted as D) [38]. The buffer continuously accumulates the state–action–reward triplets generated during the interaction between the model and the environment and repeatedly samples them for training. This significantly improves sample reuse efficiency. Moreover, the experience replay mechanism breaks the temporal correlation between samples by enabling random sampling from the historical experience pool. This contributes to balancing bias and variance during policy updates, enhancing training stability and reducing the risk of oscillations and overfitting in the learning process.

Figure 4. Experience Replay Mechanism in Deep Q-Networks.

Unlike classical deep reinforcement learning algorithms (e.g., DQN), this study does not impose a maximum storage limit on the experience replay buffer. This design is based on the observation that traffic flow prediction is not a sequential decision-making problem, and the historical experiences are generally independent. In contrast to sequential decision tasks—where redundancy and interference in the experience buffer can degrade performance—in traffic flow prediction, the accumulation of historical samples does not introduce a significant negative impact. This approach helps to maximize data utilization by ensuring that every interaction experience can be fully leveraged during subsequent policy optimization.

If the experience buffer $\mathcal{D}$ has not yet reached a predefined minimum capacity threshold during training, the model is trained using only the newly collected experiences from the current batch. Once the number of stored experiences meets the threshold, the training process switches to Prioritized Sampling based on the historical samples in the buffer, and a mini-batch training strategy is adopted to enhance training efficiency and improve the model’s generalization ability.

3.2.2. Baseline Model

Although the R² metric is effective for evaluating model fitting performance, it can exhibit considerable fluctuations during training due to significant differences in traffic flow periodicity, volatility, and data quality across different road segments. For instance, segments with more predictable and regular patterns tend to yield higher R² values. In contrast, other segments, affected by noise or greater variability, may show lower R² values, even when the predictions are reasonably accurate. In the Deep Contextual Bandit (DCB) learning process, if the same candidate model produces highly variable prediction results across different road segments, the sample variance increases, which may negatively affect the convergence rate of the learning algorithm. To address this issue, this paper introduces a relative gain mechanism by incorporating the prediction results of a baseline model, thereby reducing sample variance. The relative gain is defined in Equation (12).

$$r=R_{action}^2-R_{baseline}^2,$$

(12)

where $R_{action}^2$ denotes the actual R² value (goodness-of-fit) of the currently selected model, and $R_{baseline}^2$ represents the R² value of the baseline model under the same contextual state. The baseline model is typically selected from the candidate models that exhibit relatively stable performance and faster training speed.

By providing a consistent and representative predictive reference, the baseline model helps mitigate prediction accuracy fluctuations caused by differences in road segment characteristics. This allows the DCB model to more accurately assess the relative predictive performance of each candidate model for the same prediction target.

3.2.3. Priority Sampling

To enhance training efficiency, this paper introduces a Priority Sampling strategy. The core idea is to prioritize experiences that exhibit significant differences in model performance, thereby accelerating the learning process for critical road segments. The strategy involves the following steps:

1.

Priority Calculation

For each newly generated experience $m$, DCB evaluates the action values $Q$ of all candidate models. To avoid the situation where poor performance by certain models across most road segments diminishes the distinctiveness of the priority scores, the top K highest action values are selected, denoted as $\{Q_1,Q_2,...,Q_K\}$. The priority of experience $m$ is then defined as the average pairwise difference among these values, as shown in Equation (13).

$${Priority}_m={\displaystyle \frac{1}{K\left(K-1\right)/2}}{\sum }_{1\le i\le j\le k}\left|Q_i-Q_j\right|.$$

(13)

A higher priority indicates greater variance in the predictive performance of different models on the given segment, suggesting higher learning value. Conversely, model selection becomes less meaningful for that segment if model performances are similar.

2.

Sampling Probability

Based on the calculated priority scores of all experiences, the sampling probability of experience $m$ is defined in Equation (14).

$$P_m={\displaystyle \frac{{Priority}_m}{{\sum }_{m^{\prime }\in \mathcal{D}}{Priority}_{m^{\prime }}}},$$

(14)

where $\mathcal{D}$ denotes the experience replay buffer.

3.

Dynamic Update Mechanism

To account for model instability in the early stages of training, the priority of each sampled experience $mϵ\mathcal{D}$ is recalculated using the current policy network after it is selected. This dynamic update ensures that the sampling strategy remains focused on the most valuable experiences under the current state of the model.

3.2.4. Double Network Structure (Double DQN)

To enhance model stability and reduce the overestimation problem during training, this paper draws on the Double DQN architecture and introduces a dual-network strategy. This strategy consists of: A policy network with parameters $\theta$, which estimates the state-action value $Q(s,a;\theta )$ based on the current state; A target network with parameters ${\theta }^{\prime }$, which periodically copies parameters from the policy network and is used to compute the target value $Q^{\prime }(s,a;{\theta }^{\prime })$.

This design avoids estimation bias caused by feedback from the same network during training. Since the DCB task does not involve subsequent states, this paper adopts an action selection method that averages the outputs of the two networks, is formulated in Equation (15).

$$\overline{Q}(s,a)={\displaystyle \frac{Q\left(s,a;\theta \right)+Q^{\prime }(s,a;{\theta }^{\prime })}{2}}$$

(15)

This strategy reduces Q-value overestimation, thereby improving training stability and effectively diminishing fluctuations during the training phase.

3.3. Model Training Framework

The training framework of the Deep Contextual Bandit (DCB) model proposed in this paper is illustrated in Figure 5. The overall process alternates between interaction and training phases to achieve continuous model performance optimization.

Figure 5. Flowchart of the proposed algorithm.

During each interaction round, the system integrates the improvement strategies into the framework. First, the environment generates the current context state $s$ (i.e., the feature vector of the target road segment). Unlike state transitions in traditional MDPs, here $s$ serves strictly as the input condition for the policy and target networks to determine the optimal action, while the reward $r$ is used exclusively for loss calculation. Subsequently, the DCB system evaluates the state-action values of candidate prediction models under the current state $s$ using two independent neural networks: the policy and target networks. The policy network’s parameters are continuously updated, while the target network’s parameters are periodically synchronized from the policy network to enhance prediction stability.

Based on the $ϵ$-greedy strategy, the system selects an action $a$—corresponding to a specific candidate traffic flow prediction model—and performs prediction on the current data window. To explicitly quantify the model’s performance, the reward $r$ is calculated as the Coefficient of Determination (R²) for the current prediction horizon:

$$r=1-{\displaystyle \frac{\sum _{t=1}^T{\left(y_t-{\widehat{y}}_t\right)}^2}{\sum _{t=1}^T{\left(y_t-\overline{y}\right)}^2}}$$

(16)

where $y_t$ is the ground truth traffic flow, ${\widehat{y}}_t$ is the prediction from the selected model, and $\overline{y}$ is the mean of the observed values in the current window. This reward $r$, along with the state $s$ and action $a$, is stored in the experience replay buffer $\mathcal{D}$.

In the training phase, the reward $r$ serves directly as the training target to guide network optimization. Specifically, we define the loss function $L\left(\theta \right)$ as the expected squared error between the estimated Q-value and the observed reward:

$$L\left(\theta \right)={\mathbb{E}}_{(s,a,r)\sim D}\left[{(r-Q(s,a;\theta ))}^2\right]$$

(17)

Here, since the Contextual Bandit problem assumes no state transition (discount factor $\gamma =0$), the standard Bellman target simplifies to the immediate reward $r$. To improve sample efficiency, the model applies the Prioritized Experience Replay strategy. The sampling priority is computed based on the Temporal Difference (TD) error $\delta =\left|r-Q\left(s,a;{\theta }^{-}\right)\right|,$ emphasizing experiences where the model’s current estimation deviates significantly from the actual performance (R²). Through continuous iteration of this process, the DCB model progressively learns to map context states to the optimal model selection strategy.

The end-to-end training process of the DCB framework is detailed in Algorithm 1. It operates in an iterative loop of interaction and training phases. Key components integrated into this process include: (1) the hybrid exploration strategy for segment and action selection; (2) the dual-network (policy and target) architecture for stable $Q$-value estimation; (3) the experience replay buffer $\mathcal{D}$ for sample reuse; and (4) the priority sampling mechanism for efficient learning. The target network parameters ${\theta }^{\prime }$ are synchronized with the policy network parameters $\theta$ every $C=100$ training steps to stabilize the learning target. The algorithm terminates when the average reward (R²) on a held-out validation set plateaus over a predefined number of epochs.

Algorithm 1. Training Procedure of DCB Framework

Input: Road segment dataset $\mathcal{R},$ candidate model pool $\left\{M_1,\dots ,M_N\right\}$, hyperparameters (Section 4.2.5), experience replay buffer $\mathcal{D}$ (initially empty), priority cache. Output: Trained policy network parameters $\theta .$ 1: Initialize policy network $Q(\cdot ;\theta )$ and target network $Q^{\prime }(\cdot ;{\theta }^{\prime })$ with weights ${\theta }^{\prime }=\theta$. 2: Pre-train candidate models and perform feature clustering as per Section 4.2.4. 3: for episode $=1$ to $E$do 4:   // --- Interaction Phase --- 5:   Select a road segment $s$: 6:     With probability $\beta$, use Priority Sampling Equation (14) to select $s$ from $\mathcal{D}.$ 7:     Otherwise, select $s$ uniformly from $\mathcal{R}.$ 8:     Extract contextual feature vector $\varphi \left(s\right)$ (Section 3.1.2). 9:     // Action selection with $ϵ$-greedy 10:   With probability $ϵ$: select a random action (model) $a$. 11:   Otherwise: $a=\arg {max}_{a\prime }\overline{\mathrm{Q}}(\varphi \left(s\right),a^{\prime })$ where $\overline{\mathrm{Q}}$ is from Equation (15). 12:   Execute model $M_a$ to predict traffic flow for segment $s$. 13:   Observe reward $r=R_{M_a}^2-R_{baseline}^2$ Equation (12). 14:   Store transition $(\varphi \left(s\right),a,r)$ in replay buffer $\mathcal{D}$. 15:   Calculate and store priority for this transition using Equation (13). 16: 17:   // --- Training Phase --- 18:   if $\left|\mathcal{D}\right|\ge$ minimum batch size then 19:     Sample a mini-batch of $B$ transitions from $\mathcal{D}$ using priority probabilities $P_m$ Equation (14). 20:     for each sampled transition $\left(\varphi \left(s_j\right),a_j,r_j\right)$ do 21:       // Compute target Q-value (simplified for contextual bandit) 22:       $y_j=r_j$ 23:       // Update priority based on current prediction 24:       Recalculate $Priorit{y}_j$ using current $Q(\cdot ;\theta )$ and Equation (14). 25:     end for 26:     // Update policy network via gradient descent 27:     Perform a gradient step on $\theta$ to minimize ${\displaystyle \frac{1}{B}}\sum _j{(y_j-Q(\varphi \left(s_j\right),a_j;\theta ))}^2$. 28:     // Periodically update target network 29:     if episode mod $C==0$ then 30:       ${\theta }^{\prime }\leftarrow \theta$ 31:     end if 32:  end if 33: end for

4. Experiments

4.1. Dataset

This study utilizes a large-scale real-world traffic flow dataset collected from the Intelligent Transportation System (ITS) of City D, China. The raw data were captured by fixed loop detectors covering 4156 road segments across the city’s extensive expressway network. The traffic volume records were aggregated at 15 min intervals, spanning a four-month period from 23:45 on 26 November 2023, to 23:45 on 26 March 2024. This specific time window is significant as it encompasses major public holidays, including New Year’s Day and the Spring Festival (Chinese New Year). Including these periods allows for the evaluation of the model’s robustness under complex, non-stationary traffic patterns caused by holiday travel surges and demand fluctuations.

Data preprocessing was first conducted to ensure data quality. Raw data were cleaned to remove outliers and abnormal records, and missing values were addressed using a historical mean-imputation strategy, where missing entries were replaced with the average flow of the corresponding time-of-day from other days for the same road segment. To enhance training stability, all traffic flow data were normalized to the range $[0,1].$ Unlike approaches relying on external weather or event data, the DCB framework extracts intrinsic contextual features to implicitly capture environmental variations without relying on external auxiliary data. Specifically for the performance verification of the DCB model, 100 representative road sections were randomly selected from the test set as the primary observation objects.

For the evaluation of candidate sub-models and the overall training process, the dataset was chronologically partitioned into a training set (the first 80%) and a test set (the subsequent 20%). Specifically, the training phase covers the period from 26 November 2023, to 29 February 2024, while the testing phase spans from 1 March 2024, to 26 March 2024. A sliding window approach was employed, utilizing historical traffic flow data from the past 6 h (24 time steps) to predict the traffic volume for the next 15 min interval. To ensure statistical reliability, all experiments were repeated five times.

4.2. Experimental Setup

All experiments were conducted on a workstation running Windows 10 Professional 64-bit, equipped with an Intel Xeon W-2245 processor (3.90 GHz, 16 threads), 32 GB of RAM, and an NVIDIA Quadro RTX 4000 graphics card with 8 GB of video memory. The experimental programs were implemented in Python 3.11, with the deep learning framework primarily based on PyTorch 1.13.1.

4.2.1. Evaluation Metrics

While the ultimate goal of the Deep Contextual Bandit (DCB) model is to reduce prediction error, R² is used as the training objective in this paper. To comparatively evaluate the effectiveness of DCB in reducing traffic flow prediction errors across large-scale road networks, two commonly used error metrics are introduced: MAE and RMSE.

1.

Mean Absolute Error (MAE)

$$MAE={\displaystyle \frac{1}{T}}{\sum }_{t=1}^T\left|x_t-{\widehat{x}}_t\right|.$$

(18)

2.

Root Mean Squared Error (RMSE)

$$RMSE=\sqrt{{\displaystyle \frac{1}{T}}{\sum }_{t=1}^T{\left(x_t-{\widehat{x}}_t\right)}^2}.$$

(19)

MAE measures the average absolute size of the error between the predicted value and the real value, which is insensitive to outliers and suitable for scenarios with a more uniform distribution of errors. RMSE represents the square root of the prediction error, with the same unit as the original data, which is more sensitive to larger errors and reflects the model’s prediction accuracy more clearly.

4.2.2. Selection of Baseline Model

This study considers computational efficiency and prediction stability when selecting the baseline prediction model. Computational efficiency relates to training speed, while stability affects the reliability of reward calculation. The XGBoost (version 1.7.5) model, which achieves a good balance between these two aspects, is ultimately chosen as the baseline model to calculate the reward in the DCB framework.

4.2.3. Action Set Models

The action set constructed in this paper consists of six classical traffic flow prediction models: Bayesian Regression (BR), K-Nearest Neighbors (KNN), XGBoost, Support Vector Regression (SVR), Random Forest (RF), and Long Short-Term Memory networks (LSTM). The architectures of KNN, SVR, DNN, and LSTM are based on the designs in Salamanis et al. [39], while Bayesian Regression and Random Forest use the standard implementations provided by scikit-learn. These models are shown in Table 2. This combination balances model diversity, generalization capability, and practical applicability for large-scale road segment prediction tasks, offering a rich set of candidates for strategy optimization.

Table 2. Six classical forecasting models.

Model Category	Model Name	Brief Description
Traditional Statistical Method	Bayesian Regression (BR)	A linear model based on Bayesian inference that can quantify prediction uncertainty. It is suitable for constructing baseline models and is implemented using scikit-learn.
Machine Learning Methods	K-Nearest Neighbors (KNN)	A prediction method based on the distance between neighboring samples. It is suitable for modeling local patterns but is sensitive to high-dimensional data.
	Support Vector Regression (SVR)	A kernel-based method for modeling nonlinear relationships, offering strong generalization capabilities.
	eXtreme Gradient Boosting (SVR)	An ensemble learning model capable of handling complex nonlinear relationships between features. It is known for its high training efficiency and stable performance.
	Random Forest (RF)	An ensemble method based on multiple decision trees, offering high robustness and noise resistance. Implemented using scikit-learn.
Deep Learning Method	Long Short-Term Memory (LSTM)	A neural network model capable of capturing long-term dependencies in time series data. It is well-suited for traffic flow prediction tasks with clear periodic or trend patterns.

4.2.4. Prediction Model Parameter Candidates

Due to the high computational cost of tuning hyperparameters for each road segment, this study introduces a clustering mechanism [16] to improve the efficiency of hyperparameter training. The specific procedure is as follows.

• Feature Clustering. First, K-Means clustering is applied to the features of each road segment. The goal is to group road segments with similar traffic dynamic characteristics into the same cluster. The optimal number of clusters is estimated using the elbow method, resulting in 14 clusters. Based on this clustering, hyperparameter configurations are determined for each model within each cluster.
• Representative Road Segment Selection. For each cluster, the road segment closest to the cluster center is selected as the representative segment. This segment is used for subsequent sub-model training and hyperparameter tuning. This approach captures diverse traffic flow patterns, enhances model generalization, and prevents the training samples from being overly concentrated in a specific pattern, effectively reducing the hyperparameter search space while maintaining diversity.
• Grid Search Hyperparameter Tuning. For the representative road segments identified above, grid search is employed to optimize the hyperparameters of the six candidate prediction models (BR, KNN, XGBoost, SVR, RF, and LSTM) separately. After obtaining the optimal hyperparameter sets for each cluster, the tuned parameters are uniformly assigned to the corresponding models for all road segments within that cluster.

The primary hyperparameters and their quantities for each prediction sub-model are listed below. The tuned parameters after training are provided in Table 3. For the complete set of parameter values across all cluster groups, see Table A1 in Appendix A.1.

Table 3. Nomenclature of model hyperparameters.

Model	Hyperparameters
BR	$\alpha 1$: prior on the precision of the weights, $\alpha 2$: prior on the precision of the noise, $l1$: regularization on the weights, and $l2$: regularization on the noise.
KNN	n: number of neighbors, w: weighting function, and algo: neighbor search algorithm.
XGboost	md: maximum tree depth, lr: learning rate, s: subsample ratio, and col: feature subsample ratio per tree.
SVR	c: regularization parameter, and e: epsilon parameter.
RF	nt: number of trees, md: maximum tree depth, and ms: minimum samples for split.
LSTM	ep: number of epochs, and lr: learning rate

4.2.5. Deep Contextual Bandit Model Hyperparameters

Directly involving all road segments in the hyperparameter search of the Deep Contextual Bandit (DCB) model would result in a substantial computational workload. To address this issue, we extend the aforementioned clustering approach: only the representative road segments derived from clustering are used in the hyperparameter tuning process. A comprehensive grid search is then performed to explore all possible combinations of hyperparameters. The final set of hyperparameters for the DCB model is presented in Table 4.

Table 4. Optimal hyperparameter values for DCB.

Hyperparameters	Value
Loss function	Mean Squared Error
Activation function	Relu
Number of layers	4
Number of neurons per layer:	128
Batch size	128
learning rate	0.0001
number of epochs	5000
Optimizer	Adam
K (Parameters in Priority Sampling)	3

These hyperparameters demonstrated good training stability and convergence across representative road segments from multiple clustering groups, thereby providing a reliable foundation for the subsequent learning and generalization of the DCB strategy.

4.3. Experimentation and Analysis

The computational performance of the proposed DCB framework was evaluated on a workstation equipped with an Intel Xeon W-2245 processor and an NVIDIA Quadro RTX 4000 GPU. In this setup, the total training process required approximately 27.07 h. Regarding resource consumption during the inference phase, the model demonstrated high efficiency, recording a peak GPU memory usage of 151.26 MB and a CPU memory increase of 1152.69 MB. Most importantly, the average inference latency was maintained at 12.6 ms per road segment, confirming that the framework is computationally lightweight and well-suited for real-time traffic flow prediction in large-scale networks.

4.3.1. Model Convergence and Prediction Performance

To evaluate the performance of the Deep Contextual Bandit (DCB) model in traffic flow prediction tasks, this study compares its performance on the test set using three key metrics: the coefficient of determination (R²), MAE, and RMSE. Figure 6 presents the aggregated trajectories of these three metrics during training and contrasts them with six sub-models: Bayesian Regression (BR), KNN, XGBoost, SVR, Random Forest (RF), and LSTM.

Figure 6. Training process of DCB.

In Figure 6, the black line represents the performance of the DCB model; The red horizontal line denotes the theoretical limit—i.e., the performance achieved by the optimal sub-model for each road segment in the test set; The colored horizontal lines indicate the performance achieved when using each corresponding sub-model alone across all segments.

• R² Performance: The red horizontal line indicates the theoretical upper bound of R². The DCB model exhibits strong convergence around the 1100th training iteration, with its R² value rapidly approaching this upper bound and remaining stable thereafter. This performance is significantly better than any individual sub-model, suggesting that DCB effectively selects more appropriate sub-models under different contextual conditions, thereby enhancing overall predictive accuracy.
• MAE Performance: The red line represents the theoretical lower bound for MAE. The MAE of the DCB model decreases steadily during training and eventually stabilizes near this lower bound, indicating strong error control capability. In comparison, other models exhibit higher MAE values and less stable behavior, lacking precision and robustness.
• RMSE Performance: The red line shows the theoretical RMSE lower bound. The DCB model again demonstrates notable advantages, with rapid convergence and lower error values. In the later stages of training, its RMSE stabilizes close to the theoretical minimum, further confirming its predictive accuracy and consistency.

Empirically, the convergence behavior observed in Figure 6 indicates that DCB achieves stability after approximately 1300 iterations, closely approaching the theoretical upper bound. To further assess robustness under non-stationarity, we analyzed the cumulative regret (Figure 7)—defined as the accumulated difference between the optimal and realized rewards. The observed sub-linear growth of the cumulative regret confirms that the learning algorithm effectively converges to the optimal policy over time. Notably, the testing phase specifically coincided with the Spring Festival, introducing severe non-stationarity through abrupt traffic surges. Even under these extreme conditions, the model maintained an R² score of approximately 80%, further validating the reliability of the bandit learning process against distributional shifts.

Figure 7. Cumulative regret curve of DCB.

In conclusion, although the reward function used during training is based solely on R², the DCB model also substantially improves other error metrics. This indicates that its strategy learning process not only enhances R² performance but also effectively balances the control of prediction errors such as MAE and RMSE, showcasing excellent multi-metric performance and strong generalization ability.

4.3.2. Accuracy of Action Selection in DCB

To evaluate model selection accuracy (i.e., action selection) within the Deep Contextual Bandit (DCB) framework, we introduce a tolerance threshold T, recognizing that in practical scenarios, the prediction performances of different models on the same road segment can be very close. Small fluctuations in R² values should not be overemphasized when judging the correctness of model selection. Therefore, when the difference (gap) in R² values between two models for a given segment is below the threshold T, their predictive performances are considered equivalent, and any model selected from this group is deemed an acceptable choice.

We define the R² scores of two candidate models $M_i$ and $M_j$ on the same road segment as $R_{M_i}^2$ and $R_{M_j}^2$, respectively, where $i,j\le K$. The gap is calculated in Equation (21).

$${gap}_{M_i,M_j}={\displaystyle \frac{\left|R_{M_i}^2-R_{M_j}^2\right|}{\max \left(R_{M_i}^2,R_{M_j}^2\right)}}\times 100\%.$$

(20)

In our experiments, we explore four threshold levels: $T\in \{2\%,3\%,4\%,5\%\}$, to assess how model selection accuracy changes with varying degrees of tolerance. This approach prevents minor performance differences from misleading the evaluation of the DCB’s decision-making accuracy and ensures a fairer assessment of its model selection strategy.

Figure 8 illustrates how the action selection accuracy of the DCB model evolves with the number of training iterations under different gap thresholds. As training progresses, the accuracy steadily increases across all values of threshold TTT, eventually stabilizing—indicating strong learning capability and convergence of the decision strategy. When T = 5%, the accuracy reaches as high as 97%; even with a stricter threshold of T = 2%, it consistently remains above 89%.

Figure 8. Accuracy of DCB.

These results demonstrate that the DCB strategy exhibits robust model discrimination and selection ability, particularly under more lenient evaluation criteria. Regardless of the chosen gap threshold, the final accuracy consistently remains high, validating the stability and effectiveness of the DCB model in selecting optimal sub-models within contextual environments.

This experiment further confirms that the DCB model not only delivers excellent predictive performance but also demonstrates strong robustness in action selection. This makes it a reliable foundation for model integration in dynamic traffic prediction scenarios.

4.3.3. Ablation Experiments

To further validate the effectiveness of each key component in the Deep Contextual Bandit (DCB) model, we conducted a series of ablation experiments by sequentially removing four core strategies from the full model and comparing their performance during training. Figure 9 illustrates the evolution of three evaluation metrics—R², MAE, and RMSE—for the original DCB model and its four ablated variants.

Figure 9. Ablation Experiment of DCB.

Specifically, the following model variants were evaluated:

• DCB_noBaseline: baseline model removal.
• DCB_noMemory: experience replay buffer removal.
• DCB_noDouble: removal of the double-model prediction mechanism.
• DCB_noPriority: priority sampling mechanism removal.

Each subplot in Figure 9 shows the overall test set metrics across training epochs:

• Total R²: The full DCB model (black curve) exhibits rapid convergence in the early training stages and maintains consistently high performance, ultimately stabilizing around 0.80. In comparison, DCB_noBaseline (cyan) and DCB_noPriority (purple) also approach the performance of the full model but converge more slowly. DCB_noMemory (green) fluctuates considerably during early training and reaches a slightly lower final R². DCB_noDouble (blue) performs the worst, with severe instability throughout training and poor convergence.
• Total MAE: The trends observed in the MAE plot are consistent with the R² analysis. The full DCB model achieves the lowest and most stable MAE (~820), reflecting superior prediction accuracy. While DCB_noBaseline and DCB_noPriority eventually reach comparable levels, they do so slowly. DCB_noMemory exhibits unstable performance, whereas DCB_noDouble suffers from persistently high and erratic error values.
• Total RMSE: The full DCB model demonstrates the best overall performance, maintaining the lowest and most stable RMSE throughout training. The DCB_noDouble variant stands out with a significantly higher RMSE and strong oscillations, while the remaining variants fall in between. Notably, DCB_noMemory remains consistently above the full model in terms of RMSE.

In summary, the ablation study confirms that all four core strategies in the DCB model contribute collaboratively to enhancing prediction accuracy and training stability. Among them, the baseline network and experience replay substantially impact model convergence and overall performance. Although the double-model prediction and priority sampling mechanisms have a relatively smaller effect, their removal still leads to notable performance degradation. These findings provide further empirical support for the robustness, rationality, and effectiveness of the full DCB model architecture.

4.3.4. Analysis of Action Set and Selection Mechanism

To explicitly distinguish the performance gains derived from dynamic model selection versus those attributed solely to model diversity, this section provides a detailed analysis of the action selection distribution. It compares the proposed framework against static ensemble strategies and model subsets.

(1)

Distribution of Selected Sub-Models

Table 5 summarizes the selection frequency of each candidate sub-model by the DCB strategy across the 100 test road segments. The distribution reveals meaningful traffic-dependent specialization learned by the agent.

Table 5. Distribution of Selected Sub-Predictive Models Under DCB Strategy.

Model	BR	KNN	XGboost	SVR	RF	LSTM
Number	32	0	41	0	18	9

As observed, XGBoost and Bayesian Regression (BR) emerged as the dominant sub-models, selected for 41% and 32% of the test segments, respectively. This preference indicates their robust fitting capabilities and generalization performance across a wide range of typical traffic scenarios. Conversely, Random Forest (RF) and LSTM were selected for 18 and 9 segments, respectively. Although less frequent, their specific selection suggests that these non-linear models provide critical performance advantages under complex dynamic or structural conditions where simpler models fail.

Notably, K-Nearest Neighbors (KNN) and Support Vector Regression (SVR) were not selected for any segment. This exclusion likely reflects their limitations in scalability and robustness within high-dimensional feature spaces, prompting the DCB agent to filter them out in favor of more efficient predictors.

(2)

Comparison with Static Ensembles and Model Subsets

To further validate the superiority of the dynamic selection mechanism, we compared the full DCB framework against: (i) restricted model subsets (Subset 1 & 2), (ii) static fusion methods (Voting and Bayesian Averaging [40]), and (iii) the baseline Cluster-based selection. The results are presented in Table 6.

Table 6. Performance Comparison of DCB vs. Static Ensembles and Model.

Method	R2 (Mean ± Std)	MAE (Mean ± Std, veh/15 min)	RMSE (Mean ± Std, veh/15 min)	Average Inference Time (s)
Subset1 (“BR”, ”XGboost”, ”RF”)	80.31 ± 0.06	836.70 ± 2.04	1309.01 ± 1.81	3.05
Subset2 (“SVR”, ”RF”, ”LSTM”)	79.52 ± 0.08	871.88 ± 9.27	1370.38 ± 13.53	3.8
Voting	80.83	845.58	1313.64	12.23
Bayesian averaging	80.99	836.46	1300.99	11.94
Cluster	80.09 ± 0.03	848.07 ± 1.16	1321.90 ± 7.91	4.02
DCB	80.59 ± 0.10	823.33 ± 1.80	1281.09 ± 4.26	1.26

(3)

Traffic-Dependent Specialization Analysis

To deeply investigate how the DCB framework adapts its selection strategy across time and space, we visualized the distribution of selected sub-models across three distinct dimensions: temporal dependency (quantified by autocorrelation), spatial topology (degree centrality), and traffic fluctuation (kurtosis).

As shown in Figure 10, the results reveal a distinct context-dependent selection pattern learned by the DCB agent. Temporally, LSTM is the preferred model for segments with high autocorrelation (57.1%), effectively capturing long-term historical dependencies, whereas XGBoost becomes dominant in low-dependency scenarios (54.3%) to capture complex non-linear mappings when historical trends are weak. Spatially, the selection strategy adapts to topological complexity. While core nodes with high degree centrality require a diverse mixture of models (with XGBoost and Bayesian Regression being equally favored, ~35% each), peripheral nodes are primarily modeled by XGBoost (54.5%). Regarding traffic fluctuation, the framework exhibits a robust mechanism for stability: in high-volatility scenarios (high kurtosis), the agent overwhelmingly selects Bayesian Regression (75.0%). This suggests that the framework prioritizes the regularization and probabilistic stability of linear models to prevent overfitting to outliers. Conversely, under stable traffic conditions, XGBoost reclaims dominance (61.9%), allowing the system to exploit fine-grained non-linear patterns when the signal-to-noise ratio is high. These correlations confirm that the DCB framework has successfully learned a “principle of adaptability”—balancing model complexity and stability according to the real-time spatiotemporal state of the traffic flow.

Figure 10. Distribution of selected sub-models across temporal, spatial, and fluctuation dimensions.

These results highlight two critical insights. First, while static ensembles (e.g., Voting, Bayesian Averaging) achieve comparable $R^2$ scores by smoothing fluctuations, DCB yields significantly lower error metrics (MAE/RMSE). This distinction indicates that dynamic selection mitigates the ‘averaging bias’ inherent in static fusion, ensuring superior point-wise precision closer to the ground truth. Second, DCB achieves this accuracy with reduced inference costs by executing a single optimal model rather than a computation-heavy ensemble. This offers a decisive practical advantage for real-time deployment in large-scale road networks, balancing high accuracy with operational efficiency.

In summary, these findings confirm that the performance gain of DCB stems from its context-aware dynamic selection capability rather than merely model diversity, making it a more efficient and accurate solution for large-scale deployment.

4.3.5. Comparison with Benchmark Models

To further validate the effectiveness of the proposed method, we compare the DCB model with several mainstream traffic flow prediction models, including Spatio-Temporal Graph Convolutional Networks (STGCN) [41], Attention-Based Spatial–Temporal Graph Convolutional Networks (ASTGCN) [42], Spatial–Temporal Synchronous Graph Convolutional Networks (STSGCN) [20], Multi-Scale Dynamic Regression (MSDR) [43], and Spatial–Temporal Wavelet Attention (STWA) [44]—all of which are representative spatiotemporal modeling approaches. The comparison results are presented in Table 7.

Table 7. Performance Comparison with Mainstream Traffic Prediction Models.

Models	R2 (Mean ± Std)	MAE (Mean ± Std, veh/15 min)	RMSE (Mean ± Std, veh/15 min)	Average Inference Time (s)
STGCN	72.76 ± 1.51	1109.66 ± 22.64	1581.12 ± 23.56	1.49
ASTGCN	80.24 ± 0.12	834.31 ± 7.95	1323.12 ± 12.09	2.46
STSGCN	78.82 ± 0.18	903.09 ± 8.01	1391.06 ± 9.25	4.61
MSDR	75.18 ± 1.00	962.27 ± 37.99	1449.34 ± 43.77	3.26
STWA	76.14 ± 0.92	1013.36 ± 32.24	1552.90 ± 44.23	2.21
DCB	80.59 ± 0.10	823.33 ± 1.80	1281.09 ± 4.26	1.26

Across the three evaluation metrics—$R^2$, MAE, and RMSE—as well as computational efficiency, the DCB model achieves the best overall performance. Specifically, it attains an $R^2$ score of 80.59 ± 0.10, which is considerably higher than all baseline methods, demonstrating its superior capability in modeling both temporal dynamics and spatial dependencies of traffic flow. Additionally, it achieves the lowest prediction errors, with an MAE of 823.33 ± 1.80 and RMSE of 1281.09 ± 4.26, where the minimal standard deviations further confirm its exceptional prediction stability compared to baselines like MSDR (±37.99 MAE) or STWA (±44.23 RMSE). Although ASTGCN performs competitively among existing methods, achieving an $R^2$ of 80.24 ± 0.12, it still falls short in terms of MAE and RMSE. Notably, regarding inference efficiency, DCB requires only 1.26 s on average, reducing the inference time by approximately 48.8% compared to ASTGCN (2.46 s) and 72.7% compared to STSGCN (4.61 s). This highlights the practical advantage of DCB in terms of predictive accuracy, robustness, and real-time feasibility in complex traffic environments.

In summary, the findings indicate that deterministically selecting a single optimal model per contextual scenario—instead of blending predictions through model averaging or weighted fusion—leads to improved prediction accuracy in heterogeneous road networks. This highlights a key advantage of the bandit-based dynamic selection framework over conventional soft fusion methods. Overall, the DCB model consistently exhibits superior performance in large-scale and heterogeneous traffic flow forecasting tasks, establishing it as a promising and practical solution for real-world dynamic traffic modeling.

5. Conclusions and Future Work

This paper proposes a methodological framework based on the Deep Contextual Bandit (DCB) for the model selection problem in large-scale traffic flow prediction tasks. The approach centers on selecting the optimal prediction model by leveraging contextual information from road segments to dynamically decide among multiple sub-models, thereby achieving accurate and efficient traffic flow forecasting. Compared to traditional static multi-model fusion methods, DCB introduces context awareness and policy learning mechanisms, resulting in stronger generalization capability and adaptability.

To further enhance model performance and training efficiency, several mechanism improvements are incorporated into the DCB framework: an experience pooling mechanism for efficient reuse of historical interaction data, a clustering-based optimization strategy to compress the hyperparameter search space and improve model transferability, and a relative gain mechanism to stabilize training. Experiments conducted on multiple real-world traffic datasets demonstrate that the proposed method outperforms traditional static multi-model approaches and some reinforcement learning baselines in key metrics such as R², MAE, and RMSE, particularly in large-scale road network scenarios where it achieves higher accuracy and stability.

Furthermore, analysis of action selection frequency reveals that DCB adaptively selects the optimal sub-model according to the contextual characteristics of individual road segments, highlighting its strong strategy discrimination ability. By demonstrating superior performance over attention-based ensembles, mixture-of-experts, and Bayesian model averaging, the DCB framework validates that a hard-selection strategy driven by contextual bandits is more effective than soft fusion for large-scale, heterogeneous traffic prediction, offering a better balance among accuracy, interpretability, and computational efficiency. Ablation studies further confirm the critical contributions of key modules—such as baseline reward, experience pooling, dual-model estimation, and prioritized experience sampling—to overall performance gains.

Nevertheless, there remains room for further improvement. Future research directions include: (1) incorporating or replacing with other advanced prediction sub-models to push the accuracy ceiling higher; (2) integrating reinforcement learning techniques such as policy gradients or actor-critic frameworks to enhance long-term decision-making; (3) employing graph neural networks to capture spatial dependencies in traffic networks better; and (4) leveraging transfer learning to improve adaptability under dynamic environmental conditions.

The innovative multi-model dynamic selection framework based on deep contextual bandits presented here offers a novel solution to the heterogeneous traffic flow prediction challenge in large-scale road networks. It effectively enhances the generalization, accuracy, and stability of traffic forecasting models, providing a promising direction for future research and practical applications. Practically, the framework enables high-precision, segment-level model adaptation. Traffic managers can leverage this capability to implement proactive congestion mitigation and dynamic route optimization, significantly enhancing urban mobility efficiency.

Appendix A.1

Appendix A.1 provides the optimal hyperparameter configurations for all candidate predictive models across the 14 identified cluster groups. Table A1 details the tuned parameter sets for Bayesian Regression (BR), K-Nearest Neighbors (KNN), eXtreme Gradient Boosting (XGboost), Support Vector Regression (SVR), Random Forest (RF), and Long Short-Term Memory (LSTM) networks. These configurations were obtained via grid search performed on the representative road segment of each cluster, as described in Section 4.2.4. The table systematically lists the key hyperparameters for every model–cluster combination, ensuring reproducibility and offering insight into how traffic-pattern heterogeneity is accommodated through differentiated parameter tuning. This appendix information supports the experimental transparency and allows readers to verify or extend the presented framework under comparable settings.

Table A1. Optimal hyperparameter values for all candidate predictive models.

Cluster Group	BR	KNN	XGboost
1	$\alpha 1=1\times {10}^{-6},\alpha 2=0.0001,$ $l1=0.0001,l2=1\times {10}^{-6}$	$n=9,w=‘distance’,$ $algo=‘auto’$	$md=6,lr=0.1,$ $s=0.8,col=0.8$
2	$\alpha 1=1\times {10}^{-6},\alpha 2=0.0001,$ $l1=0.0001,l2=0.0001$	$n=7,w=‘distance’,$ $algo=‘ball\_tree’$	$md=3,lr=0.3,$ $s=0.8,col=0.6$
3	$\alpha 1=1\times {10}^{-6},\alpha 2=0.0001,$ $l1=0.0001,l2=1\times {10}^{-6}$	$n=9,w=‘uniform’,$ $algo=‘auto’$	$md=3,lr=0.1,$ $s=0.8,col=0.8$
4	$\alpha 1=0.0001,\alpha 2=1\times {10}^{-6},$ $l1=1\times {10}^{-6},l2=0.0001$	$n=9,w=‘distance’,$ $algo=‘auto’$	$md=3,lr=0.1,$ $s=0.9,col=0.8$
5	$\alpha 1=1\times {10}^{-6},\alpha 2=1\times {10}^{-6},$ $l1=0.0001,l2=0.0001$	$n=9,w=‘distance’,$ $algo=‘kd\_tree’$	$md=3,lr=0.1,$ $s=0.8,col=0.8$
6	$\alpha 1=1\times {10}^{-6},\alpha 2=0.0001,$ $l1=0.0001,l2=1\times {10}^{-6}$	$n=9,w=‘distance’,$ $algo=‘kd\_tree’$	$md=6,lr=0.1,$ $s=0.9,col=0.8$
7	$\alpha 1=0.0001,\alpha 2=1\times {10}^{-6},$ $l1=1\times {10}^{-6},l2=0.0001$	$n=9,w=‘uniform’,$ $algo=‘ball\_tree’$	$md=3,lr=0.3,$ $s=0.7,col=0.6$
8	$\alpha 1=1\times {10}^{-6},\alpha 2=0.0001,$ $l1=0.0001,l2=1\times {10}^{-6}$	$n=9,w=‘distance’,$ $algo=‘auto’$	$md=3,lr=0.3,$ $s=0.8,col=0.8$
9	$\alpha 1=0.0001,\alpha 2=1\times {10}^{-6},$ $l1=1\times {10}^{-6},l2=0.0001$	$n=9,w=‘distance’,$ $algo=‘auto’$	$md=3,lr=0.1,$ $s=0.9,col=0.6$
10	$\alpha 1=0.0001,\alpha 2=0.0001,$ $l1=1\times {10}^{-6},l2=0.0001$	$n=9,w=‘uniform’,$ $algo=‘auto’$	$md=3,lr=0.1,$ $s=0.7,col=0.8$
11	$\alpha 1=0.0001,\alpha 2=1\times {10}^{-6},$ $l1=1\times {10}^{-6},l2=0.0001$	$n=9,w=‘distance’,$ $algo=‘kd\_tree’$	$md=3,lr=0.1,$ $s=0.8,col=0.8$
12	$\alpha 1=0.0001,\alpha 2=1\times {10}^{-6},$ $l1=1\times {10}^{-6},l2=0.0001$	$n=9,w=‘uniform’,$ $algo=‘ball\_tree’$	$md=3,lr=0.1,$ $s=0.9,col=0.8$
13	$\alpha 1=1\times {10}^{-6},\alpha 2=0.0001,$ $l1=0.0001,l2=1\times {10}^{-6}$	$n=9,w=‘uniform’,$ $algo=‘kd\_tree’$	$md=3,lr=0.1,$ $s=0.9,col=0.8$
14	$\alpha 1=1\times {10}^{-6},\alpha 2=0.0001,$ $l1=0.0001,l2=1e-6$	$n=9,w=‘distance’,$ $algo=‘kd\_tree’$	$md=3,lr=0.1,$ $s=0.9,col=0.8$
Cluster Group	SVR	RF	LSTM
1	$c=0.1,e=0.01$	$nt=150,md=30,$ $ms=2$	$ep=100,lr=0.01$
2	$c=0.1,e=2$	$nt=100,md=20,$ $ms=10$	$ep=150,lr=0.001$
3	$c=0.1,e=0.01$	$nt=150,md=10,$ $ms=10$	$ep=150,lr=0.001$
4	$c=0.1,e=0.01$	$nt=50,md=20,$ $ms=10$	$ep=150,lr=0.001$
5	$c=0.1,e=0.01$	$nt=150,md=30,$ $ms=10$	$ep=50,lr=0.001$
6	$c=0.1,e=0.01$	$nt=150,md=none,$ $ms=10$	$ep=150,lr=0.001$
7	$c=0.1,e=0.01$	$nt=100,md=20,$ $ms=10$	$ep=50,lr=0.001$
8	$c=0.1,e=2$	$nt=100,md=none,$ $ms=10$	$ep=100,lr=0.001$
9	$c=0.1,e=0.01$	$nt=150,md=10,$ $ms=10$	$ep=150,lr=0.001$
10	$c=1,e=0.01$	$nt=50,md=10,$ $ms=10$	$ep=100,lr=0.001$
11	$c=0.1,e=0.01$	$nt=100,md=20,$ $ms=2$	$ep=150,lr=0.001$
12	$c=1,e=0.01$	$nt=50,md=10,$ $ms=2$	$ep=100,lr=0.01$
13	$c=0.1,e=0.01$	$nt=150,md=10,$ $ms=10$	$ep=150,lr=0.01$
14	$c=0.1,e=0.01$	$nt=150,md=20,$ $ms=10$	$ep=150,lr=0.001$

Table A1. Optimal hyperparameter values for all candidate predictive models.

Cluster Group	BR	KNN	XGboost
1	$\alpha 1=1\times {10}^{-6},\alpha 2=0.0001,$ $l1=0.0001,l2=1\times {10}^{-6}$	$n=9,w=‘distance’,$ $algo=‘auto’$	$md=6,lr=0.1,$ $s=0.8,col=0.8$
2	$\alpha 1=1\times {10}^{-6},\alpha 2=0.0001,$ $l1=0.0001,l2=0.0001$	$n=7,w=‘distance’,$ $algo=‘ball\_tree’$	$md=3,lr=0.3,$ $s=0.8,col=0.6$
3	$\alpha 1=1\times {10}^{-6},\alpha 2=0.0001,$ $l1=0.0001,l2=1\times {10}^{-6}$	$n=9,w=‘uniform’,$ $algo=‘auto’$	$md=3,lr=0.1,$ $s=0.8,col=0.8$
4	$\alpha 1=0.0001,\alpha 2=1\times {10}^{-6},$ $l1=1\times {10}^{-6},l2=0.0001$	$n=9,w=‘distance’,$ $algo=‘auto’$	$md=3,lr=0.1,$ $s=0.9,col=0.8$
5	$\alpha 1=1\times {10}^{-6},\alpha 2=1\times {10}^{-6},$ $l1=0.0001,l2=0.0001$	$n=9,w=‘distance’,$ $algo=‘kd\_tree’$	$md=3,lr=0.1,$ $s=0.8,col=0.8$
6	$\alpha 1=1\times {10}^{-6},\alpha 2=0.0001,$ $l1=0.0001,l2=1\times {10}^{-6}$	$n=9,w=‘distance’,$ $algo=‘kd\_tree’$	$md=6,lr=0.1,$ $s=0.9,col=0.8$
7	$\alpha 1=0.0001,\alpha 2=1\times {10}^{-6},$ $l1=1\times {10}^{-6},l2=0.0001$	$n=9,w=‘uniform’,$ $algo=‘ball\_tree’$	$md=3,lr=0.3,$ $s=0.7,col=0.6$
8	$\alpha 1=1\times {10}^{-6},\alpha 2=0.0001,$ $l1=0.0001,l2=1\times {10}^{-6}$	$n=9,w=‘distance’,$ $algo=‘auto’$	$md=3,lr=0.3,$ $s=0.8,col=0.8$
9	$\alpha 1=0.0001,\alpha 2=1\times {10}^{-6},$ $l1=1\times {10}^{-6},l2=0.0001$	$n=9,w=‘distance’,$ $algo=‘auto’$	$md=3,lr=0.1,$ $s=0.9,col=0.6$
10	$\alpha 1=0.0001,\alpha 2=0.0001,$ $l1=1\times {10}^{-6},l2=0.0001$	$n=9,w=‘uniform’,$ $algo=‘auto’$	$md=3,lr=0.1,$ $s=0.7,col=0.8$
11	$\alpha 1=0.0001,\alpha 2=1\times {10}^{-6},$ $l1=1\times {10}^{-6},l2=0.0001$	$n=9,w=‘distance’,$ $algo=‘kd\_tree’$	$md=3,lr=0.1,$ $s=0.8,col=0.8$
12	$\alpha 1=0.0001,\alpha 2=1\times {10}^{-6},$ $l1=1\times {10}^{-6},l2=0.0001$	$n=9,w=‘uniform’,$ $algo=‘ball\_tree’$	$md=3,lr=0.1,$ $s=0.9,col=0.8$
13	$\alpha 1=1\times {10}^{-6},\alpha 2=0.0001,$ $l1=0.0001,l2=1\times {10}^{-6}$	$n=9,w=‘uniform’,$ $algo=‘kd\_tree’$	$md=3,lr=0.1,$ $s=0.9,col=0.8$
14	$\alpha 1=1\times {10}^{-6},\alpha 2=0.0001,$ $l1=0.0001,l2=1e-6$	$n=9,w=‘distance’,$ $algo=‘kd\_tree’$	$md=3,lr=0.1,$ $s=0.9,col=0.8$
Cluster Group	SVR	RF	LSTM
1	$c=0.1,e=0.01$	$nt=150,md=30,$ $ms=2$	$ep=100,lr=0.01$
2	$c=0.1,e=2$	$nt=100,md=20,$ $ms=10$	$ep=150,lr=0.001$
3	$c=0.1,e=0.01$	$nt=150,md=10,$ $ms=10$	$ep=150,lr=0.001$
4	$c=0.1,e=0.01$	$nt=50,md=20,$ $ms=10$	$ep=150,lr=0.001$
5	$c=0.1,e=0.01$	$nt=150,md=30,$ $ms=10$	$ep=50,lr=0.001$
6	$c=0.1,e=0.01$	$nt=150,md=none,$ $ms=10$	$ep=150,lr=0.001$
7	$c=0.1,e=0.01$	$nt=100,md=20,$ $ms=10$	$ep=50,lr=0.001$
8	$c=0.1,e=2$	$nt=100,md=none,$ $ms=10$	$ep=100,lr=0.001$
9	$c=0.1,e=0.01$	$nt=150,md=10,$ $ms=10$	$ep=150,lr=0.001$
10	$c=1,e=0.01$	$nt=50,md=10,$ $ms=10$	$ep=100,lr=0.001$
11	$c=0.1,e=0.01$	$nt=100,md=20,$ $ms=2$	$ep=150,lr=0.001$
12	$c=1,e=0.01$	$nt=50,md=10,$ $ms=2$	$ep=100,lr=0.01$
13	$c=0.1,e=0.01$	$nt=150,md=10,$ $ms=10$	$ep=150,lr=0.01$
14	$c=0.1,e=0.01$	$nt=150,md=20,$ $ms=10$	$ep=150,lr=0.001$