A Distributed Machine Learning-Based Scheme for Real-Time Highway Traffic Flow Prediction in Internet of Vehicles

Abstract

Abnormal traffic flow prediction is crucial for reducing traffic congestion. Most recent studies utilized machine learning models in traffic flow detection systems. However, these detection systems do not support real-time analysis. Centralized machine learning methods face a number of challenges due to the sheer volume of traffic data that needs to be processed in real-time. Thus, it is not scalable and lacks fault tolerance and data privacy. This study designs and evaluates a scalable distributed machine learning-based scheme to predict highway traffic flows in real-time. The proposed system is segment-based where the vehicles in each segment form a cluster. We train and validate a local Random Forest Regression (RFR) model for each vehicle’s cluster (highway-segment) using six different hyper parameters. Due to the variance of traffic flow patterns between segments, we build a global Distributed Machine Learning Random Forest (DMLRF) regression model to improve the system performance for abnormal traffic flows. Kappa Architecture is utilized to enable real-time prediction. The proposed model is evaluated and compared to other base-line models, Linear Regression (LR), Logistic Regression (LogR), and K Nearest Neighbor (KNN) regression in terms of Mean Square Error (MSE), Root Mean Square Error (RMSE), Mean Absolute Error (MAE), R-squared (R²), and Adjusted R-Squared (AR²). The proposed scheme demonstrates high accuracy in predicting abnormal traffic flows while maintaining scalability and data privacy.

1. Introduction

Transportation systems provide huge amounts of data that can be obtained from different resources in real-time such as sensors, detectors, smart cards, and video cameras [1,2,3]. By utilizing the big traffic data, researchers can discover the reasons for traffic accidents and congestion, redundant traffic jams in certain areas, traffic delay at intersections, and several redundant issues on transportation systems [4,5,6,7,8,9,10,11]. However, the amount of traffic data increases rapidly to the level of Petabyte, which makes centralized machine learning prediction methods [12,13] face a major drawback in terms of scalability limitation.

Several big data techniques are utilized for processing and mining such as Artificial Intelligent tools, Machine Learning, and Data Mining [14]. Big data techniques are used in different fields of study, and researchers achieved outstanding results and success [15,16,17,18]. In transportation, decision makers should provide optimal solutions in real-time since vehicles are moving very fast. Apache Spark (3.x) and Hadoop (3.3.x) can process and store large amounts of data with a low computation time due to Hadoop Map-Reduce and Spark in-memory parallel distributed computing. These two big data platforms are used widely in industry and academia [19,20]. Studying the redundant issues of the traffic flow in historical data, helps big data scientists to provide optimal traffic solutions for drivers [21,22,23,24,25,26,27,28].

This research proposes a scalable real-time distributed machine learning-based scheme for Internet of Vehicles to predict highway traffic average speed, and detect abnormal traffic flow patterns. The proposed system consists of two main stages, which are the learning stage and the real-time stage. In the learning stage, a highway is divided into small complex segments and each Complex Segment (CS) consists of several segments. Each CS is located between two exits. We train and validate a local Random Forest Regression (RFR) model for each vehicle cluster (highway segment) using six different hyper-parameters: max depth, min sample split, min samples leaf, number of estimators, bootstrap sample, and max features. Randomize Search Cross Validation (RSCV) technique is utilized to determine the six hyper-parameters’ values of each segment’s model. By utilizing Stacking ensemble learning, all RFR models are used as inputs to build the Distributed Machine Learning Random Forest (DMLRF) model. Kappa Architecture (KA) is utilized to enable real-time traffic flow predictions [29]. The proposed system is implemented in Google Cloud Platform (GCP), which contains a private computing cluster with 17 nodes where 16 nodes were assigned as worker nodes (each highway’s segment is assigned to one computing node) and one node will be the master node that controls the processing [30]. In the real-time stage, the Discretized Stream (DStream) of Apache Spark Streaming is utilized to process real-life traffic data records and store them as batches. The real-time stream records will be sent to the streaming layer of KA where Spark Streaming can process the real-life data and then use the saved DMLRF model to provide real-time traffic flow predictions. This research utilizes real-life traffic data obtained from Florida Department of Transportation District 4 (FDOT-D4) [31]. The contribution of this study can be organized as follows:

• Build a scalable distributed machine learning system that can predict highway traffic flows. The scheme divide a highway into small complex segments, where each complex segment is located between two exits to control the level of congestion and allow vehicles to exit the highway before entering the congestion zone.
• Build a distributed machine learning model to improve the system performance for abnormal traffic flows. The distributed machine learning model is trained based on different hyper-parameters’ values, where these values are determined based on the traffic flow patterns for each segment of the complex segment. So each segment has a model with its adjusted hyper-parameters’ values based on RSCV technique. Then all models participate to build the proposed distributed learning model. Stacking ensemble learning method is utilized to build the DMLRF model.

The remainder of this paper is organized as follows, Section 2 describes the most recent related work. Section 3 presents the preliminaries of this study. Section 4 shows the research methodology, model design, and challenges. Section 5 demonstrates details of a real-life traffic dataset and all preprocessing steps. Section 6 presents our distributed machine learning architecture workflow and its components. Section 7 shows the performance evaluation and results. Section 8 concludes the paper and provides the future work.

2. Related Work

Distributed machine learning architecture has attracted researchers in the past due to its high scalability, and reliability. However, very limited papers have discussed traffic flow prediction while using distributed machine learning and parallel analysis to deal with big traffic data challenges. Providing an optimal traffic flow prediction for a large-scale traffic network where numerous vehicle detection systems are geographically installed on highways to record traffic and collect its data is challenging.

Recent studies have investigated diverse Artificial Intelligent (AI) and deep learning techniques for improving traffic flow prediction and event detection in vehicular networks. Tan et al. [32], presented a comprehensive analysis of machine learning applications in vehicular networks, including mobility management, handover optimization, and predictive routing to address dynamic network challenges in high-mobility scenarios. Olugbade et al. [33], explored AI and ML-based incident detection systems in road transport, highlighting significant advancements in real-time vehicle monitoring, route optimization, and predictive fleet maintenance, all of which enhance road safety. Abdullah et al. [34], proposed a soft GRU-based recurrent neural network model for predicting congestion in smart cities, utilizing historical traffic data and sensor inputs to enhance traffic flow and reduce congestion. These studies demonstrate the growing role of AI-driven methodologies in optimizing traffic management and improving transportation efficiency.

A distributed machine learning method was developed in [35] to predict traffic flow and deal with big traffic data challenges. A deep convolutional neural network model was implemented to be used in their parallel training approach. The framework was implemented by using Apache Spark cluster to perform parallel distributed machine learning. Real traffic data was obtained from California Department of Transportation Performance Measurement System (PeMS) to be utilized for this study. The main dataset, collected from 12 highways, was decomposed into sub datasets to be utilized by different Spark nodes (machines) to perform parallel computations. 9 months of traffic data records in 2016 were used for training and one month was used for testing. The models trained on the sub datasets and provided local training parameters that were sent to the master node to average these parameters and provide global learning parameters. These global learning parameters were sent back to the slave nodes as sub-sequence iteration processes. This process continued until the best accuracy was achieved. The performance of the model was evaluated by using MSE, RMSE, and MAE in different time intervals. Based on the results, the proposed model improved the efficiency of the training process.

A Distributed Automatic Long Short-Term Memory Customization Algorithm (DALC) for predicting highway traffic speed flow was developed in [36]. The authors started with designing the algorithm for a single detector based on Markov decision process formulation to define the best two hyperparameters manually. The two hyperparameters were the number of hidden layers and epochs. Based on the Markov decision process formulation, they designed the ALC algorithm for a single detector. Then they proposed that each detector should have its own algorithm that can work in parallel due to different traffic speed patterns. After analyzing five weekdays of traffic speed records starting from 16 October 2017, the traffic patterns in the morning were different than the traffic patterns in the night. So, they proposed that each detector should have two models. They utilized a dataset obtained from California Department of Transportation Performance Measurement System (PeMS) that was collected from 60 detectors. They utilized Apache Hadoop YARN and Spark to build a private cluster that consisted of 30 nodes where each node had two DALC algorithms. Their algorithm was compared with five different distributed machine learning algorithms that were built by Apache Spark where these algorithms utilized the same dataset. The proposed algorithm outperformed the other algorithms in terms of average absolute relative error.

To implement a traffic flow prediction model that satisfies real world application requirements, Boukerche, and Wang focused on improving the prediction model features by implementing a hybrid deep learning model that combines Graph Convolutional Network (GCN) and the deep aggregation structure to predict traffic flow [37]. This approach used parallel training that combined several edge devices with a central system to improve the accuracy and efficiency of the system. To improve the performance of the model during different lengths of time lag, a refinement learning approach was applied. The proposed system used traffic data obtained from PeMS and was evaluated in terms of MAE, R square, and RMSE.

In [38], the authors propose hybrid designs that integrate Convolutional Neural Networks (CNNs), Graph Convolutional Networks (GCNs), and transformer-based models for large-scale road networks. These hybrid models effectively combine spatial and temporal analyses, leveraging attention mechanisms to dynamically capture dependencies in traffic data. The study highlights significant improvements in prediction accuracy and scalability, demonstrated through evaluations on datasets such as PeMS-Bay and Metr-La, with models like SAGCN-SST and GMAN showing notable performance.

Alsubai et al. [39], present the Improved Arithmetic Optimization Algorithm-based Deep Learning Traffic Congestion Control (IAOADL-TCC) model, which employs a hybrid CNN-LSTM architecture to optimize traffic flow prediction. The model applies a hyperparameter tuning approach using arithmetic optimization to enhance the CNN-ALSTM framework, achieving a notable average accuracy of 98.03% on a Kaggle road traffic dataset. The IAOADL-TCC model outperformed previous approaches, including UAV-CNN, SVM, and fuzzy logic models, by effectively capturing spatio-temporal patterns. Another study in [40], explores the development of traffic flow prediction methods, transitioning from statistical models like ARMA to advanced deep learning approaches such as LSTM, CNN, and GCN. Hybrid models, including IGA-LSTM and DAFF-Net, have improved effectiveness by capturing spatio-temporal patterns and addressing inefficiencies. GCN-based models, such as FAST-GCN, effectively manage non-Euclidean traffic data, leading to notable improvements in accuracy. However, challenges persist in handling sparse data and ensuring real-time adaptability.

Fouladgar came up with a decentralized deep learning method in order to come up with scalable and real-time feedback of traffic flow predictions [41]. They used dataset obtained from Caltrans Performance Measurement System (CPMS) in California where data of 48 days from 51 different locations were used to train their model and the next 12 days were selected for validation. To avoid the class imbalance problem of their dataset where heavy traffic was significantly lower than the light traffic, they obtained a regularized euclidean loss function. CNN and LSTM were used to build their decentralized deep learning architecture. The accuracy of both models was high, but it decreased during rush hours due to the unbalanced dataset.

An improved hybrid traffic flow prediction was proposed in [42]. The hybrid model combined a decomposition and prediction process to predict traffic flow. The process started as follows: a decomposition ensemble empirical mode was utilized to decompose an original nonlinear and complex highway traffic flow data. Then, a new reconstructed component was obtained from an improved weighted permutation entropy (IWPE). To create the reconstructed component, a gray wolf optimizer was used to optimize the least-squares support vector machine. The proposed model provided useful traffic flow information based on their analysis and results.

Building on these studies, this research proposes a scalable, real-time distributed machine learning-based scheme for the Internet of Vehicles to predict highway traffic average speed and detect abnormal traffic flow patterns. The application of distributed machine learning to highway traffic flow prediction is still in its infancy. Only a small fraction of existing work utilize distributed machine learning and build their scheme on a local level of modeling using small datasets. In this study, we propose a scalable distributed machine learning-based scheme to predict highway traffic flows in real-time. The system’s models are trained on the first seven year’s data and validated on the last year’s data. To enhance the performance of the model for the abnormal traffic flows, we introduced a global stage where the model is further trained on different hyper parameter’s values based on the different traffic flow patterns. Randomize Search Cross Validation is performed to determine the hyper parameter’s values due to its efficiency of tuning random forest regression model’s hyper-parameters.

3. Preliminaries

3.1. Kappa Architecture

Jay Kerps introduced the Kappa Architecture (KA) in 2014 to reduce the code overwhelming of the Lambda Architecture [29,43]. Lambda architecture is a scalable, fault-tolerant framework with three layers. The batch-processing layer aggregates historical data but is time-consuming, while the stream-processing layer handles real-time data for quick insights. The serving layer merges both outputs for efficient querying. KA consists of one pipeline machine learning process, where all the data is considered as a stream. KA eliminates the need for the batch layer of lambda architecture, which consists of stream and service layers only. KA as represented in Figure 1 works as follows: real-time data will be processed by a message passing system such as Kafka to create the real-life streaming and send it to the streaming layer. In the streaming layer, Apache Spark ML-lib can be utilized to build machine learning models for real-time predictions. The results of machine learning will be sent to the service layer where all the results can be stored in most of the well-known storage management systems such as Hadoop Base, Cassandra, and MySQL. In the service layer, real-time analysis and decisions will be provided to users. We use KA in our research to build a real-time prediction system. Apache Spark and Spark Streaming component can be used to implement KA, which is a big data tool engine that includes different Application Programming Interfaces (APIs) such as MLlib, Spark Streaming, Spark SQL, and Spark Graphics [44]. Spark has been used recently to build big data applications for ITS due to its ability to process large scale distributed data in real-time. Spark is known for in-memory computation and the capability of processing the data 100 times faster than Apache Hadoop. In every Spark application, a Spark context will be created to manage all tasks. Once Spark context creates Resilient Distributed Datasets (RDDs), the cluster manager will distribute the RDDs among the available worker nodes in the cluster to be processed in parallel and will provide the results in real-time. Figure 2 demonstrates Apache Spark architecture.

3.2. Ensemble Learning and Hyper-Parameters Tuning

3.2.1. Ensemble Learning

Most modern applications require their problems to be solved with high accuracy, which can be achieved by using a centralized machine learning architecture. However, when the centralized method is not an option to solve a big data problem and the distributed machine learning is capable of dealing with it, the learning process is carried out in two stages: local stage and global stage. In the local stage, the machine learning models are expected to perform low than a centralized learning model that has access to all the data. In the global stage, the machine learning models are aggregated using ensemble method to improve the performance [45]. More information on ensemble learning can be found in [46]. The Stacking ensemble learning technique is applied in our proposed architecture to build our distributed learning model. The following are well-known ensemble methods [47].

• Boosting: When some data are misclassified by a model during the training process, a new model will train the misclassified data in order to improve the classification accuracy.
• Bagging: Is the process of combining different machine learning models into one model.
• Bucketing: Selecting the best model that achieved the highest accuracy evaluation metrics.
• Stacking: Is the process of reducing the variance of machine learning classifiers by taking the output of multiple classifiers/regressors as inputs of a new classifier/regressor.
• Random Forests: Increases the overall performance by averaging the prediction of multiple decision trees into an individual tree [48].

3.2.2. Hyper-Parameters Tuning

Is considered as a technique to improve machine learning algorithm’s performance. Choosing hyper-parameters and values of these parameters can be done randomly. However, randomize search cross validation and grid search cross validation can be adopted to determine the best values for the selected hyper-parameters automatically [49,50]. These two techniques test an algorithm several times based on the number of cross validations for each combination of hyper-parameter values. After a certain number of iterations, the best hyper-parameter’s values for the used dataset are determined and utilized for training the algorithm. The difference between these two approaches is that grid search cross validation consumes more time in determining the values of hyper-parameters since it validates all the values in a grid search manner. However, RSCV selects the values randomly. The RSCV is applied in our system to select the best hyper-parameters values for our local random forest models. More details on these two techniques are available in this study [51].

4. Research Methodology

4.1. Research Methodology

Intelligent Transportation Systems aim to provide a high level of safety and efficiency for drivers, and monitoring highway traffic in real-time is one of its highest priority applications. However, highways extend for hundreds of miles and vehicles move very fast making monitoring and predicting traffic flow patterns in real-time by utilizing a centralized system very challenging [52,53]. This research proposes a scalable distributed machine learning-based scheme for Internet of Vehicles to predict highway traffic average speed and detect abnormal traffic flow patterns. The experimental environment included the use of Python (3.10) for data preprocessing, Google Cloud Platform (GCP) for deployment, and Apache Spark for data processing and real-time analytics using Kappa Architecture. The proposed model was evaluated using Mean Squared Error (MSE), Root Mean Squared Error (RMSE), Mean Absolute Error (MAE), R-Squared (R²), and Adjusted R-Squared (AR²) metrics. We assume that the I-95 highway is divided into Complex Segments (CSs) where each CS is located between two exits as represented in Figure 3. Each CS consists of 4 segments and each segment has a Road Side Unit (RSU). In each segment (cluster), a leader vehicle (cluster head) will collect traffic information from the vehicles in that segment using Vehicle to Vehicle (V2V) communication. This information will be sent to the RSU, where machine learning models are implemented and ready to provide prediction results in real-time. This method allows us to break down a complex problem into small sub-problems where different machine learning algorithms can collaborate and provide the traffic prediction results in parallel. Since the geographical location of each cluster provides different traffic flow patterns, different learning algorithms trained on different hyper-parameter’s values are utilized to build a distributed machine learning model. These hyperparameter’s values are determined based on the different traffic flow patterns. So, each segment has specific hyper-parameter’s values based on its historical traffic flow data. The block diagram in Figure 4 provides an overview of the DMLRF model’s structure for predicting the average speed of highway traffic. More details on the proposed model is available in Section 6.

4.2. Model Design

4.2.1. Random Forest Regression

Random forest regression combines a large set of regression trees and considers as an ensemble learning algorithm. Starting from a root to a leaf of the tree, a set of conditions that are hierarchically organized represents by a regression tree [54]. From the original training dataset, the random forest trains bootstrap samples using sub-datasets. Algorithm 1 works as follows: for each tree in the forest, a bootstrap sample is selected from T where $T^{\left(i\right)}$ denotes the ith bootstrap. Then, using a decision tree to learn a modified decision tree. The modification steps works as follow: at each node of the tree, some subset of the features $z\subseteq Z$ where Z is the set of features are selected to examine a possible split instead of using all the features. Rather than splitting the node on Z, the algorithm use z to define the split on the best features. Determining on which feature to split is the most computation expensive of decision tree learning. By averaging all the trees, the predicted value of an observation is calculated. Algorithm 1 is utilized to build the base models in Step 1(b) of Algorithm 2.

Algorithm 1 Random Forest Base-line Model

Precondition: A training set T: = $(x_1,y_1),....,(x_n,y_n)$, features Z, and number of tree in forest N. 1 function Random-Forest(T, Z) 2     $H\leftarrow \varnothing$ 3     for $i\in 1,.....,B$ do           $T^i\leftarrow$ a bootstrap sample of T           $D{T}_i\leftarrow$ RandomizeTreeLearn $(T^i,Z)$           $H\leftarrow H\cup \left\{D{T}_i\right\}$ 4   end for 5 return H 6 end function 7 function RandomizeTreeLearn $(T,Z)$    At each node:    $z\leftarrow$ very small subset of Z    split on best feature in z 8 return the learned tree 9 end function

Algorithm 2 DMLRF model creation

Input: Split traffic dataset D into K subsets D = {$D_1,D_2,D_3,....,D_k$}, such that $D_i$ is the data subset of Segment i. Combine segments between exits into J complex segments Output: A Distributed Machine Learning Random Forest (DMLRF) model for real-time traffic flow prediction. 1 Step 1: for i ← 1 to K: (a) determine the hyperparameters’ values using RSCV for data subset $D_i$. (b) First level training: train random forest regression (RFR) model for data subset $D_i$. End for 2 Step 2: for i ← 1 to J Second level training: stack the output of individual estimators (RFR) and use a meta model (DMLRF) to compute the final prediction for Complex Segment i. End for 3 return DMLRF

4.2.2. Stacking Ensemble Learning

In this section, we will discuss the proposed prediction model. As we can see from Algorithm 2, the main traffic dataset D is divided into sub datasets D = {$D_1,D_2,D_3,.....,D_k$} where K denotes the number of segments, and each segment’s input parameters are total volume, average occupancy, and average delay. The output parameter of the segments is average speed. For each segment in ($D_k$), an RFR model will be utilized as a base regressor as demonstrated in Algorithm 1. In Step 1, the RFR models utilized the adjusted hyperparameter’s values that are determined based on the RSCV technique to train the base RFR models. Due to the variance between traffic flow patterns of sub datasets, the hyperparameter’s values for RFR models are different. In Step 2, meta regression model is developed. Stacking ensemble learning technique is used to build the DMLRF model, which combines the outputs of individual estimators (RFRs) and uses a meta model (DMLRF) to compute the final prediction where this model is trained by using cross-validated predictions of the base estimators [55]. This technique takes the output of RFR models as input of DMLRF model.

4.3. Research Challenges

The first challenge is to analyze the big traffic data. For instance, our traffic data exceeds 18 million traffic flow records obtained from 16 detectors that are located on Florida I-95 highway. The second challenge is to maintain high accuracy for the utilized models due to the distributed architecture where the main data will be divided into small datasets. So, the machine learning models will be trained on small fractions of the main dataset. The third challenge is the high need for computational resources. Since our architecture is based on a distributed learning framework, every segment (cluster) has a machine learning model that needs to be implemented and evaluated, which consumes more computational resources. So, each segment has a separate worker node (machine) that is responsible for all the analysis and evaluations of its machine learning model.

5. Data Description and Preprocessing

5.1. Data Description

In this research, we utilize a real-life traffic dataset, which we directly obtained from the Florida Department of Transportation District 4 Regional Transportation Management Center (FDOT-D4 RTMC). It is commonly referred to as SunGuide traffic data. While it is not publicly available, it can be requested from FDOT-D4 RTMC. Over 400 vehicle detection systems were used to collect the traffic data from three main highways of Florida (I-95, I-75, and I-595). The detectors are located on every half a mile on the three highways in both directions. The highway traffic flow dataset covers 8 years of traffic’s records starting from 2008 until November of 2015. It provides real-time traffic flow information including, occupancy, average speed, total volume, average delay, and other valuable information for each lane of the freeways. The FDOT-D4 collected the traffic data based on a 15-min time interval, which gave us more opportunity for subsequent data processing.

Table 1. Important Parameters Of Traffic Data.

#	Parameters	Description
1	Detector ID	The ID number of the detector
2	Total Volume	The total number of vehicles detected within a 15-min interval that was recorded by the available detector.
3	Average Speed	The average speed of vehicles that are calculated based on a 15-min interval for all the lanes.
4	Archive Lane ID	ID of each lane on each bound of the highway. Each lane on each segment has a different lane id.
5	Average Occupancy	Average number of vehicles that occupy the segment.
6	Detector Lat and log	Latitude and Longitude of the detector.
7	Traffic Timestamp	The time-stamp of the traffic record (Year-Month-Day, Hour:Minute:Second).
8	Average Latency	The average latency that recorded by a detector.

illustrates the important parameters and their descriptions of road traffic records. In this study, we focused on traffic flow data obtained from 16 detectors that are located on I-95 highway for 8 years starting from 2008. The total number of traffic flow observations of all detectors is 18,812,799 million records.

5.2. Data Preprocessing and Transformation

To provide accurate prediction results for any machine learning algorithm, data preprocessing and transformation are very essential steps that have to be considered. Removing null values and detecting outliers will impact algorithm’s accuracy dramatically. Traffic data usually suffer from class imbalance issues, which can be fixed by applying over-sampling or under-sampling techniques such as SMOTE, Near-Miss, and ROSE [56,57,58]. More details on preprocessing and transformation for our data-sets are available in our previous studies [59,60]. This research proposes a scalable distributed machine learning-based scheme for the Internet of Vehicles to predict highway traffic average speed and detect abnormal traffic flow patterns. We trained and validated our proposed model using seven years of traffic data from 2008 to 2014, which constitutes 99.8% of the total dataset. The training-validation split was 80% for training and 20% for validation. We then tested our model on one week of traffic flow patterns from 2015, which accounts for 0.2% of the original dataset.

6. Proposed System Architecture and Components

6.1. Proposed System Architecture

The proposed system consists of two main stages, which are the learning stage and the real-time stage. The main task of the learning stage is to build and update our proposed DMLRF algorithm. As demonstrated in Figure 5, the FDOT-D4 traffic historical dataset is divided into small parts based on available detectors. Each detector covers half a mile, so the highway is divided into half-mile segments. RFR algorithm is assigned for each segment, and each algorithm has six different hyper-parameters. The hyper-parameters are max depth, min sample split, min samples leaf, number of estimators, bootstrap sample, and max feature. A full description for each hyper parameter is available in

Table 2. Random Forest hyper-parameters description.

Hyper-Parameters	Description
N estimators	Number of trees in the forest
Max features	Max number of features considered for splitting a node
Max depth	Max number of levels in each decision tree
Min samples split	Min number of data points placed in a node before the node is split
Min samples leaf	Min number of data points allowed in a leaf node
Bootstrap	Method for sampling data points (with or without replacement)

. RSCV technique is utilized to determine the six hyper-parameter’s values of each segment’s model. Due to the variances of highway traffic flow data patterns in these segments, each RFR algorithm will be trained on different hyper-parameter values. By utilizing Stacking ensemble learning, all RFR models are used as inputs to build the DMLRF model. So, the DMLRF model will be built based on different hyper-parameters’ values, which makes it a distributed machine learning model.

In the real-time stage as illustrated in Figure 5, the Discretized Stream (DStream) of Apache Spark Streaming is utilized to process real-life traffic data records and store them as batches. The real-time stream records will be sent to the streaming layer of KA where Spark Streaming can process the real-life data and then use the saved DMLRF model to provide real-time traffic flow predictions. Prediction results will be sent to the serving layer of KA for analysis. In the serving layer, the system will calculate the Speed Reduction Index (SRI) value to determine a threshold value. The SRI formula returns values from 0 to 10 where 0 means very normal traffic flow and 10 means vehicles are not moving. 4 is the beginning of congestion, which is our threshold value. If the SRI value exceeds the threshold value, the system will calculate the delay time of traffic on each segment. A total delay time will be calculated for multiple complex segments located adjacent and between two exits in a highway to be shared with vehicles that are still out of the calculated area. The Internet of Vehicles communication schemes can be utilized to send and receive traffic beacon information (Average Seed, Total Volume, and Average Delay) between a cluster head and cluster nodes. Approaching vehicles will decide to either stay on the highway or use alternative roads based on the results. If the SRI value exceeds 9 for each segment, the total delay may exceed one hour and 30 min for some complex segment of the I-95 highway. The SRI formula is presented in Equation (1). The results of the traffic flow predictions are transmitted and used to update the historical database, ensuring that the system remains consistently updated.

$$SRI=1-(1-{\displaystyle \frac{AveragePredictedSpeed}{Speedlimit}}\times 10)$$

(1)

6.2. Proposed System Components

Our proposed architecture consists of two components and software modules. The ultimate objective of this system is to provide a high level of safety and increase the efficiency of the transportation system. The following points summarize the system components:

• Kappa Architecture: KA is a real-time streaming component that can process the real-life data that is obtained from a cluster (segment) head node. This is a one pipeline machine learning process that consists of two main layers: The streaming and serving layers.
• Distributed machine learning model: It is the proposed model that is built based on different random forest algorithms with different hyper-parameters values. The Stacking ensemble learning approach is utilized to build our proposed model.

7. Performance Evaluation and Results

In this section, we demonstrate the efficiency of our model’s performance on abnormal traffic flows.

7.1. Classification Models

We have evaluated our models based on the most related features to the target parameter that are extracted by using Boruta Feature Selection method [61]. RFR algorithm was adopted in this study due to its efficient performance and its capability of feature selection where this algorithm achieved outstanding results in our previous studies [59,60]. After data preparation, the RFR algorithms are trained on 18,812,799 traffic flow records for 16 detectors separately where the target parameter is average speed and the input parameters are total volume, average occupancy, and average delay. Stacking ensemble learning approach was adopted to implement the DMLRF model where the RFR models are used as inputs to train the final model based on different hyper-parameter values. In this study, we implemented other regression base line models to compare it with our proposed model. The models are Linear Regression (LR), Logistic Regression (LogR), and K Nearest Neighbor (KNN). All the Models are trained validated on seven years of traffic data starting from 2008 to 2014 and one week was utilized for testing and is obtained from 2015, which is from 11 June 2015 to 17 June 2015. This week was selected for having more abnormal traffic flow cases.

7.2. Performance Metrics

Mean Squared Error (MSE), Root Mean Squared Error (RMSE), R Squared, Adjusted R Squared, and Mean Absolute Error (MAE) are utilized as evaluation metrics to compare different prediction models. All the evaluation metrics are explained in Equations (2)–(6).

$$MSE={\displaystyle \frac{1}{n}}\sum _{i=1}^n{\left(X_i-Z_i\right)}^2.$$

(2)

where n represents a number of data points, $X_i$ presents the observed values, and $Z_i$ shows the predicted values. MSE measures the average of squared errors between the prediction results and actual values.

$$RMSE=\sqrt{{\displaystyle \frac{1}{n}}{\mathsf{\Sigma }}_{i=1}^n{\left(X_i-Z_i\right)}^2}$$

(3)

where $Z_i$ are the observations, $X_i$ are the predicted values of a variable, and n represents the number of observations. RMSE increases the error rate when the difference between the prediction results and actual values of data is bigger

$$MAE={\displaystyle \frac{{\sum }_{i=1}^n|y_i-x_i|}{n}}$$

(4)

where $y_i$ represents the predicted value, $x_i$ presents the true value, and n is the total number of data points. The MAE illustrates the difference between the prediction results and the real data.

$${\mathrm{R}}^2=1-\left[\sum _{\mathrm{i}=1}^{\mathrm{n}}{\displaystyle \frac{{\left({\mathrm{y}}_{\mathrm{i}}-{\mathrm{x}}_i\right)}^2}{{\left({\mathrm{y}}_{\mathrm{i}}-{\mathrm{x}}_i\right)}^2}}\right]$$

(5)

R-squared or ($R^2$) is used to define the percentage of variance for the predicted value (Y) from the independent variable (X). The output values range from 0 to 1, which mean that they range from 0% to 100%.

$$Adjusted{R}^2=1-\left({\displaystyle \frac{(1-R^2)(N-1)}{N-p-1}}\right)$$

(6)

Adjusted $R^2$ shows how a regression model is impacted by using useless variables. If the regression model used an unrelated variable, the output value will decrease. Where (N) represents the number of samples in a data set, and (P) is the number of features.

7.3. Results and Discussion

Our proposed distributed machine learning system effectively enhances traffic flow prediction. Our proposed system achieved notable results of traffic flow prediction for future traffic forecasting especially on predicting abnormal traffic flow. In this study, we evaluated 4 complex segments and each complex segment has 4 segments and is located between two exits. Each segment covers half a mile of distance, so each CS consists of 2 miles. Normal travel time of half a mile is 28 s if a vehicle is traveling at 65 Mph based on Equation (7). So normal travel time for one CS is 112 s or almost 2 min.

$$NormalTravelTime=(D_c/S)\times 3600$$

(7)

where $Dc$ is the length of a segment, which is 0.5 mile. S represents the speed limit, which is 65 Mph. 3600 is the total number of seconds in one hour.

The effective performance of the DMLRF model in predicting both high and low values can be attributed to robust data preprocessing, segment-based modeling, optimized hyper-parameters using Randomized Search Cross-Validation (RSCV), and the use of Stacking ensemble learning. These factors collectively contribute to the model’s ability to accurately capture the unique traffic patterns across different segments.

7.3.1. Complex Segment 1

The proposed model achieved the highest results where the $R^2$ and adjusted $R^2$ achieved over 80 percent of accuracy in terms of predicting highway traffic flow for Segment C of CS1. We observed that traffic flow patterns in this complex segment become more abnormal for the segments that are close to the exit. Figure 6 shows the DMLRF predicted delay time compared with actual delay time for all segments of CS1. The results for Segment C of Complex Segment 1 were compared with those of other baseline regression models in terms of predicting the weekly traffic flow across all complex segments, as shown in Table 3. The proposed model outperform the other models in all the segments. The proposed model achieved 85% in terms of $R^2$ and $A{R}^2$ for Segment C, which is the highest accuracy. Moreover, the proposed model achieved the lowest error in terms of MSE, RMSE, and MAE, which is presented in Segment B. Figure 7 compared the performance of the DMLRF with actual data of traffic flow for all segments in CS1. Figure 8 presents the performance of DMLRF model in terms of predicting a week of traffic flow for all the segments of CS1 and is compared with the actual SRI value. If the SRI value of predicted average speed is 4 (the threshold value) for each detector of CS1, the total travel time will be around 4 min. So the total delay is 2 min. However, if the SRI value reaches 9 to 10, the total travel time will be from 24 min to 2 h. So an early warning message should be provided for all approaching vehicles before entering this area. For this complex segment, the highest SRI value was 4 and occurred in Segment C. Segment C is located at the beginning of a curved road and is close to an exit, which increases the level of congestion in this area of I-95 highway. Next, analysis and discussion will be provided for adjacent complex segments.

7.3.2. Complex Segment 2

To provide more realistic scenarios, we obtained the prediction results of the DMLRF model for several complex segments located adjacent. This will allow drivers to prepare for their trip in advance by determining the CS that has the highest delay time and exit the highway before entering that CS. RMSE and MAE are close to zero for all the algorithms, but the $R^2$ and adjusted $R^2$ for Segments A and B are lower than 60%. However, the traffic patterns for these two segments are mostly normal. The accuracy of the DMLRF model improves when traffic flow patterns are abnormal and achieves good results under normal conditions. Figure 9 shows the comparison of DMLRF model performance in terms of predicting a week of traffic flow with actual SRI values of CS2. The actual SRI values increased when the predicted average speed values decreased. Table 3 compares the performance of the proposed model for Segment D of Complex Segment 2 with that of other baseline regression models. Overall, the proposed model outperformed the other models in all evaluation metrics and for all segments. In Segment D, the DMLRF model achieved the highest accuracy, which is 76%. LogR and KNN models performance were very low when compared to the DMLRF model for all segments. LR presented better performance than LogR and KNN in terms of all evaluation metrics for most of the segments.

7.3.3. Complex Segment 3

Traffic flow patterns in this complex segment were mostly normal for all the segments. The DMLRF performed more efficiently than other base-line models in terms of all evaluation metrics for all segments as represented in Table 3. however, the performance in this complex segment of the proposed model was lower than the previous complex segments in terms of $R^2$ and $A{R}^2$. The $R^2$ and $A{R}^2$ results were 59% and 60% respectively. However, MAE, MSE and RMSE results were very low especially in Segment B. The predicted average speed values and actual average speed values were very close. We observed that there is no curved road in this complex segment and the actual average speed were mostly over 65 mph.

Table 3. The Prediction Results of the DMLRF Model and Baseline Models for Segments A, B, C, and D in CS1, CS2, CS3, and CS4.

Complex Segment	Algorithm	Segment	MSE	RMSE	MAE	R2	AR2
	DMLRF	Segment A	7.9	2.8	2.1	61%	61%
	LR	Segment A	8.4	2.9	2.2	59%	58%
	LogR	Segment A	9.9	3.1	2.3	15%	14%
	KNN	Segment A	10.9	3.3	2.6	7%	7%
	DMLRF	Segment B	1.3	1.1	0.9	75%	75%
	LR	Segment B	4.3	2.0	1.7	18%	17%
	LogR	Segment B	6.5	2.5	1.4	22%	22%
	KNN	Segment B	1.5	1.2	1.0	71%	71%
Complex Segment 1	DMLRF	Segment C	3.5	1.8	1.5	85%	85%
	LR	Segment C	10.5	3.2	2.7	56%	55%
	LogR	Segment C	21.1	4.5	3.5	12%	8%
	KNN	Segment C	4.9	2.2	1.7	79%	79%
	DMLRF	Segment D	5.0	2.2	1.8	71%	71%
	LR	Segment D	7.9	2.8	2.2	54%	54%
	LogR	Segment D	9.2	3.0	2.2	46%	13%
	KNN	Segment D	10.5	2.6	3.2	38%	38%
	DMLRF	Segment A	7.0	2.6	2.0	52%	52%
	LR	Segment A	11.4	3.3	2.4	22%	23%
	LogR	Segment A	15.1	3.8	2.5	11%	11%
	KNN	Segment A	10.3	3.2	2.4	30%	30%
	DMLRF	Segment B	5.2	2.2	1.7	52%	52%
	LR	Segment B	6.7	2.6	1.9	38%	38%
	LogR	Segment B	7.7	2.7	2.0	30%	17%
	KNN	Segment B	8.4	2.9	2.2	23%	23%
Complex Segment 2	DMLRF	Segment C	5.6	2.3	1.8	69%	68%
	LR	Segment C	7.6	2.7	2.1	57%	57%
	LogR	Segment C	8.4	2.9	2.1	52%	16%
	KNN	Segment C	11.6	3.4	2.5	35%	35%
	DMLRF	Segment D	8.6	2.9	2.3	76%	76%
	LR	Segment D	18.4	4.2	3.2	49%	49%
	LogR	Segment D	26.3	5.1	3.7	28%	9%
	KNN	Segment D	33.0	5.7	4.5	9%	8%
	DMLRF	Segment A	5.9	2.4	1.8	35%	35%
	LR	Segment A	7.3	2.7	2.0	20%	19%
	LogR	Segment A	7.1	2.6	1.9	21%	17%
	KNN	Segment A	13.2	3.6	2.8	13%	13%
	DMLRF	Segment B	6.4	2.5	1.9	59%	60%
	LR	Segment B	11.4	3.3	2.5	30%	30%
	LogR	Segment B	15.7	3.9	2.8	13%	13%
	KNN	Segment B	7.7	2.7	2.1	50%	50%
Complex Segment 3	DMLRF	Segment C	8.6	2.9	2.2	50%	50%
	LR	Segment C	16.4	4.0	3.1	5%	5%
	LogR	Segment C	7.7	2.7	2.0	55%	17%
	KNN	Segment C	19.3	4.4	3.2	12%	12%
	DMLRF	Segment D	6.8	2.6	1.9	58%	58%
	LR	Segment D	11.3	3.3	2.4	31%	30%
	LogR	Segment D	14.2	3.7	2.6	13%	13%
	KNN	Segment D	10.2	3.1	2.3	37%	37%
	DMLRF	Segment A	9.7	3.1	2.3	44%	43%
	LR	Segment A	13.9	3.7	2.7	20%	19%
	LogR	Segment A	16.4	4.0	2.8	14%	14%
	KNN	Segment A	11.9	3.4	2.6	31%	31%
	DMLRF	Segment B	5.4	2.3	1.9	80%	80%
	LR	Segment B	7.2	2.6	2.0	73%	72%
	LogR	Segment B	8.1	2.8	2.1	69%	20%
	KNN	Segment B	13.8	3.7	2.8	48%	48%
Complex Segment 4	DMLRF	Segment C	10.7	3.2	2.5	70%	70%
	LR	Segment C	12.3	3.5	2.8	65%	65%
	LogR	Segment C	12.2	3.5	2.7	65%	10%
	KNN	Segment C	20.0	4.4	3.3	43%	43%
	DMLRF	Segment D	6.8	2.6	2.0	81%	81%
	LR	Segment D	9.7	3.1	2.5	72%	72%
	LogR	Segment D	12.6	3.5	2.8	64%	10%
	KNN	Segment D	13.7	3.7	2.8	61%	61%

Table 3. The Prediction Results of the DMLRF Model and Baseline Models for Segments A, B, C, and D in CS1, CS2, CS3, and CS4.

Complex Segment	Algorithm	Segment	MSE	RMSE	MAE	R2	AR2
	DMLRF	Segment A	7.9	2.8	2.1	61%	61%
	LR	Segment A	8.4	2.9	2.2	59%	58%
	LogR	Segment A	9.9	3.1	2.3	15%	14%
	KNN	Segment A	10.9	3.3	2.6	7%	7%
	DMLRF	Segment B	1.3	1.1	0.9	75%	75%
	LR	Segment B	4.3	2.0	1.7	18%	17%
	LogR	Segment B	6.5	2.5	1.4	22%	22%
	KNN	Segment B	1.5	1.2	1.0	71%	71%
Complex Segment 1	DMLRF	Segment C	3.5	1.8	1.5	85%	85%
	LR	Segment C	10.5	3.2	2.7	56%	55%
	LogR	Segment C	21.1	4.5	3.5	12%	8%
	KNN	Segment C	4.9	2.2	1.7	79%	79%
	DMLRF	Segment D	5.0	2.2	1.8	71%	71%
	LR	Segment D	7.9	2.8	2.2	54%	54%
	LogR	Segment D	9.2	3.0	2.2	46%	13%
	KNN	Segment D	10.5	2.6	3.2	38%	38%
	DMLRF	Segment A	7.0	2.6	2.0	52%	52%
	LR	Segment A	11.4	3.3	2.4	22%	23%
	LogR	Segment A	15.1	3.8	2.5	11%	11%
	KNN	Segment A	10.3	3.2	2.4	30%	30%
	DMLRF	Segment B	5.2	2.2	1.7	52%	52%
	LR	Segment B	6.7	2.6	1.9	38%	38%
	LogR	Segment B	7.7	2.7	2.0	30%	17%
	KNN	Segment B	8.4	2.9	2.2	23%	23%
Complex Segment 2	DMLRF	Segment C	5.6	2.3	1.8	69%	68%
	LR	Segment C	7.6	2.7	2.1	57%	57%
	LogR	Segment C	8.4	2.9	2.1	52%	16%
	KNN	Segment C	11.6	3.4	2.5	35%	35%
	DMLRF	Segment D	8.6	2.9	2.3	76%	76%
	LR	Segment D	18.4	4.2	3.2	49%	49%
	LogR	Segment D	26.3	5.1	3.7	28%	9%
	KNN	Segment D	33.0	5.7	4.5	9%	8%
	DMLRF	Segment A	5.9	2.4	1.8	35%	35%
	LR	Segment A	7.3	2.7	2.0	20%	19%
	LogR	Segment A	7.1	2.6	1.9	21%	17%
	KNN	Segment A	13.2	3.6	2.8	13%	13%
	DMLRF	Segment B	6.4	2.5	1.9	59%	60%
	LR	Segment B	11.4	3.3	2.5	30%	30%
	LogR	Segment B	15.7	3.9	2.8	13%	13%
	KNN	Segment B	7.7	2.7	2.1	50%	50%
Complex Segment 3	DMLRF	Segment C	8.6	2.9	2.2	50%	50%
	LR	Segment C	16.4	4.0	3.1	5%	5%
	LogR	Segment C	7.7	2.7	2.0	55%	17%
	KNN	Segment C	19.3	4.4	3.2	12%	12%
	DMLRF	Segment D	6.8	2.6	1.9	58%	58%
	LR	Segment D	11.3	3.3	2.4	31%	30%
	LogR	Segment D	14.2	3.7	2.6	13%	13%
	KNN	Segment D	10.2	3.1	2.3	37%	37%
	DMLRF	Segment A	9.7	3.1	2.3	44%	43%
	LR	Segment A	13.9	3.7	2.7	20%	19%
	LogR	Segment A	16.4	4.0	2.8	14%	14%
	KNN	Segment A	11.9	3.4	2.6	31%	31%
	DMLRF	Segment B	5.4	2.3	1.9	80%	80%
	LR	Segment B	7.2	2.6	2.0	73%	72%
	LogR	Segment B	8.1	2.8	2.1	69%	20%
	KNN	Segment B	13.8	3.7	2.8	48%	48%
Complex Segment 4	DMLRF	Segment C	10.7	3.2	2.5	70%	70%
	LR	Segment C	12.3	3.5	2.8	65%	65%
	LogR	Segment C	12.2	3.5	2.7	65%	10%
	KNN	Segment C	20.0	4.4	3.3	43%	43%
	DMLRF	Segment D	6.8	2.6	2.0	81%	81%
	LR	Segment D	9.7	3.1	2.5	72%	72%
	LogR	Segment D	12.6	3.5	2.8	64%	10%
	KNN	Segment D	13.7	3.7	2.8	61%	61%

7.3.4. Complex Segment 4

In complex segment 4, the abnormal traffic flow patterns occurred in Segments C and D where these two segments are located close to an exit, which is similar to the traffic flow patterns in CS1. This leads to an observation that abnormal traffic flow on I-95 highway usually occurred at the curved road that has an exit within the next half a mile. The proposed model achieved 80% in terms of $R^2$ and $A{R}^2$ and outperformed all other base-line models in all the segments.

8. Conclusions

Real-time highway traffic flow prediction is considered one of the highest priority applications for intelligent transportation systems since abnormal traffic flow can have severe impacts on people’s time, money, and lives. Providing traffic flow information in real-time will assist drivers to reach their destination faster and safer. However, traffic data is very challenging to process and evaluate in real-time due to its huge size. In this study, we designed and evaluated a scalable distributed machine learning-based scheme to predict highway traffic flow in (near-) real-time. The system maintains security and privacy since the data will not be shared at a centralized node. Stacking ensemble learning is utilized to implement the DMLRF model that takes the RFR models with different hyper-parameters’ values as inputs. KA and Apache Spark are utilized to process the data and provide the prediction results in real-time. A dataset obtained from FDOT-D4 is used to build all the models in this research to predict highway traffic flows. The proposed model was evaluated and compared to other base-line models, LR, LogR, and KNN in terms of RMSE, MSE, MAE, $R^2$, and adjusted $R^2$. The proposed scheme demonstrated high accuracy in predicting abnormal traffic flows and outperformed the base-line models while maintaining scalability and data privacy. For future work, we will design and evaluate a real-time accident prediction scheme using distributed machine learning.