An Intelligent Ensemble-Based Detection of In-Vehicle Network Intrusion

Abstract

The Controller Area Network (CAN) bus has been implemented in most modern Vehicles. Various attacks can be launched against the CAN bus protocol because it is designed without security mechanisms. It is essential to develop a highly accurate intrusion detection system (IDS) for CAN bus attacks. We design an effective ensemble learning-based IDS scheme for detecting and classifying DoS, fuzzing, replay, and spoofing attacks. These are common CAN bus attacks that can threaten the safety of a vehicle’s driver, passengers, and pedestrians. For this purpose, we utilize supervised machine learning in combination with ensemble methods. We first perform data balancing and feature selection. We build and fine-tune random forest, Xtreme gradient boosting, and decision tree supervised learning models. We then combine these models with voting, stacking, and bagging ensemble methods. The results obtained demonstrate the effectiveness of the proposed scheme when trained on real-life CAN traffic datasets to detect and classify these four attacks. The stacking method achieved the highest performance in terms of accuracy, precision, recall, F1-score, and area-under-the-curve receiver operator characteristic (ROC-AUC). The stacking method outperformed other recently proposed methods with an F1-score, precision, recall, and accuracy of 0.993, 0.993, 0.993, and 0.986, respectively.

1. Introduction

Vehicles are increasingly becoming a necessity in modern society. The emerging vehicular network technology supports inter-vehicle and vehicle-to-everything communication and has the potential to enhance the safety and efficiency of transportation systems [1,2,3,4,5,6,7,8]. In-vehicle networks (IVNs) support intra-vehicle communication between vehicle interconnected components [9,10]. To meet the growing need for advanced functionality, vehicles are becoming more equipped with electronic equipment. Electrical Control Units (ECUs) are significant pieces of equipment for monitoring and controlling various components. The average vehicle has between 70 and 100 ECUs, each of which transmits over 2500 signals to other units within the vehicle [11]. IVNs, which are comprised of various bus protocols, have been adopted as the framework for coordinating communication among ECUs. Enabled by actuators, sensors, and processors, IVNs facilitate information exchange services depending on the needs of drivers. Vehicle manufacturers adopt a high number of ECUs to enhance various functionalities [11,12,13]. For IVNs, the communication protocols utilized include Controller Area Network (CAN), Local Interconnect Network (LIN), FlexRay, Ethernet, and Media-Oriented Systems Transport (MOST). The common key feature among the protocols is the transmission of messages with pre-defined time limits and message delivery reports. IVNs mainly use the CAN bus protocol, which focuses on the reliability of communication and functional safety. CAN is often adopted for multiple functionalities and control, including real-time communication for engine management, brake controls, airbag control, and control of the body system. The protocol transmits packets between ECUs through interconnected buses.

It is essential to protect the communication within a vehicle since intra-vehicular communication affects the safety of drivers, passengers, and the vehicles themselves. The CAN protocol, designed for robustness, flexibility, speed, and low cost, has become a cause of concern due to a lack of security mechanisms. These mechanisms include a lack of authentication and encryption that let attackers disrupt the protocol and messages’ confidentiality, integrity, and availability. The authors in [14,15,16,17,18] reported that the CAN protocol’s weak security makes it a target for different attacks. The most common attack vectors include DoS, fuzzing, replay, and spoofing, which are critical due to how they can render the protocol system ineffective [19]. The protocol’s broadcast transmission technique, security vulnerabilities, and unsecured priority scheme are weaknesses that further expose the CAN protocol to cyber threats. Successful attacks against the CAN protocol through injection or other vectors can enable an attacker to take control of the bus system, disable the brakes, affect acceleration and steering, or turn the lights on/off, among others. As part of a hacking experiment, a Jeep Cherokee’s functions were hacked and controlled [20].

Various attacks can be launched against the CAN bus protocol because it was designed without security mechanisms. It is essential to develop highly accurate detection mechanisms for CAN bus attacks. Conventional security mechanisms, including authentication, firewalls, and encryption, are not suitable for CAN due to its structural limitations. These limitations include the absence of support for such security methods [21,22,23]. The 8-byte size of the CAN data frame poses limits on cryptographic message authentication. Another proposed method utilizes a convincing honeypot at the wireless gateway, but its efficacy depends on its ability to deceive assailants with its realism. In addition, a suggested firewall architecture for vehicular gateways may not offer complete protection against a variety of potential risks to the network [24]. Encryption and authentication-based IVN security solutions, firewall structures, and IVN Intrusion Detection Systems (IDSs) are security solutions discussed in [25]. Encryption and authentication are inapplicable due to the limited resources of IVNs, whereas firewalls are impracticable due to a long vehicle lifecycle and numerous attack surfaces. The authors recommend utilizing an IDS for IVNs to defend against intrusions in resource-constrained environments. According to the study in [26], automotive IDS, including those that utilize machine learning algorithms, have the capability to fulfill the requirements of the industry in terms of being cost-effective, lightweight, and working in real time. The study evaluates a variety of intrusion detection technologies, including machine learning approaches, and suggests their implementation in the automotive industry. One of the security mechanisms that has received attention among researchers is the use of IDSs. These mechanisms have been researched and created to advance security against the increasing number of attacks leveled against networks. IDSs monitor networks to identify malicious activities, such as information theft, censorship, or protocol tampering, with the major focus being the detection of anomalous behavior and recording information. IDSs are implemented as network-level defense mechanisms for computer resources. The characteristics and capabilities of IDSs make them suitable for detecting anomalies in common networks. However, in networks that face new and complicated cyberattacks, IDSs need integration with novel technologies to enhance their effectiveness and accuracy in the detection results. For instance, ensemble learning can be used to obtain better detection results for IDSs. The main objective of ensemble learning is to enhance the accuracy of classification by combining multiple classifiers into one model. The ensemble learning method allows for multiple mechanisms of learning with varying strengths to support one another, which leads to increased prediction reliability. In [18,27,28,29,30,31], the authors demonstrated that the utilization of ensemble methods has resulted in enhanced accuracy in intrusion detection compared to the utilization of individual classifiers.

In this study, we propose a scheme that utilizes well-tuned hyperparameters, feature selection, supervised models, and ensemble methods to build an effective IDS framework for classifying DoS, fuzzing, replay, and spoofing attacks on the CAN bus. We combine Random Forest (RF), Xtreme Gradient Boosting (XGBoost), and Decision Tree (DT) models using stacking, voting, and bagging methods. The following is an illustration of our contributions:

• We develop and test three supervised models that include hyperparameter tuning, balancing of data, and feature selection. The models are RF, DT, and XGBoost.
• With three different ensemble methods, we combine the three supervised models to enhance our scheme’s ability to detect DoS, fuzzing, replay, and spoofing attacks. These ensemble methods are stacking, voting, and bagging.
• We evaluate our scheme’s performance using accuracy, precision, recall, F1-score, and area-under-the-curve receiver operator characteristic (ROC-AUC). Our scheme detected DoS, fuzzing, replay, and spoofing attacks with a higher detection accuracy score of 0.986 compared to the most recent study proposed for detecting attacks in the CAN bus.

This study’s remaining sections are organized as follows: Section 2 describes CAN’s background and attack types. Section 3 discusses the related work. The models proposed are illustrated in Section 4. Section 5 contains the results and analysis. Section 6 finally concludes the paper.

2. CAN’s Background and Types of Attacks

2.1. Background of CAN Bus

Robert Bosh GmbH developed the CAN for vehicles in 1986 [32]. The CAN bus protocol transmits data and messages to all network nodes using a broadcast technique. CAN has various advantages that make it suitable for IVNs. The CAN bus has a flexible design and is suitable for fast and low-latency real-time communication in vehicles. Moreover, it has a high level of reliability, noise resistance, and fault tolerance. CAN utilizes a priority-based strategy that prevents communications interruptions from lower-priority messages for higher-priority messages within the bus. CAN is essential and irreplaceable as the core of automotive IVN communication in modern vehicles. Replacing it would require a complete redesign of the IVN architecture [33,34,35].

A CAN packet has seven fields in its structure, which are explained in the following: Start of the Frame, which uses a single bit to indicate the start of CAN message transmission to all connected nodes; Arbitration Field, which is referred to as the CAN ID, and indicates the importance of the message received. The CAN ID message has a low value, indicating its high priority. Control Field, which contains a data length code (DLC), shows the data field’s total bytes. Data Field: the actual data transfer is the responsibility of this field. Cyclic Redundancy Check (CRC) Field: utilized in the process of validating data packets. Acknowledge (ACK) Field verifies that the receiver nodes in the network have successfully received the CAN packets. End of the Frame shows the completion of the message of CAN. Figure 1 depicts the CAN bus’s frame structure.

2.2. Attack Types

The CAN protocol is limited by a lack of authentication and encryption as key security features. The lack of authentication causes violations of integrity and priority-based arbitration and exposes every node in the network to all transmitted messages. An attacker can exploit this vulnerability by launching a sniffing attack on the CAN bus, analyzing the internal CAN data of a target vehicle, and launching injection attacks to control the vehicle. The CAN protocol also gives priority to the arbitration mechanism. The protocol uses an ID-based priority scheme, which attackers can exploit by injecting high-priority messages to launch denial-of-service (DoS) attacks and prevent access to other services. The simplicity and high adaptability of the CAN protocol are vulnerabilities. This allows an attacker to inject deceptive data into the bus network [36,37]. The authenticity of messages sent using the CAN protocol is also not guaranteed. As a message-based protocol, CAN sends messages without source or destination addresses since its frame format lacks sender/receiver information. Injection attacks are, therefore, easy to launch against the protocol. Such injection attacks typically exploit the onboard diagnostics II (OBD-II) interface to introduce malicious CAN messages into the vehicles and disable some functionalities. The attacks originate from internal sources such as the OBD or external sources such as Wi-Fi, cellular links, and Bluetooth devices [35,38]. The lack of encryption, on the other hand, violates the principle of confidentiality. CAN is not encrypted because of real-time communication requirements. The risk of having no encryption is that attackers can launch malicious code to monitor all bus-broadcast communications, analyze their content, and use the information to hack the IVN. A compromised ECU can further take advantage of the broadcast property of CAN by monitoring and listening to all CAN frames or sending any frame across the network.

CAN has security vulnerabilities that threaten the safety of vehicles, drivers, and passengers. The following are the security vulnerabilities in the CAN bus: Lack of Boundary Defense and Segmentation: The ECUs in IVNs are organized into sections, but these sections lack boundaries because gateways connect them. Therefore, a malicious user with access can take control of the system. Lack of Authentication: CAN has no authentication infrastructure, allowing any device to connect to the bus and join the network without the sender’s ID being verified. Each node that joins the network may use CAN’s broadcast communication feature to send malicious data to other nodes. Unencrypted Traffic: The messages exchanged between CAN nodes are in plain text, indicating that the protocol does not take the confidentiality requirement into account. Availability Check: Before allowing any message to be sent, the CAN protocols check to see if carriers are available. If any carrier is busy, the protocol prevents all ECUs from transmitting and rechecks the carriers after a certain period of time. This mechanism makes the protocol vulnerable to DoS attacks [15,16,18]. The authors in [39] evaluated classification techniques based on ML for the purpose of DoS detection. The implementation of machine learning techniques improved the performance of classification for detecting DoS attacks.

Attackers may target several security requirements in vehicular communication environments, including confidentiality, availability, authenticity, data integrity, and non-repudiation [40,41,42,43,44,45].

Table 1. Affected security requirements.

Attack	Availability	Integrity	Authenticity	Non-Repudiation	Confidentiality
DoS [40]	+
Fuzzing [41,42]	+	+
Replay [43]		+	+	+	+
Spoofing [44]			+	+

“+” as it affects the security requirements.

depicts the type of attack and the affected security requirements.

Message injection attacks, including DoS, spoofing, replay, and fuzzing, are the most popular inter-vehicle attacks [41]. DoS: The attacker renders communication services unavailable by increasing the message frequency of the CAN bus. The attacker will inject high-frequency messages with a high priority that are either smaller than a normal message or have a CAN ID of 0x000. A malicious node can increase bus occupancy in order to cause delays or even prevent other communications. Figure 2 demonstrates an example of a DoS attack in which the attacker sends two messages with the high-priority CAN ID 0x000 to cause delays for the CAN ID 0x209 transmitted by ECU A.

Fuzzing: The attacker injects random ID, DLC, and data fields while impersonating legitimate traffic using CAN IDs that appear legitimate to compromise the CAN bus system, causing disruptions such as shaking of the steering wheel, irregular signal lights, and automated gear shifts [37]. This attack approach involves sending many randomly compromised IDs and data into the network at any time, disrupting regular communications without the victim node’s knowledge. The lack of authentication and data integrity in the CAN bus protocol led to this attack. Figure 3 shows an example of attacker-generated random messages with CAN IDs 0x200, 0x307, and 0x540, that are received by ECU B.

Replay: The objective of this type of attack is to capture legitimate CAN messages in transmission and then later re-transmit them into the vulnerable CAN bus. This is an influential Man in the Middle attack that intercepts, records, and rebroadcasts network messages. This attack can result in actions that are not authorized or the dissemination of false information throughout the network. This type of attack occurs because the CAN bus data transmission is broadcast. An example of a replay attack is depicted in Figure 4, and it includes the attacker transmitting the CAN IDs that belong to the target ECUs, A and C, and are received by ECU B.

Spoofing: The attacker is able to inject messages with specified CAN IDs belonging to authorized nodes, as no authentication is provided for communications on the CAN bus. Payload-focused spoofing is an attack where attackers intercept and analyze CAN communications to read and alter essential component structures. They quickly inject malicious information into transmission gaps by masquerading as a typical node with the same ID [46]. This attack has the possibility of reducing communication performance. Figure 5 demonstrates an example of a spoofing attack in which the attacker uses CAN ID 0x209, which belongs to ECU A, and sends it to the target ECU B. This attack has the potential to alter the frequency and the content of the targeted ID.

3. Related Work

In a review of IDS for vehicular networks, Nandy et al. [47] emphasized the importance of addressing cyber-physical threats and exploring IDS mechanisms for vehicular communication from multiple perspectives. They identified gaps in real-world validation and highlighted the significance of implementation and testing to verify IDS effectiveness. To detect in-vehicle attacks, the authors in [48] presented a deep convolutional neural network (DCNN) approach called Reduced Inception-ResNet. Compared to Naive Bayes (NB), Support Vector Machine (SVM), Decision Tree (DT), and Artificial Neural Network (ANN), this method outperformed the others in terms of detection accuracy. However, more improvement is needed to investigate the performance of various types of CAN bus attacks, and the imbalance of data between malicious and normal samples prevents these models from performing accurate intrusion detection techniques. In [49], a lightweight ML algorithm based on RNN is proposed for IDS on the CAN bus network. To obtain the highest effective results in detection performance, the authors combined the security control unit (SECCU) and the simplified attention (SIMATT) algorithms. The proposed model outperformed Long Short-Term Memory (LSTM), Gated Recurrent Unit (GRU), GAN-Based Intrusion Detection System (GIDS), and RNN in terms of performance metrics. The authors in [50] proposed a CAN-bus intrusion-based LSTM deep learning model. The model can detect three types of attacks: DoS, fuzzing, and spoofing. However, no feature selection or data balance is used. In [51], the authors proposed an IDS that utilizes RNN and LSTM. The selected model provided intrusion detection against CAN bus spoofing attacks; however, further research is required to detect additional types of cyberattacks. In [52], the authors presented CANintelliIDS, a method for detecting CAN bus intrusions based on CNN and attention-based gated recurrent unit (AGRU) models. The main focus of the model was detecting DoS, fuzzing, and impersonation attacks. In terms of performance metrics, the model outperformed RF, SVM, CNN, and Logistic Regression. In [53], CAN-bus intrusion based on the application of Recurrence Quantification Analysis (RQA) to detect spoofing attacks is proposed. The RQA-based approach obtained good detection accuracy. The author focused on spoofing attacks, but other attacks need to be considered. CNN and LSTM are types of deep learning models that have been integrated and proposed as methods for defending IVNs against a variety of cyberattacks [54]. These attacks include replay, DoS spoofing, and fuzzing attacks. They were able to detect all four attacks with an accuracy score of 97.30%. In [55], the authors proposed a graph-based IDS, and they utilized KNN and SVM to detect and identify attack messages broadcast on the CAN bus. This was conducted by turning the messages received from the CAN bus into a temporal graph; however, no hyperparameter tuning or data balancing were performed. IDS models have shown potential in previous research; however, they frequently lack an integrated framework that combines feature selection, data balancing, and systematic hyperparameter optimization across diverse attack types. Additionally, most studies focus on specific types of attacks or employ a single ensemble strategy. In contrast, our study integrates three ensemble methodologies, stacking, bagging, and voting, alongside rigorous preprocessing techniques to detect all four major CAN bus attacks (DoS, fuzzing, spoofing, and replay). This work aims to evaluate a comprehensive and multifaceted ensemble IDS on real-world CAN data. Authors in [56] have introduced an enhanced SVM-based IDS model for the CAN bus system. On the actual dataset, the model performed well and was robust to only DoS attacks on electric vehicles. Several studies [51,53,56] focus on a single type of attack, such as spoofing or DoS attacks. The CAN bus, however, is vulnerable to various attacks. While Refs. [51,52,53,54,55] accomplished feature selection, and Refs. [48,49,50,52,53,54] implemented model tuning, none of the related work used data balancing and ensemble methods.

Table 2. Our proposed method compared to recently proposed promising approaches.

Citation	Model	Feature Selection	Balanced	Hyperparameters	Ensemble Methods	Type of Attacks
[48]	DCNN	Not reported	−	+	−	DoS, fuzzing, and spoofing
[49]	RNN	Not reported	Not reported	+	−	DoS, fuzzing, and impersonation
[50]	LSTM	−	−	+	−	DoS, fuzzing, and spoofing
[51]	RNN	Manual feature extraction	−	−	−	spoofing
[52]	CNN and GRU	deep feature selection via CNN	−	+	−	DoS, fuzzing, and impersonation
[53]	ML	RQA	−	+	−	spoofing
[54]	CNN-LSTM	Implicit via CNN layers	−	+	−	DoS and fuzzing
[55]	SVM KNN	Heuristic Feature Selection	−	−	−	DoS, fuzzing, and spoofing
Ours	Stacking	Extra Trees	+	+	+	DoS, fuzzing, spoofing, and replay

“+” as a strength and “−” as a weakness.

presents a comparison between our method and some recently proposed methods. Ensemble learning, data balance, and hyperparameter tuning are in this table. The novelty lies in the combination of these components, with stacking using XGBoost (version 1.7.6) as a meta-learner being something rarely explored in prior CAN IDS research. This scheme achieves higher performance across all metrics as seen in the Results section, particularly in detecting classifying all four major CAN bus attacks.

4. Proposed Scheme

We propose a multi-classification scheme capable of detecting four types of attacks on CAN bus networks. We use machine learning to design and evaluate an effective IDS framework. This framework utilizes multi-classification algorithm-based ensemble methods to enhance the IDS’s ability to identify intrusions. We used three supervised ML models (RF, DT, and XGBoost) along with three ensemble classifiers (stacking, voting, and bagging). Well-known evaluation metrics were used to assess the effectiveness of the proposed approach. Accuracy, precision, recall, F1-score, and ROC were the metrics utilized in this study. Figure 6 is an overview of our proposed model’s workflow, which includes the CAN bus input dataset, data preprocessing, supervised models building, ensemble modeling, attacks classification, and scheme evaluation.

4.1. CAN Bus Dataset

The CAN intrusion dataset from IEEE DataPort is utilized for this study [57]. The dataset is selected since it was collected in 2020. Furthermore, the dataset has full labels, which is essential for our proposed supervised learning models. Millions of CAN bus messages were collected by the authors of the dataset from a vehicle’s real data, and then four attacks were launched: DoS, spoofing, replay, and fuzzing. The features used in the dataset are Timestamp, Arbitration-ID, DLC, Data, and Target. Timestamp is the time recorded. Arbitration-ID is an identification for CAN messages. DLC indicates the data length code. The CAN data field is Data. Target determines the types of attacks (normal, DoS, spoofing, replay, and fuzzing). The distribution of normal and attack messages is shown in

Table 3. The distribution of normal and attack messages.

Message Type	Count	Percentage (%)
Normal	3,372,743	91.846523
DoS	154,180	4.198629
Fuzzing	89,879	2.447585
Replay	47,593	1.296052
Spoofing	7756	0.211211
Total	3,672,151	100

.

4.2. Data Preprocessing

Preprocessing is used in data mining to transform raw datasets into a usable structure. Data is frequently insufficient and inconsistent in the real world. Preprocessing the data can be conducted in a variety of ways. Label encoding was implemented during the preprocessing phase of the data. It is the step before feeding data into a model, wherein numerical values are assigned to each dataset. Also, we cleaned and removed duplicate and null values. The mean–median method was used for filling null values. Feature selection was utilized to illustrate the significance of the dataset’s features [58]. The extra tree technique was utilized in order to identify the most important features and eliminate the features that had no significance in the dataset that was being analyzed. Due to their randomization, Extra Trees provide diversity, robustness to noise, and efficiency, reducing overfitting and identifying feature importance. The features used in the dataset are Timestamp, Arbitration- ID, and Data. We used a K-fold cross-validation process with K equals 10. We take the issue of data imbalance across classes into consideration. The main objective of utilizing balanced data during model training is to enhance the classification accuracy of the models. We use the near-miss as a method of undersampling that can help fix the imbalanced data issue. This method reduces the size of data for the majority class [59].

4.3. Supervised ML

In the supervised models building phase, we used three supervised models that provided the most accurate results in our previous work [29]. These models are RF [60], DT [61], and XGBoost [62]. ML model performance can be improved through model tuning. Random search is used to find the most suitable set of hyperparameters. For the models, we performed hyperparameter tuning. To prevent overfitting, we utilized a K-fold cross-validation process with K equals 10.

4.4. Ensemble Classifiers Learning

Ensemble learning uses techniques that allow different ML algorithms to work together on a single prediction to enhance accuracy. It has the potential to outperform any single algorithm. Ensemble learning for classification is several models that are trained to solve a classification problem. Models’ predictions are combined to make a final prediction. Using ensemble learning, the performance of a single classifier can be improved by reducing overfitting, making the classifier more robust, and improving its ability to generalize. In the ensemble modeling phase, we utilized three ensemble techniques: stacking, bagging, and voting. Ensemble learning with additional cross-validation provides reliable performance estimates and can improve generalization by using varied model predictions.

Stacking is an ensemble learning technique. The purpose of stacking is to achieve a more accurate final model by combining the strengths of multiple base models. It builds its models in two stages. These stages are called base learner and meta-learner. Stacking runs the single learners (base-learners) and combines them by training the meta-learner to make the final predictions using the predictions generated by the base-learners. The meta-classifier learns how to best combine the predictions of the base learners [63]. Any learning algorithm can be employed as a single learner or meta-learner. In our model, we used the RF, DT, and XGBoost models as base models and the XGBoost model as a meta model.

Bagging represents a parallel ensemble method. In this method, parallel learning produces single learners. Bagging generates many dataset subsets (bags) by bootstrapping. The final prediction results are determined by averaging all predictions for subsets. To improve performance, the bagging classifier reduces the variance of the base classifiers. Its main objective is to enhance the precision and stability of supervised models [64,65]. To reduce noise and avoid overfitting, we combined the bagging classifier with the XGBoost model.

A voting ensemble method is a technique for combining the predictions of multiple models to enhance the system’s overall performance. Different models can make different types of errors, and by combining their predictions, the ensemble can reduce errors overall [66]. The steps and evaluation of our proposed IDS scheme are depicted in Figure 7.

4.4.1. Attacks Classification

The target of our scheme during the classes phase is to identify the types of attacks (normal, DoS, spoofing, replay, and fuzzing). These attacks are described in Section 2.2.

4.4.2. Scheme Evaluation

In this phase, the evaluation of our scheme’s performance is carried out based on several metrics, including accuracy, recall, precision, receiver operating characteristic (ROC), and F1-score. These metrics are defined in relation to true positives (TPs), false positives (FPs), true negatives (TNs), and false negatives (FNs). For the ROC, specificity (True Positive Rate, or TPR) and sensitivity (True Negative Rate, or TNR) are also considered.

$$Accuracy={\displaystyle \frac{TP+TN}{TP+TN+FP+FN}}$$

(1)

$$Precision={\displaystyle \frac{TP}{TP+FP}}$$

(2)

$$Recall={\displaystyle \frac{TP}{TP+FN}}$$

(3)

$$F1={\displaystyle \frac{2\ast Precision\ast Recall}{Precision+Recall}}={\displaystyle \frac{2\ast TP}{2\ast TP+FP+FN}}$$

(4)

$$Specificity={\displaystyle \frac{TN}{FP+TN}}$$

(5)

$$Sensitivity=Recall$$

(6)

4.5. Environment Tools

A Jupyter Notebook is used as the development environment for this work. This environment is one of the applications that Anaconda offers. Python is used for the implementation of the scheme. Python is selected because it is efficient, scalable, stable, and supports the metrics used in our work. All experiments were conducted in a Python 3.8 environment using scikit-learn and Pandas libraries.

5. Results and Discussion

This research’s main objective is to detect and classify intrusion detection attacks. DoS, fuzzing, spoofing, and replay attacks are four common CAN bus attacks that can pose a threat to the safety of a vehicle’s driver, passengers, and pedestrians. We built a scheme that detects these common CAN bus attacks and classifies their type. We utilized supervised machine learning in combination with stacking, voting, and bagging ensemble methods for this purpose. Initially, we developed and evaluated three promising supervised learning techniques. After performing feature selection and data balancing, we utilized RF, DT, and XGBoost and fine-tuned these models. These models were selected due to their effective multithreading capabilities, and these tree structure algorithms provide minimal computational time. Furthermore, their ability to execute in parallel significantly decreases the time frame of model training while improving accuracy [67].

In terms of precision, recall, and F1-score, feature selection enhanced the performance of all models (RF, DT, and XGBoost). We obtained F1-scores higher than 0.979, precision scores higher than 0.980, and recall scores higher than 0.979. The XGBoost model with feature selection has the highest precision, recall, and F1-score, followed by the RF model with feature selection. Also, using feature selection with ensemble methods (stacking, bagging, and voting) improved the performance in terms of precision, recall, and F1-score. The stacking method has the highest performance with an F1-score of 0.993, precision of 0.993, and recall of 0.993. Voting has the second-highest performance with an F1-score of 0.981, precision of 0.982, and recall of 0.981. The bagging method has the lowest performance with an F1-score of 0.980 and precision and recall of 0.980. Figure 8 shows that applying feature selection to the RF, DT, XGBoost, stacking, bagging, and voting models increases their precision, recall, and F1-score performance.

Cross-validation is a technique for determining the performance of machine learning models. Cross-validation can provide a more accurate estimate of model performance than a single train-test split by performing multiple iterations of training and testing on different subsets of the data [68]. In our proposed scheme, the dataset is divided using 10-fold cross-validation, where each fold is utilized once as test data and the remaining nine as training data. This process is repeated ten times to ensure robust evaluation by averaging the performance across all iterations. In this study, the accuracy of the models with feature selection is compared to that of the models without feature selection using 10-fold cross-validation. Without feature selection, stacking achieved the highest accuracy of 0.986. XGBoost and bagging achieved an accuracy of 0.982 and 0.976, respectively. The accuracy of the voting model is 0.975, while the accuracy of the RF and DT models is 0.973 and 0.971, respectively. The stacking method has the highest performance among all the models without feature selection. However, feature selection has improved the performance of most of the models. Stacking and DT have the same accuracy because both of them are already able to select significant features effectively. Both the XGBoost model and the bagging model have an accuracy of 0.985. Following these models are voting, with an accuracy of 0.979, and RF, with an accuracy of 0.978. Using 10-fold cross-validation,

Table 4. The evaluation results using 10-fold cross-validation of the supervised models and ensemble methods without feature selection.

Model	Accuracy
	1	2	3	4	5	6	7	8	9	10	Mean
RF	0.993	0.997	0.993	0.992	0.994	0.996	0.983	0.992	0.975	0.819	0.973
DT	0.987	0.993	0.990	0.994	0.993	0.993	0.981	0.988	0.974	0.820	0.971
XGBoost	1.00	1.00	1.00	0.998	0.998	0.998	0.998	1.00	0.997	0.833	0.982
Stacking	1.00	1.00	1.00	1.00	0.998	1.00	0.998	0.998	0.996	0.876	0.986
voting	0.988	0.998	0.998	0.996	0.994	0.993	0.990	0.990	0.979	0.821	0.975
Bagging	0.993	1.00	0.994	0.993	0.993	0.996	0.984	0.998	0.981	0.823	0.976

and

Table 5. The evaluation results using 10-fold cross-validation of the supervised models and ensemble methods with feature selection.

Model	Accuracy
	1	2	3	4	5	6	7	8	9	10	Mean
RF	0.993	0.997	0.997	0.990	0.987	0.996	0.990	0.990	0.987	0.856	0.978
DT	0.967	0.992	0.990	0.987	0.992	0.993	0.984	0.985	0.966	0.858	0.971
XGBoost	1.00	1.00	1.00	0.997	0.997	1.00	1.00	0.997	1.00	0.858	0.985
Stacking	1.00	1.00	1.00	1.00	1.00	1.00	0.998	1.00	1.00	0.868	0.986
voting	0.987	0.994	0.996	0.992	0.994	0.996	0.996	0.996	0.981	0.855	0.979
Bagging	1.00	1.00	1.00	0.997	0.998	1.00	0.998	0.997	0.998	0.865	0.985

compare the accuracy of the models without feature selection to that of the models with feature selection. The accuracy of some methods remains relatively unchanged both before and after feature selection because these approaches already incorporate built-in feature selection processes. Figure 9 demonstrates that feature selection consistently reduces testing and training times for all models.

The results for all the models (RF, DT, XGBoost, Stacking, Voting, and Bagging) show that all models perform well in detecting flooding and spoofing attacks with high precision, recall, and F1-score. Stacking, XGBoost, and voting achieve the highest performance for the fuzzing attack, reaching a value of 1.00 for precision, recall, and F1-score. These results were achieved through the use of feature selection for detecting these attacks. These are followed by bagging, RF, and DT models with feature selection. The stacking model with feature selection has the highest performance for normal traffic, with precision, recall, and F1-score values of 1.00, 0.97, and 0.98, respectively. With precision, recall, and F1-scores of 1, 0.94, and 0.97, respectively, the XGBoost and RF models with feature selection have the second-highest performance. The stacking model with feature selection has the highest performance for the replay attack, with precision, recall, and F1-score values of 0.97, 1.00, and 0.99, respectively. With precision, recall, and F1-scores of 0.94, 1.00, and 0.97, respectively, the XGBoost model with feature selection has the second-best performance. The stacking model with feature selection achieves the highest performance in detecting all types of attacks, making it the best model among those considered. The XGBoost model with feature selection is the second-best model, as it detects all types of attacks effectively. The performance of the RF and DT models is comparable across all attacks, with the exception of the fuzzing attack, where they perform slightly worse. The bagging and voting methods have the lowest performance among those evaluated. The evaluation results for each attack using the supervised models and ensemble methods without and with feature selection are shown in

Table 6. The evaluation results for each attack using the supervised models and ensemble methods without feature selection.

Model	Attacks	Precision	Recall	F1-Score
RF	Flooding	1.00	1.00	1.00
	Fuzzing	0.99	1.00	1.00
	Normal	1.00	0.92	0.96
	Replay	0.92	0.99	0.96
	Spoofing	1.00	1.00	1.00
DT	Flooding	1.00	1.00	1.00
	Fuzzing	1.00	0.99	0.99
	Normal	1.00	0.92	0.96
	Replay	0.92	0.99	0.95
	Spoofing	0.99	1.00	0.99
XGBoost	Flooding	1.00	1.00	1.00
	Fuzzing	1.00	1.00	1.00
	Normal	1.00	0.92	0.96
	Replay	0.93	1.00	0.96
	Spoofing	1.00	1.00	1.00
Stacking	Flooding	1.00	1.00	1.00
	Fuzzing	1.00	1.00	1.00
	Normal	1.00	0.93	0.97
	Replay	0.94	1.00	0.97
	Spoofing	1.00	1.00	1.00
Bagging	Flooding	1.00	1.00	1.00
	Fuzzing	0.99	1.00	0.99
	Normal	0.99	0.92	0.96
	Replay	0.92	0.98	0.95
	Spoofing	1.00	1.00	1.00
Voting	Flooding	0.99	1.00	0.99
	Fuzzing	1.00	0.99	0.99
	Normal	1.00	0.89	0.94
	Replay	0.90	0.99	0.95
	Spoofing	1.00	1.00	1.00

and

Table 7. The evaluation results for each attack using the supervised models and ensemble methods with feature selection.

Model	Attacks	Precision	Recall	F1-Score
RF	Flooding	1.00	1.00	1.00
	Fuzzing	0.99	1.00	1.00
	Normal	1.00	0.93	0.96
	Replay	0.93	1.00	0.96
	Spoofing	1.00	1.00	1.00
DT	Flooding	1.00	1.00	1.00
	Fuzzing	1.00	0.99	0.99
	Normal	1.00	0.93	0.96
	Replay	0.99	1.00	0.99
	Spoofing	0.93	1.00	0.96
XGBoost	Flooding	1.00	1.00	1.00
	Fuzzing	1.00	1.00	1.00
	Normal	1.00	0.94	0.97
	Replay	0.94	1.00	0.97
	Spoofing	1.00	1.00	1.00
Stacking	Flooding	1.00	1.00	1.00
	Fuzzing	1.00	1.00	1.00
	Normal	1.00	0.97	0.98
	Replay	0.97	1.00	0.99
	Spoofing	1.00	1.00	1.00
Bagging	Flooding	1.00	1.00	1.00
	Fuzzing	0.99	1.00	0.99
	Normal	0.99	0.92	0.95
	Replay	0.93	0.98	0.95
	Spoofing	1.00	1.00	1.00
Voting	Flooding	0.99	1.00	0.99
	Fuzzing	1.00	1.00	1.00
	Normal	1.00	0.92	0.96
	Replay	0.93	0.99	0.96
	Spoofing	1.00	1.00	1.00

, respectively.

Figure 9 shows that incorporating feature selection into models and ensemble methods demonstrates that it reduces training and testing time for all models. The additional training time required for the stacking method with feature selection is a trade-off for improved performance. The stacking strategy requires the longest training time. Stacking is a technique that requires two distinct stages of training. In our experiment, there are three base classifiers and one meta-learner. Incorporating feature selection into models and ensemble methods reduces training and testing time. The use of tree-based models (RF, DT, and XGBoost) is well-suited for real-time deployment due to their efficient parallel processing capabilities and low inference latency. Although stacking involves a two-stage architecture that incurs a higher training cost, its real-time prediction performance remains practical. In addition, computational overhead is significantly reduced by selecting features, as evidenced by the shorter training and testing times seen in Figure 9. These factors collectively support the suitability of the proposed ensemble IDS for deployment in resource-constrained in-vehicle environments.

Table 8. Hyperparameter configurations for supervised learning models.

Model	Hyperparameter	Value
Random Forest	max_features	‘auto’
	n_estimators	100
	max_depth	20
	min_samples_leaf	2
	min_samples_split	5
XGBoost	n_jobs	−1
	max_depth	3
	n_estimators	300
	objective	‘multi:softprob’
Decision Tree	max_depth	20
	max_features	‘log2’

presents the hyperparameter configurations used for each supervised learning model in our study.

In Figure 10 for the stacking method, we visualize multiclassification using the AUC-ROC curve at each classification threshold. The False Positive Rate and the True Positive Rate are presented on the x-axis and the y-axis, respectively. It is preferable to have a high AUC-ROC score because this indicates the robustness of the model. Specifically, an AUC-ROC value near 1.0 indicates a robust model. In Figure 10, 0 represents DoS attacks, 1 represents fuzzing attacks, 2 represents normal messages, 3 represents replay attacks, and 4 represents spoofing attacks.

In

Table 9. Our method compared to promising current approaches.

Citation	Attacks	Model	Precision	Recall	F1-Score	ROC
[69]	Flood	KNN	0.99	0.99	0.99	-
		Accuracy			0.96
		SVM	0.99	0.99	0.99	-
		Accuracy			0.97
	Fuzzing	SVM	1.00	0.96	0.97	-
		Accuracy			0.97
		KNN	0.99	0.96	0.97	-
		Accuracy			0.96
[53]	Spoofing	KNN-based RQA, K = 8				0.959
		Accuracy	0.914
		DT-based RQA				0.85
		Accuracy	0.853
		KNN-based RQA, K = 7				0.950
		Accuracy	0.927
[54]	Benign	CNN-LSTM	0.99	1.00	99	1.00
	Flood		0.66	11	18	0.77
	Fuzzing		97	1.00	0.99	0.91
	Accuracy		0.9730
[55]	DoS	SVM	0.99	0.99	0.99	0.99
		KNN	0.99	0.99	0.99	0.99
	Fuzzing	SVM	0.99	0.99	0.99	0.99
		KNN	0.99	0.98	0.99	0.98
	Spoofing	SVM	0.97	0.93	0.95	0.93
		KNN	0.97	0.93	0.95	0.93
	Mix attack	KNN	0.98	0.96	0.97	0.96
		Accuracy	0.9792
		SVM	0.98	0.96	0.97	0.96
		Accuracy	0.9799
Our	Flood	Stacking	1.00	1.00	1.00	1.00
	Fuzzing		1.00	1.00	1.00	1.00
	Normal		1.00	0.97	0.98	1.00
	Replay		0.97	1.00	0.99	1.00
	Spoofing		1.00	1.00	1.00	1.00
	Accuracy		0.986

, a comparison is presented of the proposed method and other promising recently proposed methods [53,54,55]. Evaluation metrics for the same dataset show promising results for our model. The experimental findings validate the effectiveness of the proposed IDS in detecting and classifying attacks against in-vehicle networks. Ensemble methods based on IDSs improve the efficiency with which these four attacks can be detected in the network’s traffic. Bagging, voting, and stacking are the three methods utilized in the development of ensemble classifiers. The results of the evaluation demonstrate that the proposed stacking method achieved the highest multiclassification performance while requiring more training and testing time. Stacking employs two stages to discover and learn how to effectively combine the individual models in the base learners with the meta learners, ultimately enhancing the performance of supervised classifiers.

Compared to deep learning-based IDS solutions (e.g., CNN-LSTM, GRU, etc.), our ensemble learning framework offers several advantages. First, it requires significantly less training data and computational resources, making it suitable for deployment in resource-constrained in-vehicle environments. Second, tree-based models are more interpretable, allowing developers to trace decisions back to input features. While deep learning methods often show strong generalization, they are prone to overfitting and require large labeled datasets, which are scarce in the automotive domain. A limitation of ensemble methods is that they may require longer training time when stacking is used, as seen in our timing analysis (Figure 9), but this tradeoff results in higher accuracy and robustness.

6. Conclusions

The CAN bus is vulnerable to various attacks as it lacks security features. We proposed a multi-classification intrusion detection system based on an ensemble learning solution to detect common CAN bus attacks (DoS, fuzzing, replay, and spoofing). We design an effective ensemble learning-based IDS scheme for classifying DoS, fuzzing, replay, and spoofing attacks. These are common CAN bus attacks that can threaten the safety of a vehicle’s driver, passengers, and pedestrians. Feature selection, data balancing, and hyperparameter tuning were all used to enhance the proposed scheme. The combination of these techniques and the ensemble learning methods resulted in a robust and accurate multi-classification IDS for common CAN bus attacks. With an F1-score of 0.993, precision of 0.993, recall of 0.993, and accuracy of 0.986, the stacking method outperformed other recently proposed methods, achieving the highest performance. However, stacking requires more training and testing time. While our results are promising, we acknowledge that the evaluation was conducted using a single dataset. Although the dataset is realistic and comprehensive in terms of attack types, this limitation may affect the generalizability of our findings. In future work, we aim to validate our IDS framework on multiple CAN bus datasets or in simulated real-time environments to ensure broader applicability and robustness.