H-RT-IDPS: A Hierarchical Real-Time Intrusion Detection and Prevention System for the Smart Internet of Vehicles via TinyML-Distilled CNN and Hybrid BiLSTM-XGBoost Models

Abstract

The integration of connected vehicles into smart city infrastructure introduces critical cybersecurity challenges for the Internet of Vehicles (IoV), where resource-constrained vehicles and powerful roadside units (RSUs) must collaborate for secure communication. We propose H-RT-IDPS, a hierarchical real-time intrusion detection and prevention system targeting two high-priority IoV security pillars: availability (traffic overload) and integrity/authenticity (spoofing), with spoofing evaluated across multiple subclasses (GAS, RPM, SPEED, and steering wheel). In the offline phase, deep learning and hybrid models were benchmarked on the vehicular CAN bus dataset CICIoV2024, with the BiLSTM-XGBoost hybrid chosen for its balance between accuracy and inference speed. Real-time deployment uses a TinyML-distilled CNN on vehicles for ultra-lightweight, low-latency detection, while RSU-level BiLSTM-XGBoost performs a deeper temporal analysis. A Kafka–Spark Streaming pipeline supports localized classification, prevention, and dashboard-based monitoring. In baseline, stealth, and coordinated modes, the evaluation achieved accuracy, precision, recall, and F1-scores all above 97%. The mean end-to-end inference latency was 148.67 ms, and the resource usage was stable. The framework remains robust in both high-traffic and low-frequency attack scenarios, enhancing operator situational awareness through real-time visualizations. These results demonstrate a scalable, explainable, and operator-focused IDPS well suited for securing SC-IoV deployments against evolving threats.

1. Introduction

The rapid evolution of Internet of Vehicles (IoV) technology has become a cornerstone of modern smart city (SC) development, contributing significantly to safer roads, improved traffic coordination, and more efficient urban mobility [1,2]. In these environments, connected vehicles maintain continuous communication with roadside units (RSUs), vehicular cloud infrastructures, and centralized traffic systems, enabling real-time decision-making for tasks such as autonomous navigation, dynamic rerouting, and collision avoidance [3,4] (see Figure 1). This constant data exchange fosters decentralized intelligence, allowing transportation networks to adapt dynamically to shifting urban conditions [5,6]. Importantly, this ecosystem spans heterogeneous computational environments. Resource-constrained on-board units (OBUs) must collaborate with more powerful RSUs and cloud infrastructures, creating a multilayered processing landscape. At the city scale, scalable big-data stacks (e.g., Hadoop/Spark) enable high-throughput processing of diverse vehicular data and have proven effective for intelligent traffic monitoring [7,8,9].

Figure 1. Conceptual overview of IoV architecture and data exchange in smart city environments.

While SC-IoV deployments enable real-time coordination and safety improvements, they introduce a new cybersecurity environment that differs from traditional IT systems in terms of attack surface, operational impact, and system constraints. Unlike static networks, IoV entails vehicle-to-vehicle (V2V), vehicle-to-infrastructure (V2I), and vehicle-to-cloud (V2C) exchanges that create distinct attack entry points spanning from CAN bus and V2X radios to RSUs and cloud backends [10]. These heterogeneous links broaden the system’s exposure to malicious interference and complicate defensive design. The layered architecture in Figure 2 illustrates representative attack vectors across IoV layers—message injection, spoofing, and traffic overload. These attacks can compromise data integrity, endanger physical safety, and disrupt urban traffic flow.

Figure 2. Attack surface across IoV communication layers in smart city environments.

IoV environments face a broad range of cyber threats, but two categories are especially critical for safety and reliability: (1) availability, which is threatened by high-rate traffic overload attacks that delay or block safety messages, and (2) integrity/authenticity, compromised by the spoofing of plausible signals such as GAS, RPM, or SPEED. These two pillars form the foundation of the proposed defense framework, as both directly affect the timeliness and trustworthiness of vehicular communication. Rule-based approaches cannot adapt to such dynamic, stealthy intrusions, whereas machine learning (ML) models can capture temporal dependencies and evolving traffic behaviors, enabling accurate real-time discrimination under varying conditions. For this initial implementation, we narrow the scope to traffic overload and spoofing attacks using CICIoV2024. This controlled focus allows us to validate the hierarchical IDPS under realistic but manageable conditions before expanding to broader threat categories.

Our dataset does not include vehicle-to-everything (V2X)/basic safety messages (BSM) exchanges; in this work we restrict evaluation to CICIoV2024 (in-vehicle controller area network bus [CAN bus] traffic) to ensure methodological alignment with IoV/CAN-specific threats.

Moreover, the operation of IoV systems faces strict resource and timing limitations. While modern high-end vehicles integrate powerful domain controllers supporting advanced driver-assistance systems, many in-vehicle nodes—particularly in low-cost or legacy models—still rely on microcontroller-based electronic control units (ECUs) with limited memory and compute resources [11]. This creates stark disparities in on-vehicle processing capacity, and security mechanisms must be tailored accordingly. The time-sensitive nature of critical intersections demands millisecond-level decision-making, as any delay in intrusion detection could trigger cascading effects such as traffic congestion or accidents. Furthermore, the combination of vehicle mobility, fluctuating connectivity, and manufacturer-specific communication protocols complicate the design of adaptable and generalized detection models [12]. Together, these constraints highlight the need for a multilayered defense strategy capable of rapid local decisions and deeper upstream analysis.

These challenges (tight real-time budgets, safety stakes, and resource constraints) require a new breed of lightweight, understandable, and high-speed Intrusion Detection and Prevention Systems (IDPS). Recent hybrid detection models combining deep learning (DL) and gradient boosting have shown promising results for securing vehicular environments [13]. Building on this insight, our framework adopts a hierarchical defense strategy: a TinyML-based distilled CNN on vehicles for ultra-fast on-device detection, a BiLSTM-XGBoost hybrid model at RSUs for deeper temporal–spatial analysis of traffic flows, and a centralized Kafka–Spark Streaming layer for city-wide monitoring, coordination, and adaptive model updates [14,15]. This tiered design balances immediacy, depth of analysis, and operational scalability under realistic smart intersection conditions.

Many existing IDPSs suffer from critical limitations, despite various efforts to secure IoV environments. Traditional rule-based approaches fail to adapt to new attack methods, while standalone DL models, despite their high accuracy, require excessive computational resources, which make them unsuitable for SC environments with time-sensitive requirements [16]. Moreover, the fast operation of lightweight models, including decision trees and SVMs, does not allow them to detect sequential patterns in vehicular communication streams [17]. These limitations underline the necessity for hybrid approaches that combine temporal modeling with efficient decision mechanisms.

To address these limitations, we propose a multi-stage hybrid IDPS that leverages (i) the temporal modeling strengths of Bidirectional Long Short-Term Memory (BiLSTM) networks, (ii) the decision efficiency of XGBoost, and (iii) vehicle-level TinyML CNN students for ultra-lightweight detection. We begin by conducting offline benchmarking on the vehicular CAN bus dataset CICIoV2024, comparing several DL architectures including CNN, BiLSTM, LSTM, GRU, and FastKan. Based on detection accuracy and runtime metrics, BiLSTM was selected as the optimal temporal encoder. The final BiLSTM-XGBoost hybrid is deployed in a Kafka–Spark Streaming pipeline, enabling real-time classification of windowed CAN bus traffic simulated using CICIoV2024. At the same time, the TinyML CNN student, which was distilled from the hybrid BiLSTM-XGBoost model, provides a first layer of rapid filtering directly on vehicles, ensuring minimal latency for obvious threats.

Our approach builds on recent findings that hybrid DL-tree ensemble models can outperform monolithic networks in high-throughput edge security tasks [18]. Before real-time deployment, multiple deep and hybrid architectures were evaluated on CICIoV2024 to guide the selection of the BiLSTM-XGBoost hybrid model for its balance between accuracy and latency. The core contributions of this study are summarized below.

• A hierarchical, multi-stage IDPS that combines a TinyML-distilled CNN on vehicles for ultra-lightweight detection with a BiLSTM-XGBoost hybrid at RSUs for deeper temporal–spatial analysis.
  (Architecture and roles are detailed in Section 4; end-to-end (e2e) evaluation is in Section 7).
• Real-time deployment within a Kafka–Spark Streaming pipeline, enabling scalable, low-latency classification and automated prevention actions at SC intersections.
  (Streaming and actions described in Section 6.2; policy parameters detailed later in Section 6.2.2; average batch latency ≈ 153 ms with resilience under bursts in Section 7).
• A practical real-time dashboard integrated into the proposed Kafka–Spark IDPS pipeline, used to monitor detection outcomes, prevention actions, and system health, thereby supporting operator situational awareness in IoV environments.
  (Modules and data sources in Section 6.2.3; live operation in Section 7).

Figure 3 provides an overview of the proposed hierarchical multi-stage IDPS architecture.

Figure 3. Overview of the proposed hybrid real-time IDPS framework.

The remainder of this paper is organized as follows: Section 2 reviews related work on intrusion detection in IoV. Section 3 discusses security challenges at SC intersections and the motivation for a hierarchical defense. Section 4 presents the system architecture and threat model, including the roles of the vehicle, RSU, and central tiers. Section 5 describes the offline development and evaluation of various models. Section 6 details the real-time implementation of the full system using Kafka–Spark, covering both TinyML deployment at the vehicle level and hybrid model integration at the RSU. Section 7 reports the experimental results across the two levels and different operational modes, including performance metrics and latency for both TinyML and hybrid components. Section 8 discusses the system’s limitations, open issues, and possible directions for future research. Finally, Section 9 concludes the paper.

2. Related Work

The IoV continues to face growing cybersecurity threats, particularly through vulnerabilities in the CAN bus and V2I communication layers. Numerous studies have explored ML and DL approaches to enhance the effectiveness of IDPS. While these efforts have improved detection accuracy, many solutions remain limited to offline experimentation, and deployment-ready real-time integration is still uncommon.

The authors of [19] proposed one of the few truly real-time IDS solutions using a Kappa Architecture. Their system combines Spark Streaming with ensemble learning models—Random Forest, XGBoost, and Decision Trees—to detect CAN bus attacks such as DoS, spoofing, fuzzing, and replay. Achieving up to 98.5% accuracy and low-latency detection (as fast as 0.26 s), the work demonstrates solid real-time processing capabilities. However, the system lacks attack prevention and operational monitoring, and its traffic simulation capabilities are restricted, which our framework addresses.

Similarly focused on real-time detection, ref. [20] introduced HistCAN, a self-supervised lightweight IDS for in-vehicle networks. HistCAN uses a hybrid encoder with historical information fusion to learn from benign CAN traffic and detect anomalies via reconstruction errors. With F1-scores up to 0.9954 and throughput near 8900 frames per second on GPU, the system is well suited for embedded deployment. Despite this efficiency, it does not incorporate data streaming, prevention logic, or live operator feedback—capabilities that our Kafka–Spark pipeline and dashboard explicitly provide.

Offline-focused studies have also advanced IoV IDS models. Ref. [21] evaluated a broad set of traditional ML models on CICIoV2024, without balancing, thus preserving real traffic priors. Using Optuna for hyperparameter tuning, XGBoost and other ensembles achieved near-perfect accuracy. Nevertheless, the study remains confined to static offline evaluation and does not include real-time processing or mitigation strategies.

Expanding on architectural design, Uddin et al. [22] proposed a scalable hierarchical IDS for IoV, organizing detection into layered classifiers with Boruta-based feature selection. Although conceptually promising for edge–cloud collaboration, the work is neither implemented in real-time nor equipped with streaming, prevention, or monitoring functionalities.

Several DL ensemble approaches focus on accuracy via multi-perspective feature extraction. El-Gayar et al. [23] proposed DAGSNet, combining DenseNet, AlexNet, GoogleNet, and SqueezeNet with wavelets and attention for feature fusion. While achieving high offline accuracy, the approach is computationally heavy, non-adaptive, and not suited to CAN bus or real-time constraints.

Ref. [24] introduced DFSENet, using lightweight ensembles to achieve fast classification with reported inference times of 12 ms. However, the model is evaluated only offline and lacks integration with real-time alerts or traffic simulation, which limits its operational readiness.

In the context of hybrid DL, Kamal and Mashaly [25] proposed a CNN-MLP model enhanced by ADASYN and SMOTE to balance IoT data. Despite strong binary and multi-class results, the study is generic to IoT, lacks CAN bus specificity, and provides neither real-time streaming nor attack response logic.

Ref. [26] evaluated multiple DL architectures—including BiLSTM, GRU, and CNN—across three CAN datasets. The models achieved near-perfect static performance, confirming the value of temporal learning. However, their setup does not incorporate real-time adaptability, prevention actions, or system-level observability.

With the rise of edge-ready security, ref. [27] proposed a TinyML-based IDS for ultra-constrained devices (e.g., Arduino UNO), exploring ensembles (Random Forest, Extra Trees, and XGBoost) for low-latency inference under tight memory/compute budgets. While demonstrating the feasibility of distributed edge detection, the study omits attack response, live streaming or simulation, and operator-oriented feedback mechanisms.

Meanwhile, ref. [28] introduced a TinyML-based IDS targeting generic IoT environments. A lightweight CNN was deployed with TensorFlow Lite for microcontrollers using quantization and pruning for efficiency. Despite strong results on NSL-KDD and BoT-IoT, the approach targets generic network traffic rather than CAN bus and lacks domain-specific optimizations such as spoofing characteristics, temporal dependencies, or coordinated-attack scenarios. It also does not include streaming, prevention mechanisms, or real-time monitoring.

In a closely related study, ref. [29] developed a TinyML-based IDS for CAN bus using a compact CNN on an nRF52840 microcontroller. A dual-branch design processes CAN ID sequences and payloads, enabling accurate detection of spoofing and fuzzing under strict resource limits. Although this validates true TinyML deployment for local anomaly detection, the approach remains detection-only and does not integrate simulation frameworks, real-time streaming, or attack response logic.

In contrast to these works, our study delivers a deployment-ready, real-time IDPS built on a Kafka–Spark Streaming backbone. It combines deep temporal encoding (BiLSTM) with fast, accurate classification (XGBoost) at the RSU tier, plus a TinyML-distilled CNN for ultra-lightweight on-vehicle detection. Beyond detection, it supports dynamic attack simulation (baseline, stealth, and coordinated), automated prevention (blocking and quarantine), and operator-facing monitoring for continuous situational awareness, closing the gap between academic prototypes and operational IoV security.

More specifically, the proposed H-RT-IDPS introduces several capabilities not jointly provided by prior IoV or edge–cloud IDS frameworks. First, we deploy a TinyML CNN student directly on the vehicle, enabling on-board detection under strict OBU memory and latency constraints—an aspect missing in existing real-time IoV IDS research. Second, we apply a teacher–student knowledge-distillation pipeline that transfers calibrated decision boundaries from a BiLSTM-XGBoost hybrid at the RSU tier to the TinyML model, whereas prior work does not leverage cross-tier learning. Third, our system integrates Kafka–Spark for real-time streaming, attack simulation, automated prevention (blocking/quarantine), and dashboard-based operator monitoring, going beyond detection-only designs. Finally, unlike studies that treat spoofing as a single generic class, we provide fine-grained subclass detection (GAS, RPM, SPEED, and STEERING) alongside traffic overload, strengthening practical applicability to safety-critical IoV environments.

Beyond IoV-specific work, recent efforts in broader IoT and cloud security have explored complementary ideas such as time-aware deep learning and hierarchical fog-based defense architectures. For example, time-aware IDS models [30] integrate temporal sensitivity into intrusion detection for cloud and IoT infrastructures, while fog-based hierarchical security frameworks [31] distribute monitoring and analysis between edge, fog, and cloud layers. These frameworks share conceptual similarities with our multi-tier design, particularly the emphasis on time-sensitive detection and tiered processing. However, they do not address CAN bus semantics, fine-grained spoofing subclasses, or on-vehicle TinyML deployment, which are central to the proposed H-RT-IDPS.

For a consolidated view, Table 1 summarizes datasets, model families, and operational readiness (real-time, prevention, monitoring, TinyML, and simulation) across recent IoV IDS works.

Table 1. Comparative analysis of recent IDS in IoV.

Study	Dataset	Model Type	Real-Time	Prevention	Monitoring	TinyML	Attack Simulation
[19]	CAN bus (Real-world)	RF, XGBoost, DT (Ensemble)	✓	✗	✗	✗	● (Partial)
[20]	Car-Hacking	Self-supervised CNN + MLP	✓	✗	✗	✗	✗
[21]	CICIoV2024	XGBoost, RF, DNN	✗	✗	✗	✗	✗
[22]	CICIoV2024	Hierarchical ML	✗ (Concept Only)	✗	✗	✗	✗
[23]	CIC-IDS-2017 + Car-Hacking	CNN-based Ensemble	✗	✗	✗	✗	✗
[24]	Unspecified Vehicular	Lightweight DL Ensemble	✗	✗	✗	✗	✗
[25]	Generic IoT	CNN-MLP Hybrid	✗	✗	✗	✗	✗
[26]	Car-Hacking, OTIDS, SA	BiLSTM, GRU, VGG-16	✗	✗	✗	✗	✗
[27]	IoT Device (Arduino UNO)	TinyML Ensemble (RF, XGB, ET)	✓	✗	✗	✓	✗
[28]	NSL-KDD, BoT-IoT	TinyML CNN	✗	✗	✗	✓	✗
[29]	CAN bus (Custom)	TinyML CNN	✓	✗	✗	✓	✗
Our Work	CICIoV2024	BiLSTM-XGBoost Hybrid	✓	✓	✓	✓	✓ (Baseline, Stealth, Coordinated)

Legend: ✓ = yes; ✗ = no; ● = partial/limited support.

3. Smart City Intersection Security

Building on the limitations observed in prior work, this section examines security implications of deploying IDPS at smart-city (SC) intersections—one of the most vulnerable and critical environments in the IoV ecosystem.

3.1. Background: In-Vehicle Architectures and IDS Placement

Vehicle electronics are transitioning from many domain-specific units to zonal layouts, where a small number of higher-capacity controllers supervise groups of ECUs [32]. We distinguish three roles:

• MCU-based ECUs(e.g., Arm Cortex-M): Small, low-power controllers for time-critical functions under tight memory/CPU budgets; suitable for lightweight screening (ID allowlists, basic rate/consistency checks, and TinyML).
• Gateway/Domain/Zonal controllers (often Cortex-A or automotive SoCs): Aggregate CAN/CAN-FD and Automotive Ethernet; host shared services and network IDS with cross-bus correlation and logging.
• Central compute/HPC: Platforms for compute-intensive functions (e.g., ADAS and IVI) and fleet-scale analytics.

In deployed systems, heavier intrusion-detection tasks reside on gateway/zonal controllers due to network visibility and compute headroom [32,33]. MCU ECUs provide first-line, low-latency screening and can run TinyML-scale anomaly filters when millisecond response is required [34]. Alerts/summaries are forwarded to the gateway for system-level correlation and action.

Placement in our framework. The vehicle tier performs TinyML screening on resource-limited ECUs; the RSU tier executes the hybrid analysis (BiLSTM feature extractor → XGBoost classifier) for real-time decisions; and the cloud tier correlates alerts across intersections and manages policy/model updates. Figure 4 summarizes the tiers and IDS placement.

Figure 4. The zonal vehicle architecture: IDS placement across MCU ECUs (lightweight screening), gateway/zonal controllers (network IDS/correlation), and central compute (analytics).

• Handoff and Synchronization Considerations:

A hierarchical architecture introduces potential blind spots at the boundary between the vehicle’s TinyML screening and the RSU’s temporal analysis. Because the OBU performs lightweight anomaly filtering, an attack that is too subtle for the TinyML CNN student yet insufficiently persistent may not trigger an explicit alert. To mitigate this, the vehicle tier periodically forwards compact summaries of recent CAN bus activity to the RSU, ensuring that the RSU always receives minimal contextual information even when no alert is raised. Synchronization across heterogeneous sampling rates is handled at the RSU: all received packets are re-timestamped upon arrival, and sliding-window formation for BiLSTM–XGBoost inference is carried out on a unified RSU-controlled timeline. Late or jittered packets are buffered within a tolerance bound before window assembly, thus preventing temporal drift between vehicles with different sensing, processing, or transmission latencies. This design minimizes misalignment and reduces blind-spot exposure at the hierarchical handoff.

3.2. Smart City Intersection Scenario and IoV Communication Flows

SC intersections are among the busiest points in urban transportation networks, where vehicles, roadside infrastructure, pedestrians, and cloud platforms interact in real-time. Each connected vehicle carries an on-board unit (OBU) that continuously shares telemetry (e.g., position, speed, and braking status), while RSUs at intersections aggregate this data, interface with traffic-light controllers, and relay information to centralized platforms. Pedestrian detectors and cameras complement this setup, forming a multi-tier IoV network across V2V, V2I, and V2C communication layers [35].

Communication at intersections typically occurs at three levels:

• V2V. Vehicles approaching the same intersection exchange basic information (speed and braking status). This enables earlier responses by revealing vehicles occluded by trucks or buildings. Vehicles near stopped cars at crosswalks receive immediate alerts, reducing crash risk.
• V2I. This layer coordinates vehicles, traffic lights, and RSUs. Emergency vehicles can request preemption (temporary green waves) and broadcast warnings to surrounding drivers. Daily drivers receive advance warnings about red lights, pedestrian crossings, and sudden road closures.
• V2C. Operating at a city-wide level, RSUs transmit data to the cloud, where it is fused across intersections. Traffic management systems then identify emerging congestion and adjust signal timing or reroute before problems escalate.

These three layers operate jointly to enhance overall performance: V2V supports rapid, safety-critical reactions; V2I manages local vehicle-infrastructure coordination; and V2C maintains city-level synchronization, helping intersections remain smooth under both routine and unexpected conditions [36].

Consequently, securing SC intersections benefits from a hierarchical, layered approach: lightweight detection on vehicles filters obvious threats at the edge; RSUs perform deeper temporal analysis and enforcement; and the cloud coordinates city-wide responses to maintain resilience [37,38].

3.3. Security Risks and the Need for Millisecond-Level Detection at Intersections

Attackers target intersections because small disruptions can have outsized consequences. False-signal injection may induce conflicting greens and crashes; fake emergency alerts can hijack traffic flow; and RSU-level traffic overload can disrupt vital V2I communications, preventing timely pedestrian/hazard warnings. Even brief interruptions can cascade into congestion, secondary collisions, and city-wide gridlock [39].

The urgency of intersection decisions amplifies these risks. Emergency braking and ambulance right-of-way decisions must occur within milliseconds; any detection or response delay leaves no recovery margin and directly affects safety.

Recent work reinforces the importance of resilient cyber–physical coordination in SC infrastructures. Authors in [40] highlight how integrating advanced computation models such as quantum computing with digital-twin technologies can strengthen the resilience of transportation and power systems against false-data injection attacks, one of the most critical threats facing modern Cyber–Physical Systems (CPS). Similarly, authors in [41] examine system-level remedial strategies for mitigating FDI attack impacts, emphasizing the need for coordinated, multi-layer defensive mechanisms in safety-critical infrastructures. These insights support the requirement for multi-tier, low-latency intrusion detection at urban intersections, aligning with the design principles of our H-RT-IDPS framework.

To address these risks, we adopt a multilayered defense:

• Vehicle tier (TinyML). The system performs first-line filtering against spoofed or inconsistent messages using lightweight on-device models.
• RSU tier (temporal hybrid). The BiLSTM-XGBoost analysis is utilized to analyze sophisticated or coordinated patterns at the intersection scale.
• Cloud tier (correlation). Aggregation across intersections to detect large-scale threats via alert correlation and global policies [6].

Table 2 summarizes the main communication types at intersections, their normal functions, potential attack vectors, and how the proposed multi-tier IDPS mitigates these threats.

Table 2. Attack types at intersections, their impact on vehicular communication, and corresponding mitigation strategies.

Communication Type	Normal Function	Types of Attack	Impact on Communication and Scenario	Mitigation (TinyML or RSU IDPS)
V2V	Vehicles exchange safety info (speed, position, braking) for collision avoidance and platooning	Spoofing fake brake/lane-change messages, false location/speed broadcasts, traffic overload flooding with fake beacons	Vehicles react incorrectly, causing sudden braking and chain collisions Flooding overwhelms OBUs, causing legitimate safety messages to be lost	TinyML inside OBUs detects inconsistent speed/position Blocks fake beacons locally before forwarding RSU monitors abnormal high-rate broadcasts
V2I	The vehicle sends sensor data to the RSU and receives traffic-light timing or hazard alerts	Traffic overload flooding RSU, fake RSU impersonation, packet tampering, or delay injection	RSU overwhelmed, leading to delayed or dropped legitimate messages Vehicles miss critical traffic-light signals, causing chaos A fake RSU sends wrong light timing, leading to collisions	RSU-level BiLSTM-XGBoost IDPS detects traffic overload or spoofed RSU messages Blocks malicious senders to maintain RSU uptime
V2C	Vehicles upload aggregated data for city-wide traffic analytics and receive routing/security updates	Alert poisoning, data flooding, message injection	Cloud mispredicts traffic, causing wrong rerouting and congestion Delay in security updates or model retraining	RSU filters all data with IDPS before forwarding to the cloud. Cloud only receives verified, clean data
V2P	Pedestrian devices broadcast crossing intent to vehicles for safety	Fake pedestrian crossing broadcasts, location spoofing, privacy attacks	Vehicles stop unnecessarily, disrupting traffic flow. Fake crossings trigger repeated rerouting or braking	RSU cross-validates pedestrian signals with multiple devices TinyML on the vehicle verifies request realism vs. camera/LiDAR sensors

3.4. Justification for a Hierarchical Defense Strategy

Securing SC intersections is best achieved with a defense-in-depth architecture that assigns each security function to the tier whose compute and latency budgets fit the task. Vehicles operate under stringent memory/CPU limits and must react within milliseconds; RSUs can sustain richer temporal analytics at the intersection scale; the cloud aggregates multi-intersection signals for city-wide situational awareness but cannot provide immediate, per-packet intervention.

Vehicle tier (on-board TinyML).

On-board units (OBUs) run lightweight models that provide the first line of defense by filtering obvious anomalies in situ (e.g., spoofed speed/braking signals or location claims). By suppressing malicious traffic before it propagates, on-vehicle filtering reduces unsafe actuation risk and lowers downstream load-critical under intersection-level deadlines. Events that exceed this tier’s capacity or show non-local patterns are forwarded to the RSU for deeper analysis.

RSU tier (intersection-scale temporal analytics).

Roadside units collect streams from multiple vehicles and analyze patterns over time. Leveraging the BiLSTM-XGBoost hybrid, this tier targets coordinated or timing-sensitive threats (e.g., spoofing combined with traffic overload bursts or infrastructure impersonation) while meeting the operational latency bounds required for live signal control. With greater processing headroom than vehicles, RSUs perform deeper spatio-temporal inference and enforce prevention at the intersection boundary. Summaries and alerts are then promoted to the cloud when cross-site correlation or longer-horizon analysis is needed.

Cloud tier (city-wide correlation and adaptation).

The cloud correlates alerts across RSUs to surface large-scale behaviors that may be invisible locally (e.g., district-level congestion spoofing), supports the model lifecycle (updates, calibration, and drift handling), and provides operator-facing situational awareness. Rather than acting as a real-time control point, it refines policy and improves lower-tier models using historical and cross-site evidence. Updated policies/models are then propagated back to RSUs and vehicles.

In combination, the three tiers deliver immediate edge responses, deeper intersection scale assessment, and city-wide coordination. This layered structure also yields graceful degradation: if one tier is impaired, the others continue to provide protection appropriate to their scope, preserving safety and continuity of operations at SC intersections.

4. System Architecture and Threat Model

4.1. Description of the Three-Tier IoV Environment

The proposed hierarchical IDPS leverages the heterogeneous compute resources of the IoV ecosystem. It comprises three interconnected tiers—vehicle, RSU, and central—that partition protection and analytics to achieve low-latency, fine-grained detection with city-wide coordination.

Figure 5 provides a conceptual overview of the three-tier architecture.

Figure 5. The three-tier architecture of the proposed hybrid real-time IDPS.

4.1.1. Vehicle Tier: TinyML-Based Detection

The vehicle tier forms the first layer of defense within the IoV security architecture. Each connected vehicle is equipped with an OBU hosting a TinyML CNN student obtained via knowledge distillation from the BiLSTM-XGBoost teacher.

In the offline phase, the BiLSTM-XGBoost hybrid demonstrated superior temporal feature extraction and high detection accuracy. Through knowledge distillation, this capability was transferred to a compact student CNN optimized for minimal memory footprint, reduced computational overhead, and ultra-low (sub-millisecond) inference latency—well suited for deployment on resource-constrained automotive ECUs.

This vehicle-level TinyML CNN student focuses on the two most safety-critical attack categories that can cause immediate disruptions:

• Spoofing attacks, such as fake speed, braking, or GPS messages that could trigger false emergency maneuvers or collisions.
• Traffic overload attacks on the CAN bus, which attempt to saturate communication channels and delay or block legitimate safety messages.

The vehicle tier detects and stops high-priority threats locally, which prevents malicious data from reaching RSUs, reduces upstream load, and mitigates risks such as chain collisions and unsafe actuations. Only messages that pass local checks are forwarded to the RSU tier for additional temporal–spatial analysis.

4.1.2. RSU Tier: BiLSTM-XGBoost Hybrid Model for Traffic Overload and Multiple Spoofing Attacks

The RSU tier serves as the intersection-edge security layer. RSUs have greater computational resources than vehicle OBUs, enabling more sophisticated models while still meeting strict real-time requirements.

Each RSU runs a hybrid detector: BiLSTM for temporal feature extraction and XGBoost for classification. The BiLSTM captures temporal dependencies in CAN bus sequences, enabling detection of gradual spoofing patterns and burst-like traffic overload attacks, while XGBoost consumes these features to deliver fast, accurate predictions.

• Hybrid decomposition.

We model the RSU detector as a composition of an encoder and a classifier:

$$\begin{array}{cc}\hfill {\pi }_i& =g\left({\varphi }_{\theta }\left(W_i\right)\right),\hfill \end{array}$$

(1)

$$\begin{array}{cc}\hfill {\widehat{y}}_i& =arg\underset{k\in \mathcal{Y}}{max}{\left[{\pi }_i\right]}_k,\hfill \end{array}$$

(2)

$$\begin{array}{cc}\hfill c_i& =\underset{k\in \mathcal{Y}}{max}{\left[{\pi }_i\right]}_k.\hfill \end{array}$$

(3)

where

• $W_i\in {\mathbb{R}}^{w\times d}$: the i-th input window (length w time steps, d channels/features).
• ${\varphi }_{\theta }:{\mathbb{R}}^{w\times d}\to {\mathbb{R}}^m$: BiLSTM encoder with parameters $\theta$; maps a window to an m-dimensional feature vector.
• $g:{\mathbb{R}}^m\to {\Delta }^{\left|\mathcal{Y}\right|-1}$: XGBoost classifier producing class probabilities on the simplex ${\Delta }^{\left|\mathcal{Y}\right|-1}:=\{p\in {\mathbb{R}}^{\left|\mathcal{Y}\right|}:p_k\ge 0,{\sum }_{k\in \mathcal{Y}}{p}_k=1\}$.
• ${\pi }_i$: predicted class-probability vector for window $W_i$; ${\left[{\pi }_i\right]}_k$ denotes the probability of class k.
• $\mathcal{Y}$: the class set (e.g., {BENIGN, DoS, GAS, RPM, SPEED, STEERING_WHEEL}).
• $K=\left|\mathcal{Y}\right|$: number of classes.
• ${\widehat{y}}_i$: predicted class label.
• $c_i$: model confidence (maximum class probability), used by the prevention policy.
• $i\in \{1,\dots ,N\}$: window index.
• $k\in \{1,\dots ,K\}$: class index.
• $w,d,m$: window length, per-timestep feature dimension, and encoder output dimension.

This separation preserves temporal expressivity (via ${\varphi }_{\theta }$) while enabling low-latency decisions and calibrated probabilities (via g).

For reproducibility, the BiLSTM encoder is implemented using a lightweight temporal architecture composed of the following layers: (1) an input layer of size $10\times 8$ (window length × CAN bus features), (2) a 1D convolutional layer with 64 filters (kernel size = 3, activation = ReLU) to enhance local temporal patterns, followed by a dropout rate of 0.25, (3) a single Bidirectional LSTM layer with 64 hidden units per direction (return_sequences = False), producing a 128-dimensional latent representation, (4) a dense layer of 64 units with ReLU activation to refine the embedding, and (5) a final dropout layer of 0.25 before the feature vector is passed to the XGBoost classifier. This encoder matches the implementation used in our experiments and provides the exact $m=128$ feature vector consumed by the downstream model.

Unlike the TinyML CNN student at the vehicle tier, which performs basic local filtering, the RSU-tier hybrid executes deeper temporal analysis over (i) traffic overload patterns on CAN-derived telemetry and (ii) payload spoofing subclasses defined in CICIoV2024-Spoofing_GAS (false accelerator signals), Spoofing_RPM (fabricated engine speed), Spoofing_SPEED (incorrect vehicle speed), and Spoofing_STEERING_WHEEL (fake steering angle). When an anomaly is detected, the RSU-tier prevention logic mitigates threats by

• Blocking CAN bus streams from vehicles identified as traffic overload sources.
• Quarantining nodes suspected of spoofing to prevent propagation of false data.
• Emitting security alerts to the central dashboard for operator awareness and system-wide monitoring.

By combining real-time detection with immediate containment actions (blocking and quarantining), the RSU tier provides robust intersection-level security and complements the ultra-fast vehicle-tier TinyML filtering.

4.1.3. Central Tier: Dashboard for Local Monitoring and Future Coordination

The central tier provides monitoring and coordination for the hierarchical IDPS. Operating in the cloud or a city-level backend, it aggregates alerts from RSUs across intersections.

This layer offers the following key functions:

• An integrated security dashboard provides operator situational awareness.
• Cross-intersection alert analysis exposes city-wide, coordinated attack campaigns.
• Adaptive updates support retraining and model distribution to TinyML and RSU tiers as new threats emerge.

The central tier does not manage traffic flow directly; rather, it strengthens operational decisions through visual pattern analysis, historical trend review, and proactive model improvement. In full SC deployments, the central tier supervises many RSUs, maintains city-wide security intelligence, and supports long-term adaptive learning against evolving threats.

4.2. Threat Model

The proposed hierarchical IDPS addresses high-priority attack vectors in the IoV, focusing on communication flows between vehicles, RSUs, and the central backend. These threats directly target two core IoV security pillars—availability and integrity/authenticity— which, if compromised, can cause immediate safety and operational failures at SC intersections.

4.2.1. Current Attack Coverage

In the current implementation, the IDPS detects and mitigates two high-impact attack families—availability (traffic overload on CAN bus) and integrity/authenticity (payload spoofing)—with spoofing further evaluated across multiple subclasses (GAS, RPM, SPEED, STEERING_WHEEL) present in CICIoV2024 (Table 3). These families were prioritized for their immediate safety impact and the availability of labeled data for rigorous evaluation; the framework remains modular for extension to additional IoV threats.

Table 3. Current attack categories covered by the proposed hierarchical IDPS.

Attack Category	Description	Impact
Traffic Overload	Floods the CAN bus with high-frequency fake messages, saturating bandwidth and preventing legitimate safety messages (e.g., brake signals) from being processed.	Vehicles may fail to receive time-critical alerts, leading to accidents, intersection malfunctions, and traffic congestion.
Multiple Spoofing Attacks	Injects fake but plausible-looking values into CAN bus messages to mislead vehicle control units or RSUs. Covered spoofing types: Spoofing_GAS—false gas pedal signals Spoofing_RPM—fake engine RPM values Spoofing_SPEED—incorrect vehicle speed Spoofing_STEERING_WHEEL—fake steering-angle data	Vehicles may execute unintended maneuvers, and RSUs may make incorrect intersection decisions, creating safety hazards for nearby traffic.

Data source. Streaming windows are constructed directly from the raw CICIoV2024 decimal-formatted CAN frames; identifiers and timing are preserved. Only payload values are altered in spoofing scenarios.

• Traffic overload: mechanism and impact.

On a CAN bus, frames with lower numeric identifiers win arbitration. In a traffic overload, an adversary injects many priority-winning frames in a short time, raising bus utilization and delaying legitimate traffic. Safety-critical signals (e.g., braking and steering assistance) may be slowed or dropped, disturbing control loops and gateway timing. Launch (our system). The data generator emits fixed-length windows of size $w=10$ (each window contains $10\times 8$ payload bytes, flattened to 80 features). In stressed profiles (coordinated mode), it increases the proportion of DoS traffic and clusters it into short bursts using arbitration-winning identifiers, creating both sustained and pulsed overloads within the sliding window used by the detector. The streaming consumer applies the same $w=10$ windowing to compute per-ID rates and inter-arrival cues, encodes each window with a BiLSTM feature extractor trained offline on CICIoV2024 to obtain a 128-dimensional latent vector, and then classifies with XGBoost; detected overload triggers BLOCK.

• Payload spoofing: subclasses and impact.

Payload spoofing preserves the message identifier but modifies selected data bytes, producing plausible yet incorrect values and violating signal integrity.

spoofing GAS: changes to accelerator-position bytes ⇒ unintended throttle; Impact: unstable longitudinal control and higher rear-end risk.

spoofing RPM: changes to engine-speed bytes ⇒ misstated powertrain status; Impact: improper shift/idle behavior and masking of stalls/overspeed.

spoofing SPEED: changes to vehicle-speed bytes ⇒ misleading dynamics/timing; Impact: faulty ABS/ESP reactions and RSU decisions at intersections.

spoofing STEERING_WHEEL: changes to steering-angle bytes ⇒ disturbed lateral control; Impact: lane-keeping drift and unsafe path planning.

Launch (our system). The generator selects windows for the spoofing subclasses while preserving identifiers and timing; only payload values differ according to subclass distributions, inducing within-window inconsistencies across correlated signals. The consumer forms the same 80-dimensional window feature vector, encodes it with the BiLSTM feature extractor (CICIoV2024-trained) to a 128-dimensional latent vector, and classifies it with XGBoost; spoofing predictions trigger QUARANTINE + ALERT, while low-confidence cases ($q_i<\tau$, $\tau =0.6$) are routed to REVIEW.

While simple counters can detect full-bus floods, adversaries can employ low-rate or pulsed traffic overloads and coordinated payload spoofing that remain under static thresholds. Our approach targets these nuances by combining per-CAN-ID rates, inter-arrival statistics, and cross-signal consistency (e.g., SPEED-RPM-GAS), learned by a TinyML CNN student at the vehicle tier and a BiLSTM-XGBoost hybrid at the RSU tier for real-time discrimination. We reframe detection as load monitoring to encompass both malicious and benign anomalies, per Sommer & Paxson [42].

To make the scope and feature design precise, we formalize the two covered families as follows:

• Formalization and Action Policy.

Data and windows: Let $X_t\in {\mathbb{R}}^d$ denote the CAN frame at time t ($d=8$ bytes: DATA₀–DATA₇). With window length w and stride s, define the sliding window $W_i=(X_{t_i},\dots ,X_{t_i+w-1})\in {\mathbb{R}}^{w\times d}$ and its label $Y_i\in \{0,\dots ,K-1\}$ (BENIGN, traffic overload, Spoofing_GAS, Spoofing_RPM, Spoofing_SPEED, Spoofing_STEERING_WHEEL). For a stream of $F_{\mathrm{frames}}$ valid frames, the number of windows is

$$N=⌊{\displaystyle \frac{F_{\mathrm{frames}}-w}{s}}+1⌋$$

(4)

where

• $X_t$: CAN frame at time index t.
• $t_i$: start index (time) of window i.
• $F_{\mathrm{frames}}$: total number of valid frames in the stream (after filtering).
• $Y_i$: class label index for window i.
• $\lfloor ·\rfloor$: floor operator (greatest integer ≤ its argument).

Hybrid predictor (RSU tier): The detector follows the hybrid composition introduced earlier:

$${\pi }_i=g\left({\varphi }_{\theta }\left(W_i\right)\right),{\widehat{y}}_i=arg\underset{k\in \mathcal{Y}}{max}{\left[{\pi }_i\right]}_k$$

(5)

Uncertainty prevention actions: Let $c_i={max}_{k\in \mathcal{Y}}{\left[{\pi }_i\right]}_k$ denote the model confidence, and let $\tau =0.6$ be the uncertainty threshold. The prevention policy is defined as

$$\pi ({\widehat{y}}_i,c_i)=\left\{\begin{array}{cc}\mathsf{BLOCK},\hfill & {\widehat{y}}_i=\mathsf{traffic}\mathsf{overload}\wedge c_i\ge \tau ,\hfill \\ \mathsf{QUARANTINE},\hfill & {\widehat{y}}_i\in \{\mathsf{GAS},\mathsf{RPM},\mathsf{SPEED},\mathsf{STEERING}\_\mathsf{WHEEL}\}\wedge c_i\ge \tau ,\hfill \\ \mathsf{REVIEW},\hfill & c_i<\tau ,\hfill \\ \mathsf{ALLOW},\hfill & \mathrm{otherwise}.\hfill \end{array}\right.$$

(6)

where ∧ denotes logical “and”, and $\tau$ is the confidence threshold used for actioning (set to $0.6$). Policy parameters are listed in Table 11, and the real-time implementation is described in Section 6.2.

Coordinated attack handling: When spoofing and traffic overload appear together, the system applies a simple priority rule: BLOCK is always taken first to protect availability, followed by QUARANTINE for spoofing cases, and then ALLOW. The RSU makes the final decision whenever its confidence is above the threshold, since it has the full temporal view of the attack. If the RSU is unavailable or its confidence is low, the vehicle’s TinyML CNN student continues applying basic local checks until the RSU returns. This avoids conflicting decisions between layers and keeps prevention stable during combined attacks.

Feature rationale (signals and coherence): Although the encoder consumes byte-level frames, streaming monitors exposed for interpretability track per-CAN-ID emission rates ${\lambda }_{\mathrm{id}}$, inter-arrival statistics $\Delta t$, and cross-signal coherence (e.g., Pearson correlation $\rho$) across correlated IDs; anomalies in these monitors corroborate ${\pi }_i$ and the chosen action $\pi$.

• where

• ${\lambda }_{\mathrm{id}}$: emission rate (messages per unit time) for a given CAN ID.
• $\Delta t$: inter-arrival time between consecutive frames of the same CAN ID.
• $\rho$: Pearson correlation used for simple cross-signal coherence checks.
• ∧: logical conjunction (AND).

The choice of BiLSTM as the encoder is motivated by its ability to capture forward and backward temporal dependencies within each 10 × 8 CAN window. Spoofing subclasses modify different but correlated payload bytes, and their distinguishing temporal signatures emerge across short sequences. The BiLSTM produces 128-dimensional latent vectors that preserve these evolving patterns and cross-signal coherence, enabling XGBoost to establish clearer decision boundaries even under varying CAN bus load or noise.

TinyML pointer: The on-vehicle CNN student $s_{\psi }$ is distilled from the RSU teacher $g\circ {\varphi }_{\theta }$; where $s_{\psi }$: TinyML student model (CNN) with parameters $\psi$ deployed on-vehicle. The learning objective is given in Section 6.1.

The vehicle-tier TinyML CNN student filters obvious spoofing and traffic overload attacks locally, while the RSU-tier BiLSTM-XGBoost hybrid performs deeper temporal analysis to detect subtle or coordinated bursts that bypass vehicle-level filtering.

4.2.2. Future Threat Extensions

While the current IDPS focuses on traffic overload and spoofing families, the modular three-tier design provides a roadmap for extending coverage to advanced multi-vector threats, such as

• RSU impersonation: attackers mimic legitimate RSUs to inject false traffic-light signals.
• Large-scale coordinated flooding: multiple compromised vehicles simultaneously transmit malicious packets to overload RSUs.
• Data poisoning: adversaries inject benign-looking patterns to degrade models over time.

These extensions leverage city-wide alert correlation at the central tier and adaptive model updates distributed to RSUs and vehicles.

4.3. Operational Modes

To evaluate robustness against the two targeted threat families, three operational modes were designed—Baseline, Stealth, and Coordinated—reflecting typical urban IoV message patterns and worst-case adversarial conditions.

• Baseline. Demonstrates typical traffic with occasional isolated attacks.
• Stealth. Uses low-frequency, camouflaged spoofing/traffic overload to evade simple filters.
• Coordinated. Presents synchronized, multi-vehicle attacks combining traffic overload floods with spoofing.

Table 4 presents the essential characteristics for each mode, including traffic composition, average message rate, and maximum burst potential.

Table 4. Operational modes for testing the RSU-tier BiLSTM-XGBoost hybrid model.

Mode	Traffic Composition	Avg. Rate (msg/s)	Bursting?	Attack Behavior
Baseline	∼75% benign, $25\%$ random attacks	10	No	Occasional isolated spoofing or traffic overload attempts without coordination.
Stealth	∼90% benign, $10\%$ rare hidden attacks	5	No	Low-frequency spoofing/traffic overload camouflaged within benign traffic to evade simple filters.
Coordinated	∼35% benign, ∼65% malicious	13	Yes	Periodic bursts combining traffic overload floods with multiple spoofing types (GAS, RPM, SPEED, STEERING_WHEEL), aiming to overwhelm RSU detection capacity.

These modes emulate a spectrum of operational conditions. Baseline reflects typical urban traffic with sporadic, low-impact attacks; Stealth simulates adversaries who blend rare attacks into benign flows; and Coordinated stresses the system with synchronized, multi-vector campaigns designed to overwhelm detection. The hybrid model detects sporadic events in Baseline with minimal false positives, exploits temporal modeling in Stealth to surface rare anomalies, and, in Coordinated, demonstrates the ability to detect complex bursts and execute blocking/quarantining actions at the RSU tier.

5. Offline Model Development and Benchmarking

5.1. Datasets and Preprocessing

We used the CICIoV2024 [43] public dataset, which contains in-vehicle CAN bus traffic (benign plus traffic overload/spoofing) and is used to model vehicle-tier threats and to construct temporal windows for TinyML/BiLSTM.

CICIoV2024 Dataset

The CICIoV2024 dataset is a modern in-vehicle CAN bus benchmark for intrusion detection, containing authentic CAN traffic under normal driving and attack conditions. Each message record includes a CAN identifier (ID), eight data bytes (DATA_0-DATA_7), and three labeling fields: label (benign/malicious), category (attack family), and specific_class (exact attack type). Following methodological guidance [42], we exclude non-vehicular corpora to avoid misaligned features and focus strictly on CAN bus IoV threats.

Table 5 summarizes the attack composition and class distribution of CICIoV2024. The dataset comprises approximately 1.4 million CAN frames, covering both benign traffic and multiple high-impact CAN bus attacks.

Table 5. Attack composition and class distribution in CICIoV2024.

Attack Type	Description	Samples
BENIGN	Normal CAN frames without malicious injection	1,223,737
Denial-of-Service (DoS)	Flooding CAN bus with high-frequency messages to delay legitimate safety signals	74,663
Spoofing_GAS	Falsifying gas pedal signals to force unintended acceleration	9991
Spoofing_RPM	Injecting fake engine RPM values	54,900
Spoofing_SPEED	Broadcasting incorrect vehicle speed readings	24,951
Spoofing_STEERING	Manipulating steering angle values to mislead driver-assist systems	19,977

This imbalance reflects real-world conditions, where malicious events are rare compared to normal traffic, requiring the use of class weights during training.

Preprocessing steps. To exploit the temporal nature of CAN traffic, we applied a sliding-window strategy:

• Normalization: All DATA_0 to DATA_7 byte values were scaled with a StandardScaler to zero mean and unit variance:

  $$x^{\prime }={\displaystyle \frac{x-\mu }{\sigma }}$$

  (7)
• Sliding-window generation: Consecutive CAN frames were grouped into 10-message windows, producing input tensors of shape $(10\times 8)$. The label of each window was determined by the majority attack type within the window.
• Label encoding: Categorical labels (BENIGN, DoS, and spoofing types) were mapped to integer classes for model compatibility.
• Class balancing: Computed class weights mitigated the effect of minority attack categories.
• Train/test split: A stratified 85/15 split ensured proportional representation of all attack types in both training and test sets.

To avoid any form of data leakage, we explicitly clarify our partitioning strategy. The train–test split is performed randomly at the frame level after shuffling the full CICIoV2024 dataset. No scenario-based or session-based partitioning is used. CICIoV2024 does not organize attacks into long campaign sessions; instead, attacks appear as short, independent CAN bus message windows. As a result, shuffling followed by random frame-level splitting does not allow any coherent attack campaign, temporal trace, or contiguous sequence of injected messages to appear in both training and testing sets. This prevents leakage of attack patterns and ensures that the reported accuracy, precision, and recall reflect a physically plausible evaluation setting for CAN bus intrusion detection.

The resulting preprocessed dataset produces the following:

$${\mathbf{X}}_{\mathrm{seq}}\in {\mathbb{R}}^{N\times 10\times 8}\mathrm{and}{\mathbf{y}}_{\mathrm{seq}}\in \{0,1,2,3,4,5\}.$$

Using Equation (4) with $F_{\mathrm{frames}}$ = 1,408,219, $w=10$, and $s=1$, we obtain

$$N=1,408,210.$$

• N = 1,408,210 is the total number of sliding windows produced (via Equation (4)).
• Each window spans 10 consecutive CAN frames; each frame has 8 normalized data bytes (DATA_0–DATA_7).
• ${\mathbf{y}}_{\mathrm{seq}}$ encodes window labels: $0=\mathrm{BENIGN}$, $1=\mathrm{traffic}\mathrm{overload}$, $2=\mathrm{Spoofing}\_\mathrm{GAS}$, $3=\mathrm{Spoofing}\_\mathrm{RPM}$, $4=\mathrm{Spoofing}\_\mathrm{SPEED}$, $5=\mathrm{Spoofing}\_\mathrm{STEERING}\_\mathrm{WHEEL}$.

The preprocessing pipeline for CICIoV2024 is summarized in Figure 6.

Figure 6. Preprocessing pipeline for CICIoV2024.

This transformation preserves the sequential dynamics of CAN bus signals, making it suitable for temporal models such as BiLSTM and GRU. Also, such preprocessing strategies and dataset refinements have also been emphasized in recent IoV/IoT intrusion-detection research, where handling class imbalance and realistic traffic distributions is critical for reliable evaluation [44].

5.2. Benchmarking Models

A wide range of DL and hybrid models were evaluated on the two datasets to determine which architecture achieves the best balance between detection accuracy and computational cost. The models analyze different patterns in IoV traffic behavior:

• CNN: Effective at detecting spatial patterns in individual CAN frames and flow-based features; limited in capturing long-range temporal patterns.
• LSTM: Specializes in sequential analysis and can detect delayed attack signatures via time-based patterns; computationally heavier than CNNs.
• GRU: A simpler recurrent alternative to LSTM with fewer parameters, trading some expressivity for speed.
• BiLSTM: Extends LSTM with forward and backward processing, which helps identify complex temporal relations and detect stealthy attacks.
• FastKAN: Uses kernel activation networks to extract lightweight features while maintaining compactness for high-speed inference.
• XGBoost: A decision-tree ensemble that delivers fast classification on tabular features; performs strongly when fed deep features, but does not model temporal dynamics directly.
• Hybrid (BiLSTM + XGBoost): BiLSTM provides temporal features, and XGBoost performs classification, yielding high accuracy with low inference latency suitable for RSU-level deployment. Recent studies further corroborate that combining temporal deep feature extractors with efficient classifiers strengthens resilience to network-level threats and timing uncertainties in connected vehicles [45,46].

Training setup and hyperparameters:

All models were trained on imbalance-aware version of CICIoV2024 using consistent preprocessing and evaluation procedures. CNN, LSTM, GRU, and BiLSTM were implemented in TensorFlow and trained for up to 10 epochs with early stopping (patience = 3, best weights restored), batch size 512, Adam optimizer, and categorical cross-entropy loss, with a validation split of 0.2. FastKAN was trained in PyTorch with a batch size of 1024 and a learning rate of 0.001. XGBoost and hybrid variants used 100 estimators, a learning rate of 0.1, and a maximum depth of 5. Inputs consisted of flattened CAN bus windows of shape $(10\times 8)$. All experiments were executed on a CPU-only laptop to reflect RSU-level constraints. The configuration in Table 6 was applied consistently on CICIoV2024 to ensure a fair and reproducible comparison within the CAN domain.

Table 6. Model training hyperparameters.

Model	Framework	Batch Size	Max Epochs	Early Stop	Learning Rate	Loss Function	Notes
CNN	TensorFlow/Keras	512	10	Yes (patience = 3)	default	Categorical cross-entropy	Conv1D (64 filters, kernel = 3), GlobalMaxPooling, class weights applied
LSTM	TensorFlow/Keras	512	10	Yes (patience = 3)	default	Categorical cross-entropy	64 units, Dropout 0.3, class weights applied
BiLSTM	TensorFlow/Keras	512	10	Yes (patience = 3)	default	Categorical cross-entropy	Bidirectional (64), Dropout 0.3, class weights applied
GRU	TensorFlow/Keras	512	10	Yes (patience = 3)	default	Categorical cross-entropy	64 units, Dropout 0.3, class weights applied
FastKAN	PyTorch	1024	10	No	0.001	CrossEntropyLoss	MLP: [80 → 64 → 6], no class weights
XGBoost	XGBoost (scikit-learn API)	-	-	-	0.1	multi:softprob	100 estimators, max_depth = 5, no class weights

For clarity and reproducibility, we additionally report the full set of XGBoost hyperparameters used in the hybrid model, including tree configuration, sampling ratios, and regularization terms. These settings are summarized in Table 7.

Table 7. Detailed XGBoost hyperparameters used in the hybrid BiLSTM–XGBoost model.

Hyperparameter	Value
n_estimators	100
max_depth	6
learning_rate (eta)	0.3
subsample	1.0
colsample_bytree	1.0
lambda (L2 regularization)	1.0
alpha (L1 regularization)	0.0
objective	multi:softprob
eval_metric	mlogloss

The full hardware and software specifications of the system used for training are provided in Table 8.

Table 8. Laptop environment specifications.

Component	Specification
Operating System	Windows 11
CPU	AMD Ryzen 7
RAM	16 GB
GPU	None (CPU-only training)
Python Version	3.11.8
TensorFlow	2.18.0 (Keras backend)
PyTorch	2.7.0+cpu
XGBoost	2.1.1 (scikit-learn compatible)

5.3. Benchmarking Results

All CICIoV2024 results in this section refer to a six-class classifier (BENIGN, DoS, Spoofing_GAS, Spoofing_RPM, Spoofing_SPEED, Spoofing_STEERING_WHEEL), not a binary benign/malicious model. We evaluate each model on CICIoV2024 using accuracy, precision, recall, F1-score, training time, and prediction latency to balance detection quality and computational efficiency.

Table 9 reports classification (accuracy, precision, recall, and F1) and execution metrics (training, prediction, and total time) on CICIoV2024, providing a comparative view of detection performance and efficiency for real-time IoV deployment.

Table 9. Benchmarking results on the CICIoV2024 dataset.

Model	Accuracy	Precision	Recall	F1-Score	Training Time (s)	Prediction Time (s)	Total Time (s)
CNN (Standalone)	0.9964	0.9970	0.9964	0.9965	75.35	1.23	76.58
LSTM (Standalone)	0.9964	0.9970	0.9964	0.9965	334.79	2.73	337.52
BiLSTM (Standalone)	0.9964	0.9970	0.9964	0.9965	150.48	3.36	153.84
GRU (Standalone)	0.9964	0.9970	0.9964	0.9965	307.28	2.63	309.91
FastKAN (Flat Window)	0.9964	0.9970	0.9964	0.9965	174.96	1.71	176.67
XGBoost (Flat Window)	0.9963	0.9967	0.9963	0.9962	27.42	0.14	27.56
CNN-XGBoost (Hybrid)	0.9964	0.9967	0.9964	0.9962	20.44	0.09	20.53
LSTM-XGBoost (Hybrid)	0.9963	0.9967	0.9963	0.9962	22.21	0.09	22.30
BiLSTM-XGBoost (Hybrid)	0.9963	0.9967	0.9963	0.9962	21.96	0.09	22.05
GRU-XGBoost (Hybrid)	0.9963	0.9967	0.9963	0.9962	22.64	0.10	22.74
FastKAN-XGBoost (Hybrid)	0.9964	0.9967	0.9964	0.9962	37.19	0.20	37.38

For standalone DL and XGBoost models, precision/recall/F1 are macro-averaged (equal weight per class). For hybrid DL-XGBoost models, they are weighted-averaged (reflecting class priors in the test set). This provides both per-class fairness and deployment realism.

All models achieve high detection performance (accuracy, precision, and recall/F1 ≥ 98.5%). Recurrent DL models (LSTM, BiLSTM, and GRU) deliver competitive accuracy but incur substantially higher training and inference times, which can constrain real-time use. Hybrid models (CNN-XGBoost, LSTM-XGBoost, and BiLSTM-XGBoost) maintain equivalent detection while reducing prediction latency to ∼0.1–0.2 s. Patterns are consistent, indicating that hybrids offer superior time-sensitive performance. Among hybrids, BiLSTM-XGBoost achieves a favorable accuracy-latency trade-off by combining temporal sequence modeling with fast tree-ensemble inference.

As summarized in Table 9, model complexity strongly influences prediction speed. While CNN, LSTM, BiLSTM, and GRU reach high detection scores, they incur higher inference latency than the hybrid variants. In contrast, CNN-XGBoost and BiLSTM-XGBoost preserve accuracy while maintaining sub-second inference consistent with real-time IoV operation. Among them, BiLSTM-XGBoost offers the best balance between temporal representation and low-latency decision-making on CICIoV2024.

Although several temporal–tree hybrids achieved nearly identical offline accuracy on CICIoV2024, the BiLSTM-XGBoost model exhibited the most stable spoofing subclass discrimination with lower variance across folds. The bidirectional encoder generated more consistent latent patterns for RPM-SPEED and GAS-STEERING_WHEEL transitions, while XGBoost provided predictable low-latency inference within the CPU-based Kafka–Spark RSU environment. For these operational reasons, BiLSTM-XGBoost was selected for real-time deployment.

We further assess class-wise behavior using confusion matrices for hybrid approaches on CICIoV2024 (Figure 7), the primary benchmark guiding our real-time design. We focus on hybrids—rather than standalone DL—because they achieve comparable accuracy with much lower latency (Figure 7).

Figure 7. Confusion matrices for hybrid models on CICIoV2024.

The matrices highlight per-class performance for BENIGN, DoS, and spoofing subclasses.

As shown in Figure 7, all hybrids demonstrate strong classification, with near-perfect detection for BENIGN, DoS, GAS, and SPEED. BiLSTM-XGBoost and CNN-XGBoost yield indistinguishable results, with minor confusions primarily between RPM and SPEED spoofing, suggesting substantial feature overlap for these subclasses. LSTM-XGBoost, GRU-XGBoost, and FastKAN-XGBoost also maintain high accuracy, with slightly higher misclassification concentrated among spoofing categories. Overall misclassified counts remain low, confirming robust discriminative capacity across hybrid variants.

These multi-class predictions are directly actionable at run time: detected DoS triggers BLOCK, whereas spoofing subclasses trigger QUARANTINE, ensuring the IDS output drives immediate response. This aligns with the security-engineering view that IDS outputs must support operator action rather than only prediction [47].

6. Real-Time Hierarchical IDPS Implementation

6.1. TinyML Model Deployment

To enable efficient real-time detection on resource-constrained devices, we employ knowledge distillation. A compact TinyML CNN student (student) is trained to replicate the predictive behavior of a larger BiLSTM-XGBoost hybrid (teacher). As shown in Figure 8, the teacher is first trained on windowed CAN bus data, and its output probabilities (soft targets) guide the student CNN. Through this process, the student not only learns to match the teacher’s final predictions but also to approximate the teacher’s confidence in each class, such as predicting 90% traffic overload, 6% spoofing, and 4% benign for a given input. This enables the student to capture subtle inter-class relationships and achieve high accuracy while maintaining low latency and minimal resource use, making it suitable for on-device deployment within the hierarchical IDPS.

Figure 8. Knowledge distillation process.

The TinyML CNN student model is implemented as a compact 1D convolutional neural network operating directly on the $10\times 8$ CAN windows used throughout this work. The network comprises two convolutional blocks: the first Conv1D layer applies 32 filters with kernel size 3 and stride 1, followed by a MaxPooling1D layer with pool size 2 to reduce the temporal resolution. The second Conv1D layer applies 64 filters with kernel size 3 (stride 1 by default). Both convolutional layers use ReLU activation. A GlobalAveragePooling1D layer aggregates the resulting temporal feature maps into a 64-dimensional embedding, which is then fed to a final dense layer with 6 units and softmax activation that outputs the per-class probabilities for {BENIGN, DoS, GAS, RPM, SPEED, STEERING_WHEEL}. In total, the TinyML CNN student contains 7398 trainable parameters and occupies approximately $119\mathrm{kB}$ in FP32 format, well within the memory budgets of contemporary automotive microcontrollers. Post-training INT8 quantized variants (dynamic and full-integer) further reduce this footprint with negligible loss in accuracy.

• Distillation objective:

Let $z^{\left(t\right)}$ and $z^{\left(s\right)}$ be teacher and student logits, and let $\sigma$ denote the softmax. With temperature $T>1$, define softened distributions $q_t=\sigma (z^{\left(t\right)}/T)$ and $q_s=\sigma (z^{\left(s\right)}/T)$.

$${\mathcal{L}}_{\mathrm{distill}}=T^2\mathrm{KL}\left(q_t\parallel q_s\right)=T^2\sum _{c\in \mathcal{Y}}{q}_t\left(c\right)log\left({\displaystyle \frac{q_t\left(c\right)}{q_s\left(c\right)}}\right).$$

(8)

where

• $z^{\left(t\right)},z^{\left(s\right)}$: teacher and student logits (pre-softmax scores).
• $\sigma$: softmax function mapping logits to probabilities.
• T: temperature parameter ($T>1$ produces a softer distribution).
• $q_t,q_s$: softened teacher and student class-probability distributions.
• $\mathrm{KL}(·\parallel ·)$: Kullback–Leibler divergence.

This transfers calibrated inter-class relations from the RSU teacher (BiLSTM-XGBoost) to the TinyML CNN student. We use pure KD (no hard-label term); teacher probabilities are calibrated (Platt scaling). After training, we apply post-training INT8 quantization for OBU constraints.

In this work, we use a pure knowledge-distillation objective: the on-vehicle TinyML CNN student is optimized solely using the temperature-scaled Kullback–Leibler (KL) divergence between teacher and student outputs, without adding any cross-entropy term on hard labels. This ensures that the student learns only from the teacher’s calibrated soft targets. During distillation, the TinyML CNN student receives exclusively the BiLSTM-XGBoost teacher’s final output logits and their corresponding temperature-scaled probabilities; no intermediate features, hidden-layer embeddings, or internal representations are distilled. The student is therefore trained entirely on these soft labels rather than on the original ground-truth labels, ensuring that it inherits the teacher’s calibrated decision boundaries rather than functioning as an independently trained small CNN.

For completeness, we compare the distilled TinyML CNN to an identical non-distilled small CNN trained only on hard labels. The non-distilled CNN achieves similar accuracy (0.9964) but does not inherit the calibrated decision boundaries of the BiLSTM-XGBoost teacher and exhibits higher output variance across spoofing subclasses. In contrast, the distilled student matches the teacher’s soft targets, maintains the same accuracy, and achieves a compact 119 kB footprint, confirming the benefit of distillation over an independently trained small CNN.

Distilling from the BiLSTM-XGBoost teacher ensures the TinyML CNN student reproduces the teacher’s calibrated probability distributions and decision boundaries, yielding tier-aligned confidence scores for escalation while meeting strict memory and latency budgets.

In addition to knowledge distillation, we further optimize the TinyML CNN student through model quantization to ensure deployment feasibility on microcontroller-based platforms such as Arduino. Specifically, we apply both static quantization with max calibration and dynamic quantization. Static quantization precomputes scaling factors for weights and activations using a representative dataset, yielding predictable inference behavior, while dynamic quantization quantizes weights ahead of time and computes activation scales on the fly. These techniques significantly reduce the model’s memory footprint with negligible loss in detection accuracy.

The TinyML CNN student is deployed as a static inference-only model within the OBU; its predictions are subsequently revalidated at the RSU tier. Explicit OTA security mechanisms and adversarial defenses are not included in this version and are planned as future enhancements.

6.2. Real-Time Pipeline: Kafka–Spark for Streaming and Classification

Deploying the hierarchical IDPS in real-time requires a robust, scalable pipeline capable of ingesting, processing, and classifying vehicular network traffic at low latency. We adopt a distributed streaming architecture built upon Apache Kafka and Apache Spark, two well-established platforms for real-time big-data analytics in industrial and SC settings [48]. This choice aligns with recent distributed, microservice-based deployments and big-data benchmarking results [49,50], while practical guidance on stream cleaning and windowing further supports the configuration [51].

6.2.1. Kafka-Based Data Ingestion

The first step is the generation and streaming of realistic vehicular CAN bus data using a Kafka-based traffic producer implemented in Python. The producer emulates diverse traffic conditions and attack scenarios typical of SC-IoV environments.

System overview:

• Raw data loading: The producer reads raw CAN bus messages from CSV files in CICIoV2024. These files are not pre-windowed; the script applies normalization and generates 10-message sliding windows on the fly to reflect temporal structure in real CAN traffic.
• Data preprocessing: All numerical CAN features (DATA_0 to DATA_7) are normalized using a standard scaler. Consecutive frames are grouped into windows of 10 messages to capture short-term temporal patterns relevant for detection.
• Configurable modes: The producer supports three simulation modes: Baseline, Stealth, and Coordinated (see Table 4). Each mode specifies (i) the benign/attack mix, (ii) the message rate (messages/s), and (iii) the frequency and intensity of attacks. Operators can switch between normal, stealthy, and high-stress conditions via a single command-line argument.
• Windowed message streaming: Each Kafka message contains a sliding window of 10 normalized CAN frames, a majority-vote label, and a timestamp, mimicking real-time vehicular data streaming in a smart city.
• Flexible parameters: Command-line arguments control the traffic mode, message rate, random seed, and related options, yielding a reproducible setup adaptable to new datasets or network conditions.

Figure 9 illustrates the overall flow.

Figure 9. Overview of the Kafka-based CAN bus traffic producer.

Key points:

• The producer ingests raw source data, performs preprocessing and windowing internally, and streams directly to Kafka for real-time analysis.
• Operational modes (Baseline, Stealth, and Coordinated) modulate traffic realism and attack diversity.
• The setup enables rigorous stress-testing of the downstream Spark-based classifier under reproducible, realistic SC traffic conditions.

Implementation details: Table 10 summarizes key parameters and defaults for the Kafka traffic producer.

Table 10. Kafka traffic producer configuration parameters (per simulation mode).

Parameter	Description	Default/Example Value
–mode	Simulation mode that controls traffic composition, rate, and attack behavior.	baseline, stealth, coordinated
–rate	Message rate (messages/s), adjusted dynamically in coordinated mode.	10 (baseline), 5 (stealth), 15–25 bursts (coordinated)
–seed	Random seed to ensure reproducibility of message sequence.	42
–benign_ratio	Proportion of benign traffic (overridable).	0.75 (baseline), 0.90 (stealth), 0.35 (coordinated)
TOPIC	Kafka topic used to publish messages.	can_traffic
BOOTSTRAP_SERVER	Kafka broker address for publishing CAN messages.	localhost:9092
Window Size	Number of CAN frames per sliding window.	10
Dataset Files	Raw input CSVs used for each class (benign + 5 attacks).	decimal_benign.csv, decimal_traffic overload.csv, etc.
Feature Columns	Normalized sensor fields in each message.	DATA_0 to DATA_7
Output Fields	JSON fields sent to Kafka.	timestamp, window, label, mode

After CAN bus windows are streamed to Kafka, Spark Streaming performs real-time processing and classification, enabling low-latency detection at the RSU tier.

6.2.2. Spark-Based Real-Time Classification

After data is streamed to Kafka, the next stage performs high-speed classification and prevention via a Spark Streaming consumer tightly integrated with the BiLSTM-XGBoost hybrid.

Architecture and workflow.

• Real-time ingestion. Spark reads each windowed CAN bus message from the Kafka topic, preserving order and structure.
• Feature extraction. Each window is passed through a pre-trained BiLSTM to obtain temporal features that help distinguish benign from malicious behavior.
• Attack prediction. XGBoost classifies the extracted features as benign, traffic overload, or a spoofing subclass, combining sequence modeling with fast, accurate inference.
• Prevention actions. Based on the prediction:

  –

  If a traffic overload attack is detected, the source is blocked.

  –

  For spoofing attacks, the responsible device is quarantined and an alert is raised.

  –

  Low-confidence events are flagged for review.

• Drift detection. ADWIN monitors for concept drift; detected shifts are logged to support subsequent model updates.
• Monitoring and logging. For every processed batch, the system records accuracy, latency, CPU/RAM usage, and attack counts; these are persisted and visualized on the dashboard.

The overall workflow is illustrated in Figure 10.

Figure 10. Pipeline of the Spark-based real-time classification and prevention module.

• Latency decomposition and throughput stability: For each processed window, e2e latency decomposes as

$$T_{\mathrm{e}2\mathrm{e}}=T_{\mathrm{ingest}}+T_{\mathrm{feat}}+T_{\mathrm{clf}}+T_{\mathrm{action}}+{\epsilon }_{\mathrm{sync}}.$$

(9)

We model the residual jitter as zero-mean with finite variance: $\mathbb{E}\left[{\epsilon }_{\mathrm{sync}}\right]=0$ and $\mathrm{Var}\left({\epsilon }_{\mathrm{sync}}\right)={\sigma }_{\mathrm{sync}}^2$. Measured latency: From streaming logs, we obtain ${\overline{T}}_{\mathrm{e}2\mathrm{e}}$ = 148.67 ms with residual jitter ${\widehat{\sigma }}_{\mathrm{sync}}$ = 0.12 ms, yielding an ≈95% prediction band of 148.67 ± 0.24 ms. Component means: ${\overline{T}}_{\mathrm{ingest}}$ = 9.77 ms, ${\overline{T}}_{\mathrm{feat}}$ = 134.61 ms, ${\overline{T}}_{\mathrm{clf}}$ = 1.80 ms, ${\overline{T}}_{\mathrm{action}}$ = 0.00 ms.

The reported e2e latency of 148.67 ms was obtained in a local RSU-simulated setup running Kafka and Spark Streaming on a single laptop. In this environment, the dominant factor is the Kafka–Spark micro-batch processing overhead, while the XGBoost inference step contributes only ≈10–13 ms. Because no physical RSU–vehicle wireless link was deployed, network delay is negligible. Under higher traffic density, latency growth is mainly driven by Spark scheduling and queue buildup, whereas inference time remains effectively constant.

Here, $T_{\mathrm{feat}}$ (BiLSTM encoding) and $T_{\mathrm{clf}}$ (XGBoost inference) are measured per micro-batch. Let ${\mu }_{\mathrm{in}}$ be the input rate and ${\mu }_{\mathrm{proc}}$ the sustained processing rate. Stability requires ${\mu }_{\mathrm{in}}<{\mu }_{\mathrm{proc}}$; otherwise the Kafka lag $L_t$ grows approximately as

$$\Delta L_t\approx max\{{\mu }_{\mathrm{in}}-{\mu }_{\mathrm{proc}},0\}\Delta t,$$

(10)

where

• $T_{\mathrm{e}2\mathrm{e}}$: e2e latency per window (ingest → features → classify → action), measured in ms.
• $T_{\mathrm{ingest}}$: ingestion/serialization + Kafka → Spark handoff time (ms).
• $T_{\mathrm{feat}}$: BiLSTM feature-extraction time (per micro-batch, ms).
• $T_{\mathrm{clf}}$: XGBoost inference time (per micro-batch, ms).
• $T_{\mathrm{action}}$: prevention/alert action time (logging, block/quarantine trigger; ms).
• ${\epsilon }_{\mathrm{sync}}$: stochastic synchronization/jitter term (e.g., brief Kafka lag, scheduler jitter); ${\sigma }_{\mathrm{sync}}$ is its standard deviation.
• ${\mu }_{\mathrm{in}}$: input arrival rate (windows/s).
• ${\mu }_{\mathrm{proc}}$: sustained processing rate (windows/s).
• $L_t$: Kafka consumer lag at time t (messages/windows behind the head).
• $\Delta L_t$: change in lag over interval $\Delta t$ (s); positive when backlog accumulates.

Equation (10) is surfaced on the dashboard’s Kafka-lag (Section 7).

Key benefits: The Spark-based stage analyzes each message upon arrival, enabling millisecond-scale reaction times. The combination of deep feature extraction and fast classification provides both high accuracy and the speed needed for deployment at SC intersections, while built-in prevention and monitoring maintain operator situational awareness under evolving threats.

Table 11 outlines the real-time Spark consumer configuration, including models, prevention logic, logging outputs, and resource monitoring.

Table 11. Spark-based real-time consumer configuration parameters.

Component	Description	Value/Setting
Kafka Topic	Kafka topic subscribed for CAN bus message ingestion.	can_traffic
Consumer Group ID	Kafka consumer group used for offset tracking.	test-consumer-group
Batch Interval	Time between batch processing cycles.	5 s
Window Shape	Shape of each incoming CAN message window.	(10, 8)
Feature Extractor	Deep temporal feature extractor model.	BiLSTM (last dense layer output)
Classifier	Final prediction model used on extracted features.	XGBoost
Uncertainty Threshold	Confidence threshold to flag low-confidence predictions.	0.6
Drift Detection	Online concept drift detection mechanism.	ADWIN (river)
Prevention Actions	Policy for detected threats.	BLOCK for traffic overload, QUARANTINE for spoofing
Resource Monitoring	System metrics captured per batch.	CPU %, RAM %, latency
Per-Batch Metrics	Evaluation metrics calculated each batch.	Accuracy, precision, recall, F1-score
Prediction Logs	File storing all classification and action results.	predictions_log.csv
Action Logs	File logging non-benign actions taken.	actions_log.csv
Drift Logs	File tracking detected drift events.	drift_log.csv
Batch Logs	File summarizing each batch’s statistics.	batch_log.csv
Error Logs	File recording shape mismatches or processing failures.	error_log.csv

Table 11. Spark-based real-time consumer configuration parameters.

Component	Description	Value/Setting
Kafka Topic	Kafka topic subscribed for CAN bus message ingestion.	can_traffic
Consumer Group ID	Kafka consumer group used for offset tracking.	test-consumer-group
Batch Interval	Time between batch processing cycles.	5 s
Window Shape	Shape of each incoming CAN message window.	(10, 8)
Feature Extractor	Deep temporal feature extractor model.	BiLSTM (last dense layer output)
Classifier	Final prediction model used on extracted features.	XGBoost
Uncertainty Threshold	Confidence threshold to flag low-confidence predictions.	0.6
Drift Detection	Online concept drift detection mechanism.	ADWIN (river)
Prevention Actions	Policy for detected threats.	BLOCK for traffic overload, QUARANTINE for spoofing
Resource Monitoring	System metrics captured per batch.	CPU %, RAM %, latency
Per-Batch Metrics	Evaluation metrics calculated each batch.	Accuracy, precision, recall, F1-score
Prediction Logs	File storing all classification and action results.	predictions_log.csv
Action Logs	File logging non-benign actions taken.	actions_log.csv
Drift Logs	File tracking detected drift events.	drift_log.csv
Batch Logs	File summarizing each batch’s statistics.	batch_log.csv
Error Logs	File recording shape mismatches or processing failures.	error_log.csv

6.2.3. Dashboard Visualization and Real-Time Monitoring

The final component is a web-based dashboard that assists operators in monitoring detection results, analyzing trends, and reacting to threats in real time. Built with the Dash framework, it integrates logs generated by the Spark consumer and the Kafka lag tracker.

Key features:

• Time and mode filtering. Filter views by preset time windows (e.g., last 5 min, last hour) and by simulation mode (Baseline, Stealth, and Coordinated).
• Kafka lag monitoring. A gauge and time series display lag deltas for early detection of bottlenecks or consumer delay.
• Evaluation metrics. Bar charts present per-batch and latest accuracy, precision, recall, and F1-score over time.
• Traffic analysis. Class distribution, benign vs. attack ratios, and attack-frequency timelines provide insight into threat intensity.
• Prevention effectiveness. Donut and heatmap charts summarize BLOCK/QUARANTINE actions and their association with attack types.
• System health. CPU and RAM usage, prediction latency, and throughput are tracked per batch.
• Drift and uncertainty. Concept drift (ADWIN) and low-confidence rates are visualized over time.
• Error and alert management. Error trends and toast-style alerts notify operators of attack surges, drift, or Kafka lag anomalies.
• Prediction logs. A live table lists recent classifications (timestamp, mode, predicted label, confidence, action, and latency).

Figure 11 illustrates a representative snapshot during baseline mode: over 4200 messages were processed with near-perfect accuracy across most batches; Kafka lag remained low and stable; and batch latency stayed below 155 ms. Real-time visualizations (class-wise predictions, attack proportions, and actions taken) provide interpretable insight into live behavior. During baseline testing, no error events or concept drift were triggered, yielding empty charts for those components and indicating stable operation under normal traffic.

Figure 11. Real-time IDPS dashboard visualization during baseline simulation mode.

Implementation overview. The dashboard consumes locally stored logs from the Spark consumer and Kafka lag tracker:

• predictions_log.csv—model predictions and confidence scores.
• actions_log.csv—records of blocks and quarantines.
• drift_log.csv—concept drift detections.
• batch_log.csv—per-batch metrics (accuracy, CPU, latency, and throughput).
• kafka_lag_delta_history.csv—Kafka lag over time.
• error_log.csv—system errors or mismatches.

Table 12 summarizes the key parameters and input files used by the dashboard, including update intervals, log sources, filtering options, and alert conditions.

Table 12. Dashboard configuration parameters.

Parameter	Description	Example/Default Value
Update Interval	Dashboard refresh rate for all figures and metrics.	5 s
Prediction Log File	Timestamped predictions from Spark consumer.	predictions_log.csv
Batch Metrics File	Per-batch metrics (accuracy, latency, throughput, resources).	batch_log.csv
Kafka Lag Source	Lag delta values for gauge and timeline charts.	kafka_lag_delta_history.csv
Actions Log File	Log of blocking/quarantine actions taken by the IDPS.	actions_log.csv
Drift Log File	Records of detected concept drifts (ADWIN).	drift_log.csv
System Resource Metrics	CPU and RAM usage collected per batch.	Extracted from batch_log.csv
Alert Thresholds	Pop-up alerts for ≥10 attacks in 30 s, concept drift, or high Kafka lag.	Hardcoded in script
Time Filter Options	Time window selection for plots.	Last 5 min, 1 h, 24 h, all
Mode Filter	Filter by simulation mode.	baseline, stealth, coordinated

The dashboard refreshes every 5 s, providing continuous status visibility and early indication of anomalies.

Table 13 presents the main tools, libraries, and system specifications used to implement the real-time hierarchical IDPS, covering traffic simulation, stream processing, model inference, drift detection, and dashboard visualization.

Table 13. Tools, libraries, and frameworks used in the proposed IDPS.

Component	Purpose	Tool/Library (Version)
Traffic Simulation	Generate real-time CAN traffic from CICIoV2024 using realistic attack modes	Python 3.11.8, KafkaProducer (kafka-python 2.0.2)
Streaming Platform	Ingest and distribute real-time data to consumers	Apache Kafka 3.4.1
Stream Processing	real-time classification, prevention, logging, and drift detection	Apache Spark 3.4.4, pyspark 3.4.4, psutil 6.1.1
Deep Feature Extraction	Extract temporal features from CAN traffic windows	TensorFlow 2.18.0, Keras 3.8.0 (BiLSTM model)
Classification Model	High-speed traffic type classification	XGBoost 2.1.1, scikit-learn 1.6.1
Concept Drift Detection	Detect changes in traffic behavior	river 0.22.0 (ADWIN)
Dashboard Backend	Visualization of real-time system metrics and outputs	Dash 3.0.4, Flask 3.0.3, Plotly 6.1.1
Kafka Lag Monitoring	Monitor consumer delay for stress detection	kafka-python 2.0.2, psutil 6.1.1
Data Handling	Logging and manipulation of metrics and predictions	Pandas 2.2.3, NumPy 2.0.2

7. Results and Discussion

7.1. TinyML-Based On-Vehicle Detection

We evaluate efficiency on CICIoV2024 by comparing the baseline BiLSTM-XGBoost hybrid, its distilled CNN, and static/dynamic quantized variants.

As shown in Table 14, knowledge distillation and post-training quantization preserve detection performance relative to the baseline hybrid (metric variations within ∼0.01%). Size reductions are substantial: distillation alone lowers the footprint by ≈75%, while INT8 quantization yields up to ≈96.9% reduction. Latency (sampled over 1000 instances) also improves: the distilled model reduces inference time by ≈70% and dynamic quantization by ≈30% versus the baseline, whereas static quantization primarily benefits size. A statically quantized model was successfully deployed on an Arduino Nano 33 BLE, confirming feasibility for ultra-low-power devices. These results indicate that lightweight, on-vehicle IDS models can be realized without sacrificing classification quality for real-time IoV use.

Table 14. Performance comparison of BiLSTM-XGBoost variants with distillation and quantization on the CICIoV2024 dataset.

Model	Accuracy	Precision	Recall	F1-Score	Size (KB)	Latency (ms)
BiLSTM-XGBoost	0.9963	0.9967	0.9963	0.9962	495.54	0.027
Distilled CNN	0.9964	0.9967	0.9964	0.9962	121.89	0.008
Static quantization (FP32 → INT8)	0.9964	0.9967	0.9964	0.9962	15.23	0.033
Dynamic quantization (FP32 → INT8 dyn.)	0.9964	0.9967	0.9964	0.9962	16.38	0.019

Why a distilled CNN rather than a plain CNN: Training a plain CNN on hard labels risks overconfident or inconsistent predictions in borderline cases. Distillation transfers the teacher’s calibrated class probabilities and inter-class structure, aligning vehicle-tier decisions with RSU-tier inference and preserving accuracy while achieving the footprint/latency needed for on-device, real-time decision-making.

Comparable real-time IDS algorithm evaluations in big-data settings further motivate lightweight designs [52].

7.2. Hybrid BiLSTM-XGBoost RSU-Level Analysis

We assess the RSU-tier hybrid across three operational modes: Baseline, Stealth, and Coordinated.

For consistency, we refer to the “DoS” label in legacy plots as “traffic overload” in the text; both denote the same load-anomaly class.

Per-batch evaluation metrics: Figure 12, Figure 13 and Figure 14 report per-batch precision, recall, F1-score, and accuracy.

Figure 12. Per-batch real-time evaluation metrics—Baseline mode.

Figure 13. Per-batch real-time evaluation metrics—Stealth mode.

Figure 14. Per-batch real-time evaluation metrics—Coordinated mode.

• Baseline: In Figure 12, the hybrid sustains 100% precision, recall, F1-score, and accuracy across evaluated batches, indicating reliable behavior under nominal traffic with minor variation.
• Stealth: In Figure 13, where rare intrusions are embedded in predominantly benign flows, short-lived dips appear as follows: precision, recall, and F1 briefly drop to about 60% while accuracy remains above 93%, followed by a rapid rebound—indicating resilience to transient anomalies.
• Coordinated: Under simultaneous traffic overload bursts and spoofing (Figure 14), metrics generally remain between 95% and 100%, evidencing stable handling of high-throughput, multi-vector intrusions.
• Overall, the hybrid exhibits stable real-time detection across modes—robust to noise, adaptive to stealth, and resilient under coordinated attacks—supporting RSU deployment in SC-IoV infrastructures.

Attack/benign traffic over time:

Figure 15 illustrates the time-series distribution of benign and spoofed messages under each mode.

Figure 15. Class-wise distribution of messages over time under Baseline, Stealth, and Coordinated operational modes. (a) Baseline mode; (b) Stealth mode; (c) Coordinated mode.

In the Baseline mode (Figure 15a), benign traffic dominates with only intermittent, low-frequency attacks. In the Stealth mode (Figure 15b), attacks are extremely sparse and camouflaged within benign flows. In the Coordinated mode (Figure 15c), synchronized bursts generate sharp rises across multiple attack classes. These temporal patterns validate the simulation design and confirm that the system differentiates normal, stealthy, and large-scale coordinated intrusions in real-time streams.

Latency analysis: we measure e2e batch prediction latency (Kafka ingestion → Spark micro-batch → inference → sink) across modes (Figure 16, Figure 17 and Figure 18).

Figure 16. Batch prediction latency—Baseline mode.

Figure 17. Batch prediction latency—Stealth mode.

Figure 18. Batch prediction latency—Coordinated mode.

Across all modes, batch latency remains low and stable. In Baseline and Stealth settings, latency averages ≈140–150 ms, indicating steady throughput under nominal and sparsely adversarial traffic. Under Coordinated conditions, we observe a brief spike to ≈415 ms at burst onset, after which latency returns to ≈148 ms. Overall, the pipeline sustains sub-second responsiveness even during heavy, multi-vector attacks, consistent with Spark Streaming’s adaptive micro-batching.

Uncertainty rate: Figure 19 shows the batch-wise uncertainty rate.

Figure 19. Batch-wise uncertainty rate (%). (a) Uncertainty rate in Baseline mode; (b) Uncertainty rate in Coordinated mode; (c) Uncertainty rate in Stealth mode.

In Baseline (Figure 19a), uncertainty typically stays below 5% with brief spikes to 7–8%. In Stealth (Figure 19b), spikes are more frequent and can exceed 12%, reflecting the difficulty of rare, camouflaged events. In Coordinated (Figure 19c), uncertainty is lowest (about 2–4.5%), consistent with repetitive, high-volume attacks. Overall, the hybrid remains confident under both nominal and intense traffic while being sensitive to low-rate stealth.

Kafka lag analysis: Figure 20 illustrates Kafka lag across modes.

Figure 20. Kafka lag per batch during real-time streaming. (a) Baseline mode; (b) Stealth mode; (c) Coordinated mode.

In Baseline (Figure 20a), lag is consistently low (16–18 messages/batch). In Stealth (Figure 20b), it increases slightly (22–29), reflecting a lower input rate and longer batch wait. In Coordinated (Figure 20c), lag fluctuates between 70–91 due to intense concurrency; the system remains stable nonetheless.

System resource usage: We monitor CPU and RAM to assess efficiency and scalability (Figure 21).

Figure 21. System resource usage (CPU and RAM) over time. (a) Baseline mode; (b) Coordinated mode; (c) Stealth mode.

• Baseline. CPU fluctuates between 50% and 60%; RAM is ∼55%.
• Coordinated. CPU rises and stabilizes at 68–70%; RAM stays below 50%, indicating a CPU-bound regime under heavy traffic.
• Stealth. CPU is lower and more variable (35–50%); RAM fluctuates mildly, consistent with sporadic loads.

Overall, RAM remains stable while CPU scales with traffic intensity, confirming capacity to handle attack bursts without resource exhaustion.

Summary of key findings: Table 15 consolidates performance across modes (accuracy, latency, uncertainty, and Kafka lag), underscoring robustness and real-time responsiveness at the RSU tier.

Table 15. Summary of real-time evaluation across operational modes.

Mode	Avg. Accuracy (%)	Avg. Latency (ms)	Avg. Uncertainty (%)	Kafka Lag Behavior
Baseline	100	146.3–151.7	∼3.0	Stable
Stealth	∼95	141.2–149.7	Peaks > 12	Low
Coordinated	95–98	142.3–144.5	∼2–4.5	Spiky

7.3. Comparative Analysis with Recent IoV IDS Frameworks

To situate H-RT-IDPS within recent work, Table 16 compares it to state-of-the-art real-time or vehicular IDS frameworks (2024–2025), considering detection accuracy, F1-score, and runtime when available.

Table 16. Performance comparison with recent IoV and CAN intrusion detection frameworks.

Ref.	Year	Dataset	Model Type	Acc. (%)	F1	Runtime/Environment
[20]	2025	CAN	CNN-MLP (self-supervised)	–	0.9954	8940 FPS (GPU), 2692 FPS (CPU)
[24]	2024	CICIDS, Car-Hacking	Dynamic Ensemble Forest	99.2/98.0	0.969	–
[29]	2024	CAV experimental	SVM + TinyML (dual layer)	97.2	–	Lightweight edge (TinyML)
[19]	2024	CAN bus	RF/XGB/DT ensemble + streaming	98.5	–	Real-time Spark Streaming
[26]	2025	Car-Hacking, OTIDS	Deep learning hybrid	≈100	0.99	–
Our work	2025	CICIoV2024	BiLSTM-XGBoost hybrid + TinyML	99.9	99.9	153 ms (latency)

H-RT-IDPS matches or exceeds recent IoV IDS performance while maintaining real-time operation. Unlike single-tier or offline approaches, it provides a hierarchical architecture: TinyML CNN students on vehicles for on-device filtering, a BiLSTM-XGBoost hybrid at RSUs for temporal learning and classification, and a Kafka–Spark pipeline for centralized prevention, drift monitoring, and dashboard visualization. This cross-tier fusion and closed-loop prevention enable adaptive, city-wide situational awareness within a unified SC-IoV ecosystem. Complementary perspectives from mobile-agent itinerary planning for information fusion highlight design patterns that can translate to RSU-cloud coordination in IoV [53].

Following Sommer & Paxson [42], we note that non-ML counters suffice for overt floods, but our hybrid excels at low-rate/pulsed variants. Future work will differentiate malicious DoS from benign overloads (e.g., device failures).

8. Limitations, Open Issues, and Future Directions

Despite achieving strong performance, the proposed H-RT-IDPS has several limitations. The current system is restricted to detecting traffic overload and spoofing (GAS, RPM, SPEED, and steering wheel), while other prevalent IoV threats such as RSU impersonation, MITM, Sybil, and adversarial attacks remain outside its present scope. This focus was intentional: traffic overload and spoofing directly compromise two fundamental IoV security pillars—availability and integrity/authenticity—and are well represented in the CICIoV2024 dataset, enabling rigorous benchmarking and real-time validation.

Another limitation concerns the dataset itself. The system relies on simulated CICIoV2024 traces, and its behavior in real vehicular environments with heterogeneous driving patterns, weather conditions, and hardware imperfections remains untested.

Furthermore, the results reported in this work reflect the CAN configuration present in CICIoV2024, which corresponds to a specific vehicle platform. Different vehicles may employ distinct CAN-ID layouts, payload encodings, and timing characteristics. Such variability can affect both the on-vehicle TinyML CNN student and the BiLSTM-XGBoost hybrid model if deployed without adaptation. In practice, generalizing the system to other vehicle models would require retraining or calibration using that vehicle’s CAN traces or applying domain-adaptation mechanisms. Exploring cross-vehicle generalization therefore represents an important direction for future research.

Scalability is another concern. All real-time experiments were conducted on a single-node Kafka–Spark setup with moderate hardware resources. While the pipeline demonstrated stable performance and low latency, city-scale deployments with dozens of RSUs and thousands of vehicles will require distributed scheduling, load balancing, and resilience mechanisms that were not evaluated in this study. Similarly, current prevention actions (blocking and quarantining) are deliberately simple and do not yet include coordinated multi-RSU decisions, adaptive firewalls, or long-term threat correlation.

Future extensions will expand the framework to cover additional IoV attack families, incorporate adaptive retraining at the edge, and adopt secure coordination mechanisms such as blockchain for tamper-resistant logging and inter-RSU trust.

Recent research on distributed IoV security has explored complementary directions such as federated learning (FL) and blockchain-based coordination. FL-based IDS frameworks for vehicular networks [54,55] enable collaborative model updates across vehicles and RSUs without sharing raw CAN traces, improving privacy but incurring additional communication overhead and lacking real-time guarantees.

Likewise, blockchain-enabled IoV security [56,57] provides tamper-resistant logging and trust management, yet these systems often omit high-frequency IDS inference or do not operate on constrained OBUs. In contrast, the proposed H-RT-IDPS focuses on hierarchical, real-time detection with TinyML CNN student and RSU-tier BiLSTM-XGBoost inference—rather than distributed training or immutable logging. FL and blockchain therefore represent complementary directions for future integration, such as federated model aggregation for adaptive retraining and blockchain-backed auditability of RSU decisions.

9. Conclusions

In this work, we developed an H-RT-IDPS, which targets the security needs of IoV systems operating in SC environments. The proposed system utilizes lightweight TinyML CNN students on vehicles for quick detection of obvious threats alongside a hybrid BiLSTM-XGBoost model running at RSUs to handle complex temporal attack patterns. The centralized Kafka–Spark Streaming pipeline enables city-wide monitoring and adaptive updates and real-time dashboard visualization to enhance operator situational awareness.

The system achieved high detection accuracy during offline evaluations and real-time experiments in baseline, stealth, and coordinated operational scenarios while maintaining low inference latency and robust system performance under high-load conditions. The system’s coverage extends to specific attack types, including traffic overload and spoofing attacks. The system faces ongoing practical deployment challenges because of its dependence on datasets and its resource constraints and basic prevention methods.

Future research directions should focus on expanding detection capabilities to more sophisticated threats while implementing federated and edge learning approaches to enhance scalability and privacy and also integrating advanced interpretability and blockchain-based secure logging mechanisms. The proposed IDPS framework needs to address its limitations while investigating outlined open issues to establish a robust, scalable, adaptive foundation for protecting emerging IoV-based SC deployments from evolving cybersecurity threats.