Cyberattack Resilience of Autonomous Vehicle Sensor Systems: Evaluating RGB vs. Dynamic Vision Sensors in CARLA

Abstract

Autonomous vehicles (AVs) rely on a heterogeneous sensor suite of RGB cameras, LiDAR, GPS/IMU, and emerging event-based dynamic vision sensors (DVS) to perceive and navigate complex environments. However, these sensors can be deceived by realistic cyberattacks, undermining safety. In this work, we systematically implement seven attack vectors in the CARLA simulator—salt and pepper noise, event flooding, depth map tampering, LiDAR phantom injection, GPS spoofing, denial of service, and steering bias control—and measure their impact on a state-of-the-art end-to-end driving agent. We then equip each sensor with tailored defenses (e.g., adaptive median filtering for RGB and spatial clustering for DVS) and integrate a unsupervised anomaly detector (EfficientAD from anomalib) trained exclusively on benign data. Our detector achieves clear separation between normal and attacked conditions (mean RGB anomaly scores of 0.00 vs. 0.38; DVS: 0.61 vs. 0.76), yielding over 95% detection accuracy with fewer than 5% false positives. Defense evaluations reveal that GPS spoofing is fully mitigated, whereas RGB- and depth-based attacks still induce 30–45% trajectory drift despite filtering. Notably, our research-focused evaluation of DVS sensors suggests potential intrinsic resilience advantages in high-dynamic-range scenarios, though their asynchronous output necessitates carefully tuned thresholds. These findings underscore the critical role of multi-modal anomaly detection and demonstrate that DVS sensors exhibit greater intrinsic resilience in high-dynamic-range scenarios, suggesting their potential to enhance AV cybersecurity when integrated with conventional sensors.

1. Introduction

Autonomous vehicles (AVs) have the potential to revolutionize transportation systems worldwide, promising to reduce traffic accidents by up to 90% and improve mobility for disabled individuals [1,2]. These vehicles rely on an array of sensors such as RGB cameras, LiDAR, radar, GPS, and the emerging event-based dynamic vision sensor (DVS) to perceive their surroundings and navigate autonomously [3,4,5,6]. However, the increasing complexity and connectivity of AV systems introduce significant cybersecurity risks that could be exploited by malicious actors [7,8,9,10,11]. The security of sensor systems is paramount, as they form the critical interface between the vehicle and its environment [12]. Recent research has demonstrated that adversaries can manipulate sensor inputs through various attack vectors, including adversarial machine learning [13,14], sensor spoofing [9,15], and denial-of-service attacks [8,16], posing serious safety threats to autonomous navigation systems.

Comprehensive security research has revealed multiple attack vectors targeting AV sensor systems [17,18]. Camera-based perception systems are vulnerable to adversarial examples that can cause misidentification of traffic signs or obstacles [13,19,20], with studies demonstrating that carefully crafted perturbations can achieve attack success rates exceeding 85% under real-world conditions [21]. LiDAR sensors face threats from spoofing attacks that inject false obstacles, leading to unnecessary emergency braking or collision avoidance maneuvers [9,22,23]. GPS systems are susceptible to spoofing attacks that can cause vehicles to deviate from their intended routes by manipulating position data [9,24,25]. Additionally, denial-of-service attacks targeting sensor communication channels can disrupt critical perception capabilities [8,26]. These vulnerabilities highlight the pressing need for comprehensive security analysis and robust defense strategies across all AV sensor modalities [8,9].

While traditional sensors such as RGB cameras and LiDAR have received significant attention regarding their security vulnerabilities [15,27], the integration of a dynamic vision sensor (DVS) into AVs introduces both new opportunities and challenges that remain largely unexplored from a cybersecurity perspective [28]. Unlike conventional frame-based cameras that capture images at fixed intervals, DVSs operate by asynchronously detecting pixel-level brightness changes with microsecond temporal resolution [29,30]. This event-driven approach provides significant advantages, including high dynamic range (> 120 dB), reduced motion blur, and lower data redundancy [31,32,33], making DVSs particularly suitable for high-speed driving scenarios and challenging weather conditions where traditional cameras fail [34,35]. However, their unique operating principles and asynchronous data streams raise critical security questions that have not been systematically addressed: How resilient are DVSs to adversarial attacks compared to traditional cameras? Can their event-based nature be exploited through novel attack vectors such as event flooding or temporal manipulation? Which defense mechanisms are most effective for securing event-based perception systems? Furthermore, how does the integration of a DVS alongside traditional sensors affect the overall security posture of autonomous vehicles?

This paper conducts a comprehensive analysis of cyberattacks targeting AV sensors, placing particular emphasis on evaluating the security implications of incorporating DVS technology into multi-modal sensor suites [36]. We systematically implement and evaluate seven distinct attack vectors: salt-and-pepper noise attacks on RGB cameras [37], event flooding attacks on DVSs, depth map tampering [38], LiDAR phantom object injection [22], GPS spoofing [24,39], denial-of-service attacks [26], and steering bias manipulation [11]. Our experimental methodology leverages the CARLA simulator (version 9.15) and the PCLA (Pretrained CARLA Leaderboard Agent) framework [40] to simulate and assess these attacks within photorealistic autonomous driving environments [41]. Building upon recent advances in DVS-based perception [42], we implement realistic attack parameters derived from existing literature and real-world attack scenarios [13,22]. The CARLA simulator provides validated sensor models and physics simulations that enable rigorous evaluation of attack impacts on autonomous driving performance [36].

Our study systematically examines autonomous agent behavior and sensor responses under various attack conditions, providing quantitative insights into the vulnerabilities and resilience characteristics of modern autonomous driving systems [43]. We implement tailored defense mechanisms for each sensor modality and develop a comprehensive anomaly detection system using the EfficientAD model from the anomalib library [44,45]. Our evaluation encompasses over 160 CARLA episodes across diverse driving scenarios, providing statistical analysis of attack impacts and defense effectiveness [8].

Important Note on DVS Integration: It is crucial to clarify that in our experimental setup, DVS cameras are used exclusively for research purposes and cybersecurity analysis. The DVS sensors are not integrated into the vehicle’s driving decision model, which relies primarily on RGB cameras and other traditional sensors. Our DVS security analysis provides insights into potential vulnerabilities and defense mechanisms for future integration scenarios, but it does not claim that DVS technology currently enhances the cybersecurity of the autonomous driving system under test.

• We provide a systematic analysis of sensor vulnerabilities in autonomous vehicles, with particular focus on the emerging DVS technology and its unique security characteristics in research contexts.
• We implement and evaluate realistic attack scenarios in the CARLA simulator with quantified parameters derived from existing literature, demonstrating measurable impacts on autonomous agent behavior and perception accuracy.
• We develop an effective anomaly detection system using the EfficientAD model from the anomalib library that achieves a > 95% detection accuracy with < 5% false positives for identifying sophisticated attacks on both RGB and DVS camera sensors.
• We provide quantitative analysis of defense mechanism effectiveness, revealing that while GPS spoofing defenses achieve near-perfect mitigation, RGB and depth sensor defenses show limited effectiveness with 30–45% trajectory drift remaining.
• We propose a comprehensive multi-layered security framework that combines sensor-specific defenses, anomaly detection, and graceful degradation strategies for resilient autonomous operation.

2. Related Work

The cybersecurity of autonomous vehicles (AVs) is a rapidly evolving field, with extensive research focused on identifying vulnerabilities and developing robust defense mechanisms. This section situates our work within the existing literature, focusing on three key areas: security for multi-sensor fusion systems, recent advances in anomaly detection [46], and the nascent field of dynamic vision sensor (DVS) security. By comparing our approach to recent studies, we highlight the novelty and specific contributions of our research.

Recent studies have increasingly focused on the vulnerabilities of multi-sensor fusion algorithms, which are critical for robust perception in AVs. For instance, recent work by Zhu et al. [47] investigates malicious attacks that specifically target the fusion mechanism, creating inconsistencies between LiDAR and camera data to deceive the perception system. Similarly, other research has explored how to generate “invisible” adversarial attacks that can simultaneously fool both camera and LiDAR systems [48]. While these studies provide crucial insights into the security of integrated systems, they primarily focus on attacking the central fusion logic. In contrast, our work provides a more granular analysis by evaluating the resilience of individual sensors before their data is fed into a fusion model. We systematically implement seven distinct attack vectors on a sensor-by-sensor basis, establishing a foundational understanding of how specific sensor modalities degrade. This approach is complementary and essential, as a resilient fusion system still depends on the integrity of its input data streams.

In parallel with security-focused research, recent work has explored the use of end-to-end reinforcement learning for developing AV control policies in simulation environments [49]. Sakhai and Wielgosz [50] demonstrated that escape maneuvers in urban driving scenarios can be learned using neural policy architectures trained entirely in CARLA. While their work focused on control under dynamic conditions, our study shifts attention to the sensor-level cybersecurity of perception inputs that feed into such learning-based agents.

The development of holistic security architectures, often leveraging artificial intelligence (AI) and blockchain, represents another significant research trend. Bendiab et al. [51] propose a comprehensive framework that uses AI for threat detection and blockchain for secure data exchange. Surveys on AI-based intrusion detection systems further highlight the popularity of machine learning for identifying anomalous vehicle behavior [52]. Our research builds upon this trend by implementing and evaluating a specific, state-of-the-art unsupervised anomaly detection model, EfficientAD, from the anomalib library [44]. A key differentiator of our study is that we train and test this model on both conventional RGB data and, uniquely, on DVS event streams. By demonstrating over 95% detection accuracy, we not only validate the model’s effectiveness but also provide a direct, quantitative comparison of its performance on these two distinct sensor types, which, to our knowledge, has not been systematically explored in this context.

Research into emerging sensors like DVS has gained significant traction, primarily for their performance advantages in challenging conditions. The work by Gehrig and Scaramuzza [32] highlights the benefits of event cameras for low-latency automotive vision, while other studies focus on their high dynamic range and reduced motion blur [28]. Sakhai et al. [53] further demonstrated the effectiveness of DVS in real-time pedestrian detection under adverse weather, reinforcing its practical value in safety-critical AV scenarios. However, the vast majority of DVS research is centered on perception algorithms and performance benchmarks rather than cybersecurity resilience. Our paper addresses this critical gap. We conduct one of the first systematic evaluations of DVS security within an AV context, implementing novel attack vectors like event flooding and directly comparing the sensor’s resilience to that of a traditional RGB camera under attack. Although our DVS integration is for research and analysis purposes and not part of the active driving model, this investigation provides crucial, forward-looking insights into the security profile of this promising technology.

In addressing the limitations inherent in simulation-based studies, we position our work as a necessary and responsible precursor to real-world implementation. The complexity, cost, and safety risks associated with testing cyberattacks on physical vehicles necessitate thorough validation in a controlled environment. By using the leading open-source CARLA simulator [36], we provide a replicable and quantifiable baseline that is essential for guiding future hardware-in-the-loop and on-road experiments, which require initial research studies based on such foundational work. Our findings, particularly on the limited effectiveness of RGB defenses (30–45% trajectory drift) and the complete mitigation of GPS spoofing, offer a clear roadmap for prioritizing the development of real-world defense mechanisms. This work, therefore, serves as a critical stepping stone, providing the initial validation and security analysis required before these advanced sensor systems can be safely deployed on public roads.

In summary, this paper distinguishes itself from prior work through three main contributions. First, it provides the first systematic, comparative analysis of RGB and DVS sensor resilience in a unified cybersecurity context using the CARLA simulator. Second, it introduces and validates a tailored, high-performance anomaly detection system for these specific sensor streams, demonstrating its practical effectiveness. Finally, by quantifying the limitations of existing defenses, it provides a crucial, foundational dataset and analysis that directly informs the next stages of research and paves the way for the development of robust, real-world security frameworks.

3. Materials and Methods

This section describes the experimental setup, methodology, and implementation details used to evaluate the resilience of autonomous vehicles against cyberattacks, with a particular focus on Dynamic Vision Sensors (DVS) and RGB cameras. We provide comprehensive information to enable reproducibility of our experiments and findings.

3.1. Experimental Framework

3.1.1. CARLA Simulator

All experiments were conducted using the CARLA simulator (version 9.15), an open-source simulator for autonomous driving research that provides a realistic urban environment with detailed physics and sensor models. CARLA was chosen for its ability to accurately simulate various sensor modalities including RGB cameras, LiDAR, radar, and event-based sensors. The simulator was run on a Linux Ubuntu 22.04 system with CUDA-enabled GPUs to support the computational requirements of the autonomous driving agents and sensor processing pipelines [36].

3.1.2. PCLA Framework

We utilized the Pretrained CARLA Leaderboard Agent (PCLA) framework, which enables the deployment of autonomous driving agents from the CARLA Leaderboard onto vehicles within the simulator [40]. This framework provides several advantages for our experimental setup:

• Ability to deploy multiple autonomous driving agents with different architectures and training paradigms
• Independence from the Leaderboard codebase, allowing compatibility with the latest CARLA version
• Support for multiple vehicles with different autonomous agents operating simultaneously

The PCLA framework includes nine different high-performing autonomous driving agents trained with 17 distinct training seeds, allowing us to evaluate the impact of cyberattacks across a diverse set of perception and control algorithms.

3.1.3. Autonomous Driving Agent

For our experiments, we focused exclusively on the following autonomous driving agent:

• NEAT AIM-MT-2D: A neural attention-based end-to-end autonomous driving agent that processes RGB images and depth information to predict vehicle controls directly. We used the NEAT variant that incorporates depth information (neat_aim2ddepth), which enhances the agent’s ability to perceive the 3D structure of the environment and improves its performance in complex driving scenarios.

This agent represents an end-to-end learning approach to autonomous driving that directly maps sensor inputs to control commands, allowing us to assess the impact of attacks on a unified perception-control architecture.

3.2. Sensor Configuration

Our experimental setup incorporated a comprehensive sensor suite to evaluate the resilience of autonomous vehicles against various attack vectors:

• RGB Cameras: Front-facing RGB camera with 800 × 600 resolution and a 100° field of view, mounted at position (1.3, 0.2, 2.3) relative to the vehicle center, providing visual input for the agent’s perception system.
• Dynamic Vision Sensor (DVS): Event-based camera with 800 × 600 resolution and a 100° field of view, mounted at position (1.3, 0.0, 2.3), with positive and negative thresholds set to 0.3 to capture brightness changes in the scene with microsecond temporal resolution.
• Depth Camera: Depth-sensing camera with 800 × 600 resolution and a 100° field of view, aligned with the RGB camera position, providing per-pixel distance measurements for 3D scene understanding.
• LiDAR: A 64-channel LiDAR sensor with an 85 m range, 600,000 points per second, and a 10 Hz rotation frequency, mounted at position (0.0, 0.0, 2.5), providing detailed 3D point cloud data of the surrounding environment.
• GPS and IMU: For localization and vehicle state estimation, enabling the agent to maintain awareness of its position and orientation within the environment.

It is important to note that in this experimental setup, the DVS cameras and LiDARs were primarily used for research purposes and data collection and were not directly integrated into the vehicle’s driving decision model.

To better illustrate the RGB sensor integration and data flow within our experimental framework, Figure 1 presents a comprehensive workflow diagram that demonstrates the sensor attachment process, data acquisition pipeline, and the subsequent processing stages. This workflow visualization provides crucial insight into how RGB sensor data is captured, processed, and utilized within the CARLA simulation environment, forming the foundation for our cybersecurity evaluation methodology.

Similarly, the dynamic vision sensor (DVS) follows a comparable workflow architecture to the RGB sensor system, with several key distinctions that reflect the unique characteristics of event-based vision technology. Figure 2 illustrates the DVS sensor attachment and data processing pipeline within our experimental framework.

The DVS data processing pipeline differs from the RGB pipeline in several critical aspects:

• Event-based data capture generates asynchronous streams of brightness change events rather than synchronized frame sequences;
• Event-to-image conversion transforms sparse temporal events into dense spatial representations for compatibility with existing vision processing frameworks;
• Specialized defense mechanisms employ spatial clustering algorithms to detect and filter noise events; and
• Importantly, DVS data streams are utilized primarily for research analysis and anomaly detection rather than direct integration into the PCLA (Pretrained CARLA Leaderboard Agent) and vehicle control model.

This architectural decision allows us to evaluate the intrinsic security characteristics of event-based vision.

Sensor data was collected at 10 Hz to match the control frequency of the autonomous driving agent. All sensor data was synchronized using the CARLA simulator’s internal clock to ensure temporal consistency across modalities. This comprehensive sensor configuration allowed us to evaluate the impact of attacks on individual sensors as well as the effectiveness of multi-sensor fusion for attack detection and mitigation.

3.3. Attack Implementation

We implemented and evaluated seven distinct cyberattacks targeting the sensor inputs of autonomous vehicles, with attack parameters carefully selected based on existing cybersecurity literature and real-world attack feasibility studies. Each attack represents a realistic threat vector that could be executed by adversaries with moderate technical capabilities:

• RGB Camera Attacks, Salt-and-Pepper Noise (80% intensity): We implemented a high-density salt-and-pepper noise attack where random pixels were replaced with either black (0) or white (255) values with equal probability [37,54]. The 80% intensity parameter represents an extreme noise level commonly used in image denoising research to test algorithm robustness under severe corruption conditions. This attack simulates severe sensor interference from electromagnetic attacks or physical sensor tampering [15,27].
• Dynamic Vision Sensor Attacks, Event Flooding (60% spurious events): We implemented an event flooding attack that injected spurious events with random positions and polarities into the DVS output stream [30,33]. The 60% flooding rate was selected to represent a significant degradation scenario that would challenge event-based processing algorithms while remaining within realistic attack capabilities [28,29,34]. This attack exploits the asynchronous nature of DVS devices and could be implemented through electromagnetic interference or direct sensor manipulation [35].
• Depth Camera Attacks, Depth Map Tampering (5 patches, 50 × 50 pixels): We implemented a patch-based depth tampering attack that modified depth values in random regions of the depth map following established adversarial patch methodologies [38,55]. The attack parameters (5 patches of 50 × 50 pixels with depth offsets of 5–15 units) were selected to balance attack effectiveness with practical implementation constraints [22,56]. This creates false distance measurements that could cause collision avoidance systems to malfunction [9].
• LiDAR Attacks, Phantom Object Injection (5 clusters, 100–500 points each): We implemented a phantom object injection attack that added false point clusters to the LiDAR point cloud at positions 5–15 m ahead of the vehicle [9,22]. The attack parameters (5 clusters with 100–500 points each) were designed to create detectable objects while remaining within the spoofing capabilities demonstrated in recent LiDAR security research [23,27,57]. This attack could be implemented using laser devices or reflective materials [15,22].
• GPS Spoofing Attacks, Position Manipulation (±50 m deviation): We implemented a GPS spoofing attack that created discrepancies between actual and perceived vehicle positions [24,25]. The ±50 m deviation threshold represents a significant but realistic spoofing scenario that would noticeably impact navigation while remaining within documented GPS attack capabilities [9,58,59]. The attack maintains separate records of actual and spoofed positions to enable quantitative analysis of navigation impact.
• Denial of Service (DoS) Attacks, Sensor Rate Limiting (50% packet loss): We implemented a DoS attack that restricted sensor update frequency to simulate network-based attacks on sensor communication channels [11]. The 50% packet loss rate represents severe network conditions that significantly impact autonomous vehicle performance, as demonstrated in recent automotive cybersecurity research [16]. This attack simulates jamming or network flooding scenarios [17,18].
• Steering Bias Attacks, Control Manipulation (±15° systematic offset): We implemented a steering bias attack that introduced systematic errors into steering commands [11,16]. The ±15° bias parameter was selected to represent a significant control deviation that would be detectable by monitoring systems while demonstrating clear attack impact [17]. This attack simulates compromised control systems or actuator manipulation [18].

All attacks were implemented by intercepting and modifying the sensor data in the callback functions before it was processed by the autonomous driving agent. This approach simulates a realistic attack scenario where an adversary has gained access to the sensor data stream but not necessarily to the internal processing algorithms of the autonomous system.

3.4. Defense Integration

To protect against the implemented attacks, we developed and integrated a multi-layered defense system that combines sensor-specific filtering techniques with machine learning-based anomaly detection. Our defense architecture implements both traditional signal processing methods and deep learning-based anomaly detection to provide comprehensive protection against sensor attacks.

3.4.1. Traditional Defense Mechanisms

We implemented sensor-specific defense mechanisms in the sensor callback functions to provide real-time protection:

• RGB Camera Defenses: We implemented a decision-based adaptive median filter that dynamically adjusted its kernel size (3 × 3 to 7 × 7) based on the detected noise level. The filter specifically targets salt-and-pepper noise by

  –

  Identifying potential noise pixels (values of 0 or 255),

  –

  Growing the filter window until valid median values are found, and

  –

  Preserving edge details by only replacing confirmed noise pixels.

• DVS Defense: We introduced a system to track event counts, setting a threshold of 5000 events per frame to identify flooding attacks, along with a dual-phase defense strategy composed of

  –

  Spatial clustering analysis using KD-tree data structures to identify dense noise patterns and

  –

  Morphological operations (dilation) for standard noise smoothing when clustering is not detected.

• Depth Camera Defense: A multi-stage validation approach including the following:

  –

  Range clipping to valid depth values (0–100 m),

  –

  Gradient-based tampering detection using Sobel operators, and

  –

  Adaptive Gaussian smoothing with intensity based on detected gradient magnitude.

• LiDAR Defense: A point cloud filtering pipeline with

  –

  Distance-based filtering (maximum range 50 m),

  –

  Density-based clustering using KD-tree analysis to detect phantom objects, and

  –

  Removal of points in clusters exceeding density thresholds.

• GPS Spoofing Defense: Route consistency validation and velocity checks:

  –

  Position validation against planned route (deviation threshold: 5 m),

  –

  Velocity consistency checks (maximum realistic speed: 20 m/s), and

  –

  Automatic reversion to true GPS coordinates when anomalies are detected.

• Steering Bias Defense: Statistical anomaly detection for control commands:

  –

  Maintains 50-tick steering command history,

  –

  Z-score analysis for outlier detection (threshold: |Z| > 3), and

  –

  Automatic correction to historical mean when a bias is detected.

• DoS Defense: Sensor update monitoring and rate limiting:

  –

  Timestamp tracking for all sensor updates,

  –

  Rate limiting enforcement (maximum 20 Hz), and

  –

  Buffering mechanisms for delayed sensor readings.

3.4.2. Machine Learning-Based Anomaly Detection

In addition to traditional filtering methods, we implemented a comprehensive anomaly detection system using the EfficientAd model from the anomalib library. This system provides an additional layer of security by learning normal sensor behavior patterns and detecting deviations that may indicate attacks.

The anomaly detection system operates on RGB and DVS sensor data, providing anomaly scores that can be used to identify potential attacks even when traditional filtering methods may not be sufficient. The system is trained exclusively on normal sensor data, allowing it to detect novel attack patterns without requiring prior knowledge of specific attack types.

These defense mechanisms work in conjunction to provide comprehensive protection against the implemented attack vectors while maintaining real-time performance requirements for autonomous driving applications.

3.5. Anomaly Detection System

We implemented a comprehensive anomaly detection system using the anomalib library to identify potential sensor attacks through machine learning-based analysis. The system uses the EfficientAD model, which demonstrated superior performance for detecting anomalies in autonomous driving sensor data.

3.5.1. EfficientAD Model Architecture

The EfficientAD model was selected as our primary anomaly detection approach due to its effectiveness in detecting sensor attacks without requiring prior knowledge of attack patterns. EfficientAD employs a student–teacher framework that identifies anomalies based on the discrepancy between teacher and student network predictions. The architecture consists of the following:

• A teacher network that learns robust feature representations from normal sensor data,
• A student network that attempts to replicate the teacher’s feature extraction capabilities, and
• A discrepancy measurement module that quantifies differences between teacher and student outputs to generate anomaly scores.

This architecture is particularly suitable for sensor attack detection, as it can identify subtle deviations from normal sensor behavior patterns without requiring examples of attacks during the training phase.

3.5.2. Training and Implementation

The anomaly detection system was implemented with the following methodology:

• Data Collection and Organization: Sensor data was collected from multiple driving scenarios under normal conditions and organized into training and testing datasets with separate directories for normal and anomalous data.
• Data Preprocessing: All sensor data underwent standardized preprocessing:

  –

  Images resized to 128 × 128 pixels for computational efficiency,

  –

  Normalization using ImageNet statistics (mean = [0.485, 0.456, 0.406], std = [0.229, 0.224, 0.225]), and

  –

  LiDAR point clouds converted to 2D bird’s-eye-view images for compatibility with the image-based model.

• Model Training Configuration:

  –

  Training performed using PyTorch 1.13.1 with CUDA 11.7 acceleration

  –

  Batch size of 1 with gradient accumulation over 4 batches for memory efficiency

  –

  Training duration of 30 epochs with validation every epoch

  –

  Separate models trained for each sensor type (RGB, DVS, depth, and LiDAR). The detailed training process and loss behavior for the DVS anomaly detection model are illustrated in Figure A1 in Appendix A.

• Real-time Analysis: The trained models are used to analyze sensor data during autonomous driving episodes, generating anomaly scores that indicate the likelihood of sensor attacks. The system can process sensor data in real-time and provide continuous monitoring of sensor integrity.

The anomaly detection system operates independently of the traditional filtering defenses, providing an additional layer of security that can detect novel attack patterns and complement the rule-based defense mechanisms.

3.6. Evaluation Methodology

We evaluated both the impact of attacks on autonomous driving performance and the effectiveness of our defense mechanisms using a comprehensive set of metrics and visualization tools:

3.6.1. Attack Impact Assessment

The impact of attacks was measured using the following metrics:

• Trajectory Analysis: We track and compare the vehicle’s actual trajectory against the planned route, calculating point-to-segment distances to measure route deviation. This analysis is particularly important for GPS spoofing attacks where perceived and actual positions may differ significantly.
• Control Stability: We analyze steering, throttle, and brake commands through detailed time series analysis. For steering bias attacks, we perform statistical analysis of steering angle distributions to detect anomalous patterns.
• Sensor Performance Metrics:

  –

  RGB Camera: Noise percentage measurements during salt-and-pepper attacks

  –

  DVS: Event count tracking to detect abnormal spikes in event generation

  –

  Depth Camera: Mean depth measurements to identify tampering

  –

  LiDAR: Point cloud density analysis to detect phantom objects

• Defense Effectiveness: Comparison of performance metrics with and without defensive measures enabled, including

  –

  Adaptive median filtering for RGB noise,

  –

  Spatial clustering analysis for DVS event flooding,

  –

  Gradient-based analysis for depth tampering,

  –

  Density-based filtering for LiDAR attacks, and

  –

  Rate limiting for DoS attacks.

3.6.2. Visualization and Analysis Tools

We developed several visualization tools to analyze attack impacts:

• Combined Video Generation: Synchronized display of multiple sensor feeds (RGB, DVS, depth, and LiDAR) with attack state indicators
• Trajectory Plots: Visualization of actual vs. perceived trajectories during GPS spoofing
• Statistical Analysis: Histograms of steering distributions and sensor metrics under different attack conditions

3.6.3. Experimental Scenarios

We evaluated performance across diverse driving scenarios:

• Urban Navigation: Complex urban environments with intersections, traffic lights, and other vehicles.
• Highway Driving: High-speed scenarios with lane changes and overtaking maneuvers.
• Dynamic Obstacles: Scenarios with pedestrians and other vehicles executing unpredictable maneuvers.

Each scenario was tested with and without attacks to establish baseline performance and measure the impact of different attack types and intensities.

All experiments were conducted using Python 3.7 with PyTorch 1.13.1 and the anomalib library. The CARLA simulator was run on a system with an Intel Core i9-10900K CPU @ 3.70 GHz, 62 GB RAM, and an NVIDIA GeForce RTX 3080 GPU with 10 GB VRAM. All equipment was sourced from x-kom.pl, Krakow, Poland.

4. Results

This section presents the results of our comprehensive evaluation of cyberattacks targeting autonomous vehicle sensors and the effectiveness of our proposed defense mechanisms, based on over 160 experimental episodes conducted in the CARLA simulator, with each episode lasting approximately 6 min across diverse scenarios. Our experimental design evaluated seven distinct attack vectors (salt-and-pepper noise, event flooding, depth map tampering, LiDAR phantom injection, GPS spoofing, denial of service, and steering bias control) as described in the Materials and Methods section. We organize our findings into several key areas: impact analysis of cyberattacks on autonomous driving performance, effectiveness evaluation of defense mechanisms, anomaly detection performance assessment, comparative analysis of RGB and DVS device security, and implications for autonomous vehicle security. Key performance indicators include route completion rates, which exceeded 90% under baseline conditions and remained above 50% under attack scenarios, along with trajectory deviation analysis.

4.1. Impact of Cyberattacks on Autonomous Driving Performance

We evaluated the resilience of autonomous vehicles to various attack vectors by analyzing their impact on driving performance, trajectory adherence, and sensor reliability. Our experiments revealed two distinct outcomes: scenarios where the vehicle completed the route despite attacks (albeit with performance degradation) and scenarios where attacks caused route abandonment or crashes.

4.1.1. Completed Routes Under Attack

In some scenarios, the autonomous vehicle demonstrated resilience by completing the designated route despite being subjected to cyberattacks. Episode 7 provides a notable example of this behavior, as illustrated in Figure 3.

While the vehicle successfully completed the route in Episode 7, the trajectory analysis reveals significant deviations from the optimal path during attacks. RGB camera attacks caused the most pronounced lateral deviations, with the vehicle weaving across lanes before correcting its course. Depth sensor attacks similarly disrupted the vehicle’s ability to maintain a smooth trajectory, though to a lesser extent than RGB attacks. These deviations highlight the vehicle’s compromised perception capabilities even when it manages to reach its destination.

Figure 4 shows the corresponding sensor outputs during normal operation and under different attack conditions for Episode 7. The RGB camera feed exhibits significant salt-and-pepper noise during attacks, severely degrading visual perception. Similarly, the depth sensor distorted distance measurements, potentially causing the autonomous system to misinterpret obstacle distances and road boundaries.

Episode 26 provides another example of route completion despite attacks, as shown in Figure 5. In this case, the vehicle deviated visibly from its intended route during RGB camera attacks but still successfully reached its destination. This highlights a nuanced vulnerability where the vehicle can complete its mission despite taking a potentially dangerous path along a simple pathway.

4.1.2. Route Failures and Crashes

In contrast to the resilience observed in Episodes 7 and 26, several scenarios demonstrated significant failures where attacks caused the vehicle to crash.

Episode 5 demonstrates an even more severe failure mode, where RGB camera attacks caused the vehicle to move erratically before ultimately crashing into a roadside fence. Figure 6 shows the vehicle’s trajectory during this episode, with the abrupt termination point indicating the crash location. The red box in the figure highlights the area where the significant deviation from the planned route occurred, illustrating the severity of the attack’s impact on vehicle navigation.

The sensor data from Episode 5 (Figure 7) reveals the severe degradation of perception capabilities that led to the crash due to the RGB salt-and-pepper noise attack that started in the second column. The RGB camera feed shows extreme noise corruption, while the fence that the vehicle ultimately collided with is visible in the sensor outputs. The red box in the figure highlights the fence structure after the collision, demonstrating the physical impact of the RGB attack on vehicle navigation.

The control inputs during Episode 5 (Figure 8) further illustrate the vehicle’s instability under attack. During RGB attacks, the control system exhibits erratic behavior, alternating between acceleration and full braking as it struggles to interpret the corrupted sensor data, especially during RGB salt-and-pepper noise attacks.

Similarly, Episode 27 demonstrates a multi-stage failure where the vehicle crashed into a traffic light in the intersection during the initial attack phase, mainly during the RGB salt-and-pepper noise attack. Although it temporarily regained its route, it crashed again during a subsequent attack even with defense mechanisms active, as shown in Figure 9 and Figure 10.

4.2. Effectiveness of Defense Mechanisms

Our proposed defense mechanisms aim to mitigate the impact of cyberattacks on the sensors of autonomous vehicles. We evaluated their effectiveness by comparing vehicle performance with and without defenses under identical attack conditions. Figure 11 illustrates the impact of our defense mechanisms on sensor data quality during attacks with defense mechanisms in Episode 7.

Figure 12 and Figure 13 provide a visual comparison of sensor outputs during attacks, demonstrating the effectiveness and limitations of our defense mechanisms. These comparisons show the sensor data quality with and without defenses enabled during identical attack scenarios.

The defense mechanisms demonstrated varying degrees of effectiveness depending on the attack type and intensity:

• RGB Camera Attacks: Our anomaly detection model for RGB cameras successfully identified attacks on the camera feed, but the subsequent filtering techniques showed limited effectiveness against sophisticated attacks. While some noise reduction was achieved, the filtered images often remained significantly compromised, resulting in continued trajectory deviations even with defenses active.
• Depth Sensor Attacks: The depth sensor anomaly detection system effectively detected anomalies in depth data, but defense mechanisms for depth sensor attacks demonstrated limited effectiveness against targeted attacks, with filtered depth maps still containing significant distortions that affected the vehicle’s perception capabilities.
• GPS Spoofing: Our GPS defense mechanism successfully detected and mitigated GPS spoofing attacks by implementing plausibility checks based on route deviation and velocity consistency. The defense mechanism checks if the reported GPS position deviates more than 5.0 m from the planned route or if the calculated speed exceeds 20.0 m/s, reverting to the true transform when these thresholds are exceeded. This was our most effective defense, as it clearly identified spoofed GPS coordinates and prevented the vehicle from making unwarranted course corrections based on falsified location data.
• Steering Bias: The steering defense mechanism uses statistical anomaly detection with z-score analysis on a rolling window of the last 50 steering commands. When a steering command’s z-score exceeds 3 (indicating a statistical outlier), the system reverts to the mean steering value from the recent history. This approach showed partial effectiveness in identifying malicious steering commands, but sophisticated attacks could still influence vehicle control in certain scenarios.

Our experiments revealed significant limitations in our current defense mechanisms for most sensor types except GPS. In Episode 27, Figure 9, the vehicle crashed during an attack even with defenses active, and across multiple episodes, the defenses for RGB and depth sensors failed to adequately protect the vehicle’s perception system. These findings highlight a critical vulnerability in current autonomous vehicle security approaches.

To address these limitations, we propose an enhanced anomaly detection system that can serve as a fallback mechanism when primary defenses fail. This system would

• Continuously monitor all sensor inputs for anomalies using our trained detection models;
• If anomalies were detected, reduce reliance on AI models for driving decisions;
• Transfer driving control to human operators for a safer experience when sensor integrity is compromised; and
• Alternatively, selectively shut down compromised sensors and cross-check with remaining functional sensors to maintain autonomous operation.

This approach acknowledges the reality that perfect defense against all attack vectors may not be achievable and instead focuses on rapid detection and appropriate fallback mechanisms to ensure safety. The implementation of such a system would be a promising direction for future research.

4.3. Anomaly Detection Performance

A critical component of our proposed enhanced defense framework is the anomaly detection system, which aims to identify malicious sensor manipulations in real-time and trigger appropriate fallback mechanisms. We evaluated the performance of our anomaly detection models for both RGB and DVS camera sensors using the anomalib library to assess their potential as a foundation for this enhanced security approach.

4.3.1. RGB Camera Anomaly Detection

The RGB camera anomaly detection model demonstrated excellent performance in distinguishing between normal operation and attack conditions. Figure 14 shows the anomaly scores for different operational states in Episode 1.

Statistical analysis of the RGB anomaly detection results revealed the following:

• Normal Operation: Mean anomaly score of 0.000000 with a standard deviation of 0.000000, establishing a clear baseline for normal behavior.
• RGB Noise Attacks: Mean anomaly scores of 0.378064 (with defense) and 0.389462 (without defense), both significantly higher than normal operation. This demonstrates the model’s ability to reliably detect RGB camera attacks regardless of whether defenses are active.
• Our RGB camera anomaly detection model is specifically designed to detect anomalies in RGB camera data only and does not process data from other sensors.

These results demonstrate that our RGB anomaly detection model achieves high sensitivity to relevant attacks while maintaining specificity against unrelated attack vectors.

4.3.2. DVS Camera Anomaly Detection

The DVS camera anomaly detection model showed different characteristics compared to the RGB model, reflecting the unique properties of event-based vision. Figure 15 illustrates the anomaly scores for different operational states in Episode 5.

Statistical analysis of the DVS anomaly detection results revealed:

• Normal Operation: Mean anomaly score of 0.608308 with a standard deviation of 0.121466, establishing a baseline that reflects the inherent variability of event-based vision data.
• DVS Noise Attacks: Mean anomaly scores of 0.55 (with defense) and 0.75 (without defense). Interestingly, the score with defense is slightly lower than normal operation, while the score without defense is significantly higher.
• It is important to note that our DVS anomaly detection model is specifically designed to detect anomalies in DVS data only and does not process data from other sensors. Additionally, DVS cameras are currently used for research purposes only and are not integrated into the vehicle’s driving decision model.

These results highlight the importance of sensor-specific anomaly detection in autonomous vehicles. The DVS anomaly detection model successfully identifies abnormal patterns in DVS data, while other sensor-specific models (such as the RGB anomaly detection model) are responsible for detecting attacks on their respective sensors. Each detection system is specialized and optimized for its particular sensor type, emphasizing the need for a multi-layered security approach.

4.4. Comparative Analysis of DVS and RGB Sensor Security

Our experiments provide valuable insights into the relative security characteristics of traditional RGB cameras and emerging DVS technology:

• Research Context: It is important to emphasize that the DVS cameras in our study were used exclusively for research purposes and were not integrated into the vehicle’s driving decision model. The data collected from DVS cameras was analyzed separately from the main autonomous driving pipeline.
• Defense Effectiveness: Our defense mechanisms were evaluated separately for each sensor type. RGB cameras, which are actively used in the driving model, required more aggressive filtering that sometimes reduced image quality. DVS defenses were studied in isolation as a research component.
• Anomaly Detection: The anomaly detection models for both sensors were designed to work exclusively with their respective sensor data. The RGB model showed clearer separation between normal and attack conditions and directly impacted vehicle safety, while the DVS model’s patterns were analyzed purely for research insights.
• Future Potential: While not currently used for driving decisions, the complementary nature of RGB and DVS cameras suggests that sensor fusion approaches could potentially enhance security in future implementations.

As shown in

Table 1. Summary of Sensor Attacks and Defense Effectiveness.

Sensor Type	Attack Type	Results Without Defense	Results with Defense
RGB Camera	Salt-and-Pepper Noise (80%)	Severe trajectory deviations; vehicle weaving across lanes; crashes in some episodes	Limited effectiveness; 30–45% trajectory drift still present; filtered images remain compromised
Dynamic Vision Sensor (DVS)	Event Flooding (60% spurious events)	False motion perception; higher anomaly scores (0.76)	Limited effectiveness; anomaly detection detects attack phase; event filter still compromised; lower anomaly scores (0.55)
Depth Camera	Depth Map Tampering (random patches)	Misinterpreted obstacle distances; affected perception of road boundaries	Limited effectiveness; filtered depth maps still contain significant distortions
LiDAR	Phantom Object Injection (5 clusters)	False obstacle detection	Partial mitigation through point cloud filtering; density-based clustering removes some phantom objects
GPS	Position Manipulation	Significant deviation between reported and actual position; incorrect routing decisions	Highly effective mitigation; route deviation and velocity consistency checks prevented course corrections based on falsified data
Sensor Update	Denial of Service (rate limiting)	Delayed/missed sensor readings; affected perception and decision-making	Partial mitigation through sensor update monitoring and buffering mechanisms
Control System	Steering Bias	Systematic errors in steering commands; trajectory deviation	Partial effectiveness; sophisticated attacks could still influence vehicle control

, these findings suggest that incorporating DVS technology alongside traditional sensors can enhance the security posture of autonomous vehicles, particularly in high-dynamic-range scenarios where RGB cameras are more likely to be compromised by environmental factors or deliberate attacks.

5. Discussion

Our comprehensive analysis of cyberattacks targeting autonomous vehicle sensors reveals critical insights into the security vulnerabilities and resilience mechanisms of these systems. In this section, we interpret our findings in the context of existing research and discuss their broader implications for autonomous vehicle security.

5.1. Implications for Autonomous Vehicle Security

The results of our experiments demonstrate that autonomous vehicles remain vulnerable to a range of sensor-based attacks despite advances in perception technologies. The successful completion of routes under attack conditions in Episodes 7 and 26, albeit with significant trajectory deviations, suggests that current autonomous driving systems possess a degree of inherent resilience. However, the catastrophic failures observed in Episodes 5 and 27, where vehicles crashed due to sensor manipulation, highlight the critical nature of these vulnerabilities [60]. An illustrative example of such a critical failure under an RGB attack is presented in Figure A2 in Appendix B.

Particularly concerning is our observation that defense mechanisms showed limited effectiveness against sophisticated attacks on RGB and depth sensors. While our GPS spoofing defenses performed well, the continued vulnerability of visual perception systems represents a significant security gap that malicious actors could exploit [47]. This suggests that the current approach of hardening individual sensors against attacks may be insufficient, and a more holistic security architecture is needed [51].

5.2. Significance of Anomaly Detection Approach

Our anomaly detection system demonstrates promising capabilities for identifying malicious sensor manipulations in real time. The RGB camera anomaly detection model achieved excellent discrimination between normal operation and attack conditions, with attack scenarios consistently producing anomaly scores significantly higher than baseline. This clear separation suggests that anomaly detection could serve as a reliable trigger for fallback mechanisms when primary defenses fail [7].

Interestingly, the DVS camera anomaly detection model exhibited different characteristics, reflecting the unique properties of event-based vision. The higher baseline variability in DVS anomaly scores indicates that event-based sensors may require more sophisticated detection algorithms that account for their asynchronous nature [28]. Nevertheless, the model successfully identified abnormal patterns during attacks, particularly when defenses were not active. Further details on the training process and the observed loss plateau indicating optimal understanding of the training data can be found in Appendix A.

These findings support the potential of anomaly detection as a foundational component of a multi-layered security approach [44]. By continuously monitoring sensor inputs for deviations from expected patterns, autonomous vehicles could rapidly identify potential attacks and implement appropriate countermeasures before safety is compromised.

5.3. Comparative Security of DVS and RGB Sensors

Our comparative analysis of DVS and RGB sensor security reveals important distinctions between these technologies. Traditional RGB cameras, while providing rich visual information, demonstrated significant vulnerability to noise-based attacks that severely degraded perception capabilities. In contrast, the event-based nature of DVS technology offers potential security advantages through its high dynamic range and reduced data redundancy [28].

The complementary characteristics of these sensor types suggest that a fusion approach could enhance overall system security [61]. RGB cameras excel in static scene understanding and color perception, while DVS sensors offer superior performance in high-dynamic-range scenarios and rapid motion detection. By integrating data from both sensor types and implementing cross-validation mechanisms, autonomous vehicles could potentially detect inconsistencies that indicate sensor manipulation [47].

However, it is important to note that our study used DVS cameras exclusively for research purposes, without integration into the vehicle’s driving decision model. Future work should explore the practical challenges of incorporating DVS technology into production autonomous vehicles, including calibration requirements, computational overhead, and integration with existing perception pipelines [62].

5.4. Limitations and Challenges

Our research has several limitations that should be acknowledged. First, all experiments were conducted in simulation using the CARLA environment, which may not fully capture the complexity and unpredictability of real-world driving scenarios. While CARLA provides realistic physics and sensor models, real-world implementation would face additional challenges such as sensor noise, environmental variability, and hardware constraints. To address the inherent limitations of simulation-based studies and bridge the gap towards real-world validation, we have procured an AWS DeepRacer agent and iniVation Dynamic Vision Sensors (DVS), detailed specifications of which are provided in Appendix C.

Second, our attack models represent a subset of possible attack vectors. More sophisticated attacks, such as adversarial machine learning techniques that target specific neural network vulnerabilities, were not fully explored in this study [48]. Such attacks could potentially bypass our anomaly detection systems by generating perturbations that appear normal while causing misclassification.

Third, our defense mechanisms showed limited effectiveness against certain attack types, particularly for RGB and depth sensors. This highlights the challenge of developing robust defenses that can maintain perception quality while mitigating attacks [62]. The trade-off between security and performance remains a significant challenge for autonomous vehicle designers [60].

Finally, the computational requirements of our anomaly detection system may present challenges for real-time implementation on resource-constrained vehicle platforms [52]. While our experiments used high-performance GPUs, production vehicles may have more limited computational capabilities, necessitating the optimization of detection algorithms.

5.5. Future Research Directions

Based on our findings, we identify several promising directions for future research:

• Enhanced Anomaly Detection: Developing more sophisticated anomaly detection models that can identify subtle attacks while maintaining low false positive rates. This could include exploring deep learning architectures specifically designed for time series sensor data and incorporating contextual information from multiple sensors [45].
• Sensor Fusion for Security: Investigating how data from complementary sensors (RGB, DVS, LiDAR, and radar) can be fused not only for improved perception but also as a security mechanism [61]. Inconsistencies between different sensor modalities could serve as indicators of potential attacks [47].
• Graceful Degradation: Designing autonomous systems that can gracefully degrade performance when attacks are detected rather than failing catastrophically. This could involve developing fallback driving policies that rely on uncompromised sensors or implementing safe stop procedures [7].
• Human-in-the-Loop Security: Exploring how human operators could be effectively integrated into the security framework, particularly for remote monitoring and intervention when anomalies are detected [60]. This raises important questions about interface design, situation awareness, and response time.
• DVS Integration: Further investigating the potential of DVS technology for enhancing autonomous vehicle security, including developing specialized perception algorithms that leverage the unique properties of event-based vision [28] and exploring hybrid RGB–DVS architectures.
• Standardized Security Evaluation: Developing standardized benchmarks and evaluation methodologies for assessing the security of autonomous vehicle sensor systems, enabling more systematic comparison of different defense approaches [62].

Addressing these research directions will require interdisciplinary collaboration between experts in computer vision, cybersecurity, autonomous systems, and human factors [51]. By advancing our understanding of sensor vulnerabilities and developing more robust defense mechanisms, we can enhance the security and trustworthiness of autonomous vehicles as they transition from research laboratories to public roads.