Symmetrical Simulation Scheme for Anomaly Detection in Autonomous Vehicles Based on LSTM Model
Abstract
:1. Introduction
1.1. Our Contributions
- We present a new dataset to simulate the False Data Injection (FDI) attacks on autonomous vehicles. The dataset was generated from the simulation model after integrating the cyberattack. False Data Injection (FDI) attacks were injected into an autonomous vehicle (A.V.) simulation-based system developed by MathWorks Inc. for research purposes. We assumed an attacker compromised a smart sensor.
- We propose an intelligent anomaly detection method based on long short-term memory (LSTM) neural networks to identify False Data Injection (FDI) attacks targeting the control system of the autonomous vehicle through a compromised sensor. The proposed anomaly detection system can classify communication traffic of autonomous vehicles into normal or anomaly data.
- We provide extensive experimental evaluation results using standard performance indication factors such as detection accuracy, precision, recall, and F Score. Ultimately the prosed system achieved an overall accuracy equal to 99.95%.
1.2. Paper Organization
2. Literature Review
2.1. Existing Related Models
2.2. Research Gap and Novelty
3. Autonomous Vehicles Simulation Model
4. System Development and Specifications
4.1. A Scheme for Generating Dataset
4.1.1. Implementation of Cyberattack
Algorithm 1. Calculating Actual Sensor Value Under FDI |
Input_1: Original Sensor Readingin meter |
Input_2: Attack Percentage (0.00001% to 100%) |
Processing: Get Original Sensor Reading |
Assume Attack Percentage = |
Compute: |
Then: |
Output: Actual Sensor Value |
4.1.2. Dataset Features
4.1.3. Anomaly Detection Label
4.2. Implementation of LSTM
4.3. Training Procedure
Adaptive Moment Estimation Optimization (ADAM)
4.4. Testing Procedure
- K-Fold Cross-Validation (already discussed in Section 4.3)
- Confusion matrix
- Evaluation metrics (precision, recall, and F1-score)
- Comparison with existing methods.
5. Conclusions
Author Contributions
Funding
Institutional Review Board Statement
Informed Consent Statement
Data Availability Statement
Acknowledgments
Conflicts of Interest
References
- Sziroczák, D.; Rohács, D. Conflict Management Algorithms Development Using the Automated Framework for Autonomous Vehicles. In Proceedings of the First Conference on ZalaZONE Related R & I Activities of Budapest University of Technology and Economics 2022, Budapest University of Technology and Economics, Budapest, Hungary, 31 March 2022. [Google Scholar]
- Rahman, S.; Aburub, H.; Mekonnen, Y.; Sarwat, A.I. A Study of EV BMS Cyber Security Based on Neural Network SOC Prediction. In Proceedings of the EEE/PES Transmission and Distribution Conference and Exposition (T & D), Denver, CO, USA, 16–19 April 2018. [Google Scholar]
- Al-Haija, Q.A.; Al Tarayrah, M.I.; Enshasy, H.M. Time-Series Model for Forecasting Short-term Future Additions of Renewable Energy to Worldwide Capacity. In Proceedings of the 2020 International Conference on Data Analytics for Business and Industry: Way Towards a Sustainable Economy (ICDABI), Sakheer, Bahrain, 26–27 October 2020; pp. 1–6. [Google Scholar] [CrossRef]
- Jang, C.K.; Lee, J.; Yi, O. Encryption scheme in portable electric vehicle charging infrastructure: Encryption scheme using symmetric key. In Proceedings of the 4th International Conference on Computer Applications and Information Processing Technology (CAIPT), Kuta Bali, Indonesia, 8–10 August 2017. [Google Scholar]
- Abu Al-Haija, Q.; McCurry, C.D.; Zein-Sabatto, S. A Real Time Node Connectivity Algorithm for Synchronous Cyber Physical and IoT Network Systems. In Proceedings of the 2020 SoutheastCon, Raleigh, NC, USA, 12–15 March 2020; pp. 1–8. [Google Scholar] [CrossRef]
- Fraiji, Y.; Azzouz, L.B.; Trojet, W.; Saidane, L.A. Cyber security issues of Internet of electric vehicles. In Proceedings of the IEEE Wireless Communications and Networking Conference (WCNC), Barcelona, Spain, 15–18 April 2018. [Google Scholar]
- Al-Haija, Q.A. On the Security of Cyber-Physical Systems Against Stochastic Cyber-Attacks Models. In Proceedings of the 2021 IEEE International IOT, Electronics and Mechatronics Conference (IEMTRONICS), Toronto, ON, Canada, 21–24 April 2021; pp. 1–6. [Google Scholar] [CrossRef]
- Yang, B.; Guo, L.; Ye, J. Real-time Simulation of Electric Vehicle Powertrain: Hardware-in-the-Loop (H.I.L.) Testbed for Cyber-Physical Security. In Proceedings of the IEEE Transportation Electrification Conference & Expo (ITEC), Chicago, IL, USA, 22–26 June 2020. [Google Scholar]
- Liu, S.; Huang, Y.; Zhang, R. On-Road Vehicle Recognition Using the Symmetry Property and Snake Models. Int. J. Adv. Robot. Syst. 2013, 10, 407. [Google Scholar] [CrossRef]
- Cui, J.; Liew, L.S.; Sabaliauskaite, G.; Zhou, F. A review on safety failures, security attacks, and available countermeasures for autonomous vehicles. Ad Hoc Netw. 2019, 90, 101823. [Google Scholar] [CrossRef]
- J3016_202104; Taxonomy and Definitions for Terms Related to Driving Automation Systems for On-Road Motor Vehicles. SAE International: Warrendale, PA, USA, 2021.
- Hartenstein, H.; Laberteaux, K. A tutorial survey on vehicular ad hoc networks. IEEE Commun. Mag. 2008, 46, 164–171. [Google Scholar] [CrossRef]
- Loukas, G. Cyber-Physical Attacks A Growing Invisible Threat; Elsevier: Amsterdam, The Netherlands, 2015. [Google Scholar]
- Musleh, A.S.; Chen, G.; Dong, Z.Y. A Survey on the Detection Algorithms for False Data Injection Attacks in Smart Grids. IEEE Trans. Smart Grid 2020, 11, 2218–2234. [Google Scholar] [CrossRef]
- Al-Haija, Q.A.; Alsulami, A.A. High Performance Classification Model to Identify Ransomware Payments for Heterogeneous Bitcoin Networks. Electronics 2021, 10, 2113. [Google Scholar] [CrossRef]
- Staddon, E.; Loscri, V.; Mitton, N. Attack Categorisation for IoT Applications in Critical Infrastructures, a Survey. Appl. Sci. 2021, 11, 7228. [Google Scholar] [CrossRef]
- Krotofil, M.; Cárdenas, A.A.; Manning, B.; Larsen, J. CPS: Driving cyber-physical systems to unsafe operating conditions by timing DoS attacks on sensor signals. In Proceedings of the 30th Annual Computer Security Applications Conference, New Orleans, LA, USA, 8–12 December 2014. [Google Scholar]
- Giraldo, J.; Urbina, D.; Cardenas, A.; Valente, J.; Faisal, M.; Ruths, J.; Tippenhauer, N.O.; Sandberg, H.; Candell, R. A survey of physics-based attack detection in cyber-physical systems. ACM Comput. Surv. 2019, 51, 1–36. [Google Scholar] [CrossRef]
- van Oorschot, P.C. Information Security and Cryptography book series (I.S.C.). In Intrusion Detection and Net-Work-Based Attacks; Springer: Cham, Switzerland, 2021; pp. 309–338. [Google Scholar]
- Abu Al-Haija, Q. Top-Down Machine Learning-Based Architecture for Cyberattacks Identification and Classification in IoT Communication Networks. Front. Big Data 2022, 4. [Google Scholar] [CrossRef]
- MathWorks. Adaptive Cruise Control System Using Model Predictive Control, MathWorks. 2017. Available online: https://www.mathworks.com/help/mpc/ug/adaptive-cruise-control-using-model-predictive-controller.html (accessed on 5 June 2022).
- Sheehan, B.; Murphy, F.; Mullins, M.; Ryan, C. Connected and autonomous vehicles: A cyber-risk classification frame-work. Transp. Res. Part A Policy Pract. 2019, 124, 523–536. [Google Scholar] [CrossRef]
- He, Q.; Meng, X.; Qu, R.; Xi, R. Machine Learning-Based Detection for Cyber Security Attacks on Connected and Autonomous Vehicles. Mathematics 2020, 8, 1311. [Google Scholar] [CrossRef]
- Khan, I.A.; Moustafa, N.; Pi, D.; Haider, W.; Li, B.; Jolfaei, A. An Enhanced Multi-Stage Deep Learning Framework for Detecting Malicious Activities From Autonomous Vehicles. IEEE Trans. Intell. Transp. Syst. 2021, 1–10. [Google Scholar] [CrossRef]
- Oucheikh, R.; Fri, M.; Fedouaki, F.; Hain, M. Deep Real-Time Anomaly Detection for Connected Autonomous Vehicles. Procedia Comput. Sci. 2020, 177, 456–461. [Google Scholar] [CrossRef]
- Narasimhamurthy, S.M.; Mehran, B. A Literature Review of Performance Metrics of Automated Driving Systems for On-Road Vehicles. Front. Future Transp. Sec. Connect. Mobil. Autom. 2021, 2, 759125. [Google Scholar] [CrossRef]
- Gupta, R.; Tanwar, S.; Kumar, N.; Tyagi, S. Blockchain-based security attack resilience schemes for autonomous vehicles in industry 4.0: A systematic review. Comput. Electr. Eng. 2020, 86, 106717. [Google Scholar] [CrossRef]
- Radanliev, P.; De Roure, D.; Walton, R.; Van Kleek, M.; Montalvo, R.M.; Maddox, L.; Santos, O.; Burnap, P.; Anthi, E. Artificial intelligence and machine learning in dynamic cyber risk analytics at the edge. SN Appl. Sci. 2020, 2, 1773. [Google Scholar] [CrossRef]
- Al-Haija, Q.A.; Al-Badawi, A. Attack-Aware IoT Network Traffic Routing Leveraging Ensemble Learning. Sensors 2022, 22, 241. [Google Scholar] [CrossRef]
- Basnet, M.; Ali, M.H. Deep Learning-based Intrusion Detection System for Electric Vehicle Charging Station. In Proceedings of the 2nd International Conference on Smart Power & Internet Energy Systems (SPIES), Bangkok, Thailand, 15–18 September 2020. [Google Scholar]
- Balouji, E.; Gu, I.Y.; Bollen, M.H.; Bagheri, A.; Nazari, M. A LSTM-based deep learning method with application to voltage dip classification. In Proceedings of the 2018 18th International Conference on Harmonics and Quality of Power (ICHQP), Ljubljana, Slovenia, 13–16 May 2018; pp. 1–5. [Google Scholar] [CrossRef]
- Hochreiter, S.; Schmidhuber, J. Long short-term memory. Neural Comput. 1997, 9, 1735–1780. [Google Scholar] [CrossRef]
- Khafaga, D.S.; Alhussan, A.A.; El-Kenawy, E.-S.M.; Ibrahim, A.; Elkhalik, S.H.A.; El-Mashad, S.Y.; Abdelhamid, A.A. Improved Prediction of Metamaterial Antenna Bandwidth Using Adaptive Optimization of LSTM. Comput. Mater. Contin. 2022, 73, 865–881. [Google Scholar] [CrossRef]
- Mahajan, S.; HariKrishnan, R.; Kotecha, K. Prediction of Network Traffic in Wireless Mesh Networks Using Hybrid Deep Learning Model. IEEE Access 2022, 10, 7003–7015. [Google Scholar] [CrossRef]
- Al-Haija, Q.A. A machine learning based predictive model for time-series modelling and analysis. Int. J. Spatio-Temporal Data Sci. 2021, 1, 270–283. [Google Scholar] [CrossRef]
- Mathworks. Long Short-Term Memory Networks. Available online: https://www.mathworks.com/help/deeplearning/ug/long-short-term-memory-networks.html (accessed on 2 July 2022).
- Kingma, D.P.; Ba, J. Adam: A Method for Stochastic Optimization. In Proceedings of the 3rd International Conference for Learning Representations, San Diego, CA, USA, 7–9 May 2015. [Google Scholar]
- Tahir, S.B.U.D.; Jalal, A.; Kim, K. Wearable Inertial Sensors for Daily Activity Analysis Based on Adam Optimization and the Maximum Entropy Markov Model. Entropy 2020, 22, 579. [Google Scholar] [CrossRef]
- Kohli, H.; Agarwal, J.; Kumar, M. An Improved Method for Text Detection using Adam Optimization Algorithm. Glob. Transit. Proc. 2022, 3, 230–234. [Google Scholar] [CrossRef]
- Abu Al-Haija, Q.; Smadi, A.A.; Allehyani, M.F. Meticulously Intelligent Identification System for Smart Grid Network Stability to Optimize Risk Management. Energies 2021, 14, 6935. [Google Scholar] [CrossRef]
- Jais, I.K.M.; Ismail, A.R.; Nisa, S.Q. Adam Optimization Algorithm for Wide and Deep Neural Network. Knowl. Eng. Data Sci. 2019, 2, 41–46. [Google Scholar] [CrossRef]
- Al-Haija, Q.A.; Al-Saraireh, J. Asymmetric Identification Model for Human-Robot Contacts via Supervised Learning. Symmetry 2022, 14, 591. [Google Scholar] [CrossRef]
- Chicco, D.; Jurman, G. The advantages of the Matthews correlation coefficient (MCC) over F1 score and accuracy in binary classification evaluation. BMC Genom. 2020, 21, 6. [Google Scholar] [CrossRef] [Green Version]
- Belkin, M.; Hsu, D.; Ma, S.; Mandal, S. Reconciling modern machine-learning practice and the classical bias–variance trade-off. Proc. Natl. Acad. Sci. USA 2019, 116, 15849–15854. [Google Scholar] [CrossRef] [Green Version]
- Fails, J.A.; Olsen, D.R., Jr. Interactive machine learning. In Proceedings of the 8th International Conference on Intelligent User Interfaces, Miami, FL, USA, 12–15 January 2003. [Google Scholar]
- Hamza, M.A.; Hassine, S.B.H.; Larabi-Marie-Sainte, S.; Nour, M.K.; Al-Wesabi, F.N.; Motwakel, A.; Hilal, A.M.; Al Duhayyim, M. Optimal Bidirectional LSTM for Modulation Signal Classification in Communication Systems. Comput. Mater. Contin. 2022, 72, 3055–3071. [Google Scholar] [CrossRef]
- Almasoud, A.S.; Eisa, T.A.E.; Al-Wesabi, F.N.; Elsafi, A.; Al Duhayyim, M.; Yaseen, I.; Hamza, M.A.; Motwakel, A. Parkinson’s Detection Using RNN-Graph-LSTM with Optimization Based on Speech Signals. Comput. Mater. Contin. 2022, 72, 872–886. [Google Scholar] [CrossRef]
- Roh, H.; Oh, S.; Song, H.; Han, J.; Lim, S. Deep Learning-based Wireless Signal Classification in the IoT Environment. Comput. Mater. Contin. 2022, 71, 5717–5732. [Google Scholar] [CrossRef]
- Sarwar, A.; Hasan, S.; Khan, W.U.; Ahmed, S.; Marwat, S.N.K. Design of an Advance Intrusion Detection System for IoT Networks. In Proceedings of the 2022 2nd International Conference on Artificial Intelligence (ICAI), Islamabad, Pakistan, 30–31 March 2022. [Google Scholar]
- Song, Y.; Hyun, S.; Cheong, Y.G. Analysis of Autoencoders for Network Intrusion Detection. Sensors 2021, 21, 4294. [Google Scholar] [CrossRef]
- Alkahtani, H.; Aldhyani, T.H. Intrusion Detection System to Advance Internet. Complexity 2021, 2021, 5579851. [Google Scholar] [CrossRef]
- Al-Haija, Q.A.; Zein-Sabatto, S. An Efficient Deep-Learning-Based Detection and Classification System for Cyber-Attacks in IoT Communication Networks. Electronics 2020, 9, 2152. [Google Scholar] [CrossRef]
Feature No. | Feature Name | Unit | Data Type |
---|---|---|---|
1 | Actual position of the ego car | m | Double |
2 | Actual velocity of the ego car | m/s | Double |
3 | Actual position of the lead car | m | Double |
4 | Actual velocity of the lead car | m/s | Double |
5 | Anomaly detection label | Normal, Anomaly | Binary |
Accuracy Parameter | Value |
---|---|
Precision | 99.93% |
Recall | 99.97% |
F1-Score | 99.95% |
Accuracy | 99.95% |
Research | Task | No. of Features | ML Model | Accuracy |
---|---|---|---|---|
Hamza et al. [45] | Detection | NA | COSBO-BiLSTM | 98.81% |
Almasoud et al. [46] | Detection | 24 | RNN-GLSTM | 96.7% |
Roh et al. [47] | Detection | 64 | CNN-LSTM | 92.03% |
Sarwar et al. [48] | Detection | 83 | Random Forest | 83% |
Song et al. [49] | Classification | 77 | Deep-learning | 97.4% |
Alkahtani et al. [50] | Classification | 80 | CNN-LSTM | 98.90 |
Al-Haija et al. [51] | Classification | 43 | CNN | 98.2% |
Ullah et al. [52] | Detection | 83 | SVM | 80% |
Proposed method | Detection | 4 | LSTM | 99.95% |
Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations. |
© 2022 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).
Share and Cite
Alsulami, A.A.; Abu Al-Haija, Q.; Alqahtani, A.; Alsini, R. Symmetrical Simulation Scheme for Anomaly Detection in Autonomous Vehicles Based on LSTM Model. Symmetry 2022, 14, 1450. https://doi.org/10.3390/sym14071450
Alsulami AA, Abu Al-Haija Q, Alqahtani A, Alsini R. Symmetrical Simulation Scheme for Anomaly Detection in Autonomous Vehicles Based on LSTM Model. Symmetry. 2022; 14(7):1450. https://doi.org/10.3390/sym14071450
Chicago/Turabian StyleAlsulami, Abdulaziz A., Qasem Abu Al-Haija, Ali Alqahtani, and Raed Alsini. 2022. "Symmetrical Simulation Scheme for Anomaly Detection in Autonomous Vehicles Based on LSTM Model" Symmetry 14, no. 7: 1450. https://doi.org/10.3390/sym14071450