An Anomaly Detection Method for Oilfield Industrial Control Systems Fine-Tuned Using the Llama3 Model
Abstract
:1. Introduction
2. Related Works
2.1. Industrial Control Device Fingerprint Extraction Methods
2.1.1. Fingerprint Extraction Method for Industrial Control Equipment
2.1.2. Content of ICS Device Fingerprint Extraction
2.2. Anomaly Detection Models for Industrial Control Devices
2.3. The Selection of Large Language Models
3. Methods
3.1. Fingerprint Extraction Method for Industrial Control Devices Based on the Industrial Control Protocol Communication Mode
3.2. Industrial Control Device Anomaly Detection Model Based on Fine-Tuned Llama3 Model
4. Experiments and Results
4.1. Experimental Hardware Configuration and Software Versions
4.2. Description of Experimental Data
4.3. Experimental Results
4.3.1. Analysis of the Effectiveness of DFs
4.3.2. Performance Analysis of the Anomaly Detection Model
4.3.3. Performance Analysis of Different Anomaly Detection Methods
5. Discussion and Conclusions
Supplementary Materials
Author Contributions
Funding
Institutional Review Board Statement
Informed Consent Statement
Data Availability Statement
Conflicts of Interest
Abbreviations
ICS | Industrial Control System |
IIOT | Industrial Internet of Things |
TTL | Time to Live |
IOT | Internet of Things |
CPU | Central Processing Unit |
CLRT | Cross-Layer Response Time |
TCP | Transmission Control Protocol |
IP | Internet Protocol |
SVM | Support Vector Machine |
DCNN | Deep Convolutional Neural Network |
DBHC | Density-Based Heuristic Clustering |
LLM | Large Language Model |
MMLU | Massive Multitask Language Understanding |
ICSDF | Industrial Control System Device Fingerprint |
GPU | Graphics Processing Unit |
PLC | Programmable Logic Controller |
SCADA | Supervisory Control and Data Acquisition |
HMI | Human–Machine Interface |
WAP | Wireless Access Point |
DDOS | Distributed Denial of Service |
References
- Alladi, T.; Chamola, V.; Zeadally, S. Industrial Control Systems: Cyberattack Trends and Countermeasures. Comput. Commun. 2020, 155, 1–8. [Google Scholar] [CrossRef]
- Yadav, G.; Paul, K. Architecture and Security of SCADA Systems: A Review. Int. J. Crit. Infrastruct. Prot. 2021, 34, 100433. [Google Scholar] [CrossRef]
- Suhaimy, N.; Radzi, N.A.M.; Ahmad, W.S.H.M.W.; Azmi, K.H.M.; Hannan, M.A. Current and Future Communication Solutions for Smart Grids: A Review. IEEE Access 2022, 10, 43639–43668. [Google Scholar] [CrossRef]
- Abosata, N.; Al-Rubaye, S.; Inalhan, G.; Emmanouilidis, C. Internet of Things for System Integrity: A Comprehensive Survey on Security, Attacks and Countermeasures for Industrial Applications. Sensors 2021, 21, 3654. [Google Scholar] [CrossRef]
- Hosseini, S.; Azizi, M. The Hybrid Technique for DDoS Detection with Supervised Learning Algorithms. Comput. Netw. 2019, 158, 35–45. [Google Scholar] [CrossRef]
- Khan, A.Y.; Latif, R.; Latif, S.; Tahir, S.; Batool, G.; Saba, T. Malicious Insider Attack Detection in IoTs Using Data Analytics. IEEE Access 2020, 8, 11743–11753. [Google Scholar] [CrossRef]
- Lima, P.M.; Alves, M.V.S.; Carvalho, L.K.; Moreira, M.V. Security Against Communication Network Attacks of Cyber-Physical Systems. J. Control. Autom. Electr. Syst. 2019, 30, 125–135. [Google Scholar] [CrossRef]
- Guan, Y.; Ge, X. Distributed Attack Detection and Secure Estimation of Networked Cyber-Physical Systems Against False Data Injection Attacks and Jamming Attacks. IEEE Trans. Signal Inf. Process. Netw. 2017, 4, 48–59. [Google Scholar] [CrossRef]
- Tan, S.F.; Samsudin, A. Recent Technologies, Security Countermeasure and Ongoing Challenges of Industrial Internet of Things (IIoT): A survey. Sensors 2021, 21, 6647. [Google Scholar] [CrossRef]
- Mitchell, R.; Chen, I.-R. A Survey of Intrusion Detection in Wireless Network Applications. Comput. Commun. 2014, 42, 1–23. [Google Scholar] [CrossRef]
- Radhakrishnan, S.V.; Uluagac, A.S.; Beyah, R. GTID: A Technique for Physical Device and Device Type Fingerprinting. IEEE Trans. Dependable Secur. Comput. 2015, 12, 519–532. [Google Scholar] [CrossRef]
- Shen, C.; Chao, C.; Tan, H.; Wang, Z.; Xu, D.; Su, X. Hybrid-Augmented Device Fingerprinting for Intrusion Detection in Industrial Control System Networks. IEEE Wirel. Commun. 2018, 25, 26–31. [Google Scholar] [CrossRef]
- Aboah Boateng, E.; Aboah Boateng, J.W. Unsupervised Machine Learning Techniques for Detecting PLC Process Control Anomalies. J. Cybersecur. Priv. 2022, 2, 220–244. [Google Scholar] [CrossRef]
- Yang, W.; Fang, Y.; Zhou, X.; Shen, Y.; Zhang, W.; Yao, Y. Networked Industrial Control Device Asset Identification Method Based on Improved Decision Tree. J. Netw. Syst. Manag. 2024, 32, 32. [Google Scholar] [CrossRef]
- Bela, G.; Enachescu, C. ShoVAT: Shodan-based Vulnerability Assessment Tool for Internet-facing Services. Secur. Commun. Netw. 2016, 9, 2696–2714. [Google Scholar]
- Yu, X.; Hu, Z.; Xin, Y. A New Approach Customizable Distributed Network Service Discovery System. Wirel. Commun. Mob. Comput. 2021, 2021, 6627639. [Google Scholar] [CrossRef]
- Li, Q.; Feng, X.; Wang, H.; Sun, L. Understanding the Usage of Industrial Control System Devices on the Internet. IEEE Internet Things J. 2018, 5, 2178–2189. [Google Scholar] [CrossRef]
- Bezawada, B.; Bachani, M.; Peterson, J.; Shirazi, H.; Ray, I. Behavioral Fingerprinting of Internet-of-Things Devices. Wiley Interdiscip. Rev. Data Min. Knowl. Discov. 2021, 11, e1337. [Google Scholar] [CrossRef]
- Skowron, M.; Janicki, A.; Mazurczyk, W. Traffic Fingerprinting Attacks on Internet of Things Using Machine Learning. IEEE Access 2020, 8, 20386–20400. [Google Scholar] [CrossRef]
- Ponomarev, S.; Atkison, T. Industrial Control System Network Intrusion Detection by Telemetry Analysis. IEEE Trans. Dependable Secur. Comput. 2015, 13, 252–260. [Google Scholar] [CrossRef]
- Fan, Z.; Wang, Q.; Jiao, H.; Liu, J.; Cui, Z.; Liu, S.; Liu, Y. PUMD: A PU Learning-Based Malicious Domain Detection Framework. Cybersecurity 2022, 5, 19. [Google Scholar] [CrossRef]
- Gao, W.; Morris, T. On Cyber Attacks and Signature Based Intrusion Detection for Modbus Based Industrial Control Systems. J. Digit. Forensics Secur. 2014, 9, 3. [Google Scholar] [CrossRef]
- Khan, I.A.; Pi, D.; Khan, Z.U.; Hussain, Y.; Nawaz, A. HML-IDS: A Hybrid-Multilevel Anomaly Prediction Approach for Intrusion Detection in SCADA Systems. IEEE Access 2019, 7, 89507–89521. [Google Scholar] [CrossRef]
- Aneja, S.; Aneja, N.; Bhargava, B.; Chowdhury, R.R. Device Fingerprinting Using Deep Convolutional Neural Networks. Int. J. Commun. Netw. Distrib. Syst. 2022, 28, 171–198. [Google Scholar] [CrossRef]
- Koball, C.; Rimal, B.P.; Wang, Y.; Salmen, T.; Ford, C. IoT Device Identification Using Unsupervised Machine Learning. Information 2023, 14, 320. [Google Scholar] [CrossRef]
- Hao, Q.; Rong, Z. IoTTFID: An Incremental IoT Device Identification Model Based on Traffic Fingerprint. IEEE Access 2023, 11, 58679–58691. [Google Scholar] [CrossRef]
- Sheng, C.; Yao, Y.; Li, W.; Yang, W.; Liu, Y. Unknown Attack Traffic Classification in SCADA Network Using Heuristic Clustering Technique. IEEE Trans. Netw. Serv. Manag. 2023, 20, 2625–2638. [Google Scholar] [CrossRef]
- Shan, Y.; Yao, Y.; Zhao, T.; Yang, W. NeuPot: A Neural Network-Based Honeypot for Detecting Cyber Threats in Industrial Control Systems. IEEE Trans. Ind. Inform. 2023, 19, 10512–10522. [Google Scholar] [CrossRef]
- Abdin, M.; Aneja, J.; Awadalla, H.; Awadallah, A.; Awan, A.A.; Bach, N.; Bahree, A.; Bakhtiari, A.; Bao, J.; Behl, H.; et al. Phi-3 Technical Report: A Highly Capable Language Model Locally on Your Phone. arXiv 2024, arXiv:2404.14219. [Google Scholar]
Fingerprint Feature Identifier | Fingerprint Feature Description |
---|---|
indicates the initial value of the TTL field in the IP protocol header of packet | |
indicates the value of the DF flag in the IP protocol header of packet | |
indicates the increment value of the ID field in the IP protocol header between packet and packet | |
indicates the value of the window size field in the TCP protocol header of packet | |
indicates the initial value of the TTL field in the IP protocol header of packet | |
indicates the value of the window scale field in the TCP protocol header of packet | |
indicates the value of the SACK permitted field in the TCP protocol header of packet | |
indicates the time interval between packet and packet | |
indicates whether packet includes the TCP timestamp option | |
indicates whether the TSecr field value in the TCP timestamp option of packet matches the TSval field value in the TCP timestamp option of packet | |
indicates the update frequency of the TSval field value in the TCP timestamp option on the Server side | |
indicates the difference between the time interval of packet and packet and the time interval of packet and packet |
Others | Vendor | Device Type | |||||||
---|---|---|---|---|---|---|---|---|---|
64 | 25,085 | 4096 | 1456 | 0 | 0 | −0.037305 | … | Rockwell Automation/ Allen-Bradley (Milwaukee, Wisconsin, United States.) | Communication Adapter |
128 | 1 | 2000 | 1418 | −1 | 0 | 0.013798 | … | Rockwell Automation/Allen-Bradley | Programmable Logic Controller |
32 | 1 | 6144 | 1024 | −1 | 0 | 0.000798 | … | Omron Corporation (Kyoto, Japan.) | |
64 | 0 | 10,000 | 1460 | 0 | 0.00014 | 0.000043 | … | swat | |
64 | 2 | 4096 | 1450 | 0 | 0 | 0.000480 | … | honeypot |
Model | CPU | Memory | GPU | Hard Drive |
---|---|---|---|---|
X3650M5 | 2×E5-2620v4 (8 C, 85 W, 2.1 GHz) | 4 × 16 = 64 GB DDR4 | GPU: NVIDIA GeForce RTX 2080 SUPER. Max memory: 7.787 G | 8 × 2.5″ slots, 2 × 2 TB SATA interface, M5210 Raid 0,1 |
End Device | Communication Mode | IP Address |
---|---|---|
SCADA | wired | 192.168.1.100 |
wireless | 192.168.1.102 | |
PLC1 | wired | 192.168.1.10 |
wireless | 192.168.1.102 | |
PLC2 | wired | 192.168.1.20 |
wireless | 192.168.1.22 | |
PLC3 | wired | 192.168.1.30 |
wireless | 192.168.1.32 | |
PLC4 | wired | 192.168.1.40 |
wireless | 192.168.1.42 | |
PLC5 | wired | 192.168.1.50 |
wireless | 192.168.1.52 | |
PLC6 | wired | 192.168.1.60 |
wireless | 192.168.1.62 | |
Engineering Station | wired | 192.168.1.201 |
HMI | wired | 192.168.1.202 |
Historian | wired | 192.168.1.200 |
Number of Samples | Number of Training Epochs | Fine-Tuned Model Accuracy | The Number of Training Epochs Derived from the Function |
---|---|---|---|
1200 | 68 | 84.1% | 78 |
73 | 87.1% | ||
78 | 98.7% | ||
83 | 99.9% | ||
88 | 99.8% | ||
2000 | 73 | 87.9% | 84 |
78 | 94.8% | ||
83 | 96.5% | ||
88 | 99.8% | ||
93 | 99.8% | ||
2800 | 78 | 91.1% | 88 |
83 | 95.1% | ||
88 | 97.7% | ||
93 | 99.7% | ||
98 | 99.7% | ||
3600 | 83 | 99.3% | 92 |
88 | 99.4% | ||
93 | 99.2% | ||
98 | 99.4% | ||
103 | 99.5% | ||
4400 | 88 | 99.0% | 95 |
93 | 99.9% | ||
98 | 99.8% | ||
103 | 99.9% | ||
108 | 99.9% | ||
5200 | 93 | 94.3% | 98 |
98 | 95.8% | ||
103 | 97.4% | ||
108 | 98.2% | ||
113 | 98.7% | ||
6039 | 98 | 93.8% | 100 |
103 | 98.0% | ||
108 | 99.0% | ||
113 | 99.2% | ||
118 | 99.8% |
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content. |
© 2024 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).
Share and Cite
Zhao, J.; Jin, Z.; Zeng, P.; Sheng, C.; Wang, T. An Anomaly Detection Method for Oilfield Industrial Control Systems Fine-Tuned Using the Llama3 Model. Appl. Sci. 2024, 14, 9169. https://doi.org/10.3390/app14209169
Zhao J, Jin Z, Zeng P, Sheng C, Wang T. An Anomaly Detection Method for Oilfield Industrial Control Systems Fine-Tuned Using the Llama3 Model. Applied Sciences. 2024; 14(20):9169. https://doi.org/10.3390/app14209169
Chicago/Turabian StyleZhao, Jianming, Ziwen Jin, Peng Zeng, Chuan Sheng, and Tianyu Wang. 2024. "An Anomaly Detection Method for Oilfield Industrial Control Systems Fine-Tuned Using the Llama3 Model" Applied Sciences 14, no. 20: 9169. https://doi.org/10.3390/app14209169
APA StyleZhao, J., Jin, Z., Zeng, P., Sheng, C., & Wang, T. (2024). An Anomaly Detection Method for Oilfield Industrial Control Systems Fine-Tuned Using the Llama3 Model. Applied Sciences, 14(20), 9169. https://doi.org/10.3390/app14209169