Next Article in Journal
Tracking Control for Quad-Rotor Using Velocity Field and Obstacle Avoidance Based on Hydrodynamics
Next Article in Special Issue
First Order and Second Order Learning Algorithms on the Special Orthogonal Group to Compute the SVD of Data Matrices
Previous Article in Journal
Comparative Evaluation of Wide-Range Soft-Switching PWM Full-Bridge Modular Multilevel DC–DC Converters
Previous Article in Special Issue
Individual Behavior Modeling with Sensors Using Process Mining
Open AccessArticle

System Log Detection Model Based on Conformal Prediction

1
College of Computer Science and Technology, Civil Aviation University of China, 300300 Tianjin, China
2
Information Security Evaluation Center, Civil Aviation University of China, 300300 Tianjin, China
3
College of Cyber Science, Nankai University, 300071 Tianjin, China
4
Cyberspace Institute of Advanced Technology, Guangzhou University, 510006 Guangzhou, China
5
Department of Computer and Information Sciences, Temple University, Philadelphia, PA 19122, USA
6
Computer Science and Engineering Department, Qatar University, Doha 2713, Qatar
*
Authors to whom correspondence should be addressed.
Electronics 2020, 9(2), 232; https://doi.org/10.3390/electronics9020232
Received: 1 January 2020 / Revised: 26 January 2020 / Accepted: 27 January 2020 / Published: 31 January 2020
(This article belongs to the Special Issue Recent Machine Learning Applications to Internet of Things (IoT))
With the rapid development of the Internet of Things, the combination of the Internet of Things with machine learning, Hadoop and other fields are current development trends. Hadoop Distributed File System (HDFS) is one of the core components of Hadoop, which is used to process files that are divided into data blocks distributed in the cluster. Once the distributed log data are abnormal, it will cause serious losses. When using machine learning algorithms for system log anomaly detection, the output of threshold-based classification models are only normal or abnormal simple predictions. This paper used the statistical learning method of conformity measure to calculate the similarity between test data and past experience. Compared with detection methods based on static threshold, the statistical learning method of the conformity measure can dynamically adapt to the changing log data. By adjusting the maximum fault tolerance, a system administrator can better manage and monitor the system logs. In addition, the computational efficiency of the statistical learning method for conformity measurement was improved. This paper implemented an intranet anomaly detection model based on log analysis, and conducted trial detection on HDFS data sets quickly and efficiently. View Full-Text
Keywords: HDFS; anomaly detection; conformal prediction; confusion matrix HDFS; anomaly detection; conformal prediction; confusion matrix
Show Figures

Figure 1

MDPI and ACS Style

Ren, Y.; Gu, Z.; Wang, Z.; Tian, Z.; Liu, C.; Lu, H.; Du, X.; Guizani, M. System Log Detection Model Based on Conformal Prediction. Electronics 2020, 9, 232. https://doi.org/10.3390/electronics9020232

AMA Style

Ren Y, Gu Z, Wang Z, Tian Z, Liu C, Lu H, Du X, Guizani M. System Log Detection Model Based on Conformal Prediction. Electronics. 2020; 9(2):232. https://doi.org/10.3390/electronics9020232

Chicago/Turabian Style

Ren, Yitong; Gu, Zhaojun; Wang, Zhi; Tian, Zhihong; Liu, Chunbo; Lu, Hui; Du, Xiaojiang; Guizani, Mohsen. 2020. "System Log Detection Model Based on Conformal Prediction" Electronics 9, no. 2: 232. https://doi.org/10.3390/electronics9020232

Find Other Styles
Note that from the first issue of 2016, MDPI journals use article numbers instead of page numbers. See further details here.

Article Access Map by Country/Region

1
Search more from Scilit
 
Search
Back to TopTop