Skip to Content
You are currently on the new version of our website. Access the old version .
MathematicsMathematics
  • Article
  • Open Access

30 June 2023

16 Pages

Similarity-Based Hybrid Malware Detection Model Using API Calls

,
,
,
,
,
,
and
1
Department of Computer Science, Northern Border University, Arar 9280, Saudi Arabia
2
Department of Computer Sciences, Faculty of Computing and Information Technology, Northern Border University, Rafha 91911, Saudi Arabia
3
Department of Educational Technologies, Imam Abdulrahman Bin Faisal University, Dammam 34212, Saudi Arabia
4
School of Computing, University Teknologi Malaysia, UTM, Johor Bahru 81310, Johor, Malaysia
This article belongs to the Special Issue Models and Algorithms in Cybersecurity

Abstract

This study presents a novel Similarity-Based Hybrid API Malware Detection Model (HAPI-MDM) aiming to enhance the accuracy of malware detection by leveraging the combined strengths of static and dynamic analysis of API calls. Faced with the pervasive challenge of obfuscation techniques used by malware authors, the conventional detection models often struggle to maintain robust performance. Our proposed model addresses this issue by deploying a two-stage learning approach where the XGBoost algorithm acts as a feature extractor feeding into an Artificial Neural Network (ANN). The key innovation of HAPI-MDM is the similarity-based feature, which further enhances the detection accuracy of the dynamic analysis, ensuring reliable detection even in the presence of obfuscation. The model was evaluated using seven machine learning techniques with 10 K-fold cross-validation. Experimental results demonstrated HAPI-MDM’s superior performance, achieving an overall accuracy of 97.91% and the lowest false-positive and false-negative rates compared to related works. The findings suggest that integrating dynamic and static API-based features and utilizing a similarity-based feature significantly improves malware detection performance, thereby offering an effective tool to fortify cybersecurity measures against escalating malware threats.

Article Metrics

Citations

Article Access Statistics

Multiple requests from the same IP address are counted as one view.