Modern Approaches to Software Vulnerability Detection: A Survey of Machine Learning, Deep Learning, and Large Language Models

Abstract

Software vulnerabilities pose significant risks to the security and reliability of modern systems, making automated vulnerability detection an essential research area. Traditional static and rule-based approaches are limited in scalability and adaptability, motivating the adoption of data-driven methods. In this survey, we present a comprehensive review of Machine Learning (ML), Deep Learning (DL), and Large Language Models (LLMs) techniques for vulnerability detection. We analyze recent advances in feature representation, fine-tuning strategies, generative approaches, and prompt engineering, while highlighting their ability to capture both syntactic and semantic properties of source code. Furthermore, we examine commonly used evaluation metrics and provide a critical discussion of key challenges, including the lack of large-scale real-world datasets, limited vulnerability coverage, class imbalance, interpretability gaps, hallucination, and high computational costs. To address these issues, we outline promising future research directions, such as neuro-symbolic hybrid methods, parameter-efficient fine-tuning, continual learning, cross-language generalization, and explainable AI for vulnerability detection. Unlike previous studies, the present work explores learning paradigms from ML to LLMs using comprehensive evaluation criteria that highlight analytical capability, feature interpretability, and code-context comprehension. By combining these factors, our study addresses the methodological gap between classic feature-based approaches and current LLM-driven reasoning frameworks, providing beneficial insights to develop robust, scalable, and trustworthy software vulnerability detection systems.

1. Introduction

Modern computer systems serve as the core for current technological infrastructure, ensuring both customers and businesses interact and operate seamlessly. However, exploitable software vulnerabilities remain a continuous danger to system security and user safety. High-profile incidents such as the “Heartbleed” [1] and “Shellshock” [2] vulnerabilities, along with the 2017 Apache Struts breach that exposed confidential financial data of 143 million users demonstrate the widespread effect and significant nature of those vulnerabilities [3,4,5,6]. Despite the development of several detection techniques, these cases happened mostly due to previous approaches’ failure to discover advanced vulnerabilities early of their exploitation. This underlines a crucial research gap: the need for more adaptable and intelligent systems that can reason about semantic and contextual connections inside source code, rather than depending simply on rule-based or pattern-matching tactics. A preventative and effective technique for preventing attacks includes identifying software vulnerabilities during deployment. As shown in Figure 1, software vulnerability analysis techniques can be broadly classified into three main categories: static, dynamic, and hybrid approaches, where static vulnerability detection approaches evaluate the program without executing it, depending heavily on the source code. Common approaches include rule- or template-based analysis [7,8,9], code similarity identification [10,11], and symbolic execution [12,13,14]. These methods are capable of covering large code sections and detecting possible faults early in the development process. However, they have been reproved for developing a high proportion of false positives, which can lead to extra human intervention in validating observations [15]. Dynamic analysis approaches, for example fuzz testing [16,17], and taint analysis [18,19], evaluation software during execution to identify flaws that static methods might overlook. Although dynamic techniques are more effective at detecting runtime behaviors and vulnerabilities that are dependent on particular execution environments, they sometimes have limited code coverage and can fail to identify hidden defects in seldom performed routes. To address the limitations inherent in both static and dynamic methods, hybrid approaches have been developed. These combine static code analysis with runtime monitoring and testing to leverage the strengths of both paradigms. The integration aims to improve detection accuracy, reduce false positives, and enhance code coverage, thereby offering a more comprehensive solution for software vulnerability analysis.

Recent advancements in data-driven vulnerability discovery have introduced novel approaches that go beyond traditional static and dynamic analysis techniques. Over the past decade, vulnerability detection research has advanced from traditional Machine Learning (ML) models, which rely on manual feature extraction, to Deep Learning (DL) architectures capable of automatically learning hierarchical code representation. As the size and complexity of software increased, DL-based approaches became more successful at capturing contextual dependencies inside source code. Building on this achievement, the introduction of Large Language Models (LLMs) is a logical next step, since these models expand DL capabilities by including large-scale pretraining, code semantics comprehension, and reasoning skills. By leveraging pattern recognition and Artificial Intelligence (AI) methods [20,21,22,23,24,25,26,27,28,29,30,31,32,33,34,35,36,37], researchers have been able to automate vulnerability detection processes, improving both scalability and efficiency. Unlike previous strategies, which depended heavily on handcrafted rules and shallow heuristics, these innovative methods allow systems to learn directly from vast datasets of susceptible and non-vulnerable code. As a result, ML, DL, and LLM-based frameworks provide formidable options for automatic, accurate, and economically viable software vulnerability detection. Furthermore, the development of neuro-symbolic methods and contrastive learning strategies shows an evolution toward hybrid systems that integrate statistical learning with logical reasoning, lowering false positives and improving interpretability. These advancements represent a fundamental shift in vulnerability detection research, with an emphasis not just on discovering defects but also on understanding vulnerability patterns, root causes, and mitigation measures. As a result, the use of ML, DL, and LLM-based methodologies represents a significant move toward more robust, proactive, and intelligent security assessment solutions for current software environments. Therefore, this study provides a comprehensive review of recent models and approaches for software vulnerability detection, with a particular focus on the patterns in source code that these models exploit. In addition to analyzing conventional machine learning–based techniques, the review also covers deep learning architectures and large language model-driven frameworks that are increasingly being adopted to capture both syntactic and semantic vulnerability indicators.

2. Related Reviews

This section discusses currently available research on software vulnerability detection. Several strategies have been proposed and intensively researched in the field of software engineering for discovering shortcomings and vulnerabilities in source code. Early research concentrated mostly on static and dynamic analysis techniques that employed manually generated rules, symbolic execution, or testing frameworks to detect security issues. Over time, the research community has shifted toward more data-driven methods, incorporating pattern recognition, machine learning, and deep learning models to automate vulnerability discovery. Shahriar et al. [24] explored several code analysis techniques, classifying current methodologies according to their methodological focus and detection scope. Malhotra et al. [25] performed a study of software defect prediction approaches that used ML techniques, highlighting their benefits and limitations in detecting fault-prone code sections. In a more comprehensive assessment, Liu et al. [26] provided a broad overview of vulnerability detection research, focusing on works that used both traditional code analysis and ML-based algorithms. Ghaffarian et al. [21] conducted an in-depth review of vulnerability detection utilizing classical machine learning approaches, highlighting features based on software metrics, expert-defined patterns, and anomaly detection. Following that, another review [27] provided a succinct overview of related findings. From a different perspective, [28] performed a thorough review that identified parallels between programming languages and natural languages and explained how such comparisons influence model design. More recently, several surveys have concentrated on deep learning–based vulnerability detection. Uddin et al. [29] introduced the “Vulnerability Detection Lifecycle” framework to systematically analyze deep learning models across stages like data construction, code representation, and deployment. Hanif et al. [30] executed a comprehensive literature evaluation of 90 materials published between 2011 and 2020, providing two different taxonomies for classifying vulnerability detection studies. Harzevili et al. recently published a thorough systematic literature analysis, providing wide insights on the use of machine learning for software vulnerability detection across multiple data types including source code, binaries, and commit metadata—over a long period of time (2011–2024) [31]. Zeng et al. [32] executed their survey based on four recent “game changer” endeavors. While their methodology recognizes crucial research areas coming from these “game changers”. Lomio et al. [33] evaluated the effectiveness of an existing ML-based SVD method for commit-level detection. However, their selection of just 9 domains seems ambiguous. Liu et al. [34] conducted a survey that compared and analyzed strategies for identifying XSS vulnerabilities. Chakraborty et al. [35] carried out an empirical investigation to assess the robustness of DL–based vulnerability detection approaches when applied to real-world datasets. Their evaluation revealed a critical limitation: although existing DL-based methods often demonstrate strong performance in controlled or benchmarked environments, their effectiveness tends to deteriorate significantly in real-world scenarios. This performance gap highlights the challenges of generalizability, where models trained on curated or synthetic datasets fail to capture the complexity, diversity, and noise inherent in actual software projects. The study underscores the importance of evaluating vulnerability detection methods not only on benchmark datasets but also across diverse, real-world software repositories, thereby ensuring that proposed techniques remain reliable and scalable in practice. Zheng et al. [36] conducted an experiment to determine the impact of various ML methodologies on vulnerability detection performance. Their findings demonstrate that the attention mechanism improves vulnerability identification by allowing models to capture fine-grained contextual relationships in source code. In contrast, the use of transfer learning did not result in significant gains in model performance. This result implies that while transfer learning has been effective in other domains such as natural language processing (NLP) and computer vision, its benefits are less evident in software vulnerability detection, possibly due to the domain-specific characteristics of programming languages and security flaws. In recent times, Zhao et al. [37] proposed Yama, utilizes PHP opcode principles to perform context and path-sensitive intraprocedural analysis. It incorporates accurate type inference, dynamic language structures, and concrete execution for executing challenging built-in functions. This study demonstrates the importance of opcode-level semantic reasoning in enhancing static vulnerability detection precision in dynamic web languages. In another study. In another, Ji et al. [38] introduced Artemis, integrates GPT-4-based source/sink identification, implicit call graph construction, rule-driven taint propagation, and path-condition-based false positive pruning. However, In

Table 1. Overview of the existing review papers.

Published Paper	Published Date	Contributions
Shahriar et al. [24]	2012	Vulnerability detection experiments employing various code analysis approaches.
Malhotra et al. [25]	2015	Programming defects identification methods using ML strategies,
Liu et al. [26]	2012	A comprehensive summary of finding vulnerabilities analysis.
Ghaffarian et al. [21]	2017	Presented fundamental labeling for work in the software vulnerabilities detection area.
Uddin et al. [29]	2025	Deep learning models with code representation, and implementation.
Lomio et al. [33]	2022	The study explored the extent to which Machine Learning has been adopted as an assistive tool for developers in vulnerability detection. By leveraging statistical models and pattern-recognition algorithms, ML enables automated identification of code anomalies, thereby reducing developers’ reliance on manual inspection and traditional static analysis methods
Hanif et al. [30]	2021	Designed two alternative taxonomies for software vulnerability detection, one representing research interests and the other highlighting methodological methods.
Harzevili et al. [31]	2024	Explored ML-based methods to analyze publication trends, dataset usage, feature representations, and model architectures, while also identifying the most frequently studied vulnerability types.
Zeng et al. [32]	2020	Highlighted and critically analyzed four seminal papers in the field, emphasizing their transformative impact on the direction of software vulnerability detection research.
Liu et al. [34]	2019	Provided a comprehensive discussion on the classification of Cross-Site Scripting (XSS) attacks, highlighting their types and implications.
Chakraborty et al. [35]	2022	Conducted a survey to evaluate the effectiveness of existing deep learning–based vulnerability detection methods when applied to real-world datasets.
Zheng et al. [36]	2021	In their study, the authors examined how various ML strategies affect the performance of source-code vulnerability detection. By comparing multiple approaches including feature-engineering based models, attention mechanisms, and transfer learning—they evaluated the relative contribution of each strategy toward identifying vulnerable code segments.
Zhao et al. [37]	2025	Provided Yama tools for PHP vulnerabilities identification, where experimental findings on 24 real-world applications (10M LOC) demonstrate a 99.1% detection accuracy and the identification of 38 zero-day vulnerabilities, higher than 9 modern static analyzers.
Ji et al. [38]	2025	Evaluated on 250 PHP applications, it achieved a 90.2% reduction in false positives and discovered 35 new SSRF vulnerabilities (24 CVEs). The framework highlights the potential of combining large language models with program analysis for vulnerability detection

, we provide a consolidated overview of the reviewed survey papers, including their publication dates and the primary concepts they addressed.

This section discusses currently available research on software vulnerability detection. Several strategies have been proposed and intensively researched in the field of software engineering for discovering shortcomings and vulnerabilities in source code. Early research concentrated mostly on static and dynamic analysis techniques that employed manually generated rules, symbolic execution, or testing frameworks to detect security issues. Over time, the research community has shifted toward more data-driven methods, incorporating pattern recognition, machine learning, and deep learning models to automate vulnerability discovery. Shahriar et al. [24] explored several code analysis techniques, classifying current methodologies according to their methodological focus and detection scope. Malhotra et al. [25] performed a study of software defect prediction approaches that used ML techniques, highlighting their benefits and limitations in detecting fault-prone code sections. In a more comprehensive assessment, Liu et al. [26] provided a broad overview of vulnerability detection research, focusing on works that used both traditional code analysis and ML-based algorithms. Ghaffarian et al. [21] conducted an in-depth review of vulnerability detection utilizing classical machine learning approaches, highlighting features based on software metrics, expert-defined patterns, and anomaly detection. Following that, another review [27] provided a succinct overview of related findings. From a different perspective, [28] performed a thorough review that identified parallels between programming languages and natural languages and explained how such comparisons influence model design. More recently, several surveys have concentrated on deep learning–based vulnerability detection. Uddin et al. [29] introduced the “Vulnerability Detection Lifecycle” framework to systematically analyze deep learning models across stages like data construction, code representation, and deployment. Hanif et al. [30] executed a comprehensive literature evaluation of 90 materials published between 2011 and 2020, providing two different taxonomies for classifying vulnerability detection studies. Harzevili et al. recently published a thorough systematic literature analysis, providing wide insights on the use of machine learning for software vulnerability detection across multiple data types including source code, binaries, and commit metadata—over a long period of time (2011–2024) [31]. Zeng et al. [32] executed their survey based on four recent “game changer” endeavors. While their methodology recognizes crucial research areas coming from these “game changers”. Lomio et al. [33] evaluated the effectiveness of an existing ML-based SVD method for commit-level detection. However, their selection of just 9 domains seems ambiguous. Liu et al. [34] conducted a survey that compared and analyzed strategies for identifying XSS vulnerabilities. Chakraborty et al. [35] carried out an empirical investigation to assess the robustness of DL–based vulnerability detection approaches when applied to real-world datasets. Their evaluation revealed a critical limitation: although existing DL-based methods often demonstrate strong performance in controlled or benchmarked environments, their effectiveness tends to deteriorate significantly in real-world scenarios. This performance gap highlights the challenges of generalizability, where models trained on curated or synthetic datasets fail to capture the complexity, diversity, and noise inherent in actual software projects. The study underscores the importance of evaluating vulnerability detection methods not only on benchmark datasets but also across diverse, real-world software repositories, thereby ensuring that proposed techniques remain reliable and scalable in practice. Zheng et al. [36] conducted an experiment to determine the impact of various ML methodologies on vulnerability detection performance. Their findings demonstrate that the attention mechanism improves vulnerability identification by allowing models to capture fine-grained contextual relationships in source code. In contrast, the use of transfer learning did not result in significant gains in model performance. This result implies that while transfer learning has been effective in other domains such as natural language processing (NLP) and computer vision, its benefits are less evident in software vulnerability detection, possibly due to the domain-specific characteristics of programming languages and security flaws. In recent times, Zhao et al. [37] proposed Yama, utilizes PHP opcode principles to perform context and path-sensitive intraprocedural analysis. It incorporates accurate type inference, dynamic language structures, and concrete execution for executing challenging built-in functions. This study demonstrates the importance of opcode-level semantic reasoning in enhancing static vulnerability detection precision in dynamic web languages. In another study. In another, Ji et al. [38] introduced Artemis, integrates GPT-4-based source/sink identification, implicit call graph construction, rule-driven taint propagation, and path-condition-based false positive pruning. However, In Table 1, we provide a consolidated overview of the reviewed survey papers, including their publication dates and the primary concepts they addressed.

Previous survey studies have mainly focused on traditional ML- and DL-based methods for vulnerability detection, often providing descriptive overviews without integrating recent developments in LLMs, prompt engineering, or neuro-symbolic reasoning. In contrast, this study contributes a unified and forward-looking synthesis that bridges these paradigms, offering a taxonomy linking datasets, model architectures, and evaluation metrics. Rather than conducting empirical experiments, the emphasis of this survey lies in conceptual integration, trend analysis, and identification of research gaps to guide future empirical and theoretical advancements in automated vulnerability detection.

3. Background

3.1. Methodology

PRISMA-ScR Reporting Statement: This scoping review was conducted following the PRISMA-ScR (Preferred Reporting Items for Systematic Reviews and Meta-Analyses extension for Scoping Reviews) guidelines to ensure transparent and comprehensive reporting of the methodology. The literature selection and screening process, including the identification, exclusion of duplicates, abstract-level screening, and final inclusion of 169 studies, is summarized in Figure 2.

In this work, we employ a methodical review procedure that utilizes findings from three major sources: software vulnerability-focused review papers, experimental research studies, and empirical surveys. Our major goal is to provide future researchers with a thorough grasp of the code patterns, models that have been commonly employed in recent years, and types of vulnerabilities discovered. We begin by considering a variety of review papers which describe historical and present techniques to vulnerability identification. It helps us to observe how older technologies, such as rule-based static analysis and classical ML, have been gradually succeeded or complemented by DL and LLM-based approaches. Recent studies demonstrate a progressive movement of completely human-centered feature engineering and toward data-driven techniques that automatically learn feature representations from source code semantics. Rather than upgrading previous static and rule-based methods, advanced ML, DL, and LLM-based methodologies have been employed as supplemental solutions to improve scalability, automation, and semantic understanding in vulnerability identification. After consolidating findings from these three categories of literature, we synthesize and discuss the challenges and limitations of the existing body of work. These challenges include issues such as dataset imbalance, false positive rates, limited generalizability across programming languages, and the scarcity of benchmark datasets particularly for languages like C/C++, Python and Java. Finally, we identify future research directions that can bridge the gaps in current studies. These future scopes include the integration of con-text-driven learning, hybrid symbolic–neural approaches, improved dataset construction pipelines, and explainable AI techniques for vulnerability detection. The workflow of our methodology is illustrated in Figure 3, which demonstrates the overall process from literature selection to the identification of future research opportunities.

3.2. Vulnerability Types

Modern software systems have grown more interconnected, data-driven, and developed from complex dependency relationships. While this environment drives advancement, it also increases the attack surface and raises the implications of minor code errors. As a result, vulnerability identification has evolved from chaotic finding bugs to methodical, data-driven methodologies that rely on program analysis and artificial intelligence. One of the most common vulnerable types is Memory-safety errors (stack/heap) buffer overflows (CWE-121/122) and generalized insufficient accesses (CWE-787) arise when updates and reads exceed an object’s bounds, typically due to unchecked copying, faulty index arithmetic, and size calculation. The implications include data corruption and control-flow disruption. In another, use-after-free, double-free, and dangling pointers, where lifetime mismanagement leads to use-after-free (CWE-416), double-free (CWE-415), and dangling pointer dereferencing. These flaws commonly provide arbitrary memory writes. Sound interprocedurally escape analysis remains difficult; sanitizers catch many situations at runtime but have coverage constraints. Integer overflow/underflow (CWE-190), width conversion errors, and size-calculation issues spread through allocator calls, resulting in undersized buffers and wrapped indices that cause memory violations. Then, Uncontrolled format strings (CWE-134) occur when attacker-controlled strings are used as format specifiers like printf (user), enabling information disclosure or memory writes (%n). Reads of uninitialized variables (CWE-457), improper pointer provenance, alignment violations, and other types of undefined behavior reduce program correctness and might result in security or dependability flaws. Race conditions (CWE-362), TOCTOU file races (CWE-367), deadlocks, and atomicity violations are caused from poor synchronization across shared state or time-of-check/time-of-use gaps. Because these flaws are schedule-dependent, pure fuzzing struggles with nondeterminism. In another, Leaks of memory or file descriptors (CWE-772), improper cleanup, and exception-unsafe resource handling reduce availability and create side channels. Improper privilege management (CWE-269), insecure search paths and load-order spoofing, unsafe temporary files, and capability misconfiguration could lead to escalation or arbitrary code execution in native systems. Insecure compiler/linker flags, dangerous defines, or brittle installer logic can re-enable undefined behavior or bypass mitigations such as disabling stack canaries. Practical detection integrates static policy checks in CI with learned heuristics over build manifests and scripts. Use of broken primitives, weak PRNGs (CWE-330), incorrect modes hard-coded keys, or missing salts undermines confidentiality and integrity. These are best identified by API-misuse checks enriched with semantic reasoning; few-shot prompts over code + API docs can guide LLMs to recognize insecure cryptographic patterns. However, Figure 3 illustrates the overall distribution of vulnerability types and corresponding C/C++ code samples.

According to Figure 4, SQL Injection (SQLi) in Python (version 3.10 & 3.9) is shown where user input is directly concatenated into a SQL query, allowing attackers to manipulate database commands. Similarly, OS Command Injection (OSCi) in Python demonstrates the danger of passing unvalidated input to system commands, enabling remote code execution. Cross-Site Scripting (XSS) in Java is illustrated where unescaped user input is reflected in a webpage, leading to malicious script execution in the browser. Path Traversal in Python highlights how unsafe file path handling allows attackers to access restricted files such as/etc/passwd. Insecure Deserialization (ID) in Python is shown with pickle.loads, where malicious payloads can execute arbitrary code. Server-Side Request Forgery (SSRF) in Java demonstrates how unvalidated URLs may allow attackers to access internal resources like AWS metadata services. Finally, Buffer Overflow in C/C++ is presented with an example of writing beyond the bounds of a buffer, a classic memory safety flaw that can lead to program crashes or arbitrary code execution. In

Table 2. Overview of the most prevalent and practically dangerous vulnerability types.

Vulnerability Type	Most Common/Dangerous	Real-World Impact & Statistics	References
SQL Injection (SQLi)	Massive data exposure through unsanitized query inputs. Widely exploited across industries.	2023 ResumeLooters campaign hit 65+ websites, stealing over 2M records. Equifax 2017 breach traced to SQLi.	[39,40]
Cross-Site Scripting (XSS)	Found in nearly one-third of web apps; enables session theft and phishing via injected scripts.	XSS present in 19.1% of web apps with medium severity in 2023 reporting.	[41,42]
Buffer Overflow	Continues to dominate the 2024 CWE Top 25 due to memory corruption risks in unmanaged languages.	The SQL Slammer worm (2003) exploited such a bug to cause global outages within minutes.	[43]
SSRF (Server-Side Request Forgery)	Prolific in cloud environments; attackers exploit internal service access.	Ranked significant in the 2023 Vulnerability Report; cloud metadata targeting is notable.	[41]
Path Traversal	Allows attackers to access restricted files via malformed path input.	Jira CVE-2019-11581 enabled path traversal to expose sensitive files.	[40,43]
Insecure Deserialization	Deserialized untrusted data can execute arbitrary code; critical risk in Python/Java ecosystems.	Apache Commons Collections exploit (2015) is a prime example used in high-profile breaches.	[44]
XML External Entity (XXE)	Misconfigured XML parsers allow file disclosure or SSRF.	Still recurring in enterprise environments due to insecure defaults and legacy systems.	[45]
OS Command Injection	Injected shell commands compromise host; extremely powerful if exploited.	Log4Shell (2021) indirectly leveraged command injection vectors; Shellshock (2014) is another critical case.	[44]

, we provide an overview of the most prevalent and practically dangerous vulnerability types, emphasizing their frequency in real-world systems and the significant security risks they pose.

3.3. Traditional Methods

3.3.1. Static Analysis

Static analysis refers to the examination of a software system or its components by analyzing their structure, syntax, and documentation, without the need to execute the program [46]. The main idea of static analysis is to analyze source code or program language without executing it in order to uncover errors, weaknesses, or trends that may lead to software vulnerabilities. Traditional static analysis methods include manual code inspection, in which human analysts thoroughly inquiry the source code to find shortcomings. While effective in some situations, this procedure is resource-intensive, time-consuming, and heavily reliant on inspectors’ skill and prior knowledge of known vulnerabilities. According to Rice’s Theorem [47], any non-trivial semantic property of programs is undecidable, meaning it cannot be determined by any general algorithm. This result is a direct consequence of the halting problem, as Rice demonstrated that determining whether a program possesses a particular can be reduced to solving the halting problem itself. Another basic behavior of static analysis is its conservatism. Because static analysis must function without executing the program, it usually errs on the side of caution, listing all possible behaviors that may occur. While this conservative technique ensures that potential vulnerabilities are not discovered, the results are overly approximate and, in certain cases, imprecise [48]. To overcome these limitations, researchers have developed a wide range of static analysis tools that automate parts of the process by applying predefined rules, heuristics, or formal methods to detect security flaws. Rule-based static analysis is based on preset vulnerability patterns or coding standards obtained from secure programming guidelines. This technique uses tools to check source code against a library of criteria such as “avoid unsafe functions like strcpy” or “sanitize user inputs before SQL queries.” The main advantage of rule-based systems is their interpretability, which allows engineers to readily understand why a warning was sent. However, these solutions are fragile, frequently failing to generalize to new vulnerabilities outside of their specified rule set. Furthermore, they could deliver false positives if the criteria are incredibly broad, or false negatives if the vulnerability does not perfectly fit the pattern. FindBugs [49], and Flawfinder [50] for C/C++ rely heavily on rule-based detection. With the rise of large-scale code repositories, researchers have explored detecting vulnerabilities by comparing code fragments for similarity [51,52]. The intuition is that vulnerable code often reappears across projects due to copy-paste programming or repeated insecure practices. Code similarity techniques use program representation models. Symbolic execution is a path-sensitive analysis approach that runs a program using symbolic inputs rather than physical values. It carefully investigates potential execution routes by solving symbolic constraints, revealing vulnerabilities like buffer overflows, path traversal, and assertion violations. Symbolic execution is very precise and may even detect zero-day vulnerabilities, but it suffers from the well-known route explosion problem, in which the number of alternative pathways increases exponentially with program size [53]. Unlike basic pattern matching, nowadays static analysis propagates facts over the CFG/ICFG to identify the least fixpoints for data-flow characteristics. Taint analysis is central, monitoring flows from sources through propagators to sinks while accounting for sanitizers, aliases, and procedure calls; a warning is delivered when a contaminated value can reach a sink over a potential path. Complementary components include abstract interpretation (ranges/nullness/size), point-to/alias reasoning for heap updates and lifetimes (use-after-free/double-free), and concurrent models (races/TOCTOU). We examine the merits (explainable source-sink traces) and limits (over-approximation, framework modeling) of ML/LLMs, positioning them as complements that rank and summarize, while symbolic execution/fuzzing validates feasibility.

3.3.2. Dynamic Approach

Unlike static analysis, which analyzes source or binary code prior to execution, dynamic analysis reviews a program while it is running. This helps researchers and security practitioners to identify vulnerabilities that appear only under certain runtime situations, such as memory corruption, improper input processing, or unexpected program states. Fuzzing and taint analysis are two of the most often used approaches. Fuzz testing (fuzzing) is an automated software testing technique where a program is bombarded with large volumes of random, malformed, or crafted inputs to trigger unexpected behavior. The core idea is that vulnerabilities such as buffer overflows, use-after-free, and input validation flaws often surface when the program processes unusual inputs. Miller et al. [54] developed the “fuzz” program in 1990, inspired by the failure of modem systems given the unpredictable input from “fuzzy” lines. The program creates flows of letter sequences. Building on this perspective, Oehlert [55] described fuzzing as a highly automated testing technique capable of methodically covering border instances. Fuzzing, according to this description, is the process of generating flawed or unexpected inputs from various sources, such as files, network protocols, and API requests, in order to find exploitable holes. Similarly, Sutton et al. [17] classified fuzzing as a kind of brute-force testing, emphasizing its emphasis on exhaustively feeding huge numbers of test cases to the application under test. To supplement these viewpoints, Lanzi et al. [56] defined fuzzing as a sort of black-box testing in which the program’s underlying logic remains opaque and the focus is primarily on monitoring input-output behaviors. Taint analysis dynamically tracks how untrusted inputs (tainted data) propagate through a program and determines whether they reach sensitivity without proper sanitization. The concept is that any data originating from an external user should be considered tainted, and its use in security-critical operations must be validated [18,57]. Hybrid approaches that combine symbolic execution with heuristics, fuzzing, or machine learning intent for solving this scaling issue.

3.4. Code Based Detection

3.4.1. ML Based Methods

Early machine learning (ML)–based approaches to software vulnerability detection frequently relied on software metrics as indicators of code quality and risk. These methods aimed to capture structural and complexity-related features of source code and use them as input for classifiers that distinguish vulnerable from non-vulnerable functions. For example, Zagane et al. [58] employed metrics such as total lines of code, cyclomatic complexity, and the number of distinct operators to quantify software artifacts at a finer granularity. This quantification enabled them to characterize code fragments and evaluate their likelihood of containing vulnerabilities using classifiers like RF, DT, and K-Nearest Neighbors (KNN). Such work demonstrated that code complexity and operator diversity may be strong indicators of potential flaws. Building upon these efforts, Salimi et al. [59] introduced the concept of vulnerable slices, which represent critical code segments that are particularly prone to defects. This slicing approach allowed for more targeted feature extraction and improved the precision of ML models by narrowing the focus to code regions most relevant to security analysis. Collectively, these studies underscore the potential of metric-based approaches for vulnerability detection, while also highlighting their limitations in handling semantic and contextual aspects of modern software systems. Several studies have investigated the use of machine learning techniques to predict when and how software components should undergo refactoring. Kosker et al. [60] trained a Naïve Bayes (NB) classifier to identify classes in object-oriented systems that required refactoring. Their work demonstrated that lightweight probabilistic models could provide actionable insights into maintainability improvements. Expanding on this line of study, Kumar and Sureka [61] used Least Squares-Support Vector Machine (LS-SVM) classifiers modified with the Synthetic Minority Oversampling Technique (SMOTE) to solve class imbalance. Their findings demonstrated that the LS-SVM model with a Radial Basis Function (RBF) kernel outperformed other configurations, giving it a promising contender for practical refactoring prediction. In another study, Nyamawe et al. [62] proposed a more comprehensive framework that leveraged both software evolution history and prior refactoring actions. Their method was structured as a two-stage classification pipeline: (1) a binary classification task to determine whether a refactoring was needed, and (2) a multi-label classification task to recommend the specific type of refactoring to apply. Figure 5, represent of traditional ML models employed for software vulnerability detection across existing studies (2015–2025). Among these, RF and SVM are the most widely adopted models.

3.4.2. DL Based Methods

With the increasing success of DL in fields such as computer vision and natural language processing, the software vulnerability detectio community began to transition toward DL-based approaches. These methods eliminated the need for manually engineered features by enabling neural networks to automatically learn semantic and syntactic representations of code. Recurrent Neural Networks (RNNs) and their variants have been widely adopted in software vulnerability detection due to their ability to model sequential dependencies in code. Among these, Long Short-Term Memory (LSTM) networks have emerged as one of the most prominent techniques. LSTMs effectively capture long-term contextual relationships in sequential data, mitigating the vanishing gradient problem faced by basic RNNs. Several studies have successfully applied LSTMs in source code vulnerability detection, demonstrating their robustness in modeling code semantics and flow [63,64,65,66,67,68]. The Bidirectional LSTM (BiLSTM), an extension of this model, improves efficiency by analyzing input sequences in both forward and backward directions, enabling for more context-aware representations of source code. As a result, BiLSTMs have been widely used in various vulnerability detection activities [15,69,70,71,72,73].

Beyond LSTM-based models, researchers have also explored other recurrent architectures for software vulnerability detection. The Bidirectional RNN (BiRNN) extends the standard RNN by processing sequences in both forward and backward directions, thereby capturing richer contextual information from source code. Despite its potential, this model has seen very limited application in the literature, with only a single study reporting its use [74]. The Gated Recurrent Unit (GRU) has emerged as a simplified alternative to LSTM, designed to address the same long-term dependency challenges while offering reduced computational complexity. This trade-off makes GRUs particularly appealing for large-scale vulnerability detection tasks where training efficiency is critical. Nevertheless, GRUs have been adopted sparingly, with only two studies leveraging this architecture for vulnerability-related analysis [67]. An extension of this model, the Bidirectional GRU (BiGRU), processes sequences in both directions to enrich contextual learning. By doing so, it provides a more comprehensive understanding of source code semantics. This architecture has been evaluated in two additional studies, highlighting its niche but growing role in vulnerability detection research [65,75]. Finally, more advanced generative approaches have also been explored. For example, SeqGAN, a Generative Adversarial Network (GAN) designed specifically for sequential data, has been applied in one study [33].

Graph Neural Networks (GNNs) are a class of models designed to detect relationships and relational structures in graph-structured data. They are especially valuable in software vulnerability identification since source code may be readily expressed as graphs like Abstract Syntax Trees (ASTs), Control Flow Graphs (CFGs), and Program Dependence Graphs. These structures store both syntactic and semantic information, allowing GNNs to accurately represent data flow and control inside a program. Most of the researchers employed basic GNNs for vulnerability detection, demonstrating their ability to detect subtle code weaknesses by leveraging structural dependencies [76,77,78,79,80,81,82,83]. A notable extension of this framework is the Gated Graph Sequence Neural Network (GGNN), which incorporates gating mechanisms to capture sequential dependencies within graph representations. This architecture enables the model to track how information propagates across program elements over time, thereby improving its ability to detect vulnerabilities arising from complex code interactions. Existing studies applied GGNNs in this context, where the gating mechanism enhanced feature representation and improved detection accuracy [84,85,86,87,88]. The Relational Graph Convolutional Network (RGCN) is designed to accommodate a variety of node connections. In software engineering, this is equivalent to modeling various interactions such as data dependencies, control dependencies, and API call interactions. By distinguishing between various relationship types, RGCNs deliver a more comprehensive description of program structure as found from the existing studies [89,90]. The Graph Convolutional Network (GCN) is a simplified version of GNNs that focuses on efficient convolutional operations on graph topologies. GCNs work effectively for applications that need only local connection and neighborhood patterns, such as finding clusters of susceptible services or modules. We discovered some studies that used GCNs, particularly in static vulnerability detection environments, and shown its efficacy for finding vulnerabilities at scale [83,84,91]. Convolutional Neural Networks (CNNs) represent one of the earliest and most widely adopted deep learning architectures applied to software vulnerability detection. Originally developed for image recognition, CNNs have demonstrated strong capabilities in scanning structured and sequential data to automatically learn discriminative patterns. In the context of source code analysis, CNNs treat code tokens, embeddings, or program slices as input sequences or matrices, enabling them to extract hierarchical features that differentiate between vulnerable and non-vulnerable components. Their effectiveness lies in the ability to capture both local and global dependencies in code, which are essential for recognizing subtle patterns of insecure constructs [92]. CNN-based techniques have been widely used in a variety of research [65,93,94,95,96,97,98,99]. These studies show that CNNs are particularly efficient in detecting vulnerabilities such buffer overflows, SQL injection, and API exploitation, where syntactic and semantic regularities might be acquired from token sequences. Furthermore, CNNs are frequently used as baselines for comparison with more complex deep learning models because to their efficiency and affordable computational cost. In the field of vulnerability detection, DGCNNs have been employed to identify complex vulnerability patterns that cannot be easily detected through linear code representations. For instance, one notable work [100] applied DGCNNs to model the relational and structural aspects of source code, thereby improving the detection of vulnerabilities associated with data flow and control flow interactions. Compared to traditional CNNs, DGCNNs provide a richer representation by considering the hierarchical and relational nature of source code, which is particularly useful for detecting vulnerabilities that emerge from non-trivial execution paths and contextual interactions.

Figure 6 provides a comprehensive overview of the deep learning models that have been most frequently employed in software vulnerability detection research. Overall, the figure highlights a clear research trend: sequence-based models (CNN, LSTM, BiLSTM) have been the most frequently employed approaches for vulnerability detection, underscoring the importance of analyzing code as sequential data. Meanwhile, graph-based approaches (GNN and its variants), though less common, represent an emerging research direction as they offer unique advantages in capturing the structural and relational properties of source code. This comparison suggests that while traditional sequential models remain dominant, graph-based models are steadily gaining traction in vulnerability detection studies.

3.4.3. Large Language Models (LLMs)

Large Language Models (LLMs) represent a significant step forward in the development of language modeling approaches [101,102]. Figure 7 illustrates the recent utilization of LLMs across various applications. Building on the Transformer architecture, these models have shown the ability to expand to unprecedented levels, frequently comprising hundreds of billions of parameters. Furthermore, training LLMs on vast and varied structures has provided them with a level of generality not possible with previous generations of models. This generalization capacity implies LLMs to adapt to new domains and tasks with minimum fine-tuning, emphasizing their importance as foundation models in current AI research [103]. Their scalability also highlights a key trend in deep learning: performance improves as model size, data, and compute resources grow; a phenomenon known as “scaling laws” [104,105]. Pan et al. [106] proposed a taxonomy that categorizes LLMs into three primary architectural groups: encoder-only, encoder–decoder, and decoder-only models.

Encoder-Only. Encoder-only models depend based only on representation learning. These models, notably BERT [107], are designed to capture deep bidirectional contextual embeddings, making them ideal for classification, feature extraction, and semantic similarity tasks. Their strength is in creating rich contextual representations of input sequences, that could further be used in downstream supervised learning problems. In order so, these mathematical representations provide fine-grained analysis of programming structures, making them extremely useful in software engineering (SE) research. Several customized encoder-only models have been developed for SE purposes. In particular, CodeBERT [108] and GraphCodeBERT [109] expand BERT-like structures to source code and data-flow graphs, respectively, whilst CuBERT [110] focuses on large-scale pretraining of Python programs. Other domain-adapted encoder models include VulBERTa [111] for vulnerability detection, CCBERT [112] for code comprehension. For vulnerability detection most of the study used BERT and CodeBERT most for the evaluation and their performances [113,114,115,116].

Encoder–Decoder. Encoder–decoder models leverage the combined strengths of both the encoder and decoder components of the Transformer architecture. The encoder processes the input sequence and transforms it into a high-dimensional structured representation, which captures both syntactic and semantic properties. The decoder then generates the corresponding output sequence conditioned on this representation. This dual structure provides high versatility, making encoder–decoder models particularly effective for tasks such as translation, summarization, defect repair, and code transformation. For instance, models like PLBART [117] and T5 [118] were originally proposed for natural language processing (NLP) but have been successfully adapted to SE tasks due to their ability to learn cross-domain semantics. Similarly, CodeT5 [119] extends the T5 architecture for programming languages, demonstrating strong performance in code summarization and vulnerability detection. Other representative models include UniXcoder [120], which unifies programming language and natural language tasks in a single framework, and NatGen [121], which is designed for code generation and related SE and SVD tasks.

Decoder-Only. Decoder-only LLMs are built solely on the decoder component of the Transformer architecture, making them inherently generative in nature. These models are designed to produce text or source code sequences conditioned on an input prompt. By relying on autoregressive token prediction, they interpret contextual information from the prompt and iteratively generate subsequent tokens. This design enables decoder-only models to construct coherent, contextually relevant, and semantically rich sequences, which makes them particularly effective for tasks such as dialogue systems, text completion, and code generation.

Their strength lies in the ability to extend and elaborate context dynamically, producing complex outputs that align with both local syntax and global semantics of the input. Significant models in the decoder-only LLMs category include the GPT family, which includes GPT-2 [122], GPT-3 [123], GPT-3.5 [124], and the most current GPT-4 [125]. In addition to general-purpose models, some architectures have been designed specifically for source code and software engineering applications. These include CodeGPT, Codex, Polycoder, Incoder, the CodeGen series, as well as specialized systems like Code Llama, and StarCoder [126,127,128,129,130,131,132,133,134]. In combination, these models highlight the rapid growth of decoder-only LLMs from wide natural language production to highly specialized tools for programming and vulnerability identification [135,136,137,138].

In Figure 8, illustrates a taxonomy of vulnerability detection approaches, classified into Traditional, ML, DL, and LLMs. Traditional approaches include static (AST, CFG, rule-based) and dynamic (fuzzing, symbolic execution) analysis. ML models rely on handcrafted features with classifiers like SVM and Random Forest. DL models use CNNs, RNNs, GNNs, and Transformer encoders to learn structural and semantic patterns. LLM models leverage prompting, fine-tuning, and hybrid reasoning with advanced code models (GPT-4, CodeLlama, CodeBERT). Each category highlights common techniques, strengths, and weaknesses.

Figure 9 shows the evolution of vulnerability detection research from standard ML to advanced LLM-based approaches. The number shows that LLM/Transformer-based models currently dominate research (31.6%), followed by DL (21.1%), with traditional ML, graph-based, and hybrid techniques contributing approximately 15–16% for each. The related line graph shows a higher trend and increased focus on LLM-driven systems, emphasizing their superior capacity to represent complex structural and conceptual relationships when compared to traditional systems.

3.5. Feature Representation

3.5.1. Graph Based Feature Representation

In this section, we review prior studies that employ different graph-based program representations such as Abstract Syntax Trees (ASTs), Control Flow Graphs (CFGs), Program Dependence Graphs (PDGs), and Data Dependence Graphs (DDGs) as inputs to deep neural networks (DNNs) for learning rich feature representations. In this section, source code is represented functionally using graphs and trees to encapsulate relationships, control flows, and program semantics. The Control Flow Graph (CFG) is a fundamental representation that describes a program’s execution order by depicting the transitions between distinct code segments [59,80,91,113,134,135,136,137]. In addition to CFGs, the Program Dependence Graph (PDG) encodes both control and data dependencies between program statements, providing a more comprehensive view of inter-statement relationships [76,79,82,139,140,141]. Among all structural forms, the Abstract Syntax Tree (AST) remains the most widely adopted, representing syntactic constructs in a hierarchical tree that supports both syntactic and semantic analysis [64,75,79,138]. Variants such as the Control Flow Abstract Syntax Tree (CFAST) enrich ASTs with control flow information for enhanced program representation [139,140,141], while the Binary AST adapts tree-based structural representations for binary-level code analysis [93].

3.5.2. Text Based Feature Representation

A recent method introduced in [142] transformed Java source files into sequences of textual tokens and utilized an N-gram model to embed these tokens into vector representations. The underlying assumption was that vulnerable patterns could be revealed by analyzing the frequency distributions of code tokens. A different approach CNN-based technique presented in [143] focused on discovering vulnerabilities in C programs at the assembly level, with the goal of directly extracting dangerous code patterns from assembly instructions. To represent these instructions as vectors, the authors proposed Instruction2vec, a model based on the Word2vec framework [144,145]. A analogous CNN-based technique for function-level vulnerability identification was described in [146]. The study made use of a huge dataset of around 12 million source code functions obtained from the Juliet Test Suite, the Debian Linux distribution, and GitHub. To preprocess the code, the authors created a bespoke C/C++ lexer that converted each function into a series of tokens. The lexer limited the vocabulary to 156 tokens, which included all C/C++ keywords, operators, and separators, and removed non-essential code components that had no effect on compilation. Another popular form is the Code Slice, which separates specific chunks of source code based on established slicing criteria such as data and control dependencies. This method allows for the extraction of significant fragments that are most likely to include susceptible patterns and has been used in various research [65,79,147,148,149,150,151,152]. In contrast, the Code Snippet format concentrates on short, continuous pieces of code, which are usually retrieved for focused study of specific code sections [71]. Other representation and embedding techniques have also been adopted in recent studies, including GloVe, Doc2Vec, Continuous Bag-of-Words (CBoW), Sent2Vec, GRU-based embeddings, FastText, and N-gram models [65,69,70,75,137,147,149,152].

3.6. Prompt Engineering Techniques

Prompt engineering is the process of designing and refining inputs (prompts) given to a LLMs so that it produces more accurate, reliable, or useful outputs. Since LLMs are sensitive to how instructions are phrased, effective prompting can drastically improve performance on tasks like text generation, reasoning, summarization, or vulnerability detection.

Chain-of-Thought (CoT). Chain-of-thought prompting asks the model to provide step-by-step reasoning before giving the final answer. This improves both interpretability and accuracy in reasoning-intensive tasks such as vulnerability analysis [151,152]. According to Figure 10, the model might reason: (1) sprintf copies input without bounds checking, (2) the destination buffer is small (10 bytes), (3) untrusted input may exceed the buffer. Based on this reasoning, the final classification is 1 (vulnerable). CoT has been shown to improve performance in complex tasks by guiding the model through intermediate reasoning steps rather than relying on a direct prediction.

Zero-Shot Prompting. Zero-shot prompting is one of the most basic yet widely used strategies, where the model is asked to perform a task without being provided with any prior examples [153]. The model relies entirely on its pre-trained knowledge to infer the answer from the given instruction. For example, according to Figure 11, the function copies user input into a fixed-size buffer using strcpy, which does not perform bounds checking. This can lead to a stack-based buffer overflow if the input string exceeds 15 characters. To analyze this vulnerability, applied a zero-shot prompt, instructing the model to identify the vulnerable line, classify the vulnerability, assign CWE identifiers, assess severity, and suggest a safer alternative. The model’s structured JSON response correctly flagged line 3 as vulnerable, labeled it a buffer overflow, mapped it to CWE-120 and CWE-242, and recommended replacing strcpy with strncpy along with explicit null termination. This illustrates how zero-shot prompting can be effectively used to generate precise, structured vulnerability reports directly from source code without additional training.

Few-Shot Prompting. Few-shot prompting is a technique used in LLMs interactions where the model is given a small number of worked examples (inputs paired with expected outputs) before being asked to analyze a new, unseen instance. Unlike zero-shot prompting where the model must rely solely on instructions, few-shot prompting provides concrete demonstrations of the desired reasoning process and output format [154]. This example in Figure 12, demonstrates the essence of few-shot prompting: the model learns from a small, labeled example (f()) how vulnerabilities are reported. The functions suffer from the same type of bug (a stack-based buffer overflow caused by strcpy), but the buffer sizes differ.

3.7. Fine Tuning

Fine-tuning enables LLMs to adapt more effectively to specialized tasks by retraining a pre-trained model on domain-specific datasets. Instead of relying solely on general-purpose knowledge, fine-tuning equips the model with the ability to recognize and reason about the unique characteristics of software security. This process is particularly important for three key reasons [155,156]. First, security vulnerabilities in source code typically manifest through recurring patterns, such as unsafe memory operations, unchecked user inputs, or flawed control flows. These subtle patterns must be explicitly learned by the model in order to distinguish vulnerable code from safe implementations. Second, programming languages differ significantly from natural language, as they follow strict syntactic and semantic rules. LLMs originally trained on mixed corpora often lack deep understanding of code structure; thus, fine-tuning helps them better parse program logic, dependency structures, and execution semantics. Third, vulnerability detection demands a high degree of precision, as even small misclassifications can lead to overlooked security flaws or excessive false alarms. Fine-tuning improves the model’s ability to deliver reliable and accurate predictions in this context, making it a crucial step for applying LLMs to vulnerability detection.

Full Fine-Tuning (FFT). Full fine-tuning refers to the process of updating all model parameters during the training phase, which allows the LLMs to completely adapt to the downstream task. However, due to the high computational and memory costs of training large-scale models, most research has focused on relatively smaller architectures, typically below 15 billion parameters, such as CodeT5, CodeBERT, and UnixCoder. For example, Ding et al. [157] evaluated five models with fewer than 7B parameters and reported limited performance, achieving only a 0.21 F1-score on the PrimeVul dataset even when both training and validation were performed within the same distribution. Similarly, Guo et al. [158] fine-tuned CodeBERT for 50 epochs using FFT, which resulted in poor generalization across datasets: only 0.099 F1-score on PrimeVul, but a significantly higher 0.66 F1-score on the Choi2017 dataset. In contrast, Haurogne et al. [159] demonstrated improved performance with FFT, achieving a 0.69 F1-score on DiverseVul, while Purba et al. [160] reported an even stronger 0.73 F1-score for the task of buffer overflow detection. These findings suggest that while FFT allows full task adaptation, its effectiveness varies greatly depending on dataset diversity, model size, and task complexity, making it computationally expensive yet sometimes suboptimal compared to more efficient fine-tuning strategies.

Parameter-Efficient Fine-Tuning (PEFT). PEFT methods aim to adapt LLMs to downstream tasks by modifying only a small subset of parameters while keeping most of the pre-trained weights frozen. This significantly reduces the computational and memory costs while maintaining competitive performance. One common PEFT technique is the use of adapters, where lightweight trainable layers are inserted between the original transformer layers. For example, Yang et al. [161] applied adapters for fault localization and reported a 60% Top-5 accuracy, demonstrating their ability to guide models toward specialized tasks without full retraining. Another widely adopted technique is LoRA (Low-Rank Adaptation), which represents weight updates as low-rank decompositions. LoRA has shown strong effectiveness across multiple vulnerability detection tasks: Du et al. [162] reported a 0.72 F1-score, while Guo et al. [158] achieved a remarkably high 0.97 F1-score on their respective datasets. An extension of this approach, QLoRA, combines LoRA with quantization techniques to further reduce memory requirements. Boi et al. [163] demonstrated that QLoRA can achieve 59% accuracy while maintaining low resource usage, highlighting its suitability for resource-constrained environments. Overall, PEFT methods strike a balance between efficiency and effectiveness, making them an increasingly attractive alternative to full fine-tuning, especially for large-scale LLMs applied to software vulnerability detection.

Generative Fine-Tuning. Generative fine-tuning adapts large language models for sequence-to-sequence tasks, where the goal is not just classification but the generation of structured outputs such as vulnerability explanations, patch suggestions, or the identification of vulnerable lines. Unlike discriminative approaches that only assign labels like vulnerable and non-vulnerable, generative fine-tuning allows models to produce richer outputs that enhance interpretability and support downstream tasks like automated repair. Yin et al. [164] demonstrated that fine-tuning pre-trained language models (LMs) for generative objectives often outperforms fine-tuning larger LLMs. For example, CodeT5+ achieved a ROUGE score of 0.722, substantially surpassing the performance of the much larger DeepSeek-Coder 6.7B, which reached only 0.425.

3.8. Evaluation Metrics

Classification Metrics. Evaluating vulnerability detection systems requires a range of metrics that capture both overall performance and specific trade-offs between false positives and false negatives [165,166]. Accuracy measures overall correctness and is defined as:

$$\mathit{A}\mathit{c}\mathit{c}\mathit{u}\mathit{r}\mathit{a}\mathit{c}\mathit{y}={\displaystyle \frac{\mathit{T}\mathit{P}+\mathit{T}\mathit{N}}{\mathit{T}\mathit{P}+\mathit{T}\mathit{N}+\mathit{F}\mathit{P}+\mathit{F}\mathit{N}}}$$

(1)

While intuitive, accuracy can be misleading in highly imbalanced datasets, since a model predicting all samples as “non-vulnerable” may still achieve high accuracy. Precision, or the positive predictive value, is given by:

$$\mathit{P}\mathit{r}\mathit{e}\mathit{c}\mathit{i}\mathit{s}\mathit{i}\mathit{o}\mathit{n}={\displaystyle \frac{\mathit{T}\mathit{P}}{\mathit{T}\mathit{P}+\mathit{F}\mathit{P}}}$$

(2)

Precision reflects the proportion of detected vulnerabilities that are truly vulnerable. High precision is essential when false alarms (false positives) are costly, such as in automated triage systems. Recall, or sensitivity, is defined as:

$$\mathit{R}\mathit{e}\mathit{c}\mathit{a}\mathit{l}\mathit{l}={\displaystyle \frac{\mathit{T}\mathit{P}}{\mathit{T}\mathit{P}+\mathit{F}\mathit{N}}}$$

(3)

It indicates how many of the actual vulnerabilities were correctly detected. High recall is critical in security applications because missing true vulnerabilities (false negatives) can lead to severe risks. F1-Score, the harmonic mean of precision and recall, balances these two metrics:

$$\mathit{P}\mathit{r}\mathit{e}\mathit{c}\mathit{i}\mathit{s}\mathit{i}\mathit{o}\mathit{n}=2.{\displaystyle \frac{\mathit{P}\mathit{r}\mathit{e}\mathit{c}\mathit{i}\mathit{s}\mathit{i}\mathit{o}\mathit{n}.\mathit{R}\mathit{e}\mathit{c}\mathit{a}\mathit{l}\mathit{l}}{\mathit{P}\mathit{r}\mathit{e}\mathit{c}\mathit{i}\mathit{s}\mathit{i}\mathit{o}\mathit{n}+\mathit{R}\mathit{e}\mathit{c}\mathit{a}\mathit{l}\mathit{l}}}$$

(4)

Specificity (true negative rate) is the complement of recall and measures how effectively the model identifies safe code:

$$\mathit{S}\mathit{p}\mathit{e}\mathit{c}\mathit{i}\mathit{f}\mathit{i}\mathit{c}\mathit{i}\mathit{t}\mathit{y}={\displaystyle \frac{\mathit{T}\mathit{N}}{\mathit{T}\mathit{N}+\mathit{F}\mathit{P}}}$$

(5)

Generation and Explainability Metrics. For tasks that go beyond binary classification, such as generating vulnerability descriptions, explanations, or suggested fixes, evaluation requires different metrics. Bilingual Evaluation Understudy (BLEU) measures n-gram precision while applying a brevity penalty to prevent overly short outputs. It is widely used to assess generated vulnerability descriptions. Recall-Oriented Understudy for Gisting Evaluation (ROUGE) emphasizes recall by measuring overlap between generated and reference texts. Variants such as ROUGE-N, ROUGE-L, and ROUGE-S are used to evaluate explanation quality. METEOR goes beyond n-gram overlap by considering synonyms, stemming, and word order. Although less common in software engineering research, it can capture semantic similarity better than BLEU or ROUGE [167].

4. Insights, and Challenges

4.1. Limited Scope

Recent vulnerability detection research has been conducted on a variety of datasets that diverge in quantity, quality, and authenticity. Existing vulnerability datasets split into three categories: synthetic, semi-real, and real-world, each with a distinctive set of properties and applicability. Synthetic datasets are manually created and designed to represent specific vulnerability patterns, whereas real-world datasets are derived from software repositories, commits, and security patches that reflect appropriate software development configurations. The Juliet Test Suite is a synthetic benchmark that includes thousands of C/C++ and Java methods identified with CWE types. It provides clear annotations and extensive vulnerability coverage, making it excellent for early-stage model training and controlled benchmarking. However, Juliet’s code fragments are artificially produced and context-independent, causing limited realism and poor applicability to real-world software. The Devign dataset, which has been generated from actual GitHub projects, comprises vulnerable-fixed code combinations that are useful for graph-based and patch-learning algorithms. Despite providing realistic code designs, it suffers from labeling noise, a small scale, and limited project variety. The dataset also contains duplicate samples. PrimeVul extends this approach by aggregating samples from several open-source repositories and providing function-level CVE- and CWE-mapped combinations, which facilitate multi-label categorization and cross-project assessment. Its breadth is constrained due to partial synthetic augmentation and inadequate commit information. Among large-scale real-world datasets, BigVul is one of the most comprehensive, comprising more than 350,000 C/C++ functions with real CVE-linked vulnerabilities mined from over 300 projects. It is widely used for LLMs pre-training and model generalization studies, though it faces class imbalance, noisy labeling, and heavy preprocessing requirements. The Reveal dataset focuses on Java projects, offering rich object-oriented code context and supporting language-specific and transfer-learning research, yet it has restricted language coverage and moderate sample size. Similarly, VUDENC combines vulnerable code with natural-language vulnerability descriptions, enabling reasoning and explanation generation for LLMs, but its limited data scale and language diversity constrain its general use. Draper VDISC contains real industrial C/C++ functions with fine-grained vulnerability metrics, suitable for interpretability and semantic-analysis research, although its medium scale and incomplete CWE labeling limit broader benchmarking. Although existing vulnerability datasets, whether synthetic, semi-real, or real-world have considerably improved research in automated vulnerability identification, the field’s future growth is dependent on enhancing and increasing these corpora. Future research should focus on developing larger, cleaner, and more balanced multilingual datasets that reflect various project structures and complicated vulnerability meanings. Integrating commit-level context, fix history, and more comprehensive data will improve the realism and reasoning capacity of future datasets. Furthermore, standardizing labeling techniques and creating consistent benchmarking frameworks would ensure consistency and comparability across studies. Combining synthetic creation with real-world mining might also assist with data shortage and imbalance concerns.

4.2. Generalization Across Languages and Domains

Most existing vulnerability detection models are trained and evaluated on a single programming language, yet real-world software systems are polyglot in nature. Models fine-tuned on one dataset often fail to generalize when applied to different languages or frameworks due to variations in syntax, semantics, and idiomatic practices. For example, vulnerabilities in memory-unsafe languages such as C/C++ differ significantly from those in managed languages like Python or Java. This challenge highlights the need for cross-language vulnerability datasets and multilingual fine-tuning strategies that enable LLMs to capture both language-specific patterns and universal vulnerability semantics.

4.3. Limited Vulnerability Type Coverage

Another important limitation in current LLM-based vulnerability detection research is the restricted coverage of vulnerability types. Most existing datasets and detection models focus primarily on a handful of common vulnerability classes such as buffer overflows, null pointer dereferences, or injection flaws. While these categories are important, real-world software systems face a much broader spectrum of security issues, including logic flaws, concurrency errors, race conditions, cryptographic misuses, insecure APIs, and supply-chain vulnerabilities. Many of these vulnerability classes are underrepresented or entirely absent in publicly available datasets due to the difficulty of labeling them and the lack of consistent ground truth. As a result, models trained on narrow datasets often demonstrate high performance for a limited set of CWE categories but fail to generalize to emerging or less frequent vulnerability types. This restricted scope creates a coverage gap between academic benchmarks and practical needs in industry, where vulnerabilities are diverse, context-dependent, and evolve rapidly with new technologies. Addressing this challenge requires the development of broader and more representative datasets that capture diverse vulnerability classes and encourage models to move beyond detecting only the most common patterns.

4.4. Limited Availability

A recurring challenge in LLM-based vulnerability detection research is the limited availability of datasets and trained models. Many studies rely on proprietary or internally curated datasets that are not released publicly due to confidentiality, licensing restrictions, or security concerns. This lack of open access hinders reproducibility and prevents other researchers from validating or extending prior work. Similarly, fine-tuned vulnerability detection models are often not made available, either because of intellectual property restrictions, the computational expense of hosting large checkpoints, or the sensitive nature of security-related models. As a result, researchers are frequently forced to reimplement baselines under different conditions, leading to inconsistencies in reported results. The unavailability of standardized datasets and pre-trained/fine-tuned models also slows progress in benchmarking, since it is difficult to make fair comparisons across methods. Addressing this challenge requires stronger community efforts toward open-source datasets, shared model repositories, and reproducible pipelines, similar to initiatives in NLP and computer vision. Such openness would accelerate innovation, foster collaboration, and improve the reliability of advances in vulnerability detection research.

4.5. Reproducibility and Benchmarking Gaps

A significant challenge in this research area is the lack of standardized, reproducible benchmarks. Many studies use different datasets, preprocessing pipelines, and evaluation metrics, making cross-comparison difficult. For example, some works report accuracy while others focus on F1-score, MCC, or ROC-AUC, often without clarifying dataset splits or deduplication strategies. This inconsistency leads to inflated claims and prevents fair evaluation of competing approaches. Establishing unified benchmarks and reproducible pipelines is therefore critical to advancing the field.

4.6. Lack of Interpretability

A major challenge in applying LLMs to vulnerability detection is the lack of interpretability and transparency in their predictions. While models such as CodeBERT, CodeT5, or LLaMA-based architectures can achieve high accuracy, they typically function as black boxes, providing little insight into why a particular code snippet is classified as vulnerable. For high-stakes domains like software security, this opacity is problematic: developers and auditors need to understand which lines of code are risky, which vulnerability category it corresponds to, and why the model reached its decision. Without interpretability, false positives can lead to wasted developer effort, while false negatives can leave critical vulnerabilities undetected. Although recent advances such as attention visualization, vulnerable line highlighting, and chain-of-thought prompting have attempted to improve explainability, these methods remain limited in consistency and reliability. In many cases, the explanations generated by LLMs may sound plausible but fail to align with the true vulnerability cause, creating a false sense of trust. This interpretability gap restricts the adoption of LLM-based vulnerability detection in industry, where accountability and explainability are critical for security audits, compliance, and developer trust. Bridging this gap requires research into more robust explainability techniques, such as neuro-symbolic reasoning, program slicing with interpretability, or integrating external knowledge into model outputs.

4.7. Real-World Performances

A persistent limitation of current LLM-based vulnerability detection systems is their performance gap between benchmark datasets and real-world software. Many models demonstrate strong results on curated datasets such as Juliet, Devign, or PrimeVul, where vulnerabilities are either synthetic or clearly labeled. However, when deployed on real-world projects—characterized by large, noisy, and heterogeneous codebases—the performance often degrades significantly. Real-world software frequently includes incomplete code fragments, project-specific libraries, macros, or mixed-language environments, which differ from the clean, self-contained samples used during training. Furthermore, vulnerabilities in production systems are often more subtle, context-dependent, and influenced by system-level interactions that are not captured in small benchmark functions. This results in models generating high false positive rates, missing rare but critical vulnerability types, or failing to generalize across projects and domains. The gap between laboratory benchmarks and industrial codebases limits the practical adoption of these systems in security auditing pipelines. Addressing this challenge requires not only larger and more representative datasets but also robust evaluation on real-world repositories, continual learning mechanisms to adapt to evolving code, and hybrid approaches that combine statistical learning with program analysis to handle complex contexts.

4.8. LLM Hallucination and Overfitting Issues

Another major challenge in applying LLMs for vulnerability detection is the tendency of models to hallucinate or overfit training data. Hallucination refers to the generation of confident but factually incorrect or irrelevant outputs, such as assigning a CWE label to code where no vulnerability exists, or producing vulnerability explanations that sound plausible but are technically inaccurate. In high-stakes security tasks, such hallucinations can be particularly dangerous, as they may mislead developers into believing that code is safe (false negatives) or raise unnecessary alarms (false positives). Overfitting, on the other hand, occurs when models memorize patterns from small or imbalanced datasets rather than learning generalizable vulnerability features. For example, models trained on benchmark datasets often latch onto surface-level token sequences rather than understanding deeper control- or data-flow dependencies, leading to degraded performance on unseen or real-world projects. These issues highlight a fundamental limitation of current LLMs: while they are powerful pattern recognizers, they lack grounding in program semantics and may generate outputs that do not align with actual execution behavior. Addressing hallucination and overfitting requires regularization techniques, adversarial evaluation, cross-dataset benchmarking, and hybrid approaches that combine LLMs with static or symbolic analysis to ensure that predictions remain both accurate and trustworthy.

4.9. Bias in LLM-Based Vulnerability Detection

Bias is an often-overlooked but significant challenge when applying LLMs to vulnerability detection. Since most LLMs are pre-trained on imbalanced and uncurated corpora, they inherit biases present in the training data. For instance, publicly available vulnerability datasets frequently overrepresent certain vulnerability types while underrepresenting others such as cryptographic misuse, concurrency flaws, or logic errors. As a result, models may learn to prioritize detecting common vulnerability patterns while ignoring less frequent but equally critical classes, leading to type bias. Similarly, project and language bias arises because many datasets are dominated by a handful of open-source projects in C/C++, with limited representation from other languages or industrial codebases. This restricts generalization across diverse ecosystems. Furthermore, evaluation practices may unintentionally reinforce bias when models are repeatedly benchmarked on the same datasets, creating over-optimistic results that fail to reflect real-world conditions. Such biases not only undermine fairness and generalizability but also reduce trust in the model’s outputs, as developers cannot be confident that all vulnerability categories are being treated equally. Addressing bias requires more balanced dataset construction, cross-language evaluations, fairness-aware training objectives, and transparent reporting of dataset limitations in future research.

4.10. GPU and Resource Limitations

Training and deploying LLMs for vulnerability detection is highly resource-intensive. State-of-the-art models often contain billions of parameters, requiring large amounts of GPU memory, high-bandwidth interconnects, and optimized distributed training frameworks. Full fine-tuning of models like CodeT5+, LLaMA-7B, or DeepSeek-Coder is often infeasible for many academic and industrial teams without access to specialized hardware. Even parameter-efficient methods such as LoRA or QLoRA reduce, but do not eliminate, the computational burden, as long training times, high memory usage, and significant energy costs remain. These GPU limitations restrict reproducibility, slow down experimentation cycles, and prevent widespread adoption of LLM-based vulnerability detectors. Moreover, inference at scale can also be prohibitively expensive if not carefully optimized. This computational bottleneck makes it difficult to bridge the gap between research prototypes and deployable industry tools.

5. Future Directions

Despite recent advances in LLM-based vulnerability detection, numerous open issues remain unresolved. Future research should address these challenges by pursuing several complementary directions.

5.1. Data-Centric Development

Gap. Public corpora overrepresent a few CWE families and synthetic cases, limiting real-world generalization.

Direction. Future research should explore numerous, diversified sources to create organized, constantly revised datasets with rigorous deduplication, leakage control (project/time divides), and long-tail coverage (logic faults, crypto unlawful use, concurrency/races). Including flow facts (sources/sanitizers/sinks), span-level locations, and uncertainty ratings will result in more equitable training and evaluation. Benchmarks ought to include IID and OOD tracks with per-CWE metrics (macro-F1, AUROC, calibration) to ensure that models are awarded for breadth and robustness instead of overall reliability.

5.2. Neuro-Symbolic and Hybrid Approaches

Gap. LLMs capture patterns but lack semantic grounding (flows, control).

Direction. A potential strategy is to integrate code models with symbolic tools (AST/CFG/PDG, taint analysis, and logical parameters). Conventional pipelines include feature fusion (injecting flow/graph features), constraint-guided decoding (rejecting predictions that break flow rules), and post hoc verification (LLM + static checker). Evaluations should compare code-only, flow-only, and code + flow settings, assess evidence quality (sanitizer source to sink routes), and report hallucination reductions at a fixed retention rate.

5.3. Efficient and Resource-Aware Fine-Tuning

Gap. High VRAM cost blocks deployment in CI/IDEs.

Direction. Deployment in CI/IDEs demands low latency and small VRAM footprints. Parameter-efficient methods (LoRA/QLoRA/adapters), compression (quantization/pruning), and distillation should be standard, with cost reporting alongside accuracy (latency@batch, tokens/s, peak VRAM, energy per KLOC). Targets like ≤8 GB VRAM and sub-150 ms per function for IDEs make results actionable. The goal is Pareto-optimal detectors that preserve ≥95% macro-F1 while doubling throughput.

5.4. Cross-Language and Multi-Modal Vulnerability Detection

Gap. Polyglot systems and mixed artifacts (code + commits + docs) are underused.

Direction. Modern codebases are multilingual and context-rich. Future models should learn universal vulnerability concepts and language-specific patterns in C/C++/Java/Python while including auxiliary modalities such as commits, issues/docs, IaC/configs, and runtime traces. Protocols should incorporate zero-shot cross-language testing, few-shot transfer, and multimodal capabilities to measure metadata benefits. Success is assessed by transfer ΔF1 and improvements in fixable-bug recall after adding commits/traces.

5.5. Explainability and Trustworthy AI

Gap. Low adoption without actionable, auditable explanations.

Direction. Adoption relies on explanations that developers can check and act on. Detectors should produce highlighted spans, CWE maps, and evidence graphs to support predictions, as well as calibrated confidence or conformal intervals. Human studies (time to triage, repair accuracy) and automated checks using static verifiers are required to validate utility. Faithfulness metrics (deletion/insertion tests) and calibrated uncertainty should be provided accurately, providing teams to trading precision for trust.

5.6. Fairness, Bias, and Robust Evaluation

Gap. Skewed labels and brittle models.

Direction. Addressing bias in datasets and models will be critical to ensure fair detection across vulnerability types, programming languages, and software domains. Standardized benchmarks, cross-dataset evaluations, and fairness-aware training objectives should become the norm. Additionally, rigorous adversarial evaluation strategies should be employed to assess robustness against code obfuscation, adversarial samples, and distributional shifts.

6. Conclusions

This survey provides a comprehensive and up-to-date overview of recent software vulnerability detection approaches, with an emphasis on Machine Learning (ML), Deep Learning (DL), and Large Language Models. While conventional rule-based and static techniques are still valuable for identifying known vulnerability patterns, their limited scalability and adaptability have prompted the change to data-driven solutions. ML and DL approaches automate feature extraction and representation learning, respectively, while LLMs enhance these capabilities by comprehending code semantics, reasoning over context, and creating patches and justifications. This review presents an additional clarification of the literature. First, it provides a uniform comparison framework that connects ML, DL, and LLM-based vulnerability detection algorithms using comparable evaluation standards. Second, it identifies previously neglected challenges such as data imbalance, interpretability, hallucination, and computing cost that limit the use of existing models. Third, it discusses upcoming topics including neuro-symbolic hybrid systems, parameter-efficient fine-tuning, continuous learning, and cross-language generalization. By articulating these insights, this review moves beyond descriptive summarization to identify concrete research opportunities and practical pathways for building next-generation vulnerability detection systems that are scalable, interpretable, and trustworthy for real-world software development.