A Survey on Large Language Models in Software Security: Opportunities and Threats

Abstract

The rise of large language models (LLMs), such as GPT-4, Codex, Code Llama, Claude 3, CodeGemma and DeepSeek, etc., is changing the way software development is approached. These models provide strong support for tasks like writing codes, analyzing bugs, and automation. At the same time, their use in software development creates both opportunities and new risks. This survey reviews how LLMs are being used to improve security practices in software development, including vulnerability detection, secure code generation, threat analysis, and patch development. It also discusses how attackers may exploit LLMs for malicious purposes, such as writing malware, carrying out phishing campaigns, or bypassing defenses. We draw on case studies that show LLMs can help uncover zero-day vulnerabilities and speed up secure coding but also highlight cases where they have been misused to generate harmful code, sometimes unintentionally. The paper examines technical challenges like bias in training data, the difficulty of interpreting model outputs, and the risks of adversarial attacks. It also considers ethical and regulatory issues related to accountability, compliance, and responsible use. By bringing together findings from recent research and industry practice, the survey outlines future directions for building safer models, developing stronger defensive frameworks, and shaping policies that balance innovation with security. Overall, the paper argues for a careful approach where LLMs are used to strengthen software security while addressing the risks they introduce through collaboration, oversight, and ongoing improvements.

1. Introduction

Large language models (LLMs) have rapidly transitioned from experimental natural language processing systems into foundational components of contemporary software engineering toolchains, with increasingly direct implications for software security. Early large-scale transformer-based models trained on mixed natural language and source code corpora demonstrated that a single architecture could generalize across program synthesis, translation, summarization, and bug explanation tasks [1]. These early results fundamentally altered assumptions about the separation between natural language understanding and program analysis. Subsequent generations of open and proprietary models, including Code Llama [2], LLaMA 2 [3], and reasoning-enhanced systems such as DeepSeek-R1 [4], expanded these capabilities across programming languages, paradigms, and increasingly complex reasoning tasks. In parallel, representation-learning approaches such as CodeBERT and GraphCodeBERT introduced structure-aware and data flow-aware pre-training strategies that significantly improved downstream performance on program analysis, vulnerability localization, and code understanding tasks [5,6].

These technical advances have catalyzed a rapidly growing body of empirical research examining the role of LLMs in software security. Multiple studies show that LLMs can assist with vulnerability detection, secure code generation, automated testing, and repair, often achieving higher recall and broader cross-language coverage than traditional static analysis or rule-based tools [7,8,9]. Dedicated datasets and benchmarks—including DiverseVul [10], VulnLLMEval [11], and CVE-Bench [12]—have enabled more systematic evaluation of detection accuracy, robustness, and repair correctness. Complementary surveys and systematization efforts situate these results within the broader evolution of automated secure coding, machine learning-based vulnerability detection, and AI-assisted software engineering [13,14,15,16,17].

At the same time, a substantial and growing body of peer-reviewed evidence demonstrates that LLM-assisted development introduces persistent and non-trivial security risks. Empirical studies of AI-based code assistants consistently report the generation of insecure patterns, including injection vulnerabilities, improper cryptographic usage, unsafe deserialization, flawed access control, and insufficient input validation [18,19,20,21,22,23,24,25]. Replication and empirical studies further indicate that these weaknesses are not isolated artifacts of early models but persist across versions, deployment contexts, and prompting strategies [26,27,28]. Such findings reinforce concerns about automation bias, over-trust in AI-generated suggestions, and the gradual accumulation of latent vulnerabilities in production systems [29,30].

Beyond code generation quality, LLMs raise deeper systemic security concerns related to data leakage, memorization, and adversarial manipulation. Prior work demonstrates that large models can unintentionally reveal sensitive or proprietary training data through extraction and model inversion attacks [31,32,33]. Analyses of widely used pre-training corpora reveal the presence of vulnerable, outdated, or mislicensed code, creating pathways for vulnerability propagation and legal risk at scale [34,35]. Adversarial studies further show that code-generating models are susceptible to backdoor insertion, prompt injection, and malicious steering, enabling the generation of harmful or policy-violating code even under nominal safeguards [36,37,38,39]. Collectively, these results suggest that LLM-related security issues are structural properties of probabilistic learning from large-scale corpora, rather than incidental defects that disappear with scale alone.

The increasing integration of LLMs into real-world development workflows amplifies both their benefits and their risks. AI-powered assistants are now embedded in DevSecOps pipelines, continuous integration systems, code review tools, and threat modeling activities [40,41,42]. Although such integration enables earlier vulnerability discovery and scalable reasoning across large codebases, it also complicates accountability, traceability, and governance. Empirical studies of human–AI collaboration show that developers often struggle to distinguish correct from flawed recommendations, particularly when explanations appear confident but are semantically incomplete [43,44]. These challenges raise fundamental questions about responsibility, auditability, and risk ownership in AI-augmented software engineering [45,46].

Consequently, explainability and auditability have emerged as first-order requirements for LLM deployment in security- and compliance-critical environments. Prior research explores explainable AI techniques for security, including structured reasoning, syntax-grounded explanations, and transparency mechanisms tailored to program analysis [47,48,49]. However, recent empirical evidence indicates that LLM-generated explanations can be misleading, incomplete, or internally inconsistent, particularly under adversarial or ambiguous conditions [50,51,52]. These limitations pose significant challenges for regulated domains that require verifiable evidence, reproducible audit trails, and formal assurance arguments, such as healthcare, finance, and critical infrastructure [53,54].

Figure 1 illustrates the rapid growth of research at the intersection of LLMs and software engineering since 2020. Although the overall publication volume has increased sharply, security-focused studies remain comparatively fewer and methodologically heterogeneous, spanning peer-reviewed empirical work, controlled experiments, and early-stage preprints. This heterogeneity complicates synthesis and underscores the need for a structured, evidence-driven assessment of current capabilities, limitations, and risks.

Motivated by these observations, this survey synthesizes existing research on LLMs in software security to clarify what is empirically established, where results diverge, and which challenges remain unresolved. The analysis is organized around three research questions:

• RQ1: How does the integration of LLMs alter traditional secure development practices across the Secure Development Lifecycle, including vulnerability detection, code review, DevSecOps, and threat modeling?
• RQ2: What latent security risks arise from widespread LLM usage, particularly regarding insecure code propagation, flawed automated repairs, and sensitive-data leakage?
• RQ3: How can explainability, traceability, and auditability of LLM-assisted security decisions be strengthened to satisfy assurance and regulatory requirements?

Rather than advocating fully autonomous use, this survey emphasizes governed, human-in-the-loop integration grounded in reproducible evaluation, curated datasets, and transparent oversight. Such an approach is essential for transitioning LLMs from promising research prototypes into trustworthy components of security-critical software engineering ecosystems.

2. Methodology

To ensure transparency and reproducibility, this review follows established guidelines for systematic literature reviews in software engineering (e.g., [55]) and adopts a protocol-based approach. The methodology includes a documented search strategy, explicit database selection (including major indexing services such as Scopus), a stepwise screening process, a verifiable protocol for deduplication and preprint handling, and a PRISMA-style flow diagram summarizing study selection (Figure 2).

2.1. Search Strategy and Databases

A structured search was conducted across six major bibliographic sources commonly used in empirical software engineering and security research:

• ACM Digital Library (conference and journal articles on software engineering, security, and programming languages).
• IEEE Xplore (software engineering, dependable systems, and security venues).
• SpringerLink (journals such as Empirical Software Engineeringand Software and Systems Modeling).
• ScienceDirect (Elsevier journals, including Journal of Systems and Software and High-Confidence Computing).
• Scopus (used as a broad multidisciplinary index to cross-check coverage and retrieve additional peer-reviewed articles).
• arXiv (preprints; treated as a supplementary source and explicitly distinguished from indexed venues).

The search covered the period from January 2020 to March 2025, capturing the emergence of transformer-based LLMs and their application to software security tasks such as vulnerability detection, secure code generation, and automated patching. Scopus was used to validate and extend coverage from ACM, IEEE, SpringerLink, and ScienceDirect; for example, running the finalized search string in Scopus alone returned more than 400 candidate documents, confirming that there is already a substantial peer-reviewed body of work on LLMs and software security.

Search Strings

Because terminology varies across studies, we designed compound Boolean queries around three conceptual groups: (1) large language models, (2) software security, and (3) code-related tasks (generation, analysis, or repair). The canonical search string (adapted to each database is syntax) was

(“large language model” OR “LLM” OR “code LLM” OR “foundation model”)

AND (“software security” OR “vulnerability” OR “secure coding”

     OR “security testing” OR “threat modeling”)

AND (“code generation” OR “automatic programming”

     OR “vulnerability detection” OR “patch generation”)

For Scopus and some publisher portals, controlled vocabulary terms (e.g., “software vulnerability”, “secure software engineering”) were additionally included where supported. We used AND between the LLM, security, and code-related concept groups to restrict results to studies that address all three dimensions, and OR within each group to capture lexical variation across venues and indexing systems.

2.2. Study Selection and Screening

The search results from all six sources were exported (in BibTeX or RIS where possible) and merged into a single reference library. Figure 2 summarizes the selection process using a PRISMA-style flow diagram customized for this review.

In total, the initial search yielded 1117 records before deduplication:

• ACM Digital Library: 132 records.
• IEEE Xplore: 155 records.
• SpringerLink: 102 records.
• ScienceDirect: 88 records.
• Scopus: 444 records.
• arXiv (preprints): 196 records.

2.2.1. Deduplication and Consistency Checks

We first removed exact and near-duplicate entries across databases using a combination of DOI matching, title and year matching, and fuzzy string similarity on titles (to account for minor formatting and capitalization differences). For records without DOIs (primarily some arXiv preprints and a small number of conference papers), we manually checked titles and authors to avoid double-counting preprint/published pairs.

This process removed 585 duplicate or near-duplicate records, leaving 532 unique records for screening. Where both an arXiv preprint and a peer-reviewed version existed (e.g., [1,2,3,56]), the peer-reviewed version was treated as the primary record, and the preprint was retained only as a pointer to the earlier version rather than as a separate study.

2.2.2. Title and Abstract Screening

Two reviewers independently screened the titles and abstracts of the 532 unique records against the inclusion and exclusion criteria (defined below). Disagreements were resolved through discussion, and a third reviewer was consulted in a small number of borderline cases (e.g., papers on general LLM evaluation with limited but non-zero security content).

During this phase, 367 records were excluded due to the following reasons:

• Not LLM-related or not focused on code (e.g., general NLP security, social media analysis): 241 records.
• No substantive security-relevant outcome (e.g., productivity-only studies of code completion): 126 records.

This left 165 records for full-text assessment.

2.3. Full-Text Assessment and Eligibility

Full texts of the 165 remaining records were retrieved via institutional access or open access sources. Twelve records could not be obtained in full text (e.g., inaccessible theses or missing proceedings) and were therefore excluded.

The remaining 153 full-text articles were assessed for eligibility. We applied the inclusion and exclusion criteria to the full text, focusing on whether the study (i) used an LLM or closely related model for a software engineering task, (ii) produced security-relevant outcomes (e.g., vulnerability detection rates, secure code generation quality, data leakage analysis), and (iii) reported empirical results (benchmarks, experiments, or case studies).

In this step, 84 articles were excluded for the following main reasons:

• No empirical evaluation (e.g., conceptual position papers, essays, or vision pieces): 46 articles.
• Insufficient security focus (e.g., developer productivity studies with only incidental mention of security): 26 articles.
• Incomplete or non-reproducible results (e.g., missing evaluation setup or datasets): 12 articles.

After these exclusions, 81 studies were included in the final synthesis.

2.4. Inclusion and Exclusion Criteria

2.4.1. Inclusion Criteria

A study was included if it met all of the following:

• Explicitly involved the use, evaluation, or analysis of an LLM (or a closely related large code model) in software engineering.
• Addressed security-relevant tasks such as vulnerability detection, patch generation, secure code generation, threat modeling, data leakage, or security/privacy assessment.
• Reported empirical evidence, such as quantitative benchmarks, experiments, or structured case studies (including industrial case reports).
• Was published in a peer-reviewed venue (e.g., [9,16,17,27,57]) or available as a preprint with sufficiently detailed methodology (e.g., [11,35,58,59,60]).

2.4.2. Exclusion Criteria

Studies were excluded if they

• Did not involve LLMs or large code models (e.g., traditional static or ML-only methods without comparison to LLMs).
• Focused on NLP or security topics unrelated to source code or software systems.
• Lacked empirical evaluation (conceptual essays, tutorials, or purely theoretical discussions).
• Provided no security-relevant outcomes (e.g., productivity-only evaluations of code completion tooling).
• Were duplicates, near-duplicates, or inaccessible in full text.

2.5. Treatment of Preprints and Indexed Versions

Because the field evolves rapidly, preprints form a substantial portion of early evidence. To maintain rigor,

• Peer-reviewed publications (journals, conferences, and workshops indexed in Scopus, ACM, IEEE, SpringerLink, or ScienceDirect) were prioritized in the narrative synthesis.
• When a preprint later appeared in a peer-reviewed venue, the published version was used as the primary record (e.g., [1,2,3]).
• Purely preprint studies were included only if they presented clear empirical methodology, datasets, and evaluation details (e.g., [35,37,56,58,59]).
• Non-academic sources (blog posts, vendor white papers, and ChatGPT-generated content) were not treated as primary evidence and were used only to contextualize tooling or practice when strictly necessary.

Overall, of the 81 included studies, 68 are peer-reviewed journal, conference, or workshop papers (e.g., [9,16,17,27,42,61,62,63]) and 13 are preprints or technical reports that provide up-to-date empirical results on emerging models and benchmarks.

2.6. Data Extraction and Coding

For each included study, we extracted the following information using a structured data extraction form:

• Bibliographic metadata (year, venue, publication type: journal, conference, workshop, preprint, or technical report).
• LLM or model family evaluated (e.g., GPT-3/3.5/4, Codex, Code Llama, LLaMA 2, DeepSeek-R1, CodeBERT, GraphCodeBERT).
• Security tasks and scenarios (e.g., vulnerability detection, secure code generation, patch synthesis, malware generation, data leakage analysis, threat modeling).
• Datasets and benchmarks used (e.g., DiverseVul [10], CVE-Bench [12], VulnLLMEval [11], proprietary industry datasets).
• Evaluation metrics (e.g., accuracy, precision, recall, F1, repair success rate, exploitability of generated code).
• Key findings, mapped to the three research questions RQ1–RQ3.

Two reviewers independently coded each paper; disagreements were resolved through discussion. Where necessary, we revisited the full text to clarify ambiguous categorizations (e.g., studies that mixed productivity and security outcomes).

2.7. Quality and Bias Assessment

To strengthen the robustness of the synthesis, we conducted a structured quality assessment for a representative subset of included studies. We followed standard SLR practices and assessed each study across three dimensions:

• Dataset Clarity and Transparency (DC): Are datasets, benchmarks, and pre-processing steps clearly described and, where possible, publicly available? (e.g., [10,11]).
• Baseline Strength (BS): Does the study compare LLMs against strong, realistic baselines (e.g., static analyzers, existing ML models, or human experts) rather than only trivial or outdated baselines [9,17]?
• Reproducibility and Reporting Quality (RT): Are prompts, hyperparameters, model versions, and evaluation protocols described in sufficient detail for replication (e.g., [27,63,64])?

Table 1. Summary of the quality assessment for five representative studies selected from the 81 papers reviewed in this work. Ratings: High, Moderate, Low. Criteria: DC = dataset clarity and availability; BS = baseline strength; RT = reproducibility and transparency.

No.	Study (Abbrev.)	DC	BS	RT	Ref.
1	Systematic literature review on AI models and code-generation security	High	Moderate	High	[17]
2	Assessing the security of GitHub (2023 version) Copilot’s generated code (replication study)	High	High	Moderate	[64]
3	DiverseVul (large-scale vulnerable code dataset)	High	Moderate	High	[10]
4	BadGPT (security vulnerabilities/backdoor attacks on LLMs)	Moderate	Low	Low	[37]
5	Cracks in The Stack (risks in The Stack v2 training dataset)	High	Moderate	Moderate	[34]

summarizes ratings for five representative studies chosen from the 81 included papers. Studies with low transparency or incomplete methodological reporting were flagged and treated cautiously in the synthesis. In our analysis, peer-reviewed studies with high DC and RT scores (e.g., [16,17,27]) formed the backbone of the argumentation, while preprints and lower-transparency studies were used primarily to highlight emerging trends or open issues.

2.8. Limitations of the Review Process

Several limitations should be acknowledged:

• Screening capacity. Screening and coding were carried out by a limited number of reviewers, which may introduce selection bias despite the use of a shared protocol and consensus discussions.
• Preprint variability. Preprints (especially from arXiv) vary in rigor. While they are important for tracking the rapidly evolving LLM landscape, conclusions based on preprints are treated as provisional.
• Heterogeneous benchmarks. Studies employ different datasets, vulnerability taxonomies, and metrics, which limits direct comparability of quantitative results across papers.
• Rapid model evolution. New model releases, fine-tuned variants, and updated APIs may outpace published evaluations, meaning that some empirical findings may become outdated quickly.

Despite these constraints, the review procedure, including the use of Scopus as a central index, cross-checking between databases and preprint servers, and explicit PRISMA-style tracking, provides a transparent and reproducible basis for synthesizing current knowledge on LLMs in secure software development.

3. Results and Discussion

This section presents the findings of our survey, structured around the three research questions (RQ1–RQ3). For each question, we synthesize evidence from recent literature, highlight trends and limitations, and connect them to the broader goals of secure software development. All figures and tables include explicit source citations, and interpretations avoid categorical claims where the underlying evidence is derived from heterogeneous or preliminary studies.

3.1. RQ1: LLM Integration and Autonomy in Secure Development

The integration of large language models (LLMs) into secure development workflows is reshaping how security tasks are performed across the Secure Development Lifecycle (SDL), DevSecOps pipelines, code review, and threat modeling. Traditional workflows have relied on static analysis, dynamic testing, and manual review, whereas LLMs introduce capabilities such as contextual reasoning, multi-language understanding, and rapid generalization across security patterns and vulnerability classes. Early code-focused LLMs like Codex, Code Llama, and LLaMA 2 showed that large models trained jointly on natural language and code can perform program synthesis, translation, and bug explanation [1,2,3]. Encoder-style models such as CodeBERT and GraphCodeBERT further improved semantic code understanding for tasks like vulnerability localization and code search [5,6]. Surveys and reviews position LLMs at the intersection of software engineering and security automation, while emphasizing uneven maturity across tasks, evaluation heterogeneity, and fragile generalization [14,15,16,17,30,57].

Figure 3 illustrates adoption and visibility of major LLM frameworks. Panel (a) highlights repository-level dependence, where foundational ML libraries dominate, while workflow/orchestration frameworks (e.g., LangChain, Transformers) show accelerating uptake as building blocks for LLM-driven coding assistants, retrieval-augmented generation (RAG) pipelines, and security triage tools [65]. This trend is consistent with evidence that LLM components are increasingly embedded into DevSecOps workflows to support prioritization, continuous assessment, and pipeline-level automation [16,40]. Niche retrieval frameworks remain smaller in scale but are relevant to security and compliance tasks involving policy lookup, evidence retrieval, and audit traceability. Panel (b) shows developer attention via GitHub “stars,” reinforcing that orchestration frameworks are becoming central to developer workflows and are likely to shape how secure SDL and code-review practices evolve.

Figure 4 indicates that vulnerability rates in LLM-generated code remain substantial and, for some tools/settings, appear to increase between 2023 and 2024. These observations align with multiple independent empirical evaluations of AI code assistants and code-generating models, including studies on GitHub Copilot and other tools that report recurring insecure patterns (e.g., injection, unsafe handling, and crypto misuse) even under security-oriented prompts [18,20,21,22]. At the same time, cross-study comparisons require caution because rates depend strongly on benchmark design, task selection, and model/tool versions [17,30].

A growing body of benchmarking work provides a more systematic picture of LLM performance on security-relevant tasks. Ullah et al. evaluate eight LLMs across 228 security scenarios using the SecLLMHolmes framework, finding that models handle simple vulnerability templates but exhibit non-deterministic responses and fail on complex multi-step reasoning, with even GPT-4 producing incorrect answers in 17% of cases from trivial variable renaming alone [63]. Tamberg and Bahsi show that LLM-based approaches can achieve strong recall for vulnerability detection but often suffer from false positives and inconsistent precision across languages [9]. These limitations are consistent with broader evidence on LLM reasoning failures and reliability gaps in complex settings [30,50]. Sultana et al. demonstrate that model rankings are highly sensitive to dataset design and labeling practices [67], while empirical studies observe gradual improvements across releases alongside persistent blind spots for complex vulnerabilities [26].

Empirical studies on code assistants reinforce this picture. Pearce et al. and Majdinasab et al. find that a non-trivial fraction of assistant-generated suggestions contain at least one security weakness, even when developers ask explicitly for secure code [27,64]. These findings complement earlier Copilot-focused evaluations and replications [18,20,22]. Studies of context and grounding further suggest that assistants can misunderstand project-specific constraints and cross-file context, which matters for secure code review and fixes [43]. User studies also indicate that developers may over-trust plausible-looking outputs, miss subtle flaws, and accept insecure suggestions at non-trivial rates [29,30,52]. Broader analyses of code generation quality similarly report that LLM outputs can introduce bugs and security-relevant defects across tasks and languages [23,24].

Beyond coding, LLMs are being explored for higher-level security activities. Sridhar et al. find that LLMs can assist in brainstorming attack paths and abuse cases, but often produce incomplete threat models that require expert curation [42]. This aligns with work that frames AI-driven threat modeling as promising but risk-prone, especially when organizations treat generated artifacts as complete or authoritative [41]. Lyu et al. argue that current models fall short of the guarantees needed for fully autonomous security-critical development, emphasizing hybrid pipelines with explicit human oversight [62]. In practice-oriented terms, the emerging consensus is that LLMs behave more like “junior reviewers” than autonomous security agents, providing useful drafts and hypotheses that must be checked systematically [30,44].

Dataset and supply-chain perspectives add another dimension. Reviews emphasize that vulnerabilities in training data, benchmark contamination, and inconsistent labeling can inflate apparent performance and weaken generalization claims [17,35]. Jahanshahi and Mockus identify vulnerabilities and licensing risks in large-scale pre-training corpora, raising both security and governance concerns for downstream systems [34]. Meanwhile, work on automated repair and neural program repair highlights that patch synthesis is difficult to evaluate robustly, and many settings omit constraints such as maintainability, test adequacy, and semantic preservation [68,69,70]. Overall, the evidence supports augmentation rather than autonomy as the dominant paradigm for LLMs in secure software development.

Table 2. LLM integration with security frameworks. Data sources: Sheng et al. (2025) [16]; Negri-Ribalta et al. (2024) [17]; Pearce et al. (2025) [27]; Majdinasab et al. (2024) [64]; Ullah et al. (2024) [63]; Deng et al.(2023) [42]; Jahanshahi and Mockus (2025) [34]; Zhou et al. (2025) [35]; Sultana et al. (2024) [67]; Bouzid and Khoury (2025) [71]; and Copilot security evaluations [18,20,22].

Framework/Practice	Tools Studied	Reported Effects	Limitations	Refs.
SDL and DevSecOps	GPT-4, Code Llama, security-tuned LLMs	Faster triage; cross-language reasoning; backlog summarization	High false positives; uneven precision; limited business-logic coverage	[16,17,40,63,67]
Code Review	Copilot (2023 version), ChatGPT (GPT-3.5/GPT-4, OpenAI, 2023), VulnLLMEval, assistant-style tools	Inline CWE detection; patch suggestions; natural language explanations	Insecure outputs persist; oversimplified or partial fixes; hallucinated rationales	[11,18,22,27,64,71]
Threat Modeling	GPT-4, Claude, other chat models	Attack-path sketches; misuse/abuse cases; STRIDE-style ideation	Limited depth; missing domain-specific threats; requires expert validation	[15,41,42,57]
Supply Chain and Datasets	The Stack v2, public code corpora	CVEs and licensing issues surfaced; data-quality analysis	Recurring vulnerable patterns; contamination and data leakage across benchmarks	[34,35,59]

summarizes LLM integration across key security frameworks and practices.

Table 3. LLM autonomy across secure development phases. Data sources: Sheng et al. (2025) [16]; Kumar (2024) [15]; Pearce et al. (2025) [27]; Majdinasab et al. (2024) [64]; Fu et al. (2025) [61]; Wang et al. (2025) [12]; Li et al. (2025) [68]; Fakih et al. (2025) [60]; Fu and Tantithamthavorn (2023) [26]; Lyu et al. (2025) [62]; Gao et al. (2024) [44]; Rzig et al.(2025) [54]; and repair and program–repair perspectives [69,70].

Lifecycle Phase	Tools Studied	Autonomy Level	Observed Gaps	Refs.
Requirements	Prompt-based assistants; RAG systems	Assistive only	Needs human-driven prioritization and scope definition	[15,16]
Design	Chat-based LLMs with security patterns	Assistive only	No formal guarantees for architectural correctness; limited threat-coverage	[41,42,44,62]
Coding	Copilot (2023 version), ChatGPT (GPT-3.5/GPT-4, OpenAI, 2023), other assistants	Partial automation	Insecure outputs persist; new vulnerabilities introduced; context gaps	[27,43,61,64]
Testing	VulnLLMEval, CVE-Bench, DiverseVul	Partial automation	Mislabeling of patched vs. vulnerable samples; benchmark leakage	[10,11,12,59]
Deployment and Repair	LLM-based auto-patch pipelines	Partial automation (∼10–30% CVEs fixed)	Repair success remains low; generalization limited; semantic correctness hard to guarantee	[60,68,69,70,72]
Maintenance	empirical vulnerability detection	Assistive	Vulnerabilities persist across releases; risk of accumulated technical debt	[26,28,54]

synthesizes autonomy across phases. Early phases benefit from drafting and pattern identification, but human validation remains indispensable [15,16,44]. During coding, partial automation is offset by insecure outputs, newly introduced weaknesses, and context/grounding failures [27,43,61,64]. Testing results show high recall but frequent misclassification and benchmark leakage [10,11,59]. Repair studies typically remain below 30% success under realistic constraints, consistent with broader program–repair surveys that emphasize semantic preservation and evaluation difficulty [12,60,68,69,70,72]. Maintenance studies indicate vulnerabilities can persist and accumulate as technical debt without systematic monitoring [26,28,54]. Overall, current evidence points to augmentation rather than autonomy as the dominant paradigm.

3.2. RQ2: Latent Security Risks of LLM Code Generation

While LLMs offer significant benefits for software development, they introduce latent risks stemming from probabilistic generation, exposure to uncurated training data, and susceptibility to adversarial manipulation. Across studies, models reproduce insecure, outdated, or unsafe patterns (e.g., weak cryptography, injection vulnerabilities, unsafe deserialization, and flawed validation) [17,57,73]. Empirical evaluations of GitHub Copilot and other AI tools repeatedly document insecure suggestions, including vulnerabilities that persist across prompts and languages [18,20,21,22,27]. More broadly, studies of LLM code generation report bugs and security defects that arise from shallow pattern-matching, limited constraint satisfaction, and fragile reasoning under context shifts [23,24,50].

Table 4. Mapping of vulnerability types in LLM-generated code to STRIDE threat categories. Data source: CVE-Bench (Wang et al. 2025) [12].

STRIDE Category	Vulnerability Type	Dataset/Benchmark	Avg. Occurrence (%)	Reference
Information Disclosure	SQL Injection	CVE-Bench	2.36%	[12]
Elevation of Privilege	Cross-Site Scripting (XSS)	CVE-Bench	20.24%	[12]
Denial of Service	HTTP Response Splitting	CVE-Bench	2.36%	[12]

maps common vulnerability classes in synthesized code to STRIDE categories. Although rates vary across benchmarks and tools, recurring web vulnerabilities (e.g., XSS) remain prominent, consistent with evidence that many assistants default to insecure patterns without explicit sanitization and context-aware constraints [12,17,30].

Beyond insecure code, LLMs pose risks related to data exposure and memorization. Figure 5 shows that while most outputs contain no leaks, identifiable information and secrets still appear in a small fraction of generations [17]. This aligns with prior work on training-data extraction, model inversion, and broader surveys of ML data leakage that show large models can unintentionally reveal sensitive or proprietary information under targeted prompting or adversarial interaction [31,32,33,74]. Analyses of pre-training corpora also identify vulnerable and mislicensed code as plausible sources for vulnerability propagation and accidental leakage [34,35,59].

Table 5. Detailed comparison of latent security risks in LLM-generated code. Data sources: Pearce et al. (2025) [27]; Fu et al. (2025) [61]; Tóth et al. (2024) [76]; Negri-Ribalta et al. (2024) [17]; Sandoval et al. (2023) [29]; Li et al. (2025) [68]; Jahanshahi and Mockus (2025) [34]; Cheng et al. (2025) [38]; Pa Pa et al (2023) [77]; Chong et al. (2024) [66]; Ji et al.(2023) [51]; Rzig et al.(2025) [54]; and Copilot security and misuse literature [18,20,22,36,39].

Risk Category	Examples Observed	Affected Tools/Models	Reported Impact	References
Insecure Code Generation	Weak cryptography; unsafe memory ops; insecure handlers; unsafe file I/O	Copilot, Codex, GPT-style tools	Security weaknesses persist across prompts and contexts; high variance across benchmarks	[18,21,22,27,61,76,78]
Propagation of Vulnerable Patterns	Hardcoded secrets; injection; unsafe file handling; insecure reuse	Codex, Copilot, chat assistants	Automation bias amplifies risk when suggestions are accepted quickly; insecure reuse at scale	[13,17,29,57]
Incomplete / Flawed Fixes	Superficial CWE patches; non-compiling or semantics-breaking repairs	VulnLLMEval, CVE-Bench, LLM4CVE	Limited semantic repair success on real CVEs; evaluation constraints matter	[11,12,60,68,69,70]
Data Leakage and Memorization	Secrets; credentials; verbatim code resurfacing	Models trained on large code corpora	Small but non-trivial leakage rates; extraction/inversion risks under targeted prompts	[17,31,32,33,34,74]
Adversarial Misuse	Prompt injection; jailbreaks; malicious code generation; hidden behaviors	Chat assistants, code completion tools	Prompt integrity can be subverted; backdoors and misuse scenarios remain plausible	[36,37,38,39,66,77]
Developer Over-Reliance and Technical Debt	Automation bias; reduced manual review; deferred refactoring	Assistant-centric workflows	Higher acceptance of insecure code; accumulation of long-term technical debt	[28,29,30,52,54]
Organizational and Governance Risks	Shadow AI usage; unclear data-handling policies; weak accountability	Enterprise deployments	Policy violations and unclear responsibility for security failures	[45,46,79]

provides a detailed comparison of latent security risks observed across studies, organized by risk category. These risks are especially acute in constrained environments such as low-end IoT devices, where baseline vulnerability rates are already high and patch deployment is difficult [75]. In such settings, LLM-introduced weaknesses may be costly or hard to remediate. Overall, the evidence indicates that latent risks stem from three principal sources: (i) replication and amplification of insecure code, (ii) incomplete or incorrect vulnerability fixes, and (iii) leakage or manipulation of sensitive content and system behavior. Socio-technical studies suggest that without explicit governance and careful human oversight, these risks can accumulate and manifest as persistent technical debt and chronic exposure to known vulnerability patterns [28,30,54].

3.3. RQ3: Explainability, Auditability, and Compliance

As LLMs become embedded in secure software development workflows, ensuring explainability, auditability, and regulatory compliance has emerged as a central challenge. Unlike traditional static and dynamic analysis tools, LLMs generate probabilistic outputs that may lack interpretable rationale or semantic traceability. This introduces barriers for developers, auditors, and regulators who must validate correctness, safety, and accountability. Reliability analyses and user studies show that LLMs can produce persuasive explanations that may be incomplete, inconsistent, or unfaithful to actual model behavior, especially under ambiguity or adversarial pressure [48,49,50,51].

Explainability techniques attempt to mitigate these issues. Ding et al survey interpretable AI approaches for secure software engineering, including rule-based rationales, localized highlighting of suspicious code regions, and templates aligned with CWE categories [48]. Systematic analyses of explainable AI highlight that effective explanations must go beyond surface-level interpretability, emphasizing properties such as faithfulness to the underlying model, robustness across inputs, and alignment with the decision context to ensure meaningful and trustworthy use [47]. However, empirical work also shows that explanation failure modes are common, including plausible but incorrect justifications and brittle chains of reasoning under prompt perturbations [50,51,52]. Structured prompting (e.g., self-checking and rule-aligned exemplars) can improve consistency, but remains sensitive to phrasing, context, and tool integration [30,57,58].

Compliance requirements amplify these challenges. In regulated sectors (e.g., HIPAA, ISO/IEC 27001, sector-specific regulation), outputs must be explainable and auditable. NIST’s SSDF-AI extension calls for AI-system-specific secure development practices, including documentation of model behavior, testing, and risk management for AI-enabled components [53]. More general governance frameworks for trustworthy AI emphasize verifiable claims, reproducible pipelines, and transparent documentation [46]. Work on governance and accountability highlights the organizational need for role clarity, audit trails, and responsibility assignment when LLMs influence security decisions [45]. These concerns align with evidence from DevSecOps integration research: adding AI components can increase pipeline complexity and blur accountability without explicit controls and monitoring [40].

Table 6. Summary of findings for RQ3: explainability and compliance of LLMs in secure development. Data sources: Ding et al (2022) [48]; Liu et al. (2024) [49]; Arrieta et al. (2020) [47]; Huang et al. (2023) [50]; Ji et al. (2023) [51]; NIST (2024) [53]; Brundage et al. (2020) [46]; Mökander et al. (2023) [45]; Yao et al. (2024) [57]; and Prompt Security Team (2025) [81].

Aspect	Approach/Technique	Observed Benefits	Limitations/Gaps	Refs.
Explainability Mechanisms	Rule-based rationales; security-focused highlighting; step-wise prompting	Improved interpretability; supports quicker identification of obvious flaws	Traces may be unfaithful; instability under prompt/context shifts	[48,49]
Compliance Alignment	Mapping to ISO/IEC 27001, HIPAA, NIST SSDF-AI	Supports traceability between outputs and controls	No standardized automated scoring; heavy reliance on expert review	[53,57]
Assurance and Evidence	Security arguments; test-and-review pipelines; continuous monitoring	Reusable evidence for audits and certifications	Limited benchmarks for AI-enabled assurance; hard to compare tools	[45,46]
Model Transparency Tools	Behavioral probes; differential prompts; token-level inspection	Supports sanity checks and localized vulnerability analysis	Overhead in large projects; no agreed metrics for transparency sufficiency	[49,50]
Governance and Organizational Controls	Secure SDLC extensions; policy/audit logs; controlled AI adoption	Mitigates shadow AI; clarifies accountability in safety-critical systems	Implementation overhead; requires sustained operational discipline	[45,46,81]

summarizes the key findings for RQ3 across explainability mechanisms, compliance alignment, and governance approaches. Adversarial research adds further concerns for explainability and auditability. Backdoor attacks show that hidden behaviors can be introduced into code-generating models and may not be detectable through surface-level explanations alone [36,37]. Prompt-injection and environment-manipulation attacks on LLM-based tools demonstrate that adversaries can corrupt the decision context itself, leading to subtle but dangerous modifications in code suggestions [38,39,66]. LLM-assisted malware generation studies and security-misuse analyses similarly highlight that guardrails and governance are necessary for safe deployment in realistic environments [39,77,80].

Concerns about repair quality reinforce the need for explainability and strong assurance. Figure 6 shows that automated repair success rates remain low across languages, consistent with repair-focused empirical studies and broader program–repair surveys that emphasize semantic correctness and evaluation rigor [12,17,60,68,69,70,72]. Even when models produce convincing rationales, explanations can mask incomplete or incorrect fixes, making human-in-the-loop review essential for safety-critical code [30,51].

Overall, achieving explainability and auditability for LLMs in secure software development will require a combination of interpretable-by-design techniques, robust evaluation datasets, explicit governance frameworks, and sustained human oversight [45,46,47,48,49,53]. Longitudinal and socio-technical evidence suggests that adoption in regulated environments depends on verifiable audit trails, reproducible evidence, and transparent role definitions for AI components within the secure development lifecycle [30,54,62]. Without improved oversight, LLM outputs risk becoming opaque artifacts that undermine both security and compliance.

Taken together, the results across RQ1–RQ3 show a heterogeneous but converging picture. LLMs can improve the speed and reach of secure development activities, yet they also introduce systematic vulnerabilities, data-governance issues, and explainability gaps that traditional tools do not fully address. Because findings are distributed across datasets, tasks, and evaluation setups, strong conclusions should not be drawn from any single study in isolation. The next section therefore provides an integrated synthesis highlighting cross-cutting themes, trade-offs, and open gaps that emerge when the evidence is considered holistically.

4. Our Findings

This section synthesizes the collective evidence across the reviewed literature to answer the three research questions (RQ1–RQ3) at a conceptual level, rather than reiterating task-level results or benchmark-specific observations already discussed in Section 3. In response to reviewer feedback, we deliberately avoid re-listing Secure Development Lifecycle (SDL) phases, DevSecOps components, or individual tools. Instead, we abstract recurring patterns, constraints, and tensions that cut across models, datasets, evaluation settings, and organizational contexts.

We organize the synthesis around five cross-cutting findings: (F1) how LLMs are actually used in secure development practice, (F2) where their benefits are most robust, (F3) where risks systematically arise, (F4) why current evaluation practice limits strong generalization, and (F5) why governance, explainability, and human oversight emerge as structural requirements rather than optional enhancements. Throughout, we prioritize peer-reviewed evidence, while incorporating preprints where they provide unique empirical insight, consistent with established SLR guidance [15,16,55,56,57,71].

4.1. F1: LLMs’ Function as High-Bandwidth Security Assistants Rather than Autonomous Engineers

Across studies addressing RQ1, a consistent and robust pattern emerges: LLMs are not replacing human security engineers or traditional assurance mechanisms, but instead function as high-bandwidth assistants embedded within existing socio-technical systems. Regardless of model family or deployment context, LLMs are integrated into IDEs, CI/CD pipelines, code review tooling, and security triage workflows, where they generate candidates, explanations, or hypotheses that are subsequently filtered and validated by humans.

This assistant-oriented role holds across multiple generations of models. Early large-scale code models demonstrated strong generative and explanatory capacity but lacked the determinism and guarantees required for autonomous security decision-making [1,2,3]. Representation-learning approaches such as CodeBERT and GraphCodeBERT improved vulnerability localization and semantic understanding, yet remained decision-support tools rather than replacements for expert judgment [5,6]. More recent reasoning-enhanced models, including DeepSeek-R1 and agentic pipelines, extend analysis across files and repositories, but empirical evaluations still situate them within assistive, not authoritative, roles [4,59,62].

Evidence from repository-level telemetry reinforces this interpretation. Dependency analysis (e.g., Figure 3) shows widespread adoption of LLM frameworks as infrastructural components rather than standalone decision engines, indicating integration into broader development and security ecosystems [65]. Empirical studies and safety-critical studies further show that organizations treating LLM outputs as authoritative tend to accumulate unresolved vulnerabilities and technical debt over time, whereas teams that frame LLMs as advisory tools achieve more stable outcomes [26,28,54]. Collectively, these findings suggest that the dominant and realistic role of LLMs today is that of *capability amplifiers*, not autonomous security engineers.

4.2. F2: Benefits Concentrate on Visibility, Coverage, and Cognitive Support

Across RQ1 and RQ3, the strongest and most reproducible benefits of LLM integration are not improvements in correctness guarantees, but expansions in visibility, coverage, and human cognitive support.

First, LLMs consistently improve the *breadth* of vulnerability detection. Across multiple datasets and benchmarks, LLM-based approaches achieve higher recall than traditional static analysis or classical ML models, particularly for medium-complexity vulnerabilities and cross-language patterns [9,10,16,17,67]. While precision varies substantially, high recall is often operationally acceptable when LLM outputs are treated as candidates for downstream triage rather than final verdicts [15,27,63]. This aligns with DevSecOps practices that emphasize early, wide-net detection combined with later-stage validation.

Second, LLMs provide cross-language and cross-layer abstraction that is difficult to replicate with rule-based tools. Joint natural language–code representations enable transfer of vulnerability concepts across programming languages, frameworks, and system layers [5,6]. Longitudinal studies indicate that this abstraction is particularly valuable in large, polyglot codebases, where security expertise is unevenly distributed [16]. Reasoning-oriented and agentic systems extend this capability to repository-level and multi-step analysis, though without eliminating the need for expert oversight [4,62].

Third, LLMs provide substantial value as cognitive scaffolding. By translating low-level code findings into natural-language explanations, mitigation suggestions, and references to standards, LLMs help developers understand and contextualize security issues [44,48]. Even when generated fixes are incomplete, accompanying explanations can improve developers’ mental models of vulnerability classes and defensive patterns [49,63,71]. This supports collaboration models in which humans retain decision authority while relying on LLMs to accelerate comprehension and exploration [44,54].

4.3. F3: Security Risks Are Structural and Persistent Across Models

Findings related to RQ2 converge on the conclusion that LLM-associated security risks are structural rather than accidental. Figure 7 aggregates latent risks reported across studies and illustrates that insecure-by-default code generation dominates observed failures, followed by propagation of vulnerable patterns and data-related risks.

Empirical evaluations consistently report that 25–40% of LLM-generated code contains at least one security weakness under typical usage conditions, with substantially higher rates under adversarial prompting [27,58,61,64,78]. Automated repair studies further show that LLM-generated patches frequently address surface symptoms while failing to preserve deeper semantic correctness or long-term maintainability [12,60,68,72]. These failure modes persist across model families and benchmark designs, indicating that they are not artifacts of specific implementations.

Training data plays a central role in these risks. Analyses of large pre-training corpora reveal embedded CVEs, outdated practices, and licensing violations that can be reproduced downstream [34,59]. This inheritance effect is especially concerning in long-lived systems such as IoT and embedded software, where remediation is costly or infeasible [75]. Adversarial research further demonstrates that prompt injection, jailbreaks, and backdoor attacks can manipulate LLM behavior without access to model weights [37,38]. Together, these findings indicate that LLM-related risks arise from probabilistic generation, dataset curation, and socio-technical deployment practices rather than isolated bugs.

4.4. F4: Evaluation Practices Limit the Strength of Generalization

A central meta-finding across RQ1 and RQ2 is that heterogeneous evaluation practices constrain how confidently results can be generalized to production settings. Although many studies follow established SLR principles, differences in search scope, inclusion criteria, datasets, and metrics yield partially incompatible evidence bases [15,16,17,55,57,71].

Empirical results span controlled lab tasks, real-world repositories, user studies, and proprietary datasets, often using incomparable success criteria [9,10,12,27,78]. Benchmark contamination and data leakage further inflate reported performance in some settings [34,35]. Empirical studies partially mitigate these issues by revealing how LLM usage reshapes security work over time, including automation bias and shifting review practices [26,28,54]. As a result, our synthesis treats single-benchmark claims as indicative rather than definitive.

4.5. F5: Governance, Explainability, and Human Oversight Are Structural Requirements

Across RQ3 and socio-technical analyses, governance and explainability emerge as first-order requirements for responsible LLM adoption. Because LLM outputs are probabilistic and persuasive, post hoc explanation and auditability are essential for detecting hallucinations, shallow fixes, and adversarial manipulation.

Work on interpretable AI demonstrates that structured explanations, syntax-grounded reasoning, and security-aligned narratives help developers interrogate LLM outputs, though they do not guarantee faithfulness [48,49]. Compliance-oriented approaches show how LLMs can support audit-ready documentation and multi-hop reasoning for regulatory frameworks, provided final judgments remain human [53]. Organizational case studies emphasize the need for access controls, logging, dataset governance, and explicit accountability structures [46,79,81].

Figure 8 synthesizes these governance trade-offs by framing LLM adoption as a balance between opportunity, risk, and unresolved gaps.

Synthesis Across F1–F5

Taken together, the evidence shows that LLMs offer substantial value by expanding visibility, accelerating reasoning, and supporting human understanding of security issues, while simultaneously introducing persistent risks rooted in probabilistic generation, dataset inheritance, and socio-technical deployment. The most robust use cases are those that treat LLMs as governed, explainable assistants with humans firmly in the loop. Fully autonomous, compliance-grade secure software engineering remains aspirational, and progress toward that goal depends as much on advances in evaluation methodology and governance as on improvements in model architecture or scale.

5. Future Work

The evidence synthesized in Section 3 and Section 4 indicates that LLMs are already reshaping security work, but not in a uniform or reliably “secure-by-default” way. Empirical evaluations of AI-assisted code generation repeatedly show that vulnerability rates can remain substantial across tools and tasks, and that security outcomes are sensitive to prompt framing, context, and developer behavior [18,19,20,21,22,23,24,25]. At the same time, studies on LLM-enabled detection and repair suggest promising gains in recall and cross-language transfer, yet expose persistent limits in multi-step reasoning, semantic faithfulness of explanations, and patch correctness [7,30,50,69]. In consequence, future research must focus on methodological rigor, longitudinal validation, and governance-aware system design, rather than treating security improvements as an automatic consequence of model scaling.

• Rigorously reproducible evaluation protocols, not one-off benchmarks. A recurring barrier to comparability is that many studies differ in dataset curation, vulnerability taxonomies, prompt templates, decoding parameters, and reporting granularity. Future research should adopt reproducible evaluation protocols with versioned model identifiers, recorded inference settings, and standardized reporting of context-window usage and toolchain dependencies. Replication should be treated as a first-class research product, especially for assistant tools whose behavior can shift across model updates [18,22,30]. In addition, evaluations should include robustness checks that vary prompt phrasing, codebase context, and language choice to capture the instability observed in grounded-context studies of LLM understanding [43,50].
• Human–AI collaboration as a measurable security control. Results across RQ1 and RQ2 suggest that the security impact of LLMs is mediated by human acceptance and review practices. Future studies should move beyond productivity metrics and explicitly measure security outcomes in realistic workflows: acceptance rates of insecure suggestions, time-to-detection of subtle flaws, and the effectiveness of structured review checkpoints. Controlled trials should test interventions such as uncertainty signaling, escalation triggers, and “secure-by-policy” templates that force developers to verify properties (e.g., input validation, authorization checks) before merging [30,40,51]. Particular attention is needed for automation bias and over-trust, which can be amplified by persuasive but incomplete explanations [50,51].
• Security-aware dataset governance and provenance at scale. Multiple strands of evidence indicate that training data composition and contamination shape both vulnerability propagation and leakage risk. Future work must establish dataset governance standards for code corpora: provenance tracking, license auditing, deduplication, vulnerability filtering, and continuous maintenance. This includes empirically validating whether removing known-vulnerable snippets and duplicated repositories reduces insecure-by-default generation without harming functional correctness. Broader ML security surveys highlight the need to treat leakage and memorization as structural risks when sensitive code is present in training or fine-tuning pipelines [32,33]. Dataset releases should provide security-centered documentation (e.g., known CVE contamination, duplication metrics, licensing risk flags) to enable reliable downstream evaluation [14,73].
• Semantics-first patch evaluation and program–repair integration. Repair remains a bottleneck: many generated patches are syntactically plausible yet semantically incomplete, especially for vulnerabilities that require cross-function invariants or precise state reasoning. Future research should integrate neural program repair insights with formal and test-based validation to evaluate patches for semantic correctness, exploitability reduction, and regression safety [69,70]. This implies moving beyond pass/fail compilation metrics toward property-based tests, differential testing, and proof-carrying evidence where feasible. Comparative studies should also benchmark hybrid pipelines that combine LLM patch proposals with deterministic analyzers and constraint solvers, aligning with broader secure-coding tool surveys that emphasize layered defenses [14,71].
• Robustness to adversarial manipulation, including backdoors and prompt injection. RQ2 and RQ3 findings show that adversarial manipulation is a realistic concern for code-generating models and agentic toolchains. Future work should systematically evaluate backdoor and data poisoning threats, including how malicious triggers survive instruction tuning and how they manifest in code suggestions [36,39]. Similarly, evaluations should cover prompt-injection and context-manipulation attacks in retrieval-augmented and tool-using assistants, where untrusted inputs can steer generation. Defensive research should develop and validate detection methods (behavioral probes, anomaly detection, provenance-based filtering) that operate under black-box constraints common in proprietary tools [39,45].
• Explainability that is faithful, auditable, and compliance-aligned. The field needs explanation mechanisms that are not only readable but also faithful and verifiable. Systematic reviews of explainable AI for security and empirical work on reasoning failures show that LLM explanations can diverge from actual model behavior, producing false confidence in fixes and threat assessments [47,50,51]. Future research should define evaluation criteria for explanation faithfulness in software security settings (e.g., alignment with data-flow evidence, consistency under perturbations, calibration to uncertainty). In parallel, compliance-driven contexts require traceability from model outputs to controls and evidence artifacts; governance-oriented work highlights accountability gaps if such traceability is absent [30,45].
• Threat modeling and DevSecOps integration with measured assurance outcomes. As LLMs are increasingly integrated into DevSecOps, future studies should test how AI affects threat modeling quality, security gate performance, and incident response readiness. AI-driven threat modeling offers opportunities (faster enumeration of attack surfaces) but also risks (shallow or biased coverage). Future work should benchmark LLM-assisted threat modeling against expert baselines and evaluate how outputs affect downstream security decisions [41]. DevSecOps-focused reviews suggest that integration success depends on process design and organizational controls, not only model capability [40].
• Governance, accountability, and socio-technical policy design. Ultimately, trustworthy deployment is constrained by governance: who is responsible for an AI-assisted change, what is logged, what is reviewed, and what is auditable. Governance and accountability challenges in LLMs remain open, particularly in enterprise settings where “shadow AI” usage, unclear data-handling policies, and vendor opacity can undermine assurance [30,45]. Future research should propose and empirically evaluate governance patterns (policy-as-code for AI usage, audit logs for AI-assisted diffs, model-risk registers, and red-team protocols) that can be adopted without prohibitive overhead.

Across these directions, a consistent implication emerges: progress toward trustworthy LLM-assisted secure development will depend as much on methodology, evaluation discipline, and governance engineering as on model architecture improvements. The most valuable near-term advances are likely to come from reproducible, longitudinal studies and verifiable hybrid pipelines that explicitly measure security outcomes and organizational assurance, rather than from isolated demonstrations on narrow benchmarks [18,30,73].

6. Conclusions

This survey synthesized the current state of research on large language models (LLMs) in software security, integrating evidence across the studies published between 2020 and 2025. We organized the review around three research questions: (RQ1) how LLM integration alters secure development practices across the lifecycle; (RQ2) which latent risks arise from widespread LLM usage; and (RQ3) how explainability, auditability, and governance can be strengthened to satisfy assurance and regulatory needs.

For RQ1, the literature converges on a practical characterization: LLMs presently function as high-bandwidth security assistants embedded in development workflows rather than autonomous security engineers. Studies examining AI-assisted coding tools and code-generation systems show that LLMs can accelerate security-relevant activities—including vulnerability triage, explanation, and candidate repair generation—and can generalize across languages more readily than many traditional approaches [7,14]. However, evidence also shows that these gains are uneven and highly context-sensitive: models often struggle with implicit invariants, cross-file dependencies, and multi-step reasoning, and they can misunderstand context even when provided with surrounding code [43,50]. Consequently, the strongest empirical support favors augmentation with structured oversight: LLM outputs are most defensible when layered with deterministic analysis, testing, and human review.

For RQ2, the surveyed evidence indicates that LLM-related security risks are systematic rather than incidental. Multiple independent empirical studies report that AI-based code assistants frequently produce insecure patterns (e.g., injection vulnerabilities, unsafe validation, weak cryptographic choices), including when developers explicitly request secure implementations [18,19,20,21,22,25]. Complementary work shows that LLM-generated code can introduce bugs with security implications and that the correctness of generated fixes is often fragile, particularly under real-world constraints [23,24,69]. Beyond code quality, broader ML security research highlights the risks of memorization, leakage, and inversion, which become especially salient when proprietary or sensitive code appears in training or fine-tuning pipelines [32,33]. Adversarial research further shows that code-generating models can be manipulated through backdoors and other attacks, reinforcing that robustness must be evaluated against adaptive threats and not only benign prompting [36,39].

For RQ3, explainability and auditability emerge as decisive barriers to high-assurance deployment. Although explainable AI for security is an active area, empirical studies emphasize that LLM explanations can be persuasive yet unfaithful, inconsistent, or incomplete, which can amplify automation bias and create false assurance in security decisions [47,50,51]. This limitation matters most in compliance-driven environments where outputs must be traceable to evidence and controls. Governance-oriented work highlights that accountability gaps can widen when LLMs are embedded into pipelines without clear logging, review policies, and responsibility boundaries [30,45]. Similarly, AI-driven threat modeling illustrates both opportunity (expanded enumeration) and risk (shallow coverage) unless outputs are systematically validated and owned by accountable roles [41].

Taken together, the surveyed literature supports a balanced conclusion: LLMs can materially improve the reach and speed of security work, but they also introduce structural risks that require deliberate controls. A credible path toward trustworthy integration demands (1) reproducible and contamination-aware evaluation frameworks, (2) curated and ethically governed datasets with provenance and leakage safeguards, (3) explanation mechanisms evaluated for faithfulness and audit utility, and (4) governance structures that define accountability, monitoring, and safe operational boundaries [30,33,45,55,73]. As the field matures, progress should be assessed not only by model capability, but by the degree to which LLM-assisted security workflows produce verifiable, longitudinally stable improvements in real systems under realistic organizational and adversarial conditions [39,40].