Security-Augmented Use Case Flow Refinement for LLM-Based Agentic Systems

Abstract

Unlike conventional software systems and large language models (LLMs) evaluated in isolation, LLM-based agentic systems introduce a compositional, workflow-level attack surface. Existing security analysis approaches provide limited support for identifying security omissions implicit in functional use case flows and deriving corresponding flow refinements for direct incorporation into use case specifications. We formulate the Security-Augmented Use Case Flow Refinement Task and propose MA-SAFR, a multi-agent framework that coordinates Red Team, Blue Team, and Critic agents for threat anchoring, security-branch construction, and validation. EvoGraphRAG adapts relation-aware retrieval through validation feedback, while Risk-Specific Security Guidance (RSSG) provides risk- and role-specific constraints. To support systematic evaluation of this task, we construct SAFR-Bench from OWASP and MITRE ATLAS, comprising 157 cases and 235 source-grounded threat–refinement pairs. Across three generation models, MA-SAFR improves all three metrics on average compared with the strongest corresponding baselines; pipeline recall and pipeline precision increase by 63.44% and 54.96%, respectively. Controlled analyses show that RSSG and feedback-based graph evolution within gEvoGraphRAG improve downstream threat-to-defense conversion. Despite remaining challenges in implicit-threat identification, multi-threat coverage, and defense alignment, the results show that MA-SAFR can support the construction of valid security-augmented use case flows.

1. Introduction

LLM-based agentic systems are increasingly being deployed in real-world application settings, where they process user-provided information, support decisions, and interact with external tools or services [1]. Unlike conventional software systems, whose security-relevant operations are largely governed by developer-defined control logic, or LLMs evaluated in isolation, whose security is typically assessed at the model input–output boundary, agentic systems introduce an additional compositional, workflow-level attack surface. Specifically, model outputs can become intermediate control signals that select tools, supply arguments, retrieve external knowledge, update memory, or change application state [2,3,4]. This coupling among model behavior, potentially untrusted information, evolving state, delegated permissions, and tool-mediated execution allows malicious instructions or erroneous decisions to cross component boundaries and propagate through tool calls, data updates, and downstream business actions [5,6]. These risks therefore need to be analyzed in relation to the workflow steps where they are introduced, triggered, propagated, controlled, or recovered from [7,8].

Many of these workflow-level risks can be traced to system behaviors captured during requirements analysis, often through use case modeling, which organizes actor–system interactions into structured functional flows [9,10,11]. These flow steps not only describe normal business behavior but may also leave security-relevant assumptions unstated and potential risk entry points unaddressed. Existing approaches to risk analysis, security requirements analysis, and AI safety evaluation can identify relevant risks and security needs from different perspectives [5,12,13,14,15]. However, their outputs are usually threat lists, misuse cases, standalone security requirements, mitigation suggestions, test reports, or expert analyses rather than flow-level refinements that can be directly incorporated into existing use case specifications. Engineers therefore still need to manually determine whether a risk is grounded in the functional flow, where it is introduced or triggered, where the defense should be inserted, whether the defense covers the threat mechanism, and how the workflow should recover, retry, or terminate after the defense is triggered. As a result, traceability between identified threats and concrete use case flow refinements remains limited, thereby providing insufficient guidance for downstream design, testing, and implementation [16,17].

To close this gap, we formulate the Security-Augmented Use Case Flow Refinement Task, which identifies security omissions implicit in functional use case flows and completes the missing security behavior as security-augmented refinements within the original use case flow structure. The task focuses on an end-to-end transformation from functional behavior to security handling logic: each security omission is grounded at a concrete flow step, and the corresponding preventive and failure-handling behavior is incorporated into the original flow while preserving the original functional intent. To address this task, we propose MA-SAFR, a multi-agent framework that analyzes functional use case flows to identify grounded threats and generate corresponding flow-level security refinements, thereby producing traceable, reviewable, and actionable security-augmented use case flows that can be directly incorporated into existing use case specifications. Given a functional use case flow, MA-SAFR uses a Red Team Agent to identify security threats grounded in the flow and anchor them to flow steps where the risks are introduced or triggered. A Blue Team Agent translates the corresponding defenses into Security Branch Points (SBPs) and Security-augmented Alternative Flows (SAFs). A Critic Agent validates whether the generated threat/refinement pairs satisfy five quality dimensions needed for flow-level refinement: threat validity, defense coverage, threat-defense traceability, branch closure, and flow consistency. To support this agent workflow, we design a security-oriented and flow-aware knowledge support mechanism comprising EvoGraphRAG and a Risk-Specific Security Guidance (RSSG) layer. EvoGraphRAG organizes external security knowledge as a graph and adapts retrieval through edge-weight updates based on Critic-derived validation feedback, while RSSG provides risk- and role-specific generation constraints for threat anchoring and SBP/SAF construction.

Based on this formulation, we investigate the following problem-oriented research questions:

• RQ1: How effectively can missing security behavior be identified and incorporated into functional use case flows of LLM-based agentic systems?
• RQ2: How does risk-specific security guidance affect threat identification and security refinement within functional use case flows?
• RQ3: How does feedback-driven adaptation of security knowledge retrieval affect the quality of security-augmented use case flow refinements?

To support systematic evaluation of the Security-Augmented Use Case Flow Refinement Task, we construct SAFR-Bench from OWASP and MITRE ATLAS, comprising 157 cases and 235 source-grounded threat–refinement pairs. Using this benchmark, we evaluate MA-SAFR across three generation models and compare it with single-agent, no-retrieval, and vanilla-RAG baselines. The results show that, on average, MA-SAFR improves upon the strongest baseline in threat-validity recall, pipeline recall, and pipeline precision, with relative gains of 63.44% and 54.96% in the latter two metrics, respectively. Further controlled analyses examine how RSSG and feedback-based edge-weight updates in EvoGraphRAG affect $SBP$/$SAF$ refinement quality, while a parameter sensitivity analysis assesses the edge-weight update configuration. External qualitative case studies provide preliminary evidence of applicability to medical and software-development flows beyond SAFR-Bench, while a computational-cost analysis reports online LLM-generation cost in terms of token consumption and observed generation time. Finally, our failure analysis identifies three remaining bottlenecks: threat identification failures, incomplete coverage of security omissions in multi-threat cases, and semantic misalignment among identified threats, generated $SBP$/$SAF$ refinements, and the original functional flow.

The main contributions of this paper are as follows:

• We formulate the Security-Augmented Use Case Flow Refinement Task, which transforms functional use case flows into security-augmented use case flows, and construct SAFR-Bench, which contains 157 cases and 235 source-grounded threat records and corresponding security-augmented alternative flows, supporting evaluation of threat discovery and anchoring, and flow refinement.
• We propose MA-SAFR, a multi-agent framework that decomposes security-augmented flow refinement into three coupled reasoning stages: threat anchoring, defense-branch generation, and quality validation through Red Team, Blue Team, and Critic agents.
• We propose a security-oriented and flow-aware knowledge support mechanism that combines EvoGraphRAG with a Risk-Specific Security Guidance (RSSG) layer. EvoGraphRAG retrieves relation-aware risk–attack–mitigation evidence and adapts retrieval preferences through feedback-based graph evolution, while RSSG provides risk- and role-specific guidance for grounding threats and constructing flow-consistent SBP/SAF refinements.

The paper is structured as follows: Section 2 reviews related work and positions our study within the existing literature. Section 3 formalizes the representation of use case flows and defines the Security-Augmented Use Case Flow Refinement Task. Section 4 presents MA-SAFR details, including the Red Team, Blue Team, and Critic agents, EvoGraphRAG, and the RSSG layer. Section 5 describes SAFR-Bench, the evaluation metrics, research questions, baseline settings, and experimental setup. Section 6 presents the comparative results and component-level analyses of RSSG and feedback-based edge-weight updates, examines parameter sensitivity, computational cost and scalability considerations, evaluates the reliability of LLM-based judging, and reports the diagnostic failure analysis and external qualitative case studies. Section 7 discusses potential internal, external, and construct threats to our study. Section 8 concludes the paper and outlines future directions.

2. Related Work

2.1. Security Requirements Analysis and Refinement

Security requirements analysis and refinement aim to identify threats in functional specifications and derive corresponding security requirements [18]. Threat-oriented methods identify and categorize threats before eliciting and validating security requirements [19]; model-driven approaches detect STRIDE threats and recommend protective requirements [20]; use-case-driven methods further derive misuse cases, security use cases, and mitigation schemes from functional scenarios [13].

Recent studies have applied LLMs to security requirements analysis, including eliciting threats from system descriptions [21] and deriving security requirements from functional specifications under the guidance of security standards [14]. Multi-agent approaches have also been explored to identify inclusivity-related concerns and refine security requirements [22].

The conventional and LLM-based approaches reviewed above primarily produce threat models, mitigation schemes, or standalone security requirements; the process of anchoring a security omission to a functional-flow step and translating the corresponding defense into a validated handling branch remains largely analyst-driven. To bridge this gap, MA-SAFR treats security-oriented use-case refinement as an end-to-end, verifiable transformation from functional behavior to security handling logic rather than as isolated artifact generation. Its Red Team Agent identifies and grounds each omission at a concrete flow step, while its Blue Team Agent converts the resulting threat record into an SBP/SAF pair that encodes the security check, trigger condition, handling actions, and closure within the original flow. The Critic Agent then validates the complete transformation for threat validity, defense coverage, threat–defense traceability, branch closure, and flow consistency.

2.2. Reflective and Feedback-Driven LLM Agent Systems

Reflection and iterative feedback have been widely explored as mechanisms for improving LLM agent behavior without updating model parameters. Reflexion converts scalar or free-form task feedback into verbal reflections and stores them in episodic memory to guide subsequent trials [23]. Self-Refine instead uses the same LLM to generate feedback on its current output and iteratively revise that output [24], while CRITIC grounds the verification–correction process in feedback obtained through interactions with external tools [25]. Together, these methods show how memory, self-evaluation, and tool-supported critique can support feedback-driven improvement.

Feedback-driven coordination has also been studied in role-specialized multi-agent systems for engineering tasks. MetaGPT encodes standardized operating procedures into a software-engineering workflow, allowing agents with distinct roles to produce and verify intermediate artifacts [26]. ChatDev similarly organizes specialized agents across software design, coding, and testing through multi-turn communication [27]. In circuit design, AutoSizer coordinates agents for circuit understanding, adaptive search-space construction, and optimization orchestration, and uses simulation feedback in a two-loop process to revise variable priorities and parameter ranges [28]. These systems demonstrate the usefulness of role specialization and iterative evaluation across engineering domains.

MA-SAFR does not introduce a fundamentally new feedback paradigm. Instead, it provides a security-requirements-specific instantiation in which Critic judgments over threat and SBP/SAF tuples are mapped to topology-preserving edge-weight updates in a source-grounded security knowledge graph. The resulting persistent preferences guide subsequent role-specific retrieval, prioritizing risk and attack evidence for the Red Team and threat-linked mitigation paths for the Blue Team without introducing new security knowledge.

2.3. Security Benchmarks for LLMs and Agentic Systems

Relevant security benchmarks can be broadly grouped by evaluation target into model-level and agent-level benchmarks. Model-level safety evaluation has focused on harmful generation, trustworthiness, and adversarial robustness. RealToxicityPrompts [29] supports the evaluation of toxic degeneration, while ToxiGen [30] provides adversarially generated examples for implicit hate-speech detection. Broader frameworks such as DecodingTrust [31] assess multiple trustworthiness dimensions, whereas HarmBench [32] standardizes automated red teaming and robust-refusal evaluation. These benchmarks primarily treat individual inputs and model responses as the evaluation unit and report toxicity, harmfulness, refusal or compliance, and attack success.

Agent-level benchmarks extend evaluation to multi-step trajectories involving external data, tools, and environment state. ToolEmu [7] examines risky tool use in LM-emulated sandboxes, AgentDojo [5] evaluates utility and security under indirect prompt injection, and Agent Security Bench (ASB) [8] systematizes attacks and defenses across agent prompts, tools, and memory. They typically evaluate agent behavior in terms of task completion, attack success, unsafe tool calls, policy violations, and sensitive-information leakage.

Although these benchmarks cover model outputs and agent behavior, they provide limited support for evaluating the transformation of security omissions implicit in functional use case flows into requirements-level flow refinements. To address this gap, SAFR-Bench organizes each case as a layered structure that links a requirement description and a functional use case flow to source-grounded threat records and corresponding SBP/SAF refinements. This structure supports evaluation of threat discovery and anchoring, defense-branch generation, and end-to-end refinement. OWASP LLM Top 10 [33] and MITRE ATLAS [34] provide the security taxonomies and source evidence used to construct these cases. Accordingly, SAFR-Bench complements model- and agent-level benchmarks by treating traceable requirements-level flow refinement as a distinct evaluation target.

2.4. Graph-Based and Adaptive RAG for Security Reasoning

Retrieval-augmented generation (RAG) augments language-model generation with external evidence retrieved from a non-parametric corpus [35]. In chunk-based RAG, source documents are segmented into independently indexed text units that are ranked by semantic relevance and supplied as generation context [35,36]. Although this representation provides domain-specific evidence, it leaves relations among security risks, attacks, mitigations, and source evidence implicit. In addition, a fixed retrieve-then-generate policy cannot adjust when or how retrieval is performed in response to query complexity or generation-time signals. These limitations motivate two complementary directions: representing external knowledge relationally and adapting retrieval to changing evidence needs.

Graph-based RAG addresses the first direction by representing external knowledge as entities and relations and retrieving relation-connected subgraphs or graph-guided chunk neighborhoods. G-Retriever [37] and KG2RAG [36] demonstrate how graph structure can support subgraph retrieval and relation-aware organization of evidence. In cybersecurity, knowledge graphs are also used to integrate heterogeneous threat intelligence and support situational awareness, attack-path analysis, and knowledge inference [38]. Together, these studies show how explicit graph structure supports relation-aware retrieval and why it is suitable for security knowledge that links risks, attacks, mitigations, and source evidence.

Adaptive and self-reflective RAG methods address the second direction by adjusting retrieval and generation in response to query- and generation-time signals. FLARE [39] retrieves additional evidence when upcoming content is uncertain, Adaptive-RAG [40] selects retrieval strategies according to query complexity, and Self-RAG [41] uses reflection tokens to assess retrieved evidence and generated outputs. However, these representative methods adapt retrieval primarily within the current inference process; they do not use downstream task validation to persistently update retrieval preferences for subsequent queries.

Building on both directions, EvoGraphRAG combines role-specific, relation-aware graph retrieval with Critic-derived validation feedback for security-oriented use case refinement. The Red Team Agent retrieves threat-oriented graph neighborhoods, whereas the Blue Team Agent follows paths from anchored threats to candidate mitigations. Validation feedback updates graph edge weights, thereby adapting retrieval preferences for later retrieval rounds. Complementing this retrieval mechanism, RSSG supplies risk- and role-specific constraints for mapping retrieved evidence to flow-grounded threat anchors and SBP/SAF refinements. Together, EvoGraphRAG and RSSG form the security-oriented and flow-aware knowledge support mechanism that links retrieval, refinement, and validation.

3. Task Formulation

3.1. Use Case Flow Representation

Use case modeling is widely used to capture interaction scenarios between actors and a target system in functional requirements and organize them into action flows. A use case specification $UCS$ usually contains one basic flow ($BF$) and zero or more alternative flows, denoted as the set $\mathcal{AF}$:

$$UCS=\langle BF,\mathcal{AF}\rangle .$$

(1)

The $BF$ describes the main success scenario in which the actor and the system interact along the normal path to achieve the use case goal, which is represented as an ordered sequence of basic flow steps $b_i$:

$$BF=\langle b_1,b_2,\dots ,b_n\rangle .$$

(2)

Each alternative flow $A{F}^{b{p}_k}\in \mathcal{AF}$ describes a deviating or exceptional path guided by a branch point $b{p}_k\in \mathcal{BP}$, where $\mathcal{BP}$ is the set of branch points and $b{p}_k$ is a flow step located in either the $BF$ or another alternative flow. The set of alternative flows is represented as

$$\mathcal{AF}=\left\{A{F}^{b{p}_k}=〈a_1^{b{p}_k},a_2^{b{p}_k},\dots ,a_r^{b{p}_k}〉\mid b{p}_k\in \mathcal{BP}\right\},$$

(3)

where $\langle a_1^{b{p}_k},a_2^{b{p}_k},\dots ,a_r^{b{p}_k}\rangle$ is the ordered sequence of steps.

3.2. Security-Augmented Use Case Flow Refinement Task

We define the Security-Augmented Use Case Flow Refinement Task as identifying flow-level security omissions in a use case specification and completing the missing security behavior as security-augmented refinements within the original use case flow structure. The goal is not to introduce a separate flow representation, but to make implicit or missing security behavior explicit while preserving the functional intent and structural conventions of the original use case.

To characterize how missing security behavior is completed within the use case flow structure, we introduce two role labels: Security Branch Point ($SBP$) and Security-augmented Alternative Flow ($SAF$). A security branch point $sb{p}_l$ is inserted at an anchored position in the existing use case flow:

$$sb{p}_l=\langle ancho{r}_l,ste{p}_l\rangle ,$$

(4)

where $ancho{r}_l$ indicates the insertion position and $ste{p}_l$ denotes the inserted security check or control. Similar to a branch point $b{p}_k$ that guides $A{F}^{b{p}_k}$, an inserted security branch point $sb{p}_l$ guides the corresponding security-augmented alternative flow $SA{F}^{sb{p}_l}$:

$$SA{F}^{sb{p}_l}=〈s{a}_1^{sb{p}_l},s{a}_2^{sb{p}_l},\dots ,s{a}_q^{sb{p}_l}〉,$$

(5)

where $\langle s{a}_1^{sb{p}_l},s{a}_2^{sb{p}_l},\dots ,s{a}_q^{sb{p}_l}\rangle$ is the sequence of security-handling steps.

The output of the Security-Augmented Use Case Flow Refinement Task is:

$$UC{S}^{+}=\langle B{F}^{+},{\mathcal{AF}}^{+}\rangle ,$$

(6)

where $B{F}^{+}$ and ${\mathcal{AF}}^{+}$ denote the updated basic flow and alternative-flow set after inserting $SBPs$ and adding the corresponding $SAFs$.

Figure 1 illustrates the refinement structure with a simplified benchmark-derived example. The task preserves the original functional flow while inserting $SBPs$ at anchored positions in the flow and adding corresponding $SAFs$ to handle detected security conditions.

4. Approach

4.1. Multi-Agent Security-Augmented Flow Refinement (MA-SAFR)

As shown in Figure 2, MA-SAFR addresses the Security-Augmented Use Case Flow Refinement Task by decomposing it into three distinct but coupled reasoning objectives. First, a refinement approach must inspect the original use case flow and identify flow-level security omissions grounded in the existing use case behavior. Second, the identified omissions must be converted into concrete flow refinements by inserting security branch points and generating the corresponding security-augmented alternative flows. The first two objectives are tightly coupled: the identified omission determines where and why a refinement is needed, while the generated refinement must mitigate the threat implied by the omission and remain consistent with the anchored behavior and surrounding flow context. Potential failures in this coupling include ungrounded threats, defenses that do not cover the corresponding threat mechanisms, traceability gaps, incomplete branches, and modifications that change the original functional intent. These risks motivate the third objective: validation over threat validity, defense coverage, threat-defense traceability, branch closure, and flow consistency.

Motivated by this analysis, we instantiate MA-SAFR by assigning these objectives to three specialized agents: a Red Team Agent for security omission identification, a Blue Team Agent for security-augmented flow generation, and a Critic Agent for refinement validation.

4.1.1. Red Team Agent

Given an original use case specification, the Red Team Agent inspects the existing use case flows to identify security omissions and outputs a threat-record set $\mathcal{TR}=\{t{r}_1,t{r}_2,\dots ,t{r}_m\}$, which serves as an intermediate representation for passing anchored threat information to the downstream agents. Each threat record takes the following structured form:

$$t{r}_j=\langle ti{d}_j,ancho{r}_j,mec{h}_j,impac{t}_j\rangle ,$$

(7)

where $ti{d}_j$ is the threat identifier, $ancho{r}_j$ denotes the flow step where the omission is anchored, $mec{h}_j$ describes how the threat can arise from the anchored behavior, and $impac{t}_j$ describes the corresponding security consequence.

Table 1. Prompt structure of the Red Team Agent.

Component	Description
Task Specification	Defines the role of the Red Team Agent, the security-omission identification objective, and the strict threat-record output interface.
Input	Provides the original functional use case flow as the primary evidence for threat identification.
Retrieved Knowledge	Provides retrieved security evidence from the knowledge base to refine threat mechanisms and impacts.
RSSG Guidance	Supplies risk-specific rules for threat coverage, duplicate merging, and anchor-step selection (Section 4.3).

summarizes the prompt structure used by the Red Team Agent.

4.1.2. Blue Team Agent

Given the threat-record set $\mathcal{TR}$ produced by the Red Team Agent, the Blue Team Agent transforms each $t{r}_j\in \mathcal{TR}$ into an $SBP$/$SAF$ refinement pair. Specifically, it conditions generation on the anchored step $ancho{r}_j$, threat mechanism $mec{h}_j$, security impact $impac{t}_j$, the original use case flow, and retrieved mitigation knowledge, and produces an $sb{p}_j$ and a corresponding $SA{F}^{sb{p}_j}$ that mitigates the identified threat. The refinement pairs $\langle sb{p}_j,SA{F}^{sb{p}_j}\rangle$ are then integrated into the original flow to obtain $UC{S}^{+}$.

Table 2. Prompt structure of the Blue Team Agent.

Component	Description
Task Specification	Defines the role of the Blue Team Agent, the flow-completion objective, and the strict $SBP$/$SAF$ output interface.
Input	Provides the original functional use case flow together with one threat record, including its anchored step, threat mechanism, and security impact.
Retrieved Knowledge	Provides retrieved security evidence from the knowledge base to support mitigation selection and defense formulation.
RSSG Guidance	Supplies risk-specific rules for $SBP$ placement, $SAF$ trigger specification, security-handling logic, and branch closure (Section 4.3).

summarizes the prompt structure used by the Blue Team Agent.

4.1.3. Critic Agent

After the Red Team and Blue Team agents produce threat records and refinement pairs, the Critic Agent validates each tuple $\langle t{r}_j,sb{p}_j,SA{F}^{sb{p}_j}\rangle$ against the original use case flow. The validation follows five dimensions:

• Threat validity checks whether each identified omission corresponds to a security-relevant concern grounded in the original flow;
• Defense coverage checks whether the generated $SBP$/$SAF$ mitigates the threat mechanism described in the corresponding threat record;
• Threat–defense traceability checks whether the threat record, inserted $SBP$, and generated $SAF$ are semantically connected;
• Branch closure checks whether the $SAF$ forms a complete security-handling path with a well-defined continuation or termination;
• Flow consistency checks whether the added security behavior preserves the original functional intent of the use case.

The Critic Agent outputs a structured validation report containing an overall decision, dimension-level outcomes, and concise feedback for threats not grounded in the original flow, weak defenses, missing traceability, incomplete branches, or inconsistent flow modifications. Both the dimension-level outcomes and the overall decision are categorical rather than numerical.

Table 3. Prompt structure of the Critic Agent.

Component	Description
Task Specification	Defines the role of the Critic Agent, the refinement-validation objective, and the structured validation output interface.
Input	Provides the original functional use case flow, the threat records generated by the Red Team Agent, and the $SBP$/$SAF$ refinements generated by the Blue Team Agent.
Validation Criteria	Checks threat validity, defense coverage, threat-defense traceability, branch closure, and flow consistency, with concise reasons for detected issues.

summarizes the prompt structure used by the Critic Agent.

The Critic Agent assesses the generated refinements, and its validation results are used to derive feedback signals for subsequent graph evolution rather than to filter, revise, or regenerate those refinements.

Using UC0125 in Figure 1 as a compact illustration, the Red Team Agent identifies embedded prompt instructions in ingested email content as a prompt-injection threat anchored at BF1. The Blue Team Agent inserts an $SBP$ after BF1 and generates an $SAF$ that quarantines the injected content, reloads trusted knowledge-base entries, and returns to BF1. The Critic Agent then checks whether the resulting threat–refinement tuple is grounded, covers the threat mechanism, remains traceable and flow-consistent, and forms a closed branch.

4.2. EvoGraphRAG

MA-SAFR relies on external security knowledge to support both threat identification and defense generation. When external security knowledge is indexed as ordinary text chunks, standard chunk-based RAG retrieves isolated snippets whose relevance is mainly determined by textual similarity. Although such snippets may mention related risks or mitigations, they do not explicitly preserve the relations among risks, attack patterns, techniques, mitigations, and source evidence. For the Security-Augmented Use Case Flow Refinement Task, this limitation is critical: the Red Team Agent needs risk and attack knowledge that can be anchored to specific flow behavior, while the Blue Team Agent needs mitigation knowledge that is traceable to the identified threat and can be translated into an $SBP$/$SAF$ refinement. Both needs require retrieval to preserve the relations between threat mechanisms, defense knowledge, and the flow context. Without relational retrieval, agents may select semantically similar but structurally irrelevant evidence, generate generic threat descriptions, or choose mitigations that are poorly aligned with the identified threat mechanism. EvoGraphRAG addresses this limitation by organizing security knowledge as a heterogeneous graph over risks, attack patterns, techniques, mitigations, and source evidence, and by retrieving role-specific graph neighborhoods for different agents. The graph structure makes risk–attack, attack–technique, risk–mitigation, and evidence-support relations explicit, while the evolutionary component updates edge weights from Critic Agent feedback so that knowledge paths leading to valid and traceable refinements are promoted and paths associated with threats not grounded in the original flow or weak defenses are down-weighted.

4.2.1. Graph Representation

EvoGraphRAG represents external security knowledge as a heterogeneous directed graph:

$$\mathcal{G}=(\mathcal{V},\mathcal{E},w),$$

(8)

where $\mathcal{V}$ is the set of security knowledge nodes, $\mathcal{E}$ is the set of typed directed edges, and $w:\mathcal{E}\to {\mathbb{R}}^{+}$ assigns a positive weight to each edge. Each node $v\in \mathcal{V}$ belongs to one of five types: Risk, AttackPattern, Technique, Mitigation, or SourceEvidence. A Risk node represents a security risk category, an AttackPattern node describes how a risk may be exploited in a concrete scenario, a Technique node captures a more specific adversarial technique, a Mitigation node represents a security control or defense strategy, and a SourceEvidence node preserves the source material supporting the corresponding knowledge item.

Edges encode the relations needed for flow-level security refinement. The relation $exploits$ connects an attack pattern to the risk it exploits, $belongs\text{\_}to$ connects a technique to its broader risk category, $implements$ connects a technique to the attack pattern it instantiates, $mitigated\text{\_}by$ connects risks, attack patterns, or techniques to candidate mitigations, and $supported\text{\_}by$ links graph nodes or relations to their source evidence. These relations allow EvoGraphRAG to retrieve not only semantically similar knowledge items, but also the surrounding risk–attack–mitigation evidence paths that explain why a knowledge item is relevant. The graph is constructed from external security knowledge sources and can be instantiated with different domain-specific security taxonomies, attack knowledge bases, and mitigation guidelines.

4.2.2. Role-Specific Retrieval and Reranking

Given the graph $\mathcal{G}$, EvoGraphRAG performs role-specific retrieval for the Red Team and Blue Team agents according to their different knowledge needs. For the Red Team Agent, the query $q_R$ is built from the original requirement text and the functional flow steps. The retriever searches over threat-oriented nodes, including Risk, AttackPattern, and Technique nodes, and then expands the initial hits to related risks, attack patterns, techniques, mitigations, and source evidence. The returned graph neighborhood supports threat naming, mechanism explanation, and flow-step anchoring.

For the Blue Team Agent, the query $q_B$ is conditioned on both the original flow and a specific threat record $t{r}_j$. It includes the anchored flow step, the threat mechanism, and the security impact. The retriever first identifies risk, attack-pattern, or technique nodes related to the threat, and then follows graph paths to candidate Mitigation nodes. The returned neighborhood is therefore mitigation-oriented and preserves the relation between the identified threat and the candidate defense knowledge used to generate the corresponding $SBP$/$SAF$ refinement.

For both roles, EvoGraphRAG reranks candidate graph items using a role-specific scoring function:

$$\begin{array}{cc}\hfill Scor{e}_{\rho }(c,q_{\rho })& ={\alpha }_{\rho }{s}_{sem}(c,q_{\rho })+{\beta }_{\rho }{s}_{ctx}(c,q_{\rho })\hfill \\ \hfill & +{\gamma }_{\rho }{s}_{graph}\left(c\right)+{\delta }_{\rho }{s}_{path}\left(c\right)\hfill \\ \hfill & +{\tau }_{\rho }{s}_{type}\left(c\right)+{\lambda }_{\rho }{s}_{src}\left(c\right)+{\mu }_{\rho }{s}_{evo}\left(c\right)\hfill \end{array}$$

(9)

where $\rho \in \{R,B\}$ denotes the retrieval role, c is a candidate graph item, and $q_{\rho }$ is the role-specific query. The term $s_{sem}$ measures semantic similarity, $s_{ctx}$ measures contextual fit with the use case flow or anchored threat context, $s_{graph}$ captures graph support from neighboring evidence, $s_{path}$ measures the relevance of graph paths to candidate mitigations, $s_{type}$ encodes role-specific node-type preference, $s_{src}$ reflects source priority, and $s_{evo}$ incorporates the current edge weights updated from validation feedback, as described in Section 4.2.3. By setting the weights $({\alpha }_{\rho },{\beta }_{\rho },{\gamma }_{\rho },{\delta }_{\rho },{\tau }_{\rho },{\lambda }_{\rho },{\mu }_{\rho })$ differently for the two roles, EvoGraphRAG produces distinct ranking preferences: Red Team retrieval prioritizes threat-relevant nodes and graph-supported risk evidence, whereas Blue Team retrieval prioritizes mitigation paths connected to the identified threat mechanism and fitted to the anchored flow context.

4.2.3. Feedback-Based Graph Evolution

The evolutionary component of EvoGraphRAG updates graph edge weights according to validation feedback. After the threat records and $SBP$/$SAF$ refinements have been generated, the Critic Agent evaluates each threat–refinement tuple along five dimensions: threat validity, defense coverage, threat–defense traceability, branch closure, and flow consistency. The validation outcome is then converted into feedback signals for the graph paths that were retrieved during generation. A knowledge path that contributes to a valid and traceable refinement receives positive feedback, whereas a path associated with a threat not grounded in the original flow, weak defense, missing traceability, incomplete branch, or inconsistent flow modification receives negative feedback.

Let ${\mathcal{F}}_t$ denote the set of feedback events collected at evolution round t. Each feedback event is associated with one or more retrieved graph edges and carries a signed reward derived from the validation outcome. For an edge $e\in \mathcal{E}$, EvoGraphRAG aggregates all feedback signals assigned to that edge into a clipped signal ${\tilde{s}}_t\left(e\right)$. The edge weight is then updated by a multiplicative rule:

$$w_{t+1}\left(e\right)=clip\left(w_t\left(e\right)·exp(\eta ·{\tilde{s}}_t\left(e\right)),w_{min},w_{max}\right),$$

(10)

where $\eta$ is the update rate, and $w_{min}$ and $w_{max}$ bound the updated edge weight. Positive feedback increases the weight of an edge, making graph paths containing that edge more likely to be retrieved in later rounds; negative feedback decreases the weight, reducing the retrieval priority of paths that repeatedly lead to invalid or weak refinements.

The graph evolution process does not rewrite node descriptions or add benchmark-derived knowledge to the graph; it only updates edge weights. Thus, EvoGraphRAG preserves the original graph structure while adapting retrieval preferences through edge-weight updates.

4.3. Risk-Specific Security Guidance Layer

Although EvoGraphRAG provides relationally structured security knowledge, it primarily determines what security knowledge is relevant to a use case flow. However, the output of the refinement task is not a standalone knowledge item or a set of explanatory sentences, but a structured refinement of the use case flow that introduces a context-dependent sequence of security actions. It must decide where the security concern emerges, where an $SBP$ should be inserted, and how the corresponding $SAF$ should organize detection, handling, and flow continuation while preserving basic control-flow continuity and data-dependency consistency. In addition, different risk types require different refinement strategies. Accordingly, MA-SAFR introduces a Risk-Specific Security Guidance (RSSG) layer that provides manually defined, risk- and role-specific guidance for mapping retrieved security evidence to flow-grounded threat anchors and structured $SBP$/$SAF$ refinements.

RSSG is organized by risk type and provides role-specific constraints for the Red Team and Blue Team agents. For the Red Team Agent, RSSG constrains threat identification and anchoring. It guides the agent to anchor a threat to the flow step where the unsafe behavior is introduced, triggered, or first becomes actionable. It also helps distinguish independent threats from duplicates by considering the anchor step, risk type, attack mechanism, exposed asset, and security impact. This improves the traceability between each threat record and the original use case flow, while reducing generic or duplicate threat generation.

For the Blue Team Agent, RSSG constrains how the identified threat is converted into a concrete flow refinement. It guides the agent to place the $SBP$ at the anchored position before the unsafe behavior causes harm. It also specifies the expected security-handling pattern of the corresponding $SAF$, such as flagging the unsafe condition, blocking or isolating the unsafe item or action, applying a risk-specific remediation, and returning to the protected flow or terminating safely. These constraints prevent the generated refinement from degenerating into generic logging, vague administrator review, or unrelated fallback behavior.

Table 4. Examples of RSSG guidance.

Risk Type	Red Team Guidance	Blue Team Guidance
Prompt injection	Anchor to the step where untrusted instructions, documents, prompts, or directives first enter the system.	Insert an $SBP$ before the model follows the untrusted content; the $SAF$ flags injected instructions, requires revision or removal, and returns to the protected step.
Sensitive information disclosure	Anchor to the step where sensitive data is read for an unauthorized context or crosses a trust boundary.	Insert an $SBP$ before disclosure; the $SAF$ redacts, minimizes, or restricts the sensitive data and resumes the protected flow when safe.
Excessive agency or tool misuse	Anchor to the step where the system invokes a tool, changes state, sends messages, or crosses a permission boundary.	Insert an $SBP$ before the high-risk action; the $SAF$ requires scoped authorization, constrains the action, or terminates the unsafe operation.

shows examples of RSSG guidance for representative risk types.

By making risk-specific anchoring and refinement rules explicit, RSSG complements EvoGraphRAG and bridges the gap between retrieved security knowledge and structured use-case-flow refinement. The full RSSG rule set used in the Red Team and Blue Team prompts is included in the public replication package.

5. Benchmark and Experiments

5.1. SAFR-Bench

We construct SAFR-Bench, a flow-level benchmark for evaluating methods on the Security-Augmented Use Case Flow Refinement Task. Unlike existing security datasets and benchmarks that primarily target toxic or unsafe model behavior [29,30,31], automated red-teaming or jailbreak test cases [15,32], cybersecurity capability evaluation [42], or prompt-injection attacks against tool-using agents [5], SAFR-Bench focuses on flow-level security refinement: it requires security concerns to be anchored to concrete use case steps and translated into security checks and security-handling branches. SAFR-Bench organizes each case into four layers: requirement description, functional use case flow, threat records, and security-augmented use case flow, and

Table 5. Schema of SAFR-Bench.

Layer	Artifact	Fields
Requirement Description	Case metadata and source text	use_case_id; dataset; source_knowledge_id; source_requirement_text
Functional Use Case Flow	Use case flow step	step_id; step_sentence; subject; verb; object; flow_from
Threat Records	Threat record	use_case_id; threat_id; threat_name; anchor_steps; threat_mechanism; security_impact; source_knowledge_id; source_evidence
Security-Augmented Use Case Flow	Security branch point (SBP)	step_id; anchor; step_sentence
	Security-augmented alternative flow (SAF)	saf_id; mitigates; entry_condition; source_evidence; SAF_steps
	SAF step	step_id; step_sentence

summarizes the key fields of these four layers. This four-layer design supports evaluation of threat anchoring, defense-branch generation, and end-to-end refinement. SAFR-Bench comprises 157 cases derived from OWASP and MITRE ATLAS sources, including 100 OWASP-derived and 57 MITRE-ATLAS-derived cases, and provides 235 threat records, each paired with a corresponding $SBP$/$SAF$ refinement.

Table 6. Overview of SAFR-Bench.

Statistic	Value
Number of cases	157
Source distribution	100 OWASP-derived cases and 57 MITRE-ATLAS-derived cases
Data split	80 training, 20 development, and 57 test cases
Use case flow steps	855 in total; 5.45 per case on average
Threat records	235 in total; 1.50 per case on average
$SBP$/$SAF$ refinement pairs	235 in total; 1.50 per case on average

summarizes the scale, source distribution, data split, and flow-refinement complexity of SAFR-Bench. To improve annotation quality, we randomly selected 60 cases for expert review by 5 engineers with 6–10 years of experience in requirements analysis or software development.

5.1.1. SAFR-Bench Construction

The construction of SAFR-Bench follows a pipeline aligned with the four-layer schema in which each annotation remains traceable to its source.

We first collect source materials from the OWASP Top 10 for Large Language Model Applications 2025 [33] and MITRE ATLAS v5.5.0 [34], two publicly maintained security knowledge resources that provide complementary coverage of recurring LLM application risks, example scenarios, and concrete adversarial techniques that can be mapped to requirement-level system behaviors. These materials span heterogeneous LLM application contexts and draw on community-curated risks, real-world attack observations, publicly documented incidents, and realistic security demonstrations, thereby providing source-level breadth across risk types, attack mechanisms, and application settings.

To approximate realistic requirements artifacts, where security risks are usually implicit in functional behavior rather than stated as attacks, we rewrite each source item or scenario into a concise requirement description for an LLM-enabled application. The rewritten description specifies the user goal, system boundary, and actors, while removing explicit attacker intent, attack labels, and mitigation hints.

Then, we construct the functional use case flow layer. Each rewritten description is parsed into an ordered use case flow, where each step describes one subject–verb–object unit and records its step identifier, action sentence, subject, verb, object, and predecessor link (flow_from). For the first step, this field is set to null. We do not inject security checks, controls, or security-handling branches into this functional-flow layer, so that the security-refinement task remains separated from the functional-flow construction step.

Next, we build threat records for each case. For OWASP-derived cases, the source risk category provides the risk category and evidence used to instantiate the threat record. For MITRE ATLAS-derived cases, we extract candidate threats from case procedures and associated techniques, then retain only threats that can be mapped to a flow step where the omission becomes actionable for detection, blocking, or constraint. Attacker-side preparation, staging, or reconnaissance activities are excluded unless they correspond to behavior that the target system can plausibly detect, block, or constrain within the functional flow. Each retained threat record contains a threat identifier, threat name, primary anchor step (anchor_steps), threat mechanism, security impact, source knowledge identifier (source_knowledge_id), and supporting evidence (source_evidence).

Finally, we construct the security-augmented use case flow layer by converting each threat record into a corresponding $SBP$/$SAF$ refinement pair. The $SBP$ is placed at the insertion position indicated by the threat anchor and expresses the security check that must occur before the protected behavior proceeds. The associated $SAF$ references the mitigated threat (mitigates), defines the entry condition (entry_condition) under which the security branch is triggered, and specifies the handling steps required to block, remediate, retry, return to the protected flow, or terminate safely. The resulting benchmark package includes the case registry, schema files, rewritten requirement descriptions, functional use case flows, threat records, and security-augmented use case flows.

5.1.2. Expert Review and Quality Control

To improve the reliability of SAFR-Bench, we conduct expert review on a subset of 60 randomly selected cases. The review is performed by five engineers with 6–10 years of experience in requirements analysis or software development. The reviewers examine the four benchmark layers jointly, and the review process is summarized as follows:

• Review coverage. The expert-reviewed subset contains 60 cases from SAFR-Bench and is checked by five experienced engineers.
• Review dimensions. The review covers requirement rewriting fidelity, functional-flow quality, threat validity, source grounding, anchor correctness, defense coverage, threat-defense traceability, branch closure, and flow consistency.
• Adjudication outputs. Cases with quality issues are revised at the corresponding layer, including rewritten requirement descriptions, corrected functional-flow steps, revised threat records, adjusted anchors, updated $SBP$ placement, rewritten $SAF$ steps, and branch-closure fixes.

These review dimensions are operationalized as layer-specific checks. For the requirement-description layer, reviewers check whether the rewritten description preserves the security-relevant functional affordances of the source scenario while removing explicit attacker intent, attack labels, and mitigation hints. For the functional use case flow layer, they check whether each step captures a coherent subject–verb–object action, whether the step order follows the rewritten requirement description, and whether the functional flow avoids injecting security controls that are not stated in the requirement. For the threat record layer, reviewers assess threat validity, source grounding, and anchor correctness, with the anchor required to identify the primary defense-actionable flow step. For the security-augmented use case flow layer, reviewers assess defense coverage, threat-defense traceability, branch closure, and flow consistency. In particular, each $SAF$ must be linked to the threat it mitigates, include an explicit entry condition triggered by the corresponding $SBP$, provide concrete handling steps, and define a valid branch closure.

Cases with identified quality issues are revised at the corresponding layer and then checked for cross-layer consistency. Anchor corrections update the associated threat records and downstream $SBP$ placement; defense revisions update the corresponding $SAF$ while preserving the original functional intent; and ambiguous but acceptable alternative anchors are recorded separately rather than replacing the canonical anchor. The resulting expert-reviewed subset provides a higher-confidence portion of SAFR-Bench for validating benchmark quality and supporting the evaluation reported in this study.

5.2. Evaluation Metrics

In the evaluation, we focus on three quality aspects of Security-Augmented Use Case Flow Refinement Task: identifying security omissions, converting the identified omissions into valid defenses, and avoiding invalid over-generation. Accordingly, we use threat-validity recall, pipeline recall, and pipeline precision.

Threat-validity recall. Threat-validity recall measures the proportion of reference threats that are correctly identified. A predicted threat is counted as valid only when it matches a reference threat with respect to both the anchor step and the security concern. More specifically, the security-concern match is judged in the context of the functional flow based on the threat mechanism, affected target, and security consequence.

Let $T{P}_i^T$ denote the number of predicted threats in case i that are matched to reference threat records, and let $F{N}_i^T$ denote the number of reference threats in case i that are missed. Threat-validity recall is computed as

$$R_{\mathrm{threat}}={\displaystyle \frac{{\sum }_{i}T{P}_i^T}{{\sum }_i(T{P}_i^T+F{N}_i^T)}}.$$

(11)

Threat-validity recall is reported because a missed reference threat cannot enter downstream refinement and therefore directly limits attainable pipeline coverage. Over-generation is evaluated at the complete-task level through pipeline precision, where unmatched predicted threats are counted as false positives.

Pipeline recall. Pipeline recall measures the proportion of reference threats for which the complete pipeline produces valid end-to-end defenses. A defense is counted as valid only when all of the following conditions hold: the corresponding predicted threat is matched to a reference threat, the generated $SBP$/$SAF$ refinement covers the reference defense intent, the inserted security check is placed at an appropriate flow step, and the associated $SAF$ provides a concrete and complete handling branch for the threat.

Let $T{P}_i^{\mathrm{pipe}}$ denote the number of valid end-to-end defenses in case i, each mitigating a distinct reference threat, and let $F{N}_i^{\mathrm{pipe}}$ denote the number of required defenses that are not validly produced for the remaining reference threats. Pipeline recall is computed as

$$R_{\mathrm{pipe}}={\displaystyle \frac{{\sum }_{i}T{P}_i^{\mathrm{pipe}}}{{\sum }_i(T{P}_i^{\mathrm{pipe}}+F{N}_i^{\mathrm{pipe}})}}.$$

(12)

Pipeline precision. Pipeline precision measures the proportion of predicted threats that yield valid end-to-end defenses through the complete pipeline. Let $F{P}_i^{\mathrm{pipe}}$ denote the number of predicted threats in case i that do not yield valid end-to-end defenses because the threat is unmatched or the corresponding defense is missing or invalid. Pipeline precision is computed as

$$P_{\mathrm{pipe}}={\displaystyle \frac{{\sum }_{i}T{P}_i^{\mathrm{pipe}}}{{\sum }_i(T{P}_i^{\mathrm{pipe}}+F{P}_i^{\mathrm{pipe}})}}.$$

(13)

5.3. Research Questions (RQs) and Comparison Settings

RQ1: How effectively can missing security behavior be identified and incorporated into functional use case flows of LLM-based agentic systems? RQ1 evaluates the overall effectiveness of the complete MA-SAFR pipeline. To isolate the key characteristics of MA-SAFR, we compare it with three baseline settings. SA-NoRAG uses a single agent to perform security-omission identification and flow refinement without external retrieval. MA-NoRAG keeps the multi-agent decomposition but disables external retrieval, allowing us to examine the effect of role-specific Red Team and Blue Team reasoning alone. MA-VanillaRAG keeps the multi-agent setting and retrieves conventional text chunks from the same security knowledge corpus, which tests whether unstructured retrieval is sufficient for the Security-Augmented Use Case Flow Refinement Task. The full setting, denoted as MA-SAFR, uses the proposed multi-agent pipeline together with EvoGraphRAG and RSSG.

RQ2: How does risk-specific security guidance affect threat identification and security refinement within functional use case flows? RQ2 studies the contribution of the RSSG layer by fixing the multi-agent architecture and EvoGraphRAG retrieval while varying whether risk-specific guidance is provided. The with-RSSG setting uses RSSG to provide risk-specific guidance for threat anchoring and $SBP$/$SAF$ refinement. The without-RSSG setting removes these risk-specific rules and replaces them with a generic, risk-agnostic scaffold that preserves the basic task instructions and strict output constraints.

RQ3: How does feedback-driven adaptation of security knowledge retrieval affect the quality of security-augmented use case flow refinements? RQ3 examines how feedback-based edge-weight updates in EvoGraphRAG affect MA-SAFR performance. We compare retrieval using the base graph edge weights with retrieval using the same graph after its edge weights have been updated using Critic-derived validation feedback from the training split. Within each model-specific comparison, all other pipeline components are held fixed, and the updated graph is frozen before evaluation on the test split.

5.4. Experiment Settings

We conduct the experiments with three generation models: Qwen3.5-Plus [43], DeepSeek-V3.2 [44], and GPT-5.2 [45]. For all comparison settings, we use Qwen3.5-Flash as the fixed LLM-as-a-judge model for both threat matching and defense validity evaluation, so that all methods are evaluated under the same judging protocol. The fixed LLM-as-a-judge model is distinct from the Critic Agent in the proposed approach and uses the same predefined prompts and structured output schemas across all comparison settings.

All generation and judging calls use temperature $0.0$ and a request timeout of 900 s. Generated outputs are parsed under a strict JSON protocol, and we do not apply retry-based regeneration, JSON repair, schema correction, or missing-field imputation.

For retrieval-based settings, we use top-3 retrieval by default. Both MA-VanillaRAG and EvoGraphRAG use the same external security knowledge corpus, which is constructed from source-level OWASP and MITRE ATLAS materials rather than from SAFR-Bench reference annotations. Specifically, the corpus excludes benchmark-specific threat records, anchor labels, $SBP$/$SAF$ refinement pairs, security-augmented use case flows, evaluation labels, and test-set validation feedback.

For the feedback-based edge-weight update in EvoGraphRAG, we use the default values $\eta =0.08$, $w_{min}=0.35$, and $w_{max}=3.0$. We focus on these three parameters because $\eta$ determines the magnitude of the weight adjustment, while $w_{min}$ and $w_{max}$ define the allowable range of the updated edge weights. To assess sensitivity around the default configuration, we vary one parameter at a time. For $\eta$, we halve and double its default value, evaluating $\eta \in \{0.04,0.08,0.16\}$ while retaining the default weight bounds. For the weight bounds, we set $w_{min}=1.0$ and $w_{max}=1.0$ in separate configurations while keeping the other bound at its default value and $\eta =0.08$, thereby restricting downward and upward weight adjustments, respectively. To isolate parameter effects under a fixed model, the sensitivity analysis was conducted using Qwen3.5-Plus. Each configuration was evaluated over five independent runs, and the mean results are reported; all other experimental settings were held fixed.

5.5. Human Validation of LLM-Based Judging

Because evaluating generated threats and security-augmented flow branches requires semantic judgment, we use an LLM-based judge for scalable assessment and further validate its reliability through an independent human study.

The validation subset is sampled from the evaluation outputs of RQ1, RQ2 and RQ3, covering different generation models, method variants, benchmark sources, and both positive and negative judging outcomes. The sampled instances involve two judging tasks: threat matching, which checks whether a generated threat corresponds to a reference threat, and defense validity, which checks whether the generated security-augmented basic flow and alternative flow effectively mitigate the matched threat without violating the original flow semantics.

For each sampled instance, the human assessor reviews the functional flow, reference information, generated output, and judging rubric, while the LLM-based judging results are hidden. The assessor then independently evaluates the corresponding threat-matching and defense-validity decisions.

We measure agreement between the LLM-based judge and human assessment using percent agreement and Cohen’s $\kappa$ [46]. The former reports the raw agreement ratio, while the latter accounts for chance agreement. Formally, percent agreement is computed as follows:

$$P_o={\displaystyle \frac{1}{N}}{\displaystyle \sum _{i=1}^N}\mathbb{I}(y_i^{LLM}=y_i^{Human}),$$

(14)

where N is the number of sampled instances. Cohen’s $\kappa$ is computed as follows:

$$\kappa ={\displaystyle \frac{P_o-P_e}{1-P_e}},$$

(15)

where $P_o$ is the observed agreement and $P_e={\sum }_c{P}_{LLM}\left(c\right)P_{Human}\left(c\right)$ is the chance agreement estimated from the marginal label distributions.

6. Results and Discussion

6.1. RQ1: How Effectively Can Missing Security Behavior Be Identified and Incorporated into Functional Use Case Flows of LLM-Based Agentic Systems?

Table 7. Comparative results of MA-SAFR and baseline settings for security-augmented use case flow refinement (RQ1): threat-validity recall, pipeline recall, and pipeline precision. Values are reported as mean ± sample standard deviation over five independent runs. Bold indicates the best results for each model. The Improvement rows report relative changes from the strongest non-MA-SAFR baseline for each model and metric.

Model	Setting	${\mathit{R}}_{\mathbf{threat}}$	${\mathit{R}}_{\mathbf{pipe}}$	${\mathit{P}}_{\mathbf{pipe}}$
Qwen3.5-Plus	SA-NoRAG	(17.28 ± 2.90)%	(6.17 ± 2.47)%	(6.17 ± 1.81)%
	MA-NoRAG	(20.99 ± 2.76)%	(12.35 ± 1.95)%	( 8.77 ± 1.04)%
	MA-VanillaRAG	(25.93 ± 1.75)%	(11.11 ± 1.51)%	(10.59 ± 1.82)%
	MA-SAFR	(34.57 ± 1.23)%	(29.63 ± 1.51)%	(21.82 ± 1.94)%
	Improvement	+33.33%	+140.00%	+106.06%
DeepSeek-V3.2	SA-NoRAG	(28.40 ± 1.95)%	(1.23 ± 1.75)%	(0.52 ± 0.75)%
	MA-NoRAG	(30.86 ± 3.15)%	(8.64 ± 3.38)%	(3.04 ± 1.10)%
	MA-VanillaRAG	(37.04 ± 2.76)%	(11.11 ± 1.95)%	(8.26 ± 1.80)%
	MA-SAFR	(23.46 ± 2.31)%	(16.05 ± 1.51)%	(11.82 ± 1.66)%
	Improvement	−36.67%	+44.44%	+43.13%
GPT-5.2	SA-NoRAG	(24.05 ± 2.00)%	(12.66 ± 2.37)%	(4.37 ± 1.05)%
	MA-NoRAG	(25.32 ± 2.83)%	(17.72 ± 2.97)%	(6.25 ± 1.46)%
	MA-VanillaRAG	(29.11 ± 1.79)%	(21.52 ± 2.69)%	(7.59 ± 1.26)%
	MA-SAFR	(31.65 ± 1.27)%	(22.78 ± 0.90)%	(8.78 ± 0.71)%
	Improvement	+8.70%	+5.88%	+15.70%

presents the comparative results of MA-SAFR and the baselines on SAFR-Bench across three evaluation metrics. Overall, these results show that MA-SAFR delivers the best end-to-end refinement performance across all three generation models, indicating that the complete MA-SAFR pipeline improves the refinement of security-augmented use case flows.

First, the SA-NoRAG/MA-NoRAG comparison suggests that role-specific agent decomposition better supports multi-stage refinement. Across all three generation models, MA-NoRAG achieves higher threat-validity recall, pipeline recall, and pipeline precision than SA-NoRAG. By separating threat identification from defense construction, the multi-agent setting allows each agent to focus on a distinct refinement stage rather than handling all subtasks in a single generation pass.

Second, the MA-NoRAG/MA-VanillaRAG comparison examines the effect of adding chunk-level retrieval to the multi-agent setting. Across all three generation models, MA-VanillaRAG achieves higher threat-validity recall and pipeline precision than MA-NoRAG. Its effect on pipeline recall is less consistent: pipeline recall improves for DeepSeek-V3.2 and GPT-5.2 but slightly decreases for Qwen3.5-Plus. This pattern suggests that retrieved text chunks can provide useful security cues, but these cues do not always translate into more complete and valid $SBP$/$SAF$ refinements.

Finally, the MA-VanillaRAG/MA-SAFR comparison examines the combined effect of replacing chunk-level retrieval with EvoGraphRAG and providing RSSG. Both settings use the same external security knowledge corpus and multi-agent decomposition; MA-VanillaRAG presents retrieved text chunks as background evidence, whereas MA-SAFR combines relation-aware graph retrieval with risk-specific refinement guidance. Across all three generation models, MA-SAFR achieves higher pipeline recall and pipeline precision than MA-VanillaRAG; threat-validity recall is also higher for Qwen3.5-Plus and GPT-5.2 but lower for DeepSeek-V3.2. For Qwen3.5-Plus and GPT-5.2, the simultaneous improvements in threat-validity recall and both pipeline metrics suggest that the combined mechanism supports both threat discovery and downstream refinement. For DeepSeek-V3.2, MA-VanillaRAG identifies more reference threats, whereas MA-SAFR achieves higher pipeline recall and pipeline precision. To examine this trade-off beyond the aggregate metrics, we further analyzed the matched-threat and valid-defense counts across the five runs. The two settings generate nearly the same number of threat records per run (109.8 for MA-VanillaRAG and 110.6 for MA-SAFR), indicating that the lower threat-validity recall of MA-SAFR is not caused simply by fewer predictions. The reduction therefore occurs at the threat-grounding stage, where fewer generated records satisfy the reference-anchor and semantic security-concern criteria. However, the mean proportion of matched reference threats converted into valid end-to-end defenses increases from $29.82\%$ under MA-VanillaRAG to $68.45\%$ under MA-SAFR, indicating that matched threats are more frequently carried through to jointly valid $SBP$/$SAF$ refinements. This pattern suggests that, for DeepSeek-V3.2, the MA-SAFR configuration favors more constrained threat interpretations while substantially strengthening risk-specific handling and branch closure in downstream refinement. This result also indicates a model-specific trade-off between broader reference-threat coverage and downstream refinement alignment, and the contributions of RSSG and feedback-based retrieval adaptation to this trade-off are examined further in RQ2 and RQ3. Overall, across the three generation models, MA-SAFR provides a more consistent benefit for downstream threat-to-defense conversion than for threat discovery.

Although MA-SAFR achieves substantial relative improvements over the baselines, its absolute pipeline performance remains modest. At the current performance level, MA-SAFR should therefore be regarded as a human-in-the-loop decision-support tool that proposes traceable candidate refinements for requirements and security review, rather than as an autonomous replacement for security analysis. Accordingly, the relative improvements demonstrate comparative gains under the benchmark, while the absolute results help clarify the method’s current scope of practical applicability.

6.2. RQ2: How Does Risk-Specific Security Guidance Affect Threat Identification and Security Refinement Within Functional Use Case Flows?

Table 8. Relative effect of RSSG on MA-SAFR compared with the without-RSSG setting (RQ2). Values are relative changes (%).

Model	${\mathbf{\Delta }}_{\mathbf{rel}}{\mathit{R}}_{\mathbf{threat}}$	${\mathbf{\Delta }}_{\mathbf{rel}}{\mathit{R}}_{\mathbf{pipe}}$	${\mathbf{\Delta }}_{\mathbf{rel}}{\mathit{P}}_{\mathbf{pipe}}$
Qwen3.5-Plus	+16.67%	+26.32%	+32.06%
DeepSeek-V3.2	−26.92%	+18.18%	+52.56%
GPT-5.2	+8.57%	+14.81%	+17.84%

reports the relative changes introduced by RSSG compared with the without-RSSG setting. Across all three generation models, RSSG yields positive relative changes in both pipeline recall and pipeline precision, indicating that it improves the coverage of reference threats by valid defenses while more effectively controlling invalid over-generation across the complete pipeline. This pattern is particularly informative because RSSG is intended not merely to expand threat discovery, but to complement EvoGraphRAG by guiding the translation of retrieved security evidence into flow refinements.

For Qwen3.5-Plus and GPT-5.2, the improvement extends to threat-validity recall, suggesting that RSSG can support both threat discovery and downstream refinement. For DeepSeek-V3.2, RSSG decreases threat-validity recall while improving both pipeline metrics. Because RSSG jointly imposes risk-specific threat-anchoring constraints and structured $SBP$/$SAF$ refinement requirements, this pattern suggests that, for DeepSeek-V3.2, RSSG narrows reference-threat coverage while strengthening threat–defense traceability and branch closure. Thus, its primary benefit in this setting lies in improving the refinement of matched threats rather than expanding threat discovery.

Taken together, under the reported settings, RSSG affects threat discovery differently across models, while its most consistent contribution lies in bridging relation-aware security knowledge retrieval and structured use-case-flow refinement, thereby improving downstream threat-to-defense conversion.

6.3. RQ3: How Does Feedback-Driven Adaptation of Security Knowledge Retrieval Affect the Quality of Security-Augmented Use Case Flow Refinements?

Table 9. Effect of feedback-based edge-weight updates in EvoGraphRAG (RQ3). Values are relative changes (%).

Model	${\mathbf{\Delta }}_{\mathbf{rel}}{\mathit{R}}_{\mathbf{threat}}$	${\mathbf{\Delta }}_{\mathbf{rel}}{\mathit{R}}_{\mathbf{pipe}}$	${\mathbf{\Delta }}_{\mathbf{rel}}{\mathit{P}}_{\mathbf{pipe}}$
Qwen3.5-Plus	+16.67%	+9.09%	+11.07%
DeepSeek-V3.2	0.00%	+6.67%	+11.30%
GPT-5.2	+6.06%	+7.41%	+6.25%

reports the relative changes introduced by feedback-based edge-weight updates. Across all three generation models, the updates yield positive changes in pipeline recall and pipeline precision, indicating a consistent benefit for downstream threat-to-defense conversion. Their effect on threat discovery is less uniform: threat-validity recall improves for Qwen3.5-Plus and GPT-5.2 but remains unchanged for DeepSeek-V3.2. For Qwen3.5-Plus and GPT-5.2, the simultaneous improvements across all three metrics suggest that the updated retrieval preferences can support both threat discovery and downstream refinement. For DeepSeek-V3.2, feedback-based edge-weight updates leave threat-validity recall unchanged while improving both pipeline metrics. Therefore, unlike Qwen3.5-Plus and GPT-5.2, DeepSeek-V3.2 obtains no additional threat-coverage benefit from the updated retrieval preferences, although it still benefits in downstream refinement. Together with RQ2, this helps explain the mixed RQ1 result for DeepSeek-V3.2: although MA-SAFR covers fewer reference threats than MA-VanillaRAG, the controlled analyses suggest that RSSG and feedback-based retrieval adaptation primarily strengthen the conversion of matched threats into valid refinements rather than consistently expanding reference-threat coverage.

Each paired comparison keeps the graph topology, retriever, source knowledge, and remaining pipeline components fixed; the observed changes can therefore be attributed more directly to feedback-based edge-weight updates, which alter the retrieval priorities of existing risk–attack–mitigation paths without rewriting node descriptions or changing relation types. This prioritization can help the agents translate identified threats into valid $SBP$/$SAF$ refinements by making paths associated with successful refinements more prominent during retrieval. Overall, feedback-based edge-weight updates have a model-dependent effect on threat discovery but a consistently positive effect on downstream refinement.

6.4. Parameter Sensitivity of EvoGraphRAG’s Edge-Weight Update

Table 10. Parameter sensitivity of EvoGraphRAG’s edge-weight update (%).

ID	Varied Parameter	$\mathit{\eta }$	${\mathit{w}}_{min}$	${\mathit{w}}_{max}$	${\mathit{R}}_{\mathbf{threat}}$	${\mathit{R}}_{\mathbf{pipe}}$	${\mathit{P}}_{\mathbf{pipe}}$
E1	$\eta$	0.04	0.35	3.0	$30.86$	$25.43$	$18.41$
E2	Default	0.08	0.35	3.0	$34.57$	$29.63$	$21.82$
E3	$\eta$	0.16	0.35	3.0	$29.88$	$24.20$	$16.98$
E4	$w_{min}$	0.08	1.00	3.0	$31.11$	$23.70$	$16.73$
E5	$w_{max}$	0.08	0.35	1.0	$31.11$	$26.17$	$19.30$

first examines the update rate $\eta$, which determines how strongly validation feedback changes the edge weights. The default value $\eta =0.08$ achieves the highest observed performance across all three metrics, while both the smaller and larger update rates yield lower results. Moreover, the reductions observed at $\eta =0.16$ are consistently larger than those at $\eta =0.04$ relative to the default, suggesting that performance is more sensitive to an overly large update rate within the tested range. This result indicates a trade-off: a smaller $\eta$ may underuse the validation feedback, whereas a larger $\eta$ may exert an excessively strong influence on retrieval preferences. Thus, $\eta =0.08$ provides the best observed balance among the tested values.

Changing either weight bound also reduces performance relative to the default configuration. Raising $w_{min}$ to $1.0$ limits the extent to which negative feedback can decrease edge weights, whereas lowering $w_{max}$ to $1.0$ limits the extent to which positive feedback can increase them. Both variants underperform the default, with the restriction on negative weight adjustments producing a larger decrease in the pipeline metrics. Under the default configuration, none of the updated edge weights reached either bound; therefore, further widening the bounds would not change the evolved graph under the current update settings. Taken together, these results suggest that, within the tested settings, retaining the capacity for both downward and upward weight adjustments yields better observed performance than restricting either direction.

6.5. Computational Cost and Scalability Considerations

Table 11. Average LLM-generation cost of the comparison settings using Qwen3.5-Plus.

Setting	Input Tokens per Case	Output Tokens per Case	Total Tokens per Case	LLM-Generation Time per Case (s)
SA-NoRAG	886.63	440.14	1326.77	8.546
MA-NoRAG	866.33	333.44	1199.77	6.847
MA-VanillaRAG	2874.49	318.82	3193.32	7.015
MA-SAFR	5503.14	443.33	5946.47	9.756

reports the average LLM-generation cost of the four Qwen3.5-Plus settings, including the input and output token consumption and generation time within the pipeline. Retrieval is executed locally and does not introduce additional LLM-token consumption. Critic-based validation is performed during offline graph adaptation, with its cost amortized across subsequent cases, whereas the resulting edge-weight updates are applied locally. The reported online cost therefore excludes this overhead.

Among the four settings, MA-SAFR has the highest total token consumption and observed LLM-generation time, while its output-token consumption remains close to that of SA-NoRAG. This pattern is expected, given that relation-aware retrieval evidence and RSSG guidance enlarge the input context. This additional cost represents the computational trade-off associated with its improved refinement performance. The overhead also varies with the number and content length of the threat records passed from the Red Agent to the Blue Agent. Nevertheless, its observed LLM-generation time is 1.14 times that of SA-NoRAG (9.756/8.546), indicating that the increase in input-token consumption does not translate proportionally into generation latency in the evaluated setting.

From a deployment perspective, MA-SAFR is amenable to existing input-side optimizations because it repeatedly reuses substantial prompt components, including role instructions, refinement constraints, and output schemas. In cache-enabled deployments, prefix caching can reuse the computation of these stable prompt components [47], while retrieved-context compression and prompt compression can reduce the actual input-token count [48,49]; together, these techniques provide practical opportunities to reduce the computational cost of MA-SAFR during deployment.

6.6. Reliability of LLM-Based Judging

Table 12. Human validation results of LLM-based judging.

Judging Task	n	Agreement	Disagreement	Percent Agreement	Cohen’s $\mathit{\kappa }$
Threat matching	60	53	7	88.33%	0.767
Defense validity	60	49	11	81.67%	0.633

reports the agreement between the LLM-based judge and the independent human assessment described in Section 5.5. Overall, the LLM-based judge shows meaningful agreement with the human assessment on both judging tasks, with higher consistency for threat matching than for defense validity.

For threat matching, the LLM-based judge agrees with the human assessment on 53 out of 60 instances, achieving an agreement rate of 88.33% and a Cohen’s $\kappa$ of 0.767. This result indicates that the judge is largely reliable in determining whether a generated threat corresponds to a reference threat. For defense validity, the agreement rate is 81.67%, with a Cohen’s $\kappa$ of 0.633. Although lower than that for threat matching, this result still indicates a degree of overall consistency in the judging protocol for large-scale evaluation.

The lower agreement on defense validity is expected because this task requires more complex semantic judgment. The assessor must determine not only whether the generated defense is relevant to the threat, but also whether the inserted $SBP$/$SAF$ can effectively mitigate the threat, preserve the original flow semantics, and form a coherent branch. Therefore, defense validity is intrinsically more subjective than threat matching. These results support the use of the LLM-based judge for scalable evaluation, while also indicating that defense-validity judgments remain a source of evaluation uncertainty.

A qualitative review shows that threat-matching disagreements mainly arose at category boundaries or when related risks were expressed at different levels of abstraction, whereas defense-validity disagreements concerned whether alternative controls sufficiently covered the reference intent and whether the $SAF$ provided complete handling and closure. The disagreements were bidirectional: the LLM-based judge was more permissive than the human assessor in 3 of the 7 threat-matching cases and 6 of the 11 defense-validity cases, and stricter in the remaining cases, suggesting no simple one-sided judging tendency in this sample. These disagreements indicate that the exact values of judge-dependent metrics are subject to evaluation uncertainty, particularly for defense validity, while the overall agreement still supports the use of the judging protocol for aggregate comparisons.

6.7. Failure Analysis

We perform a case-by-case diagnostic analysis of the failed or partially covered cases observed in our experiments. The analysis shows that threat identification failure accounts for the largest share, followed by incomplete coverage of security omissions in multi-threat cases and semantic misalignment.

The dominant limitation is threat identification failure, reflecting the difficulty of inferring implicit security threats from the functional behavior represented by use case flow steps. To reduce such omissions, future work could employ multi-pass flow-to-threat reasoning, in which the functional flow is revisited from complementary risk perspectives to identify security omissions not captured in an initial pass.

A second limitation is incomplete coverage of security omissions in multi-threat cases. In some partially covered cases involving chain-like or multiple related attack opportunities, MA-SAFR identifies the most salient threat but does not address all security omissions in the use case. This pattern points to the need for explicit modeling of threat dependencies, attack propagation paths, and residual risks, so that related threats and attack opportunities remaining after a control can be analyzed jointly rather than independently.

The third limitation concerns semantic misalignment among the identified threat, the generated $SBP$/$SAF$, and the original use case flow. In such cases, the refinement may describe a plausible security action while remaining inconsistent with the corresponding threat mechanism or the control-flow logic of the original use case. Addressing such inconsistencies may require a tighter threat–defense–flow review-and-revision loop that jointly examines the anchored threat mechanism, proposed defense, and resulting flow behavior and revises the candidate when misalignment is detected.

6.8. External Qualitative Case Studies

To complement the benchmark-based evaluation and examine whether the proposed refinement approach can be applied beyond the OWASP and MITRE ATLAS cases, we conduct two external qualitative case studies. The first is derived from an external clinical task in MedAgentBench involving retrieval from an electronic health record system (EHR) and medication ordering [50]. The second concerns Replit, an AI-powered software development platform, and is reconstructed from its official description of a former workflow in which development activities and live customer data shared a single database [51]. The source materials are normalized into our functional-flow representation without adding target threats or security controls.

We compare MA-SAFR with MA-VanillaRAG using Qwen3.5-Plus, temperature 0.0, retrieval depth $k=3$, and five runs per setting. Because the cases do not provide reference threat records or $SBP$/$SAF$ refinements, we perform qualitative analysis rather than compute benchmark metrics.

MedAgentBench case. The source task asks a clinical agent to retrieve a patient’s most recent serum magnesium observation within the preceding 24 h and, if the value is low, create an intravenous (IV) magnesium replacement order according to the supplied dosing instructions. We normalize the task into the following functional flow:

• BF1, the clinician requests magnesium assessment and conditional IV magnesium replacement;
• BF2, the clinical agent queries the EHR using the patient identifier and magnesium code;
• BF3, the EHR returns the matching observations and their measurement times;
• BF4, the clinical agent filters the observations to the preceding 24 h and selects the most recent eligible value;
• BF5, the clinical agent classifies the deficiency severity;
• BF6, the clinical agent selects the corresponding dose and infusion duration;
• BF7, the clinical agent submits the active IV magnesium medication request;
• BF8, the EHR creates the order and returns its status.

If no eligible observation is available or the selected value is not low, the task terminates without submitting an order.

Replit case. Replit reported that its former workflow used a single database for development activities and live customer data. We normalize the account into the following functional flow:

• BF1, the user asks Replit Agent to modify a database-backed application;
• BF2, Replit Agent changes the application code and database state during development;
• BF3, Replit Agent previews and tests the updated application;
• BF4, Replit Agent modifies during development the same database that stores live customer data;
• BF5, the user requests deployment;
• BF6, Replit deploys the updated application using the same database;
• BF7, the live application continues to read and write customer data in that database.

Table 13. Representative threat records and security-flow refinements for the external qualitative cases.

Case and Setting	Threat Record	Generated $\mathit{SBP}$/$\mathit{SAF}$ Refinement
MedAgentBench MA-SAFR	Excessive agency in automated medication ordering, anchored at BF7. The clinical agent can directly submit an active IV medication order without an independent authorization or safety boundary.	SBP: Before BF7, validate the proposed dose and infusion duration against approved safety limits and require clinician verification before submission. SAF: Block and quarantine an unsafe or unverified order, present its parameters and rationale to the clinician, and retry BF7 after confirmation or correction.
MedAgentBench MA-VanillaRAG	Incorrect magnesium-deficiency classification, anchored at BF5. An erroneous severity classification can cause an inappropriate dose and medication order; therefore, the data should be comprehensively validated to prevent such errors.	SBP: Before BF5, expose the selected value for verification. SAF: Halt the workflow when a mismatch is detected, and continue to BF6 after correction.
Replit MA-SAFR	Live-data corruption through development changes, anchored at BF4. Agent-generated database operations can be applied directly to the database containing live customer data.	SBP: Before BF4, validate database operations and block destructive commands such as unscoped deletion, truncation, or schema removal. SAF: Quarantine the unsafe operation, request a corrected change, revalidate it, and return to BF4 after the operation satisfies the safety policy.
Replit MA-VanillaRAG	Malicious code injection via an untrusted user request, anchored at BF2. Hidden prompt-injection instructions in the initial request can cause Replit Agent to generate malicious code or execute unauthorized database commands during application modification.	SBP: Before BF2, submit the proposed code and database changes to an external non-LLM guardrail that scans for malicious patterns, unauthorized database instructions, and prompt-injection artifacts. SAF: If suspicious instructions or database commands are detected, block the proposed changes, prevent the agent from accessing the development environment with the tainted context, alert the user, and require a revised clean request before the workflow can continue.

presents one representative schema-valid output from a repeatedly observed threat theme for each case and setting. The threat records and refinements are condensed for readability without changing their security intent.

Table 13 highlights three differences between the two methods. First, graph-based retrieval leads MA-SAFR to follow explicit risk–mitigation relations and produce specific, traceable controls, such as medication-order authorization and destructive database-operation checks. Flat semantic retrieval allows MA-VanillaRAG to make broader inferences; however, the resulting threat, $SBP$, and $SAF$ may be only loosely connected. In the medical case, the threat detected by MA-VanillaRAG concerns an incorrect severity classification, whereas the $SBP$ merely exposes the selected value without specifying the verification criterion, and the $SAF$ invokes a mismatch without defining how it is detected or corrected.

Second, although both methods employ Red and Blue agents, MA-SAFR maintains a clearer separation between diagnostic threats and prescriptive refinements. In the medical case, MA-VanillaRAG places the recommendation for comprehensive data validation inside the Red-Agent threat record, whereas such prescriptive content should be produced by the Blue Agent.

Third, the RSSG constraints in MA-SAFR keep its $SBP$/$SAF$ refinements close to the affected flow step and provide explicit blocking, correction, and re-entry behavior. Without these constraints, MA-VanillaRAG allows broader design changes. This flexibility can be beneficial, but it can also produce under-specified scope expansion. For example, in the Replit case, we observed that MA-VanillaRAG expanded a local refinement of the BF2 code-modification step into a system-level security redesign by introducing an external guardrail service to scan generated code for broadly defined malicious patterns and prompt-injection artifacts, without specifying concrete detection criteria.

For requirements engineers, each of the above threat anchors and $SBP$/$SAF$ pairs translates an implicit security omission into a reviewable candidate change at the corresponding flow step and provides concrete guidance for specifying preventive and failure-handling behavior.

However, the effectiveness of MA-SAFR is influenced by the coverage and domain relevance of its security knowledge graph. When domain-specific knowledge is limited, graph retrieval may overemphasize generic agent-security patterns and produce precise, traceable controls that do not fully address the domain-specific root cause. For example, in another MedAgentBench run, MA-SAFR applied a generally valid agent-security data-minimization control but masked partial timestamps required by the clinical task.

7. Threats to Validity

7.1. External Validity

Threats to external validity arise mainly from two aspects. First, SAFR-Bench comprises 157 cases and 235 threat records derived from the OWASP LLM Top 10 2025 and MITRE ATLAS v5.5.0, providing source-level breadth across heterogeneous LLM application risks and attack scenarios. However, its requirement descriptions and use case flows are reconstructed artifacts and may underrepresent the ambiguity, incompleteness, domain terminology, and organizational context of real requirements. Although the external qualitative case studies in Section 6.8 extend the evaluation beyond SAFR-Bench, they only partially alleviate this limitation, as their limited scale cannot establish broad generalizability across application domains and requirement styles.

Second, experimental generalizability is affected by model coverage and dependence on external security knowledge. The experiments cover three representative LLMs, and improvements across these evaluated models yield preliminary evidence of cross-model generalizability. However, the results may not generalize to other proprietary models, smaller open-source models, future model versions, or different decoding configurations. In addition, EvoGraphRAG is sensitive to the coverage and structure of its external security knowledge graph; changes in knowledge sources, graph construction, relation schemas, or risk taxonomies may affect retrieval and downstream refinement. Although SAFR-Bench and the external security knowledge used for retrieval are both derived from OWASP and MITRE ATLAS, the retrieval corpus excludes benchmark-specific threat records and anchor labels, reference $SBP$/$SAF$ refinements, evaluation labels, and test-set validation feedback, avoiding direct overlap with the reference outputs. Future work will further evaluate MA-SAFR on more diverse requirement datasets from additional application domains and extend SAFR-Bench to cover broader security risks and emerging attack scenarios.

7.2. Internal Validity

Threats to internal validity first arise from LLM generation variability and potential confounding among comparison settings. LLM outputs are sensitive to prompt design and may vary across repeated runs even under identical inputs. To mitigate this threat, we kept the prompt templates fixed within each setting and set the temperature to 0.0 for all generation models. To further reduce run-specific variation, we report results averaged over five independent runs. All methods were evaluated using consistent generation and judging settings. In particular, MA-VanillaRAG and EvoGraphRAG used the same external security knowledge corpus and retrieval depth, while the RQ2 RSSG ablation and the RQ3 controlled edge-weight comparison varied only the target component with the remaining pipeline fixed.

A second concern is potential bias in LLM-based evaluation. Prior work has demonstrated the effectiveness of LLM-as-a-judge [52]. We further validated our judging protocol through an independent human assessment, obtaining Cohen’s $\kappa$ values of 0.767 for threat matching and 0.633 for defense validity. In addition, we used Qwen3.5-Flash, which was not among the three evaluated generation models, as the fixed judge for all methods. These results support the reliability of the judging protocol, although uncertainty remains for complex defense-validity decisions, as discussed in Section 6.6.

Benchmark construction represents another potential source of internal bias. Scenario rewriting, flow parsing, threat-record construction, and $SBP$/$SAF$ annotation involve human judgment and may therefore introduce annotation bias. We mitigated this risk through source grounding, a unified annotation schema, cross-layer consistency checks, and expert review of 60 randomly selected cases by 5 experienced engineers. Nevertheless, because only a subset of SAFR-Bench was reviewed, annotation errors may remain in the unreviewed cases. Future work will extend expert review to the full benchmark and incorporate additional independent reviewers and adjudication to further reduce residual annotation bias.

7.3. Construct Validity

Threats to construct validity mainly concern how valid threats, valid defenses, and refinement quality are operationalized. A predicted threat is considered valid only when it matches a reference threat with respect to both its anchor step and security concern. Retaining the canonical primary anchor strengthens flow-level traceability and keeps comparisons consistent, but may undercount predictions with plausible alternative anchors or reasonable formulations at different levels of abstraction. Similarly, an end-to-end defense is considered valid only if the corresponding threat is matched, the generated $SBP$/$SAF$ pair covers the reference defense intent, the $SBP$ is appropriately placed, and the $SAF$ specifies concrete handling. Although aligned with Security-Augmented Use Case Flow Refinement Task, this definition may not capture global policies, architectural constraints, or runtime guardrails. The Critic Agent evaluates five dimensions: threat validity checks whether an omission is security-relevant and grounded in the flow; defense coverage assesses whether the refinement mitigates the threat mechanism; threat-defense traceability links the threat, $SBP$, and $SAF$; branch closure requires a complete continuation or termination path; and flow consistency preserves the original functional intent. For quantitative evaluation, we use three complementary metrics: threat-validity recall measures threat coverage; pipeline recall measures successful threat-to-defense conversion; pipeline precision accounts for invalid over-generation. However, these task-level measures do not directly assess real-world attack resistance, output readability, engineering adoptability, implementation and maintenance costs, or user-experience impact. Within the scope of Security-Augmented Use Case Flow Refinement Task, the evaluation metrics operationalize key aspects of the validation dimensions for flow-level security refinement.

8. Conclusions and Future Work

This paper formulated the Security-Augmented Use Case Flow Refinement Task, which aims to transform security omissions implicit in the functional use case flows of LLM-enabled agentic systems into traceable and actionable security refinements. To address this task, MA-SAFR coordinates three specialized agents: a Red Team Agent for threat anchoring, a Blue Team Agent for $SBP$/$SAF$ construction, and a Critic Agent for refinement validation. The framework combines EvoGraphRAG for relation-aware retrieval and feedback-based graph evolution with the RSSG layer, which provides risk- and role-specific guidance for grounding threats and constructing $SBP$/$SAF$ refinements. The SAFR-Bench supports the systematic evaluation of threat discovery, threat anchoring, and end-to-end security-augmented use case flow refinement.

Across the three evaluated generation models, MA-SAFR achieved the strongest end-to-end refinement performance among the compared settings, indicating the comparative benefit of the complete framework under the evaluated settings. The controlled RSSG comparison shows that RSSG consistently improves the conversion of identified threats into valid $SBP$/$SAF$ refinements. The controlled edge-weight comparison further shows that feedback-based edge-weight updates consistently improve downstream threat-to-defense conversion, although their effect on threat discovery varies across models. The sensitivity analysis shows that the default edge-weight update configuration achieves the highest observed performance among the tested settings, while the computational-cost analysis characterizes the online generation overhead and the two external qualitative cases provide preliminary evidence of applicability beyond SAFR-Bench. The failure analysis highlights three remaining challenges: identifying security omissions implicit in functional behavior, covering all security omissions in multi-threat cases, and maintaining semantic alignment among identified threats, generated defenses, and the original flow.

Future work will investigate three complementary directions across more diverse application domains: multi-pass flow-to-threat reasoning to reduce missed threats; coverage-aware refinement for multi-threat and attack-chain scenarios by modeling threat dependencies, attack propagation paths, and residual risks; and tighter threat–defense–flow review and revision to reduce semantic misalignment.