PromptSentinel-X: A Leakage-Aware and Context-Aware Framework for Prompt-Injection Detection in Large Language Model-Powered Web Agents

Abstract

Large language model (LLM)-powered web agents combine privileged instructions with user requests, webpages, retrieved documents, tool outputs, memory, and conversation history, creating prompt-injection risks that static text classification may not capture. This study presents PromptSentinel-X, a leakage-aware and context-aware screening framework. The primary benchmark contained 1581 English records from the Prompt Injection Malignant dataset; 30,015 external prompt-injection, jailbreak, and benign hard-negative records were reserved for transfer and stress testing. The framework uses prompt-family-aware partitioning, trusted–untrusted context segmentation, calibrated risk prediction, robustness analysis, and deployment-oriented routing. On a 465-record group-aware test set, PromptSentinel-X achieved 0.9849 accuracy, 0.8887 macro-F1, 0.9851 attack recall, a 0.0050 benign false-positive rate, 0.9971 area under the receiver operating characteristic curve (AUROC), 0.9836 area under the precision-recall curve (AUPRC), and 0.0153 expected calibration error. Random splitting produced a higher baseline macro-F1 but 144 leakage warnings. Context-aware macro-F1 declined from 0.8301 for static prompts to 0.6306 for multi-turn, 0.3844 for retrieval-augmented generation, and 0.3200 for memory scenarios. PromptSentinel-X is positioned as a calibrated screening and escalation component, not a stand-alone authorization mechanism. Broader tool, memory, multimodal browser, multilingual, and end-to-end agent studies remain necessary.

1. Introduction

LLM-powered web agents have become a strategic marker of the transition from conversational artificial intelligence toward autonomous, tool-using, web-mediated systems. A conventional chatbot primarily responds to a user message; a web agent may browse pages, retrieve documents, summarize external content, inspect files, invoke APIs, update memory, select tools, and execute multi-step tasks across heterogeneous digital environments. This shift changes the security problem. The web no longer functions only as a source of information that a model summarizes; it becomes an operational substrate through which untrusted text can influence the agent’s reasoning, planning, memory, tool selection, and downstream actions.

Prompt-injection detection has therefore become a central safety-control problem for the Agentic Web. In traditional software, program code, database content, user input, and network responses are separated by technical boundaries. In LLM systems, the same natural-language sequence can behave as data, context, instruction, policy conflict, retrieval evidence, or tool metadata. Prompt injection exploits this ambiguity by introducing instructions that compete with the user’s legitimate task, the developer’s intended policy, or the system’s privileged instruction hierarchy [1]. The resulting failure is not merely that a model classifies a text incorrectly; it may cause an agent to follow an untrusted instruction, disclose protected context, select the wrong tool, approve unsafe content, poison memory, or execute an unauthorized action.

Recent work on AI-agent security argues that agent architectures alter assumptions about code–data separation, authority boundaries, tool access, and execution predictability, producing confidentiality, integrity, and availability failures that are not reducible to ordinary chatbot misuse [2]. Survey evidence on LLM-based agents similarly emphasizes that planning loops, memory, tool use, and multi-agent coordination expand the attack surface from model-only failures to agent compromise [3,4]. In parallel, 2026 studies on prompt-injection surveys, AI browsers, tool-selection attacks, coding assistants, and hidden-state jailbreak detection have converged on a common conclusion: web-agent defense requires contextual, calibrated, and deployment-aware detection rather than a single lexical filter or static classifier [5,6,7,8,9,10,11].

Against this background, the present study develops PromptSentinel-X as an independently evaluated defensive prompt-risk detection framework for LLM-powered web agents. The empirical core is the Kaggle Prompt Injection Malignant dataset [12], used as the in-domain benchmark for jailbreak and prompt-injection detection. The study extends the analytical surface from static prompt classification to agentic-web conditions through an expanded risk taxonomy, group-aware splitting, hard-negative evaluation, external benchmark adapters, multi-view representation learning, context-aware segmentation, target-LLM hidden-state analysis, calibrated ensemble inference, robustness assessment, error analysis, and deployment-oriented cascade policies.

1.1. LLM-Powered Web Agents and the Emergence of the Agentic Web

LLM-powered agents have expanded the role of language models from passive text generators to interactive systems that can reason, plan, retrieve information, use external tools, and complete multi-step tasks. Early demonstrations of web-augmented and tool-augmented language models showed that task performance could improve when models searched the web, consulted external tools, and interleaved reasoning with actions [13,14,15]. Subsequent surveys formalized this shift into a broader agent framework, where the language model serves as a decision core connected to perception modules, memory, planning routines, tools, and external environments [16,17]. This architecture enables natural-language systems to operate beyond static chat, but it also introduces security risks because agents must interpret and act on information from sources with varying levels of trust.

The Agentic Web is used in the present study to describe web-mediated environments in which LLM agents consume online content and act across multiple software surfaces. In this setting, webpages, search results, retrieved documents, browser accessibility trees, tool responses, memory entries, API outputs, and conversation histories may all become part of the model’s operating context. The browser or retrieval pipeline therefore functions as a semantic runtime, where HTML, metadata, accessibility labels, comments, search snippets, tool descriptions, and stored memory can influence downstream decisions. As a result, the model is not only a generator but also an interpreter of mixed-trust inputs and a controller of actions across browser environments, software-development workflows, enterprise copilots, RAG pipelines, and multi-agent systems.

Recent work illustrates both the capability and the risk of this shift. WebArena demonstrated the difficulty of realistic web-agent operation by evaluating agents in interactive websites that require navigation, reading, action planning, and tool-like execution [18]. AgentDojo extended this concern to security by evaluating agents that use tools over untrusted data and by showing how prompt injection can hijack behavior in realistic tasks such as email management, travel booking, and e-banking-style workflows [19]. More recent agent-security surveys similarly emphasize that LLM agents face distinctive risks because they operate through multi-step inputs, opaque internal reasoning, external tools, changing environments, and interactions with untrusted entities [20,21]. A 2026 IEEE Access taxonomy of semantic and agentic threats in AI browsers argues that previously inert DOM elements, hidden text, ARIA labels, metadata, and JSON structures can become covert channels for prompt injection, model steering, data exfiltration, and unintended autonomous behavior [22]. This directly motivates the present study’s emphasis on browser, RAG, tool, memory, system-boundary, and multi-turn attack surfaces.

The Agentic Web also changes how security controls should be evaluated. Static benchmarks may indicate whether an input resembles a known jailbreak, but web agents must decide whether a content segment is a user request, developer instruction, retrieved evidence, browser-visible data, hidden webpage instruction, tool output, or memory artifact. A detector that ignores source privilege and context may miss indirect attacks, while an overly broad detector may block benign technical work. Prompt-injection detection for web agents therefore needs to preserve distinctions between trusted and untrusted segments, evaluate source-specific risk, and support calibrated responses rather than relying only on binary labels.

This literature establishes the need to study prompt-injection detection at the level of agentic workflows rather than isolated prompts. A model that performs well on a static benchmark may still fail when adversarial instructions are embedded in webpage text, retrieved passages, tool outputs, or memory records. Conversely, a detector that blocks all suspicious wording may reduce some attacks while making legitimate technical work, security education, debugging, and multilingual assistance unusable. The Agentic Web therefore requires a safety framework that connects benchmark validity, representation learning, context segmentation, over-defense analysis, calibration, and deployment feasibility.

1.2. Recent 2026 Evidence on Prompt Injection in Agentic Systems

The 2026 literature has moved prompt-injection research from generic LLM safety toward agentic systems with tools, memory, protocols, and browser-mediated interaction. Li et al. describe how AI-agent architectures alter code–data separation, authority boundaries, and execution predictability, with indirect prompt injection, confused-deputy behavior, and cascading failures emerging as central risks in long-running workflows [2]. Tamuka et al. synthesize attacks on LLM-based agents and identify planning loops, memory, tool use, and multi-agent systems as structural sources of vulnerability; importantly, their review concludes that persistent gaps remain in generalization under adaptive attackers and in evaluation of agent–tool interaction risk [3].

Several 2026 surveys broaden this systems view. Kim et al. systematize the attack and defense landscape of agentic AI and argue that the combination of LLMs with non-AI software components creates security challenges different from those of traditional software or standalone LLMs [4]. Ferrag and colleagues bridge input-level prompt injection with protocol-layer exploitation in LLM-agent workflows, emphasizing host-to-tool and agent-to-agent communications [21]. Geng et al. provide a consolidated survey of prompt-injection attack methods, root causes, and defense strategies, while Gulyamov et al. focus specifically on LLM and AI-agent systems, including tool-based injection, MCP-related poisoning, and defense mechanisms [5,6]. Together, these sources establish that prompt injection has become a system-level agentic vulnerability rather than a narrow conversational weakness.

The most recent empirical and attack-specific studies sharpen this conclusion. Shi et al. introduce ToolHijacker, an NDSS 2026 attack against LLM-agent tool selection in which a malicious tool document manipulates both retrieval and selection phases [23]. Khodayari et al. present a large-scale study of indirect prompt injection in webpages and HTTP responses, showing that prompt-based interference is already present in the web ecosystem and that many injections are positioned in non-rendered or machine-targeted content [10]. Maloyan and Namiot synthesize vulnerabilities in agentic coding assistants, including skills, file systems, shell access, and protocol ecosystems [7]. Abdelnabi and Bagdasarian recast prompt injection through contextual integrity, arguing that simple data–instruction separation is insufficient for future agents because contextual manipulation can make unsafe flows appear legitimate [8].

1.3. Prompt Injection as an Instruction-Control and Contextual-Integrity Vulnerability

Prompt injection is commonly described as an attack in which adversarial instructions cause an LLM to deviate from its intended behavior. OWASP identifies prompt injection as the first major risk category for LLM applications, including both direct and indirect forms [24]. However, for web agents, this definition must be interpreted as an instruction-control vulnerability. The central question is not only whether a text contains malicious words but whether the agent incorrectly treats untrusted content as authoritative instructions.

This distinction is technically important because agentic systems frequently concatenate multiple sources of text into one model context. A user query, retrieved document, webpage, tool result, prior conversation, and memory record may be jointly processed even though they carry different trust levels. Indirect prompt-injection studies show that LLMs can fail to distinguish informational context from actionable instructions when external content is placed into the prompt [25,26]. This weakness becomes more severe when the model has tool authority, persistent memory, or browser action capability because a successful injection can affect downstream execution rather than only produce an unsafe answer.

The present study therefore interprets prompt injection as a failure of trusted–untrusted boundary management. A benign technical question, a cybersecurity discussion, a quoted adversarial example, a roleplay scenario, or a multilingual translation can contain suspicious vocabulary without malicious intent. Conversely, a webpage comment, tool description, or memory entry can contain an instruction-like sequence whose risk depends on the agent’s privilege and context. This is why PromptSentinel-X evaluates fine labels, coarse risk categories, context segments, calibrated confidence, hard negatives, and action-level deployment policies rather than relying on lexical triggers alone.

1.4. Direct, Indirect, Browser, Tool, Memory, and Multi-Agent Attack Surfaces

Direct jailbreaks remain important because they test whether a user can explicitly pressure a model to ignore constraints or produce prohibited behavior. The Kaggle Prompt Injection Malignant dataset captures this in-domain detection problem through benign conversation, malicious jailbreak, and suspicious roleplay or act_as categories [12]. Nevertheless, LLM-powered web agents face a wider threat model. The adversarial instruction may be direct, embedded in retrieved documents, hidden in browser content, encoded in tool output, staged across multiple turns, stored in memory, or propagated across agents.

Browser-mediated injection is especially relevant to the Agentic Web. Johnson et al. showed that LLM-based web-browsing agents can be manipulated through adversarial triggers embedded in webpage HTML or accessibility-tree content [27]. Garcia et al. extend this concern to AI-enabled browsers, arguing that AI browsers semantically interpret DOM elements and therefore convert hidden or structural web artifacts into model-consumable inputs [22]. Khodayari et al. further show that indirect prompt injections already exist in the wild across webpages and HTTP responses, with many placed in non-rendered HTML, metadata, headers, comments, or structured page elements [10].

Tool-mediated injection introduces another layer of risk. Shi et al. show that the tool-selection stage itself can be attacked through malicious tool documents, which means an agent may choose a harmful tool before any downstream tool-call validation occurs [23]. This matters because many agentic workflows use retrieval and selection to decide which API, plugin, MCP tool, database connector, or file operation should be invoked. Memory and multi-agent surfaces create related challenges: injected instructions may persist across sessions or propagate between agents through communication channels. These risks justify the present study’s extended taxonomy, context-aware segmentation, and cascade thresholds for tool, browser, memory, RAG, and multi-turn scenarios.

1.5. Benchmarking, Leakage Control, and Evaluation Validity

Prompt-injection research depends on benchmark quality. A detector may appear strong if paraphrases, near-duplicates, or related prompt families appear in both training and test partitions. In web-agent deployment, however, the system encounters novel webpages, new tool descriptions, unseen retrieval documents, unfamiliar user tasks, and adapted attack styles. Therefore, evaluation must control leakage at the family level and report macro-F1, class-wise recall, attack recall, benign false-positive rate, calibration error, and deployment outcomes rather than accuracy alone.

Recent benchmarks provide important foundations but also expose limitations. AgentDojo evaluates prompt injection attacks and defenses in dynamic tool-using LLM-agent environments [19]. InjecAgent targets indirect prompt injections in tool-integrated agents [28]. BIPIA evaluates indirect prompt injection in externally supplied content and highlights the inability of LLMs to reliably distinguish data from instructions [25]. HarmBench and JailbreakBench provide broader harmful-behavior and jailbreak robustness evaluation [29,30]. The recent SoK on jailbreak guardrails emphasizes that guardrail evaluation must consider security, efficiency, and utility rather than a single attack-detection score [31]. Deployable detector studies such as PromptShield additionally illustrate the importance of operational prompt-injection screening [32].

The present study uses the Kaggle Malignant dataset as the primary in-domain benchmark while adapting the evaluation design toward agentic-web conditions. Its normalized in-domain file contains 1581 English records: 1312 benign conversation samples, 199 malicious jailbreak samples, and 70 suspicious roleplay or act_as samples. This imbalance is methodologically important because benign traffic dominates real agent operation, while malicious prompts are sparse but high-impact. Group-aware splitting, external benchmark adapters, hard negatives, and robustness slices are therefore used to avoid inflated performance and to test whether detector behavior survives paraphrase, obfuscation, multilingual shift, long-context placement, RAG injection, textual browser-content injection, and cross-source variation.

1.6. Detection, Representation Learning, and Hidden-State Signals

Prompt-injection detection cannot be reduced to keyword matching. Recent studies show that attacks can be obfuscated, paraphrased, embedded in external content, or distributed across context. At the same time, benign samples may contain security-related vocabulary, quoted examples, code snippets, prompt-engineering discussion, or educational analysis. Detection therefore requires semantic representation, intent modeling, context segmentation, uncertainty estimation, and careful false-positive control.

Several 2026 studies support this direction. Kadali and Papalexakis argue that jailbreak attacks leave identifiable traces in internal representations and propose latent-space analysis for lightweight detection and inference-time mitigation [33]. Alshammari and Alsaleh propose a hybrid SIEM and one-class SVM framework for detecting prompt-injection attacks in generative AI systems [9]. Guardrail mismatch studies show that external safety monitors may fail when the guardrail model and protected target model differ in capabilities, assumptions, or threat sensitivity [34]. These findings motivate detectors that combine multiple views rather than relying on one prompt-level classifier.

PromptSentinel-X follows this multi-view logic. The architecture combines a configurable text encoder, projection-space representation learning, fine-label and coarse-risk heads, context-aware segmentation, optional target-LLM hidden-state features, calibrated inference, and ensemble or cascade variants. The projection space supports interpretable risk geometry; the fine-label head supports error analysis; the coarse-risk head supports operational decisions; context features distinguish trusted and untrusted segments; hidden-state features test whether the target model’s internal activations contain complementary risk signals; and calibration converts raw scores into action thresholds for allow, warn, block, sandbox, or human review.

1.7. Over-Defense, Calibration, and Deployment Practicality

A practical web-agent guardrail must control both missed attacks and over-defense. Excessive blocking can make an agent unusable for legitimate security research, debugging, compliance analysis, prompt engineering, multilingual assistance, or educational discussion. Under-blocking can permit prompt injection to modify tool choice, disclose private context, or execute unauthorized actions. This tension is central to Agentic Web deployment because the same input can be safe or unsafe depending on source, intent, privilege, tool availability, and downstream action.

The 2026 literature increasingly frames this as a security–utility trade-off. Li et al. identify deterministic policy enforcement, sandboxed execution, and layered mitigation as necessary complements to input-level defenses for high-consequence agent actions [2]. Tamuka et al. identify monitoring, guardrails, provenance, and standardized evaluation as key defense categories while emphasizing persistent gaps under adaptive attacks [3]. Abdelnabi and Bagdasarian further caution that stricter separation rules can block legitimate contextual flows while still failing against manipulation that makes a prohibited flow appear appropriate [8]. These arguments support a calibrated detection architecture rather than a brittle binary filter.

Recent defenses reinforce that deployment should be evaluated as a security-utility problem rather than only as attack detection. Structured-query approaches explicitly separate instructions from data, provenance-marking methods make untrusted content more salient, and task-alignment defenses verify whether agent instructions and tool calls serve the user’s stated objective [35,36,37,38]. Agent-level benchmarks likewise show that security must be evaluated jointly across tool use, memory, execution, utility, and attack success rather than inferred from a single prompt-classification score [39,40]. These findings support the present study’s use of hard negatives, calibration, escalation, and runtime controls outside the detector.

Deployment analysis is therefore part of the scientific contribution. A detector used at every inbound message must be fast, stable, and conservative about false positives. A detector placed before high-impact tool execution may justify additional latency, hidden-state extraction, or ensemble confirmation. A detector applied to retrieved webpages or memory updates may require trusted–untrusted segmentation and uncertainty-based escalation. The present study reports latency, throughput, model size, false-block rate, missed-attack rate, attack recall, benign false-positive rate, escalation rate, and cascade thresholds to connect benchmark performance with operational feasibility.

1.8. Positioning of the Present Study

The present study is positioned at the intersection of prompt-injection detection, web-agent safety, context-aware guardrails, and deployment-oriented evaluation (refer to

Table 1. Recent 2026 literature alignment with the present Agentic Web study.

Evidence Layer	Representative 2026 Sources	Main Insight	Design Response in the Present Study
LLM-agent security architecture	Li et al. (2026) [2]; Kim et al. (2026) [4]; Tamuka et al. (2026) [3]	Agent architectures alter authority boundaries, tool access, memory, and execution predictability.	Frames prompt injection as a system-level Agentic Web control problem rather than only unsafe text.
Prompt-injection surveys	Geng et al. (2026) [5]; Gulyamov et al. (2026) [6]	Recent surveys consolidate attack methods, root causes, direct/indirect injection, tool-based injection, and defense mechanisms.	Supports the extended taxonomy and the distinction between benign, suspicious, and malicious coarse-risk states.
AI-browsers and real web content	Garcia et al. (2026) [22]; Khodayari et al. (2026) [10]	AI browsers and web agents interpret DOM, metadata, headers, comments, and non-rendered content as model inputs.	Motivates context-aware segmentation for browser, RAG, and hidden web-content surfaces.
Tool and coding-agent attacks	Shi et al. (2026) [23]; Maloyan and Namiot (2026) [7]	Tool-selection, tool-description, skill, protocol, file-system, and shell-access surfaces can be manipulated.	Motivates tool-output, tool-selection, and high-impact action-gating analysis.
Guardrails and hidden-state detection	Wang et al. (2025) [31]; Kadali and Papalexakis (2026) [33]; Alshammari and Alsaleh (2026) [9]	Guardrail evaluation requires security-efficiency-utility evidence, and hidden representations can contain jailbreak signals.	Motivates multi-view detection, target-LLM hidden-state features, calibration, and deployment metrics.
Contextual integrity and security-utility limits	Abdelnabi and Bagdasarian (2026) [8]	Contextual manipulation can defeat simple data–instruction separation while strict separation can over-block legitimate flows.	Motivates hard negatives, ambiguous-case handling, and calibrated escalation rather than rigid binary filtering.

). It evaluates PromptSentinel-X as an independent defensive framework using the Kaggle Prompt Injection Malignant dataset as the in-domain benchmark and structured agentic-web evaluation surfaces to approximate browser, retrieval, tool, memory, and multi-turn risks. The study is retrospective and computational, and it avoids printing harmful prompts verbatim. Its scientific objective is to produce auditable evidence about detection validity, context sensitivity, hard-negative behavior, robustness, calibration, and deployment feasibility.

PromptSentinel-X is narrower than end-to-end web-agent benchmarks such as ASB and WASP and multimodal environments such as VisualWebArena, EIA, and VPI-Bench [39,40,41,42,43]. Those studies evaluate agent behavior in executable or visually grounded environments, whereas the present work evaluates textual prompt-risk classification and structured context records. This comparison defines the intended contribution: an independently evaluated screening and routing layer that can be integrated into, but does not replace, an end-to-end agent security architecture.

This positioning is important because the current literature has not converged on a single sufficient defense. Survey work identifies diverse risks across prompt manipulation, tool abuse, protocol exploitation, memory compromise, data exfiltration, and multi-agent propagation [2,3,4,5,6,21]. Empirical studies show that browser content, tool selection, and hidden representations each expose different safety signals and failure modes [7,10,22,23,33]. Benchmark studies emphasize leakage control, utility–security trade-offs, and evaluation under realistic agent conditions [19,25,28,31]. PromptSentinel-X responds to these demands by treating prompt-injection detection as a layered safety-control framework rather than as a one-model leaderboard.

1.9. Limitations and Contributions of the Present Study

The existing evidence remains limited in five respects. First, direct-jailbreak benchmarks underrepresent indirect prompt injection in webpages, retrieved documents, tool descriptions and outputs, memory, system–developer boundary conflicts, and staged multi-turn interactions. Second, random record-level partitions can place related families, paraphrases, or near-duplicates in training and test data. Third, aggregate accuracy can conceal minority-class, false-positive, calibration, and escalation failures. Fourth, benign hard negatives remain undermeasured despite their importance for legitimate security, debugging, educational, multilingual, and prompt-engineering work. Fifth, classifier metrics are often disconnected from latency, throughput, model access, tool privilege, and downstream enforcement; for details, refer to

Table 2. Methodological limitations in prior evidence and responses in the present study.

Limitation in Prior Evidence	Why It Matters for Web-Agent Safety	Design Response in the Present Study
Direct-prompt bias in existing evaluation	Static jailbreak benchmarks may underrepresent browser, retrieval-augmented generation (RAG), tool-use, memory, and multi-turn injection surfaces, although modern LLM agents increasingly browse, reason, retrieve, invoke tools, and operate in realistic web environments where indirect prompt injection can arise through external content rather than direct user prompts alone [13,14,15,16,17,18,19,22,25,26,27,28,40].	The study maps prompts into an agentic-web taxonomy and evaluates context-aware slices across user, browser, RAG, tool, memory, and multi-turn scenarios.
Single-view detector limitations	Text-only classifiers may miss latent, semantic, contextual, or model-internal risk signals; prior work on agentic threats, guardrails, hidden representations, and contrastive learning suggests that robust evaluation should compare multiple representational and decision views rather than relying on one static detector [3,4,5,6,19,31,32,33,44,45].	PromptSentinel-X evaluates encoder, contrastive projection, hidden-state, context, ensemble, and cascade views.
Leakage-prone validation	Random splits can place related prompt families, paraphrases, or near-duplicates across train and test sets, leading to inflated estimates of generalization and weakening claims about performance on unseen attack families or deployment distributions [35,46,47,48].	Group-aware and family-aware splitting are used as primary scientific estimates, with random splitting treated only as a sensitivity comparison.
Undermeasured over-defense	Guardrails may block benign security, debugging, roleplay, quoted, multilingual, or educational content; therefore, attack recall alone is insufficient because deployable systems must also preserve legitimate use and avoid excessive false blocking [11,24,29,30,31,32,34].	Hard negatives and benign false-block metrics are included alongside attack recall and missed-attack rate.
Insufficient context awareness	A model may miss the difference between privileged instruction, user request, retrieved data, browser content, tool output, and memory, even though secure agent behavior depends on preserving instruction hierarchy and source authority across mixed-trust context [1,2,8,21,24,26,36,37,38].	Trusted–untrusted segmentation, overlap features, scenario labels, and calibrated uncertainty are incorporated.
Deployment limitation	Offline accuracy does not determine whether a guardrail can be placed before high-impact actions, because agent safety depends on tool-selection risk, action severity, latency, escalation capacity, and operational placement within agent workflows [2,7,20,21,22,23,32,39].	Latency, throughput, model size, cascade thresholds, escalation rate, and action strategy are analyzed.
Unsafe reporting risk	Prompt-injection papers can unintentionally publish reusable adversarial strings, while responsible LLM-security evaluation requires balancing reproducibility with reduced dissemination of operationally harmful prompt content [5,6,24,29,30,35].	The manuscript reports sanitized identifiers, metadata, aggregate metrics, and error categories without printing harmful prompts verbatim.

.

The present study responds to these limitations through a leakage-mitigated, context-aware, calibrated, and deployment-oriented evaluation protocol. It combines a normalized risk taxonomy, prompt-family-aware partitioning, external stress tests, benign hard negatives, multi-view detector analysis, trusted–untrusted segmentation, calibration, robustness slices, statistical validation, and action-oriented deployment metrics.

The evaluation logic used in the present study is consistent with prior domain-independent benchmarking work emphasizing that model performance should not be judged by a single aggregate score or by algorithmic novelty alone. In renewable-energy forecasting, Zhuhadar showed that model interpretation changed when statistical baselines, machine-learning and foundation-model comparators, uncertainty diagnostics, hierarchical coherence, scenario evaluation, and decision-focused planning metrics were analyzed together rather than reduced to a single leaderboard [49]. The same principle applies to prompt-injection detection for LLM-powered web agents: internal accuracy must be interpreted alongside leakage control, context-specific performance, calibration, robustness, false-block behavior, and deployment feasibility. A related clinical machine-learning study further illustrates why validation design and intended-use framing are essential when benchmark results may appear strong. Zhuhadar evaluated ICU admission-time diabetes phenotyping using leakage-mitigated preprocessing, feature ablations, grouped hospital validation, calibration assessment, threshold-aware evaluation, and SHAP-based interpretability, concluding that the model was best understood as a screening-oriented phenotyping aid rather than a stand-alone diagnostic tool [50]. The present study adopts the same caution for Agentic Web security: PromptSentinel-X is evaluated as a calibrated screening, escalation, and review-triage component, not as an autonomous authorization mechanism or proof that prompt injection has been eliminated.

The contribution should therefore be interpreted as integrated evaluation evidence and a routing framework rather than as a universal prompt-injection defense. The study measures whether risk screening remains useful under family novelty, class imbalance, context shift, and operational constraints; it does not demonstrate complete protection of a live web agent.

1.10. Study Objectives and Contributions

Guided by these limitations, the study asks whether PromptSentinel-X can provide credible and operationally useful prompt-risk evidence for large language model-powered web agents under leakage-controlled validation. The evaluation addresses five questions:

(1)

How does fine-label and coarse-risk performance change when prompt families, paraphrases, and near-duplicates are controlled?

(2)

How does detection vary across static, textual browser, retrieval, tool, memory, and multi-turn contexts, and how much uncertainty is created by low-support categories?

(3)

Do hard negatives, multilingual or obfuscated transformations, and external sources reveal over-defense or source-shift failures?

(4)

What complementary evidence is provided by projection-space geometry, target-model hidden states, model combinations, calibration, and component-sensitivity analyses?

(5)

Which inference profiles are feasible for screening, escalation, sandbox routing, or review under latency, throughput, false-positive, and missed-attack constraints?

The main contributions are:

• A normalized fine/coarse taxonomy and provenance schema for prompt-risk records.
• Family-aware splitting and leakage auditing as the primary generalization estimate.
• A modular detector/evaluation framework combining text, representation, context, optional hidden-state, calibration, and routing views.
• Joint assessment of context sensitivity, hard-negative over-defense, robustness, statistical uncertainty, and deployment cost.
• A responsible-reporting protocol that uses sanitized identifiers and aggregate error categories instead of publishing reusable harmful prompts.

2. Materials and Methods

This retrospective computational study evaluated PromptSentinel-X as a defensive and audit-oriented guardrail for LLM-powered web agents. All analyses were implemented in a versioned and auditable Python workflow, with dataset construction, split assignment, model development, calibration, robustness testing, error analysis, deployment assessment, and safety reporting maintained as analytically distinct stages. The experiments were conducted using Python 3.12.13. The software environment included PyTorch 2.12.0, Transformers 5.9.0, SentenceTransformers 5.5.1, scikit-learn 1.8.0, pandas 3.0.3, NumPy 2.4.6, Matplotlib 3.10.9, UMAP-learn 0.5.12, datasets 4.8.5, accelerate 1.13.0, evaluate 0.4.6, sentencepiece 0.2.1, protobuf 7.35.0, and huggingface-hub 1.16.1. The design therefore treated prompt-injection detection as a mixed-trust safety-control problem rather than as an isolated text-classification task.

No live users were exposed to experimental guardrails, no public systems were targeted, and no model was permitted to execute consequential actions during evaluation. The Kaggle Prompt Injection Malignant dataset served as the in-domain benchmark, while structured agentic-web scenarios and external benchmarks were retained as separate stress-test layers. All comparisons used common schema, split, metric, calibration, and reporting definitions. Raw harmful prompts and complete adversarial contexts were excluded from the manuscript and replaced by sanitized identifiers, labels, hashes, metadata, and aggregate summaries.

2.1. Study Design, Threat Model, and Key Innovations

The methods proceed from the threat model and data design to architecture and training, evaluation, error analysis, deployment, reproducibility, and responsible release. The study introduces three linked methodological innovations. First, it treats prompt-family novelty, rather than random record novelty, as the primary unit of generalization and audits residual family, paraphrase, and near-duplicate overlap. Second, it separates trusted and lower-trust context sources before prediction and maps fine-grained mechanisms to operational benign, suspicious, and malicious states. Third, it connects calibrated risk and uncertainty to deployment actions and evaluates those actions jointly with attack recall, benign false positives, latency, throughput, and escalation. The contribution is therefore an integrated evaluation and routing protocol rather than a new foundation model or a claim of complete prompt-injection prevention.

Key Methodological Innovations and Corresponding Evidence

The methodological design of PromptSentinel-X is organized around three complementary innovations: leakage-aware validation, provenance-aware risk modeling, and calibrated deployment routing (Scheme 1).

First, leakage-aware validation treats prompt-family novelty, rather than random record novelty, as the primary basis for estimating generalization. This innovation was implemented through family-level data partitioning, supplemented by paraphrase and near-duplicate auditing. Its evidentiary basis consists of the comparison between random and group-aware splits, the analysis of group-based prediction slices, and external stress testing. Together, these evaluations help determine whether model performance remains credible when related prompt families and semantically similar examples are prevented from inflating test results.

Second, provenance-aware risk modeling distinguishes trusted or privileged instructions from lower-trust content before risk prediction. This distinction was implemented through trusted–lower-trust context segmentation and a combined fine-grained and coarse-risk taxonomy. Evidence for this innovation is provided by scenario-specific and relationship-specific performance metrics, benign hard-negative evaluation, and sanitized error analyses. These analyses assess whether the framework can account for both the semantic characteristics of an input and the source, privilege, or contextual relationship through which that input reaches the agent.

Third, calibrated deployment routing translates model-estimated risk and uncertainty into proportionate operational responses. Depending on the predicted risk, uncertainty, and potential consequences of the pending action, an input may be routed to allow, warn, review, block, sandbox, or escalation states. The evidence supporting this innovation includes expected calibration error, attack recall, benign false-positive rate, escalation behavior, latency, and throughput. These measures connect predictive performance to deployment practicality and help position PromptSentinel-X as a screening and decision-support layer rather than as a stand-alone authorization mechanism.

The study was operationalized through the five-stage pipeline shown in Figure 1. First, mixed-trust inputs were segmented by source and privilege, distinguishing system and developer instructions from user messages, browser content, retrieved passages, tool outputs, and memory. Second, each record was mapped to a fine-grained label, a coarse risk state, and an agentic attack surface. Third, the detector combined textual, contextual, metadata, and optional target-model hidden-state views. Fourth, validation-derived calibration converted model outputs into risk and uncertainty estimates. Finally, an action policy routed each case to allow, warn, review, block, sandbox, or escalation states according to the authority and consequence of the pending agent action.

The investigation was structured as a design-and-evaluation study rather than as the development of a new foundation model. Its methodological contribution lies in integrating leakage control, provenance-aware segmentation, multi-view risk estimation, calibrated uncertainty, benign hard-negative behavior, robustness testing, and operational routing within a single versioned and auditable guardrail protocol. Figure 1 provides the organizing framework; the following subsections define the data, models, validation procedures, and deployment controls used to instantiate it.

2.2. Data Sources, Taxonomy, and Leakage-Aware Evaluation Design

Dataset construction and evaluation design comprised five linked components: (i) normalization of the Malignant corpus, (ii) extension of the label space to an agentic-web risk taxonomy, (iii) adapter-based external evaluation, (iv) family-aware split construction and leakage screening, and (v) assembly of benign hard negatives. External records were not silently pooled into the principal in-domain training claim; each source retained its dataset, attack-surface, transformation, and split metadata.

2.2.1. Original Malignant Corpus and Normalized Schema

The primary in-domain benchmark was the Kaggle Prompt Injection Malignant dataset, a public defensive corpus curated for jailbreak and prompt-injection research. Following schema normalization, the analytic file contained 1581 English-language records: 1312 benign conversation samples, 199 malicious jailbreak samples, and 70 suspicious roleplay or act_as samples. For details, refer to

Table 3. Dataset construction, taxonomy, and analytical roles.

Construction Layer	Operational Definition	Methodological Implication
Original Malignant	Kaggle in-domain corpus with benign conversation, jailbreak, and roleplay or act_as records.	Provides the controlled benchmark for supervised prompt-risk detection.
Extended taxonomy	Fine labels mapped to benign, suspicious, and malicious coarse classes.	Supports calibrated actions rather than a single binary decision.
Agentic attack surfaces	User, RAG, browser, tool, memory, system/developer boundary, and multi-turn contexts.	Aligns evaluation with LLM-powered web-agent workflows.
Metadata schema	Stable sample identifiers, family identifiers, label fields, language, transformation type, split, and context metadata.	Enables reproducible joining of predictions, metrics, and sanitized error reports.
Safety-preserving reporting	Raw harmful prompts are replaced with identifiers, hashes, labels, and aggregate summaries.	Preserves defensive reproducibility while reducing unsafe dissemination.
External benchmark adapters	Evaluation-only records from prompt-injection, jailbreak, indirect-injection, and hard-negative sources.	Tests transfer across source, style, and attack-surface shifts without contaminating the in-domain claim.
Family-aware split controls	Stable family identifiers keep paraphrases and related variants within one partition; leakage screens audit residual overlap.	Produces a more conservative estimate of generalization to unseen prompt families.
Benign hard negatives	Legitimate security, debugging, quoted, multilingual, prompt-engineering, and roleplay content.	Measures over-defense and category-specific false blocking.

. The observed imbalance was retained for the main analysis because it approximates ordinary agent traffic, in which benign content predominates while malicious instructions remain comparatively rare but safety-critical. The normalized in-domain dataset was represented as follows.

Equation (1). In-domain prompt-risk dataset.

$${\mathcal{D}}_M=\{\left(x_i,y_i,z_i,g_i,a_i,r_i,s_i,c_i\right){\}}_{i=1}^N, N=1581$$

(1)

In Equation (1), ${\mathcal{D}}_M$ denotes the normalized in-domain Malignant dataset, $N=1581$ is the number of records, and $i$ indexes the individual records. For record $i$, $x_i$ denotes the modeling text or structured detector input, $y_i$ denotes the fine-grained label, $z_i$ denotes the coarse-risk label, $g_i$ denotes the family identifier used for leakage control, $a_i$ denotes the agentic attack surface, $r_i$ denotes the transformation type, $s_i$ denotes the assigned partition, and $c_i$ denotes optional structured context.

2.2.2. Extended Risk Taxonomy and Agentic Attack Surfaces

The original three-class label space was preserved for controlled in-domain comparison and mapped into an expanded taxonomy suitable for web-agent safety analysis. Fine labels distinguished ordinary conversation, benign hard negatives, suspicious roleplay or persona shifting, direct jailbreaks, direct and indirect prompt injection, prompt leakage, tool abuse, data exfiltration, obfuscation, and ambiguous or borderline cases. Each sample also retained an attack-surface designation covering user, browser, retrieval-augmented generation (RAG), tool, memory, system/developer boundary, or multi-turn context. Fine labels were mapped to the coarse operational states benign, suspicious, and malicious. The coarse-risk mapping was defined as follows.

Equation (2). Fine-to-coarse risk mapping.

$$z_i=\varphi \left(y_i\right), \varphi :{\mathcal{Y}}_f\to {\mathcal{Y}}_c=\{B,S,M\}$$

(2)

In Equation (2), $y_i\in {\mathcal{Y}}_f$ is the fine-grained label for record $i$, $z_i\in {\mathcal{Y}}_c$ is its coarse-risk label, and $\varphi$ is the deterministic mapping from the fine-label space ${\mathcal{Y}}_f$ to the coarse-risk space ${\mathcal{Y}}_c$. The symbols $B$, $S$, and $M$ denote benign, suspicious, and malicious, respectively.

2.2.3. External Benchmark Adapters

External benchmark adapters were used to examine cross-source generalization beyond the Malignant distribution. The inventory included Open-Prompt-Injection, CAPTURE-style indirect prompt-injection data, HarmBench, JailbreakBench, and InjecGuard/NotInject-style benign hard negatives. These sources were treated as evaluation stressors rather than as pooled training evidence for the primary in-domain claim, thereby preserving a clear distinction between internal discrimination and external robustness; for details, refer to

Table 4. PromptSentinel-X components and evaluation roles.

Method Layer	Main Implementation	Role in the Study
Encoder and classifier heads	Configurable encoder with fine-label and coarse-risk heads.	Measures supervised prompt-risk classification.
Projection head	Normalized projection space trained with contrastive supervision.	Supports separable risk geometry and uncertainty-aware routing.
Context segmentation	Trusted and untrusted fields separated before feature extraction.	Evaluates indirect prompt injection across RAG, browser, tool, memory, and multi-turn surfaces.
Hidden-state detector	Cached target-LLM hidden states with lightweight classifiers.	Tests whether internal model representations provide complementary risk signals.
Calibrated ensemble	Probability averaging, voting, stacking, and cascade inference.	Converts model scores into deployable action policies.
Robustness evaluation	Perturbation, multilingual, obfuscation, long-context, roleplay, RAG, and cross-source slices.	Tests whether performance survives realistic agentic-web variation.

.

External records were admitted only when usable text and label metadata could be mapped to the common schema. Records without usable text were skipped with scoped status messages rather than manually reconstructed, and qualifying examples retained an external-test assignment unless a prespecified ablation required another treatment. This rule prevented undocumented repair or training contamination from obscuring the provenance of cross-source results.

No sample-level manual reannotation was performed by this study. Label harmonization was deterministic at the dataset and label level. Each source was mapped through adapter rules that recorded the source dataset, original available label or status field when present, mapped fine-grained label, mapped coarse-risk label, split assignment, and exclusion status. Records with missing usable text were excluded and counted.

Records whose labels could not be mapped from available metadata were assigned only through the prespecified adapter default for that source; no post hoc manual relabeling was applied.

Automated checks verified unique sample identifiers, allowable labels, source totals, and consistency between fine-grained and coarse labels. The adapter inventory, exclusion counts, source summaries, and label summaries are included with the sanitized reproducibility artifacts. Because no sample-level manual adjudication was performed, no inter-rater agreement statistic is reported.

2.2.4. Leakage-Aware Family Splits

Family-aware partitioning was selected because random record-level splitting can produce optimistic estimates when dependent, paraphrased, or near-duplicate samples cross partitions. Leakage in machine-learning evaluation has been shown to inflate apparent generalization and blocked or grouped validation is recommended when observations share hierarchical or family structure [46,47,48]. Prompt-injection benchmarking also formalizes the need to separate injected tasks and attack families when comparing defenses [35].

Family identifiers were constructed before model evaluation. For the Malignant reproduction, original jailbreak and act_as prompts were assigned sequential source-family identifiers, while conversation examples received unique row-level family identifiers. Paraphrase variants were grouped by assigning each paraphrase to the nearest original prompt within the same class using cosine similarity over MiniLM sentence embeddings.

The group-aware split then used original_family_id with GroupShuffleSplit, keeping all members of a family in the same split. Residual near-duplicate warnings were identified after lowercasing and alphanumeric normalization using difflib.SequenceMatcher similarity across train/test text pairs, with a fixed threshold of 0.92. These procedures and thresholds were fixed and applied before model evaluation.

The primary scientific partition was group-aware: all records sharing the same prompt-family identifier were assigned to a single partition, including source prompts and assigned paraphrase variants. This design reduced the risk that a model would be evaluated on wording closely related to its training examples. Random splitting was retained only as a sensitivity comparator, because deployment requires generalization to previously unseen prompts, sources, contexts, languages, and attack families. The family-exclusion constraint was specified as follows.

Equation (3). Group-aware split constraint.

$$g_i=g_j\Rightarrow s_i=s_j, {\mathcal{G}}_{tr}\cap {\mathcal{G}}_{te}=\mathrm{⌀}$$

(3)

In Equation (3), $g_i$ and $g_j$ are family identifiers, $s_i$ and $s_j$ are partition assignments, and ${\mathcal{G}}_{\mathrm{t}\mathrm{r}}$ and ${\mathcal{G}}_{\mathrm{t}\mathrm{e}}$ are the sets of families assigned to the training and test partitions, respectively. The symbol $\mathrm{⌀}$ denotes the empty set. Thus, records sharing a family identifier must receive the same partition assignment, and the training and test family sets must be disjoint.

2.2.5. Benign Hard Negatives

Benign hard negatives were intentionally non-malicious but lexically or structurally challenging. Categories included cybersecurity discussion, debugging text, educational security analysis, quoted or encoded material, prompt-engineering questions, multilingual examples, and benign roleplay. Their purpose was to quantify over-defense: a guardrail that blocks these legitimate activities may appear conservative while being operationally unsuitable for software-development, educational, compliance, or security-analysis workflows. The category-specific false-block rate for hard negatives was defined as follows.

Equation (4). Hard-negative false-block rate.

$${\mathrm{F}\mathrm{B}\mathrm{R}}_k={\displaystyle \frac{1}{\left|{\mathcal{H}}_k\right|}}{\sum }_{i\in {\mathcal{H}}_k}1\{A_i\in {\mathcal{A}}_F\}$$

(4)

In Equation (4), ${\mathrm{F}\mathrm{B}\mathrm{R}}_k$ is the false-block rate for benign hard-negative category $k$, ${\mathcal{H}}_k$ is the set of examples in that category, and $\left|{\mathcal{H}}_k\right|$ is its support. $A_i$ is the action assigned to example $i$, ${\mathcal{A}}_F$ is the set of restrictive actions counted as false alarms, and $\mathbf{1}\{\cdot \}$ is the indicator function, which equals 1 when its condition is satisfied and 0 otherwise.

2.3. PromptSentinel-X Architecture and Training

PromptSentinel-X was implemented as a modular multi-view prompt-risk detector for LLM-powered web agents. The framework combined encoder-based classification, projection-space representation learning, dual fine-label and coarse-risk prediction, provenance-aware context features, optional target-LLM hidden states, post hoc calibration, model combination, and deployment-oriented routing. Components were evaluated independently and in combination under shared split and metric definitions so that architectural effects could be distinguished from dataset and policy effects.

2.3.1. PromptSentinel-X Architecture

The base architecture comprised a configurable text encoder, a projection head, a fine-label classifier, a coarse-risk classifier, calibration hooks, and optional feature-fusion modules. The encoder produced the contextual text representation, the projection head mapped that representation into a normalized metric-learning space, and the two classifier heads generated mechanism-specific and operational risk distributions.

The principal PromptSentinel-X result used the retained embedding-mode configuration. Inputs were 384-dimensional MiniLM sentence embeddings associated with paraphrase-multilingual-MiniLM-L12-v2; however, the exact Hugging Face/Sentence Transformer model revision and tokenizer revision were not pinned in the retained artifacts. Because the principal run used precomputed embedding features rather than raw-text transformer tokenization, no tokenizer was invoked during the model training.

The PromptSentinel-X model used an embedding encoder with output dimension 384, followed by a projection head with dimensions 384 → 512 → 256 → 128. The projection head used LayerNorm, GELU activations, and dropout of 0.1. The fine-label classifier was a linear 128 → 3 head for jailbreak, act_as, and conversation; the coarse-risk classifier was a linear 128 → head for benign, suspicious, and malicious.

Training used AdamW with learning rate 0.001, weight decay 0.0001, batch size 128, class weighting enabled, and random seed 42. The run was configured for 8 epochs with early stopping patience of 3 and stopped after 4 epochs. Checkpoint selection used validation fine-label macro-F1. The loss combined fine-label cross-entropy with weight 1.0, coarse-risk cross-entropy with weight 0.5, and supervised contrastive loss with weight 0.1 and temperature 0.1. Focal loss was disabled (gamma = 0.0) and center loss was disabled (weight = 0.0). Post hoc temperature scaling and OOD statistics were enabled.

The focal PromptSentinel-X model used the same in-domain split sizes as the target-LLM hidden-state experiment. The group-aware split first assigned 1116 Malignant records to the non-test side and preserved 465 records as the held-out family-aware test set. Because the saved group-aware split-assignment file did not include a validation subset, the shared split builder created validation data from the 1116 non-test records using a row-level stratified train_test_split with validation size 0.15 and random seed 42. This produced 948 training records and 168 validation records. The secondary train/validation split was stratified by fine label but was not group-aware; 28 original_family_id values crossed training and validation. The 465-record family-aware test set remained untouched for final evaluation and had no original_family_id overlap with either training or validation. The core forward pass was represented as follows.

Equation (5). Encoder, projection, and dual-head prediction.

$$\begin{array}{ll}{e}_i& =E_{\theta }\left(x_i\right),\\ p_i& ={\displaystyle \frac{G_{\psi }\left(e_i\right)}{{∥G_{\psi }\left(e_i\right)∥}_2}},\\ q_i^f& =\mathrm{s}\mathrm{o}\mathrm{f}\mathrm{t}\mathrm{m}\mathrm{a}\mathrm{x}\left(W_f{e}_i+b_f\right),\\ q_i^c& =\mathrm{s}\mathrm{o}\mathrm{f}\mathrm{t}\mathrm{m}\mathrm{a}\mathrm{x}\left(W_c{e}_i+b_c\right)\end{array}$$

(5)

In Equation (5), $E_{\theta }$ is the encoder, $G_{\psi }$ is the projection head, $e_i$ is the encoder representation of input $x_i$, and $p_i$ is the L2-normalized projection vector. The matrices $W_f$ and $W_c$, together with the bias vectors $b_f$ and $b_c$, parameterize the fine-label and coarse-risk classifiers. The vectors $q_i^f$ and $q_i^c$ contain the corresponding predicted class probabilities.

2.3.2. Multi-View Representation

The multi-view design avoided making a safety decision from one textual representation alone. The text view captured lexical and semantic evidence; the projection view represented learned risk geometry; the context view encoded provenance, segment relationships, and trusted–untrusted overlap; the optional hidden-state view captured target-model activations; and the metadata view retained source, attack surface, transformation, and routing information. These views could be concatenated, stacked, averaged, or used sequentially in a cascade. The fused representation was defined as follows.

Equation (6). Multi-view feature fusion.

$$v_i=\left[e_i\hspace{0.25em};\hspace{0.25em}{p}_i\hspace{0.25em};\hspace{0.25em}{c}_i\hspace{0.25em};\hspace{0.25em}{h}_i\hspace{0.25em};\hspace{0.25em}{m}_i\right], {\tilde{v}}_i=F_{\omega }\left(v_i\right)$$

(6)

In Equation (6), $v_i$ is the concatenated multi-view feature vector, $e_i$ and $p_i$ are the encoder and projection representations, $c_i$ contains context-aware segment features, $h_i$ contains optional target-LLM hidden-state features, and $m_i$ contains metadata features. The semicolons inside the brackets denote feature concatenation. $F_{\omega }$ is an optional fusion module, and ${\tilde{v}}_i$ is the resulting fused representation.

2.3.3. Training Objectives

Training jointly optimized supervised classification and representation structure. The supervised contrastive term encouraged samples sharing a configured fine or coarse risk relation to occupy nearby regions of projection space while separating samples with dissimilar labels. This objective was intended to improve the organization of low-support and boundary cases, including benign roleplay, suspicious persona shifting, and malicious jailbreak behavior [44]. The supervised contrastive loss was defined as follows.

Equation (7). Supervised contrastive objective.

$$s_{ij}=p_i^{\mathrm{\top }}{p}_j, {\mathcal{L}}_{sc}=-{\displaystyle \frac{1}{N}}{\displaystyle {\sum }_{i=1}^N}{\displaystyle \frac{1}{\left|P\left(i\right)\right|}}{\sum }_{j\in P\left(i\right)}\mathrm{l}\mathrm{o}\mathrm{g}{\displaystyle \frac{\mathrm{e}\mathrm{x}\mathrm{p}\left(s_{ij}/\tau \right)}{\sum _{a\in A\left(i\right)}\mathrm{e}\mathrm{x}\mathrm{p}\left(s_{ia}/\tau \right)}}$$

(7)

In Equation (7), $s_{ij}=p_i^{\mathsf{T}}{p}_j$ is the projection-space cosine similarity between the normalized vectors $p_i$ and $p_j$. ${\mathcal{L}}_{\mathrm{s}\mathrm{c}}$ is the supervised contrastive loss, $N$ is the number of samples, $P\left(i\right)$ is the positive set for anchor $i$, $\left|P\left(i\right)\right|$ is the number of positive examples for that anchor, $A\left(i\right)$ is the comparison set excluding the anchor, and $\tau$ is the temperature parameter.

Equation (8). Total multi-objective training loss.

$${\mathcal{L}}_{tot}={\lambda }_f{\mathcal{L}}_{ce}\left(y_i,q_i^f\right)+{\lambda }_c{\mathcal{L}}_{ce}\left(z_i,q_i^c\right)+{\lambda }_s{\mathcal{L}}_{sc}+{\lambda }_a{\mathcal{L}}_{aux}$$

(8)

In Equation (8), ${\mathcal{L}}_{\mathrm{t}\mathrm{o}\mathrm{t}}$ is the total training loss, ${\mathcal{L}}_{\mathrm{c}\mathrm{e}}$ is the cross-entropy loss, $y_i$ and $z_i$ are the fine-grained and coarse-risk targets, and $q_i^f$ and $q_i^c$ are their corresponding predicted probability vectors. ${\mathcal{L}}_{\mathrm{s}\mathrm{c}}$ is the supervised contrastive loss, ${\mathcal{L}}_{\mathrm{a}\mathrm{u}\mathrm{x}}$ represents optional auxiliary losses, and ${\lambda }_f$, ${\lambda }_c$, ${\lambda }_s$, and ${\lambda }_a$ are configurable loss weights.

2.3.4. Context-Aware Segmentation

Context-aware segmentation preserved provenance and authority before risk features were computed. System instructions and developer policy were treated as privileged segments; user messages, retrieved passages, browser content, tool outputs, memory records, and conversation history were treated as lower-trust or untrusted segments unless explicitly authorized by policy. The method therefore modeled not only what a segment said but also where it originated and whether it was permitted to influence the agent’s behavior.

Scope of browser representation. In this study, browser content denotes text and metadata already exposed to the detector through the structured context records. The evaluation does not ingest raw DOM topology, CSS visibility rules, screenshots, rendered pixels, OCR output, or visual overlays, and it does not independently classify ARIA attributes unless their text is present in the provided record. Accordingly, the results are described as textual browser-content injection detection and should not be generalized to non-textual or multimodal browser attacks. The structured context representation was specified as follows.

Equation (9). Context segment set.

$${\mathcal{C}}_i=\{S_i^{sys},S_i^{dev},U_i,R_i,B_i,T_i,M_i,H_i\}$$

(9)

In Equation (9), ${\mathcal{C}}_i$ is the structured context associated with record $i$. $S_i^{\mathrm{s}\mathrm{y}\mathrm{s}}$ and $S_i^{\mathrm{d}\mathrm{e}\mathrm{v}}$ are the privileged system and developer segments, $U_i$ is the user query, $R_i$ is retrieved context, $B_i$ is textual browser content, $T_i$ is tool output, $M_i$ is memory content, and $H_i$ is conversation history. A trusted–untrusted overlap score was computed to identify untrusted content that semantically resembles instructions.

Equation (10). Trusted–untrusted similarity score.

$${\rho }_i=\underset{\mathcal{l}\in {\mathcal{L}}_i,\hspace{0.33em}p\in {\mathcal{P}}_i}{\mathrm{m}\mathrm{a}\mathrm{x}}\mathrm{c}\mathrm{o}\mathrm{s}\left(\mathrm{E}\mathrm{n}\mathrm{c}\left(\mathcal{l}\right),\mathrm{E}\mathrm{n}\mathrm{c}\left(p\right)\right)$$

(10)

In Equation (10), L_i is the set of lower-trust segments for record i, P_i is the set of privileged system and developer instruction segments, and Enc(.) is the text encoder. The cosine function denotes cosine similarity, and P_i is the maximum semantic similarity between any lower-trust segment and any privileged segment. A high value is treated as a cautionary routing signal rather than as proof of maliciousness.

2.3.5. Target-LLM Hidden-State Detector

A target-LLM hidden-state detector was evaluated as a complementary risk sensor when local access to an instruction-tuned model was available. Hidden-state vectors were extracted at prespecified layers and summarized through mean or last-token pooling before classification with logistic regression, linear support-vector machines, or shallow multilayer perceptrons. Features were cached to separate extraction cost from downstream classifier comparison.

Hidden-state features were cached for all 1581 in-domain Malignant records, but the downstream lightweight classifiers did not use a separate random split or cross-validation. They used the same leakage-aware group split defined by original_family_id. The parent group-aware partition contained 1116 non-test records and 465 held-out test records. For classifier fitting, the 1116 non-test records were further divided into 948 classifier-training records and 168 validation records using the shared split builder with random seed 42, while the 465-record group-aware test set was preserved unchanged. Logistic regression, linear SVM, and shallow MLP classifiers were trained on cached hidden-state vectors from the 948 training records and evaluated on the same 465-record family-aware test set. No separate record-level random partition, cross-validation procedure, or external test partition was used for the internal hidden-state ablation scores. The hidden-state detector was represented as follows.

Equation (11). Hidden-state risk detector.

$$h_i^{\left(\mathcal{l},p\right)}={\mathrm{P}\mathrm{o}\mathrm{o}\mathrm{l}}_p\left(H_{\mathcal{l}}\left(x_i\right)\right), q_i^h=\mathrm{s}\mathrm{o}\mathrm{f}\mathrm{t}\mathrm{m}\mathrm{a}\mathrm{x}\left(W_h{h}_i^{\left(\mathcal{l},p\right)}+b_h\right)$$

(11)

In Equation (11), $H_{\mathcal{l}}\left(x_i\right)$ is the hidden-state sequence produced for input $x_i$ at layer $\mathcal{l}$, ${\mathrm{P}\mathrm{o}\mathrm{o}\mathrm{l}}_p$ is pooling strategy $p$, and $h_i^{\left(\mathcal{l},p\right)}$ is the resulting pooled hidden-state vector. $W_h$ and $b_h$ are the parameters of the hidden-state classifier, and $q_i^h$ is the resulting hidden-state risk-probability vector.

2.3.6. Calibration and Model Combination

Model combination was evaluated after individual-view prediction through probability averaging, majority voting, stacked classification, and a deployment-style cascade. Calibration was applied because raw probabilities can be overconfident under class imbalance or distribution shift. The calibrated outputs were interpreted as risk evidence for later action selection rather than as self-sufficient authorization decisions.

For the ensemble analyses, probability averaging and majority voting used seven prespecified prediction views: MiniLM/Siamese, ModernBERT-base transformer classifier, BGE-M3 embedding classifier, target-LLM hidden-state detector, Prompt Guard 2, Llama Guard, and PromptSentinel-X. These views were aligned by sample_id, and all seven provided predictions for the same 465 records in the group-aware test set. Probability averaging used the arithmetic mean of scalar calibrated risk scores. Majority voting first selected the coarse label by vote count and then selected the fine label among model predictions compatible with that coarse label. Ties were resolved deterministically by sorting tied labels lexically after sorting by descending vote count.

The cascade thresholds were retained in the synthesis configuration step, but the archived artifacts do not document a validation-threshold sweep. Therefore, these values are reported as fixed configured thresholds rather than validation-selected thresholds. The cascade used a low escalation threshold of 0.25 and a high escalation threshold of 0.75: scores in the open interval (0.25, 0.75) were considered ambiguous and routed from the fast model to PromptSentinel-X, and then to the slow safety judge if still ambiguous. Final action thresholds were risk score < 0.35 for allow, 0.35 ≤ risk score < 0.65 for warn, and risk score ≥ 0.65 for block, unless uncertainty exceeded 0.45, in which case the action was human-review. Uncertainty was computed as 1 − |score − 0.5| × 2, and model disagreement raised uncertainty to at least 0.5. Probability averaging was defined as follows.

Equation (12). Ensemble probability aggregation.

$${\bar{q}}_i={\displaystyle {\sum }_{m=1}^M}{\alpha }_m{q}_i^{\left(m\right)}, {\alpha }_m\ge 0, {\displaystyle {\sum }_{m=1}^M}{\alpha }_m=1$$

(12)

In Equation (12), $M$ is the number of component models, $q_i^{\left(m\right)}$ is the probability vector produced by component model $m$ for sample $i$, and ${\alpha }_m$ is the nonnegative weight assigned to that model. The weights sum to one, and ${\overline{q}}_i$ is the aggregated ensemble probability vector. Under equal-weight averaging, ${\alpha }_m=1/M$. Here, $M$ denotes the number of models and should not be confused with the malicious coarse-risk label used in Equation (2).

Temperature scaling was used as a calibration layer when validation predictions supported it [45].

Equation (13). Temperature-scaled calibration.

$$q_i^{cal}=\mathrm{s}\mathrm{o}\mathrm{f}\mathrm{t}\mathrm{m}\mathrm{a}\mathrm{x}\left({\displaystyle \frac{o_i}{T}}\right), T>0$$

(13)

In Equation (13), $o_i$ is the uncalibrated logit vector for sample $i$, $T>0$ is the temperature estimated from validation predictions, and $q_i^{\mathrm{c}\mathrm{a}\mathrm{l}}$ is the resulting temperature-scaled calibrated probability vector.

Equation (14). Expected calibration error.

$$\mathrm{E}\mathrm{C}\mathrm{E}={\displaystyle {\sum }_{b=1}^B}{\displaystyle \frac{\left|{\mathcal{B}}_b\right|}{N}}\left|\mathrm{a}\mathrm{c}\mathrm{c}\left({\mathcal{B}}_b\right)-\mathrm{c}\mathrm{o}\mathrm{n}\mathrm{f}\left({\mathcal{B}}_b\right)\right|$$

(14)

In Equation (14), $\mathrm{E}\mathrm{C}\mathrm{E}$ is the expected calibration error, $B$ is the number of confidence bins, ${\mathcal{B}}_b$ is the set of samples assigned to bin $b$, and $\left|{\mathcal{B}}_b\right|$ is the number of samples in that bin. $N$ is the total number of evaluated samples, $\mathrm{a}\mathrm{c}\mathrm{c}\left({\mathcal{B}}_b\right)$ is the empirical accuracy in bin $b$, and $\mathrm{c}\mathrm{o}\mathrm{n}\mathrm{f}\left({\mathcal{B}}_b\right)$ is the mean predicted confidence in that bin.

2.3.7. Robustness Evaluation

Robustness evaluation examined whether detector behavior remained useful under perturbations and transfer conditions relevant to web agents. Prespecified categories included paraphrase, typographical and spacing noise, multilingual input, obfuscation, long context, roleplay confusion, RAG injection, textual browser-content injection, tool-output injection where available, multi-turn staging, and cross-dataset transfer. Results were stratified by category and support rather than collapsed into a single pooled robustness score. Robustness degradation for category was defined as follows.

Equation (15). Category-specific robustness degradation.

$${\Delta }_k^r=M\left({\mathcal{D}}_0\right)-M\left({\mathcal{D}}_k\right)$$

(15)

In Equation (15), ${\Delta }_k^{\mathrm{r}\mathrm{o}\mathrm{b}}$ is the observed robustness degradation for category $k$, ${\mathcal{D}}_0$ is the reference evaluation set, ${\mathcal{D}}_k$ is the transformed or external robustness slice, and $\mathcal{S}\left(\cdot \right)$ is the selected evaluation score. For higher-is-better measures, a positive value of ${\Delta }_k^{\mathrm{r}\mathrm{o}\mathrm{b}}$ denotes deterioration relative to the reference set.

2.4. Evaluation Protocol, Component Sensitivity, and Statistical Validation

Ablation analysis evaluated the sensitivity of the evidence to loss functions, model components, and dataset design. Each comparison used the same split definitions, metric conventions, and sanitized reporting rules as the focal model. Where a fully controlled component-removal retraining was unavailable, the comparison was explicitly labeled as an artifact-based or proxy analysis and was interpreted as sensitivity evidence rather than causal attribution. The generic ablation effect was defined as follows.

Equation (16). Component-sensitivity effect.

$${\Delta }_m=M\left({\mathcal{M}}_0\right)-M\left({\mathcal{M}}_{-m}\right)$$

(16)

In Equation (16), ${\mathcal{M}}_0$ is the full configuration, ${\mathcal{M}}_{-m}$ is the configuration in which component $m$ is removed or replaced, and $\mathcal{S}(\cdot )$ is the selected evaluation score. The quantity ${\Delta }_m$ is the observed score difference between the two configurations. Unless identical controlled retraining was performed, ${\Delta }_m$ should be interpreted as component-sensitivity evidence rather than as a causal effect.

2.4.1. Loss-Function Ablations

Loss-function comparisons included cross-entropy-only training, cross-entropy with supervised contrastive learning, focal-loss variants, center or proxy-style clustering terms, and calibration-related auxiliary objectives. Evaluation emphasized macro-F1, class-wise recall, projection diagnostics, calibration, and hard-negative behavior because a loss that improves aggregate accuracy can still worsen minority-class recovery or benign false blocking.

2.4.2. Model-Component Ablations

Model-component analyses removed or replaced the projection head, coarse-risk head, context segmentation, hidden-state view, calibration layer, and ensemble or cascade mechanism. The objective was to determine which components supported class separation, operational risk mapping, context sensitivity, uncertainty handling, and deployability rather than to select a component solely from one aggregate score.

2.4.3. Dataset-Component Ablations

Dataset-component analyses examined the influence of external-test coverage, benign hard negatives, family-aware splitting, paraphrase-family control, and context-specific scenario data. Malignant-only conditions measured in-domain dependence, no-hard-negative conditions assessed over-defense, and random-split conditions quantified the optimism introduced when related prompt families could cross partition boundaries.

2.4.4. Statistical Validation

Statistical validation was performed on saved, sample-aligned predictions using five group-based prediction slices, nonparametric bootstrap summaries, McNemar tests for paired correctness, paired bootstrap comparisons for macro-F1 differences, Wilson score intervals for support-based proportions, calibration curves, and sanitized error analysis. These analyses used stable sample identifiers and did not require raw prompt text.

The group-aware component was limited to the five prediction slices. Saved predictions were merged with original_family_id, and five-fold GroupKFold was applied to construct family-disjoint evaluation slices. These folds were used as evaluation slices only; models were not retrained within the five folds. The reported group-slice mean and standard deviation were computed from the five slice-level macro-F1 values. The interval reported for the group-slice macro-F1 summary was a 200-repetition percentile bootstrap interval over the mean of these five slice-level macro-F1 values, using random seed 42 and the 2.5th and 97.5th percentiles. Because this interval was bootstrapped from only five slice-level values, it should be interpreted as a descriptive stability summary rather than as a full family-level bootstrap confidence interval.

The retained implementation did not perform a cluster bootstrap or family-level resampling for all reported bootstrap intervals. Accuracy intervals for individual model views were computed by resampling record-level correctness indicators with replacement. Paired bootstrap comparisons were computed after aligning model predictions by sample_id, resampling individual records with replacement, and recomputing macro-F1 differences. These procedures used percentile intervals, were not stratified, did not use BCa correction, and did not preserve prompt-family clusters during resampling. Consequently, these bootstrap intervals may underestimate uncertainty when related prompt-family members remain statistically dependent. They are therefore interpreted as exploratory uncertainty summaries over saved predictions rather than as fully leakage-aware confirmatory confidence intervals.

For support-based proportions, 95% Wilson score intervals were computed from the observed numerator and denominator. These intervals were reported for scenario accuracy and, where both class counts were available, for attack recall and benign false-positive rate. Low-support rows were interpreted descriptively even when the observed point estimate was high.

The retained implementation did not apply a multiple-comparison adjustment. Pairwise tests were therefore interpreted as exploratory analyses over prespecified model comparisons rather than as familywise-error-controlled confirmatory tests. Models evaluated on nonidentical support were not treated as directly equivalent. Sanitized error-category counts were analyzed alongside the statistical summaries to connect average performance with recurrent failure modes.

2.5. Error Analysis and Support-Aware Interpretation

Error analysis used saved predictions joined to sanitized sample identifiers, labels, hashes, attack-surface metadata, transformation types, confidence values, and split assignments. The objective was to identify actionable failure mechanisms without reproducing reusable adversarial content. Errors were organized into false negatives, false positives, ambiguous cases, and multilingual failures.

2.5.1. False Negatives

False negatives were malicious or high-risk inputs assigned to benign or otherwise insufficiently restrictive outcomes. They were categorized by mechanism, including direct jailbreak, indirect injection, malicious roleplay, prompt leakage, tool abuse, data exfiltration, obfuscation, long-context placement, and multi-turn staging. For deployment interpretation, errors were additionally stratified by the authority and reversibility of the affected agent action. The category-specific false-negative rate was defined as follows.

Equation (17). Category-specific false-negative rate.

$$FN{R}_k={\displaystyle \frac{F{N}_k}{F{N}_k+T{P}_k}}$$

(17)

In Equation (17), ${\mathrm{F}\mathrm{N}\mathrm{R}}_k$ is the false-negative rate for category $k$, ${\mathrm{F}\mathrm{N}}_k$ is the number of false negatives in that category, and ${\mathrm{T}\mathrm{P}}_k$ is the number of true positives in that category.

2.5.2. False Positives

False positives were benign inputs assigned to malicious, block, sandbox, or unnecessary-review outcomes. Analysis focused on legitimate security discussion, trigger-word cases, debugging, quoted or encoded text, prompt-engineering questions, multilingual examples, and roleplay. These categories measured the usability cost of over-defense in technical, educational, compliance, and security workflows. The benign false-positive rate was defined as follows.

Equation (18). Benign false-positive rate.

$$FP{R}_B={\displaystyle \frac{F{P}_B}{F{P}_B+T{N}_B}}$$

(18)

In Equation (18), ${\mathrm{F}\mathrm{P}\mathrm{R}}_B$ is the benign false-positive rate, ${\mathrm{F}\mathrm{P}}_B$ is the number of benign examples incorrectly assigned to restrictive actions, and ${\mathrm{T}\mathrm{N}}_B$ is the number of benign examples correctly handled as non-malicious.

2.5.3. Ambiguous Cases

Ambiguous cases were retained as a methodological category rather than discarded as annotation noise. They included roleplay with uncertain intent, dual-use security questions, benign text containing adversarial vocabulary, educational examples, quoted unsafe material, and cases whose risk depended on tool privilege or surrounding context. Their analysis combined calibrated probability, ensemble disagreement, trusted–untrusted overlap, and out-of-distribution indicators. Uncertainty was defined as follows.

Equation (19). Predictive uncertainty score.

$$u_i=1-\underset{k}{\mathrm{m}\mathrm{a}\mathrm{x}}{q}_i^{cal}\left(k\right)$$

(19)

In Equation (19), $u_i$ is the predictive-uncertainty score for sample $i$, $q_i^{\mathrm{c}\mathrm{a}\mathrm{l}}\left(k\right)$ is the calibrated probability assigned to class $k$, and the maximum is taken over all candidate classes. A high uncertainty score does not by itself indicate maliciousness; instead, it supports warning, sandboxing, richer-model evaluation, or human review.

2.5.4. Multilingual Failures

Multilingual evaluation was analyzed separately because language shift can alter both lexical cues and semantic intent. The slice included non-English, translated, and code-mixed examples as well as multilingual or obfuscated transformation metadata. The analysis assessed attack recall, benign false-positive behavior, and the tendency to over-route legitimate multilingual content to restrictive actions, with interpretation conditioned on language-specific support.

2.6. Deployment and Reproducibility

Deployment analysis translated benchmark performance into operational constraints for LLM-powered web agents. The evaluated variants comprised a fast single-view screening detector, PromptSentinel-X, probability-averaged ensemble inference, and cascade inference. Safety metrics were interpreted jointly with latency, throughput, model size, missed-attack rate, benign false-positive rate, calibration, and escalation burden.

2.6.1. Latency and Throughput

Per-sample inference latency and throughput were measured under the preprocessing and feature-availability assumptions of each deployment variant. The protocol distinguished low-cost inbound screening from richer multi-view or ensemble evaluation. This distinction reflects the different timing requirements of user-message screening, retrieved-document filtering, browser-content inspection, memory-write gating, and pre-tool-execution authorization. Mean inference latency was defined as follows.

Equation (20). Mean inference latency.

$$L_{ms}={\displaystyle \frac{1000}{R}}{\displaystyle {\sum }_{r=1}^R}\left(t_r^e-t_r^s\right)$$

(20)

In Equation (20), $L_{\mathrm{m}\mathrm{s}}$ is the mean inference latency in milliseconds, $R$ is the number of repeated runs, $t_r^s$ is the start time of run $r$, and $t_r^e$ is its end time. The factor 1000 converts elapsed time from seconds to milliseconds.

2.6.2. Cascade Thresholds

Cascade thresholds were fixed in the retained synthesis configuration. The archived artifacts did not preserve evidence of a validation-threshold sweep; consequently, these values are reported as configured operating points rather than as validation-selected optima.

Let ${\pi }_i^{\mathrm{m}\mathrm{a}\mathrm{l}}$ denote the calibrated malicious-risk probability for sample $i$, and let $u_i$ denote its predictive uncertainty. Inputs below a conservative allow threshold were passed to the agent. Inputs above a block threshold were blocked or sandboxed. Inputs between thresholds, or above an uncertainty threshold, were escalated to a richer detector, sandboxed tool execution, or human review. The basic action policy was specified as follows.

Equation (21). Calibrated cascade action policy.

$$A_i=\left\{\begin{array}{ll}{a}_0,& {\pi }_i^{\mathrm{m}\mathrm{a}\mathrm{l}}<{\tau }_A\wedge u_i<{\tau }_U,\\ a_1,& u_i\ge {\tau }_U\vee \left({\tau }_A\le {\pi }_i^{\mathrm{m}\mathrm{a}\mathrm{l}}<{\tau }_B\right),\\ a_2,& {\pi }_i^{\mathrm{m}\mathrm{a}\mathrm{l}}\ge {\tau }_B\wedge u_i<{\tau }_U\end{array}\right.$$

(21)

In Equation (21), $A_i$ is the action assigned to sample $i$, ${\pi }_i^{\mathrm{m}\mathrm{a}\mathrm{l}}$ is its calibrated malicious-risk probability, and $u_i$ is its predictive uncertainty. The actions $a_0$, $a_1$, and $a_2$ denote allow, warn/review or escalation, and block/sandbox, respectively. The parameters ${\tau }_A$, ${\tau }_B$, and ${\tau }_U$ are the allow, block, and uncertainty thresholds. Under this policy, high-uncertainty cases are routed to warning, review, or escalation even when their risk score is high, whereas high-risk, low-uncertainty cases are routed directly to block or sandbox.

2.6.3. Escalation Strategy

The escalation strategy implemented staged risk control. Stage 1 applied a fast detector to ordinary user and browser inputs. Stage 2 invoked PromptSentinel-X or another richer view for uncertain cases, elevated trusted–untrusted similarity, contextual scenarios, or high-impact actions. Stage 3 routed unresolved or high-severity cases to sandboxing, explicit confirmation, human review, or policy-specific blocking. Escalation was treated as both a safety mechanism and an operational cost. The escalation rate was defined as follows.

Equation (22). Escalation rate.

$$ER={\displaystyle \frac{1}{N}}{\displaystyle {\sum }_{i=1}^N}1\left\{A_i\in {\mathcal{A}}_E\right\}.$$

(22)

In Equation (22), $\mathrm{E}\mathrm{R}$ is the escalation rate, $N$ is the number of evaluated samples, $A_i$ is the action assigned to sample $i$, and ${\mathcal{A}}_E$ is the set of escalation actions, including warning, sandboxing, human review, or richer-model routing. The indicator function $1\{\cdot \}$ equals 1 when $A_i\in {\mathcal{A}}_E$ and 0 otherwise.

2.6.4. Computational Environment and Training Cost

The archived artifacts available for this revision did not retain hardware identifiers, software-version manifests, wall-clock training time, GPU-hours, or peak memory. These values are therefore reported as not recorded rather than estimated, and their absence is treated as a reproducibility limitation. Online latency, throughput, and model-artifact sizes were measured separately and are reported in Section 3.9; future releases should preserve the complete environment manifest, hyperparameters, seeds, training logs, and offline feature-extraction cost. The retained artifacts identify the trained prediction outputs, split assignments, model architecture, and most training hyperparameters, but they do not preserve the exact external embedding-model revision, tokenizer revision, hardware identifiers, software-version manifest, training time, GPU-hours, or peak memory. For this reason, we describe the release as a versioned and auditable Python workflow.

2.7. Ethics, Safety, and Responsible Release

The study was designed exclusively for defensive prompt-injection detection and safety evaluation. It did not generate new attacks, optimize jailbreak success, test public targets, or publish operational attack strings. Safety controls were integrated into data handling, reporting, artifact release, and intended-use statements rather than added only after analysis.

2.7.1. No Harmful Prompts Printed Verbatim

Raw harmful prompts, malicious retrieved content, unsafe tool-output strings, and complete adversarial conversation histories were not reproduced verbatim. Qualitative analysis used sanitized identifiers, labels, hashes, transformation metadata, attack-surface categories, and aggregate error descriptions. This approach preserved analytical traceability while reducing the risk that the manuscript could function as an attack catalogue.

2.7.2. Controlled Release

Controlled-release principles governed code, data, model, and reporting artifacts. Public-facing materials may include dataset source references, schemas, label mappings, split definitions, aggregate metrics, configuration files, and reproduction commands but should exclude raw harmful prompts, credentials, private user data, unsafe tool traces, and exploit-oriented instructions. Access to external benchmarks remains subject to their original licenses and safety requirements.

2.7.3. Defensive-Only Use

PromptSentinel-X was evaluated as a defensive guardrail for prompt-risk screening, indirect-injection detection, benign hard-negative testing, context-segmentation research, calibrated routing, sandbox selection, and human-review triage. Generating new attacks, improving jailbreak success, or bypassing deployed safeguards was outside the intended scope and should not be supported by released artifacts.

2.7.4. Reproducibility and Reporting

Reproducibility was supported through stable sample identifiers, explicit label mappings, saved split assignments, configuration snapshots, cached expensive features, common metric definitions, and versioned output artifacts. Optional models or external files that were unavailable were reported as such rather than replaced with synthetic results. The manuscript documents dataset definitions, split logic, model components, loss terms, thresholds, validation procedures, deployment variants, limitations, and sanitized error categories so that the analytical claims can be independently audited without distributing reusable adversarial content.

3. Results

3.1. Dataset Composition and Leakage-Aware Validation

The in-domain benchmark comprised 1581 English-language records from the Malignant dataset [12]: 1312 benign conversation records (83.0%), 199 malicious jailbreak records (12.6%), and 70 suspicious roleplay or act_as records (4.4%). The original class imbalance was retained because it more closely approximates routine agent traffic, in which benign inputs predominate while malicious instructions remain comparatively infrequent but operationally consequential. Because this distribution can make overall accuracy and weighted-F1 appear favorable even when minority classes are unstable, the analysis emphasized macro-F1, class-wise recall, attack recall, benign false-positive rate, and calibration alongside aggregate accuracy; refer to Figure 2; refer to

Table 5. Ablation and statistical-validation protocol.

Ablation Family	Components Evaluated	Interpretation Criterion
Loss functions	Cross-entropy, supervised contrastive loss, focal loss, center/proxy terms, calibration-related terms.	Determines whether representation learning and imbalance handling improve safety metrics.
Model components	Projection head, coarse-risk head, context segmentation, hidden-state detector, calibration, ensemble/cascade.	Identifies which architectural elements support class separation, deployment actions, and uncertainty handling.
Dataset components	External tests, hard negatives, group-aware splits, leave-family-out splits, context scenarios.	Determines whether reported performance depends on data design or leakage-prone evaluation.
Validation layer	Bootstrap intervals, group-slice summaries, paired tests, calibration curves, sanitized error categories.	Reduces overclaiming from a single score or one train-test split.

,

Table 6. Error-analysis, deployment, and safety protocol.

Analytical Layer	Metrics or Controls	Reason for Inclusion
False negatives	Missed-attack rate, attack recall, category-specific false-negative rate, action severity.	Identifies high-risk failures that could alter agent behavior or tool use.
False positives	Benign false-positive rate and hard-negative false-block rate.	Measures over-defense and usability loss in legitimate technical workflows.
Ambiguity	Calibrated uncertainty, ensemble disagreement, overlap scores, out-of-distribution indicators.	Supports warning, sandboxing, and human review instead of brittle binary decisions.
Deployment feasibility	Latency, throughput, model size, escalation rate.	Determines where the guardrail can be placed in an agentic workflow.
Safety controls	Sanitized reporting, controlled release, defensive-only scope.	Preserves reproducibility without disseminating harmful prompt content.

,

Table 7. Dataset composition and evaluation-design summary.

Evidence Layer	Category	Records	Share/Families	Analytical Role
In-domain corpus	Benign conversation	1312	83.0%	Primary training and internal evaluation
In-domain corpus	Malicious jailbreak	199	12.6%	Primary high-risk class
In-domain corpus	Suspicious roleplay/act_as	70	4.4%	Minority ambiguity class
External tests	Open-Prompt-Injection	16,673	-	Direct prompt-injection transfer
External tests	CAPTURE-style	5537	-	Indirect/RAG-style transfer
External tests	HarmBench	4624	-	Jailbreak and harmful-behavior transfer
External tests	JailbreakBench	2503	-	Jailbreak-family transfer
External tests	InjecGuard/NotInject-style	678	-	Benign hard-negative evaluation
Primary split	Group-aware	1116 train/465 test	984/422	No family or paraphrase overlap
Sensitivity split	Random	1106 train/475 test	1006/452	Comparator only; leakage warnings retained
Stress split	Leave-family-out	1504 train/77 test	1378/28	Complete family holdout
Stress split	Temporal/style	1406 train/175 test	1406/90	Transformed-style holdout

Note. Percentages apply to the 1581-record in-domain corpus. Family counts are shown for split rows.

,

Table 8. Same-split baseline and PromptSentinel-X classification performance.

Model	Accuracy	Macro Precision	Macro Recall	Macro-F1	Weighted-F1
MiniLM embedding logistic regression	0.9828	0.8898	0.9840	0.9308	0.9832
Siamese triplet baseline	0.9763	0.7990	0.8814	0.8352	0.9776
DeBERTa-v3-base	0.9871	0.6368	0.6667	0.6510	0.9809
ModernBERT-base	0.9871	0.9353	0.9171	0.9258	0.9869
ModernBERT-large	0.9871	0.8675	0.9171	0.8870	0.9875
XLM-RoBERTa-large	0.9828	0.6342	0.6557	0.6447	0.9766
BGE-M3 embeddings + classifier	1.0000	1.0000	1.0000	1.0000	1.0000
Prompt Guard 2	0.9656	0.6538	0.6120	0.6303	0.9585
Granite Guardian	0.4925	0.3940	0.4602	0.3091	0.5550
Llama Guard	0.8581	0.6193	0.3388	0.3186	0.7946
PromptSentinel-X	0.9849	0.8420	0.9803	0.8887	0.9864

and

Table 9. Fine-label classification report for PromptSentinel-X on the 465-record group-aware test set.

Class/Summary	Support	Precision	Recall	F1-Score
Malicious jailbreak	61	0.9831	0.9508	0.9667
Suspicious roleplay/act_as	6	0.5455	1.0000	0.7059
Benign conversation	398	0.9975	0.9899	0.9937
Macro average	465	0.8420	0.9803	0.8887
Weighted average	465	0.9897	0.9849	0.9864

Note. Overall accuracy was 0.9849. At the coarse-risk level, attack recall was 0.9851, the benign false-positive rate was 0.0050, AUROC was 0.9971, AUPRC was 0.9836, and expected calibration error was 0.0153.

.

External benchmark adapters contributed 30,015 evaluation records: 16,673 from Open-Prompt-Injection, 5537 from CAPTURE-style indirect prompt-injection data, 4624 from HarmBench, 2503 from JailbreakBench, and 678 benign hard negatives from an InjecGuard/NotInject-style source. The 678 InjecGuard/NotInject-style records were one source of benign hard negatives. The later over-defense analysis used a broader benign hard-negative evaluation set that also included eligible benign records from Open-Prompt-Injection and JailbreakBench, plus a small set of generated benign hard-negative templates. These records were maintained as external-test evidence rather than pooled into the primary Malignant training set. The resulting design therefore separated in-domain discrimination from cross-source robustness and over-defense evaluation. The primary scientific partition was group-aware. It assigned 1116 records from 984 prompt families to training and 465 records from 422 nonoverlapping families to testing. The random comparator contained 1106 training and 475 test records, whereas the leave-family-out and temporal/style partitions contained 77 and 175 test records, respectively. This partition structure made prompt-family novelty, rather than repeated variants of the same family, the central unit of generalization. The partitioning rationale follows prompt-injection benchmark formalization and grouped-validation guidance [35,46,47,48].

Random splitting produced a more optimistic estimate than group-aware splitting. The random split achieved 0.9853 accuracy and 0.9491 macro-F1, whereas the group-aware split achieved 0.9763 accuracy and 0.8352 macro-F1. The leakage audit explains this decrease: random splitting generated 144 warnings, including family and paraphrase overlap, while group-aware splitting removed those two leakage classes while retaining 38 near-duplicate warnings. The reduction was therefore interpreted as a more conservative estimate of generalization to unseen prompt families rather than as evidence of model deterioration. This distinction is especially relevant to LLM-powered web agents, which encounter novel webpages, retrieved documents, tool outputs, languages, and attack formulations rather than repeated variants of benchmark prompts.

3.2. Same-Split Baselines and PromptSentinel-X Internal Classification

The same-split baseline comparison revealed substantial variation across model families (Table 8). BGE-M3 embeddings followed by a classifier achieved perfect internal accuracy and macro-F1. Logistic regression on MiniLM embeddings and ModernBERT-base also performed strongly, with macro-F1 values of 0.9308 and 0.9258, respectively, whereas the Siamese triplet baseline reached 0.8352. The evaluated guardrail and judge-style comparators varied considerably: Prompt Guard 2 reached 0.6303 macro-F1, Granite Guardian reached 0.3091, and Llama Guard reached 0.3186. The divergence between near-ceiling accuracy and substantially lower macro-F1 for several models shows that aggregate accuracy did not guarantee balanced recovery of benign, suspicious, and malicious classes. PromptSentinel-X was evaluated on the 465-record group-aware test set. The model achieved 0.9849 accuracy, 0.8420 macro precision, 0.9803 macro recall, 0.8887 macro-F1, and 0.9864 weighted-F1. At the coarse-risk level, attack recall was 0.9851 and the benign false-positive rate was 0.0050. These values indicate strong internal discrimination while also showing why the low-support suspicious class must be interpreted through class-specific metrics rather than through overall accuracy alone.

The fine-label confusion matrix contained 458 correct predictions among 465 test records. PromptSentinel-X correctly classified 58 of 61 jailbreak records, all six suspicious roleplay records, and 394 of 398 benign conversation records. Misclassifications were concentrated at the suspicious boundary: two jailbreak records were assigned to act_as, three benign conversations were assigned to act_as, one jailbreak was assigned to conversation, and one conversation was assigned to jailbreak. This pattern explains the combination of high macro recall and lower macro precision, because the suspicious class had only six test examples and received several false-positive assignments.

At the coarse-risk level, 396 of 398 benign records were allowed as benign, all six suspicious records remained within the suspicious category, and 56 of 61 malicious records were classified as malicious. Four additional malicious records were routed to suspicious rather than benign, leaving only one malicious record assigned to the benign class. Thus, the operational attack recall of 0.9851 was achieved with a benign false-positive rate of 0.0050. The confusion structure (refer to Figure 3) supports graduated handling of uncertain content, because most residual malicious errors were conservative escalations to suspicious rather than unrestricted benign decisions. The suspicious class contained only six test examples. Its observed recall of 1.0000 has a 95% Wilson interval of 0.6097–1.0000, and its precision of 0.5455 has an interval of 0.2801–0.7873. These intervals reinforce that the class estimate is unstable and should be interpreted as preliminary rather than as evidence of reliable suspicious-case discrimination.

The learned representation formed three visually distinct regions (Figure 4a). Quantitative cluster diagnostics were consistent with the visualization: the silhouette score was 0.7881, the Davies–Bouldin index was 0.3925, and the mean distance between class centroids was 13.8839. The closest centroid pair remained separated by 10.3718 units. These results indicate that the projection objective produced structured risk geometry rather than an undifferentiated embedding space. The training curve in (Figure 4b) showed a rapid reduction in loss during the first two epochs, followed by smaller improvements, while validation macro-F1 changed only modestly after the initial phase. The absence of a late validation collapse supported selection of the stabilized checkpoint, although family-aware and external results remained the primary evidence against memorization.

3.3. Target-LLM Hidden-State Detection

The target-LLM experiment (refer to Figure 5) evaluated whether latent representations from Qwen2.5-0.5B-Instruct contained separable prompt-risk information. Hidden states were cached for all 1581 in-domain records at 896 dimensions under two layers (−1 and −4), two pooling strategies (last-token and mean pooling), and three lightweight classifiers. Performance was uniformly high, but mean pooling was the most consistent configuration (

Table 10. Target-LLM hidden-state ablation results.

Layer	Pooling	Classifier	Accuracy	Macro-F1
−1	Last token	Logistic regression	0.9978	0.9969
−1	Last token	Linear SVM	0.9892	0.9848
−1	Last token	Shallow MLP	0.9914	0.9623
−1	Mean	Logistic regression	1.0000	1.0000
−1	Mean	Linear SVM	1.0000	1.0000
−1	Mean	Shallow MLP	0.9957	0.9468
−4	Last token	Logistic regression	0.9935	0.9654
−4	Last token	Linear SVM	0.9914	0.9623
−4	Last token	Shallow MLP	0.9914	0.9647
−4	Mean	Logistic regression	1.0000	1.0000
−4	Mean	Linear SVM	0.9978	0.9969
−4	Mean	Shallow MLP	1.0000	1.0000

Note. Hidden states were cached for all 1581 in-domain records, but lightweight classifiers used the same group-aware split as the primary evaluation: 948 training records, 168 validation records, and the unchanged 465-record group-aware test set. No separate random split or cross-validation was used. The results are internal to the Malignant benchmark and should be interpreted as latent separability under this split, not as proof of cross-model or cross-domain generalization.

).

These results used the same group-aware split as the primary in-domain evaluation: cached hidden-state features were available for all 1581 records, but classifiers were trained on 948 records from the group-aware training side and evaluated on the unchanged 465-record group-aware test set, with 168 records retained as validation. Thus, the perfect scores observed for several mean-pooling configurations should be interpreted as internal family-split separability within the Malignant benchmark, not as evidence of cross-dataset or cross-validation performance.

Mean-pooled representations produced perfect accuracy and macro-F1 for logistic regression at both evaluated layers, for linear SVM at layer −1, and for the shallow MLP at layer −4. Last-token pooling remained strong but was less stable, with macro-F1 ranging from 0.9623 to 0.9969. The best hidden-state detector matched the BGE-M3 internal result and exceeded the MiniLM embedding logistic-regression baseline (0.9308 macro-F1) and the Siamese triplet baseline (0.8352). The result demonstrates that the target model encoded risk-relevant information in its intermediate representation; however, the perfect internal score was not used as a universal deployment claim because the detector depends on model access, representation stability, and recalibration after model changes.

3.4. Context-Aware Detection Across Agentic Surfaces

The context-aware dataset contained 2850 structured records and was partitioned using a record-level stratified split with random seed 42. First, 30% of records were assigned to the held-out test set using candidate_risk_label stratification. The remaining non-test records were then split again, with 15% of that non-test pool assigned to validation, also stratified by candidate_risk_label. This produced 1695 training records, 300 validation records, and 855 test records, corresponding to 59.47%, 10.53%, and 30.00% of the structured context dataset, respectively. Existing source split labels were not preserved for this context-specific experiment.

This context-aware split was record-level rather than family-aware. Exact source_sample_id values did not overlap across train, validation, and test partitions, but original_family_id was not used as a grouping constraint. As a result, related context variants may have crossed partitions; in the retained split, 12 family identifiers crossed train and validation, 17 crossed train and test, and four crossed validation and test. Context-aware results are therefore interpreted as structured scenario evidence rather than as fully family-leakage-controlled context generalization. The 855 records used in

Table 11. Context-aware performance by agentic scenario. (A) Scenario accuracy with 95% Wilson score intervals derived from the integer correct counts reported for each scenario.

Scenario	Support	Accuracy	Macro-F1	Weighted-F1
RAG injection	81	0.9259	0.3844	0.9082
Tool-output injection	1	1.0000	1.0000	1.0000
Textual browser-content injection	172	0.8721	0.7594	0.8571
Memory-related injection	35	0.7429	0.3200	0.6857
Multi-turn prompt staging	210	0.8429	0.6306	0.8146
Static user prompt	356	0.8399	0.8301	0.8250
(A)
Scenario	Support	Accuracy	95% Wilson CI
RAG injection	81	0.9259	0.8477–0.9656
Tool-output injection	1	1.0000	0.2065–1.0000
Textual browser-content injection	172	0.8721	0.8139–0.9140
Memory-related injection	35	0.7429	0.5793–0.8584
Multi-turn prompt staging	210	0.8429	0.7875–0.8859
Static user prompt	356	0.8399	0.7982–0.8743

Note. Tool-output injection (n = 1) is descriptive only; its interval is extremely wide. Memory-related injection (n = 35) is exploratory. Scenario accuracy intervals quantify uncertainty in accuracy, not macro-F1. Intervals quantify scenario accuracy, not macro-F1. The tool-output row is descriptive, and the memory row is exploratory.

and

Table 12. Context relationship classification performance.

Relationship Label	Support	Precision	Recall	F1-Score
Benign context	276	0.8818	0.9457	0.9126
Override attempt	224	0.8918	0.9196	0.9055
Leakage attempt	16	0.5000	0.2500	0.3333
Hijack attempt	151	0.8106	0.7086	0.7562
Tool-abuse attempt	19	0.9048	1.0000	0.9500
Data-exfiltration attempt	75	0.9265	0.8400	0.8811
Obfuscated context	19	1.0000	0.8421	0.9143
Roleplay or borderline	75	0.8193	0.9067	0.8608

Note. Relationship scores are class-specific. Leakage attempts (n = 16) are exploratory and do not support a statistically robust deployment claim; hijack and other low-support categories should also be interpreted together with support.

are the held-out test split created from this dataset, not the subset with retrieved context. The equality between the full-dataset retrieved-context coverage count, 855 records, and the test-set size, 855 records, is coincidental. Schema coverage was computed over all 2850 structured records, whereas scenario and relationship metrics were computed only on the 855 held-out test records.

Figure 6 shows that context-aware detection was materially more difficult than static prompt classification. The principal diagonal remained dominant for high-support categories, but errors increased among semantically adjacent and low-support risks. The relationship analysis recovered benign context, override attempts, tool abuse, data exfiltration, obfuscation, and roleplay with F1 scores above 0.86. In contrast, leakage attempts reached only 0.3333 F1, driven by recall of 0.25, and hijack attempts reached 0.7562 F1. These results locate the primary context-level weakness in distinguishing information-leakage and hijacking behavior from related instruction patterns.

Scenario-level results varied markedly. Static prompts produced the strongest supported macro-F1 (0.8301), followed by textual browser-content injection (0.7594), multi-turn staging (0.6306), RAG injection (0.3844), and memory-related injection (0.3200). The tool-output result was based on one qualifying record and is descriptive only; its accuracy interval spans 0.2065–1.0000. Memory accuracy also remained uncertain (95% interval 0.5793–0.8584). Trusted–untrusted similarity shifted upward for suspicious and malicious records but overlapped across classes, so it is used as a routing signal rather than a deterministic rule. These findings show that textual context-aware detection is more difficult than static classification and that RAG, memory, staged multi-turn, tool-output, raw-DOM, ARIA-structure, and visual-browser settings require broader, balanced evaluation before deployment claims can be generalized; refer to Figure 7.

3.5. Hard Negatives and Over-Defense

A deployable guardrail must avoid both missed attacks and excessive restriction of legitimate activity. Over-defense was evaluated on 8276 records comprising 4276 benign hard negatives and 4000 attack references. The 4276 benign hard negatives in the over-defense evaluation were broader than the 678 InjecGuard/NotInject-style records reported in the external-adapter inventory. The over-defense set combined all benign records that met the hard-negative evaluation criteria: 678 InjecGuard/NotInject-style records, 3470 benign Open-Prompt-Injection records, 100 benign JailbreakBench records, and 28 generated benign hard-negative templates. No in-domain Malignant benign records were included in the over-defense evaluation subset. Thus, the additional 3598 records beyond the InjecGuard/NotInject-style source consisted of 3570 other external benign records and 28 generated benign hard-negative templates (

Table 13. Over-defense model summary.

Model	Benign FPR	Attack Recall	Missed Attack	Macro-F1	AUROC	AUPRC	ECE
Intent-sensitive TF-IDF	0.0685	0.4873	0.5128	0.6986	0.7631	0.8122	0.0556
Trigger-word baseline	0.0030	0.1180	0.8820	0.4585	0.6368	0.6285	0.3145

Note. FPR at 95% TPR was 0.9677 for the intent-sensitive model and 1.0000 for the trigger-word baseline, confirming that neither simple detector provided an acceptable high-recall operating point.

and Figure 8).

Two deliberately simple detectors were used to expose the limitations of keyword-oriented filtering: an intent-sensitive TF-IDF logistic-regression model and a trigger-word baseline. The intent-sensitive model improved attack ranking but increased benign blocking, whereas the trigger-word model preserved a low false-positive rate by missing most attacks.

The trigger-word baseline achieved a benign false-positive rate of 0.0030 but detected only 11.8% of attacks. The intent-sensitive model increased attack recall to 48.7% and macro-F1 to 0.6986, but its benign false-positive rate rose to 6.85%. False blocking by the intent-sensitive model was highest for benign security discussion (16.30%), benign trigger-word cases (10.24%), encoded or quoted text (9.72%), multilingual examples (6.86%), debugging text (5.41%), and benign roleplay (4.22%). The prompt-engineering subset had no false blocks but contained only eight examples. These category-specific results show that a seemingly moderate overall false-positive rate can conceal substantial usability loss in legitimate technical and security workflows.

Figure 9 shows that the intent-sensitive detector outperformed the trigger-word baseline on both threshold-independent measures, with AUROC of 0.7631 versus 0.6368 and AUPRC of 0.8122 versus 0.6285. Its probability estimates were also substantially better calibrated: expected calibration error was 0.0556, compared with 0.3145 for trigger-word filtering. Nevertheless, the ROC and precision-recall curves show that high recall required accepting a prohibitive false-positive rate. These findings establish hard-negative evaluation as a necessary complement to attack-only testing and demonstrate why trigger vocabulary alone was not used as the final guardrail decision rule.

3.6. Robustness and Cross-Dataset Generalization

Table 14. Robustness results by perturbation and transfer category. (A) Wilson intervals for robustness attack recall and benign false-positive rate, reported with the corresponding attack and benign support.

Category	Samples	Attack	Benign	Macro-F1	Attack Recall	Benign FPR	ECE
Paraphrase	31,596	26,036	5560	0.8642	1.0000	0.0004	0.1185
Typo/noise	500	199	301	0.8211	1.0000	0.0033	0.1038
Multilingual	524	352	172	0.7143	1.0000	0.0000	0.1200
Obfuscation	675	503	172	0.7500	1.0000	0.0000	0.1200
Long context	1059	1059	0	0.8879	0.9991	-	-
RAG injection	5623	5614	9	0.8125	1.0000	0.0000	0.1196
Roleplay confusion	10,469	6221	4248	0.7827	1.0000	0.0000	0.1199
(A)
Category	Attack recall (95% CI)				Benign FPR (95% CI)
Paraphrase/external pool	1.0000 (0.9999–1.0000); 26,036 attacks				0.0004 (0.0001–0.0013); 5560 benign
Typo/noise	1.0000 (0.9811–1.0000); 199 attacks				0.0033 (0.0006–0.0186); 301 benign
Multilingual	1.0000 (0.9892–1.0000); 352 attacks				0.0000 (0.0000–0.0218); 172 benign
Obfuscation	1.0000 (0.9924–1.0000); 503 attacks				0.0000 (0.0000–0.0218); 172 benign
Long context	0.9991 (0.9947–0.9998); 1059 attacks				Not estimable; no benign records
RAG injection	1.0000 (0.9993–1.0000); 5614 attacks				0.0000 (0.0000–0.2991); 9 benign
Roleplay confusion	1.0000 (0.9994–1.0000); 6221 attacks				0.0000 (0.0000–0.0009); 4248 benign

Note. Values rounded to four decimals. Long-context evaluation contained no benign examples, so benign FPR and binary calibration were not estimable. Paraphrase and cross-dataset summaries coincide because both category filters selected the same de-duplicated combined robustness pool: 1581 in-domain Malignant records plus 30,015 external benchmark records. This 31,596-record pool should therefore be interpreted as an in-domain plus external robustness inventory, not as an external-only pool.

and Figure 10 illustrate the results of the robustness. It was evaluated by perturbation type and attack surface rather than through a single pooled statistic. The evaluated slices included paraphrase, typographical noise, multilingual inputs, obfuscation, long context, RAG injection, roleplay confusion, and cross-dataset transfer. Multi-turn staging, tool-output injection, and adaptive detector-aware evaluation had no qualifying records in the robustness-stage inventory and were therefore reported as unavailable rather than assigned imputed scores.

The robustness inventory contained 31,596 unique source records after de-duplication by sample identifier. This count combines the complete 1581-record in-domain Malignant corpus with 30,015 external benchmark records. Although the loader also accepted the context-aware dataset, those records used source identifiers already present in the in-domain or external normalized files and therefore did not add new unique source records after de-duplication. Accordingly, the 31,596-record robustness pool is a combined in-domain plus external pool, not an external-only pool. The interval analysis changes the interpretation of zero observed errors. For RAG injection, zero false positives among only nine benign records yields an upper 95% bound of approximately 0.2991; therefore, the observed zero benign FPR does not establish a low population false-positive rate. In contrast, zero false positives among 4248 roleplay-confusion benign records yields a much tighter upper bound of approximately 0.0009. Robustness should therefore be interpreted jointly with class composition and interval width.

Attack recall remained at or near 1.0 across the evaluated slices, but macro-F1 varied from 0.7143 for multilingual inputs to 0.8879 for long-context attacks. Typographical noise reached 0.8211 macro-F1, RAG injection reached 0.8125, roleplay confusion reached 0.7827, and obfuscation reached 0.7500. The discrepancy between near-perfect attack recall and lower macro-F1 indicates that robustness was recall-dominant rather than uniformly balanced across classes. Results from highly skewed slices, particularly RAG injection with only nine benign records and long-context evaluation with no benign records, were therefore interpreted together with sample composition rather than as evidence of complete immunity to those attack forms.

3.7. Ensemble, Cascade, and Component-Sensitivity Analyses

Model combination produced different safety and operational profiles (

Table 15. Ensemble and cascade performance.

Mode	Support	Accuracy	Macro-F1	Attack Recall	Benign FPR	ECE	Escalation
Probability average	465	0.9914	0.9216	0.9701	0.0000	0.1171	-
Majority vote	465	0.9978	0.9968	0.9851	0.0000	0.0301	-
Stacked classifier	163	1.0000	1.0000	1.0000	0.0000	0.0183	-
Cascade	465	0.9806	0.8935	0.9254	0.0050	0.0163	0.0172

Note. Probability averaging, majority voting, and cascade were evaluated on the full 465-record group-aware test set using seven aligned prediction views: MiniLM/Siamese, ModernBERT-base, BGE-M3, target-LLM hidden-state detector, Prompt Guard 2, Llama Guard, and PromptSentinel-X. The stacked classifier used the same seven risk-score columns but was evaluated only on a 163-record stratified internal holdout created from the 465-row ensemble feature table using a 65/35 split with random seed 42. It trained on the remaining 302 records. The stacked classifier did not use out-of-fold predictions and should be interpreted as an exploratory internal stacking analysis rather than a complete-test-set comparison. Cascade thresholds were fixed in the retained configuration rather than selected through a retained validation sweep. Escalation used the ambiguous score interval (0.25, 0.75). The final action thresholds were as follows: allow for scores < 0.35, warn for scores from 0.35 to <0.65, and block for scores ≥ 0.65.

and Figure 11). Majority voting achieved the strongest result on the common 465-record test set, with 0.9978 accuracy and 0.9968 macro-F1. Probability averaging achieved 0.9216 macro-F1 and a zero benign false-positive rate, whereas the cascade achieved 0.8935 macro-F1 with a 1.72% escalation rate. The stacked classifier produced perfect scores on a smaller 163-record subset and was not compared as if it had been evaluated on the complete test set.

Figure 11 shows that majority voting provided the strongest complete-test metric, while the cascade traded some macro-F1 for low escalation and strong calibration. The available component comparisons (

Table 16. Component-sensitivity and artifact-based proxy comparisons (non-causal).

Configuration	Macro-F1	Attack Recall	Benign FPR	ECE	Interpretation
Full deployment cascade	0.8935	0.9254	0.0050	0.0163	Observed deployment configuration
No contrastive objective *	0.9258	0.9403	0.0000	0.2914	ModernBERT proxy; not controlled retraining
No hard negatives *	0.6986	0.4873	0.0685	0.0556	Over-defense proxy
Malignant-only *	0.8352	-	-	-	Group-split in-domain proxy
No context segmentation *	0.8887	0.9851	0.0050	0.0153	Static PromptSentinel-X proxy
No hidden-state fusion *	0.8887	0.9851	0.0050	0.0153	Current focal model did not fuse hidden states
No calibration *	0.8887	-	-	0.0216	Pre-calibration score diagnostic
Random split only	0.9491	-	-	-	Leakage-prone sensitivity comparator

Note. * identify artifact-based or proxy comparisons rather than retraining of an otherwise identical architecture. These rows quantify the sensitivity of the available evidence but do not isolate the causal effect of contrastive learning, hard-negative training, context segmentation, hidden-state fusion, or calibration. Direct component attribution requires controlled retraining under shared seeds and group-aware folds.

) do not support causal attribution. In particular, the no-contrastive row uses a different ModernBERT artifact, the no-hard-negative row uses an over-defense proxy, and the focal model did not fuse the hidden-state detector. These comparisons identify where the evidence changes, but they do not establish which single component caused the change. A controlled one-component-at-a-time factorial study is reserved for future work.

3.8. Statistical Validation and Sanitized Error Analysis

The statistical analyses provide exploratory uncertainty summaries over the saved prediction artifacts rather than fully family-aware confirmatory confidence intervals. Five group-based prediction slices produced mean macro-F1 of 0.9158 with standard deviation 0.0532. Because only five slice-level values were available, the corresponding interval should be interpreted descriptively rather than as a robust confidence interval. Record-level bootstrap intervals for model accuracy and paired macro-F1 differences summarize variability under an individual-record resampling assumption and therefore do not fully account for within-family dependence. Pairwise tests were used to identify large descriptive differences and unresolved comparisons, not to establish familywise-error-controlled superiority (

Table 17. Statistical validation and paired model comparisons.

Validation Item	Estimate	Interval/ p-Value	Interpretation
Five group slices: PromptSentinel-X macro-F1	0.9158 (SD 0.0532)	0.8723 to 0.9595	Variation across family-based slices
Bootstrap accuracy: PromptSentinel-X	0.9849	0.9720 to 0.9957	Narrow record-level exploratory interval; within-family dependence was not preserved
PromptSentinel-X vs. BGE-M3: McNemar	b01 = 7; b10 = 0	p = 0.0156	BGE-M3 made fewer paired errors
PromptSentinel-X—BGE-M3: macro-F1	−0.1168	−0.2243 to −0.0333	Difference favored BGE-M3
PromptSentinel-X vs. transformer: McNemar	b01 = 6; b10 = 5	p = 1.0000	No paired-error difference
PromptSentinel-X—transformer: macro-F1	−0.0350	−0.1351 to 0.0640	Interval included zero
PromptSentinel-X vs. Llama Guard: McNemar	b01 = 4; b10 = 63	p = 1.11 × 10−14	Difference favored PromptSentinel-X
PromptSentinel-X—Llama Guard: macro-F1	0.5636	0.4513 to 0.6557	Large positive difference

Note. b01 and b10 are discordant-pair counts from the saved prediction comparison. Confidence intervals are bootstrap or group-slice intervals, as indicated.

).

Figure 12 shows that the observed calibration curve tracked the diagonal broadly but showed deviations in the low- and high-confidence regions. PromptSentinel-X expected calibration error was 0.0153 in the final comparison, and the deployment cascade achieved 0.0163. These small aggregate values support threshold-based routing, but the bin-level deviations show that calibration was not exact throughout the score range. The sanitized error log recorded five false-negative tags and two false-positive tags. It also recorded five malicious-roleplay-missed tags, one long-context-failure tag, and two other-mismatch tags. Because error tags could overlap, these counts were interpreted as categories rather than as 15 unique cases. The concentration of false-negative tags in malicious roleplay was consistent with the lower precision and limited support of the suspicious boundary class. All qualitative error reporting used sanitized identifiers, aggregate categories, labels, hashes, source metadata, and transformation types; raw harmful prompts, unsafe tool outputs, and complete adversarial conversation histories were not reproduced.

3.9. Deployment Feasibility and Final Model Comparison

Deployment metrics translated benchmark performance into operational placement and exposed a clear latency–throughput trade-off (

Table 18. Deployment-oriented performance of the evaluated inference variants.

Variant	Latency (ms)	Pred./s	Size MB	Macro-F1	Attack Recall	Benign FPR	Missed Attack	Escalation
Fast screening model	2.0	500.00	0.0187	0.9308	0.9851	0.0151	0.0149	0.0000
PromptSentinel-X	12.0	83.33	1.9630	0.8887	0.9851	0.0050	0.0149	0.0000
Probability ensemble	45.0	22.22	-	0.9216	0.9701	0.0000	0.0299	0.0000
Cascade	18.0	55.56	-	0.8935	0.9254	0.0050	0.0746	0.0172

and Figure 13). The fast-embedding classifier required approximately 2 ms per prediction and processed 500 predictions per second. PromptSentinel-X required 12 ms and processed 83.33 predictions per second, while the probability ensemble required 45 ms and processed 22.22 predictions per second. The cascade occupied an intermediate position at 18 ms and 55.56 predictions per second.

PromptSentinel-X reduced the benign false-positive rate from 0.0151 for the fast-screening model to 0.0050 while preserving the same attack recall of 0.9851, at the cost of increasing latency from 2 to 12 ms. Probability averaging eliminated benign false positives on the reported test set but doubled the missed-attack rate relative to PromptSentinel-X and reduced throughput to 22.22 predictions per second. The cascade provided moderate latency and low escalation, but its missed-attack rate of 0.0746 was the highest among the deployment variants.

These results indicate that the appropriate variant depends on whether the control point prioritizes low latency, minimum benign disruption, maximum recall, or selective escalation. These operating points suggest a placement hierarchy: the fast model is most suitable for inbound screening and low-risk filtering; PromptSentinel-X is better suited to selected agentic decision points and untrusted-content inspection; the cascade is appropriate for uncertain cases and pre-tool-execution routing; and the probability ensemble is most defensible for offline audit or high-assurance checks where additional latency is acceptable.

Figure 14 confirms that no model dominated every criterion. BGE-M3 and the target-LLM hidden-state detector reached perfect internal macro-F1, but their deployment value remains conditional on distribution shift and, for hidden states, local model access. MiniLM embedding logistic regression and ModernBERT-base were the strongest practical baselines, whereas PromptSentinel-X combined 0.9851 attack recall, a 0.0050 benign false-positive rate, and a 1.963 MB model artifact. Majority voting led the complete-test macro-F1 comparison, while probability averaging eliminated benign false positives at lower throughput. Family-aware, hard-negative, and context-aware tests remained the principal constraints on interpreting near-ceiling internal scores.

4. Discussion

In this public benchmark and multi-layer stress-test study, PromptSentinel-X demonstrated strong prompt-risk detection under prompt-family-aware validation while also exposing the limits of treating prompt injection as a conventional static text-classification problem. The analysis was designed as a defensive safety-control evaluation for LLM-powered web agents rather than as evidence of universal attack prevention or a stand-alone authorization mechanism. The workflow retained the original imbalance of the 1581-record Malignant corpus, reserved 30,015 external records for transfer and stress testing, separated related prompt families across the primary train and test partitions, compared multiple encoder and guardrail families, evaluated target-LLM hidden states, preserved trusted–untrusted context boundaries, measured benign hard-negative behavior, assessed calibration and robustness, and quantified deployment trade-offs. This layered design strengthens the interpretation of the findings because it reduces the likelihood that high internal scores were produced solely by prompt-family overlap, attack-only evaluation, or one favorable model and test split.

The principal PromptSentinel-X result was a strong but nonuniform operating profile on the 465-record group-aware test set. The model achieved accuracy of 0.9849, macro-F1 of 0.8887, weighted-F1 of 0.9864, attack recall of 0.9851, and a benign false-positive rate of 0.0050. Fine-label performance was strongest for malicious jailbreak and benign conversation records, with F1-scores of 0.9667 and 0.9937, respectively. The suspicious roleplay or act_as class reached recall of 1.0000 but precision of 0.5455 and F1 of 0.7059. This boundary class contained only six test examples and also received false-positive assignments from neighboring benign and malicious categories. The model therefore separated clearly malicious and clearly benign inputs reliably, but the distinction among benign persona use, ambiguous roleplay, and adversarial instruction shifting remained less stable and should not be inferred from the overall accuracy alone.

The coarse-risk confusion structure is more informative for operational use than fine-label accuracy by itself. Of 61 malicious records, 56 were classified as malicious, four were routed to the suspicious category, and only one was assigned to benign. At the same time, 396 of 398 benign records remained benign, and all six suspicious records remained within the suspicious category. Most residual malicious errors were therefore conservative escalations rather than unrestricted benign decisions. For an LLM-powered agent, this distinction supports graduated responses such as warning, richer-model inspection, sandboxed execution, reduced tool privilege, or human review. However, the classifier cannot determine the appropriate action in isolation. The final policy must also account for the requested capability, sensitivity of accessible data, reversibility of the action, user authorization, and the consequences of both false blocking and missed attacks.

The same-split model comparison shows that the contribution of PromptSentinel-X should not be expressed as universal numerical superiority. BGE-M3 embeddings followed by a classifier and several mean-pooled target-LLM hidden-state configurations achieved perfect internal accuracy and macro-F1. MiniLM embedding logistic regression and ModernBERT-base also produced higher macro-F1 values than PromptSentinel-X, at 0.9308 and 0.9258, respectively. In contrast, specialized guardrail comparators varied widely, with substantially lower class-balanced performance for Prompt Guard 2, Granite Guardian, and Llama Guard in the reported setup. Pairwise tests reinforced this qualified interpretation: BGE-M3 made fewer paired errors than PromptSentinel-X, the comparison with the reported transformer classifier did not resolve a statistically meaningful difference, and PromptSentinel-X strongly outperformed Llama Guard. The value of PromptSentinel-X therefore lies in its combined operating profile—high attack recall, low benign disruption, calibrated probabilities, structured representation learning, compatibility with context-aware analysis, and moderate deployment cost—rather than in claiming that its focal classifier leads every internal leaderboard.

Prompt-family leakage was one of the most consequential validity findings. For the corresponding split-sensitivity baseline, random splitting produced accuracy of 0.9853 and macro-F1 of 0.9491, whereas group-aware splitting produced accuracy of 0.9763 and macro-F1 of 0.8352. The random partition generated 144 warnings, including 52 family-overlap warnings, 52 paraphrase-overlap warnings, and 40 near-duplicate warnings. Group-aware splitting eliminated the family and paraphrase overlaps and retained 38 near-duplicate warnings. The lower family-aware score should therefore be interpreted as a more conservative estimate of performance on previously unseen prompt families, not as deterioration caused by the split itself. This distinction is essential for Agentic Web deployment, where systems encounter new webpages, retrieved passages, tool descriptions, languages, and attack formulations rather than repeated benchmark variants. Because near-duplicate warnings remained, the design is best described as leakage-mitigated rather than fully leakage-free.

The projection-space diagnostics provide evidence that the learned representation organized the three risk classes into a structured geometry. The UMAP analysis was supported by a silhouette score of 0.7881, a Davies–Bouldin index of 0.3925, and a mean class-centroid distance of 13.8839. This organization may be useful for uncertainty-aware routing because samples near class boundaries can be escalated instead of being forced into a high-confidence binary action. The training trajectory also stabilized without a late collapse in validation macro-F1. Nevertheless, the current component-sensitivity evidence does not isolate the causal contribution of supervised contrastive learning. The reported no-contrastive condition used a ModernBERT proxy rather than controlled retraining of an otherwise identical PromptSentinel-X configuration, and that proxy achieved numerically higher macro-F1. The results therefore support representation separability, but they do not establish that the contrastive term alone caused the observed classification performance. A controlled factorial ablation remains necessary before attributing gains to individual loss components.

The target-LLM hidden-state experiment identifies a potentially valuable complementary detection channel. Mean-pooled representations from Qwen2.5-0.5B-Instruct achieved perfect internal accuracy and macro-F1 under several lightweight classifiers, while last-token pooling remained strong but more variable. This result suggests that prompt-risk information was already encoded in the target model’s latent representation and could be monitored with a comparatively small downstream classifier. Such monitoring may be useful at high-assurance control points, including pre-tool authorization, protected memory writes, or operations involving sensitive data. Its scope is nevertheless constrained by the requirement for local or exposed model states, dependence on the evaluated architecture and layer, and possible drift after fine-tuning, quantization, model updates, or provider-side revisions. The perfect internal scores should therefore be interpreted as evidence of latent separability within this benchmark, not as proof of cross-model or cross-domain transfer.

The near-perfect BGE-M3 and hidden-state scores show separability within the evaluated benchmark and split, but they should not be read as equivalent to independent cross-platform generalization. The external adapters differ in label definitions and class balance and were used as stress tests rather than as a preregistered replication. Independent evaluation on executable agent benchmarks and unseen providers remains necessary, particularly after model updates or representation drift.

The context-aware findings provide the clearest evidence that static prompt detection is insufficient for the Agentic Web. The structured evaluation contained 2850 records, but support differed substantially across fields and scenarios. Static user prompts reached macro-F1 of 0.8301 and textual browser-content injection reached 0.7594. Performance declined to 0.6306 for multi-turn staging, 0.3844 for RAG injection, and 0.3200 for memory-related injection. The nominally perfect tool-output result was based on a single qualifying scenario and cannot support a general claim about tool-mediated defense. Relationship-level classification was strong for benign context, override attempts, tool abuse, data exfiltration, obfuscation, and roleplay or borderline behavior, but leakage attempts reached only 0.3333 F1 and hijack attempts reached 0.7562 F1. The principal unresolved problem is therefore not explicit jailbreak recognition; it is the discrimination of indirect leakage, hijacking, and context-dependent instruction control from semantically similar legitimate content.

Support-aware uncertainty reinforces the evidence boundary. The one-record tool-output result is descriptive, the memory-related estimate is exploratory, and the leakage-attempt relationship contains only 16 examples. These rows identify failure modes worth studying but do not establish reliable tool, memory, or leakage defense. In addition, the browser-context records represent textual content available to the detector, not raw DOM structure, CSS visibility, ARIA hierarchy, screenshots, or visual overlays. The current evidence therefore supports textual prompt-risk screening, not comprehensive browser-agent perception security.

Trusted–untrusted similarity added useful but limited context. Suspicious and malicious records tended to show greater semantic overlap with privileged instructions, yet the distributions overlapped substantially. Similarity to system or developer language is therefore not equivalent to maliciousness. Benign documentation, policy text, software instructions, security guidance, and quoted material may also resemble privileged instructions. In deployment, provenance and authority must remain distinct from semantic similarity: browser content, retrieved text, tool output, and memory should be treated as data unless the runtime explicitly authorizes those sources to influence the agent’s instruction hierarchy. PromptSentinel-X can provide a risk and uncertainty estimate, but source isolation, instruction provenance, tool permissions, output validation, and memory-write controls must be enforced by the surrounding application.

The hard-negative analysis demonstrates why attack recall must be interpreted jointly with usability. A trigger-word baseline produced a benign false-positive rate of 0.0030 but detected only 11.8% of attacks. The intent-sensitive TF-IDF model improved attack recall to 48.7% and macro-F1 to 0.6986, but its benign false-positive rate increased to 6.85%. Category-specific false blocking was especially high for benign security discussion, trigger-word cases, quoted or encoded text, multilingual examples, debugging content, and benign roleplay. These categories are central to legitimate web-agent use in software development, security analysis, education, documentation review, incident response, and multilingual assistance. The findings show that a detector may appear conservative because it rarely blocks benign traffic while missing most attacks or appear protective because it raises recall while making legitimate workflows unusable. Hard-negative testing and category-specific false-block rates should therefore be treated as primary evidence rather than as optional diagnostics.

Calibration and threshold behavior further explain why simple filtering was inadequate. The intent-sensitive baseline outperformed trigger-word matching on AUROC, AUPRC, and expected calibration error, yet its false-positive rate at 95% true-positive rate remained 0.9677; the corresponding value for trigger-word filtering was 1.0000. Neither simple detector offered a credible high-recall operating point. PromptSentinel-X achieved an expected calibration error of 0.0153, and the deployment cascade achieved 0.0163, supporting risk-based routing more strongly than the simple baselines. However, the reliability curve still departed from ideal calibration in the lowest- and highest-confidence regions. These probabilities should therefore be interpreted as deployment-specific decision signals rather than invariant security probabilities. Thresholds require recalibration when the model version, input population, attack prevalence, tool privilege, or acceptable review burden changes.

The robustness results were encouraging for attack sensitivity but did not establish uniformly balanced robustness. Attack recall remained at or near 1.0 across paraphrase, typographical noise, multilingual inputs, obfuscation, long context, RAG injection, roleplay confusion, and cross-dataset transfer. Macro-F1, however, ranged from 0.7143 for multilingual inputs to 0.8879 for long-context attacks, with obfuscation at 0.7500, roleplay confusion at 0.7827, RAG injection at 0.8125, and typographical noise at 0.8211. This gap indicates recall-dominant behavior rather than consistent class balance. Some slices were also highly skewed: the RAG slice contained only nine benign records, and the long-context slice contained no benign records, preventing estimation of benign false-positive rate and binary calibration. In addition, the paraphrase and cross-dataset summaries were computed over the same grouped external-test pool and should not be interpreted as independent replications. The evidence supports broad attack sensitivity under several transformations but not immunity to adaptive, multilingual, obfuscated, or indirect attacks.

The ensemble and cascade analyses show that model combination can improve selected outcomes while introducing new trade-offs. Majority voting achieved the strongest complete-test result, with 0.9968 macro-F1, attack recall of 0.9851, and no benign false positives on the 465-record test set. Probability averaging also eliminated benign false positives but reduced attack recall to 0.9701, increased the missed-attack rate to 0.0299, and produced an expected calibration error of 0.1171. The cascade achieved a macro-F1 of 0.8935, attack recall of 0.9254, benign false-positive rate of 0.0050, and an escalation rate of 1.72%, but it also had the highest reported missed-attack rate, 0.0746. The stacked classifier achieved perfect scores on a smaller 163-record subset and is not directly comparable with complete-test results. Ensemble gains thus depend on support size, model-error correlation, calibration, threshold selection, and the policy applied after prediction; combination alone does not guarantee the most appropriate safety profile.

Deployment measurements clarify where the evaluated variants may fit within an agentic workflow. The fast-screening model required approximately 2 ms per prediction and processed 500 predictions per second, but its benign false-positive rate was 0.0151. PromptSentinel-X reduced that rate to 0.0050 while preserving an attack recall of 0.9851, with latency of 12 ms, throughput of 83.33 predictions per second, and a reported model artifact of 1.963 MB. The probability ensemble required 45 ms per prediction and processed 22.22 predictions per second, while the cascade occupied an intermediate position at 18 ms and 55.56 predictions per second but sacrificed attack recall. These results support a privilege-aware placement hierarchy: the fast model is most suitable for high-volume inbound screening and low-risk filtering; PromptSentinel-X is better suited to selected agentic decision points and untrusted-content inspection; the cascade is most defensible for uncertain cases and pre-tool-execution routing; and the probability ensemble is better reserved for offline audit or high-assurance checks where additional latency is acceptable. These placements remain hypotheses until validated under the target serving environment and workload.

The statistical validation should be interpreted as exploratory support rather than confirmatory proof of stable superiority. The five group-based prediction slices provided a descriptive check on variation across family-disjoint evaluation slices, while the bootstrap accuracy and paired macro-F1 intervals were computed by resampling individual records. Because those bootstrap procedures did not resample prompt families as clusters, they may underestimate uncertainty when related prompt-family members remain dependent. Accordingly, the statistical results are used to qualify the descriptive model ranking: BGE-M3 showed fewer paired errors than PromptSentinel-X, the PromptSentinel-X versus transformer comparison was not clearly resolved, and PromptSentinel-X substantially outperformed Llama Guard in the retained prediction artifacts. These findings should be read as exploratory evidence from the saved evaluation outputs rather than as definitive family-level inferential claims.

The statistical analyses provide exploratory consistency checks for the main internal findings while limiting claims of family-level uncertainty or universal model superiority. Five group-based prediction slices produced a mean macro-F1 of 0.9158, standard deviation of 0.0532, and an interval from 0.8723 to 0.9595. PromptSentinel-X accuracy was 0.9849 with a bootstrap interval from 0.9720 to 0.9957. Paired comparisons favored BGE-M3, did not resolve a difference between PromptSentinel-X and the reported transformer comparator, and strongly favored PromptSentinel-X over Llama Guard. Sanitized error analysis identified malicious roleplay as a recurring false-negative category, together with one long-context failure and a small number of other mismatches. Because the error tags could overlap, these counts represent failure categories rather than unique cases. The pattern is consistent with the class-level and context-level evidence: explicit jailbreak and ordinary conversation examples were comparatively tractable, whereas ambiguous roleplay and context-sensitive instructions remained more difficult.

Study Limitations and Future Work

Several limitations constrain generalization. First, the in-domain corpus contains 1581 English-language records, is strongly imbalanced, and includes only six suspicious examples in the principal test set. Second, family-aware splitting removed family and paraphrase overlap but retained near-duplicate warnings and depends on the quality of the family identifiers. Third, external adapters harmonize sources with different collection methods, label definitions, and class balances; pooled stress-test performance is not an independent cross-platform replication. Fourth, context support is uneven: tool-output injection has one qualifying result, memory-related injection has 35 records, and leakage attempts have 16 relationship examples. Fifth, the browser analysis uses textual or structured fields rather than raw DOM topology, CSS-hidden content, ARIA hierarchy, screenshots, or visual overlays.

Additional limitations concern causal and operational claims. Several component comparisons are artifact-based proxies rather than controlled one-component-at-a-time retraining, and the focal model does not directly fuse the hidden-state detector. Calibration and cascade thresholds were evaluated retrospectively. Latency, throughput, and size are environment-dependent, and complete training hardware, software-version, wall-clock-time, GPU-hour, and peak-memory logs were not retained. Most importantly, the study evaluates classification and routing, not whether a functioning web agent avoids unauthorized actions. Detection can inform a policy, but provenance enforcement, least-privilege tool access, plan and output validation, protected memory writes, confirmation, and sandboxing must be implemented by the surrounding runtime.

Future work should begin with independently collected, balanced scenario sets for tool descriptions and outputs, memory writes and poisoning, leakage, RAG, browser content, and staged multi-turn attacks, with difficult benign controls for every surface. Multimodal browser evaluation should cover raw DOM trees, ARIA and accessibility structure, CSS visibility, hidden text, screenshots, visual overlays, and computer-use agents in environments such as ASB, WASP, VisualWebArena, EIA, and VPI-Bench [39,40,41,42,43]. Controlled factorial retraining should remove one component at a time under identical architectures, shared group-aware folds, seeds, and hyperparameters. Hidden-state detectors should be tested across model families and versions. Prospective end-to-end studies should report attack success, task completion and utility, unauthorized-action prevention, review burden, drift, sandbox failures, and latency under realistic concurrent load. Broader multilingual and code-mixed coverage, adaptive detector-aware attacks, and post-update recalibration are also required.

5. Conclusions

This study evaluated PromptSentinel-X as a leakage-aware, context-aware screening framework for prompt-injection risk in large language model-powered web agents. The primary analysis used 1581 English-language records from the Prompt Injection Malignant dataset, with 30,015 external records reserved for transfer, robustness, and benign hard-negative assessment. On the 465-record group-aware test set, PromptSentinel-X achieved 0.9849 accuracy, 0.8887 macro-F1, 0.9851 attack recall, a 0.0050 benign false-positive rate, and 0.0153 expected calibration error. The suspicious roleplay class remained uncertain because only six test records were available.

Leakage control materially affected interpretation. A random-split baseline reached 0.9491 macro-F1 but generated 144 leakage warnings, whereas family-aware splitting removed family and paraphrase overlap and produced a more conservative estimate. Context-aware performance was also substantially weaker than static classification: macro-F1 was 0.8301 for static prompts, 0.7594 for textual browser-content injection, 0.6306 for multi-turn staging, 0.3844 for RAG injection, and 0.3200 for memory-related injection. The tool-output result was based on one qualifying record and is descriptive only. Hard-negative testing further showed that simple trigger-word and intent-sensitive filters could not simultaneously provide high attack recall and low benign disruption.

The results support PromptSentinel-X as one layer in a broader security architecture for screening, escalation, sandbox routing, and human-review triage. It should not be treated as a stand-alone authorization mechanism. Runtime protection still requires provenance preservation, least-privilege tool access, validation of plans and outputs, protected memory writes, and confirmation for consequential actions.

The evidence remains limited by an English-only, imbalanced in-domain corpus; very small suspicious, tool-output, leakage, and memory subsets; proxy rather than controlled component removals; textual rather than multimodal browser evaluation; and classifier-level rather than end-to-end agent testing. Future work should expand balanced tool, memory, leakage, DOM and ARIA, visual, multilingual, adaptive, and live-agent benchmarks and assess whether detection reduces attack success while preserving task utility under realistic workloads.