When Cost Objectives Delete Capability: Accuracy-Constrained Tool Selection for Multi-Component Intrusion Detection in IoT Networks

Abstract

Agentic systems built on large language models are increasingly being allowed to decide for themselves which diagnostic tools to run and the obvious way to keep the resulting expense under control is to train that decision against a cost objective. We evaluate this question in a routed multi-component detection pipeline in which evidence fusion is deterministic so that the results characterize the tool selection layer in place of a full language-model-mediated inference loop. We show that cost-only optimization is unsafe in a repeatable and specific way. When the informative tools in a registry are also the expensive ones then a cost-driven policy removes exactly the detectors which the system exists to use and multiclass attack attribution collapses on both the datasets we study, with most attack families never predicted at all. The loss is invisible to the summary metric most often reported in this field because attack vs. normal detection remains high on an attack-heavy evaluation split even when the underlying predictions carry no information. We propose a constrained formulation in which the tools that carry class discriminative signal form a mandatory core placed outside the policy’s reach, while the remaining choice minimizes the measured invocation cost, which is subjected to an explicit relative accuracy floor. Classifier capacity is offered at three tiers rather than being fixed, so the floor decides how much model to buy for each traffic group rather than only which auxiliary tools to drop. On both the corpora the constrained policy runs at a small fraction of the cost of calling every tool and is statistically equivalent to it under a pre-specified margin. On both it converged exactly to its mandatory core so the demonstrated benefit is the preservation of signal-bearing tools and the correct choice of classifier capacity rather than dynamic selection among many useful alternatives. Removing the core reproduces the original collapse. Sweeping the floor traces an explicit cost-accuracy frontier and exposes its practical limit. The rarest attack family, and not the average across families, determines how far the tolerance can safely be relaxed since macro-averaging distributes a severe loss on one class across all of them. Every invocation cost reported here is timed, and we report indirect prompt injection as a null result.

1. Introduction

Security teams responsible for Internet of Things (IoT) deployments face a workload problem with a different shape from the one most intrusion detection research addresses. Deciding whether a flow looks unusual is comparatively easy, and a classifier fitted to labelled traffic does it dependably. The expensive judgement comes afterwards: naming the family of attack under way, working out what it is likely to do to the devices involved, and settling on a proportionate response. That judgement still falls to human analysts, and the alert volume of even a moderately sized estate outruns what an analyst team can absorb [1,2].

Agents built on large language models look like promising candidates for the second task, since they can weigh evidence of different kinds and explain the reasoning behind a verdict [3,4]. The obstacle is economic rather than technical. An agent running every available diagnostic on every incident costs several times as much per incident as a single classifier, and at IoT flow rates the arithmetic stops working. The natural response is to let the agent learn when each tool is worth running, a form of cost-aware inference that has drawn attention in the tool-use literature [5,6].

This paper reports what we found when we built that mechanism and measured it carefully, on two corpora with every cost figure obtained by timing. The first result is a failure with a clean and transferable cause, and we regard it as the more valuable half of what we have to offer. A policy trained to minimize invocation cost, with nothing in its objective tying it to detection quality, worked out that the classifiers in its registry were also its most expensive tools and stopped calling them. Multiclass attack attribution collapsed on both corpora, and most attack families were never predicted at all.

What makes this collapse hard to see is that it barely registers in the metric that is most reported. Under an attack-heavy evaluation split, attack versus normal detection stayed high in every configuration including the broken one, because a detector that unconditionally answers “attack” already scores well on such a split. Several configurations reported binary scores agreeing to sixteen decimal places, which is the signature of an identical prediction vector rather than of several systems performing equally well. Once the split was rebalanced the broken configuration was exposed, its binary detection falling below what a constant prediction achieves. The number that appeared to certify the pipeline had been measuring nothing.

Both halves of this failure were anticipated in the methodological literature. It has been argued for more than a decade that the closed-world assumptions behind machine learning sit with operational intrusion detection and that published accuracies frequently do not survive contact with deployment [7]. A catalogue of recurring pitfalls in security machine learning names inappropriate baselines and metrics that flatter a system without measuring the capability anyone cares about [8]. What we encountered is a concrete instance of the baseline pitfall, and we surfaced it only because we eventually computed the majority-class score explicitly instead of assuming it was low.

This work makes five contributions. First, we document the failure and its cause in reproducible detail on two independent corpora, on the view that a negative result with an identified mechanism is more useful to practitioners than another incremental accuracy figure. Second, we propose a constrained formulation for tool invocation in which the tools carrying class-discriminative signal are removed from the policy’s reach entirely and the remaining choice minimizes cost subject to an explicit relative accuracy floor. Third, we offer classifier capacity at graduated tiers rather than fixing it, so that the floor decides how much model to buy for each traffic group rather than only which auxiliary tools to discard; this turns a degenerate selection problem into a real one. Fourth, we measure every invocation cost by timing the tools on real incidents, separating the calculations a tool actually performs from the fixed overhead paid on every class regardless of which tool is chosen. Fifth, we sweep the accuracy floor to trace the cost-accuracy frontier it defines and identify where that frontier stops being safe, like the rarest attack family, not the average across families, determines how far the tolerance can be relaxed.

The rest of the paper is organized as follows. Section 2 places the work against IoT intrusion-detection datasets and evaluation practice, agentic tool use, cost-aware inference, and research on indirect prompt injection. Section 3 describes both corpora, the preprocessing decisions that two separate defects forced upon us, the measured cost model, the pipeline and the constrained formulation. Section 4 reports the achievable ceilings, detection and cost results, the core-removal experiment, the accuracy-floor frontier and injection robustness. Section 5 draws out the general lessons and states the limits on our claims. The paper concludes with Section 6.

2. Related Work

2.1. IoT Intrusion-Detection Datasets and Evaluation Practice

The TON_IoT collection was assembled to supply heterogeneous and realistically generated telemetry, an operating system and network traffic for evaluating security applications [1,9]. Its network component provides Zeek-derived flow features covering nine attack categories alongside benign traffic [1]. A study of the collection’s internal heterogeneity argued that the absence of feature standardization across its constituent sources complicates comparison between papers [10], an observation our own results reinforce from an unexpected direction, since the two corpora we use differ so sharply in achievable accuracy that a result on one cannot be read as evidence about the other.

Bot-IoT was built at the same institution to concentrate on botnet activity within IoT networks and supplies a huge, labelled corpus whose extreme class imbalance has itself become a subject of study [11]. The other dataset remain in circulation. UNSW-NB15 introduced a hybrid of synthetic and captured traffic described by forty-nine features [12]. CICIDS2017 emphasized realistic background traffic and protocol diversity [13]. Edge-IIoTset extended coverage toward federated and edge deployment [14], and NBaIoT supplied device-level botnet traces captured from commercial hardware [15]. Multiclass evaluation of these corpora is itself an active problem, and recent work has examined how scalable frameworks behave when several attack families must be separated rather than just detected [16].

Survey work on this literature is extensive. Network-based intrusion-detection datasets have been catalogued alongside the properties that determine their suitability [17], and the field’s techniques, datasets and open challenges have been reviewed at length [18]. Learning-based detection under IoT constraints [19], machine learning across the IoT threat surface [20] and the broader landscape of machine and deep learning for IoT security [21] have all been surveyed within the last few years. A critical review of detection practice for IoT set out what a universal and resilient system would have to provide [22], and a later one combined a threat taxonomy with an assessment of how current datasets shape and sometimes distort the systems trained on them [23]. An older survey of data-mining methods for intrusion detection remains a useful reference point for how little the evaluation conventions have changed in the interval [24].

A theme recurs across the methodological contributions in this area: evaluation practice lags behind modelling practice. A comparison of deep learning approaches across datasets found that reported performance depends heavily on preprocessing choices that papers rarely describe in full [25]. An examination of whether models trained on public intrusion-detection corpora transfer at all found the transfer weak [26]. A case for a standard feature set was made precisely because incompatible feature definitions make cross-corpus claims unverifiable [27]. The gap between machine learning results reported in cybersecurity papers and the requirements of operational deployment has been surveyed directly [28], and supervised methods evaluated within a single corpus have been shown to generalize poorly once the corpus changes [29]. Our finding that a saturated binary metric can conceal a total collapse in multiclass attribution belongs to this tradition of results about measurement rather than about models.

The specific measurement issue we encountered has been treated formally. Receiver operating characteristic curves give an over-optimistic impression on imbalanced data where precision-recall analysis does not [30]. The distortions that imbalance introduces into learning and into reported metrics have been surveyed [31] and studied systematically across classifier families [32]. A systematic analysis of performance measures makes clear why a summary binary score and a macro-averaged multiclass score can diverge as far as ours did [33]. Synthetic resampling remains the standard response to imbalance [34], although we deliberately downsample rather than synthesize for reasons given in Section 3.3. How imbalance propagates into metrics derived from the confusion matrix has been analyzed directly [35], and predictive modelling under imbalanced distributions has been surveyed more broadly [36].

2.2. Large Language Model Agents and Tool Invocation

The pattern of interleaving reasoning traces with external actions has been adopted, in one variant or another, by most contemporary agent frameworks [3]. Later work showed that a model can learn where in a generation to insert an API call without task-specific supervision [5] and that structured planning improves completion of multi-step tasks [6]. The resulting design space has been surveyed from the perspective of agent capability [4], of agent architecture [37], and of the coordination problems that arise once several such agents operate together [38].

Applications to security followed quickly. The dual-use character of generative models in cybersecurity and privacy has been surveyed, noting both defensive applications and the offensive capability the same models confer [39]. Security and privacy considerations across the language model lifecycle have been reviewed [40] and the attack surface these systems introduce when embedded in larger pipelines has been catalogued [41]. Generative approaches to cyber threat detection have been surveyed specifically [42], and a systematic review spanning more than three hundred studies has since mapped how these models are being applied across cybersecurity tasks [43].

Multi-agent organization of this work draws on a longer tradition. Multi-agent system architectures and the coordination problems they raise have been surveyed [44], as have trust management mechanisms for settings in which individual agents may be unreliable or compromised [45]. The trust-weighted consensus we employ is a straightforward instance of that class. To our knowledge, however, the particular interaction we study here between a learned tool-invocation policy and the class-discriminative capability of the tools that policy is allowed to decline has not previously been isolated and measured.

2.3. Cost-Aware Inference and Model Capacity Selection

The idea that inference cost should be matched to the difficulty of the instance is not new to agentic systems. Cascade architectures route easy inputs to cheap models and hard ones to expensive models, an approach studied under several names across vision and language [46,47]. Anytime and budgeted prediction frameworks formalize the same tradeoff as an explicit optimization [48], and the adaptive setting in which a cheap model defers to an expensive one is the closest formal analogue to what our accuracy floor chooses between [49]. Our tiered classifier registry is a deliberately simple instance of this idea, and the contribution is not the tiering itself but the observation that any accuracy floor placed over it behaves as a tunable control on the cost–accuracy frontier with a measurable point beyond which further relaxation stops being safe.

2.4. Indirect Prompt Injection

Instructions embedded in model input can override directives supplied at the system level, and the indirect form of this attack, in which adversarial content arrives through a retrieval channel rather than from the user, means that any agent consuming external documents inherits the exposure [50]. Automated procedures now exist for generating such payloads at scale, which removes the assumption that an attacker must handcraft each attempt [51]. The resulting taxonomy of prompt injection attacks and the defenses proposed against them has been surveyed [52], and the problem has been examined specifically in the context of tool-using agents [53]. An agentic intrusion-detection system retrieving threat intelligence is precisely the configuration these works describe, which is why we designated the threat-intelligence agent as the attack surface and evaluated it against both the payload family the filter was written for and a different family containing none of the imperative markers it matches. We report the outcome as a null result in Section 4.6 and are explicit that a null result is not evidence of defensive efficacy.

2.5. Base Learners and Statistical Methodology

The tool registry uses random forests [54] and isolation forests [55] as supervised and unsupervised detectors respectively, since both are conventional choices in this setting. An extension of isolation forests addressing a bias in the original splitting rule has been proposed [56], and the anomaly-detection landscape has been surveyed more broadly [57]. Empirical evidence supports treating random forests as a strong default across tabular classification problems [58], which is the role they play here, and an analysis of which hyperparameters matter most for random forests informs our tier design [59].

For inference we use paired bootstrap resampling [60] together with the paired test appropriate for classifiers evaluated on a shared partition [61]. Because our TON_IoT test partition is large, differences are far too small to matter operationally or reach nominal statistical significance, so we report relative effect sizes beside every p-value rather than relying on significance thresholds alone. Concept drift is a further consideration for any deployed detector and has been surveyed from the adaptation perspective [62] and from the learning perspective [63].

2.6. Summary and Key Takeaways

Four gaps emerge from this survey and shape what follows. First, evaluation practice in IoT intrusion detection is known to be fragile, but the specific failure in which a summary binary metric conceals a complete collapse of multiclass attribution has not been documented with its cause identified and replicated. Second, the agentic tool-use literature treats tool selection largely as an efficient problem, and the possibility that a cost-driven policy might discard the very tools carrying the task-relevant signal has not been isolated experimentally. Third, cost figures in this literature are frequently asserted rather than measured, which makes efficiency claims unverifiable. Fourth, robustness evaluations for agentic security systems report defense efficacy without first establishing whether the attack had any effect to prevent.

Table 1. Position of this work relative to the most closely related studies. We are not aware of prior work that measures the interaction between the cost-driven tool-invocation policy and the class-discriminative capability of the tools it is allowed to decline with costs obtained by timing and results replicated across corpora.

Study	Focus	Tool Policy	Costs Timed	Ceiling Reported	Majority Baseline
[19]	IoT IDS survey	N/A	no	no	no
[25]	deep learning for IDS	N/A	no	no	partial
[26]	cross-corpus transfer	N/A	no	no	no
[27]	standard feature sets	N/A	no	no	no
[29]	generalization	N/A	no	N/A	no
[49]	adaptive cost	learned	yes	no	no
[53]	agent injection	N/A	no	N/A	N/A
This work	agentic IoT IDS	constrained	yes	yes	yes

“N/A” means the dimension does not apply to that study’s design. “Partial” means the quantity is discussed but not reported alongside each result.

places our contribution against the most closely related work.

The gap this work addresses can be stated in one sentence. Cost-aware inference has been studied extensively as an efficiency problem, and tool selection for agents has been widely studied as a capability problem, but the case in which the two conflict, where the informative tools are also the expensive ones so that the optimizing cost removes capability, to our knowledge has not been isolated, measured with time costs and replicated across corpora.

3. Materials and Methods

3.1. Corpora

We evaluate on two independent corpora so that every claim can be checked for replication. Both are public and they differ enough in difficulty that a result appearing on only one of them should be reported as specific to that corpus rather than general.

The first is the network component of TON_IoT, distributed as a single curated train-and-test file of 211,043 labelled flows [1,10]. After removing identifiers, timestamps, free-text fields and the label column, 29 usable features remain. Keeping source and destination addresses would let a classifier memorize the attack infrastructure of the capture rather than learn transferable behavior, a form of leakage that inflates reported accuracy without any operational capability [8].

The second is Bot-IoT, taken in its full release of 73,370,443 flows across 74 files [11]. After removing identifiers and two columns that never vary, 19 usable features remain. The corpus is extraordinarily imbalanced, with benign traffic accounting for 9543 flows. We exclude the theft category which contains 1587 records and would otherwise cap the per-family sample size for the entire study. We state this plainly because it is a judgement rather than a technicality, and a reader may reasonably prefer the alternative.

3.2. Traffic Grouping

Flows are grouped so that each group receives its own fitted detectors, which preserves the multi-domain routing structure of the agent architecture. TON_IoT is grouped by application layer service, and Bot-IoT by transport protocol, since it carries no service field. This is a genuine difference in what a group means between the two corpora, and we state it rather than glossing over it.

The group earns its own detectors only if it contains at least five thousand flows and at least four distinct classes on TON_IoT, or three on Bot-IoT. Both tests matter. A group with many flows but a single class scores a trivial macro-F1 of one and tells us nothing, and a group with several classes but fewer flows receives too few validation incidents for tier comparison to mean anything. Services failing either test are merged into a residual group, and if the residual group is itself too small it is folded into the large viable group since a group with no fitted detectors produces no evidence at all and every one of its incidents would default to benign.

This procedure yields three groups on TON_IoT and, after rebalancing, two on Bot-IoT, as summarized in

Table 2. Traffic groups after viability filtering and rebalancing. Each group receives independently fitted detectors at three capacity tiers.

Corpus	Group	Train	Test	Live Features	Classes
TON_IoT	Unclassified	22,895	23,194	20	10
TON_IoT	DNS	9610	9581	17	9
TON_IoT	HTTP	5786	5800	16	8
TON_IoT	pooled	64,291	38,575	29	10
Bot-IoT	UDP	7313	4395	19	4
Bot-IoT	TCP	6320	3786	18	4
Bot-IoT	Pooled	13,633	8181	19	4

A feature counts as live for a group when it varies within that group’s traffic. Protocol specific fields are necessarily absent elsewhere and including them as constants would present a column of zeros to detectors.

. On TON_IoT, six services including FTP and SSL are merged into the residual group. FTP in particular carried 1065 flows of a single class. On Bot-IoT the ICMP group passes the test on the full corpus with 32,322 flows but falls to 101 flows after rebalancing and is folded into UDP at that point. Group viability must therefore be checked against the partition actually used and not against the source corpus.

3.3. Class Composition and Rebalancing

Two aspects of our original protocol needed correction, and we describe both because each on its own invalidated a headline number and each is easy to reproduce in any study of this kind.

The first concerns is class balance. Our initial TON_IoT partition contained 164 attack flows for every 25 benign fractions. On such a partition, a constant “attack” prediction achieves a binary F1 of 0.9292, which is indistinguishable from what our system was reporting. We therefore rebalance both corpora toward a target benign fraction of 0.35. Because benign flows are the limiting resource in both cases and overwhelmingly so in Bot-IoT, we downsample attack families rather than oversampling benign traffic. Synthetic oversampling [34] would place fabricated instances into a partition meant to measure behavior on real traffic, and we judged the resulting ambiguity worse than the loss of data.

Families are capped at the size the benign supply allows, except that a family smaller than the cap keeps its natural size rather than dragging every other family down to it. On TON_IoT this preserves man-in-the-middle at 1043 flows while other families are capped at 10,317; forcing uniformity would have discarded roughly nine tenths of the usable data to accommodate a single class. That decision turns out to matter for the results since the rare class is what determines how far the accuracy floor can be relaxed.

Table 3. Evaluation partitions for both corpora. The majority-class binary F1 is the score a constant prediction achieves and is quoted beside every binary result in Section 4.

Corpus	Pool	Train	Validation	Test	Benign Fraction	Majority Binary F1
TON_IoT	128,583	64,291	25,717	38,575	0.350	0.7876
Bot-IoT	27,267	13,633	5453	8181	0.350	0.7839

and Figure 1 give the resulting composition.

3.4. Measured Cost Model

The second correction concerns cost accounting, and it is the one we consider most important to state plainly. Our original cost function used hand-written constants, and a policy optimized against it reduced tool cost by a factor of nearly ten while increasing token consumption and end-to-end latency, so that the apparent saving was a transfer of expenditure into a category the objective could not see. Worse, the constants had never been checked against reality. An efficiency claim resting on assumed costs cannot be verified, and we now regard asserting costs as a methodological defect in its own right.

Every cost reported in this paper is therefore timed. Each tool is measured on real validation incidents, with warm-up passes discarded and the median of repeated timed passes taken, so that a single garbage-collection pause or scheduler interruption cannot define a published figure. The measurement separates two quantities that a naive approach runs together. The marginal compute is the work the tool actually performs, measured in batch and divided by batch size. The fixed overhead is the per-call cost of framework validation, array construction and dispatch, paid once on every invocation regardless of which tool is chosen.

That distinction is not academic. On our hardware the fixed overhead is 3.64 ms per call on TON_IoT and 3.56 ms on Bot-IoT, two orders of magnitude larger than the marginal compute of even the most expensive classifier. Timing single invocations therefore returns nearly identical figures for the cheapest and most expensive classifiers, which flattens exactly the distinction the tiered registry depends upon. Normalized cost units are based on marginal compute, since that is the term the policy’s decision depends on, and the overhead is reported alongside.

Costs are normalized so that the cheapest measured tool equals one unit, and tool compute and language model tokens share the unit so that the policy cannot reduce one by inflating the other. Measurements were taken on an Apple Silicon workstation running macOS 15.5 and Python 3.14.6.

Table 4. Timed invocation costs in normalized cost units, by corpus and group. Costs are marginal batched compute per incident, normalized so that the cheapest tool equals 1.0.

Corpus	Group	Lite	Mid	Full	Isoforest	Rules
TON_IoT	unclassified	318	905	1770	610	1.1
TON_IoT	DNS	678	1847	3547	1404	2.0
TON_IoT	HTTP	807	2095	3941	1754	2.4
Bot-IoT	UDP	279	694	1403	556	1.0
Bot-IoT	TCP	310	769	1546	610	1.1

Classifier tiers scale roughly 1:2.4:4.9 across both corpora. Rule checks are three orders of magnitude cheaper than any classifier, which is why removing them saves almost nothing: the classifier is the cost. To make the unit concrete, one normalized cost unit corresponds to approximately 65 nanoseconds of marginal compute on the reference hardware. The constrained policy’s 1120 units are therefore about 73 microseconds of classifier compute per incident, against about 225 microseconds when every tool is called. Absolute figures are the properties of this machine, and the ratios between capacity tiers are the transferable quantity.

gives the measured registry.

3.5. Tiered Tool Registry

Each group receives three classifiers at graduated feature budgets rather than one. The lite tier uses the five features ranked highest by mutual information, with sixty trees at depth eight; the mid tier uses twelve features with one hundred and fifty trees at depth sixteen; the full tier uses every live feature with three hundred unrestricted trees. Feature ranking is computed once per group and reused across tiers, so the tiers are nested rather than independently selected.

This design decision is not cosmetic. Our earlier registry contained one classifier per group at a single capacity, plus auxiliary detectors that measured at or below zero marginal value. Selection was therefore degenerate: calling the classifiers was trivially optimal, and the learned policy converged on that subset every time, which made the constrained formulation impossible to evaluate because there was nothing to choose between. With graduated capacity the policy faces a real accuracy-per-unit-cost tradeoff, and the accuracy floor becomes a control parameter rather than a formality.

Because the tiers are the same model at different capacities, they are alternatives rather than complements. Calling several tiers of the same group’s classifier lets two weaker tiers outvote the strongest, which is dilution rather than corroboration, so the pipeline keeps only the widest tier present per group at invocation time. Auxiliary detectors, comprising an isolation forest, a quantile rule check and a distributional drift check, are retained because a registry containing only classifiers would beg the question the policy is meant to answer.

3.6. Pipeline Architecture

Before describing the pipeline, we state its scope precisely because the distinction matters for how the results should be read. The configuration evaluated in this paper routes each incident to a group of fitted detectors and fuses their output deterministically. No language model participates in the reported experiments, and token cost is therefore zero by construction rather than by measurement. What follows is an evaluation of the tool selection layer, not of a language-model-mediated interference loop, and we are careful throughout Section 4 and Section 5 to claim no more than that.

We use four terms precisely and distinguish them here. A tool is a fitted detector that consumes an incident and returns evidence. In this work, a classifier has one of three capacities: an isolation forest, a rule check or a drift check. A component is one of the five stages that routes, enriches, analyzes, validates and fuses that evidence together. The policy is the procedure that decides which tools a component may call. A language model would be an alternative implementation of the fusion component and is discussed as such among the limitations, but it does not play a role in the experiments reported here.

The pipeline comprises five agents together with a security layer that operates across all of them, shown in Figure 2. A monitoring agent performs initial triage and casts a preliminary vote from inexpensive detector output. A threat-intelligence agent retrieves external context, and this is the component we designate as the injection surface for the experiments in Section 4.6. An analysis agent determines the attack family and probable impact. A validation agent checks the accumulated votes for internal inconsistency. A supervisor agent fuses all votes through trust-weighted consensus into a final verdict.

Evidence entering the consensus is weighted by kind rather than counted uniformly, with fitted classifiers assigned four times the weight of anomaly scores. This is not a tuning convenience but a correction for a defect we observed directly. Under uniform averaging, three weak detectors agreeing on an incorrect label could outvote one confident classifier and the resulting verdicts were worse than those of the classifier alone.

A second and less obvious rule governs the fusion. Anomaly scores, rule checks and drift checks are binary. They answer whether an incident looks unusual, not which family it belongs to. They may therefore never name a final label. An earlier implementation allowed them to vote for a literal “attack” token, which is not one of the corpus classes. Binary evidence is now spread across the attack classes in proportion to whatever classifier evidence exists, and where no classifier was called it can support only the benign decision, which is the honest answer since binary evidence cannot perform attribution.

The security layer runs across all stages and comprises three mechanisms: a per-agent tool firewall restricting which detectors each agent may call, memory compartments that quarantine retrieved intelligence exhibiting instruction-like structure, and dynamic trust scoring that reduces the consensus weight of agents whose votes diverge persistently from the median.

3.7. Constrained Policy Formulation

Let T denote the tool registry and c(t) the timed normalized cost of tool t. The unconstrained formulation minimizes total invocation cost subject only to a budget, and it is this formulation that produces the collapse reported in Section 4.4. It is worth saying plainly that the policy behaves correctly with respect to the objective it is given. The objective is incomplete.

The constrained formulation instead solves

$$\mathrm{m}\mathrm{i}\mathrm{n}\sum \mathrm{c}\left(\mathrm{t}\right) \mathrm{s}\mathrm{u}\mathrm{b}\mathrm{j}\mathrm{e}\mathrm{c}\mathrm{t} \mathrm{t}\mathrm{o} {\mathrm{F}}_1\left(\mathrm{S}\right)\ge \left(1-\mathsf{\epsilon }\right){\mathrm{F}}_1\left(\mathrm{T}\right),\mathrm{C}\subseteq \mathrm{S}$$

(1)

where S is the selected subset, C a mandatory core placed outside the policy’s reach entirely, and ε a relative tolerance. Three properties matter. The accuracy floor is expressed relative to the performance of the full registry rather than as an absolute target, so it remains meaningful on corpora where the achievable ceiling is low. The mandatory core is enforced structurally, by removing those tools from the set the policy may act upon, rather than by penalizing their omission, because a penalty can always be outweighed by a large enough cost saving, and the largest savings available are precisely those from skipping the most expensive tool. And because the objective is a minimization, the policy stops once the floor is satisfied, rather than spending any budget that remains. An earlier implementation kept buying tools with positive marginal value and thereby re-added the expensive tier the floor had just replaced, which made the constrained policy cost more than its own core for slightly less accuracy.

Membership in C is decided by measurement rather than by assertion and the procedure has three stages. First, for each traffic group, every classifier tier is evaluated on its own against the group’s validation incidents and the cheapest tier scoring at least one minus the tolerance times the best tier’s score is admitted. Second, each admitted member is checked against the core rather than against the full registry where we ask what is lost if this tool goes and nothing replaces it and not what is lost if it goes while its siblings remain. That distinction is easy to get wrong and quietly empties the core, because leave-one-out testing over a registry containing redundant alternatives reports that every tier is dispensable. Third, the marginal value of each remaining candidate is estimated given the core, again within its own group since a group’s classifier fires only on that group’s traffic.

Restricting tier comparison to a group’s own incidents is equally necessary. A group’s classifier fires only on that group’s traffic, so scoring it against the whole validation set measures mostly rows it never saw, and the comparison degenerates into near-ties.

3.8. Evaluation Design and Experimental Conditions

We state the separation of partitions explicitly because the policy is itself fitted. Tier comparison, core membership, marginal values and decision thresholds are calculated on the validation partition alone. The test partition informs no policy decision and is used only to produce the results reported in Section 4. Decision thresholds are calibrated independently for each condition on the validation partition, hereby maximizing balanced accuracy. Different tool subsets produce different score distributions, so a threshold appropriate to one is not appropriate to another; in an earlier protocol, calibrating once under exhaustive invocation and reusing that threshold drove three conditions below the majority-class baseline on its own.

We report macro-averaged F1 over all classes as the primary metric, since it weights each attack family equally and therefore registers the disappearance of a rare family that an accuracy-based measure would not. Binary detection F1 is reported alongside, always with the majority-class baseline adjacent and binary balanced accuracy as a secondary check because F1 on an imbalanced partition remains hard to interpret even when the baseline is stated [33]. Confidence intervals come from bootstrap resampling with two thousand replicates and paired comparisons between conditions apply the same resampling to the paired difference. Because a non-significant difference is not evidence of equivalence, we assess equivalence directly using two one-sided tests with a margin of δ = 0.005 macro-F1. The margin was fixed in advance as the smallest difference we would regard as optionally meaningful on these corpora and equivalence is declared when the bootstrap interval for the paired difference lies entirely within the δ change. Fitting and fusion are deterministic, so the five random seeds vary only the bootstrap draw. Therefore we report a single figure per condition rather than a mean and standard deviation across seeds, since the latter would overstate what the repetition establishes.

Seven conditions are evaluated, summarized in

Table 5. Experimental conditions. The core-removal condition is exempt from the automated quality check, since its purpose is to reproduce a failure under controlled conditions.

Condition	Tools Called	Core Enforced	Role
Exhaustive	All, widest tier per group	Yes	Cost upper bound
Domain-all	All in group	Yes	Routing baseline
Fixed	One full tier per group	Yes	Unmeasured engineer’s choice
Random	Uniform random subset	No	Sanity check
Core-only	Mandatory core alone	Yes	Minimal sufficient set
Constrained	Core under accuracy floor	Yes	Method under evaluation
Core removed	Cost-minimal subset	No	Reproduces the failure

. Exhaustive invocation calls every registered tool, taking the widest classifier tier per group, and establishes the cost upper bound. Domain-restricted invocation calls all tools associated with the incident’s group. Fixed selection uses one full-tier classifier per group and represents the configuration a careful engineer would reach without measurement. Random selection draws a subset uniformly and serves as a sanity check, since a policy that fails to beat random selection is not a policy. Core-only calls the mandatory core alone. The constrained policy is the method under evaluation. Finally, a core-removal condition reproduces the unconstrained formulation deliberately, under a budget below the cheapest classifier, so that real cost pressure applies. It is retained precisely because it demonstrates the cause rather than because it performs well.

4. Results

4.1. Achievable Ceilings

Before evaluating any agent configuration, we established what each corpus permits. Fitting classifiers directly to the labels, with no agent layer of any kind, yields a pooled macro-F1 of 0.9578 on TON_IoT and 0.999 on Bot-IoT. Every agent result below should be read as a fraction of these figures, rather than of unity, since no orchestration strategy can extract information that the detectors do not have access to.

We stress the word pooled. An earlier version of this analysis calculated a weighted mean of per-group macro-F1 scores and obtained 0.8755 on TON_IoT, which we initially took as the ceiling. That figure is not comparable to an agent result, because averaging macro-F1 across groups containing different class subsets is a different statistic from macro-F1 over pooled predictions where the HTTP group contains eight classes and the residual group ten, so per-group average implicitly reweights the classes. Reporting the incomparable figure would have made our agent appear to exceed the achievable maximum by eight points, an impossible result that should be read as a bug rather than a finding. Establishing a ceiling is only useful if the ceiling is calculated as the same statistic being compared against it.

The gap between the two corpora is itself informative and is shown in Figure 3. Bot-IoT is close to saturated with four classes and nineteen features, its ceiling is 0.999, and its binary ceiling is exactly 1.00. TON_IoT, with ten classes including one rare family, tops out at 0.9578. A method that appears strong on Bot-IoT alone has demonstrated very little, and we use it as a replication corpus rather than as a primary result throughout.

Table 6. Achievable macro-F1 under different fitting regimes. The pooled figure is the comparator for all agent results; the averaged figure is a different statistic and is included only to document a comparison error we made and corrected.

Corpus	Pooled	Averaged	Global	Binary	Comparable to Agents
TON_IoT	0.9552	0.8755	0.9578	0.9974	pooled and global only
Bot-IoT	0.9999	0.9990	0.9999	1.0000	pooled and global only

reports the achievable ceiling for both corpora under each fitting regime.

4.2. Detection Performance and Cost on TON_IoT

Table 7. Detection performance and cost on the TON_IoT network corpus. Mean over five seeds, n = 38,575 test flows. Measured macro-F1 ceiling 0.9578; majority-class binary F1 0.7876.

Condition	Macro-F1	% of Ceiling	Binary F1	Cost (NCU)	Tools	Dead
Exhaustive	0.9549	99.70	0.9976	3451	3.6	0
Domain-all	0.9549	99.70	0.9976	3451	3.6	0
Fixed	0.9555	99.76	0.9978	2542	1.0	0
Random	0.8888	92.80	0.9366	2355	2.4	0
Core-only	0.9542	99.62	0.9974	1120	1.0	0
Constrained	0.9542	99.62	0.9974	1120	1.0	0
Core removed	0.0511	5.33	0.0217	2.0	1.6	8/10

reports all seven conditions on TON_IoT. The constrained policy attains a macro-F1 of 0.9542 at 1120 normalized cost units per incident against 0.9549 at 3451 units for exhaustive invocation. The accuracy difference is 0.0007, relative difference of 0.077 percent while the cost difference is a factor of 3.08. The bootstrap interval for this paired difference is [−0.0031, +0.0014] over 38,575 paired instances, which lies entirely within the pre-specified equivalence margin of 0.005, so the two conditions are statistically equivalent rather than merely indistinguishable. Figure 4 shows the resulting cost-accuracy frontier.

The paired difference between the constrained policy and exhaustive invocation is −0.0007 macro-F1, with a bootstrap interval of [−0.0031, +0.0014] over 38,575 paired instances. This interval lies entirely within the pre-specified equivalence margin of 0.005, so the two are statistically equivalent rather than merely indistinguishable. The corresponding p-value of 0.526 confirms that no difference is detected but it is the equivalence result and not the failure to reject that supports the claim.

Table 8. Paired comparisons against exhaustive invocation on TON_IoT. Differences are in macro-F1, from 2000 paired bootstrap replicates over the shared test partition.

Condition	Δ Macro-F1	95% CI	p	Relative (%)	Interpretation
Domain-all	0.0000	[0.0000, 0.0000]	1.000	0.00	identical
Fixed	+0.0006	[−0.0002, +0.0018]	0.278	+0.06	equivalent
Random	−0.0650	[−0.0716, −0.0585]	<0.001	−6.81	moderate loss
Core-only	−0.0007	[−0.0031, +0.0014]	0.526	−0.08	equivalent
Constrained	−0.0007	[−0.0031, +0.0014]	0.526	−0.08	equivalent
Core removed	−0.9039	[−0.9100, −0.8968]	<0.001	−94.65	catastrophic

reports the full set of paired comparisons with relative effect sizes beside every p-value, because with a partition this large a difference far too small to matter operationally can still reach nominal significance, and the fixed condition shows the converse being nominally better by 0.0006 with p = 0.278.

An honest reading of Table 7 requires acknowledging that the constrained policy and the core-only condition are identical on this corpus, agreeing exactly on every metric. This is not coincidence. The core is chosen so that it already satisfies the accuracy floor, and because the objective is minimization the policy correctly stops there rather than spending further budget. The contribution of the formulation is therefore not the discovery of an elaborate subset but the selection of the right capacity and the avoidance of a catastrophic subset, and the core-removal experiment in Section 4.4 is what makes the second half of the claim visible. Figure 5 shows the marginal-value estimates that produced the selection.

4.3. Replication on Bot-IoT

Table 9. Detection performance and cost on Bot-IoT. Mean over five seeds, n = 8181 test flows. Measured macro-F1 ceiling 0.9999; majority-class binary F1 0.7839.

Condition	Macro-F1	% of Ceiling	Binary F1	Cost (NCU)	Tools	Dead
Exhaustive	0.9990	99.91	0.9991	2051	3.0	0
Domain-all	0.9990	99.91	0.9991	2051	3.0	0
Fixed	0.9999	100.00	0.9999	1469	1.0	0
Random	0.9376	93.77	0.9724	1404	2.1	0
Core-only	0.9997	99.98	0.9997	294	1.0	0
Constrained	0.9997	99.98	0.9997	294	1.0	0
Core removed	0.2422	24.22	0.7266	1.0	1.0	2/4

reports the same seven conditions on Bot-IoT. The constrained policy attains 0.9997 at 294 normalized cost units against 0.9990 at 2051 for exhaustive invocation, a factor of 6.98 cheaper while scoring marginally higher, with a paired difference of +0.0007 and a bootstrap interval of [0.0000, 0.0015] that again falls within the equivalence margin. The direction of that difference is worth a sentence. Exhaustive invocation includes the auxiliary detectors, and on a corpus where the classifier is essentially perfect those detectors can only add noise to the consensus. Invoking fewer tools is better here, not merely cheaper.

Two findings replicate cleanly across the corpora and one does not. Detection collapses in both cases when the core is removed, and the cheapest classifier tier suffices in every group of both corpora. The accuracy-floor frontier, however, cannot be exercised on Bot-IoT at all since every tier scores exactly 1.0000 on validation, so the cheapest is selected at every tolerance and the sweep is flat. We report Bot-IoT as a replication of the underlying behavior rather than as a second demonstration of the frontier and treat any claim that requires headroom as TON_IoT-specific. Bot-IoT therefore carries the replication of the collapse and of the capacity finding, but it cannot exercise the accuracy cost frontier at all. With a ceiling of 0.999 and every tier scoring identically on validation, the cheapest tier satisfies any tolerance we might set and the sweep is flat by construction rather than by result. We draw no frontier conclusion from this corpus.

4.4. Removing the Core Reproduces the Failure

Removing the mandatory core while holding every other component fixed, under a budget below the cheapest classifier so that genuine cost pressure applies, reduces macro-F1 from 0.9542 to 0.0511 on TON_IoT, a relative loss of 94.65 percent, and from 0.9997 to 0.2422 on Bot-IoT, a relative loss of 75.75 percent. Both are significant at p < 0.001 by paired bootstrap. Figure 6 places the core-removal experiment beside every other condition.

The per-class breakdown in Figure 7 shows what the aggregate figure conceals. On TON_IoT the collapsed configuration scores exactly zero on nine of ten classes, retaining only partial performance on benign traffic at 0.5089. Its binary F1 of 0.0217 sits far below the 0.7876 that a constant “attack” prediction would achieve, which is the clearest possible signal that the configuration is not merely degraded but actively anti-correlated with the labels. On Bot-IoT the pattern is the same in kind: denial of service and reconnaissance fall to zero, distributed denial of service retains 0.2536, and binary F1 of 0.7266 again falls below the 0.7839 baseline.

Figure 8 reports binary detection against the majority-class baseline, which is the form in which we believe binary results in this literature should be presented. Presented this way, the collapse is unmistakable.

4.5. The Accuracy Floor as a Control Parameter

Sweeping the tolerance ε traces the frontier the constraint defines.

Table 10. Accuracy-floor sweep on TON_IoT. Tier assignments are listed for the DNS, HTTP and residual groups respectively. Cost ratios are against exhaustive invocation at 3451 NCU.

ε	Tier Assignment	Macro-F1	Cost (NCU)	Cost Ratio	Rel. Loss (%)
0.005	full + lite + mid	0.9534	1536	2.25 times	0.16
0.010	full + lite + mid	0.9534	1536	2.25 times	0.16
0.020	lite + mid + mid	0.9542	1120	3.08 times	0.07
0.050	lite + lite + mid	0.9411	768	4.49 times	1.45
0.100	lite + lite + mid	0.9411	768	4.49 times	1.45
0.200	lite + lite + lite	0.9360	482	7.16 times	1.98

and Figure 9 report the result on TON_IoT, where four distinct operating points emerge, each corresponding to a different assignment of classifier capacity across the three traffic groups.

The second is the operational limit, shown in Figure 10. Relaxing the tolerance from 0.02 to 0.05 costs only 1.45 percent of macro-F1 in aggregate, which looks like an acceptable trade for a further 46-percent cost reduction. Inspecting the per-class effect tells a different story. At ε = 0.02 the rarest family, man-in-the-middle at 1043 training flows, is unharmed and in fact marginally improves, from 0.8120 under exhaustive invocation to 0.8139. At ε = 0.05 it falls to 0.7308, a relative loss of ten percent, while every other family loses under one percent. Because macro-F1 averages over ten classes, a ten-percent loss on one of them contributes roughly one percent to the aggregate and is easily mistaken for uniform mild degradation. The rarest class, not the average, is what determines how far the floor can be relaxed.

4.6. Indirect Prompt Injection: A Null Result and What It Implies for Evaluation Design

We evaluated indirect prompt injection across three arms on 300 incidents per corpus: a clean control, an injected arm with adversarial content planted in retrieved threat intelligence, and a defended arm with the security layer active. Two payload families were used, one comprising the imperative forms the defensive filter was written against and one comprising paraphrases containing no imperative markers.

On both corpora, in both payload families and in both arms, no verdict changed. The verdict-change fraction, the binary-flip fraction, the change in false-negative rate and the change in false-positive rate are all exactly zero, on TON_IoT and on Bot-IoT alike.

Table 11. Indirect prompt injection results on both corpora, n = 300 incidents each. Both arms and both payload families returned no change from the clean control.

Corpus	Payload Family	Arm	Verdicts Changed
TON_IoT	in-distribution	injected	0.00%
TON_IoT	in-distribution	defended	0.00%
TON_IoT	held-out	injected	0.00%
TON_IoT	held-out	defended	0.00%
Bot-IoT	in-distribution	injected	0.00%
Bot-IoT	in-distribution	defended	0.00%
Bot-IoT	held-out	injected	0.00%
Bot-IoT	held-out	defended	0.00%

records the result.

We report this as a null result and decline to draw the conclusion it superficially invites. The attack did not move verdicts, so the defense therefore had no opportunity to demonstrate efficacy and nothing here should be read as evidence that it works. Why the attack failed is a separate question that these experiments cannot answer.

We note this partly as a methodological point. A robustness evaluation that reports only the defended arm on a system whose undefended arm was already unaffected will report perfect defensive efficacy while having measured nothing, which is structurally the same error as the saturated binary metric described in Section 1.

4.7. Key Findings

Two distinct mechanisms are at work in these results and it is worth separating them. The recovery from collapse is attributable to the mandatory core which the core-removal condition isolates directly; with the core enforced the pipeline reaches the achievable ceiling and without it detection falls below a constant prediction. The cost reduction is attributable to the capacity decision made by the accuracy floor which the tolerance sweep isolates. Relaxing the floor here buys progressively cheaper classifier tiers along a measured frontier. Adaptive selection among comparably useful tools is a third mechanism that these experiments do not exercise, because on both corpora the selected subset was the core itself. With that separation in place, five findings emerge. A cost-driven tool-invocation policy eliminates the tools carrying class-discriminative signal, losing 94.65 percent of macro-F1 on TON_IoT and 75.75 percent on Bot-IoT, with binary detection falling below a constant prediction in both cases. The constrained formulation recovers essentially all of the achievable accuracy at 3.08 times lower cost on TON_IoT and 6.98 times lower on Bot-IoT, with the difference from exhaustive invocation statistically equivalent under a pre-specified margin on both corpora. The saving arises from classifier capacity selection rather than auxiliary tool pruning, since auxiliary detectors are three orders of magnitude cheaper than classifiers and pruning them saves almost nothing. The accuracy floor behaves as a tunable control parameter tracing an explicit frontier, whose operational limit is set by the rarest class rather than by the aggregate metric. And indirect prompt injection produced no measurable effect on either corpus, which we report as a null result rather than as defensive success.

5. Discussion

5.1. Cost Objectives Require Explicit Accuracy Constraints

The central lesson generalizes well beyond our particular architecture. A policy optimizing an objective that references only cost will eliminate whichever components are most expensive, and in a diagnostic pipeline the most expensive components are frequently those carrying the most information. The policy in our unconstrained experiment behaved correctly with respect to the objective it was given, and the fault lay in an objective that omitted the very quantity the system exists to produce.

Any deployment granting an agent discretion over its own tool invocations should therefore encode an accuracy floor structurally rather than relying on the optimizer to preserve a capability it is not rewarded for preserving. The distinction between structural enforcement and penalty-based discouragement is not a detail. A penalty term can always be outweighed by a sufficiently large cost saving and the savings available from skipping the most expensive tool are precisely the largest on offer. Removing those tools from the action space entirely, as we do, makes the failure unreachable rather than merely unattractive.

We emphasize that the mandatory core must be determined by measurement. In earlier experiments on a different corpus, one group’s classifier exhibited a negative marginal contribution, so forcing it into the core on the intuition that classifiers are always valuable would have cost both accuracy and compute. The leave-one-out procedure of Section 3.7 is inexpensive relative to the cost of getting this wrong, provided it is applied correctly, validating each core member against the core rather than against the full registry and within its own group rather than across the whole validation set. Both errors are easy to make and both silently destroy the core.

5.2. The Accuracy Floor Is a Control Parameter with a Rare-Class Limit

Our most practically useful result is that the tolerance ε traces an explicit and measurable cost-accuracy frontier. At ε = 0.02 the system runs at 3.08 times lower cost than exhaustive invocation for a 0.077-percent accuracy difference. At ε = 0.20 it runs at 7.16 times lower cost for a 1.98-percent difference. An operator can select a point on this curve according to their alert volume and their tolerance for missed attributions, and the selection is defensible because the curve is measured rather than argued.

The limit on that relaxation is the finding we did not anticipate and consider the most transferable. Macro-averaging distributes a catastrophic loss on one rare class across all classes, so the aggregate metric understates the damage by roughly the number of classes. On our corpus, moving from ε = 0.02 to ε = 0.05 appears to cost 1.45 percent of macro-F1 while actually costing the man-in-the-middle family ten percent of its F1. In an operational setting a ten-percent degradation on the rarest attack family is likely to matter more than a one-percent degradation on the aggregate, because rare families are frequently the ones an analyst most needs correctly attributed. We therefore recommend that any capacity-reduction decision be validated against per-class scores for the rarest classes specifically and not against a macro-average alone.

The non-monotonicity between ε = 0.005 and ε = 0.02 makes a related point. The tighter tolerance buys the wider DNS classifier because it scored better on validation, and that model scored slightly worse on test. A relative accuracy floor with a non-trivial tolerance absorbs exactly this kind of estimation noise; a floor tight enough to chase every validation point does not and inherits the variance of the validation estimate along with its signal.

5.3. Cost Must Be Measured, Not Asserted

Our earlier cost table consisted of hand-written constants, and the ratio between them determined our headline efficiency claim. When we replaced those constants with measurements, three things changed. The ratio between the cheapest and most expensive tool grew by three orders of magnitude, because rule checks execute in tens of nanoseconds while classifiers execute in tens of microseconds. The identity of the saving changed, and pruning auxiliary tools, which the assumed costs had made appear worthwhile, turned out to save almost nothing; essentially all of the available saving came from choosing classifier capacity, and the headline ratio itself changed substantially.

The measurement procedure also required care that we did not initially apply. Timing single invocations conflates the marginal compute of the tool with the fixed per-call overhead of the surrounding framework, and on our hardware that overhead is roughly 3.6 ms, which is two orders of magnitude larger than the marginal compute of even the most expensive classifier. A single-call measurement therefore returns nearly identical figures for a sixty-tree and a three-hundred-tree model, which flattens exactly the distinction the tiered registry depends upon. Separating the two, reporting the overhead alongside and basing the cost unit on marginal compute resolves this. We consider the practice of asserting rather than measuring invocation costs a methodological defect in the same family as omitting a majority-class baseline where both make a reported quantity unverifiable.

5.4. Aggregate Metrics Can Conceal Total Capability Loss

In our original protocol, binary detection F1 remained near 0.93 while multiclass macro-F1 fell by a factor of eight. The binary figure was not merely uninformative but actively misleading, because it exceeded the score of a deliberately weakened baseline we had included as a sanity check. A system that appears to beat its own negative control should prompt investigation.

The diagnostic that resolved the question was elementary since computing the F1 of a constant “attack” prediction on the evaluation partition obtained 0.9292, with three conditions reporting that value to sixteen decimal places. Identical floating-point output across conditions that invoke different tools is not evidence of consistent performance but of identical prediction vectors, and in retrospect this was the clearest signal available. We recommend that the majority-class score be reported adjacent to every binary detection result in this literature, a recommendation consistent with the methodological arguments of [7,8] and with the metric analyses of [30,33]. The cost of this practice is one number per table.

5.5. Limitations

Six limitations bound our claims, stated in the order of how much they constrain the conclusions. First, the evidence fusion evaluated here is deterministic rather than language-model-mediated. Token cost is therefore zero by construction and our cost figures characterize the tool layer alone. We do not claim to have measured the efficiency of an agentic pipeline in the full sense, and a token-inclusive evaluation may alter the relative ordering of conditions, since configurations invoking fewer tools may require more reasoning to compensate. The cost accounting infrastructure is in place and the decomposition is reported precisely so that this evaluation can be conducted and compared directly.

Second, our injection result is null on both corpora and we have therefore established nothing about defensive efficacy. The explanation we find plausible, that strong and mutually corroborating classifier evidence renders a single compromised agent unable to move a trust-weighted consensus, requires a matched weak-evidence configuration to test. Constructing one, for instance from a feature-poor telemetry corpus, is the natural next experiment. Third, our injection payloads are static. An adaptive adversary with knowledge of the filter would very likely fare better, and our held-out paraphrase family, while a genuine step beyond evaluating only the payloads the filter was written for, is not an adaptive attack. Fourth, the accuracy-floor frontier is demonstrated on one corpus. A third corpus would strengthen the frontier result only if it has headroom. Bot-IoT does not: with a ceiling of 0.999, every capacity tier is adequate, so the floor has nothing to choose between and a further saturated corpus would extend the replication without advancing the frontier. What is needed is a corpus difficult enough that capacity matters with several classes of unequal frequency, since our results indicate that the rarest class rather than the average determines how far the tolerance can be relaxed. Cross-corpus transfer in this field is known to be poor [26,27], and we make no transfer claim. Fifth, the constrained policy converges to its own mandatory core on both corpora, so the selection mechanism beyond core determination is untested here. A registry containing several comparably informative and comparably priced tools would exercise it more genuinely than ours does. Sixth, we do not evaluate concept drift. A deployed detector faces traffic whose distribution shifts, and both the mandatory core and the tier assignment that defines it would require periodic computations [62,63]. The procedure is cheap enough to run on a schedule, but we have not measured how quickly the estimates go stale.

5.6. Deployment Considerations

Two practical questions follow from these results, and we can answer one of them from the measurements. On latency, the constrained policy’s 1120 normalized cost units corresponds to roughly 73 microseconds of marginal compute per incident, to which the fixed per-invocation overhead of 3.64 milliseconds must be added. The overhead dominates by a factor of about fifty, which means the practical gain from capacity selection in an online setting is throughput under batching rather than single incident latency. An operator processing incidents one at a time would see little benefit, and the one batching a few hundred would see most of it.

On drift, we can only frame the question. Both the mandatory core and the tier assignment are derived from validation data, so a distribution shift that is large enough to change which tools carry signal would require re-computation. The procedure is inexpensive, taking minutes rather than hours, so scheduling it is not an obstacle. What we cannot say from these data is how quickly the estimates go stale, because we evaluated no temporal split. That measurement on a corpus with reliable timestamps is the natural companion to the work reported here.

6. Conclusions and Future Work

We set out to build a cost-efficient agentic intrusion-detection system for IoT networks and found, first, a failure mode worth reporting on its own terms. A tool-invocation policy trained against cost alone learns to skip the only tools carrying class discriminative signal, losing 94.65 percent of macro F1 on TON_IOT and 75.75 percent on Bot-IoT, while binary detection falls below the score of a constant prediction in both cases. The failure is invisible to the aggregate metric most commonly reported in this literature, and we surfaced it only by computing the majority class baseline explicitly.

The constrained formulation we propose places signal bearing tools outside the policy’s action space and minimizes measured invocation cost subject to an explicit relative accuracy floor with classifier capacity tiered so that the floor selects how much model to buy for each traffic group. On TON_IoT it attains a macro-F1 of 0.9542 against a measured ceiling of 0.9578 at 1120 normalized cost units per incident versus 3451 for exhaustive invocation, a difference of 0.077 percent that is statistically equivalent under a pre-specified margin of 0.005 macro-F1. On Bot-IoT it is 6.98 times cheaper while scoring marginally higher than exhaustive invocation. A core-removal experiment reproduces the original collapse on both corpora, which shows that the constraint and not some other part of the redesign, is what makes the recovery possible.

Three secondary results appear to us to have wider application. The tolerance parameter traces an explicit cost–accuracy frontier from 2.25 to 7.16 times cost reduction, which lets an operator choose a defensible point rather than accept a designer’s default. That frontier has an operational limit set by the rarest class, which loses ten percent of its F1 at a tolerance where the macro-average loses one and a half percent, so capacity decisions must be validated per class rather than in aggregate. Furthermore, replacing assumed invocation costs with measured ones changed both the identity of the saving and its magnitude, which persuades us that asserted costs should be treated as a methodological defect rather than a convenience.

We close on what these results do not settle. The pipeline evaluated here fuses evidence deterministically, so the cost figures characterize the tool layer and not a language-model-mediated loop. The frontier rests on one corpus, since the second is saturated. The injection evaluation is null and establishes nothing about defense. And we have not measured how quickly the core or the tier assignment goes stale under drift, which is the question a deployment would ask first. What we do claim is narrower and, we think, durable. Where a policy is permitted to decline tools on cost grounds and the informative tools are the expensive ones, cost minimization will remove capability and an accuracy floor enforced structurally rather than by penalty is what prevents it.