Abstract
Downsampling an imbalanced training set to a fixed data budget requires deciding how many observations each class retains. Under aggressive compression, even Square-Root allocation can leave rare classes with only a few training observations. We introduce PAIR (Pair-Capacity Adaptive Intervention and Repair), a conditional support-repair operator that keeps capacity-aware Square-Root allocation as the parent solution. PAIR raises a structural-tail quota only when it falls below a K-dependent floor, optionally strengthened by a training-only representation criterion, and then restores the exact budget by minimum-divergence repair; otherwise, it returns the Square-Root quotas and sampled rows exactly. On NF-ToN-IoT, UNSW-NB15, and CICIDS2017 with budgets of 1000–5000, PAIR is active in four of nine settings and identical to Square-Root in the other five. Held-out Macro-F1 rises from 0.5227 to 0.5466 on NF-ToN-IoT at , where fixed Power is higher (0.5519), and from 0.6363, 0.6788, and 0.7626 to 0.6714, 0.7726, and 0.7945 on CICIDS2017, the highest of seven compared methods. PAIR exceeds Square-Root in 39 of 40 active repeated development-split comparisons, and a synthetic study confirms the closed-form activation condition. The CICIDS2017 gains concentrate in classes with two to seven held-out observations, while Accuracy and Weighted-F1 change by less than 0.002, indicating rare-event recovery.
Keywords:
fixed-budget sampling; undersampling; square-root sampling; class imbalance; long-tailed learning; network intrusion detection MSC:
68T05; 68T10; 68T09
1. Introduction
Large network-intrusion benchmarks provide broad coverage of benign and malicious traffic, but their scale can make repeated preprocessing, model development, and comparative evaluation expensive. A fixed retained-data budget arises in practice when many candidate models must be trained and compared on the same benchmark, when training must fit within limited compute or memory, or when storing and labeling training examples is costly [1]. In each case, the size of the training subset is set in advance, and the remaining design decision is how to spend it. When the retained training set is constrained to a fixed size, data reduction becomes more than a row-selection problem: the available budget must also be distributed across classes. In highly imbalanced multiclass data, this allocation can determine whether rare attack categories retain enough genuine observations to remain represented during training.
Distribution-preserving downsampling and training-budget allocation address different objectives. Martinez-Fuentes et al. proposed proportional stratified random downsampling for NF-ToN-IoT V1 within a reproducible preprocessing framework [2]. Their procedure reduced approximately 1.38 million flows to about 50,000 while closely preserving the source class distribution and statistical properties. As expected from a proportional objective, Ransomware was reduced from 142 observations to 5 and MITM from 1295 to 46. The resulting subset is highly faithful to the original distribution, but the same fidelity also preserves extreme scarcity in the smallest classes.
This distinction becomes consequential when the reduced subset is used directly for multiclass learning. Under a strict budget, assigning additional observations to one class necessarily removes budget from another. Weiss and Provost showed that, when training data are costly, the class distribution of a fixed-size training sample can materially affect classification performance [1]. The relevant question is therefore not only which observations to retain, but also how much of the available budget each class should receive.
Common count-based rules span a continuum from distribution preservation to class equalization. Proportional allocation follows the observed class frequencies, whereas balanced allocation gives similar quotas to classes whenever capacities permit. Square-root allocation provides an intermediate, sublinear rule that reduces head-class dominance without imposing complete balance and has been studied explicitly in long-tailed recognition [3]. This intermediate behavior makes Square-Root a useful parent allocation for fixed-budget reduction. We call Square-Root the parent because PAIR returns it unchanged unless a repair is triggered and otherwise stays as close to it as possible.
The limitation considered in this work arises under severe compression. A Square-Root parent can still assign only a few observations to an ultra-rare class, even though the overall allocation is substantially less head-heavy than proportional sampling. A more tail-heavy global Power rule can increase such support, but it simultaneously changes quotas across the full class distribution. This motivates a narrower design problem: retain the Square-Root allocation as the default solution and introduce a repair only when a structural-tail quota enters a predefined low-support regime.
Recent data-pruning and coreset studies similarly emphasize that aggressive reduction can expose class-dependent failure modes. Coverage-centric Coreset Selection addresses loss of data coverage at high pruning rates [4], while class-proportional coreset selection allocates the selection budget across classes in proportion to class size, with a per-class minimum, and applies a difficulty-based selector within each class [5]. Multiclass resampling methods also incorporate local structure and class relations [6]. Class awareness, minimum retention, and structure-aware selection are therefore established ingredients. PAIR addresses a different problem: conditional repair of an already specified parent allocation that is left unchanged when no repair is needed.
We propose PAIR (Pair-Capacity Adaptive Intervention and Repair), a conditional support-repair operator built around capacity-aware Square-Root allocation. PAIR identifies a structural tail from the training supports and evaluates whether each tail quota falls below a deterministic K-dependent support floor. For an active class, a secondary representation-derived quantity can strengthen the lower bound. The resulting constraints are enforced by a minimum-divergence repair that restores the exact global budget while remaining close to the Square-Root reference. If no class is active, PAIR returns the Square-Root quota vector and, under paired class-wise prefix sampling, the same retained observations.
The activation rule is used as a structural eligibility criterion for allocation repair; predictive adequacy is evaluated empirically rather than encoded in the trigger. Likewise, the representation refinement is computed entirely from training data and does not use classifier loss, validation scores, or held-out performance. PAIR therefore separates the allocation decision from the downstream learner while allowing the predictive effect of that allocation to remain learner-dependent.
The main contributions are as follows:
- We formulate PAIR as a conditional fixed-budget operator that preserves a capacity-aware Square-Root parent exactly outside a defined low-support regime.
- We combine a K-dependent support floor with a secondary training-only representation refinement and enforce the resulting active bounds through a budget-preserving minimum-divergence repair.
- We establish operator-level properties relevant to implementation and reproducibility, including feasibility conditions, deterministic integer realization, explicit activation algebra, exact budget conservation, and exact quota/sample identity in the inactive state.
- We evaluate PAIR against Proportional, fixed Power, Square-Root, K-Center, class-proportional coreset selection (CCS-CP), and a fixed-budget adaptation (SOUP-Q) of similarity-based oversampling and undersampling preprocessing (SOUP) on common held-out observations, and we test the parent-relative differences with paired tests over repeated development splits. We complement this comparison with trigger sensitivity, component ablation, rare-class reliability, cross-learner analyses, and a controlled synthetic study of the activation boundary (Section 7.5).
The remainder of the paper is organized as follows. Section 2 reviews fixed-budget allocation, distribution-preserving reduction, coreset selection, and multiclass imbalance-aware sampling. Section 3 defines the allocation setting and Square-Root parent. Section 4 presents PAIR, and Section 5 gives its analytical properties. Section 6 and Section 7 describe the experimental design and results, followed by the discussion and conclusions.
2. Related Work
The literature review is targeted to the design choices that define the present problem: distribution-preserving reduction, fixed-budget class allocation, aggressive data pruning and coreset selection, multiclass imbalance-aware resampling, and data-dependent control of minority sample quantity. The aim is to locate PAIR within these lines of work; the review is not a systematic survey.
2.1. Distribution-Preserving Reduction and Imbalanced Intrusion Detection
Dataset reduction in network security serves at least two distinct purposes. A preprocessing pipeline may seek a smaller dataset that remains representative of the source distribution, or it may seek a compact training subset optimized for learning under a resource constraint. The first objective naturally favors preservation of class proportions; the second can place greater weight on absolute class support.
Martinez-Fuentes et al. pursued the first objective for IoT cybersecurity benchmarks by applying proportional stratified downsampling to NF-ToN-IoT V1 [2]. Their method preserves relative class frequencies and statistical characteristics while substantially reducing dataset size. For extreme minority classes, proportionality also preserves scarcity, which motivates considering class-budget allocation separately when the reduced data are used for model training.
Imbalanced intrusion-detection research addresses the learning consequences of unequal class frequencies through data-level, algorithm-level, and hybrid strategies [7,8]. Dynamic undersampling, synthetic generation, cost-sensitive learning, and ensemble methods modify either the sample composition or the learning objective. DUEN, for example, incorporates classification hardness into iterative undersampling within boosting [9], while hybrid resampling and ensemble approaches have been studied for multiclass intrusion detection [10]. Recent intrusion-detection studies continue in both directions. MCH-Ensemble trains decision-tree, XGBoost, and LightGBM base learners on balanced, randomly undersampled subsets and combines them in a random-forest meta-model, evaluated on UNSW-NB15, CIC-IDS2017, and WSN-DS [11]. Hybrid pipelines combining ADASYN/SMOTE, edited nearest neighbors, and class weighting have been applied to IoT intrusion data such as IoT-23 and NF-BoT-IoT-v2 [12]. These methods rebalance or synthesize training data but neither fix the total training budget nor guarantee a minimum support for ultra-rare classes. PAIR operates one stage earlier: it determines class quotas for a fixed number of genuine training observations before the downstream learner is fitted.
2.2. Fixed-Budget Class Allocation and Long-Tailed Sampling
The allocation of a limited training sample across classes has long been recognized as a distinct design variable. Weiss and Provost showed that the class distribution of a fixed-size training set can influence classifier performance when training examples are costly [1]. Under this constraint, class allocation is inherently coupled because any increase in one quota must be offset elsewhere.
Long-tailed learning provides several global allocation rules for this trade-off. Natural-frequency sampling preserves the empirical distribution, class-balanced sampling emphasizes tail classes, and sublinear rules occupy intermediate positions. Kang et al. evaluated square-root sampling together with instance-balanced, class-balanced, and progressively balanced strategies [3]; related work has also examined frequency-based reweighting and effective-sample corrections [13] and the effect of resampling and threshold adjustment under class imbalance [14]. Sampling- and reweighting-based rebalancing remain active topics. Lin and Yuan combine information-theoretic representation learning with an information-preservable sampler that draws balanced, informative subsets to correct majority-biased decision boundaries [15]. PHASE is a dynamic reweighting framework that combines class-level weight allocation with sample-level difficulty [16]. In the present study, Square-Root is used as the parent because it moderates head dominance while retaining part of the original frequency structure; PAIR keeps this simple frequency-based rule and makes any modification conditional on a detected structural tail.
Power allocations provide a direct global alternative. Changing the exponent continuously redistributes budget between head and tail classes and can be advantageous in particular regimes. PAIR addresses a different decision: whether an existing Square-Root quota should be repaired locally while the global exponent stays fixed.
2.3. Coreset Selection and Class-Aware Data Pruning
Coreset selection approaches fixed-budget reduction at the sample-selection level. Geometry-based K-Center methods seek broad feature-space coverage and therefore provide a useful label-blind reference for representative subset construction [17,18]. Coverage-centric Coreset Selection (CCS) was developed for high pruning rates, where importance-only selection can lose broad data coverage [4]. Its formulation illustrates a recurring theme in aggressive reduction: methods that work well at modest pruning can exhibit different behavior when the retained set becomes very small. Instance-level selectors such as InfoMax optimize sample informativeness and redundancy [19], but do not explicitly control per-class counts, which can leave rare classes with very few observations at small budgets.
Class-aware coreset work brings the allocation decision closer to the setting studied here. Tsai et al. showed that data-difficulty distributions can separate by class, including on network-security benchmarks, and introduced class-proportional variants of multiple coreset strategies [5]. Their framework allocates the selection budget across classes in proportion to class size, subject to a per-class minimum, and then applies a difficulty-based selector within each class; the class-level difficulty differences motivate this within-class selection but do not set the class budgets. DRoP shows that existing pruning methods can yield highly class-biased models and that class-wise pruning ratios combined with random within-class selection improve worst-class accuracy [20]; NUCS assigns class-specific budgets according to category difficulty [21], and recent work on 3D dataset pruning uses per-class retention quotas to preserve long-tail coverage and to tune the trade-off between overall and mean class accuracy [22]. These studies support the premise that, under severe reduction, the per-class count is itself a primary design decision. PAIR shares the concern that aggressive reduction can underrepresent difficult or rare classes, but differs in its parent-preserving formulation: the Square-Root allocation remains unchanged unless a predefined structural support condition becomes active.
2.4. Multiclass Relations and Sample-Quantity Adaptation
Multiclass imbalance is shaped not only by frequency ratios but also by overlap, local density, and relations among classes [23]. SOUP, for example, combines neighborhood information with inter-class similarity to guide oversampling and undersampling [6]. More recently, NDESO moves noisy observations toward their class centroids, guided by their distances to neighbors, before multiclass oversampling [24]. A comparative review of multiclass resampling and ensemble methods found that not all of them reliably improve performance and that some perform worse than the baseline [25], which motivates conservative, condition-dependent intervention. Pairwise and error-correcting multiclass formulations likewise emphasize that each class is learned in the presence of multiple competitors [26,27]. PAIR does not adopt these prediction architectures or pairwise similarity scores; the number of classes enters only through the structural support trigger used for allocation eligibility.
Other work adapts the amount of minority sampling to data characteristics. Lee and Kim determine oversampling size using classification-complexity measures [28], illustrating that minority support can be treated as a data-dependent quantity. PAIR extends this general principle to a strict fixed-budget setting: a class-specific structural refinement can strengthen an active lower bound, after which the remaining budget is reconciled against the Square-Root parent.
2.5. Position of the Present Work
The preceding methods address complementary decisions in constrained data reduction. Proportional sampling prioritizes distributional fidelity; Square-Root and Power rules define global class allocations; K-Center and CCS emphasize representative sample selection; class-proportional coresets combine class allocation with within-class selection; and SOUP uses multiclass structure during hybrid resampling. PAIR is positioned as a conditional operator on top of an existing allocation: it modifies a structural-tail quota only in its active support regime and otherwise retains the parent exactly. This regime-specific formulation is consistent with the broader observation that imbalance treatments need not be uniformly optimal across data conditions [29].
The active repair is implemented with generalized Kullback–Leibler (KL) divergence and bounded projection, drawing on established Bregman and I-divergence projection theory [30,31]. These tools provide the optimization mechanism for maintaining proximity to the Square-Root reference under new lower bounds. The methodological contribution lies in the conditional composition of parent allocation, structural activation, support-floor refinement, budget-preserving repair, and exact inactive identity.
Table 1 summarizes this distinction. PAIR does not introduce class awareness, minimum retention, or structure-aware sampling as standalone concepts; it uses them within a parent-preserving fixed-budget operator whose defining decision is whether the Square-Root allocation should be repaired at all.
Table 1.
Positioning of PAIR relative to representative data-reduction and multiclass-resampling strategies. The table describes the primary construction decision of each method; it does not compare predictive performance.
3. Problem Formulation
Let the training pool be
where and K is the number of classes. Let denote the number of available training observations from class c, so that .
We consider a strict retained-data budget B, where . A class-allocation rule produces an integer quota vector satisfying
The feasible set is therefore
The fixed-budget constraint creates a direct trade-off: increasing the retained support of one class necessarily reduces the budget available to other classes.
Square-Root is used as the parent allocation. For budget B, let the continuous capacity-aware target be
where is chosen so that
The capacity term prevents a class from receiving more observations than exist in the source pool. When no class saturates, Equation (4) reduces to
The continuous targets are converted by deterministic capacity-aware apportionment into .
PAIR does not construct a second global distribution from scratch. Let denote a structural-tail set obtained from the training supports, and let denote the subset for which an intervention is active at a particular budget. Their construction is part of PAIR and is introduced in Section 4. For each active class, a method-specific lower bound may be imposed:
The value is a training-derived structural constraint that defines the minimum support enforced in an active class (see Remark 2).
The resulting conditional problem is to find that satisfies the active lower bounds while deviating minimally from the Square-Root parent. The repair is local in the sense that new constraints are introduced only for active tail classes; budget conservation necessarily induces compensating changes in donor classes, that is, classes whose quotas decrease to fund the repair.
If no intervention is active, the desired solution is
Equation (8) defines the inactive state of the operator and provides the reference against which active repairs are measured.
The construction additionally requires exact budget conservation after integerization, compliance with genuine class capacities, and training-only computation of every activation quantity. The downstream learner is fitted after the allocation has been fixed. PAIR is therefore classifier-agnostic at the construction stage, while the predictive response to a repaired subset can still depend on the learner.
4. Proposed PAIR Method
PAIR operates as a conditional transformation of the Square-Root allocation defined in Section 3. A structural-tail class enters the repair only when its Square-Root quota falls below the K-dependent support threshold. For an active class, the lower bound may be strengthened by a training-only representation estimate, the robust random-prefix (RRP) floor . The resulting bounds are then reconciled with the fixed budget by a KL repair around the Square-Root reference (Section 4.3). The representation estimate acts as a secondary refinement of an already active constraint.
Figure 1 summarizes the flow. The common split is created first, and all structural quantities are computed from training data. If no class is active, PAIR returns the Square-Root parent unchanged. If a class is active, PAIR applies the active support floor and performs the budget-preserving repair. The test partition is not used in any allocation decision.
Figure 1.
PAIR workflow. Training data determine the Square-Root parent, the structural tail, and the active support floor. An inactive case returns the Square-Root allocation unchanged. In an active case, a secondary training-only representation estimate may strengthen the K-dependent support floor before the budget-preserving KL repair. Test observations are used only for evaluation.
4.1. Structural Tail and Secondary Representation Refinement
Let be the permutation of the class labels that sorts the training supports in ascending order, with ties broken deterministically by class label, and write . Thus, the parenthesized index denotes the original class label that occupies rank position j in this order, whereas the plain index j denotes the rank position itself, so that . For each adjacent pair of rank positions, define the multiplicative support gap
The boundary is
and the structural tail is
Here and are indexed by rank position: is the position of the boundary, and is the structural gap between the largest tail class and the smallest non-tail class. The tail is a set of original class labels, and all class-indexed quantities that follow, such as and for , use the original label c. The structural tail is therefore the low-support prefix separated by the largest adjacent multiplicative gap. If several boundaries attain the same maximum gap, the smallest tail prefix is selected. This definition identifies a support regime from the training distribution.
PAIR also computes a secondary representation-derived floor for each . This refinement uses equal-feature Gower dissimilarity [32], a deterministic greedy facility-location sequence, and up to five random-prefix folds that are cross-fitted on training data only (cross-fitted folds). Within each fold, the operating point balances normalized coverage gain against retained support; the final RRP value is the maximum fold-local requirement capped by the corresponding point-estimate envelope,
Appendix A gives the loss definitions, random-prefix expectation, preprocessing conventions, and edge cases. RRP is computed entirely from training-set geometry and can only strengthen a class for which the K-dependent trigger is already active.
4.2. K-Dependent Support Trigger and Active Floor
PAIR uses a deterministic support trigger for structural-tail classes. For , define the K-dependent minimum support
Operationally, PAIR activates when a structural-tail Square-Root quota falls below this floor. Because is the smallest integer satisfying the pair-count reference, the two forms are equivalent:
The name PAIR refers to this pair-capacity criterion, which compares the number of unordered within-class pairs, , with the competing class relations of a class; is the smallest support whose internal pairs can cover all relations. This construction provides a deterministic design convention for intervention eligibility, whose sensitivity is evaluated in Section 7.4. It depends only on the observed number of classes, increases monotonically with K, and does not use classifier loss or validation performance. Its role is to mark a structural low-support regime (see Remark 2). For , PAIR defines and leaves the trigger inactive. In the three benchmarks studied here, gives for NF-ToN-IoT and UNSW-NB15, whereas gives for CICIDS2017. The resulting floors are small at these class cardinalities and grow gradually as K increases. Section 7.4 evaluates narrower, wider, linear, and unconditional trigger variants.
The active set is therefore
For an active class, PAIR applies
For , the lower bound is the Square-Root quota itself,
Thus, the representation refinement cannot change a class whose K-dependent trigger is inactive.
For non-tail classes, only the basic survival lower bound is used:
If , PAIR returns the Square-Root parent before any repair is performed.
Remark 1
(Activation condition). PAIR intervenes when for a structural-tail class t that still has unused observations (). The condition uses the continuous, unsaturated Square-Root quota of Equation (6) as an approximation; the exact trigger is the integer condition defining the active set in Equation (15), so the two can differ by one unit near the boundary. A triggered tail class whose observations are all already used leaves the quotas unchanged (capacity-saturated cells in Section 7.5).
4.3. Fixed-Budget Repair and Sample Realization
When one or more PAIR constraints are active, the added tail mass must be compensated elsewhere. Rather than specifying a hand-coded donor ordering, PAIR uses a bounded minimum-divergence repair under generalized KL divergence around the continuous Square-Root reference, referred to below as the KL repair:
Generalized KL divergence provides a standard convex measure of deviation from the Square-Root reference. The projection therefore implements the design objective of satisfying active support constraints while preserving the parent allocation as closely as possible under the fixed budget.
Computational cost. All structural quantities of PAIR (class supports, structural tail, K-dependent floor, RRP estimate, and the repaired quota vector) depend only on the training pool and the budget. They are computed once per training pool and budget during preprocessing and are reused unchanged for every sampling seed; the seed changes only the within-class random permutation from which the fixed quotas are realized, as for Square-Root. In the recorded environment, this one-time structural setup took 1.28 s on NF-ToN-IoT and less than 0.01 s on UNSW-NB15 and CICIDS2017, because the representation refinement operates on within-class distance matrices of the tail classes (MITM has 829 development-training observations, whereas the UNSW-NB15 and CICIDS2017 tail classes have 104 and 7–23); this is negligible relative to loading the data and fitting the downstream learner (Appendix B).
Structural lower bounds are applied only when they are jointly feasible. If the full structural bounds exceed the target budget, PAIR replaces them with basic one-observation survival bounds for nonempty classes. If these survival bounds are feasible, the same bounded repair is solved under the weaker bounds. If B is smaller than the number of nonempty classes, universal class survival is mathematically infeasible; the implementation records this regime explicitly and returns the Square-Root parent without claiming a survival guarantee. No observations are fabricated, and the global budget is never exceeded.
The continuous solution is converted to an integer quota vector by deterministic capacity-aware integerization. Residual units are assigned by largest fractional remainder; remainder ties are resolved using the continuous projected quota and then a fixed class-label ordering. The resulting vector satisfies the applicable lower bounds, genuine class capacities, and the exact global budget. Square-Root and Power use the same deterministic apportionment convention, so integer ties do not add sampling randomness.
The intervention mass is
Since both vectors sum to B, D is the total number of quota units transferred from donor classes to recipient classes.
PAIR changes class quantities only; it does not add a quality-based row selector. For each seed, one random permutation is generated within each class. Square-Root and PAIR both use prefixes of these same class-wise permutations. Therefore, identical quotas imply identical retained rows. If the trigger is inactive for every class, the complete PAIR training subset is the Square-Root subset, not an independent random realization.
Algorithm 1 summarizes the complete procedure.
| Algorithm 1 PAIR fixed-budget sampling |
|
5. Mathematical Properties of the PAIR Operator
The analysis below characterizes PAIR as a mapping from training supports and a fixed budget to an integer quota vector. Standard generalized-KL projection, Karush–Kuhn–Tucker (KKT) conditions, and scalar root finding [30,31] are specialized to establish feasibility, the form of the active repair, deterministic integer realization, the K-dependent activation threshold, and exact identity with Square-Root in the inactive state.
Let , let be the applied lower bound for , and let be the continuous Square-Root reference. Define
Theorem 1
(Feasible active repair and clipped form). The active repair is feasible if and only if
When feasible, the generalized-KL objective has a unique primal minimizer, and that minimizer has the standard clipped form
for a scale chosen so that .
Proof.
The feasibility condition follows by summing the coordinate bounds; sufficiency follows by convex interpolation between the lower- and upper-bound vectors. The generalized-KL objective is strictly convex on positive coordinates, so the primal minimizer is unique. Standard KKT conditions give a common multiplicative scale on interior coordinates and clipping at active bounds, which yields Equation (23). The scalar budget equation is monotone and is solved by standard bracketing and bisection. □
Theorem 1 specializes established I-divergence projection theory to the PAIR constraint set and shows that the active repair is well posed whenever the lower bounds are jointly feasible.
Proposition 1
(Integer realization). Let be a feasible continuous repair with integer lower and upper bounds and integer budget B. Bound-aware largest-remainder apportionment returns an integer quota vector satisfying
with for every class.
Proof.
Flooring each coordinate respects integer bounds. The remaining integer residual equals the sum of fractional parts and is therefore no larger than the number of positive fractional remainders. Assigning one unit to the largest admissible remainders reaches the exact budget without crossing an upper bound. □
Proposition 2
(K-dependent support threshold). For , the smallest integer satisfying is
The threshold is nondecreasing in K and changes only when crosses a triangular number.
Proof.
Solving for its positive root and taking the least admissible integer gives Equation (25). Monotonicity and the triangular transition points follow directly from the defining inequality. □
Theorem 2
(Inactive-case identity). If the active set in Equation (15) is empty, then
Under the paired class-wise random-prefix realization used in this study,
Thus, an inactive cell is identical to Square-Root at both the quota and sample levels.
Proof.
PAIR bypasses the repair when and copies the Square-Root integer quota vector. Both methods then take equal-length prefixes from the same seed-specific class-wise permutations, so the retained rows are identical class by class. □
Because both quota vectors have total budget B, any positive intervention mass is balanced by an equal reduction elsewhere. This follows from the budget identity alone.
Remark 2
(Interpretation of activation). Equation (14) defines structural eligibility for support repair. It does not guarantee that the repaired support is sufficient for prediction: the trigger, the K-dependent floor, and the RRP refinement use no classifier loss, validation score, or held-out performance. Predictive adequacy is therefore assessed empirically through the trigger-sensitivity, learner, and held-out analyses, and Appendix B reports a controlled case in which an active repair lowers target-class F1.
6. Experimental Methodology
6.1. Evaluation Logic
The primary development comparison evaluates PAIR against Proportional sampling, a fixed global Power allocation, and its direct Square-Root parent. The held-out common-test comparison additionally includes K-Center, class-proportional coreset selection (CCS-CP), and the fixed-budget SOUP-derived adaptation (SOUP-Q), while leaving PAIR unchanged. The remaining experiments examine specific aspects of the method and introduce no further competing samplers. Repeated splits assess partition sensitivity, trigger variants assess activation sensitivity, component ablations identify the role of the two structural floors, Random Forest (RF) and deep neural network (DNN) runs assess learner dependence, and class-level analyses quantify the reliability of results for ultra-rare classes.
6.2. Datasets
Three public multiclass intrusion-detection benchmarks are used: NF-ToN-IoT V1 [33], UNSW-NB15 [34], and CICIDS2017 [35]. They were selected because they contain markedly different support spectra and include classes that remain rare even in large source datasets.
NF-ToN-IoT V1 contains approximately 1.38 million flows across classes. The full source distribution ranges from Injection (468,539) and DDoS (326,345) to MITM (1295) and Ransomware (142). UNSW-NB15 also has classes in the official 175,341-record training split; Worms has only 130 training observations, compared with 56,000 Normal and 40,000 Generic records. CICIDS2017 has classes and is more extreme: among approximately 2.83 million flows, Heartbleed has 11 source observations, Web Attack–SQL Injection has 21, and Infiltration has 36, while BENIGN exceeds 2.27 million. Consequently, the K-dependent PAIR floor is 5 for NF-ToN-IoT and UNSW-NB15 and 6 for CICIDS2017 (Table 2).
Table 2.
Development partitions and dataset-specific K-dependent PAIR floors. Structural tails are derived from development-training support only.
The three primary budgets are . They correspond to aggressive retention: approximately 0.113–0.567% of the NF-ToN-IoT development-training pool, 0.713–3.565% of UNSW-NB15, and 0.055–0.276% of CICIDS2017.
6.3. Data Splits and Held-Out Evaluation
Downsampling is applied only to training data. For NF-ToN-IoT and CICIDS2017, a stratified 20% outer holdout is set aside for the final test evaluation, while the remaining data are used for development. For UNSW-NB15, the official test set is used as the held-out test set, and the official training data are used for development. Within the development data, all sampling methods are trained on the same development-training pool and evaluated on the same development-test observations. The held-out test observations are not used to define the structural tail, estimate RRP floors, choose the activation rule, or fit the downstream learner.
Because class-level estimates can be unstable for very small test supports, the support of each structural-tail class is reported explicitly (Table 3).
Table 3.
Development-test support of structural-tail classes.
6.4. Primary Comparison Methods, Budgets, and Seeds
The primary development table intentionally contains only four allocation rules:
- Proportional: , representing distribution-preserving reduction;
- Power: , a fixed globally more tail-heavy allocation;
- Square-Root: , the direct PAIR parent;
- PAIR: the proposed conditional Square-Root repair.
The main severe-compression grid is with five paired sampling seeds . Quota-based methods obey the same global budget and genuine class capacities. Balanced, K-Center, and a second Power exponent () are reported as secondary comparisons in Appendix G.
6.5. Support-Trigger Sensitivity Analysis
To assess sensitivity to the activation criterion, we compare the proposed K-dependent support trigger with ROOTK (), LINEARK−1 (), half and double perturbations of the pair threshold, and unconditional tail activation. All other parts of PAIR are kept unchanged. These variants are used only to assess the sensitivity of the activation rule; the complete results are reported in Appendix G.
6.6. Repeated Development-Split Robustness and Tail Stability
To assess sensitivity to the development partition, the Square-Root–PAIR comparison is repeated over 10 development splits. Each split uses three paired sampling seeds (42, 52, and 62) and all three budgets. Structural-tail membership, RRP floors, Square-Root quotas, activation, and PAIR quotas are recomputed from the training portion of each split.
The three paired-seed differences are averaged within each split. We report the mean and standard deviation across splits, split-level wins/ties/losses, a corrected repeated-holdout interval, and the paired tests described in Section 6.9. Because the repeated test partitions overlap, the interval and the overlap-corrected test use the variance correction of Nadeau and Bengio [36]. Structural-tail and activation frequencies are also reported.
6.7. Component Ablation and Rare-Class Reliability
The internal floor analysis uses four explicitly labeled ablation states: Square-Root, K-floor only, RRP-floor only (applied only when the trigger is active), and Full PAIR. These are not presented as additional proposed samplers. The K-floor-only ablation preserves the Square-Root parent and, when the trigger is active, enforces only . The RRP-only ablation analogously enforces . Full PAIR uses Equation (16). Five paired seeds are used.
For CICIDS2017, reliability is analyzed directly from prediction counts. Per class and seed, we record TP, FP, FN, TN, support, precision, recall, F1, and exact Clopper–Pearson 95% intervals for recall. The contribution of class c to a Macro-F1 difference is . This decomposition is descriptive and is used to identify whether an aggregate Macro-F1 gain reflects broad-distribution change or recovery of a few ultra-rare classes.
6.8. Downstream Learners
XGBoost is the primary downstream evaluator. The XGBoost configuration uses 300 trees, maximum depth 6, learning rate 0.1, subsample 0.8, column subsample 0.8, minimum child weight 5, , regularization 0, regularization 1, multiclass log-loss evaluation, and histogram tree construction. The sampling rule never reads XGBoost loss or predictions.
RF [37] and the DNN serve only as secondary learner-robustness checks. RF uses 500 trees and one common training-only configuration selected by out-of-bag (OOB) error for each paired Square-Root/PAIR cell. The DNN uses 128- and 64-unit hidden layers, batch normalization, dropout 0.30, Adam with learning rate , sparse categorical cross-entropy, batch size 512, a maximum of 20 epochs, and patience 3. No development-test or held-out test performance is used to construct PAIR quotas.
6.9. Metrics and Pairing
Macro-F1 is the primary predictive metric because each class contributes equally. Macro-Recall, Macro-Precision, Weighted-F1, Accuracy, and per-class precision/recall/F1 are also reported with fixed test support. Aggregate metrics and their differences are reported to four decimals; class-level recall and F1 values and exact interval bounds are reported to three. All reported differences are computed from unrounded means and may therefore differ in the last digit from differences of the rounded table entries. Classes with support below five are interpreted descriptively, those with support 5–9 are treated cautiously, and those with support at least 10 receive standard class-level interpretation. No class is removed from Macro-F1.
Sampling comparisons are paired by construction. For a given seed, quota-based methods share class-wise random prefixes. Therefore, an inactive Square-Root–PAIR cell has identical quotas and retained rows, not merely statistically similar samples. Five-seed summaries are reported as mean ± SD with paired wins, ties, and losses. Paired hypothesis tests are applied to the active cells of the repeated development-split analysis, where each of the 10 splits contributes one paired difference. We use an exact two-sided Wilcoxon signed-rank test, which treats the split-level differences as independent, and a corrected resampled t-test that inflates the variance to account approximately for the overlap of repeated holdout partitions (test/train ratio 0.2/0.8) [36]. Both tests are Holm-corrected over the active cells. Seed-level tests on the held-out test set are reported for completeness only. With five paired seeds, the two-sided exact Wilcoxon signed-rank test used here has a minimum attainable p-value of 0.0625, even when all five differences are positive, so the 5% threshold cannot be reached at the seed level.
6.10. Held-Out Common-Test Evaluation
The held-out evaluation uses the same fixed test observations for Proportional, Power (), Square-Root, K-Center, SOUP-Q, CCS-CP, and PAIR. XGBoost, budgets , and seeds are the same as in the primary evaluation. NF-ToN-IoT uses the reserved 275,855-row outer holdout, CICIDS2017 the reserved 566,149-row outer holdout, and UNSW-NB15 the official 82,332-row test set. Square-Root and PAIR reuse the previously evaluated parent-relative runs; the additional Proportional, Power, and K-Center models are trained only on the corresponding training subsets and evaluated on these same held-out observations. No held-out metric is used to alter PAIR, choose the Power exponent, configure K-Center, or tune CCS-CP.
As a class- and structure-aware comparator from the multiclass resampling literature, we additionally evaluate a fixed-budget SOUP-derived adaptation (SOUP-Q) of SOUP [6]. SOUP equalizes class sizes by removing unsafe majority observations and duplicating safe minority observations, where the safety level of an observation is the mean, over its k nearest neighbors, of for its own class c and the neighbor’s class . SOUP-Q departs from the original procedure in two respects. First, the original procedure determines its own resampled class size and permits repeated observations, whereas our protocol prescribes a budget of B distinct source observations; SOUP-Q therefore uses SOUP’s common per-class target as capacity-capped equal quotas without oversampling, with chosen so that (largest-remainder integerization); this quota vector is identical to capacity-capped Balanced allocation, the development Balanced baseline of Appendix G. Second, the reference implementation [38] recomputes safety levels while classes are processed sequentially, on the progressively modified dataset, whereas SOUP-Q computes them once, with exact nearest neighbors (the default of the reference implementation) on the original standardized training pool. SOUP’s distinct contribution in SOUP-Q is therefore its within-class selection. In each class, we retain the observations with the highest safety level and break ties with a seed-specific random order. SOUP-Q is trained and evaluated in the same way as the other methods on the same held-out observations; the software versions of these runs and a same-environment refit check are documented in Appendix H.
We also evaluate class-proportional coreset selection (CCS-CP) [5], the class-aware coreset method that allocates a fixed budget of distinct observations across classes before selecting within each class. Each class receives observations; if these budgets exceed B, they are reduced one unit at a time, starting from the largest classes, without falling below m. Within each class, CCS-CP applies Coverage-centric Coreset Selection (CCS) [4], which removes the fraction of the hardest observations according to the area under the margin (AUM) and samples the remaining observations evenly across 50 equal-width AUM strata. CCS-CP is compatible with our fixed-budget subset-selection objective. We implemented CCS-CP from its description in the original article, using the published class-budget rule and the public CCS code for the within-class selection. Before the evaluation, we fixed the implementation choices for the settings that the paper leaves open. The minimum is set to , the same K-dependent base-floor value used by PAIR (a sensitivity run with is reported in Supplementary Table S8). The class quotas and the effective lower bounds still differ, because PAIR applies its floor only to structural-tail classes in the active regime and may strengthen it through RRP, whereas CCS-CP applies its minimum to every class. At NF-ToN-IoT , for example, Ransomware receives 5 observations under CCS-CP and 10 under PAIR. AUM is accumulated over 20 epochs of a multilayer perceptron with hidden layers of 128 and 64 units. We select for each dataset and budget with the hierarchical search of the CCS-CP paper on an inner split of the training pool. During this search, the perceptron and its standardization are fitted only on the inner training partition (80%), and candidate coresets are drawn only from this partition. The frozen XGBoost trained on each candidate (sampling seed 42) is evaluated by accuracy on the stratified 20% inner validation partition, following the selection criterion of the original CCS-CP protocol. Once is fixed, we refit the standardization and the perceptron on the full training pool to compute the AUM used for the final coreset selection. The held-out observations are used in neither stage. We complete any rounding shortfall below B by largest remainder, and sampling within strata uses the same seed-specific within-class random orders as the other quota methods. CCS-CP is trained and evaluated in the same way as the other methods, in the same software environment as SOUP-Q.
The complete held-out Accuracy, Macro-F1, Weighted-F1, Macro-Recall, and Macro-Precision results for all seven methods are provided in Supplementary Table S1. Supplementary Tables S2–S4 report class-wise recall together with the fixed test support for every class in NF-ToN-IoT, UNSW-NB15, and CICIDS2017, respectively.
6.11. Leakage and Feasibility Controls
All PAIR quantities are computed from training data before the downstream learner is fitted. Structural-tail membership uses training class counts, Gower preprocessing and RRP geometry use training observations only, the activation rule uses K and the Square-Root quota, and the KL repair uses only the parent allocation, capacities, and structural lower bounds. If the structural lower bounds are infeasible under the global budget, PAIR falls back to basic class-survival bounds when those bounds are feasible; this fallback was not required in the reported experiments.
7. Results
Results are reported in two stages. Development analyses characterize the parent-relative behavior of PAIR through the primary allocation benchmark, repeated splits, trigger sensitivity, component ablation, rare-class analysis, and learner robustness. The final common-test evaluation then compares Proportional, fixed Power, Square-Root, K-Center, the fixed-budget SOUP-derived adaptation (SOUP-Q), class-proportional coreset selection (CCS-CP), and PAIR on the same held-out observations.
7.1. Primary Development Benchmark
Table 4 reports five-seed development Macro-F1 means for the four primary allocation rules. Power uses the fixed exponent and provides a global tail-emphasis control, whereas Square-Root is the parent allocation from which PAIR is derived. At NF-ToN-IoT , Power attains the highest mean, while PAIR is higher than Square-Root.
Table 4.
Five-seed development XGBoost Macro-F1 means for the primary fixed-budget benchmark. Power denotes the fixed global exponent . Bold indicates the best value in each row; exact ties are both bold.
The NF-ToN-IoT result illustrates the difference between global reallocation and conditional repair. Power changes the complete quota vector through a single exponent, whereas PAIR retains the Square-Root allocation except where an active tail constraint requires repair. The more head-heavy control and the Balanced/K-Center development baselines are reported in Appendix G.
Table 5 isolates the direct parent-relative development result. NF-ToN-IoT activates only at , while UNSW-NB15 is inactive throughout the main grid. CICIDS2017 is active at all three budgets.
Table 5.
Five-seed development Square-Root–PAIR results. W/T/L denotes PAIR wins, ties, and losses at the paired sampling-seed level.
7.2. Budget-Dependent Activation and the Inactive Case
Table 6 shows the structural intervention separately from prediction. D is the total positive quota mass transferred relative to Square-Root. An inactive cell is returned before projection and therefore has exactly the Square-Root quota vector and, under paired prefix sampling, the same retained subset. Figure 2 shows the fraction of the budget moved, , and the mean PAIR-minus-Square-Root Macro-F1 difference as functions of the budget.
Table 6.
PAIR activation in the main development budget grid.
Figure 2.
Budget-dependent development behavior under XGBoost. (a) Fraction of the fixed budget moved relative to Square-Root, . (b) Mean PAIR-minus-Square-Root Macro-F1. Predictive performance is not used by the trigger.
7.3. Repeated-Split Robustness and Structural-Tail Stability
The repeated-split experiment directly tests whether the development pattern is an artifact of one partition. Table 7 summarizes the 10 split-level differences after averaging the three paired sampling seeds within each split. The four active regimes reproduce positive mean differences across splits; NF-ToN-IoT and CICIDS2017 and 2500 win in all 10 splits, while CICIDS2017 wins in 9 of 10. In every inactive regime, the difference is zero in all 10 splits. Across the four active regimes, 39 of the 40 split-level differences are positive. Because the development splits overlap, the primary analysis uses the Nadeau–Bengio overlap correction, which corrects for their dependence only approximately: the corrected 95% intervals exclude zero in all four active cells, and the overlap-corrected t-test gives Holm-adjusted (Table 7; Supplementary Table S6). As supplementary descriptive evidence, the exact Wilcoxon test, which treats the 10 split-level differences as independent and therefore does not account for their dependence, gives for NF-ToN-IoT and CICIDS2017 and 2500, and for CICIDS2017 (Holm-adjusted in each case). Both tests give Holm-adjusted , but given the overlap, we interpret them as evidence of reproducibility across development partitions rather than as exact inference.
Table 7.
Repeated development-split robustness. Corrected intervals and corrected t-tests use the Nadeau–Bengio variance correction, which accounts only approximately for overlapping holdout partitions; the Wilcoxon test treats splits as independent. p-values are Holm-adjusted over the four active cells (full results: Supplementary Table S6).
The tail declaration is also stable across these partitions. NF-ToN-IoT selects Ransomware and MITM in 10/10 splits, UNSW-NB15 selects Worms in 10/10, and CICIDS2017 selects Heartbleed, SQL Injection, and Infiltration in 10/10. Activation is equally stable at the regime level: all CICIDS2017 budgets and NF-ToN-IoT are active in 10/10 splits, whereas both larger NF-ToN-IoT budgets and every UNSW-NB15 budget are inactive in 10/10. NF-ToN-IoT transfers only 4–6 quota units across the 10 splits; the CICIDS2017 transfer masses remain 14, 6, and 1 for , 2500, and 5000.
7.4. Support-Trigger Sensitivity
The trigger-sensitivity analysis isolates the effect of the activation rule while keeping the remaining PAIR construction fixed. Table 8 reports three representative cells; the complete seven-rule matrix is provided in Appendix G.
Table 8.
Representative five-seed trigger-sensitivity cells. ROOTK is a narrower activation rule, and ALWAYS-TAIL removes the trigger.
Narrower rules miss several active repairs: ROOTK remains inactive on NF-ToN-IoT and CICIDS2017 , and it activates fewer structural-tail classes at CICIDS2017 . At the opposite extreme, ALWAYS-TAIL moves substantially more quota mass without a corresponding improvement in mean Macro-F1; on NF-ToN-IoT , it transfers 46 units compared with four for PAIR and yields a slightly lower mean. More permissive variants often converge to the same repaired quota as PAIR. Together, these results support the trigger as a conservative structural operating point in the present benchmarks while showing that predictive response remains regime-dependent. Appendix B reports the controlled K-axis analysis, including an active case with a negative predictive response.
7.5. Controlled Synthetic Study of the Activation Boundary
To test the activation condition beyond the three benchmarks, we generated synthetic long-tailed data in which two tail classes lie below head classes at a controlled structural gap . The grid varies , K, and B over 128 cells and uses the five paired seeds, the frozen XGBoost configuration, and the full PAIR operator (design details in Appendix C). The closed-form condition of Remark 1 predicted whether the quotas change in 127 of the 128 cells; the exception is a one-unit integer-rounding case (Figure 3). In the 28 active cells, in which the PAIR quotas differ from Square-Root, the mean PAIR-minus-Square-Root Macro-F1 difference was at class separation . Averaged over the tested active cells, the difference increased with K and decreased with B, and four active cells had small negative means. At , activation was unchanged, and the mean active-cell difference was ; class separability therefore changes the size of the benefit but not the activation boundary.
Figure 3.
Controlled synthetic study (; Appendix C): closed-form activation boundary (line; shaded region: trigger predicted to fire) against observed cells. Filled markers are active cells (quotas changed) colored by mean ΔMacro-F1; diamonds are triggered cells in which the tail class is already fully used, so the quotas are unchanged.
7.6. Component Ablation and Internal Floor Roles
Table 9 reports only the four active development cells; all inactive cells are exact Square-Root ties for every ablation state. The component controls are parent-preserving: a component floor is never allowed to reduce an existing Square-Root quota.
Table 9.
Five-seed Macro-F1 component ablation in active development cells. The ablation states are diagnostic components of PAIR, not additional proposed methods.
The ablation separates the roles of the two support components in the development pool. In all three active CICIDS2017 cells, the K-dependent floor alone reproduces the Full PAIR quota vector and Macro-F1. RRP becomes uniquely binding only for NF-ToN-IoT at , where it raises the Ransomware floor beyond the K-dependent value. Its incremental effect over the floor-only ablation is small and varies across seeds, indicating that RRP functions as a secondary data-dependent refinement; the development CICIDS2017 results are driven by the K-dependent floor. In the held-out training pools, RRP is also binding for CICIDS2017 Infiltration (Section 8.2).
7.7. Tail-Class Readout and Learner Dependence
NF-ToN-IoT at is the cleaner development active case because Ransomware and MITM have 23 and 207 common-test observations, respectively. PAIR increases the Ransomware training quota from 4 to 8 while retaining the MITM quota at 12. Averaged over five seeds, Ransomware recall increases from 0.070 to 0.730 and F1 from 0.079 to 0.297; MITM F1 remains essentially unchanged (0.322 versus 0.321).
Table 10 evaluates the same active allocations under XGBoost, Random Forest, and DNN. The results distinguish classifier-agnostic subset construction from learner-dependent predictive response: the sign and magnitude of PAIR-minus-Square-Root Macro-F1 vary across downstream models.
Table 10.
Cross-learner development PAIR-minus-Square-Root Macro-F1. Positive values favor PAIR.
The largest RF reversal occurs at CICIDS2017 , where the repaired allocation raises Macro-Recall but reduces Macro-Precision enough to lower Macro-F1. Appendix E reports the corresponding training-only RF decision-rule sensitivity analysis. These results show that the allocation mechanism is independent of classifier feedback, while the realized predictive trade-off depends on the learner and decision rule.
7.8. Rare-Class Reliability on CICIDS2017
The CICIDS2017 development gains are concentrated in classes with extremely small test support. Table 11 decomposes the total Macro-F1 difference into the contribution of Heartbleed and the three structural-tail classes together. Heartbleed has only two development-test observations, SQL Injection three, and Infiltration six. The exact class-level recall intervals are therefore wide even when the observed recall change is large.
Table 11.
Development CICIDS2017 Macro-F1 decomposition. Percentages are shares of the total mean PAIR-minus-Square-Root Macro-F1 difference; values above 100% indicate that positive tail gains are partially offset by negative changes elsewhere.
The CICIDS2017 differences are therefore concentrated in rare-event recovery. Weighted-F1 and Accuracy change much less than Macro-F1: across the ten repeated development splits, the mean CICIDS2017 changes are at most 0.0004 in Accuracy and Weighted-F1, against 0.0324–0.0578 in Macro-F1 (Supplementary Table S6; held-out values: Section 8.4 and Supplementary Table S1). This is consistent with gains that arise from a small number of tail observations and leave aggregate performance on the test set nearly unchanged. All classes remain included in Macro-F1; the small supports affect the precision of the class-level estimates.
7.9. Held-Out Common-Test Results
Table 12 reports five-seed Macro-F1 means on the same held-out observations for all seven allocation methods. Accuracy, Weighted-F1, Macro-Recall, and Macro-Precision for every method–budget cell are given in Supplementary Table S1, and the class-wise recall of every class together with its fixed test support is given in Supplementary Tables S2–S4 for NF-ToN-IoT, UNSW-NB15, and CICIDS2017, respectively.
Table 12.
Held-out XGBoost Macro-F1 (mean ± SD over five seeds) on the same fixed test observations. Power uses the fixed exponent ; SOUP-Q denotes the fixed-budget SOUP-derived adaptation (Section 6.10), and CCS-CP denotes class-proportional coreset selection with (Section 6.10). Δ is PAIR minus Square-Root. CICIDS2017 Macro-F1 includes three classes with only 2, 4, and 7 held-out observations (Section 8.4), so its differences carry substantial class-level uncertainty.
The held-out comparison reproduces the development intervention map while placing PAIR beside the full baseline set. At NF-ToN-IoT , Power has the highest mean Macro-F1 (0.5519), followed closely by PAIR (0.5466), with PAIR exceeding its Square-Root parent by 0.0240. At the two larger NF-ToN-IoT budgets and all three UNSW-NB15 budgets, the PAIR trigger is inactive, and the method ties Square-Root exactly. In CICIDS2017, PAIR records the highest mean Macro-F1 among the seven methods at all three budgets, with parent-relative differences of 0.0351, 0.0938, and 0.0319.
Supplementary Table S1 provides the corresponding Accuracy, Weighted-F1, Macro-Recall, and Macro-Precision values. These measures reveal distinct trade-offs: Power can produce high Macro-Recall in CICIDS2017 while remaining lower in Macro-Precision and Macro-F1, whereas Proportional and K-Center can retain comparatively strong Accuracy in some cells without recovering the rarest classes. The common-test analysis therefore separates tail-sensitive performance from majority-dominated aggregate behavior.
SOUP-Q has a lower held-out Macro-F1 than both PAIR and Square-Root in all nine cells (Table 12). The difference to PAIR ranges from at NF-ToN-IoT , where SOUP-Q wins three of the five paired seeds, to between and on CICIDS2017, where it loses in all fifteen paired comparisons. The tail-class allocations in Table 13 explain this pattern. SOUP-Q raises every class to the common per-class target, capped at its capacity. It therefore uses all available observations of the CICIDS2017 tail classes and, at , of Ransomware and Worms; where a tail class is not exhausted, it receives 8–25 times its Square-Root quota (MITM 100, 265, and 543 instead of 12, 31, and 61; Ransomware 100 instead of 4 and Worms 100 instead of 10 at ). It consequently attains a higher Macro-Recall than PAIR in every cell and the highest of all methods on CICIDS2017 (0.9225–0.9359, against 0.7972–0.8431 for PAIR); only the fixed Power allocation reaches a higher Macro-Recall in five NF-ToN-IoT and UNSW-NB15 cells. However, the equalized quotas remove most of the budget from the bulk classes. On CICIDS2017, BENIGN recall falls to 0.869–0.970 and Macro-Precision to 0.4016–0.5172, against 0.6320–0.7760 for PAIR. Supplementary Table S7 compares SOUP-Q with a control that uses the same quotas but random within-class selection. Because the SOUP-Q quotas coincide with capacity-capped Balanced allocation, this control is capacity-capped Balanced allocation on the held-out evaluation, the counterpart of the development Balanced baseline in Appendix G. The source of the SOUP-Q deficit differs between datasets. On CICIDS2017, the random-selection control with the same quotas is itself 0.2117–0.2459 below PAIR, while the safety-based selection of SOUP-Q is slightly better than random selection ( to ); this supports attributing the CICIDS2017 deficit to the allocation. On NF-ToN-IoT, the selection effect is mixed ( at , and at and 5000), and the control is also below PAIR at all three budgets, so the deficit is again mainly allocation-related. On UNSW-NB15, by contrast, the safety-based selection itself reduces Macro-F1 relative to the same-quota random control in all 15 paired seeds (, , and at , 2500, and 5000) and accounts for 28–50% of the gap to PAIR; at , for example, PAIR reaches 0.4302, the control 0.3990, and SOUP-Q 0.3680.
Table 13.
Allocation outcomes for the structural-tail classes on the held-out evaluation: training quota (mean held-out recall over five seeds) under Square-Root, PAIR, SOUP-Q, and CCS-CP. is the training-pool capacity of the class and its fixed held-out support. CICIDS2017 recalls rest on 2–7 held-out observations and are descriptive. PAIR moves of 1000 observations on NF-ToN-IoT at and , 7, and 1 on CICIDS2017 at , 2500, and 5000 (); D is computed from the full quota vectors and equals the sum of the tail-class increases. Held-out quotas can differ from the development quotas in Table A1 because the training pools differ; for example, Ransomware has here and 91 in development, which gives an RRP floor of 10 instead of 8. In these held-out pools, RRP is binding for Ransomware (10 versus the K-dependent floor of 5) and for CICIDS2017 Infiltration at and 2500 (7 versus 6); Heartbleed and SQL Injection remain at the K-dependent floor.
CCS-CP has a lower held-out Macro-F1 than PAIR in all nine cells and in all 45 paired seed comparisons (Table 12). The difference to PAIR ranges from to on UNSW-NB15, from to on NF-ToN-IoT, and from to on CICIDS2017. Because its class budgets follow the training supports apart from the minimum m, CCS-CP stays close to Proportional sampling; its Macro-F1 differs from Proportional by to and lies within 0.03 of it in six of the nine cells. Although CCS-CP enforces a per-class minimum, several low-support classes remain difficult to recover under the evaluated budgets (Table 13). On CICIDS2017, each structural-tail class receives six observations; Heartbleed is recalled at 0.600–1.000, whereas SQL Injection and Infiltration reach at most 0.050 and 0.229, against 0.100–0.300 and 0.714–0.771 under PAIR. Bot also receives six observations at every budget and is never detected, against a recall of 0.695–0.949 under PAIR. On NF-ToN-IoT and UNSW-NB15, the structural-tail classes receive the minimum of five observations at every budget, and Ransomware, MITM, and Worms are rarely detected (mean recall at most 0.250). Because proportional budgets retain more majority-class observations, CCS-CP attains a higher Accuracy than PAIR in four of the six NF-ToN-IoT and UNSW-NB15 cells (Supplementary Table S1). Supplementary Table S8 compares CCS-CP with a control that uses the same class budgets but random within-class selection. The AUM-based selection is worse than this control in all nine cells ( to ; 36 of 45 paired seeds) and accounts for 6–40% of the gap to PAIR. The control itself remains 0.0381–0.0470 (UNSW-NB15), 0.0881–0.1321 (NF-ToN-IoT), and 0.2296–0.3853 (CICIDS2017) below PAIR, so the larger part of the CCS-CP deficit is allocation-related. With , CCS-CP is also below PAIR in all 45 paired comparisons, and on CICIDS2017 at , its Macro-F1 is 0.2563, close to that of Proportional sampling (0.2651; Supplementary Table S8).
The CICIDS2017 held-out set contains only two Heartbleed, four SQL Injection, and seven Infiltration observations. The Macro-F1 difference is therefore driven by recovery of very rare events. Table 14 makes the denominators explicit. Heartbleed is recovered in both held-out observations under PAIR in all five seeds, yet the exact 95% recall interval for remains ; the corresponding Square-Root result is with interval .
Table 14.
Held-out CICIDS2017 rare-class readout at . All three classes have fewer than 10 held-out observations, so their recall estimates move in steps of and are descriptive only. Recall means are over the five paired sampling seeds. “PAIR TP” gives the seed-level range of true positives on the same held-out observations; the exact 95% Clopper–Pearson intervals are evaluated at the lowest and highest seed-level counts (all budgets: Supplementary Table S5).
The exact Clopper–Pearson intervals quantify the uncertainty associated with these small denominators.
8. Discussion
8.1. When to Expect Activation and Benefit
Because the trigger depends only on class counts, whether PAIR will intervene can be checked before any model is trained. For a structural-tail class t with training support , the continuous Square-Root quota is approximately , and PAIR intervenes when this quota falls below while the class still has unused observations (; Remark 1). Activation therefore becomes more likely as (i) the budget B decreases, (ii) the class count K increases, raising , and (iii) the tail class becomes rarer relative to the rest of the distribution, that is, as decreases relative to . In the synthetic design, this ratio is controlled by the structural gap . The controlled synthetic study in Section 7.5 confirms this condition: it predicted whether the quotas change in 127 of 128 cells, the exception being a one-unit integer-rounding case.
The same study indicates when a benefit can be expected. Gains appeared only inside the active region. Averaged over the tested active cells, they increased with K and with the number of reallocated observations and decreased as B grew; these are average trends rather than guarantees for individual cells. In the synthetic design at , gains were small or null when and when the tail class contained so few observations that it was already fully used. PAIR is most relevant for many-class problems with a few extremely rare classes under severe compression, as in CICIDS2017.
UNSW-NB15 illustrates the opposite case. Its only structural-tail class, Worms, already receives 10 observations under Square-Root at , twice the floor , so no repair is needed at any of the studied budgets. When PAIR is inactive, it confirms that the Square-Root allocation already meets the support floor and returns exactly the same quotas and retained rows. Applying PAIR in such a setting therefore costs only the negligible structural setup and cannot change the result.
8.2. Stability of Activation, Trigger Choice, and Refinement
Consistency. The intervention map is the same in the development and held-out stages: NF-ToN-IoT and all three CICIDS2017 budgets are active, and the remaining cells are inactive (Table 6, Table 7 and Table 12). Activation therefore follows the class-support structure and the budget, not a particular partition. Predictive effects remain dataset- and learner-dependent, in line with broader observations that resampling strategies interact with the data distribution and model class [29].
Trigger choice. The proposed trigger lies between under-activation and unconditional intervention (Table 8 and Appendix G). Narrower rules miss repairs that are associated with positive parent-relative differences, wider rules mostly converge to the same repaired quotas, and ALWAYS-TAIL moves more budget without a better mean.
Role of RRP. In the development pool, the K-dependent floor alone accounts for the CICIDS2017 repairs, and RRP is uniquely binding only for NF-ToN-IoT Ransomware at (Table 9). In the held-out training pools, RRP also raises CICIDS2017 Infiltration to 7 at and 2500 and NF-ToN-IoT Ransomware to 10 at , above K-dependent floors of 6 and 5 (Table 13). RRP thus acts as a secondary, geometry-derived refinement of an active floor.
8.3. Comparison with Global Allocation Baselines
The baseline results distinguish conditional repair from global redistribution. Proportional allocation preserves the source frequency structure, the fixed Power rule increases global tail emphasis, Square-Root provides the parent compromise, and K-Center emphasizes label-blind geometric coverage. PAIR modifies only the active portion of the Square-Root allocation.
This distinction is visible on NF-ToN-IoT at , where Power attains slightly higher held-out Macro-F1 than PAIR (0.5519 versus 0.5466). In this regime, a stronger global shift toward the tail is numerically favorable. In CICIDS2017, by contrast, PAIR achieves the highest held-out Macro-F1 at all three budgets while moving only a small fraction of the fixed budget relative to Square-Root. K-Center performs substantially lower in Macro-F1 under the severe budgets considered here, suggesting that label-blind coverage and class-quota repair address different aspects of aggressive reduction. The fixed-budget SOUP-derived adaptation (SOUP-Q), whose quotas coincide with capacity-capped Balanced allocation, achieves higher Macro-Recall than PAIR but lower mean Macro-F1 in all nine held-out cells. On CICIDS2017, this is accompanied by lower Macro-Precision and BENIGN recall. Together with Power, SOUP-Q illustrates that a global shift toward the tail can recover rare classes at a cost elsewhere, whereas PAIR obtains its tail gains by moving at most 1.5% of the budget on the held-out training pools. CCS-CP has the opposite limitation. Although it enforces a per-class minimum, several low-support classes remain difficult to recover under the evaluated budgets, and because its remaining budget follows the training supports, its Macro-F1 remains close to Proportional sampling.
8.4. Rare-Event Recovery and Aggregate Performance
CICIDS2017 provides the clearest active regime, but also the most restrictive statistical setting. Most of the development Macro-F1 difference is attributable to the structural-tail classes, particularly Heartbleed, and the held-out denominators are only two Heartbleed, four SQL Injection, and seven Infiltration observations. The associated exact intervals are consequently wide. The improvement is best understood as recovery of rare events that the parent allocation misses; majority-dominated performance changes little.
The held-out secondary metrics in Supplementary Table S1 support this interpretation. In CICIDS2017 at , PAIR raises Macro-F1 from 0.6788 to 0.7726 (), whereas Accuracy changes from 0.9940 to 0.9939 and Weighted-F1 from 0.9943 to 0.9942; at , Macro-F1 rises by 0.0351 while Accuracy and Weighted-F1 decrease by 0.0006 and 0.0004. Across the four active held-out cells, the largest absolute change in Accuracy or Weighted-F1 is 0.0016 (NF-ToN-IoT, ), less than one tenth of the corresponding Macro-F1 change. Because Accuracy and Weighted-F1 are dominated by the majority classes, these near-zero and occasionally negative changes are what a gain confined to a few ultra-rare classes would produce. NF-ToN-IoT offers a complementary active case with larger rare-class denominators. Its repeated-split results are consistently positive at , while one of the five held-out sampling seeds is negative, showing that the benefit is reproducible at the regime level but not deterministic at the individual seed level.
8.5. Learner and Decision-Rule Dependence
Cross-learner analysis further separates allocation behavior from predictive realization. The same PAIR quotas that improve XGBoost Macro-F1 can produce negligible or negative differences under Random Forest and smaller mixed responses under DNN. At CICIDS2017 , for example, Random Forest gains Macro-Recall but loses enough Macro-Precision to reduce Macro-F1. The training-only RF decision-rule analysis in Appendix E shows that calibration can also change this trade-off.
Accordingly, classifier-agnostic refers to how PAIR constructs the subset: no classifier predictions, gradients, losses, or validation scores enter the allocation. The performance obtained from that subset remains a property of the interaction between allocation, learner, and decision rule.
8.6. Relation to Existing Intrusion-Detection and Coreset Methods
PAIR occupies a different stage of the reduction pipeline from methods that incorporate model difficulty or sample-level geometry directly into selection. DUEN uses difficulty information within boosting [9]; class-proportional coreset selection combines class allocation with within-class selection [5] and is evaluated as CCS-CP in Table 12; SOUP uses local safety and class similarity [6], and an explicitly defined fixed-budget adaptation of it, SOUP-Q, is evaluated in Table 12. PAIR instead treats Square-Root as a parent class allocation and introduces a support repair only in an active structural-tail regime. Its defining property is therefore conditional parent preservation, which class awareness alone does not provide.
8.7. Limitations
The present evidence has the following boundaries.
- Scope of datasets and learners. The empirical study covers three intrusion-detection benchmarks, so external validity beyond this domain remains to be established. The full seven-method held-out comparison is conducted with XGBoost, while Random Forest and DNN serve as development-stage learner-robustness analyses.
- Statistical limits. CICIDS2017 contains extremely small held-out supports for its rarest attacks, which limits the precision of class-level recall and F1 estimates. Repeated development splits overlap; we therefore use an overlap-corrected test alongside the Wilcoxon test, but the resulting p-values remain approximate. PAIR is active in four of the nine dataset–budget settings, and on the held-out test set its benefit over Square-Root rests on five paired seeds per cell. With five paired seeds, the seed-level Wilcoxon test cannot reach the 5% level (Section 6.9). In NF-ToN-IoT at and CICIDS2017 at , the held-out mean difference is comparable to the seed-to-seed standard deviation.
- Design choices. The K-dependent trigger is a structural design convention whose sensitivity has been evaluated within the studied range, and alternative structural summaries could produce different active floors. The RRP refinement is a secondary, training-only estimate whose incremental effect is modest and seed-dependent (Section 8.2).
- Baseline implementations. The original SOUP procedure determines its own resampled class size and permits repeated observations; its native output therefore does not enforce our prespecified budget of B distinct source observations. We instead evaluate the explicitly defined SOUP-Q adaptation under the shared fixed-budget protocol, which uses capacity-capped equal quotas without oversampling and computes safety levels once on the original training pool instead of recomputing them during sequential class processing. Because we did not evaluate the original procedure, the SOUP-Q results do not establish comparative performance against it. For observations with identical feature vectors but different labels, which are frequent in NF-ToN-IoT, their safety levels depend on the neighbor tie-breaking rule. DUEN was not evaluated because it constructs its training samples dynamically inside its own boosting ensemble and therefore yields no single fixed-budget subset that could be compared under our protocol; we also did not locate a publicly available implementation. Our paper-based CCS-CP reimplementation fixes the settings that the paper leaves open (the per-class minimum, the stratum count, and the completion of rounding shortfalls). It also computes AUM with the same multilayer-perceptron architecture for all datasets instead of the dataset-specific networks of the original study, so its results depend on these documented choices.
- Global reallocation. The fixed Power allocation outperforms PAIR in some cells, confirming that parent preservation and global predictive optimization are distinct objectives.
9. Conclusions
This study introduced PAIR as a conditional support-repair operator for Square-Root allocation under a fixed retained-data budget. The method preserves Square-Root as the default allocation, identifies a structural low-support regime from the training data, applies a K-dependent floor with a secondary representation-derived refinement when active, and restores the budget through the KL repair. When the trigger is inactive, quota and sample identities with Square-Root are exact.
Across NF-ToN-IoT, UNSW-NB15, and CICIDS2017, activation is concentrated in the most severely compressed regimes. Repeated development splits and the held-out common-test evaluation identify NF-ToN-IoT and all three CICIDS2017 budgets as active, with the inactive case in the remaining main cells. In the held-out seven-method comparison, PAIR achieves the highest mean Macro-F1 in all three CICIDS2017 budgets, while the fixed Power allocation is slightly higher on NF-ToN-IoT at . The CICIDS2017 differences are concentrated in ultra-rare classes and are accompanied by wide class-level uncertainty, whereas aggregate Accuracy and Weighted-F1 change comparatively little.
The ablation results further clarify the mechanism. In the development pool, the K-dependent support floor accounts for the CICIDS2017 repairs, while the representation-derived refinement is uniquely binding in the NF-ToN-IoT regime. In the held-out training pools, the refinement additionally raises CICIDS2017 Infiltration above the K-dependent floor. Taken together, the results position PAIR as a regime-specific extension of Square-Root sampling for severe fixed-budget tail-support loss. Its practical value lies in combining targeted intervention with exact parent preservation outside the active regime, while leaving the predictive consequences to be evaluated for the data distribution and learner at hand.
Supplementary Materials
The following supporting information can be downloaded at https://www.mdpi.com/article/10.3390/math14213988/s1: Table S1: Held-out overall metrics; Table S2: NF-ToN-IoT held-out class recall and fixed test support; Table S3: UNSW-NB15 held-out class recall and fixed test support; Table S4: CICIDS2017 held-out class recall and fixed test support; Table S5: CICIDS2017 classes with fewer than 10 held-out observations; Table S6: Paired tests of PAIR minus Square-Root Macro-F1; Table S7: SOUP-Q versus a control with identical quotas and random within-class selection; Table S8: CCS-CP: tuned cutoff, sensitivity to the per-class minimum, and random-selection control.
Author Contributions
Conceptualization, S.K.; methodology, S.K.; software, S.K.; validation, S.K., M.Z.K., K.K. (Kaplan Kaplan) and K.K. (Kerem Küçük); formal analysis, S.K.; investigation, S.K.; data curation, S.K.; writing—original draft preparation, S.K.; writing—review and editing, S.K., M.Z.K., K.K. (Kaplan Kaplan) and K.K. (Kerem Küçük); visualization, S.K.; supervision, M.Z.K. and K.K. (Kerem Küçük); project administration, K.K. (Kerem Küçük). All authors have read and agreed to the published version of the manuscript.
Funding
This research received no external funding.
Institutional Review Board Statement
Not applicable.
Informed Consent Statement
Not applicable.
Data Availability Statement
The study uses publicly available benchmark intrusion-detection datasets. The source code of PAIR and of the experiments is available from the corresponding author upon reasonable request. The Supplementary Materials contain Table S1 (overall held-out metrics), Tables S2–S4 (class-wise held-out recall with fixed test support), Table S5 (exact intervals for CICIDS2017 classes with fewer than 10 held-out observations), Table S6 (paired tests), Table S7 (SOUP-Q control with random within-class selection), and Table S8 (CCS-CP tuning, sensitivity to the per-class minimum, and random-selection control).
Conflicts of Interest
The authors declare no conflicts of interest.
Appendix A. RRP Structural Refinement
This appendix gives the reproducibility details of the secondary representation refinement. For a structural-tail class c, PAIR computes equal-feature Gower dissimilarities. Numerical features are median-imputed and range-normalized from structural-training data; categorical variables use equality/mismatch distance with missing values represented as a separate category. A deterministic greedy facility-location sequence is used to define
with normalized representation gain and retention cost
The point estimate is the smallest maximizer of .
To align the refinement with the final random-prefix sampler, each tail class is then evaluated with up to five structural cross-fitting folds. For a held-out observation whose distances to n structural-training observations are ordered as , retaining a uniformly random prefix of size k is equivalent to drawing a uniform k-subset. If J is the rank of the nearest retained observation,
This finite expectation is evaluated directly rather than estimated by Monte Carlo. Each fold produces a random-prefix operating point within the point-estimate search envelope, and the final refinement is
Singleton or degenerate classes use a floor of one. If cross-fitting cannot support the required search interval, the point estimate is retained. This refinement never activates PAIR on its own.
Appendix B. Controlled Trigger Falsification and Runtime
A nested synthetic multiclass experiment changes K while holding feature dimension and rare-class geometry fixed. Inactive cells show exact quota, subset, prediction, and metric identity. The experiment also contains an active case at and in which mean target-class F1 changes by across the five seeds. This counterexample illustrates Remark 2.
Runtime is secondary to the predictive analysis; the values below are given to four decimals, whereas the main text rounds them to two. In the recorded development environment, the one-time structural setup required 1.2777 s on NF-ToN-IoT, 0.0051 s on UNSW-NB15, and 0.0036 s on CICIDS2017. Across active cells, the largest absolute PAIR–Square-Root difference in downstream fit time was 0.0467 s, and the largest prediction-time difference was 0.0389 s. These values are reported only as implementation context because the timing scopes of alternative reduction methods are not directly comparable.
Appendix C. Controlled Synthetic Study of the Activation Boundary
To show how activation and benefit depend on data characteristics, we generated synthetic long-tailed multiclass data with parametrically controlled supports. Each class follows a 20-dimensional Gaussian with mean , where is the c-th standard basis vector of , so that all pairs of class means are at distance s. The two tail classes () have equal training support , which equals for every grid value. The head classes have supports , , rounded to the nearest integer, with : for , and the head supports are 2000, 3000, and 4500; for , and the head supports range from 2000 to 20,000. Because for every grid value, the largest adjacent support ratio is the tail–head boundary, so the structural gap equals . Because the head-class supports are fixed, varying g changes the tail-class size directly; the gap axis can therefore equivalently be read as a tail-size axis. The grid is , , and (128 cells), with 200 test observations per class, the five paired seeds of the main study, the frozen XGBoost configuration, and the full PAIR operator including the RRP refinement. The data are drawn once from a fixed random stream per class (NumPy RandomState with seed for class c, at both separations), which first generates the 200 test observations and then 20,000 training observations, of which the first are used; the training sets are therefore nested across gaps, and for a given K and s, the test set is identical across all values of g and B. The data are not regenerated per sampling seed: the five sampling seeds set only the XGBoost seed and the within-class random orders from which both methods take their quotas; the RRP folds use a fixed structural seed (20260828). The full 128-cell grid was run at separation ; only the 28 active cells were repeated at , because activation depends only on class counts and is therefore identical at both separations. The budget is included only to map the activation boundary under more severe compression; the real-data evaluation retains the prespecified budgets of 1000–5000.
PAIR left the quotas unchanged in 100 of 128 cells, including every cell with ; in all of them, quotas, retained rows, predictions, and metrics were identical to Square-Root. Here, the 28 active cells are the cells in which the PAIR quotas differ from Square-Root (); the trigger of Equation (15) fires in 33 cells, and in the remaining 5, all at with , the tail classes are already fully used under Square-Root, so the quotas are unchanged. The closed-form condition of Remark 1 predicted whether the quotas change in 127 of 128 cells; the exception is a one-unit rounding case at , , , where the continuous quota 5.68 is below the floor 6 but the integer quota equals 6. Figure 3 compares the predicted boundary with the observed activation.
Within the 28 active cells at , the mean PAIR-minus-Square-Root Macro-F1 difference was , positive in 24 cells and in 110 of 140 seed-level comparisons (Figure A1). The largest differences, and , occurred at for , and , , where tail-class F1 increased by 0.1693 and 0.2614. Averaged over the tested active cells, the difference was larger in cells that reallocated more observations, increased with K (, , and for , 15, and 20) and decreased with B (, , and for , 1000, and 2500), while head-class Macro-F1 changed by at most 0.0045. Four active cells had small negative means ( to , each within one standard deviation), all at or with only four tail observations available. At , the mean active-cell difference was , positive in all 28 cells (up to ). As expected, since activation depends only on class counts, class separability changed the size of the benefit but not the activation boundary; the run was repeated only for the 28 active cells.
Figure A1.
Synthetic study (): mean PAIR-minus-Square-Root Macro-F1 (percentage points) and seed-level W/T/L as a function of the structural gap , budget B, and class count K. Hatched cells are unchanged-quota cases (trigger not active, or active with the tail class already fully used), in which PAIR returns the Square-Root quotas and rows.
Appendix D. PAIR Activation and Quota Details
Table A1 summarizes the inputs to each active PAIR allocation in the main budget grid. The development-training support , Square-Root quota, the RRP structural-floor estimate, the closed-form K-dependent floor, the binding component, and the final PAIR quota are shown together. This table summarizes the active constraints within PAIR and shows how the binding floor changes across active cells; the KL repair may redistribute donor mass over several coordinates.
Table A1.
Active PAIR tail constraints in the main budget grid. Values are computed before downstream model fitting.
Within the active development cells, the RRP component is uniquely binding only for NF-ToN-IoT Ransomware, where ; in the held-out training pools, it is also binding for CICIDS2017 Infiltration (Table 13). In CICIDS2017, the closed-form K-dependent floor either binds or ties the RRP estimate. For comparison, MITM in NF-ToN-IoT has at and an RRP structural-floor estimate of 54, but the PAIR trigger is inactive because ; the quota therefore remains 12. Worms in UNSW-NB15 similarly remains at its Square-Root quota of 10 even though its RRP estimate is 11. These cases show that the RRP estimate is an internal candidate floor, not an unconditional correction rule.
The total intervention masses for the active dataset–budget cells are for NF-ToN-IoT at and , 6, and 1 for CICIDS2017 at , 2500, and 5000, respectively. All other main-budget cells have . At budgets of at least 10,000 in the additional sensitivity analysis, NF-ToN-IoT and CICIDS2017 also return the Square-Root allocation unchanged.
Appendix E. Random Forest Decision-Rule Sensitivity
The RF robustness analysis uses one common training-only selected configuration for each paired Square-Root/PAIR cell. Hyperparameter selection is based on out-of-bag predictions from the sampled training subsets; the common test set is not used for model selection. The same selected RF configuration is then applied to both sampling methods.
For CICIDS2017 at , we also report a calibrated RF decision rule. “Calibrated RF” uses the same fitted RF, hyperparameters, and Square-Root/PAIR training subsets; only the predicted class probabilities are adjusted with training-only class-prior information before the final class decision. The shared calibration strength is selected from training data only, so the common test set is not used. Calibration therefore changes the downstream decision rule, not PAIR or its quotas (Table A2).
Table A2.
CICIDS2017 at : five-seed Macro-F1 under the standard RF decision rule and the training-only calibrated RF sensitivity check. Δ is PAIR minus Square-Root.
This appendix check is intentionally narrow. The standard RF result remains the primary robustness result. The calibrated row only shows that the sign of the CICIDS2017 effect is decision-rule-sensitive; it is not used to select, tune, or redefine PAIR.
Appendix F. Cross-Learner Active-Cell Detail
Table A3 reports the actual Square-Root and PAIR Macro-F1 means for the active severe-budget cells. Presenting both values avoids reducing the robustness analysis to signed differences alone.
Table A3.
Five-seed Macro-F1 means in active cells for the three downstream learners. RF values use the default RF argmax decision rule.
The table reinforces the distinction between construction and predictive realization. PAIR’s quotas are identical regardless of the downstream learner, but the sign and magnitude of the predictive response differ. The large CICIDS2017 RF loss at occurs together with higher Macro-Recall and lower Macro-Precision, showing that added rare support can shift the classifier’s operating trade-off between recall and precision. The compact calibration check in Table A2 is reported separately because it changes only the RF decision rule.
Appendix G. Additional Baselines and Support-Trigger Sensitivity
Table A4 reports additional Balanced, K-Center, and Power () baselines. Table A5 reports the complete trigger-sensitivity results.
Table A4.
Additional baseline results on the development data.
Table A5.
Complete five-seed trigger-sensitivity Macro-F1 means. Only the activation trigger changes.
Appendix H. Implementation Details
The primary results use five paired sampling seeds (42, 52, 62, 72, and 82), while the repeated-split analysis uses 10 development splits with three paired sampling seeds per split. Square-Root and PAIR use the same class-wise random-prefix realization for each paired seed. RRP details are given in Appendix A. Integer apportionment is deterministic: PAIR residual ties are resolved by fractional remainder, continuous projected quota, and then fixed class-label order; corresponding Square-Root and Power ties use their deterministic Hamilton ordering. If the full structural lower bounds are infeasible, basic class-survival bounds are used when feasible. If B is smaller than the number of nonempty classes, the implementation records class-survival infeasibility and returns the Square-Root parent without asserting universal survival. No fallback was required in the reported experiments. The development runs recorded Python 3.13.9, XGBoost 3.3.0, scikit-learn 1.9.0, NumPy 2.2.2, and pandas 2.2.3, the versions recorded in the project environment; the reused held-out runs did not record package versions. The SOUP-Q and CCS-CP runs used Python 3.13.16 and XGBoost 3.4.1 (scikit-learn 1.9.1, NumPy 2.5.3, pandas 3.0.5). Refitting Square-Root and PAIR for all held-out cells and seeds in the SOUP-Q environment reproduced the training subsets exactly and changed the five-seed Macro-F1 means by at most 0.0011 (individual seeds by at most 0.0069 for CICIDS2017 Square-Root); CICIDS2017 PAIR reproduced exactly. These five-seed mean differences are small relative to the differences between SOUP-Q and the other methods.
References
- Weiss, G.M.; Provost, F. Learning when training data are costly: The effect of class distribution on tree induction. J. Artif. Intell. Res. 2003, 19, 315–354. [Google Scholar] [CrossRef] [Scilit]
- Martinez-Fuentes, V.; Arroyo, A.; Granados López, D.; Herrero, A. Toward AI-driven IoT cybersecurity: A preprocessing framework for benchmark datasets. Int. J. Inf. Secur. 2026, 25, 62. [Google Scholar] [CrossRef] [Scilit]
- Kang, B.; Xie, S.; Rohrbach, M.; Yan, Z.; Gordo, A.; Feng, J.; Kalantidis, Y. Decoupling representation and classifier for long-tailed recognition. In Proceedings of the International Conference on Learning Representations (ICLR), Virtual, 26 April–1 May 2020. [Google Scholar]
- Zheng, H.; Liu, R.; Lai, F.; Prakash, A. Coverage-Centric Coreset Selection for High Pruning Rates. In Proceedings of the International Conference on Learning Representations (ICLR), Kigali, Rwanda, 1–5 May 2023. [Google Scholar]
- Tsai, E.; Zheng, H.; Prakash, A. Class-Proportional Coreset Selection for Difficulty-Separable Data. In Proceedings of the IEEE/CVF International Conference on Computer Vision Workshops, Honolulu, HI, USA, 20 October 2025; pp. 6869–6878. [Google Scholar]
- Janicka, M.; Lango, M.; Stefanowski, J. Using information on class interrelations to improve classification of multiclass imbalanced data: A new resampling algorithm. Int. J. Appl. Math. Comput. Sci. 2019, 29, 769–781. [Google Scholar] [CrossRef] [Scilit]
- Chen, W.; Yang, K.; Yu, Z.; Shi, Y.; Chen, C.L.P. A survey on imbalanced learning: Latest research, applications and future directions. Artif. Intell. Rev. 2024, 57, 137. [Google Scholar] [CrossRef] [Scilit]
- Milosevic, M.S.; Ciric, V.M. Extreme minority class detection in imbalanced data for network intrusion. Comput. Secur. 2022, 123, 102940. [Google Scholar] [CrossRef] [Scilit]
- Ren, H.; Tang, Y.; Dong, W.; Ren, S.; Jiang, L. DUEN: Dynamic ensemble handling class imbalance in network intrusion detection. Expert Syst. Appl. 2023, 229, 120420. [Google Scholar] [CrossRef] [Scilit]
- Le, T.-T.-H.; Shin, Y.; Kim, M.; Kim, H. Towards unbalanced multiclass intrusion detection with hybrid sampling methods and ensemble classification. Appl. Soft Comput. 2024, 157, 111517. [Google Scholar] [CrossRef] [Scilit]
- Oh, S.; Sohn, S.; Kim, C.; Park, M. MCH-Ensemble: Minority Class Highlighting Ensemble Method for Class Imbalance in Network Intrusion Detection. Appl. Sci. 2025, 15, 12647. [Google Scholar] [CrossRef] [Scilit]
- Kamal, H.; Mashaly, M. Robust Intrusion Detection System Using an Improved Hybrid Deep Learning Model for Binary and Multi-Class Classification in IoT Networks. Technologies 2025, 13, 102. [Google Scholar] [CrossRef] [Scilit]
- Cui, Y.; Jia, M.; Lin, T.-Y.; Song, Y.; Belongie, S. Class-balanced loss based on effective number of samples. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Long Beach, CA, USA, 16–20 June 2019; pp. 9268–9277. [Google Scholar] [CrossRef] [Scilit]
- Buda, M.; Maki, A.; Mazurowski, M.A. A systematic study of the class imbalance problem in convolutional neural networks. Neural Netw. 2018, 106, 249–259. [Google Scholar] [CrossRef] [Scilit] [PubMed]
- Lin, F.; Yuan, X. Long-Tailed Recognition via Information-Preservable Two-Stage Learning. Adv. Neural Inf. Process. Syst. 2025, 38, 47035–47064. [Google Scholar]
- Li, J.; Duan, J.; Shao, C.; Yu, H. PHASE: Progressive Hierarchical Adaptation for Sample-Efficient Rebalancing in Long-Tail Classification. Symmetry 2025, 17, 2040. [Google Scholar] [CrossRef] [Scilit]
- Ding, H.; Yu, H.; Wang, Z. Greedy strategy works for k-center clustering with outliers and coreset construction. In Proceedings of the 27th Annual European Symposium on Algorithms (ESA 2019), Munich/Garching, Germany, 9–11 September 2019; Leibniz International Proceedings in Informatics (LIPIcs); Schloss Dagstuhl–Leibniz-Zentrum für Informatik: Dagstuhl, Germany, 2019; Volume 144, pp. 40:1–40:16. [Google Scholar] [CrossRef]
- Gonzalez, T.F. Clustering to minimize the maximum intercluster distance. Theor. Comput. Sci. 1985, 38, 293–306. [Google Scholar] [CrossRef] [Scilit]
- Tan, H.; Wu, S.; Huang, W.; Zhao, S.; Qi, X. Data Pruning by Information Maximization. In Proceedings of the International Conference on Learning Representations (ICLR), Singapore, 24–28 April 2025; pp. 50133–50155. Available online: https://proceedings.iclr.cc/paper_files/paper/2025/hash/7d848891e365ca623dc8352db9782585-Abstract-Conference.html (accessed on 18 October 2026).
- Vysogorets, A.; Ahuja, K.; Kempe, J. DRoP: Distributionally Robust Data Pruning. In Proceedings of the International Conference on Learning Representations (ICLR), Singapore, 24–28 April 2025; pp. 5715–5739. Available online: https://proceedings.iclr.cc/paper_files/paper/2025/hash/110efb00ba39be543b75d097a0aae531-Abstract-Conference.html (accessed on 18 October 2026).
- Zhang, H.; Xing, Z.; He, R.; Yang, W.; Ma, C.; Tan, W.; Yan, B. Non-Uniform Class-Wise Coreset Selection for Vision Model Fine-tuning. arXiv 2025, arXiv:2504.13234v2. [Google Scholar]
- Zhao, X.; Shang, X.; Liu, J.; Shen, Z. Exploring 3D Dataset Pruning. In Proceedings of the 43rd International Conference on Machine Learning (ICML 2026), Seoul, Republic of Korea, 6–11 July 2026. [Google Scholar]
- Fernández, A.; López, V.; Galar, M.; del Jesús, M.J.; Herrera, F. Analysing the classification of imbalanced data-sets with multiple classes: Binarization techniques and ad-hoc approaches. Knowl.-Based Syst. 2013, 42, 97–110. [Google Scholar] [CrossRef] [Scilit]
- Putrama, I.M.; Martinek, P. Neighbor Displacement-Based Enhanced Synthetic Oversampling for Multiclass Imbalanced Data. Comput. Mater. Contin. 2025, 83, 5699–5727. [Google Scholar] [CrossRef] [Scilit]
- Fachrie, M.; Musdholifah, A.; Pulungan, R. Effectiveness of data resampling and ensemble learning in multiclass imbalance learning. Artif. Intell. Rev. 2025, 58, 368. [Google Scholar] [CrossRef] [Scilit]
- Fürnkranz, J. Round Robin Classification. J. Mach. Learn. Res. 2002, 2, 721–747. [Google Scholar]
- Allwein, E.L.; Schapire, R.E.; Singer, Y. Reducing multiclass to binary: A unifying approach for margin classifiers. J. Mach. Learn. Res. 2000, 1, 113–141. [Google Scholar]
- Lee, D.; Kim, K. An efficient method to determine sample size in oversampling based on classification complexity for imbalanced data. Expert Syst. Appl. 2021, 184, 115442. [Google Scholar] [CrossRef] [Scilit]
- Moniz, N.; Monteiro, H. No Free Lunch in imbalanced learning. Knowl.-Based Syst. 2021, 227, 107222. [Google Scholar] [CrossRef] [Scilit]
- Bregman, L.M. The relaxation method of finding the common point of convex sets and its application to the solution of problems in convex programming. USSR Comput. Math. Math. Phys. 1967, 7, 200–217. [Google Scholar] [CrossRef] [Scilit]
- Csiszár, I. I-divergence geometry of probability distributions and minimization problems. Ann. Probab. 1975, 3, 146–158. [Google Scholar] [CrossRef] [Scilit]
- Gower, J.C. A general coefficient of similarity and some of its properties. Biometrics 1971, 27, 857–871. [Google Scholar] [CrossRef] [Scilit]
- Sarhan, M.; Layeghy, S.; Moustafa, N.; Portmann, M. NetFlow datasets for machine learning-based network intrusion detection systems. In Big Data Technologies and Applications (BDTA 2020); Lecture Notes of the Institute for Computer Sciences, Social Informatics and Telecommunications Engineering; Springer: Cham, Switzerland, 2021; Volume 371, pp. 117–135. [Google Scholar] [CrossRef] [Scilit]
- Moustafa, N.; Slay, J. UNSW-NB15: A comprehensive data set for network intrusion detection systems. In Proceedings of the Military Communications and Information Systems Conference (MilCIS), Canberra, Australia, 10–12 November 2015; pp. 1–6. [Google Scholar] [CrossRef] [Scilit]
- Sharafaldin, I.; Lashkari, A.H.; Ghorbani, A.A. Toward generating a new intrusion detection dataset and intrusion traffic characterization. In Proceedings of the 4th International Conference on Information Systems Security and Privacy (ICISSP), Funchal, Portugal, 22–24 January 2018; pp. 108–116. [Google Scholar] [CrossRef] [Scilit]
- Nadeau, C.; Bengio, Y. Inference for the generalization error. Mach. Learn. 2003, 52, 239–281. [Google Scholar] [CrossRef] [Scilit]
- Breiman, L. Random Forests. Mach. Learn. 2001, 45, 5–32. [Google Scholar] [CrossRef] [Scilit]
- Grycza, J.; Horna, D.; Klimczak, H.; Lango, M.; Pluciński, K.; Stefanowski, J. multi-imbalance: Open Source Python Toolbox for Multi-class Imbalanced Classification. In Machine Learning and Knowledge Discovery in Databases. Applied Data Science and Demo Track (ECML PKDD 2020); Lecture Notes in Artificial Intelligence; Springer: Cham, Switzerland, 2021; Volume 12461, pp. 546–549. [Google Scholar] [CrossRef] [Scilit]
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content. |
© 2026 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license.



