ADDAEIL: Anomaly Detection with Drift-Aware Ensemble-Based Incremental Learning
Abstract
:1. Introduction
- (a)
- Distribution-based detection: Using robust statistical tests, the Kolmogorov–Smirnov (KS) test and Mann–Whitney U (MWU) test, ADDAEIL accurately captures both nuanced global shape and location shifts in data distributions across sliding windows.
- (b)
- Performance-based detection: ADDAEIL directly evaluates the distributional changes in path lengths of individual base detectors in iForest, as concept drift affects the decision paths within the tree structure and governs anomaly scoring. By analyzing these changes, it identifies which base detectors are most affected by concept drift from a structural perspective. This design provides a structure-based performance evaluation mechanism for drift detection without introducing any additional model training.
2. Related Work
2.1. Time Series Anomaly Detection Under Concept Drift
2.2. Concept Drift Detection Strategies
2.3. Adaptive Ensemble Models and Incremental Learning
3. Methodology
3.1. Framework Overview
- Module 1: Distribution-based Drift Detection. In the distribution-based drift detection module, the KS test and MWU test are used to simultaneously capture shape and location shifts of the distribution across consecutive windows. The two tests act as complementary statistical measures for drift detection and jointly provide a more complete estimation of drift severity, which can be mapped to the replacement ratios. In other words, these tests can be used to estimate the replacement ratio for model updating according to the drift level in each window.
- Module 2: Performance-based Drift Detection. In the performance-based drift detection module, the internal structure of the iForest model is monitored to rank the base detectors according to their replacement priorities. Concept drift will have an impact on the result decision path of the tree structure in Isolation Forest. Analyzing path length deviation will give a very good indication of model performance degradation. Therefore, the framework provides a structure-based detector ranking without requiring additional model training during drift detection.
- Module 3: Model Update Strategy. For the model update strategy, a proportion of base detectors is replaced according to the replacement ratio estimated by the distribution-based drift detection module and the base detector ranking from the performance-based drift detection module. The remaining base detectors are preserved, which allows the framework to adaptively retain more useful historical knowledge.
3.2. Module 1: Distribution-Based Drift Detection
- KS test [24]: The Kolmogorov–Smirnov (KS) test quantifies the maximum distance between the empirical cumulative distribution functions (ECDFs) of two samples:
- MWU test [26]: The Mann–Whitney U (MWU) test evaluates whether two samples originate from the same distribution by comparing their central tendencies, particularly the median. The test is non-parametric and based on rank statistics. The U statistic is computed as follows:Here, and are the sums of ranks for elements in and , respectively, and are the sample sizes. The corresponding p-value is computed as follows:The MWU test is particularly sensitive to changes in distribution center, making it complementary to the KS test, which is more responsive to differences in overall shape.
Algorithm 1 Distribution-Based Drift Detection (Module 1) |
|
3.3. Module 2: Performance-Based Drift Detection
Algorithm 2 Performance-Based |
|
3.4. Module 3: Model Update Strategy
Algorithm 3 Model Update Strategy (Module 3) |
|
4. Results
4.1. Experimental Setup
- No Update: A static iForest trained only once on the initial data .
- Distribution-only: Where drift detection and updates are only conducted based on the results of the KS and MWU tests in module 1. The replaced base detectors are randomly selected from the base model.
- Performance-only: Where updates are conducted based on the results of the path-length-based detector in module 2. The replacement ratio is set to be fixed with the value 0.3.
- ADDAEIL (Proposed): The full version of the proposed framework, combining both drift detection results.
4.2. Datasets
4.3. Evaluation Metrics
- AUC: The metric measures the ability of a model to rank anomalies higher than normal instances, without assuming a decision threshold.
- F1-score: The geometric mean of Precision and Recall, reflecting the trade-off between the two when converting continuous anomaly scores to binary labels.
- Precision: The proportion of correctly identified anomalies among all anomalies detected.
- Recall: The proportion of true anomalies that have been detected by the model.
4.4. Model Performance Evaluation
- ECOD [16]: A recent static anomaly detector based on empirical copulas. Although not tailored for streaming scenarios, ECOD is widely used as an unsupervised baseline and reflects the performance of non-adaptive methods under evolving conditions.
- LODA [10]: A semi-online ensemble-based method that uses random projections and sparse histograms for lightweight anomaly scoring. While it does not update its model after deployment, it supports automatic bin sizing during training and offers some passive robustness to minor distributional changes.
- MemStream [11]: A streaming anomaly detector that adapts to new data through a memory buffer mechanism. It can handle concept drift at the data level but lacks structural modeling of detector degradation.
- LSTM-VAE [18]: A deep learning-based anomaly detector that employs an LSTM-based variational autoencoder for capturing temporal dependencies in time series data. Although capable of modeling complex sequential patterns, it requires offline training and lacks adaptation to evolving data distributions.
- STAD [22]: An anomaly detection approach that applies KS and AD tests within the latent space of an autoencoder to capture subtle changes in data representation under concept drift. However, it does not monitor the performance of the model or internal structural alignment, and its sensitivity to drift is limited by the feature representation capabilities of the encoder.
- AEADF [15]: An unsupervised ensemble-based anomaly detection framework for streaming time series with concept drift. It employs a hybrid drift detection mechanism combining histogram-based distributional comparison (e.g., Hellinger distance) and output-level evaluation. The framework uses a fixed update strategy anchored to a static initial model, without support for incremental model updates. This limits its adaptability to gradually evolving data streams.
- ADDAEIL (proposed): The proposed framework uses statistically grounded and structure-based drift detection, combined with an adaptive model update strategy. ADDAEIL enables incremental and interpretable adaptation in non-stationary streaming settings.
4.5. Ablation Study on Update Strategies
4.6. Sensitivity Analysis on Key Hyperparameters
4.7. Edge Computing Suitability
- Training Time: 0.117 s.
- Latency per window (i.e., testing time): 0.15–0.21 s.
- Maximum Memory Usage: 340 MB.
- Model Size: 0.24 MB.
5. Conclusions
Author Contributions
Funding
Data Availability Statement
Conflicts of Interest
References
- Cook, A.A.; Misirli, G.; Fan, Z. Anomaly Detection for IoT Time-Series Data: A Survey. IEEE Internet Things J. 2020, 7, 6481–6494. [Google Scholar] [CrossRef]
- Chandola, V.; Banerjee, A.; Kumar, V. Anomaly detection: A survey. ACM Comput. Surv. 2009, 41, 1–58. [Google Scholar] [CrossRef]
- Lu, J.; Liu, A.; Dong, F.; Gu, F.; Gama, J.; Zhang, G. Learning under Concept Drift: A Review. IEEE Trans. Knowl. Data Eng. 2019, 31, 2346–2363. [Google Scholar] [CrossRef]
- Li, J.; Malialis, K.; Panayiotou, C.G.; Polycarpou, M.M. Unsupervised Incremental Learning with Dual Concept Drift Detection for Identifying Anomalous Sequences. In Proceedings of the 2024 International Joint Conference on Neural Networks (IJCNN), Yokohama, Japan, 30 June–5 July 2024; IEEE: Piscataway, NJ, USA, 2024; pp. 1–8. [Google Scholar]
- Chen, Y.; Liu, Z.; Zhang, B.; Fok, W.; Qi, X.; Wu, Y.C. Mgfn: Magnitude-contrastive glance-and-focus network for weakly-supervised video anomaly detection. AAAI Conf. Artif. Intell. 2023, 37, 387–395. [Google Scholar] [CrossRef]
- Elwell, R.; Polikar, R. Incremental learning of concept drift in nonstationary environments. IEEE Trans. Neural Netw. 2011, 22, 1517–1531. [Google Scholar] [CrossRef]
- Chambers, L.; Gaber, M.M.; Ghomeshi, H. Deepstreamensemble: Streaming adaptation to concept drift in deep neural networks. Int. J. Mach. Learn. Cybern. 2025, 16, 3955–3976. [Google Scholar] [CrossRef]
- Museba, T.; Nelwamondo, F.; Ouahada, K.; Akinola, A. Recurrent adaptive classifier ensemble for handling recurring concept drifts. Appl. Comput. Intell. Soft Comput. 2021, 2021, 5533777. [Google Scholar] [CrossRef]
- Liu, F.T.; Ting, K.M.; Zhou, Z. Isolation-Based Anomaly Detection. ACM Trans. Knowl. Discov. Data 2012, 6, 1–39. [Google Scholar] [CrossRef]
- Pevný, T. Loda: Lightweight on-line detector of anomalies. Mach. Learn. 2016, 102, 275–304. [Google Scholar] [CrossRef]
- Bhatia, S.; Jain, A.; Srivastava, S.; Kawaguchi, K.; Hooi, B. Memstream: Memory-based streaming anomaly detection. In Proceedings of the ACM Web Conference 2022, Virtual, 25–29 April 2022; pp. 610–621. [Google Scholar]
- Yang, L.; Manias, D.M.; Shami, A. Pwpae: An ensemble framework for concept drift adaptation in iot data streams. In Proceedings of the 2021 IEEE Global Communications Conference (GLOBECOM), Madrid, Spain, 7–11 December 2021; pp. 1–6. [Google Scholar]
- Bayram, F.; Ahmed, B.S.; Kassler, A. From concept drift to model degradation: An overview on performance-aware drift detectors. Knowl.-Based Syst. 2022, 245, 108632. [Google Scholar] [CrossRef]
- Agrahari, S.; Singh, A.K. Concept drift detection in data stream mining: A literature review. J. King Saud-Univ.-Comput. Inf. Sci. 2022, 34, 9523–9540. [Google Scholar] [CrossRef]
- Li, D.; Nair, N.; Kevin, I.; Wang, K.; Sakurai, K. Ensemble based unsupervised anomaly detection with concept drift adaptation for time series data. In Proceedings of the 2024 IEEE Smart World Congress (SWC), Nadi, Fiji, 2–7 December 2024; IEEE: Piscataway, NJ, USA, 2024; pp. 434–441. [Google Scholar]
- Li, Z.; Zhao, Y.; Hu, X.; Botta, N.; Ionescu, C.; Chen, G. Ecod: Unsupervised outlier detection using empirical cumulative distribution functions. IEEE Trans. Knowl. Data Eng. 2022, 35, 12181–12193. [Google Scholar] [CrossRef]
- Lei, T.; Gong, C.; Chen, G.; Ou, M.; Yang, K.; Li, J. A novel unsupervised framework for time series data anomaly detection via spectrum decomposition. Knowl.-Based Syst. 2023, 280, 111002. [Google Scholar] [CrossRef]
- Lin, S.; Clark, R.; Birke, R.; Schönborn, S.; Trigoni, N.; Roberts, S. Anomaly detection for time series using vae-lstm hybrid model. In Proceedings of the ICASSP 2020—2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Barcelona, Spain, 4–8 May 2020; IEEE: Piscataway, NJ, USA, 2020; pp. 4322–4326. [Google Scholar]
- Lai, K.H.; Wang, L.; Chen, H.; Zhou, K.; Wang, F.; Yang, H.; Hu, X. Context-aware domain adaptation for time series anomaly detection. In Proceedings of the 2023 SIAM International Conference on Data Mining (SDM), Paul Twin Cities, MN, USA, 27–29 April 2023; pp. 676–684. [Google Scholar]
- Gama, J.; Medas, P.; Castillo, G.; Rodrigues, P. Learning with drift detection. In Proceedings of the Advances in Artificial Intelligence–SBIA 2004: 17th Brazilian Symposium on Artificial Intelligence, Sao Luis, Brazil, 29 September–1 October 2004; Springer: Berlin/Heidelberg, Germany, 2004. Proceedings 17. pp. 286–295. [Google Scholar]
- Bifet, A.; Gavalda, R. Learning from time-changing data with adaptive windowing. In Proceedings of the 2007 SIAM International Conference on Data Mining, Minneapolis, MN, USA, 26–28 April 2007; pp. 443–448. [Google Scholar]
- Li, B.; Gupta, S.; Müller, E. State-transition-aware anomaly detection under concept drifts. Data Knowl. Eng. 2024, 154, 102365. [Google Scholar] [CrossRef]
- Gama, J.; Žliobaitė, I.; Bifet, A.; Pechenizkiy, M.; Bouchachia, A. A survey on concept drift adaptation. ACM Comput. Surv. (CSUR) 2014, 46, 1–37. [Google Scholar] [CrossRef]
- Berger, V.W.; Zhou, Y. Kolmogorov–smirnov test: Overview. In Wiley Statsref: Statistics Reference Online; Wiley: Cambridge, MA, USA, 2014. [Google Scholar]
- Joyce, J.M. Kullback-Leibler Divergence. In International Encyclopedia of Statistical Science; Lovric, M., Ed.; Springer: Berlin, Germany, 2011; pp. 720–722. [Google Scholar] [CrossRef]
- MacFarland, T.W.; Yates, J.M. Mann–Whitney U Test. In Introduction to Nonparametric Statistics for the Biological Sciences Using R; Springer: Cham, Switzerland, 2016; pp. 103–132. [Google Scholar] [CrossRef]
- Goldenberg, I.; Webb, G.I. Survey of distance measures for quantifying concept drift and shift in numeric data. Knowl. Inf. Syst. 2019, 60, 591–615. [Google Scholar] [CrossRef]
- Cerqueira, V.; Gomes, H.M.; Bifet, A.; Torgo, L. STUDD: A student–teacher method for unsupervised concept drift detection. Mach. Learn. 2023, 112, 4351–4378. [Google Scholar] [CrossRef]
- Kolter, J.Z.; Maloof, M.A. Dynamic weighted majority: An ensemble method for drifting concepts. J. Mach. Learn. Res. 2007, 8, 2755–2790. [Google Scholar]
- Korycki, L.; Krawczyk, B. Unsupervised drift detector ensembles for data stream mining. In Proceedings of the 2019 IEEE International Conference on Data Science and Advanced Analytics (DSAA), Washington, DC, USA, 5–8 October 2019; IEEE: Piscataway, NJ, USA, 2019; pp. 317–325. [Google Scholar]
- Laptev, N.; Amizadeh, S. Yahoo Anomaly Detection Dataset S5. 2015. Available online: https://www.researchgate.net/publication/336315311_Unsupervised_Drift_Detector_Ensembles_for_Data_Stream_Mining (accessed on 6 May 2025).
- Lavin, A.; Ahmad, S. Evaluating real-time anomaly detection algorithms–the Numenta anomaly benchmark. In Proceedings of the 2015 IEEE 14th International Conference on Machine Learning and Applications (ICMLA), Miami, FL, USA, 9–11 December 2015; IEEE: Piscataway, NJ, USA, 2015; pp. 38–44. [Google Scholar]
- Jamshidi, E.J.; Yusup, Y.; Kayode, J.S.; Kamaruddin, M.A. Detecting outliers in a univariate time series dataset using unsupervised combined statistical methods: A case study on surface water temperature. Ecol. Inform. 2022, 69, 101672. [Google Scholar] [CrossRef]
- Ahmad, S.; Lavin, A.; Purdy, S.; Agha, Z. Unsupervised real-time anomaly detection for streaming data. Neurocomputing 2017, 262, 134–147. [Google Scholar] [CrossRef]
- Gu, M.; Fei, J.; Sun, S. Online anomaly detection with sparse Gaussian processes. Neurocomputing 2020, 403, 383–399. [Google Scholar] [CrossRef]
- Vázquez, F.I.; Hartl, A.; Zseby, T.; Zimek, A. Anomaly detection in streaming data: A comparison and evaluation study. Expert Syst. Appl. 2023, 233, 120994. [Google Scholar] [CrossRef]
- Ding, C.; Zhao, J.; Sun, S. Concept drift adaptation for time series anomaly detection via transformer. Neural Process. Lett. 2023, 55, 2081–2101. [Google Scholar] [CrossRef]
Method | Type | Drift-Aware | Incremental | Description |
---|---|---|---|---|
ECOD [16] | Offline | × | × | Static copula-based method. Common unsupervised benchmark. |
LODA [10] | Semi-Online, Ensemble | × Robust to minor drift | × | Random projection ensemble. Passive robustness to minor drift. |
MemStream [11] | Online | ✓ Partial | ✓ Partial | Memory-based streaming model. Learns from recent data. |
LSTM-VAE [18] | Deep Learning | × | × | LSTM-based variational autoencoder. Requires offline training. Lacks adaptation to drift. |
STAD [22] | Deep Learning | ✓ | × | Autoencoding with drift detection in latent space via KS and AD tests. No performance based drift evaluation. |
AEADF [15] | Online, Ensemble | ✓ Partial | ✓ Partial | Fixed-update hybrid drift. detector |
ADDAEIL (Proposed) | Online, Ensemble | ✓ | ✓ | Hybrid drift detection with incremental adaptive updates. |
Dataset | ECOD | LODA | MemStream | |||||||||
AUC | F1 | Recall | Precision | AUC | F1 | Recall | Precision | AUC | F1 | Recall | Precision | |
Yahoo | 0.66 | 0.52 | 0.79 | 0.60 | 0.54 | 0.18 | 0.53 | 0.16 | 0.85 | 0.075 | 0.064 | 0.11 |
NAB | 0.80 | 0.59 | 0.93 | 0.55 | 0.78 | 0.38 | 0.79 | 0.34 | 0.84 | 0.0023 | 0.004 | 0.016 |
CEMACS | 0.91 | 0.82 | 0.81 | 0.92 | 0.90 | 0.78 | 0.75 | 0.93 | 0.84 | 0.73 | 0.86 | 0.73 |
Average | 0.79 | 0.64 | 0.84 | 0.69 | 0.74 | 0.45 | 0.69 | 0.48 | 0.84 | 0.27 | 0.31 | 0.29 |
Dataset | LSTM-VAE | STAD | ||||||||||
AUC | F1 | Recall | Precision | AUC | F1 | Recall | Precision | |||||
Yahoo | 0.62 | 0.53 | 0.79 | 0.52 | 0.64 | 0.52 | 0.77 | 0.51 | ||||
NAB | 0.81 | 0.62 | 0.97 | 0.58 | 0.76 | 0.57 | 0.93 | 0.54 | ||||
CEMACS | 0.92 | 0.88 | 0.88 | 0.93 | 0.91 | 0.88 | 0.86 | 0.96 | ||||
Average | 0.97 | 0.68 | 0.88 | 0.68 | 0.77 | 0.66 | 0.65 | 0.67 | ||||
Dataset | AEADF | ADDAEIL (Proposed) | ||||||||||
AUC | F1 | Recall | Precision | AUC | F1 | Recall | Precision | |||||
Yahoo | 0.82 | 0.58 | 0.95 | 0.54 | 0.92 | 0.77 | 0.98 | 0.75 | ||||
NAB | 0.84 | 0.55 | 0.95 | 0.48 | 0.85 | 0.61 | 0.95 | 0.58 | ||||
CEMACS | 0.93 | 0.88 | 0.87 | 0.94 | 0.94 | 0.90 | 0.92 | 0.92 | ||||
Average | 0.86 | 0.67 | 0.92 | 0.65 | 0.90 | 0.76 | 0.95 | 0.75 |
Dataset | iForest (No Update) | Performance-only | Distribution-only | |||||||||
AUC | F1 | Recall | Precision | AUC | F1 | Recall | Precision | AUC | F1 | Recall | Precision | |
Yahoo | 0.62 | 0.31 | 0.64 | 0.29 | 0.91 | 0.74 | 0.98 | 0.72 | 0.89 | 0.71 | 0.98 | 0.68 |
NAB | 0.83 | 0.47 | 0.95 | 0.41 | 0.84 | 0.59 | 0.95 | 0.55 | 0.85 | 0.60 | 0.94 | 0.57 |
CEMACS | 0.92 | 0.89 | 0.88 | 0.85 | 0.94 | 0.89 | 0.91 | 0.92 | 0.94 | 0.90 | 0.91 | 0.93 |
Average | 0.79 | 0.56 | 0.82 | 0.52 | 0.90 | 0.74 | 0.95 | 0.73 | 0.89 | 0.74 | 0.94 | 0.73 |
Dataset | ADDAEIL (Proposed) | |||||||||||
AUC | F1 | Recall | Precision | |||||||||
Yahoo | 0.92 | 0.77 | 0.98 | 0.75 | ||||||||
NAB | 0.85 | 0.61 | 0.95 | 0.58 | ||||||||
CEMACS | 0.94 | 0.90 | 0.92 | 0.92 | ||||||||
Average | 0.90 | 0.76 | 0.95 | 0.75 |
Parameter Setting | AUC | F1-Score | Recall | Precision |
---|---|---|---|---|
Default (baseline) | 0.92 | 0.77 | 0.98 | 0.75 |
Varying Window Size W | ||||
0.92 | 0.80 | 0.99 | 0.77 | |
0.91 | 0.71 | 0.94 | 0.71 | |
Varying Number of Base Detectors N | ||||
0.92 | 0.77 | 0.98 | 0.75 | |
0.92 | 0.77 | 0.98 | 0.75 | |
Varying Update Ratio Range | ||||
0.92 | 0.76 | 0.98 | 0.74 | |
0.93 | 0.79 | 0.98 | 0.77 |
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content. |
© 2025 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).
Share and Cite
Li, D.; Nair, N.-K.C.; Wang, K.I.-K. ADDAEIL: Anomaly Detection with Drift-Aware Ensemble-Based Incremental Learning. Algorithms 2025, 18, 359. https://doi.org/10.3390/a18060359
Li D, Nair N-KC, Wang KI-K. ADDAEIL: Anomaly Detection with Drift-Aware Ensemble-Based Incremental Learning. Algorithms. 2025; 18(6):359. https://doi.org/10.3390/a18060359
Chicago/Turabian StyleLi, Danlei, Nirmal-Kumar C. Nair, and Kevin I-Kai Wang. 2025. "ADDAEIL: Anomaly Detection with Drift-Aware Ensemble-Based Incremental Learning" Algorithms 18, no. 6: 359. https://doi.org/10.3390/a18060359
APA StyleLi, D., Nair, N.-K. C., & Wang, K. I.-K. (2025). ADDAEIL: Anomaly Detection with Drift-Aware Ensemble-Based Incremental Learning. Algorithms, 18(6), 359. https://doi.org/10.3390/a18060359