Automated Cyber Threat Intelligence Extraction from Distributed Honeypots: A Hybrid Machine Learning Approach

1. Summary

Thank you very much for taking the time to review this manuscript. Please find the detailed responses below.

2. Point-by-point response to Comments and Suggestions for Authors

Comment 1: Contradiction Between “Big Data” Claims and Actual Dataset Size: The authors repeatedly state that the core problem their framework addresses is the “Big Data” challenge inherent in modern CTI ecosystems. However, their proprietary dataset consists of only 30,000 security events collected over a short period of six weeks. Furthermore, the data originates from just four honeypot agents deployed in only two regions (the US and Saudi Arabia). This limited scope is insufficient to represent a true “Big Data” environment, which severely limits the generalizability of their results.

Response 1: We thank the reviewer for this important observation and agree that the term “Big Data” was applied too generally and it’s not reflecting the number of security events collected in this paper.

In the revised manuscript, we have replaced broad references to “Big Data” with more accurate language such as “high-volume alert streams”. The event/alert was collected from security lab which was built by the university.

Comment 2: Over-reliance on Aggressive Random Under-sampling: The study notes that the threat stream is highly imbalanced, with over 95% of events being authentication attacks targeting SSH services. Discarding up to 95% of the real-world telemetry means throwing away a massive amount of contextual background noise. Consequently, evaluating the models on this artificially balanced data likely inflates the performance metrics and fails to prove the framework’s effectiveness in a real, heavily imbalanced SOC environment.

Response 2: We agree with this concern. Random under-sampling was necessary to allow the supervised classifier to learn decision boundaries across all attack technique classes; without balancing, minority classes would be effectively invisible to the model. To address the reviewer's comments, we have expanded the Limitations and Future Work section (Section 6.5) in the revised manuscript to transparently acknowledge this limitation.

One limitation of this study is the use of random under-sampling to address severe class imbalance, as SSH authentication attacks accounted for more than 95% of the collected telemetry. While this approach was necessary to ensure adequate representation of minority attack classes, it reduced the preservation of real-world traffic distributions and may have resulted in optimistic performance estimates. In practical deployments, high-volume SSH brute-force traffic can be filtered prior to classification, allowing the model to focus on more diverse threat activities. Future work will investigate cost-sensitive learning and synthetic over-sampling techniques, such as SMOTE-NC, to better preserve real-world traffic characteristics while maintaining minority-class detection performance.

Comment 3: Inconsistent Feature Configurations Across Datasets: The researchers used a different set of features to evaluate Dataset 3 compared to Datasets 1 and 2. Specifically, they removed the geographical metadata and used only the “Contextual” configuration for Dataset 3 because they observed a “homogeneous source-country distribution” during preprocessing. Altering the methodology and feature inputs post-hoc compromises the scientific validity of cross-dataset performance comparisons.

Response 3:

We thank the reviewer for this important feedback. We would like to clarify a key point: Datasets 1, 2, and 3 are structurally identical, comprising the same features and data format, and differ only in their temporal collection window (Phase 1: May 24–26; Phase 2: June 1–2; Phase 3: July 1–2, 2025).

All three datasets contain the full set of available features, including geographical metadata. The decision to apply the Contextual configuration (without Geo) to Dataset 3 was not a structural limitation but a data-driven preprocessing decision made prior to model training. Specifically, a distribution analysis of Dataset 3 revealed that attack events during the July 1–2 window originated predominantly from a single geographic region, resulting in near-zero variance in the geo-encoded country feature. Under these conditions, retaining the Geo feature would introduce noise rather than discriminative signal, which is contrary to sound feature engineering practice.

This decision was therefore driven entirely by the statistical properties of that specific temporal window, and not by any attempt to optimize cross-dataset performance. To improve transparency, we have revised Section 5.1 of the manuscript to make this rationale explicit and to reinforce the caveat that this configuration difference should be considered when interpreting cross-dataset comparisons.

Dataset 3 was evaluated using the Contextual configuration, while Datasets 1 and 2 used the Contextual + Geo configuration. This decision was based on preprocessing analysis, which revealed limited variation in the source-country distribution of Dataset 3. Consequently, geographical metadata was excluded from the final model configuration. This difference should be considered when interpreting cross-dataset performance comparisons.

Revised Text: All three datasets are structurally identical and share the same features and data formats, differing only in their temporal collection periods. The contextual configuration (no geographic metadata) applied to Dataset 3 was not due to structural constraints but was based on a preprocessing distribution analysis performed prior to model training. Attack events during the July 1–2 period (Dataset 3) occurred primarily in a single geographic region, resulting in near-zero differences in geographically encoded country features. Since maintaining near-zero variance features generates noise rather than identifiable signals, this exclusion follows standard feature engineering practices; nevertheless, these configuration differences must be considered when interpreting cross-dataset performance comparisons.

Comment 4: Limited Baseline Comparisons: The XGBoost classifier is only compared against Random Forest, and HDBSCAN is only compared against K-means. The authors fail to benchmark their hybrid framework against any modern, state-of-the-art techniques discussed in their literature review.

Response 4: We thank the reviewer for this suggestion. In the revised manuscript, LightGBM and CatBoost have been added as additional gradient boosting baselines, evaluated under identical data splits and feature configurations as XGBoost and Random Forest. The hyperparameter configurations for all models are summarized in the updated Table 2. The extended performance comparison including all four models with 95% Bootstrap Confidence Intervals is presented in the new Table 4. All three gradient boosting models consistently outperform Random Forest across all datasets and metrics. No single gradient boosting model dominates across all configurations: CatBoost achieves the highest Balanced Accuracy across all three datasets (0.7895, 0.8178, 0.7693), while XGBoost leads in F1-Macro in Dataset 2 (0.7183) and LightGBM performs best in Dataset 3 across multiple metrics. These findings confirm that classifier performance is dataset-dependent, and practitioners are encouraged to select the most suitable model based on their operational environment.

Revised text: Table 2 has been updated to include hyperparameter configurations for all supervised models (XGBoost, LightGBM, CatBoost) and HDBSCAN. Table 4 has been added to Section 5.1 presenting the extended performance comparison of all four models with 95% Bootstrap CIs. Section 4.4.1 has been revised as follows: "To rigorously validate the supervised classification track, we conducted a comparative analysis across four models: XGBoost, LightGBM, CatBoost, and Random Forest. Random Forest represents a widely used bagging-based ensemble baseline in cybersecurity, while XGBoost, LightGBM, and CatBoost are state-of-the-art gradient boosting frameworks. As shown in Table 4, all three gradient boosting models consistently outperform Random Forest across all datasets and metrics. However, no single gradient boosting model dominates across all configurations, indicating that classifier selection should be tailored to the specific dataset and SOC environment. XGBoost was used as the primary classifier in this study's pipeline; however, the framework is designed to accommodate any of the evaluated gradient boosting models."

Comment 5: Unverified IP-to-Location Ground Truth: The framework relies heavily on geographical metadata derived from IP addresses. The authors admit that IP-based geolocation can be easily masked by attackers using VPNs or proxies, yet the methodology does not implement any validation mechanism, weakening claims about accurate threat attribution.

Response 5: We fully agree with this limitation. In the revised manuscript, we have: (1) reframed geographical metadata throughout as a probabilistic contextual signal rather than a ground-truth attribution mechanism; (2) added a clear discussion in Section 6.5 recommending that geo-attribution be treated as one weighted feature within a broader multi-signal risk-scoring system, complemented by AS reputation data and behavioral consistency checks; and (3) removed language implying definitive geographic attack attribution.

“Revised text in Section 6.5: Geographical metadata derived from IP-to-location mapping should be treated as a probabilistic contextual signal rather than a definitive attribution mechanism. Future iterations should integrate AS reputation scoring and cross-session behavioral fingerprinting as secondary validation signals.”

English Comments : Missing Definite Article: In the caption for Figure 1, the text states, "...illustrating process of supervised classification...".  It is missing an article before the noun phrase. It should be corrected to "...illustrating the process of supervised classification...".

Uncountable Noun Modifier Error: The document mentions agents deployed "to collect various threat intelligence". Because "intelligence" is an uncountable noun, using the adjective "various" directly before it is grammatically awkward. It would be more accurate to write "...collect various types of threat intelligence".

Awkward Phrasing / Missing Prepositions: The sentence "...finding ML techniques to be effective with satisfactory survey" is grammatically incomplete and awkwardly phrased. It should likely be revised to "...effective in a satisfactory survey" or "...effective according to a satisfactory survey".

Improper Capitalization: The text notes, "Since the Dataset consists of three distinct temporal phases...". The word "Dataset" is a common noun in this context and should not be capitalized mid-sentence. It should be corrected to "dataset".

Lexical Collocation (Word Choice) Error: The phrase "The advantages organizations can collect by setting up a honeypot..." uses an unnatural verb. In English, organizations do not "collect" advantages. A grammatically sound correction would be, "The advantages organizations can gain..." or "...can derive...".

Typographical Error: When naming the agents in the US Zone, one is spelled as "deepkake_v2_us". Given that the corresponding agent in the Saudi Arabia Zone is properly named "deepfake_v1", "deepkake" is almost certainly a misspelling of "deepfake".

Response English Corrections: We thank the reviewer for the careful language review. All six corrections have been implemented in the revised manuscript: (1) added missing article "the" in Figure 1 caption; (2) corrected "various threat intelligence" to "various types of threat intelligence"; (3) revised awkward phrasing in the unsupervised learning section; (4) corrected improper capitalization of "Dataset" to "dataset"; (5) replaced "collect" with "gain" in the honeypot advantages sentence; (6) corrected typographical error "deepkake" to "deepfake."

1. Summary

Thank you very much for your detailed and rigorous review. Your comments have helped us identify several important inaccuracies and methodological gaps. Please find our point-by-point responses below.

2. Point-by-point response to Comments and Suggestions for Authors

Comment 1: All the references to [5] are false:

1.         “the MITRE ATT&CK technique (rule.mitre.technique) [5].”. MITRE is not mentioned in [5];

2.         “This hybrid architecture …[5].”. The term “hybrid” is not used in [5];

3.         And so other references to [5].

Response 1: We sincerely thank the reviewer for pointing out the citation errors. Upon close review, we confirmed that Landauer et al. [5] was incorrectly cited in several parts where the content did not support the claims of this paper.

Specifically, we have made the following corrections: (1) The part cited with "rule.mitre.technique" has been replaced with [36], which is a direct reference to the MITRE ATT&CK framework. (2) The part cited to support the description of the hybrid architecture has been removed, as Landauer et al. [5] does not use the term "hybrid" in that context. All remaining citations for [5] have been verified and retained only if they are directly related to the system log clustering methodology.

We have carefully reviewed all citations once again to ensure consistency between the cited content and the claims throughout the revised manuscript.

Comment 2: “Existing studies on CTI have largely focused on classifying malware types or identifying MITRE ATT&CK tactics and techniques.”. I do not understand why have you joined “MITRE ATT&CK tactics and techniques”, since you did not mentioned MITRE in the review of related work. It means that your statement is not correct.

Response 2: We thank the reviewer for identifying this inconsistency. We agree that the original statement was not fully aligned with the scope of the literature reviewed in this study, as MITRE ATT&CK-based classification was not specifically examined in the related work section. To improve consistency and accuracy, we have revised the sentence as follows:

“Revised text in Section 1.1: Existing CTI research has primarily focused on classifying malware types or identifying known attack signatures using supervised learning methods, paying limited attention to identifying behavioral attribution and clustering geographical sources

Comment 3: “increases [3].” You are advised always to place a reference number alongside author names, e.g., “Montasari et al. [3]”.

Response 3: We have updated all in-text citations throughout the revised manuscript to include the corresponding author name(s) alongside the reference number.

“Revised example: ...as Montasari et al. [3] emphasize, without automated mechanisms to cluster and prioritize threats, the window of exposure to cyber risks significantly increases.”

Comment 4: “The primary research question this paper aims to answer.” If you declare “the primary research question”, what is the secondary research question?

Response 4: We agree with this observation. The term "primary" implies the existence of secondary research questions, which are not defined in this study. In the revised manuscript, we have replaced "primary research question" with "central research question" to more accurately reflect the scope of the study.

Comment 5: “We contribute a proprietary dataset.” If you do not make this dataset public, it is not a contribution.

Response 5: We acknowledge this concern. The original framing of "proprietary dataset" as a standalone contribution was misleading, as the dataset is not publicly released. In the revised manuscript, we have reframed this contribution as follows:

"Unlike studies relying on synthetic or outdated traffic, this study validates the proposed framework using a dataset collected directly from a distributed honeypot infrastructure, capturing authentic adversarial behaviors and diverse attack vectors from live threat landscapes. The dataset is available on request from the corresponding author, subject to applicable data sharing agreements and institutional data governance policies.

Comment 6: The methodology reference [29] is false, since the term “structured process” is not used in [29] and the process in [29] employs different steps.

Response 6: We agree with the reviewer's observation. Upon review, [29] was incorrectly applied to describe our methodology pipeline, as the steps in that work differ from those employed in this study and the term "structured process" does not appear in that reference.

In the revised manuscript, we have removed this citation from the sentence, as the description refers to our own proposed methodology and does not require external references.

Comment 7: Literature review is missing a review of research works on investigation of honeypots, which is the key term of the manuscript. Just two very outdated publications on honeypots are cited in the subsection “Honeypots”.

Response 7:

We thank the reviewer for this important observation. We agree that the honeypot subsection relied heavily on outdated references. In the revised manuscript, we have substantially updated Section 2.1.4 by incorporating recent honeypot research. Specifically, Franco et al. [13] conducted a comprehensive survey of honeypots and honeynets across IoT, Industrial IoT, and cyber-physical systems, demonstrating their critical role in modern threat intelligence gathering. Additionally, Vetterl and Clayton [14] proposed a virtual honeypot framework for capturing CPE and IoT attacks, highlighting the evolving role of honeypots in modern threat landscapes. These additions provide a more current and comprehensive foundation that better reflects the state of honeypot research and supports the data collection methodology employed in this study. More recently, El Kouari et al. [15] proposed a robust IIoT cybersecurity architecture integrating vertical honeypots across all Industry 4.0 levels with Wazuh for log transmission and CTI integration, further demonstrating the expanding role of honeypots in modern industrial threat intelligence environments."

Comment 8: “We have selected XGBoost as our primary supervised learning algorithm due to its superior” – you have to provide a reference for such a statement.

Response 8: We agree that this statement required a supporting reference. In the revised manuscript, we have added a citation to the original XGBoost paper by Chen and Guestrin [33], which introduced the XGBoost framework and demonstrated its superior scalability and performance on high-dimensional tabular datasets. This citation substantiates our selection of XGBoost as the primary supervised learning algorithm.

Comment 9: STIX format for CTI sharing is closely related to CTI. Why is it left without attention?

Response 9:

We thank the reviewer for raising this important point. STIX (Structured Threat Information eXpression) is indeed a critical standard for CTI sharing. In the revised manuscript, we have introduced STIX in the Introduction, acknowledging its role in enabling the sharing of threat information in standardized, machine-readable formats. However, as the primary focus of this study is on automated threat extraction and classification using hybrid machine learning, a full implementation of STIX-formatted output falls outside the scope of the current work. Accordingly, we have added STIX integration as a future research direction in the Conclusion, where we note that extracted IoCs and cluster intelligence could be serialized into STIX-formatted bundles for seamless integration with threat sharing platforms such as MISP or OpenCTI.

Added text in future research direction:

STIX Integration: Future work will explore ways to serialize extracted IoC and cluster intelligence into STIX format bundles to enable seamless integration with threat sharing platforms such as MISP or OpenCTI.

Comment 10: “While previous studies often relied on simpler models like KNN” – it is not true. The performed review considers works using many different classifiers.

Response 10: We agree. Our literature review cites studies employing Random Forest, ANN, SVM, Logistic Regression, and Decision Trees, making the original statement inaccurate. In the revised manuscript, we have rewritten this sentence:

“Revised text in Section 2.3: Previous studies have utilized various classifiers such as KNN, Random Forest, SVM, and ANN, but there is still a lack of systematic evaluation of gradient boosting methods for CTI log classification under conditions of severe class imbalance.”

Comment 11: “that gradient boosting architectures offer better generalization” – How many comparisons have you made to declare such a conclusion? Just with Random Forest, it is not enough.

Response 11: We agree that the original statement overgeneralized beyond the scope of our comparison. In the revised manuscript, we have removed the claim that "gradient boosting architectures offer better generalization" and instead reframed this as a gap in the literature: prior studies have utilized various classifiers such as KNN, Random Forest, SVM, and ANN, but a systematic evaluation of gradient boosting methods specifically for CTI log classification under severe class imbalance conditions remains lacking. This framing more accurately reflects the motivation for our study without making unsupported comparative claims.

Comment 12: “By transitioning to a dynamic dataset” – Why do you call the dataset dynamic?

Response 12: We agree that “dynamic” was unclear and potentially misleading. The dataset is not dynamically updated in real-time; rather, it was collected from a live threat stream. In the revised manuscript we have corrected this:

“Revised text in Section 3: By utilizing this live-capture dataset, rather than relying on synthetic or pre-processed benchmark datasets,”

Comment 13: “4.4. Phase 3: Hybrid” – Why did phases 1 and 2 deserve no attention?

Response 13: Phases 1 and 2 were indeed present in the original manuscript as Section 4.1 (Data Pre-processing and Sampling) and Section 4.2 (Hybrid Feature Engineering and Encoding), but were not explicitly labeled with phase numbers.

In the revised manuscript, we have renamed these subsections to

"Phase 1: Data Pre-processing and Sampling" and "Phase 2: Hybrid Feature Engineering and Encoding" respectively, ensuring consistency with the Phase 3 label already applied to Section 4.4.

Comment 14: “HDBSCAN can discover clusters of varying shapes and sizes while explicitly identifying low-density noise points [24].” Wrong reference. [24] is devoted to K-means.

Response 14: Upon review, cite [24] is indeed devoted to K-means clustering and was incorrectly applied to describe HDBSCAN.

In the revised manuscript, we have replaced this citation with the original HDBSCAN paper by Campello et al. [37], which introduced the hierarchical density-based clustering algorithm and accurately supports the described properties of HDBSCAN.

Comment 15: “controlled experiments” – What specifics do you assign to the term “controlled”, since experiments are always controlled?

Response 15: We agree that the term "controlled" was vague. In the revised manuscript, we have replaced it with

"54 systematic experiments conducted under identical data splits,"

which more precisely reflects that the comparability of results was ensured through consistent experimental conditions including fixed train-test splits.

Comment 16: How did you get the values in Table 2? If averages between iterations – you have to calculate confidence intervals for all metrics and both classifiers.

Response 16: We thank the reviewer for this important methodological observation. To clarify, the "54 iterations" mentioned in the manuscript referred to the systematic exploration of different combinations (3 datasets × 3 feature configurations × 3 classification targets × 2 models) during the initial evaluation phase, rather than repeated runs of the same configuration for averaging.

Therefore, the performance values reported are not averages across iterations. Instead, the final performance metrics are derived from a single stratified 80/20 test split. To address your valid concern regarding statistical reliability, we have computed 95% Bootstrap Confidence Intervals (1,000 resamples of the test set predictions, random_state=42) for all metrics across all four evaluated models (Random Forest, XGBoost, LightGBM, and CatBoost). This ensures that the single-split evaluation is statistically robust.

Revised text: The methodology in Section 5.1.1 has been refined to explicitly distinguish between the exploratory combinatorial experiments and the final single-split bootstrap validation. Table 4 has been added to Section 5.1 with the following note: "† 95% Bootstrap Confidence Intervals (1,000 resamples, random_state=42). Dataset 1: RF BAcc [0.5935, 0.6743], XGBoost BAcc [0.7348, 0.7883], LightGBM BAcc [0.7262, 0.7878], CatBoost BAcc [0.7719, 0.8057]. Dataset 2: RF BAcc [0.6070, 0.6992], XGBoost BAcc [0.7085, 0.8267], LightGBM BAcc [0.5959, 0.8005], CatBoost BAcc [0.7147, 0.8858]. Dataset 3: RF BAcc [0.5360, 0.6347], XGBoost BAcc [0.6710, 0.7615], LightGBM BAcc [0.6770, 0.7663], CatBoost BAcc [0.7330, 0.7907]. Full CI values for all metrics are available upon request."

Comment 17: What tool have you used for feature importance ranking?

Response 17: Feature importance ranking was conducted using XGBoost's scikit-learn compatible feature_importances attribute, which derives each feature's importance score based on its contribution across all decision tree splits within the trained XGBoost model. This method was selected for its computational efficiency and direct interpretability. The revised manuscript explicitly states the tool used in Section 5.1 to ensure reproducibility.

Comment 18: “9 seconds” – If you provide timing, you have to provide characteristics of your computer.

Response 18: We thank the reviewer for this observation. We fully agree that specific timing values are inherently dependent on numerous variables—including hardware configuration, software environment, dataset size, data structure, and runtime conditions such as memory and CPU load—making precise figures difficult to generalize across deployment environments. Accordingly, in the revised manuscript, we have replaced specific timing claims in the text with qualitative expressions that convey operational feasibility without overstating precision (e.g., "sub-second", "sub-minute"). However, as Tables 5 and 6 retain empirically measured execution times for transparency and reproducibility, we have added the hardware and software specifications used in our experiments to Section 5.1.2. All experiments were conducted on a system equipped with an Intel Core i7 CPU, 16 GB RAM, running Windows 11 (64-bit), without GPU acceleration, using Python 3.11.7 (Anaconda distribution). These specifications are provided solely as a reference baseline; readers are advised that absolute timing values may vary across different environments, while the sub-minute feasibility claim is expected to hold for any comparable modern computing setup.

Revised text : The operational feasibility of the proposed hybrid framework was validated through inference latency measurements to assess its suitability for high-velocity, large-scale data environments. All experiments were conducted on a system equipped with an Intel Core i7 CPU, 16 GB RAM, running Windows 11 (64-bit), without GPU acceleration, using Python 3.11.7 (Anaconda distribution). In a distributed threat stream environment, minimizing processing time is as critical as classification accuracy for proactive defense. All three gradient boosting classifiers — XGBoost, LightGBM, and CatBoost — achieved sub-second inference latency per feature configuration across all three datasets, with CatBoost demonstrating the fastest inference, followed by XGBoost and LightGBM. All models remained well within the sub-minute operational threshold required for real-time SOC environments. Given the high-throughput nature of modern SIEM logs, this confirms that any of the evaluated gradient boosting classifiers can be integrated into live SIEM pipelines to provide near-real-time triage, significantly reducing the mean time to detect (MTTD) and allowing SOC analysts to prioritize critical alerts without introducing significant latency to the security pipeline.

Comment 19: “XGBoost, through its gradient boosting mechanism, effectively captures minority class patterns that traditional models like Random Forest might overlook.” XGBoost is a traditional model, as well, but a traditional boosting model.

Response 19: We agree with the reviewer's observation. XGBoost is indeed a traditional boosting model, and characterizing Random Forest as a "traditional model" in contrast to XGBoost was inaccurate. In the revised manuscript, we have replaced this phrasing with a more precise distinction based on algorithmic approach: "bagging-based ensemble methods such as Random Forest," which accurately reflects the fundamental difference between bagging and boosting strategies without mischaracterizing either model.

1. Summary

We sincerely thank Reviewer for their careful and constructive evaluations. Reviewer raised complementary concerns regarding experimental rigor, novelty articulation, framework integration clarity, and literature comprehensiveness. We have addressed each point in full below.

2. Point-by-point response to Comments and Suggestions for Authors

Comment 1: Clustering quality for K-means is judged solely by the Silhouette Score, which increases almost monotonically across the tested K range without exhibiting a clear criterion for optimal K selection; fixing K at 40 thus lacks statistical support.

Response 1: We thank the reviewer for this observation. We clarify that the optimal K value for each scenario was determined using the Elbow method [38], which identifies the point of diminishing returns in within-cluster sum of squares as K increases. The Silhouette Score was applied as a secondary validation metric to confirm cluster cohesion, not as the primary selection criterion. This clarification has been added to Section 5.2.2 of the revised manuscript.

Comment 2: XGBoost and HDBSCAN are presented as parallel tracks in the proposed architecture, yet the paper never clarifies how these tracks are fused, complemented, or cascaded at the decision level. For instance, should samples confidently classified as known attacks by XGBoost still enter HDBSCAN for sub-clustering?

Response 2: We sincerely thank the reviewer for this insightful and highly practical observation regarding SOC operations. We agree that in a real-world deployment, a cascaded (sequential) approach is operationally optimal.

In the context of this current study, the two tracks were designed to operate independently in parallel. This experimental design was intentionally chosen to rigorously evaluate the standalone discovery capabilities of HDBSCAN—specifically, to observe its ability to identify hidden campaigns even within high-volume alert categories, without being constrained or biased by the upstream classifier's decisions.

However, we fully agree with the reviewer that for practical SOC implementation, a cascaded architecture is highly desirable. In an operational setup, samples classified with high confidence as known attacks by the gradient boosting model (e.g., routine SSH brute-force attacks) would be filtered out or routed directly to automated mitigation. Only low-confidence, unclassified, or anomalous samples would then enter HDBSCAN for sub-clustering and zero-day discovery. This cascaded approach would significantly reduce computational overhead and maximize operational efficiency.

To address this important point, we have revised Section 4.5.2 in the manuscript to clarify the distinction between the parallel experimental design used in our evaluation and the cascaded pipeline recommended for real-world deployment. Furthermore, we have explicitly added the development of this cascaded integration pipeline to the Future Research Directions (Section 7.2).

Comment 3: The main topic of this paper is cyber threat extraction. Therefore, the latest research in the fields of cyber threat and attack (an adversarial attack detection method based on bidirectional consistency discrimination for deep learning-based soft sensors; AI-based holistic framework for cyber threat intelligence management, etc.) should be incorporated into the Introduction to enhance the objectivity, comprehensiveness and timeliness of the paper.

Response 3: We thank the reviewer for this suggestion. In the revised manuscript, we have incorporated two recent works into the Introduction: Guo et al. [4], who proposed an adversarial attack detection method based on bidirectional consistency discrimination, and Spyros et al. [5], who introduced an AI-based holistic framework for CTI management. These additions enhance the comprehensiveness and timeliness of the literature coverage.

Comment 4: The supervised experiments compare XGBoost only against Random Forest, rendering the baseline setting insufficient. Given that more advanced algorithms such as LightGBM and CatBoost exist in the gradient boosting domain, and common deep learning baselines including TabNet are omitted, the relative advantage of XGBoost lacks adequate support. The authors are advised to supplement at least two widely recognized strong baselines under identical data splits and feature configurations, reporting macro F1 and Balanced Accuracy uniformly.

Response 4: We agree that the baseline comparison was insufficient. In the revised manuscript, LightGBM and CatBoost have been added as baselines under identical data splits and feature configurations. Balanced Accuracy and F1-Macro with 95% Bootstrap CIs are reported uniformly in the new Table 4. All three gradient boosting models consistently outperform Random Forest across all datasets and metrics. No single gradient boosting model dominates across all configurations: CatBoost achieves the highest Balanced Accuracy across all three datasets (0.7895, 0.8178, 0.7693), while XGBoost leads in F1-Macro in Dataset 2 (0.7183) and LightGBM performs best in Dataset 3 across multiple metrics. These findings confirm that classifier performance is dataset-dependent, and no single model is universally optimal for all CTI environments. The proposed framework is designed to accommodate any gradient boosting classifier, and practitioners are encouraged to evaluate all models on their own SIEM data to identify the most suitable option for their operational environment.

Revised text: Table 2 has been updated to include hyperparameter configurations for all supervised models (XGBoost, LightGBM, CatBoost) and HDBSCAN. Table 4 has been added to Section 5.1 presenting the extended baseline comparison with Balanced Accuracy and F1-Macro for all four models with 95% Bootstrap CIs. Section 6.1 has been updated to reflect the dataset-dependent nature of classifier performance: "No single gradient boosting model dominates across all configurations. CatBoost achieves the highest Balanced Accuracy across all three datasets (0.7895, 0.8178, 0.7693), while XGBoost leads in F1-Macro in Dataset 2 (0.7183) and LightGBM performs best in Dataset 3 across multiple metrics. Practitioners are therefore encouraged to evaluate all gradient boosting models on their own SIEM data and select the most suitable classifier for their operational context."

1. Summary

We sincerely thank Reviewer for their careful and constructive evaluations. Both reviewers raised complementary concerns regarding experimental rigor, novelty articulation, framework integration clarity, and literature comprehensiveness. We have addressed each point in full below.

2. Point-by-point response to Comments and Suggestions for Authors

Comment 1: The paper addresses an important problem in CTI automation and the hybrid use of XGBoost and HDBSCAN is interesting and practical for SOC environments. However, the novelty compared to existing hybrid ML-based CTI frameworks should be clarified more clearly.

Response 1: We thank the reviewer for this positive assessment. We agree that novelty relative to existing hybrid frameworks was not sufficiently articulated. In the revised manuscript, we have added a dedicated novelty paragraph at the end of Section 2.3 that differentiates our framework on three dimensions: (1) cyclical temporal encoding combined with geo-contextual enrichment as a unified feature engineering stage; (2) HDBSCAN for noise-aware density-based campaign discovery without requiring a pre-specified cluster count; and (3) live-capture distributed honeypot telemetry as the evaluation environment.

“Added in Section 2.3: In contrast to existing hybrid CTI frameworks that typically combine supervised classifiers with static clustering algorithms on benchmark datasets, our approach introduces: (i) cyclical temporal encoding with geo-contextual enrichment, (ii) HDBSCAN for noise-aware density-based campaign discovery, and (iii) live-capture distributed honeypot telemetry providing authentic adversarial diversity unavailable in synthetic datasets.”

Comment 2: The feature engineering section is well designed, especially the temporal encoding and geo-contextual features. Still, converting IP addresses directly into integers may not be the best representation method and should be better justified.

Response 2: We appreciate this feedback. The rationale for integer IP conversion is: (1) it preserves subnet-level proximity; (2) it avoids the dimensionality explosion of one-hot encoding; and (3) it is computationally efficient and adopted in prior SIEM log classification studies. We acknowledge this does not capture hierarchical subnet structure and note that future work will explore AS-level embeddings.

“Added in Section 4.2: Source IP addresses are converted to their integer equivalents to preserve implicit subnet proximity in the feature space. Future work will explore AS-level embeddings as a richer network infrastructure representation.”

Comment 3: The experimental evaluation is extensive, but the paper does not provide enough implementation details for reproducibility. Important hyperparameters for XGBoost and HDBSCAN should be included.

Response 3: We agree that implementation details were insufficient for reproducibility. In the revised manuscript, we have added a dedicated hyperparameter table (Table 2. Hyperparameter Configurations for All Supervised Models and HDBSCAN) in Section 4.4.1, documenting the full configurations for both All Supervised Models and HDBSCAN used in all experiments.

Comment 4: The authors conclude that HDBSCAN outperforms K-means, but in several cases K-means achieved higher silhouette scores. This discussion should be explained more carefully.

Response 4: We agree this required more careful explanation. In the revised manuscript, we have added a clarification in Section 6.2:

"Although K-means achieves higher Silhouette Scores in Scenarios 3 and 4, this is partly an artifact of the fixed K=40, which forces over-segmentation into spherical clusters. HDBSCAN's data-driven cluster discovery and explicit noise identification carry direct operational value for SOC analysts, as they eliminate the need for a pre-specified cluster count and automatically suppress non-conforming background noise."

Comment 5: The paper is generally well written, but some sections in the discussion and conclusion are repetitive. The manuscript would benefit from more concise presentation and clearer figure explanations.

Response 5: We thank the reviewer for this observation. In the revised manuscript, we have made two improvements: (1) streamlined the Discussion by removing repetitive content between Sections 6.3 and 6.4, where noise suppression statistics were described twice; and (2) revised Figure captions (Figures 2, 4, and 5) to provide clearer and more informative descriptions, including specific metrics, model names, and key findings referenced in each visualization.

1. Summary

We sincerely thank Reviewer for their careful and constructive evaluations. Reviewer raised complementary concerns regarding experimental rigor, novelty articulation, framework integration clarity, and literature comprehensiveness. We have addressed each point in full below.

2. Point-by-point response to Comments and Suggestions for Authors

Comment 1: Clustering quality for K-means is judged solely by the Silhouette Score, which increases almost monotonically across the tested K range without exhibiting a clear criterion for optimal K selection; fixing K at 40 thus lacks statistical support.

Response 1: We thank the reviewer for this observation. We clarify that the optimal K value for each scenario was determined using the Elbow method [38], which identifies the point of diminishing returns in within-cluster sum of squares as K increases. The Silhouette Score was applied as a secondary validation metric to confirm cluster cohesion, not as the primary selection criterion. This clarification has been added to Section 5.2.2 of the revised manuscript.

Comment 2: XGBoost and HDBSCAN are presented as parallel tracks in the proposed architecture, yet the paper never clarifies how these tracks are fused, complemented, or cascaded at the decision level. For instance, should samples confidently classified as known attacks by XGBoost still enter HDBSCAN for sub-clustering?

Response 2: We sincerely thank the reviewer for this insightful and highly practical observation regarding SOC operations. We agree that in a real-world deployment, a cascaded (sequential) approach is operationally optimal.

In the context of this current study, the two tracks were designed to operate independently in parallel. This experimental design was intentionally chosen to rigorously evaluate the standalone discovery capabilities of HDBSCAN—specifically, to observe its ability to identify hidden campaigns even within high-volume alert categories, without being constrained or biased by the upstream classifier's decisions.

However, we fully agree with the reviewer that for practical SOC implementation, a cascaded architecture is highly desirable. In an operational setup, samples classified with high confidence as known attacks by the gradient boosting model (e.g., routine SSH brute-force attacks) would be filtered out or routed directly to automated mitigation. Only low-confidence, unclassified, or anomalous samples would then enter HDBSCAN for sub-clustering and zero-day discovery. This cascaded approach would significantly reduce computational overhead and maximize operational efficiency.

To address this important point, we have revised Section 4.5.2 in the manuscript to clarify the distinction between the parallel experimental design used in our evaluation and the cascaded pipeline recommended for real-world deployment. Furthermore, we have explicitly added the development of this cascaded integration pipeline to the Future Research Directions (Section 7.2).

Comment 3: The main topic of this paper is cyber threat extraction. Therefore, the latest research in the fields of cyber threat and attack (an adversarial attack detection method based on bidirectional consistency discrimination for deep learning-based soft sensors; AI-based holistic framework for cyber threat intelligence management, etc.) should be incorporated into the Introduction to enhance the objectivity, comprehensiveness and timeliness of the paper.

Response 3: We thank the reviewer for this suggestion. In the revised manuscript, we have incorporated two recent works into the Introduction: Guo et al. [4], who proposed an adversarial attack detection method based on bidirectional consistency discrimination, and Spyros et al. [5], who introduced an AI-based holistic framework for CTI management. These additions enhance the comprehensiveness and timeliness of the literature coverage.

Comment 4: The supervised experiments compare XGBoost only against Random Forest, rendering the baseline setting insufficient. Given that more advanced algorithms such as LightGBM and CatBoost exist in the gradient boosting domain, and common deep learning baselines including TabNet are omitted, the relative advantage of XGBoost lacks adequate support. The authors are advised to supplement at least two widely recognized strong baselines under identical data splits and feature configurations, reporting macro F1 and Balanced Accuracy uniformly.

Response 4: We agree that the baseline comparison was insufficient. In the revised manuscript, LightGBM and CatBoost have been added as baselines under identical data splits and feature configurations. Balanced Accuracy and F1-Macro with 95% Bootstrap CIs are reported uniformly in the new Table 4. All three gradient boosting models consistently outperform Random Forest across all datasets and metrics. No single gradient boosting model dominates across all configurations: CatBoost achieves the highest Balanced Accuracy across all three datasets (0.7895, 0.8178, 0.7693), while XGBoost leads in F1-Macro in Dataset 2 (0.7183) and LightGBM performs best in Dataset 3 across multiple metrics. These findings confirm that classifier performance is dataset-dependent, and no single model is universally optimal for all CTI environments. The proposed framework is designed to accommodate any gradient boosting classifier, and practitioners are encouraged to evaluate all models on their own SIEM data to identify the most suitable option for their operational environment.

Revised text: Table 2 has been updated to include hyperparameter configurations for all supervised models (XGBoost, LightGBM, CatBoost) and HDBSCAN. Table 4 has been added to Section 5.1 presenting the extended baseline comparison with Balanced Accuracy and F1-Macro for all four models with 95% Bootstrap CIs. Section 6.1 has been updated to reflect the dataset-dependent nature of classifier performance: "No single gradient boosting model dominates across all configurations. CatBoost achieves the highest Balanced Accuracy across all three datasets (0.7895, 0.8178, 0.7693), while XGBoost leads in F1-Macro in Dataset 2 (0.7183) and LightGBM performs best in Dataset 3 across multiple metrics. Practitioners are therefore encouraged to evaluate all gradient boosting models on their own SIEM data and select the most suitable classifier for their operational context."