Review Reports - Cross-Dataset Generalization of Deep Learning-Based Detectors for Intracranial Hemorrhage Subtype Localization on Noncontrast Head CT: A Comparative Study

Round 1

Reviewer 1 Report

Comments and Suggestions for Authors

I have thoroughly reviewed your article titled "Localization of Intracranial Hemorrhage Subtypes on Noncontrast Head CT using Transformer Architecture." While the article is generally well-structured and valuable, particularly for its comparative analysis between internal and external datasets, it contains some significant methodological and presentational shortcomings. Sample images from the dataset are not provided. Although the authors largely explain the performance degradation with “dataset characteristics,” this concept is not differentiated quantitatively or empirically. Furthermore, methodologically, model training and hyperparameter adjustments are largely left to supplementary materials, and sufficient transparency is not provided in the main text. The Swin-RT-DETR model is defined as a backbone replacement in an existing architecture, and this approach is quite common in the literature. Moreover, the reasons why the model was specifically developed with this architecture or what its advantages are compared to alternative transformer-based approaches are not discussed in detail. Although the findings are comprehensively presented in the results section, the interpretation of the statistical analyses could be more in-depth. While the interpretability analysis is partially presented in the study, this analysis remains limited. The presentation of attention maps is insufficient. Furthermore, clinical integration and real-world use cases are addressed quite superficially. These shortcomings weaken the study's claim to clinical applicability. I would also like to point out that the figure captions are excessively long. The introduction section is very weak, lacking a literature review. Many studies on this topic exist in the literature. There are too many subheadings. Table and figure explanations are absent. The conclusion is written very superficially.

Author Response

Reviewer 1:

I have thoroughly reviewed your article titled "Localization of Intracranial Hemorrhage Subtypes on Noncontrast Head CT using Transformer Architecture."

While the article is generally well-structured and valuable, particularly for its comparative analysis between internal and external datasets, it contains some significant methodological and presentational shortcomings.

Response: We appreciate this balanced assessment. We have revised the manuscript to address the methodological and presentational concerns, with particular emphasis on external validation, domain shift, metric interpretation, model rationale, and clinical limitations.

Sample images from the dataset are not provided.

Response: We have added representative sample CT images and annotations for each ICH subtype from the BHX and RSNA+ datasets in the revised figure set (Figure 4). The examples include typical subtype morphology and selected failure modes.

Although the authors largely explain the performance degradation with “dataset characteristics,” this concept is not differentiated quantitatively or empirically.

Response: Thank you for this important comment. We agree that the previous version insufficiently quantified the term “dataset characteristics.” To address this limitation, we added a new dataset-characterization analysis comparing BHX and RSNA+ in terms of subtype prevalence, normalized bounding-box size, bounding-box aspect ratio, lesion burden, annotation density, spatial distribution, width–height distribution, and representative imaging examples. These results are now presented in Figure 4 and described in the Results section. The added analysis demonstrates that BHX and RSNA+ differ not only in data source but also in annotation geometry, lesion burden, subtype distribution, and spatial localization patterns. We have revised the Discussion to clarify that the observed external performance degradation likely reflects a combination of imaging-domain shift and annotation-domain shift, rather than architecture-specific limitations alone.

Furthermore, methodologically, model training and hyperparameter adjustments are largely left to supplementary materials, and sufficient transparency is not provided in the main text.

Response: Thank you for this constructive comment. We agree that the previous version did not provide sufficient methodological transparency in the main text. To address this issue, we added a summary table of the major training hyperparameters for all evaluated models, including optimizer, learning rate and scheduler, batch size, training epochs, weight decay, loss functions, and data augmentation strategies.

The Swin-RT-DETR model is defined as a backbone replacement in an existing architecture, and this approach is quite common in the literature. Moreover, the reasons why the model was specifically developed with this architecture or what its advantages are compared to alternative transformer-based approaches are not discussed in detail.

Response: Thank you for these important comments. We agree that Swin-RT-DETR should not be overstated as a fundamentally new architecture, because it was implemented as a backbone-level modification of the RT-DETR framework.

We have revised the Methods section to describe it more precisely as a Swin Transformer–based RT-DETR variant, in which the original HGNetV2 backbone was replaced with a Swin Transformer Base backbone while the remaining RT-DETR detection framework was preserved.
We also added a clearer rationale for this architectural choice. Swin Transformer was selected because its hierarchical representation and shifted-window self-attention mechanism can provide multiscale contextual feature extraction, which is relevant for ICH subtypes with heterogeneous size, morphology, and spatial distribution. RT-DETR was selected because it supports efficient end-to-end object detection using multiscale encoding and Transformer-based object queries. The purpose of including Swin-RT-DETR was therefore to evaluate the potential contribution of Transformer-based hierarchical feature extraction to ICH localization, rather than to claim a completely novel detector design.
We further revised the Discussion to acknowledge that this study did not exhaustively compare all Transformer-based detectors, and that alternative Transformer architectures should be evaluated in future work.

Although the findings are comprehensively presented in the results section, the interpretation of the statistical analyses could be more in-depth.

Response: Thank you for this constructive comment. We agree that the previous version reported the statistical comparisons but did not sufficiently interpret their implications.

In the revised Results and Discussion sections, we expanded the statistical interpretation at both the image and patient levels. Specifically, we now emphasize that architecture-level differences were more distinct during internal validation but became less consistent under external validation in both training directions. This attenuation suggests that, under cross-dataset conditions, domain shift and annotation heterogeneity may exert a stronger influence on localization performance than architecture-specific differences alone.
We also revised Section 3.2 to provide a more precise subtype-specific interpretation of the statistical comparisons. In particular, we clarified that Swin-RT-DETR did not uniformly outperform all competing models. Faster R-CNN with a ResNeXt101 backbone achieved comparable IVH BB-DSC and BB-IoU and significantly higher IPH BB-DSC and BB-IoU than Swin-RT-DETR, whereas Swin-RT-DETR showed significantly higher BB-DSC and BB-IoU for SAH, SDH, and EDH. We therefore revised the wording to state that the advantage of Swin-RT-DETR was subtype-dependent rather than uniform across all hemorrhage categories.
In addition, we added patient-level analyses to complement the image-level comparisons. These results showed trends consistent with the image-level findings: internal validation performance was higher than external validation performance in both BHX-to-RSNA+ and RSNA+-to-BHX directions.
We revised the Discussion to clarify that patient-level aggregation may reduce slice-level variability but does not eliminate the observed cross-dataset performance degradation.
Finally, we added a limitation noting that per-slice statistical comparisons may be affected by within-patient or within-scan correlation, and that future studies should incorporate patient-level aggregation, scan-level bootstrapping, or mixed-effects modeling to provide more robust statistical inference.

While the interpretability analysis is partially presented in the study, this analysis remains limited. The presentation of attention maps is insufficient.

Response: Thank you for this helpful comment. We agree that the previous version did not present the interpretability analysis in sufficient detail. In the revised manuscript, we expanded the interpretability analysis by adding representative true-positive, false-positive, and false-negative examples for each ICH subtype on both the BHX internal test set and the RSNA+ external test set. The revised figure now shows the corresponding attention maps together with prediction boxes and ground-truth boxes, allowing clearer assessment of whether model activation overlapped with hemorrhage-related regions or non-specific image areas. We also revised the Results section to describe subtype-specific failure modes, including false-positive activation in anatomically plausible but non-hemorrhagic regions and false-negative cases involving small, subtle, diffuse, or low-contrast hemorrhages.

Furthermore, clinical integration and real-world use cases are addressed quite superficially. These shortcomings weaken the study's claim to clinical applicability.

Response: Thank you for this important comment. We agree that the previous manuscript did not sufficiently discuss clinical integration and may have overstated the clinical applicability of the evaluated models.

In the revised Discussion, we expanded the clinical interpretation and clarified that the current external validation performance, particularly the low BB-DSC values for IVH, SAH, and EDH, does not support stand-alone clinical deployment.
We revised the manuscript to frame the potential clinical role of these models more cautiously. Specifically, we now describe possible use cases as adjunctive rather than autonomous, including radiologist triage support, visual assistance for suspected hemorrhage regions, quality-control review, and research-oriented dataset annotation support.
We also emphasize that any clinical implementation would require prospective validation across institutions, scanner protocols, patient populations, and annotation standards, as well as evaluation of reader performance, workflow impact, false-positive burden, and patient-level outcomes.
The revised Discussion and Conclusion therefore present the study primarily as evidence of the fragility of cross-dataset ICH localization models, rather than as validation of a clinically deployable system.

I would also like to point out that the figure captions are excessively long.

Response: We shortened the figure captions and moved explanatory detail to the Results or Methods text.

The introduction section is very weak, lacking a literature review.

Response: Thank you for this important comment. We substantially rewrote the Introduction to provide a more structured literature review.

The revised Introduction now summarizes prior CNN-based one-stage and two-stage object detection methods, segmentation-based approaches, and Transformer or hybrid CNN–Transformer architectures in medical imaging.
We also clarified the distinction between image-level classification, bounding-box localization, and segmentation, emphasizing that high classification performance does not necessarily imply clinically useful spatial localization.
In addition, we revised the Introduction to define the unresolved gap more clearly. Specifically, we now emphasize that most previous ICH localization studies relied primarily on internal validation using single-source datasets, leaving cross-dataset robustness insufficiently characterized. The revised Introduction therefore frames the study around external validation, dataset shift, annotation heterogeneity, and morphology-dependent localization performance, rather than architecture novelty alone.

Many studies on this topic exist in the literature.

Response: Thank you for this comment. We agree that the previous Introduction did not sufficiently cover the existing literature on ICH detection and localization. In the revised manuscript, we expanded the literature review to include additional relevant studies, covering 3D CNN approaches for ICH classification [1], efficient segmentation pipelines [2], multi-channel input strategies [3], commercially available AI tools evaluated in clinical settings [4], and large language model–based approaches [5]. These references are now integrated into the Introduction to provide a broader context for the present work and to better position the study's contribution relative to existing methods.

There are too many subheadings.

Response: Thank you for this comment. We revised the manuscript structure to reduce excessive subdivision and improve readability. In the Results section, we consolidated overlapping subsections and reorganized the findings into three broader sections: dataset and annotation characteristics, model performance under internal and external validation, and qualitative/interpretability analysis. We retained the Methods subheadings because they provide necessary methodological transparency and reproducibility. The Discussion was already presented as a continuous narrative without subheadings, so no additional Discussion subheadings were removed.

Table and figure explanations are absent.

Response: We added explanatory text before and after the main tables and figures, emphasizing how each result supports the study question.

The conclusion is written very superficially.

Response: We rewrote the conclusion to emphasize the main finding: internal architectural gains do not ensure external robustness, and dataset shift and subtype morphology are major barriers to clinically reliable ICH localization.

Reviewer 2 Report

Comments and Suggestions for Authors

The manuscript addresses an important and timely problem: whether contemporary object-detection architectures can provide reliable localisation of intracranial haemorrhage subtypes on non-contrast CT, particularly under cross-dataset validation. Its central strength is the explicit comparison between internal BHX validation and external RSNA+ testing, which reveals a clinically meaningful collapse in performance across architectures. This is the most valuable contribution of the work, because it shifts the emphasis from architectural superiority to dataset shift, annotation heterogeneity, and external validity. The proposed Swin-RT-DETR performs strongly under internal validation, but its advantage largely disappears externally, with DSC falling markedly across haemorrhage subtypes. This finding is important and should be foregrounded more forcefully as a cautionary message for clinical translation.

However, the manuscript currently overstates the novelty of the architecture while underdeveloping the methodological implications of the external-validation failure. For a high-impact clinical journal, the work would need a clearer clinical framing, stronger statistical treatment, more transparent annotation analysis, and a more rigorous discussion of whether bounding-box detection is the appropriate paradigm for thin, diffuse, or multifocal haemorrhages such as SAH and SDH.

Major Comments

The study design is conceptually sound but incomplete for the claim that architecture-specific advantages diminish under domain shift. The authors train and optimise on BHX and test externally on RSNA+, but they do not perform the reciprocal experiment: training on RSNA+ and testing on BHX. Without this bidirectional validation, it remains unclear whether the observed degradation reflects intrinsic model fragility, BHX-specific overfitting, RSNA+ annotation differences, or asymmetry in dataset composition. A reciprocal analysis would substantially strengthen the central conclusion.

The exclusion of multi-class labelled images is insufficiently justified. In clinical practice, haemorrhage subtypes frequently coexist, and excluding multi-class cases may artificially simplify the task while reducing ecological validity. This decision may also distort subtype distributions and bias performance estimates. The authors should quantify how excluded cases differ from included cases and provide a sensitivity analysis including multi-class images or, at minimum, explain why the final model should be considered clinically relevant despite this restriction.

The statistical analysis is not yet adequate for the structure of the data. The manuscript appears to analyse per-image metrics as independent observations, but CT slices are nested within patients or scans. This violates independence assumptions and may inflate statistical significance. A high-impact journal would expect patient-level clustering, mixed-effects modelling, bootstrapped confidence intervals by scan, or at least patient-level aggregation. Reporting numerous Bonferroni-corrected p values does not compensate for the more fundamental issue of correlated observations.

The interpretation of DSC and IoU for bounding boxes requires clarification. Dice and IoU are traditionally segmentation metrics; when applied to bounding boxes, they measure box overlap rather than lesion-level anatomical accuracy. This distinction is crucial, especially for SAH and SDH, where rectangular boxes may include substantial normal tissue. The authors should explicitly define whether DSC and IoU were computed on bounding boxes or masks, and should avoid language implying true lesion segmentation unless pixel-level masks were used.

The manuscript should move beyond reporting performance degradation and diagnose its causes. The authors speculate that annotation variability, morphology, and dataset characteristics explain the external failure, but they do not measure these factors systematically. They should compare slice thickness, scanner parameters, windowing, haemorrhage size, box aspect ratio, lesion burden, subtype prevalence, and annotation density between BHX and RSNA+. A stratified analysis by lesion size and bounding-box geometry would make the paper considerably more informative.

The clinical significance of the reported performance remains uncertain. External DSC values for Swin-RT-DETR are very low for IVH, SAH, and EDH, and only modest for IPH and SDH. The authors should state plainly that these external results are not compatible with stand-alone clinical deployment. The conclusion should be tempered: the study demonstrates the fragility of current localisation systems more than it validates a clinically useful model.

The qualitative figures are useful but selective. Representative true positives, false positives, and false negatives should be sampled using prespecified criteria rather than illustrative choice. The authors should include failure modes stratified by subtype and dataset, preferably with examples of small haemorrhages, extra-axial blood, motion artefact, calcification, post-operative change, and mimics. This would make the interpretation clinically credible.

Minor Comments

The abstract should better reflect the negative external-validation result. The current framing still suggests architectural benefit, whereas the dominant finding is poor generalisation. The conclusion should state that external validation revealed limited robustness and that architectural gains observed internally may not translate across datasets.

Tables 1 and 2 are difficult to read and should be redesigned. The authors should separate internal and external validation into distinct tables, report confidence intervals, and highlight best-performing models without excessive statistical symbols. A summary table showing absolute and relative performance loss by subtype would improve clarity.

Figure 3 is visually dense and insufficiently explanatory. The figure should either be simplified or accompanied by a clearer description of what is genuinely novel in Swin-RT-DETR compared with standard RT-DETR plus Swin backbone substitution.

Several grammatical issues require correction, including “was relatively infrequently,” “although our multi-dataset evaluation provides insight into generalization behavior; however,” and inconsistent spelling of haemorrhage/hemorrhage. The manuscript should adopt one language convention throughout.

The reference list includes very recent and possibly future-dated citations. The authors should verify all bibliographic details carefully before publication.

Comments on the Quality of English Language

The manuscript would benefit from sharper language. Phrases such as “state-of-the-art” should be used cautiously unless the comparison includes the strongest available models and reproducible benchmarking. The term “localisation” should be consistently distinguished from “detection” and “segmentation.”

Author Response

Reviewer 2:

Response: Thank you for this thoughtful and constructive assessment. We appreciate the reviewer’s recognition that the central contribution of this study lies in the comparison between internal and external validation and in demonstrating the fragility of ICH localization performance under cross-dataset conditions. We agree that the previous manuscript did not foreground this cautionary message strongly enough and may have placed excessive emphasis on architecture-level comparison.

In the revised manuscript, we reframed the study to emphasize cross-dataset generalization, dataset shift, annotation heterogeneity, and clinical translation limitations. Specifically, we revised the title, Abstract, Introduction, Results, Discussion, and Conclusion to make clear that the main finding is not the uniform superiority of Swin-RT-DETR, but rather the substantial attenuation of architecture-specific advantages under external validation. We also added bidirectional cross-dataset validation, patient-level analysis, and dataset-characterization analysis to better support this interpretation.

We now state more explicitly that the low external BB-DSC values for several subtypes, particularly IVH, SAH, and EDH, do not support stand-alone clinical deployment. The revised Discussion presents the evaluated models as investigational and potentially adjunctive tools, pending prospective validation of reader performance, workflow impact, false-positive burden, and patient-level outcomes. Overall, the revised manuscript foregrounds the cautionary message that external validation and dataset characterization are essential before ICH localization models can be considered for clinical integration.

However, the manuscript currently overstates the novelty of the architecture while underdeveloping the methodological implications of the external-validation failure.

Response: We toned down claims of model novelty and expanded the domain-shift discussion. Swin-RT-DETR is now presented as a representative hybrid Transformer-based detector used to test generalization under external validation.

For a high-impact clinical journal, the work would need a clearer clinical framing, stronger statistical treatment, more transparent annotation analysis, and a more rigorous discussion of whether bounding-box detection is the appropriate paradigm for thin, diffuse, or multifocal haemorrhages such as SAH and SDH.

Response: Thank you for this important and constructive comment. We agree that the previous manuscript required stronger clinical framing, more cautious statistical interpretation, more transparent annotation analysis, and a clearer discussion of the limitations of bounding-box localization for morphologically complex hemorrhage subtypes.

First, we revised the Introduction, Discussion, and Conclusion to provide a clearer clinical framing. The revised manuscript now emphasizes that the evaluated models are not ready for stand-alone clinical deployment, particularly given the low external BB-DSC values for several subtypes. We now frame potential use cases more cautiously as investigational and adjunctive, such as radiologist triage support, visual prompting, quality-control review, or research annotation assistance. We also added discussion that clinical usefulness should be interpreted according to the intended use case, because no universally accepted minimum BB-DSC, BB-IoU, or mAP@50 threshold currently defines clinical readiness.
Second, we strengthened the statistical interpretation by adding both image-level and patient-level analyses. The new patient-level results showed trends broadly consistent with the image-level findings, with higher internal performance than external performance in both cross-dataset directions. We also revised the Results and Discussion to emphasize that architecture-level differences were attenuated under external validation, suggesting that domain shift and annotation heterogeneity may exert a stronger influence than architecture-specific effects. In addition, we added a limitation noting that slice-level observations may not be fully independent and that future studies should incorporate scan-level bootstrapping, hierarchical resampling, or mixed-effects modeling.
Third, we added a quantitative dataset-characterization analysis comparing BHX and RSNA+ in terms of subtype prevalence, bounding-box size, aspect ratio, lesion burden, annotation density, spatial distribution, width–height distribution, and representative imaging examples. These results are now presented in Figure 4. This analysis provides empirical evidence that the two datasets differ not only in image source but also in annotation geometry, lesion burden, subtype distribution, and spatial localization patterns.
Finally, we expanded the discussion of whether bounding-box detection is an appropriate paradigm for thin, diffuse, or multifocal hemorrhages. We now explicitly distinguish bounding-box localization from pixel-level segmentation and refer to DSC and IoU as BB-DSC and BB-IoU to clarify that these metrics quantify rectangular box overlap rather than anatomical lesion-contour accuracy. We also discuss that SAH and SDH are inherently challenging for bounding-box detection because thin, crescentic, multifocal, or diffuse hemorrhage patterns are poorly represented by a single rectangular box. The revised manuscript therefore states that segmentation-based or hybrid detection–segmentation approaches may be more appropriate for precise lesion delineation when harmonized pixel-level annotations are available.

Major Comments

The study design is conceptually sound but incomplete for the claim that architecture-specific advantages diminish under domain shift. The authors train and optimise on BHX and test externally on RSNA+, but they do not perform the reciprocal experiment: training on RSNA+ and testing on BHX. Without this bidirectional validation, it remains unclear whether the observed degradation reflects intrinsic model fragility, BHX-specific overfitting, RSNA+ annotation differences, or asymmetry in dataset composition. A reciprocal analysis would substantially strengthen the central conclusion.

Response: Thank you for this important suggestion. We agree that a one-directional external validation design is insufficient to determine whether the observed performance degradation is specific to the BHX-to-RSNA+ setting or reflects a broader limitation in cross-dataset robustness. To address this concern, we added a reciprocal cross-dataset experiment in the revised manuscript. Specifically, in addition to the original experiment in which models were trained and optimized on BHX and externally tested on RSNA+, we performed a reciprocal experiment in which the same model architectures were trained and optimized on RSNA+ and externally tested on BHX. In both directions, the external test dataset was used only for final evaluation and was not used for model selection, hyperparameter tuning, or early stopping. The revised study design is now described in Section 2.2 and illustrated in Figure 2.

The reciprocal experiment showed that external validation performance also decreased when the training and testing directions were reversed. This bidirectional degradation indicates that the observed performance loss was not solely attributable to BHX-specific optimization or RSNA+-specific testing conditions. Instead, the results support the interpretation that dataset composition, annotation geometry, subtype distribution, and hemorrhage morphology substantially influence external localization robustness.

We also added image-level and patient-level analyses for both validation directions, presented in Figures 5 and 6, and precision–recall curve analyses across the four training/testing settings in Figure 7. These additional analyses strengthen the central conclusion that architecture-specific advantages observed during internal validation are attenuated under cross-dataset conditions. We have revised the Results, Discussion, and Conclusion accordingly to emphasize bidirectional cross-dataset robustness rather than unidirectional external validation alone.

The exclusion of multi-class labelled images is insufficiently justified. In clinical practice, haemorrhage subtypes frequently coexist, and excluding multi-class cases may artificially simplify the task while reducing ecological validity. This decision may also distort subtype distributions and bias performance estimates. The authors should quantify how excluded cases differ from included cases and provide a sensitivity analysis including multi-class images or, at minimum, explain why the final model should be considered clinically relevant despite this restriction.

Response: Thank you for this important comment. We agree that excluding multi-class labeled images simplifies the localization task and may reduce ecological validity, because coexisting hemorrhage subtypes are common in clinical practice and may provide useful anatomical context for subtype differentiation.

In the revised manuscript, we clarified that the single-class design was used as a controlled experimental simplification to reduce label ambiguity and enable consistent subtype-specific comparisons across datasets and architectures. We also quantified the excluded multi-class cases in Figure 1. Multi-class labeled images accounted for 24.33% of the BHX dataset (3,667/15,075 images) and 24.44% of the RSNA+ test set (3,459/14,155 images), indicating that approximately three-quarters of images in both datasets were single-class cases.

We did not perform a sensitivity analysis including multi-class images in the current revision because coexisting hemorrhage subtypes often produce overlapping or adjacent bounding boxes, which would introduce ambiguity when assigning subtype-specific localization errors and calculating BB-DSC or BB-IoU for individual categories. Including multi-class images without a dedicated multi-label evaluation framework could therefore confound the architecture-level comparison and obscure whether performance differences arise from model design, subtype coexistence, or annotation overlap.

We have added this issue as a limitation and revised the Discussion to state that the reported results should be interpreted as a controlled subtype-specific evaluation rather than a complete representation of real-world ICH localization complexity. Future studies should incorporate multi-label or multi-class detection frameworks that can simultaneously localize coexisting hemorrhage subtypes and better reflect clinical CT interpretation.

The statistical analysis is not yet adequate for the structure of the data. The manuscript appears to analyse per-image metrics as independent observations, but CT slices are nested within patients or scans. This violates independence assumptions and may inflate statistical significance. A high-impact journal would expect patient-level clustering, mixed-effects modelling, bootstrapped confidence intervals by scan, or at least patient-level aggregation.

Response: Thank you for this important comment. We agree that slice-level statistical comparisons may be affected by within-patient or within-scan correlation. To address this concern, we added patient-level performance analyses in addition to the image-level comparisons. The new patient-level results are presented in Figure 6 and show trends broadly consistent with the image-level findings: internal validation performance was higher than external validation performance in both BHX-to-RSNA+ and RSNA+-to-BHX directions.

We also revised the Discussion and Limitations to clarify that patient-level aggregation partially reduces the influence of slice-level variability but does not fully replace more rigorous statistical approaches such as scan-level bootstrapping, hierarchical resampling, or mixed-effects modeling. Therefore, the reported p values are now interpreted more cautiously as exploratory model-level comparisons rather than definitive patient-level inference.

Reporting numerous Bonferroni-corrected p values does not compensate for the more fundamental issue of correlated observations.

Response: Thank you for this important comment. We agree that Bonferroni correction addresses multiple pairwise comparisons but does not resolve the non-independence of correlated observations, particularly when multiple CT slices originate from the same patient or scan. In the revised manuscript, we therefore interpret the Bonferroni-corrected p values more cautiously and no longer present them as definitive patient-level inference.

To partially address this concern, we added patient-level performance analyses in addition to the original image-level comparisons. The patient-level results, now presented in Figure 6, showed trends broadly consistent with the image-level findings: internal validation performance was higher than external validation performance in both BHX-to-RSNA+ and RSNA+-to-BHX directions. These findings support the conclusion that cross-dataset generalization remained limited, even after aggregation beyond individual slices.

We revised the Limitations section to address the issue of correlated observations more explicitly. We now state that several primary statistical comparisons were based on image-level metrics and that multiple CT slices from the same patient or scan may not be fully independent, potentially inflating statistical significance. Although patient-level analyses were added and showed trends broadly consistent with the image-level results, we acknowledge that patient-level aggregation alone does not fully model within-patient correlation or uncertainty. We therefore clarified that Bonferroni-corrected p values should be interpreted as exploratory model-level comparisons rather than definitive patient-level inference, and we recommend scan-level bootstrapping, hierarchical resampling, generalized estimating equations, or mixed-effects modeling in future studies.

The interpretation of DSC and IoU for bounding boxes requires clarification. Dice and IoU are traditionally segmentation metrics; when applied to bounding boxes, they measure box overlap rather than lesion-level anatomical accuracy. This distinction is crucial, especially for SAH and SDH, where rectangular boxes may include substantial normal tissue. The authors should explicitly define whether DSC and IoU were computed on bounding boxes or masks, and should avoid language implying true lesion segmentation unless pixel-level masks were used.

Response: Thank you for this important comment. We agree that the previous terminology could have led to confusion between bounding-box localization and pixel-level lesion segmentation. In the revised manuscript, we explicitly clarify that DSC and IoU were computed from predicted and ground-truth bounding boxes, not from pixel-level masks. Accordingly, we now refer to these metrics as bounding-box Dice similarity coefficient (BB-DSC) and bounding-box intersection over union (BB-IoU) throughout the manuscript.

We revised the Performance Evaluation section to define BB-DSC and BB-IoU as measures of rectangular box overlap rather than lesion-contour accuracy. We also revised the Results and Discussion to avoid language implying true segmentation performance. In the Discussion, we further emphasize that this distinction is particularly important for SAH and SDH, where thin, crescentic, multifocal, or diffuse hemorrhage patterns may be poorly represented by a single rectangular bounding box and may include substantial normal tissue. Therefore, BB-DSC and BB-IoU should be interpreted as localization-overlap metrics rather than indicators of anatomical lesion delineation.

The manuscript should move beyond reporting performance degradation and diagnose its causes. The authors speculate that annotation variability, morphology, and dataset characteristics explain the external failure, but they do not measure these factors systematically. They should compare slice thickness, scanner parameters, windowing, haemorrhage size, box aspect ratio, lesion burden, subtype prevalence, and annotation density between BHX and RSNA+. A stratified analysis by lesion size and bounding-box geometry would make the paper considerably more informative.

Response: Thank you for this constructive suggestion. We agree that the manuscript should move beyond reporting performance degradation and should empirically characterize possible sources of domain shift. To address this concern, we added a dataset-characterization analysis comparing BHX and RSNA+ with respect to subtype prevalence, normalized bounding-box size, bounding-box aspect ratio, lesion burden, annotation density, spatial distribution, width–height distribution, and representative CT examples. These analyses are now presented in Figure 4 and summarized in the Results section.

The added analysis demonstrates clear differences between BHX and RSNA+ in annotation geometry, lesion burden, subtype distribution, and spatial localization patterns. We also revised the morphology-related descriptions throughout the manuscript to distinguish subtype-specific geometric patterns more precisely. SAH is now described primarily as irregular, diffuse, or multifocal, whereas SDH is described as elongated or crescentic in an extra-axial distribution. This distinction was incorporated into the Results and Discussion to clarify why bounding-box localization may affect these subtypes differently.

We further revised the Discussion to emphasize that the observed external validation failure likely reflects both imaging-domain shift and annotation-domain shift, rather than architecture-specific limitations alone. We acknowledge that scanner parameters, acquisition protocols, and population-level variables could not be fully compared because these metadata were not consistently available across the public datasets. This limitation has been explicitly added to the revised manuscript.

The clinical significance of the reported performance remains uncertain. External DSC values for Swin-RT-DETR are very low for IVH, SAH, and EDH, and only modest for IPH and SDH. The authors should state plainly that these external results are not compatible with stand-alone clinical deployment. The conclusion should be tempered: the study demonstrates the fragility of current localisation systems more than it validates a clinically useful model.

Response: Thank you for this important comment. We agree that the previous manuscript did not state the clinical implications of the low external performance clearly enough. In the revised Discussion and Conclusion, we now explicitly state that the external BB-DSC values, particularly for IVH, SAH, and EDH, are not compatible with stand-alone clinical deployment.

We have tempered the interpretation throughout the manuscript. Rather than presenting Swin-RT-DETR as a clinically deployable localization model, we now frame the study as demonstrating the fragility of current ICH localization systems under cross-dataset validation. We also clarify that the potential role of these models, at the current level of performance, should be considered investigational and adjunctive rather than autonomous. Possible future use cases may include radiologist triage support, visual prompting, quality-control review, or research annotation assistance, but only after prospective validation of reader performance, workflow impact, false-positive burden, and patient-level outcomes.

The revised Conclusion now emphasizes that external validation, dataset characterization, patient-level evaluation, and task-specific clinical benchmarks are required before automated ICH localization models can be considered for real-world clinical integration.

The qualitative figures are useful but selective. Representative true positives, false positives, and false negatives should be sampled using prespecified criteria rather than illustrative choice. The authors should include failure modes stratified by subtype and dataset, preferably with examples of small haemorrhages, extra-axial blood, motion artefact, calcification, post-operative change, and mimics. This would make the interpretation clinically credible.

Response: Thank you for this valuable suggestion. We agree that the qualitative analysis should be presented more systematically rather than relying on selectively chosen illustrative examples. In the revised manuscript, we updated the qualitative evaluation strategy by selecting representative true-positive, false-positive, and false-negative cases using predefined criteria across hemorrhage subtypes and datasets.

Specifically, the revised qualitative figure now includes examples stratified by subtype and dataset source, with clinically relevant success and failure modes, including small hemorrhages, extra-axial blood adjacent to the skull, motion-related image degradation, calcification or bone-related mimics, postoperative changes, and other hemorrhage mimics. These examples are now presented in Figure 8.

We also revised the qualitative evaluation section to describe the main failure patterns more explicitly. False-negative cases commonly involved small, subtle, diffuse, or low-contrast hemorrhages, whereas false-positive cases often arose from anatomically plausible mimics, including sulcal hyperdensity, calcification, skull-adjacent structures, postoperative changes, or motion-related artifacts. We further discuss these findings in relation to the limitations of bounding-box localization and the challenges of external validation. These revisions improve the transparency, reproducibility, and clinical interpretability of the qualitative analysis.

Minor Comments

The abstract should better reflect the negative external-validation result. The current framing still suggests architectural benefit, whereas the dominant finding is poor generalisation. The conclusion should state that external validation revealed limited robustness and that architectural gains observed internally may not translate across datasets.

Response: Thank you for this important comment. We agree that the previous Abstract placed too much emphasis on architectural benefit and did not sufficiently foreground the negative external-validation result. We substantially revised the Abstract to emphasize bidirectional cross-dataset generalization, external validation failure, and attenuation of architecture-specific advantages under domain shift.

Specifically, the revised Abstract now states that Swin-RT-DETR showed subtype-dependent internal advantages rather than uniform superiority. We also added the reciprocal RSNA+-to-BHX validation results and patient-level analysis to show that performance degradation occurred in both cross-dataset directions. In the Conclusions, we now state that dataset characteristics, annotation heterogeneity, and domain shift strongly influenced ICH localization performance, and that internally observed architectural gains diminished under bidirectional external validation. The revised Abstract therefore presents the study primarily as evidence of limited cross-dataset robustness rather than validation of a clinically deployable model.

Tables 1 and 2 are difficult to read and should be redesigned. The authors should separate internal and external validation into distinct tables, report confidence intervals, and highlight best-performing models without excessive statistical symbols. A summary table showing absolute and relative performance loss by subtype would improve clarity.

Response: Thank you for this constructive suggestion. We agree that the original tables were difficult to read because they contained many subtype-specific metrics and statistical symbols in a dense format. In the revised manuscript, we redesigned the presentation of model performance to improve readability and interpretability.

Specifically, we separated internal and external validation results into clearer summary displays and added box-plot figures for both image-level and patient-level evaluations. These are now presented in Figure 5 and Figure 6. The revised figures highlight model-level performance patterns without excessive statistical symbols, while Bonferroni-corrected significant differences from Swin-RT-DETR are indicated only where needed. We also added reciprocal RSNA+-to-BHX cross-dataset results to provide a bidirectional evaluation.

In addition, we added supplementary summary tables reporting absolute and relative performance loss by subtype, together with confidence intervals, for both image-level and patient-level analyses. These results are provided in Tables S3–S10, with Table S2 reporting model parameters and runtime performance. We believe these revisions make the internal and external validation results easier to interpret and more clearly demonstrate the magnitude of cross-dataset performance degradation.

Figure 3 is visually dense and insufficiently explanatory. The figure should either be simplified or accompanied by a clearer description of what is genuinely novel in Swin-RT-DETR compared with standard RT-DETR plus Swin backbone substitution.

Response: Thank you for this helpful comment. We agree that the previous Figure 3 was visually dense and did not clearly distinguish the implemented architectural modification from the standard RT-DETR framework. In the revised manuscript, we simplified Figure 3 and revised the accompanying description to clarify that Swin-RT-DETR is a Swin Transformer–based RT-DETR variant rather than a fundamentally new detector architecture.

Specifically, we now state that the main architectural modification was replacement of the original HGNetV2 backbone in RT-DETR with a Swin Transformer Base backbone, while the remaining RT-DETR detection framework was preserved. We also revised the Methods and Discussion to emphasize that the purpose of including Swin-RT-DETR was to evaluate the potential contribution of Transformer-based hierarchical feature extraction to ICH localization, not to claim a novel detector design. This clarification better aligns the figure and text with the comparative focus of the study.

Several grammatical issues require correction, including “was relatively infrequently,” “although our multi-dataset evaluation provides insight into generalization behavior; however,” and inconsistent spelling of haemorrhage/hemorrhage. The manuscript should adopt one language convention throughout.

Response: Thank you for pointing this out. We carefully revised the manuscript for grammar, wording, and consistency. The specific grammatical issues identified by the reviewer were corrected. For example, “was relatively infrequently” was revised to “was relatively infrequent,” and the sentence containing “although our multi-dataset evaluation provides insight into generalization behavior; however” was rewritten to remove the redundant conjunction.

We also standardized the manuscript to American English throughout. Accordingly, we consistently use “hemorrhage” rather than “haemorrhage,” and we reviewed related terminology, including “localization,” “noncontrast,” “bounding-box,” and subtype names, to ensure consistent usage across the Abstract, main text, figure captions, tables, and supplementary materials.

The reference list includes very recent and possibly future-dated citations. The authors should verify all bibliographic details carefully before publication.

Response: Thank you for this important comment. We carefully rechecked the reference list and corrected bibliographic details where necessary, including author names, article titles, journal names, publication years, volume numbers, article numbers, and DOI information. We also verified the very recent references to ensure that they correspond to formally published or indexed articles rather than unpublished future-dated citations.

In particular, the citation that may have appeared future-dated has now been updated with complete bibliographic information, including journal volume, article number, publication date, and DOI. We also removed or corrected incomplete bibliographic entries and standardized the reference formatting throughout the manuscript according to the journal style.

The manuscript would benefit from sharper language. Phrases such as “state-of-the-art” should be used cautiously unless the comparison includes the strongest available models and reproducible benchmarking. The term “localisation” should be consistently distinguished from “detection” and “segmentation.”

Response: Thank you for this helpful comment. We agree that the wording in the previous version could be made more precise. In the revised manuscript, we replaced broad terms such as “state-of-the-art” with more specific wording, including “representative contemporary detectors” or “evaluated detection architectures,” unless the statement referred directly to a cited method. We also revised the terminology throughout the manuscript to distinguish more clearly among detection, bounding-box localization, and segmentation.

Specifically, we now use “detection” to refer to the identification and classification of hemorrhage regions, “bounding-box localization” to refer to spatial localization using rectangular boxes, and “segmentation” only when referring to pixel-level lesion delineation. We also renamed DSC and IoU as BB-DSC and BB-IoU where appropriate to clarify that these metrics quantify bounding-box overlap rather than pixel-level lesion segmentation accuracy. This revision improves terminological precision and avoids overstating the scope of the evaluated models.

Reviewer 3 Report

Comments and Suggestions for Authors

This manuscript presents a study for intracranial hemorrhage subtype localization on non-contrast head CT with a focus on cross-dataset generalization of Transformer-based architectures. However, several issues undertermine its contribution.

Domain shift is acknowledged but not explored in depth. Is the drop due to differences in annotation protocol, scanner differences or population biases?
A comparison with a pure Transformer detector and a modern CNN architecture would better isolate the contribution of the attention mechanism vs architectural advancements for the hybrid Swin-RT-DETR.
There should be a discussion of the minimum benchmark that would be considered useful in a clinical context .
Why not start with a segmentation framework, which would arguably be better suited for lesion delineation.
If trained without multi-class labels , the task is easier, but it discards valuable anatomical context that could aid in differentiating subtypes.
The authors should be more modest about the superiority of Swin-RT-DETR architecture, its performance under cross-dataset conditions is really bad.
The title might mislead a reader into expecting a model design paper. Perhaps you can reframe it to emphasize the comparative study and the key finding that domain shift is more important than architecture.

Author Response

Reviewer 3:

Domain shift is acknowledged but not explored in depth. Is the drop due to differences in annotation protocol, scanner differences or population biases?

Response: Thank you for this important comment. We agree that the previous manuscript acknowledged domain shift but did not sufficiently examine its possible sources. To address this issue, we added a quantitative dataset-characterization analysis comparing BHX and RSNA+ in terms of subtype prevalence, normalized bounding-box size, bounding-box aspect ratio, lesion burden, annotation density, spatial distribution, width–height distribution, and representative imaging examples. These results are now presented in Figure 4 and described in the Results section.

The added analysis demonstrates that BHX and RSNA+ differ substantially in annotation geometry, lesion burden, subtype distribution, and spatial localization patterns. These findings suggest that the observed performance degradation likely reflects not only imaging-domain shift but also annotation-domain shift and subtype-specific morphology. We have revised the Discussion accordingly to clarify that cross-dataset performance loss should not be attributed to model architecture alone.

We also acknowledge that scanner parameters, acquisition protocols, and population-level variables could not be fully compared because these metadata were not consistently available across the two public datasets. This limitation has been explicitly added to the revised Discussion.

A comparison with a pure Transformer detector and a modern CNN architecture would better isolate the contribution of the attention mechanism vs architectural advancements for the hybrid Swin-RT-DETR.

Response: Thank you for this insightful comment. We agree that comparison with a pure Transformer detector and additional modern CNN architectures would better isolate the contribution of the attention mechanism from broader architectural differences. In the present study, Swin-RT-DETR was included as a representative Transformer-based RT-DETR variant to evaluate whether hierarchical Transformer-based feature extraction could improve ICH localization compared with commonly used CNN-based one-stage and two-stage detectors. However, because Swin-RT-DETR differs from the other models in both backbone design and detection framework, the current comparison cannot attribute performance differences specifically to self-attention mechanisms alone.

We have revised the Methods and Discussion to describe Swin-RT-DETR more cautiously as a representative Transformer-based detector rather than evidence of attention-specific superiority. We also added this issue as a limitation, noting that future studies should include pure Transformer detectors, additional modern CNN detectors, and controlled backbone-ablation experiments to more rigorously disentangle the effects of attention mechanisms, multiscale feature representation, and detector design.

There should be a discussion of the minimum benchmark that would be considered useful in a clinical context.

Response: Thank you for this important comment. We agree that the clinical usefulness of an automated localization model should be interpreted in relation to an intended use case and a minimum acceptable performance benchmark. In the revised Discussion, we added a more explicit discussion of clinical performance requirements. We now clarify that there is no universally accepted minimum BB-DSC, BB-IoU, or mAP@50 threshold for clinical deployment of ICH localization models, and that the required benchmark would likely differ according to the clinical application, such as triage support, visual prompting, lesion follow-up, or quantitative volumetric assessment.

We also revised the manuscript to state that the low external BB-DSC values observed for several subtypes, particularly IVH, SAH, and EDH, do not support stand-alone clinical deployment. At the current level of external performance, the evaluated models should be considered investigational and potentially useful only as adjunctive tools after further validation. We added that future studies should define task-specific clinical benchmarks and evaluate reader performance, false-positive burden, workflow impact, and patient-level outcomes in prospective multi-institutional settings.

Why not start with a segmentation framework, which would arguably be better suited for lesion delineation.

Response: Thank you for this important comment. We agree that segmentation frameworks are better suited for precise lesion delineation, particularly for thin, diffuse, or irregular hemorrhage patterns such as SAH and SDH. In the present study, we focused on bounding-box object detection because the publicly available BHX and RSNA+ datasets provide bounding-box annotations rather than harmonized pixel-level segmentation masks. This allowed us to perform a controlled cross-dataset comparison across multiple detector architectures using consistent annotation formats.

We have revised the Introduction and Discussion to clarify that bounding-box localization should not be interpreted as true lesion segmentation. We now explicitly define BB-DSC and BB-IoU as bounding-box overlap metrics rather than pixel-level lesion-conformity metrics. We also added discussion that the poor performance for SAH and SDH partly reflects the limitation of rectangular bounding boxes for thin, crescentic, multifocal, or diffuse hemorrhage patterns. Future studies should incorporate segmentation or hybrid detection–segmentation frameworks when pixel-level annotations are available, particularly for subtypes requiring accurate lesion delineation or volumetric assessment.

If trained without multi-class labels, the task is easier, but it discards valuable anatomical context that could aid in differentiating subtypes.

Response: Thank you for this important comment. We agree that excluding multi-class labeled images simplifies the localization task and may reduce ecological validity, because hemorrhage subtypes can coexist in clinical practice and their spatial relationships may provide useful anatomical context for subtype differentiation. In the revised manuscript, we clarified that the use of single-class labeled images was a controlled experimental simplification intended to reduce label ambiguity and allow consistent subtype-specific comparison across datasets and architectures.

We also revised the Discussion and Limitations to acknowledge that this design may underestimate the complexity of real-world ICH localization and may discard clinically relevant contextual information from coexisting hemorrhage subtypes. Future studies should incorporate multi-label or multi-class modeling strategies that allow simultaneous detection of coexisting hemorrhage subtypes and better reflect clinical CT interpretation.

The authors should be more modest about the superiority of Swin-RT-DETR architecture, its performance under cross-dataset conditions is really bad.

Response: Thank you for this important comment. We agree that the superiority of Swin-RT-DETR should not be overstated. In the revised manuscript, we changed the wording to clarify that Swin-RT-DETR showed subtype-dependent advantages under internal validation rather than uniform superiority across all hemorrhage categories. We also emphasized that these architecture-related advantages were substantially attenuated under external validation.

In addition, we added bidirectional image-level and patient-level analyses to better characterize cross-dataset robustness. These analyses showed that external performance degradation occurred in both BHX-to-RSNA+ and RSNA+-to-BHX directions. We therefore revised the Discussion and Conclusion to present Swin-RT-DETR as a representative Transformer-based detector for comparative evaluation, rather than as a clinically deployable or consistently superior model. The revised manuscript now emphasizes the fragility of cross-dataset ICH localization models and the importance of dataset characterization, external validation, and patient-level evaluation before clinical translation.

The title might mislead a reader into expecting a model design paper. Perhaps you can reframe it to emphasize the comparative study and the key finding that domain shift is more important than architecture.

Response: Thank you for this helpful suggestion. We agree that the previous title could have led readers to expect a model design paper focused primarily on Swin-RT-DETR. To better reflect the revised focus of the manuscript, we changed the title to “Cross-Dataset Generalization of Deep Learning Detectors for Intracranial Hemorrhage Subtype Localization on Noncontrast Head CT: A Comparative Study.” The revised title emphasizes the comparative nature of the work and the central finding that cross-dataset generalization and dataset shift are key determinants of localization performance.

Round 2

Reviewer 1 Report

Comments and Suggestions for Authors

Thank you for the successful revision.

Author Response

Comments and suggestions for authors:

Thank you for the successful revision.

Response: Thank you very much for your encouraging comment. We sincerely appreciate your time and constructive feedback throughout the review process.

Reviewer 2 Report

Comments and Suggestions for Authors

I am happy with the new version of the manuscript. However, the quality of the figures could be improved to enhance readability.

Comments on the Quality of English Language

Author Response

Comments and suggestions for authors:

I am happy with the new version of the manuscript. However, the quality of the figures could be improved to enhance readability.

Response: Thank you very much for your positive assessment of the revised manuscript and for this helpful suggestion. We revised the figures to improve overall readability and visual clarity.

Specifically, Figures 1 and 2 were redrawn to present the dataset-selection process and bidirectional cross-dataset validation workflow more clearly. In Figure 1, we clarified the inclusion and exclusion flow, corrected the dataset counts, and improved the labeling of single-class and excluded multi-class images. In Figure 2, we redesigned the workflow to better distinguish training/validation sets, internal validation, external validation, detector architectures, model inference, and performance evaluation.

In addition, Figure 4 was rearranged to improve the readability of the dataset-characterization analysis. We enlarged panel labels, standardized subplot titles and legends, improved spacing between panels, corrected axis labels, and increased the resolution of representative CT examples. These changes make the subtype prevalence, bounding-box geometry, lesion burden, annotation density, spatial distribution, and representative imaging examples easier to interpret.

For the remaining figures, we increased image resolution, enlarged key labels and annotations, improved visual contrast, standardized panel labels and legends, and adjusted figure legends to make the results easier to interpret. These revisions were made to ensure that the dataset-characterization analyses, performance comparisons, precision–recall curves, and interpretability examples are more clearly presented in the revised manuscript.

Comments on the Quality of English Language

Response: Thank you for this helpful comment. We revised the manuscript to use more precise and cautious language throughout. Broad terms such as “state-of-the-art” were replaced with more specific wording, including “representative CNN-based object detection architectures,” “representative detector architectures,” or “evaluated models,” where appropriate.

We also revised the terminology to consistently distinguish detection, bounding-box localization, and segmentation. In the revised manuscript, “detection” refers to identifying and classifying hemorrhage regions, “bounding-box localization” refers to spatial localization using rectangular bounding boxes, and “segmentation” is reserved for pixel-level lesion delineation. To avoid confusion with segmentation metrics, we now use the terms bounding-box Dice similarity coefficient (BB-DSC) and bounding-box intersection over union (BB-IoU), and we explicitly state that these metrics were computed from predicted and ground-truth bounding boxes rather than pixel-level masks.

In addition, we performed minor copyediting to improve clarity and consistency. Specifically, we revised the ethics statement, standardized bounding-box metric terminology, corrected the description of the Swin-RT-DETR framework, corrected punctuation and spacing errors, revised the morphology-related Discussion sentence, changed “clinical implications for these findings” to “clinical implications of these findings,” and revised the abbreviation definition of mAP@50. These changes improve linguistic precision while keeping the structure and interpretation of the manuscript unchanged.

Reviewer 3 Report

Comments and Suggestions for Authors

Having reviewed the author’s revised draft, I agree to its approval.

Author Response

Comments and suggestions for authors:

Having reviewed the author’s revised draft, I agree to its approval.

Response: Thank you very much for your favorable assessment and support for approval. We sincerely appreciate your time and constructive review of our manuscript.