Review Reports

Round 1

Reviewer 1 Report

Comments and Suggestions for Authors

1. “However, Bethesda categories III (AUS) and IV (FN/SFN) constitute a well-recognized diagnostic “gray zone” with variable malignancy risk, frequently leading to repeat biopsies or diagnostic surgery that ultimately prove benign [6,7]. ”It is recommended that the potential risks associated with unnecessary surgery be discussed here; for example, research has shown that some patients may regret their decision due to postoperative complications following thyroid surgery (PMID: 40057484). Please cite it.
2. The AI's accuracy (73.1%) differs from that of Radiologist 1 (71.2%) by only 1.9%. Is this small difference statistically significant (p-value)?
3. The conclusion mentions “reducing unnecessary repeat biopsies.” Since this study is retrospective and did not involve a prospective comparison of decision-making (such as changes in physician decisions guided by AI), it is not possible to directly conclude that “biopsies are reduced.” Please revise the wording to read “has the potential to serve as supplementary information for decision support.”
4. It is recommended that the limitations section emphasize that subgroup analyses based on the small sample size in this study may lead to overinterpretation of the statistical results.

Author Response

Reviewer 1

1. “However, Bethesda categories III (AUS) and IV (FN/SFN) constitute a well-recognized diagnostic “gray zone” with variable malignancy risk, frequently leading to repeat biopsies or diagnostic surgery that ultimately prove benign [6,7]. ”It is recommended that the potential risks associated with unnecessary surgery be discussed here; for example, research has shown that some patients may regret their decision due to postoperative complications following thyroid surgery (PMID: 40057484). Please cite it.

 

We have added a short discussion of the harms of potentially unnecessary surgery to the second paragraph of the Introduction, including the risks of recurrent laryngeal nerve injury, hypoparathyroidism, and lifelong hormone replacement, and we now cite the study by Li et al. on decision regret after thyroidectomy. The revised text reads: “Such surgery is not without cost. Thyroidectomy carries risks of recurrent laryngeal nerve injury, hypoparathyroidism, and lifelong thyroid hormone replacement. Even among patients operated on for low-risk thyroid cancer, a notable proportion later report regret about their decision, particularly after postoperative complications [8]. Better preoperative risk information for indeterminate nodules could therefore help clinicians and patients weigh the benefits and harms of surgery more carefully.” (Introduction)

 

2. The AI's accuracy (73.1%) differs from that of Radiologist 1 (71.2%) by only 1.9%. Is this small difference statistically significant (p-value)?

 

The difference is not statistically significant. In the revised analysis, the reference standard is the histopathology of the index nodule itself (please see the note on the reference standard at the beginning of this letter). With this reference standard, the accuracy was 71.2% for the TRAICK ACR TI-RADS output and 61.5% for Radiologist 1 (exact McNemar test, p = 0.267), and their AUCs did not differ (0.762 vs. 0.777; DeLong test, p = 0.783). With the coding used in the first submission (73.1% vs. 71.2%), the difference was also not significant (p = 1.000; Supplementary Table S2). We have added these results to the Abstract and to a new “Paired comparisons” subsection in the Results, and all paired comparisons are now given in Supplementary Table S1. We have removed all statements suggesting that TRAICK performed better than Radiologist 1 overall; the only significant difference between them was a higher specificity of TRAICK (65.5% vs. 37.9%; p = 0.021). (Abstract; Results; Supplementary Tables S1 and S2)

 

3. The conclusion mentions “reducing unnecessary repeat biopsies.” Since this study is retrospective and did not involve a prospective comparison of decision-making (such as changes in physician decisions guided by AI), it is not possible to directly conclude that “biopsies are reduced.” Please revise the wording to read “has the potential to serve as supplementary information for decision support.”

 

We have removed the statement about reducing unnecessary repeat biopsies from the Abstract, Discussion, and Conclusions and revised the wording as suggested. The Conclusions now state: “These findings suggest that TRAICK has the potential to provide supplementary information to support preoperative decision-making in indeterminate thyroid nodules. They do not show superiority over radiologists or FNAB, generalizability to other settings, or a reduction in repeat biopsies or surgery.” We also added a paragraph to the Discussion noting that changes in clinical decisions, repeat biopsy rates, and surgery rates were not assessed and would need to be tested in prospective studies. (Discussion, Conclusions)

 

4. It is recommended that the limitations section emphasize that subgroup analyses based on the small sample size in this study may lead to overinterpretation of the statistical results.

 

We have added this point to the first item of the Limitations: “The confidence intervals were wide, the paired comparisons were exploratory and not adjusted for multiple testing, and several subgroups contained very few nodules; the subgroup results, in particular, may be overinterpreted and should be read with caution.” In addition, malignancy rates in the subgroups are now reported with 95% confidence intervals (Table 4), and a note on the small subgroup sizes has been added to the table footnote. (Discussion, Strengths and limitations)

 

 

Reviewer 2 Report

Comments and Suggestions for Authors

The manuscript entitled “Artificial Intelligence–Assisted TIRADS and Bethesda Prediction for Indeterminate Thyroid Nodules: A Proof-of-Concept Clinical Validation Study” addresses a timely and clinically relevant topic. The management of Bethesda III–IV thyroid nodules remains challenging, and the possibility of integrating ultrasound-based artificial intelligence into the diagnostic pathway is certainly of interest. The use of postoperative histopathology as the reference standard, the comparison with experienced radiologists, and the simultaneous exploration of imaging-derived TIRADS and Bethesda-like outputs are potentially valuable aspects of the study.

However, several methodological and interpretative issues currently limit the strength of the conclusions. In particular, the manuscript sometimes moves from a small retrospective proof-of-concept evaluation to claims regarding superiority, generalizability, and clinical utility that are not fully supported by the present design. I believe that the study may become considerably stronger after substantial revision, particularly by clarifying the diagnostic framework, improving statistical reporting, and more cautiously contextualizing the findings.

Major comments

1. The diagnostic framework should clearly distinguish the two different TRAICK outputs.
   The system generates both an ACR-TIRADS category and an imaging-derived “Bethesda” category, but these outputs are sometimes discussed almost interchangeably. According to the Methods, binary malignancy for imaging assessment is defined as TIRADS ≥4, whereas Bethesda ≥V is used for the cytology-like output . However, Table 1 reports a single set of diagnostic performance measures for TRAICK alongside the two radiologists, apparently based on the TIRADS output , while several subsequent claims regarding “conservative behavior” derive from the Bethesda-like output. These represent different classification tasks and should be analyzed and reported separately. I would strongly recommend providing separate confusion matrices and diagnostic performance estimates for the TIRADS and Bethesda-like outputs, with explicit denominators and thresholds. The terminology “virtual cytology” should also be used cautiously, since an imaging-derived risk category is conceptually different from a cytological diagnosis.
2. The comparison with FNAB/Bethesda classification requires substantial reconsideration.
   The study specifically includes only nodules with Bethesda III or IV cytology . Therefore, FNAB cannot be treated as a conventional binary diagnostic comparator in the same manner as AI or TIRADS. In particular, Bethesda III should not be described as a “false-negative” cytological result: it is an indeterminate diagnostic category rather than a negative test. The statement that 16 malignant nodules initially classified as Bethesda III represent “cytological false-negative cases” should therefore be revised . Similarly, the statement that no nodules “classified as malignant by FNAB” were reassigned to low-risk categories is difficult to interpret because Bethesda V–VI cases were excluded by study design. I suggest reframing this component as a reclassification/concordance analysis within an indeterminate cytology cohort, rather than as evidence that TRAICK outperforms FNAB. The Abstract and Conclusions should be revised accordingly.
3. Selection and spectrum bias deserve greater emphasis.
   All patients had indeterminate cytology and subsequently underwent surgery, resulting in a malignancy prevalence of 65.4% . This is therefore a highly selected surgical cohort rather than a representative population of patients with Bethesda III–IV nodules. The consequences extend beyond PPV and NPV: spectrum selection may also influence sensitivity, specificity, and the apparent performance of radiologists, particularly because the benign nodules reaching surgery are likely to represent clinically or sonographically selected cases. The manuscript currently interprets the low specificity of the radiologists as evidence of a “pronounced tendency toward overdiagnosis” ; this conclusion should be softened. A patient flow diagram showing the total number of Bethesda III–IV nodules evaluated during the study period, the number proceeding to surgery, exclusions, and reasons for surgical treatment would greatly improve transparency.
4. Statistical reporting needs to be strengthened, particularly given the small sample size.
   The Methods state that McNemar's test and DeLong's test were used for comparison of diagnostic performance and ROC curves , but the corresponding pairwise statistical results are not clearly reported. At present, the manuscript repeatedly describes TRAICK as having “higher” or “superior” performance based predominantly on point estimates. With only 52 nodules, uncertainty around these estimates is likely substantial. Please report 95% confidence intervals for sensitivity, specificity, accuracy, PPV, NPV, kappa, and AUC, together with the results of the stated paired comparisons. Confidence intervals would also be preferable to p-values alone for the kappa analyses. If the AI system produces continuous probability estimates, the authors should clarify whether ROC analyses were based on these probabilities or on the final ordinal categories.
5. More information is needed on model development and true independence of validation.
   The manuscript refers to Appendix A for model architecture, training, and development , but this information is essential for judging the validity of the present analysis. The authors should clearly report the origin and size of the development dataset, training/validation/test splitting strategy, whether splitting occurred at patient level, potential overlap between training data and the current institution or population, image-selection procedures, number of images per nodule, segmentation process, threshold selection, calibration, and whether version 1.9.4 was locked before analysis of the present cohort. Importantly, the absence of local retraining does not by itself demonstrate “genuine generalizability”, as currently suggested . Generalizability requires independent external evaluation across different populations, institutions and ideally ultrasound platforms. The recently published STARD-AI guideline may provide a useful framework for improving transparent reporting of these aspects: Sounderajah V, Guni A, Liu X, et al. The STARD-AI reporting guideline for diagnostic accuracy studies using artificial intelligence. Nat Med. 2025;31:3283–3289. doi:10.1038/s41591-025-03953-8.
6. The radiologist comparison should be interpreted more cautiously.
   Only two radiologists were included, and one of them (D.Y.) appears both to have substantial involvement in the clinical ultrasound assessment and to have participated in development of the TRAICK system ; this relationship is appropriately disclosed in the Conflict of Interest statement . Nevertheless, this should be considered when interpreting the reader comparison. The current study supports comparison with two experienced readers in this dataset, rather than a general conclusion regarding performance versus “experienced radiologists”. Ideally, future validation should include readers fully independent from the development team. The authors should also confirm that radiologists and TRAICK received equivalent imaging information and clarify whether the human assessment was performed on static selected images or the complete ultrasound examination.
7. Imaging–pathology matching in multifocal disease is an important potential source of misclassification.
   The manuscript states that, in multifocal cases, the index lesion was selected according to sonographic suspicion, size and FNAB results , while acknowledging the absence of formal nodule-to-nodule histopathological verification . This may be particularly relevant in a diagnostic accuracy study, since malignancy detected elsewhere in the thyroid could potentially be attributed to the ultrasound lesion analyzed by the AI system. The authors should report how many patients had multifocal disease and, if feasible, perform a sensitivity analysis restricted to nodules with secure imaging–cytology–histology correspondence.
8. The Discussion should be updated and the reported performance placed within the most recent evidence.
   The authors appropriately cite the recent systematic review and meta-analysis by Jassal et al., which is particularly relevant because it specifically addresses AI for indeterminate thyroid nodules. However, the literature has evolved rapidly. More recent meta-analyses have explored AI diagnostic performance, external validation and comparison with radiologists, and would provide an important contemporary context for the present results. Relevant examples include: Zhan J, Zhang J, Zhu S, et al. Diagnostic performance of ultrasound characteristics-based artificial intelligence models for thyroid nodules: a systematic review and meta-analysis. Front Oncol. 2025;15:1614603. doi:10.3389/fonc.2025.1614603; Gatta E, Gatta R, Morandi R, et al. Machine Learning for diagnosis of malignant thyroid nodules based on thyroid ultrasound: Systematic review and meta-analysis of studies with external datasets. Eur J Radiol Open. 2026;16:100716. doi:10.1016/j.ejro.2025.100716; De Cicco R, Dias FA, Vieira VB, Souza R, Briaunys Milan T. Artificial intelligence-based systems as auxiliary tools for thyroid nodule malignancy detection on ultrasound: a systematic review and meta-analysis of diagnostic accuracy. Gland Surg. 2026;15(7):186. doi:10.21037/gs-2026-1-0126. These broader meta-analyses should not be directly compared numerically with the present highly selected Bethesda III–IV cohort, but they are particularly useful for discussing external validation, heterogeneity, and AI–radiologist comparisons. This distinction should itself be clearly acknowledged.
9. The clinical implementation discussion could be strengthened by considering communication of uncertainty and shared decision-making.
   This is particularly relevant for the proposed concept of “virtual cytology”. Patients with Bethesda III–IV nodules already face substantial diagnostic uncertainty; introducing an additional probabilistic AI-derived risk estimate may reduce uncertainty in some cases, but may also generate discordant information, for example, when cytology, ultrasound and AI-derived risk categories point in different directions. Therefore, the clinical value of such a system should not be considered only in terms of diagnostic discrimination, but also in terms of how this information is communicated and incorporated into patient-centred decision-making. Recent work specifically addressing structured communication of uncertainty in patients with indeterminate thyroid cytology may offer a useful perspective for expanding this discussion: Melcarne R, Consorti F, Grani G, Engel TD, Iannuzzi E, Ammendolia C, Giacomelli L, Durante C, Biffoni M. Designing a SPIKES-based protocol for communicating uncertainty in indeterminate thyroid cytology: a mixed-method analysis within a pilot study. Updates Surg. 2026;78(1):75–87. doi:10.1007/s13304-025-02346-3. This perspective could help the authors frame TRAICK not simply as a tool producing an additional diagnostic label, but as a potential component of a broader shared decision-making pathway.
10. Some clinical implications currently go beyond the outcomes assessed in this study.
    The Conclusions suggest that TRAICK may reduce unnecessary repeat biopsies and refine management . These are clinically meaningful hypotheses, but repeat biopsy rates, surgery avoidance, changes in management and patient outcomes were not evaluated in the present cohort. These statements should therefore be reframed as potential benefits requiring prospective assessment. A future study evaluating management impact, or, where appropriate, decision-curve/net-benefit analysis, would represent an important next step. Similarly, statements that all TIRADS 5 lesions were malignant should be interpreted in light of the very small denominator, and expressions such as “strong evidence” or “reliability” should be avoided when based on only a few observations.

Minor comments

The title may be slightly too strong in describing this as a “Clinical Validation Study”; “retrospective proof-of-concept evaluation” would more accurately reflect the present design. The terminology should consistently use ACR TI-RADS. The reference for the Bethesda System should be updated to the 2023 edition, which revised category terminology and risks of malignancy: Ali SZ, Baloch ZW, Cochand-Priollet B, et al. The 2023 Bethesda System for Reporting Thyroid Cytopathology. Thyroid. 2023;33(9):1039–1044. doi:10.1089/thy.2023.0141. The treatment of NIFTP as “benign” should be explicitly justified and the number of NIFTP cases reported; if present, a sensitivity analysis considering alternative dichotomization would be informative. In Table 3, confidence intervals around malignancy rates would help avoid overinterpretation of very small subgroups. Figure 4 would be more informative if a corresponding confusion matrix for TRAICK were shown. Finally, “prognostic significance” in the Discussion should be replaced by “diagnostic significance” or “diagnostic discrimination”, since no prognostic outcome is being assessed.

Overall assessment

This is an interesting proof-of-concept study addressing an important problem in thyroid nodule management. The simultaneous generation of ultrasound-based TIRADS and Bethesda-like risk information is potentially innovative. However, the manuscript would benefit substantially from a clearer separation of these two outputs, more complete statistical reporting, a more rigorous discussion of selection bias and model validation, and more cautious interpretation of clinical implications. With these revisions, the paper could provide a useful contribution to the evolving literature on AI-assisted evaluation of indeterminate thyroid nodules.

Author Response

Reviewer 2

The manuscript entitled “Artificial Intelligence–Assisted TIRADS and Bethesda Prediction for Indeterminate Thyroid Nodules: A Proof-of-Concept Clinical Validation Study” addresses a timely and clinically relevant topic. The management of Bethesda III–IV thyroid nodules remains challenging, and the possibility of integrating ultrasound-based artificial intelligence into the diagnostic pathway is certainly of interest. The use of postoperative histopathology as the reference standard, the comparison with experienced radiologists, and the simultaneous exploration of imaging-derived TIRADS and Bethesda-like outputs are potentially valuable aspects of the study.

However, several methodological and interpretative issues currently limit the strength of the conclusions. In particular, the manuscript sometimes moves from a small retrospective proof-of-concept evaluation to claims regarding superiority, generalizability, and clinical utility that are not fully supported by the present design. I believe that the study may become considerably stronger after substantial revision, particularly by clarifying the diagnostic framework, improving statistical reporting, and more cautiously contextualizing the findings.

 

We agree that the original manuscript sometimes went beyond what a retrospective study can support. In this revision we analyzed the two TRAICK outputs separately, redid all statistics with full reporting of confidence intervals and paired comparisons, and revised the interpretation throughout. The title now describes the study as a retrospective proof-of-concept evaluation. In addition, as explained in the note at the beginning of this letter, the reference standard is now the histopathology of the index nodule, and all results were recalculated. Our point-by-point responses follow.

 

Major comments

1. The diagnostic framework should clearly distinguish the two different TRAICK outputs.

The system generates both an ACR-TIRADS category and an imaging-derived “Bethesda” category, but these outputs are sometimes discussed almost interchangeably. According to the Methods, binary malignancy for imaging assessment is defined as TIRADS ≥4, whereas Bethesda ≥V is used for the cytology-like output . However, Table 1 reports a single set of diagnostic performance measures for TRAICK alongside the two radiologists, apparently based on the TIRADS output , while several subsequent claims regarding “conservative behavior” derive from the Bethesda-like output. These represent different classification tasks and should be analyzed and reported separately. I would strongly recommend providing separate confusion matrices and diagnostic performance estimates for the TIRADS and Bethesda-like outputs, with explicit denominators and thresholds. The terminology “virtual cytology” should also be used cautiously, since an imaging-derived risk category is conceptually different from a cytological diagnosis.

 

The two outputs address different classification tasks and are now analyzed and reported separately throughout the manuscript. The Methods define a separate prespecified threshold for each output (TR4–TR5 for the ACR TI-RADS output; categories V–VI for the Bethesda-like output), and neither threshold was optimized on the study data. Table 1 now reports the two outputs in separate rows, with numerators, denominators and 95% confidence intervals. Figure 5 shows separate confusion matrices for Radiologist 1, Radiologist 2, the TRAICK ACR TI-RADS output and the TRAICK Bethesda-like output. Figure 6 shows separate ROC curves for the two outputs. We removed the term “virtual cytology” and the statements about “conservative behavior”. The Bethesda-like output is now described as follows: “The Bethesda-like output is an estimate of cytological risk based on image features. It is not a cytological diagnosis, and it is intended to complement FNAB rather than replace it.” (Introduction, Methods, Results)

 

2. The comparison with FNAB/Bethesda classification requires substantial reconsideration.

The study specifically includes only nodules with Bethesda III or IV cytology . Therefore, FNAB cannot be treated as a conventional binary diagnostic comparator in the same manner as AI or TIRADS. In particular, Bethesda III should not be described as a “false-negative” cytological result: it is an indeterminate diagnostic category rather than a negative test. The statement that 16 malignant nodules initially classified as Bethesda III represent “cytological false-negative cases” should therefore be revised . Similarly, the statement that no nodules “classified as malignant by FNAB” were reassigned to low-risk categories is difficult to interpret because Bethesda V–VI cases were excluded by study design. I suggest reframing this component as a reclassification/concordance analysis within an indeterminate cytology cohort, rather than as evidence that TRAICK outperforms FNAB. The Abstract and Conclusions should be revised accordingly.

 

FNAB is no longer treated as a binary diagnostic comparator. We removed the description of Bethesda III nodules as “cytological false-negative cases” and the statement about nodules “classified as malignant by FNAB”. This part of the study is now presented as a reclassification analysis within the indeterminate cytology cohort, stratified by final histopathology. The Methods state: “Because all nodules had indeterminate (Bethesda III–IV) cytology by design, FNAB cytology was not treated as a binary diagnostic comparator. Instead, we described how the TRAICK Bethesda-like output redistributed nodules within each cytological category (reclassification analysis).” The Results state that these findings are not a comparison of diagnostic accuracy between TRAICK and FNAB, and the Discussion states that they “should not be read as evidence that TRAICK outperforms FNAB”. We revised the Abstract and Conclusions accordingly; the Conclusions now state that the findings “do not show superiority over radiologists or FNAB”. (Methods, Results, Discussion; Conclusions)

 

3. Selection and spectrum bias deserve greater emphasis.

All patients had indeterminate cytology and subsequently underwent surgery, resulting in a malignancy prevalence of 65.4% . This is therefore a highly selected surgical cohort rather than a representative population of patients with Bethesda III–IV nodules. The consequences extend beyond PPV and NPV: spectrum selection may also influence sensitivity, specificity, and the apparent performance of radiologists, particularly because the benign nodules reaching surgery are likely to represent clinically or sonographically selected cases. The manuscript currently interprets the low specificity of the radiologists as evidence of a “pronounced tendency toward overdiagnosis” ; this conclusion should be softened. A patient flow diagram showing the total number of Bethesda III–IV nodules evaluated during the study period, the number proceeding to surgery, exclusions, and reasons for surgical treatment would greatly improve transparency.

 

As explained in the note at the beginning of this letter, with the revised index-nodule reference standard the malignancy rate is 44.2% (23 of 52 nodules), which is still higher than expected for Bethesda III–IV nodules. We agree that this is a selected surgical cohort and have made this clearer in several places. First, we added a flow diagram (Figure 1) showing the 93 nodules with Bethesda III–IV cytology during the study period, the 58 that were operated on and the 35 that were not, the 6 exclusions with their reasons, and the 52 nodules included. Second, the reasons for surgery are now given in the Methods. Third, we added a paragraph on spectrum bias: “Because only operated nodules could be verified by histopathology, the study population is a selected surgical cohort. It is likely to contain a higher proportion of malignant nodules than the full population of patients with Bethesda III–IV cytology, and the nonmalignant nodules that reached surgery were probably selected on clinical or sonographic grounds. This spectrum may affect sensitivity and specificity, not only the predictive values, and it was taken into account when interpreting the results.” Fourth, we removed the phrase “pronounced tendency toward overdiagnosis”. The Discussion now explains that the low specificity of the radiologists “may partly reflect this selection rather than a general tendency to overcall”. Finally, spectrum bias is listed as the second limitation. (Methods, Discussion; Limitations)

 

 

4. Statistical reporting needs to be strengthened, particularly given the small sample size.

The Methods state that McNemar's test and DeLong's test were used for comparison of diagnostic performance and ROC curves , but the corresponding pairwise statistical results are not clearly reported. At present, the manuscript repeatedly describes TRAICK as having “higher” or “superior” performance based predominantly on point estimates. With only 52 nodules, uncertainty around these estimates is likely substantial. Please report 95% confidence intervals for sensitivity, specificity, accuracy, PPV, NPV, kappa, and AUC, together with the results of the stated paired comparisons. Confidence intervals would also be preferable to p-values alone for the kappa analyses. If the AI system produces continuous probability estimates, the authors should clarify whether ROC analyses were based on these probabilities or on the final ordinal categories.

 

Sensitivity, specificity, accuracy, PPV and NPV are now reported with numerators, denominators and Wilson 95% confidence intervals (Table 1). κ values are now reported with 95% confidence intervals rather than p values alone (Tables 1 and 2), and AUCs with DeLong 95% confidence intervals (Table 1, Figure 6). All paired comparisons are now reported in the Results and in Supplementary Table S1. These are the exact McNemar test for accuracy, sensitivity and specificity, and the DeLong test for AUC. We now use words such as “higher” only when a difference was statistically significant. For example, TRAICK did not differ significantly from Radiologist 1 in accuracy (p = 0.267) or AUC (p = 0.783) but had a higher specificity (p = 0.021), and it had a higher accuracy (p = 0.013), specificity (p < 0.001), and AUC (p = 0.042) than Radiologist 2. TRAICK does not produce a continuous probability and reports final categories only, so all ROC analyses were based on the ordinal categories. This is stated in the Methods, in the Figure 6 legend and in the Limitations: “The system reports final categories only; it does not export a continuous malignancy probability.” (Methods, Results;  Supplementary Table S1)

 

5. More information is needed on model development and true independence of validation.

The manuscript refers to Appendix A for model architecture, training, and development , but this information is essential for judging the validity of the present analysis. The authors should clearly report the origin and size of the development dataset, training/validation/test splitting strategy, whether splitting occurred at patient level, potential overlap between training data and the current institution or population, image-selection procedures, number of images per nodule, segmentation process, threshold selection, calibration, and whether version 1.9.4 was locked before analysis of the present cohort. Importantly, the absence of local retraining does not by itself demonstrate “genuine generalizability”, as currently suggested . Generalizability requires independent external evaluation across different populations, institutions and ideally ultrasound platforms. The recently published STARD-AI guideline may provide a useful framework for improving transparent reporting of these aspects: Sounderajah V, Guni A, Liu X, et al. The STARD-AI reporting guideline for diagnostic accuracy studies using artificial intelligence. Nat Med. 2025;31:3283–3289. doi:10.1038/s41591-025-03953-8.

 

We added the requested information to the Methods and to Appendix 1:

• Origin and size: 9,185 gray-scale ultrasound images from several institutions (including Istanbul Medeniyet University Göztepe Training and Research Hospital and 17 hospitals of the Acıbadem Healthcare Group, to which our institution belongs).
• Splitting: approximately 52% training, 22% validation and 26% test. The test set was separated at the patient level, so no test patient contributed images to training or validation; the validation set was drawn at random from the remaining images and was therefore not separated at the patient level. This applies to the ACR TI-RADS feature classifiers; the segmentation model was developed with a separate, image-level split (Appendix 1, Section 7). We acknowledge in the Limitations that the internal test results in Appendix 1 may be optimistic. This does not apply to the present cohort.
• Overlap: hospitals of the Acıbadem Healthcare Group, including our own, contributed development images from other patients. None of the 52 patients in the present cohort contributed images to the development dataset, and the cohort was not used for training, validation, or testing. The present analysis is therefore an evaluation in patients independent of the development data rather than a fully external evaluation, and we now state this in the Methods and the Limitations.
• Image selection and number of images: four to five gray-scale images are stored per nodule in routine practice. One pure gray-scale image per nodule, without calipers, Doppler, elastography or markers, was selected by a reader blinded to cytology and histopathology.
• Segmentation: a YOLOv8-L instance-segmentation module, trained from scratch, locates and segments the nodule (Figure 3). On an internal test set of 227 images, the mean Dice coefficient against the annotators' outlines was 0.826 (95% CI 0.789–0.859), and the nodule was detected in 94.3% of the images (Appendix 1, Section 7).
• Training: Appendix 1 now gives the training settings (framework, optimizer, learning rate, batch size, early stopping and loss), the data augmentation used, and the definition of the performance metrics in Table A2. Synthetic images were used only to train the segmentation model and were not used for validation or testing.
• Thresholds and calibration: feature-level outputs are converted to ACR TI-RADS points and summed using the standard ACR thresholds. The Bethesda-like output is the category with the highest predicted probability, with no additional threshold. The analysis thresholds were prespecified, and the model was not recalibrated on these data.
• Model version: the analysis was performed with the model as provided by the development team. It was not retrained, fine-tuned or recalibrated on the present data, and no thresholds were adjusted. This is now stated in the Methods: “The model was not retrained, fine-tuned, or recalibrated on these data, and no thresholds were adjusted.” (Methods, TRAICK AI System)

We also removed the claim of “genuine generalizability”. The Discussion now states that “a single site does not by itself establish generalizability, which will require validation in multiple independent centers”. We cited the STARD-AI guideline [14] and provide the completed checklist, with page and line numbers, as Supplementary File S1. (Methods, Discussion; Limitations; Appendix 1; Supplementary File S1)

 

 

6. The radiologist comparison should be interpreted more cautiously.

Only two radiologists were included, and one of them (D.Y.) appears both to have substantial involvement in the clinical ultrasound assessment and to have participated in development of the TRAICK system ; this relationship is appropriately disclosed in the Conflict of Interest statement . Nevertheless, this should be considered when interpreting the reader comparison. The current study supports comparison with two experienced readers in this dataset, rather than a general conclusion regarding performance versus “experienced radiologists”. Ideally, future validation should include readers fully independent from the development team. The authors should also confirm that radiologists and TRAICK received equivalent imaging information and clarify whether the human assessment was performed on static selected images or the complete ultrasound examination.

 

First, the Methods now describe the role of D.Y. in full. He was involved in TRAICK development and performed the original examinations. A washout interval of at least 3 months was used, and he did not run the TRAICK analysis or the statistical analysis. Radiologist 2 has no affiliation with TRAICK, and all results are reported separately for each reader. Second, on the imaging information: both radiologists and TRAICK assessed the same single static gray-scale image of each nodule, without cine loops or clinical reports. The Discussion notes that this differs from routine real-time examination. Third, the Discussion now reads: “Our results therefore describe a comparison with two readers in this dataset and should not be generalized to radiologists as a group. Future validation studies should include multiple readers who are fully independent of the development team.” We also updated the Conflicts of Interest statement to match the Methods. (Methods, Discussion; Conflicts of Interest)

 

7. Imaging–pathology matching in multifocal disease is an important potential source of misclassification.

The manuscript states that, in multifocal cases, the index lesion was selected according to sonographic suspicion, size and FNAB results , while acknowledging the absence of formal nodule-to-nodule histopathological verification . This may be particularly relevant in a diagnostic accuracy study, since malignancy detected elsewhere in the thyroid could potentially be attributed to the ultrasound lesion analyzed by the AI system. The authors should report how many patients had multifocal disease and, if feasible, perform a sensitivity analysis restricted to nodules with secure imaging–cytology–histology correspondence.

 

We added a new Methods subsection on imaging–pathology correlation in multifocal disease. The index nodule was the nodule that had undergone FNAB, and it was matched to the histopathology report by lobe, location and size. Three operated nodules were excluded because the lobe contained several adjacent nodules and the match could not be made with confidence. Four of the 52 included patients had more than one nodule on ultrasound, and in each of them the index nodule could be matched to the histopathology report. In six included cases, papillary microcarcinoma was found outside the index nodule; on pathology review, the index nodule itself was benign in five of these cases and a well-differentiated tumor of uncertain malignant potential (WT-UMP) in one. We agree with the reviewer that malignancy elsewhere in the gland should not be attributed to the imaged nodule. These six nodules are therefore now classified according to the histology of the index nodule (nonmalignant) in the primary analysis. As the reviewer suggested, we also repeated the analysis after excluding these six nodules, so that only nodules with a secure imaging–cytology–histology match were included (n = 46). The AUC was 0.820 for the TRAICK ACR TI-RADS output, 0.802 for Radiologist 1 (p = 0.732) and 0.634 for Radiologist 2 (p = 0.006), and 0.851 for the Bethesda-like output. A further sensitivity analysis using the original coding, in which these nodules were counted as malignant, is also reported. The main findings did not change. (Methods; Results; Supplementary Table S2)

 

8. The Discussion should be updated and the reported performance placed within the most recent evidence.

The authors appropriately cite the recent systematic review and meta-analysis by Jassal et al., which is particularly relevant because it specifically addresses AI for indeterminate thyroid nodules. However, the literature has evolved rapidly. More recent meta-analyses have explored AI diagnostic performance, external validation and comparison with radiologists, and would provide an important contemporary context for the present results. Relevant examples include: Zhan J, Zhang J, Zhu S, et al. Diagnostic performance of ultrasound characteristics-based artificial intelligence models for thyroid nodules: a systematic review and meta-analysis. Front Oncol. 2025;15:1614603. doi:10.3389/fonc.2025.1614603; Gatta E, Gatta R, Morandi R, et al. Machine Learning for diagnosis of malignant thyroid nodules based on thyroid ultrasound: Systematic review and meta-analysis of studies with external datasets. Eur J Radiol Open. 2026;16:100716. doi:10.1016/j.ejro.2025.100716; De Cicco R, Dias FA, Vieira VB, Souza R, Briaunys Milan T. Artificial intelligence-based systems as auxiliary tools for thyroid nodule malignancy detection on ultrasound: a systematic review and meta-analysis of diagnostic accuracy. Gland Surg. 2026;15(7):186. doi:10.21037/gs-2026-1-0126. These broader meta-analyses should not be directly compared numerically with the present highly selected Bethesda III–IV cohort, but they are particularly useful for discussing external validation, heterogeneity, and AI–radiologist comparisons. This distinction should itself be clearly acknowledged.

 

We added the three meta-analyses to the Discussion [10–12] and corrected our description of the Jassal et al. meta-analysis [13]. As the reviewer suggested, we also state the distinction explicitly: “These pooled estimates come mostly from broader or mixed populations and should not be compared numerically with the present small surgical cohort of Bethesda III–IV nodules. They do, however, highlight two points that also apply to our study. First, performance measured in a single center may not be maintained on data from other institutions and ultrasound platforms. Second, although no patient of the present cohort contributed to model development, hospitals of the same group did, and a single site does not by itself establish generalizability, which will require validation in multiple independent centers.” (Discussion)

 

9. The clinical implementation discussion could be strengthened by considering communication of uncertainty and shared decision-making.

This is particularly relevant for the proposed concept of “virtual cytology”. Patients with Bethesda III–IV nodules already face substantial diagnostic uncertainty; introducing an additional probabilistic AI-derived risk estimate may reduce uncertainty in some cases, but may also generate discordant information, for example, when cytology, ultrasound and AI-derived risk categories point in different directions. Therefore, the clinical value of such a system should not be considered only in terms of diagnostic discrimination, but also in terms of how this information is communicated and incorporated into patient-centred decision-making. Recent work specifically addressing structured communication of uncertainty in patients with indeterminate thyroid cytology may offer a useful perspective for expanding this discussion: Melcarne R, Consorti F, Grani G, Engel TD, Iannuzzi E, Ammendolia C, Giacomelli L, Durante C, Biffoni M. Designing a SPIKES-based protocol for communicating uncertainty in indeterminate thyroid cytology: a mixed-method analysis within a pilot study. Updates Surg. 2026;78(1):75–87. doi:10.1007/s13304-025-02346-3. This perspective could help the authors frame TRAICK not simply as a tool producing an additional diagnostic label, but as a potential component of a broader shared decision-making pathway.

 

We added a new subsection and cited the suggested study [21]: “In practice, AI-derived risk categories will sometimes agree and sometimes conflict with cytology and with the radiologist’s assessment. Patients with indeterminate cytology already face considerable uncertainty, and discordant information may increase rather than reduce this uncertainty if it is not communicated carefully. Structured approaches to communicating uncertainty in indeterminate thyroid cytology have recently been proposed [21], and AI-derived information should be incorporated into such shared decision-making rather than presented as an additional diagnosis.” (Discussion)

 

10. Some clinical implications currently go beyond the outcomes assessed in this study.

The Conclusions suggest that TRAICK may reduce unnecessary repeat biopsies and refine management . These are clinically meaningful hypotheses, but repeat biopsy rates, surgery avoidance, changes in management and patient outcomes were not evaluated in the present cohort. These statements should therefore be reframed as potential benefits requiring prospective assessment. A future study evaluating management impact, or, where appropriate, decision-curve/net-benefit analysis, would represent an important next step. Similarly, statements that all TIRADS 5 lesions were malignant should be interpreted in light of the very small denominator, and expressions such as “strong evidence” or “reliability” should be avoided when based on only a few observations.

 

We removed the statements about reducing repeat biopsies and refining management from the Abstract and Conclusions. The Discussion now reads: “The present study did not assess whether TRAICK changes clinical decisions, repeat biopsy rates, surgery rates, or patient outcomes. These potential benefits remain hypotheses. They should be tested in prospective studies of management impact, and decision-curve or net-benefit analyses in larger, unselected cohorts would help to define the range of risk thresholds at which the system might be clinically useful.” The TR5 result is now given with its denominator and confidence interval (7/8; 87.5%, 95% CI 52.9–97.8), and the text notes that the subgroups were small and the intervals wide. We also removed the expressions “strong evidence” and “reliability”. (Discussion; Conclusions; Results)

 

 

Minor comments

The title may be slightly too strong in describing this as a “Clinical Validation Study”; “retrospective proof-of-concept evaluation” would more accurately reflect the present design. The terminology should consistently use ACR TI-RADS. The reference for the Bethesda System should be updated to the 2023 edition, which revised category terminology and risks of malignancy: Ali SZ, Baloch ZW, Cochand-Priollet B, et al. The 2023 Bethesda System for Reporting Thyroid Cytopathology. Thyroid. 2023;33(9):1039–1044. doi:10.1089/thy.2023.0141. The treatment of NIFTP as “benign” should be explicitly justified and the number of NIFTP cases reported; if present, a sensitivity analysis considering alternative dichotomization would be informative. In Table 3, confidence intervals around malignancy rates would help avoid overinterpretation of very small subgroups. Figure 4 would be more informative if a corresponding confusion matrix for TRAICK were shown. Finally, “prognostic significance” in the Discussion should be replaced by “diagnostic significance” or “diagnostic discrimination”, since no prognostic outcome is being assessed.

• Title: Changed as suggested to “…: A Retrospective Proof-of-Concept Evaluation”. (Title; Abstract; Methods)
• ACR TI-RADS: The term is now used consistently throughout the manuscript, keywords, tables, figures and Appendix 1.
• Bethesda 2023: The 2017 reference was replaced by the 2023 edition [7], and category names now follow this edition. (Introduction; Methods)
• NIFTP: Six of the 52 nodules were NIFTP and one was a WT-UMP, and these numbers are now reported. The WHO Classification of Tumours (5th edition) does not consider these tumors carcinomas but low-risk neoplasms that are usually treated by lobectomy alone [15]. In the first submission, NIFTP cases had been coded as recorded in the study database (five as malignant and one as benign), which was inconsistent. In the revised primary analysis, all low-risk neoplasms are classified as nonmalignant, in line with the WHO classification. As the reviewer suggested, we repeated the analysis with all of them classified as malignant: the AUC was 0.654 for the TRAICK ACR TI-RADS output, 0.688 for Radiologist 1 (p = 0.592) and 0.526 for Radiologist 2 (p = 0.081). In this analysis, the AUCs of both radiologists and the TRAICK ACR TI-RADS output decreased, whereas the AUC of the Bethesda-like output increased slightly (from 0.822 to 0.838). TRAICK still did not differ significantly from Radiologist 1, but its AUC advantage over Radiologist 2 was no longer statistically significant. (Methods; Results; Supplementary Table S2)
• Table 3: This table is now Table 4. Each malignancy rate is given as n/N with a Wilson 95% CI and a test for trend, and the footnote notes that small subgroups should be read with caution. (Table 4)
• Figure 4: This figure is now Figure 5 and shows confusion matrices for both radiologists and for both TRAICK outputs. (Figure 5)
• “Prognostic significance”: Replaced. The Discussion now reads: “These rates indicate diagnostic discrimination in this cohort…” (Discussion)

 

Overall assessment

This is an interesting proof-of-concept study addressing an important problem in thyroid nodule management. The simultaneous generation of ultrasound-based TIRADS and Bethesda-like risk information is potentially innovative. However, the manuscript would benefit substantially from a clearer separation of these two outputs, more complete statistical reporting, a more rigorous discussion of selection bias and model validation, and more cautious interpretation of clinical implications. With these revisions, the paper could provide a useful contribution to the evolving literature on AI-assisted evaluation of indeterminate thyroid nodules.

 

• Separation of the two outputs. The ACR TI-RADS and Bethesda-like outputs are now analyzed and reported separately, each with its own prespecified threshold, confusion matrix, performance estimates and ROC curve (Table 1; Figures 5 and 6).
• Statistical reporting. All estimates are now given with numerators, denominators and 95% confidence intervals, and all paired McNemar and DeLong comparisons are reported (Tables 1 and 2; Supplementary Table S1).
• Selection bias and model validation. We added a patient flow diagram (Figure 1) and a description of the selected surgical cohort and its effect on sensitivity and specificity. We also report how the model was developed, that it was not retrained or recalibrated on the present data, and that no patient of the present cohort contributed to its development (Methods; Appendix 1; Supplementary File S1). Sensitivity analyses address multifocal disease and the classification of NIFTP (Supplementary Table S2).
• Clinical implications. The Discussion and Conclusions were rewritten more cautiously. Possible clinical benefits are now presented as hypotheses that need prospective testing, and we discuss how AI-derived information should be communicated within shared decision-making.

 

 

Reviewer 3 Report

Comments and Suggestions for Authors

1.  Comparison with the radiologists
The authors state that TRAICK performed better than the radiologists. However, the study includes only 52 nodules. In addition, although McNemar and DeLong tests are mentioned in the Methods, the results of these comparisons are not reported. Please provide the corresponding p-values and avoid using terms such as “superior” unless the differences are statistically significant.

3. High malignancy rate
The malignancy rate in this cohort was very high (65.4%), probably because only patients who underwent surgery were included. This may affect PPV and NPV and limits the applicability of the results to routine clinical practice. The authors should emphasize this point when discussing the clinical value of the model.

3. Conclusions should be more cautious
The authors suggest that TRAICK may reduce unnecessary repeat biopsies. However, this was not directly evaluated in the present study. This statement should therefore be presented as a possible future clinical benefit rather than a conclusion of the current study.

 

Supplement 1. What is the main question addressed by the research?
The study investigates whether the TRAICK AI system can improve the assessment of cytologically indeterminate thyroid nodules (Bethesda III–IV) by generating TIRADS and Bethesda-like predictions directly from ultrasound images and comparing its performance with experienced radiologists and final histopathology.
2. Do you consider the topic original or relevant to the field? Does it address a specific gap in the field? Please also explain why this is/is not the case.
Yes. The topic is clinically relevant because the management of Bethesda III–IV thyroid nodules remains a common diagnostic problem. The study also addresses an interesting gap, as most AI-based ultrasound systems focus on TIRADS classification, whereas TRAICK provides both TIRADS and Bethesda-like predictions from ultrasound images.  
3. What does it add to the subject area compared with other published material?
The main addition is the simultaneous generation of TIRADS and Bethesda-like categories from ultrasound images in patients who already have indeterminate cytology. The use of surgical histopathology as the reference standard is also a strength.  
4. What specific improvements should the authors consider regarding the methodology?
The statistical comparison between TRAICK and the radiologists should be reported more completely. Although McNemar and DeLong tests are described in the Methods, the corresponding results and p-values should be presented. In addition, the effect of the highly selected surgical cohort and the high malignancy prevalence should be considered more carefully when interpreting diagnostic performance, particularly PPV and NPV.  
5. Are the conclusions consistent with the evidence and arguments presented and do they address the main question posed? Please also explain why this is/is not the case.
Some statements are stronger than the available evidence. In particular, the suggestion that TRAICK may reduce unnecessary repeat biopsies was not directly evaluated in this study.  
6. Are the references appropriate?
Yes. I have no major concerns regarding the reference list.
7. Any additional comments on the tables and figures.
The tables and figures are generally clear and helpful.   Table 3 includes several subgroups with very small numbers of nodules, and the reported malignancy rates, particularly the 100% rates in some TIRADS categories, should therefore be interpreted cautiously (I would prefer numbers ) .

Author Response

Reviewer 3

1. Comparison with the radiologists

The authors state that TRAICK performed better than the radiologists. However, the study includes only 52 nodules. In addition, although McNemar and DeLong tests are mentioned in the Methods, the results of these comparisons are not reported. Please provide the corresponding p-values and avoid using terms such as “superior” unless the differences are statistically significant.

 

All paired comparisons are now reported with p values. TRAICK did not differ significantly from Radiologist 1 in accuracy (71.2% vs. 61.5%; McNemar p = 0.267), sensitivity (p = 0.250), or AUC (0.762 vs. 0.777; DeLong p = 0.783), but had a higher specificity (65.5% vs. 37.9%; p = 0.021). Compared with Radiologist 2, TRAICK had a higher accuracy (p = 0.013), specificity (p < 0.001), and AUC (p = 0.042), with no significant difference in sensitivity. We removed the word “superior” and now state only that the AUC of TRAICK did not differ significantly from that of Radiologist 1. The Conclusions state that the findings “do not show superiority over radiologists or FNAB”. (Abstract; Results; Conclusions)

 

2. High malignancy rate

The malignancy rate in this cohort was very high (65.4%), probably because only patients who underwent surgery were included. This may affect PPV and NPV and limits the applicability of the results to routine clinical practice. The authors should emphasize this point when discussing the clinical value of the model.

 

The Methods now explain that the study population is a selected surgical cohort, which is likely to contain a higher proportion of malignant nodules than the full population of patients with Bethesda III–IV cytology. They also note that this may affect sensitivity and specificity as well as the predictive values. A new flow diagram (Figure 1) shows that 35 of the 93 nodules with indeterminate cytology were not operated on. With the revised, index-nodule reference standard, the malignancy rate is 44.2%, which is still higher than expected for Bethesda III–IV nodules. This point is listed as the second limitation: “This selected surgical cohort had a malignancy rate of 44.2%, higher than expected for Bethesda III–IV nodules, and spectrum bias may have affected sensitivity and specificity as well as the predictive values; 35 of the 93 nodules with indeterminate cytology were not operated on and could not be evaluated.” We have also added this point to the discussion of clinical value: “In addition, the predictive values observed here reflect the malignancy rate of this surgical cohort (44.2%), which is higher than the expected risk of malignancy of Bethesda III–IV nodules [7], and would be expected to differ in routine practice.” (Methods; Discussion; Limitations)

 

3. Conclusions should be more cautious

The authors suggest that TRAICK may reduce unnecessary repeat biopsies. However, this was not directly evaluated in the present study. This statement should therefore be presented as a possible future clinical benefit rather than a conclusion of the current study.

 

We removed the statement about reducing repeat biopsies from the Abstract and Conclusions. The Conclusions now state that TRAICK “has the potential to provide supplementary information to support preoperative decision-making” and that the findings do not show “a reduction in repeat biopsies or surgery”. The Discussion presents these possible benefits as hypotheses for prospective studies: “These potential benefits remain hypotheses. They should be tested in prospective studies of management impact…” (Discussion, Conclusions)

 

Supplement 1. What is the main question addressed by the research?

The study investigates whether the TRAICK AI system can improve the assessment of cytologically indeterminate thyroid nodules (Bethesda III–IV) by generating TIRADS and Bethesda-like predictions directly from ultrasound images and comparing its performance with experienced radiologists and final histopathology.

 

In the revised manuscript, we have stated the aim more precisely to match the retrospective proof-of-concept design. The aim was to evaluate the diagnostic performance of the two TRAICK outputs in surgically resected Bethesda III–IV nodules, using postoperative histopathology as the reference standard. We also compared the ACR TI-RADS output with two radiologists and described how the Bethesda-like output redistributed nodules within the indeterminate cytology categories. Whether TRAICK actually improves clinical assessment or management was not tested, and we now state this clearly as a question for prospective studies. (Introduction, Methods, Discussion)

 

2. Do you consider the topic original or relevant to the field? Does it address a specific gap in the field? Please also explain why this is/is not the case.

Yes. The topic is clinically relevant because the management of Bethesda III–IV thyroid nodules remains a common diagnostic problem. The study also addresses an interesting gap, as most AI-based ultrasound systems focus on TIRADS classification, whereas TRAICK provides both TIRADS and Bethesda-like predictions from ultrasound images. 

 

We have kept this point in the Introduction, which notes that most AI approaches focus only on ACR TI-RADS classification and that few have been evaluated specifically in cytologically indeterminate nodules [13]. To avoid overstating the second output, we now describe it as an imaging-derived, Bethesda-like risk category. It is not a cytological diagnosis and is intended to complement FNAB rather than replace it. (Introduction; Methods)

 

3. What does it add to the subject area compared with other published material?

The main addition is the simultaneous generation of TIRADS and Bethesda-like categories from ultrasound images in patients who already have indeterminate cytology. The use of surgical histopathology as the reference standard is also a strength. 

 

We agree that the main contribution is the simultaneous generation of ACR TI-RADS and Bethesda-like categories in nodules that already have indeterminate cytology, with surgical histopathology as the reference standard. The Strengths paragraph now notes the separate evaluation of the two TRAICK outputs in nodules with indeterminate cytology and the use of postoperative histopathology as the reference standard for all nodules, together with the evaluation of the model in patients who were not part of its development data (Discussion)

 

4. What specific improvements should the authors consider regarding the methodology?

The statistical comparison between TRAICK and the radiologists should be reported more completely. Although McNemar and DeLong tests are described in the Methods, the corresponding results and p-values should be presented. In addition, the effect of the highly selected surgical cohort and the high malignancy prevalence should be considered more carefully when interpreting diagnostic performance, particularly PPV and NPV. 

 

First, all paired comparisons between TRAICK and the radiologists are now reported, using the exact McNemar test for accuracy, sensitivity and specificity and the DeLong test for AUC. All estimates are given with 95% confidence intervals (Results, Paired comparisons; Table 1; Supplementary Table S1). Second, the Methods now describe the study population as a selected surgical cohort and explain that the high malignancy rate may affect sensitivity and specificity as well as PPV and NPV. A flow diagram showing operated and non-operated nodules has been added (Figure 1), and this issue is discussed in the Discussion and listed in the Limitations. Please also see our responses to Comments 1 and 2.

 

5. Are the conclusions consistent with the evidence and arguments presented and do they address the main question posed? Please also explain why this is/is not the case.

Some statements are stronger than the available evidence. In particular, the suggestion that TRAICK may reduce unnecessary repeat biopsies was not directly evaluated in this study. 

 

We removed the statement about reducing repeat biopsies from the Abstract and Conclusions. The Conclusions now say that TRAICK “has the potential to provide supplementary information to support preoperative decision-making” and that the findings do not show superiority over radiologists or FNAB, generalizability to other settings, or a reduction in repeat biopsies or surgery. The Discussion presents these possible benefits as hypotheses that need prospective testing. Please also see our response to Comment 3. (Discussion, Conclusions)

 

6. Are the references appropriate?

Yes. I have no major concerns regarding the reference list.

 

In response to the other reviewers, we updated the Bethesda reference to the 2023 edition and added recent references on decision regret after thyroid surgery, the STARD-AI guideline, three recent meta-analyses, and communication of uncertainty in indeterminate cytology. References that were no longer cited were removed, and the list is now numbered in order of citation. (References)

 

7. Any additional comments on the tables and figures.

The tables and figures are generally clear and helpful.   Table 3 includes several subgroups with very small numbers of nodules, and the reported malignancy rates, particularly the 100% rates in some TIRADS categories, should therefore be interpreted cautiously (I would prefer numbers ) .

 

This table is now Table 4. Each malignancy rate is given as numbers (n/N) with a 95% confidence interval, for example 5/5 (95% CI 56.6–100) and 2/3 (20.8–93.9) for TR5. The footnote, the Results and the Discussion all note that several subgroups contained fewer than 10 nodules and that these rates should be interpreted with caution. In addition, the statistical figures were redrawn, and new figures were added: a patient flow diagram (Figure 1), confusion matrices (Figure 5), ROC curves (Figure 6) and a reclassification plot (Figure 7). (Results; Discussion; Table 4; Figures 1 and 5–7)

Author Response File: Author Response.docx

Round 2

Reviewer 2 Report

Comments and Suggestions for Authors

The authors improved the robustness and rigor of the study by following the reviewer's suggestions. I believe the manuscript is now suitable for publication. I have no further comments. Thanks.