Review Reports
- Enes Zafer 1,2,* and
- Mehmet Yıldız 3
Reviewer 1: Anonymous Reviewer 2: Anonymous Reviewer 3: Anonymous
Round 1
Reviewer 1 Report
Comments and Suggestions for AuthorsTitle: Who should have an HbA1c? Development and external validation of a dysglycaemia prediction model using only routinely available data
Introduction
The Discussion explains the intended clinical setting and the role of model predictions in testing decisions. A brief summary of this intended use in the Introduction, particularly following lines 104–106, would strengthen the clinical rationale. This should identify how predictions might be accessed, for example through electronic health record integration or a clinical calculator, and clarify that the approach is intended for patients with the necessary existing data rather than as a general replacement for questionnaire-based screening.
Materials and Methods
Data Source and Participants
The cross-sectional nature of the study is acknowledged in the Discussion but should also be stated explicitly at the beginning of the Methods. A suitable description would be “a clinical prediction model development and temporal external validation study based on a secondary analysis of repeated cross-sectional NHANES data.”
Predictors
The statement in lines 146–159 that all 48 predictors are available from a routine clinical encounter should be qualified. Variables such as the family income-to-poverty ratio, hip and arm circumference, daily alcohol intake in grams, and high-sensitivity C-reactive protein may not already be recorded in the intended clinical setting.
The Discussion appropriately acknowledges that measurements collected for research may differ from routinely recorded clinical data. The wording in the Methods, Abstract, and Conclusions should be consistent with this limitation and identify the settings in which the required predictors would be available without additional measurements or information collection.
Statistical Analysis: Complex survey design
In lines 224–238, confidence intervals for the AUC are calculated using outcome-stratified bootstrap resampling, whereas survey strata and primary sampling units are considered in the descriptive analyses. It is unclear how the stratified and clustered NHANES sampling design was addressed when estimating uncertainty in performance measures and between-model differences. The authors should specify the bootstrap resampling unit and explain how the complex survey design was incorporated into these analyses.
The Abstract states that survey weights were used throughout, whereas lines 253–255 and the Results identify weighted rather than unweighted model fitting as a sensitivity analysis. The authors should specify whether weights were used in primary model fitting, hyperparameter selection, performance evaluation, and calibration assessment, and ensure consistency across sections.
Comparator
The Methods and Discussion appropriately acknowledge that the ADA risk test could not be evaluated in the validation cycle. The Abstract and Conclusions should therefore explicitly distinguish the model’s temporally validated performance from its superiority over the ADA risk test, which was assessed only in the development cycle.
Lines 187–200 explain that unobserved ADA items were assigned zero points according to the cited scoring rule. However, genuinely missing measurements should be distinguished from structural questionnaire skips, such as the gestational diabetes item in women who have never been pregnant.
Using multiple imputation for the full model but zero scores for missing ADA items may influence comparative performance. A sensitivity analysis evaluating the model–ADA performance difference in the same participants under an alternative missing-data strategy is recommended.
Results
The Discussion explains the clinical trade-offs associated with different thresholds. However, it remains unclear whether the highlighted threshold of 0.30 was prespecified or selected after examining performance.
Although a confidence interval is provided for the sensitivity difference at matched specificity, uncertainty is not reported for the differences in testing rates and case detection at the 0.30 threshold. Confidence intervals for these comparative differences should be provided. The example involving 100,000 adults should be clearly labelled as a survey-based projection rather than an observed implementation outcome.
The Discussion clarifies that gradient boosting performed better in the two sensitivity analyses with statistically significant between-model differences. The corresponding effect sizes and confidence intervals should be briefly reported in the Results, with reference to the relevant supplementary table, so that readers can assess their magnitude and precision.
Discussion
Lines 328–331 appropriately explain that the performance difference between the reweighted ADA model and the full model reflects the net effect of adding predictors while removing three questionnaire items. The statement in lines 202–209 that this comparison “isolates” the contribution of additional variables should be revised, and related statements in the Abstract, Highlights, Discussion, and Conclusions should remain consistent with this qualification.
Several statements in the Discussion also exceed the comparisons presented. Describing approximately 0.76 as the performance “ceiling” of a seven-item instrument and concluding that a questionnaire “adds nothing” once laboratory results are available (lines 458–461) are not directly established by the analyses. In particular, the incremental contribution of the omitted questionnaire items when added to the full model was not evaluated.
The statements that differing studies “cannot both guide practice” (lines 518–519) and that questionnaires containing a prior-glucose item are difficult to outperform whereas those without it are not (lines 544–545) should also be qualified. Differences in populations, outcomes, predictor sets, and validation procedures limit these cross-study conclusions. More cautious wording should distinguish directly demonstrated findings from possible explanations.
References
The reference list contains bibliographic inaccuracies and gaps in coverage of the relevant literature and should be reviewed and corrected by the authors.
Author Response
Please see the attachment.
Author Response File:
Author Response.pdf
Reviewer 2 Report
Comments and Suggestions for AuthorsIntroduction:
It explains the rational well, however it is too convoluted and is heavy with information loosing the reader. I would suggest shortening it into more understandable point to rational the studies need.
Furthermore, there is language that I would suggest to improve to a more scientific/formal way. For example lines 102-104 “There is a second question underneath the first. Where flexible machine learning is compared with regression on tabular clinical data under equivalent conditions, it usually does not win”
Methods:
Lines139-141, explaines why some methods were chosen. Does not belong in methods rather it should be a part of the discussion along with explanations why it was chosen that way.
Lines 162-164 is better placed in the discussion, and repeats the statement from line 130.
Discussion in lines 475-491 is the main issue with the new model, this model relies on other laboratory data as stated in the methods in previous paragraphs in the discussion. It excludes glucose which is the most basic lab test and it on its own is enough to define diabetes or pre diabetes (in two separate tests). Thus, is adding this model with all added complexities worth the mild difference in decision making?
Having the patient come again for HbA1c screening after they already were examined for plenty other tests caries with it further costs that are not addressed such as the extra visit, extra work time lost and so on. Might be added to limitations
Author Response
Please see the attachment.
Author Response File:
Author Response.pdf
Reviewer 3 Report
Comments and Suggestions for AuthorsSee my comments attached.
Comments for author File:
Comments.pdf
Author Response
Please see the attachment.
Author Response File:
Author Response.pdf