Review Reports

Round 1

Reviewer 1 Report

Comments and Suggestions for Authors

This manuscript presents the Diabetes-QMS Benchmark, a multi-level industrial dataset for temporal quality monitoring and calibration in glucose and HbA1c diagnostic-device manufacturing. The authors provide a detailed account of dataset construction, technical validation, and the importance of leakage-aware, temporally rigorous evaluation. The manuscript has certain value for methodological studies of statistical process control and machine learning-based quality monitoring. However, several issues are suggested to be addressed to further improve the manuscript. Detailed comments are provided below.

1. In related work section, authors reviewed literature on predictive quality modelling, anomaly detection and conformal prediction, but lacks a systematic overview of publicly available relevant datasets. It is suggested to add discussion on existing open datasets and analyze their limitations. Explicit comparison should be provided to illustrate how the proposed Diabetes‑QMS Benchmark differs from existing resources and what unique strengths it offers
2. The characteristics of the dataset should be more deeply analyzed, and the rationale for selecting specific machine learning algorithms should be explained. Machine learning algorithms are developing rapidly, and many alternatives exist for calibration modelling and anomaly detection. The authors should justify why Random Forest was selected as the representative algorithm, and discuss whether other approaches might be more suitable given the hierarchical and temporal structure of the data. Additionally, the nature of "factory calibration" should be clarified. Is it a single global function, a lot-specific model, or a more complex adaptive procedure? This clarification is essential because it directly affects the fairness and interpretability of the comparisons presented in the technical validation section.
3. The strongest instability observed in 2025 should be further analyzed and explained. The manuscript repeatedly identifies 2025 as a period of pronounced process deterioration, but it does not clarify whether this drift occurred naturally or was artificially introduced during benchmark construction. The authors should explicitly state the origin of this instability and provide justification. 

Author Response

Dear Reviewer, please kindly find attached our detailed point-by-point response to your comments. We sincerely appreciate your valuable feedback and constructive suggestions, which have helped us improve the manuscript.

Author Response File: Author Response.pdf

Reviewer 2 Report

Comments and Suggestions for Authors

This manuscript presents the Diabetes-QMS Benchmark, a multi-level industrial dataset designed to support methodological research in glucose and HbA1c diagnostic-device manufacturing. The manuscript is generally well structured, the methodological rationale is clearly explained, and particular attention has appropriately been given to temporal structure, production-lot dependence, target leakage, and the distinction between retrospective methodological evaluation and prospective or regulatory applications. The technical validation is comprehensive and the conclusions are generally supported by the analyses presented.

I suggest that the authors address the following six points:

1. Please correct the correspondence information and a small textual duplication. The correspondence field on the first page still contains the placeholder “e-mail@e-mail.com”, while the entries following “Tel.” appear to be email addresses rather than telephone numbers. In addition, immediately after Table 1, the sentence describing the 800 records is accidentally duplicated/merged (“690 HbA1c The public release therefore…”). Please correct these items and perform a final check for similar residual template or editing errors.

2. Please provide a brief rationale for the study-defined analytical thresholds. The manuscript appropriately states that the 5% CV and 10% maximum-error limits for the glucose QC component and the ±6% relative-error criterion for HbA1c are benchmark-specific and should not be interpreted as formal ISO, NGSP, or regulatory conformity limits. However, it would strengthen the reproducibility and interpretability of the benchmark to explain briefly how these particular numerical thresholds were selected—for example, whether they originated from internal manufacturing specifications, historical practice, predefined study conventions, or another rationale. No extensive discussion is required.

3. Please make the analytical population used for the leave-one-year-out HbA1c analysis completely explicit. The Methods indicate that HbA1c calibration analyses use the predefined 635-measurement complete-case subset, whereas the temporal results are reported for 2023–2026. I suggest stating directly in the paragraph describing leave-one-year-out validation, and preferably also in the caption or footnote to Table 5, that this analysis was conducted on the 635 complete-case observations from 39 production lots covering 2023–2026. This would prevent readers from wondering why 2022, which is represented in the full dataset, does not appear among the held-out years.

4. Please add a few additional details needed for exact computational reproducibility. The manuscript already reports Python 3.12, the Random Forest tree number and random seed, and several other modelling parameters. Nevertheless, the authors should also report the exact scikit-learn version and clarify relevant model settings that otherwise rely on software defaults. The number of bootstrap resamples used to construct the 95% confidence interval should also be stated. For split-conformal prediction, please describe how the training/calibration/evaluation subsets were formed. Similarly, the expression “adaptive source experiment” should be clarified and sufficient details should be provided to understand and, where possible, reproduce the Adaptive Conformal Inference analysis.

5. Please improve the readability of the multi-panel figures. Figures 2 and 4–6 contain important information, but several axis labels, legends, annotations, and numerical elements are relatively small at normal manuscript viewing size. Increasing font sizes and/or the dimensions or resolution of these figures would considerably improve readability. If necessary, individual panels could be enlarged or reorganized while retaining the current analytical content.

6. Please strengthen transparency concerning the roles of the funder and industrial data provider. The manuscript states that the study was funded by A-Mediqal LLP and that Axel & A LLP provided the de-identified manufacturing data. For complete transparency, the Funding and/or Conflict of Interest statements should briefly specify the role, if any, of these organisations in study design, data collection/provision, analysis, interpretation, manuscript preparation, and the decision to publish. If they had no role beyond funding or data provision, this can simply be stated explicitly.

 

Author Response

Dear Reviewer, please kindly find attached our detailed point-by-point response to your comments. We sincerely appreciate your valuable feedback and constructive suggestions, which have helped us improve the manuscript.

Author Response File: Author Response.pdf

Round 2

Reviewer 1 Report

Comments and Suggestions for Authors

Authors have addressed my comments well.  I have no further concerns.

Reviewer 2 Report

Comments and Suggestions for Authors

I have carefully reviewed the revised version of the manuscript. The authors have adequately addressed the issues raised during the review process, and the main principles and recommendations suggested by the reviewers have been appropriately incorporated into the revised manuscript. The methodological scope, interpretation of the results, validation strategy, and limitations are now presented with appropriate clarity and caution.

I have no further comments or objections to the publication of the manuscript in its present form.