Review Reports

Round 1

Reviewer 1 Report

Comments and Suggestions for Authors

In my opinion, this article is valuable and holds potential. The use of longitudinal MRI descriptors within a survival-analysis framework is meaningful and goes beyond conventional baseline-only approaches. The comparison of classical, machine-learning, and deep-learning models is also relevant. However, several methodological points require clarification before the conclusions can be fully supported.

• The temporal descriptors were calculated using MRI measurements obtained up to the event or censoring time. Further clarification of the intended prediction time and how this design supports prediction at the initial diagnosis is necessary.
• The OASIS3 analysis wass conducted using five-fold cross-validation within the external cohort and a 23-variable model. Please clarify whether the ADNI-trained models were applied without refitting. If not, please reconsider the term “external validation.”
• The conversion definitions differ between ADNI and OASIS-3. Could the authors explain whether conversion to AD in ADNI and CDR total ≥ 1 in OASIS-3 represent comparable outcomes?
• The benchmark uses the combined 24-variable input, but the incremental contribution of the temporal MRI descriptors is not directly quantified. Comparison of clinical variables only, clinical variables with baseline MRI, and the combined temporal feature set will be better.
• DeepSurv was trained for a fixed 100 epochs, whereas DeepHit used early stopping (we can see in Table 1). In order to ensure a fairer comparison between the deep-learning models, using  comparable stopping criteria will be better.
• DeepSurv produced a valid IBS result in only one-fold. Please resolve thiss numerical issue or remove the value from the direct calibration comparison.
• Similar C-index values do not necessarily confirm that performance is “feature-limited rather than architecture-limited.” I suggest presenting this interpretation more cautiously and limiting it to the current experimental setting.
• The definition of the variability feature is unclear. The text refers to variability around the fitted trend, whereas eqs. 16 appears to calculate variability around the mean. Please correct this inconsistency.
• ADNI required at least three MRI sessions, whereas OASIS-3 required only two. Please justify this difference and, if feasible, consider a sensitivity analysis using the same criterion in both cohorts.
• Please provide further details on participants excluded by the repeated-MRI criterion and discuss whether this selection may have affected the analytical cohort.
• Some clinical claims by author, I think appear stronger than the current design supports. For example, the manuscript refers to prediction “at the initial MCI diagnosis,” recommends models for “clinical deployment,” and describes the approach as a “viable decision-support tool.” Please moderate these statements and align them with the longitudinal data requirements and exploratory validation design.
• Although the exact combination of longitudinal MRI and survival modelling is relatively specialized, several recent and closely related studies are not discussed. Please update the literature review and clarify how the present temporal-descriptor benchmark differs from recent longitudinal and deep-survival approaches.

Author Response

Please see the attachment.

Author Response File: Author Response.pdf

Reviewer 2 Report

Comments and Suggestions for Authors

Revision required

1. External validation (OASIS-3)
With n = 78 and only 28 events, the EPV is approximately 1 in the training folds - far below thresholds
for reliable validation. The absence of ADAS-13 in particular makes the ΔC-index values difficult to
interpret. The language should be changed from generalisability and cross-cohort performance to
exploratory/hypothesis-generating. Mention that the decrease in performance is a combination of
domain shift and the missing predictor. Validation on NACC with the full set of covariates is
mandatory before making any clinical claims.
2. No hyperparameter optimisation for DeepSurv / DeepHit
No hyperparameter tuning was performed for DeepSurv/DeepHit - defaults were used. The deep
learning models are sensitive to these choices. The performance of these models is likely
underestimated. Change the discussion to talk about these underestimations or use nested cross
validation to find the optimal parameters.
3. Inconsistent EPV statements
On page 8, the EPV is stated as 250/24 = 10.4. On page 9, the EPV is stated as 13.8. This needs to be
corrected or removed.
4. Proportional-hazards violations + Table 3 footnote
Hippocampus-σ and Ventricles-σ violate the proportional hazards assumption. Either the Cox model
can be stratified on these variables, or a time-varying coefficient Cox model can be used. The footnote
for the DeepSurv IBS in Table 3 is not included - add it in the table.
5. Over-long figure captions
The figure captions for figures 1 and 5 restate the results section. These can be shortened to a
sentence stating what is shown in the figure and the key takeaway.
6. Code availability
The code is available from the corresponding author upon reasonable request. This can be improved
by depositing the code on GitHub or Zenodo. Change the data availability statement to reflect this.
7. Competing-risk framing
The discussion mentions death as a competing risk and the Fine-Gray cannot be used with
ADNIMERGE. Change the paragraph to state that Fine-Gray will be performed on NACC to turn a
weakness into a commitment for the next step.
8. Residual typos / phrasing
- There is an extra ‘r’ in the Acknowledgements: “r ADNI is funded”. Remove it.
- There are repeated sentence fragments in the discussion. Remove them.
- Check the use of “time-weighted cumulative mean” and “time-weighted mean” for consistency.

Author Response

Please see the attachment.

Author Response File: Author Response.pdf

Reviewer 3 Report

Comments and Suggestions for Authors

1. The prediction-time definition is unclear and potentially invalid. The longitudinal MRI descriptors are calculated using measurements up to the individual's event/censoring time. This means that converters can contribute MRI information obtained immediately before conversion. Although this prevents post-event leakage, it does not represent information available at baseline and can lead to substantial optimism. The authors should reformulate the problem as dynamic/landmark prediction or restrict baseline models to baseline-accessible predictors.
2. The greater than or equal to 3-MRI inclusion criterion creates potential selection and survivorship bias. Of 993 subjects, only 661 were retained. Similar baseline demographics do not establish absence of selection bias. The authors should characterize excluded subjects and examine whether MRI availability/follow-up is associated with conversion and survival time.
3. The manuscript does not demonstrate incremental value of longitudinal MRI. Clinical-only, baseline-MRI, longitudinal-MRI, and combined models should be compared. Without such ablations, the claim that temporal MRI descriptors provide additional prognostic information is not established.
4. The claim that performance is feature-limited rather than architecture-limited is not supported. All models receive the same 24-dimensional hand-engineered representation, which inherently limits the opportunity for deep models to learn temporal representations. This benchmark demonstrates similarity among survival algorithms on engineered features, not architecture-independence of the underlying prediction problem.
5. The comparison with the current literature is insufficient. A recent ADNI survival study using 902 MCI subjects reported an RSF C-index of 0.878, exceeding the 0.863 reported here. The authors should provide a comprehensive comparison with recent MCI->AD survival and longitudinal MRI studies and clearly explain why their performance should be regarded as competitive/SOTA.
6. The machine-learning benchmark is not sufficiently controlled. No formal hyperparameter optimization was performed, and DeepSurv and DeepHit were trained under different optimization/early-stopping protocols. This makes conclusions regarding the relative performance and generalizability of model families difficult to support.
7. The OASIS-3 experiment is underpowered for the conclusions drawn. Only 28 events are available among 78 subjects. The very large uncertainty in DeepHit and DeepSurv performance makes it inappropriate to conclude that classical models are intrinsically more generalizable. This should be presented as exploratory only.
8. The OASIS-3 comparison lacks covariate parity. ADAS-13, the strongest predictor in the ADNI model, is unavailable in OASIS-3. Therefore, the observed performance drop cannot be cleanly attributed to domain shift. A common predictor set should be used for a fair external comparison.
9. Competing risks are not adequately handled. Death and non-AD dementia may prevent observation of AD conversion. Treating these outcomes as ordinary censoring can bias cumulative-incidence estimates. A competing-risk sensitivity analysis would substantially strengthen the study.
10. The Cox model violates proportional hazards for two predictors. The authors should address these violations using time-varying coefficients, stratification, or another appropriate survival formulation rather than relying on conventional hazard-ratio interpretation.
11. The reported feature-level statistical significance is exploratory. Eleven of 24 variables are declared significant without multiplicity correction. FDR-adjusted results or a clearly exploratory framing are needed.
12. Clinical utility is overstated. A C-index/time-dependent AUC and Kaplan–Meier separation do not establish clinical utility. Decision-curve analysis, calibration plots, clinically meaningful thresholds, and prospective/multicohort validation would be required before describing the model as suitable for memory-clinic decision support.
13.  

Author Response

Please see the attachment.

Author Response File: Author Response.pdf

Round 2

Reviewer 1 Report

Comments and Suggestions for Authors

The authors have substantially addressed the previous concerns, and the manuscript has improved considerably. The remaining issues primarily concern interpretation, terminology, and consistency between the response letter and the revised manuscript. In my opinion, the following points should be addressed before publication.

• The authors should revise the terminology used for the OASIS-3 analysis. Some occurrences of “external validation,” “transferability,” and “generalisation” remain, although the models were refitted within OASIS-3. Please ensure that the terminology is consistent with the exploratory nature of this analysis.
• Please further moderate the remaining clinical claims. In my opinion, statements implying reliable clinical identification or clinical deployment should remain consistent with the requirement for multiple longitudinal MRI sessions.
• Please further soften the statement that performance is bounded by feature quality rather than model architecture. The narrow C-index range indicates no clear performance separation among the evaluated models in the present setting, but does not directly establish the cause of this convergence.
• Please use more cautious wording regarding selection bias. Significant differences were observed between included and excluded participants in age, MMSE, ADAS-13, and follow-up duration. In my opinion, these findings do not exclude the possibility of selection bias.
• The authors should correct the description of Configuration A2. The response letter and Table 4 indicate six clinical variables plus six baseline MRI variables (12 total), whereas the Results text refers to 18 baseline MRI variables.

Author Response

Please see the attachment

Author Response File: Author Response.pdf

Reviewer 2 Report

Comments and Suggestions for Authors

The revised version appears okay for publication.

Author Response

Thank you!