Machine Learning-Driven Probability of Permanent Pacemaker Implantation After Transcatheter Aortic Valve Replacement

Round 1

Reviewer 1 Report

Comments and Suggestions for Authors

The reviewer’s comments to the MS titled as “Machine Learning-Driven Probability of Permanent Pacemaker Implantation After Transcatheter Aortic Valve Replacement” as follows

1. Small typographical errors
2. Please consult an expert in artificial intelligence
3. Reduce text burden leaving the message intact
4. Reduce references because many references are not justified to the context
5. Please mention number of balloon expandable and self-expanding valves, the PPI outcome util one month follow up
6. Please mention the number of valves from the different companies, there comparison for PPI
7. Please provide a flow chart to show what are the different kinds of complications encountered
8. How the sample size was decided
9. Of course ,a statistician may be consulted

Author Response

Dear Editor, please find attached a Word document containing our responses to the reviewer.

Author Response File: Author Response.pdf

Reviewer 2 Report

Comments and Suggestions for Authors

Dear Authors,

 

Thank you for the opportunity to review this interesting manuscript evaluating a machine learning–based model for predicting permanent pacemaker implantation (PPI) after TAVR.

 

The topic is clinically relevant because post-TAVR conduction disturbances remain one of the most important complications affecting procedural planning, monitoring strategies, and long-term outcomes. I particularly appreciated the authors’ effort to combine conventional statistical methodology with explainable machine learning approaches such as SHAP analysis, which improves transparency and interpretability of the predictive model.

 

The manuscript is generally well organized, and the prospective nature of the cohort represents an important strength. The discussion is also well connected to current TAVR literature, especially regarding the role of anatomical and prosthesis-related factors in conduction disturbances.

 

However, several methodological and interpretative issues should be clarified before the manuscript can be considered further.

 

1. Limited number of events and risk of overfitting

 

Although the overall cohort size is acceptable (n = 179), the number of PPI events is relatively small (n = 17).

 

This is particularly important given the relatively high number of candidate predictors included in the machine learning pipeline. Even with cross-validation and SMOTE correction, the possibility of model instability and overfitting remains substantial.

 

I strongly recommend expanding the limitations section to discuss:

 

low event-per-variable ratio,

instability of feature ranking,

and the exploratory nature of the derived model.

 

Currently, the Discussion occasionally presents the model performance somewhat too confidently relative to the available event count.

 

2. Interpretation of model performance

 

The reported overall accuracy of 0.944 appears impressive at first glance.

 

However, because the dataset is markedly imbalanced (17 PPI vs. 162 non-PPI cases), accuracy alone may overestimate true predictive utility.

 

I appreciate that PR-AUC was used as the primary ranking metric, which is methodologically appropriate in this context.

 

Nevertheless, the manuscript should place greater emphasis on:

 

sensitivity,

precision,

PR-AUC,

and false-negative performance,

 

rather than primarily highlighting overall accuracy and weighted F1-score.

 

The relatively modest precision for the PPI class (0.667) should also be discussed more explicitly in terms of potential false-positive clinical implications.

 

3. Validation strategy

 

The authors used stratified 3-fold cross-validation.

 

Given the small number of outcome events, additional justification for choosing 3 folds instead of repeated CV or bootstrap validation would strengthen the methodological section.

 

Importantly, the lack of external validation remains a major limitation and should be emphasized more clearly throughout the manuscript, particularly when discussing possible clinical implementation.

 

4. Feature selection strategy

 

The manuscript would benefit from a clearer explanation of:

 

how candidate variables were selected,

whether any preselection or dimensionality reduction was performed,

and how multicollinearity among anatomical variables was handled.

 

Several predictors appear anatomically related (e.g., annulus measurements, sinus diameters, valve size), which raises concerns regarding correlated feature dominance in the SHAP analysis.

 

5. Clinical interpretability of SHAP findings

 

The SHAP analysis is one of the strongest aspects of the study.

 

However, interpretation remains somewhat descriptive.

 

For example:

 

What approximate valve size ranges were associated with increased risk?

Did larger annular dimensions consistently increase risk?

Were there nonlinear relationships?

Were interaction effects explored?

 

Providing several clinically interpretable examples would significantly improve translational relevance.

 

6. Missing established electrophysiological predictors

 

The Introduction appropriately discusses predictors such as RBBB and membranous septum anatomy.

 

However, these variables do not appear to play a major role in the final model. This discrepancy deserves additional discussion.

 

In particular, the relatively low contribution of baseline conduction disturbances in SHAP analysis is somewhat unexpected compared with prior TAVR literature and should be interpreted more carefully.

 

7. Calibration assessment

 

The manuscript focuses primarily on discrimination metrics. If available, inclusion of calibration analysis (calibration curve, Brier score, or calibration slope/intercept) would substantially strengthen the manuscript.

 

Even a brief acknowledgment of calibration limitations would be valuable.

 

8. Results section

 

The Results section is comprehensive but somewhat repetitive, particularly across the SHAP and permutation importance descriptions.

 

Moderate condensation would improve readability.

 

9. Terminology consistency

 

The manuscript alternates between:

 

“permanent pacemaker implantation,”

“PPI,”

and “pacemaker implantation.”

 

Consider standardizing terminology throughout the text.

 

10. Figure readability

 

Figures are informative overall; however:

 

axis labels,

font size,

and color contrast

 

could be improved slightly for publication quality, especially in the SHAP summary plot.

 

11. Language polishing

 

The manuscript is generally well written, although several sentences are overly long and occasionally difficult to follow.

 

Minor English editing would improve clarity and readability.

 

Overall, this is a clinically relevant and thoughtfully designed exploratory study addressing an important complication after TAVR. The integration of explainable machine learning methods is a notable strength, and the findings regarding the predominance of anatomical and prosthesis-related variables are biologically plausible and clinically interesting.

 

However, important limitations related to the small event count, potential overfitting, and lack of external validation require more cautious interpretation.

 

With moderate revision focused on methodological transparency and balanced interpretation, the manuscript could become suitable for publication in Diagnostics.

 

Author Response

Dear Editor, please find attached the Word document containing our responses to the reviewer.

Author Response File: Author Response.pdf

Reviewer 3 Report

Comments and Suggestions for Authors

• Only 17 PPI events (9.5%) in a high-dimensional dataset raise serious concerns about overfitting and generalizability (even with SMOTE and 3-fold CV). Please discuss the events-per-variable ratio.
• Correct statistical inconsistency.
• Provide exact final feature list, feature selection process (if any), handling of missing data/multicollinearity, hyperparameter details, and full model comparison results.
• Clarify whether membranous septum length, pre-existing RBBB, implantation depth, and baseline conduction disturbances were included. Compare SHAP findings more explicitly with recent ML models.
• Improve the formatting of Table 1 and its footnotes for better readability.
• Ensure high-resolution quality for Figures 1–4.
• Correct “SHapley Additive ex-Planations” → “exPlanations” and perform professional English language editing (awkward phrasing and minor grammatical issues present).
• Strengthen the limitations paragraph and mention clinical implementation barriers.

Comments on the Quality of English Language

The English language is generally clear and scientifically appropriate, with a good command of technical, medical, and machine-learning terminology. However, there are frequent instances of awkward phrasing, non-idiomatic expressions, and minor grammatical issues (particularly article usage). Specific errors should be corrected. Professional language editing by a native English-speaking editor is strongly recommended before final acceptance.

Author Response

Dear Editor, please find attached the word document our responses to the reviewer.

Author Response File: Author Response.pdf

Round 2

Reviewer 2 Report

Comments and Suggestions for Authors

Dear authors,

Thank you for the careful revision and detailed point-by-point responses.

The revised version has improved substantially and most methodological concerns raised during the previous review round have been appropriately addressed. The manuscript now presents a more balanced interpretation of the machine learning results and provides improved methodological transparency.

The following revisions are particularly appreciated:

1. Clearer acknowledgment of the exploratory nature of the model and the limited event count.
2. Expanded discussion regarding possible overfitting and low event-per-variable ratio.
3. Improved presentation of class-specific metrics with reduced emphasis on overall accuracy.
4. Additional justification for the use of stratified 3-fold cross-validation.
5. Clearer explanation of feature selection strategy and handling of correlated anatomical variables.
6. More cautious interpretation of SHAP findings.
7. Explicit recognition of the absence of external validation and calibration analysis.

These modifications strengthen the scientific rigor of the study and improve interpretability.

There are only 3 points I want you to correct.

1. Although the lack of calibration analysis is now acknowledged, inclusion of even a simple calibration figure or Brier score (if available) would further strengthen confidence in model performance.
2. The finding that anatomical and prosthesis-related variables dominated the model is clinically interesting. However, established electrophysiological predictors such as baseline RBBB, conduction abnormalities, implantation depth, and membranous septum–related factors remain less prominent than expected. A brief additional discussion regarding this discrepancy would further improve the manuscript.
3. Figure quality has improved, but the SHAP summary plot may still benefit from slightly larger labels for final publication formatting.

Overall, the revised manuscript is considerably stronger than the initial submission. The topic is clinically relevant, the prospective design is valuable, and the integration of explainable machine learning methods remains an important strength.

In my opinion, the manuscript is now suitable for publication after minor revision.

Author Response

The responses to round 2 are in the pdf document.

Author Response File: Author Response.pdf