A Decision-Aid Model for Predicting Triple-Negative Breast Cancer ICI Response Based on Tumor Mutation Burden

Round 1

Reviewer 1 Report

Comments and Suggestions for Authors

 

The authors provided a very detailed and insightful presentation of the problem, the challenges it poses, related work in the past and their solution, justifying it with concrete and accurate arguments.

 

However, it would have been beneficial to also include a literature review section with all the related studies to the paper and also providing a table summarising these studies. At the end of the section, a proper discussion should be included, highlighting the drawbacks/gaps of the studies selected and how the present paper is aiming to fill in these gaps.

 

The authors provided a thorough and detailed methodology section. However, it would have been beneficial to also include appropriate tables and diagrams summarising the content to assist the reader in understanding the paper in greater depth and with more ease and not just in the Appendix.

 

The figures were missing from the paper. Please include them in future versions.

 

Also, it would have been beneficial to include more details on the datasets used.

 

SHapley Additive explanations (SHAP) analysis is more towards explaining XAI. There are other feature importance approaches which could also be explored such as bio-inspired algorithms or suggested in future work.

 

Please include more information on the machine learning part, and specifically why you chose the specific ML models. Also, consider using deep learning architectures.

 

 

 

 

 

 

 

Author Response

1- However, it would have been beneficial to also include a literature review section with all the related studies to the paper and also providing a table summarising these studies. At the end of the section, a proper discussion should be included, highlighting the drawbacks/gaps of the studies selected and how the present paper is aiming to fill in these gaps.  

We appreciate your suggestion and inform you that we have included a comprehensive literature review section. Our review systematically compares these studies based on key factors such as model construction (DL vs. ML) and TMB threshold selection. A summary table outlining these aspects was also added with our newly proposed approach for direct comparison (Table 1). We further highlight the contribution of our approach such as the integration of histological features and clinical data, survival analysis, and the emphasis on biological interpretation (Lines 96-174).

 

2- The authors provided a thorough and detailed methodology section. However, it would have been beneficial to also include appropriate tables and diagrams summarising the content to assist the reader in understanding the paper in greater depth and with more ease and not just in the Appendix.

Thank you for the valuable suggestion, we agree that incorporating tables and overall flows enhances and facilitates comprehension of our methods. According to your suggestion, we added a detailed workflow (Figure 1) outlining our approach in 5 steps: (1) Data acquisition and preparation, (2) TMB threshold selection and survival analysis, (3) Image processing and features extraction, (4) Model construction, (5) Biological insights. Additionally, we have incorporated tables to further support and summarize our methodology:  Table 1 in the literature review to summarize and facilitate the comparison of our approach with related studies in the field, Table 2 with a detailed description of the dataset used in our study, Table 3 to show hyperparameters obtained for each model.

 

3-The figures were missing from the paper. Please include them in future versions.

Thank you for your feedback. We have adhered to the journal's submission guidelines and uploaded the figures separately from the manuscript as required. However, we will ensure that the figures are uploaded in the review panel.

 

4- Also, it would have been beneficial to include more details on the datasets used.  

In response to your comment, we have included a Table with a detailed description of our dataset. (Table 2)

 

5- SHapley Additive explanations (SHAP) analysis is more towards explaining XAI. There are other feature importance approaches which could also be explored such as bio-inspired algorithms or suggested in future work. 

Thank you for your insightful suggestion. We understand the value of feature selection algorithms in improving the model’s efficiency, interpretability and overall performance and used two independent methods to select important features. While SHAP is primarily used to interpret model prediction, its feature importance score can be leveraged for feature selection based on relative SHAP value. We acknowledge the limits of the methods used in our study and therefore added a Genetic algorithm, a technique inspired by natural selection, to further screen features. The technique proposed a combination of 39 optimal features while maintaining and balancing model performance. We then selected common features from 39 features selected by SHAP, GA and model and used these to create the integrated model.

 

6- Please include more information on the machine learning part, and specifically why you chose the specific ML models. Also, consider using deep learning architectures.

We appreciate your feedback. Based on your comment, we have detailed the ML section in methods with a more comprehensive explanation of the methods used. We especially elaborated on the choice of ML models with the hyperparameters used and the strategy applied for balancing our dataset and the scaling techniques. (Lines 235-253)

About your remark on adding deep learning techniques. We acknowledge that Deep Learning methods could be incorporated into our models to compare a wide range of AI methods. However, our primary aim was to assess and evaluate the potential and effectiveness of ML algorithms in predicting TMB in TNBC. Nonetheless, we have included the use of complex deep learning models as a future research perspective.

Author Response File: Author Response.pdf

Reviewer 2 Report

Comments and Suggestions for Authors

Bendani et al use published TNBC data to combine histological features and clinical data to stratify TNBC patients into high and low TMB groups. Using various machine learning models, with best performance by a random forest classifier, the authors achieve good predictive performance. Additionally, the authors perform genomic mutation analysis, differential expression gene (DEG) analysis, and pathway enrichment analyses to further investigate the biological underpinnings of the TMB predictions.

 

Major issues:

1.     Histological features vs. established predictors: the authors need to provide a clearer comparison of the importance of histological features relative to well-known predictors such as lymph node involvement and age. These established features are known to be associated with clinical outcomes in TNBC, and it is crucial to contextualize the significance of the newly proposed histological features against these predictors.

2.     Model overfitting: given the relatively small cohort size, overfitting is a significant concern. The authors should detail the steps taken to control for overfitting more comprehensively. For example, running the classifier with TMB values randomly shuffled between patients to demonstrate that this significantly lowers model performance.

3.     Contextualization with existing literature: the authors do not contextualize their finding with the literature about TNBC mutation burden, some important papers in the field include:

a.     Lee, J.V., Housley, F., Yau, C. et al. Combinatorial immunotherapies overcome MYC-driven immune evasion in triple negative breast cancer. Nat Commun 13, 3671 (2022)

b.     Shah, S., Roth, A., Goya, R. et al. The clonal and mutational evolution spectrum of primary triple-negative breast cancers. Nature 486, 395–399 (2012).

 

Minor comments:

1.     Figure 2, please give the features comprehensible names.

2.     Provide more details on hyperparameter tuning for each model and justification for TMB threshold selection.

Author Response

Major issues:

1- Histological features vs. established predictors: the authors need to provide a clearer comparison of the importance of histological features relative to well-known predictors such as lymph node involvement and age. These established features are known to be associated with clinical outcomes in TNBC, and it is crucial to contextualize the significance of the newly proposed histological features against these predictors.

Thank you for your valuable comment. We have added, in response to your suggestion, a comparison of lymph node status and age with the histological features selected from our predictive model. Based on SHAP analysis, we explored feature importance in the prediction of TMB, discussed how lymph node status and age predict prognosis, and supported our findings with relevant studies. Furthermore, our findings highlighted the complementary effect of these predictors and further enhanced the model’s predictive potential.  (Lines 374-402)

 

2- Model overfitting: given the relatively small cohort size, overfitting is a significant concern. The authors should detail the steps taken to control for overfitting more comprehensively. For example, running the classifier with TMB values randomly shuffled between patients to demonstrate that this significantly lowers model performance.

Thank you for the insightful suggestion. We recognize that overfitting is a significant concern, especially in small cohorts, and we have used several techniques to address this issue. We used a cross-validation technique and ensured no data leakage between the training and testing sets. Moreover, the random forest reduces overfitting by averaging the predictions of multiple trees trained on different subsets of the data which reduces the variance and enhances model performance and generalizability. Furthermore, when comparing the performance of our model, no significant gap was recorded between our metrics in training and testing sets, which suggests that overfitting was not a concern. Additionally, and as recommended, we performed permutation testing (Figure below) with 1000 permutations and 3 cross-validation and obtained a p-value of 0.001, further demonstrating that our model is strong and reliable.

 

 

3- Contextualization with existing literature: the authors do not contextualize their finding with the literature about TNBC mutation burden, some important papers in the field include:

1. Lee, J.V., Housley, F., Yau, C. et al. Combinatorial immunotherapies overcome MYC-driven immune evasion in triple negative breast cancer. Nat Commun 13, 3671 (2022)
2. Shah, S., Roth, A., Goya, R. et al. The clonal and mutational evolution spectrum of primary triple-negative breast cancers. Nature 486, 395–399 (2012).

Thank you for your remark. In our discussion, we discussed the genomic mutation load of TMB groups in TNBC, their DEGs, and their respective enriched pathways. Based on your feedback, we have added a thorough discussion and contextualized our findings on TMB in TNBC by incorporating important papers in the field, including the interesting ones provided. (Lines 339-357)

Minor comments:

1- Figure 2, please give the features comprehensible names.

Thank you for pointing that out, we have modified the feature names into more comprehensible ones.

 

2- Provide more details on hyperparameter tuning for each model and justification for TMB threshold selection.

Thank you for your thoughtful suggestions. For the hyperparameter tuning, we have provided a table summarizing, for each model, the range of hyperparameters used and optimal parameters selected. Additionally, we explained the use of some key parameters in lines 235-253 to provide a better understanding of the choices made for the model optimization.

For the TMB threshold selection, we explained in lines 135-145 of the literature review section and line 87 that various thresholds have been used by researchers, including the mean, median, FDA-approved 10 mutations/Mb, and others. As detailed in lines 211-218 in the methods section, to select the optimal threshold for TMB we computed the Cox proportional hazards regression and Kaplan-Meier analysis to calculate the hazard ratio of TMB and assessed the statistical significance of the different thresholds. Table S1 shows the hazard ratio and p-value of each threshold.

Author Response File: Author Response.pdf

Round 2

Reviewer 1 Report

Comments and Suggestions for Authors

The authors have successfully made all the necessary changes to the feedback received and have significantly improved the overall quality of the manuscript.