Review Reports
- Nahuel Paesano 1,
- Violeta Catalá 1 and
- Juan Morote 6,8,*,†
- et al.
Reviewer 1: Anonymous Reviewer 2: Natale Calomino Reviewer 3: Ahmed Abdlkadir
Round 1
Reviewer 1 Report
Comments and Suggestions for AuthorsThis manuscript presents an external validation comparing two MRI-based predictive models (BCN-MRI PM and ERSPC-MRI PM) for clinically significant prostate cancer (csPCa) detection in an Ibero-American cohort of 540 men. The study addresses a clinically relevant question, and the head-to-head comparison using calibration, discrimination, decision curve analysis, and clinical utility curves is methodologically appropriate. Revisions are required before acceptance for publication.
- The study is described as both “a prospective analysis of 540 men” (Abstract, line 39) and “a retrospective validation and performance comparison” (Methods, line114), which is a contradiction.
- The csPCa detection rate of 55.7% is exceptionally high compared to the development cohorts of both models (BCN-MRI PM 36.4%, ERSPC-MRI PM 35.9%) and to the recalibrated ERSPC cohort (~20%), yet the implications of this prevalence discrepancy for model transportability are insufficiently addressed. A detection rate this high suggests a highly selected referral population (e.g., 51.3% PI-RADS 4, 25.4% PI-RADS 5), and the superior performance of the BCN-MRI PM may partly reflect that its development population more closely resembles this enriched cohort.
- The use of Cohen’s Kappa to assess correlation between two continuous probability estimates (line 158, 204) is methodologically inappropriate, as Kappa is designed for categorical agreement and requires arbitrary categorization of continuous probabilities that loses information and produces results sensitive to chosen cut-points. The reported Kappa of 0.143 is uninterpretable without specifying how probabilities were categorized. The authors should instead use Pearson or Spearman correlation, or the concordance correlation coefficient (CCC) for absolute agreement on the probability scale.
- To further strengthen the depth and contextualization of the background and discussion regarding PC driver mutation genes, predictive values of PSA combined with MRI, multi-omics fo PC prognosis, it would be highly beneficial for the authors to discuss several additional studies(PMIDs: 35127694, 40843359, 32609397) in this field.
- The three participating centers differ substantially in biopsy approach: IAF uses cognitive MRI-TRUS fusion, while CB and HCUCH use software-based fusion; CB performs mapping at 5-mm intervals whereas IAF and HCUCH use 2–4 cores per lesion (up to 3 lesions) targeted biopsy plus 12-core systematic. These differences in biopsy technique affect csPCa detection rates independently of model performance, yet the authors do not adjust for biopsy protocol in their comparisons. The heterogeneity in biopsy scheme is confounded with center, making it impossible to disentangle model performance from detection methodology.
- The manuscript states that “All prostate biopsies were performed via the transperineal route” (line 137), which is a strength, but the development cohorts of both models predominantly used transrectal biopsy. The authors attribute part of the ERSPC-MRI PM miscalibration to this route difference (line 374) but do not discuss whether the transperineal approach may have systematically altered the sensitivity of csPCa detection in ways that affect both models differentially. Given that transperineal biopsy detects more anterior lesions, the authors should discuss whether the model variables (especially prostate volume and PI-RADS score) interact differently with biopsy route.
Author Response
I would like to thank the Reviewer for their comments and suggestions. I have addressed all of them and made appropriate modifications to the manuscript, which are highlighted in red.
This manuscript presents an external validation comparing two MRI-based predictive models (BCN-MRI PM and ERSPC-MRI PM) for clinically significant prostate cancer (csPCa) detection in an Ibero-American cohort of 540 men. The study addresses a clinically relevant question, and the head-to-head comparison using calibration, discrimination, decision curve analysis, and clinical utility curves is methodologically appropriate. Revisions are required before acceptance for publication.
1. The study is described as both “a prospective analysis of 540 men” (Abstract, line 39) and “a retrospective validation and performance comparison” (Methods, line114), which is a contradiction.
Response: You are correct. Indeed, this was a prospective study designed to validate the BCN predictive models in an Ibero-American population. However, the present analysis is a retrospective evaluation of cases collected from three participating centers in which the risks of clinically significant prostate cancer (csPCa) were calculated using both the ERSPC-MRI and BCN-MRI predictive models. To avoid confusion, we have revised the manuscript to clearly describe the study as a retrospective analysis in Lines 37 and 163.
2. The csPCa detection rate of 55.7% is exceptionally high compared to the development cohorts of both models (BCN-MRI PM 36.4%, ERSPC-MRI PM 35.9%) and to the recalibrated ERSPC cohort (~20%), yet the implications of this prevalence discrepancy for model transportability are insufficiently addressed. A detection rate this high suggests a highly selected referral population (e.g., 51.3% PI-RADS 4, 25.4% PI-RADS 5), and the superior performance of the BCN-MRI PM may partly reflect that its development population more closely resembles this enriched cohort.
Response: The reviewer's observation is valid. In the original development studies, the distribution of PI-RADS categories differed between the BCN-MRI and ERSPC-MRI cohorts. Specifically, the proportions were 24.9% vs. 18% for PI-RADS <3, 21% vs. 18% for PI-RADS 3, 41.3% vs. 40% for PI-RADS 4, and 12.6% vs. 18% for PI-RADS 5, respectively (References 32 and 22). As expected, the distribution of PI-RADS categories in the Ibero-American validation cohort also differed from those observed in both development cohorts. This represents a challenging validation scenario for both predictive models and may have contributed to the differences in their performance. Nevertheless, both models were evaluated under the same conditions, allowing a fair comparison of their discriminative ability in this external validation cohort.
3. The use of Cohen’s Kappa to assess correlation between two continuous probability estimates (line 158, 204) is methodologically inappropriate, as Kappa is designed for categorical agreement and requires arbitrary categorization of continuous probabilities that loses information and produces results sensitive to chosen cut-points. The reported Kappa of 0.143 is uninterpretable without specifying how probabilities were categorized. The authors should instead use Pearson or Spearman correlation, or the concordance correlation coefficient (CCC) for absolute agreement on the probability scale.
Response: We apologize for this oversight. The reviewer is correct. We have revised the Statistical Methods section (Line 206) to specify the appropriate statistical test used. In addition, we have corrected the Results section (Line 253 to report the Pearson correlation coefficient with its 95% confidence interval. The reported results are consistent with those obtained using the kappa statistic and do not affect the interpretation of the findings.
3. To further strengthen the depth and contextualization of the background and discussion regarding PC driver mutation genes, predictive values of PSA combined with MRI, multi-omics for PC prognosis, it would be highly beneficial for the authors to discuss several additional studies (PMIDs: 35127694, 40843359, 32609397) in this field.
Response: We appreciate the reviewer's comment. Accordingly, we have revised the Discussion to acknowledge this important point. Specifically, we have added the following statement in Lines 473-474: "Multi-omics analyses are expected to provide a more refined characterization of clinically significant prostate cancer (csPCa) detected in prostate biopsies, with improved correlation with the true whole-prostate pathology [56–58]." We have added the proposed references.
4. The three participating centers differ substantially in biopsy approach: IAF uses cognitive MRI-TRUS fusion, while CB and HCUCH use software-based fusion; CB performs mapping at 5-mm intervals whereas IAF and HCUCH use 2–4 cores per lesion (up to 3 lesions) targeted biopsy plus 12-core systematic. These differences in biopsy technique affect csPCa detection rates independently of model performance, yet the authors do not adjust for biopsy protocol in their comparisons. The heterogeneity in biopsy scheme is confounded with center, making it impossible to disentangle model performance from detection methodology.
Response: Since we identified heterogeneity in the optimal thresholds across PI-RADS categories and participating centers, the sample size was insufficient to perform subgroup analyses according to both PI-RADS category and center. This represents a limitation of the study and has been acknowledged in the limitations section. In our opinion, external validations should ideally be conducted separately for each center and within each PI-RADS category, which would require substantially larger validation cohorts, as stated in the Limitations section. Nevertheless, the overall sample size was adequate to support the primary validation analyses and the conclusions derived from the overall cohort.
5. The manuscript states that “All prostate biopsies were performed via the transperineal route” (line 137), which is a strength, but the development cohorts of both models predominantly used transrectal biopsy. The authors attribute part of the ERSPC-MRI PM miscalibration to this route difference (line 374) but do not discuss whether the transperineal approach may have systematically altered the sensitivity of csPCa detection in ways that affect both models differentially. Given that transperineal biopsy detects more anterior lesions, the authors should discuss whether the model variables (especially prostate volume and PI-RADS score) interact differently with biopsy route.
Response: The reviewer is correct. However, the same scenario applied of both predictive models. The only difference is that the BCN-MRI PM had previously been validate in population assessed using PI-RADS v 2.1 and the transperineal approach in detecting csPCa. Importantly, the advantage of the transperineal approach in detecting clinically significant prostate cancer (csPCa), particularly in anterior lesions, was the same for both predictive models and therefore does not explain the observed difference in performance.
Reviewer 2 Report
Comments and Suggestions for AuthorsOur fellow researchers have produced a study whose aim: Our objective was to validate and compare the performance of the BCN-MRI PM predictive model and the ERSPC-MRI PM predictive model in an Ibero-American population, besides this main one the researchers have identified secondary objectives such as: (1) to describe the baseline characteristics of the overall study population and evaluate the heterogeneity between the participating centers; (2) to compare the calibration, discrimination and net benefit of the models for the detection of csPCa; (3) to compare the overall clinical efficacy of both models, especially at high sensitivity thresholds for the detection of csPCa; and (4) to compare the clinical utility of both predictive models based on PI-RADS scores and participating centers. The title is well harmonized with the whole study and the abstract is a good summary of the whole work, The next session introduces in a good way the object of study. Significant importance is given to screening and the diagnostic phase of the disease as the cornerstone of the oncology process. Consistent with the title of the paper, the concept of prediction based on magnetic resonance imaging is introduced. The methods are well-written; colleagues retrospectively examined a cohort of 450 people, for whom we requested the minimum and maximum ages and the average number of biopsies performed. The precise description makes the study easily replicable. The statistics are also well-conducted and well-described. The results are well-described; the tables with clear captions are particularly important. In any case, these, along with a selected bibliography, prepare for the discussion. The latter highlights the factors that lead us to focus on the data our colleagues present in this paper. A couple of questions naturally arise for colleagues: the first is whether the lymph nodes, which were not mentioned at all, are important in building a predictive model; The second is whether artificial intelligence, which has forcefully entered surgical practice (doi.org/10.3390/jcm15062208, to be read and cited in the bibliography), could be helpful in building a predictive model, given the amount of data collected by colleagues at the three centers where the study was conducted. We agree with the limitations of their paper; the strengths are clear in the conclusions. Compliments to the authors for the fundamental iconography and accompanying captions. Excellent English. Good bibliography.
Author Response
I would like to thank the Reviewer for their comments and suggestions. I have addressed all of them and made appropriate modifications to the manuscript, which are highlighted in red.
Comment
Our fellow researchers have produced a study whose aim: Our objective was to validate and compare the performance of the BCN-MRI PM predictive model and the ERSPC-MRI PM predictive model in an Ibero-American population, besides this main one the researchers have identified secondary objectives such as: (1) to describe the baseline characteristics of the overall study population and evaluate the heterogeneity between the participating centers; (2) to compare the calibration, discrimination and net benefit of the models for the detection of csPCa; (3) to compare the overall clinical efficacy of both models, especially at high sensitivity thresholds for the detection of csPCa; and (4) to compare the clinical utility of both predictive models based on PI-RADS scores and participating centers. The title is well harmonized with the whole study and the abstract is a good summary of the whole work, The next session introduces in a good way the object of study. Significant importance is given to screening and the diagnostic phase of the disease as the cornerstone of the oncology process. Consistent with the title of the paper, the concept of prediction based on magnetic resonance imaging is introduced. The methods are well-written; colleagues retrospectively examined a cohort of 450 people, for whom we requested the minimum and maximum ages and the average number of biopsies performed. The precise description makes the study easily replicable. The statistics are also well-conducted and well-described. The results are well-described; the tables with clear captions are particularly important. In any case, these, along with a selected bibliography, prepare for the discussion. The latter highlights the factors that lead us to focus on the data our colleagues present in this paper. We agree with the limitations of their paper; the strengths are clear in the conclusions. Compliments to the authors for the fundamental iconography and accompanying captions. Excellent English. Good bibliography.
Response: Thank you very much for your review.
A couple of questions naturally arise for colleagues:
- The first is whether the lymph nodes, which were not mentioned at all, are important in building a predictive model.
Response: This study analyzes two models for predicting csPCa in prostate biopsy specimens. It does not address the clinical staging of PCa or true final pathology observed patients undergoing radical prostatectomy with or without extended pelvic lymph node dissection. Currently, pelvic lymph node status is assessed based on MRI, including the presence of enlargement lymph nodes. Currently in patients with high-risk disease, PSMA PET/CT is recommended to detect metastatic disease at any site, including lymph nodes.
- The second is whether artificial intelligence, which has forcefully entered surgical practice (doi.org/10.3390/jcm15062208, to be read and cited in the bibliography), could be helpful in building a predictive model, given the amount of data collected by colleagues at the three centers where the study was conducted.
Response: As stated at the end of Discussion, the combination of appropriate machine learning algorithms and federated networks represents a promising approach for enabling the continuous online updating of predictive models, which remains one of the greatest challenges in this field (lines 480-483). This interesting study has now been included as Reference 63.
Reviewer 3 Report
Comments and Suggestions for AuthorsThe current multicenter retrospective study aims to compare the validation and performance of the Barcelona and ERSPC-MRI predictive models in patients suspected of having prostate cancer within an Ibero-American population. Below, I outline several comments and revision points that should be addressed in the next manuscript version before consideration for publication.
- The current title does not explicitly reflect the main outcome of interest, namely patients suspected of having prostate cancer. Please revise the title accordingly to improve clarity.
- Abstract:
- Background: Specify the studies outcome of interest
- Methods: Correct typo error in “met-hods”
- Methods: What was the main adapted reference standard for you studied outcome (i.e., clinical, biochemical profile, or combination of thereof).
- Methods: Briefly summarize the main adopted statistical test and the software utilized in your analysis.
- Results: Mention the total number of MRI exams, total number of biopsies offered, and age distribution of included cohort.
- Introduction:
- Briefly summarize PI-RADS scoring categorization in a new table.
- Please acknowledge whether prior studies of Ibero-American Population exist in the current literature.
- Materials and Methods:
- Outline the range of examined MRI sequences and mention who were in charge of MRI scan review.
- Dedicate a new subheading to outline the main collected data variables and mention the software utilized for data collection and organization.
- Remove double spacing in line 165.
- Results:
- Are all patients therapy naïve?
- Table 1: What about history of ongoing BPH? Please elaborate with number/percentages
- Briefly describe number and range of offered therapy lines in your cohort.
- Discussion:
- Acknowledge key uncontrolled confounders, such as variability in MRI scanners, differences in reader expertise, and inter-center reporting variability, as study limitations.
- Discuss the fact that the current predictive models do not incorporate all available diagnostic parameters. Other emerging scoring systems, such as the PRIMARY score based on PSMA hybrid imaging and the miPSMA score, should also be considered in future studies for evaluating patients with suspected prostate cancer.
Author Response
I would like to thank the Reviewer for their comments and suggestions. I have addressed all of them and made appropriate modifications to the manuscript, which are highlighted in red.
The current multicenter retrospective study aims to compare the validation and performance of the Barcelona and ERSPC-MRI predictive models in patients suspected of having prostate cancer within an Ibero-American population. Below, I outline several comments and revision points that should be addressed in the next manuscript version before consideration for publication.
- The current title does not explicitly reflect the main outcome of interest, namely patients suspected of having prostate cancer. Please revise the title accordingly to improve clarity.
Response: Thank you. The title has been revised and added “in men suspected of having prostate cancer”
- Abstract:
- Background: Specify the studies outcome of interest. Response: The sentence has been changed “External validation of predictive models is needed in new populations after their implementation”
- Methods: Correct typo error in “met-hods”. Response: Corrected
- Methods: What was the main adapted reference standard for you studied outcome (i.e., clinical, biochemical profile, or combination of thereof). Response: The reference standard outcome was the rate of avoided biopsies. The sentence “Reference standard outcome was the rate of avoided biopsies” has been added.
- Methods: Briefly summarize the main adopted statistical test and the software utilized in your analysis. Response: This sentence has been added: “Calibration, discrimination of csPCa, net benefit, and clinical utility were analyzed with R software”
- Results: Mention the total number of MRI exams, total number of biopsies offered, and age distribution of included cohort. Response: These sentences have been added: “All participants underwent MRI and prostate biopsy. The median age was 66 years (95% CI 61-71)”
- Introduction:
- Briefly summarize PI-RADS scoring categorization in a new table. Response: This sentence has been added in lines 73-74, “The risk of csPCa is categorized into five levels: very low (1), low (2), intermediate (3), high (4), and very high (5)”
- Please acknowledge whether prior studies of Ibero-American Population exist in the current literature. Response: Only one previous validation of the BCN-MRI PM exists in Ibero-American population and none for the ERSPC-MRI PM. This sentence is added in lines 92-93, and European and Ibero-American populations exist in line 107.
- Materials and Methods:
- Outline the range of examined MRI sequences and mention who were in charge of MRI scan review. Response: Examined sequences of MRI are mentioned in lines 138-139, “MRI examinations were performed according to the recommendations of the European Society of Urogenital Radiology and included T2-weighted, diffusion-weighted, and dynamic contrast enhanced sequences [38
- Dedicate a new subheading to outline the main collected data variables and mention the software utilized for data collection and organization. Response: This new subheading has been included: “2.3 Data variables. Variables included in the BCN-and ERSPC-MRI PM were: age (years), prior negative prostate biopsy, DRE, serum PSA, prostate volume derived from MRI, and PI-RADS. Family history of PCa was also required in the BCB-MRI PM. Participant centers reported an anonymized dataset in Excel to the coordinator center”
- Remove double spacing in line 165. Response: Done.
- Results:
- Are all patients therapy naïve?. Response: Yes
- Table 1: What about history of ongoing BPH? Please elaborate with number/percentages. Response: The only data we had was the percentage of men who underwent treatment with 5-ARI due to symptomatic BPH.
- Briefly describe number and range of offered therapy lines in your cohort. Response: Unfortunately, we have not this information since it was out our objective.
- Discussion:
- Acknowledge key uncontrolled confounders, such as variability in MRI scanners, differences in reader expertise, and inter-center reporting variability, as study limitations. Response: “Uncontrolled confounders, such as variability in MRI scanners, differences in reader expertise, and inter-center reporting va-riability, were also limitations” was added in line 435-437
- Discuss the fact that the current predictive models do not incorporate all available diagnostic parameters. Other emerging scoring systems, such as the PRIMARY score based on PSMA hybrid imaging and the miPSMA score, should also be considered in future studies for evaluating patients with suspected prostate cancer. Response: This sentence has been added in lines 440-442, “Other emerging scoring systems, such as the PRIMARY score based on PSMA hybrid imaging and the miPSMA score, should also be considered in future studies for evaluating men with suspected prostate cancer”. Reference 59 has been also added.
Author Response File:
Author Response.docx
Round 2
Reviewer 1 Report
Comments and Suggestions for AuthorsAuthors have addressed all comments provided by reviewers and now it is acceptable for publication in current version.
Reviewer 2 Report
Comments and Suggestions for AuthorsWe reviewed our colleagues' study and the changes made based on the reviewers' suggestions. The current draft, rewritten after careful revisions to various sections of the paper, has ensured its readiness, especially given the topic: prostate cancer. In our opinion, the paper should be offered to a broad audience of surgeons who treat this disease. The final section of the discussion, where the dissertation concerns the application of artificial intelligence, is particularly commendable. We trust the Editor will share our views on this matter. Endorse for publication