External Evaluation of a Predictive Model of Suboptimal Cytoreduction in Advanced Ovarian Cancer

Round 1

Reviewer 1 Report (Previous Reviewer 5)

Comments and Suggestions for Authors

Dear authors,

Thanks for your work

Please accept the following notes

1- Why was the control group not selected from the same time period to avoid bias?

2- Please define" Intestinal sub-obstruction"

3- The model describes PCI only as a predictive factor in addition to intestinal obstruction. So, what did the model add? PCI is well known to be the main predictive factor for resectability

4- The discussion should be more concise

5- The added benefit of using the model is not clear to the traditionally used PCI

Author Response

Thank you for your comments. I will try to answer your questions clearly and concisely.

1- Why was the control group not selected from the same time period to avoid bias?

The model was defined using a database of patients treated by the reference unit during the period 2013-2016, as described in the corresponding article in the bibliography (Llueca et al 2019).
Verification of a clinical model consists of three steps: creation of the rule, verification or validation of the rule, and evaluation of the impact of the rule on clinical behavior (impact analysis). The validation process may
require several studies to fully verify the accuracy of the rule in different centers.
The aim of the current study is this validation, carried out with data from other hospitals, which were collected prospectively. 

2- Please define" Intestinal sub-obstruction"

Intestinal sub-obstruction is a partial blockage of the small or large intestine, allowing limited passage of food, liquids, and gases, unlike complete obstruction. Its symptoms include colicky abdominal pain, nausea, vomiting, bloating, and often diarrhea due to the passage of liquid contents, requiring medical diagnosis by imaging (CT scan or X-ray). In our study it was diagnosed during the CT scan.

3- The model describes PCI only as a predictive factor in addition to intestinal obstruction. So, what did the model add? PCI is well known to be the main predictive factor for resectability.

4- The discussion should be more concise

As your suggest i've modified the discussion to make it more concise. I hope you agree. 

I have attached the new version of the article for you to review. In any case, I have copied the changes below.

Although this external validation study did not allow direct mathematical validation of the predictive model, it provides valuable insight into how the model has influenced clinical practice and patient selection in referral centres. The higher rate of complete and optimal cytoreduction in the experimental group (95.2% vs 86.3%) suggests improved surgical outcomes in highly experienced centres. This likely reflects stricter patient selection, partly guided by the model. However, intraoperative visual assessment of cytoreduction remains subjective. Emerging biomarkers such as circulating tumour DNA (ctDNA) (12), which correlates with residual tumour burden, may offer a more objective tool for postoperative evaluation.

Ascites was more frequent in the experimental group (49.4% vs 27.5%), although its direct impact on achieving complete cytoreduction remains unclear. While ascites is associated with advanced disease and poorer prognosis, it does not consistently preclude complete resection (13). The reason for its higher prevalence in the experimental cohort could not be determined.

A particularly relevant—and paradoxical—finding was the complete absence of intestinal sub-obstruction in the experimental group (0% vs 8%), a significant difference. As this condition is a strong adverse prognostic factor and contributes two points to the predictive model, the most plausible explanation is that high-risk patients were redirected to neoadjuvant therapy. Thus, although the model’s sensitivity for identifying “high-risk” cases could not be directly validated, its influence on clinical decision-making indirectly supports its effectiveness in preventing suboptimal surgery.

Accurate prediction tools are essential in advanced ovarian cancer. While FIGO stage is prognostically relevant, it does not adequately describe tumour burden or distribution. The Peritoneal Carcinomatosis Index (PCI) provides a more precise assessment by quantifying tumour extent across 13 abdominopelvic regions and is generally more informative than other scoring systems as Eisenkop and Fagotti (14,15). In this study, CT enterography was preferred for preoperative PCI assessment due to its improved correlation with surgical findings. Despite ongoing exploration of MRI and PET-CT, conventional CT—particularly CT enterography—remains the most practical and cost-effective imaging modality for evaluating peritoneal carcinomatosis in the cytoreductive setting (16,17,18).

The main limitation of this study was the very low number of suboptimal surgeries (four cases), which reduced statistical power. This was further influenced by the COVID-19 pandemic, which led to surgical delays and increased use of neoadjuvant therapy. Even with the planned sample size, the number of events would have been insufficient for robust ROC-based validation, rendering formal statistical discrimination unfeasible.

Notably, no patient in the experimental group was classified as “high risk” by models R3 or R4, suggesting selection bias. It is likely that the model was used prospectively to identify high-risk candidates and redirect them to alternative strategies, such as neoadjuvant chemotherapy. Although this limited conventional statistical validation, it demonstrates the model’s practical value in reducing suboptimal surgeries and associated morbidity.

In summary, while formal mathematical validation was not achievable due to the low event rate and the model’s influence on patient selection, this limitation reflects its integration into clinical practice. The findings suggest that the model contributes to improved patient selection and optimized surgical outcomes in specialized centres.

 

5- The added benefit of using the model is not clear to the traditionally used PCI

This model has another feature that could not be demonstrated in the current study, which is that is a dynamic model, that predicts either SCS or CCS and OCS depending on the prevalence of the SCS of the surgical team, and it changes with time depending on incorporating new therapeutic approaches and technical innovations. Combining sensitivity and specificity of the model with the actual prevalence of cytoreduction in a certain scenario (Bayes’ theorem), we obtained the predictive values. Positive predictive value for SCS is a simple probability but it is the one that must aid to determine the decision to be taken according to its magnitude.

It is explained in the original article mentioned above.

I hope you like my answers and will consider approving the draft of the article.

Reviewer 2 Report (Previous Reviewer 2)

Comments and Suggestions for Authors

REVIEW 2:

This multicenter controlled study is presented for review. The study aimed at developing a predictive model for predicting the risk of suboptimal cytoreduction in advanced ovarian cancer.

 

Compared to the previously submitted version of this article, Figures 1 and 2 have been added to improve comprehension. Tables duplicating Table 2, which reflects the clinical and pathological characteristics of patients in the study groups, have been removed. The "Discussion" chapter has been expanded to provide a more detailed discussion of the study's limitations.

Conclusion. Accept for publication.

Author Response

Thank you for your acceptance.

Round 2

Reviewer 1 Report (Previous Reviewer 5)

Comments and Suggestions for Authors

Thanks for your replies and the modifications performed to the manuscript

This manuscript is a resubmission of an earlier submission. The following is a list of the peer review reports and author responses from that submission.

Round 1

Reviewer 1 Report

Comments and Suggestions for Authors

General Assessment

This study aims to externally validate the predictive models (R3 and R4) developed by Llueca et al. for estimating the likelihood of suboptimal cytoreduction in advanced ovarian cancer within a multicenter prospective cohort. The topic holds high clinical relevance, as appropriate patient selection and reduction of surgical morbidity remain key challenges in gynecologic oncology.

The study design—prospective, multicenter, using REDCap for data management, and ethically approved—is commendable. However, the failure of statistical validation and the overwhelming influence of selection bias substantially weaken the study’s original hypothesis. Nonetheless, interpreting the findings as a form of “indirect clinical validation” is thought-provoking; the work provides valuable methodological lessons regarding model implementation in real-world settings.

Major Comments

1. Study Design and Rationale

Strengths: The multicenter nature of the study and inclusion of highly experienced surgical teams enhance the credibility and robustness of the collected data.

Weaknesses: The description of the study as a “prospective analytical study with retrospective data comparison” is confusing. The design essentially represents a retrospective cohort comparison between two time periods, and the term “prospective” should apply only to the newly collected cohort.

Recommendation: The study design and patient flow should be clarified according to STROBE guidelines, clearly detailing inclusion/exclusion criteria and handling of missing data (e.g., patient screening, exclusions, and attrition).

2. Sample Size and Statistical Power

Major issue: Although a target of 129 patients was calculated, only 83 were ultimately included, which substantially compromises the statistical power of the study.

Additionally: The number of outcome events (suboptimal surgeries, n = 4) is too low for meaningful statistical validation. Consequently, ROC curves, sensitivity, specificity, and AUC analyses cannot be reliably interpreted.

Recommendation: This limitation should be explicitly discussed with an a priori power analysis and a “futility analysis.” Exact binomial confidence intervals should be reported to accurately reflect the wide uncertainty due to low event rates.

3. Selection Bias and Interpretation

The authors acknowledge the presence of selection bias and attribute it to the clinical adoption of the predictive model itself. This interpretation is intriguing but should be presented as a hypothesis of association rather than a causal inference.

Moreover, the proportion of high-risk patients redirected to neoadjuvant chemotherapy is not quantified.

Recommendation: Include a clear flow diagram illustrating the number of patients initially evaluated, excluded (e.g., intestinal sub-obstruction, ECOG > 2), and included in the analysis to enhance transparency.

4. Variables and Model Performance

Although PCI was assessed using three methods (CT, laparoscopy, and intraoperative evaluation), inter-observer variability is not discussed. This variability could critically influence the model’s reproducibility and external validity.

While model scores are presented in tabular form, graphical visualization (histogram or box plot) would provide a clearer overview of score distribution and model behavior.

Recommendation: Present logistic regression coefficients and include a calibration plot (e.g., Hosmer–Lemeshow test) to support model performance assessment.

5. Discussion Depth and Literature Context

The discussion is generally comprehensive; however, the comparison with prior validation tools (e.g., Fagotti score, Eisenkop index, Predictive Index Value for Ovarian Cancer Surgery) remains limited.

Recommendations:

• The distinctions, advantages, and limitations of the current model relative to existing scoring systems should be explicitly emphasized.

• The impact of the COVID-19 pandemic on patient recruitment and surgical practice should be quantitatively supported and discussed.

Minor Comments

• Language and Style: While the manuscript is grammatically sound, the phrase “mathematical validation not feasible” is repeated excessively. A more formal and precise phrasing such as “statistical external validation could not be performed due to event scarcity” is preferable.

• Tables: Tables 2–5 lack sufficient explanatory captions. The meaning of the column “Valid n” should be clearly defined.

• Ethics: Although IRB approval (protocol 7/2019) is reported, details of multicenter ethical approvals or local institutional clearances are missing.

• References:

  • References [9] and [12] are by the same research group, which may introduce self-citation bias; this should be acknowledged.

  • The literature review ends in 2024; inclusion of the ESMO 2025 Ovarian Cancer Guidelines would improve the currency and completeness of the references.

Comments on the Quality of English Language

The manuscript is generally well-written and grammatically correct, but certain recurring expressions (e.g., “mathematical validation not feasible”) should be replaced with more academic alternatives such as “statistical external validation could not be performed due to event scarcity.” Minor stylistic edits would enhance readability and precision.

Author Response

Dear reviewer, thank you for your comments.

I will respond to your requests point by point:

I added to the manuscript the emphasization of the  distinctions and advantages of the current model, but I write down here for you.

The sources highlight that the UMCOAP model offers higher precision than existing tools:

• Eisenkop Model: Only evaluates 5 abdominal regions.
• Fagotti Score: Uses 7 parameters; while useful, it can lead to unnecessary surgeries for patients with intermediate scores (between 2 and 8). Moreover, actually its only way to perform it is laparoscopically.
• Anatomical Detail: By using 13 regions, the UMCOAP model describes the size and location of lesions more accurately, facilitating a more objective standardization of treatment. In addition, the PCI is per se an independent prognostic factor.

 (van de Vrie R, Kruitwagen RFPM, Powell ME, van der Heijden RHA, Stoeckle E, Stuart GC, et al. Diagnostic laparoscopy before primary cytoreductive surgery or neoadjuvant chemotherapy for advanced ovarian cancer. Cochrane Database Syst Rev. 2019;3:CD009567. doi:10.1002/14651858.CD009567.pub3

Llueca, A., Climent, M. T., Escrig, J., Carrasco, P., Serra, A., Gomez-Quiles, L., Játiva, R., Cebrian, G., Bosso, V., Villarin, A., Maiocchi, K., Delgado-Barriga, K., Rodrigo-Aliaga, M., Ruiz, N., Herrero, C., Frances, A., Beato, I., Ferrer, C., Aracil, J. P., Boldo, E., Boldo, A., and Adell, R. (2021). Validation of three predictive models for suboptimal cytoreductive surgery in advanced ovarian cancer. Scientific Reports, 11:8111.)

The article is based on a doctoral thesis registered for commencement in September 2019, when collaborating hospitals were contacted for data collection.

From March 2020 onwards, due to the global COVID pandemic, most hospitals, including ours, reduced or even suspended all peritoneal carcinomatosis operations, referring patients to chemotherapy.

As patients who undergo laparoscopy and CT scans prior to surgery are usually primary surgeries, no data was collected by the collaborating hospitals until well into 2022, and given the time limit, despite having requested an extension, this was decisive in not reaching the required number of patients. The doctoral thesis was submitted in February 2025, which is why the ESMO guidelines for that year are not included.

Our group has been publishing on ovarian carcinomatosis for years, so it is impossible not to use references from our group in the comments, especially if the study is based on the validation of a model described by us.

The English has been revised.

Reviewer 2 Report

Comments and Suggestions for Authors

REVIEW:

This multicenter controlled study is presented for review. The study aimed at developing a predictive model for predicting the risk of suboptimal cytoreduction in advanced ovarian cancer.

Certainly, the problem raised by the authors is extremely relevant, and  clinical or clinical laboratory models would be in demand  for oncologists.

However, the authors ultimately failed to develop a clinically and radiologically meaningful model, failed to evaluate its sensitivity, specificity, etc., and did not present ROC analysis data.

The article is replete with numerous fundamental methodological, stylistic, and other errors. I will highlight only a few:

1) The description of the study methods is completely insufficient. The authors state that the multicenter group was recruited from clinics specializing in the treatment of advanced ovarian cancer. However, the article does not describe the current approach to the scope of methods for diagnosis and staging of ovarian cancer (ultrasound and its variants, computed tomography of three zones (chest, abdomen, and pelvis) with intravenous contrast. Enterography is not included in the diagnostic standards for ovarian cancer! MRI of the pelvis and abdomen with intravenous contrast should be performed.

2) There is no description of PCI calculation (CT-PCI, laparoscopic PCI, intra-operative PCI).

3) The groups formed in the article (in different subchapters of the article) are designated as the control group, experimental group, MODEL, and CURRENT, which introduces uncertainty.

4) The assessment of clinical-CT partial intestinal obstruction is not described (in separate subchapters and Table 1, for example, this parameter is called "Absence of intestinal sub-obstruction" - this is unacceptable).

5)  The treatment of these patients is not described. The studies included patients with ovarian cancer complicated by pleural effusion (15% and 12% in both subgroups, respectively). It is unclear at what stage cytoreductive surgery was attempted in these patients? These patients are generally not amenable to surgical treatment.

6) Tables 3 and 4 do not contain significant information and should be removed. The significance level should be reflected in Table 2.

 

Conclusion. It is recommended that the article be rejected, as it requires extensive reworking of the existing results/deep statistical analysis.

Comments on the Quality of English Language

The English in the manuscript needs serious correction

Author Response

Dear reviewer, thank you for your comments.

I will respond to your requests point by point:

 

1. We assume that readers of this article are familiar with the usual diagnosis of ovarian cancer (physical examination, ultrasound, and thoracoabdominal-pelvic CT scan), so we do not believe it necessary, given the limited length of the text, to describe it.

As said at the text, all patients included underwent a thoracoabdominal-pelvic CT scan, with or without bowel preparation (entero-CT), in which the PCI was also quantified.

As described in the article referenced in point 18, CT enterography, which optimises luminal distension, has shown better correlation with surgical and pathological PCI than conventional CT and was therefore the preferred imaging technique in this study to quantify preoperative PCI.

 

2. Our group has been publishing articles on ovarian carcinomatosis for years, in which we have discussed the quantification of the peritoneal carcinomatosis index at length. We have modified the manuscript by adding a reference to the original article for Sugarbaker index. Below is a transcription of the paragraph mentioned.

Peritoneal cancer index (PCI) is used to quantitatively assess cancer distribution in the peritoneum based on calculating the sizes of lesions in 13 abdominopelvic regions. The sizes of the lesions are then converted to scores of 0–3: a lesion size score (LSS) of 0 defines no visible tumor burden in the peritoneum, while an LLS of 1, 2, or 3 describes lesions with a maximum diameter of 0.5, 5.0, and >5 cm or lesión confluence, respectively. PCI is calculated by adding the LSS for all regions, giving a maximum PCI of 39 (13×3) (Jacquet P, Sugarbaker PH. Peritoneal-plasma barrier. In 1996. p. 53–63.)

 

3. Following your request, terms that could lead to misunderstanding have been removed from the final manuscript.
4. Following your request, the term ‘absence of intestinal subobstruction’ has been removed and replaced with ‘no intestinal subobstruction’.
5. In the management of peritoneal carcinomatosis, if there is no positive pleural cytology, despite the presence of effusion at this level, it is not considered stage IV. In addition, many working groups, if the amount of tumour at the abdominal level is manageable, perform primary cytoreduction followed by adjuvant treatment. Several recent articles support this approach (Jochum, Floriane et al.

American Journal of Obstetrics & Gynecology, Volume 232, Issue 2, 194.e1 - 194.e11; Liu, H et al. Front. Oncol., 24 July 2023, Sec. Gynecological Oncology Volume 13 - 2023 | https://doi.org/10.3389/fonc.2023.1103357).

 

6. Tables has been removed.

 

English has been revised.

 

We sincerely hope that, in light of these modifications, you will find the revised version more satisfactory and that it may merit a renewed and thorough evaluation. We would greatly appreciate it if you could reassess the manuscript considering the revisions made.

 

Thank you again for your consideration.

Reviewer 3 Report

Comments and Suggestions for Authors

Dear Authors, 

external validation of different protocols or predictive models is of utmost importance, that is why I appreciate your efforts and will suggest your manuscript for publication. However, I would like to kindly ask you to smoothen the text and explain some inconsistiencies. 

1. Please provide in the introduction an explanation why you chose this model bearing in mind that there are also other models, that are clinically useful and used broadly. Actually, till now, I was not aware about these two names, although the model itself was quite familiar for me. I suppose that other non-hispanic readers may have similar impressions.
2. Second aim of the study seems to be created as "post-hoc" analysis - please rephrase that. 
3. Please clear all abbreviations, expecially from tables - CCS, OCS, SCS and use them consistently through the text or better use them in full each time
4. It is not clear for me if in your experimental group you assessed the model retrospectively despite prospective consenting patients or based on the model you excluded some patients from the surgery at all? Here we need clearly statemant, maybe with the help of graphical presentation of the flow of the patient? It is almost impossible that among 83 patients with stage III and IV no one had partial bowel obstruction.
5. Discussion section - is of high importance in this paper and is well written!

Thank you.

Author Response

Dear reviewer, thank you for your comments.

I will respond to your requests point by point:

1. The article is based on my doctoral thesis, which was written

 using the model developed by my mentors and published in 2019 by our group.  That is why this model was selected for external validation, as this is the next step after the internal validation described in the previous article. If you think it is necessary to include this comment, I will include it

 

2. We have replaced the paragraph describing the primary and secondary objectives with this one: Compare the characteristics of the population currently undergoing surgery for ovarian cancer with those of the population used to develop the model, which underwent this surgery between 2013 and 2016, focusing on those aspects that have been shown to influence suboptimal surgeries, in order to achieve the secondary objectives: Validate the suboptimal surgery model for ovarian cancer proposed by Llueca et al. in 2019 (8) and objectify that the knowledge derived from this predictive model has led to a change in the selection of patients eligible for surgical treatment.
3. It has been changed at the manuscript, adding at the end this abbreviations.
4. Patients in the experimental group were recruited prospectively and data were collected prospectively, but since the data were compared with a cohort of patients whose data had been reviewed retrospectively, we considered the data analysis to be retrospective.
5. Thank you for your comment!

 

I hope you like these changes and that they help the article get accepted.

Reviewer 4 Report

Comments and Suggestions for Authors

It is appreciable that that you have presented the negative results of not acheiving the objectives

You may compare the CT PCI, Laparoscopic PCI with PCI at Laparotomy as a secondary outcome measure

Author Response

Dear reviewer, thank you for your comments.

This article is based on a doctoral thesis, so only the results derived from it have been

discussed, but I appreciate your comment and it will be taken into account for

future publications by our group on this topic.

Reviewer 5 Report

Comments and Suggestions for Authors

Dear authors,

Thanks for your work

1- I suggest omitting "Challenges and Lessons from " from the title

2- The aim of the work was not achieved in view of the very small number of suboptimal debulking. Further data collection should extend till reaching an accepted number

3- Please explain the timing of diagnostic laparoscopy. The same surgery day, or a separate date?

4- Why the xipho-pubic incision for all patients?

5- Achieving complete cytoreduction for such a large number with stage IIIC & IV ovarian cancer is an achievement. Pleas explain

Author Response

Dear reviewer, thank you for your comments.

I will respond to your requests point by point:

 

• Following your suggestion, the title of the article has been changed.
• The article is based on a doctoral thesis registered for commencement in September 2019, when collaborating hospitals were contacted for data collection. From March 2020 onwards, due to the global COVID pandemic, most hospitals, including ours, reduced or even suspended all peritoneal carcinomatosis operations, referring patients to chemotherapy. As patients who undergo laparoscopy and CT scans prior to surgery are usually primary surgeries, no data was collected by the collaborating hospitals until well into 2022, and given the time limit, despite having requested an extension, this was decisive in not reaching the required number of patients.
• It depends on each hospital. In the patients used to construct the model, laparoscopy was performed separately from cytoreductive surgery. In the patients included in the prospective study, we are aware that some hospitals performed it during the same surgical procedure, but many others did so separately. The date of the laparoscopy is not recorded in the data collection database.
• As described in the procotol, a midline xipho-pubic incision was systematically performed to ensure adequate staging and complete exploration of both the inframesocolic and supramesocolic compartments, thereby maximizing the likelihood of achieving complete cytoreduction, as recommended in international guidelines for advanced ovarian cancer European Society of Gynaecological Oncology (ESGO).

du Bois A, Querleu D, Vergote I, et al.

Consensus guidelines for ovarian cancer surgery.

International Journal of Gynecological Cancer. 2016;26(4):S1–S25. doi:10.1097/IGC.0000000000000609

• The data is collected for each hospital via Redcap. One of the comments in the thesis is indeed that this number of complete cytoreductions is quite an achievement. We cannot control which patients our colleagues have included in the data collection. We understand that, as they are state-level reference units for this type of surgery, they have very good results, but it is not unreasonable to think that some of the collaborators have selected the patients they have included in the study. The only ones who have included all the patients who underwent this type of surgery in the period described are the reference units in the Valencian Community. The rest have included significantly fewer patients.