Review Reports - Regression Analysis of Rectal Cancer and Possible Application of Artificial Intelligence (AI) Utilization in Radiotherapy

Round 1

Reviewer 1 Report

This paper describes a study that uses data from 20 patients with rectal cancer, and analysis, using machine learning techniques (basically regression analysis) how different variables affect treatment success - quantified as reduction in tumor volume. The study is interesting, and certainly informative, but some aspects needs to be clarified.

While machine learning falls within 'artificial intelligence' (AI) I am not sure the title reflects accurately the study with only 20 patients. It is certainly a nice start, but not what it is typically mean by AI (e.g. deep learning strategies).
Line 46 and on - are age and sex also important when considering treatment? How about other comorbidities? Which ones are most important?
Materials and methods. There are some discrepancies between the text and the data showed in tables. For instance line 83 mentions 1.8 Gy per fraction for all patients, but Table 1 shows different doses per fraction for different patients. Lines 90 to 93 are confusing - did you use in total 6 images per patient (including the pre-treatment image, and 5 subsequent images for weeks 1 to 5)? If that is the case, the total number of images was 120, correct? Please clarify and also clarify what image was used as the pre-treatment one, and when it was acquired.
Lines 105 and 122 - please link reference.
Line 107 - mentions that 4 patient resulted in negative TVRR, or tumor volume increase. However, when examining Table 1 there are 5 patients with tumor increase (patients 6, 11, 12, 14 and 19). In Table 2, however, the volume change for patient 14 does not show as negative as it should. Please correct or clarify.
The authors briefly mention about the patients for which the tumor enlarges after therapy - I think this needs to be discussed more (perhaps in the paper discussion section). Is this common? In this case, it seems to be 1/4 or 25% of patients (5 out of 20). Furthermore, there are other patients for which the tumor barely changes, like patient 20. How accurate is the contouring algorithm? And how good is the planning software? Or do these patients present other characteristics/co-morbidities that make them non-responders? What do doctors do in this case? Stop or change treatment? How long after radiation started? Please discuss this issue in more depth.
Line 123-125 - how was the patient dose, dose per fraction and number of fractions determined? Based on the planning software? How does the software determines it?
Line 141 - I don't think NP was defined. What is it?
Figure 1: What is the 'importance score'? How was it calculated? Could you please give the equation for it or more details about it?
Figure 2. I don't understand the PDP and ICE plots - how were they obtained? What are they exactly? What is plotted in the horizontal axis? - For example, what is plotted in the horizontal axis for 'gender' I was expected only two values, male or female... Please clarify.
Lines 179 and 180 - dose per fraction appears as both a variable that does affect and that does not affect volume change... please correct.
Figure 3. Does this figure mean that at the same dose per fraction the treatment is less effective in younger people? Why? Is this a known fact? Please discuss more thoroughly.
Figure 4. The dependence is difficult to visualize, could you please change the point of view? So treatment success depends more on age than weight? Is this true? Is this a known fact too? Please discuss more thoroughly.
Figure 5. Does this figure imply that a greater dose per fraction is less effective (same weight)? and weight is actually less important? Is this a known fact? Please discuss more thoroughly.
Equation 3 - please provide the values for the constants. Do they make sense?
Lines 220-221 is this the R^2? (0.164 is very low).
Line 274-275 - feasibility of employing machine learning instead? And perhaps AI as a future possibility?

Author Response

We appreciate you and the reviewers for your precious time in reviewing our paper and providing valuable comments. The authors have carefully considered the comments and tried our best to address every one of them. We hope the manuscript after careful revisions meet your high standards. The authors welcome further constructive comments if any.

Below we provide the point-by-point responses. All modifications in the manuscript have been highlighted in red.

Response to Reviewer 1

While machine learning falls within 'artificial intelligence' (AI) I am not sure the title reflects accurately the study with only 20 patients. It is certainly a nice start, but not what it is typically mean by AI (e.g. deep learning strategies). The aim of this study to investigate using AI in adaptive radiation therapy treatment planning by predicting the Tumor Volume Reduction Rate (TVRR). That is the plan for future study, employing machine learning will required a greater number of participants and the main purpose of this study is to investigate if there is any relation between the Tumor Volume Reduction Rate and the mentioned biomarkers and open the gate for further investigation in this area which has not been investigated before.

Line 46 and on - are age and sex also important when considering treatment? How about other comorbidities? Which ones are most important? In this study we investigated each biomarker and combination of multiple markers. As mentioned in line 120, The relation between the target volume reduction rate and a patient’s clinical variables such as age, Total Dose (Gy), gender, number of treatment fractions, dose per fraction, and patient weight was analyzed.
Materials and methods. There are some discrepancies between the text and the data showed in tables. For instance line 83 mentions 1.8 Gy per fraction for all patients, but Table 1 shows different doses per fraction for different patients.

True: the text should be modified to 1.8 – 2 Gy per fraction for all patients for a total dose between 45Gy – 59.4 Gy. Thank you very much for the reminder. We have made revisions accordingly.

Lines 90 to 93 are confusing - did you use in total 6 images per patient (including the pre-treatment image, and 5 subsequent images for weeks 1 to 5)? If that is the case, the total number of images was 120, correct? Please clarify and also clarify what image was used as the pre-treatment one, and when it was acquired. )? Yes however, 1^st value is the pre-treatment volume (GTV) delineated on the planning CT which is usually taken around 2 - 3 weeks prior to 1^st fraction, the rest of 5 volumes are the ones delineated using the weekly CBCT from week 1 to week 5. Thus the total number of images is 100 CBCT (5 CBCT x 20 patients) + 20 GTV

Lines 105 and 122 - please link reference. Thank you very much for the reminder. We have made revisions accordingly.

Line 107 - mentions that 4 patient resulted in negative TVRR, or tumor volume increase. However, when examining Table 1 there are 5 patients with tumor increase (patients 6, 11, 12, 14 and 19). In Table 2, however, the volume change for patient 14 does not show as negative as it should. Please correct or clarify. Thank you very much for the reminder. We have made revisions accordingly. It should be -42.8 for patient 14.
The authors briefly mention about the patients for which the tumor enlarges after therapy - I think this needs to be discussed more (perhaps in the paper discussion section). Delineation of GTV includes the full circumference of the involved segment of rectum by the tumor, for example: if the tumor is semi-circumferential we contour the whole circumference of the involved rectum at that level which means that the volume of GTV could change when the rectum is dilated but it doesn’t mean that the tumor enlarges. Its only the rectal filling which determine the volume of the affected segment of the rectum Is this common? In this case,This is common despite the instruction given to patients regarding bowel preparation before each radiotherapy session including prescription of laxatives, some patients fail to follow these instructions or they may suffer from chronic constipation or other co-morbidities. it seems to be 1/4 or 25% of patients (5 out of 20). Furthermore, there are other patients for which the tumor barely changes, like patient 20. How accurate is the contouring algorithm? Contouring of rectum using CBCT has its own limitations due to the resolution and quality of the images in showing soft tissues compared to planning CT images. And how good is the planning software? Considered one of the best planning systems available in the market Or do these patients present other characteristics/co-morbidities that make them non-responders? Has nothing to do with the treatment response as the change in the diameter of the rectum is mainly due to the variation of rectal filling throughout the treatment course. Quality of CBCT images is not good enough to visualize the tumor itself What do doctors do in this case? Stop or change treatment? How long after radiation started? Please discuss this issue in more depth. when diameter of the rectum is too big usually (>20 cm) the patient is asked to evacuate the rectum and come again for CT simulation / CBCT or in some cases rectal enema is prescribed before CT sim and/or before each treatment session

Line 123-125 - how was the patient dose, dose per fraction and number of fractions determined? Based on the planning software? How does the software determines it? Total dose of 45Gy – 50.4Gy with 1.8 to 2 Gy / fraction is an internationally approved regimen as the standard dose for neoadjuvant treatment with concurrent chemotherapy for locally advanced rectal cancer. Only one patient with adenocarcinoma involving anal canal and rectum received a total dose of 59.4 Gy is used (anal canal protocol) was included in this study
Line 141 - I don't think NP was defined. What is it? Its Nano-particles (NPs). Thanks, corrected.
Figure 1: What is the 'importance score'? How was it calculated? Could you please give the equation for it or more details about it? Mentioned in line 138, “A small p-value of the test statistic implies that the corresponding variable is essential. The output score is –log(p). Therefore, a considerable score value indicates that the corresponding variable is important”
Figure 2. I don't understand the PDP and ICE plots - how were they obtained? What are they exactly? What is plotted in the horizontal axis? - For example, what is plotted in the horizontal axis for 'gender' I was expected only two values, male or female... Please clarify.

Individual Conditional Expectation (ICE) plots demonstrate how the prediction of an instance varies when an attribute changes. Because it focuses on a general average rather than particular cases, the partial dependency plot for the average influence of a feature is a global technique. Individual conditional expectation (ICE) plots are the equivalent of a PDP for individual data instances. In contrast to partial dependency plots, an ICE plot visualizes the prediction's dependence on a characteristic for each occurrence individually, resulting in one line per instance instead of one line overall. An ICE plot's lines are averaged to form a PDP.

Lines 179 and 180 - dose per fraction appears as both a variable that does affect and that does not affect volume change... please correct. Thank you very much for the reminder. We have made revisions accordingly. It should be “ The figure shows that gender, number of treatment fractions and total dose do not affect volume change within the patient group studied. However, dose per fraction, age and weight appear to correlate with percentage volume change within this patient group.“
Figure 3. Does this figure mean that at the same dose per fraction the treatment is less effective in younger people? yes Why? Still under investigation Is this a known fact? Please discuss more thoroughly.
Figure 4. The dependence is difficult to visualize, could you please change the point of view? So treatment success depends more on age than weight? Age and weight combination show better treatment outcome as a combination. Is this true? Is this a known fact too? Still under investigation Please discuss more thoroughly.
Figure 5. Does this figure imply that a greater dose per fraction is less effective (same weight)? and weight is actually less important? This figure imply that a greater dose per fraction is more effective (same weight) and weight is actually is important, the lower dose per fraction reflect lower trement outcome as the weight increased Is this a known fact? Still under investigation Please discuss more thoroughly.
Equation 3 - please provide the values for the constants. Do they make sense?

Thank you very much for the reminder. We have made revisions accordingly. “TVRR ~ 1 + Age +Dose Per Fraction +Weight”

Lines 220-221 is this the R^2? (0.164 is very low). “The Root Mean Squared Error (RMSE) of the linear regression is 0.164.” it is (RMSE) not R^2. Where R^2 describe the correlation between input and output the RMSE means the difference between the observed and the predicted values.
Line 274-275 - feasibility of employing machine learning instead? And perhaps AI as a future possibility? That is the plan for future study, employing machine learning will required a greater number of participants and the main purpose of this study is to investigate if there is any relation between the Tumor Volume Reduction Rate and the mentioned biomarkers and open the gate for further investigation in this area which has not been investigated before.

Author Response File: Author Response.pdf

Reviewer 2 Report

First of all thanks to the authors. Some suggestions to improve this work. The topic is interesting and the article is well written. However, some clarifications are needed:
- The author should provide an extensive list of inclusion and exclusion criteria.
- Line 82 has concomitant chemotherapy been administered?
- Line 84 the CBCTs have been revised by an experienced radiation oncologist? Which were the criteria for accepting a CBCT? Have the CBCTs been picked on the same day every week, ensuring the right time span between two subsequent CBCTs, or have they been picked randomly?
- Line 105 "Error! Reference source not found" is written instead of the table reference
- Line 106 taking into account the hypothesis beneath the negative TVRR of the four patients, may these factors have affected even the other results? Further investigations should be performed in order to understand the interplay of the different confounding factors.
- Table 2 gender "male" should be in capital letters such as "Female"
- Was the weight assessed week by week or just at the beginning of the treatment?
- Line 122 "Error! Reference source not found" is written instead of the table reference
- Would you think that using a different test group could be useful to externally validate the model?

Author Response

Below we provide the point-by-point responses. All modifications in the manuscript have been highlighted in red.

Response to Reviewer 2

First of all thanks to the authors. Some suggestions to improve this work. The topic is interesting and the article is well written. However, some clarifications are needed:

- The author should provide an extensive list of inclusion and exclusion criteria.

Inclusion criteria:

Age 18-80 year old patients
Locally advanced Adenocarcinoma of the rectum
Received neoadjuvant Long course chemo-radiotherapy
Treated with Elekta LINAC machines using Monaco Treatment Planning System
Daily CBCT was performed for image verification throughout the whole treatment course

Exclusion creiteria:

Age less than 18 or older than 80
Squamous cell carcinoma or other rare pathology of rectal tumors
Treated with short course protocol (25 Gy in 5 fraction, 5 Gy per fraction)
Treated with non Elekta System
Other image verification protocol where only weekly CBCT is performed

We have made revisions accordingly.

- Line 82 has concomitant chemotherapy been administered? Yes. In all patients

- Line 84 the CBCTs have been revised by an experienced radiation oncologist? Yes Which were the criteria for accepting a CBCT? Quality of CBCT images and Visibility of rectum and soft tissue should be good enough to delineate the rectum.

Have the CBCTs been picked on the same day every week, ensuring the right time span between two subsequent CBCTs, or have they been picked randomly? Yes in most of the cases CBCT of 1st day of the week was picked except for some cases where the quality of the image of 1st day CBCT was an issue, then CBCT of next consecutive day was picked.

- Line 105 "Error! Reference source not found" is written instead of the table reference. Thank you very much for the reminder. We have made revisions accordingly.

- Line 106 taking into account the hypothesis beneath the negative TVRR of the four patients, may these factors have affected even the other results? Further investigations should be performed in order to understand the interplay of the different confounding factors. We agree with the reviewer, in the future work a larger data will be used. In this study we aim at the major factor that may effect the output of the treatment.

- Table 2 gender "male" should be in capital letters such as "Female" Thank you very much for the reminder. We have made revisions accordingly.

- Was the weight assessed week by week or just at the beginning of the treatment? just at the beginning of the treatment

- Line 122 "Error! Reference source not found" is written instead of the table reference Thank you very much for the reminder. We have made revisions accordingly.

- Would you think that using a different test group could be useful to externally validate the model? That is the plan for future study, employing machine learning will required a greater number of participants and the main purpose of this study is to investigate if there is any relation between the Tumor Volume Reduction Rate and the mentioned biomarkers and open the gate for further investigation in this area which has not been investigated before.

Author Response File: Author Response.pdf

Round 2

Reviewer 1 Report

It appears that the authors uploaded a previous revised version, which does not include all revisions. They mentioned that revisions are highlighted in red, yet only the added inclusion criteria is highlighted. Perhaps other changes were not highlighted? But in looking at the paper, it does not seem that some of the changes requested were actually made.

For clarification: When I asked to discuss further, I meant not just explain in the response to reviewers, but actually discuss more thoroughly in the text of the paper - this does not seem to be done.

With respect to the paper title: it should ideally reflect current work, not envisioned future work. So please change the title to reflect the paper content, in which regression analysis (not AI) was used.

Author Response

Dear Editor,

We appreciate you and the reviewers for your precious time in reviewing our paper and providing valuable comments. Below we provide the point-by-point responses. All modifications in the manuscript have been highlighted in red.

Response to Reviewer 1