Estimation of the Prostate Volume from Abdominal Ultrasound Images by Image-Patch Voting

Round 1

Reviewer 1 Report

This submission presents an end-to-end deep learning based on Abdominal Ultrasound (AUS) for automated prostate volume estimation studies, enabling an expert-in-the-loop system. In particular, a Quadruplet Deep Convolutional Neural Network (QDCNN) was proposed.
The experiments were conducted on an AUS dataset from 305 patients with both transverse and sagittal planes; this dataset included MRI images for 75 of these patients.

The manuscript is overall interesting and well prepared, but the novelty and clinical relevance need to be pointed out. The experimental results are reasonable and clearly presented.
Moreover, careful proofreading would be beneficial.

My main concerns are listed in what follows.

1) Abstract: The final sentence "Our data set and project code will be made available for public use along with the expert markings." should be removed.
With this regard, when do the Authors plan to share the code and the dataset?
Moreover, the conclusive remarks have to be improved for summarizing the main findings. 

2) Considering medical decision-making tasks allowing for expert interaction, these recent and relevant articles should be introduced and discussed:
- Rundo, L., Pirrone, R., Vitabile, S., Sala, E., & Gambino, O. (2020). Recent advances of HCI in decision-making tasks for optimized clinical workflows and precision medicine. Journal of Biomedical Informatics, 103479. DOI: 10.1016/j.jbi.2020.103479
- Lutnick, B., Ginley, B., Govind, D., McGarry, S. D., LaViolette, P. S., Yacoub, R., ... & Sarder, P. (2019). An integrated iterative annotation technique for easing neural network training in medical image analysis. Nature Machine Intelligence, 1(2), 112-119. DOI: 10.1038/s42256-019-0018-3

3) Figure 3: Please use the multiplication sign '×' consistently throughout the labels in the figure.

4) Concerning end-to-end deep learning approaches for prostate cancer detection, please consider to discuss these interesting applications:
- Lapa, P., Castelli, M., Gonçalves, I., Sala, E., Rundo, L. (2020). A hybrid end-to-end approach integrating conditional random fields into CNNs for prostate cancer detection on MRI. Applied Sciences, 10(1), 338. DOI: 10.3390/app10010338
- Wang, Z., Liu, C., Cheng, D., Wang, L., Yang, X., & Cheng, K. T. (2018). Automated detection of clinically significant prostate cancer in mp-MRI images based on an end-to-end deep neural network. IEEE Transactions on Medical Imaging, 37(5), 1127-1139. DOI: 10.1109/tmi.2017.2789181

5) Lines 363-364: Please use '.' instead of ',' as a decimal separator.

6) The use of the subset of MRI scans would be justified in the proposed analysis. Also, the MR images shown in Figure 2 should be better linked to the US images in Figure 1.

7) Figure 10: The t-SNE plots need further explanation, especially with reference to the color semantics.

8) Section 5: conclusive remarks need to be extended and clarified, as well as a feasible plan for future work should be provided.

9) As future work, regarding the use of deep learning approaches for data integration, these papers should be discussed to complement the rationale of the study in the Introduction:
- Rundo, L., Militello, C., Vitabile, S., Russo, G., Sala, E., & Gilardi, M. C. (2020). A Survey on Nature-Inspired Medical Image Analysis: A Step Further in Biomedical Data Integration. Fundamenta Informaticae, 171(1-4), 345-365. DOI: 10.3233/FI-2020-1887
- Kothapalli, S. R., Sonn, G. A., Choe, J. W., Nikoozadeh, A., Bhuyan, A., Park, K. K., ... & Gambhir, S. S. (2019). Simultaneous transrectal ultrasound and photoacoustic human prostate imaging. Science Translational Medicine, 11(507). DOI: 10.1126/scitranslmed.aav2169

Author Response

Dear Reviewers, 

Thank you for your instructive suggestions. We have carefully read your reviews and revised our manuscript. Here are your reviews with our explanations in bold. We have also created a copy of our revised manuscript with the revisions highlighted, please see the attachment. 

 

Review 1: 

This submission presents an end-to-end deep learning based on Abdominal Ultrasound (AUS) for automated prostate volume estimation studies, enabling an expert-in-the-loop system. In particular, a Quadruplet Deep Convolutional Neural Network (QDCNN) was proposed. 
The experiments were conducted on an AUS dataset from 305 patients with both transverse and sagittal planes; this dataset included MRI images for 75 of these patients. 

The manuscript is overall interesting and well prepared, but the novelty and clinical relevance need to be pointed out. The experimental results are reasonable and clearly presented. 
Moreover, careful proofreading would be beneficial. 

• Author response: We rearranged the first paragraph of the introduction to mention the clinical relevance. Lines 19-27. 

My main concerns are listed in what follows. 

1) Abstract: The final sentence "Our data set and project code will be made available for public use along with the expert markings." should be removed. 
With this regard, when do the Authors plan to share the code and the dataset? 
Moreover, the conclusive remarks have to be improved for summarizing the main findings.  

• Author response: We removed the last sentence of the abstract. We added our data set and project code to the GitHub repository mentioned at the end of the introduction section. 

2) Considering medical decision-making tasks allowing for expert interaction, these recent and relevant articles should be introduced and discussed: 
- Rundo, L., Pirrone, R., Vitabile, S., Sala, E., & Gambino, O. (2020). Recent advances of HCI in decision-making tasks for optimized clinical workflows and precision medicine. Journal of Biomedical Informatics, 103479. DOI: 10.1016/j.jbi.2020.103479 
- Lutnick, B., Ginley, B., Govind, D., McGarry, S. D., LaViolette, P. S., Yacoub, R., ... & Sarder, P. (2019). An integrated iterative annotation technique for easing neural network training in medical image analysis. Nature Machine Intelligence, 1(2), 112-119. DOI: 10.1038/s42256-019-0018-3 

• Author response: We discussed these relevant articles in the last paragraph of the previous work section. Lines 169-171. 

3) Figure 3: Please use the multiplication sign '×' consistently throughout the labels in the figure. 

• Author response: We used the multiplication sign ‘x’ only for the pixel sizes in this figure. 

4) Concerning end-to-end deep learning approaches for prostate cancer detection, please consider to discuss these interesting applications: 
- Lapa, P., Castelli, M., Gonçalves, I., Sala, E., Rundo, L. (2020). A hybrid end-to-end approach integrating conditional random fields into CNNs for prostate cancer detection on MRI. Applied Sciences, 10(1), 338. DOI: 10.3390/app10010338 
- Wang, Z., Liu, C., Cheng, D., Wang, L., Yang, X., & Cheng, K. T. (2018). Automated detection of clinically significant prostate cancer in mp-MRI images based on an end-to-end deep neural network. IEEE Transactions on Medical Imaging, 37(5), 1127-1139. DOI: 10.1109/tmi.2017.2789181 

• Author response: We added these articles as samples of end-to-end systems to the last paragraph of the previous work section. Line 163. 

5) Lines 363-364: Please use '.' instead of ',' as a decimal separator. 

• Author response: We corrected the decimal separators. 

6) The use of the subset of MRI scans would be justified in the proposed analysis. Also, the MR images shown in Figure 2 should be better linked to the US images in Figure 1. 

• Author response: We linked these two figures in a sentence at the 3rd page. Lines 65-67. 

7) Figure 10: The t-SNE plots need further explanation, especially with reference to the color semantics. 

• Author response: We added explanation to the last paragraph of the ablation study section. Lines 395-398. 

8) Section 5: conclusive remarks need to be extended and clarified, as well as a feasible plan for future work should be provided. 

9) As future work, regarding the use of deep learning approaches for data integration, these papers should be discussed to complement the rationale of the study in the Introduction: 
- Rundo, L., Militello, C., Vitabile, S., Russo, G., Sala, E., & Gilardi, M. C. (2020). A Survey on Nature-Inspired Medical Image Analysis: A Step Further in Biomedical Data Integration. Fundamenta Informaticae, 171(1-4), 345-365. DOI: 10.3233/FI-2020-1887 
- Kothapalli, S. R., Sonn, G. A., Choe, J. W., Nikoozadeh, A., Bhuyan, A., Park, K. K., ... & Gambhir, S. S. (2019). Simultaneous transrectal ultrasound and photoacoustic human prostate imaging. Science Translational Medicine, 11(507). DOI: 10.1126/scitranslmed.aav2169 

• Author response: A paragraph of future work is added at the end of the conclusions section. Lines 428-431. 

 

 

Author Response File: Author Response.pdf

Reviewer 2 Report

The authors propose a system for estimating the prostate volume from abdominal ultrasound images. The method is mainly based on a neural network that combines image patch information at different scales for estimating a set of anatomic landmarks, which are then used to calculate the prostate volumes. The authors also created a database and plan to realease it for the research community. The manuscript structure is adecuate, and the proposed approach seems to outperform a fully manual methodology. However, some aspects should be enhanced.

Minor issues:
- Lines 26-29: This affirmation should be supported by a reference paper.
- Section 2: From my point of view, there is a confusion with the verb tenses used in this section. Sometimes the authors use the past simple, other times they use the present. Both tenses can be used together, however in this case I think the combination makes the reading quite hard.
- Line 216: The authors comment that their method was pretrained. More information should be given in this regard. Which dataset they used? Why they decided to freeze only the last two layers?
- Section 4: The authors refers to Table 1 with values that do not appear in that table (i.e. 2.81cm3 in line 301 or 3.73cm3 in line 302). I understand that these values are calculated from the information of that Table, but the link should be more clear. Perhaps the formula used to calculate those values should be given.
- Table 1: The caption of this table should state which unit are its values in.
- Tables 1 and 2: These tables should be merged into one table, using the naming convention mean+-sd. It makes no sense for them to appear separately. Moreover, it is redundant to show a symmetric matrix. Please, remove the upper or the lower triangle for a better comprehension.
- Table 2. I bet these values do not correspond strictly to the standard deviation values of the Table 1. Table 1 shows the mean values of the AVD metrics, while Table 2 shows the standard deviation values of the AVD metrics. If the authors accept a suggestion, they can use the terms MAVD (Mean Absolute Value Difference) and SDAVD (Standard Deviation of the Absolute Value Difference), or use simply AVD (Absolute Value Difference) and then refer to the mean and standard deviation of AVD.

Major concerns:
- Section2: The authors refers to a previous work carried out by them targeted at the same objective. However, no other reference to this work can be seen in the manuscript. The improvements of the proposed system with respect to the previous ones should be highlighted 
- The first paragraph of the Section 4.2 states that the images were rescaled to remove scale differences between them. I think this is a very unrealistic testing scenario. In a real case, the images will present scaling differences. I guess the proper experiment should be leave the images in their original scale, and normalize the height, width and length measurements using the pixel sizes provided by the US device. In fact, the multiscale patch system should be helpful for dealing with scaling variance.
- Section 4: The authors show information on the standard deviation of the Absolute Value Difference. However, they do not use this information to compare the different tested approach. This metric should be taking into account, as it provides useful information on the stability of the estimations.
- Section 4: The authors claim that a comparison against any other automatic method is not possible. However, thay also mention a previous work focused on the same objective. Therefore, a comparison of the proposed system not only to a baseline, but also to the previous system, is desirable.
- Section 4.3: The ablation study is very poor. Only two test cases regarding the method are tested (use of QDCNN or not) are used. Moreover, a Table with the incremental use of the method components is needed to easily understand the author decisions.

 

Author Response

Dear Reviewers, 

Thank you for your instructive suggestions. We have carefully read your reviews and revised our manuscript. Here are your reviews with our explanations in bold. We have also created a copy of our revised manuscript with the revisions highlighted, please see the attachment. 

 

•  

 

Review 2: 

The authors propose a system for estimating the prostate volume from abdominal ultrasound images. The method is mainly based on a neural network that combines image patch information at different scales for estimating a set of anatomic landmarks, which are then used to calculate the prostate volumes. The authors also created a database and plan to realease it for the research community. The manuscript structure is adecuate, and the proposed approach seems to outperform a fully manual methodology. However, some aspects should be enhanced. 

Minor issues: 

- Lines 26-29: This affirmation should be supported by a reference paper. 

• Author response: We added proper references. Lines 28-31. 

- Section 2: From my point of view, there is a confusion with the verb tenses used in this section. Sometimes the authors use the past simple, other times they use the present. Both tenses can be used together, however in this case I think the combination makes the reading quite hard. 

• Author response: We checked Section 2 and revised using past tense. 

- Line 216: The authors comment that their method was pretrained. More information should be given in this regard. Which dataset they used? Why they decided to freeze only the last two layers? 

• Author response: We clarified the sentence and cited the pretrained model library. Lines 220-222. Marcelino Pedro's article, "Transfer learning from pre-trained models." (Towards Data Science (2018)) gives more information on the usage of pretrained models. 

- Section 4: The authors refers to Table 1 with values that do not appear in that table (i.e. 2.81cm3 in line 301 or 3.73cm3 in line 302). I understand that these values are calculated from the information of that Table, but the link should be more clear. Perhaps the formula used to calculate those values should be given. 

• Author response: We added the calculations in parenthesis through Sections 4.1 and 4.2. 

- Table 1: The caption of this table should state which unit are its values in. 

• Author response: We added the unit statement to the title of Table 1. 

- Tables 1 and 2: These tables should be merged into one table, using the naming convention mean+-sd. It makes no sense for them to appear separately. Moreover, it is redundant to show a symmetric matrix. Please, remove the upper or the lower triangle for a better comprehension. 

• Author response: Actually, we tried these suggestions but adding the std values made the table very crowded and unreadable, and giving a triangle table made it difficult to compare the values. 

- Table 2. I bet these values do not correspond strictly to the standard deviation values of the Table 1. Table 1 shows the mean values of the AVD metrics, while Table 2 shows the standard deviation values of the AVD metrics. If the authors accept a suggestion, they can use the terms MAVD (Mean Absolute Value Difference) and SDAVD (Standard Deviation of the Absolute Value Difference), or use simply AVD (Absolute Value Difference) and then refer to the mean and standard deviation of AVD. 

• Author response: Thank you for the suggestion, we defined SDAVD term (Lines 339-340.) and updated the title of Table 2 accordingly. 

Major concerns: 

- Section2: The authors refers to a previous work carried out by them targeted at the same objective. However, no other reference to this work can be seen in the manuscript. The improvements of the proposed system with respect to the previous ones should be highlighted  

• Author response: At the end of the previous work section, we discuss differences between our current work and previous work. Lines 154-161. 

- The first paragraph of the Section 4.2 states that the images were rescaled to remove scale differences between them. I think this is a very unrealistic testing scenario. In a real case, the images will present scaling differences. I guess the proper experiment should be leave the images in their original scale, and normalize the height, width and length measurements using the pixel sizes provided by the US device. In fact, the multiscale patch system should be helpful for dealing with scaling variance. 

• Author response: The pixel size is always available from the device so it makes sense to use this information to avoid the unnecessary pixel scale search process. We made clarifying addition to the sentence. Line 333. 

- Section 4: The authors show information on the standard deviation of the Absolute Value Difference. However, they do not use this information to compare the different tested approach. This metric should be taking into account, as it provides useful information on the stability of the estimations. 

• Author response: We talked about the std values in the paragraph. Lines 339-343. 

- Section 4: The authors claim that a comparison against any other automatic method is not possible. However, thay also mention a previous work focused on the same objective. Therefore, a comparison of the proposed system not only to a baseline, but also to the previous system, is desirable. 

• Author response: The previous work compared automatic segmentation with manual segmentations using a border distance metric. There is not a volume result that we can compare with ours. 

- Section 4.3: The ablation study is very poor. Only two test cases regarding the method are tested (use of QDCNN or not) are used. Moreover, a Table with the incremental use of the method components is needed to easily understand the author decisions. 

• Author response: We trained a twin network similar to the proposed model but gets two patches as input and contains two resnet models. We discussed the test results in the ablation study section and added a bar chart to compare the results with the baseline and proposed systems. 

 

 

Author Response File: Author Response.pdf

Reviewer 3 Report

Dear authors,

thank you for this interesting manuscript. The topic is worthwhile as imaging plays an ever-bigger role in modern medicine. However, modern imaging such as MRI can be cost intensive, so relatively low-cost, and widely available options such as ultrasound should be considered where possible. From a urologist’s point of view prostate volume is important when planning conservative and interventional treatment in patients with benign prostate enlargement as well as when considering further diagnostics in the detection of prostate cancer.  

Regarding diagnostics I have a correction: PSA is measured in a patient’s blood and not calculated from prostate volume as stated in the introduction. If you know the patient’s PSA-level and prostate size it allows to calculated PSA-density. However, this is usually not what the indication for prostate biopsy is based upon but could be used in patient counselling. I would not go as far to say that prostate volume leads to biopsies being omitted. I would suggest looking up current guidelines on prostate cancer diagnostics for further information and to adjust that section in your manuscript. I would also add some more information on clinical relevance and the introduction to highlight the topics importance and catch the reader’s attention.

I would suggest turning the TRUS image in figure 1 by 180° as in normal clinical practice.

From my point of view a lot of effort was put into planning a practical approach to automated prostate volume detection with abdominal ultrasound. The experimental set up with a well-defined control group with two different expert readers and MRI as a possible control is well chosen.

You report that the average intra-expert MAVD values for MR images is 2.81 cm³ while it is 3.73 cm³ for the AUS images. In my clinical opinion that is only a very small difference and does not seem clinically relevant in everyday life. Also inter expert MAVD difference between AUS-AUS and MRI-MRI of around 5 cm³ each is not clinically relevant.

I would suggest doing statistical analysis and comparing the different groups to test for statistically significant differences in MAVD. That would make the results far more interesting to the reader and for further evaluation in the future.

Author Response

Dear Reviewers, 

Thank you for your instructive suggestions. We have carefully read your reviews and revised our manuscript. Here are your reviews with our explanations in bold. We have also created a copy of our revised manuscript with the revisions highlighted, please see the attachment. 

 

 

Review 3: 

Dear authors, 

thank you for this interesting manuscript. The topic is worthwhile as imaging plays an ever-bigger role in modern medicine. However, modern imaging such as MRI can be cost intensive, so relatively low-cost, and widely available options such as ultrasound should be considered where possible. From a urologist’s point of view prostate volume is important when planning conservative and interventional treatment in patients with benign prostate enlargement as well as when considering further diagnostics in the detection of prostate cancer.   

Regarding diagnostics I have a correction: PSA is measured in a patient’s blood and not calculated from prostate volume as stated in the introduction. If you know the patient’s PSA-level and prostate size it allows to calculated PSA-density. However, this is usually not what the indication for prostate biopsy is based upon but could be used in patient counselling. I would not go as far to say that prostate volume leads to biopsies being omitted. I would suggest looking up current guidelines on prostate cancer diagnostics for further information and to adjust that section in your manuscript. I would also add some more information on clinical relevance and the introduction to highlight the topics importance and catch the reader’s attention. 

• Author response: Thank you for the suggestions. We updated the first paragraph of the introduction accordingly. Lines 19-27. 

I would suggest turning the TRUS image in figure 1 by 180° as in normal clinical practice. 

• Author response: We turned the TRUS image in Figure 1 by 180o. 

From my point of view a lot of effort was put into planning a practical approach to automated prostate volume detection with abdominal ultrasound. The experimental set up with a well-defined control group with two different expert readers and MRI as a possible control is well chosen. 

You report that the average intra-expert MAVD values for MR images is 2.81 cm3 while it is 3.73 cm3 for the AUS images. In my clinical opinion that is only a very small difference and does not seem clinically relevant in everyday life. Also inter expert MAVD difference between AUS-AUS and MRI-MRI of around 5 cm3 each is not clinically relevant. 

I would suggest doing statistical analysis and comparing the different groups to test for statistically significant differences in MAVD. That would make the results far more interesting to the reader and for further evaluation in the future. 

• Author response: We mentioned the statistical analysis as a future work at the end of the conclusions section. Lines 430-431. 

 

 

Author Response File: Author Response.pdf

Round 2

Reviewer 2 Report

The authors addressed the issues discussed

Reviewer 3 Report

My remarks from the first review were addressed. However, I would suggest doing statistical analysis for this manuscript not for a later project as this would increase the reader's interest in the topic.