A Knowledge-Enhanced Hierarchical Reinforcement Learning-Based Dialogue System for Automatic Disease Diagnosis

Round 1

Reviewer 1 Report

Comments and Suggestions for Authors

 

 

 

The main idea of the paper is to propose a new model for disease diagnosis that utilizes deep reinforcement learning and knowledge enhancement to improve performance. The proposed model is called Knowledge Enhanced Hierarchical Reinforcement Learning (KNHRL) based Dialogue System for Automatic Disease Diagnosis. The paper describes the architecture of the model and evaluates its performance through experiments.

The study is based on a synthetic dataset, which may not fully capture the complexity and variability of real-world disease diagnosis scenarios. Therefore, the generalizability of the proposed model to real-world scenarios needs to be further investigated. Open source code must be provided for fair evaluation.

Author Response

We feel great thanks for your professional review work on our article. As you are concerned, there are several problems that need to be addressed.

We fully understand your doubt that the synthetic dataset may not fully capture the complexity and variability of real-world disease diagnosis scenarios. But the number of diseases and user goals in the real-world dataset is still limited. Due to the limited availability of real diagnostic datasets, we utilized an artificially synthesized dialogue dataset proposed for disease diagnosis.

In the future, we will consider validating the model's performance on real datasets. Additionally, our code is still being further organized, and we promise that open source code will be provided in the future.

Reviewer 2 Report

Comments and Suggestions for Authors

The paper presents Knowledge Enhanced Hierarchical RL model for strategy learning in the medical dialogue system for disease diagnosis. The authors integrate medical knowledge and disease symptom relation into a dialogue model. Overall paper is of good quality and is interested for readers.

Nevertheless, I have some suggestions for improvements or clarifications:

1. The most obvious concern is regarding the technical novelty of the paper. Apparently, there is no research contribution in the paper. In other words, RL methods are already available and the present work becomes the application of RL.

2. The formulation of the problem into an MDP model is also not clear. It is suggested to add a section on technical background and explain basic introduction to MDP an RL. It will help readers to understand the problem formulation into an MDP model.

3. Although the problem is important but the abstract and introduction sections lacks to highlight the importance of the work. It is suggested to rewrite both section and focus on highlighting your work.

4. Related work section is also brief. Please clearly indicate the limitation of each cited work in this section with respect to the proposed work.

5. The methodology part seems shallow. It needs to be extended including more detailed information on the process and method.

6. The last and important concern is regarding the results. Only two table are provided. It is suggested to add more results by adding more performance metrics. Then we will be able to understand the clear advantage of the proposed work over the existing methods.

 

Comments on the Quality of English Language

Moderate editing of English language required

Author Response

We feel great thanks for your professional review work on our article. As you are concerned, there are several problems that need to be addressed. According to your nice suggestions, we have made extensive corrections to our previous draft, the detailed corrections are listed below.

1. Thank you very much for your advice. We want to explain to you about the innovative points of the paper. Our hierarchical strategy alleviates the problem of large action space of reinforcement learning. In addition, the method we proposed simulates the real scene of clinical practice, and assigns patients to different workers through highlevel policy, thereby reducing the action space and improving training efficiency.
2. Your suggestion is very helpful to us. We have added a basic introduction to MDP, which can help readers to understand the problem formulation into an MDP model.
3. As suggested by the reviewer, we have revised the introduction sections to better emphasize the importance of our research problem. We have taken care to provide a clearer focus on the significance of our work.
4. We have revised and improved the related work section. We have supplemented some related work and highlighted the remaining issues.
5. We have refined and modified the methodology section.
6. As you suggested, We added an experimental result: the learning curves of the KNHRL model and the recurrent KR-DQN model on the synthetic dataset, which respectively show the changes in the success rate for the dataset during the learning process of the two models.

Reviewer 3 Report

Comments and Suggestions for Authors

This paper proposes a Knowledge Enhanced Hierarchical Reinforcement Learning (KNHRL) based Dialogue System for Automatic Disease Diagnosis. The proposed model utilizes deep reinforcement learning and knowledge enhancement to improve the accuracy of disease diagnosis. The model consists of three main components: a dialogue manager, a knowledge base, and a disease diagnosis module. Although the authors claimed that the proposed model has the potential to be applied in various domains such as healthcare, finance, and customer service, there are several aspects need to be improved.

 

1. More quantitative results are expected. I agree that the authors spent long pages on their methodology, but the results from this study are inadequate. Table 1 presents the results of the proposed model and several baseline models on the dataset used for evaluation, while Table 2 presents the results of ablation experiments designed to test the effectiveness of different components of the proposed model. While the paper provides a detailed description of the experimental setup and evaluation metrics used, which helps to contextualize the results presented in the tables, it is important to note that the results are relatively less than expected. With only two tables we may potentially constrain ourselves in a little box to understand this study.

2. What are the limitations in more detail? The authors could provide a more detailed discussion of the limitations of the proposed model, including potential sources of bias and areas for future research.

3. What is the test in real-world settings? The authors could consider testing the proposed model in a real-world setting to assess its practical applicability and potential impact on patient outcomes.

4. In this study, a very important topic is the discussion of ethical considerations. The authors could provide a more detailed discussion of the ethical considerations associated with the use of the proposed model in healthcare settings, including issues related to privacy, informed consent, and potential biases in the data.

Comments on the Quality of English Language

The English of this paper is okay, and additional proofreading is suggested to check for the final manuscript. 

Author Response

We feel great thanks for your professional review work on our article. As you are concerned, there are several problems that need to be addressed. According to your nice suggestions, we have made extensive corrections to our previous draft, the detailed corrections are listed below.

1. Your suggestion is very helpful to us. We have added an experimental result: the learning curves of the KNHRL model and the recurrent KR-DQN model on the synthetic dataset, which respectively show the changes in the success rate for the dataset during the learning process of the two models.
2. As you suggested, we added a more discussion of the limitations of the proposed model.
3. We fully understand your doubt that the simulated data may not fully capture the complexity and variability of real-world disease diagnosis scenarios. But the number of diseases and user goals in the real-world dataset is still limited. Due to the limited availability of real diagnostic datasets, we utilized an artificially synthesized dialogue dataset for disease diagnosis. In the future, we will consider validating the model's performance on real datasets.
4. Thank you very much for your advice. We have added the discussion of ethical considerations in the ethics statement section.

Reviewer 4 Report

Comments and Suggestions for Authors

Disease diagnosis is very important topic but also very risky. Correct diagnosis is crucial for saving people health and life. This paper proposes a knowledge enhanced hierarchical reinforcement learning model KNHRL for this task. 

Authors compared their approach with several other models used in those applications. Proposed model outperformed other approaches. Comaprison was performed on simulated data which is the biggest weakness of this research. 

Structure of proposed approach was clearly presented on a simple Graph.

Model itself presented by Formulas step by step. Low level and high level approach is clearly presented by Figures.

Due to simulated data the usefulness of such model and conclusions from this model is rather limited. 

Would suggest to get at least one example of application on real data.

Comments on the Quality of English Language

minor spellings

Author Response

We feel great thanks for your professional review work on our article. As you are concerned, there are several problems that need to be addressed.

We fully understand your doubt that the simulated data may not fully capture the complexity and variability of real-world disease diagnosis scenarios. But the number of diseases and user goals in the real-world dataset is still limited. Due to the limited availability of real diagnostic datasets, we utilized an artificially synthesized dialogue dataset for disease diagnosis. In the future, we will consider validating the model's performance on real datasets.

Round 2

Reviewer 2 Report

Comments and Suggestions for Authors

The authors have addressed most of my comments in the revised version. The technical background to MDP and RL is still week and I suggest to improve it before final publication. You may refer to article " A gentle Introduction to Reinforcement Learning and Its Applications in Different Fields".

  Comments on the Quality of English Language

Minor editing of English language required

Author Response

We sincerely appreciate the valuable comments. As you suggested, we have improved the technical background to MDP and RL.

Reviewer 4 Report

Comments and Suggestions for Authors

No more comments from my side.

Comments on the Quality of English Language

No comments

Author Response

We sincerely appreciate your valuable comments once again.