Explainable AI (XAI) Applied in Machine Learning for Pain Modeling: A Review

Round 1

Reviewer 1 Report

1. The quality of figures-1 is poor and the table-1 is not well aligned. Entire Table-1 should be on the same page.

2. The authors have presented a survey titled “Explainable AI (XAI) Applied in Machine Learning for Pain 2 Modelling: A Review ”. There was a recent thorough survey on the application of AI-based methods in healthcare “Reinforcement learning for intelligent healthcare applications: A survey” The authors should clearly compare the two reviews and state clearly what are the new contributions/differences.

3. First, while the basic background information could be useful, what is missing in the present paper is:  how to carry out such an application; that is to say, how to translate and formulate an existing pain-related problem, be it diagnostics, or treatment, or public health -both are missing- in nature, into an XAI-workable computational problem. This is a major piece of the puzzle that is missing in this paper. And yet, it is something that can be most important and useful for researchers, in either computing or healthcare, to know or learn from reading a survey paper.  

To address point-3, the following need to be further provided and explained: 

-When and whether a problem (or a type of problem) at hand is well suited to XAI in general, and certain specific DL variants in particular;

-If so, how to define its constructs.

-How define, learn, and evaluate the reward functions;

-How to determine or learn the right trade-off between exploration and exploitation (e.g., how to empirically and experimentally acquire/refine the parameters involved);

-How to assess and enhance the performance of the DL programs (e.g., how to enhance the learning trackability and verifiability);

-What and how data (as well as, empirical observations and conceptual models) could be incorporated in addressing the above points.

-Second, another major issue with the present paper is in its rather ad hoc and incomplete nature of the contents, in terms of its scope, logical flow and organization, and technical discussions. 

Apart from introducing the basic XAI methodology, it would be more meaningful and insightful to elaborate and discuss systematically the natures and TYPES of problems that each of the DL methods or variants are aimed to handle, and how well, or limited, they are capable, or incapable, of doing so.

-Similarly, in providing and elaborating on the taxonomy of the applications, the organization and discussions should address:  (a) from the domain perspective, what are their shared characteristics, needs, or desired systems behaviors in context; or alternatively, (b) from the AI or problem-solving perspective, what are their common or distinct problem-complexity and the characteristics of AI methods for dealing with such identified complexity.

Third, no methodology or technology is an island. This is especially true in the case of DL.  To make any DL healthcare application work, related technologies will be essential and crucial, which include but are not limited to:  data collection (from device-level to behavioral-level), signal processing, data preprocessing/analytics (e.g., imbalance, incompleteness, heterogeneity, or multi-modality), and time-varying nature, as well as model-based, multi-factor and multi-scale dependency characterization (physiological, environmental, or behavioral), and human-in-the-loop embedding.

-Last but foremost, while it is necessary for a good survey study to systematically address the above-mentioned three aspects, it would be most important to holistically integrate them into one theme, one suggested methodology, i.e., your recommended life-cycle of a Machine Learning application Pain  Modelling, if, say, someone after reading wants to work on his/her own problem at hand: 

What are the steps to follow, after your survey;

Where and how the above-mentioned issues can be addressed; 

When and how he/she would be able to develop and deploy a workable solution.  

While doing so, your current definitions and mentioned examples could then be used as a good source of references.

Author Response

Reviewer #1:

1. The quality of figures-1 is poor and the table-1 is not well aligned. Entire Table-1 should be on the same page.

Answer:

Thanks for the review. The quality of figure-1 is adjusted, Table-1 is well aligned and placed on the same page. Please refer to Page 6 Line 233 for Table-1 and Page 7 Line 276 for Figure-1.

2. The authors have presented a survey titled “Explainable AI (XAI) Applied in Machine Learning for Pain 2 Modelling: A Review ”. There was a recent thorough survey on the application of AI-based methods in healthcare “Reinforcement learning for intelligent healthcare applications: A survey” The authors should clearly compare the two reviews and state clearly what are the new contributions/differences.

Answer:

Thanks for the review. The paper titled “Reinforcement learning for intelligent healthcare applications: A survey” is a survey paper on overall healthcare applications of a specific Machine learning technique called Reinforcement Learning (RL) is explained. As this is the first review to discuss the Pain and explainable AI together. We have referred the paper and included some of the differences and future research to be carried out to bridge the gap between the explainable AI in reinforcement learning.  We have explained this on Page 10 Lines 370-393.

3. First, while the basic background information could be useful, what is missing in the present paper is: how to carry out such an application; that is to say, how to translate and formulate an existing pain-related problem, be it diagnostics, or treatment, or public health -both are missing- in nature, into an XAI-workable computational problem. This is a major piece of the puzzle that is missing in this paper. And yet, it is something that can be most important and useful for researchers, in either computing or healthcare, to know or learn from reading a survey paper.

Answer:

Thanks for the review. This paper collects the information on what is pain and its detection using AI models, how to combine the salient features for each pain type that are of high importance for the researchers and combines a research review that is addressed in diagnostics, treatment and public health. As pain is highly inter-variated and subjective feeling that makes it very difficult for the end user of the Artificial Intelligent model to know how the model end with the decision, explainable AI models necessity to give an idea for the research scope in this area. Future reviews can address the XAI models that is workable with the pain related problem by referring this review paper.

To address point-3, the following need to be further provided and explained: 

 -When and whether a problem (or a type of problem) at hand is well suited to XAI in general, and certain specific DL variants in particular

-If so, how to define its constructs.

-How define, learn, and evaluate the reward functions;

-How to determine or learn the right trade-off between exploration and exploitation (e.g., how to empirically and experimentally acquire/refine the parameters involved);

-How to assess and enhance the performance of the DL programs (e.g., how to enhance the learning trackability and verifiability);

-What and how data (as well as, empirical observations and conceptual models) could be incorporated in addressing the above points.

Answer:

Thanks for the review. The above questions are explained in the paper as the Importance of XAI for specifically to pain in Page 5 from Lines 203-231. Different DL models are mentioned for its performance with the features in Page 6 in Table-1.

-Second, another major issue with the present paper is in its rather ad hoc and incomplete nature of the contents, in terms of its scope, logical flow and organization, and technical discussions. 

 Answer:

Thanks for the review. To discuss this, the nature of Pain is explained and AI in particular related to these types of pain with its workflow to XAI models, the scope and importance is discussed.

Apart from introducing the basic XAI methodology, it would be more meaningful and insightful to elaborate and discuss systematically the natures and TYPES of problems that each of the DL methods or variants are aimed to handle, and how well, or limited, they are capable, or incapable, of doing so.

 Answer:

Thanks for the review. This question was discussed along with the available DL methods in Page 10 Lines 395-429

-Similarly, in providing and elaborating on the taxonomy of the applications, the organization and discussions should address:  (a) from the domain perspective, what are their shared characteristics, needs, or desired systems behaviors in context; or alternatively, (b) from the AI or problem-solving perspective, what are their common or distinct problem-complexity and the characteristics of AI methods for dealing with such identified complexity.

  Answer:

Thanks for the review. This question was discussed along with the available DL methods in domain perspective. The data available for a DL model is limited in pain. The models are very hard to understand the inner feature meaning for many problems. Thus it is focused in this paper. To how can any machine learning research can be worked in pain.

Third, no methodology or technology is an island. This is especially true in the case of DL.  To make any DL healthcare application work, related technologies will be essential and crucial, which include but are not limited to:  data collection (from device-level to behavioral-level), signal processing, data preprocessing/analytics (e.g., imbalance, incompleteness, heterogeneity, or multi-modality), and time-varying nature, as well as model-based, multi-factor and multi-scale dependency characterization (physiological, environmental, or behavioral), and human-in-the-loop embedding.

Answer:

Thanks for the review. There is work related to pain evaluation using the real-time images which works with higher accuracy but the output is not a valid result as we need a working model in AI that is not biased, trustful in terms of the decision making.

-Last but foremost, while it is necessary for a good survey study to systematically address the above-mentioned three aspects, it would be most important to holistically integrate them into one theme, one suggested methodology, i.e., your recommended life-cycle of a Machine Learning application Pain  Modelling, if, say, someone after reading wants to work on his/her own problem at hand: 

 Answer:

Thanks for the review. The work with pain related models and its decision making ability to clearly explain the result of how it end with that result is unclear with many machine learning algorithms. So if someone after reading this paper can work with some reinforcement learning methods to check the effectiveness of the model for pain problem.

What are the steps to follow, after your survey;

Where and how the above-mentioned issues can be addressed; 

When and how he/she would be able to develop and deploy a workable solution.  

 Answer:

Thanks for the review. The work with pain related models and its decision making ability to clearly explain the result of how it end with that result is unclear with many machine learning algorithms. So if someone after reading this paper can work with some reinforcement learning methods to check the effectiveness of the model for pain problem. Thus the researcher can get the idea of a working solution.

While doing so, your current definitions and mentioned examples could then be used as a good source of references.

Author Response File: Author Response.docx

Reviewer 2 Report

I am really grateful for reviewing this manuscript. In my opinion, this manuscript can be published once some revision is done successfully. I would like to suggest the authors to list the explainable features in terms of variable importance in Table 1. This is expected to help the readers to draw effective clinical/policy implications. 

Author Response

Reviewer #2:

I am really grateful for reviewing this manuscript. In my opinion, this manuscript can be published once some revision is done successfully. I would like to suggest the authors to list the explainable features in terms of variable importance in Table 1. This is expected to help the readers to draw effective clinical/policy implications.

Answer: 

Thank you for your review. We have explained the variable importance of the explainable features of Table-1 that relates with the different pain types in Table-4 in Page 10.

Author Response File: Author Response.docx

Round 2

Reviewer 1 Report

The authors have address my comments.