Robotic Psychology: A PRISMA Systematic Review on Social-Robot-Based Interventions in Psychological Domains

Round 1

Reviewer 1 Report

[ Major ]

The current draft does not qualify as a systematic review due to a lack of depth in the analysis. The paper has a good introduction to the field and a robust inclusion and exclusion criterion. The review, however, is not integrated and condensed and does not provide the authors' unique insight to be added to robotic psychology as a science. That is, the manuscript does not contribute anything significant to the HRI field but is a mere summary of the literature.

The draft (from Page 3) introduces an exhaustive list of counts (e.g., one paper, two papers) that belongs to a category (e.g., studies that involve ASD). This may be a good way to introduce the distribution of interests in the literature. However, an integrated review paper should introduce original frameworks and paradigms based on the literature and identify areas of opportunities for future research.

[ Minor ]

- Both the intro and the selection criterion are well sought out and written.

- Page 3 - 9 is an awkward way to organize the findings. Instead of having an exhaustive list of "regarding the study using A," consider organizing studies by themes. For example, if thematic clusters are "social skills, mood, and well-being" (from the abstract), then consider using them to organize the findings.

- Page 10 (Risk of Bias) is thorough and well written.

- Page 10 (4. Discussion) is poorly written, only introducing a superficial summary of the field.

- Please avoid the use of "and/or" in an academic paper.

Author Response

Reviewer’s comment: The current draft does not qualify as a systematic review due to a lack of depth in the analysis. The paper has a good introduction to the field and a robust inclusion and exclusion criterion. The review, however, is not integrated and condensed and does not provide the authors' unique insight to be added to robotic psychology as a science. That is, the manuscript does not contribute anything significant to the HRI field but is a mere summary of the literature. The draft (from Page 3) introduces an exhaustive list of counts (e.g., one paper, two papers) that belongs to a category (e.g., studies that involve ASD). This may be a good way to introduce the distribution of interests in the literature. However, an integrated review paper should introduce original frameworks and paradigms based on the literature and identify areas of opportunities for future research.

Authors’ answer: Thank you for allowing us to reflect on this matter. In order to answer the critical issues the reviewer highlighted, we revised the paper trying to pointing out our contribution to the HRI field. More specifically, on the one hand, we re-organized the result section putting more emphasis on selected studies target application and frameworks (Lines 462-546), and on the other, we re-wrote the discussion section (Lines 593-607, 620-670) in order to identify areas of opportunities for future research and to present insights and guidelines about the most appropriate technology to be used while targeting a given psychological domain. 

Reviewer’s comment: - Both the intro and the selection criterion are well sought out and written. Page 3 - 9 is an awkward way to organize the findings. Instead of having an exhaustive list of "regarding the study using A," consider organizing studies by themes. For example, if thematic clusters are "social skills, mood, and well-being" (from the abstract), then consider using them to organize the findings. - Page 10 (Risk of Bias) is thorough and well written.

Authors’ answer: Thank you very much for sharing your thoughts about this issue. We actually edited the entire result section organizing the results also based on the target application (ASD, anxiety, etc.) while still maintaining the previous organization of the findings (Lines 462-546). We hope that having this double lens for reading and interpreting the results can be appreciated by the readers as it has been appreciated by some of the reviewers.

Reviewer’s comment: - Page 10 (4. Discussion) is poorly written, only introducing a superficial summary of the field. - Please avoid the use of "and/or" in an academic paper.

Authors’ answer: We totally agreed with the reviewer about the discussion quality. For this reason, and also to answer the reviewer’s previous points, we almost entirely re-thought our discussion section (Lines 593-607, 620-670). Finally, we got rid of "and/or" throughout the paper as suggested. 

Reviewer 2 Report

The paper presents a systematic review of the available literature regarding social robots and robot-based psychotherapy. Although a review can never be complete, the paper very well addresses the set goals.
A conclusion section would be necessary to close the paper.

Author Response

Reviewer’s comment: The paper presents a systematic review of the available literature regarding social robots and robot-based psychotherapy. Although a review can never be complete, the paper very well addresses the set goals. A conclusion section would be necessary to close the paper.

Authors’ answer: Thank you very much for your kind words of appreciation and for taking your time to improve our manuscript. As the reviewer suggested, we added a conclusion section to summarize our findings and properly close the paper (Lines 678-686).

Reviewer 3 Report

The manuscript presents a systematic review to investigate the application of social robots in the domain of psychotherapy. The authors present their methodology and results from the review. The paper is really informative and can be really useful for researchers in this field.

There are some parts that need to be improved:

• I am not sure about the term "Social Robot and Robot-based Psychotherapy" in the title. I think that the references cover a broader application context, e.g., are ASD interventions or social skill training considered to be psychotherapy? In the abstract you mentionr a more suitable term: "Social robot-based interventions in psychological domains.". You should reconsider the use of terms and edit these through the manuscript.
• At the results section, you organize the references based on the robot used. It would be interesting to add a paragraph organizing the results based on the target application, as included at the table (ASD, anxiety, etc.). The current presentation is really informative but the manuscript could include guidelines to select the most appropriate technology based on the requirements. The paper is highly informative due to the references included. However,  some concluding remarks/guidelines or research trends and needs, are needed to justify the purpose of such a review.

Author Response

Reviewer’s comment:  The manuscript presents a systematic review to investigate the application of social robots in the domain of psychotherapy. The authors present their methodology and results from the review. The paper is really informative and can be really useful for researchers in this field. There are some parts that need to be improved: I am not sure about the term "Social Robot and Robot-based Psychotherapy" in the title. I think that the references cover a broader application context, e.g., are ASD interventions or social skill training considered to be psychotherapy? In the abstract you mentionr a more suitable term: "Social robot-based interventions in psychological domains.". You should reconsider the use of terms and edit these through the manuscript.

Authors’ answer: Thank you very much for highlighting this point that actually could have been confusing for the readers. As you recommend we got rid of the term "Social Robot and Robot-based Psychotherapy" in the title and we replaced that with "Social robot-based interventions in psychological domains". We also tried to avoid using the term “psychotherapy” throughout the paper. Nevertheless, we took the chance for making clearer our justification of why we selected Social Robot and Robot-based Psychotherapy as our keywords both for our paper and search terms (Section 2.1, Lines 98-103). 

Reviewer’s comment: At the results section, you organize the references based on the robot used. It would be interesting to add a paragraph organizing the results based on the target application, as included at the table (ASD, anxiety, etc.). The current presentation is really informative but the manuscript could include guidelines to select the most appropriate technology based on the requirements. 

Authors’ answer: Thank you very much for sharing with us this idea that we found brilliant. We actually edited the entire result section organizing the results based on the target application (Lines 462-546). By doing that, we were able to present insights and guidelines 

about the most appropriate technology to be used in each scenario in a more consequential way in the discussion section (Lines 593-607, 620-670). 

Reviewer’s comment: The paper is highly informative due to the references included. However,  some concluding remarks/guidelines or research trends and needs, are needed to justify the purpose of such a review.

Authors’ answer: We agreed on this point and thus we added both a conclusion section to summarize our findings and properly close the paper (Lines 678-686) and few lines about research trends and needs in the discussion section (Lines 593-607).  

Round 2

Reviewer 1 Report

I recognize and appreciate the authors' effort to improve the paper; the newly added paragraph such as "overall studies showed.." (Page 10) is a condensed and integrated summary. The discussion and conclusion are vastly improved. However, this level of integration should be permeated through the whole text. 

3. Results remain the same. The manuscript still includes an exhaustive list of counts. The paper does not add anything significant beyond raw data. The authors could have turned the counts into a graph and use the visuals to support a narrative pointing out the research trend, identifying the gaps, and suggesting an original research framework for the researchers of social robot interventions to pursue.

For example,

"Moreover, two studies did not observe improvements in joint attention [30,35]; on the contrary, one study found that the use of social cues for prompting joint attention with the robot increased the children’s performance [33]."

Instead of reading off the counts of study, the authors should have reported that "studies involving joint attention remains inconclusive." and further discuss what is necessary for HRI research to improve the psychological construct (i.e., joint attention). I suggest a rework of Page 3 - 9.

Author Response

Reviewer’s comment: I recognize and appreciate the authors' effort to improve the paper; the newly added paragraph such as "overall studies showed.." (Page 10) is a condensed and integrated summary. The discussion and conclusion are vastly improved. However, this level of integration should be permeated through the whole text.  3. Results remain the same. The manuscript still includes an exhaustive list of counts. The paper does not add anything significant beyond raw data. The authors could have turned the counts into a graph and use the visuals to support a narrative pointing out the research trend, identifying the gaps, and suggesting an original research framework for the researchers of social robot interventions to pursue. For example, "Moreover, two studies did not observe improvements in joint attention [30,35]; on the contrary, one study found that the use of social cues for prompting joint attention with the robot increased the children’s performance [33]." Instead of reading off the counts of study, the authors should have reported that "studies involving joint attention remains inconclusive." and further discuss what is necessary for HRI research to improve the psychological construct (i.e., joint attention). I suggest a rework of Page 3 - 9.

Authors’ answer: Thank you very much for your words of appreciation about the paper improvements and for giving us the chance to improve it further.  In order to meet the request of keeping the level of integration “stable” through the whole text, we completely re-wrote both the section 3.1 and 3.2. In section 3.1, as the reviewer suggested, we condensed in a graph and few lines of explanation the research trends emerging from the included studies. In 3.2, instead, we discussed findings coming from the selected studies suggesting research frameworks for the researchers of social robot interventions to pursue and what actually we know working and what does not. In the latter case, we tried to provide some reason about why this may happen. Thank you once again for your precious guidance in this editing journey of our paper. We really hope to have accomplished fully what the reviewer asked us. All the bests, the authors. 

Round 3

Reviewer 1 Report

Dear Authors,

I have carefully reviewed the 2nd revision, especially from page 3. The paper has improved and is now publishable, conditional on the following:

- Since Figure 2 and the supporting narrative starts with a quantitative usage in social robots (NAO, Paro, etc.), are there any functional characteristics or differential appearances that may have affected the findings? For example, NAO has been used the most, but why? Aside from being a social robot research platform for a long time, which features of NAO helped social intervention studies? NAO is human-like, whereas PROBO is animal-like. Was PROBO used to study the animal-like features in a social intervention study?

- Page 5: "more session of interaction, as well as larger samples" this is an important insight, so more details would be helpful for the readership. For example, did the studies lack power analysis? How many independent variables (IV) or levels within the IV were there (in general or on average) that would require more N (Number of Participants)? 

- Page 6 - 8 needs more paragraphs and proper use of "indent" for better readability.

- There are a lot of "indeed" (adverb). Please exercise restraint when using excessive meta adverbs. "Actually" should be avoided in an academic paper.

Author Response

Reviewer’s comment: I have carefully reviewed the 2nd revision, especially from page 3. The paper has improved and is now publishable, conditional on the following:

Since Figure 2 and the supporting narrative starts with a quantitative usage in social robots (NAO, Paro, etc.), are there any functional characteristics or differential appearances that may have affected the findings? For example, NAO has been used the most, but why? Aside from being a social robot research platform for a long time, which features of NAO helped social intervention studies? NAO is human-like, whereas PROBO is animal-like. Was PROBO used to study the animal-like features in a social intervention study?

Authors’ answer: Thank you for recognizing our efforts in improving the paper. Section 3.1 has been modified as suggested by including a justification for why NAO has been used the most, especially in the case of children with ASD. We also highlighted which features of PROBO and PARO make these robots suitable for psychological interventions. Unfortunately, for all the robots, we can assume based on theoretical models and approaches which features are more important, but there is still no empirical evidence (e.g., PARO intervention with or without the fur). All the modified parts are green. 

Reviewer’s comment: Page 5: "more session of interaction, as well as larger samples" this is an important insight, so more details would be helpful for the readership. For example, did the studies lack power analysis? How many independent variables (IV) or levels within the IV were there (in general or on average) that would require more N (Number of Participants)?

Authors’ answer: Thank you for highlighting this issue. The statement has been expanded accordingly. The studies to which we referred are severely underpowered due to sample size. In one case we had 5 and 9 children (and two levels on IV), while the other had only 5 participants that participated in all the three conditions. By performing a post hoc power analysis on our part using the information coming from the papers, we observed that even for large effect sizes the power was always under an acceptable threshold (<.80). 

Reviewer’s comment: Page 6 - 8 needs more paragraphs and proper use of "indent" for better readability. There are a lot of "indeed" (adverb). Please exercise restraint when using excessive meta adverbs. "Actually" should be avoided in an academic paper.

Authors’ answer: The manuscript has been modified to increase readability by separating paragraphs with proper use of “indent”. Moreover, we replaced or got rid of the indeed adverb when it was possible. “Actually” was also removed from the manuscript as suggested.