Modelling the Trust Value for Human Agents Based on Real-Time Human States in Human-Autonomous Teaming Systems

Round 1

Reviewer 1 Report

 The authors proposed a trust model to estimate human trust during human-autonomous teams' systems. The manuscript addresses a relevant topic, and it is very well-structured. The main limitation is the sample size. The study was developed only with 6 healthy male participants.

Lines 31-32: The authors mentioned: "Previous studies proposed trust-based approaches to explore either human or teammate trust for the optimization of interactions among agents in specified tasks". However, they could consider a recent paper (published in the current year) about this issue: https://doi.org/10.3390/robotics11030059

The discussion section shall be improved, comparing the authors' results with previous studies. And the limitations of the current study have to be discussed, pointing future work.

Author Response

The authors thank the reviewer for the comments. All comments have been addressed accordingly, please refer to the attached response letter.  All changes have been marked in red color in the revised manuscript. 

Author Response File: Author Response.pdf

Reviewer 2 Report

Paper deals with important task. The authors proposed a multievidence human trust model applied an adaptive fusion method based on fuzzy reinforcement learning to fuse multievidence from eye trackers, heart rate monitors and human awareness.

Paper has solid scientific novelty and great practical value.

Suggestions:

1.       It would be good to add clear point-by-point the main contributions at the end of the Introduction section

2.       It would be good to add the remainder of this paper

3.       Fig 1-3, 5-6, 8-9 are very small. Please increase it

4.       It would be good to mension about SGTM Neural-Like Structure and T-Controller in the Section 2.2.2.

5.       The conclusion section should be extended using: 1) numerical results obtained in the paper; 2) limitations of the proposed approach; 3) prospects for future research.

6.       Some of references are outdated. Please fix it using 3-5 years old papers in high-impact journals.

Author Response

The authors thank the reviewer for the comments. All comments have been addressed accordingly, please refer to the attached response letter.  All changes have been marked in red color in the revised manuscript. 

Author Response File: Author Response.pdf

Reviewer 3 Report

The paper presents a trust model that takes as input human cognition signals, such as eye tracking and heart rate data in order to generate trust values. In particular, reinforcement learning was used to calculate the weights and a fuzzy inference system to calculate the rewards. The information is then fused and a human trust value is generated. This generated trust value can be then used by autonomous agents for decision making.

The topic is interesting. However, it could be improved by providing more information about the context in the introduction.  

The paper presents original contribution and also an experimental evaluation of the proposed trust model.  

What the paper is missing is a separate section on related work. Even though the introduction presents some existing work, this is not adequate. A separate related work section is required that discusses the related work and how the proposed work is differs/compares with the related work. The authors may want to discuss about computational trust models, please see, for example, Tjøstheim, T. A., Johansson, B., & Balkenius, C. (2019, September). A computational model of trust-, pupil-, and motivation dynamics. In Proceedings of the 7th International Conference on Human-Agent Interaction (pp. 179-185). Also, the authors mention trust and trustworthiness without defining these and how they are related with each other in the context of their work, please see, for example, Pavlidis, M., Mouratidis, H., Islam, S., & Kearney, P. (2012, May). Dealing with trust and control: a meta-model for trustworthy information systems development. In 2012 Sixth International Conference on Research Challenges in Information Science (RCIS) (pp. 1-9). IEEE.

In terms of the experimental evaluation the authors could discsuss if there were any threat to the validity and how these were mitigated.

In terms of paper presentation, the paper is well written.

Author Response

The authors thank the reviewer for the comments. All comments have been addressed accordingly, please refer to the attached response letter.  All changes have been marked in red color in the revised manuscript. 

Author Response File: Author Response.pdf