Review Reports

Round 1

Reviewer 1 Report

Comments and Suggestions for Authors

The authors propose an original methodology focused on the behavioral processes of HRI in uncontrolled environments by implementing a dual-level observational approach.  The description and basics of the methodology are insightfully presented and correlate with the context of the example analyzed. The goals and objectives of the study are described clearly. The statistical methods used appear to have been chosen truthfully for the sample size.

The authors rightly pointed out the limitations of their study, particularly the lack of research on how these engagement states can be used to inform real-time robot behavior. A second major limitation is the lack of analysis of engagement resulting from repeated encounters.

In conclusion, the article is interesting, making an original contribution to the evaluation of HRI behavior by proposing a structured methodology.

The article is written in careful technical language and contains no factual errors. The subject matter covered is interesting and the authors' contribution is original. I postulate in favor of publishing it in the form it was delivered for review.

Author Response

We would like to sincerely thank the reviewer for the thoughtful and encouraging comments. We greatly appreciate the recognition of the originality and relevance of our dual-level observational approach for evaluating human-robot interaction (HRI) in uncontrolled public environments such as museums.

We are honored by the reviewer’s recommendation for publication without further revision and deeply value the supportive assessment. The constructive and affirmative feedback will guide our continued efforts to refine adaptive HRI systems grounded in real-world behavioral observation.

Thank you once again for your careful review and positive evaluation of our work.

Reviewer 2 Report

Comments and Suggestions for Authors

Review of the paper: “Observation of Human-Robot Interactions at a Science Museum: A Dual-Level Analytical Approach”

This paper presents a dual-level analytical approach to investigating engagement in human-robot interactions (HRI) in a public museum setting. The authors combine (1) statistical analysis of demographic user groups and (2) temporal behavior analysis to understand patterns of engagement.

A key contribution of the paper is the behavior coding scheme to annotating the engagement and the process of creating it. Based on related work and an iterative process in which behaviors were placed in temporal relations using a grammatical structure, a coding scheme was ultimately developed, which included both physical proximities and interaction attempts.

This new coding scheme was applied to a dataset with n=290 visitors of a museum. The main results indicate that children engage more with the robot, than adults. Furthermore, with a temporal analysis (HHM-based) classified three distinct engagement states.

 

The paper addresses an important and underexplored area—qualitative assessment of HRI "in the wild"—and contributes a valuable methodological framework. However, a number of clarifications and revisions would improve the paper’s transparency, reproducibility, and analytical rigor.

 

Major Comments

1. Clarification on Tables 1 and 3: It is unclear how Table 1 maps onto Table 3. Specifically, does Table 1 account for individual vs. group-level interactions (e.g., "touch" performed by one person vs. several)? Was this distinction also captured in the coding scheme?
2. Group-Level observation (Variable): Unfortunately, it's not entirely clear what the relevant variables are. I think they're the same as for individual-level observations, but these are described in Chapter 4.3.1. 
3.  Group-Level observation (Statistical analysis): Furthermore, it is very surprising that the tests for age become significant at the high SD. In addition, a few key details are currently missing, such as:
   1. the N of the individual groups
   2. the effect size of the tests
   3. Brief explanation of why the Mann-Whitney U test was chosen. Since you have several dependent variables that describe the engagement, a MANOVA or a multivariate regression would be more appropriate in my opinion.
4. Individual-level observation: the interpretation of the five HHM states (Figure 4) is not entirely comprehensible, it is not understandable how one gets from 5 states to 3. Here are some suggestions:
   1. How were observation sequences segmented (i.e., how was the time series of behaviors divided for the HMM analysis)?
   2. Why was no distinction made between physical proximity and interaction attempts in the individual-level analysis?
   3. Figure 4: Replace the label code_encode with meaningful names. If the label is taken from page 9, line 333, then this would mean that the people in state LE_low performed the behavior "touch" most often.

 

Minor Comments

• There are some redundant or duplicated statements (e.g., p. 5, lines 138 and 144) that should be edited for conciseness.
• Page 13, lines 430 and 438 both refer to figure 5. Is this a guideline or a process example?
• Group-level: why was it not possible to let people specify their gender and age themselves, since everyone filled out a consent form anyway

 

Author Response

Thank you for your valuable comments. Please see the attachment.

Author Response File: Author Response.pdf

Reviewer 3 Report

Comments and Suggestions for Authors The topic of the paper is interesting. Before acceptance, the paper requires minor changes and additions.

1. Introduction (lines 22-43): no references are given! Therefore, all claims in the first three paragraphs must be covered by relevant references.

2. Introduction (lines 40-41): "Despite these advantages, prior studies in HRI..." What are these studies? I ask the authors to cite relevant references.

3. Introduction (lines 44-49): the motivation and goal are clearly stated and that is fine. I ask the authors to show the structure of the paper in the next paragraph.

4. Related Works: the references are not arranged in the order: 1, 2, 3, 4..., but as: 1, 2, 40, 3... I ask the authors to check all references in the paper, as well as their order when first cited.

5. Related Works: I suggest that authors create a “Summary” within this section by changing lines 107-129 to Summary, while lines 130-136 should be removed, as they do not belong in this section.

6. Table 1 is given within “Related Works”, but it was not previously given in the text, so the question arises why it was given? First, the Table/Figure should be introduced in the text, while at the end of the paragraph, the Table/Figure should be given immediately below.

7. Behavior Coding Scheme (lines 138-148): “two stages” are mentioned at the beginning and end of the paragraph; in my opinion, it is redundant to be mentioned twice.

8. Reference list: check that all references are formed in accordance with the template; also, use standard abbreviations for cited journals.

Author Response

Thank you for your valuable comments. Please see the attachment.

Author Response File: Author Response.pdf

Round 2

Reviewer 2 Report

Comments and Suggestions for Authors

The authors have revised the paper appropriately and have addressed all comments. The paper has significantly improved in quality and can, in my opinion, be published as is.