Skip to Content
You are currently on the new version of our website. Access the old version .
Applied SciencesApplied Sciences
  • Article
  • Open Access

6 August 2025

30 Pages

Are Robots More Engaging When They Respond to Joint Attention? Findings from a Turn-Taking Game with a Social Robot

,
,
and
Department of Systems Engineering and Automation, University Carlos III of Madrid, Av. de la Universidad, 30, 28911 Leganes, Madrid, Spain
*
Author to whom correspondence should be addressed.
This article belongs to the Special Issue Emerging Technologies for Assistive Robotics

Abstract

Joint attention, the capacity of two or more individuals to focus on a common event simultaneously, is fundamental to human–human interaction, enabling effective communication. When considering the field of social robotics, emulating this capability might be necessary for promoting natural interactions and thus improving user engagement. Responding to joint attention (RJA), defined as the ability to react to external attentional cues by aligning focus with another individual, plays a critical role in promoting mutual understanding. This study examines how RJA impacts user engagement during human–robot interaction. The participants play a turn-taking game against a social robot under two conditions: with our RJA system active and with the system inactive. Auditory and visual stimuli are introduced to simulate real-world dynamics, testing the robot’s ability to detect and follow the user’s focus of attention. We use a twofold approach to evaluate the system’s impact on the user’s experience during the interaction. On the one hand, we use head pose telemetry to quantify attentional aspects of engagement, including measures of distraction and focus during the interaction. On the other hand, we use a post-experimental questionnaire incorporating the User Engagement Scale Short Form to assess engagement. The results regarding telemetry data reveal reduced distraction and improved attentional consistency, highlighting the system’s ability to maintain attention on the current task effectively. Furthermore, the questionnaire responses show that RJA significantly enhances self-reported engagement when the system is active. We believe these findings confirm the value of attentional mechanisms in promoting engaging human–robot interactions.

1. Introduction

Attention is a core biological and cognitive mechanism that allows organisms to coordinate their actions by selectively prioritizing relevant sensory information across different modalities. This information includes but is not restricted to peripheral visual and auditory stimuli, allowing individuals to analyze and respond to their environment [1]. Individuals generate, maintain, and coordinate their intentions through attention and alignment on shared goals or topics to achieve mutual understanding. This alignment considers detecting and interpreting non-verbal cues, such as gaze direction, manual pointing, facial expressions, or verbal signals [2].
Joint attention (JA) emerges as a distinctive form of attention coordination, defined as when two or more individuals deliberately focus on a common event, object, or experience, which is usually referred to as the focus of attention (FoA) [3]. Several mechanisms are considered within the frameworks of JA processes, often performed by following the partner’s gaze, pointing gestures, or verbal cues, allowing individuals to set up a common frame of reference. As stated by Flom et al. [4], gaze following implies an ability to detect the direction of a partner’s gaze and, in response, to orient one’s own eyes and head accordingly to infer their intentions. A similar process occurs with pointing gestures, which also serve to direct attention. These mechanisms play an essential role during development: for example, as children acquire JA capabilities, they begin to develop language skills, interpret social signals, and effectively navigate their environment alongside their caregivers [5].
JA has been classified in various ways throughout the literature, reflecting its multifaceted nature and role in social cognition. Moore et al. [6] classify JA into three levels and two types, reflecting increasing complexity and social coordination. The levels are described as follows: (i) Dyadic representation implies a direct eye-to-eye look between two individuals that is necessary for social contact but does not share an external focus. (ii) Shared gaze is achieved when two individuals coordinate their attention with respect to some third-element object and a shared reference point is established. In this case, the individuals are not expected to notice the other’s intentionality regarding the shared reference point. Finally, (iii) triadic representation is the most complex level, where individuals focus on the same object and mutually acknowledge each other’s attention. Referring to the types, the literature divides JA into responding to joint attention (RJA) and initiating joint attention (IJA) [7,8,9]. RJA refers to the ability of an individual to recognize and follow an object or event referred to by another person. This means responding to certain cues. For example, when one individual looks at an object and/or points to it, the viewer directs his or her attention in the indicated direction. IJA is the capacity to turn another person’s attention to a specific object or event. Continuing with the previous example, one would point to a far object and look simultaneously at his or her interaction partner to ensure that he or she also noticed it. RJA and IJA are the building blocks of effective communication, allowing an individual to infer the perspective, intention, and subjective state of another person, which is usually understood as intersubjective understanding [10].
One way to maintain JA and make it successful, whether in a shared object-focused activity or a collaborative action, would be that the individuals involved are engaged during the interaction [11]. Engagement goes beyond just paying attention and involves how much participants are mentally, emotionally, and physically involved in the activity. Generally speaking, high engagement is desired since focused work, collaboration, and meaningful exchange only ensue with sufficiently high levels, especially in dynamic and long-lasting activities [12]. Engagement plays a significant role in guaranteeing naturalness and effectiveness during human–robot interaction (HRI). Robots maintaining user engagement give the impression of being competent, relate to them on a social scale, and possess high degrees of social competence [13]. These situations are relevant for applications including education, therapies, and entertainment, whose impacts depend directly on the sustained engagement a user expresses [14]. Similarly, in the field of social robotics, where the purpose of these robots is to establish relationships and coexist with humans [15], JA is of interest as a capacity to be considered to allow the robots to respond correctly to user cues, not only promoting natural interactions but also generating trust and acceptance by the users [16]. RJA, in particular, allows robots to react in real time to user behaviors, demonstrating awareness and adaptability. However, the effects of JA capabilities in a social robot in terms of engagement remain unexplored.
Most existing research on JA has focused on implementing specific attentional mechanisms in robots, such as gaze following [17], imitation [18], or turn-taking [19], demonstrating the technical advances and their integration. Nevertheless, while most of these studies have successfully replicated this capability, analyzing the user perception of the robots regarding JA and engagement is still an area of interest. Most importantly, this will be necessary for understanding effectiveness beyond technical functionality. Bridging the gap between replicating attentional mechanisms and assessing their effects on user experience could provide valuable insights into how to create more engaging and socially competent robots.
In a previous study, we presented a biologically inspired responsive joint attention system (RJAS) that enables a social robot to perform RJA by recognizing and following user attentional cues such as gaze and gestures [20]. Subsequent evaluations of this system revealed that, when active, it enhanced the user’s perception of the robot’s social presence, as well as its perceived competence and warmth [21]. Building on these results, the present work shifts the focus towards user engagement. Although the experimental framework remains consistent with our previous studies—centered on the “Odds and Evens” turn-taking game and using the same two experimental conditions (RJAS Active vs. RJAS Inactive)—the current study introduces a novel analytical perspective.
This work draws from principles in developmental and cognitive psychology related to joint attention, implementing them in a social robot designed to detect and respond to user social cues such as gaze direction and pointing gestures. We expect these capabilities to enable the robot to behave in a socially attentive manner, potentially shaping how it is perceived during interaction. The main contribution of this work is to examine whether this socially responsive behavior affects user engagement, specifically if users stay attentive, avoid distraction, and remain involved in the interaction during a collaborative task.
To do so, we assess engagement through a dual approach: from a questionnaire, using the User Engagement Scale Short Form (UES-SF) [22]; and using a telemetry system, referred to in this work as the user behavior tracking (UBT) system. The described telemetry system monitored head pose orientation to indicate user distractions and attentional focus during the interaction. This combination of self-reports and behavioral data is still relatively uncommon in HRI studies, where engagement is often assessed through either questionnaires or manual observational grids. However, both approaches present limitations: self-reports suffer from time-discrepancy issues, and observational methods can be affected by annotator subjectivity [23]. By integrating self-report data with automatic telemetry-based perception, our study offers a more complete and time-sensitive perspective on how JA behaviors affect user engagement throughout the interaction.
The rest of this paper is organized as follows. Section 2 reviews the related works on JA, engagement, and their applications in social robotics. Section 3 describes the functioning of the system. Section 4 details the experimental setup. Section 5 presents the evaluation metrics and study procedure. Section 6 presents the results, analyzing both self-reported and behavioral measures of engagement. Section 7 discusses the implications of our findings and highlights limitations and future work. Finally, Section 8 concludes the study by summarizing the key contributions and outlining potential research directions.

3. Responsive Joint Attention System

The RJAS [20] architecture is designed to incorporate JA capabilities into social robotic platforms to increase their naturalness and responsiveness by detecting user social cues. The RJAS was implemented in a social robot, allowing it to respond to user attention dynamically. Three interrelated modules compose the RJAS (see Figure 1): the User Feature Detectors (UFDs), the Visual Attention Module (VAM), and the Bio-inspired Behavioral Module (BBM). The general pipeline of the system allows the robot to capture user features, prioritize the stimuli, and generate both physical and verbal responses.
Figure 1. Overview of the RJAS. The system considers detector stimuli input and applies weights to compute the FoA, which the motion controller uses to reorient the robot and perform verbal expressions.
Discerning the architecture in the perceptual (UFD and VAM) and actuation parts (BBM): The UFD module is responsible for gathering visual sensory stimuli data and identifying user features relevant to attention coordination, like the face for maintaining eye contact, head orientation for gaze-following, and arm position for detecting manual pointing and hand gestures. For that purpose, the following pre-trained detectors are implemented, selected for their compatibility with embedded AI accelerators commonly used in robotics platforms:
  • Face Detector: Locates the user’s face and calculates its position in real-world coordinates (highlighted in a green bounding box). Implemented using the OpenVINO model face-detection-retail-0004 https://github.com/openvinotoolkit/open_model_zoo/blob/master/models/intel/face-detection-retail-0004/ (accessed on 30 July 2025).
  • Head Pose Detector: Uses the red bounding box of the detected head to estimate the user’s head orientation to determine their gaze direction (represented with a white arrow). The orientation vector is discretized into one of nine possible directions: frontal, left, right, up, down, up–left, up–right, down–left, and down–right. Implemented using the OpenVINO model head-pose-estimation-adas-0001 https://github.com/openvinotoolkit/open_model_zoo/blob/master/models/intel/head-pose-estimation-adas-0001/ (accessed on 30 July 2025).
  • Body Pose Detector: Tracks the user’s body pose by computing several landmarks that compose the user’s skeleton. The coordinates of the arms are analyzed to determine their direction, which is also discretized using the same nine-direction scheme as the head pose (body landmarks are highlighted with green arrows and red dots). Based on the lightweight PoseNet architecture https://github.com/google-coral/project-posenet (accessed on 30 July 2025).
  • Hand Gesture Detector: Recognizes the orientation and movements of the user’s fingers, enabling the robot to interpret manual signals, such as gestures (the hand’s finger landmarks are currently highlighted in the figure). Implemented using MediaPipe Hands https://ai.google.dev/edge/mediapipe/solutions/vision/hand_landmarker (accessed on 30 July 2025).
The VAM dynamically assigns intensity values to each detected stimulus, enabling the system to prioritize stimuli based on the interaction context. The Stimuli Intensity Manager operates on a rule-based system that assigns higher intensity values to the user’s gaze and arm signaling direction than to the user’s face. This prioritization reflects the system’s focus on attending to external cues that indicate the user’s FoA shifts. Intensity values, ranging from 0 to 1, are defined empirically based on human attentional behavior and dynamically adjusted. For instance, primary JA cues such as gaze direction and arm pointing are assigned a value of 1 when the user is not facing the robot, or when they are pointing with their arms, while the user’s face maintains a baseline intensity of 0.5, acting as a default attentional anchor. In specific phases involving hand gestures, the intensity of hand cues is increased to 0.8 to reflect their relevance during the activity. This rule-based assignment was inspired by how humans naturally prioritize different types of attention-related stimuli [51]. When one of these stimuli is detected, the Stimuli Ranking Module ranks the stimuli and computes the final FoA according to their intensity to direct the robot’s attention to the highest-priority stimulus (e.g., the user’s head orientation direction or manual pointing). After the robot attends to the detected stimulus, the bio-inspired Inhibition of Return (IOR) [52] mechanism implemented decreases the intensity of that stimulus, reprioritizing the user’s face as the main focus. This process allows the robot to alternate naturally between following the user’s attention and reestablishing eye contact, ensuring the natural JA closed-cycle interaction.
Regarding actuation, the BBM translates the VAM’s output into physical and verbal behaviors, imitating natural human attentional responses. The human vestibular system [53] inspires the motion controller, allowing the robot to perform coordinated and fluid movements of its eyes, head, and torso (see Figure 2). The robot’s movements are based on standard techniques, such as inverse kinematics for joint control and a PID controller for adjusting the base during user tracking. The bio-inspired aspect lies in how the robot coordinates its eyes, head, and torso, mimicking the oculocephalic reflex and compensatory head movements. This enables human-like attention shifts through sequential activation of its degrees of freedom. For example, when a user’s gaze shifts, the robot adjusts its eyes, head, and torso if necessary. In case the FoA is within the robot’s field of view, it will only move its eyes toward the FoA (see Figure 2a). If the eyes reach their limits, the robot will turn its head toward the direction of the FoA while completing a counter-movement with the eyes to maintain the robot’s gaze centered (see Figure 2b). Finally, the robot reorients the body towards the FoA, with the head readjusting to ensure it remains properly oriented. (see Figure 2c). If the FoA moves and leaves the robot’s field of view, it will be followed by its torso camera until it reaches its maximum position or the center of the FoA.
Figure 2. BBM controller phases imitating the vestibular system mechanism. (a) The robot moves its eyes toward the FoA if it is within its field of view. (b) It turns its head toward the direction of the FoA while conducting a counter-movement with the eyes. (c) Finally, the robot reorients the body towards the FoA, and the head adjusts accordingly to maintain alignment.
In addition to the movement, the Attention Verbal Expression Manager handles the robot’s verbal responses related to JA events. It is designed to acknowledge the user’s distractions and redirect their attention to the ongoing task. The expressions employed by the robot are thought to avoid open-ended questions or responses that might shift the user’s focus away from the activity. Table 1 gathers the phrase list, uttered in a neutral tone, ensuring the user perceives the robot as attentive without disrupting the interaction flow. Those are randomly selected from a predefined set to enhance naturalness, avoiding repetition to prevent predictability. This approach reinforces the robot’s awareness of the user’s behavior while ensuring the interaction remains centered on the primary activity to maintain user engagement.
Table 1. Verbal phrases the robot uses to acknowledge user distractions during the interaction.
Our RJAS system extends prior implementations by integrating multiple user cues, such as head orientation and arm gestures, into a unified attentional framework. It incorporates an attentional layer for dynamic stimulus prioritization, and an inhibition mechanism based on IOR to reduce the relevance of previously attended stimuli over time. The complete pipeline runs in real time, combining detection, attention regulation, and robot actuation through a vestibular-inspired motion controller and a verbal expressivity layer. Except for the low-level controller, the system is hardware-agnostic and has been designed for proper integration across platforms [20].

4. Experimental Setup

This section describes all the elements that form part of our experimental setup. First, we provide a description of the robotic platform employed during the user study. Second, we explain the telemetry system that enables passive analysis of user engagement. Finally, we present the interactive game integrated into the HRI scenario, along with the distractor system designed to capture the user’s attention.

4.1. Social Robot Mini

Mini is a desktop social robot designed for close face-to-face interactions, developed at the RoboticsLab of the University Carlos III of Madrid [54]. Mini features a friendly appearance, as shown in Figure 2, with a design intended for HRI. Its physical structure includes five degrees of freedom distributed across its head, neck, torso, and arms, enabling it to perform natural and dynamic gestures. Expressive LEDs are embedded in its cheeks, heart, and mouth to emulate effective human expressions and interaction and enhance its social presence. To perceive the user and its environment, Mini is equipped with an Intel RealSense D435i RGB-D https://www.intelrealsense.com/depth-camera-d435i/ (accessed on 30 July 2025), which captures color and depth information. Additionally, sensory capabilities include capacitive touch sensors on its torso and arms to detect physical interactions and microphones in its head and body to capture ambient sound and the user’s voice. Two internal speakers placed in Mini’s head are used for verbal communication, and two screens in its eyes allow Mini to emulate eye blinking and gaze. Moreover, Mini has an auxiliary tablet that can play multimedia content and acts as an additional way to communicate with the robot. To reduce CPU usage during computational tasks, Mini integrates AI accelerators such as the Intel Movidius Neural Compute Stick 2 (NCS2) https://www.intel.la/content/www/xl/es/products/sku/140109/intel-neural-compute-stick-2/specifications.html (accessed on 30 July 2025) and Google Coral TPU https://coral.ai/products/accelerator (accessed on 30 July 2025), allowing it to perform real-time operations. Specifically, the head pose estimation model runs on the NCS2, while the body pose estimation is delegated to the Coral TPU. The face detector and the hand detector run on CPU.
Mini’s software architecture is built upon the Robot Operating System (ROS) [55] and is composed of a set of modular components that collectively support perception, decision-making, and actuation during human–robot interaction. The system starts by acquiring sensory data through the robot’s RGB-D camera, which feeds into a series of visual detectors responsible for extracting user-related features such as head position, orientation, and hand gestures. These features are processed by the Perception Manager, which standardizes the information and transmits it to the RJAS. The RJAS prioritizes the most salient stimuli based on these cues, estimates the user’s FoA, and triggers a responsive behavior that includes coordinated eye, head, and torso movements to reorient the robot towards that FoA. This attentional information is also sent to the human–robot Interaction Manager, which coordinates verbal responses and expressive gestures, and to the Decision-Making System, which selects the appropriate skill to execute based on the current interaction state. The robot has multiple skills that consist of interactive games and behaviors that enhance engagement and stimulate the user both cognitively and physically. The selected behavior is then carried out by the Actuation module, which manages the robot’s motor actions and expressive outputs. For a more detailed explanation of the integration of the RJAS within Mini’s architecture, see [20].
Regarding the real-time performance of the RJAS pipeline, we measured the CPU usage of the perceptual stack when executed on the robot’s on-board processor (Intel i7-1165G7) without graphical display output and with one user in front of the robot. Average CPU loads for each detector were as follows: face detector (4%), head pose detector (4.5%), body pose detector (3.5%), and hand gesture detector (7%). The RGB-D camera acquisition process consumed approximately 2%. The VAM and BBM, which are responsible for stimulus prioritization and motor control, respectively, together accounted for less than 2% of the CPU load. These values suggest that RJAS is lightweight enough to operate in real time within a tabletop scenario using a single-user setup, even on moderate consumer-grade CPUs.

4.2. User Behavior Tracking

The UBT system was developed to measure user engagement during the interaction and analyze user attention and distraction time. This system aimed to complement self-reported engagement measures by providing a detailed dataset on attentional dynamics during the HRI to understand how the robot’s responsive behavior influenced user attention and engagement.
The logs contained the following variables related to user behavior. For each one, the timestamp of the interaction was recorded:
  • Head Pose Orientation: To determine the attentional focus of the user, the direction of their head was recorded at each sampling interval to identify whether the user was looking at either the robot or the game, or elsewhere, for example, toward a distractor.
  • Distractor Activation Time: Events were timestamped according to the onset of visual or auditory distractors.
  • Type of Distractor: Each distractor was categorized as visual-animated images on a side monitor or audio-recorded sounds played through speakers.
The UBT sampling rate was set to 0.16 s (due to the camera’s 6 FPS setting) as a compromise between accurate head orientation tracking and the need to maintain smooth performance for both the system and the robot. From the raw telemetry data, several measures were derived to characterize the user’s interaction dynamics:
  • Total Number of Distractions ( N d ): A distraction is defined as when the participant’s head orientation shifted away from the robot or the game area toward a distractor and remained in that position for at least a minimum duration. Consecutive samples with distracted orientation were treated as part of the same distraction event.
  • Total Distraction Time ( T d ): Also derived from the head pose orientation data and timestamps, the cumulative time the user’s head direction indicated distraction (e.g., not looking at the robot).
  • Total Interaction Time ( T i ): The duration from the first timestamp that marks the start of the game to the final timestamp when the robot concluded the interaction.
To determine whether the user was distracted, we set a minimum duration threshold of 1.5 s for their head to remain oriented away from the robot or the game. This value was defined empirically, considering the average human reaction time to stimuli (between 250–500 ms) [56], and accounting for Hick’s Law [57], which states that reaction time increases with the number and complexity of stimuli. In our case, the interaction included concurrent multimodal elements, such as the robot’s movements and speech, the tablet interface, and external visual or auditory distractors, which justified a longer window. Moreover, the threshold helps to avoid false positives from brief saccadic eye or head movements, which typically last under 200 ms [58], ensuring that only sustained attentional shifts are classified as genuine distractions.
Distraction time is computed cumulatively over all time steps in which the head pose classifier does not return the “frontal” label. This label is assigned when the estimated yaw angle lies within ±15° and the pitch angle within ±10°, corresponding to the user roughly facing forward. Any orientation falling outside these angular limits is interpreted as an attentional shift and is included in the distraction time calculation.

4.3. Interaction Environment

To evaluate the impact of the RJAS on user engagement, we designed an interactive environment where the participants played the game “Odds and Evens” with the robot Mini. This game has been used because of its simplicity and suitability for maintaining user engagement without demanding excessive cognitive effort from the user. Its intuitive rules make it very unobtrusive: the user can focus on the interaction with the robot rather than fiddle with complicated mechanics, which makes an optimal scenario for RJAS evaluation.
The game has distinct steps (see Figure 3), starting with the robot presenting itself and explaining the rules. This is followed by a calibration phase, where the user is prompted to display different number combinations with their fingers, helping them to learn how to correctly present their hands to the robot, which is then able to recognize the gestures accurately. During the selection and reminder phase, the robot either randomly invites the user to choose whether to play as “odds” or “evens” or makes the choice itself and then provides a brief reminder of the rules. In each round, the robot actively joins in the interaction by performing a countdown, showing its choice on the tablet screen, and asking the user to display their hands. If the user does not show their hands on time or if there are problems in detection, the robot restarts the round. Otherwise, it determines the outcome and checks the winner. This loop runs five times per match for two matches. Finally, the robot concludes the game with a farewell.
Figure 3. Workflow of the gaming experience. The green boxes represent the common game stages in the scenario. Distinctive icons represent the different stimuli (audio and visual). This figure is similar to the experimental setup previously mentioned in a related study [21].
The auditory and visual stimuli were triggered at various game stages, such as the selection and reminder, countdown, and check-winner phases, to make scenarios similar to real life and test the robot’s ability to respond to the user’s distractions. These stimuli were thought to temporarily catch the user’s attention to challenge RJAS in re-engaging the user effectively. To ensure replicability while reducing the risk of learning effects, distractors were presented at predefined non-consecutive rounds across the two gameplay sessions. Their timing was deliberately spread across different game phases to minimize predictability and avoid repetitive sequences.
The auditory stimuli comprised sounds like drum beats, which lasted for 2 s, bells lasting 5 s, glass breaking lasting for 1 s and repeated twice, and an alarm clock sounding for 5 s. Visual stimuli, presented as animated GIFs, included a moving vehicle and a cartoon chase scene. These animations appeared within the user’s peripheral vision on a monitor and remained on until the round was over. Both types of stimuli were selected according to the normative literature about human attention capture. Typically, auditory stimuli are more effective than visual ones [59]. The duration and timing of the stimuli were empirically adjusted to avoid overloading the user while allowing them to stay focused on the game.

5. User Study

This section outlines the methodology used to evaluate the impact of the RJAS, integrated into the social robot Mini, on user engagement during a game-based interaction. We conducted a between-subjects user study with two experimental conditions defined by the state of the RJAS:
  • RJAS Active: Volunteers play the “Odds and Evens” game with the robot while the RJAS is active. During the interaction, a series of visual and auditory distractors are introduced in order to shift the participant’s attention away from the task. When the user seems distracted, the robot first turns towards the direction of the user’s gaze, then turns back to the user and verbally acknowledges the distraction.
  • RJAS Inactive: The same game is played, but the robot does not respond to user distractions. Its behavior remains static in terms of attention, offering no gaze reorientation or verbal cues in response to shifts in user focus.
The core hypothesis is that the participants in the RJAS Active condition will show higher engagement levels than those in the RJAS Inactive condition. To assess this, we rely on self-reported measures via post-interaction questionnaires and objective data obtained through the UBT, a telemetry system that captures attentional behavior during the task.

5.1. Participants

A total of 40 participants took part in the experiment, including 26 women and 14 men. Demographic information (age and gender) was collected at the start of the session. The age distribution was as follows: 19 participants were aged 18–24, 11 were 25–34, 4 were 35–44, 5 were 45–54, and 1 participant was aged 55–64.
The participants were randomly assigned to one of the two experimental conditions (RJAS Active or RJAS Inactive). Among them, 18 participants had previously interacted with the robot Mini, while 22 had not. Table 2 shows the distribution of participants across gender, age groups, and prior experience with the robot, separated by experimental condition. We believe the sample was well balanced across these demographic variables, with a comparable number of participants in each condition and no major disparities in background characteristics that could systematically bias the results. As a result of the assignment, both conditions included 20 participants.
Table 2. Distribution of participants by gender, age group, and experimental condition. Values are reported as count (percentage of total); values in brackets indicate how many had prior interaction with the robot Mini.
The University Carlos III of Madrid approved this experiment (under the name PI_perception_JA_1) It was performed under the Spanish Organic Law 3/2018 of December 5th on the Protection of Personal Data and Guarantee of Digital Rights (LOPDGDD) and the European General Data Protection Regulation (Regulation (EU) 2016/679, abbreviated GDPR), and informed consent was obtained from all participants and/or their legal guardians.

5.2. Procedure

The experiment took place in a controlled environment, with the robot Mini positioned on a table next to its tablet. To avoid false positives, the tablet was placed in front of it so it would not be misinterpreted as a distraction. The tablet also allows the participants to follow game-related instructions naturally. Distractions were provided via two speakers placed behind an opaque screen positioned behind the robot, ensuring that the source of the sounds was not visually accessible. In addition, a monitor was placed on the right side of the table to display visual stimuli in the form of short animations. This setup, illustrated in Figure 4, was designed to simulate real-world distractions by using contextually irrelevant but attention-grabbing stimuli.
Figure 4. Layout of the experimental environment, including the robot Mini with its auxiliary tablet, and the distractor sources. Auditory distractions were played from speakers placed behind an opaque screen, and the visual stimuli were shown on a monitor positioned to the right of the user.
The session began with the experimenter providing the participant with a general briefing about the experiment, without disclosing specific details that could bias the results. Following the introduction, the participant was seated and asked to complete a data protection form containing basic demographic information and a unique identifier for data pseudonymization. After reviewing and signing the informed consent form, the participants were invited to begin the interaction with Mini. At that point, the researcher left the space to allow the interaction to proceed autonomously. The interaction consisted of the “Odds and Evens” game, which comprised two matches of best-of-five rounds. During the game, distractors were introduced at predetermined points in the match sequence to simulate interruptions, as explained in Section 4.3 and shown in Figure 3. After completing the full session, the participants were asked to fill out a post-experiment questionnaire to assess their interaction engagement. In addition to the questionnaire, the participants were also asked in person to comment on their overall experience during the experiment. They were encouraged to provide constructive feedback (to minimize the impact of the participant’s politeness), and their responses were noted down for further qualitative analysis.
A video of a full session, including two matches and the presentation of all distractors under the RJAS Active condition, was recorded for demonstration purposes. Video of a user with the RJAS Active condition: https://youtu.be/LN4IIWeibWc (accessed on 30 July 2025).

5.3. Measures

We adopted a mixed-methods approach to assess user engagement, combining self-report data and behavioral tracking measurements. This combination allows for a more comprehensive understanding of how the RJAS influences the interaction, capturing participants’ perceived experience and attentional responses during the task. The following subsections describe the tools and metrics used to evaluate engagement in the study.

5.3.1. User Behavioral Tracking Metrics

The computation of these engagement metrics was based on the data provided by the UBT system described in Section 4.2. This system allowed us to measure participants’ attentional behavior throughout the interaction by gathering data from the head pose orientation, distraction events, and overall interaction time. We derived two engagement-related metrics from these data: distraction frequency and distraction duration ratio.
Recent literature emphasizes attention as a core cognitive component of engagement in HRI. As noted by Sorrentino et al. [23], the cognitive facet of engagement comprises conscious processes such as effort, investment, and attention, with attention being the component most frequently associated with engagement. Following this idea, we interpret both the frequency of distraction events and the ratio of time spent distracted as indicators of attentional disengagement. We believe these metrics will allow us to evaluate changes in user attention throughout the interaction, thus complementing self-reported assessments with behavioral data.
The first metric is the distraction frequency ( f d ). This metric reflects how often the participants disengaged from the task. As Equation (1) shows, it is obtained as the number of distraction events divided by the total interaction time:
f d = N d T i
where N d is the total number of distractions and T i is the total interaction time (in seconds). Then, we also defined the distraction duration ratio ( ρ d ). This metric quantifies the proportion of the session during which the participants were not paying attention to the robot or the game. It is computed as the total distraction time divided by the total interaction time, as shown by Equation (2):
ρ d = T d T i
where T d is the total time spent distracted (in seconds), and T i , is again the total interaction time (also in seconds).

5.3.2. Self-Reported Metrics—Post-Experimental Questionnaire

Regarding the self-reported measure metrics, we used the UES-SF to assess user engagement. The UES-SF has demonstrated sufficient reliability and validity and is commonly applied in digital interaction studies [60,61,62]. The questionnaire comprises 12 items rated on a 5-point Likert scale, ranging from 1 (strongly disagree) to 5 (strongly agree). These items are evenly distributed across four dimensions: Focused Attention, Perceived Usability, Aesthetic Appeal, and Reward Factor, with three items per category. Focused Attention refers to the degree to which users become absorbed in the interaction, often losing awareness of time or their surroundings. It reflects cognitive immersion and deep concentration during the task. Perceived Usability captures users’ feelings of control, effort, and frustration during the interaction, where higher scores indicate a smoother and more intuitive experience. Aesthetic Appeal relates to users’ appreciation of the system’s visual and sensory design, reflecting how visually engaging and pleasant the interface is perceived. Finally, Reward Factor assesses how worthwhile, enjoyable, and personally valuable the experience is, encompassing aspects of intrinsic motivation and user satisfaction. In addition to the questionnaire, the participants were briefly interviewed about their overall experience. They were encouraged to provide honest and constructive feedback, which was noted for qualitative analysis.

5.4. Hypotheses

The independent variable in this study is the activation of the RJAS in the robot. We examine self-reported and telemetry-dependent variables to evaluate their impact on user engagement. Self-reported engagement is assessed through the UES-SF questionnaire, which provides an overall score and four subscales: Focused Attention, Perceived Usability, Aesthetic Appeal, and Reward Factor. During the interaction, telemetry engagement is captured through attention-related metrics obtained via the UBT system. Based on previous findings in HRI and JA, we formulate the following hypotheses:

5.4.1. User Behavioral Measures (UBT)

  • H1: Participants in the RJAS Active condition will exhibit a lower distraction frequency ( f d ) than those in the RJAS Inactive condition. The robot’s RJA mechanisms are expected to reduce the number of distraction events.
  • H2: Participants in the RJAS Active condition will show a smaller distraction duration ratio ( ρ d ) than participants in the RJAS Inactive condition. This reflects the effectiveness of the RJAS in maintaining user attention throughout the task.

5.4.2. Self-Reported Measures (UES-SF Questionnaire)

  • H3: Participants in the RJAS Active condition will report higher Total Engagement scores than those in the RJAS Inactive condition. This aggregated score reflects the general level of user involvement as measured by the UES-SF.
To better understand how the observed differences in Total Engagement were distributed across engagement components, we additionally explored the four dimensions that constitute the UES-SF:
  • H4: Participants in the RJAS Active condition will report higher Focused Attention scores than those in the RJAS Inactive condition.
  • H5: Participants in the RJAS Active condition will report higher Perceived Usability scores than those in the RJAS Inactive condition.
  • H6: Participants in the RJAS Active condition will report higher Aesthetic Appeal scores than those in the RJAS Inactive condition.
  • H7: Participants in the RJAS Active condition will report higher Reward Factor scores than those in the RJAS Inactive condition.
These analyses were conducted to provide insight into the nature of the overall engagement effect and were not used for confirmatory hypothesis testing.

6. Results

This section presents the analyses conducted following the experiments, including both quantitative and qualitative approaches. The quantitative analyses were performed using IBM SPSS Statistics (version 26), with a significance threshold set at α = 0.05 . Additionally, post hoc power analyses were conducted using the G*Power software (2020 version) to assess the statistical power achieved with our sample. The qualitative analysis derives from brief semi-structured interviews conducted with the participants after the experiment, offering contextual depth to the statistical findings.

6.1. Quantitative Results

Given the between-subjects design of our study, which contrasted two experimental conditions, we conducted independent samples t-tests for variables that met the assumption of normality. When this assumption was broken, we employed the non-parametric Mann–Whitney U test. The analyses aimed to compare the mean scores across multiple dimensions of participants’ engagement while interacting with the robot between the two conditions: interaction with the robot when the RJAS was active versus when it was inactive.

6.1.1. User Behavioral Measures (UBT)

This subsection presents the results obtained from the UBT system, which served as the objective assessment of user engagement. Based on head pose telemetry data, we computed two key indicators of attentional disengagement: distraction frequency ( f d ) and distraction duration ratio ( ρ d ). These metrics are designed to reflect how often and for how long the participants looked away from the robot, offering a behavioral perspective on the impact of the RJAS on user attention during the interaction.
First, regarding the distraction frequency ( f d ), outliers were identified using boxplot analysis with a 1.5 × I Q R (interquartile range) rule. Specifically, any data point lying more than 1.5 × I Q R above the third quartile or below the first quartile was considered an outlier and removed from the analysis. This method, known as Tukey’s rule, is widely used for ensuring that the measures are not skewed by anomalously high or low data points. Four participants in the Active condition exceeded the upper bound for f d and were excluded from the analysis. As a result, the final sample included 16 participants in the Active condition and 20 in the Inactive condition. Shapiro–Wilk tests confirmed that distraction frequency was normally distributed in both the Inactive ( W = 0.933 , p = 0.179 ) and Active ( W = 0.937 , p = 0.318 ) conditions.
Next, an independent-samples t-test revealed that the participants in the RJAS Active condition ( M = 0.0091 , S D = 0.0044 ) exhibited significantly lower distraction frequency than those in the Inactive condition ( M = 0.0226 , S D = 0.0161 ), t ( 22.52 ) = − 3.564 , p = 0.002 . The mean difference was − 0.0134 , with a large effect size ( d = 1.08 ) and high statistical power ( 1 − β = 0.94 ) to detect this effect.
Looking at the distraction duration ratio ( ρ d ), we also observe some outliers from the original 20–20 group. Two participants from each condition were excluded as a result of the boxplot analysis. As a result, the final sample included 18 participants in the Inactive condition and 18 in the Active condition. The Shapiro–Wilk tests revealed that the distraction duration ratio was normally distributed in the Active condition ( W = 0.948 , p = 0.393 ) but not in the Inactive condition ( W = 0.888 , p = 0.036 ). Although the normality test indicated a significant deviation from normality in the Inactive condition, the Kolmogorov–Smirnov result was non-significant ( p = 0.054 ), and visual inspection of the histogram revealed no severe normality infractions. The distribution was slightly right-skewed but showed no extreme values or multimodality. Given the approximate symmetry and the robustness of the t-test under soft normality infractions, particularly with balanced group sizes, we proceeded with the parametric analysis.
In this case, the t-test revealed that the participants in the RJAS Active condition ( M = 0.0554 , S D = 0.0252 ) exhibited significantly lower ρ d than those in the Inactive condition ( M = 0.1015 , S D = 0.0774 ), t ( 20.56 ) = − 2.405 , p = 0.026 . The mean difference was − 0.0461 , showing a large effect size ( d = 0.80 ) and acceptable statistical power ( 1 − β = 0.76 ) to detect this effect. All the results for both UBT metrics are shown in Figure 5.
Figure 5. Bar chart with the average scores and SDs for each UBT metric. Significance levels are indicated as * for p < 0.05 and ** for p < 0.01 .

6.1.2. Self-Reported Measure (UES-SF)

After analyzing the objective results, this section describes the results obtained from the UES-SF questionnaire, which served as the self-reported measure of user engagement. The UES-SF evaluates participants’ perceived engagement through four dimensions (Focused Attention, Perceived Usability, Aesthetic Appeal, and Reward Factor) as well as a Total Engagement score. These metrics provide insight into the participants’ internal experiences during the interaction with the robot, complementing the behavioral perspective captured by the UBT system.
Concerning Total Engagement, one participant from the RJAS Inactive condition and two from the Active condition exceeded the bounds for overall engagement. As a result, the final sample included 18 participants in the Inactive condition and 19 in the Active condition. Shapiro–Wilk tests confirmed that the overall engagement scores were normally distributed in both the Inactive ( W = 0.939 , p = 0.275 ) and Active ( W = 0.952 , p = 0.428 ) conditions. The subsequent independent-samples t-test revealed that the participants in the RJAS Active condition ( M = 4.16 , S D = 0.39 ) reported significantly higher overall engagement scores than those in the Inactive condition ( M = 3.93 , S D = 0.28 ), t ( 35 ) = 2.102 , p = 0.043 . The resulting Cohen’s d reflects a medium-to-large effect size ( d = 0.69 ), and the sample shows moderate statistical power to detect the effect ( 1 − β = 0.66 ). Figure 6 shows the results for the UES-SF Total Engagement.
Figure 6. Bar chart with the average scores and SDs of the total user engagement measured by the UES-SF. Significance level is indicated as * for p < 0.05 .
After analyzing the overall engagement score, we proceeded to examine each UES-SF dimension separately. These results were examined to better understand the nature of the overall engagement effect and were not subjected to confirmatory hypothesis testing. Therefore, no correction for multiple comparisons was applied to the p-values associated with these exploratory tests.
For Focused Attention, outliers were identified, resulting in the exclusion of one participant from the RJAS Active condition and two from the Inactive condition. The final sample thus included 19 and 18 participants, respectively. This dimension also passed normality tests in both conditions (Inactive: W = 0.960 , p = 0.604 ; Active: W = 0.917 , p = 0.099 ). Group means revealed higher Focused Attention scores in the RJAS Active condition ( M = 3.89 , S D = 0.63 ) compared to the Inactive condition ( M = 3.44 , S D = 0.56 ). An independent-samples t-test confirmed that this difference was statistically significant, t ( 35 ) = 2.30 , p = 0.028 , with a mean difference of 0.45 . The effect size was medium-to-large ( d = 0.75 ), with a statistical power of 0.73 to detect this effect.
For Perceived Usability, one outlier was excluded from each group, resulting in 19 participants per condition. The normality assumption was slightly violated in the Active condition ( W = 0.896 , p = 0.041 ), while it held in the Inactive condition ( W = 0.907 , p = 0.066 ). However, visual inspection of the distribution and the robustness of the t-test justified proceeding with a parametric comparison. The participants in the Active condition reported slightly higher usability scores ( M = 4.37 , S D = 0.53 ) than those in the Inactive condition ( M = 4.33 , S D = 0.54 ), but this difference was not statistically significant ( p = 0.842 ). The effect size was negligible ( d = 0.07 ), and the statistical power to detect such an effect was very low ( 1 − β = 0.07 ).
Regarding Aesthetic Appeal, no outliers were identified and all the participants were retained in the analysis ( n = 20 per condition). Normality was confirmed in both groups (Inactive: W = 0.918 , p = 0.090 ; Active: W = 0.929 , p = 0.151 ), and variances were assumed equal. The participants in both conditions reported similar levels of Aesthetic Appeal (Inactive: M = 3.87 , S D = 0.94 ; Active: M = 3.85 , S D = 0.71 ), and no significant difference was found ( p = 0.950 ). The effect size was negligible ( d = − 0.02 ), with very low statistical power to detect such a small effect ( 1 − β = 0.06 ).
Finally, for Reward Factor, four outliers were excluded from the Inactive group, while no outliers were detected in the Active group, resulting in a final sample of 20 participants in the Active condition and 16 in the Inactive condition. Shapiro–Wilk tests indicated significant deviations from normality in both groups (Inactive: W = 0.808 , p = 0.003 ; Active: W = 0.822 , p = 0.002 ). Given the non-normal distributions and unequal group sizes, a non-parametric Mann–Whitney test was conducted. The participants in the Inactive condition reported slightly higher reward scores ( M = 4.63 , S D = 0.27 ) than those in the Active condition ( M = 4.43 , S D = 0.66 ), but this difference was not statistically significant ( p = 0.789 ). The effect size was negligible ( r = 0.05 ), with low statistical power to detect the effect ( 1 − β = 0.07 ). Figure 7 displays the distribution of scores for each UES-SF dimension across the conditions.
Figure 7. Bar charts with the average scores and SDs of each user engagement dimension measured by the UES-SF. Significance level is indicated as * for p < 0.05 .
A summary of the main results across all the engagement measures, including sample sizes, means, standard deviations, significance levels, and statistical power, is provided in Table 3.
Table 3. Summary of results across engagement measures, dimensions, and experimental conditions. Statistically significant values ( p < 0.05 ) and powers above 0.65 are shown in bold.

6.1.3. Correlational Analysis

Although not directly hypothesized, in order to assess whether self-reported engagement aligns with observable behavior, we conducted exploratory correlational analyses to examine the relationship between the self-reported and behavioral engagement metrics. We believe this analysis might offer a more comprehensive understanding of user experience, in addition to revealing unexpected relationships worth testing in confirmatory studies in the future.
A Pearson correlation was conducted to examine the relationship between participants’ self-reported engagement (UES-SF Total Engagement score) and the distraction frequency measured by the UBT system. The analysis revealed a moderate negative correlation ( r = − 0.32 , p = 0.068 , n = 33 ), suggesting that higher reported engagement tended to be associated with fewer observed distractions, although the result did not reach significance. The statistical power for detecting this effect was moderate ( 1 − β = 0.46 ).
With respect to the relationship between participants’ self-reported engagement (UES-SF Total Engagement score) and the distraction duration ratio, the analysis also revealed a moderate negative correlation ( r = − 0.31 , p = 0.085 , n = 33 ), indicating that higher engagement scores tended to be associated with less time spent distracted. However, this correlation also did not reach statistical significance. The power for this test was similarly moderate ( 1 − β = 0.42 ).
We additionally explored whether any specific UES-SF dimensions were associated with distraction metrics. Among them, Focused Attention exhibited a negative correlation with distraction frequency ( r = − 0.25 , p = 0.158 , n = 33 ), suggesting that those participants who felt more absorbed tended to exhibit fewer observable distractions, although this trend was not statistically significant and the power to detect this effect was limited ( 1 − β = 0.30 ). No correlations were found between the remaining UES-SF dimensions and the objective engagement metrics, indicating that alignment between self-reported and behavioral data may be strongest for attentional aspects of engagement.

6.2. Qualitative Observations

In addition to the post-experiment questionnaire, we conducted brief semi-structured interviews with the participants to gain deeper insight into their experiences. These interviews were carried out informally at the end of each session and were designed to let the participants speak freely, without prompting or directing their response. While these accounts are subjective and not generalizable, they offer valuable context regarding users’ perceptions and emotional responses to the robot’s behavior.
All the comments were transcribed and categorized by two independent raters according to thematic similarities. A light thematic coding was employed to identify recurrent topics and sentiments, grouping the comments into five predefined categories: positive, negative, constructive suggestions, error reports, and neutral observations. This classification was based on both the tone and content of each statement. The subjective nature of these interviews introduces potential biases, including social desirability effects as participants might be inclined to offer favorable opinions when speaking directly to the experimenters. Nonetheless, many provided critical feedback or pointed out unexpected behaviors, suggesting a reasonable level of honesty.
Among the forty participants interviewed, fourteen provided clearly positive feedback, especially within the RJAS Active condition (ten of them). The participants frequently described the robot as attentive and socially present. For instance, one user noted that “the robot followed me too much, very attentive”, and even praised its design, stating it had a “friendly appearance” and that they liked “the pink colour” of the plush body (ID = 44, RJAS Active). These impressions align with the increased scores in engagement observed in the quantitative analysis.
The participants also commented on how the robot reacted to attentional shifts. One participant stated that “the robot noticed when I was distracted”, adding that they perceived its responses as deliberate and appropriate (ID = 51, RJAS Active). Similarly, another remarked that “I realised the RJAS was working”, highlighting their awareness of the system’s responsiveness (ID = 73, RJAS Active). Such experiences support the notion that the system contributed meaningfully to users’ sense of being seen and understood.
Several remarks highlighted the RJAS’s role in shaping the interaction. One participant appreciated how “the robot took into account that I was getting distracted”, reinforcing the impression of reciprocal attention (ID = 72, RJAS Active). Another participant shared that “the robot paid attention to me”, even though some stimuli like the screen were more distracting (ID = 82, RJAS Active). These perceptions suggest the robot’s attentional behavior added a layer of social awareness to the experience.
While the majority of the comments in the RJAS Active condition were positive, some pointed out areas for improvement. For instance, one participant described the robot’s behavior as “passive-aggressive with distractions” and noted that “when the face was occluded, it failed”, although they also acknowledged that “the RJAS worked well and aligned with distractions” (ID = 47, RJAS Active). Another mentioned early difficulties in hand detection, saying that “in one case, it did not detect a hand”, but still concluded that “the RJAS worked great” (ID = 49, RJAS Active). These constructive remarks suggest that, while the core attentional behavior was well received, its reliability in edge cases could be improved.
In contrast, the participants in the RJAS Inactive condition generally reported a more neutral or disconnected experience. One participant commented that “the robot is dispensable” and felt that the robot “barely turned around” (ID = 55, RJAS Inactive). Another described it as “doing its own thing” and even remarked that “the plush version was a bit creepy” (ID = 60, RJAS Inactive). These reflections reinforce the importance of active attentional behaviors in shaping engagement and social connectedness.
Overall, the qualitative feedback supports the quantitative findings: the participants in the RJAS Active condition experienced the robot as more responsive, engaging, and socially present. While some technical aspects, such as hand detection and reaction timing, could benefit from refinement, the RJAS system demonstrably enhanced the interaction by aligning the robot’s behavior with the user’s attentional state.

7. Discussion

As the previous section shows, the findings partially support our hypotheses. Regarding the UBT metrics, H1 was supported as the participants in the RJAS Active condition showed significantly lower distraction frequency ( f d ) than those in the Inactive condition, with a large effect size and high statistical power. This aligns with our expectation that the robot’s RJA behavior would reduce the number of attentional disengagement events, helping the participants to remain more focused during the interaction. Similarly, H2 was also supported: the distraction duration ratio ( ρ d ) was significantly lower for the Active condition, again with a large effect and acceptable power. These results confirm that the RJAS not only reduced how often users looked away but also helped to shorten those disengagement episodes, reinforcing the system’s effectiveness in sustaining user attention throughout the task (the game).
Turning to self-reported engagement, H3 was also supported, with significantly higher total UES-SF scores observed in the RJAS Active condition compared to the Inactive group. This result showed a medium-to-large effect size and moderate statistical power, supporting the practical relevance of the finding despite the limited sample size. This indicates that, as hypothesized, the participants perceived the interaction as more engaging when the robot responded to their attentional behavior. The dimension-specific exploratory hypotheses yielded more nuanced results. H4 was supported as the participants reported significantly higher Focused Attention in the Active condition. This aligns with the expectation that the robot’s attentional responsiveness would improve task immersion. However, H5–H7 were not supported. Perceived Usability, Aesthetic Appeal, and Reward Factor did not differ significantly between the conditions, and the associated effect sizes and statistical power were negligible to low. These findings may indicate that attentional responsiveness, while effective in enhancing Focused Attention, may not be sufficient to influence broader engagement components such as Perceived Usability, Aesthetic Appeal, or Reward Factor. These aspects could depend more on the overall interaction flow—in this case, influenced by the game—the perceived expressiveness and appearance of the robot, or the user’s expectations and prior experiences, which may have been relatively similar across the conditions. Nevertheless, the low statistical power observed for these dimensions suggests that future studies with larger sample sizes might reveal significant effects.
These findings build upon previous work, where we demonstrated that RJA mechanisms increase the perceived social presence of the robot [20]. The current results go one step further, suggesting that this increased social presence may translate into higher perceived value of the interaction itself. In this case, the participants were not only more aware of the robot’s social cues but also more invested in the shared task. The attentional responses of the robot may have enhanced the meaningfulness of the interaction, reinforcing the participant’s engagement with the game and the robot as a collaborative partner.
As a complementary step beyond our main hypotheses, we also performed exploratory correlational analyses to investigate potential relationships between self-reported and behavioral engagement metrics. Although neither of the objective engagement measures reached statistical significance in their correlation with the UES-SF Total Engagement score, both showed moderate negative associations in the expected direction. Specifically, higher self-reported engagement was associated with lower distraction frequency ( r = − 0.32 , p = 0.068 ) and shorter distraction duration ( r = − 0.31 , p = 0.085 ). The statistical power for detecting these effects was moderate ( 1 − β = 0.46 and 0.42 , respectively), which may have limited the ability to achieve significance. These trends, while not definitive, suggest a relevant alignment between the self-reported experience and observable behavior. We also assessed whether participants’ scores on the UES-SF dimensions were associated with the behavioral measures of distraction recorded by the UBT system. While again no statistically significant correlations were observed, we noted some consistent patterns. For instance, Focused Attention showed a moderate negative correlation with distraction frequency ( r = − 0.25 , p = 0.158 ), with low statistical power ( 1 − β = 0.30 ), suggesting that those participants who felt more absorbed tended to exhibit fewer observable distractions. No meaningful correlations were found for the remaining UES-SF dimensions, indicating that the convergence between behavioral and self-reported data may be more specific to attentional aspects of engagement. Regarding the lack of statistical significance in these analyses, we believe it may be related to the sample size, and future studies with a larger sample may help to confirm the robustness of these relationships.
In sum, we consider that these correlational results provide preliminary support for the idea that self-reported and observed behavioral measures reflect similar aspects of engagement. Moreover, while not conclusive, the observed patterns also suggest that UBT-derived metrics may capture meaningful aspects of user engagement, highlighting their potential utility as unsupervised behavioral indicators in HRI contexts. In the more general context of joint attention mechanisms, this opens up a promising research line. If real-time behavioral engagement can be monitored through UBT or similar systems, this capability could enable robots to dynamically adjust their behavior when engagement drops. Thus, objective engagement tracking might serve as the final component in closing the loop of shared attention in HRI, complementing IJA and RJA with EJA as a new design principle for adaptive social robots.
Finally, the qualitative feedback obtained from the semi-structured interviews reinforces the quantitative findings regarding engagement. Many participants in the RJAS Active condition described the robot as attentive and responsive, frequently noting its ability to track their attention and react appropriately to distractions. These impressions align with the observed increase in engagement and suggest that the RJAS contributed to a sense of mutual awareness during the interaction. At the same time, a few participants reported moments where the robot’s reactions felt exaggerated or mistimed, particularly when interpreting minor movements as distractions. These observations point to potential areas for refinement in the system’s sensitivity thresholds and head pose detector precision. In contrast, the participants in the Inactive condition often described the robot as passive or disconnected, which supports the notion that attentional responsiveness plays a key role in preventing user disengagement and impacts perceived interaction quality in HRI.

Limitations and Future Work

A key limitation of this study lies in the use of head pose telemetry as a proxy for attentional engagement. Although non-intrusive and practical, this method is constrained by the resolution of the RGB sensor and the accuracy of the head pose estimation, with mean absolute error values of 5.4° ± 4.4° for yaw, 5.5° ± 5.3° for pitch, and 4.6° ± 5.6° for roll https://github.com/openvinotoolkit/open_model_zoo/blob/master/models/intel/head-pose-estimation-adas-0001/README.md (accessed on 30 July 2025). The accuracy limitation was especially noticeable during interactions involving participants wearing glasses, which sometimes caused detection errors. These issues led to misclassifications of minor movements as distractions, introducing noise into the data. The qualitative feedback also indicated that some participants noticed the robot reacting to small or unintended movements, occasionally perceived as exaggerated or mistimed. This highlights the need to fine-tune the sensitivity thresholds and improve robustness. Future iterations of the RJAS and the UBT system could benefit from integrating complementary signals such as eye gaze, blink rate, or facial expressions to provide a more reliable and nuanced view of engagement.
Another consideration regarding the current state of the UBT system is the sampling rate of the head pose detector, which operates at 6 FPS due to the hardware constraints of the robot’s processing architecture. This frame rate may miss very brief head movements or micro-shifts in gaze. However, its impact on distraction metrics is likely minimal as the system is designed to consider only sustained head orientations lasting at least 1.5 s, thereby deliberately excluding brief movements such as eye saccades or momentary glances. Still, the occasional omission of short gaze shifts could slightly affect the total distraction time measure. Future work could explore optimized processing or external sensors to achieve finer-grained attention tracking.
Beyond these technical considerations, there are also methodological limitations that should be acknowledged. Concerning the temporal scope of the study, measuring engagement in a single short interaction (such as a 15 min gameplay session) may not fully capture the dynamics of user involvement over time. Engagement is inherently multidimensional and can fluctuate depending on context, task complexity, and novelty effects. Thus, while our results reflect meaningful differences between experimental conditions, they should be interpreted with caution when generalizing to longer-term or repeated interactions. Future work should explore engagement over extended periods and across a wider variety of tasks beyond gamified settings in order to assess whether the observed benefits of the RJAS persist and translate to broader HRI scenarios.
Another methodological limitation is that the specific impact of each expressive modality was not assessed independently. Future work could investigate how variations in prosody, such as pitch, intonation, or speech rate, affect the perceived responsiveness and effectiveness of the robot in recovering attention. Additionally, evaluating the system on alternative platforms with different embodiment features, including the presence or absence of facial LEDs, could help to clarify the role of aesthetic elements in shaping user engagement and distraction.
In addition, while the sample included 40 participants with comparable group sizes, some variability remained in demographic factors such as age, gender, and prior experience with the robot (see Table 2). The participants were randomly assigned to conditions, which helped to reduce the risk of systematic bias. Some variability in age distribution was present across the groups, but no clear asymmetries were observed. Still, future studies with larger and more stratified samples could offer greater control over demographic factors and help to clarify their potential influence.
In parallel, future work could also build on the current findings by exploring the impact of the RJAS on specific population groups, such as elderly individuals with mild cognitive impairment or children with ASD. Although our results suggest that the system is effective across a diverse sample, targeted studies could further validate its applicability in specific contexts. This highlights a promising direction for future research on the concept of joint engagement (JE), particularly in populations such as children with ASD, where social robotics may serve as a valuable tool to support shared attention and interaction. JE is a key developmental process in which two individuals share attention and interest toward an object or activity [63]. This phenomenon is especially relevant in therapeutic and educational contexts for children with ASD, where difficulties staying engaged with others often slow the development of social and communication skills. Interventions such as JASPER (joint attention, symbolic play, engagement, and regulation) have shown encouraging results in helping children to develop JE and related skills through structured but natural play-based interactions. More importantly, JE has been identified as a mediating mechanism for the development of IJA skills, which in turn are linked to later gains in language and communication [64].
In this sense, the RJAS system presented in our study can be viewed as a potential first step toward incorporating JASPER-like principles into robotic platforms. By enabling the robot to respond dynamically to attentional disengagement and reorient the user’s focus, we take a first step toward supporting more meaningful JA experiences. A future direction could involve integrating the system with behavioral models that explicitly target IJA (already mentioned before), allowing the robot not only to respond to the user’s attention but to actively prompt and support it in ways consistent with JASPER’s goals. This might open the door to therapeutic applications in child–robot interaction, where the robot’s ability to detect engagement in real time and adapt accordingly could help to reinforce the social skills necessary for language learning and overall developmental improvement.

8. Conclusions

In this study, we examined the impact of the RJAS on user engagement, a multidimensional construct encompassing cognitive, emotional, and behavioral involvement during interaction. The participants played the “Odds and Evens” turn-taking game against a social robot under two experimental conditions (RJAS Active vs. Inactive). While the user was playing the game, auditory and visual stimuli were introduced to simulate real-world dynamics, testing the robot’s ability to detect and follow the user’s focus. We used a twofold approach to evaluate the system’s impact on the user’s engagement during the interaction: using the self-reported measurements from the UES-SF, as well as the UBT, a telemetry system. The UBT system monitored head pose orientation to indicate user distractions and attentional focus during the interaction. Combining self-reports with behavioral data remains relatively rare in HRI research, where engagement is typically measured using either questionnaires or manually coded observations.
Taken together, the results demonstrate that the activation of the RJAS led to clear improvements in both behavioral and self-reported indicators of user engagement. The participants in the Active condition were less distracted and reported higher engagement, particularly in terms of attentional focus. While some engagement dimensions remained unaffected, possibly due to limited statistical power, overall, the results support the idea that RJA mechanisms enhance the user’s involvement in the interaction. The correlational analyses further suggested a meaningful alignment between the self-reports and observed behavior, and the qualitative feedback reflected these findings, highlighting the robot’s perceived attentiveness. Overall, the results consistently indicate that activating the RJAS had a positive effect on user engagement during interaction with the robot.
Future research should explore whether the benefits of the RJAS hold over longer interactions and a wider range of tasks, moving beyond short game-based sessions. Enhancing the accuracy of the telemetry system by adding complementary signals, such as eye gaze or facial expressions, could improve how well engagement is detected. Another relevant direction is testing the RJAS in specific user groups, such as children with ASD, where supporting joint engagement and prompting IJA behaviors may have practical value. Lastly, adding real-time engagement monitoring to the robot’s control system could make it possible for the robot to respond more effectively when users lose focus, completing the interaction loop between IJA, RJA, and EJA.

Author Contributions

Conceptualization, J.G.-M., J.J.G.-M., J.C.C., and Á.C.-G.; methodology, J.G.-M. and J.J.G.-M.; software, J.G.-M.; validation, J.G.-M. and J.J.G.-M.; formal analysis, J.J.G.-M.; investigation, J.G.-M.; resources, J.G.-M. and J.J.G.-M.; data curation, J.G.-M. and J.J.G.-M.; writing—original draft preparation, J.G.-M. and J.J.G.-M.; writing—review and editing, J.G.-M., J.J.G.-M., J.C.C., and Á.C.-G.; visualization, J.G.-M. and J.J.G.-M.; supervision, J.C.C. and Á.C.-G.; project administration, J.C.C. and Á.C.-G.; funding acquisition, J.C.C. and Á.C.-G. All authors have read and agreed to the published version of the manuscript.

Funding

The research leading to these results has received funding from the following projects: Robots sociales para mitigar la soledad y el aislamiento en mayores (SOROLI), PID2021-123941OA-I00, funded by Agencia Estatal de Investigación (AEI), Spanish Ministerio de Ciencia e Innovación. Robots sociales para reducir la brecha digital de las personas mayores (SoRoGap), TED2021-132079B-I00, funded by Agencia Estatal de Investigación (AEI), Spanish Ministerio de Ciencia e Innovación. Mejora del nivel de madurez tecnológica del robot Mini (MeNiR) PDC2022-133518-I00, funded by MCIN/AEI/10.13039/501100-011033 and by the European Union NextGeneration EU/PRTR. Portable Social Robot with High Level of Engagement (PoSoRo) PID2022-140345OB-I00, funded by MCIN/AEI/10.13039/501100-011033 and ERDF A way of making Europe.

Institutional Review Board Statement

Ethical review and approval were waived for this study due to its low-risk nature and the exclusive involvement of voluntary participants who provided informed consent. According to Spanish regulations, Law 14/2007 of 3 July on Biomedical Research regulates research ethics committees and applies only to studies involving invasive procedures, biological samples, or genetic analyses within the healthcare system—none of which are present in this study. Given that Law 14/2007 refers to Law 41/2002 in its final provision, and since it does not apply to our study, the applicable framework is Law 41/2002 of 14 November, which governs patient autonomy and clinical information. This law sets out the principles of informed consent, participant rights, and data protection, which were fully respected throughout the study. Additionally, the study complies with Organic Law 3/2018 of 5 December on the Protection of Personal Data and Guarantee of Digital Rights (LOPDGDD) and with the General Data Protection Regulation (Regulation (EU) 2016/679, GDPR), under which informed consent was obtained from all participants and/or their legal guardians.

Data Availability Statement

The raw data supporting the conclusions of this article will be made available by the authors on request.

Acknowledgments

During the preparation of this work, the authors used GPT-4o, from OpenAI, and Grammarly, from Grammarly Inc., to improve the work’s readability and language. After using these tools, the authors reviewed and edited the content as needed and take full responsibility for the publication’s content.

Conflicts of Interest

The authors declare no conflicts of interest.

References

  1. Picton, T.W.; Campbell, K.B.; Baribeau-Braun, J.; Proulx, G.B. The neurophysiology of human attention: A tutorial review. In Attention and Performance VII; Routledge: Abingdon, UK, 2022; pp. 429–467. [Google Scholar]
  2. Burgoon, J.K.; Manusov, V.; Guerrero, L.K. Nonverbal Communication; Routledge: Abingdon, UK, 2021. [Google Scholar]
  3. Moll, H. What We Do and Don’t Know About Joint Attention. Topoi 2024, 43, 247–258. [Google Scholar] [CrossRef] [Scilit]
  4. Flom, R.; Lee, K.; Muir, D. Gaze-Following: Its Development and Significance; Psychology Press: Hove, UK, 2017. [Google Scholar]
  5. O’Madagain, C.; Tomasello, M. Joint attention to mental content and the social origin of reasoning. Synthese 2021, 198, 4057–4078. [Google Scholar] [CrossRef] [Scilit]
  6. Moore, C.; Dunham, P.J.; Dunham, P. Joint Attention: Its Origins and Role in Development; Psychology Press: Hove, UK, 2014. [Google Scholar]
  7. Huang, C.M.; Thomaz, A.L. Joint attention in human-robot interaction. In Proceedings of the 2010 AAAI Fall Symposium Series, Arlington, VA, USA, 11–13 November 2010. [Google Scholar]
  8. Mundy, P. RJA/IJA (Initiating/responding to joint attention). In Encyclopedia of Autism Spectrum Disorders; Springer: Berlin/Heidelberg, Germany, 2021; pp. 3997–4004. [Google Scholar]
  9. Cho, Y. The Relationship Between CJA, IJA, RJA of Joint Attention and Language in Infancy. Ph.D. Thesis, Seoul National University Graduate School, Seoul, Republic of Korea, 2015. [Google Scholar]
  10. Cipolletta, S.; Mascolo, M.F.; Procter, H. Intersubjectivity, joint action and sociality. J. Constr. Psychol. 2022, 35, 904–929. [Google Scholar] [CrossRef] [Scilit]
  11. Yuill, N.; Yuill, N. Engagement and Joint Attention. In Technology to Support Children’s Collaborative Interactions: Close Encounters of the Shared Kind; Springer: Berlin/Heidelberg, Germany, 2021; pp. 21–37. [Google Scholar]
  12. Paliga, M. Human–cobot interaction fluency and cobot operators’ job performance. The mediating role of work engagement: A survey. Robot. Auton. Syst. 2022, 155, 104191. [Google Scholar] [CrossRef] [Scilit]
  13. Avelino, J.; Garcia-Marques, L.; Ventura, R.; Bernardino, A. Break the ice: A survey on socially aware engagement for human–robot first encounters. Int. J. Soc. Robot. 2021, 13, 1851–1877. [Google Scholar] [CrossRef] [Scilit]
  14. Alam, A. Social robots in education for long-term human-robot interaction: Socially supportive behaviour of robotic tutor for creating robo-tangible learning environment in a guided discovery learning interaction. ECS Trans. 2022, 107, 12389. [Google Scholar] [CrossRef] [Scilit]
  15. Breazeal, C.; Dautenhahn, K.; Kanda, T. Social robotics. In Springer Handbook of Robotics; Springer: Berlin/Heidelberg, Germany, 2016; pp. 1935–1972. [Google Scholar]
  16. Woertman, S.E. Joint Attention in Human-Robot Interaction. Bachelor’s Thesis, University of Twente, Enschede, The Netherlands, 2024. [Google Scholar]
  17. Admoni, H.; Scassellati, B. Social eye gaze in human-robot interaction: A review. J.-Hum.-Robot. Interact. 2017, 6, 25–63. [Google Scholar] [CrossRef] [Scilit]
  18. Zabidi, S.A.M.; Yusof, H.M.; Sidek, S.N.; Ghazali, A.; Rashidan, M. Application of Robots in Improving Joint Attention and Imitation Skills for Children with Autism: A Comprehensive Review. J. Mech. Eng. Res. Dev. 2022, 45, 96–122. [Google Scholar]
  19. Skantze, G.; Hjalmarsson, A.; Oertel, C. Turn-taking, feedback and joint attention in situated human–robot interaction. Speech Commun. 2014, 65, 50–66. [Google Scholar] [CrossRef] [Scilit]
  20. García-Martínez, J.; Gamboa-Montero, J.J.; Castillo, J.C.; Castro-González, Á.; Salichs, M.A. Implementation of a Biologically Inspired Responsive Joint Attention System for a Social Robot. Adv. Intell. Syst. 2025, 7, 2400650. [Google Scholar] [CrossRef] [Scilit]
  21. García-Martínez, J.; Gamboa-Montero, J.J.; Castillo, J.C.; Castro-González, Á. Analyzing the Impact of Responding to Joint Attention on the User Perception of the Robot in Human-Robot Interaction. Biomimetics 2024, 9, 769. [Google Scholar] [CrossRef] [Scilit] [PubMed]
  22. O’Brien, H.L.; Cairns, P.; Hall, M. A practical approach to measuring user engagement with the refined user engagement scale (UES) and new UES short form. Int. J.-Hum.-Comput. Stud. 2018, 112, 28–39. [Google Scholar] [CrossRef] [Scilit]
  23. Sorrentino, A.; Fiorini, L.; Cavallo, F. From the Definition to the Automatic Assessment of Engagement in Human–Robot Interaction: A Systematic Review. Int. J. Soc. Robot. 2024, 16, 1641–1663. [Google Scholar] [CrossRef] [Scilit]
  24. Nagai, Y.; Hosoda, K.; Morita, A.; Asada, M. A constructive model for the development of joint attention. Connect. Sci. 2003, 15, 211–229. [Google Scholar] [CrossRef] [Scilit]
  25. Boucenna, S.; Gaussier, P.; Hafemeister, L. Development of joint attention and social referencing. In Proceedings of the 2011 IEEE International Conference on Development and Learning (ICDL), Frankfurt am Main, Germany, 24–27 August 2011; Volume 2, pp. 1–6. [Google Scholar]
  26. Scassellati, B. Imitation and mechanisms of joint attention: A developmental structure for building social skills on a humanoid robot. In Proceedings of the International Workshop on Computation for Metaphors, Analogy, and Agents, University of Aizu, Aizuwakamatsu, Japan, 6–10 April 1998; Springer: Berlin/Heidelberg, Germany, 1998; pp. 176–195. [Google Scholar]
  27. Brooks, R.A.; Breazeal, C.; Marjanović, M.; Scassellati, B.; Williamson, M.M. The Cog project: Building a humanoid robot. In Proceedings of the International Workshop on Computation for Metaphors, Analogy, and Agents, University of Aizu, Aizuwakamatsu, Japan, 6–10 April 1998; Springer: Berlin/Heidelberg, Germany, 1998; pp. 52–87. [Google Scholar]
  28. Ito, M.; Tani, J. Joint attention between a humanoid robot and users in imitation game. In Proceedings of the International Conference on Development and Learning (ICDL), La Jolla, CA, USA, 20–22 October 2004. [Google Scholar]
  29. Ishida, T. Development of a small biped entertainment robot QRIO. In Proceedings of the Micro-Nanomechatronics and Human Science, and The Fourth Symposium Micro-Nanomechatronics for Information-Based Society, Nagoya, Japan, 31 October–3 November 2004; IEEE: New York, NY, USA, 2004; pp. 23–28. [Google Scholar]
  30. Chew, J.Y.; Wang, X. Joint Attention Estimation during Multi-party Facilitation Using Multi-Modal Fusion. In Proceedings of the Companion of the 2024 ACM/IEEE International Conference on Human-Robot Interaction, Boulder, CO, USA, 11–15 March 2024; HRI ’24. pp. 322–326. [Google Scholar]
  31. Huang, C.M.; Thomaz, A.L. Effects of responding to, initiating and ensuring joint attention in human-robot interaction. In Proceedings of the 2011 Ro-Man, Atlanta, GA, USA, 31 July–3 August 2011; IEEE: New York, NY, USA, 2011; pp. 65–71. [Google Scholar]
  32. Westlund, J.M.K.; Dickens, L.; Jeong, S.; Harris, P.L.; DeSteno, D.; Breazeal, C.L. Children use non-verbal cues to learn new words from robots as well as people. Int. J.-Child-Comput. Interact. 2017, 13, 1–9. [Google Scholar] [CrossRef] [Scilit]
  33. Scassellati, B.; Boccanfuso, L.; Huang, C.M.; Mademtzi, M.; Qin, M.; Salomons, N.; Ventola, P.; Shic, F. Improving social skills in children with ASD using a long-term, in-home social robot. Sci. Robot. 2018, 3, eaat7544. [Google Scholar] [CrossRef] [Scilit]
  34. Pereira, A.; Oertel, C.; Fermoselle, L.; Mendelson, J.; Gustafson, J. Responsive joint attention in human-robot interaction. In Proceedings of the 2019 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), Macau, China, 4–8 November 2019; IEEE: New York, NY, USA, 2019; pp. 1080–1087. [Google Scholar]
  35. Cao, H.L.; Simut, R.E.; Desmet, N.; De Beir, A.; Van De Perre, G.; Vanderborght, B.; Vanderfaeillie, J. Robot-assisted joint attention: A comparative study between children with autism spectrum disorder and typically developing children in interaction with NAO. IEEE Access 2020, 8, 223325–223334. [Google Scholar] [CrossRef] [Scilit]
  36. Mishra, C.; Skantze, G. Knowing where to look: A planning-based architecture to automate the gaze behavior of social robots. In Proceedings of the 2022 31st IEEE International Conference on Robot and Human Interactive Communication (RO-MAN), Napoli, Italy, 29 August–2 September 2022; IEEE: New York, NY, USA, 2022; pp. 1201–1208. [Google Scholar]
  37. Teow, K. Joint Attention in Human-Robot Interaction, a Case of Eye Gaze-Leading Task. Bachelor’s Thesis, University of Twente, Enschede, The Netherlands, 2024. [Google Scholar]
  38. Mazzola, C.; Morocutti, L.; Pedrizzi, H.; Poslon, X.D.; Lucny, A.; Wingert, S.M.; Nagai, Y.; Rea, F.; Sciutti, A. Sketch it for the Robot! How Child-Like Robots’ Joint Attention Affects Humans’ Drawing Strategies. In Proceedings of the 2024 IEEE International Conference on Development and Learning (ICDL), Austin, TX, USA, 20–23 May 2024; IEEE: New York, NY, USA, 2024; pp. 1–8. [Google Scholar]
  39. O’Brien, H.L.; Toms, E.G. What is user engagement? A conceptual framework for defining user engagement with technology. J. Am. Soc. Inf. Sci. Technol. 2008, 59, 938–955. [Google Scholar] [CrossRef] [Scilit]
  40. O’Brien, H.; Cairns, P. Why Engagement Matters: Cross-Disciplinary Perspectives of User Engagement in Digital Media; Springer: Berlin/Heidelberg, Germany, 2016. [Google Scholar]
  41. O’Brien, H.L.; Toms, E.G. The development and evaluation of a survey to measure user engagement. J. Am. Soc. Inf. Sci. Technol. 2010, 61, 50–69. [Google Scholar] [CrossRef] [Scilit]
  42. Hall, J.; Tritton, T.; Rowe, A.; Pipe, A.; Melhuish, C.; Leonards, U. Perception of own and robot engagement in human–robot interactions and their dependence on robotics knowledge. Robot. Auton. Syst. 2014, 62, 392–399. [Google Scholar] [CrossRef] [Scilit]
  43. Rich, C.; Ponsler, B.; Holroyd, A.; Sidner, C.L. Recognizing engagement in human-robot interaction. In Proceedings of the 2010 5th ACM/IEEE International Conference on Human-Robot Interaction (HRI), Osaka, Japan, 2–5 March 2010; pp. 375–382. [Google Scholar]
  44. Sidner, C.L.; Lee, C.; Kidd, C.D.; Lesh, N.; Rich, C. Explorations in engagement for humans and robots. Artif. Intell. 2005, 166, 140–164. [Google Scholar] [CrossRef] [Scilit]
  45. Ishii, R.; Nakano, Y.I. Estimating User’s Conversational Engagement Based on Gaze Behaviors. In Proceedings of the Intelligent Virtual Agents, Tokyo, Japan, 1–3 September 2008; Prendinger, H., Lester, J., Ishizuka, M., Eds.; Lecture Notes in Computer Science. Springer: Berlin/Heidelberg, Germany, 2008; pp. 200–207. [Google Scholar]
  46. Ivaldi, S.; Anzalone, S.; Rousseau, W.; Sigaud, O.; Chetouani, M. Robot initiative in a team learning task increases the rhythm of interaction but not the perceived engagement. Front. Neurorobotics 2014, 8, 1–16. [Google Scholar] [CrossRef] [Scilit] [PubMed]
  47. Sanghvi, J.; Castellano, G.; Leite, I.; Pereira, A.; McOwan, P.W.; Paiva, A. Automatic analysis of affective postures and body motion to detect engagement with a game companion. In Proceedings of the 6th International Conference on Human-Robot Interaction, Lausanne, Switzerland, 8–11 March 2011; HRI ’11. pp. 305–312. [Google Scholar]
  48. Scassellati, B. How Social Robots Will Help Us to Diagnose, Treat, and Understand Autism. In Proceedings of the Robotics Research, Hiroshima, Japan, 26–29 November 2007; Thrun, S., Brooks, R., Durrant-Whyte, H., Eds.; Springer: Berlin/Heidelberg, Germany, 2007. Tracts in Advanced Robotics. pp. 552–563. [Google Scholar]
  49. Boucenna, S.; Anzalone, S.; Tilmont, E.; Cohen, D.; Chetouani, M. Learning of Social Signatures Through Imitation Game Between a Robot and a Human Partner. IEEE Trans. Auton. Ment. Dev. 2014, 6, 213–225. [Google Scholar] [CrossRef] [Scilit]
  50. Anzalone, S.M.; Boucenna, S.; Ivaldi, S.; Chetouani, M. Evaluating the Engagement with Social Robots. Int. J. Soc. Robot. 2015, 7, 465–478. [Google Scholar] [CrossRef] [Scilit]
  51. Marques-Villarroya, S.; Castillo, J.C.; Fernández-Rodicio, E.; Salichs, M.A. A bio-inspired exogenous attention-based architecture for social robots. Expert Syst. Appl. 2024, 249, 123623. [Google Scholar] [CrossRef] [Scilit]
  52. Posner, M. Components of visual orienting. In Attention and Performance X: Control of Language Processes/Lawrence Erlbaum; Lawrence Erlbaum: London, UK, 1984. [Google Scholar]
  53. Day, B.L.; Fitzpatrick, R.C. The vestibular system. Curr. Biol. 2005, 15, R583–R586. [Google Scholar] [CrossRef] [Scilit]
  54. Salichs, M.A.; Castro-González, Á.; Salichs, E.; Fernández-Rodicio, E.; Maroto-Gómez, M.; Gamboa-Montero, J.J.; Marques-Villarroya, S.; Castillo, J.C.; Alonso-Martín, F.; Malfaz, M. Mini: A new social robot for the elderly. Int. J. Soc. Robot. 2020, 12, 1231–1249. [Google Scholar] [CrossRef] [Scilit]
  55. Quigley, M.; Conley, K.; Gerkey, B.; Faust, J.; Foote, T.; Leibs, J.; Berger, E.; Wheeler, R.; Ng, A.Y. ROS: An open-source Robot Operating System. In Proceedings of the ICRA Workshop on Open Source Software, Kobe, Japan, 12–17 May 2009; Volume 3, p. 5. [Google Scholar]
  56. Yoshida, K.T.; Kiernan, J.X.; Okamura, A.M.; Nunez, C.M. Exploring human response times to combinations of audio, haptic, and visual stimuli from a mobile device. In Proceedings of the 2023 IEEE World Haptics Conference (WHC), Delft, The Netherlands, 10–13 July 2023; IEEE: New York, NY, USA, 2023; pp. 121–127. [Google Scholar]
  57. Proctor, R.W.; Schneider, D.W. Hick’s law for choice reaction time: A review. Q. J. Exp. Psychol. 2018, 71, 1281–1299. [Google Scholar] [CrossRef] [Scilit] [PubMed]
  58. Fischer, B.; Ramsperger, E. Human express saccades: Extremely short reaction times of goal directed eye movements. Exp. Brain Res. 1984, 57, 191–195. [Google Scholar] [CrossRef] [Scilit]
  59. Spence, C.; Driver, J. Audiovisual links in exogenous covert spatial orienting. Percept. Psychophys. 1997, 59, 1–22. [Google Scholar] [CrossRef] [Scilit]
  60. Mackamul, E.B.; Esteves, A. A Look at the Effects of Handheld and Projected Augmented-reality on a Collaborative Task. In Proceedings of the Symposium on Spatial User Interaction, Berlin, Germany, 13–14 October 2018; SUI ’18. pp. 74–78. [Google Scholar]
  61. Speakman, R.; Hall, M.M.; Walsh, D. User Engagement with Generous Interfaces for Digital Cultural Heritage. In Proceedings of the Digital Libraries for Open Knowledge, Porto, Portugal, 10–13 September 2018; Méndez, E., Crestani, F., Ribeiro, C., David, G., Lopes, J.C., Eds.; Lecture Notes in Computer Science. Springer: Cham, Switzerland, 2018; pp. 186–191. [Google Scholar]
  62. Donnermann, M.; Lein, M.; Messingschlager, T.; Riedmann, A.; Schaper, P.; Steinhaeusser, S.; Lugrin, B. Social robots and gamification for technology supported learning: An empirical study on engagement and motivation. Comput. Hum. Behav. 2021, 121, 106792. [Google Scholar] [CrossRef] [Scilit]
  63. Kasari, C.; Gulsrud, A.C.; Wong, C.; Kwon, S.; Locke, J. Randomized Controlled Caregiver Mediated Joint Engagement Intervention for Toddlers with Autism. J. Autism Dev. Disord. 2010, 40, 1045–1056. [Google Scholar] [CrossRef] [Scilit] [PubMed]
  64. Shih, W.; Shire, S.; Chang, Y.C.; Kasari, C. Joint engagement is a potential mechanism leading to increased initiations of joint attention and downstream effects on language: JASPER early intervention for children with ASD. J. Child Psychol. Psychiatry 2021, 62, 1228–1235. [Google Scholar] [CrossRef] [Scilit] [PubMed]
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content.

Article Metrics

Citations

Article Access Statistics

Multiple requests from the same IP address are counted as one view.