How Human–Robot Interaction Can Influence Task Performance and Perceived Cognitive Load at Different Support Conditions

Abstract

Cognitive load refers to the mental resources used for executing simultaneous tasks. Since these resources are limited, individuals can only process a specific amount of information at a time. Daily activities often involve mentally demanding tasks, which is why social robots have been proposed to simplify them and support users. This study aimed to verify whether and how a social robot can enhance the performance and support the management of cognitive load. Participants completed a baseline where a cognitive activity was carried out without support, and three other conditions where similar activities of increasing difficulty were collaboratively made with the NAO robot. In each condition, errors, time, and perceived cognitive load were measured. Results revealed that the robot improved performance and perceived cognitive load when compared to the baseline, but this support was then thwarted by excessive levels of cognitive load. Future research should focus on developing and designing collaborative human–robot interactions that consider the user’s mental demand, to promote effective and personalized robotic help for independent living.

1. Introduction

Human working memory can handle a limited amount of information when executing simultaneous tasks [1]. This is the basic assumption implied in the concept of cognitive load (CL), which refers to the mental resources used to manage a behavioral plan [2]. There are a number of types of CL such as intrinsic, extraneous, and germane [3]. Intrinsic CL results from the complexity of the learning task and depends on the number of sources of information, so that a higher complexity increases CL. Extraneous CL is linked to the way the task is presented and to possible external confounders, such as distractions, noise, and instructional design. The easier the task is to understand the lower the CL becomes. Germane CL, on the other hand, reflects the cognitive effort invested in processing, understanding, and integrating new information. However, in recent perspectives [4,5], intrinsic and germane dimensions may be merged; therefore, intrinsic and extraneous should be considered as the most relevant components of CL [6].

Each activity that we complete in life demands a certain number of cognitive resources, which means CL is always experienced in daily life activities in various quantities. Activities of personal autonomy can be challenging for some populations, like older people, as they are usually dual-task and therefore produce more CL [7], which can hinder their health and reduce life expectancy [8]. Maintaining independence is crucial for overall health, as the loss of independence is associated with numerous chronic conditions [9], cognitive decline [10], and general disability [11]. Supporting vulnerable individuals thus represents one of the most pressing health challenges in promoting a positive quality of life [12]. With this in mind, there is strong interest in human–robot collaborations [13] as a potential tool to help the management of daily activities to maintain an individual’s independence [14].

Several systematic reviews, meta-analyses, and empirical studies have been conducted demonstrating that using robotics can support daily activities and, therefore, independence. For instance, Socially Assistive Robotics (SAR) can be utilized to support learning by increasing cognitive and affective outcomes, while achieving results similar to those of human tutoring [15]. They have also been utilized successfully in the treatment of Autism Spectrum Disorder by recognizing and responding to human social cues with structured rehabilitative behaviors [16]. SAR have also been reported to support the reduction in symptoms of depression by providing companionship and health-monitoring robots [17]. Robots have been also used to assess cognitive functions through robotic psychometric assessments administered interactively [18], to increase mobility in infants thanks to play-based, child–robot interactions [19], and to rehabilitate after a stroke with personalized robotic exercise coaching [20]. Finally, social anxiety has been reduced by using artificial agents for symptom management, social skills development, and improvement in overall quality of life in a cognitive-behavioral framework [21]. However, new healthcare technologies, while beneficial, have not been evaluated considering the role of CL, which can be either increased by the effort of using an additional tool, or reduced thanks to an actual supportive interaction.

Indeed, these technologies can significantly impact the CL of users as they often require processing multiple streams of information simultaneously, which can be challenging especially for those experiencing possible cognitive declines or frailty. Increased CL from complex interfaces can lead to stress and anxiety, potentially hindering the effective use of such technologies. For instance, systematic reviews revealed that the effective use of such tools with older adults with chronic diseases, like cognitive impairment, must consider at least five domains: demographic and socioeconomic, health-related, dispositional, technology-related, and social. Among the health-related factors, cognitive limitations due to aging are included, while technology-related factors deal with the ergonomics of interfaces [22]. Such factors impact directly on the intention to use new devices, either promoting or penalizing their integration in daily life, so they need to be carefully analyzed [23]. Therefore, the implementation of user-friendly, intuitive interfaces is crucial to enhance the usability and adoption of healthcare technologies. In this context, SAR have been increasingly studied to simplify tasks [24]. However, most of the research in this area has focused on collaborative robots (i.e., co-bots) in work settings [25], leaving a gap in understanding the use of technology for aiding in everyday life in a CL framework. After careful analysis of the literature, indeed, within the CL framework artificial agents were studied only as professional aids, for instance in surgery [26], industry [27], and construction [28]. Most of these contributions were either proof-of-concept or exploratory, and did not compare different measures and support conditions. Personal autonomy in daily life for vulnerable populations, finally, was not the main final goal.

Therefore, we conducted the current study with adults to preliminarily verify whether and how a supportive human–robot interaction (HRI) could impact on the performance and the perceived effort due a complex activity, while the same cognitive skills required for daily autonomy were increasingly elicited. Specifically, participants completed a cognitive task at three different levels of CL, while supported by a SAR (Figure 1). Number of errors, time of completion, and perceived fatigue were measured for rating the performance and the effectiveness of support. As an important addition, we also included a baseline where the task was executed without the support of SAR. Our overall goal was to understand under which conditions an external robotic support could be helpful for outcome improvement and mental demand management, hypothesizing that SAR are beneficial when compared to not assisted tasks, but only when user’s cognitive features are taken into account for a tailored interaction. Therefore, considering the lack of studies involving SAR in a similar framework, we explored how this technology can help independent living, and how this support impacts objective and subjective measures of CL.

2. Materials and Methods

2.1. Ethical Considerations

Ethical approval was obtained by Nottingham Trent University Ethics Committee with reference number 729 in March 2024. All participants signed an informed consent prior to the experiment. All procedures were run in accordance with the Declaration of Helsinki.

2.2. Sample

The sample consisted of 42 adult volunteers (17 males, 25 females, mean age: 54.3 ± 21.1 year). Participants were aged between 18 and 65 years and were enrolled in or were staff at Nottingham Trent University, in the UK. The older participants were community-dwelling inhabitants of the same urban area recruited by social media advertisement. Each person was asked to provide information on their health status, as the research involved healthy subjects only.

2.3. Tools

The cognitive task was inspired by the B form of Trail Making Test (TMT) [29]. TMT was chosen over other cognitive measures because it stresses several mental abilities, like attention, visual search, executive functions, and working memory. Therefore, it subjects the participant to a trial in which they use multiple skills simultaneously, resembling most daily activities. Our task, in particular, consisted of a 29.7 × 42 cm paper sheet displaying 52 circles spread out randomly. Half of the circles contained English letters from A to Z, while the other half showed numbers from 1 to 26 (Figure 2). Overall, four parallel forms (i.e., with different random dispositions of circles) of the same task were prepared for all participants, each corresponding to one of the four conditions of the study. Resulting objective performance parameters were number of correct/incorrect pairs joined, and time of execution.

As a supportive agent, the NAO 6 humanoid robot (United Robotics Group, Santa Monica, CA 90404, USA) was used. The reason for this choice is that it is a commercially available platform meeting our key requirements, including the ability to speak, listen, and engage in social interactions. Additionally, NAO was recognized as a social robot that tends to be well-received by older and vulnerable adults, as highlighted in prior research [14]. It is a 58 cm high social robot designed to interact with different interfaces, such as speech production, speech recognition, and limb gestures. It was programmed with Choregraphe software (version 2.8.7) and Python code to interact and collaborate with participants in three out of the four conditions of the study. Specifically, the robot’s behavior was structured through block-based visual programming with a node flow interface. The robot started with tracking the face of the participant and mimicking eye-contact, introducing itself, and giving the first instructions. The speech recognition function was also activated, so that the robot could repeat the initial explanations upon request. The first part of the interaction was meant as a vocal trial that varied according to human feedback. However, after that initial warm-up, the NAO did not repeat its instructions so as not to interfere with the cognitive task, but the listening function was continuously activated to regulate the pace and turn-taking of the interactions. The NAO moved on with the task when the participant stated “Okay”, or “Done”. Please refer to Figure 3 for additional information on the flowchart of interaction.

Perceived cognitive load (PCL) was measured with the NASA-TLX questionnaire [30]. It is a rapid self-administered psychometric tool investigating different dimensions: Mental Demand, Physical Demand, Temporal Demand, Performance, Effort, and Frustration. Each of them is rated by the respondent on a scale of 20 intervals, with higher scores indicating a higher strain. As NASA-TLX revealed optimal psychometric qualities over other alternatives [31], it was chosen as the best option for measuring subjective cognitive load in this study.

2.4. Procedure

Each participant completed the experiment individually through four conditions, one as a baseline and three with the robot at different cognitive load levels (low, medium, and high). Overall, the procedure lasted about 15 min. After signing an informed consent, participants were invited to sit at a desk with the first sheet.

In the baseline (first condition), the participants were asked to connect with a marker, pairs of one number, and one letter as fast as they could, starting from the last number and the last letter displayed on the paper and proceeding backwards until all the circles were joined. This setup was to elicit CL by engaging semantic memory, working memory, visual search, and processing speed. Indeed, the participant had to rapidly retrieve the information on backward alphabet and numbers, keep in mind that information, and look for the correct circles to connect.

In the following three conditions, the NAO robot was used as an interactive support. Indeed, at the beginning of the assisted conditions it was programmed to welcome the participants, explain the task, repeat the instructions upon request, and to provide verbally the pairs to join, requiring participants to only remember, search, and connect the correct circles, without any effort for retrieving them from the semantic memory (Figure 4). Therefore, the robotic conditions involved working memory, visual search, and processing speed only, allowing a direct comparison with the baseline that relied also on semantic memory, with a higher impact on CL. During the activity, NAO also waited for the participant to connect the circles and to say “Okay” or “Done” before saying the next pair. To avoid any learning effects, the pairs were constantly randomized. This was made in three different levels of cognitive load: in the low level (second condition), NAO provided one pair at a time; in the medium level (third condition), it provided two to four pairs at a time; and in the high level (fourth condition), it provided five to seven pairs at a time.

At the end of each condition, from baseline to high level with the robot, the NASA-TLX questionnaire was administered to measure PCL.

During the scoring phase, two independent experimenters calculated the number of correct pairs, the number of errors (wrong pairs + omissions), and the time of execution, as well as the scores obtained at NASA-TLX for each condition.

2.5. Statistical Analyses

Linear mixed models, considering participants as random intercepts, were performed to test for the effect of the task condition on performance (number of errors), time in seconds to complete the task and perceived cognitive load (NASA-TLX), which underwent Type III ANOVA. Sex and age were considered as covariates. Effect size has been estimated through partial eta squared (${\eta }_{\mathit{p}}^2$). Post hoc comparisons were performed on the estimated marginal means and corrected using the Bonferroni method. A power analysis was performed using G*Power to test for within-between interactions through ANOVA with repeated measures, considering a medium effect size, an alpha error of 0.05, and a statistical power of 0.80. With two groups and four measurements, the analysis indicated that a total sample size of 32 was required.

3. Results

Means (M) and standard deviations (SD) of errors, time, and PCL across the four conditions are displayed in

Table 1. Means (M) and standard deviations (SD) of errors, time, and perceived cognitive load (PCL) across conditions.

Condition	Errors (M, SD)	Time (s) (M, SD)	PCL (M, SD)
1. Alone: Baseline	2.12, 5.04	415, 117	49.8, 15.1
2. Robot: Low CL	2.05, 2.90	363, 113	42.1, 18.5
3. Robot: Medium CL	10.8, 4.50	357, 79.9	61.8, 15.6
4. Robot: High CL	20.3, 3.29	250, 61.9	69.9, 12.9

.

The task condition had a significant effect on participants’ performance (F(3, 123) = 251.06, p < 0.001, ${\eta }_p^2$ = 0.86), their time to complete the task (F(3, 123) = 36.11, p < 0.001, ${\eta }_p^2$ = 0.47), and their perceived cognitive load (F(3, 123) = 63.03, p < 0.001, ${\eta }_p^2$ = 0.61). While sex did not emerge as a significant covariate, age was found to be a significant covariate for each independent variable (F(1, 39) = 17.35, p < 0.001, ${\eta }_p^2$ = 0.31; F(1, 39) = 21.62, p < 0.001, ${\eta }_p^2$ = 0.36; F(1, 39) = 4.09, p = 0.04, ${\eta }_p^2$ = 0.10, respectively).

Post hoc analysis indicated that participants showed a decrease in performance with an increase in difficulty level. Results indicated significant differences between the low and medium levels (M_dif = −8.76, SE = 0.77, t(123) = −11.29, p < 0.001), the medium and high levels (M_dif = −9.50, SE = 0.77, t(123) = −12.24, p < 0.001), and the low and high levels (M_dif = −18.26, SE = 0.77, t(123) = −23.53, p < 0.001). No difference was found between the baseline and low level of cognitive load (M_dif = 0.07, SE = 0.77, t(123) = 0.092, p = 0.99).

Time to complete the task showed a significant reduction across conditions, except between the low and medium levels. Post hoc analysis indicated a significant difference between the baseline and low level (M_dif = 51.8, SE = 16.2, t(123) = 3.19, p = 0.009), no difference between the low and medium levels (M_dif = 5.9, SE = 16.2, t(123) = 0.36, p = 0.98), and a significant decrease between the low and high levels (M_dif = 106.6, SE = 16.2, t(123) = 6.56, p < 0.001).

Interestingly, participants reported a significant decrease in PCL between the baseline condition and the low level of the task (M_dif = 7.66, SE = 2.2, t(123) = 3.48, p = 0.003). However, the opposite was found for the medium level, which had a significantly higher PCL compared to baseline (M_dif = −12.03, SE = 2.2, t(123) = −5.46, p < 0.001) and the low level (M_dif = −19.69, SE = 2.2, t(123) = −8.95, p < 0.001). This increase in PCL further escalated at the high level, as indicated by the significant increase between the medium and high levels (M_dif = −8.07, SE = 2.2, t(123) = −3.66, p = 0.002).

Please refer to Figure 5 for the estimated means and standard errors.

4. Discussion

To the best of our knowledge, this was the first study to examine the impact of social robot-provided interactive support at different levels of cognitive load with a baseline condition in which the task was completed without interactive assistance. This comparison enabled us to establish a benchmark for understanding the actual impact of the support provided. Furthermore, as previously mentioned, this was the first comprehensive study to examine the role of a social robot in reducing CL in the context of independent living.

According to our results, when the task was completed alone there was no significant difference in performance (i.e., number of errors) compared to the robotic support at low CL. This similar accuracy, however, was achieved at the cost of increased execution time in the baseline. Indeed, in that condition the participant had to retrieve the information on correct pairs from the semantic memory, significantly slowing the completion of the task. This phenomenon is well documented in the CL literature, as demanding activities lead to a worse time performance, fatigue and possibly to compromised outcomes [32].

The effective help provided by the SAR was confirmed also by the decrease in perceived cognitive load between baseline and the low CL robot condition of the study. This could be due to two main factors. First, the SAR relieved the participant from the effort of retrieving information from the semantic memory. In an ecological scenario of daily life, a robotic support could remind the user of unfinished tasks or forgotten memories, so that the human would not need to strive to either to remember or keep in mind useful data. Second, the interaction with the robot, which gave the instructions only when the user said “Okay” or “Done”, probably allowed a more effective management of the task itself. Dividing a complex procedure into smaller chunks and increasing the learner’s perception of control on the whole process, indeed, reduces CL [33]. This means that the multimodal interaction with a SAR could be a means to rationalize tasks and promote a reduction in perceived effort.

However, our results showed also that CL interfered with task performance even when supported by SAR. This effect was evident through both objective measures (number of errors) and subjective assessments (perceived mental demand). Indeed, as CL increased, accuracy declined across conditions, cognitive effort rose, and execution time decreased. This last evidence is particularly important. While the reduction in time from the baseline to the SAR condition under low CL was accompanied by maintained accuracy and lower perceived cognitive load—indicating more efficient task management—the subsequent apparent speed of execution was linked to poorer performance and greater effort. This suggests that participants developed what in experimental psychology is called “learned helplessness”, where individuals cease attempting to resolve a difficult situation due to repeated failure and a perceived lack of control [34]. In our case, participants prematurely ended the task by saying “Okay” to the robot without completing the pairing, as the effort became overwhelming and information could not be retained.

The increase in CL at higher difficulty levels with robotic support is a crucial finding in our study. This result is particularly significant given that potential confounding variables were accounted for to minimize their impact. For example, all participants had no prior interaction with social robots and demonstrated normal cognitive functioning. Therefore, although SAR can support daily activities [35], our study indicated that SARs are effective in aiding tasks only when interactions are tailored to consider the user’s cognitive effort. Indeed, excessive CL nullifies any advantage provided by such support. This is particularly important given that research has shown that older adults, for example, experience greater cognitive load than younger adults when performing tasks that require significant mental effort, highlighting the need for tailored interventions to effectively support certain populations [36].

Indeed, all partial eta squared values exceeded 0.14, indicating that a substantial proportion of the total variance in the dependent variables was explained by the independent variable. This demonstrates that the level of CL and its management has a practical and significant impact on parameters such as performance, time, and fatigue. The pursuit of effective support strategies represents a real challenge in daily life, rather than being merely a statistical observation.

These findings open research directions. It is important to identify which aspects of human–robot interaction affect CL, so that SAR can be implemented effectively. Additionally, it is crucial to understand how to design artificial agents to measure the cognitive effort of users in real-time. According to our results, this goal could be reached by developing systems that track the number of errors and amount of time during a task of a given difficulty. It would allow for the adjustment of HRI parameters, reducing stress and improving performance. This approach would promote a person-centered strategy in assistive robotics that considers psychological variables.

Indeed, even if SAR have been demonstrated to be effective tools to screen and promote users’ health conditions and quality of life, it is still the human interactor who must understand, learn, and adapt to the artificial agents’ HRI features. For this reason, SAR is often a limited strategy for actual support, as it lacks flexibility and personalization. Specific characteristics, therefore, are more an obstacle rather than a valuable source of information. For an autonomous system supporting independent living, it is essential to consider human needs and preferences dynamically. Recent studies moved towards that direction, for instance by creating real-time multimodal human–robot collaboration with gestures and speech [37].

Despite the interesting findings, this study has some limitations, such as the small sample size, the use of one robotic model only, the lack of a comparison with human–human interaction, and the absence of biological markers of cognitive load that could have corroborated the subjective perception of participants. The concept of “learned helplessness”, even if scientifically convincing in this context, would need a qualitative exploration for being confirmed. In addition, our study should be extended to other younger and older participants, as our findings indicated age as a significant covariate, and the sample seemed skewed towards middle-aged adults. Therefore, results could not be generalized well to young and elderly subjects. However, even if these aspects were relevant, the use of a baseline offered an important benchmark to draw useful results on the role played by SAR.

5. Conclusions

This study contributed to understanding the impact of supportive robotic interactions in the management of CL, and how mental effort can influence the performance in assisted tasks. The robot was demonstrated to be an effective tool when compared to the condition without help, with decreased time of execution and reduced PCL. However, with higher levels of CL, this positive effect was lost leading to worse performance, fatigue, and probable learned helplessness.

Our findings could allow the development of a technology used in the daily activities of vulnerable populations. A SAR as a companion that can provide dynamic CL-based supportive cues for activities like cooking, dressing, planning a routine, and executing general tasks according to the user’s needs is the scenario we envision. The robot could learn from previous interactions and could give the human periodical reports on their autonomy.

Future studies should explore how machine learning can adapt SAR behaviors based on user feedback and performance metrics, creating a responsive and evolving interaction model. Additionally, using multimodal feedback mechanisms, like visual and auditory cues, could improve SAR effectiveness in reducing CL and enhancing task performance. This could prevent overwhelming users in high-load conditions. By addressing these challenges, researchers can develop personalized solutions that better support daily activities and promote independent living.