Next Article in Journal
Comparison of Machine Learning Approaches for Robust and Timely Detection of PPE in Construction Sites
Next Article in Special Issue
Got It? Comparative Ergonomic Evaluation of Robotic Object Handover for Visually Impaired and Sighted Users
Previous Article in Journal
Design and Analysis of VARONE a Novel Passive Upper-Limb Exercising Device
Previous Article in Special Issue
Are Friendly Robots Trusted More? An Analysis of Robot Sociability and Trust
 
 
Search for Articles:
Title / Keyword
Author / Affiliation / Email
Journal
Article Type
 
 
Advanced Search
 
Section
Special Issue
Volume
Issue
Number
Page
 
Journals
Robotics
Volume 13
Issue 2
10.3390/robotics13020030
robotics-logo
Submit to this Journal Review for this Journal Propose a Special Issue
► Article Menu

Article Menu

share Share announcement Help format_quote Cite question_answer Discuss in SciProfiles thumb_up
...
Endorse textsms
...
Comment
first_page
settings
Order Article Reprints
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Review

What Affects Human Decision Making in Human–Robot Collaboration?: A Scoping Review

by
Yuan Liu
1,2,
Glenda Caldwell
1,2,*,
Markus Rittenbruch
1,3,
Müge Belek Fialho Teixeira
1,2,
Alan Burden
1,2 and
Matthias Guertler
1,4
1
Australian Cobotics Centre, Brisbane, QLD 4000, Australia
2
School of Architecture and Built Environment, Faculty of Engineering, Queensland University of Technology, Brisbane, QLD 4000, Australia
3
School of Design, Faculty of Creative Industries, Education and Social Justice, Queensland University of Technology, Brisbane, QLD 4059, Australia
4
School of Mechanical and Mechatronic Engineering, Faculty of Engineering and Information Technology, University of Technology Sydney, Sydney, NSW 2007, Australia
*
Author to whom correspondence should be addressed.
Robotics 2024, 13(2), 30; https://doi.org/10.3390/robotics13020030
Submission received: 30 December 2023 / Revised: 3 February 2024 / Accepted: 6 February 2024 / Published: 9 February 2024
(This article belongs to the Special Issue Human Factors in Human–Robot Interaction)
Browse Figures
Versions Notes

Abstract

:
The advent of Industry 4.0 has heralded advancements in Human–robot Collaboration (HRC), necessitating a deeper understanding of the factors influencing human decision making within this domain. This scoping review examines the breadth of research conducted on HRC, with a particular focus on identifying factors that affect human decision making during collaborative tasks and finding potential solutions to improve human decision making. We conducted a comprehensive search across databases including Scopus, IEEE Xplore and ACM Digital Library, employing a snowballing technique to ensure the inclusion of all pertinent studies, and adopting the PRISMA Extension for Scoping Reviews (PRISMA-ScR) for the reviewing process. Some of the important aspects were identified: (i) studies’ design and setting; (ii) types of human–robot interaction, types of cobots and types of tasks; (iii) factors related to human decision making; and (iv) types of user interfaces for human–robot interaction. Results indicate that cognitive workload and user interface are key in influencing decision making in HRC. Future research should consider social dynamics and psychological safety, use mixed methods for deeper insights and consider diverse cobots and tasks to expand decision-making studies. Emerging XR technologies offer the potential to enhance interaction and thus improve decision making, underscoring the need for intuitive communication and human-centred design.

1. Introduction

As a socio-technical system, collaborative robots (cobots) are designed to improve productivity, flexibility, and ergonomics, and to increase customised production rather than mass production. Since the fourth industrial revolution (Industry 4.0), concerns regarding the human workers’ role in the production environment [1] have increased causing Human–robot Collaboration (HRC) to become an emerging area of robotic and cobotic research in recent years. A popular discussion about the next industrial revolution (Industry 5.0) is human–robot co-working [2], which emphasises bringing human workers back to the production process loop [3]. An HRC system combines human soft skills such as decision making, intelligence, problem-solving, adaptability and flexibility with robots’ precision, repeatability, and the ability to work in dangerous environments [4].
For this reason, cobots are adopted to work and interact safely with humans on shared tasks, in a shared workspace simultaneously [5,6,7]. Cobots have enormous potential for their increased use in many industries. To introduce industrial cobots clearly, [8] proposed a framework which categorises the interaction between humans and robots into four types (Figure 1). First type is the full automatization of conventional industrial robots, and the latter three types are categorised based on the interaction between humans and cobots: coexistence, cooperation, and collaboration. In the coexistence scenario, humans and cobots work sequentially in different workspaces. In the cooperation scenario, the humans and cobots work in a shared space, and the tasks of the humans and cobots are linked. In the collaboration scenario, which is the highest level of interaction, humans and cobots work in the shared space on the shared tasks simultaneously.

1.1. Human Decision Making

Since humans and robots can work together as a team in HRC, they both have the authority to make decisions. Robots can make decisions by using algorithms based on models such as Markov decision processes (MDP) [11], partially observable Markov decision processes (POMDP) [12], Bayesian Decision Model (BDM) [13], Adaptive Bayesian policy selection (ABPS) [14], and others. Therefore, the performance of robot decision making relies on algorithms. With the emergence of technologies such as Artificial Intelligence (AI) and Machine Learning (ML), robots can make decisions on their own and be more intelligent in some specific situations [15]. Different from robots, humans make decisions based on normative inference, influenced by their previous experiences, unconscious drives, and emotions [16]. Robots are good at making decisions in stable and predictable situations, while human decision making is essential for handling complex and dynamic situations [17].
One example of a dynamic situation where humans are working with robots is robot-assisted surgery. In this scenario, a surgeon makes decisions based on their knowledge, experience, and the current situation in the operation theatre. Other factors such as communication within the surgical team, situation awareness, and workload are related to the surgeon’s decision making [18]. For instance, during a procedure, a surgeon must carefully choose the right instrument based on the specific needs of the task and the current conditions. They must evaluate the completion of each step satisfactorily, be ready with an alternative plan if necessary, and always consider the most appropriate next action. The quality of the surgeon’s decisions regarding patient care, incision placement, and procedure steps directly affects the overall success of the surgery. Another dynamic example is a search and rescue task. In this scenario, humans make decisions based on the data collected from a robot where effective information can aid humans to make decisions necessary for the search. For instance, in an Urban Search and Rescue (USAR) task [19], robots provide data continuously to human operators, and human operators analyse the data, update search strategy, and reassign tasks to ensure the efficiency and safety.
Therefore, collaboration between humans and robots necessitates that human operators apply their expertise to make situationally appropriate decisions. This decision making process aligns with the concept of the Naturalistic Decision Making (NDM) theory proposed by Klein et al. in 1993 [20]. This theory elucidates decision-making processes in environments that are both significant and familiar to humans, portraying them as expert decision makers with domain-specific knowledge and experience. Klein et al. further introduced the Recognition-Primed Decision making (RPD) model as a framework for understanding how effective decisions were made [20,21].
This model starts with an assessment of whether the situation is familiar. If it is not, the individual seeks more information and re-assesses the situation. If the situation is familiar, the model predicts that the individual will have expectancies about what is normal for that situation. If these expectancies are not violated, the individual engages in a mental simulation of the action, essentially predicting the outcome of an action without actually performing it. If the mental simulation suggests that the action will succeed, the individual implements the action. If not, they modify the plan and re-evaluate its potential success through another mental simulation. This process repeats until a workable plan is formulated.
RDP model, along with the NDM theory, has been applied to various real-world domains, including Unmanned Air Vehicles (UAV) operations by Yesilbas and Cotter in 2019 [22] and human–agent collaboration by Fan et al. in 2005 [23], demonstrating its broad applicability.

1.2. Cognitive Workload and Human Decision Making

In human decision making, cognitive workload plays a critical role, especially in environments where humans interact with complex systems or technology, such as robotics [24]. This factor is frequently examined alongside human decision making due to its profound impact on performance and outcomes [25]. Effective decision making is a complex cognitive process that necessitates the optimal distribution of an individual’s attention and mental capacity. The quality of decisions heavily relies on the ability to analyse information, evaluate possible outcomes, and choose the best course of action [26,27]. However, cognitive overload can significantly impede this process. When an operator faces an excess of information or task demands that exceed their cognitive resources, they are likely to experience mental fatigue [28,29], which can lead to reliance on simplifying strategies, known as heuristics. While heuristics can be useful for quick judgments, they often ignore much of the available data and the nuance required for high-quality decision making [30,31].
For example, using tools designed to minimize cognitive effort in pattern recognition can significantly improve group-decision outcomes by allowing better resource allocation decisions in dispersed groups [32]. Similarly, the frontal network in the brain responds to uncertainty in decision making tasks by modulating cognitive resource allocation, and the cognitive control in navigating uncertain situations is important [33]. In complex operations, cognitive readiness, which includes situation awareness, problem-solving, and decision making, is essential for effective resource allocation. In addition, supporting long-term anticipation in decision making can significantly improve performance in complex environments, and cognitive support tools can enhance the anticipation of future outcomes [34].
For this reason, this scoping review is conducted in order to identify the factors that affect human decision making and cognitive workload during human–robot collaborative tasks. The research questions guiding this research are as follows:
RQ1. What are the factors that impact human decision making and cognitive workload during HRC tasks?
RQ2. What are the potential solutions to address these factors?

2. Methods

This Scoping review is guided by PRISMA Extension for Scoping Reviews (PRISMA-ScR) [35]. Papers selected for this scoping review focused on human decision making and cognitive workload in HRC tasks. Peer-reviewed journal articles, conference papers and book chapters were included if they were published from 2019 to 2023 and written in English. Quantitative, qualitative, and mixed-method studies were included in order to consider different aspects of human decision making and cognitive workload in HRC. Papers were excluded if they did not fit into the theme of this study, which focused on autonomous mobile robot, telerobot, AI bot, social robot, military robot, machine learning, reinforcement learning, robot decision making, algorithmic decision making, decision making modelling, robot’s planning process, conversation or voice agent and design process (Table 1).
Due to the multidisciplinary and state-of-the-art nature of the topic, the search was conducted in the following different databases from 2019 to November 2023: Scopus, IEEE Xplore and ACM digital library. As listed in Table 2, keywords were identified in two sets. The first set of keywords was related to human–robot collaboration, and the second set of keywords is related to human decision making and cognitive workload. The Boolean operator “AND” is used between the two sets, and the “OR” operator was used within each set. Searches covered titles, abstracts, and full texts in the databases. The search strategy was drafted by Yuan Liu and further refined through team discussion. The final search strategy is (“cognitive workload” OR “human decision”) AND (“human–robot collaboration” OR cobot* OR “collaborative robot*” OR “human–robot interaction”). The final search was conducted on November 8, 2023, and search results were exported into Paperpile, and duplicates were removed by the Paperpile filter.
After the removal of duplicates, two reviewers screened the same 271 publications according to the title and abstract. After that, the full text of the selected records was scanned to remove publications that were inconsistent with the research questions. Each exclusive criterion was discussed among all authors. To achieve comprehensive coverage of related and up-to-date studies, 5 relevant publications were snowballed. The Preferred Reporting Items for Systematic Reviews and Meta-Analysis (PRISMA) chart (Figure 2) was adopted to illustrate the steps of paper identification and selection.
The data was charted in three phases, phase (1) main characteristics, phase (2) study design and setting, and phase (3) findings related to the research questions. For main characteristics, data including title, year of publication, country, and study aims were extracted (Appendix A). For study design and setting, methodology, sample size, experiment setting and measures for assessment were extracted. Finally, the key findings related to research questions including factors related to human decision making and cognitive workload, types of human–robot interaction, cobots, tasks and interfaces, and potential solutions were identified.

3. Results

The search for literature was conducted in November 2023. A total of 41 publications were included after the screening process, 5 publications were snowballed, and a final of 46 studies were included (Appendix A).

3.1. Study Characteristics

As shown in Figure 3, the selected papers were published between 2018 and 2023. The majority were published in the last three years (n = 31, 67.4%). Most papers are peer-reviewed journal articles (n = 26), 19 are conference papers, and only 1 is a book chapter (Figure 4). Most of the papers were conducted in the USA (n = 17, 37.0%) and Italy (n = 15, 32.6%). The rest were conducted in Germany (n = 3), Switzerland (n = 3), Netherlands (n = 2), China (n = 2), Sweden (n = 1), Mexico (n = 1), Portugal (n = 1), and India (n = 1) (Figure 4).

3.2. Study Design and Setting

3.2.1. Study Settings

As Table 3 shows, among all the studies, most studies were conducted in the laboratory setting (n = 28, 60%), the others were literature review (n = 9, 20%), studies conducted in the virtual environment (n = 5, 11%), a study through an online survey (n = 1, 2%), and a study conducted in realistic scenarios (n = 1, 2%). A total of 12 studies involved participants less than 20, 5 studies involved 20–50 participants, and 3 studies involved participants more than 50.

3.2.2. Measures

Among all the selected studies, subjective measures and biometric measures were the main tools used to measure human factors. In total, 14 studies only used subjective measurements, 5 studies only adopted biometric measurements, and 10 studies applied both (Table 3).
For subjective measurement methods from the studies listed in Table 1, the NASA Task Load Index (NASA-TLX) stood out as a golden standard for workload and the most utilized measure employed in 17 studies. Other methods include various questionnaires for assessing Fluency Perception, Technology Acceptance, System Usability, and User Experience. Additional techniques such as direct observation, video recording analysis, semi-structured interviews, the System Acceptance Scale for measuring system acceptance, the Short Stress State Questionnaire (SSSQ) for measuring stress, and the Trust in Industrial Human–robot Interaction Scale were also applied. Furthermore, preferences for automation levels, Likert-scale surveys, diverse pre- and post-experiment questionnaires, verbal reports, and online surveys also contributed to the subjective measurements in several studies.
In the selected studies, a diverse array of biometric measurements was employed to analyse human physiological and behavioural responses. Eye-tracking emerged as the most frequently used method, applied in six studies to assess visual attention and physiological reactions. Electroencephalography (EEG) was a primary tool for monitoring brain activity, while Electromyography (EMG) and Heart Rate Variability (HRV) provided insights into muscle actions and heart rhythms. Other methods included Electrodermal Activity (EDA) for skin conductance, Wrist Motion (MOV), Skin Response, and Motion Capture for movement analysis. Additionally, functional Near-Infrared Spectroscopy (fNIRS), along with assessments of Upper-limb Kinematics and Muscle Activity, contributed to a comprehensive set of biometric evaluations (Table 3).

3.3. Human–Robot Interaction

3.3.1. Types of Interaction and Tasks

As illustrated in Figure 5, among all the studies, most of the interaction type between humans and robots is collaboration (n = 34, 72.9%), the others are cooperation (n = 8, 16.7%), coexistence (n = 3, 6.2%), and teleoperation (n = 2, 4.2%).
Figure 5 shows that the studies reviewed primarily focused on assembly tasks [38,39,48,53,54,57,58,61,63,64,72,73,76,77,80] with other frequent activities including pick-and-place [44,55,60,65,79], screwing [41,70,71], handling [38,41], sorting [51,79], lifting [41,49], transferring [36,70], and surgical procedures [47,69]. These tasks represent the core applications of collaborative robots in current research. Additionally, a range of specialized tasks such as surface polishing [38], welding [38], guiding [66], inspecting [70], weighing [70], and search-and-rescue [46] were explored. Innovative applications extended to collect-and-delivery [52], space exploration, wheelchair control, autonomous driving [47], matching [75], testing, and tracking moving objects, showcasing the diverse potential of cobots across various industries.

3.3.2. Types of Cobots

As listed in Table 4, among all the studies, 6 degrees of freedom (6-DoF) cobots were the most used (n = 16). Other types of cobots were 4 degree of freedom cobots (4-DoF) (n = 1), 7 degree of freedom (7-DoF) cobots (n = 6), surgical robots (n = 1), dual-arm robots (n = 3), and virtual robots (n = 1). The majority of studies on 6-DoF cobots predominantly featured various models from Universal Robots, including the UR3, UR5, and UR10, along with their enhanced versions. The UR3 and its iterations were the most cited, followed by the UR5 and UR10 series. In addition to these, SCHUNK’s PowerBall LWA 4P was also mentioned. For 7-DoF cobots, models like the KUKA LWR4+, Franka Emika Panda, JACO robotic arm, and UFACTORY xArm 7 were utilized, with some studies exploring dual-arm coordination using two KUKA LWR4+. Dual-arm cobots like Baxter and ABB YuMi were used to simulate more complex interactions. Other unique cobot types included a 4-DoF cobot, the surgical robot da Vinci, and virtual robots in simulated environments, indicating a diverse range of robotic platforms used in HRC research.

3.4. Factors Related to Human Decision Making

Within the selected studies, 24 factors that influence human decision making during tasks were identified. Generally, these factors affecting human decision making are categorized into four groups: human factors, robot factors, communication factors, and environmental factors. (Figure 6, Table 5).

3.4.1. Human Factors

In the realm of human factors impacting decision making in HRC, cognitive workload receives the most attention, as evidenced by its focus in 18 studies. Trust, operator ability, human characteristics, and acceptance are also prominent, having been extensively studied across multiple research works. Additionally, physical workload, stress levels, and emotional responses such as satisfaction or frustration are deemed critical. These factors, together with the perception of the situation and environment, play significant roles in influencing human decision making during collaborative tasks with robots.

3.4.2. Robot Factors

Robot factors that influence human decision making in collaborative tasks encompass a spectrum of the robot’s physical characteristics and actions. Key aspects such as the force exerted by the robot, its speed, and the distance maintained from human operators are crucial, directly impacting task performance and safety protocols. Additionally, the frequency and nature of robot errors, as well as the trajectory and movement patterns, are vital considerations. The role of the robot, whether as a leader or a follower, alongside the degree of automation implemented, plays a significant part in shaping the human–robot interaction dynamic.

3.4.3. Communication Factors

Communication factors include user interface design, control modality, feedback mechanisms, usability, human intent prediction, and mutual awareness. These elements facilitate interaction and are foundational for intuitive operation and effective teamwork between humans and robots, as evidenced by numerous studies. The user interface is particularly emphasized, as it directly impacts the efficiency and satisfaction of the operator. Feedback and mutual awareness are also integral, ensuring that both humans and robots can respond adaptively to each other’s actions and intentions.

3.4.4. Environmental Factors

Environmental factors include the dynamics of the situation, task complexity, workspace design, and physical safety. Among the selected studies, task complexity received the most emphasis. The design of the workspace was also deemed crucial, while the dynamics of the situation and physical safety were likewise noted as important considerations.

3.5. User Interface

Figure 7 shows that, among all selected studies, the Graphical User Interface (GUI) [37,39,46,54,57,60,63,66,69,71,72,73,75,77,80] was the most utilized type of user interface, implemented in 15 studies. Other interfaces include physical interaction [49,57,69,71,72,73,77] and touch interfaces [46,55,65,71], which facilitate direct engagement with robots. Additional methods such as gesture recognition [38,39,63,65,72,73], voice control [36,38,63,64,65], and eye gaze tracking [65,68] were employed, enhancing the versatility of interactions. Haptic feedback [62] and extended reality (XR) interfaces [38,39,46,52,59,60,63] were noted for their immersive and tactile capabilities. Furthermore, EMG-based interfaces [37,39] and Brain–Computer Interfaces (BCIs) [36,39,40] have been adopted, indicating progress in user command and collaboration with robots. Beyond the commonly used GUI, XR-based interfaces have garnered more attention than other emerging technologies.

3.6. Technologies Related to Human Decision Making

Numerous studies have highlighted a variety of advanced technologies such as XR, gesture control, voice control, and AI-based perception, which significantly influence human decision making in collaborative environments.
Some studies mentioned a multimodal interface which combines different sensory input to improve interaction to enhance human decision making and collaboration. For example, gesture recognition [38,57,63,65,72], speech recognition [38,63,65], cognitive signals like EEG [36], ECG [55], fNIRS [75], and force sensory [36] were used in some studies.
XR-base interfaces including AR [38,52,59,63], VR [46], and MR [60] were discussed in several studies. These technologies can help improve human decision making in HRC by assisting with visual and other perceptions.
Other technologies such as Brain–Computer Interface (BCI) [40], eye gaze [65,68], and AI-based perception [57,72] were also mentioned. Such technologies have the potential to reduce cognitive workload and enhance users’ preference and needs therefore improving their decision making.

4. Discussion

This scoping review aims to identify the factors related to human decision making during HRC tasks and find potential solutions to reduce cognitive workload and improve human decision making, therefore enhancing human–robot collaboration.

4.1. Key Findings

For the study design and setting, almost all the selected studies were conducted in laboratory settings and virtual environments, and only 1 study was conducted in realistic scenarios. While laboratory studies are valued for their controlled conditions, which ensure repeatability and safety, they frequently do not fully encapsulate the complex and dynamic nature of real-world environments. The simplified settings within a laboratory may overlook essential variables that naturally occur outside of these controlled environments. This oversight can result in significant gaps in the applicability of research findings, as studies may not accurately reflect the complexities and unpredictability encountered in everyday situations. Consequently, this limitation can lead to a disconnect between theoretical research outcomes and their practical implementation in real-world scenarios, underscoring the need for complementary field studies to validate laboratory findings.
Besides, most of the studies used 6-DoF robot. However, the integration of diverse cobots like exoskeletons and dual-arm systems is crucial for creating more adaptable, efficient, and human-centric collaborative environments. Furthermore, a multimodal human–robot collaboration, which combines single-arm, dual-arm, and wearable robots, can cater to a broader range of tasks and environments, ranging from the precision of assembly lines to the unpredictable conditions of construction sites or the sensitive needs of patient care.
The specific tasks performed among all the studies were mostly assembly (34%) and pick-and-place (11%), and other tasks were less frequent (7% screwing, 7% handling, 4.5% sorting, 5% lifting, 5% transferring, 5% surgery, 2% surface polishing, 2% welding, 2% guiding, 2% inspecting, weighing, 2% search-and-rescue, 2% collect-and-delivery, 2% space exploration, 2% wheelchair control, 2% autonomous driving, 2% matching, 2% testing, 2% tracking moving object).
Regarding the methods of measurement in all the selected studies, 14 studies employed only subjective methods, 10 studies utilized both subjective and biometric measures, and 5 studies relied exclusively on biometric measurements. Among the subjective measures, the NASA-TLX was the most frequently used, appearing in 17 studies. Other methods were employed less frequently: 3 studies implemented the System Usability Scale, 2 employed the Short Stress State Questionnaire (SSSQ), 2 measured trust using the Trust in Industrial Human–robot Interaction Questionnaire, 1 measured User Experience, 1 used direct observation, 1 analysed video recordings, 1 conducted semi-structured interviews, 1 used the System Acceptance Scale, 1 used a questionnaire on the Preference of Automation Level, 1 used a Likert-scale survey, 1 study used a questionnaire on fluency perception, 1 used the Technology Acceptance Model (TAM) questionnaire, 1 used various pre- and post-experiment questionnaires, 1 collected verbal reports from participants, and 1 conducted an online survey for participants to rate the design of an AR interface. Among the biometric measures utilized in research, eye-tracking was the most frequently employed, being featured in 6 studies. Heart Rate Variability (HRV) followed with 5 studies, and Electroencephalography (EEG) with 4. Other methods were used less frequently, with 1 study each employing Electromyography (EMG), Electrodermal Activity (EDA), Metal Oxide Varistor (MOV), skin response, motion capture, functional Near-Infrared Spectroscopy (fNIRS), upper-limb kinematics, and muscle activity.
In the realm of human decision making, cognitive workload is the most extensively discussed factor, featuring in 39% of the studies; it significantly influences decision making during tasks [45]. User Interface (UI) was highlighted in 24% of the studies as a relevant factor, while human trust was mentioned in 13%. Furthermore, operator’s ability, feedback, task complexity, human characteristics, and robot attributes were each the focus of 13% of the studies. Lesser discussed factors include mutual awareness, acceptance, physical workload, robot errors, automation level, and control modality, each of them addressed in 8% of the studies. Human stress, robot movement and trajectory, robot roles, and workspace design were each considered in 7% of the studies. Usability, intent prediction, user satisfaction or frustration, and situation perception were featured in 4% of the studies. Lastly, the dynamics of the situation and safety of the physical environment were noted in 2% of the studies. While current research has considered factors from various perspectives, some aspects have not been mentioned. For example, these include the social aspect, multi-human–robot teams, psychological safety, cultural sensitivity, and others. Social factors and perceptions of psychological safety are crucial elements in HRC. Integrating these elements into the design and collaborative processes of HRC can enhance teamwork and productivity, as well as ensure the smooth adoption of robotic technologies.
Among all the studies, the conventional Graphical User Interface (GUI) was the most used, featuring in 15 studies. Emerging interfaces, such as XR-based interfaces, were present in 7 studies, as were physical interfaces. Gesture recognition was included in 6 studies, voice control in 5, touch interfaces in 4, Brain–Computer Interfaces (BCIs) in 3, and eye gaze tracking in 2. Additionally, EMG-based interfaces were explored in 2 studies, while foot interfaces and haptic feedback were each the subject of 1 study. Several studies mentioned multi-model interfaces.

4.2. Potential Solutions

To improve human decision making, some studies suggested potential solutions. Clear and intuitive communication was the most suggested solution. Norton et al. suggested that clear and effective communication of a robot’s proficiency to human partners can improve decision making and manage cognitive workload [44]. Ajoudani et al. suggested the use of shared communication modalities and robotic learning techniques for gradual mutual adaptation [37]. Villani et al. suggested developing user-friendly interfaces with the application of cognitive engineering principles [38]. Intuitive interaction and well-designed interfaces were also suggested as a solution in several studies [42,57,59]. However, Kalatzis et al. indicated that while user interfaces can aid in HRC tasks, their complexity (especially in mixed reality UIs) can increase cognitive workload and negatively impact task performance, situational awareness, and trust [60]. Designers should consider these factors and potentially develop less cumbersome and more intuitive user interfaces to enhance HRC. Besides, Saren et al., indicated the need for careful selection of modalities based on the task complexity and situational context [65].
Appropriate data visualisation and feedback is another potential solution. Szafir suggested applying data visualization to the design of robot interfaces [46]. Cleaver et al. claimed that appropriate visualization of the robot’s path can improve human decision making and reduce cognitive workload by providing enough information without overwhelming the user [52]. Zhou et al. suggested that intraoperative workload feedback and user-driven autonomous assistance based on cognitive load sensing could improve human decision making and reduce cognitive workload [69]. Lemasurier et al., suggested that effective signalling of robot intent, like the LED bracelet, can be a solution [51]. Zhu et al. suggested a haptic shared control architecture, which integrates human and robot control with fuzzy logic inference in tracking tasks [62].
Human-centred design is also discussed and suggested. For example, considerations for a more human-centred HRC design [17]; understanding the bidirectional nature of anticipatory actions and improving the accuracy of intent prediction [68]; using simulations to predict and plan for human behaviour and developing tools for coupling human models [50]; designing with consideration for operator states, appropriate task allocation, and the use of adaptive interfaces [45]; and designing collaborative system features that can adapt to individual user preferences and needs [72].
Adapting robots’ behaviour or role to humans in real-time is a dynamic approach. Lagomarsino et al., suggested adapting the robot’s trajectory based on real-time assessment of human cognitive load [55]. Bales and Kong suggested optimizing the information presentation and interaction patterns between humans and robots based on real-time cognitive state assessments [74]. Hostettler et al., suggested adapting the speed of the robot to individual users [78]. In addition, adjusting the robot’s role in real-time based on cognitive workload was discussed in several studies [76,80].
Other suggestions are integration of trust [48], safety [42,57,73], visibility and predictability of robot actions [57], and ensuring user acceptance [77].

4.3. Limitations

This study has several limitations. It reviewed only papers published in English; future studies would benefit from including articles in non-English languages for a more comprehensive analysis. Furthermore, the literature search was confined to three databases: Scopus, IEEE Xplore, and ACM Digital Library. To obtain more exhaustive results, expanding the search to additional databases is recommended for subsequent research.

5. Conclusions

This scoping review investigates current research on factors that may impact human decision making while collaborating with robots during tasks, and it discusses the potential solutions to address these factors. Human factors and communication factors are the most frequently discussed topics. Within human factors, cognitive workload receives the most attention, highlighting its significant impact on decision-making processes. As for communication factors, the design of the user interface is identified as the most influential factor affecting human decision making. However, there is a lack of attention on other aspects, such as social aspects, human–robot teams, psychological safety, and cultural sensitivity.
The majority of the studies were conducted in laboratory settings. Future studies should be expanded in realistic scenarios to investigate the complexity and dynamic situations in the real world. The assessment of cognitive workload was conducted using subjective and biometric measurements. The most frequently used subjective approach is the NASA-TLX, while eye tracking, EEG, and HRV are the most utilized biometric measures. However, other factors, including trust, stress, acceptance, fluency perception, and user experience, were assessed only through subjective approaches. Future studies could employ a mixed-methods approach, incorporating both subjective and objective measures to assess these aspects more comprehensively.
Related to human decision making and cognitive workload investigation, the most frequently used cobots were 6-DoF single arms, primarily for assembly and pick-and-place tasks. Future research would benefit from incorporating a wider variety of cobots, such as surgical robots, dual-arm robots, and those with a higher DoF. Moreover, the scope of tasks should extend beyond manufacturing to include other industries like agriculture, medicine, and health care. Investigating a broader range of tasks, such as surgery, welding, surface polishing, and agricultural activities like spraying and harvesting, would be advantageous.
Among the emerging user interfaces, XR-based interfaces are a growing trend. Technologies such as gesture recognition, voice control, Brain–Computer Interfaces (BCIs), and eye tracking hold significant potential for application. As for suggested solutions, they encompass intuitive and clear communication, appropriate data visualization, meaningful feedback, the real-time adaptation of robot behaviour, and human-centred design principles, all of which are integral to the design of user interfaces. These novel interaction methods can significantly enhance human–robot interaction by facilitating more intuitive and human-like communication. For instance, when robots can understand and respond to human cues in a way that mirrors human-to-human interaction, it reduces the cognitive load on humans, allowing them to make decisions more effectively.
In summary, cognitive workload and user interface design are the most prominent factors influencing human decision making in HRC tasks. However, there is a notable lack of research on social dynamics and psychological safety. Future studies would benefit from realistic scenarios with context-rich data and real-world applicability. Furthermore, a mixed-methods approach, integrating subjective and objective measures would be used to provide a more comprehensive understanding. Additionally, exploring a variety of cobots and task scenarios will broaden the scope of human decision-making research in the domain of HRC. Furthermore, XR-based interfaces and related technologies represent an emerging trend with significant potential to improve user interaction, emphasizing the importance of intuitive communication and human-centred design in the design and development of future interfaces.

Author Contributions

Conceptualization, Y.L., G.C., M.R. and A.B.; methodology, Y.L., M.R., M.B.F.T. and M.G.; writing—original draft preparation, Y.L.; writing—review and editing, G.C. and A.B.; supervision, G.C., M.R., M.B.F.T., A.B. and M.G. All authors have read and agreed to the published version of the manuscript.

Funding

The authors would like to acknowledge the support received through the following funding schemes of the Australian Government: ARC Industrial Transformation Training Centre (ITTC) for Collaborative Robotics in Advanced Manufacturing under Grant IC200100001.

Data Availability Statement

No new data were created in this study. This scoping review analysed studies and articles sourced from databases such as Scopus, IEEE Xplore, and ACM Digital Library, all sources cited in this review are publicly available and can be accessed through the respective databases.

Conflicts of Interest

The authors declare no conflict of interest.

Appendix A

Table A1. Studies’ main characteristics.
Table A1. Studies’ main characteristics.
Ref.Year TypeTitleCountryAims
[37]2018Journal ArticleProgress and prospects of the human–robot collaborationItalyReview the current state-of-the-art in HRC, focusing on intermediate human–robot interfaces, robot control modalities, system stability, benchmarking, and relevant use cases.
[48]2018Journal ArticleCollaborative assembly in hybrid manufacturing cells: An integrated framework for human–robot interactionUSADevelop an integrated framework for effective human–robot interaction (HRI).
[38] 2018Journal ArticleSurvey on human–robot collaboration in industrial settings: Safety, intuitive interfaces and applicationsItalyExplore collaborative robotics solutions where human workers and robots share skills, focusing on combining the advantages of robots (accuracy, speed, repeatability) with the flexibility and cognitive skills of human workers.
[63] 2019Conference PaperA human-in-the-loop cyber-physical system for collaborative assembly in smart manufacturingItalyPresent a natural human–machine interface (NHMI) that integrates human decision making capabilities into the cybernetic control loop of a smart manufacturing assembly system.
[39]2019Journal ArticleSymbiotic human–robot collaborative assemblySwedenProvides an overview of symbiotic human–robot collaborative assembly and highlights future research directions.
[64]2019Conference PaperHuman Prediction for the Natural Instruction of Handovers in Human Robot CollaborationGermanyPresents an approach to integrate robotic handover assistance into collaborative assembly tasks using natural communication.
[49]2019Conference PaperHuman Features-Based Variable Admittance Control for Improving HRI and Performance in Power-Assisted Heavy Object ManipulationUSAAddress the limitations of current power assist robotic system (PARS) for lifting objects and to propose a variable admittance control (VAC) based on weight perception and kinematic and kinetic features to improve HRI and performance.
[36]2019Conference PaperAn effective model for human cognitive performance within a human–robot collaboration frameworkUSAProposes a novel time-variant human cognitive performance modelling approach for human–robot collaborative actions.
[66]2019Journal ArticleObjective Assessment of Human Workload in Physical Human–robot Cooperation Using Brain MonitoringUSAAssess human workload in physical human–robot cooperation, improving the reliability and generalizability of workload classifiers by selecting EEG features common between different tasks.
[67]2020Conference PaperHuman decisions for robot integration task allocation in a plan based building assignmentGermanyExplore how inexperienced individuals adapt to using a cooperative robot.
[40]2020Journal ArticlePassive Brain–Computer Interfaces for Enhanced Human–robot InteractionNetherlandsReview the state of the art in passive Brain–Computer Interface (BCI) technology in human–robot interaction (HRI).
[41]2020Conference PaperHuman–robot Collaboration Systems: Components and ApplicationsMexicoPresent a literature review analysis identifying trends in Human–robot Collaboration (HRC) in the manufacturing sector.
[68] 2020Conference PaperExamining the Effects of Anticipatory Robot Assistance on Human Decision MakingUSAInvestigates whether a robot’s anticipatory assistance influences a person’s decision making during a task. It aims to measure intent and examine if anticipatory robot actions affect user decisions.
[69]2020Journal ArticleMultimodal Physiological Signals for Workload Prediction in Robot-assisted SurgeryUSADemonstrate a computational framework to predict user workload during telerobotic surgery using wireless sensors to monitor surgeons’ cognitive load and predict their cognitive states.
[42]2020Journal ArticleCollaborative Robotics: A SurveyItalyProvide an overview of collaborative robotics, emphasizing the close interaction between humans and robots in industrial settings.
[70]2021Journal ArticleContact-initiated shared control strategies for four-arm supernumerary manipulation with foot interfacesSwitzerlandExplore the effectiveness of contact-initiated shared control strategies to improve the subjective fluency of human–robot interaction and reduce the task load on participants.
[50]2021Conference PaperA Cognitive Human Model for Virtual Commissioning of Dynamic Human–robot TeamsGermanyDevelop a generic and configurable cognitive human model for virtual commissioning.
[46]2021Conference PaperConnecting human–robot interaction and data visualizationUSAHighlight the importance of integrating data visualization knowledge into robot interface design to enhance data analysis and decision making in human–robot interaction (HRI).
[51]2021Journal ArticleMethods for Expressing Robot Intent for Human–Robot Collaboration in Shared WorkspacesUSAImprove interaction experiences between humans and robots working in close proximity, such as in factory settings. It reports on a user study that tested various signals a robot might use to communicate its intent to move, thereby enhancing safety and efficiency.
[52]2021Conference PaperDynamic Path Visualization for Human–robot CollaborationUSADevelop a method that conveys a robot’s future navigation route in a quick and intuitive manner using augmented reality in simulation.
[71]2021Journal ArticleFacing with Collaborative Robots:The Subjective Experience in Senior and Younger WorkersItalyUnderstand the subjective experience of younger and senior workers interacting with an industrial collaborative robot (cobot).
[47]2021Journal ArticleAutonomy in Physical Human–robot Interaction: A Brief SurveyItalyCollect and discuss the latest results in the field of shared control (SC) and shared autonomy (SA), with a particular focus on Physical Human–robot Interaction (pHRI).
[53]2021Journal ArticleHuman–robot collaboration: Optimizing stress and productivity based on game theoryItalyPropose a novel paradigm that enables a robot to adapt its behaviour online to optimize human physiological stress and productivity in real-time.
[43]2021Journal ArticleErgonomics and human factors as a requirement to implement safer collaborative robotic workstations: A literature reviewPortugalConduct a literature review to understand the integration of ergonomics and human factors (E&HF) as a requirement in the implementation of collaborative robots (Cobots) to reduce work-related Musculoskeletal Disorders (WMSD) risk.
[72]2022Conference PaperHuman–robot Collaboration During Assembly Tasks: The Cognitive Effects of Collaborative Assembly Workstation FeaturesItalyExplore the effects of collaborative robotic system features on workers’ perceived cognitive workload, usability, and visual attention.
[73]2022Conference PaperEvaluation of Variables of Cognitive Ergonomics in Industrial Human–robot Collaborative Assembly SystemsItalyEvaluate cognitive ergonomics variables in human–robot collaborative assembly systems (CASs) within the context of Industry 4.0.
[54]2022Journal ArticlePick the Right Co-Worker: Online Assessment of Cognitive Ergonomics in Human–robot Collaborative AssemblyItalyPropose an online and quantitative method to assess the cognitive workload induced by interaction with a co-worker, which can be either a human operator or an industrial collaborative robot with different control strategies.
[55]2022Conference PaperRobot Trajectory Adaptation to Optimise the Trade-off between Human Cognitive Ergonomics and Workplace Productivity in Collaborative TasksItalyPropose a human–robot interaction framework that adapts the robot’s behaviour online according to the operator’s cognitive workload and stress to address the balance between worker comfort and safety and the productivity of collaborative robots (CoBots) in industrial settings.
[56]2022Journal ArticleCross-Task Cognitive Workload Recognition Based on EEG and Domain AdaptationChinaProposes a new framework for EEG-based cross-task cognitive workload recognition using domain adaptation methods.
[44]2022Journal ArticleMetrics for Robot Proficiency Self-assessment and Communication of Proficiency in Human–robot TeamsUSADevelop metrics to evaluate the characteristics and performance of robot systems that can self-assess their proficiency in accomplishing tasks.
[74]2022Journal ArticleNeurophysiological and Behavioral Differences in Human-Multiagent Tasks: An EEG Network PerspectiveUSAUnderstand the cognitive state of humans as they interact with multiagent systems and to use this understanding to improve collaboration between humans and robots.
[75]2022Conference PaperTowards Brain Metrics for Improving Multi-Agent Adaptive Human–robot Collaboration: A Preliminary StudyUSAInvestigate brain metrics of workload and collaboration in teams consisting of humans and robots, focusing on two brain regions: the right prefrontal cortex (rPFC) and the right superior temporal sulcus (rSTS) during collaborative tasks.
[45]2022Journal ArticleHuman Factors Considerations and Metrics in Shared Space Human–robot Collaboration: A Systematic ReviewUSASystematically review the literature to evaluate the most frequently addressed operator human factor states in shared space human–robot collaboration (HRC), the methods used to quantify these states, and the implications of these states on HRC.
[17]2022Journal ArticleWhat about the human in human robot collaboration?: A literature review on HRC’s effects on aspects of job qualityNetherlandsReview the effects of human–robot collaboration (HRC) on job quality, defined as the quality of the working environment and its impact on employee well-being.
[57]2022Journal ArticleDevelopment and evaluation of design guidelines for cognitive ergonomics in human–robot collaborative assembly systemsItalyDevelop and evaluate design guidelines for cognitive ergonomics in human–robot collaborative assembly systems (CASs).
[76]2023Book ChapterEnhancing the Quality of Human–robot Cooperation Through the Optimization of Human Well-Being and ProductivityItalyEnhance the quality of human–robot cooperation (HRC) by optimizing human well-being and productivity.
[77]2023Journal ArticleAssessing the Relationship between Cognitive Workload, Workstation Design, User Acceptance and Trust in Collaborative RobotsItalyAssesses the relationship between cognitive workload, workstation design, user acceptance, and trust in collaborative robots.
[58] 2023Journal Article UHTP: A User-Aware Hierarchical Task Planning Framework for Communication-Free, Mutually Adaptive Human–robot CollaborationUSADevelop a User-aware Hierarchical Task Planning (UHTP) framework for communication-free, mutually adaptive human–robot collaboration.
[78]2023Conference PaperPupillometry for Measuring User Response to Movement of an Industrial RobotSwitzerlandMeasure users’ cognitive workload (CWL) responses to robot movements using pupillometry.
[79]2023Conference PaperMultisensory Evaluation of Human–robot Interaction in Retail Stores-The Effect of Mobile Cobots on Individuals’ Physical and Neurophysiological ResponsesUSAAssess individuals’ physical and neurophysiological responses to a mobile cobot in a retail environment, with a focus on understanding how these interactions affect short-term adaptation and to inform the cobot’s behavioural and control algorithms.
[59]2023Conference PaperEnhancing Human–robot Collaboration by Exploring Intuitive Augmented Reality Design RepresentationsUSADevelop systematic design guidelines for AR interfaces in Human–robot Interaction (HRI) systems and evaluating these designs to improve user understanding of the robot’s intents, trust, and safety in the work environment.
[60]2023Conference PaperA Multimodal Approach to Investigate the Role of Cognitive Workload and User Interfaces in Human–robot CollaborationUSARefine interactions between humans, machines, and robots by developing human-centered design solutions to enhance HRC performance, trust, and safety.
[65]2023Journal ArticleComparing alternative modalities in the context of multimodal human–robot interactionIndiaEvaluate the impact of alternative input modalities on user performance and perceived cognitive workload in human–robot interaction, using a fixed-base robot for object picking and dropping in single-task scenarios, and a mobile robot for driving in dual-task scenarios.
[61]2023Conference PaperTowards the modelling of defect generation in human–robot collaborative assemblyItalyDevelop suitable defect generation models for predicting defects in manufacturing processes and planning effective quality controls, specifically in a human–robot collaborative environment, to compare quality performances with purely manual assembly.
[80]2023Journal ArticleThe effect of cognitive workload on decision authority assignment in human–robot collaborationSwitzerlandInvestigate the effect of decision authority schemes in HRC tasks on performance, overall perceived workload, and subjective preference criteria at different levels of induced cognitive workload.
[62]2023Journal ArticleA Haptic Shared Control Architecture for Tracking of a Moving ObjectChinaPropose a haptic shared control framework that integrates human and robotic control to track moving objects.

References

  1. Baratta, A.; Cimino, A.; Gnoni, M.G.; Longo, F. Human Robot Collaboration in Industry 4.0: A Literature Review. Procedia Comput. Sci. 2023, 217, 1887–1895. [Google Scholar] [CrossRef] [PubMed]
  2. Demir, K.A.; Halil, C. The next Industrial Revolution: Industry 5.0 and Discussions on Industry 4.0, Industry 4.0 from the Management Information Systems Perspectives. In Industry 4.0 from the MIS Perspective; Peter Lang GmbH: Frankfurt, Germany, 2018. [Google Scholar]
  3. Raja Santhi, A.; Muthuswamy, P. Industry 5.0 or Industry 4.0S? Introduction to Industry 4.0 and a Peek into the Prospective Industry 5.0 Technologies. Int. J. Interact. Des. Manuf. 2023, 17, 947–979. [Google Scholar] [CrossRef]
  4. Gervasi, R.; Mastrogiacomo, L.; Franceschini, F. A Conceptual Framework to Evaluate Human-Robot Collaboration. Int. J. Adv. Manuf. Technol. 2020, 108, 841–865. [Google Scholar] [CrossRef]
  5. Colgate, J.E.; Wannasuphoprasit, W.; Peshkin, M.A. Cobots: Robots for Collaboration with Human Operators; Kwon, Y.W., Davis, D., Chung, H.H., Eds.; ASME: New York, NY, USA, 1996; Volume 58. [Google Scholar]
  6. Akella, P.; Peshkin, M.; Colgate, E.; Wannasuphoprasit, W.; Nagesh, N.; Wells, J.; Holland, S.; Pearson, T.; Peacock, B. Cobots for the Automobile Assembly Line. In Proceedings of the 1999 IEEE International Conference on Robotics and Automation (Cat. No.99CH36288C), Detroit, MI, USA, 10–15 May 1999; Volume 1, pp. 728–733. [Google Scholar]
  7. Guertler, M.; Tomidei, L.; Sick, N.; Carmichael, M.; Paul, G.; Wambsganss, A.; Moreno, V.H.; Hussain, S. When Is a Robot a Cobot? Moving beyond Manufacturing and Arm-Based Cobot Manipulators. Proc. Des. Soc. 2023, 3, 3889–3898. [Google Scholar] [CrossRef]
  8. Kopp, T.; Baumgartner, M.; Kinkel, S. Success Factors for Introducing Industrial Human-Robot Interaction in Practice: An Empirically Driven Framework. Int. J. Adv. Manuf. Technol. 2021, 112, 685–704. [Google Scholar] [CrossRef]
  9. Burden, A.G.; Caldwell, G.A.; Guertler, M.R. Towards Human–Robot Collaboration in Construction: Current Cobot Trends and Forecasts. Constr. Robot. 2022, 6, 209–220. [Google Scholar] [CrossRef]
  10. ISO 10218-1:2011. Available online: https://www.iso.org/standard/51330.html (accessed on 29 December 2023).
  11. Puterman, M.L. Chapter 8 Markov Decision Processes. In Handbooks in Operations Research and Management Science; Elsevier: Amsterdam, The Netherlands, 1990; Volume 2, pp. 331–434. [Google Scholar]
  12. Kaelbling, L.P.; Littman, M.L.; Cassandra, A.R. Planning and Acting in Partially Observable Stochastic Domains. Artif. Intell. 1998, 101, 99–134. [Google Scholar] [CrossRef]
  13. Ma, W.J. Bayesian Decision Models: A Primer. Neuron 2019, 104, 164–175. [Google Scholar] [CrossRef]
  14. Can Görür, O.; Rosman, B.; Sivrikaya, F.; Albayrak, S. FABRIC: A Framework for the Design and Evaluation of Collaborative Robots with Extended Human Adaptation. ACM Trans. Hum.-Robot. Interact. 2023, 12, 38. [Google Scholar] [CrossRef]
  15. Soori, M.; Arezoo, B.; Dastres, R. Artificial Intelligence, Machine Learning and Deep Learning in Advanced Robotics, a Review. Cogn. Robot. 2023, 3, 54–70. [Google Scholar] [CrossRef]
  16. Jonassen, D.H. Designing for Decision Making. Educ. Technol. Res. Dev. 2012, 60, 341–359. [Google Scholar] [CrossRef]
  17. Baltrusch, S.J.; Krause, F.; de Vries, A.W.; van Dijk, W.; de Looze, M.P. What about the Human in Human Robot Collaboration?: A Literature Review on HRC’s Effects on Aspects of Job Quality. Ergonomics 2022, 65, 719–740. [Google Scholar] [CrossRef] [PubMed]
  18. Randell, R.; Honey, S.; Alvarado, N.; Pearman, A.; Greenhalgh, J.; Long, A.; Gardner, P.; Gill, A.; Jayne, D.; Dowding, D. Embedding Robotic Surgery into Routine Practice and Impacts on Communication and Decision Making: A Review of the Experience of Surgical Teams. Cogn. Technol. Work 2016, 18, 423–437. [Google Scholar] [CrossRef]
  19. Liu, Y.; Nejat, G. Multirobot Cooperative Learning for Semiautonomous Control in Urban Search and Rescue Applications. J. Field Robot. 2016, 33, 512–536. [Google Scholar] [CrossRef]
  20. Klein, G.A.; Orasanu, J.; Caldenwood, R. Decision Making in Action: Models and Methods; Praeger: Westport, CT, USA, 1993; ISBN 9780893919436. [Google Scholar]
  21. Klein, G. Naturalistic Decision Making. Hum. Factors 2008, 50, 456–460. [Google Scholar] [CrossRef]
  22. Yesilbas, V.; Cotter, T.S. Application of Naturalistic Decision Making to the Domain of Unmanned Air Vehicles Operations. In Proceedings of the International Annual Conference of the American Society for Engineering Management, Philadelphia, PA, USA, 23–26 October 2019; American Society for Engineering Management (ASEM): Huntsville, AL, USA, 2019; pp. 1–7. [Google Scholar]
  23. Fan, X.; Sun, S.; McNeese, M.; Yen, J. Extending the Recognition-Primed Decision Model to Support Human-Agent Collaboration. In Proceedings of the Fourth International Joint Conference on Autonomous Agents and Multiagent Systems, Utrecht, The Netherlands, 25–29 July 2005; Association for Computing Machinery: New York, NY, USA; pp. 945–952. [Google Scholar]
  24. Gonzalez, C. Task Workload and Cognitive Abilities in Dynamic Decision Making. Hum. Factors 2005, 47, 92–101. [Google Scholar] [CrossRef] [PubMed]
  25. Simone, V.D.; Pasquale, V.D.; Giubileo, V.; Miranda, S. Human-Robot Collaboration: An Analysis of Worker’s Performance. Procedia Comput. Sci. 2022, 200, 1540–1549. [Google Scholar] [CrossRef]
  26. Ewusi-Mensaxh, K. Evaluating Information Systems Projects: A Perspective on Cost-Benefit Analysis. Inf. Syst. 1989, 14, 205–217. [Google Scholar] [CrossRef]
  27. Kumar, A.; Maskara, S. Coping up with the Information Overload in the Medical Profession. J. Biosci. Med. 2015, 03, 124–127. [Google Scholar] [CrossRef]
  28. Amadori, P.V.; Fischer, T.; Wang, R.; Demiris, Y. Predicting Secondary Task Performance: A Directly Actionable Metric for Cognitive Overload Detection. IEEE Trans. Cogn. Dev. Syst. 2022, 14, 1474–1485. [Google Scholar] [CrossRef]
  29. Edgcumbe, D.R. Transcranial Direct Current Stimulation and Decision-Making: The Neuromodulation of Cognitive Reflection. Ph.D. Thesis, University of East London, London, UK, 2018. [Google Scholar]
  30. Gigerenzer, G.; Brighton, H. Homo Heuristicus: Why Biased Minds Make Better Inferences. Top. Cogn. Sci. 2009, 1, 107–143. [Google Scholar] [CrossRef] [PubMed]
  31. Parpart, P.; Jones, M.; Love, B.C. Heuristics as Bayesian Inference under Extreme Priors. Cogn. Psychol. 2018, 102, 127–144. [Google Scholar] [CrossRef] [PubMed]
  32. Hayne, S.C.; Smith, C.A.P.; Turk, D. The Effectiveness of Groups Recognizing Patterns. Int. J. Hum. Comput. Stud. 2003, 59, 523–543. [Google Scholar] [CrossRef]
  33. Satterthwaite, T.D.; Green, L.; Myerson, J.; Parker, J.; Ramaratnam, M.; Buckner, R.L. Dissociable but Inter-Related Systems of Cognitive Control and Reward during Decision Making: Evidence from Pupillometry and Event-Related FMRI. Neuroimage 2007, 37, 1017–1031. [Google Scholar] [CrossRef] [PubMed]
  34. Lafond, D.; DuCharme, M.B.; Gagnon, J.-F.; Tremblay, S. Support Requirements for Cognitive Readiness in Complex Operations. J. Cogn. Eng. Decis. Mak. 2012, 6, 393–426. [Google Scholar] [CrossRef]
  35. Tricco, A.C.; Lillie, E.; Zarin, W.; O’Brien, K.K.; Colquhoun, H.; Levac, D.; Moher, D.; Peters, M.D.J.; Horsley, T.; Weeks, L.; et al. PRISMA Extension for Scoping Reviews (PRISMA-ScR): Checklist and Explanation. Ann. Intern. Med. 2018, 169, 467–473. [Google Scholar] [CrossRef] [PubMed]
  36. Rabby, K.M.; Khan, M.; Karimoddini, A.; Jiang, S.X. An Effective Model for Human Cognitive Performance within a Human-Robot Collaboration Framework. In Proceedings of the 2019 IEEE International Conference on Systems, Man and Cybernetics (SMC), Bari, Italy, 6–9 October 2019. [Google Scholar]
  37. Ajoudani, A.; Zanchettin, A.M.; Ivaldi, S.; Albu-Schäffer, A.; Kosuge, K.; Khatib, O. Progress and Prospects of the Human–Robot Collaboration. Auton. Robots 2018, 42, 957–975. [Google Scholar] [CrossRef]
  38. Villani, V.; Pini, F.; Leali, F.; Secchi, C. Survey on Human–Robot Collaboration in Industrial Settings: Safety, Intuitive Interfaces and Applications. Mechatronics 2018, 55, 248–266. [Google Scholar] [CrossRef]
  39. Wang, L.; Gao, R.; Váncza, J.; Krüger, J.; Wang, X.V.; Makris, S.; Chryssolouris, G. Symbiotic Human-Robot Collaborative Assembly. CIRP Ann. 2019, 68, 701–726. [Google Scholar] [CrossRef]
  40. Alimardani, M.; Hiraki, K. Passive Brain-Computer Interfaces for Enhanced Human-Robot Interaction. Front. Robot. AI 2020, 7, 125. [Google Scholar] [CrossRef]
  41. Parra, P.S.; Calleros, O.L.; Ramirez-Serrano, A. Human-Robot Collaboration Systems: Components and Applications. In Proceedings of the 7th International Conference of Control Systems, and Robotics (CDSR’20), Niagara Falls, ON, Canada, 9–11 November 2020. [Google Scholar]
  42. Vicentini, F. Collaborative Robotics: A Survey. J. Mech. Des. 2020, 143, 1–29. [Google Scholar] [CrossRef]
  43. Cardoso, A.; Colim, A.; Bicho, E.; Braga, A.C.; Menozzi, M.; Arezes, P. Ergonomics and Human Factors as a Requirement to Implement Safer Collaborative Robotic Workstations: A Literature Review. Safety 2021, 7, 71. [Google Scholar] [CrossRef]
  44. Norton, A.; Admoni, H.; Crandall, J.; Fitzgerald, T.; Gautam, A.; Goodrich, M.; Saretsky, A.; Scheutz, M.; Simmons, R.; Steinfeld, A.; et al. Metrics for Robot Proficiency Self-Assessment and Communication of Proficiency in Human-Robot Teams. J. Hum.-Robot Interact. 2022, 11, 29. [Google Scholar] [CrossRef]
  45. Hopko, S.; Wang, J.; Mehta, R. Human Factors Considerations and Metrics in Shared Space Human-Robot Collaboration: A Systematic Review. Front. Robot. AI 2022, 9, 799522. [Google Scholar] [CrossRef] [PubMed]
  46. Szafir, D.; Szafir, D.A. Connecting Human-Robot Interaction and Data Visualization; Association for Computing Machinery: New York, NY, USA, 2021; pp. 281–292. [Google Scholar]
  47. Selvaggio, M.; Cognetti, M.; Nikolaidis, S.; Ivaldi, S.; Siciliano, B. Autonomy in Physical Human-Robot Interaction: A Brief Survey. IEEE Robot. Autom. Lett. 2021, 6, 7989–7996. [Google Scholar] [CrossRef]
  48. Sadrfaridpour, B.; Wang, Y. Collaborative Assembly in Hybrid Manufacturing Cells: An Integrated Framework for Human–Robot Interaction. IEEE Trans. Autom. Sci. Eng. 2018, 15, 1178–1192. [Google Scholar] [CrossRef]
  49. Rahman, S.M.M. Human Features-Based Variable Admittance Control for Improving HRI and Performance in Power-Assisted Heavy Object Manipulation. In Proceedings of the 2019 12th International Conference on Human System Interaction (HSI), Richmond, VA, USA, 25–27 June 2019. [Google Scholar]
  50. Riedelbauch, D.; Luthardt-Bergmann, D.; Henrich, D. A Cognitive Human Model for Virtual Commissioning of Dynamic Human-Robot Teams. In Proceedings of the 2021 Fifth IEEE International Conference on Robotic Computing (IRC), Taichung, Taiwan, 15–17 November 2021. [Google Scholar]
  51. Lemasurier, G.; Bejerano, G.; Albanese, V.; Parrillo, J.; Yanco, H.A.; Amerson, N.; Hetrick, R.; Phillips, E. Methods for Expressing Robot Intent for Human–Robot Collaboration in Shared Workspaces. J. Hum.-Robot Interact. 2021, 10, 40. [Google Scholar] [CrossRef]
  52. Cleaver, A.; Tang, D.V.; Chen, V.; Short, E.S.; Sinapov, J. Dynamic Path Visualization for Human-Robot Collaboration; Association for Computing Machinery: New York, NY, USA, 2021; pp. 339–343. [Google Scholar]
  53. Messeri, C.; Masotti, G.; Zanchettin, A.M.; Rocco, P. Human-Robot Collaboration: Optimizing Stress and Productivity Based on Game Theory. IEEE Robot. Autom. Lett. 2021, 6, 8061–8068. [Google Scholar] [CrossRef]
  54. Lagomarsino, M.; Lorenzini, M.; Balatti, P.; De Momi, E.; Ajoudani, A. Pick the Right Co-Worker: Online Assessment of Cognitive Ergonomics in Human-Robot Collaborative Assembly. IEEE Trans. Cogn. Dev. Syst. 2022, 15, 1928–1937. [Google Scholar] [CrossRef]
  55. Lagomarsino, M.; Lorenzini, M.; De Momi, E.; Ajoudani, A. Robot Trajectory Adaptation to Optimise the Trade-off between Human Cognitive Ergonomics and Workplace Productivity in Collaborative Tasks. In Proceedings of the 2022 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), Kyoto, Japan, 23–27 October 2022. [Google Scholar]
  56. Zhou, Y.; Xu, Z.; Niu, Y.; Wang, P.; Wen, X.; Wu, X.; Zhang, D. Cross-Task Cognitive Workload Recognition Based on EEG and Domain Adaptation. IEEE Trans. Neural Syst. Rehabil. Eng. 2022, 30, 50–60. [Google Scholar] [CrossRef]
  57. Gualtieri, L.; Fraboni, F.; De Marchi, M.; Rauch, E. Development and Evaluation of Design Guidelines for Cognitive Ergonomics in Human-Robot Collaborative Assembly Systems. Appl. Ergon. 2022, 104, 103807. [Google Scholar] [CrossRef]
  58. Ramachandruni, K.; Kent, C.; Chernova, S. UHTP: A User-Aware Hierarchical Task Planning Framework for Communication-Free, Mutually-Adaptive Human-Robot Collaboration. J. Hum.-Robot Interact. 2023. [Google Scholar] [CrossRef]
  59. Eze, C.; Crick, C. Enhancing Human-Robot Collaboration by Exploring Intuitive Augmented Reality Design Representations; Association for Computing Machinery: New York, NY, USA, 2023; pp. 282–286. [Google Scholar]
  60. Kalatzis, A.; Rahman, S.; Prabhu, V.G.; Stanley, L.; Wittie, M. A Multimodal Approach to Investigate the Role of Cognitive Workload and User Interfaces in Human-Robot Collaboration; Association for Computing Machinery: New York, NY, USA, 2023; pp. 5–14. [Google Scholar]
  61. Puttero, S.; Verna, E.; Genta, G.; Galetto, M. Towards the Modelling of Defect Generation in Human-Robot Collaborative Assembly. Procedia CIRP 2023, 118, 247–252. [Google Scholar] [CrossRef]
  62. Zhu, Y.; Yang, C.; Tu, Z.; Ling, Y.; Chen, Y. A Haptic Shared Control Architecture for Tracking of a Moving Object. IEEE Trans. Ind. Electron. 2023, 70, 5034–5043. [Google Scholar] [CrossRef]
  63. Ruiz Garcia, M.A.; Rojas, R.; Gualtieri, L.; Rauch, E.; Matt, D. A Human-in-the-Loop Cyber-Physical System for Collaborative Assembly in Smart Manufacturing; Butala, P., Govekar, E., Vrabic, R., Eds.; Elsevier B.V.: Amsterdam, The Netherlands, 2019; Volume 81. [Google Scholar]
  64. Lambrecht, J.; Nimpsch, S. Human Prediction for the Natural Instruction of Handovers in Human Robot Collaboration. In Proceedings of the 2019 28th IEEE International Conference on Robot and Human Interactive Communication (RO-MAN), New Delhi, India, 14–18 October 2019. [Google Scholar]
  65. Saren, S.; Mukhopadhyay, A.; Ghose, D.; Biswas, P. Comparing Alternative Modalities in the Context of Multimodal Human–Robot Interaction. J. Multimodal User Interfaces 2023, 18, 69–85. [Google Scholar] [CrossRef]
  66. Memar, A.H.; Esfahani, E.T. Objective Assessment of Human Workload in Physical Human-Robot Cooperation Using Brain Monitoring. J. Hum.-Robot Interact. 2019, 9, 13. [Google Scholar] [CrossRef]
  67. Klaer, V.; Wibranek, B. Human Decisions for Robot Integration Task Allocation in a Plan Based Building Assignment. In Proceedings of the Companion of the 2020 ACM/IEEE International Conference on Human-Robot Interaction, Cambridge, UK, 23–26 March 2020. [Google Scholar]
  68. Newman, B.A.; Biswas, A.; Ahuja, S.; Girdhar, S.; Kitani, K.K.; Admoni, H. Examining the Effects of Anticipatory Robot Assistance on Human Decision Making. In International Conference on Social Robotics; Springer: Cham, Switzerland, 2020; Volume 12483. [Google Scholar]
  69. Zhou, T.; Cha, J.S.; Gonzalez, G.; Wachs, J.P.; Sundaram, C.P.; Yu, D. Multimodal Physiological Signals for Workload Prediction in Robot-Assisted Surgery. J. Hum.-Robot Interact. 2020, 9, 12. [Google Scholar] [CrossRef]
  70. Amanhoud, W.; Hernandez Sanchez, J.; Bouri, M.; Billard, A. Contact-Initiated Shared Control Strategies for Four-Arm Supernumerary Manipulation with Foot Interfaces. Int. J. Rob. Res. 2021, 40, 986–1014. [Google Scholar] [CrossRef]
  71. Rossato, C.; Pluchino, P.; Cellini, N.; Jacucci, G.; Spagnolli, A.; Gamberini, L. Facing with Collaborative Robots: The Subjective Experience in Senior and Younger Workers. Cyberpsychol. Behav. Soc. Netw. 2021, 24, 349–356. [Google Scholar] [CrossRef]
  72. Fraboni, F.; Gualtieri, L.; Millo, F.; De Marchi, M.; Pietrantoni, L.; Rauch, E. Human-Robot Collaboration During Assembly Tasks: The Cognitive Effects of Collaborative Assembly Workstation Features. In Proceedings of the 21st Congress of the International Ergonomics Association (IEA 2021), Virtual, 13–18 June 2021; Volume 223. [Google Scholar]
  73. Gualtieri, L.; Fraboni, F.; De Marchi, M.; Rauch, E. Evaluation of Variables of Cognitive Ergonomics in Industrial Human-Robot Collaborative Assembly Systems. In Proceedings of the 21st Congress of the International Ergonomics Association (IEA 2021) Virtual, 13–18 June 2021; Volume 223. [Google Scholar]
  74. Bales, G.; Kong, Z. Neurophysiological and Behavioral Differences in Human-Multiagent Tasks: An EEG Network Perspective. J. Hum.-Robot Interact. 2022, 11, 42. [Google Scholar] [CrossRef]
  75. Howell-Munson, A.; Doherty, E.; Gavriel, P.; Nicolas, C.; Norton, A.; Neamtu, R.; Yanco, H.; Wu, Y.-N.; Solovey, E.T. Towards Brain Metrics for Improving Multi-Agent Adaptive Human-Robot Collaboration: A Preliminary Study; Association for Computing Machinery: New York, NY, USA, 2022; p. 11. [Google Scholar]
  76. Messeri, C. Enhancing the Quality of Human-Robot Cooperation Through the Optimization of Human Well-Being and Productivity. In SpringerBriefs in Applied Sciences and Technology; Springer: Cham, Switzerland, 2023; pp. 15–25. [Google Scholar]
  77. Panchetti, T.; Pietrantoni, L.; Puzzo, G.; Gualtieri, L.; Fraboni, F. Assessing the Relationship between Cognitive Workload, Workstation Design, User Acceptance and Trust in Collaborative Robots. Appl. Sci. 2023, 13, 1720. [Google Scholar] [CrossRef]
  78. Hostettler, D.; Bektaş, K.; Mayer, S. Pupillometry for Measuring User Response to Movement of an Industrial Robot; Association for Computing Machinery: New York, NY, USA, 2023; p. 52. [Google Scholar]
  79. Luo, Y.; Chen, Y.; Hu, B. Multisensory Evaluation of Human-Robot Interaction in Retail Stores—The Effect of Mobile Cobots on Individuals’ Physical and Neurophysiological Responses; Association for Computing Machinery: New York, NY, USA, 2023; pp. 403–406. [Google Scholar]
  80. Karakikes, M.; Nathanael, D. The Effect of Cognitive Workload on Decision Authority Assignment in Human–Robot Collaboration. Cogn. Technol. Work 2023, 25, 31–43. [Google Scholar] [CrossRef]
Robotics 13 00030 g001
Figure 1. Types of human–robot relationships from full automation to coexistence, cooperation, and collaboration [9]. Adopted from [8] and ISO 10218-1 [10].
Figure 1. Types of human–robot relationships from full automation to coexistence, cooperation, and collaboration [9]. Adopted from [8] and ISO 10218-1 [10].
Robotics 13 00030 g001
Robotics 13 00030 g002
Figure 2. PRISMA flow chart.
Figure 2. PRISMA flow chart.
Robotics 13 00030 g002
Robotics 13 00030 g003
Figure 3. Number of papers per year.
Figure 3. Number of papers per year.
Robotics 13 00030 g003
Robotics 13 00030 g004
Figure 4. (a) Types of publications; (b) countries of publications.
Figure 4. (a) Types of publications; (b) countries of publications.
Robotics 13 00030 g004
Robotics 13 00030 g005
Figure 5. Types of interaction and tasks: (a) types of interaction; (b) majority of tasks.
Figure 5. Types of interaction and tasks: (a) types of interaction; (b) majority of tasks.
Robotics 13 00030 g005
Robotics 13 00030 g006
Figure 6. Factors related to human decision making in HRC.
Figure 6. Factors related to human decision making in HRC.
Robotics 13 00030 g006
Robotics 13 00030 g007
Figure 7. Types of user interfaces.
Figure 7. Types of user interfaces.
Robotics 13 00030 g007
Table 1. Selecting criteria.
Table 1. Selecting criteria.
Inclusive CriteriaExclusive Criteria
journal articles
conference papers
book chapters
published from 2019 to 2023
written in English
quantitative, qualitative, and mixed-method		autonomous mobile robot, telerobot, AI bot, social robot, military robot
machine learning, reinforcement learning
robot decision making, algorithmic decision making, decision making modelling, robot planning process
conversation or voice agent
design process
Table 2. Sets of keywords used in the search.
Table 2. Sets of keywords used in the search.
Human Decision making and Cognitive WorkloadHRC
“cognitive workload” OR “human decision”“human–robot collaboration” OR cobot* OR “collaborative robot*” OR “human–robot interaction”
Table 3. Study design and setting.
Table 3. Study design and setting.
Research TypeStudySettingSample SizeMeasuresFactors Were Measured
Conceptual Research[36]Virtual Environment----Cognitive workload
Review[17,37,38,39,40,41,42,43,44,45]--------
Survey and Analysis[46,47]--------
Design and Development[48]Laboratory--NASA-TLXCognitive workload
[49]Laboratory--NASA-TLXCognitive workload
[50]Virtual Environment----
[51]Laboratory92NASA-TLXCognitive workload
[52]Virtual Environment36NASA-TLXCognitive workload
[53]Laboratory--HRVStress
[54]Laboratory--NASA-TLX, HRV, Skin responseCognitive workload
[55]Laboratory--HRVCognitive workload
[56]Laboratory--EEGCognitive workload
[57]Laboratory14System Acceptance Scale, NASA-TLX, SSSQ, Trust in Industrial Human–robot Collaboration Scale, System Usability Scale, observation, video recording, semi-structured interviewTrust, acceptance, Satisfaction, Stress, Cognitive workload
[58]Laboratory, Virtual environment35NASA-TLXCognitive workload
[59]Online Survey14Online Survey--
[60]Laboratory15HRV, NASA-TLXCognitive workload
[61]Laboratory52----
[62]Laboratory--Eye-tracking, Likert-scale surveyControl method
Implementation[63]Laboratory------
[64]Laboratory--NASA-TLXCognitive workload
Evaluation[65]Laboratory8NASA-TLXCognitive workload
Empirical Research[66]Laboratory18EEG, NASA-TLXCognitive workload
[67]Laboratory12----
[68]Laboratory99----
[69]Virtual environment12EEG, EMG, HRV, EDA, MOV, NASA-TLXCognitive workload
[70]Laboratory--Questionnaire of fluency perception, NASA-TLXCognitive workload
[71]Laboratory20TAM questionnaire, System Usability Scale, User Experience, NASA-TLXAcceptance, Usability, Cognitive workload
[72]Laboratory14Eye-tracking, NASA-TLX, System Usability ScaleFeedback, Cognitive workload, Usability
[73]Laboratory--Observation, video recording, semi-structured interviewUsability, Trust, Satisfaction, physical workload, Cognitive workload
[74]Laboratory--Eye-tracking, EEG, Motion CaptureCognitive workload
[75]Laboratory12fNIRS, pre- and post-experiment questionnairesPhysical workload, Cognitive workload
[76]Realistic Scenarios47HRVStress
[77]Laboratory--System Acceptance Scale, System Usability Scale, NASA-TLX, SSSQ, Trust in Industrial
Human–robot Interaction Questionnaire, Eye-tracking		Acceptance, Usability, Cognitive workload, Stress, Trust
[78]Laboratory7Eye-tracking, verbal reportCognitive workload
[79]--8Upper-limb kinematics, Muscle activity, Eye-trackingCognitive workload
[80]Laboratory21NASA-TLX, Preference of Automation LevelCognitive workload
Table 4. Types of Cobots.
Table 4. Types of Cobots.
CobotStudy
UR3, UR3e (6-DoF)[57,60,61,63,72,73,77]
UR5, UR5e (6-DoF)[62,64,75,79]
UR10, UR10e (6-DoF)[62,67,71]
UR16e (6-DoF)[55]
SCHUNK PowerBall LWA 4P (6-DoF)[66]
KUKA LWR4+ (7-DoF)[70]
Franka Emika Panda (7-DoF)[54,80]
JACO robotic arm (7-DoF)[58]
UFACTORY xArm 7 (7-DoF)[78]
Baxter (dual-arm)[51]
ABB YuMi (dual-arm)[53,76]
Dobot (4-DoF)[65]
da Vinci surgical robot[69]
virtual cobot[50]
Table 5. Factors Related to Human Decision making.
Table 5. Factors Related to Human Decision making.
Human FactorsRobot FactorsCommunication FactorsEnvironment Factors
trust [17,45,48,57,73,77]physical attribute of the robot [39,70,72]user interface [37,38,42,46,51,60,63,64,73]dynamic of the situation [75]
acceptance [17,57,71,77]robot errors [36,55]control modality [37,65,71]task complexity [36,55,66,69,80]
human characteristics [39]trajectory and movement [53,55,78]feedback [39,48,51,52,62,72]design of workspace [39,73,77]
physical workload [50]role of the robot [61,76,80]usability [71,73]physical safety [42]
cognitive workload [40]automation level [17,44,47,72]human intent prediction [59,68]
operator’s ability [39,41,67,74,76,79] mutual awareness [17,41,46,68]
stress [57,76,77]
feeling of satisfaction or frustration [57,73]
perception of the situation and environment [45,49]
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content.

Share and Cite

MDPI and ACS Style

Liu, Y.; Caldwell, G.; Rittenbruch, M.; Belek Fialho Teixeira, M.; Burden, A.; Guertler, M. What Affects Human Decision Making in Human–Robot Collaboration?: A Scoping Review. Robotics 2024, 13, 30. https://doi.org/10.3390/robotics13020030

AMA Style

Liu Y, Caldwell G, Rittenbruch M, Belek Fialho Teixeira M, Burden A, Guertler M. What Affects Human Decision Making in Human–Robot Collaboration?: A Scoping Review. Robotics. 2024; 13(2):30. https://doi.org/10.3390/robotics13020030

Chicago/Turabian Style

Liu, Yuan, Glenda Caldwell, Markus Rittenbruch, Müge Belek Fialho Teixeira, Alan Burden, and Matthias Guertler. 2024. "What Affects Human Decision Making in Human–Robot Collaboration?: A Scoping Review" Robotics 13, no. 2: 30. https://doi.org/10.3390/robotics13020030

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop