Evaluating Levels of Automation in Human–Robot Collaboration at Different Workload Levels

: This study explored how levels of automation (LOA) inﬂuence human robot collaboration when operating at different levels of workload. Two LOA modes were designed, implemented, and evaluated in an experimental collaborative assembly task setup for four levels of workload composed of a secondary task and task complexity. A user study conducted involving 80 participants was assessed through two constructs especially designed for the evaluation (quality of task execution and usability) and user preferences regarding the LOA modes. Results revealed that the quality of task execution and usability was better at high LOA for low workload. Most of participants also preferred high LOA when the workload increases. However, when complexity existed within the workload, most of the participants preferred the low LOA. The results reveal the beneﬁts of high and low LOA in different workload situations. This study provides insights related to shared control designs and reveals the importance of considering different levels of workload as inﬂuenced by secondary tasks and task complexity when designing LOA in human–robot collaborations.


Introduction
Human-robot collaboration (HRC) involves one or more humans working with one or more robots to accomplish a certain task or a specific goal [1]. Significant research has focused on interaction aspects for designing robotic systems for use by or with humans [2][3][4][5][6]. This research, which focuses on factors that affect HRC [1,7] at different levels of automation, specifically evaluates the influence of workload.
The level of automation (LOA) of the system, defined as the degree to which the robot and the human are involved in the collaborative task [8][9][10][11], influences the characteristics of the dynamics of the collaboration, the behavior of the robots, actions to be taken, as well as autonomy of the human in the collaboration [12,13]. Workload addresses the actual and perceived amount of work that the human operator experiences as related to the effort invested in the task [14,15]. It can be described in terms of the elements that constitute the cost of accomplishing the goal for the human operator in the HRC [16]. These elements could be task-related (such as mental, temporal, and physical demands [17], operator-related (such as skill, strategy, experience [18]) or machine-related (such as poorly designed controls, feedback, inappropriate, or inadequate automation [15]. Workload consequences could be reflected in the stress, fatigue or frustration experienced by the human operator [16], depletion of attentional, cognitive or response resources [15], as well as in performance changes [19]. Workload can also be influenced by task complexity as characterized in terms of the stimuli involved in the task for inputs, as well as the behavioral requirements the human operator should emit in order to achieve a specific level of performance [20]. It could depend on the objective complexity derived from the task properties and on the subjective complexity which is influenced by the human operator's perception [21]. The task properties include the component complexity-number of distinct

Experimental System
The experimental system included a 4 degree of freedom DOBOT Magician robotic arm (https://www.dobot.cc/dobot-magician/product-overview.html, accessed on 30 May 2021) equipped with a suction gripper, user interface (presented on a computer), cubes to be assembled and the human operator ( Figure 1). The DOBOT Magician (135 mm high, 158 mm wide with a 320 mm radius and 500 g payload) connects to the computer through a USB connection and was programmed for the two LOA modes using the Python programming language.  The HRC assembly task simulates a work scenario where participants are expected to assemble blocks made from cubes brought to them by a robot according to a configuration presented to them through a user interface. The task was performed in two LOA modes, at four workload levels. The workload levels, detailed below, are composed of different combinations of a secondary task and task complexity.
The user communicates with the robot through a user interface implemented on a GUI screen (Figure 2). This was designed to be friendly to promote ease of use as the human interacts with the robot through the GUI [34][35][36]. The configuration to be assembled is displayed on the GUI screen when starting the task. The robot brings the cubes in a sequence one after another from a predetermined place according to the specific LOA the robot is operating in. The robot releases the cube when it reaches the front of the participant. The participants are expected to assemble the cubes when received from the robot and place these cubes in a marked area on the desk in front of them.

Levels of Automation (LOA) Modes
The automation design focuses on the decision and action aspects of the overall pro cess taken either by the robot or the user. This specifies the degree of control the user o robot in the decision of action(s) to be taken and the execution of the actions. It is cond tioned in two levels for this study: (a) Low LOA-the user has autonomy to select the type and order of cubes. The robo supports the user by bringing the type of cube the user selected via the user interfac (b) High LOA-the robot has autonomy to bring the specific type of cube and in the orde preprogrammed in its operation. The user simply demands for a cube through th user interface and the robot brings the type of cube suitable for the specific configu ration assembled.

Levels of Workload
The workload design focuses mainly on the physical and cognitive workload in duced through the selection of the right cubes to assemble in the minimum possible tim This is the main task. Workload is increased in two ways: through a secondary task an by increasing task complexity.
The secondary task influence was depicted through an off-the shelf well known cog nitive game, the "RUSH HOUR" (https://www.thinkfun.com/products/rush-hour/, ac cessed on 30 May 2021) thinking game ( Figure 3). It involves arranging toy cars in a wa to get a specific car out of a gridlock. There are tabs at each stage showing how to arrang the cars and finding a way to get the required red car out at different stages. The automation design focuses on the decision and action aspects of the overall process taken either by the robot or the user. This specifies the degree of control the user or robot in the decision of action(s) to be taken and the execution of the actions. It is conditioned in two levels for this study: (a) Low LOA-the user has autonomy to select the type and order of cubes. The robot supports the user by bringing the type of cube the user selected via the user interface. (b) High LOA-the robot has autonomy to bring the specific type of cube and in the order preprogrammed in its operation. The user simply demands for a cube through the user interface and the robot brings the type of cube suitable for the specific configuration assembled.

Levels of Workload
The workload design focuses mainly on the physical and cognitive workload induced through the selection of the right cubes to assemble in the minimum possible time. This is the main task. Workload is increased in two ways: through a secondary task and by increasing task complexity.
The secondary task influence was depicted through an off-the shelf well known cognitive game, the "RUSH HOUR" (https://www.thinkfun.com/products/rush-hour/, accessed on 30 May 2021) thinking game ( Figure 3). It involves arranging toy cars in a way to get a specific car out of a gridlock. There are tabs at each stage showing how to arrange the cars and finding a way to get the required red car out at different stages.  In the main task, where cubes are assembled, the default se the assembly differ only by color. The users are required to asse particular configurations characterized by differences in color pa The task complexity influence was depicted by introducing bly that differ in color and in numbers on a particular side (Figu quired to assemble the cubes in color patterns as done in the low tion, but in addition, they must ensure that the specific number cubes match the required configuration per time. The task comp additional information cue (presence of numbers) and their spatia of the number in the configuration). It represents component an plexity induced through the number and type of sub-actions to lecting the right cubes and assembling along with the coordina secondary task.  Four levels of workload were designed using these factors: (a) Low workload (LWL)-the users perform only the main task out reference to the numbers on the cubes) to match the s quired. The workload involves some physical demand of arr demand of thinking about the type of cube that would match tion and some temporal demand related to completing the tas time.
(b) Medium workload 1 (MWL1)-the users perform only the the cubes but with reference to the numbers on the cubes. with increased task complexity (or high workload without s (c) Medium workload 2 (MWL2)-the users perform the main t out references to the numbers on the cubes) simultaneously It depicts the high workload level without complexity inclu In the main task, where cubes are assembled, the default setting is that the cubes for the assembly differ only by color. The users are required to assemble the cubes to match particular configurations characterized by differences in color pattern ( Figure 4a).

The red car
The game In the main task, where cubes are assembled, the default setting is that the cubes for the assembly differ only by color. The users are required to assemble the cubes to match particular configurations characterized by differences in color pattern (Figure 4a).
The task complexity influence was depicted by introducing the cubes for the assembly that differ in color and in numbers on a particular side (Figure 4b). The users are required to assemble the cubes in color patterns as done in the low task complexity condition, but in addition, they must ensure that the specific numbers on a particular color of cubes match the required configuration per time. The task complexity is increased by the additional information cue (presence of numbers) and their spatial consideration (position of the number in the configuration). It represents component and coordinative task complexity induced through the number and type of sub-actions to be performed while selecting the right cubes and assembling along with the coordination of the actions in the secondary task.  Four levels of workload were designed using these factors: (a) Low workload (LWL)-the users perform only the main task, assembling cubes (without reference to the numbers on the cubes) to match the specific configuration required. The workload involves some physical demand of arranging the cubes, mental demand of thinking about the type of cube that would match the required configuration and some temporal demand related to completing the task in the shortest possible time.
(b) Medium workload 1 (MWL1)-the users perform only the main task of assembling the cubes but with reference to the numbers on the cubes. It depicts the LWL level with increased task complexity (or high workload without secondary task).
(c) Medium workload 2 (MWL2)-the users perform the main task of assembling (without references to the numbers on the cubes) simultaneously with the secondary task. It depicts the high workload level without complexity included (or the LWL with a secondary task).
(d) High workload (HWL)-the users perform the main task of assembling the cubes (with reference to the numbers on the cubes) along with a secondary task. This combines both secondary task and increased task complexity. The task complexity influence was depicted by introducing the cubes for the assembly that differ in color and in numbers on a particular side ( Figure 4b). The users are required to assemble the cubes in color patterns as done in the low task complexity condition, but in addition, they must ensure that the specific numbers on a particular color of cubes match the required configuration per time. The task complexity is increased by the additional information cue (presence of numbers) and their spatial consideration (position of the number in the configuration). It represents component and coordinative task complexity induced through the number and type of sub-actions to be performed while selecting the right cubes and assembling along with the coordination of the actions in the secondary task.

Experimental Design
Four levels of workload were designed using these factors: (a) Low workload (LWL)-the users perform only the main task, assembling cubes (without reference to the numbers on the cubes) to match the specific configuration required. The workload involves some physical demand of arranging the cubes, mental demand of thinking about the type of cube that would match the required configuration and some temporal demand related to completing the task in the shortest possible time. (b) Medium workload 1 (MWL1)-the users perform only the main task of assembling the cubes but with reference to the numbers on the cubes. It depicts the LWL level with increased task complexity (or high workload without secondary task). (c) Medium workload 2 (MWL2)-the users perform the main task of assembling (without references to the numbers on the cubes) simultaneously with the secondary task. It depicts the high workload level without complexity included (or the LWL with a secondary task). (d) High workload (HWL)-the users perform the main task of assembling the cubes (with reference to the numbers on the cubes) along with a secondary task. This combines both secondary task and increased task complexity.

Experimental Design
The experimental design includes two independent variables: LOA and levels of workload. A between-within participant experimental design was conducted with the LOA as the within variable while level of workload was the between variable. Four groups were designed depicting the different levels of workload. Each participant was randomly assigned to one of the four groups and experienced both LOA modes (Table 1).

Level of Automation (LOA)
Low LOA

Condition 1a
The user chooses via a GUI screen which color of cube the robot will bring him. The user concentrates only on the main task, without reference to the numbers written on the cubes.

Condition 2a
The user chooses via a GUI screen which color of cube the robot will bring him. The user concentrates only on the main task, which has increased complexity (through the numbers written on the cubes).

Condition 3a
The user chooses via a GUI screen which color of cube the robot will bring him. The user performs a main + secondary task simultaneously, without reference to the numbers written on the cubes.

Condition 4a
The user chooses via a GUI screen which color of cube the robot will bring him. The user concentrates on performing a main + secondary task simultaneously, with an increased task complexity (must refer to the numbers written on the cubes).

High LOA
Condition 1b The robot brings the cubes to the user in a predefined order. The user concentrates only on the main task, without reference to the numbers written on the cubes.

Condition 2b
The robot brings the cubes to the user in a predefined order. The user concentrates only on the main task, which has increased complexity (through the numbers written on the cubes).

Condition 3b
The robot brings the cubes to the user in a predefined order.
The user concentrates on performing a main + secondary task simultaneously, without reference to the numbers written on the cubes.

Condition 4b
The robot brings the cubes to the user in a predefined order. The user concentrates on performing a main + secondary task simultaneously, with increased task complexity (must refer to the numbers written on the cubes).

Study Hypotheses
The model for the study ( Figure 5) and the hypotheses describing the proposed connection between the constructs, user preferences and the study variables (LOA and levels of workload) along for the rationale for the hypotheses are presented as follows: We suspect that at all workload levels, high LOA will enable the users to perform efficiently and effectively since the high LOA involves the robot carrying out most aspects of the main task which would likely improve performance [37]. Therefore, we propose: Hypothesis 1. Quality of task (QoT) execution will be higher with high LOA than with low LOA for all workload levels.
Several meta-studies conducted regarding levels of automation [38], ref. [39] seem to suggest that the workload experienced by users is influenced by the LOA of the system, particularly in situations of routine performance. This does not discountenance the effect of task complexity but seems to point to the effect level of workload may have in low task complexity. Since a major component of usability is the users' perception of the system use [40] along with effectiveness and efficiency, which high LOA will likely increase, we posit: Hypothesis 2. Usability will be higher with high LOA than with low LOA for all workload levels. We suspect that at all workload levels, high LOA will enable the users to perform efficiently and effectively since the high LOA involves the robot carrying out most aspects of the main task which would likely improve performance [37]. Therefore, we propose: Hypothesis 1. Quality of task (QoT) execution will be higher with high LOA than with low LOA for all workload levels.
Several meta-studies conducted regarding levels of automation [38], ref. [39] seem to suggest that the workload experienced by users is influenced by the LOA of the system, particularly in situations of routine performance. This does not discountenance the effect of task complexity but seems to point to the effect level of workload may have in low task complexity. Since a major component of usability is the users' perception of the system use [40] along with effectiveness and efficiency, which high LOA will likely increase, we posit: Hypothesis 2. Usability will be higher with high LOA than with low LOA for all workload levels.
Research has revealed that as automation increases, workload is expected to decrease, particularly if the automation is properly designed and does not provide new challenges and tasks related to monitoring or other forms of engagement [39]. Moreover, in the design of adjustable robot autonomy in human-robot systems, research shows that as task complexity increases, robot effectiveness is likely to reduce if the robot is operating at higher autonomy [41]. Users seem to intuitively understand that autonomous systems could encounter difficulties in more complex situations with high uncertainty [42] Therefore, in terms of user preferences, we propose: Hypothesis 3. Participants will prefer high LOA to low LOA for high workload and low LOA to high LOA when task complexity is increased.

Participants
Eighty undergraduate industrial engineering third year students (44 females, 36 males, mean age = 26, SD = 1.4) participated in the study. All students had experience with Research has revealed that as automation increases, workload is expected to decrease, particularly if the automation is properly designed and does not provide new challenges and tasks related to monitoring or other forms of engagement [39]. Moreover, in the design of adjustable robot autonomy in human-robot systems, research shows that as task complexity increases, robot effectiveness is likely to reduce if the robot is operating at higher autonomy [41]. Users seem to intuitively understand that autonomous systems could encounter difficulties in more complex situations with high uncertainty [42] Therefore, in terms of user preferences, we propose: Hypothesis 3. Participants will prefer high LOA to low LOA for high workload and low LOA to high LOA when task complexity is increased.

Participants
Eighty undergraduate industrial engineering third year students (44 females, 36 males, mean age = 26, SD = 1.4) participated in the study. All students had experience with both computers and robots. Participation was voluntary and every participant received compensation in the form of a bonus point contributing to a credit in an academic course. The participants completed a preliminary questionnaire which included demographics questions for the participants and the negative attitudes towards robots scale (NARS) [43].
The NARS results revealed that 21.06% of the participants had a negative attitude towards situations and interactions with robots while 63.65% were neutral about it. 26.58% had highly negative attitudes towards the social influence of robots, 47.61% had a low attitude and 25.81% were neutral about it. 65.82% had a highly negative attitude towards the concept of robots having emotions, 8.87% were indifferent about it while 25.31% had a low negative attitude towards it.

Experimental Procedure
Explanation was provided to the participants noting the robot would operate differently in the two trials. To avoid bias, the details of each trial in terms of LOA was not explained to them. They were told that a post-trial and final questionnaire will be provided to express their observations, assessments, and preferences. Then, the participant experienced two experimental trials in which they collaborated with the robot to assemble the configuration that appeared during the GUI in a specific LOA (high/low) in random order. After each trial, they completed a post-trial questionnaire regarding their experience with the robot. At the end of the two trials, each participant completed a final questionnaire where they indicated their preferred level of automation. The experimental design and protocol were approved by the departmental ethical committee.

Objective Measures
Effectiveness: Accuracy of the robot during the task-calculated from the number of times the robot erred in bringing the cubes (e.g., failed to catch a cube, brought an incorrect cube). These are system errors to portray the context of a system whose performance may not be absolutely optimum at all times. Performance in the secondary task was measured as the number of stages they passed in the secondary task (for the participants that experienced the higher workload).
Efficiency: Total time (in seconds) that it took the participant to complete the task for each trial. In the higher level of automation, the total time was constant since depended on robot motions only.

Subjective Measures
The subjective measures were collected through questionnaires that included questions regarding the participants' experience with the robot. The post-trial questionnaire was prepared as a 5-point Likert scale ranging from "1 = strongly disagree" to "5 = strongly agree" through which participants were expected to express their experience and assessments. The questionnaire included NASA-TLX questions [17] to assess perceived workload in relation to the system efficiency. The raw NASA-TLX scores were added without the weights to provide an estimate of the overall workload (RTLX aggregation technique). The post-trial questionnaire also included questions from the technology acceptance model (TAM) to assess perceived ease of use [44]. The final questionnaire assessed user preferences regarding LOA modes and their perceptions as they collaborate with the robot at specific LOA modes.

Constructs
The dependent variables were defined through two constructs: QoT execution and usability. These constructs were derived from the objective and subjective measures explained above (mapping is provided in Figure 6). They were adapted to the context of human-robot collaboration from the ISO 9241-151 guideline [40,45] as follows: . 2021, 11, x FOR PEER REVIEW 8 of 16 Quality of task (QoT) execution. The extent to which specific goals in a task are accomplished to a specified degree of accuracy for a specified time period [46]. This construct involves effectiveness and efficiency of the collaboration. Effectiveness of the collaboration was evaluated by the accuracy and completeness of the task which the human and robot cooperate to execute. The efficiency of the collaboration depends on resources such as time and human effort spent to achieve the required goal [47].
Usability. The extent to which the robotic system can be used to achieve specified goals with effectiveness, efficiency and satisfaction in a specified context of use (adapted from [40]). This construct, in this study, is composed of effectiveness, efficiency in addition to satisfaction derived from the perceived ease of use, perceived workload and perceived reliability of the system. All these variables could affect the degree to which the human operator believes that working with the robot will be free of difficulty or great effort. This is an adaption from [44] in the information technology domain to the context of HRC. They constitute the user's perception regarding use of the system and is essential to ensure that the human can successfully team up with the robot to achieve such collaboration [35]. A negative user perception could lead to disuse of the support the robot can provide in the collaboration [48]. In the current study, the usability construct was comprised of the QoT measures, along with other user perceptions on ease of use, workload, and reliability.

Analysis
A generalized linear mixed model (GLMM) was applied to analyze the data with the LOA, and workload as independent variables. To combine variables for the constructs, multivariate analyses of variance (MANOVA) was used. The analyses considered all the constituent variables within constructs and combined them into a composite variable.

Quality of task (QoT) execution.
The extent to which specific goals in a task are accomplished to a specified degree of accuracy for a specified time period [46]. This construct involves effectiveness and efficiency of the collaboration. Effectiveness of the collaboration was evaluated by the accuracy and completeness of the task which the human and robot cooperate to execute. The efficiency of the collaboration depends on resources such as time and human effort spent to achieve the required goal [47].
Usability. The extent to which the robotic system can be used to achieve specified goals with effectiveness, efficiency and satisfaction in a specified context of use (adapted from [40]). This construct, in this study, is composed of effectiveness, efficiency in addition to satisfaction derived from the perceived ease of use, perceived workload and perceived reliability of the system. All these variables could affect the degree to which the human operator believes that working with the robot will be free of difficulty or great effort. This is an adaption from [44] in the information technology domain to the context of HRC. They constitute the user's perception regarding use of the system and is essential to ensure that the human can successfully team up with the robot to achieve such collaboration [35]. A negative user perception could lead to disuse of the support the robot can provide in the collaboration [48]. In the current study, the usability construct was comprised of the QoT measures, along with other user perceptions on ease of use, workload, and reliability.

Analysis
A generalized linear mixed model (GLMM) was applied to analyze the data with the LOA, and workload as independent variables. To combine variables for the constructs, multivariate analyses of variance (MANOVA) was used. The analyses considered all the constituent variables within constructs and combined them into a composite variable. Tukey's honestly significant difference (Tukey's HSD) test were used as the post-hoc test for multiple comparison. The tests were designed as two-tailed with a significance level of 0.05. The items in the user preferences questionnaire were analyzed using ANOVA to assess the effect of workload on their preferences for the LOA mode they experienced.

Results
Results of the assessments using the constructs (QoT execution and usability), details of the user preference regarding the LOA modes and a comparison within the workload groups are presented below.

QoT Execution
The interaction of LOA and workload had significant effect (F (3, 152) = 5.198, p = 0.002) on the QoT execution. The QoT execution was higher at the high LOA when the workload was low compared to other LOA-workload combinations, confirming H1. LOA (F (3, 150) = 45.15, p < 0.001) and workload (F (3, 152) = 18.725, p < 0.001) were also significant as main effects on the QoT execution. The high LOA produced better QoT execution compared to the low LOA. Best results were obtained for low workload as expected. When the workload is high, the high LOA also produced a better QoT execution compared to the low LOA. Details of the constituent variables in the QoT execution (effectiveness and efficiency) are presented below:

Usability
The interaction of LOA and workload on usability was not significant (

User Preferences
A one-way ANOVA revealed that there was a significant difference between workload groups (F (3, 76) = 9.276, p < 0.001). When comparing LWL and HWL, high LOA was preferred. However, when comparing between MWL1 and MWL2, low LOA was preferred for the MWL1 (confirming H3). More details regarding user preferences for the LOA modes between the workload groups are depicted in Figure 7.

Comparison between Workload Groups for Different LOA Modes
Multiple comparison made between the different workload groups with details on each LOA mode for groups that were significantly different are presented in Table 2. Results revealed that at low LOA: QoT execution is higher when workload is lower; usability is higher when a secondary task is involved, and user preference tended towards low LOA when complexity increases. However, at high LOA: QoT execution was the same for all workload types except when complexity is involved; usability was higher when a secondary task is involved, and user preference tended towards high LOA when a secondary task is involved.

User Preferences
A one-way ANOVA revealed that there was a significant difference between workload groups (F (3, 76) = 9.276, p < 0.001). When comparing LWL and HWL, high LOA was preferred. However, when comparing between MWL1 and MWL2, low LOA was preferred for the MWL1 (confirming H3). More details regarding user preferences for the LOA modes between the workload groups are depicted in Figure 7.

Comparison between Workload Groups for Different LOA Modes
Multiple comparison made between the different workload groups with details on each LOA mode for groups that were significantly different are presented in Table 2. Results revealed that at low LOA: QoT execution is higher when workload is lower; usability is higher when a secondary task is involved, and user preference tended towards low LOA when complexity increases. However, at high LOA: QoT execution was the same for all workload types except when complexity is involved; usability was higher when a secondary task is involved, and user preference tended towards high LOA when a secondary task is involved.

Discussion
The main influences and interacting influences of LOA in HRC in an assembly task context, considering different levels of workload is summarized in Table 3. Table 3. Summary of findings.

Constituent Measures Significant Effects Finding
QoT execution Efficiency; effectiveness LOA (p < 0.001); workload (p < 0.001); LOA*workload (p = 0.002) LOA and workload had significant effect on the QoT execution. The QoT execution was higher at the high LOA.

Usability
QoT execution measures; perceived ease of use, perceived reliability, perceived workload LOA (p < 0.001); Workload (p < 0.001) The usability was higher at high LOA. The workload had more influence on the constituent variables, with the LWL resulting in higher usability.

User preferences
User choices regarding LOA modes Workload (p < 0.001) Most of the participants preferred the high LOA for both LWL and HWL. In the medium workload levels, the low LOA was preferred for the MWL1 where some task complexity was involved

Influence of LOA
In HWL situations, where additional resources are needed to complete the task in the least possible time and with minimal effort, high LOA is preferred. This corresponds with the observations made in the meta-analyses conducted in [38,39] where several automationrelated data where analyzed. It also agrees with the characteristics of the suggested line of solution in workload demands amidst multiple resources as elaborated upon in [37]. However, in cases where complexity is involved, as seen in the results for the LOA preference of participants in the medium workload category, a low LOA can be considered. Most participants seem to prefer a low LOA when the task complexity is high. This confirms H3, and is also in agreement with previous studies where it was stated that a higher LOA may not always give a positive outcome in situations where uncertainties, and higher probabilities of failure exist [38,39]. In high complex tasks where high component and coordinative complexity increases the probabilities of failure [23,49], humans usually have a higher potential to better manage unknown or unexpected situations [50,51]. This reinforces the significance of evaluating LOA modes alongside different workload situations as emphasized in [52] for various contexts and causes of workload. It also calls for further assessments using these constructs.

Workload Considerations
Workload had significant influence on most of the measures. The significant effects were seen in effectiveness and efficiency leading to reduced QoT execution in situations where the workload was high. This is consistent with the literature highlighting the contribution of task-related demands (such as mental, temporal, and physical demands, including complexity demands involved in the HRC task) to workload, which could negatively influence resources available to complete task at hand [15].
The medium workload category more clearly reflects some of the differences in additional workload which can be induced by a secondary task or task complexity. Secondary task inclusion (depicted in MWL2) seems to produce a higher perception of workload compared to complexity in the task (depicted through MWL1). This could explain the reason why most users preferred the high LOA (which autonomously executes more aspects of the task) compared to the low LOA for MWL2. The LOA option seems to provide more mental space for the users to execute other tasks, particularly when the automation functioned well, as suggested in [38,39].
This difference in the medium workload category also brings into prominence the relevance of task complexity, specifically the influence reflected through the perceived reliability where MWL1 (reflecting higher complexity) condition was perceived more reliable compared to MWL2 (reflecting secondary task influence). This could be a result of higher uncertainty and failure probabilities which complexity induces as elaborated in [53,54]. It is therefore understandable that users preferred low LOA to the high LOA in this level of workload (where the task complexity exists) where they seem to have an increased sense of control over the operation [55]. This enables them to better manage the higher uncertainties in this condition (through the low LOA) compared to relying on the robot (through the high LOA). The results reveal that both objective and subjective complexity considerations as noted in [21] should be considered along with the suitable LOA modes for such HRC assembly tasks. This consequently affects the QoT execution and usability of the system.

Limitations
Evaluation was performed with users who had experience with computers and robots. We expect these results to be amplified with users who have experience in real industrial setting. We are also cognizant of potential differences in the subjective assessment of the students in comparison to professionals in an industrial setting since this plays a role in the perception of the users working alongside a robot in a work setting [56]. We therefore consider the results obtained with caution, with the perception that these could be relatively equivalent to assessment with novice operators and different from expert or professional assessments.
The LOA and levels of workload design is simplified for research purpose and not fully representative of the degree of automation, workload levels demanded in more industrial settings. The results obtained, therefore, serve as building blocks and insights for further developments where more detailed automation, workload and complexity conditions are tested in sample industrial settings. Some other social aspects of interacting (such as verbal [57] and non-verbal communication methods [58]) with the robot for the collaborative work were not explicitly investigated in this study. However, further research should also investigate the interplay of the socio-technical aspects of the collaboration while also considering economic and societal issues to understand fuller dimensions of improved HRC in industry [56,59,60].

Conclusions
This paper presented the influence of LOA on a human robot collaborative assembly task considering different workload levels. The user study yielded valuable insights into participants' preferences and influence of LOA and workload. The study also introduced two constructs for the evaluation: quality of task (QoT) execution and usability. The evaluation obtained through these constructs highlighted their potential for use in HRI studies. The study has served to provide support tools to further align manufacturing strategies and automation decisions putting into consideration level of workload to further improve productivity.
The QoT execution construct also pointed to the significance of combining efficiency and effectiveness together as a single variable. It revealed the influence of the LOA and workload in the extent to which goal of the task was accomplished under specified degree of accuracy and duration of the task. The usability construct was significant in revealing the combined effect of QoT execution and user perceptions of the ease of use, workload, and system reliability. The interactive effect of LOA and levels of workload on this construct pointed to the added value which user perceptions contribute when combined with the QoT measure.
We recommend a high LOA to support the user when the workload is high. A high LOA could reduce the stress or pressure of additional secondary tasks which the robot could support in. This was observed in the outcome of the user preferences which tended towards higher LOA when the workload was high. It also agrees with the observations of [38] in their meta-analyses considering the influence of LOA on workload. High LOA, when designed effectively, helps to extend the capabilities of the user to attend to other tasks concurrently as noted by [42,61]. However, lower LOA is helpful when high task complexities are involved, for which failure performance may occur as also noted in [39]. An adaptive LOA design that takes these outcomes into consideration is therefore recommended for further investigation.
There may be significant differences in the influence of these variables when observed in other settings, with different forms of robots, tasks and robot feedback modalities [62] and with the perception of different users as emphasized in [63]. Future work should evaluate different forms of increased workload. The workload design can be fine-tuned to portray distinct types of workload demands such as physical, cognitive and temporal demands during the task. Evaluation should also be conducted with other forms of tasks e.g., with a mobile robot delivering items and with other populations. Ongoing research is aimed at performing studies with older adults for daily living tasks and for nonprofessional users, putting into consideration the influence of demographics on the changes automation brings [64]. LOA has proven to influence performance for older adults [12]. We expect the effect of the levels of workload to amplify with them. The change of preferences and the differences in the reaction and performance of the older adults should be examined with different LOA options for different workload levels.