Dynamic Graphical Instructions Result in Improved Attitudes and Decreased Task Completion Time in Human–Robot Co-Working: An Experimental Manufacturing Study

Collaborative robots offer opportunities to increase the sustainability of work and workforces by increasing productivity, quality, and efficiency, whilst removing workers from hazardous, repetitive, and strenuous tasks. They also offer opportunities for increasing accessibility to work, supporting those who may otherwise be disadvantaged through age, ability, gender, or other characteristics. However, to maximise the benefits, employers must overcome negative attitudes toward, and a lack of confidence in, the technology, and must take steps to reduce errors arising from misuse. This study explores how dynamic graphical signage could be employed to address these issues in a manufacturing task. Forty employees from one UK manufacturing company participated in a field experiment to complete a precision pick-and-place task working in conjunction with a collaborative robotic arm. Twenty-one participants completed the task with the support of dynamic graphical signage that provided information about the robot and the activity, while the rest completed the same task with no signage. The presence of the signage improved the completion time of the task as well as reducing negative attitudes towards the robots. Furthermore, participants provided with no signage had worse outcome expectancies as a function of their response time. Our results indicate that the provision of instructional information conveyed through appropriate graphical signage can improve task efficiency and user wellbeing, contributing to greater workforce sustainability. The findings will be of interest for companies introducing collaborative robots as well as those wanting to improve their workforce wellbeing and technology acceptance.


Introduction
A recognised challenge facing the future of the workplace is the need for a more sustainable workforce [1]. Many developed countries (Australia, Canada, France, Germany, Italy, Japan, South Korea, the United Kingdom, and the United States) predict a decline in working-age populations, and identify automation as playing an increasingly essential role in providing the productivity boost required by the global economy [2]. Collaborative robots offer a particular opportunity to support workforce sustainability, by shielding workers from hazardous operations, repetitive motions, and physically demanding operations, whilst at the same time retaining the human workers' skills and attributes. However, sustainability goes beyond mere productivity, and requires interdisciplinary study at the intersection of the social, economic, and environmental dimensions [3]. Of key consideration are the human factors that affect the deployment, acceptance, and efficient usage of newly developed technologies.
For example, in the European Union, the number of older people in workforce is increasing, putting pressure on public funds and resulting in an increase in the retirement age [4]. As the 100-year-old citizen becomes more common, society will have to consider the needs of the 70-year-old care provider. The deployment of collaborative robots to undertake the necessary repetitive, physical, and demanding activities is expected to lead to greater workforce resilience, less physical injuries and strains, and longer retention in the workforce. However, increased use of automation may also result in changes in social and psychological (both emotional and cognitive) demands on users, if not appropriately managed [5].
A lack of confidence from the workforce is a key barrier to technology achieving anticipated positive impacts [6]. Moreover, as the autonomy and complexity of robots increases, so too does the need for confidence and understanding of their capacities, in order for humans to effectively collaborate with them [7]. Trust is a key element affecting an individual's confidence towards robots [8][9][10][11], while from an organisational aspect, empowerment, communication, controllable workflows, and user knowledge and experience are key for technology acceptance [12,13]. Conversely, a lack of understanding of task-related procedures can be a barrier for technology acceptance among the workforce [12], with rejection issues exacerbated further if users feel they do not have enough information or training to undertake a task with the new technology [6]. A lack of perceived control and confidence in one's ability to accomplish a task using new technology can lead to decreased concentration on the task, changes in situational awareness, and an increased number of accidents [14,15], but this can be countered by establishing effective measures of communication, and making the user aware of the state of the robot [15]. As empowerment and knowledge are important factors in technology acceptance, and thus contribute to workforce sustainability, it is important to reflect what individual factors could affect this acceptance.
As previously mentioned, controllable workflow and experience [13] are two of the key factors for technology acceptance. Control of the workflow and experience can be related to an individual's outcome expectancies-the belief that one's efforts will result in one's desired rewards is an important part of expectancy theory for predicting human behaviour [16]. Although there is a debate about the similarities and differences between outcome expectancy and self-efficacy [17], outcome expectancy is related to self-concept and frame-of-reference (comparison of oneself to others on social and dimensional criteria), while self-efficacy relates to a general notion of one's abilities [18]. Thus, knowing how to perform the task (either by having done it before or/and having the information on how to perform it) may boost one's outcome expectancy from working with a robot, and hence have a positive impact on the integration of robotic technology in the work environment.
With the rapid growth in industry 4.0, technological changes to the work environment, and in particular changes to the employees' roles at work, can be challenging and stressful [19]. Understanding the likely emotional impact of work/task change (such as concerns that robot deployment may lead to redundancies [20]), and how employees may individually experience and regulate emotions, may be beneficial to industries in avoiding substituting one sustainability challenge (such as an aging workforce) with another (such as an apathetic younger workforce). Well-established models of emotion regulation [21,22] point to a dual-component emotion regulation process of cognitive reappraisal and expressive suppression. Cognitive reappraisal supports reframing emotional experiences and can lower their intensity [23], supporting socially appropriate expressions [21,22,24]; in contrast, expressive suppression relies on behavioural masking (while not addressing the experience) that may be less effective in regulating expression [23]. Additional strains of having to manage emotional demands, particularly for those who tend to use less-effective methods such as expressive suppression, may even lead to some mental health issues [25][26][27]. In terms of robotics, the role of regulating emotions has been investigated in terms of attempting to regulate it [28,29]; however, the question of how emotion regulation affects human-robot interactions in manufacturing has been left unaddressed.
Prior research in collaborative robotics for manufacturing contexts indicates that participants who had access to information on a robot's operational state had positive emotional experience during successful co-working trials, whereas this was not the case for those without information available [30]. The authors of [30] used age as a covariate when examining emotional experience and expression (although age is known to improve emotion regulation capabilities [31]) rather than specifically looking at emotional regulation strategies. Nevertheless, the above-mentioned work highlights that even in low-stakes contexts (i.e., students participating in a simulated manufacturing task), emotions can be meaningfully shaped when working with collaborative robots. In real manufacturing contexts, new technology acceptance and anxiety towards changing work situations may deeply impact and be impacted by interaction. Given the differences in efficacy and individual regulation strategies [23], these matters may be more pronounced for those implementing expressive suppression, while cognitive reappraisal might encourage people to embrace changes and consider how they may benefit.
Lack of knowledge and control of the workflow in a work environment is costly in a number of ways: from stress, burnout, and fatigue, to physical injuries and accidents [32][33][34]. Possible solutions to avoid this, or at least to minimise harm, include efficient information communication. By knowing task demands and individual responsibilities, people are more able to cope with work pressures as well as concentrate on the task they are performing, and thus avoid potential injuries [14]. Although information transfer is possible through a variety of mediums, signage-the graphical display of signs and symbols to communicate a message [35]-is one of the most cost-effective, convenient, and adaptable [36]. In the study reported in this paper, signage was used to convey simplified task-related information relating to different sub-steps between a person and a robotic arm [37]. Some studies suggest that the combination of signage modalities, i.e., dynamic videos with static text-based information, helps to aid engagement and task performance [38], while other studies point to the greater benefit of new information retention via dynamic modes [39], as it can aid information transfer from concrete to abstract levels [40,41]. Furthermore, dynamic signage that is screen-based (such as billboards or road maintenance signs) is: (i) easier to adapt to changing situations (turn on and off depending on the need), (ii) less costly in terms of replication and curation, and (iii) adjustable depending on the targeted population [42]. Dynamic signage also promotes faster and more accurate decision-making in high-pressure situations, such as emergency evacuation, with environmental stressors (e.g., high background noise) [43]. In the current study, we explored the use of both static and dynamic instructional signage. The results [41,44,45] suggest that participants benefited more from dynamic signage than static signage, when completing essential tasks.
In this study, we investigate how dynamic information communication during a human-robot collaborative task may impact on user expectancies and attitudes towards collaborative robots. The study builds upon and extends our previous work [30] along methodological and participant population lines. In terms of methodology, the current study used refined dynamic signage, co-created with industry users [20], that changes in response to users' actions. In terms of the participant population, the current study recruited employees of an industrial partner (our previous study recruited university staff and students). Moreover, the experiment described in this study was conducted on our industrial partner's premises, and in relation to their first deployment of collaborative robots, and staff were aware of these plans and were motivated to participate.
The reported study aimed to quantify the effects of dynamic signage on user task performance (completion time and accuracy), and the possible changes in attitudes towards robots. Therefore, an experimental study was conducted to explore the following hypothesis:

Hypothesis 1 (H1).
Users' performance will improve in the experimental condition (dynamical signage) compared to the control condition (no signage). This effect was observed in the previous study [30] undertaken with a student population.
Furthermore, workforce sustainability relates to wellbeing and, in the manufacturing context, safety and technology engagement are essential to increase this wellbeing [8][9][10][11][12][13]. An appropriate level of trust in automation is necessary for increased efficiency and safety in the work environment [46]; however, high outcome expectancy might be necessary for successful technology adoption and appropriate use [47][48][49]. Therefore, we predict: Hypothesis 2 (H2). The experimental group participants presented with dynamic signage will have lower scores on the Robot Anxiety Scale and Negative Attitudes towards Robots Scale, and their outcome expectancy of using robots will increase after the interaction with a robot.
The expected relationship between variables is presented in Figure 1, where it indicates that the effects of signage are expected to have a direct effect on task performance (H1) and a direct effect on HRI attitudes (NARS and RAS), as well as on the user outcome expectancies as a result of their performance (H2). The strength and novelty of this study comes from its participant population (shopfloor operators on a welding process) and that it is a controlled experiment on an active manufacturing premises. Our selected methodology (controlled experiment) allows us to make statistical inferences about how dynamic screen-based signage could enhance humanrobot collaboration in terms of task efficacy and user wellbeing. Therefore, the results can be applicable not only in the manufacturing sector, but in any sector where humanrobot collaboration is introduced and information about tasks is needed to encourage the development of a more sustainable workforce.
The paper is structured as follows: Section 2 will present the methods and materials used to explore the effects of dynamic signage in Study 1 (field study), with results provided in Section 3. Section 4 details methods and materials comparing dynamic and static signage effects (combining this and a previous study), with the results provided in Section 5. Section 6 will discuss our findings relating to the effects of signage on user task completion, efficiency, and wellbeing, with conclusions drawn in Section 7.

Materials and Methods-Study 1 2.1. Design
The aim of the study was to quantify the changes on the task performance and user wellbeing in response to graphical signage, and the researchers felt that experimental design was the best approach, allowing them to conduct inferential statistical analysis to generalise the findings. The study used a 2 × 2 mixed design, comprising two independent conditions that both completed surveys on two occasions as repeated measures (pre-and post-HRI). The two independent conditions were: presence of task-relevant, screen-based, dynamic signage (experimental condition), and no signage present (control condition). Participants in both groups completed four brief surveys both before and after interaction with the robot (see Section 2.7: Measures). The study was approved by the University of Sheffield Ethics Committee prior to commencing.

KUKA iiwa Lightweight Arm
A KUKA Intelligent Industrial Work Assistant (iiwa; KUKA Roboter GmbH) was used for the human-robot co-working task in this study. The KUKA iiwa is particularly suited to human-robot co-working scenarios, allowing direct human interaction due to its configurable safety measures. It is a versatile robot and configurable to afford a wide variety of user interactions; however, this versatile design does not communicate constraints on user interaction (e.g., [50]). As such, it may become necessary to provide supplemental information (such as graphical signage) to guide users in successful interaction.
The KUKA iiwa was controlled via [51] API, set to be operated in compliant safe mode 'T1' with limits on speed and a requirement for experimenter monitoring throughout the HRI scenario.

Graphical Signage
For the current study, a paper-based graphical signage [52] was updated following collaborative design workshops with employees from the industry partner [20]. As with the paper-based signage, key information on HRI, such as when it is safe to touch the robot, the expected speed of robot movement, and operational area, was presented visually. The updated dynamic screen-based signage provided real-time information to the user about robot operational processes (e.g., signage updating to reflect a change from the robot moving with operator guidance to moving autonomously).
A computer monitor (20-inch screen diameter) was positioned on the right side of the robot at 70 cm away from the desk edge, where participants were standing. This screen was present across conditions. For the control condition, the screen was turned off throughout the interaction with the robot. For the experimental condition, the screen-based dynamical graphic signage was presented on the computer monitor. At the start of the interaction, animated gifs portraying information about the robotic arm were presented for 30 s (i.e., the speed and reach of the robot, direction of robot movement (x and y axes) and required applied force from the user to navigate the robot). During the trials, the signage indicated when participants should manually move the robot, and when the robot was completing a process autonomously ( Figure 2). As Figure 2 depicts from the left to the right side: Once the trial started, the signage would indicate that it is safe to touch the robot and navigate the robotic arm over the tube containing one of the bolts (green signage with moving hand towards robot). Once the participant would position the robotic arm over a particular tube, the signage would change from green to red where the hand is not touching the robot. Following this, each robotic arm movement (readjusting for the exact tube position, collecting the bolt from the tube, returning the bolt to the user) would be accompanied by a red sign. The signage once again would change to green once it was the users turn to navigate the robotic arm over the next tube containing a bolt. Figure 2. Sequence of dynamic signage display relating to "touch" and "do not touch" the robot.

Participants
Participants were drawn from the industry partner's manufacturing workforce. The industry partner produces a high-volume of low-value components used in building and infrastructure, where the component assembly process would be classified as low-skilled manufacturing. The partner did not make use of robots in the manufacturing process at the time of the study but was underway of installing its first collaborative robotics cell. This participant population was ideal for the study as a manufacturing workforce that had not interacted with robotics before.
Forty workers in manufacturing with no exposure to the graphical signage before the experiment and no prior experience in using robots participated in the study. Participants were invited to volunteer for the study by placing an advertisement in the company a week before the set trial date, and on the day asking on the shopfloor for the operators to volunteer (12 from morning shift, 23 from the afternoon shift, 2 from night shift, and 3 others (2 supervisors and 1 work experience trainee)). Participants were randomly allocated to a condition. One participant's data was excluded from the analysis because the pre-test questionnaire was left blank; overall, data from 39 participants was included in the final analysis (21 Experimental, 18 Control condition). Participants' demographic and baseline data are presented in Table 1.

Task and Procedure
The study was conducted onsite at the industrial partner's factory (Keighley, UK), in a process development room to achieve a realistic working experience. The experiment consisted of three main stages: (1) completing pre-trial measures, (2) interacting with the robot in the manufacturing-type task, and (3) completing post-trial measures.
Before starting the experiment, participants received a study information sheet and signed their consent to take part. Participants were also verbally informed about the procedures, their rights, and provided with an opportunity to ask questions concerning the experiment.
Once the consent was signed, participants completed basic demographic information (age, sex, and tenure at the company) and the pre-trial questionnaires regarding their attitudes towards robotics, experience in using robotics, confidence in ability to complete the task, and emotion regulation (see Section 2.6. Measures and Table 1 for baseline and demographic details).
After participants completed the questionnaires, they were directed to a second room containing the KUKA iiwa robotic arm for the HRI co-working scenario ( Figure 3). Participants were given verbal instructions for the task: "On the table in front of you, there are 16 narrow tubes; six of these contain M5 bolts. These bolts need to be collected, however they are inaccessible to people, and, although the robot can reach and pick them, it cannot identify which tubes contain bolts. You can spend up to 10 min on this task". Participants engaged in the HRI scenario on an individual basis, with the experimenter present in the room solely to ensure safe interaction-the experimenter did not offer further instruction on the task.
As outlined in the verbal instructions, the bolt collection task could be completed only through collaborative working between the participant and the robot. Nonetheless, both the task's goal and physical actions required by the user to achieve the goal were simple [30]. The complexity in the task lies in participants exploring and learning how and when to interact with the robot. For example: a user manually guiding the robot with excessive force (how) or pushing the robot while it moves autonomously (when) will enact a safety feature, locking the arm in place and requiring a reset of the system. Participants in the experimental condition were presented with the graphical signage (see Section 2.3 Graphical Signage) on how to interact (delivered at the start of the scenario), and when to interact (delivered throughout the scenario). Participants in the control condition were not presented with the graphical signage. The HRI scenario took 10 min or until the participants had collected all the bolts, whichever came sooner. The experiment was video recorded to obtain behavioural measures.
After the human-robot interaction task was completed, participants were directed to another room to complete the same questionnaires as those pre-interaction concerning attitudes towards robots. Participants also completed measures indicating their recollection of the task and recognition of the graphical signage (experimental condition only). Participants were debriefed after completing the questionnaires. Participation in the experiment took approximately 30 min on average and was completed in a single session.  Participants' performance in the collaborative task was recorded in terms of their task accuracy, overall response time, and trial response time. Task accuracy was calculated by dividing the number of successfully completed trials by the number of attempted trials overall (failed attempts could include directing the KUKA arm to the wrong location or causing the arm to lock in place). Overall task response time was simply calculated by the total duration it took participants to collect all bolts, including delays due to operator errors. This was capped at a maximum of 600 s, at which point the experimenter ended the HRI phase of the study. Trial response time was recorded for each of the trials, and comparisons between trials for each participant may give indications of the degree and rate of task improvement due to practice.

Attitudes towards Robotics
Participants' attitudes towards robots at pre-and post-interaction were assessed with three factors drawn from two scales: Negative Attitudes towards Robots Scale (NARS [53]) and Robot Anxiety Scale (RAS [54]).
The factors from NARS concerning negative attitudes towards interaction with robots (S1) and towards social influences of robots (S2) were used. Participants indicated their level of agreement on each statement in the questionnaire on a five-point scale (from 1strongly disagree to 5-strongly agree). The factor from RAS measuring anxiety towards the behavioural characteristics of robots was used. Participants indicated how anxious they feel about each out of four statements on a six-point scale (from 1-I do not feel anxiety at all to 6-I feel very anxious). Reliability in both scales was high (Cronbach's α: NARS pre-trial = 0.78, post-trial = 0.88; RAS pre-trial = 0.96, post-trial = 0.96).
Participants' expectancies for completing the task pre-and post-interaction were assessed with the Outcome Expectancy Scale [55]. Participants indicated their level of agreement on statements concerning their ability to complete the HRI task on a five-point scale (from 1-strongly agree to 5-strongly disagree). Items in this scale were adapted to reflect the HRI scenario, replacing the words "software package" with "robot". Reliability in this scale was high (Cronbach's α for pre-trial = 0.91, post-trial = 0.912).

Individual Differences at Baseline
A further five pre-trial questionnaires were used to account for any baseline differences of participants' experience with robotics and computers, their self-efficacy, risk-taking, and emotional regulation.
The Experience with Robots scale [56] assessed how often participants have engaged with robotics either directly, such as attending robotics-related events or building a robot, or indirectly, such as through media. Participants indicated their answers on a 6-point scale (0, 1, 2, 3, 4, 5, or more times), which was moderately reliable (Cronbach's α = 0.63). Participants' experience with computers was assessed via self-reports of the number of hours per week participants use a computer for assignments/work, for browsing/socialising, and for playing computer games (indicating which category of games they prefer). Programming expertise was self-assessed on a 5-point scale (1-very inexperienced, 5-very experienced). A measure of participants' self-efficacy towards the task was adapted from [55], replacing the term "software package" with "robot". Participants indicated on ten ten-point items (ranging from 1-not at all confident to 10-totally confident) their confidence in successfully completing the HRI scenario. Reliability in the scale was high (Cronbach's α = 0.92).
The Risk-Taking Index (RTI [57]) assessed participants' everyday risk-taking attitudes now, and in the past, on a five-point scale (1-never, 5-very often). Two statements assessing participants' past and present safety risks were administered pre-trial (Cronbach's α = 0.93).
Participants further completed [22] a questionnaire on emotion regulation strategies of expressive suppression and cognitive reappraisal. Participants indicated their agreement to each of ten statements on a seven-point scale (1-strongly disagree to 7strongly agree), and the scale was moderately reliable (Cronbach's α for reappraisal = 0.73, suppression = 0.78). As with all preceding questionnaires for the study, this questionnaire was delivered in pen and pencil format.

Visual Attention towards Signage
The total duration for participant's gaze towards the digital display was recorded, and the display was present across both conditions: with or without the dynamical signage, depending on the condition. Total gaze duration towards the digital display was calculated using automatic classification of gaze direction using Noldus FaceReader 5.0.

Statistical Analysis
The Study 1 dependent variables of task completion accuracy, response time, the Robot Anxiety Scale scores, the Negative Attitudes Towards Robots Scale scores, the Emotion Regulation Questionnaire scores, the Risk-Taking scores, Robot Experience and Computer Usage scores, and the Self-efficacy towards the task questionnaire scores were normally distributed according to the Kolmogorov-Smirnov test (p > 0.05); therefore, all the following analyses were completed with a parametric test. As a first step of the analysis in Study 1, the differences between the two experimental groups (dynamic signage and control) were compared at a baseline level. The analysis investigated the relationship between graphical signage toward task performance (response time and accuracy), as well as the relationship between graphical signage and HRI attitudes (outcome expectancy, RAS, and NARS), as moderated by task performance, was investigated ( Figure 4). Any significant differences observed at this point were considered as covariates for comparisons of the dependent measures, where applicable. A two-tailed alpha of 0.05 was considered as significant for all comparisons. Results for baseline and demographic measures are considered 'of interest' where p < 0.10.

Baseline Differences
Preliminary checks were run to examine the pre-trial distribution of participants across the two participant groups for demographic measures (sex, age, and tenure) and all pretrial measures. There were no significant differences in demographics across conditions at pre-trial.
There was a difference of interest across conditions in the responses to the emotion regulation questionnaire (t(36) = 1.811, p = 0.078, Cohen's d s = 0.588): experimental condition participants had higher expressive suppression scores compared to control condition participants. There were no further significant differences between groups for any of the remaining pre-trial measures (t(37) ≤ 1.44, p ≥ 0.159, Cohen's d s ≤ 0.462). All further analyses were conducted with expressive suppression as a covariate, unless otherwise specified.

Visual Attention towards Signage
To control for the possibility that the experimental group did not pay attention to the signage and the occurring differences between two groups are not due to the presence of signage, the analysis of gaze duration towards the signage was analysed. The digital display for the graphical signage was present for both conditions, although the presence of the dynamic signage on the display was only for the experimental condition. Gaze duration is expressed as percentage of total time spent on the task. The independent t-test revealed that experimental group participants had a significantly longer gaze duration towards the digital display compared to control group participants (t(37) = 2.69, p = 0.011, Cohen's d s = 0.86; M exp = 31.53%, SD exp = 14.43; M cont = 19.84%, SD cont = 12.45). This result indicates that experimental group participants were looking at the signage, and the following differences between the groups could be attributed to the effect of the signage present during the task.

Accuracy
Participant accuracy is calculated by the number of successfully completed trials as a fraction of attempted trials. Effects of the dynamic signage on task accuracy were examined using a univariate ANCOVA (with expressive suppression as a covariate). The results did not indicate any significant difference between conditions (F(1, 35) = 0.45, p = 0.505, η p 2 = 0.013): mean accuracy for the dynamic signage condition was 0.67 (SD = 0.18) and for the control condition was 0.71 (SD = 0.20).

Response Time
The impact of the signage on participants' (successful) trial response time was examined using linear mixed model ANCOVA (between-subject: condition, within-subject: trial number (1-6), covariates: tube position-this covariate is included because participants could collect the bolts in any order, and bolts distal from the default position for the robot after handover to the participant take (linearly) longer to reach than those proximaland expressive suppression). The analysis showed a significant main effect of condition (F(1, 179) = 10.28, p = 0.002, η p 2 = 0.054) and a main effect of trial (F(5, 132) = 2.65, p = 0.025, η p 2 = 0.091), as well as a significant condition and trial interaction effect (F(5, 132) = 2.34, p = 0.045, η p 2 = 0.081; Figure 5). Post-hoc comparison between conditions for each trial showed that participants in the experimental condition were significantly quicker than those in the control condition during trials two and four (F(1, 37) = 9.77, p = 0.004, η p 2 = 0.209; F(1, 30) = 4.62, p = 0.041, η p 2 = 0.133). There were no significant differences between groups for the remaining trials (F(1,24) ≤ 1.83, p ≥ 0.191, η p 2 ≤ 0.071).

NARS and RAS Scales
Further analysis was performed to research the condition effect on Negative Attitudes and Anxiety. First, the change between participants' post-trial scores compared to pretrial scores was calculated. Second, moderated regressions with independent variables, separately accuracy and response time, moderator condition, and covariate expressive suppression, were performed on the dependent variables of NARS and RAS change. The predictive model is specified in Figure 4.
The results showed that accuracy in the task predicts the change of negative attitudes towards robots, moderated by condition (F(4, 33) = 3.29, p = 0.023, R 2 = 0.29); with increasing accuracy, the post-trial NARS decreased compared to the pre-trial NARS. However, this was significant only in the experimental group (t = 2.66, p = 0.012, b = −11.28), but not the control group (t = 1.89, p = 0.067, b = −8.43), see Figure 6.
There was a significant correlation between participants' ratings of the signage effectiveness and their task accuracy scores (r = 0.436, p = 0.048). There were no other significant correlations between participants' ratings of the signage effectiveness and their task accuracy, average response time, or sign recognition (maximum r ≤ −0.218, p ≥ 0.343).

Outcome Expectancy
To measure whether participants' outcome expectancy (higher score indicating more positive outcome expectancy or confidence in achieving positive results) was affected by their performance in the task (accuracy and response time) and moderated by group (experimental vs. control, a moderated regression with independent variable of response time, moderator variable group, and covariate expressive suppression was run on the dependent variable change in outcome expectancy). The analysis showed that response time significantly predicts the change in outcome expectancy (F(4, 33) = 3.58, p = 0.016, R 2 = 0.30). Looking into the group results, the control group showed that post-trial outcome expectancy increased with the decreasing response time (t = 3.74, p < 0.001, b = −0.1453); however, the experimental group showed a consistent perceived outcome expectancy score compared pre-trial and post-trial (t = 0.10, p = 0.922, b = −0.0055). The same model with accuracy as a predictor did not show significant results (F(3, 35) = 1.83, p = 0.160, R 2 = 0.14).

Method-Study 2
Study 1 describes a comparison of an experimental condition (dynamic signage) against a control condition (no signage). To evaluate the effectiveness of the dynamic graphical signage relative to a more traditional static signage, current results were compared against pre-existing data [30]. In that study, participants completed the same collaborative task (pick and place) with the same collaborative KUKA iiwa arm; however, the signage provided for the study was presented statically on a paper printout, positioned on the desk on the left side of the robot (Figure 7). This prior study presented the same visual information on how to interact with the robot throughout the trials as the current study but did not dynamically present information in response to robot status (i.e., it gave no information on when to interact). The dataset consisted of 90 participants over three conditions: 30

Design
This study used an independent measures design: participants interacted with the robot in the presence of either a static or dynamic graphical language. This study compared the data from the experimental condition of Study 1 against the experimental condition of a pre-existing dataset [30].

Participants
Data from the experimental condition in Study 1 and the pre-existing dataset [30] were analysed. This comprises 51 participants overall (21 for the Dynamic and 30 for the Static conditions). Although participants came from the different populations, their average scores on age, NARS, or RAS were not significantly different (p > 0.07), thus allowing the comparison between groups.

Visual Attention towards Signage
For both the current and pre-existing datasets, the total duration for participants' gaze towards the graphical signage was recorded. The graphical language was present for both studies and placed in the same workspace area, and it differed only in its static or dynamic nature. Total gaze duration towards the graphical language was calculated using automatic classification of gaze direction using Noldus FaceReader 5.0.

Effectiveness of Graphical Signs
Participants for both studies (in which the allocation condition meant graphical signage was present) completed two additional measures regarding the graphical signage after their completion of the HRI scenario: perceived effectiveness of the signage and recognition of the graphical signage.
First, the Reference [58] questionnaire, Experimental and Survey Studies on the Effectiveness of Dynamic Signage Systems, was adapted for use from the context of fire-safety signage to reflect the manufacturing-like HRI scenario. This questionnaire assessed participants' perceived effectiveness of signs in general and the effectiveness of the signage for the specific scenario. Participants indicated how much they agree with each of the eight statements on a 5-point scale (from 1-strongly agree to 5-strongly disagree), and reliability for the measures was high (Cronbach's α general effectiveness = 0.90, specific effectiveness = 0.95).
The second measure comprised a simple recognition exercise of the signage displayed during the scenario, and the worksheet for the recognition task included distractor signs that had been developed to mimic the design of the task-relevant signs. These questionnaires were delivered in pen and pencil format.

Statistical Analysis
The Study 2 dependent variables of visual attention towards signage, signa recognition, and effectiveness of signs were normally distributed according to the Kolmogorov-Smirnov test (p > 0.05); therefore, all the following analyses comparing the static and dynamic signage were completed with a parametric test. A two-tailed alpha of 0.05 was considered as significant for all comparisons.

Visual Attention towards Signage
Comparison of gaze duration towards the dynamic and static signage revealed that participants with access to the static signage had a greater gaze duration, expressed as a percentage of total time of trial towards signage, than those who had access to the dynamic signage (t(33.15) = 5.11, p ≤ 0.001, Cohen's d = 1.47; M dyn = 31.53%, SD dyn = 14.43 s; M stat = 12.93%, SD stat = 10.15). Gaze duration is expressed as percentage of total time spent on the task.

Sign Recognition
Independent t-test analysis on a d' sensitivity analysis for recognition of the dynamic and static graphical signage presents a significant difference between groups for recall of signs (t(49) = 2.30, p = 0.026, Cohen's d = 0.65), with an overall sign recognition accuracy of 82.5%. Participants who observed the dynamic signage remembered the graphical signage with greater accuracy than those who observed the static signage (M dyn = 2.42 SD dyn = 0.51; M stat = 2.08, SD stat = 0.53).

Effectiveness of Graphical Signs
The questionnaire measuring signage effectiveness with dynamic signage and the experimental group with static signage was analysed with the independent sample ttest. The analysis revealed that dynamic signage was evaluated to be more effective compared to static signage (t(49) = 5.57, p ≤ 0.001). On average, participants who saw the dynamic signage rated it as more effective than those who the saw static signage (M dyn = 3.68, SD = 0.70; M stat = 2.33, SD = 0.96). However, there was no correlation between accuracy, response time, and sign recollection (neither d', false alarm, nor hit rate).

Discussion
This study explored the effects of screen-based dynamic graphical signage on users' attitudes towards robots and their task performance in an HRI manufacturing scenario. Broadly, the results indicate that the graphical signage has positive effects on both task performance and user attitudes. Users who had access to the dynamic graphical signage showed a better task performance than those who had no signage to support them. This effect was observed for the response-time metric of task performance but not for task accuracy. In contrast with [30], in which participants who saw graphical signage had greater accuracy than those who did not, the presence of the graphical signage did not significantly affect user accuracy in the current HRI scenario (Study 1). Furthermore, user attitudes towards the robot and outcome expectancy have been affected because of their performance and the study condition: the NARS scores were influenced by the task completion accuracy, showing that higher accuracy resulted in a greater decrease in negative attitudes towards robot scores, however only in the experimental group. The control group participants showed a positive relationship between increasing outcome expectancies and decreasing task completion time. Moreover, in comparing participants' self-reports on the effectiveness of the signage, in the current study, the dynamic signage received higher scores than the static signage from the archival data. Finally, the self-report scores on the dynamic signage effectiveness were significantly higher than scores on the static signage effectiveness from the archival study.
Task performance and its efficiency are among the main drives for technology introduction in manufacturing [59,60]. The current study shows that while there was no significant effect of dynamic signage on the user task completion accuracy, there was a positive effect on the task completion time. This result only partially supports our hypothesis and there are several possible explanations for this, some of which are directly addressed in Study 2. It is plausible that the current participants simply did not attend to or recognise the dynamic signage to the degree seen in an earlier study with static signage [30]. While participants looked towards the dynamic signage for a shorter duration than the static signage, they did recognise the signage with over 80% accuracy. Furthermore, participants reported the dynamic signage to be significantly more effective than those who saw the static signage rated it to be. From this, we conclude that dynamic signage was observed and considered by participants as appropriate to the study, and there may be populationlevel effects. Potentially, the signage did not effectively support task accuracy due to the participants' already high potential to accurately complete the task.
Unlike prior work exploring context creation in HRI (e.g., [30]), this study draws from a manufacturing workforce population. This population was chosen specifically to examine the impact of signage amongst people most likely to encounter such materials in the future, particularly as they perform similar operations as those devised for the scenario. As a result, participants in the current study may well be highly practiced in ensuring accuracy in manufacturing: ceiling effects in participants' precision and experience with machinery may limit the impact of the graphical signage in improving accuracy in the HRI scenario. Nonetheless, a critical element of effective manufacturing is working efficiently and processes that reduce the time taken to complete tasks are valuable: the graphical signage supports faster working in the HRI scenario, without a compromise on accuracy.
The main effect of the dynamic graphical signage on response time may arise by creating context for the interactions during the HRI scenario. The signage indicated both how and when participants should interact with the robot. The dynamical nature of the signs meant that these could respond to changes in the robot's state, indicating when the user should take bolts from the robot, when they should push the robot, and when the robot is autonomously completing its part of the task. These may have helped to reduce the time needed to complete a single trial without time-consuming deliberations by the user. This result is consistent with past studies in evacuation environments [33,34], showing that information in a graphical format results in a quicker navigation of unfamiliar environments, in particular with dynamic signage being beneficial [43]. Furthermore, knowledge of when and how to co-work with the robot can increase control of the workflow and the knowledge [12,13], which are important for positive impacts on the workforce sustainability.
The effects of graphical signage were observed not only on the task performance, but also on their attitudes towards robots. Participants' ratings of negative attitudes towards robots decreased as a function of their increasing accuracy on the task; critically, this decrease was significant only for those who saw the graphical signage. Positive changes in attitudes towards robots after interaction are consistent with past studies [61], although the current work identifies that the context of the interaction (crafted by the presence of signage) may have a significant influence. The results from the current study further resemble findings from [30], in which participant scores on the Robot Anxiety Scale scores also decreased as a function of increasing task accuracy-again, in the experimental condition only. Of note, there was a strong positive correlation between participant ratings on the NARS and RAS scales in the current study. Potential future work could closer examine the underlying phenomena that these scales explore, identifying commonalities between the constructs targeted. In terms of the current study, an interesting issue is raised: context of the HRI experience (as shaped by the presence of signage) may play a key role in converting successful interaction experiences into positive user attitudes.
Finally, the study investigated how participants' outcome expectancy differs before and after interaction with a robot. The change between pre-and post-trial expectancy was highly influenced by performance (response time) in the control group; the slower they were at performing the task, the lower their outcome expectancies became, and vice versa. Outcome expectancies are related to motivation [62], intention to perform certain behaviours [63], and job satisfaction [64]. This indicates that the control group's motivation to work with robots in the future was affected by their performance. On the other hand, for participants who were presented with graphical information on how to operate the robot, outcome expectancies remained stable. This might be influenced by whether participants attribute the performance in the task due to them not performing well ("I am slower therefore I do not see positive outcomes") or attributing this externally to technology ("this is how the process works and not a reflection on my ability, therefore my expectations about the outcomes remain the same"). Overall, this result emphasises the need of successful initial experiences while working with robots. However, information communication about robots (in this case in graphical format) is important in the initial stages of new technology deployment as the users are new to the process and have not reached the top of their performance potential. In these early stages, to maintain stable motivation it is necessary for the users to see the potential benefits of working with the technology past initial adoption stages. At the same time, positive outcome expectancy has an impact on greater resilience in hazardous environments [65] and even coping with disease [66].
The results cannot be explained by baseline group differences. In the experiment, participant groups were controlled for demographic information (age, tenure at the company, experience with robots, emotion regulation, and self-efficacy in working with robot measures). The only trend difference between participant groups was in expressive suppression, with experimental group participants having higher scores in this measure. Therefore, expressive suppression was taken as a covariate in further analyses.
Collaborative robotics can provide a solution to increasing issues around workforce aging and sustainability [45]. This technology has the potential to reduce physical strain and increase the safety of low-skilled workers, as well as providing some psychological benefits, such as increased positive expectations of one's performance and resilience in the work environment. The current study provides evidence that by supporting robot users with dynamic information about robot operation, we can not only increase their speed on the task without affecting accuracy, but also decrease negative attitudes towards robots. Furthermore, maintaining outcome expectancies independent of the performance in the task might be a possible solution for how to increase acceptance of the collaborative robot technology in the first instance. The strength of the current study lies in the volunteer population being drawn from a low-skilled manual workforce. With a low number of studies investigating "social" aspects of human-robot collaboration in manufacturing, there is a clear lack of an interdisciplinary and holistic investigation [67]. Providing robot users with efficient information on how to work with collaborative robots affects their work efficiency, wellbeing, but also affects the success of human-robot collaboration in industry.

Limitations and Future Directions
The current study, despite shedding light on how dynamic signage can affect task performance and operator wellbeing, has several limitations. First, although the target population was shopfloor employees, the experiment was conducted on a manufacturingtype task and not the exact process our participants are working during their shifts in the factory. This has limitations of ecological validity due to additional observation equipment (cameras, experimenter being onsite) as well as introducing a new process the participants are not skilled at. Future studies should investigate the effects of signage over a longer period on the actual workstation. This leads to the second possible limitation: the interaction with a robot lasted up to 10 min, so the findings can be generalised only for the robot introduction and short-term effects. The possibility that over a long time, the signage would become less effective or even redundant, needs to be explored with a longitudinal study. Finally, in order to explore the unexpected result of task performance-significant differences in response time but not task completion accuracy-the dynamic signage effectiveness scores were compared with the archival data of static signage effectiveness. The participant population in both datasets was different, and although we controlled for participants' age, prior interaction, RAS, and NARS scores, this does not eliminate the possibility that differences emerged due to some other characteristics of participants. A future study comparing dynamic and static signage on the same manufacturing employees would clarify this question.

Conclusions
Collaborative robots offer a solution to increasing issues around workforce aging and sustainability [45]. The technology has the potential to reduce physical strain and increase safety for workers, as well as providing some psychological benefits such as increased job satisfaction and psychological wellbeing.
In the reported study, we investigated how dynamic signage can aid human-robot collaboration in manufacturing applications. The findings show that the use of dynamic signage can reduce the task completion time without compromising on accuracy. Results further indicate that the presence of graphical signage had generally positive effects on participants' attitudes and expectancies towards the robot.
Participants' negative attitudes towards robots decreased after the HRI scenario, as a function of their task performance, and this effect was more pronounced for participants presented with the graphical signage. In addition, the graphical signage maintained the participants' expected outcomes, post-HRI scenario. Users who did not have access to the signage reported worse outcome expectancies as a function of their response time: longer response times resulted in participants reporting a lower estimation of their performance than their peers.
Although the study has some limitations (the task is only representative of an actual work process, and we did not capture long-term effects), it provides evidence to show how task performance and attitudes towards robots can be aided by the provision of dynamic collaboration information. The strength of the study comes from the participant population-all expected future users of collaborative robots. However, the study has been designed as a controlled experiment-the results can be generalised outside our participant population to other manufacturing sectors and situations where human-robot collaboration is being introduced. These results support our hypothesis that the provision of dynamic instructional information, conveyed through appropriate graphical signage, can improve task efficiency and user wellbeing, contributing to greater workforce sustainably.