Evaluating Physical Stress across Task Difficulty Levels in Augmented Reality-Assisted Industrial Maintenance

Abstract

Many studies have revealed the benefits of augmented reality (AR) to enhance productivity and fulfill a quality function in industry. Yet, the potential adverse effects of AR, such as physical stress, have not been extensively studied. This study investigates the effect of task difficulty (simple versus complex maintenance tasks) on the overall performance and physical stress of individuals while interacting with multimedia instructions (i.e., AR via HoloLens versus. paper-based guidance). Twenty-eight healthy male participants with a mean age of 32.12 years (standard deviation of 2.45 years) were randomly assigned to two groups: AR-based group and paper-based group. The metrics of total maintenance time, NASA-TLX scores, and heart rate variability (HRV) were used to evaluate overall performance, perceived workload, and physical stress, respectively. The findings suggest that AR-based instruction for a complex task significantly reduced both total maintenance time (by 14.94%) and physical stress in comparison with paper-based instruction. Paper-based instruction was found to destabilize heart rhythm, increase heart rate, and cause a shift in autonomic balance due to heightened sympathetic nerve activation. This study concludes that AR-based instruction is more efficient and less stressful than paper-based instruction, indicating that the AR system may offer independent benefits in reducing physical workloads.

1. Introduction

Industrial maintenance and assembly tasks are extremely challenging operations that require physical abilities such as fine motor control and bimanual coordination. These tasks can overwhelm employees who must perform many actions requiring a lot of effort in a short period [1]. AR systems are now extensively used as an effective guiding tool for extremely challenging maintenance tasks [2,3,4,5,6,7,8,9,10,11,12]. Compared to traditional assistance techniques for delivering instructions and job-relevant information, AR tools have been found to be more engaging, leading to increased productivity and fulfilling a quality control function [13,14,15,16,17,18,19]. Furthermore, for more challenging tasks requiring strategic planning and intensive problem solving, AR methods have shown considerable benefits over conventional methods by enhancing the cognitive comprehension of tasks [6]. Previous studies have primarily focused on the efficacy of AR in terms of task completion time, success rates, and various subjective measures [9,10,11,20,21,22,23,24,25,26,27,28,29,30,31,32]. It is, however, uncertain whether using AR instruction can lead to adverse effects such as motion sickness, increased mental workload, increased physical stress, and visual fatigue [33,34,35,36,37]. As the AR industry grows, it is essential to address potential safety concerns like physical stress associated with AR instruction.

Physical stress can be defined as a complex state that can influence an individual’s ability to remain alert and attentive [38] depending on task characteristics, conditions, and the individual. In the field of human factors research, one area of special importance is measuring physiological stress utilizing physiological indicators. It is, therefore, quite essential that the measurement methods used are accurate. The most frequent physiological method used for measuring physical stress is heart rate variability (HRV), which is extracted from an electrocardiogram (ECG) [39,40,41]. An ECG represents the graphical record of an individual’s heart’s electrical activity, produced by the depolarization and repolarization of the atria and ventricles [42]. HRV is the variation in the time intervals between consecutive heartbeats [43] and can be used as a sensitive diagnostic technique for monitoring the activity of the autonomic nervous system (ANS) in stressful situations [16,44]. The central nervous system’s autonomic branches affect the electrical activity of the heart through the sympathetic (SNS) and parasympathetic (PNS) divisions of the ANS, affecting heart rate (HR) and beat-to-beat rhythm. PNS activity promotes relaxation and regeneration, and SNS activity mobilizes the body for activities such as the ‘‘fight or flight’’ response [45,46]. The ANS plays a crucial role in stress and workload [37], making it an intriguing source of information to investigate. Signal fluctuations in the heart’s rhythm can be sent to the brain via the vagal nerve terminals, leading to increased emotional stress and a reduced cognitive capacity for processing visual information [44,47,48,49]. It has been found that the cognitive and neurological functions of humans can influence their cardiac rhythm [50]. Conversely, the sensory input of the heart can also affect performance, cognitive functions, and working memory [47].

Previous studies have used HRV indicators to examine the physical stress associated with stereoscopic displays [40,44,49,51,52]. For instance, Tumler et al. [16] objectively evaluated user strain during picking tasks in an industrial warehouse while using either an AR system or a simple paper list instruction, with measurements based on HRV. Their results showed that long-term use (two hours) of the AR system did not significantly increase overall strain. The variation in stress induced by the AR system was similar to that produced by the paper list method. Park et al. [44] assessed 3D visual fatigue using heart rhythm and humans’ autonomic balance. Their study showed that viewing 3D video destabilizes heart rhythm, increases heart rate, and causes a shift in autonomic balance due to increased activation of the sympathetic nerves. Ramadan and Alhaag [49] evaluated the physical stresses associated with viewing 3D and 2D displays using HRV. Their results indicated that viewing the 3D displays significantly increased physical stresses compared to 2D displays. This was evidenced by significant increases in HR and decreases in the standard deviation of heart rate (STDHR), mean time between heartbeats (m-RR intervals), very low frequency (VLF) power, and LF/HF power when viewing 3D displays compared to 2D displays. Yuan et al. [52] studied visually induced motion sickness (VIMS) and the visual impacts of head-mounted displays (HMDs). They showed that the experience of HMDs caused higher visual discomfort (i.e., simulator sickness and visual strain) compared to the experience of TV and desktop displays. Their evaluation, however, was limited to research using HMDs for virtual reality. Gao et al. [51] assessed visual fatigue when viewing virtual magic cube tasks through optical see-through HMDs (OST-HMDs) compared to viewing real objects in the real world. They found that participants were more likely to experience discomfort when viewing virtual objects presented by OST-HMDs. Also, their study provided suggestions that HRV measures could be used to explore how visual fatigue is induced by AR content.

Increasing task difficulties has been reported to cause an increase in HR [53]. Also, HR has been found to increase during multitasking conditions [54] or when an additional memory load is introduced [55]. RR intervals have been observed to decrease during complex tasks compared to simple tasks [56]. An increased mean HR was found during a Stroop test, and a reduction in high-frequency (HF) power was noted, with a large reduction in low-frequency (LF) power seen during tasks of high complexity [57]. The same findings regarding LF power were also reported by Splawn and Miller [39] and Lehrer et al. [58] with regard to increased task difficulties. The LF/HF ratio of HRV spectra is a proxy for the degree of sympathovagal balance present in healthy individuals [59]. It has been suggested that the standard deviation of RR intervals (SDRR) is the best indicator of total HRV (overall autonomic activity). While LF has been primarily utilized as a measurement of sympathetic modulation activity, HF, RMSSD, and PNN50 have been used as indicators of vagal modulation activity [44,49].

In the comparison of AR against traditional instruction in previous studies, there is a substantial paucity of experiential data from user studies assessing the potentially negative consequences of AR-based training techniques. Furthermore, the comparisons and evaluations were mostly accomplished with non-industrial products using tasks with low levels of difficulty and few variations, which may not be representative of real industrial settings [60,61,62]. Consequently, the conclusions drawn from such studies are limited and specific, and may not apply to the wide variety of tasks in industrial settings, particularly where there is a high degree of product variability. To gain a deeper understanding of the features of AR-based instruction methods compared to paper-based approaches, it is beneficial and important to address more thoroughly the potential negative impact of AR-based instruction methods using objective techniques.

To better reflect actual industrial situations, the design of the experiment should involve a variety of complex tasks such as fitting, screwing, fastening, etc. Hence, this research is essential as it will contribute to the body of knowledge by advancing our understanding of how to evaluate the effects of AR systems on a user’s physical stress while performing maintenance tasks. As a result, the main goal of this research is to assess the impact of various guidance techniques (i.e., AR-based and paper-based guidance) and task difficulty on human performance and physical stress. Human performance as a function of different training methods was measured by total guidance time, while physical stress was evaluated using HRV.

This research aims to answer the following question: How do AR-based guidance techniques influence human physical stress, perceived workload, and user task performance (i.e., task completion time) across a range of task difficulties? For maintenance guidance, it was hypothesized that AR-based engagement would be more efficient and successful than traditional paper-based techniques. Therefore, AR-based interactions were expected to result in lower physical stress and perceived workload and improved performance in the experimental maintenance tasks compared to paper-based interactions.

2. Materials and Methods

2.1. Participants

A total of 28 male college students were recruited from King Saud University. The participants had a mean age (SD) of 32.12 (2.45) years, weighed an average of 71 (9.80) kg, and had a mean height of 168.13 (3.72) cm. According to the results of the self-report screening survey, all the participants were in good health, had normal vision, did not have any diseases of the nervous system, had no history of neck or shoulder pain issues, and had no other type of musculoskeletal injury. Furthermore, none reported experiencing any kind of sleep disruption during the prior two weeks.

Participants were requested not to eat or smoke for at least two hours prior to the data collection session. They were also instructed to avoid any strenuous physical activity the day before the experiment and maintain their typical sleep pattern (approximately six hours of sleep each night). Only individuals who had either no previous information or a minimal amount of prior knowledge about the activity that needed to be completed were asked to take part in the experiment.

2.2. Experiment Variables

2.2.1. Maintenance Tasks

This study focused on the overhaul maintenance activities of a piston pump, including the repair of the gearbox (a complex task) and the inspection of the seal of the pump housing (a simple task). The piston pump housing and the gearbox in their fully assembled state are shown in Figure 1.

Figure 2 presents the exploded view of the piston pump gearbox (for the complex task). In this complex task, individuals were required to fix the piston pump gearbox and inspect all its seals. This task involved 32 sets of components that needed to be assembled in 40 stages using a soft hammer, screwdriver, jig, puller, and open-ended wrench. Figure 3 shows an exploded view of the pump housing (for the simple task). The simple task required individuals to verify all pump housing seals during the disassembly of their parts. This task was less demanding than the complex task as it involved fewer steps (26), unspecialized hand tools, and lightweight components. The degree of complexity and the details of the steps for the two tasks are thoroughly described and evaluated experimentally in studies by Alhaag et al. [1] and Alessa et al. [63].

2.2.2. Instruction Methods

Both paper-based and AR-based instructions were created for this study to instruct the individuals on how to properly execute the maintenance activities. Paper-based instructions were provided on a folder of A4 paper, with one page dedicated to every two steps, and were adapted directly from the piston pump’s manual. Each step of the paper instructions had photos, written details, and the necessary tools that were required for each step; see Figure 4.

For the complex task, paper-based instructions spanned 22 pages and included 40 individual steps to carry out. For the simple task, paper-based instructions spanned 14 pages and had a total of 26 individual steps. An example of one participant employing the paper-based instruction method to complete the simple task is shown in Figure 5.

The AR-based instructions were developed using Creo Illustrate and Vuforia Studio software (2019 version) for generating the maintenance steps and the AR experience, respectively. In addition, JavaScript was utilized to create the essential characteristics of AR apps. Figure 6 gives an overview of the implementation of the AR-based maintenance instructions. For displaying the AR instruction, the Vuforia View app stored on the Microsoft HoloLens (1st gen) was used for scanning specific quick responses (QRs), as shown in Figure 7.

The 3D animated maintenance sequence was delivered to the participants’ surroundings using the first-generation Microsoft HoloLens (Microsoft, 2019), as presented in Figure 8. In this method, participants could experience the digital model interacting with the real environment. For instance, they placed the model in the recognized marker 130 cm away from them on the maintenance desk.

The interaction technique for the proposed AR application employed head movement tracking, letting users manipulate the maintenance 3D animation sequence using 3D buttons. To trigger the maintenance steps, the users merely needed to move their heads to manipulate a pointer and point to a 3D button. The user could navigate through the instruction sequence in the application using 3D buttons to obtain guidance on the current step, move to the next or previous step, and rapidly go to any step. Figure 9 presents an example of one participant performing the complex and simple tasks via the AR method.

To minimize the impact of varying AR systems and instruction formats on the reproducibility of this study, the experimental protocol incorporated an AR system that is a market leader known for setting industry standards. Additionally, a standardized presentation style for both digital and manual instructions was employed to reduce design variability. However, it is recognized that different AR systems and instructional formats could yield different outcomes. Therefore, the results should be interpreted with the understanding that they might vary under alternative conditions, and further research is encouraged to investigate a range of technologies and presentation styles to better ascertain the generalizability of these findings.

2.3. Experimental Design

This experiment was represented by a 2 × (2 × 14) mixed design. The difficulty of the maintenance tasks, with two levels (simple and complex tasks), was treated as a within-subject factor. The instruction techniques, with two levels (AR-based instructions and paper-based instructions), were treated as between-subject factors. Task complexity was counterbalanced across participants to avoid order effects. The dependent variables for this design included the overall task time, the perceived workload as measured by the NASA TLX, and heart rate variability (HRV).

2.4. Response Variables

2.4.1. Human Performance

Performance was measured in terms of total task time, which represents the overall duration taken to accomplish the work while following the guidelines. This duration was captured using a camera system in conjunction with a stopwatch.

2.4.2. Perceived Workload (NASA-TLX)

The NASA-Task Load Index developed by Hart [64] was utilized to assess participants’ levels of mental workload. The questionnaire includes six questions, each designed to measure different factors such as mental and physical workload, temporal demands, performance, effort, and frustration level. Participants used a workload rating scale of 0–20 (1 being low and 20 being excessive) to score each element. The total estimated workload was calculated by averaging the six indicators.

2.4.3. Physical Stress

The HRV parameters of the ECG were used to assess user stress during the performance of the tasks under the different instruction methods. A one-channel MT-ECG-1 preamplifier (Mega Electronics Ltd., Kuopio, Finland) with three lead wires was used to record the ECG signals. Ag/AgCl solid, adhesive, pre-gelled electrodes were positioned on the participants’ chests to reduce the occurrence of muscle contraction artifacts during the task [65] (see Figure 10). Mega Win 3.0.1 software (Mega Electronics Ltd., Kuopio, Finland) was used to acquire the ECG signals at a sampling rate of 1000 Hz with a resolution of 12 bits. Before any further processing, the quality of the signal was visually inspected.

To decrease high-frequency noise, the ECG signals were preprocessed via a 40 Hz cutoff low-pass filter. In addition, a 0.3 Hz cutoff high-pass filter was implemented to attenuate baseline wander. ECG signals were processed using Kubios HRV Analysis Software version 2.2 [63]. The processing workflow of the ECG data is outlined in Figure 11. The ECG processing started with the previously filtered signals and the detection of the QRS complexes(the depolarization of the ventricles).

Time series analyses of HRV began with the elimination of artifacts caused by erroneous RR intervals (i.e., falsely detected QRS complexes) [59,66]. Artifacts in recorded ECG signals can cause significant distortion to ECG results [59]. The important part of the analysis focused on achieving two types of parameters: time domain and frequency domain features. A useful reference about the interpretation, measurement, and medical use of HRV can be seen in [59,67]. Artifact beats were filtered out from the recorded ECGs by deleting RR intervals that deviated by more than 25% from one another [66]. For the data length to stay constant (i.e., with the same number of beats), the removed RR intervals were substituted with values obtained by linear interpolation. Low-frequency disturbances in the baseline trend were eliminated using the smoothness prior approach with a Lambda of 1000 [68,69].

Within the HRV time domain, three types of variables were analyzed in this study: statistical, geometrical, and nonlinear. For statistical parameters, the mean RR interval (m-RR, in ms), standard deviation of normal-to-normal RR intervals (SDRR), mean heart rate (m-HR, in BPM), standard deviation of heart rate (SDHR), root mean square of successive RR interval differences (RMSSD, in ms), and percentage of RR intervals differing >50 ms from the preceding one (pNN50) were computed. Geometrical features involved calculations on the HRV triangular index and the triangle base used to represent the RR time series histogram (TINN). Nonlinear HRV features such as sample entropy (SaEn), SD1, SD2, and approximate entropy (ApEn) were also calculated.

A frequency domain analysis was also performed on the ECG data using fast Fourier transformations (FFT). [70]. The non-equidistant beat-by-beat time series was converted into a new time series sampled at uniform intervals using a quadratic interpolation at 4 Hz [71,72]. Subsequently, any possible trends in the signals were eliminated by detrending, and the variations in the signals were extracted by suppressing the mean [73]. Power spectra of the periodic oscillations were obtained using the Welch’s periodogram function with a 256 s window and 50% overlap, allowing for a frequency resolution of 0.015 to 0.02 Hz [72]. Frequency domain parameters were computed by dividing the RR time series PSD into three frequency bands, with the spectral power being the area below the power spectral density (PSD) for each period.

Frequencies below 0.04 Hz were considered very low frequency (VLF). The area under the PSD from 0.04 Hz to 0.15 Hz was computed to yield the low-frequency power (LF), and the same process from 0.15 Hz to 0.4 Hz yielded the high-frequency power (HF). The low-to-high frequency power ratio (LF/HF) of HRV spectra is often used as an indicator of sympathogal balance in healthy individuals. Specifically, LF was utilized to represent sympathetic modulation activity, while HF was considered to represent vagal modulation activity [59]. From the frequency domains, the relative powers of the LF bands and HF bands and the low-to-high frequency (LF/HF) ratio were used in the analysis.

2.5. Experimental Procedure

Each participant, before participating in the experiment, read and signed an informed consent form approved by the Institutional Review Board at King Saud University (code: E-19-4467). Upon arrival, a general health questionnaire was given, and the study’s rationale was thoroughly explained. Participants also completed surveys regarding their prior experience with AR in a maintenance context. Subsequently, participants were randomly divided into two groups. The first group received AR training using Microsoft HoloLens1 (first version, Microsoft, 2019) on the maintenance steps, including a 10–15 min familiarization session with HoloLens and its features [6,10]. A scooter disassembly and assembly demo was used for this process, and they were taught the proper use of the toolkit. The second group received written instructions on A4 paper for the tasks. After becoming comfortable with the assigned guidance methods, either HoloLens or paper instructions, participants returned the next day to perform the maintenance tasks using their respective methods.

On the following day, Ag/AgCl electrodes for the ECGs were attached to participants’ chests. Baseline data consisting of three minutes of ECG signal were acquired while they were in a resting position. ECG signals were also collected during each training procedure and maintenance activity. Participants in Group 1 (the AR-based guidance group) were randomly assigned either a complex or a simple task to complete using AR instructions, as shown in Figure 12. Tasks were assigned in various sequences using a counterbalancing strategy to eliminate task order impact, with either AB or BA sequences (A and B representing complex and simple tasks, respectively). In the AB sequence, fifty percent of the participants in Group 1 first performed a complex task. The BA sequences were given to the remaining participants to carry out the simple task first. In this session, individuals completed half of the experiment (a simple or complex task).

The second session of the experiment required participants to return to the lab two days later to complete their remaining task (simple or complex) following AR instructions (for Group 1) or paper instructions (for Group 2), as shown in Figure 12. Signals were recorded for an additional three minutes at the end of each session. After that, participants filled out NASA Tlx questionnaires, measuring their personal perception of the task and its execution in terms of perceived effort. Participants were acknowledged once all sessions were completed and the electrodes were taken off.

2.6. Statistical Analysis

SPSS Statistics software (version 23.0; IBM Inc., Armonk, NY, USA) was used for statistical analysis, with a significance level of 0.05. The normality assumption across all experimental conditions was verified using Kolmogorov–Smirnov tests [74]. All response variables were not statistically significant (p > 0.05) based on the Kolmogorov–Smirnov tests. Mauchly’s test of sphericity was conducted on all responses across all variables to ensure that the variance homogeneity assumption was met. Given its binary nature (two levels), sphericity analysis confirmed that the primary assumption was satisfied.

To examine the main and interaction effects of the instructional approach and task complexity on task completion time, HRV, and perceived workload (NASA TLX), a two-way analysis of variance (ANOVA) with a repeated measures methodology was performed. Simple effect analysis further investigated the significance of the observed interaction effects. To identify significantly differing task difficulty permutations, a paired t-test was performed. Pairwise evaluations of the interaction effects using an independent sample t-test were carried out to determine the substantially different permutations of the training strategies. Calculations were made to determine the descriptive statistics (mean and standard deviation) of each dependent variable. In addition, the effect size was determined using the partial eta-squared value (η²) representing the proportion of total variation in the dependent variables attributable to the specific independent variable.

3. Results

3.1. Performance

The mixed ANOVA showed that task difficulty significantly impacted total time {F (1,26) = 286.30, p = 0.00, η² = 0.92}. Maintenance time increased from simple to complex tasks {mean: 13.34 versus 26.62}, regardless of instruction method. Time to completion was also significantly affected by the interaction between the method of instruction and task complexity {F (1,26) = 7.23, p = 0.012, η² = 0.22}. Higher completion times for complex tasks were found with both AR-based guidance (24.48 versus 13.30, t (13) = 29.18, p = 0.00) and paper-based guidance (28.76 versus 13.36, t (13) = 10.09, p = 0.00). The AR instruction group finished the complex task faster than the paper instruction group. While the AR group completed the simple task slightly faster than the paper-based group, the difference was not statistically significant (p > 0.05). Overall, the study found that AR reduced completion time for complex tasks.

3.2. Perceived Workload (NASA-TLX) Scores

The NASA TLX workload questionnaire measured participants’ perceived workload [64]. As shown by the arrows in Figure 13, a higher total score reflected a larger perceived load, except for the performance subscale, where a higher total score indicated a lower self-reported level of success with the task. The main effect of task complexity was significant on the mental load {F (1,26) = 939.92, p = 0.000, η² = 0.97}, physical load {F (1,26) = 202.59, p = 0.000, η² = 0.90}, temporal load {F (1,26) = 202.08, p = 0.000, η² = 0.90}, performance {F (1,26) = 15.30, p = 0.001, η² = 0.40}, effort {F (1,26) = 38.91, p = 0.000, η² = 0.60}, and frustration {F (1,26) = 125.00, p= 0.000, η² = 0.83} subscales as well as overall mean NASA-TLX scores {F (1,26) = 672.45, p = 0.000, η² = 0.96}. Throughout the complex task, scores increased across the subscales. It was found that there was a considerable influence that instruction technique had on performance {F (1,26) = 25.46, p = 0.000, η² = 0.50}, frustration {F (1,26) = 70.08, p = 0.00, η² = 0.73}, and overall mean NASA-TLX scores {F (1,26) = 9.34, p = 0.005, η² = 0.26}, with these scores being higher for paper-based instruction.

All NASA-TLX subscales, except for effort and the overall mean, showed statistically significant interactions between task complexity and instruction technique (p > 0.5). The interaction effects of task complexity and instruction method on the NASA-TLX results are presented in Figure 13. The paired t-test revealed that as task complexity increased, so did average scores of mental and physical load, performance, and frustration for both approaches of instruction. When participants were given written instructions for a complex task, they reported significantly higher levels of physical workload, performance, and frustration. There was no statistically significant difference in mental load between AR and paper-based guidance for either simple or complex tasks (p > 0.05). Also, physical load did not significantly differ across groups when performing the simple task.

3.3. Physical Stress

The HRV parameters from the ECGs were used to assess user stress during task performance under the different instruction methods. Statistical analysis results for all HRV parameters are demonstrated in

Table 1. Mean (SD) and statistical results for HRV responses across experimental tasks. Bold p-values indicate a significant difference. Bolded p-value, and background color indicate a significant effect. Note: η² is partial eta squared.

Variable	Mean (SD)				Statistics p (η2)
Instruction Methods	Paper-Based Instruction		AR-Based Instruction		Instruction (η2)	Complexity (η2)	Interaction
Task Complexity	High Difficulty	Low Difficulty	High Difficulty	Low Difficulty
m-RR (ms)	677.91 (50.55)	774.22 (66.32)	752.16 (102.48)	767.42 (73.20)	0.22 (0.06)	0.00 (0.58)	0.002 (0.35)
SDRR (ms)	16.65 (2.29)	18.38 (1.64)	18.86 (2.30)	18.96 (2.54)	0.08 (0.99)	0.08 (0.11)	0.12 (0.9)
m-HR (bpm)	89.49 (6.31)	78.27 (6.51)	80.46 (11.66)	77.90 (8.16)	0.13 (0.09)	0.00 (0.60)	0.00 (0.37)
SDHR (bpm)	5.11 (0.97)	3.59 (0.72)	4.09 (1.34)	3.85 (1.14)	0.31 (0.04)	0.00 (0.38)	0.00 (0.24)
RMSSD (ms)	12.31 (1.97)	16.87 (2.77)	18.85 (6.05)	19.17 (5.07)	0.01 (0.23)	0.00 (0.50)	0.00 (0.39)
NN50 (ms)	4.21 (4.15)	4.14 (4.37)	3.79 (3.31)	2.21 (1.67)	0.32 (0.04)	0.23 (0.06)	0.27 (0.05)
PNN50 (%)	0.21 (0.22)	0.37 (0.34)	0.28 (0.30)	0.30 (0.34)	0.98 (0)	0.21 (0.06)	0.35 (0.92)
HRVI	5.19 (0.72)	5.86 (0.712)	5.75 (1.11)	6.02 (1.18)	0.25 (0.051)	0.02 (0.19)	0.32 (0.04)
TINN (ms)	202.14 (54.59)	158.57 (31.40)	162.14 (35.18)	158.57 (41.34)	0.15 (0.079	0.00 (0.24)	0.02 (0.19)
VLF (%)	23.82 (7.75)	20.11 (7.21)	19.63 (8.41)	17.61 (10.08)	0.233 (0.054)	0.089 (0.11)	0.61 (0.01)
LF (%)	68.46 (7.54)	56.26 (16.04)	54.03 (23.20)	53.10 (20.62)	0.18 (0.07)	0.01 (0.23)	0.03 (0.18)
HF (%)	31.37 (7.56)	43.57 (16.02)	45.93 (23.01)	46.71 (20.58)	0.17 (0.071)	0.01 (0.22)	0.02 (0.18)
LF/HF	2.34 (0.75)	1.59 (0.99)	1.73 (1.47)	1.66 (1.37)	0.53 (0.02)	0.00 (0.27)	0.02 (0.21)
SD1	9.27 (0.92)	12.57 (1.63)	14.86 (1.51)	14.01 (2.26)	0.00 (0.61)	0.00 (0.42)	0.00 (0.68)
SD2	43.80 (8.15)	39.65 (7.47)	41.62 (6.55)	43.11 (6.75)	0.73 (0.00)	0.53 (0.02)	0.19 (0.07)
ApEn	1.31 (0.13)	1.40 (0.07)	1.34 (0.13)	1.40 (0.12)	0.77 (0.01)	0.00 (0.34)	0.21 (0.059)
SampEn	1.47 (0.19)	1.60 (0.16)	1.60 (0.23)	1.66 (0.27)	0.20 (0.06)	0.03 (0.18)	0.34 (0.035)

. For time domain characteristics, the RMSSD and SD1 were significantly affected by the instruction approach (p < 0.05). The m-RR, m-HR, SDHH, RMSSD, HRV triangular index, TINN, SD1, ApEn, and SampEn were significantly affected by task complexity (p < 0.05). Statistical analysis of the interaction effects indicated that m-RR, m-HR, SDHH, RMSSD, TINN, and SD1 were all significantly different depending on whether the instruction was paper- or AR-based and whether the task was simple or complex (p < 0.05). Figure 14 shows the significant interaction effects of the instruction methods and task complexity on m-RR interval, m-HR, SDHR, RMSSD, TINN, and SD1. The results revealed that for the paper group, performing the complex task resulted in significantly lower m-RR, RMSSD, and SD1 values and higher m-HR, SDHR, and TINN values compared to the simple task. However, using AR-based instruction for either the complex or the simple task did not yield any significantly different results (p > 0.05). Comparing paper-based and AR-based instructions for the complex task showed a statistically significant difference (p < 0.05).

For the frequency domain variables of HRV, task complexity had statistically significant effects on LF, HF, and the LF/HF ratio (p < 0.05). Interaction effects between instruction methods and task complexity significantly influenced LF, HF, and the LF/HF ratio (p < 0.05). Figure 15 shows these interaction effects on LF, HF, and the LF/HF ratio, respectively. Pairwise tests revealed that performing the complex maintenance task using paper-based instruction resulted in higher HF/LF ratios and LF values and lower HF values compared to the simple task (p < 0.05). No significant change was seen in LF, HF, or the LF/HF ratio when using AR-based instruction for either simple or complex tasks (p > 0.05). However, there was a significant difference between AR and paper-based instruction for the complex task (p < 0.05), but no significant difference was found for the simple task (p > 0.05).

4. Discussion

The goal of this research was to compare physical workloads, human performance, and perceived workloads during maintenance tasks, considering the variables of instructional methods (paper-based versus AR-based) and task complexity (simple versus complex). Current methodologies of tracking user interactions with AR systems are largely subjective, necessitating the development of sensitive and accurate methodologies to evaluate user workload during routine maintenance tasks. The findings from this study significantly contribute to our understanding of how physical workloads vary when using AR systems for training. Specifically, they provide insights into the differential impacts of paper-based and AR-based instructions on workload and performance during maintenance and assembly tasks. These insights are crucial for developing more effective and user-friendly AR training systems for complex and simple tasks alike.

4.1. Performance

This study compared the effectiveness of AR-based instructions to paper-based methods in terms of task completion time. Notably, completion times varied between maintenance tasks, with a significant increase (50%) from simple to complex tasks across all instruction types. Both AR (24.48 vs. 13.30 min) and paper-based (28.76 vs. 13.34 min) instructions required considerably more time to complete complex tasks than simple ones. However, while completion times for simple tasks were similar with both methods, completion times with AR-based instruction were significantly faster for complex tasks. These results indicate that AR-based instruction during complex activities can greatly shorten maintenance times. It may be claimed that paper-based instructions may be less efficient for challenging tasks because they require the user to browse the paper and extract information from the material. In contrast, AR-based instruction systems offer a variety of features that can be displayed upon request. Therefore, while AR did not enhance performance for simple tasks, it proved significantly more efficient for more challenging tasks.

This study supports previous findings [21,75,76,77,78]. Wiedenmaier et al. [21] observed that paper manuals and AR support were equally effective for simple tasks, but AR support outperformed paper-based support for complex tasks. Deshpande and Kim [79] discovered that using paper instructions for assembly activities with a modest level of complexity was more time-consuming than following AR instructions. Similarly, Mengoni et al. [75] reported minor improvements when applying AR to tasks of low complexity compared to traditional approaches, while AR was found to be more effective when applied to more complicated activities, as indicated by Alves et al. [77]. Bendzioch et al. [78] investigated the influence of product difficulty (difficult, medium, and easy) on task completion time with different instruction types (AR, tablet, projection, and paper instructions). They found that compared to paper instructions, using AR glasses resulted in time reductions of 3.16%, 9.51%, and 9% for easy, medium, and difficult tasks, respectively [78].

4.2. Perceived Workload (NASA-TLX) Scores

The results indicated that using paper-based methods for complex tasks increased the mean score ratings of physical demand, performance, and frustration compared to AR-based instruction. For the complex task, the AR instruction method reduced physical workload by 12.18%, performance improved by 8.69%, and frustration improved by 60.14%. AR instruction resulted in greater temporal demand ratings and worse performance ratings for the simple task than paper instruction. AR instruction for the simple task increased temporal demand by 43.63% and decreased performance and frustration by 27.64% and 24.97%, respectively, compared to paper-based instruction. Overall, the AR-based method reduced performance, frustration, and mean scores by 18.13%, 50.82%, and 6.50%, respectively. Additionally, it was observed that the average ratings for mental workload, physical workload, temporal demand, and frustration all rose as the task difficulty went up, regardless of which instruction method was used. Some previous studies have shown that the utilization of AR for the purpose of delivering visual assistance and task-specific information has been found to effectively decrease cognitive load and result in enhanced performance [15,80,81]. Conversely, several studies have indicated that AR may be seen as mentally demanding and consequently result in diminished performance [82,83]. The outcomes of this study align with the findings from previous research [15,80,81]. It was demonstrated that factors such as the type of AR display device used, the relevance and timeliness of the content, the manner of information presentation, and user and task characteristics all profoundly influence mental effort and task performance [84]. The observed discrepancies in results could potentially be attributed to variations in study types, technologies employed, and research strategies.

4.3. Physical Stress

HRV, as previously outlined, was used to assess the impacts of task complexity and instruction method on physical stress. The physical stress analyzed in this study represents the measurable portion of physical resources expended during the performance of the two maintenance tasks using different instruction methods. HRV analysis, derived from electrocardiogram (ECG) data, is a highly responsive diagnostic technique used to assess the activity of the autonomic nervous system (ANS) under stressful situations [85]. The ANS’s branches modulate the heart’s electrical activity via the sympathetic (SNS) and parasympathetic (PNS) divisions, influencing heart rate and rhythm. PNS activity promotes relaxation and regeneration and SNS activity mobilizes the body for activities such as the ‘‘fight or flight’’ response [45,46].

The time domain of HRV showed significant effects of task complexity on m-RR, m-HR, SDHH, RMSSD, HRV triangular index, and TINN. As task difficulty escalated from simple to complex, m-HR, SDHR, HRV triangular index, and TINN indicators increased, while m-RR and RMSSD indicators decreased, irrespective of the instructional approaches employed. This indicates that increased task complexity results in higher physical activity levels, leading to enhanced sympathetic activity through central and autonomic nervous system integration, a finding consistent with several other studies [86,87,88]. Paper-based instruction was associated with lower values of RMSSD compared to AR-based instruction, regardless of task complexity. The statistical analysis showed that for the paper-based instruction group, the complex task significantly reduced m-RR and RMSSD values and increased m-HR, SDHR, and TINN values compared to the simple task. Comparing the two instruction methods during a complex task, AR instruction significantly reduced physical workload level by an average of 13.66% (10.95% for m-RR, 10.09% for m-HR, 19.96% for SDHR, and 53.13% for RMSSD). However, no changes in physical workload were observed when comparing the two instructional methods for a simple task. In addition, using AR for both complex and simple tasks resulted in the same physical workload.

A higher workload has been reported to be associated with lower values of m-RR and higher values of m-HR [89], and a decrease in RMSSD of HRV has been used as an indicator of fatigue and stress [90,91]. The variations in the time domains of HRV in this study acted as indicators of a high physical workload that suggest increased sympathetic activity. The observed reductions in RMSSD indicated a decrease in vagal modulation of the HR [90,91,92]. Based on the time domain of HRV parameters, the results indicated that the utilization of AR training for the complex task led to a notable decrease in workload when compared to the traditional paper-based instruction approach. These results align with other research demonstrating a correlation between alterations in task difficulty and corresponding variations in physical load [93,94,95,96,97,98]. Furthermore, previous studies have reported that HR increases when task difficulty [53,99,100,101] or stressful conditions [102,103] increase. Theoretically, based on the measures used in this study, the complex task is associated with high stress levels, which explains the changes observed in the HRV time domain variables.

HRV analysis through nonlinear domains focuses on the complex time behavior of heart rhythm. The results demonstrated a statistically significant effect of task complexity on SD1, ApEn, and SampEn. These measures exhibited a drop in values during complex tasks in comparison to simple tasks, irrespective of the instructional techniques employed. Our findings are in line with those reported in previous studies [98,104]. Anishchenko et al. [104] found that nonlinear metrics decreased significantly during stressful conditions. The results also showed that for paper-based instruction, the SD1 value was significantly reduced when performing the complex maintenance task. However, with AR instruction, the SD1 values for both simple and complex tasks remained unchanged. The findings of this study indicate that participants experienced higher levels of stress when engaging in paper-based instruction during the complex tasks compared to both the simple task and the AR instruction-based technique tasks. Hence, the use of AR instruction for executing complex tasks substantially alleviated the physical workload. The observed reduction in complexity measurements indicates a shift towards more stability and periodicity in HR during periods of stress. This shift might be associated with enhanced regularity, the separation of interconnected networks, and the inactivation of control mechanisms within the cardiovascular system [105,106]. Such a decrease in HR complexity in response to stressful situations may indicate diminished flexibility and fitness of the cardiac pacemaker [107].

For the frequency domain variables of HRV, the results indicated that task complexity significantly influenced LF, HF, and the LF/HF ratio. As the task difficulty increased, HF decreased while LF and the LF/HF ratio rose. The results also showed that LF and HF/LF values were higher for the complex maintenance task while employing paper-based instruction, but HF values were lower. Moreover, performing the complex task using the paper instruction technique resulted in a higher HF/LF ratio and LF values and lower HF values compared to AR-based instruction. There was no significant difference between AR and paper-based instruction for the simple tasks, and no changes were seen between the complex and simple tasks when using AR-based instruction. This result is similar to those regarding the time domain of HRV, suggesting that the AR-based instruction technique was effective in dealing with increased maintenance task difficulty. The findings from the frequency domain parameters also showed that the mental and physical workloads for the challenging task were lowered by 46.27% when using AR instruction. The findings suggest that AR instruction is superior to paper instruction in boosting cognitive processing during complex tasks.

The parasympathetic nervous system, which influences the effect of breathing on HR and is represented as HF power [108], is characterized by respiratory sinus arrhythmia [109]. Parasympathetic modulation causes a rise in HR [108], observed while carrying out the challenging task using paper-based instruction. A further decline in HF power was noted during the complex task using paper-based instruction. This decline can be attributed to alterations in the breathing pattern resulting from the adoption of uncomfortable postures, such as frequent bending and reaching while utilizing paper instructions. The observed rise in LF power during the complex task involving paper-based instruction could also be attributed to the adoption of uncomfortable body positions. According to a study by [109], changes in body position were shown to greatly enhance power within the LF range. Furthermore, elevated LF is noted in several scenarios, including a 90-degree tilt of the head when the body is in a supine position, standing, experiencing mental stress, moderate exercise among healthy individuals, and participating in physical activity [59]. This could explain the higher LF power values when performing a complex task using paper instructions. Perini and Veicsteinas [94] observed a 2.5-fold rise in the LF/HF ratio during high-intensity exercise compared to resting levels. The LF/HF ratio was employed as an indicator of sympathetic activity and HF power was utilized to indicate parasympathetic activity in the evaluation of work stress [98]. Collins et al. [98] observed a significant increase in the LF/HF ratio and a decrease in HF power among those experiencing high levels of strain throughout workdays. This study also indicates that the LF/HF ratio exhibited an increase during the execution of complex tasks utilizing paper-based instruction, accompanied by a drop in HF power. These outcomes align with previously reported findings [98,110].

Overall, HRV indices are sensitive indicators of physical stress. The fluctuations in HRV delineate the duration of specific information extractions during engagement in cognitively demanding tasks. This phenomenon was further supported by an interaction effect seen with the instructional approach, which can be attributed to contrasting inclinations in the processing of difficult tasks. These findings revealed that using AR instruction significantly reduced the physical stress for the complex task, implying that AR instruction is better than paper instruction for complex tasks. In addition, the physical workload of the AR users decreased while participants performed the maintenance tasks, whether simple or complex. As the two tasks were performed in an AR setting, the HRV metrics remained approximately constant. In contrast, the HRV metrics increased when performing the complex task in a paper-based setting. This means that the instruction methods and thus the workloads increased with complex tasks but remained relatively constant for simple tasks. It is likely that the individuals had developed a level of familiarity with the nature of simple tasks and the corresponding processes of decision making, searching, and maintenance. This familiarity served to diminish the necessity for the further extraction of information. The AR system independently reduced the physical workload. The elevation in physical load resulted in a disruption of the homeostatic equilibrium, leading to subsequent allostatic adaptations. These adaptations, in turn, influenced the temporal spacing between R-peaks, therefore affecting HR and HRV parameters. HRV is recognized not only as a measure of ANS equilibrium but also as an indicator of central–peripheral neural feedback and the integration between the central nervous system (CNS) and the ANS [47,111].

There are some limitations to the current study that warrant mention. This research did not track how the participant’s upper body moved. Monitoring kinematic variances between individuals may help in understanding the variety of stiffness responses to complex tasks and different instructional approaches. In addition, all participants in this study were male. Future work should consider incorporating women and other physiological signals, such as upper arm muscular activity, into the evaluation. Furthermore, the age range, education level, and formation of the participants in this study were quite limited. Future investigations could benefit from exploring the implications of our findings for broader age groups and different participants with different education levels and formations. Also, while the current study provides insights into the effects of AR instructions, it is limited to short-term use. The long-term implications of daily, extended use of AR in occupational settings remain to be explored.

Another limitation of this study is the relative inexperience of the target participants. While this choice was deliberate to simulate a training scenario for new workers or for introducing new complex tasks, it may not fully represent the experiences of seasoned operators. To better generalize our findings, future research should include experienced participants. This will allow for a more comprehensive understanding of AR technology’s utility across different skill levels, particularly for training on new or complex tasks within industrial settings. A notable limitation arose from the specific selection and use of interface types (visual and auditory), technological tools (such as paper, HMD, laptop, and HHD), and research design choices (including between, within, or mixed designs). In this study, a mixed experimental design was implemented, incorporating one between-subject variable (instruction methods) and one within-subject variable (task complexity). Task complexity was counterbalanced across participants to avoid order effects. For future work, it would be worthwhile to consider other experimental designs that ensure both groups experience similar mental and physical loads associated with wearing the display, including potential restrictions on field of view (FOV) and head mobility. This could be achieved by having both groups perform one task with the AR display and one task with physical instructions, counterbalancing the order of the tasks and the use of AR or paper instructions. Future research should also consider employing tasks of equal cognitive complexity, such as assembling a Lego set or arranging pieces of an image (i.e., a puzzle).

The outcomes and conclusions derived in this study are largely contingent on these particular choices. Future studies involving a different combination of these elements might yield divergent results, emphasizing the unique context of this user study.

5. Conclusions

To address the knowledge gap in the literature regarding the efficiency and benefits of using AR tools as a training method, this study aimed to perform experiments on human subjects with different training methods and different levels of task complexity. The current study adopted the ECG measure approach to evaluate the workload of AR users on maintenance tasks. The findings indicate that for more challenging tasks, AR instruction has an essential benefit over traditional paper-based instruction in reducing both the time taken for maintenance and physical stress. AR resulted in a time reduction of 14.94% for complex tasks. Moreover, utilizing paper-based instruction led to an unstable heart rhythm, an increase in HR, and a shift in autonomic balance, primarily due to increased activation of the sympathetic nerves. The findings revealed that physical stress for AR users was reduced while they performed the two maintenance tasks. With both tasks performed in an AR setting, the HRV metrics remained approximately constant. This study concludes that AR-based instruction is more efficient and less stressful than paper-based instruction, indicating that AR systems may offer independent benefits in reducing physical workloads. Both subjective and physiological measures (ECGs) support these findings. HRV indices and frustration ratings strongly suggested an increase in physical stress when the participants performed a complex task using paper-based instruction. The current study adopted the ECG measure approach to evaluate the workload of AR users on maintenance tasks. This aids in understanding the benefits and drawbacks of AR instructional techniques, which are transforming traditional instruction systems. Moreover, the results of this study can help manufacturing stakeholders better understand the benefits of AR technology as an instruction tool for manual tasks. Furthermore, the present study’s results provide a foundation for the appropriate application of this technology in industrial sectors across the globe.