A User-Centered Evaluation of a VR HMD-Based Harvester Training Simulator

Abstract

Skilled operation of forestry harvesters is essential for ensuring safety, efficiency, and sustainability in logging practices. However, conventional training methods are often prohibitively expensive and limited by access to specialized equipment. This study delivers one of the first user-centered validations of a low-cost, VR HMD-based forestry harvester simulator, directly addressing access and scalability barriers in training. With 26 participants, we quantify cognitive load, usability, user experience, and simulator sickness using established instruments. An increase in cognitive load was seen from baseline tutorial to each training module (NASA-TLX: $18.65\to 34.26\to 38.43$; rm-ANOVA, p < 0.001). Usability was ‘Good’ (with a mean SUS score: 76.63), hedonic UX ranked in the top decile (UEQ-S), and simulator sickness was moderate (mean SSQ score: 28.91), while task success remained high across all modules. These results indicate early-stage feasibility and usability of a low-cost VR HMD harvester simulator for student-focused introductory instruction, and they provide actionable design guidance (e.g., managing extraneous load, comfort safeguards) advancing evidence-based VR HMD-based training in the forest engineering and harvesting domain. Our findings validate the potential of VR-HMD as a tool for forestry education capable of addressing training accessibility gaps and enhancing learner motivation through immersive experiential learning.

1. Introduction

Training skilled forest harvester operators is essential for sustainable logging, operational efficiency, and safety. However, traditional field-based training methods remain costly, equipment-intensive, and logistically demanding, often requiring supervised sessions and extended downtime [1]. These challenges are exacerbated by outdated curricula, limited funding, and a lack of specialized vocational programs, leading to a widening gap between industry demands and workforce readiness [2]. Despite increasing demand for mechanized operators, many training programs still rely on conventional training, meaning hands-on, field-based instruction where trainees operate real harvesters under supervision. High costs further limit access to advanced instruction, and underfunded schools often lack the equipment or partnerships needed for practical learning. As a result, forestry vocational programs suffer from low enrollment, even as student interest remains high [3].

Forestry simulators provide a safer, more controlled training alternative, but their adoption is hindered by high costs and hardware requirements [4]. Advances in consumer-grade Virtual Reality (VR), however, offer a promising solution [5]. Affordable Head-Mounted Displays (HMDs) now enable immersive, scalable training experiences, making high-quality forestry education more accessible, especially in underserved regions [6].

This paper presents a user-centered evaluation of a VR HMD-based harvester training simulator, directly addressing critical gaps in forestry harvester operator education. Specifically, recent research highlighted major limitations in conventional operator training, such as high economic cost, safety risks, and lack of scalability as well as the paucity of forestry-specific research assessing the effectiveness of simulator-based instruction [4,7]. In particular, it underscored the need for empirical studies of forestry training applications built on modern, consumer-grade VR hardware that evaluate cognitive load, usability, user experience, and simulator sickness.

To advance understanding in this underexplored area, we designed and developed an immersive training application, which runs on the Meta Quest 2 headset, incorporating two core modules: machine navigation and head/boom manipulation [8]. Guided by this design, and informed by cognitive load and instructional design theory, we designed and conducted a research study, presented in this paper, that investigated the following research questions:

• RQ1: What level of cognitive load do users experience while operating the VR HMD-based harvester training simulator?
• RQ2: What is the perceived usability and overall user experience of the system?
• RQ3: What is the severity of simulator sickness during operation of the VR HMD-based harvester simulator?

Grounded in the premise that VR learning experiences aligned with instructional design principles and effective cognitive load management tend to yield better task performance and more positive user experiences, we examined these questions through the following working hypotheses:

Hypothesis 1 (H1). 

The harvester training modules, while more demanding than a simple VR tutorial, are expected to elicit acceptable levels of cognitive load compatible with successful task execution.

Hypothesis 1 (H2). 

Higher perceived usability and more positive user experience will be associated with better task performance in the training modules.

Hypothesis 1 (H3). 

The VR training will induce measurable simulator sickness, but symptom severity will remain within acceptable limits for VR HMD-based training (SSQ Total Score < 40, per Caserman et al. [9]).

To empirically test these hypotheses, we designed a study involving 26 participants that was conducted to evaluate perceived cognitive workload, usability, user experience, and simulator sickness. This work presents one of the first comprehensive evaluations of a forestry harvester specific VR training system, utilizing validated Human Computer Interaction (HCI) methods. By validated HCI methods, we refer to established human-centered evaluation practices for interactive systems that combine (i) standardized, validated self-report instruments and (ii) objective/behavioral performance measures. In this study, these include NASA Task Load Index (NASA-TLX) for perceived workload, System Usability Scale (SUS) for usability, User Experience Questionnaire—Short (UEQ-S) for user experience, Simulator Sickness Questionnaire (SSQ) for simulator sickness, along with task success and completion-time metrics, and post-session qualitative feedback. Together, these methods provide a comprehensive assessment of effectiveness, efficiency, satisfaction, and comfort in an immersive training system. The findings demonstrate that low-cost VR solutions can provide a viable and scalable complement to traditional training infrastructure, especially in high-risk, resource-constrained domains such as forestry operations.

The remainder of the paper is structured as follows: Section 2 presents a literature review of research on VR in heavy machinery training for forestry operations and describes the gap in forestry operator education. Section 3 outlines the theoretical framework and contextualizes the research questions that guide this study. Section 4 describes the overall research design detailing the experimental setup whereas Section 5 provides information about participant recruitment. Section 6 details the user experience within the VR environment, describing the training modules and the interactive components. Section 7 explains the metrics and instruments used for evaluation in this study. Section 8 focuses on the step-by-step experimental procedure followed in the study. The Results section, presented in Section 9, outlines the key findings and statistical analyses, followed by Section 10, the Discussion section, which interprets these results in relation to prior research and theoretical implications. Finally, Section 11 summarizes the main contributions, discusses limitations, and outlines future research directions.

2. Related Work

Recent research has explored the role of immersive technologies such as VR HMDs for training in domains involving heavy machinery and safety-critical operations, often with the goal of reducing cost while increasing safety by creating a virtual, interactive environment. VR HMDs have been widely adopted for training in aviation [10], construction [11], military [12], medicine [13], and many other domains offering realistic operational contexts. These studies have validated this technology across various parameters, including mental workload, user performance, user experience, quality of interaction, motion sickness, presence, usability, immersion, educational utility, and overall training effectiveness within their respective domains. Moreover, being small and portable makes VR HMDs convenient for users who need to train in remote locations. Collectively, these findings suggest that VR HMDs are an effective medium for skill development and task training.

However, research on validating immersive and interactive technologies within the forestry sector for training remains limited. Existing studies have primarily focused on evaluating system performance or user skill outcomes, with comparatively little emphasis on user-centered factors. The early work by Ovaskainen et al. [14,15] evaluated desktop-based forestry and mechanized training systems primarily in terms of productivity, cycle time, and operational efficiency, providing valuable benchmarks for system-level performance but limited insight into user experience or cognitive workload. Similarly, research by Lapointe et al. [1] and Schwegman et al. [16] focused on comparing interaction techniques and system configurations, assessing effectiveness and accuracy under controlled conditions. More recent studies evaluated simulator-based training for logging equipment by examining operator performance on real machines, trainee perceptions during real-world operation, and cost efficiency relative to traditional training methods [4]. Similarly, Lee et al. [17] conducted a comparative evaluation of simulator-based and machine-based training for novice operators, demonstrating comparable operational proficiency after short-duration training, while additionally highlighting benefits related to trainee confidence and reduced stress. Another study examined the use of a virtual reality simulator to assess bimanual motor skills in harvester operator candidates, demonstrating that early performance metrics such as run time and cutting height can reliably predict operator skill and support the selection process [18]. While these studies contribute to understanding system usability at a functional level, user-centered factors such as mental workload, user experience, usability, and perceived comfort were only minimally addressed. Moreover, most prior work centers on non-immersive or semi-immersive simulators, leaving a gap in understanding how fully immersive VR HMD-based simulators can support effective operator training in forestry. Industry-grade forestry simulators developed by manufacturers such as John Deere [19], Ponsse [20], and Komatsu [21] currently dominate operator training. These semi-immersive simulators achieve a good degree of realism through the integration of physical controls such as steering wheels, pedals, and shifters, coupled with synchronized visual and auditory feedback on multi-screen displays. In addition, force-feedback mechanisms replicate the vibration and movement of an actual harvester, effectively meeting the training requirements of professional operators [22,23]. Ovaskainen et al. [14] found that operator techniques on the Timberjack simulator closely resembled real forest work, while Lapointe et al. [1] reported significant gains in productivity and cost reduction after desktop-based simulator training. Recent comparisons of John Deere and Ponsse simulator training with real machine training similarly showed no significant differences in overall performance but substantial cost savings [4,17]. Additionally, further research highlights psychological benefits of simulation-based harvester operator training, such as increased confidence and reduced stress, anxiety, frustration and mental workload when compared to conventional training [17,24]. Studies also show improvements in performance, productivity, and operational quality [1,25]. In addition, simulator-based training has been shown to support organizational and instructional oversight in forestry education. The ProForSim project emphasized that harvester simulators provide a consistent and controllable training environment, enabling instructors to monitor trainee education across heterogeneous logging contexts [26]. But, again, all of these findings are based on setups that are not fully immersive. Therefore, it remains unclear whether higher levels of immersion would enhance or hinder the psychological and performance outcomes of forestry harvester trainees.

Furthermore, these systems rely on single- or multi-screen displays coupled with physical control interfaces. While such configurations offer advantages, including reduced simulator sickness, integrated haptic feedback, and a wide field of view (FOV), they typically require specialized hardware and occupy substantial physical space. As a result, they are costly and less accessible for small educational institutions and small-scale training programs. In addition, their limited portability and reliance on proprietary control units restrict opportunities for broad deployment and experimental use.

To the best of our knowledge, there is currently no forestry harvester training system that solely relies on a low-cost VR HMD and has been validated from a user-centered perspective. This gap is particularly significant given its growing maturity, affordability, and accessibility. It is timely to evaluate their potential for accessible and scalable forestry harvester operator training. Therefore, the present study aims to evaluate the cognitive load, usability, user experience, and simulator sickness associated with training on a low-cost VR HMD-based simulator. By empirically assessing these parameters, this study provides insights into the feasibility and readiness of VR HMD for forestry harvester operator training.

3. Theoretical Framework

3.1. Cognitive Load in VR

Cognitive load plays a central role in how learners acquire, retain, and apply new skills, particularly in high-risk, skill-intensive domains such as heavy machinery operation. Because working memory has limited capacity, learning effectiveness depends on the mental effort required to process task-relevant information. Cognitive load is defined as the mental effort imposed on working memory during learning or task execution [27]. While the Cognitive Load Theory (CLT) distinguishes three components: intrinsic load, arising from task complexity; extraneous load, introduced by inefficient instructional design; and germane load, reflecting effort devoted to schema construction and automation [28]. Effective instructional design aims to manage intrinsic load, minimize extraneous load, and support germane load through strategies such as content segmentation, multimodal cues (e.g., visual and auditory), and signaling of critical information [29]. These principles are particularly salient in immersive VR environments, where design choices can either mitigate or amplify cognitive demands.

Empirical findings on the effects of VR HMD-based training on cognitive load are mixed and strongly task-dependent. Several studies report increased cognitive demands relative to less immersive modalities. Frederiksen et al. [30] found that VR HMD-based simulation induced a 7.9% higher cognitive load than Desktop-VR, attributing this to increased task interactivity (raising intrinsic load) and immersive environmental distractions (increasing extraneous load). Initial performance deficits were observed in VR but diminished with repeated practice, suggesting novelty effects. Similarly, Hebbar et al. [31] showed that multitasking requirements in VR flight simulation significantly increased mental demand, underscoring the role of task complexity. Elevated cognitive load in VR has also been linked to poorer long-term motor memory retention and reduced context transfer [32]. Huang et al. [33] further demonstrated that while immersive VR can enhance motivation, it may overburden working memory when conceptual material requires sustained attention.

Conversely, other studies indicate that VR HMD-based training does not inherently increase cognitive load and may even reduce it under appropriate instructional conditions. Wenk et al. [34] reported no significant differences in perceived cognitive load between VR/AR HMD-based and 2D modalities, while observing advantages for VR/AR in usability, motivation, and embodiment. Likewise, Liu et al. [35] found lower cognitive load in VR-based primary science education compared to traditional classroom instruction, attributing this to visually rich yet simplified instructional design. Additional evidence shows that targeted design strategies, such as real-time feedback [36], visual signaling [37], and textual cues for spatial guidance [38], can effectively reduce extraneous load in immersive environments.

Overall, these findings clarify that the impact of VR HMD-based training on cognitive load is not determined by immersion alone, but by the interaction of task complexity, interaction density, technology novelty, learners’ prior VR experience, and confounding factors such as simulator sickness. Despite growing interest in cognitive load within immersive systems, this issue remains underexplored in the context of forestry harvester operation training, a domain characterized by high cognitive demands and substantial risk of overload. Assessing cognitive load in VR HMD-based forestry harvester simulators is therefore essential to determine whether immersion supports or hinders learning and to identify design factors that mitigate excessive mental effort.

3.2. Usability and User Experience in VR HMD-Based Training

Usability was introduced in 1986 as a core concept in HCI [39]. It is commonly defined as a multidimensional quality of interactive systems, encompassing ease of learning, efficiency of use, memorability, error tolerance and recovery, and overall user satisfaction [40]. Prior usability research on VR-based training has largely focused on safety-critical domains such as aviation and driving, where studies consistently report positive outcomes [41,42,43]. A common characteristic of these systems, however, is the use of specialized physical peripherals (e.g., yokes, pedals, steering wheels) that closely replicate real-world controls.

In contrast, relatively few studies have examined usability when generic VR handheld controllers serve as the primary input modality. Mapping complex machine operations onto abstract controller-based interactions presents substantial design challenges; when control mappings fail to preserve operational logic, constraints, or proprioceptive cues, both perceived usability and skill transfer may be compromised. Empirical evidence addressing this issue remains limited. Fussell et al. [44] reported positive usability outcomes for a flight training application using an HTC Vive Pro with handheld controllers. However, the operational demands, control complexity, and environmental context of forestry harvester operation differ markedly from those of aviation or driving simulators, warranting domain-specific investigation. Consequently, systematic usability evaluation is a prerequisite for integrating VR HMD-based harvester simulators into formal training programs.

In this study, usability was assessed using SUS, a widely adopted and validated instrument for measuring perceived usability across diverse systems, with demonstrated flexibility through minor contextual adaptations [45,46]. SUS was selected for its brevity, robustness, and ability to produce a single quantitative score that supports cross-study comparison. Prior work has also shown SUS to be sensitive to variations in interface design, interaction techniques, and user experience in immersive VR environments [47,48,49].

Beyond usability, user experience (UX) is a key determinant of technology adoption and sustained use. Well-designed user experiences promote initial acceptance and long-term engagement with emerging technologies [50]. Contemporary frameworks conceptualize UX as comprising task type, environment, and application context [51]. Accordingly, in this work the primary focus was placed on task-related interaction techniques, navigation, selection and manipulation, and system control, which are fundamental to coordinated harvester operation.

These interaction techniques are implemented using the OpenXR input framework, providing a standardized and extensible basis for VR selection and manipulation. While HCI research increasingly emphasizes multisensory interaction beyond vision and audition [52], the present study focuses on visual, auditory, and tactile feedback, as these modalities most directly align with the operational demands of forestry machinery training. The system was developed through an iterative design process incorporating formative feedback from computer science students with prior VR experience to refine interaction design and usability [8].

User experience in VR can be assessed using several standardized instruments, including the User Experience Questionnaire (UEQ) [53], meCUE [54], AttrakDiff [55], and the Game Experience Questionnaire [56]. A recent review indicates that the UEQ has been the most frequently used UX instrument since 2017, including in immersive VR contexts [57]. The short version of the UEQ (UEQ-S) has gained particular traction due to its reduced length while retaining core dimensions of pragmatic and hedonic quality [58]. UEQ-S was therefore selected to minimize participant fatigue, an important consideration in cognitively and physically demanding VR sessions, while still capturing essential aspects of user experience.

3.3. Simulator Sickness in VR Training

A well-documented limitation of immersive VR HMD-based training is simulator sickness, characterized by symptoms such as nausea, disorientation, and oculomotor discomfort [59]. It primarily arises from sensory conflict between visual input and vestibular (inner-ear) cues [60,61]. In the literature, simulator sickness, cybersickness, and visually induced motion sickness are often used interchangeably to describe similar symptom profiles [62]. In this study, the term simulator sickness refers specifically to motion sickness induced by VR HMD-based simulation.

Reported prevalence rates suggest that 80–95% of users experience some level of discomfort during VR HMD exposure [63,64]. Comparative studies across display modalities, including desktop VR, multi-sided projection systems, and VR HMDs, consistently show that increasing immersion is associated with higher simulator sickness severity [65], a concern that is particularly salient for fully immersive training applications. Simulator sickness has been associated with reduced learning effectiveness, impaired task performance, decreased motivation, and diminished spatial ability [66,67,68]. In severe cases, symptoms can prevent task completion; Cobb et al. [69] reported that approximately 5% of users discontinue VR tasks due to discomfort. Simulator sickness is both polygenic and polysymptomatic, reflecting the influence of multiple interacting factors, including individual susceptibility as well as hardware and software characteristics [70]. Numerous mitigation strategies have been proposed, such as interpupillary distance (IPD) calibration, increased display resolution, field-of-view (FOV) reduction, and virtual camera or image manipulation [71,72,73,74]. However, many of these approaches require hardware or software modifications that increase system cost and technical complexity [75].

Other approaches, including user acclimatization, symptom decay with repeated exposure, and pharmacological interventions, have also been explored, yet no single method has proven universally effective given the multifactorial nature of simulator sickness [76,77,78]. Importantly, discomfort may lead users to adopt compensatory behaviors, such as reducing head movement or avoiding aggressive maneuvers [79]. If such adaptations transfer to real-world operation, they may pose safety risks. Consequently, it is essential to characterize simulator sickness in immersive forestry harvester training and to use these insights to inform the design of safer, more comfortable training environments and targeted mitigation strategies. Given its prevalence and impact, reliable measurement of simulator sickness is critical. The SSQ remains the most widely used and validated instrument for this purpose [59]. Although alternative measures, such as the Virtual Reality Sickness Questionnaire [80], the Cybersickness in Virtual Reality Questionnaire [81], the Motion Sickness Susceptibility Questionnaire [82], and the MIsery SCale [83], have been proposed, they are less extensively validated and not yet widely adopted [84]. Furthermore, SSQ scores have been shown to correlate with physiological indicators such as heart rate variability during VR exposure, supporting their validity beyond subjective self-report [85]. For these reasons, the SSQ continues to be regarded as the most robust and reliable instrument for assessing simulator sickness in VR HMD-based experiences [86,87,88].

4. Research Design

The research design was structured to systematically investigate cognitive load, usability, user experience, and simulator sickness during operation of a VR HMD-based harvester training simulator. To this end, we employed a mixed-methods, user-centered evaluation that integrates validated subjective instruments with behavioral performance measures. This approach aligns with established HCI practice for evaluating interactive systems, combining standardized questionnaires, performance logging, and qualitative user feedback to triangulate findings. The study followed a clearly defined, reproducible sequence of stages (Figure 1), including participant briefing, VR acclimatization, completion of two training modules, and post-task assessment.

Figure 1. Study design and evaluation method illustrating the experiment phase (participant recruitment, pre-test, VR sessions, post-test), followed by data analysis and results addressing the research questions.

In our previous work [8], we conducted a careful analysis to identify the most relevant features and interactions necessary for simulating realistic harvester operations in VR. The objective was to ensure that the simulator replicates key operational tasks to provide an experience closely aligned with real-world operation and enables effective user-centered evaluation. Two primary interaction modules were implemented:

• Harvester navigation: Maneuvering the tracked harvester forward, backward, left, and right at appropriate speeds to reach target locations.
• Head/boom control: Extending, retracting, raising, and lowering the boom; rotating the harvester head for alignment; and opening or closing the claws to grip or release objects.

To evaluate current study hypotheses, a within-subjects experimental design was employed, with each participant completing both training modules. The independent variable was the training module, while the dependent variables included cognitive load, usability, user experience, and simulator sickness. This design reduces inter-participant variability and enables direct comparison across task conditions. Quantitative data were collected using standardized instruments (Section 7), complemented by qualitative feedback from post-session interviews, yielding a comprehensive user-centered assessment.

The training modules were presented in a fixed instructional order, navigation followed by head/boom control, reflecting the prerequisite structure of real-world harvester operation, where basic vehicle control precedes articulated manipulation. Counterbalancing module order would therefore violate task dependencies and reduce ecological validity relative to authentic training pipelines. The procedural flow (Figure 1) and detailed protocol (Section 8) were intentionally designed to mirror this curricular progression.

5. Participants

Participants were recruited using a multi-channel outreach strategy that included class announcements, laboratory mailing lists, printed flyers, and the university’s research participant pool. Participation was entirely voluntary.

A total of 26 students participated in the study, representing a diverse academic background as shown in Table 1. The use of a student sample in this study was purposeful rather than incidental. Because the VR harvester training simulator is intended for integration into student training programs, students represent the appropriate and intended user population for early-stage evaluation.

Table 1. Participant Demographics (N = 26).

Characteristic	N (%)
Gender
Male	14 (53.85%)
Female	10 (38.46%)
Other	2 (7.69%)
Age
Mean	24.46
Standard Deviation	4.03
Median	23.5
Prior VR Experience
None/Never	10 (38.462%)
Basic/Rarely	8 (30.769%)
Intermediate/Frequently	8 (30.769%)
Eyewear
None	16 (61.54%)
Glasses	10 (38.5%)
Education
Master’s degree	12 (46.15%)
Bachelor’s degree	8 (30.77%)
Associate degree/Some college/High school	6 (23.08%)

All study procedures adhered to the ethical guidelines approved by the university’s Institutional Review Board (IRB). To ensure participant safety, eligibility criteria required that students (a) had no history of seizures, epilepsy, motion sickness, migraines, or other neurological conditions, (b) did not have any implanted medical devices, and (c) were in an optimal physical and mental state at the time of the study. The students were required to sign an informed consent form followed by a demographic questionnaire prior to participation. The demographic data of the participants has been detailed as shown in Table 1.

6. Training System Walkthrough

The simulator presents a realistic forest environment with a tracked harvester model positioned on a flat terrain surrounded by trees (see Figure 2). The John Deere—959MH tracked harvester CAD model is sourced from an online 3D model marketplace. To enhance immersion, ambient sound is incorporated, including harvester engine noise and subtle background music audible from within the cabin. These elements create an engaging and contextually relevant setting for operator training.

Figure 2. Simulated forest environment featuring the John Deere 959MH tracked harvester model.

Upon launching the application, users are seated inside the harvester cabin on the operator chair. A welcome canvas appears in front of the user as shown in Figure 3, offering a selection interface with three interactive training modules: navigation, head/boom movements, and tree processing. Each module comes with a training session followed by a practice session. This study focuses on the first two modules.

Figure 3. Welcome screen showing module selection for Navigation, Head/Boom Movements, and Tree Processing.

6.1. Learning Modules of the Interactive Tutorial System

A consistent instructional framework is employed across all training modules. Each module begins with an introductory card outlining the learning objectives for that module. Subsequent cards present a title along with a corresponding instruction as seen in Figure 4. These are presented as on-screen text and read aloud using Wit.ai’s Text-to-Speech functionality, using the Wit.ai API version current at the time of development in 2024 (as Wit.ai defaults to the latest API version when no explicit version parameter is specified). Additionally, visual cues appear on the controllers to assist users in identifying the correct inputs. Real-time feedback is provided to the user through on-screen prompts, guiding correct interactions during the task.

Figure 4. Instructional interface within the harvester simulator showing labeled UI elements including the step indicator, title, action instructions, and real-time feedback text.

6.2. Module 1—Navigation

The module starts with a training session. The objective of the training session is to enable users to maneuver the harvester in all four directions: forward, backward, left, and right. These basic navigation skills are essential for operating the machine effectively. As detailed in Section 6.1, the tutorial system teaches the user how to do this with real-time feedback. The instructions presented during the navigation tutorial session are summarized in Table 2 in the order in which they are given. Once this training session is completed, the user has the option to continue to the practice session.

Table 2. Navigation Training Instruction Cards (each action is evaluated based on completion time in seconds; skipped actions are marked as NA).

Instruction Card No.	Instruction	Measured as
Instruction card 1	Select the Navigation module from Welcome canvas	Time (s)/NA
Instruction card 2	Read Navigation module instructions	Time (s)/NA
Instruction card 3	Move the harvester forward	Time (s)/NA
Instruction card 4	Move the harvester backward	Time (s)/NA
Instruction card 5	Move the harvester left	Time (s)/NA
Instruction card 6	Move the harvester right	Time (s)/NA
Instruction card 7	Return to Home, Restart, or move on to Practice session	Time (s)/NA

In the practice session, users apply the skills acquired during training to navigate a winding forest path. As seen in Figure 5, the path is marked with directional arrows and is surrounded by trees, requiring users to reach a designated endpoint without colliding with trees or toppling the harvester. Users are expected to complete a set of tasks during the navigation practice session as detailed in Table 3.

Figure 5. View from inside the harvester cabin during the navigation practice module. Directional arrows guide the user along the designated forest path to practice movement control.

Table 3. Navigation Practice Session Tasks (each task is evaluated as Pass, Fail, or NA).

Task No.	Task	Measured as
Task 1	Select Navigation practice session	Pass/Fail/NA
Task 2	Understand the rules of the Navigation practice session and click on start	Pass/Fail/NA
Task 3	Drive the harvester to the destined position	Pass/Fail/NA
Task 4	Check your performance evaluation	Pass/Fail/NA
Task 5	Return to Home; practice session is completed	Pass/Fail/NA

Upon successful completion, a performance evaluation is presented based on several predefined metrics such as total time taken to complete the session, time bonus awarded based on a predefined threshold, fuel remaining at the end of the session, fuel bonus based on efficiency threshold, number of arrows collected during the run, number of collisions and instances of toppling. A final performance score is calculated and displayed as a percentage.

6.3. Module 2—Head/Boom Movements

The objective of this module is to allow the user to develop the ability to extend and retract the boom to control the horizontal reach, while raising and lowering the boom to adjust the vertical positioning of the harvester head, and rotating the head left or right to align with the target tree properly. Additionally, the session covers the opening and closing of the harvester claws, which are used to securely grip or release objects as needed during operation. The instructions presented during the head/boom tutorial session are summarized in Table 4 in the order in which they are presented. In the practice session, users apply the skills acquired during training to precisely grab objects from a tree and place them in a basket without dropping them (see Figure 6). Users are expected to complete a set of tasks during the head/boom practice session that are detailed in Table 5. Upon successful completion, a performance evaluation is presented based on several predefined metrics such as total time taken to complete the session, time bonus awarded based on a completion threshold, number of target objects collected, and number of times the object was dropped. A final performance score is calculated and displayed as a percentage.

Table 4. Head/Boom Training Instruction Cards (each action is evaluated based on completion time; skipped actions are marked as NA).

Instruction Card No.	Instruction	Measured as
Instruction card 1	Select the Head/Boom module from Welcome canvas	Time/NA
Instruction card 2	Read Head/Boom module instructions	Time/NA
Instruction card 3	Tilt the harvester head	Time/NA
Instruction card 4	Bring back the harvester head to its original position	Time/NA
Instruction card 5	Close harvester claw	Time/NA
Instruction card 6	Open harvester claw	Time/NA
Instruction card 7	Virtual joystick	Time/NA
Instruction card 8	Identify virtual joysticks	Time/NA
Instruction card 9	Grab virtual joysticks	Time/NA
Instruction card 10	Raise harvester boom	Time/NA
Instruction card 11	Lower harvester boom	Time/NA
Instruction card 12	Extend harvester boom	Time/NA
Instruction card 13	Retract harvester boom	Time/NA
Instruction card 14	Rotate harvester head left	Time/NA
Instruction card 15	Rotate harvester head right	Time/NA

Figure 6. View from inside the harvester cabin during the head/boom practice module, featuring a tree with target objects and a collection basket.

Table 5. Head/Boom Practice Session Tasks (each task is evaluated as Pass, Fail, or NA).

Task No.	Task	Measured as
Task 1	Select Head/Boom practice session	Pass/Fail/NA
Task 2	Understand the rules of the Head/Boom practice session and click on start	Pass/Fail/NA
Task 3	Move harvester closer to tree	Pass/Fail/NA
Task 4	Move the harvester boom near the target object	Pass/Fail/NA
Task 5	Grab the target object	Pass/Fail/NA
Task 6	Move the harvester boom close to basket	Pass/Fail/NA
Task 7	Drop the object in the basket	Pass/Fail/NA
Task 8	Complete picking all 3 objects	Pass/Fail/NA
Task 9	Check performance evaluation	Pass/Fail/NA
Task 10	Return to Home; practice session is completed	Pass/Fail/NA

7. Measures

In order to measure the cognitive load of the users, unweighted NASA-TLX was employed [89]. This subjective assessment tool evaluates perceived task demand across six subscales: mental demand, physical demand, temporal demand, performance, effort, and frustration. Each subscale is rated within a range of 100 points with 5-point steps, where ‘1’ corresponds to ‘Very Low/Perfect’ and 20 corresponds to ‘Very High/Failure’.

Next, the SUS was used to measure the usability of the system. It is a standardized and widely adopted tool developed by Brooke et al. [46] and provides a high-level subjective view of usability. The SUS comprises a 10-item questionnaire that alternates between positive and negative statements about the system. The five-point Likert scale ranges from Strongly Disagree (1) to Strongly Agree (5).

Then, to measure user experience, the short version of the UEQ-S was utilized [53]. The questionnaire consisted of 8 items, where each item is to be rated on a seven-point semantic differential scale. The data is rescaled to a standardized range of $-3$ to $+3$ during analysis. The value $-3$ represents the most negative evaluation, 0 a neutral response, and $+3$ the most positive evaluation. The items are categorized into two main dimensions: Pragmatic Quality (goal-oriented attributes) and Hedonic Quality (non-goal-oriented attributes).

User’s experience with simulator sickness was measured using SSQ, a widely adopted research tool to quantify and compare simulator sickness in virtual environments [59]. Participants provided subjective severity ratings of 16 symptoms on a scale from 0 (no perception) to 3 (severe perception).

Lastly, the task performance of the user was also recorded. For the training session, quantitative metrics focused on task duration. Specifically, the time spent on each instructional card was recorded. If a task could not be completed due to a system malfunction, if the participant required assistance from the evaluator, or if a step was skipped, the time was marked as ‘Not Applicable’ (NA). Otherwise, the duration in seconds was documented for each step (See Table 2 and Table 4). For the practice session, task performance was evaluated using a simple success criterion. Each task was marked as ‘Pass’ if the participant completed it independently, ‘Fail’ if they required evaluator assistance or chose to exit the session, and ‘NA’ if the task could not be completed due to technical issues (See Table 3 and Table 5).

User performance during the practice sessions was also evaluated using a set of predefined metrics. For example: the total time taken to complete the session, time-based performance bonus, amount of fuel consumed, fuel efficiency bonus, etc. This systematic recording allowed for a comprehensive assessment of user performance and system usability across both guided and unguided scenarios. The questionnaires (NASA-TLX, SUS, UEQ-S) handed out to participants are provided as Supplementary Materials.

8. Procedure

The study was conducted in the Extended Reality Lab at Oregon State University, Corvallis, OR, USA. Hardware and software configurations were held constant across sessions to ensure experimental reliability. The environment was generally quiet, with only minimal background noise that did not interfere with task performance. Prior to the experiment, participants completed informed consent and a demographic questionnaire as part of the pre-test procedure.

The VR experiment comprised three sequential phases: (1) a baseline VR task, (2) the navigation module, and (3) the head/boom control module (Figure 1). In the first phase, participants completed the ‘First Steps for Quest 2’ application from the Meta Store to establish a uniform baseline of familiarity with the Meta Quest 2 interface and interaction mechanics. This brief tutorial supported acclimatization to the VR environment, including controller handling, spatial navigation, and basic interactions. For VR novices, it served as an introduction; for experienced users, it functioned as a refresher. Participants followed in-application guidance and explored freely. This familiarization phase lasted approximately five minutes, after which participants completed the NASA-TLX to assess perceived workload. Before the second phase, participants were shown a 360° overview of the tracked harvester model using Unity’s Scene mode. Key components, including the cabin, boom, head, and tracks, were highlighted to support structural understanding of the machine. This demonstration was added based on pilot-study feedback indicating the need for improved conceptual grounding. During the experiment, participants were seated in a sturdy chair with a table to support their arms, approximating the seated operational posture of real harvester operators. Sessions were conducted using a Meta Quest 2 HMD connected to a VR-ready desktop via a link cable. An evaluator observed each session, documenting interaction patterns, user behavior, and challenges encountered. Participants were encouraged to verbalize their thoughts using a think-aloud protocol, providing insight into cognitive strategies and usability issues. A dual-recording setup captured both participants’ physical behavior and the first-person VR view (Figure 7).

Figure 7. Dual recording setup capturing the participant and the first-person view inside the VR headset.

In the second phase, participants completed the navigation module, consisting of a guided training tutorial followed immediately by a practice session. Task performance metrics (Section 7) were recorded throughout. Upon completion, participants completed the NASA-TLX to assess workload associated with navigation tasks.

In the third phase, participants engaged with the head/boom control module, again completing both training and practice sessions. Task performance was recorded across both sessions. Following completion, participants completed the NASA-TLX, SSQ, UEQ-S, and SUS. Finally, participants responded to open-ended questions designed to elicit qualitative feedback; these questions are provided in the Supplementary Materials.

9. Results

All 26 participants took part in the study and completed the full set of post-session questionnaires. These included the NASA-TLX, SUS, UEQ-S and SSQ. Although participant 8 and participant 13 (n = 2) discontinued the practice sessions due to simulator-induced nausea, and participant 20 (n = 1) was unable to complete the head/boom practice session due to technical difficulties, all three were able to provide complete responses across all four instruments: NASA-TLX, SUS, UEQ-S and SSQ.

9.1. Cognitive Load

Participants completed the NASA-TLX questionnaire following each of the three test modules: (1) a baseline VR interaction task using the First Steps for Quest 2 application, (2) the navigation module comprising a tutorial and a practice session, and (3) the head/boom control module, also consisting of a tutorial and a practice session.

Cognitive load was analyzed using a repeated measures ANOVA to examine the effect of module type (baseline, navigation, and head/boom) on participants’ perceived workload, with module serving as the sole predictor. Dependent variables were the overall cognitive load score for each module. The analysis revealed a statistically significant main effect of module on cognitive load, F(2, 50) = 40.597, p < 0.001, indicating that the perceived workload varied across conditions. The partial eta squared (${\eta }_p^2$ = 0.619) suggested a large effect size, with approximately 61.9% of the variance in workload scores accounted for by the module type. Post hoc pairwise comparisons with Bonferroni correction revealed that both the navigation and head/boom modules elicited significantly higher workload than the Baseline module (Navigation vs. Baseline: p < 0.001, d = 1.01; Head/Boom vs. Baseline: p < 0.001, d = 1.28). However, the difference in workload between the navigation and head/boom modules was not statistically significant (p = 0.230, d = 0.27), suggesting a plateau in perceived cognitive load beyond the navigation module (see Table 6).

Table 6. Repeated Measures ANOVA and Post Hoc Comparisons for Cognitive Load Across Modules.

Within Subjects Effects
Source	Sum of Squares	df	Mean Square	F	p	${\eta }_p^2$
Modules	5651.335	2	2825.668	40.597	<0.001	0.619
Residuals	3480.146	50	69.603
Post Hoc Comparisons
Comparison	Mean Diff	SE	df	t	Cohen’s d	${\mathit{p}}_{\mathit{bonf}}$
Baseline vs. Navigation	−15.609	2.283	25	−6.838	−1.013	<0.001
Baseline vs. Head/Boom	−19.776	2.400	25	−8.240	−1.283	<0.001
Navigation vs. Head/Boom	−4.167	2.257	25	−1.847	−0.270	0.230

Descriptive statistics indicated a progressive increase in mean workload ratings across modules as well. Participants reported the lowest workload during the baseline task (M = 18.65, SD = 15.23), followed by the navigation module (M = 34.26, SD = 15.45), and the highest workload during the head/boom module (M = 38.43, SD = 15.54). This trend is further reflected in the subscale-level breakdown across nearly all NASA-TLX attributes as shown in Figure 8. While physical demand scores showed smaller differences across modules, the general pattern suggests that the added complexity of the later modules contributed to elevated perceived workload.

Figure 8. Graph showing the mean values for NASA-TLX workload data for the Baseline, Navigation and Head/Boom module with error bars representing standard deviation.

A repeated-measures ANOVA was conducted to see whether prior VR experience influenced cognitive load across the Baseline, navigation, and head/boom modules as seen in Table 7.

Table 7. Repeated Measures ANOVA Assessing Cognitive Load Across Task Modules and the Influence of Prior VR Experience.

Within Subjects Effects
Source	Sum of Squares	df	Mean Square	F	p
Module × VR Experience	574.972	4	143.743	2.276	0.075
Residuals	2905.174	46	63.156
Between Subjects Effects
Source	Sum of Squares	df	Mean Square	F	p
VR Experience	277.171	2	138.586	0.227	0.799
Residuals	14,050.642	23	610.897

The module × VR experience interaction was not statistically significant, F(4, 46) = 2.28, p = 0.075, suggesting that changes in cognitive load across tasks did not differ based on VR experience level. The between-subjects effect of VR experience was also non-significant, F(2, 23) = 0.23, p = 0.799, indicating that overall cognitive load did not differ by prior VR exposure. Descriptive comparisons showed no consistent pattern of higher or lower cognitive load across VR experience groups, with considerable variability within each category (see Table 8).

Table 8. Descriptive statistics of mean Cognitive Load scores for each module based on participants’ prior VR experience.

Module	VR Experience	N	Mean CL	SD	SE	Coefficient of Variation
Baseline	None	10	$23.333$	$18.588$	$5.878$	$0.797$
	Basic	8	$19.583$	$16.214$	$5.733$	$0.828$
	Intermediate	8	$11.875$	$6.038$	$2.135$	$0.508$
Navigation	None	10	$35.583$	$15.441$	$4.883$	$0.434$
	Basic	8	$31.667$	$19.003$	$6.719$	$0.600$
	Intermediate	8	$35.208$	$13.182$	$4.660$	$0.374$
Head/Boom	None	10	$39.500$	$16.879$	$5.338$	$0.427$
	Basic	8	$34.583$	$16.862$	$5.962$	$0.488$
	Intermediate	8	$40.938$	$13.660$	$4.830$	$0.334$

Partial Pearson correlations, controlling for VR experience, showed strong positive associations among the three cognitive load measures: Baseline–Navigation (r = 0.745, p < 0.001), Baseline–Head/Boom (r = 0.730, p < 0.001), and Navigation–Head/Boom (r = 0.725, p < 0.001). These relationships indicate that, independent of VR experience, participants who reported higher cognitive load in one task tended to report higher cognitive load in the others.

Overall, the results demonstrate that modules significantly influenced cognitive load, whereas prior VR experience did not, and individual differences in cognitive load were consistent across tasks even after controlling for VR experience.

Across both navigation and head/boom module, Spearman’s correlations between cognitive load and total time taken to complete each module (training + practice) were non-significant as seen in Table 9.

Table 9. Summary of Spearman’s Correlations Between Cognitive Load (CL) and Task Completion Times.

Variable	Spearman’s Rho	p-Value
Navigation Module
Navigation CL × Total time taken for Navigation module	0.153	0.455
Head/Boom Module
Head/Boom CL × Total time taken for Head/Boom module	−0.151	0.461

This indicates that even participants who reported higher cognitive load did not necessarily take more time to complete the modules. Figure 9 provides a participant-level visualization of these patterns. Importantly, success rates remained consistently high across both modules (reported in Section 9.5), indicating that elevated cognitive load did not meaningfully impair task performance.

Figure 9. Relationship Between Cognitive Load and Task Duration for Each Participant for Navigation and Head/Boom module.

9.2. System Usability

The overall mean SUS score was found to be 76.63 (SD = 12.21), with individual scores ranging from 42.5 to 97.5. The distribution of scores revealed a 1st quartile value of 70, a median of 75, and a 3rd quartile value of 85. Based on the interpretation scale proposed by Bangor et al. [90] and contextualized by Sauro et al. [91], this score falls within the ‘Good’ adjective range, corresponds to a grade of ‘B’, and is located in the 3rd quartile of usability benchmarks (see Figure 10).

Figure 10. System Usability Score for the harvester system compared against contextualization scale based on formula from [46]. The box represents the interquartile range, with the internal solid line indicating the median SUS score (75). The whiskers denote the minimum (42.5) and maximum (97.5) observed scores. The dotted horizontal line indicates the mean SUS score (76.63).

On the acceptability scale, the system was rated as ‘Acceptable’, and on the Net Promoter Score (NPS) scale, it was categorized as ‘Passive’. The system also scored ‘Above Average’ when compared to industry benchmarks, indicating strong usability perception among participants. These calculations were generated using the SUS Analysis Toolkit developed by Blattgerste et al. [92] and have been summarized in the Table 10 and Table 11.

Table 10. SUS Score Summary.

Mean	SD	Min	Max	Q1	Median	Q3
76.63	12.21	42.5	97.5	70	75	85

Table 11. SUS Interpretation Scales.

Adj	Grade	Quartile	Acc.	NPS	Benchmark
Good	B	3rd	Acceptable	Passive	Above Average

To provide more granular insights, the mean and standard deviation for each SUS item were also computed and are presented in Figure 11.

Figure 11. Graph showing the mean values for each SUS question.

Analyzing individual SUS items provides deeper insights into participants’ subjective perceptions of the system. Questions 3, 5, 7, and 9 all positively worded and received high mean ratings (M ≥ 4), suggesting that participants found the system easy to use, well-integrated, quick to learn, and felt confident while using it. These are strong indicators of high usability. In contrast, negatively worded Questions 2, 4, 6, 8 (M ≤ 2) received low average scores, implying that participants generally disagreed with statements indicating complexity, the need for technical support, inconsistency, and awkwardness to use. This further supports the perception of a usable system. Slightly higher scores on the negatively worded Question 10 (M = 2.15)—“I need to learn a lot of things before I could get going with this product”, suggesting that while most users did not perceive the system as inherently difficult to begin using, the cumulative complexity of mastering multiple controls and functionalities within a limited time frame likely contributed to the elevated ratings on this item. For the positively worded Question 1—“I think I would like to use this product frequently”, participants reported a neutral to slightly positive response (M = 3.35, SD = 0.94), suggesting moderate willingness to reuse the simulator.

In addition to the item-level analysis, statistical tests were conducted to examine whether participants’ prior VR experience influenced their usability ratings. A one-way ANOVA revealed no significant differences in total SUS scores across the three VR experience groups, none, basic, and intermediate, F(2, 23) = 0.39, p = 0.68. Consistent with this, a Spearman’s rank-order correlation showed no significant association between VR experience (treated as an ordinal variable) and SUS score, $\rho$ = 0.16, p = 0.45. These findings indicate that prior familiarity with VR did not meaningfully impact participants’ perceptions of the system’s usability. Another Spearman rank-order correlation was conducted to examine the relationship between simulator sickness and perceived usability. The analysis revealed a significant moderate negative correlation, $\rho$ = −0.45, p = 0.021, indicating that participants who experienced higher simulator sickness tended to report lower usability ratings.

9.3. User Experience

UEQ-S responses were analyzed using the UEQ-S Data Analysis Tool, following the official guidelines provided by the questionnaire authors [53]. Figure 12 presents the mean scores for each item, grouped by pragmatic and hedonic quality. According to standard interpretation guidelines, scores above $0.8$ are considered positive, scores below $-0.8$ negative, and scores between $-0.8$ and $0.8$ neutral. As seen in the results, all UEQ-S items for the VR harvester simulator achieved positive evaluations, indicating a generally favorable user experience.

Figure 12. UEQ-S Mean Value Per Item: Individual score results from the UEQ-S.

Additionally, the 95% confidence intervals were computed for each UEQ-S dimension. Notably, participants gave consistently high ratings in categories related to system efficiency, clarity, interestingness, and supportiveness. These findings are further corroborated by qualitative feedback collected from open-ended questions, where several participants described the harvester simulator as ‘fun’ and ‘exciting’.

To contextualize the results, the mean scores for pragmatic quality, hedonic quality, and overall user experience were compared against the standardized UEQ benchmark dataset as show in Figure 13. The simulator scored above average in terms of pragmatic quality, meaning approximately 25% of systems in the benchmark perform better, while 50% perform worse. In the domain of hedonic quality, the simulator ranked in the top 10% of all benchmarked systems, suggesting a highly enjoyable and stimulating user experience. The overall user experience score falls in the ‘Good’ category, where only 10% of systems perform better, and 75% perform worse. These results indicate that the VR harvester simulator offers a well-balanced user experience combining both functional effectiveness and user enjoyment, making it suitable for immersive training and educational use.

Figure 13. User Experience Score for the harvester system compared against the benchmark dataset based on full UEQ scores.

Overall UEQ-S scores differed by prior VR experience. Participants with Basic VR experience reported the highest scores (M = 1.96, SD = 0.72), followed by None (M = 1.55, SD = 0.27) and Intermediate (M = 1.16, SD = 0.55). As seen in Table 12, one-way ANOVA confirmed a significant effect of VR experience, F(2, 23) = 4.53, p = 0.022. Post hoc comparisons showed that the Basic group scored significantly higher than the Intermediate group (mean difference = 0.796, p = 0.017), while differences between Basic vs. None and Intermediate vs. None were not significant. Pearson correlation did not reveal a significant linear association, r = −0.25, p = 0.22. These results indicate that participants with moderate prior VR experience perceived the system most positively, whereas more experienced users rated the user experience lower, possibly reflecting higher expectations.

Table 12. One-way ANOVA Assessing UEQ Scores Across Prior VR Experience with Post Hoc Comparisons.

One Way ANOVA
Source	Sum of Squares	df	Mean Square	F	p
VR Experience	2.536	2	1.268	4.528	0.022
Residuals	6.441	23	0.280	–	–
Post Hoc Comparisons (Tukey)
Comparison	Mean Difference	SE	df	t	${\mathit{p}}_{\mathit{tukey}}$
Basic vs. Intermediate	0.796	0.265	23	3.009	0.017
Basic vs. None	0.404	0.251	23	1.610	0.262
Intermediate vs. None	−0.392	0.251	23	−1.562	0.282

A Spearman rank-order correlation was conducted to examine the relationship between simulator sickness and overall user experience. The analysis revealed a significant moderate negative correlation, $\rho$= −0.47, p = 0.017, indicating that participants who experienced higher simulator sickness tended to report lower overall user experience. Further examining the UEQ subscales revealed that the negative relationship was stronger for hedonic experience ($\rho$ = −0.393, p = 0.047) than for pragmatic, task-focused experience ($\rho$ = −0.234, p = 0.250).

9.4. Simulator Sickness

While most participants reported no symptoms, mild to moderate levels of general discomfort, eye strain, nausea, and a sense of fullness in the head were fairly common (see Figure 14). The scores for each individual symptom are categorized into three non-mutually exclusive categories representing symptoms for nausea (N), oculomotor disturbance (O), and disorientation (D). Table 13 shows the means of SSQ subscale scores and the final total severity score. The highest average was observed in the disorientation subscale (M = 39.08). This observation reflects the D > N > O (39.083 > 21.282 > 20.408) profile commonly associated with cybersickness, where disorientation symptom dominates, followed by nausea and oculomotor. The symptom hierarchy differs from that seen in other forms of motion sickness, such as seasickness or space sickness [93]. The observed total mean SSQ score is 28.91 as reported in Table 13.

Figure 14. Graph illustrating the distribution of participant-reported severity ratings for each SSQ symptom.

Table 13. Mean SSQ scores.

	Nausea	Oculomotor	Disorientation	Total Score
Mean	$21.282$	$20.408$	$39.083$	$28.913$
Std. Deviation	$28.849$	$21.840$	$47.085$	$32.356$
Percentile point	87	78	93	88

A breakdown of SSQ subscale scores by gender, eyewear use, and prior VR experience is presented in Table 14. Female participants reported higher symptom severity across all subscales than male participants, with the largest difference observed in disorientation scores (57.07 vs. 27.84), consistent with the pattern illustrated in Figure 15. Figure 16 shows that participants who wore eyeglasses exhibited comparable subscale scores to those who did not, suggesting minimal impact of eyewear use on simulator sickness symptoms. Interestingly, participants with no prior VR experience reported the lowest nausea score, while intermediate users had slightly lower disorientation scores; oculomotor scores were almost similar across experience levels, suggesting that prior VR exposure had limited influence on overall simulator sickness (see Figure 17).

Table 14. Simulator sickness questionnaire subscale scores.

		Nausea		Oculomotor		Disorientation		Total Score
Group	n	M	SD	M	SD	M	SD	M	SD
Gender
Female	10	32.44	41.02	26.53	27.74	57.07	55.08	41.41	43.61
Male	14	12.95	13.29	16.78	17.38	27.84	40.49	20.84	20.41
Other	2	23.85	33.73	15.16	21.44	27.84	39.37	24.31	34.38
Eyewear
None	17	24.131	34.581	20.065	23.354	39.304	48.028	29.92	36.54
Glasses	9	15.900	12.620	21.056	19.975	38.667	48.109	27.01	24.36
VR Experience
None/Never	10	15.26	16.95	23.50	21.87	47.33	52.99	30.3	29.84
Basic/Rarely	8	26.23	46.38	20.85	30.79	41.76	58.59	31.7	48.2
Intermediate/Frequently	8	23.85	19.08	16.12	10.28	26.10	24.03	24.31	15.35

Figure 15. SSQ subscale scores for gender.

Figure 16. SSQ subscale scores for eyewear.

Figure 17. SSQ subscale scores for VR experience.

To statistically examine group differences, tests for normality were first conducted. A Shapiro–Wilk test revealed that all SSQ subscales and total scores significantly deviated from normality: nausea, W(26) = 0.71, p < 0.001; oculomotor disturbance, W(26) = 0.79, p < 0.001; disorientation, W(26) = 0.78, p < 0.001; and total severity score, W(26) = 0.79, p < 0.001. Given the non-normal distributions, nonparametric tests were employed for subsequent analyses. A Kruskal–Wallis H test examining differences in total SSQ scores across gender groups indicated no statistically significant difference after Bonferroni correction, H(2) = 12.40, p = 0.54. Similarly, the effect of VR experience on total SSQ scores was not statistically significant, H(2) = 0.73, p = 0.69. Finally, a Mann–Whitney U test comparing participants who wore eyewear to those who did not also revealed no significant difference in total scores, U = 71.50, p = 0.81. So, while individual variability existed, particularly in gender and VR familiarity the overall level of simulator sickness was moderate and consistent across participant subgroups.

9.5. Task Performance

Task performance was recorded for all 26 participants across both the training and practice sessions. During the training session, the time spent on each instructional card was recorded in seconds for all steps that the participants completed independently. Steps that were skipped, or could not be completed due to technical issues, or required evaluator assistance were marked as ‘NA’. For the navigation module, there were 7 instruction cards and the average time spent per instruction card was 11.53 s (SD = 3.9 s). The total time to complete the module was, on average, 79.23 s (SD = 25.31 s). The NA rate, which represents the proportion of NA instances relative to the total number of recorded tasks, was encountered in only 1.44% of instances. For the head/boom module, there were 15 instruction cards and the average time spent per instructional card was 11.69 s (SD = 3.81 s). The total time to complete the module was, on average, 170.31 s (SD = 56.59 s). Steps were marked as NA in 1.97% of instances. These metrics are summarized in Table 15 below.

Table 15. Training Session: Average Time per Instructional Card and Total Module Time.

Module	Instruction Cards	Avg Time per Card (s)	SD (s)	Avg Completion Time (s)	SD (s)	NA Rate (%)
Navigation	7	11.53	3.9	79.23	25.31	1.09
Head/Boom	15	11.69	3.81	170.31	56.59	2.05

In the practice session, task performance was evaluated using a success criterion. Each task was categorized as ‘Pass’ if completed independently, ‘Fail’ if the participant required evaluator assistance or exited the task, and ‘NA’ if the task could not be completed due to technical issues. For the navigation practice session, 5 tasks were evaluated. The overall success rate of task completion was 94.61%, the failure rate was 5.38%, and the NA rate was 0%. The participants completed the practice session in an average time of 197.63 s (SD = 70.94 s). The final scores were calculated for 24 participants (excluding 2 participants who quit due to simulator sickness), with an average score of 80.67% (SD = 17.87%). For the head/boom practice session, participants were evaluated on the basis of 10 tasks. The overall success rate was 91.53%, the failure rate was 7.3%, and the NA rate was 1.15%. The participants completed the head/boom practice session in an average time of 299.27 s (SD = 196.79 s). The final scores were calculated for 23 participants (excluding 2 participants who quit due to simulator sickness and 1 due to technical issues), yielding a mean of 82.60% (SD = 9.02%). These metrics are summarized in Table 16 below.

Table 16. Practice Session: Success, Failure, and NA Rates.

Module	Tasks Evaluated	Success Rate (%)	Failure Rate (%)	NA Rate (%)	Avg Completion Time (s)	SD (s)	Avg Final Score (%)	SD (%)
Navigation	5	94.61	5.38	0.00	197.63	70.94	80.67	17.87
Head/Boom	10	91.53	7.30	1.15	297.21	192.72	79.79	16.38

10. Discussion

10.1. Cognitive Load

The first research question examined the level of cognitive load experienced during use of the VR HMD-based harvester training simulator. NASA-TLX results showed higher cognitive load during the harvester training modules compared to the baseline VR task, reflecting fundamental differences in task complexity. The baseline module consisted of a simple tutorial on Meta Quest 2 controls, whereas the harvester modules required complex, multi-step machinery operations. A meta-analysis of 237 NASA-TLX studies reported that 80% of workload scores fall between 26.08 and 68.00 [94]. Although this analysis was not VR-specific, it provides a useful comparative reference. Interpreted against these percentile distributions, the observed workload levels for the VR harvester modules (Navigation: M = 34.26; Head/Boom: M = 38.43) lie in the lower-to-middle range of reported values, consistent with moderate workload. As noted by Grier et al. [94], assessments of acceptability must also consider task context, user expertise, and performance outcomes.

Consistent with Cognitive Load Theory (CLT), the intrinsic complexity of harvester operation tasks produced higher cognitive load relative to baseline. Importantly, this increase did not impair performance, as indicated by the high task success rates (Table 16). High SUS scores further suggest that the system’s design effectively supports task execution. Together, these findings support Hypothesis H1, indicating that although more demanding than a basic VR tutorial, the harvester modules impose cognitive load levels compatible with successful task performance.

Qualitative analysis of participant behavior and feedback identified several contributors to cognitive load. From a CLT perspective, the transition from guided instruction to autonomous control increased intrinsic load, as participants were required to recall multiple control mappings for navigation and head/boom manipulation without external cues. Prior work shows that recall-based interaction is cognitively more demanding than recognition-based interaction, which relies primarily on familiarity judgments [95]. Similarly, AR and VR studies have demonstrated that recall-based interfaces impose higher cognitive demands than recognition-based or visually cued designs [96]. Because participants moved directly from instruction to practice without rest or repetition, working memory demands were elevated before procedural knowledge could consolidate. Consistent with VR skill-acquisition research [97], repeated exposure and distributed practice may reduce this burden by promoting automation; longitudinal measurement of cognitive load would help clarify these dynamics.

Additional cognitive load arose from extraneous spatial–perceptual challenges. Distance underestimation was frequently observed during object placement in the practice session, a well-documented limitation of VR environments [98]. This issue could likely be mitigated through stronger depth cues, such as dynamic shadows, which are currently absent. Enhancing environmental cues may therefore improve spatial accuracy while reducing unnecessary mental effort.

Finally, novelty effects and limited prior VR exposure likely contributed to elevated cognitive load, particularly during practice sessions. Similar effects have been reported in first-time VR users, where initial cognitive demands increase due to concurrent task and interface learning [99]. These effects typically diminish with familiarity and sensorimotor adaptation. Gradual task introduction, scheduled breaks, and adaptive tutorial support may therefore help balance immersion with cognitive manageability in future training designs.

10.2. System Usability and User Experience

The second research question examined perceived usability and overall user experience of the VR HMD-based harvester training system. Both SUS and UEQ scores were rated highly and compared favorably with established benchmarks, reflecting the system’s careful design, including structured instructional sequencing, visual fidelity, and overall robustness. Strong task performance, evidenced by high success rates, low error rates, and efficient completion times, aligns with prior findings that intuitive and engaging systems support more effective task execution. This convergence of subjective experience and objective performance supports Hypothesis H2.

Qualitative feedback further identified design features that enhanced engagement, particularly multimodal feedback such as ambient audio, haptic cues, and adaptive visual guidance, which collectively improved immersion and task confidence. Participants also noted that novelty effects contributed positively to engagement and motivation. Despite these strengths, a recurring usability concern involved the separation of navigation controls, with left/right and forward/backward movements distributed across both controllers. Many users attempted to operate movement with a single controller, reflecting expectations shaped by conventional game control schemes. This finding underscores the need to balance operational fidelity with intuitive interaction design; repeated tutorial exposure may help users internalize less familiar control mappings.

Physiological factors, particularly simulator sickness, also influenced perceived usability and user experience. Participants reporting greater simulator sickness consistently rated the system as less usable, indicating that usability perceptions in immersive VR are closely linked to physical comfort. In high-immersion contexts, discomfort can overshadow interface qualities and limit users’ ability to evaluate functional aspects of the system. Although the association between simulator sickness and overall user experience was not statistically significant, a clear negative trend was observed. Analysis of UEQ-S subscales showed a stronger negative effect on hedonic (enjoyment-oriented) experience than on pragmatic (task-focused) experience, suggesting that simulator sickness primarily diminishes experiential and affective qualities rather than task-related perceptions. These results highlight the importance of explicitly accounting for bodily comfort when designing and evaluating immersive VR systems, even when task performance remains high.

10.3. Simulator Sickness

Given the well-known challenge of simulator sickness, the third research question examined its severity during use of the VR HMD-based harvester simulator. Simulator sickness arises from interacting technological and individual factors; while advances in VR technology may mitigate some technical causes, individual differences remain influential [100]. Although no universal solution exists, understanding underlying mechanisms is essential for developing effective mitigation strategies. A review of SSQ total scores for VR HMDs suggests that values of 40 or higher are more appropriate for classifying a VR experience as a “bad simulator” [9]. The observed scores in this study remained below this threshold, supporting Hypothesis H3.

The elevated disorientation score (M = 39.08) is consistent with participant reports indicating that simulator sickness occurred primarily during the navigation practice session, where users drove the harvester along a wavy path. Such tasks likely induced visual–vestibular mismatches and rotation-related effects known to trigger disorientation through sensory conflict among visual, proprioceptive, and inertial cues [60]. To mitigate disorientation, future simulations should consider simplifying navigation paths, limiting acceleration and rotation, and keeping session durations short. Post-session interviews further indicated that participants with frequent gaming experience, whether in VR or non-VR contexts, reported lower simulator sickness, suggesting a habituation effect.

Two female participants discontinued the session due to simulator sickness. Descriptive analyses showed higher mean SSQ values for female participants across subscales, although no statistically significant gender differences were observed in total SSQ scores. Extreme values from these participants may have inflated group means, and the modest sample size limited statistical power. Nonetheless, this descriptive trend aligns with prior reports of greater simulator sickness susceptibility among females [101]. Gender-related differences may reflect multiple factors, including prior VR experience, field of view (FOV), interpupillary distance (IPD), exposure duration, and general health. Stanney et al. [102] identified IPD non-fit as a primary contributor to gender differences in simulator sickness. The Meta Quest 2 used in this study supports an IPD range of 56–70 mm, reported to accommodate approximately 95% of adults [103]. Whether IPD mismatch contributed to participant withdrawal cannot be determined, as individual IPD measurements were not collected. Broader accommodation ranges (approximately 50–75 mm) have been recommended to better support adult users [104]. Notably, both female participants who experienced severe symptoms were novice VR users, suggesting that limited habituation may have increased susceptibility. A limitation of this study is the absence of pre-task assessment of individual simulator sickness susceptibility.

Eyewear-related discomfort emerged as an additional ergonomic concern. Among participants who normally wear glasses, 40% chose not to wear them during VR exposure due to discomfort, even with an eyeglass spacer, and some reported eye strain and blurred vision. Although no significant SSQ differences were observed between eyewear and non-eyewear users, these observations underscore the need for improved HMD ergonomics. Future designs may benefit from more adaptable eyeglass accommodations and customizable headset sizing to better support individual IPD and eyewear needs [105].

10.4. Task Performance

Task performance across both training and practice sessions indicates that participants were generally able to understand and execute simulator operations effectively. During training, engagement with the instructional cards was consistent, with mean completion times of approximately 11–12 s per card for both the navigation and head/boom modules. This consistency suggests well-calibrated instructional pacing and content load, allowing sufficient time for information processing and action. The low NA rates (1.09% for navigation and 2.05% for head/boom) further indicate clear instructions and minimal technical or usability barriers. As expected, the head/boom training required approximately twice the completion time of the navigation training, reflecting greater operational complexity and a higher number of instructional steps. This difference is consistent with the increased cognitive and motor demands associated with coordinating multiple degrees of freedom and control mappings.

Performance in the practice sessions further supports the simulator’s usability and learnability. Participants achieved high success rates (94.61% for navigation and 91.53% for head/boom), demonstrating effective task execution following guided training. Modest failure rates (5.38% and 7.3%, respectively) likely reflect isolated challenges in control precision or spatial coordination rather than systemic design limitations. The near absence of NA values underscores both technical stability and task clarity.

Differences in practice-session completion times again mirror task complexity. Navigation tasks were completed in approximately 198 s on average, whereas the more demanding head/boom tasks required nearly 300 s. Greater variability in head/boom completion times (SD = 192.72 s) suggests individual differences in spatial motor learning, with some participants adopting more cautious control strategies.

Final performance scores (80.67% for navigation and 79.79% for head/boom) indicate satisfactory proficiency after a single training exposure. The relatively narrow standard deviation in the head/boom module suggests convergence toward consistent performance once control mappings were understood. Instances of participant withdrawal due to simulator sickness further emphasize the importance of comfort and acclimatization considerations in future training designs.

10.5. Participant Population and Generalizability

This study evaluated early-stage feasibility using a student cohort, aligned with the intended near-term deployment in student training contexts. However, the results should not be interpreted as representative of professional harvester operators. Operators may show lower workload and faster performance due to domain schemas and prior machine-control experience, but they may also apply stricter expectations regarding control fidelity and realism. Findings most likely to generalize across populations include (i) comfort constraints and sickness triggers during continuous navigation/rotation, (ii) interaction-design issues related to controller mapping and feedback, and (iii) the need for stronger depth cues during precision manipulation. In contrast, absolute NASA-TLX levels, completion times, and hedonic UX scores may vary substantially with domain expertise and habituation.

11. Conclusions

This study evaluated an affordable, HMD-based harvester training simulator against three questions concerning cognitive load, perceived usability, user experience, and simulator sickness. Across 26 participants, workload increased from a baseline tutorial to each training module (NASA-TLX: $18.65\to 34.26\to 38.43$; repeated-measures ANOVA, p < 0.001), while perceived usability was ‘Good’ (SUS = 76.63) with strong hedonic experience (UEQ-S, top-decile benchmark). Simulator sickness was non-trivial (SSQ = 28.91, D > N > O), with two withdrawals, yet most users completed practice sessions with high success (≈95% Navigation; ≈91% Head/Boom). Taken together, these findings indicate early-stage feasibility for student-focused training on consumer HMDs, alongside clear comfort constraints that must be engineered down before longer or more complex modules are deployed.

Interpreted against our hypotheses, the results support H1 (task-appropriate instructional scaffolding can keep workload in a manageable range), also support H2 (higher usability and user experience are associated with stronger task performance), and support H3 as well (SSQ score did not go beyond 40 that would categorize the system as a ‘bad simulator’).

For VR HMD-based heavy machinery training, our data suggest the following:

• Reduce extraneous load: provide recognition-aiding, on-controller prompts that fade as skill consolidates; pace tutorials and allow brief repetition before practice.
• Design for comfort: limit angular acceleration and rotation, offer optional FOV vignetting/rest-frames, schedule short bouts with planned breaks, and calibrate fit/IPD, especially for users who wear glasses.
• Improve spatial cues: add shadows/occlusion and other depth cues to support precise placement and reduce unnecessary effort in manipulation tasks.

Based on these findings, our next steps are (1) implement comfort-focused navigation controls (e.g., reduced angular acceleration, optional comfort vignette/rest frames, and scheduled micro-breaks), (2) improve precision manipulation by adding depth cues (e.g., shadows/occlusion) and refining object feedback, (3) test alternative control mappings and (when feasible) integrate low-cost physical control surrogates to increase fidelity, (4) replicate this user-centered evaluation with forestry students and trainee operators, and (5) conduct longitudinal studies measuring retention and, where possible, transfer to real or high-fidelity simulator tasks. Additionally, targeted strategies for reducing simulator sickness, whether through design, calibration, or adaptive interfaces, must be explored to support longer training durations.

Overall, the findings support system feasibility and learnability in a single-session exposure, showing that participants can complete the guided modules and associated practice tasks while reporting moderate workload and acceptable usability/UX. Ultimately, this research contributes to a growing body of evidence that immersive technologies can democratize vocational training, making high-risk, high-cost skill development more equitable, engaging, and widely accessible. This study does not evaluate learning outcomes, retention, or real-machine skill transfer; these constitute the focus of planned longitudinal evaluations with domain learners and trainee operators and are planned as next steps. Current results provide evidence of good system learnability, stable performance across participants, and task progression that appropriately balances complexity and cognitive demand. Future iterations will build on these insights by integrating adaptive pacing or progressive difficulty to personalize training experiences further and support long-term skill retention. Specifically, future work will build on these findings by conducting evaluations within established curricular programs using formal, summative assessment protocols already adopted in training curricula. The evaluation will be extended to domain-specific learners, including forestry students and trainee operators, and will be carried out in a formal training setting with controlled comparisons between desktop-based VR systems and VR HMD-based implementations.