Usability and User Experience in an Industrial Metaverse: A Mixed-Methods Study of the Necoverse Point Cloud Inspection System for Shipbuilding

Abstract

Industrial metaverse systems enable shared, immersive environments for coordinating complex, data-intensive industrial workflows; however, ensuring effective and usable interaction remains a key barrier to professional adoption. This study examines immersive point cloud- and CAD-based inspection tasks in an industrial metaverse context using a mixed-methods evaluation that combines perceived usability ratings, cognitive workload assessment (NASA-TLX), validated presence and flow instruments, qualitative interviews, and structured observation. The results indicate that users generally experienced smooth navigation, manageable cognitive workload, and a meaningful sense of spatial presence, supporting focused and task-oriented engagement. At the same time, execution-level challenges—particularly related to tool discoverability, annotation flexibility, system feedback clarity, and interaction ergonomics—introduced workflow friction for some users. By triangulating quantitative, qualitative, and observational evidence, the study derives actionable design recommendations, including adaptive onboarding, improved feedback mechanisms, and refinements to interaction design. Overall, the findings provide empirical insight into how usability, cognitive workload, presence, and flow jointly shape user experience in industrial metaverse inspection environments and inform the development of more robust, user-centered industrial systems.

1. Introduction

In recent years, the metaverse—understood as a network of interconnected immersive virtual environments (IVEs)—has expanded beyond entertainment into professional training and collaborative work. The effectiveness of such environments depends on their ability to foster presence, which emerges from usability, intuitive interaction, visual clarity, and responsive system behavior [1]. Augmented and virtual reality (AR, VR) technologies underpin these immersive environments, with VR in particular enabling users to perform realistic tasks in safe, controlled settings [2]. Prior work in education and training demonstrates that immersive environments can enhance engagement, learning outcomes, and collaboration through social interaction and embodied digital avatars [3,4,5,6]. In this study, Necoverse—Point Cloud Visualization & Inspection System is an industrial metaverse platform developed to support immersive, collaborative workflows such as digital twin–based inspection, design review, and training in industrial contexts. The Necoverse system serves as the evaluation platform used throughout this paper. Also, the term Necoverse system is used consistently throughout this paper.

This shift is increasingly visible in industrial contexts. The industrial metaverse integrates immersive communication, hands-on interaction, and digital twin technologies into shared virtual environments that support complex, real-world workflows and multi-user collaboration [2]. Industries such as construction, shipbuilding, and energy rely on these environments to coordinate distributed teams and integrate sensor-based data with high-fidelity digital representations of physical assets [7]. By combining digital twins, VR, and point cloud data, industrial metaverse systems enable engineering-grade analysis, simulation, and inspection tightly coupled to physical reality [8,9,10]. Empirical evidence suggests that such platforms can reduce errors and streamline workflows such as quality assurance and design validation, while also supporting operator training and safety-critical learning scenarios [11,12,13,14,15].

In this paper, the etaverse is defined as an integrated technological paradigm emerging from the convergence of artificial intelligence, spatial computing, and extended reality—a triadic synthesis herein denoted as AI–SC–XR—reflecting prevailing definitions in contemporary metaverse research. Extending this general conceptualization to industrial contexts, the industrial metaverse is theorized as a value-generating sociotechnical system wherein data originating from physical industrial environments are continuously captured, integrated, analyzed, and operationalized to optimize processes and enhance overall production effectiveness.

Despite this potential, industrial metaverse systems continue to struggle with the specific demands of professional use. Unlike consumer-oriented extended reality (XR) applications—typically designed for entertainment or general-purpose interaction—industrial contexts require precise multi-user collaboration, high-fidelity 3D visualization, and task-specific workflows [16]. Usability principles derived from entertainment or general-purpose XR often fail to transfer effectively, leaving gaps in our understanding of user experience in industrial metaverse environments [17]. In these settings, cognitive load becomes a critical factor influencing performance and safety [18,19], while the integration of digital twins further necessitates evaluation approaches tailored to industrial use cases [20].

Addressing these challenges requires empirical research that informs design guidelines for industrial metaverse systems that are not only technologically capable but also usable, cognitively manageable, and engaging [21,22]. Presence and flow represent key experiential dimensions of such immersive systems, while cognitive workload captures the mental effort required to interact effectively with complex environments. Together, these constructs provide a holistic framework for evaluating usability and user experience in industrial metaverse applications and form the basis of the evaluation model adopted in this study.

In high-stakes industrial scenarios, interaction design directly affects operational efficiency and safety [22]. Industrial metaverse environments place heightened demands on interface clarity, system feedback, and interaction reliability due to their support for shared, data-intensive workflows and distributed inspection activities [23]. Core interaction elements such as navigation, controller input, voice-based interaction, and annotation tools therefore play a decisive role in shaping usability, cognitive workload, and user engagement [24]. Given the complexity of industrial inspection workflows, systematic evaluation is essential for understanding how interaction design influences user performance and experience, and for supporting the development of industrial metaverse systems that can sustain efficiency, safety, and long-term adoption in real-world operational contexts [25,26].

While the study is grounded in a specific industrial metaverse system, the aim is to derive generalizable insights into usability, cognitive workload, presence, and flow that are applicable across industrial metaverse inspection contexts. This study makes the following contributions. First, it provides an integrated evaluation of usability, cognitive workload, presence, and flow within an industrial metaverse inspection workflow, moving beyond isolated assessments of individual user experience constructs. Second, it demonstrates the value of a mixed-methods, triangulated evaluation approach that combines validated quantitative instruments with qualitative interviews and structured observation to reveal execution-level interaction breakdowns that may not be captured through questionnaires alone. Third, by grounding the evaluation in point cloud– and CAD-based industrial inspection tasks, the study derives generalizable, design-relevant insights that extend beyond the specific system under study and inform the development of more usable and cognitively efficient industrial metaverse environments.

The remainder of this paper is structured as follows. Section 2 reviews related work on industrial metaverse systems and relevant user experience constructs. Section 3 introduces the Necoverse system. Section 4 describes the study methodology, followed by the results in Section 5. Section 6 discusses the findings and design implications, and Section 7 concludes the paper.

2. Literature Review

2.1. Industrial Metaverse

The convergence of AR and VR technologies has enabled the emergence of the industrial metaverse—a shared, immersive, and persistent digital space integrating physical and virtual systems. Recent research defines the industrial metaverse as a physics-based environment in which humans, machines, and data interact to enhance physical-world processes [27,28]. Often described as a parallel reality supported by digital identities, these environments generate new opportunities across industrial sectors such as manufacturing, automotive, and healthcare [29,30].

Digital twins constitute a core component of the industrial metaverse, enabling real-time simulation and monitoring of physical assets, as demonstrated in industrial deployments by companies including Siemens (Siemens AG, Germany) and BMW (BMW AG, Germany) [31,32]. Collaborative platforms such as NVIDIA Omniverse (NVIDIA Corporation, Santa Clara, CA, USA) exemplify this paradigm by supporting shared industrial design, simulation, and robotics workflows [32,33], while enterprise-focused solutions such as ProVerse (ProVerse Interactive Oy, Finland) provide immersive collaboration environments tailored to industrial needs [34].

Together, these developments position the industrial metaverse as a strategic driver of industrial efficiency, innovation, and data-centric collaboration [35,36,37]. However, in complex and specialized domains such as shipbuilding inspection, adoption remains constrained by unresolved human-factor challenges related to usability, cognitive workload, and collaborative workflow support, motivating the need for empirical, user-centered evaluation.

2.2. Usability

Usability, a foundational concept in Human–Computer Interaction (HCI), refers to the extent to which users can achieve specified goals effectively, efficiently, and satisfactorily within a system [38]. Classical usability principles—such as learnability, efficiency, and error prevention—have informed established evaluation methods and heuristic frameworks, including standardized instruments like the System Usability Scale (SUS) and structured approaches such as the User-Centered Design and Usability Engineering Lifecycle [39,40,41,42].

In industrial metaverse systems, these principles remain relevant but must be interpreted in the context of immersive interaction. Such environments introduce interaction characteristics—including spatial navigation, embodied interaction, and real-time multimodal feedback—that extend beyond assumptions underlying traditional two-dimensional interfaces [43]. In high-stakes industrial contexts, these factors are particularly consequential, as usability directly influences operational efficiency, safety, and error prevention [44,45].

Accordingly, recent research emphasizes not the replacement of established usability methods, but their context-sensitive application to industrial metaverse environments [46]. Effective usability evaluation must account for domain-specific conditions such as interaction with large-scale 3D data, integration with digital twins, and cognitively demanding inspection workflows. As a result, usability assessment in industrial metaverse systems increasingly relies on mixed-methods approaches that combine validated quantitative instruments with qualitative and observational insights to capture experiential, cognitive, and contextual dimensions of use [45,46]. Understanding how conventional usability constructs manifest in immersive industrial environments therefore remains a central research challenge.

2.3. Cognition and Cognitive Workload in Industrial Metaverse Systems

Immersive technologies underpinning industrial metaverse systems place distinct demands on human cognition by altering users’ perception of reality and engaging fundamental cognitive processes such as attention, memory, and decision-making [47,48]. In immersive industrial environments, attentional regulation directly influences task focus and immersion, while continuous and information-rich sensory input shapes perception and emotional responses [47,48]. At the same time, users rely heavily on memory to navigate complex virtual spaces, interpret spatial information, and make frequent operational decisions, underscoring the importance of interface designs that support—rather than hinder—core cognitive processes [49].

When the cumulative demands of immersive environments exceed users’ available mental resources, cognitive load increases and becomes a primary bottleneck to effective performance. Elevated cognitive load has been shown to impair decision-making, increase error likelihood, and accelerate mental fatigue, particularly in complex or safety-critical tasks [50]. From a design perspective, this highlights the role of cognitive affordances—interface features that actively support users’ reasoning and attention—in aligning industrial metaverse systems with human cognitive capabilities and mitigating overload [51]. Consequently, evaluating cognitive workload requires not only measuring perceived mental effort but also understanding its impact on task performance and immersive experience.

Foundational contributions from NASA to cognitive engineering provide well-established tools for assessing workload in high-stakes environments. The NASA Task Load Index (NASA-TLX) is a widely adopted and robust method for measuring subjective cognitive workload across multiple dimensions [52]. When combined with frameworks addressing situational awareness, these approaches offer a validated basis for evaluating systems that must preserve performance and safety under high cognitive demand [53]. Their relevance extends beyond aerospace, with demonstrated applicability in immersive training, human–robot collaboration, and other high-demand domains where controlled cognitive load and shared situational awareness are critical [52,53].

These principles are directly applicable to industrial metaverse systems, which are characterized by immersive, information-dense environments, continuous interaction, and time-critical collaborative workflows. Poorly structured or excessive cognitive demands can degrade attention, reduce task effectiveness, and increase the risk of operational errors, whereas interaction designs that minimize unnecessary workload support sustained engagement, accurate decision-making, and reliable performance [54]. Accordingly, prioritizing users’ cognitive state is a foundational requirement for the safe and effective deployment of industrial metaverse applications. The established validity of the NASA-TLX therefore provides a strong methodological justification for its use in this study to evaluate the cognitive demands associated with collaborative point cloud inspection tasks.

2.4. Presence Theory in Industrial Metaverse Systems

Presence, a core phenomenon of human cognition, refers to the psychological acceptance of a virtual environment as a place of experience, commonly described as the feeling of “being there.” Foundational frameworks distinguish multiple dimensions of presence, including spatial presence (the sensation of existing within a virtual environment), social presence (the perception of others as being present), and self-presence (the experience of inhabiting a virtual body), each contributing to immersive experience in distinct ways [55,56]. Early work established that presence emerges from the interaction between a system’s objective immersive properties and users’ subjective psychological responses [57], a model later extended to include social and narrative dimensions such as copresence and plausibility illusion—the perceived believability of events within the virtual environment [56].

Subsequent research emphasizes that presence is not determined by technological fidelity alone. While visual quality and responsive tracking remain important, factors such as narrative coherence, user intention, and personal motivation play an equally significant role [58]. Empirical and meta-analytical findings further indicate that congruence between users’ expectations and environmental design strengthens multiple dimensions of presence, often as strongly as system-level specifications [55,56,58].

In industrial metaverse systems, presence extends beyond individual immersion to directly influence teamwork, communication, and task coordination. Embodied Social Presence Theory (ESPT) suggests that avatars and digital proxies simulate physical copresence, supporting authentic social interaction in shared virtual environments [59]. Prior studies show that such embodiment not only enhances individual immersion but also fosters trust, cooperation, and improved collective performance during collaborative tasks [60]. Accordingly, the convergence of spatial, social, and self-presence represents a critical enabler for effective collaboration and decision-making in immersive industrial contexts. In collaborative inspection workflows, fostering a strong sense of shared presence is therefore hypothesized to support effective teamwork and communication.

2.5. Flow Theory in Industrial Metaverse Systems

Flow, a psychological state defined by Csikszentmihalyi [61], is characterized by deep concentration, intrinsic enjoyment, and a merging of action and awareness during challenging activities. Industrial metaverse systems provide favorable conditions for flow due to their immersive, interactive, and task-oriented nature. In immersive environments, flow has been described as a state of intense task focus accompanied by reduced awareness of external distractions and altered time perception, resulting in sustained absorption in the activity at hand [62,63].

Extensive research demonstrates that flow positively influences user motivation, satisfaction, and sustained engagement. In virtual reality games and simulations, strong relationships have been observed between flow, user enjoyment, and prolonged interaction durations [64]. These findings are particularly relevant for training and skill-development contexts, where flow has been associated with improved learning outcomes and stronger skill retention. Critically, the emergence of flow depends on a balance between task challenge and user skill, supported by clear goals and immediate, meaningful feedback [62,64].

In industrial metaverse systems, flow acts as a key mediator of task performance and long-term system adoption. Design elements such as intuitive interaction techniques, responsive feedback mechanisms, and coherent shared virtual spaces have been shown to foster stronger flow experiences [65,66]. Beyond enhancing intrinsic engagement, flow also shapes users’ perceptions of system usefulness, reinforcing continued use and acceptance [66]. In industrial inspection and collaborative operational tasks, inducing flow can support sustained focus, improved accuracy, and more resilient workflows. Accordingly, flow represents a central experiential metric for evaluating user experience in industrial metaverse applications.

2.6. Research Gaps

Advances in immersive technologies have enabled industrial metaverse systems that support digital twins, immersive inspection, and collaborative problem-solving. Despite this progress, a critical research gap remains at the intersection of usability, cognition, presence, and flow in industrial metaverse environments. While the importance of these constructs is well established, prior work has largely examined them in isolation—focusing separately on usability [44], cognitive workload [49], or experiential factors such as presence and engagement [60]. As a result, an integrated understanding of how these factors jointly shape user experience and performance in complex, collaborative industrial workflows remains limited.

This gap is compounded by the predominance of consumer-oriented or generic immersive contexts in existing research, which inadequately reflect the requirements of specialized industrial environments. Challenges central to professional use—such as multimodal interaction for precise error detection, collaboration-supporting communication mechanisms, and the integration of data-intensive representations like point clouds—remain underrepresented. Moreover, empirical findings are seldom translated into validated, actionable design guidance capable of improving operational trust, accuracy, and efficiency in industrial metaverse workflows.

As industrial metaverse systems become increasingly embedded in real-world practice, systematic, human-centered research that explicitly examines the interdependencies between usability, cognitive workload, presence, and user engagement is urgently needed. To address this need, the present study is guided by research questions aimed at deriving integrated, design-relevant insights into user experience and interaction quality in industrial metaverse systems.

2.7. Research Questions

To address the identified research gaps, this mixed-methods study investigates user experience through a structured usability evaluation of an industrial metaverse system called Necoverse for shipbuilding inspection, focusing on point cloud and CAD model visualization tasks. The following research questions guide the quantitative and qualitative evaluation:

• RQ1: What is the perceived usability of the Necoverse system for point cloud inspection?
• RQ2: What level of cognitive workload do users experience when using the Necoverse system?
• RQ3: To what extent do users experience a sense of presence within the industrial metaverse environment?
• RQ4: To what extent do users experience flow while using the industrial metaverse system?
• RQ5: What relationships exist among perceived usability, cognitive workload, presence, and flow in the industrial metaverse system?
• RQ6: What usability challenges and opportunities emerge from user interactions and qualitative feedback during system use?
• RQ7: What actionable, empirically grounded design recommendations can be derived to improve usability and user experience in industrial metaverse systems?

By integrating quantitative measures with qualitative analysis, the study aims to derive evidence-based design guidance for usability and user experience in industrial metaverse inspection systems.

3. Necoverse—Point Cloud Visualization & Inspection System

The Necoverse system is an industrial metaverse application developed within the Business Finland–funded Necoverse system project, part of the broader Necoleap ecosystem initiated by Meyer Turku [67,68]. Built on an industrial metaverse platform developed at Turku University of Applied Sciences and commercialized by ProVerse Interactive Oy [34], the system supports climate-neutral cruise ship design and collaborative industrial inspection. Within the Necoverse system environment, users can directly compare as-built point cloud scans with as-designed CAD models in a shared virtual space—an essential capability in shipbuilding and construction, where even minor deviations can lead to costly late-stage corrections.

At a technical level, the system integrates three core components. First, LiDAR scanners generate dense three-dimensional point cloud datasets consisting of Cartesian coordinates and associated attributes. Surface luminosity serves as the baseline attribute, while optional color information—implemented in one software version—enhances visual interpretation. Although surface normals are not currently integrated, the existing data support accurate digital representations of physical environments within the industrial metaverse.

Second, real-time interaction with large-scale point cloud data is enabled through computational optimizations implemented in the Unity game engine (version 2022.3.7f1 (LTS)). Techniques such as spatial partitioning, level-of-detail management, and custom point cloud shaders maintain visual clarity and stable performance during professional use. Camera-facing shader logic dynamically adjusts point size and appearance based on viewing distance and angle, ensuring consistent readability in dense scenes.

Third, the user interface supports multiple visualization modes to accommodate different inspection needs. Point cloud data can be rendered in a grayscale mode optimized for structural analysis, with an optional glow overlay to aid spatial orientation and layer verification. A fully colored point cloud mode is available in an alternative version when photorealistic interpretation or material context is required.

A defining feature of the system is the integration of point cloud data with CAD models. Overlay visualization enables real-time identification of misalignments, missing components, and deviations from design tolerances. However, CAD data preparation currently requires manual preprocessing, including file conversion and structural reorganization, and alignment between point clouds and CAD models relies on manual registration based on user expertise—an important practical constraint in large-scale industrial workflows.

Beyond visualization, the system functions as a collaborative inspection workspace. Multi-user sessions are supported in both virtual reality and desktop modes, allowing geographically distributed teams to collaborate in real time. Integrated voice communication and speech-to-text functionality support documentation during inspection activities. A virtual smartphone interface enables users to manage layers, capture geo-referenced snapshots, and create spatially anchored annotations.

Through these capabilities, the Necoverse system supports a transition from isolated, manual inspection practices to connected, scalable, and high-precision industrial workflows. By enabling side-by-side comparison of as-built and as-designed conditions within an industrial metaverse environment, the system facilitates early issue detection, shared situational understanding, and more efficient inspection processes. Figure 1, Figure 2, Figure 3 and Figure 4 present example views of the system in use.

The combination of spatial navigation within large-scale point cloud data, multimodal interaction (controller input, voice, virtual smartphone), overlay-based comparison, and collaborative communication creates a complex interaction environment that places demands on usability, cognitive resources, spatial awareness, and sustained task engagement. These characteristics motivate the evaluation framework adopted in this study, which assesses perceived usability, cognitive workload (NASA-TLX), presence, and flow as complementary dimensions of user experience during industrial metaverse inspection tasks. Figure 1, Figure 2, Figure 3 and Figure 4 illustrate the Necoverse system workflow, including a user joining a collaborative session, inspecting construction progress, and examining the building model. The solid components shown from the interior perspective in Figure 2 represent the detailed structural elements, while the blue wireframe indicates the overall building structure. The system also supports collaborative interaction, inspection, and photo capture during the review process.

4. Method

4.1. Design and Procedures

A structured usability study was conducted to evaluate the Necoverse system in terms of usability, cognitive workload, presence, and flow during point cloud–based inspection tasks involving CAD model in fbx-format, that is commonly used file format for 3D models in Unity game engine, comparison within a shared industrial metaverse environment. Thirty participants took part in the study. They were primarily engineering students with prior exposure to virtual reality technologies but no experience with industrial metaverse systems or professional inspection workflows. Participants were between 20 and 27 years of age, and all were from Finland. Participants were recruited through university advertisements in accordance with institutional ethical guidelines. Participation was voluntary, informed consent was obtained, and participants could withdraw at any time.

In this study, each session followed a three-stage protocol—tutorial, inspection task, and post-session questionnaires—and lasted approximately 45 to 60 min. In the tutorial phase, participants were introduced to the Necoverse system and its hardware, including a Meta Quest 3, produced by VR Meta Platforms Technologies, LLC, CA, USA headset and hand controllers. Core interaction techniques were demonstrated, including smooth locomotion and teleportation via analog sticks, direct object manipulation, and use of a smartphone-style interface that served as the primary interaction tool for switching point cloud and CAD layers.

During the inspection phase, participants completed a standardized engineering scenario designed to replicate a construction inspection task. They created a virtual meeting using speech-to-text (STT) and retrieved the smartphone interface from a dispenser to manage overlays, switch layers, and record annotations. Navigation within the Necoverse system was supported by both smooth locomotion and teleportation, with the inspection model placed in an open virtual space.

Participants compared point cloud data with CAD models to identify mismatches and documented detected issues by capturing virtual images and dictating annotations via STT. All records were geo-referenced and automatically archived in the session log. As text input was not available in the VR environment, voice input served as the primary documentation method. Upon completing the task, participants exited the session. The full list of user tasks is summarized in

Table 1. Point Cloud User Tasks in Evaluation.

Point Cloud Tasks	Description
Set up the meeting	Using the meeting creation interface to name and create the meeting.
Create the meeting and enter	After creating the meeting, you can wait for others to join or start the meeting (enter).
Grab the mobile controller/interface	Collect the virtual smartphone from the dispenser located beside the meeting panel by grabbing it with the virtual hand.
Use the mobile phone/ Select the model/Press the buttons	The user can use the phone interface by pressing buttons on its display with the index finger of their free hand.
Walk inside model (building)	The user navigates the virtual environment either by “walking,” which involves smooth movement using the left controller’s analog stick, or by “teleporting.” In the teleportation method, pushing the right analog stick forward displays an indicator extending from the user’s right hand. Releasing the stick instantly moves the user to the indicated position. Since the only collision detection is with the floor, users can freely move throughout the environment. The building model itself is positioned in an open space adjacent to the starting area.
Find the error	This represents a hypothetical scenario in which an architect employs the tool to assess the construction state by navigating the environment, recording notes from multiple locations, and comparing point cloud scans against CAD models. Such an example was offered to participants who appeared confused about the task. Yet, most participants intuitively carried out similar inspection activities on their own.
Take picture	Capture a virtual photograph of the inspection area or identified issue within the XR environment using the virtual smartphone interface. The image is automatically saved and can be referenced in annotations and meeting records.
Communicate	All interactions use speech-to-text (STT) input, including in VR, with text-to-speech (TTS) used to render spoken annotations as audio.
Go back to the meeting	After everything else users were prompted to return back to the starting area to end the meeting. Most people also ended up returning the phone back to the “dispenser” unprompted; this was also completely unnecessary as the phone just disappears when the meeting is ended.

.

The inspection task was designed to reflect core structural elements of real industrial inspection workflows, including navigation within large-scale spatial data, comparison of as-built point cloud scans with as-designed CAD models, identification of deviations, and documentation of findings through geo-referenced images and annotations. To accommodate laboratory constraints and first-time users, the scenario was intentionally simplified and did not require domain-specific engineering judgment. However, the task preserved the fundamental perceptual, cognitive, and interaction demands characteristic of industrial inspection activities.

The inspection task was designed as an exploratory usability and user experience evaluation rather than a performance benchmarking exercise. As such, the study did not define a fixed set of correct deviations, time constraints, or performance targets. This design choice allowed participants to focus on interaction, navigation, and documentation workflows without introducing artificial correctness criteria that could confound usability and workload assessment.

The study was conducted in a controlled laboratory environment to ensure consistency and minimize confounding variables. The VR headset was connected to a dedicated computer running the Necoverse system. Pre- and post-task questionnaires were administered via Google Forms on separate laptops, and an iPad was used by researchers for real-time observational note-taking. Access to the laboratory was restricted to authorized researchers throughout the study.

All procedures complied with institutional ethical guidelines. Participants provided informed consent prior to participation, all data were anonymized, and participants could withdraw at any time without penalty. Figure 5 shows a usability evaluation session with a participant and a researcher.

Table 2. Experimental Procedure for Necoverse System.

Phase	Tasks	Duration	Research Roles
Pre-Test	Introduction and a quick tutorial on the Necoverse system Pre-test questionnaire.	10–15 min	One researcher introduced and guided the participant. One researcher asked the pre-test questionnaire.
In-Test	Task execution with the minimal guidance from the researcher.	20 min	One researcher noted down and observed the interaction experiences and usability challenges encountered by the participant.
Post-Test	Post-test questionnaires ○ Usability ○ NASA-TLX ○ Presence and Flow Post-test interview	20 min	One researcher conducted the post-test section and asked the post-test questionnaire and interview questions.

shows the step-by-step experimental procedure for the Necoverse system.

4.2. Measures

A suite of standardized psychometric instruments was administered post-task to quantitatively assess user experience in the Necoverse system. Perceived usability was assessed using the Single Ease Question (SEQ) [69], a validated single-item ease-of-use measure presented on a 7-point Likert scale ranging from “Extremely Difficult” to “Extremely Easy,” capturing participants’ overall impressions of system clarity and operational efficiency immediately after task completion. The SEQ has been shown to provide reliable and sensitive assessments of perceived ease of use and is widely applied in usability testing, particularly when multiple instruments are administered or when minimizing respondent burden is important [69]. In the present study, the SEQ was selected to capture users’ overall perception of task ease while avoiding respondent fatigue given the multi-instrument evaluation design. Cognitive workload was assessed using the NASA-TLX, which measures six dimensions—mental demand, physical demand, temporal demand, effort, frustration, and performance—on 0–100 scales, where higher scores indicate greater demand. The Performance subscale uses an inverted scale (0 = perfect performance, 100 = failure), such that lower scores reflect better perceived task success [70,71]. Presence was evaluated using items derived from the presence questionnaire proposed by Usoh et al. [72], with minimal contextual adaptation limited to referencing the evaluated Necoverse system. The items targeted spatial presence, environmental realism, and bodily awareness and were used for descriptive assessment rather than as a newly validated scale. The presence items were analyzed descriptively rather than as a psychometrically revalidated scale, consistent with prior exploratory XR usability research. We used a 7-point Likert scale (e.g., 1 = not at all; 7 = very much). Flow was measured using the Flow Short Scale (FSS) [73], assessing absorption, fluency, and perceived control during goal-directed task execution.

In addition to self-report instruments, task execution was monitored through structured observation during all sessions. Researchers recorded whether participants were able to complete the inspection workflow independently, locate and operate core interaction tools, and successfully produce inspection artifacts such as geo-referenced images and voice-based annotations. Observational notes also captured instances of assistance requests, interaction breakdowns, and navigation difficulties. These behavioral indicators were used to contextualize subjective usability and workload ratings but were not intended as quantitative performance metrics. In addition to self-reported measures, structured observation was used to assess task understanding, interface element discoverability, and control understanding using a 7-point Likert-type rating scale (1 = strongly disagree, 7 = strongly agree).

5. Results

A comprehensive usability evaluation was conducted to assess user interaction with the Necoverse system during point cloud–to–CAD model inspection tasks. Quantitative data were collected using standardized usability rating scales to characterize the overall user experience.

5.1. Usability Results

The usability evaluation yielded a mean rating of (M = 5.13; SD = 1.07) on a 7-point scale, positioning the Necoverse system between “Somewhat Easy” and “Easy.” Of the 30 participants, 14 rated the system as “Very Easy” and 10 as “Somewhat Easy,” while three reported a neutral experience, two rated it “Somewhat Difficult,” and one rated it “Difficult.” No participant selected “Extremely Difficult.” Overall, 24 of 30 participants (80%) evaluated the system as either “Somewhat Easy” or “Very Easy,” indicating generally high perceived usability. Although the majority of responses were positive, the presence of a small number of neutral or difficult ratings suggests opportunities for targeted refinements to improve inclusivity and robustness. Taken together, these results indicate that the Necoverse system demonstrates strong usability for realistic industrial inspection tasks.

5.2. Presence Results

As shown in

Table 3. Descriptive statistics for presence questionnaire items.

No	Presence Items	Mean	SD
1	I experienced a sense of ‘being there’ in the Necoverse System, as though it were a real place.	4.70	1.30
2	During the experience, there were instances when the Necoverse System was perceived as real.	3.00	1.60
3	In retrospect, I perceive the Necoverse System as a place I visited, rather than a collection of visual images.	4.50	1.50
4	While immersed in the experience, I was more aware of being in the Necoverse System than in the physical world around me.	5.30	1.20
5	My recollection of the Necoverse System is comparable to memories of real-world locations, particularly in aspects such as vividness, color, spatial dimensions, and structural detail.	3.70	1.90
6	Throughout the experience, I frequently had the sensation of physically standing within the Necoverse System.	4.70	1.60

, participants reported a moderately high overall sense of presence in the Necoverse system (M = 4.32; SD = 0.98), indicating a generally meaningful immersive experience with variability across presence dimensions.

Spatial and bodily presence were rated positively. Participants reported a strong sense of “being there” in the industrial metaverse environment (M = 4.70; SD = 1.30) and a moderate sensation of physically standing within the virtual space (M = 4.70; SD = 1.60). Attentional engagement was particularly pronounced, with participants reporting greater awareness of the virtual environment than of the physical world during immersion (M = 5.30; SD = 1.20).

In contrast, perceptual realism and memory-related aspects of presence were rated more moderately. The environment was only intermittently perceived as real during the experience (M = 3.00; SD = 1.60), and recollections of the virtual environment were less consistently comparable to memories of real-world locations (M = 3.70; SD = 1.90), despite participants generally perceiving the environment as a place they had visited (M = 4.50; SD = 1.50).

Overall, these results indicate that the Necoverse system effectively fostered attentional, spatial, and bodily presence, while perceptual realism and memory vividness remained comparatively weaker. This pattern suggests that functional immersion was achieved despite limitations in perceptual fidelity, highlighting opportunities for enhancing realism and the durability of spatial memories. Figure 6 illustrates the mean presence scores reported by participants while using the Necoverse system.

5.3. Flow Results

5.3.1. Flow Items

The flow questionnaire assessed participants’ engagement during interaction with the Necoverse system across fluency, absorption, and worry dimensions. As shown in

Table 4. Descriptive Statistics for Flow Questionnaire Items.

Flow Items	Mean	SD
1. I feel just the right amount of challenge.	4.80	1.50
2. My thoughts/activities run fluidly and smoothly.	5.70	0.90
3. I don’t notice time passing.	5.10	1.60
4. I have no difficulty concentrating.	5.80	1.20
5. My mind is completely clear.	5.70	1.10
6. I am totally absorbed in what I am doing.	5.70	1.10
7. The right thoughts/movements occur of their own accord (They happen effortlessly when in flow or alignment).	5.30	1.20
8. I know what I have to do each step of the way.	5.20	1.50
9. I feel that I have everything under control.	5.50	1.30
10. I am completely lost in thought.	3.20	1.40

, the overall flow score indicated a moderately high level of engagement during the inspection task (M = 5.20; SD = 0.70), suggesting a sustained and functional flow experience. Subscale-specific results are reported below. Figure 7 presents the mean scores of the flow-related items reported by participants while interacting with the Necoverse system.

5.3.2. Fluency Items and Scores

As shown in

Table 5. Descriptive Statistics for Fluency Subscale Items.

Fluency Items	Mean	SD
My thoughts/activities run fluidly and smoothly.	5.70	0.90
I have no difficulty concentrating.	5.80	1.10
My mind is completely clear.	5.70	1.10
The right thoughts/movements occur of their own accord (They happen effortlessly when in flow or alignment).	5.30	1.20
I know what I have to do each step of the way.	5.20	1.50
I feel that I have everything under control.	5.50	1.30

, the Fluency subscale of the FSS indicated a high level of cognitive smoothness and perceived control during interaction with the Necoverse system (M = 5.50; SD = 0.80). High ratings for concentration, mental clarity, and smooth execution suggest sustained attentional engagement and effective cognitive–motor coordination. Slightly greater variability in items related to automaticity and stepwise clarity indicates individual differences in perceived procedural effort, suggesting opportunities for further interface refinement to support more uniformly effortless interaction.

5.3.3. Flow—Absorption Items

As shown in

Table 6. Descriptive Statistics for the FSS Absorption Subscale Items.

Absorption Items	Mean	SD
I feel just the right amount of challenge.	4.80	1.50
I don’t notice time passing.	5.10	1.60
I am totally absorbed in what I am doing.	5.70	1.00
I am completely lost in thought.	3.20	1.40

, the Absorption subscale of the FSS indicated a moderately high level of immersion during the Necoverse system inspection task (M = 4.70; SD = 0.90). Participants reported strong task-focused engagement and moderate temporal dissociation, while low ratings for dissociative absorption (“lost in thought”) suggest that immersion remained primarily goal-directed rather than detached from task demands. Overall, these findings indicate that the system supported meaningful absorption without compromising attentional focus on inspection objectives.

5.3.4. Flow—Worrying Items

As shown in

Table 7. Descriptive Statistics for the Flow Worrying Items.

Flow Worrying Items	Mean	SD
Something important to me is at stake here (whether I feel that achieving the goal or completing the task is personally significant or impactful to me).	3.30	1.40
I won’t make any mistake here.	4.20	1.70
I am worried about failing.	2.30	1.60

, the Worry subscale indicated low task-related anxiety during the Necoverse system inspection task (M = 3.28; SD = 1.10). Participants reported low fear of failure and moderate confidence in task performance, suggesting a relaxed psychological state. Overall, low worry levels likely supported sustained attention and uninterrupted engagement, creating favorable conditions for a productive flow experience.

Taken together, the flow results indicate that the Necoverse system supported a productive, goal-directed form of flow characterized by high fluency, moderate absorption, and low task-related anxiety.

5.4. NASA-TLX Results Interpretation

The NASA-TLX was used to assess perceived workload across six dimensions during the Necoverse system inspection task (

Table 8. Descriptive Statistics for NASA-TLX Subscales.

NASA Items	Mean	SD
How mentally demanding was the Necoverse system task?	22.8	19.9
How physically demanding was the Necoverse system task?	14.7	17.6
How rushed or hurried did you feel during the Necoverse system task?	4.8	8.6
How successful were you in completing the Necoverse system task?	28.6	31.0
How much effort did the Necoverse system task require?	19.5	13.2
How frustrated did you feel while using the Necoverse system task?	11.1	18.2

). Overall, results indicate a generally low workload experience, with some individual variability.

Mental and physical demands were rated as low on average (M = 22.8, SD = 19.9; M = 14.7, SD = 17.6), consistent with modest cognitive requirements and limited physical effort. Temporal demand was minimal (M = 4.8, SD = 8.6), indicating that participants did not feel rushed. Effort and frustration were also low (M = 19.5, SD = 13.2; M = 11.1, SD = 18.2), suggesting that the task was not experienced as demanding or emotionally taxing. For the Performance subscale, it should be noted that the NASA-TLX uses an inverted scale in which lower scores indicate better perceived performance (0 = Perfect, 100 = Failure) [70,71]. Consistent with the inverted Performance scale described in Section 4.2, the observed mean score (M = 28.6, SD = 31.0) indicates that participants generally perceived themselves as successful in completing the inspection task, although the relatively high standard deviation reflects notable individual variability in self-assessed performance.

To further characterize overall workload, aggregate NASA-TLX scores were classified into workload ranges. Most participants (66.7%) fell within the very low workload range (0–20), while an additional 26.7% reported low-to-moderate workload (21–40). Only 6.7% experienced moderate workload (41–60), and no participants reported high workload levels. Overall, more than 93% of participants experienced the task as low or moderately demanding, confirming that the Necoverse system supported inspection activities within a manageable and low-stress workload profile.

5.5. Interview Results

Qualitative analysis was conducted on written responses from 30 participants to an open-ended question addressing overall experience and perceived improvement needs. Following Braun and Clarke’s six-phase thematic analysis framework [74], responses were inductively coded and iteratively synthesized into higher-order themes. Verbatim quotes were used selectively to preserve participants’ perspectives while maintaining analytical concision.

5.5.1. Overall Experience and Engagement

Participants reported a strongly positive overall experience, frequently describing the system as immersive, engaging, and practically valuable (22 mentions). Users emphasized the realism of the environment and its perceived applicability to real-world industrial contexts. As one participant noted, “The project seemed very cool and something that I see a big use for… Overall I had a good experience and didn’t encounter big problems.”

5.5.2. Onboarding and Learning Curve

Eight participants reported initial challenges during onboarding, particularly in locating the virtual smartphone and understanding the task sequence. These issues were often resolved through external guidance, prompting suggestions for clearer in-system cues and contextual instructions. One participant commented, “After receiving the phone it was not immediately clear that I had to use it… maybe some kind of notification would help.”

5.5.3. Control Ergonomics and Interaction Design

Interaction-related issues were among the most frequently cited concerns (15 mentions). Participants reported accidental button activations, unnatural finger postures, and occlusion caused by virtual hands or tools, which affected precision and comfort. As one user explained, “It’s rather easy to click buttons by accident, and holding your index finger up away from the controller feels a bit unnatural.” Suggested improvements included separating movement controls and refining the ergonomics of the virtual smartphone.

5.5.4. Motion Sickness and Comfort

Five participants reported dizziness or disorientation linked to locomotion mechanics, particularly joystick-based turning and overlapping movement controls. Participants proposed snap turning and alternative navigation schemes to mitigate discomfort. One participant stated, “Turning with the joystick made me feel dizzy… implementing snap turning might be a good idea.”

5.5.5. Technical Reliability and Stability

Technical issues were reported by eleven participants, including speech-to-text failures, application freezes, and crashes. These disruptions negatively affected immersion and workflow continuity. A participant highlighted the impact of such issues, stating, “The freeze after uploading pictures is very disorienting.” These findings underscore the importance of system robustness for professional deployment.

5.5.6. Immersion and Environmental Fidelity

Participants generally perceived the environment as realistic and engaging (7 mentions), while also identifying limitations related to point cloud completeness, visual artifacts, and a sparse surrounding context. One participant noted, “The surrounding area being black made it feel a bit empty… having more context could make it more immersive.” Suggestions focused on improving data fidelity and enriching environmental context.

5.5.7. Guidance Versus Independent Use

Eight participants reflected on the role of facilitator guidance, acknowledging its usefulness while noting that it could mask underlying usability issues. One participant observed, “Having someone guiding you is helpful but can mask some problems… extra guidelines would be nice for first-time use.” This highlights the need for integrated, context-sensitive guidance to support autonomous use.

5.5.8. Summary

Overall, the qualitative findings indicate that the system’s strengths in immersion, engagement, and perceived utility outweigh its limitations. However, recurring themes identify clear priorities for improvement, including onboarding support, interaction ergonomics, locomotion comfort, technical stability, and environmental fidelity. These user-grounded insights provide actionable guidance for refining collaborative industrial metaverse systems.

5.6. Correlation Analysis

A Pearson correlation analysis was conducted to examine relationships among perceived usability, cognitive workload (NASA-TLX), presence, and flow within the Necoverse system. The results reveal a coherent pattern of associations characterizing user experience during point cloud inspection tasks.

Moderate positive correlations were observed between usability and flow (r = 0.50, p = 0.005) and between presence and flow (r = 0.50, p = 0.005), indicating that higher perceived usability and a stronger sense of presence were both significantly associated with greater task engagement. In contrast, the relationship between usability and presence was negligible and not statistically significant (r = 0.03, p = 0.86), suggesting that perceived immersion was largely independent of interface usability.

Cognitive workload showed a moderate negative correlation with usability (r = −0.40, p = 0.029), indicating that increased workload was significantly associated with reduced ease of use. A smaller negative relationship was observed between workload and flow (r = −0.31, p = 0.099), though this association did not reach statistical significance. The relationship between workload and presence was weak and non-significant (r = −0.22, p = 0.25).

Overall, these findings indicate that usability and presence contribute independently to flow, while cognitive workload is associated with reduced flow mainly through its relationship with perceived usability. This pattern is consistent with qualitative and observational findings identifying onboarding complexity, control discoverability, and interaction ergonomics as key sources of cognitive demand, underscoring the importance of minimizing unnecessary workload in industrial metaverse system design.

5.7. Observation Results

Structured observation provided complementary behavioral evidence regarding participants’ ability to execute the inspection task and interact with the Necoverse system. The observation ratings were based on a 7-point Likert-type scale (1 = strongly disagree, 7 = strongly agree), aligned with the response format used in the questionnaire measures. Ratings were provided by a researcher observing each session in real time, focusing on participants’ demonstrated task understanding, ability to locate interface controls (discoverability), and understanding of control functionality during task execution. Structured observation of user interactions provided complementary quantitative and qualitative insights into system usability. Quantitative ratings revealed a clear contrast between participants’ conceptual understanding of the task and their ability to locate interface controls. Participants demonstrated a strong understanding of task objectives (M = 5.36; SD = 1.00), indicating that they generally knew what they were expected to accomplish within the Necoverse system environment. In contrast, the discoverability of interactive elements was substantially lower (M = 3.59; SD = 0.80), suggesting frequent difficulty in locating key controls. Once controls were found, participants’ understanding of their function was moderately high (M = 4.48; SD = 1.04), indicating that the primary usability barrier lay in control visibility rather than control logic.

Qualitative observations contextualized these findings by highlighting recurring interaction challenges. Participants frequently exhibited spatial disorientation and difficulty interpreting the environment, reflecting insufficient spatial cues and affordances. Unclear system status and imprecise control feedback were also common, with users showing uncertainty about interaction states and occasionally triggering unintended inputs. User performance varied notably based on prior VR experience, with novice users requiring frequent facilitator assistance while experienced users navigated the system more independently, underscoring the need for adaptive onboarding. Despite these challenges, participants consistently demonstrated strong engagement with the system’s core concept, expressing enthusiasm for its potential real-world application.

Taken together, the observational findings depict a system with strong conceptual clarity and high engagement potential that is constrained primarily by spatial navigation, control discoverability, and system feedback limitations. Addressing these issues would likely enhance usability while preserving the high levels of task engagement observed during inspection activities.

Although these observations were not quantified as formal performance metrics, they indicate that participants were generally able to complete the inspection workflow and produce inspection outputs, with most difficulties arising from interaction discoverability rather than task understanding.

6. Discussion

This study examines how an industrial metaverse system supports point cloud inspection in terms of usability, cognitive workload, presence, and flow. By integrating quantitative measures with qualitative and observational evidence, the findings clarify how such systems are experienced in practice and where key design constraints currently emerge.

It is important to note that the present study did not include objective performance measures such as task completion time or deviation detection accuracy. The evaluation was intentionally focused on usability, cognitive workload, presence, and flow as foundational human-factor conditions that must be established before inspection performance can be meaningfully assessed. Observational evidence indicates that participants were generally able to complete the inspection workflow and generate inspection artifacts, suggesting functional task execution. However, the study does not claim to evaluate inspection efficiency or accuracy, which remain important directions for subsequent research.

Regarding task representativeness, the inspection scenario was designed not to replicate the full complexity of professional shipbuilding inspection, but rather to isolate foundational interaction, cognitive, and experiential factors within an industrial metaverse environment. This approach aligns with established human–computer interaction practice, in which usability and workload are assessed using structurally representative tasks before domain-specific complexity is introduced [45,46]. Accordingly, the findings are most appropriately interpreted as evidence of interaction viability and experiential suitability rather than inspection performance optimization.

The absence of explicit performance outcome measures reflects a deliberate methodological choice. Given that participants lacked professional inspection experience and the evaluation was exploratory in nature, introducing performance benchmarks would have risked conflating domain expertise with interaction quality. The study therefore prioritizes understanding whether the industrial metaverse environment can support inspection activities with manageable cognitive demand, adequate usability, and sustained engagement—conditions that constitute necessary preconditions for reliable performance assessment.

The results indicate that the Necoverse system provides viable interaction foundations and experiential support for industrial inspection workflows under controlled laboratory conditions with non-expert users. Participants engaged effectively with inspection activities, maintained spatial understanding, and operated within generally manageable cognitive and physical demands. Notably, strong engagement and presence were achieved despite moderate perceptual realism, suggesting that functional immersion—driven by task relevance and spatial coherence rather than photorealistic fidelity—can be sufficient for industrial inspection use. This finding aligns with prior research indicating that presence emerges from the interaction between system properties and users’ psychological engagement with task-relevant content, not from technological fidelity alone [57,58]. At the same time, the findings underscore that interaction design quality, system feedback, and technical robustness remain decisive factors for scalable and unsupervised deployment.

A central outcome of this study is that experiential engagement and operational usability, while related, function as partially independent dimensions in industrial metaverse systems. Usability and presence were both positively associated with flow, whereas cognitive workload acted as a constraining factor—primarily through its negative association with perceived ease of use. This pattern is consistent with established flow theory, which holds that optimal experience depends on a balance between challenge and skill, supported by clear goals and immediate feedback [61,62]. Critically, these results indicate that immersive environments do not inherently compensate for usability shortcomings, nor does high usability alone guarantee engagement. Effective user experience instead emerges from a balanced orchestration of interaction clarity, spatial coherence, and workload management.

The findings further reveal that usability challenges were concentrated at the level of execution rather than conceptual understanding. Participants generally understood inspection goals and system intent, yet encountered friction related to tool discoverability, ambiguous system states, ergonomic constraints, locomotion comfort, and technical instability. These difficulties disproportionately affected less experienced users, indicating that usability was unevenly distributed across experience levels. Nevertheless, participants consistently expressed enthusiasm for the system’s core concept and its perceived industrial value, highlighting strong adoption potential once execution-level barriers are addressed.

From a broader perspective, this study contributes empirical evidence on how usability, cognitive workload, presence, and flow jointly shape user experience in industrial metaverse environments. The results suggest that industrial effectiveness depends less on maximal visual realism and more on cognitive efficiency, interaction transparency, and system reliability. Moreover, the mixed-methods approach demonstrates the value of triangulating subjective self-report data with observational and qualitative evidence to uncover interaction breakdowns that questionnaires alone may not capture. This methodological contribution responds to calls in the literature for context-sensitive evaluation approaches tailored to the unique demands of immersive industrial environments [45,46].

Regarding RQ1, participants perceived the Necoverse system as generally usable for point cloud inspection, with most users able to understand and execute inspection tasks effectively. For RQ2, cognitive workload remained low and manageable, indicating that system interaction did not impose excessive mental or physical demands. In relation to RQ3, participants reported a meaningful sense of presence—particularly in spatial awareness and attentional engagement—even though perceptual realism was rated more moderately. Addressing RQ4, the system supported a productive flow experience characterized by focused, goal-directed engagement and low task-related anxiety. Concerning RQ5, usability and presence independently contributed to flow, while cognitive workload acted as a constraining factor primarily through its influence on perceived ease of use. With respect to RQ6, qualitative and observational data revealed that usability challenges emerged mainly at the execution level—including tool discoverability, interaction ergonomics, system feedback clarity, and locomotion comfort—rather than from misunderstandings of task goals. Finally, for RQ7, the integrated findings informed actionable design recommendations for industrial metaverse systems, encompassing adaptive onboarding, enhanced interaction feedback, improved technical robustness, ergonomic interaction design, and support for flexible documentation workflows.

Taken together, these findings provide design-relevant insights grounded in observed user behavior rather than abstract usability principles. They offer empirically informed guidance for developing industrial metaverse systems that support accurate inspection, sustained engagement, and long-term professional adoption.

This study makes the following contributions:

• A consolidated empirical account of how usability, cognitive workload, presence, and flow manifest during immersive industrial inspection activities;
• Demonstration of how combining self-report, observational, and qualitative data can expose interaction frictions that are not evident through questionnaires alone;
• Practice-oriented design insights to inform the development of industrial metaverse inspection systems that support efficient interaction and manageable cognitive demand.

6.1. Design Implications for Industrial Metaverse Systems

The findings point to several overarching principles that should guide the design of future industrial metaverse systems.

First, cognitive frictionlessness must be a primary design objective. Users should be able to devote their mental resources to inspection and analysis rather than to understanding how to operate the system. This requires predictable interactions, visible affordances, and the elimination of hidden or ambiguous functions that increase cognitive overhead.

Second, usability and environmental fidelity should be treated as independent design axes. The results show that effective task engagement does not require photorealistic environments, but poor interaction design cannot be offset by visual realism. For industrial inspection, clarity of data representation, control precision, and workflow efficiency should take precedence unless realism directly supports decision-making.

Third, scaffolded onboarding is essential for accessibility and scalability. Observational and interview data revealed that novice users frequently relied on facilitator guidance, indicating a steep initial learning curve. Embedding adaptive, context-sensitive guidance that fades with experience can improve learnability without constraining expert efficiency.

Fourth, technical robustness is foundational to professional trust. Even infrequent failures—such as crashes, freezes, or unreliable input modalities—disrupt workflow continuity and undermine confidence. Stability, recoverability, and transparent system states must therefore be treated as core features rather than secondary optimizations.

Finally, ergonomic comfort and physiological safety are prerequisites for sustained use. Locomotion discomfort and awkward interaction postures directly limit the feasibility of prolonged professional deployment. Configurable movement options and ergonomically aligned interaction design are essential to support long-term use in industrial contexts.

6.2. Design Recommendations

Based on these principles, the following recommendations are proposed to strengthen industrial metaverse systems for professional inspection tasks:

• Introduce adaptive onboarding through embedded tutorials and progressive guidance to reduce reliance on external facilitation.
• Improve tool discoverability by making critical interaction elements persistently visible and spatially anchored.
• Enhance system feedback with explicit visual and auditory indicators for all system states to reduce uncertainty.
• Prioritize technical reliability, including robust handling of input failures, freezes, and recovery from interruptions.
• Support motion comfort and accessibility through configurable locomotion options such as snap turning.
• Improve environmental and point cloud fidelity where it directly supports inspection accuracy and spatial interpretation.
• Expand annotation modalities to support flexible, precise documentation aligned with professional inspection workflows.

6.3. Limitations and Future Work

Several limitations should be acknowledged. First, the study was conducted in a controlled laboratory environment with facilitator support, which likely attenuated onboarding and discoverability challenges relative to independent operational use. Second, the participant sample—comprising engineering students rather than professional inspectors—may not fully represent the expertise, expectations, or constraints encountered in real-world production environments. Future work should therefore prioritize longitudinal field studies conducted in active industrial contexts with domain experts under extended-use conditions. Such studies should examine sustained usability, workflow integration, and the role of intelligent assistance—including automated deviation detection and contextual guidance—in enhancing inspection accuracy and efficiency within industrial metaverse systems.

On the technical development side, planned improvements informed by the present findings include replacing handheld controllers with hand- and finger-tracking–based interaction, implementing snap turning to address reported locomotion discomfort, and integrating spoken-word annotations in users’ native languages with support for over 50 languages. Additional enhancements include a rectangular 3D spatial visualization indicating the location and viewing direction of captured images, supplementary environmental layers—such as a nature layer replacing the current black void with natural ground and vegetation—and the introduction of a photorealistic 3D model layer using techniques such as Gaussian splatting.

While the SEQ provides a reliable measure of perceived task-level usability [69], it captures only a global impression and does not differentiate among usability subdimensions such as learnability, efficiency, or error tolerance. Future studies should therefore complement it with validated multi-item instruments such as the System Usability Scale (SUS) [41] to enable more granular usability profiling and cross-system benchmarking.

Future work should also incorporate objective performance indicators—such as deviation detection accuracy, inspection completeness, task completion time, and interaction log analysis—to examine how usability, workload, presence, and flow relate to measurable inspection effectiveness under operational conditions. Such evaluations should be grounded in clearly defined ground truth data and conducted with expert participants to ensure that performance outcomes reflect genuine inspection capability rather than novice exploration behavior.

7. Conclusions

This study demonstrates that industrial metaverse systems can support point cloud–based inspection workflows by combining high perceived usability, manageable cognitive workload, and engaging user experience under controlled laboratory conditions. The findings indicate low perceived workload, strong task fluency, and meaningful spatial and attentional presence, suggesting that immersive inspection workflows can be cognitively sustainable and experientially effective from a human-factors perspective. By integrating quantitative measures with qualitative and observational evidence, this work moves beyond isolated usability assessments to provide actionable, empirically grounded design guidance for industrial metaverse systems.

At the same time, the results reveal that key challenges persist primarily at the level of execution rather than conceptual understanding. Issues related to onboarding, system feedback clarity, interaction ergonomics, locomotion comfort, and technical reliability continue to constrain scalability and independent use. These findings emphasize that experiential engagement alone is insufficient for robust industrial adoption and that interaction quality and system robustness must be addressed jointly to support reliable and repeatable workflows.

By grounding the evaluation in real user behavior, this research establishes a foundation for assessing interaction feasibility and user experience in industrial metaverse inspection environments. While the study was conducted with non-expert participants in a controlled setting with facilitator support, it identifies clear directions for future work. Longitudinal field studies involving domain experts, extended system use, and the integration of context-aware assistance—such as automated deviation detection or adaptive guidance—will be essential to evaluate long-term usability, workflow integration, and operational impact in real industrial contexts. Addressing the identified usability barriers will support the development of industrial metaverse systems that enable accurate, efficient, and intuitive inspection workflows in practice.