Next Article in Journal
Circular Economy and Sustainability in Higher Education: A Comparative Study of Knowledge and Student Perceptions in Peru, Colombia, and Mexico
Next Article in Special Issue
AI-Based Online Education Systems Integrating Real-Time Affective Computing: A Design-Oriented Conceptual Framework
Previous Article in Journal
Imprisonment and the Redistribution of Harm Across Families and Wider Relationships
Previous Article in Special Issue
Evaluation of a Digital Twin Metaverse Classroom in Higher Education
 
 
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

Usability and User Advocacy of a Digital Twin-Inspired Metaverse Orientation System: An Exploratory Pilot Study

by
Jia-Hui Tan
1,
Soon-Nyean Cheong
1,2,*,
Chee-Onn Wong
1,2 and
Ahmad Hishamuddin Bin Mohamed
2,3
1
Faculty of Creative Multimedia, Multimedia University, Persiaran Multimedia, Cyberjaya 63100, Selangor, Malaysia
2
Centre of Excellence for Immersive Experience, Multimedia University, Persiaran Multimedia, Cyberjaya 63100, Selangor, Malaysia
3
Telekom Malaysia Research & Development Sdn. Bhd., TM Innovation Centre, Cyberjaya 63000, Selangor, Malaysia
*
Author to whom correspondence should be addressed.
Soc. Sci. 2026, 15(7), 414; https://doi.org/10.3390/socsci15070414
Submission received: 15 May 2026 / Revised: 10 June 2026 / Accepted: 19 June 2026 / Published: 24 June 2026

Abstract

University orientation programmes are a primary mechanism through which new students become familiar with campus facilities, academic spaces, and institutional procedures. However, many orientation activities are delivered as single in-person sessions, limiting opportunities for students to revisit spatial and procedural information after the event. To help address this constraint, a digital twin-inspired metaverse orientation application, the Digital Twin Metaverse Orientation (DTMO), was designed in Unity and hosted on Spatial.io as a spatially faithful virtual replica of a faculty environment. An exploratory pilot evaluation was conducted with 30 university students from multiple faculties after a facilitator-guided orientation session. The System Usability Scale (SUS), Net Promoter Score (NPS), and two open-ended questions were used to examine perceived usability, recommendation intention, and the reasons underpinning recommendation decisions. The application obtained a mean SUS score of 86.83, corresponding to an excellent perceived-usability rating, and an NPS of 53.33, indicating positive immediate recommendation intention. Qualitative responses suggested that participants valued the DTMO for engagement, accessibility, ease of navigation, and support for spatial familiarisation, while some participants emphasised that it should complement rather than replace physical orientation. These pilot findings indicate promising user reception in a small, guided-session sample, but they do not establish orientation effectiveness, learning transfer, wayfinding performance, retention, belonging, institutional integration, or sustained use. Further research with broader samples and outcome-based measures is therefore needed.

Graphical Abstract

1. Introduction

University orientation programmes are a primary mechanism through which incoming students become familiar with the layout of faculty facilities and the academic procedures they will follow (Schilling et al. 2022). Effective orientation can support a smoother academic transition, but many faculty-level programmes are delivered as single scheduled in-person sessions. Students who miss the session, enrol late, or need repeated exposure to new spaces and procedures may therefore have limited access to structured orientation support (Schilling et al. 2024; Tüzün and Özdinç 2015). More flexible orientation models that remain available after the scheduled event may contribute to more inclusive student onboarding in higher education (Schilling et al. 2022), in line with SDG 4’s emphasis on equitable access to educational resources. This single-exposure limitation motivates the search for digital alternatives that provide asynchronous and repeated access to the faculty environment. Immersive digital environments offer one such modality because they allow students to navigate information-rich spaces independently of time and location constraints (Hutson et al. 2023). Metaverse environments have attracted increasing attention in higher education for their potential to support avatar-based engagement and immersion (Sripan and Jeerapattanatorn 2025). Tüzün and Özdinç (2015) compared three-dimensional virtual orientation with physical orientation and reported equivalent gains in conceptual knowledge as well as stronger spatial route recall in the virtual condition, although usability was not evaluated with standardised instruments. Reviews of metaverse-based learning consistently report motivation, engagement, and experiential gains as common outcomes, while also identifying the lack of comparable evaluation frameworks as a persistent methodological gap (Sripan and Jeerapattanatorn 2025). In line with these concerns, Pyae et al. (2023) documented interface and navigation barriers among first-time users of a metaverse learning experience, which indicates a need for careful onboarding and usability evaluation.
This gap is important because usability has been identified as a condition for sustained engagement with metaverse applications. In their review, Al-Kfairy et al. (2024) note that navigation and operational challenges can reduce perceived usability and undermine continued-use intentions. The System Usability Scale (SUS) (Brooke 1996) provides a validated ten-item composite usability score on a 0–100 scale, while the Net Promoter Score (NPS) (Reichheld 2003) measures recommendation intention by classifying respondents as Promoters, Passives, or Detractors. Uribe et al. (2024) reported substantial variation in usability across metaverse platforms despite broadly similar learning outcomes, which points to the need for platform-specific and standardised assessment. Sasmito et al. (2019) showed that SUS and NPS can be used together to evaluate an interactive system; however, the available literature (Sripan and Jeerapattanatorn 2025) does not yet show this paired framework being applied to an immersive digital orientation environment. To address this evaluation gap, the present study designed and developed the Digital Twin Metaverse Orientation (DTMO) on Spatial.io as a one-to-one immersive replica of a university faculty space. The DTMO is a spatially faithful virtual replica rather than a live-synchronised digital twin. Its orientation information is embedded directly in the environment so that participants can revisit the space independently after the initial facilitator-guided session. This persistent-content design positions the DTMO as a reusable, location-independent orientation support rather than a substitute for physical orientation. The study therefore applies SUS and NPS as paired descriptive measures and addresses three research questions.
RQ1: How was the DTMO designed and developed to support repeated, spatially navigable orientation beyond a single in-person session?
RQ2: How do participants perceive the usability of the Digital Twin Metaverse Orientation for student orientation, as measured by the System Usability Scale?
RQ3: How willing are participants to recommend the Digital Twin Metaverse Orientation for orientation purposes, and what qualitative factors underpin their recommendation decisions, as captured by the Net Promoter Score?
This study contributes exploratory usability and user-advocacy evidence for the DTMO in a higher education orientation context. Rather than treating SUS and NPS as evidence of orientation effectiveness, the study uses the instruments to characterise perceived usability, immediate recommendation intention, and the qualitative reasons that shape advocacy. The pilot evaluation produced a mean SUS score of 86.83 and an NPS of 53.33, with open-ended responses identifying engagement, accessibility, ease of navigation, and spatial familiarisation as the main drivers of recommendation. The findings therefore provide an initial user-centred evaluation basis for future studies that examine spatial, social, and procedural onboarding outcomes more directly.

2. Literature Review

This review is organised around the study’s evaluation problem: the limits of existing digital orientation tools, what metaverse-based and digital twin-inspired environments make possible and where they fall short, and the need for standardised measures of usability and user advocacy. Universities usually deliver digital orientation through websites, learning management system modules, and video-based presentations (Schilling et al. 2022). Such formats support information delivery. What they rarely provide is room for students to navigate actively or explore a space. The shortfall becomes clearest in complex facilities such as laboratories or specialist learning spaces, where limited exposure before arrival can leave students underprepared spatially (Hutson et al. 2023). Beyond this, students expect onboarding information to remain available when needed, rather than only during one scheduled session (Schilling et al. 2024). Traditional formats therefore widen access to information, yet they do little to support spatial familiarisation or repeated, exploratory engagement with the institution’s physical layout.
The metaverse is usually described as an immersive, shared virtual environment in which users interact through avatars and experience heightened social presence within persistent digital spaces. Within education, such platforms have largely been studied for how they shape engagement and experiential learning (Sripan and Jeerapattanatorn 2025). Alfaisal et al. (2024), reviewing 41 studies, report that the Technology Acceptance Model dominates as the framework for predicting learners’ intention to adopt metaverse systems, and university students frequently serve as respondents. Within this framework, usability and recommendation intention help predict whether adoption is sustained over time. Tüzün and Özdinç (2015) give a relevant case: among 55 students, three-dimensional virtual orientation produced conceptual gains similar to physical orientation but better spatial route recall. No standardised usability instruments were used in that study.
Adoption is a further question, beyond engagement. Avatar-mediated embodiment and co-presence can raise social presence and help students explore a space, though they also add interaction demands that may lower perceived usability (Sripan and Jeerapattanatorn 2025; Al-Kfairy et al. 2024). Repeatable web-based access also brings in students who cannot make a scheduled session (Hutson et al. 2023). But it relies on adequate devices and a stable connection, which can leave students on the wrong side of the digital divide at a disadvantage. When orientation is viewed through adoption and acceptance rather than engagement alone, usability, social influence, and equitable access become conditions for sustained use.
A digital twin is a digital representation of a physical entity that stays in structured correspondence with its real-world counterpart (Jones et al. 2020). Beyond visual resemblance, full digital twin architectures usually incorporate data links, monitoring, or bidirectional exchange. The present system draws on this idea in its design, yet because it lacks automated bidirectional data exchange, it is formally a digital model and not a digital twin (Jones et al. 2020). Its value for orientation comes from spatial fidelity. A one-to-one virtual replica lets students explore spatially organised information inside a recognisable real-world structure. Orientation does more than deliver information. It also involves becoming familiar with the physical and academic environment in which future activities take place (Tüzün and Özdinç 2015). When used in this way, digital twin-inspired environments can support familiarisation and spatial preparation before physical arrival (Hutson et al. 2023; Manokeaw et al. 2025).
Usability evaluations of metaverse platforms in educational settings report substantial variation across platforms, even where learning outcomes are broadly similar (Uribe et al. 2024). Spatial.io itself has been rated positively for perceived usefulness, ease of use, and satisfaction in student perception studies (Susilana et al. 2024). Previous work has thus examined digital orientation tools, metaverse-based learning environments, and virtual orientation experiences, but these strands remain only partially integrated. Existing digital orientation tools mainly support information access rather than navigable spatial familiarisation, and prior virtual orientation studies have tended to focus on learning or route recall rather than standardised usability evaluation. Digital twins and digital twin-inspired models have been used for educational visualisation and preparation, but orientation-specific systems that replicate actual faculty spaces and are tested with a paired usability-advocacy framework remain rare. The present study addresses this gap with a faculty-based DTMO that corresponds spatially to a real learning environment and is evaluated using both SUS and NPS.
Standardised instruments are required for benchmarkable evaluation of interactive systems. Brooke (1996) developed the System Usability Scale, a widely validated 10-item instrument to yield a composite usability score on a 0–100 scale. Bangor et al. (2009) then established a grading interpretation and proposed an adjective rating scale that correlates composite scores with descriptors ranging from “Worst Imaginable” to “Excellent”. To achieve the highest tier, a score at or above 80.3 is needed, which corresponds to Grade A. Lewis and Sauro (2009) identified a two-factor measurement structure comprising a primary usability subscale and a learnability subscale, each demonstrating acceptable internal consistency. Reichheld (2003) introduced the Net Promoter Score, a single-item measure of recommendation likelihood using a 0–10 scale, in which respondents are classified into three groups: Promoters (9–10), Passives (7–8), and Detractors (0–6). In addition, Kara et al. (2022) examined its utility in higher education and established that, when paired with open-ended responses, the instrument captures both recommendation levels and the reasons underlying those decisions. The combination of both instruments as complementary evaluation measures has been demonstrated by Sasmito et al. (2019), and together they constitute a concise, validated framework for evaluating the operability and advocacy dimensions of immersive orientation systems, as shown in Figure 1.
Existing digital orientation tools widen access to information but do not necessarily build spatial or interactive familiarity, and the metaverse and digital twin-inspired approaches that might address this are seldom evaluated with standardised, user-centred instruments. The present study applies such a framework, pairing usability and advocacy measures on a faculty-based virtual replica designed for spatial familiarisation and repeatable orientation access.

3. Materials and Methods

This section describes the design, development, and evaluation of the DTMO using the ADDIE instructional design model (Branch 2009): Analysis, Design, Development, Implementation, and Evaluation. Analysis came first, establishing the system requirements. To examine perceived usability and recommendation intention, an evaluation framework combining the SUS and NPS was chosen in response to the lack of standardised evaluation in comparable metaverse orientation research. The main functional requirement was a three-dimensional navigable environment allowing free spatial movement rather than passive viewing of static images or videos. The environment was therefore designed to reproduce physical measurements and asset locations at a one-to-one scale, consistent with the study’s digital twin-inspired rationale.
In the Design phase, environment blueprinting and content planning were carried out for the Faculty of Creative Multimedia orientation. Classroom dimensions, photographs, and equipment placement were recorded to guide the one-to-one digital replica, and orientation content was organised around six faculty course introductions. This content covered classroom and equipment familiarisation, course-related facilities, educational tools and teaching materials, and basic learning-space familiarisation supported by photographs. Orientation guidance was embedded in the environment so that participants could interact with both the academic information and the represented physical space, and they could revisit it after the initial facilitated session, allowing repeated access beyond a single orientation event. The embedded content thus mitigates the single-session constraint of in-person orientation by keeping orientation information available for repeated, independent access. The orientation flow comprised guided spatial exploration within the DTMO, a facilitator-led faculty briefing, and group exercises with quizzes.
Development came next, following the three-stage pipeline shown in Figure 2: asset creation, environment setup, and deployment. Physical classroom measurements and photographs guided the construction of one-to-one Blender (version 5.1; Blender Foundation, Amsterdam, The Netherlands) geometry, with assets optimised for web-based rendering. The scene was then built in Unity (version 2021.3.44f1; Unity Technologies, San Francisco, CA, USA) using baked lightmaps and mesh colliders to support stable avatar movement, and finally deployed as a custom Spatial.io (Spatial Systems, Inc., New York, NY, USA) environment through the Spatial Creator Toolkit, which provided interactive elements, voice communication, and multi-user synchronisation.
For implementation, the system was delivered through Spatial.io in a web browser, with no local installation required. Pilot sessions ran in groups of four in a Faculty of Creative Multimedia classroom, each group attending a separate scheduled session under facilitator supervision, and participants accessed the DTMO individually through web browsers, as shown in Figure 3. Each session included a tutorial space for learning Spatial.io navigation and interaction controls, a guided tour of the digital replica classroom, a facilitator-led faculty briefing, and a group quiz. The full session lasted about 45 min: 5 for the tutorial, 5 for guided exploration, 10 for the faculty course briefing, 10 for the group activity, and 15 for questionnaire completion.
The evaluation used two instruments. Perceived usability was measured with the System Usability Scale, which has ten items on a five-point Likert scale. Following Brooke (1996), an overall score from 0 to 100 was calculated and then interpreted with the Bangor et al. (2009) adjective scale, on which scores above 80.3 fall in Grade A (“Excellent”). User advocacy was measured with the Net Promoter Score (Reichheld 2003). Participants who rated the system 9–10 counted as Promoters, 7–8 as Passives, and 0–6 as Detractors, and NPS was the percentage of Promoters minus the percentage of Detractors. The two optional open-ended items asked why participants would or would not recommend it. A single researcher coded them through inductive thematic analysis: codes were drawn directly from participants’ responses, grouped into themes, and compared across NPS segments. Because the qualitative component was exploratory and rested on one coder, formal inter-rater reliability was not computed. That constraint is acknowledged among the study limitations.
All participants were informed of the study’s purpose, the voluntary nature of participation, their right to withdraw without consequence, and the confidentiality of their responses. Digital informed consent was then obtained before participation began. Thirty participants were recruited from the university community by convenience sampling. To reduce prior-knowledge effects, students from the Faculty of Creative Multimedia were excluded. Basic computer literacy was required, though prior experience with three-dimensional environments was not. For an exploratory pilot centred on usability and user advocacy, convenience sampling was an appropriate choice, since feasibility and rapid recruitment were the priorities here. That choice nonetheless limits generalisability, so the findings should be read as preliminary. Each participant first completed a pre-session demographic questionnaire before entering the tutorial space to learn Spatial.io navigation and interaction controls. They then explored the DTMO’s digital replica classroom under guidance. After joining the facilitator-led orientation briefing, they completed a breakout group quiz on the orientation content and, finally, answered the post-session questionnaire, which comprised the SUS items, the NPS item, and the optional open-ended items.
Descriptive statistics summarised the demographic data. SUS responses were scored with Brooke’s standard procedure, analysed in SPSS (version 27; IBM Corp., Armonk, NY, USA), and reported as means and standard deviations, while NPS was computed with the standard formula (% Promoters − % Detractors). The optional open-ended responses were analysed through inductive thematic coding based on recurring response patterns. The complete evaluation instruments are provided in Appendix A.
Generative AI tools supported language editing and clarity during manuscript preparation, assisting only with grammar and stylistic refinement. All ideas, data collection, analysis, interpretation, and conclusions were the authors’ own, and the authors take full responsibility for the content of the manuscript.

4. Results

This section reports participant demographics together with the usability and advocacy findings. As Table 1 shows, the 30 participants were predominantly male (86.67%) and mostly first-year students from five non-FCM faculties, and their prior experience with three-dimensional environments varied widely. This variation matters when reading the usability results.
Table 2 reports the SUS descriptive statistics. On the Bangor et al. (2009) grading scale, set out in Table 3, the mean of 86.83 corresponds to Grade A (≥80.3) and the lowest score of 65 to the marginal band. Internal consistency was α = 0.673.
Participants’ likelihood-to-recommend ratings produced a Net Promoter Score of 53.33. As Table 4 shows, 63.33% were Promoters, 26.67% Passives, and 10.00% Detractors. Promoters were drawn mainly by engagement and enjoyment, accessibility, and ease of use; Passives liked the system but flagged technical limits and its inability to replace physical attendance; and the one Detractor who commented cited limited functionality relative to existing video-based communication tools.

5. Discussion

The findings are discussed in relation to the research question and then summarised. For RQ1, the DTMO addressed two limitations identified in the Introduction: physical orientation happens in a single session, and many existing digital alternatives are passive. Its one-to-one spatial fidelity kept the virtual model aligned with the physical classroom and was meant to build the navigational familiarity that static images, videos, or non-navigable tours cannot give. Orientation slides sat inside the environment so participants could revisit content on their own after the facilitator-guided session. This made orientation support a reusable, location-independent resource rather than a one-time event. The qualitative data fit this rationale. Among Promoters, 57.9% cited accessibility and remote access, and 42.1% cited spatial familiarisation, which suggests that some participants recognised both the access and spatial-preparation functions of the system. The Grade A SUS outcome further suggests that the tutorial-supported onboarding helped reduce the interface and navigation difficulties reported in comparable first-time metaverse deployments (Pyae et al. 2023).
For RQ2, the DTMO earned a mean SUS of 86.83 (SD = 9.33), which is an “Excellent” rating on the Bangor et al. (2009) scale. This stands out because participants had never used the platform and came from five faculties with varied backgrounds and prior three-dimensional experience. The lowest score (65) fell in the marginal band (51.0–67.9), the mean fell in the excellent range, and no score reached the not-acceptable range. Because usability scores for metaverse learning tend to shift with platform and task (Uribe et al. 2024), the high average is most safely interpreted as strong perceived usability following a guided session, likely helped by the system’s spatial fidelity, the structured onboarding, and the Spatial.io interface itself. The spread of scores (SD = 9.33) likely reflects differences in prior experience and navigation skill across participants. The reliability coefficient, α = 0.673, was just under the usual 0.70 mark. The SUS is known to have two factors, usability and learnability (Lewis and Sauro 2009), which may explain some of this, though the value remains below threshold. Item-level statistics are therefore reported, and the issue is treated as a limitation, particularly given the small sample.
For each SUS item, Table 5 gives the corrected item–total correlation and the alpha-if-deleted value. Two items were weak: SUS4 and SUS9 correlated only 0.05 and 0.04 with the total, and removing either would lift the coefficient to 0.698 or 0.694, close to but still under the 0.70 threshold. The other eight items ranged from 0.23 to 0.62, and six reached or passed the conventional 0.30 criterion. The sub-threshold alpha therefore reflects these two weak items rather than any general breakdown of the SUS responses in this dataset.
For RQ3, the system received an NPS of 53.33, with 63.33% Promoters and 10.00% Detractors. As a contextual reference, Kara et al. (2022) reported 51.2% Promoters, 33.5% Passives, and 15.2% Detractors (NPS ≈ 36) across undergraduate business programmes at three universities (N = 493). These figures are not directly comparable. They differ in context, since the Kara et al. (2022) baseline captures programme-level advocacy, whereas the present rating was an immediate post-session judgement of one tool. Sample size diverges sharply as well (N = 30 versus N = 493), and the recommendation construct itself is not identical, so the reference marks only the direction of the present result rather than its magnitude. Even with those caveats, the pilot ran higher on Promoters (63.33% versus 51.2%) and lower on Detractors (10.00% versus 15.2%), yielding a higher overall score (53.33 versus about 36). The reasons participants gave for advocacy were experiential and instrumental alike. Engagement and enjoyment led among Promoters (68.4%), followed by accessibility and remote access (57.9%), ease of use (47.4%), and spatial familiarisation (42.1%). Of these, spatial familiarisation carries the most weight for the system’s orientation purpose, although it is a participant-reported theme rather than evidence of measured wayfinding gains. Among Passives, the most common reservations were technical concerns (50.0%) and doubts about replacing physical orientation (37.5%). These reservations reinforce positioning the DTMO as a complement to physical orientation rather than a replacement.
The SUS and NPS were collected from the same 30 participants, which allows the two to be interpreted together at a descriptive level. A high mean SUS and a positive NPS point in the same direction. Participants generally found the system easy to use, and after the guided session, they were willing to recommend it. This aligns with the paired-instrument approach of Sasmito et al. (2019). The link between usability and advocacy gains further support from the fact that 47.4% of Promoters cited ease of use as a reason for recommending. Aggregate figures, however, cannot yield individual-level correlations, and so the relationship is best read as a descriptive, qualitative alignment rather than a statistically established association. Interpreted in this way, the DTMO emerges as a promising preparatory complement to physical orientation, pending further validation for institutional deployment.
From this reading, three practical implications follow. The system should first be positioned as a preparatory complement rather than a substitute. A second implication concerns deployment: the accessibility, remote-access, and spatial-familiarisation themes point toward a model that supports repeatable pre-familiarisation for students in remote locations, as well as for those unable to attend scheduled sessions. Third, since participants varied in their prior three-dimensional experience, tiered onboarding and orientation protocols should be calibrated to the proficiency distribution of the target user group. At the institutional level, then, the DTMO is best presented as a preparatory tool. Framing it this way avoids the expectation mismatch seen among Passives, who enjoyed the system but doubted that it could replace physical attendance.

6. Conclusions and Future Work

The study set out to develop and evaluate the DTMO as a student orientation tool, judged on three grounds: design adequacy, perceived usability, and user advocacy. Its core idea was to relax the single-session limit of in-person orientation by keeping orientation content inside a one-to-one virtual replica that students can revisit after the guided session. On usability, the system reached a mean SUS of 86.83, a Grade A or “Excellent” rating on the Bangor et al. (2009) scale, with no participant in the not-acceptable range. The NPS was 53.33, with 63.33% of participants classified as Promoters. Qualitative analysis identified engagement, accessibility, ease of use, and spatial familiarisation as the main participant-reported drivers of recommendation.
What these findings show is positive user reception, not orientation effectiveness. The favourable usability and advocacy scores, and the qualitative responses linking ease of use and spatial familiarisation to recommendation, point to the DTMO as a promising and perceptually accessible orientation support tool. However, they reflect only perceived usability and immediate recommendation intention after a guided pilot session, and they do not establish learning transfer, retention, wayfinding performance, belonging, institutional integration, or sustained use.
A few limitations bound these conclusions. The system does not perform the real-time synchronisation or bidirectional data exchange of a full digital twin, so it is best understood as a digital twin-inspired digital model. The sample was also small and convenience-based, skewed towards male participants (86.67%), and the qualitative coding had no formal inter-rater reliability check. The findings are therefore preliminary and not representative of the wider student body. Future work could relate usability and advocacy at the individual level, add pre- and post-tests of orientation knowledge and spatial familiarity, compare passive screen-sharing briefings with immersion-based in-world experiences, widen the evaluation to more institutions and students, and apply the dual-instrument framework across other metaverse platforms (Uribe et al. 2024; Susilana et al. 2024) to enable comparative evaluation of immersive orientation tools.

Author Contributions

Conceptualization, S.-N.C. and C.-O.W.; methodology, J.-H.T.; software, J.-H.T.; validation, S.-N.C., C.-O.W. and J.-H.T.; formal analysis, J.-H.T.; investigation, J.-H.T.; resources, S.-N.C., C.-O.W. and A.H.B.M.; data curation, J.-H.T.; writing—original draft preparation, J.-H.T.; writing—review and editing, S.-N.C. and C.-O.W.; visualization, J.-H.T.; supervision, S.-N.C., C.-O.W. and A.H.B.M.; project administration, S.-N.C.; funding acquisition, S.-N.C. and A.H.B.M. All authors have read and agreed to the published version of the manuscript.

Funding

This research was funded by Telekom Malaysia Research & Development Sdn. Bhd., grant number RDTC/251158. The APC was funded by Telekom Malaysia Research & Development Sdn. Bhd.

Institutional Review Board Statement

Ethical review and approval were not required for this study because it involved a minimal-risk educational usability evaluation using anonymised questionnaire responses, with no identifiable private information collected within or linked to the research responses, as described under Guideline 1(4)(c) of the Ministry of Health Malaysia Medical Review & Ethics Committee (MREC), “Guidelines for Ethical Review of Clinical Research or Research Involving Human Subjects.”

Informed Consent Statement

Written informed consent was obtained from all participants before participation.

Data Availability Statement

The survey instruments are reproduced in Appendix A. The evaluation protocol, recruitment information, anonymised coding sheet, and analysis materials are available from the corresponding author upon reasonable request. The DTMO system assets and Unity/Spatial project files are not publicly available because of institutional branding, privacy, and platform licensing restrictions.

Acknowledgments

The authors would like to acknowledge the financial support provided by the TM R&D grant, RDTC/251158, which played a vital role in the successful development of the prototype at Multimedia University. During the preparation of this manuscript, the authors used Claude Opus 4.8 (Anthropic) for the purposes of language editing (grammar and stylistic refinement). The authors have reviewed and edited the output and take full responsibility for the content of this publication.

Conflicts of Interest

The authors declare no conflict of interest. The funding sponsors had no role in the design of the study; in the collection, analyses, or interpretation of data; in the writing of the manuscript; and in the decision to publish the results.

Appendix A

The instruments were delivered through Microsoft Forms. Demographic items (Table A1) were collected before the session. The System Usability Scale (Table A2), the Net Promoter Score item, and the two open-ended questions (Table A3) were collected immediately after the facilitator-guided session.

Appendix A.1. Pre-Session Demographic Questionnaire

Table A1. Pre-session demographic items.
Table A1. Pre-session demographic items.
ItemResponse Options
IDA self-chosen nickname, used only to match the pre- and post-session responses
AgeUnder 18; 18–20; 21 and older
GenderFemale; Male
Year of studyFoundation; Year 1; Year 2; Year 3; Post-graduate; PhD
FacultyComputing & Informatics (FCI); AI and Engineering (FAIE); Engineering & Technology (FET); Information Science & Technology (FIST); Creative Multimedia (FCM); Cinematic Arts (FCA); Management (FOM); Business (FOB); Law (FOL); Applied Communication (FAC)
Experience with 3D virtual worlds or games (non-VR; e.g., Minecraft, Genshin Impact)Never; Rarely; Sometimes; Often; Very Often

Appendix A.2. System Usability Scale (SUS; Brooke 1996)

The standard ten-item SUS (Brooke 1996) was administered on a five-point scale (1 = strongly disagree, 5 = strongly agree). Following Brooke’s scoring procedure, negatively worded items (2, 4, 6, 8, and 10) were reverse-scored so that higher values indicate more favourable usability. The reliability and item-level statistics reported in Table 5 were computed on these scored values.
Table A2. System Usability Scale items.
Table A2. System Usability Scale items.
No.Item
1I think that I would like to use the Digital Twin Metaverse Orientation Classroom frequently.
2I found the Digital Twin Metaverse Orientation Classroom unnecessarily complex.
3I thought the Digital Twin Metaverse Orientation Classroom was easy to use.
4I think that I would need the support of a technical person to be able to use the Digital Twin Metaverse Orientation Classroom.
5I found the various functions in the Digital Twin Metaverse Orientation Classroom were well integrated.
6I thought there was too much inconsistency in the Digital Twin Metaverse Orientation Classroom.
7I would imagine that most people would learn to use the Digital Twin Metaverse Orientation Classroom very quickly.
8I found the Digital Twin Metaverse Orientation Classroom very cumbersome to use.
9I felt very confident using the Digital Twin Metaverse Orientation Classroom.
10I needed to learn a lot of things before I could get going with the Digital Twin Metaverse Orientation Classroom.

Appendix A.3. Net Promoter Score (NPS; Reichheld 2003) and Open-Ended Items

Table A3. Net Promoter Score and open-ended items.
Table A3. Net Promoter Score and open-ended items.
No.Item
1How likely are you to recommend the Digital Twin Metaverse Orientation Classroom to a friend or colleague for Faculty Orientation Briefing? (0 = not at all likely; 10 = extremely likely)
2(Open-ended) If you would recommend this application, what are your main reasons?
3(Open-ended) If you would NOT recommend this application, what are your main reasons?

References

  1. Alfaisal, Raghad, Haslinda Hashim, and Ummu Husna Azizan. 2024. Metaverse system adoption in education: A systematic literature review. Journal of Computers in Education 11: 259–303. [Google Scholar] [CrossRef]
  2. Al-Kfairy, Mousa, Ayham Alomari, Mahmood Al-Bashayreh, Omar Alfandi, and Mohammad Tubishat. 2024. Unveiling the Metaverse: A survey of user perceptions and the impact of usability, social influence and interoperability. Heliyon 10: e31413. [Google Scholar] [CrossRef] [PubMed]
  3. Bangor, Aaron, Philip Kortum, and James Miller. 2009. Determining what individual SUS scores mean: Adding an adjective rating scale. Journal of Usability Studies 4: 114–23. [Google Scholar]
  4. Branch, Robert Maribe. 2009. Instructional Design: The ADDIE Approach. New York: Springer. [Google Scholar] [CrossRef]
  5. Brooke, John. 1996. SUS: A “Quick and Dirty” Usability Scale. In Usability Evaluation in Industry. Abingdon: Taylor & Francis, pp. 207–12. [Google Scholar]
  6. Hutson, James, Robert Steffes, and Joe Weber. 2023. Virtual learning environments and digital twins: Enhancing accessibility, diversity, and flexibility in training secondary educational administrators. Metaverse 4: 16. [Google Scholar] [CrossRef]
  7. Jones, David, Chris Snider, Aydin Nassehi, Jason Yon, and Ben Hicks. 2020. Characterising the Digital Twin: A systematic literature review. CIRP Journal of Manufacturing Science and Technology 29: 36–52. [Google Scholar] [CrossRef]
  8. Kara, Ali, Alma Mintu-Wimsatt, and John E. Spillan. 2022. An application of the net promoter score in higher education. Journal of Marketing for Higher Education 34: 478–501. [Google Scholar] [CrossRef]
  9. Lewis, James R., and Jeff Sauro. 2009. The factor structure of the system usability scale. In Human Centered Design. Lecture Notes in Computer Science. Berlin/Heidelberg: Springer, pp. 94–103. [Google Scholar] [CrossRef]
  10. Manokeaw, Sattaya, Pattaraporn Khuwuthyakorn, Ying-Chieh Chan, Naruephorn Tengtrairat, Manissaward Jintapitak, Orawit Thinnukool, Chinnapat Buachart, Thepparit Sinthamrongruk, Thidarat Kridakorn Na Ayutthaya, Natee Suriyanon, and et al. 2025. A dynamic digital twin framework for sustainable facility management in a smart campus: A case study of Chiang Mai University. Technologies 13: 439. [Google Scholar] [CrossRef]
  11. Pyae, Aung, Werner Ravyse, Mika Luimula, Emiliana Pizarro-Lucas, Pedro L. Sanchez, Ignacio P. Dorado-Diaz, and Aung Khant Thaw. 2023. Exploring user experience and usability in a metaverse learning environment for students: A Usability Study of the Artificial Intelligence, Innovation, and Society (AIIS). Electronics 12: 4283. [Google Scholar] [CrossRef]
  12. Reichheld, F. Frederick. 2003. The One Number you Need to Grow. Harvard Business Review 81: 46–54. [Google Scholar] [PubMed]
  13. Sasmito, Ginanjar Wiro, La Ode Mohamad Zulfiqar, and M. Nishom. 2019. Usability Testing based on System Usability Scale and Net Promoter Score. Paper presented at 2019 International Seminar on Research of Information Technology and Intelligent Systems (ISRITI), Yogyakarta, Indonesia, December 5–6; pp. 540–45. [Google Scholar] [CrossRef]
  14. Schilling, Hannes, Britta Wittner, and Simone Kauffeld. 2022. Current Interventions for the digital onboarding of First-Year Students in Higher Education Institutions: A scoping review. Education Sciences 12: 551. [Google Scholar] [CrossRef]
  15. Schilling, Hannes, Stephanie Hirschberger, and Simone Kauffeld. 2024. A focus group study of students’ expectations of digital onboarding tools in higher education. Frontiers in Education 9: 1287387. [Google Scholar] [CrossRef]
  16. Sripan, Thanapat, and Pattarawat Jeerapattanatorn. 2025. Metaverse-based learning: A comprehensive review of current trends, challenges, and future implications. Contemporary Educational Technology 17: ep584. [Google Scholar] [CrossRef] [PubMed]
  17. Susilana, Rudi, Laksmi Dewi, and Gema Rullyana. 2024. Exploring student perceptions of virtual learning experiences: A study of the use of the Spatial.io 3D Metaverse platform. Journal of Education Technology 8: 673–83. [Google Scholar] [CrossRef]
  18. Tüzün, Hakan, and Fatih Özdinç. 2015. The effects of 3D multi-user virtual environments on freshmen university students’ conceptual and spatial learning and presence in departmental orientation. Computers & Education 94: 228–40. [Google Scholar] [CrossRef]
  19. Uribe, Valentina, Pablo Figueroa, and Vivian Gomez. 2024. The influence of metaverse environment design on the quality of experience in virtual reality classes: A comparative study. Frontiers in Education 9: 1451859. [Google Scholar] [CrossRef]
Figure 1. Dual-instrument evaluation framework for immersive orientation systems.
Figure 1. Dual-instrument evaluation framework for immersive orientation systems.
Socsci 15 00414 g001
Figure 2. Digital Twin Production Pipeline for Spatial.io Orientation.
Figure 2. Digital Twin Production Pipeline for Spatial.io Orientation.
Socsci 15 00414 g002
Figure 3. DTMO activities.
Figure 3. DTMO activities.
Socsci 15 00414 g003
Table 1. Demographic Profile of Participants (N = 30).
Table 1. Demographic Profile of Participants (N = 30).
CategoryItem/GroupFrequency (n)Percentage (%)
GenderFemale413.33
Male2686.67
Age GroupUnder 1813.33
18–201550.00
21 and older1446.67
Year of StudyFoundation26.67
Year 11550.00
Year 2620.00
Year 3723.33
FacultyFaculty of Computing & Informatics (FCI)1240.00
Faculty of Cinematic Arts (FCA)826.67
Faculty of Applied Communication (FAC)516.67
Faculty of Management (FOM)413.33
Faculty of AI and Engineering (FAIE)13.33
Prior 3D ExperienceNever310.00
Rarely826.67
Sometimes516.67
Often310.00
Very Often1136.67
Total N = 30100%
Table 2. Descriptive Statistics SUS.
Table 2. Descriptive Statistics SUS.
ScaleMinMaxMeanStd. DeviationCronbach’s Alpha
SUS65.00100.0086.839.330.673
Table 3. SUS Grade Classification and Adjective Rating Scale.
Table 3. SUS Grade Classification and Adjective Rating Scale.
GradeSUS Score RangeAdjective DescriptorAcceptability
A>80.3ExcellentAcceptable
B68.0–80.3GoodAcceptable
C51.0–67.9OKMarginal
D37.6–50.9PoorNot Acceptable
F<37.5AwfulNot Acceptable
Table 4. NPS Distribution and Qualitative Analysis: Thematic Coding Results.
Table 4. NPS Distribution and Qualitative Analysis: Thematic Coding Results.
NPS Categoryn%Top ThemesFrequency (n)% of Category
PROMOTERS (9–10)1963.33Engagement and Fun1368.4
Accessibility/Remote Access1157.9
Easy to Use/User-Friendly947.4
Spatial Familiarisation842.1
Novel/Innovative Experience631.6
PASSIVES (7–8)826.67Fun but Limited562.5
Technical Concerns450.0
Good for Specific Use Cases337.5
Cannot Replace Physical337.5
DETRACTORS (0–6)310.00Limited Functionality133.3
No Response Provided266.7
Table 5. SUS item-level descriptive statistics and internal consistency (N = 30).
Table 5. SUS item-level descriptive statistics and internal consistency (N = 30).
ItemMeanSDCorrected Item–Total CorrelationCronbach’s α If Item Deleted
SUS14.370.960.2540.672
SUS24.430.730.4390.628
SUS34.530.570.4530.633
SUS44.370.720.0530.698
SUS54.570.730.5030.616
SUS64.470.860.6160.582
SUS74.570.630.3020.655
SUS84.470.730.5190.612
SUS94.670.610.0410.694
SUS104.300.790.2260.671
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content.

Share and Cite

MDPI and ACS Style

Tan, J.-H.; Cheong, S.-N.; Wong, C.-O.; Mohamed, A.H.B. Usability and User Advocacy of a Digital Twin-Inspired Metaverse Orientation System: An Exploratory Pilot Study. Soc. Sci. 2026, 15, 414. https://doi.org/10.3390/socsci15070414

AMA Style

Tan J-H, Cheong S-N, Wong C-O, Mohamed AHB. Usability and User Advocacy of a Digital Twin-Inspired Metaverse Orientation System: An Exploratory Pilot Study. Social Sciences. 2026; 15(7):414. https://doi.org/10.3390/socsci15070414

Chicago/Turabian Style

Tan, Jia-Hui, Soon-Nyean Cheong, Chee-Onn Wong, and Ahmad Hishamuddin Bin Mohamed. 2026. "Usability and User Advocacy of a Digital Twin-Inspired Metaverse Orientation System: An Exploratory Pilot Study" Social Sciences 15, no. 7: 414. https://doi.org/10.3390/socsci15070414

APA Style

Tan, J.-H., Cheong, S.-N., Wong, C.-O., & Mohamed, A. H. B. (2026). Usability and User Advocacy of a Digital Twin-Inspired Metaverse Orientation System: An Exploratory Pilot Study. Social Sciences, 15(7), 414. https://doi.org/10.3390/socsci15070414

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop