Enhancing Safety Training Performance Using Extended Reality: A Hybrid Delphi–AHP Multi-Attribute Analysis in a Type-2 Fuzzy Environment

Abstract

Safety training effectively addresses the inexperience of and lack of knowledge among construction workers, which are some of the most significant contributors to workplace accidents on construction sites. This paper aims to understand the effectiveness of different extended reality (XR) technologies in imparting important construction safety training to construction workers in a virtual environment compared to conventional classroom training sessions. A group of experts were engaged to understand the most effective learning criteria and the impact of XR visualizations, and their responses were analysed using the interval type-2 fuzzy Delphi (IT2FD) method. Following this, a cohort of engineering students were subjected to construction safety training in traditional, augmented reality (AR) and virtual reality (VR) environments. Their feedback was collected using an online questionnaire and the responses were analysed using the interval type-2 fuzzy analytic hierarchy process (IT2F–AHP). The results revealed that addressing the virtual interface design of the training to maintain the attention of trainees and ensuring the virtual environment’s resemblance to the actual site and task were the most important factors in ensuring effective knowledge retention by the trainees. AR visualizations were most effective at imparting knowledge, and their interactive nature allowed trainees to retain the learned knowledge.

1. Introduction

The construction industry is among the highest contributors to work-related injuries and on-the-job fatalities among all major industries [1]. Being “struck by moving objects” and “fall from a height” are two of the most significant contributing causes towards incidents on construction sites [2,3]. In order to encourage safe practices and ensure safe working conditions for construction workers, it is necessary to increase their safety awareness through systematic safety training [4]. Current safety training sessions are delivered using traditional classroom lectures, textbook learning, and on-site safety briefings. These training methods are very broad in their approach and lack engagement, which results in the insufficient absorption of information by construction workers, which ultimately does not translate to long-term information retention. To facilitate better retention of safety concepts and their application on a work site, the safety training method must allow the workers to feel motivated [5].

Increased adoption of Construction 4.0 standards has enabled the use of immersive building information models (BIM), which can provide a base for the creation of site-specific safety training using immersive and interactive media, such as virtual reality (VR) and augmented reality (AR) head-mounted displays. Some recent studies have demonstrated measurable improvements in hazard identification and risk assessment among construction workers and heavy machinery operators using immersive technologies [6,7,8]. Several studies have tried to evaluate the delivery of immersion, interaction, and realism to demonstrate the ability of devices in the realm of extended reality (XR) to reproduce visuals and conditions akin to those of real construction sites [9,10,11,12]. Others that employ VR-based visualizations have evaluated the utility of this technology combined with the gamification of tasks to train heavy machinery operators [13,14]. However, all of these studies lack evaluations the effectiveness of these safety training sessions on the knowledge acquisition of construction workers, which ultimately translates into safe construction practice on construction sites.

Motivation is an essential factor in translating the concepts taught in training to the workplace: “Motivation refers broadly to what people desire, what they choose to do, and what they commit to do.” [15]. To understand the motivational outcome of a safety training session, it is necessary to evaluate the delivery of the four main components of the ARCS model—attention, relevance, confidence, and satisfaction [16]. The attention–relevance–confidence–satisfaction (ARCS) model is a robust evaluation technique employed for evaluating the motivational outcome of a training session or an instructional module. The ARCS model has been used by several studies that evaluate the criteria under investigation using the Instructional Materials Motivation Survey (IMMS) technique [15]. However, since the IMMS survey is training-centric, outcomes from a study employing the IMMS survey are focused on the instructions or material of the training rather than on the medium employed to deliver the training. Therefore, a novel approach has been proposed in this paper to evaluate knowledge acquisition delivered via each medium of training with the same content or material used for the safety training of construction workers.

This study aims to contribute to the existing body of knowledge by analysing the impact of XR-based training methods versus conventional training methods on the knowledge acquisition of construction workers by answering the following research questions:

• What are the most influential criteria for evaluating a trainee’s knowledge acquisition towards applying safety measures in their workplace? This question aims to understand the factors influencing the effectiveness of construction safety training. The effectiveness of training can be measured by understanding knowledge acquisition and retention among trainees. Insights gained from this understanding can help in designing highly effective construction safety training.
• To what extent do Extended Reality (XR) training methods impact criteria important for knowledge acquisition among trainees compared to traditional training methods? This question explores the limitations and advantages of using emerging technologies within the realm of XR visualizations for task information delivery. To cover technologies spanning the XR spectrum, both the extreme ends of the spectrum—Virtual Reality (VR) and Augmented Reality (AR)—have been addressed in this study. Understanding these technologies’ usability and limitations can enable the development of user-centric safety training sessions aimed at enhancing the performance of construction workers in a particular construction task.
• How does the decision of modelling criteria weight priorities change the outcome of the multi-attribute decision-making analysis? This question explores the robustness of the applied methodology in analysing the effectiveness of different criteria and training methodologies on the knowledge acquisition of trainees. Since the findings of this study depend on the method of the analysis employed, it is important to understand if the modeler’s decision to apportion different priority weights than those applied in this study would change the study’s outcome.

2. Literature Review

The enhanced ability of construction workers to understand and recognize hazards is fundamental for a high safety performance on construction sites. It has been demonstrated that unrecognized hazards can lead to unexpected injuries or even fatalities in a fast-paced construction environment [17,18]. Multiple studies have suggested the employment of safety training sessions to improve this aspect of workers’ safety performance. A study exploring the reasons behind unsafe work behavior on construction sites describes the lack of safety orientation and training as one of the leading causes of unsafe behavior [19]. Another study demonstrated improvements in safety performance with increased safety discussions and the distribution of safety training materials to construction workers [20]. Improvement in learning performance is an important measure of effectiveness for any kind of training [21]. It is necessary to understand the extent of the knowledge acquired, which in this situation, translates to the effective application of safety practices on construction sites, as the safety training of construction workers is directly correlated with better safety performance on construction sites.

2.1. Knowledge Acquisition and Application towards Safe Working Practices

The importance of acquired knowledge and motivation in the safety performance of an individual has been established by Neal et al. [22]. In their study, they demonstrated that for the design of effective safety training, it is important to target an employee’s knowledge and motivation. However, motivation was found to be the more effective factor for safety performance practice on the job. Motivation has also been recognised as an essential driver for implementing acquired knowledge by J.M. Keller [15]. Many studies have utilised the Instructional Materials Motivation Survey (IMMS) devised to analyse the four main factors—attention, relevance, confidence, and satisfaction (ARCS)—that form the basis of the motivation gained from an instructional discourse.

The ARCS model has been identified as an essential and effective motivational model frequently used for evaluating serious games in education and training [23]. A systematic review by Krath et al. identified the ARCS motivational model as helpful in explaining the effect of gamification in education and training. The ARCS model has been used to evaluate the effectiveness of educational training programs in multiple studies. Many researchers have developed extended reality (XR)-based training systems and evaluated their contribution to learning using the ARCS model. A system for imparting elementary science education to primary students was developed by Su et al. [24]. They evaluated the relationship between learning achievement and motivation and found a positive relationship when mobile-based gamification was used for imparting knowledge to students. In a similar study, the effect of gamification on the learning and performance of trainees was studied by Wu et al. [25]. They proposed a learning framework based on Bloom’s technology for education in STEAM (Science, Technology, Engineering, Arts, and Math) fields. They employed the technological acceptance method (TAM) in conjunction with the ARCS model to assess motivation and learning. The use of game-based learning for improved motivation and academic performance among engineering students was studied by Pando Cerra et al. using the ARCS model [26]. With the employment of XR games for the visualization of complex engineering concepts, game-based learning is seeing increased application in training workers across multiple industries.

The architecture, engineering, and construction (AEC) industries have rapidly evolved in terms of construction methods to become efficient. However, there is a need for effective training methods that allow workers to work safely on construction sites. In this context, some notable studies have focused on the training and education of workers and employed the ARCS model for the evaluation of outcomes. A motivational model-based virtual reality approach was used to evaluate the training effectiveness of Spherical video-based virtual reality in professional safety training by Hwang et al. [27]. The ARCS model was also used to evaluate the effectiveness of video lessons based on an XR platform for safety management education by Yang et al. [28]. This study demonstrated the effectiveness of XR technologies in increasing motivation toward learning and suggested improvements to the design of VR/MR educational applications. Similarly, teaching model integration with VR technology was studied by Hou et al. for improvements in effective communication for education and learning in the AEC industries [29].

2.2. Implementation of XR-Based Training in AEC Industry

Building information models (BIM) and other 3D visualizations have gained traction with the broader acceptance of the Building 4.0 platform in the AEC industry. Engineering design, construction management, project management, and operational management activities are some of the more critical areas of a building’s life cycle that have benefitted from BIM-based visualizations. XR-based visualizations for enhancing safety during the construction phase of a building have mainly been the focus of various studies in recent years [30,31]. VR-based visualizations based on BIM models were used by Shringi et al. to demonstrate a VR-based safety training platform to train tower crane operators to install prefabricated modules in building construction [6]. A VR-based virtual environment depicting a workshop was used by Teizer et al. for the safety training of ironworkers [32]. They used real-time location tracking for hazards along with 3D immersive visualizations for training ironworkers to identify hazards better and increase their productivity. In another study, a BIM was used by Park et al. to create VR and AR visualizations for training workers on-site to identify fall hazards [33]. VR visualizations with a VR head-mounted display (VR-HMD) were used by Li et al. to train workers to avoid “struck-by” and fall hazards while installing precast façade panels [34]. Virtual reality visualizations were also used by Hou et al. for training maintenance workers in an industrial facility [35]. The study demonstrated increased productivity in maintenance tasks with better safety practices implemented. In another novel study, the walking pattern of construction workers was studied by Shi et al. using a VR-HMD to assess their fall risk behaviour before and after safety training [36]. In a similar study, a VR-HMD was used by Habibnezhad et al. for a gait analysis of ironworkers to evaluate their risk of falling from height based on their gait patterns [37].

Another critical area that has benefited from XR-based safety training sessions is heavy machinery operations. A machinery operator’s knowledge acquisition from VR-based training is an important factor that determines on-site safety performance and has been evaluated by multiple studies using XR-based visualizations [38]. The training efficiency of a VR system was analysed by Guo et al. [39] for tower crane dismantlement safety training with the use of a physics-based game engine, while the safety performance of a hook attachment to load during tower crane lift operations was studied using VR by Li et al. [40] in a similar manner. A VR-based serious game was also used by Choi et al. to evaluate operators’ situational awareness during forklift operations [41]. The effectiveness of VR-based operator training for robotic teleoperation was evaluated by Adami et al. by evaluating the knowledge acquisition of trainees in operating a demolition robot [42]. Since high-fidelity virtual environments can closely mimic conditions on real construction sites, operators’ abilities to timely identify safety hazards and take evasive action using virtual construction environments have been evaluated in multiple studies [43,44]. The hazard recognition ability of experienced and novice construction workers was studied by Dzeng et al. in a virtual environment using a head-mounted eye tracker [45], while the role of virtual safety training in enhancing risk recognition and improving risk perceptions was studied by Namian et al. [46]. The importance of using VR-HMDs for enhancing depth perception among novice heavy machinery operators was studied by Shringi et al. [47]. Similarly, the change in hazard recognition ability of construction workers post virtual safety training has also been evaluated in multiple studies [48,49,50].

Table 1. Notable safety training studies in the XR realm.

Reference	Area of Application	Experiment Setup	Visualization Technology	Factors Analysed	Data Collection	Analysis Methods
[8]	Hazard recognition performance	High-risk brown-field construction work	VR HMD	Hazard Identification, safety behaviour	System feedback, survey (in gameplay)	Descriptive statistics
[11]	Hazard identification	Construction worksite	VR (LCD)	Presence, Hazard Identification	System feedback, Survey	Nonparametric Mann-Whitney U and Kruskal-Wallis, in conjunction with Dunn post hoc tests
[28]	Incident Investigation		VR (LCD) and Traditional (Recorded video)	Perceived Effectiveness, Motivation (ARCS)	Survey	Descriptive statistics (t-test)
[30]	Impact of VR training	Roofing tasks	VR HMD	Accident causality, VR Effectiveness and VR Engagement	In-game survey	Descriptive statistics
[31]	Learning effectiveness of VR training	Safety training retention	VR HMD	Effectiveness of training based on experience, personality and job position	System feedback (in gameplay)	Descriptive statistics (MANOVA)
[32]	Safety training in workshop environment	Assembly of steel structures in workshop	VR without Headset	Worker Productivity	Survey and system	Simple statistical analysis
[33]	On-site worker training	Fall from height	VR (BIM), AR	Hazard Identification	Survey	Statistical Average of Q&A Performance
[34]	On-site work training	Struck by and fall hazards in precast façade installation	VR Headset	Training efficiency (Time)	System feedback	Scoring comparison + simple statistical analysis of likert scale feedback
[35]	Productivity training	LNG plant maintenance	VR and AR	Productivity and Learning	System feedback, Survey	t-test
[36]	Walking behaviour on site	Plank across buildings	VR headset + Kinect	Safety behaviour	System feedback	ANOVA
[37]	Fall from height (Risk evaluation)	Gait analysis of Ironworkers	VR HMD	Postural Control for fall analysis	Motion trackers	Descriptive statistics (t-test, Mann-whitney U)
[38]	Hazard recognition performance	Construction worksite	VR HMD	Hazard Identification	System feedback (in gameplay)	Descriptive statistics (t-test)
[39]	Tower crane dismantlement	Tower crane dismantlement	VR Headset + Keyboard + Nintendo Wii	Training efficiency	System feedback	Scoring comparison + simple statistical analysis of likert scale feedback
[40]	Tower crane lifting operation	Hook attachment to load	VR (LCD) + Keyboard + Nintendo Wii	Safety Performance	System feedback	Descriptive statistics
[41]	Forklift operations	Forklift operations	VR HMD	Situational Awareness	System feedback, survey	ANOVA
[42]	Machinery operation	Demolition robot operation	VR HMD	Knowledge acquisition, safety behaviour & operational skills	System feedback with controller and VR treadmill, survey	Descriptive statistics (ANOVA), Grounded theory (Qualitative)
[43]	Hazard Identification in VR	Industrial operations (Maintenance and Construction)	VR	Hazard Recognition	System feedback	t-test of HR Index
[44]	Hazard Identification in VR	Multiple hazards	VR withour Headset	Hazard Identification accuracy	System feedback	ANOVA
[45]	Hazard Identification in VR	Multiple hazards	VR HMD + Eye Tracker	Hazard Identification accuracy, miss rate, and time	System feedback	Simple statistical analysis
[46]	Hazard identification	-	-	Hazard identification and Safety risk perception	Surveys	t-test of HR Index
[47]	Hazard identification	Construction worksite	VR (LCD)	Usability and Realism, Hazard Identification	System feedback, Survey	Descriptive statistics
[48]	Safety Management	Safety manager training	VR LCD	Hazard Identification	User feedback for “Use Case” development	Observational analysis
[49]	Safety Education	Basic occupational safety and health-related information	VR LCD	Effectiveness of safety training	Survey	ANOVA

lists the analysis methods and target areas for some of the key studies focusing on the implementation of XR-based training.

2.3. Multi-Attribute Decision-Making and Interval Type-2 Fuzzy Sets

The concept of fuzzy sets was introduced by Zadeh in 1965 to deal with the vagueness in the membership of a given function [51]. In contrast to classical set theory, fuzzy sets do not have a bivalent membership function. Instead, fuzzy sets allow for membership within an interval between two real numbers to cater to vagueness. This allows for a gradual assessment of a variable’s membership within the interval. Fuzzy sets are used in decision-making and linguistic research in which their application allows researchers to convert the linguistic terms or responses to be evaluated with a degree of certainty.

As an extension to the concepts of fuzzy sets and to cater to another dimension of uncertainty, type-2 fuzzy sets were introduced by Zadeh in 1975 [52]. Type-2 fuzzy sets have a third dimension in the form of upper and lower membership degrees representing uncertainty. However, owing to complexities in the calculations involving type-2 fuzzy sets, a special case of generalized type-2 fuzzy sets, named interval type-2 fuzzy (IT2F) sets, has seen more application since IT2F sets are more manageable in terms of calculations [53]. IT2F sets have been successfully implemented with multi-criteria decision-making (MCDM) methods involving responses from participants based on a linguistic scale. The essential definitions and mathematical operations of IT2F sets are detailed in Section 3.1.

Multi-criteria decision-making (MCDM) methods have seen widespread application in research spanning social sciences, engineering, medicine, and many other areas. MCDM methods are employed to arrive at an optimum solution for a given situation or problem. The analytic hierarchy process (AHP) and the Delphi method are some of the more commonly used methods that have been used for solving decision-making problems in the field of safety. A fuzzy AHP and fuzzy TOPSIS were used by Abbasinia et al. for analysing the causes of accidents originating from causes related to governance, environmental, and individual factors [54]. In contrast, the same methods were used for a five-dimensional safety risk assessment for ranking existing project safety risks by Mohandes et al. [55]. A fuzzy AHP and fuzzy TOPSIS were also employed for the assessment of workplace conditions and ranking of the safety performance of multiple construction companies by Basahel et al. [56]. A fuzzy rough number AHP and VIKOR have been used to consider uncertainty in a risk assessment of construction hazards based on ranking failure modes by Zhu et al. [57]. A fuzzy AHP was used for evaluating the efficacy of VR devices for virtual fire evacuation training in a building evacuation scenario by Bourhim et al. [58]. A fuzzy AHP was also used for equipment selection and risk rating on construction sites in similar studies [59,60,61].

IT2F sets were implemented for the development of safety risk analysis methodology using a Bayesian network for determining occupational risk in manually executed industrial processes by Yasli et al. [62]. IT2F sets were used with an AHP analysis to evaluate the effectiveness of training imparted to electric overhead traveling (EOT) crane operators by Dhalmahapatra et al. [63]. In contrast, an IT2F–AHP was used for the evaluation of a VR mine safety training system in a similar study by Zhang et al. [12].

The Delphi method was introduced by Dalkey et al. [64] in 1969 and is a popular method used in various fields for group decision making. Karsak proposed the fuzzy Delphi method for prioritizing factors important in determining customer requirements [65]. A fuzzy Delphi was used by Li et al. for evaluating customer requirements in open design [66] while the fuzzy Delphi method was used by Ma et al. for evaluating the perception of safety experts to derive safety performance indicators [67]. Similarly, in the safety and risk evaluation field, the fuzzy Delphi method was used along with the analytic network process (ANP) for the risk analysis of tunnelling works by Liu et al. [68]. All of these above-described studies are tabulated in

Table 2. Safety studies employing fuzzy analysis methods.

Reference	Context of Application	Analysis Methods
[12]	Evaluation of VR mine safety training system by ranking the components of the system using interval type-2 fuzzy AHP.	Interval Type-2 Fuzzy AHP
[54]	Analysing the causes of accidents by selecting and ranking the causes grouped under governance, environmental, and individual factors.	Fuzzy AHP, Fuzzy TOPSIS
[55]	Five-dimensional safety risk assessment for evaluating project safety risks using fuzzy AHP and fuzzy TOPSIS.	Fuzzy AHP, Fuzzy TOPSIS
[56]	Use of fuzzy AHP and fuzzy TOPSIS for the assessment of workplace safety conditions.	Fuzzy AHP, Fuzzy TOPSIS
[57]	Ranking failure modes for risk assessment based on a failure mode effects analysis with fuzzy rough numbers and VIKOR.	Fuzzy rough number VIKOR
[58]	Evaluation of system and user interfaces of virtual fire evacuation training using a fuzzy AHP analysis	Fuzzy AHP
[59]	Using a fuzzy Delphi and fuzzy BWM for construction workers’ risk assessments	Fuzzy Delphi, Fuzzy BWM
[60]	Development of a project hazard index using fuzzy AHP for a construction safety evaluation based on the hazardous trades involved.	Fuzzy AHP
[61]	Use of a fuzzy AHP to rank equipment based on selected criteria for appropriate loader selection	Fuzzy AHP
[62]	The development of occupational safety risk analysis methodology using a Bayesian network and interval yype-2 fuzzy sets.	Interval Type-2 Fuzzy Analysis
[63]	The assessment of VR simulator effectiveness by a fuzzy analysis of presence, usability, and sickness during virtual training.	Interval Type-2 Fuzzy AHP
[66]	Using a fuzzy Delphi to evaluate customers’ requirements in open design.	Fuzzy Delphi
[67]	The evaluation of safety perception of experts using a fuzzy Delphi to derive safety performance indicators.	Fuzzy Delphi
[68]	Use of a fuzzy ANP for the risk evaluation of tunnelling project.	Fuzzy ANP

for ease of reference.

3. Methodology

This study compares knowledge retention among construction workers and trainees based on three different approaches: VR-based immersive training, AR-based immersive training, and traditional training involving the presentation of content on overhead projected slideshows. Training content was based on an Australian industrial standard—Preventing Falls in Housing Construction—Code of Practice [69]. The safety team prepared an immersive environment for demonstration of concepts using Unity3D [70] to simulate the safety risks, their outcomes, and applicable safety protection devices. Figure 1 describes the workflow of the study from the selection of criteria to the outcome of their analysis. As described in Figure 1, this study is divided into four major stages. The first stage involved the selection of criteria based on the IMMS questionnaire for analysing trainees’ knowledge retention from the different types of training. In stage 2, the derived criteria from the IMMS questionnaire and ARCS model [15] were further distilled using interval type-2 fuzzy Delphi method to arrive at the final set of criteria to be analysed during stages 3 and 4. After obtaining the final set of criteria and their sub-criteria, construction safety training was administered to the population of trainees in stage 3, and their inputs on knowledge retention were collected by using a post-survey questionnaire. Finally, in Stage 4, participant responses were analysed using interval type-2 fuzzy AHP to arrive at the most critical criteria that should be incorporated in safety training to enhance knowledge retention. This knowledge retention can then help improve construction workers’ safety performance and reduce workplace incidents on construction sites.

3.1. Interval Type-2 Fuzzy Sets

Interval type-2 fuzzy (IT2F) sets were introduced by Zadeh as an extension to classical fuzzy sets in order to deal with uncertainty along with vagueness more accurately [52,53]. Some of the definitions of IT2F sets are given in this section.

Definition 1. 

Interval Type-2 Fuzzy Set.

An IT2F set, $\stackrel{\approx }{\mathrm{A}}$, in the universe of discourse $X$ can be represented by a type-2 membership function ${\mathsf{\mu }}_{\stackrel{\approx }{\mathrm{A}}}$ as defined below:

$$\stackrel{\approx }{\mathrm{A}}=\left\{\left(\left(\mathrm{x},\mathrm{u}\right),{\mathsf{\mu }}_{\stackrel{\approx }{\mathrm{A}}}\left(\mathrm{x},\mathrm{u}\right)\right)|\forall \mathrm{x}\in \mathrm{X},\forall \mathrm{u}\in {\mathrm{J}}_{\mathrm{x}}\subseteq \left[0,1\right],0\le {\mathsf{\mu }}_{\stackrel{\approx }{\mathrm{A}}}\left(\mathrm{x},\mathrm{u}\right)\le 1\right\}$$

where $J_x$ denotes an interval [0,1]. The type-2 fuzzy set can also be represented as follows:

$$\stackrel{\approx }{\mathrm{A}}={{\displaystyle \iint }}_{x\in X, u\in J_x}{\mathsf{\mu }}_{\stackrel{\approx }{\mathrm{A}}}\left(x,u\right)/\left(x,u\right)$$

where $J_x\subseteq \left[0,1\right]$ and ${\displaystyle \iint }$ denotes union over all admissible x and u.

Let $\stackrel{\approx }{\mathrm{A}}$ be a type-2 fuzzy set in the universe of discourse $X$ represented by the membership function ${\mathsf{\mu }}_{\stackrel{\approx }{\mathrm{A}}}$. If all ${\mathsf{\mu }}_{\stackrel{\approx }{\mathrm{A}}}\left(x,u\right)=1$, then $\stackrel{\approx }{\mathrm{A}}$ is called an interval type-2 fuzzy set. An interval type-2 fuzzy set is a special kind of type-2 fuzzy set and can be represented as follows:

$$\stackrel{\approx }{\mathrm{A}}={{\displaystyle \iint }}_{x\in X, u\in J_x}1/\left(x,u\right), J_x\subseteq \left[0,1\right]$$

The upper and lower membership functions of an IT2F set are type-1 membership functions.

Considering the nature of safety incidents, to bring more certainty to outcome of participant response analysis, authors have employed trapezoidal IT2F sets for this study. Further details are presented in the following definition.

Definition 2. 

Trapezoidal Interval Type-2 Fuzzy Set.

A trapezoidal IT2F set is denoted as:

$${\stackrel{\approx }{\mathrm{A}}}_i=({\tilde{\mathrm{A}}}_i^U,{\tilde{\mathrm{A}}}_i^L)=\left(a_{i1}^U,a_{i2}^U,a_{i3}^U,a_{i4}^U;H_1({\tilde{\mathrm{A}}}_i^U),H_2({\tilde{\mathrm{A}}}_i^U)\right),\left(a_{i1}^L,a_{i2}^L,a_{i3}^L,a_{i4}^L;H_1({\tilde{\mathrm{A}}}_i^L),H_2({\tilde{\mathrm{A}}}_i^L)\right)$$

where $H_j({\tilde{\mathrm{A}}}_i^U)$ denotes the membership value of the element $a_{i\left(j+1\right)}^U$ in the upper trapezoidal membership function ${\tilde{\mathrm{A}}}_i^U$, 1 ≤ j ≤ 2, and $H_j({\tilde{\mathrm{A}}}_i^L)$ denotes the membership value of the element $a_{i\left(j+1\right)}^L$ in the lower trapezoidal membership function ${\tilde{\mathrm{A}}}_i^L$, 1 ≤ j ≤ 2, $H_1({\tilde{\mathrm{A}}}_i^U)\in \left[0,1\right]$, $H_2({\tilde{\mathrm{A}}}_i^U)\in \left[0,1\right]$, $H_1({\tilde{\mathrm{A}}}_i^L)\in \left[0,1\right]$, $H_2({\tilde{\mathrm{A}}}_i^L)\in \left[0,1\right]$ and 1 ≤ i ≤ n. Graphical representation of the membership function of a generic trapezoidal IT2F set is demonstrated in Figure 2.

Definition 3. 

Addition of Trapezoidal Interval Type-2 Fuzzy Sets.

Let ${\stackrel{\approx }{\mathrm{A}}}_1$ and ${\stackrel{\approx }{\mathrm{A}}}_2$ be two trapezoidal IT2F sets such that

$${\stackrel{\approx }{\mathrm{A}}}_1=({\tilde{\mathrm{A}}}_1^U,{\tilde{\mathrm{A}}}_1^L)=\left(a_{11}^U,a_{12}^U,a_{13}^U,a_{14}^U;H_1({\tilde{\mathrm{A}}}_1^U),H_2({\tilde{\mathrm{A}}}_1^U)\right),\left(a_{11}^L,a_{12}^L,a_{13}^L,a_{14}^L;H_1({\tilde{\mathrm{A}}}_1^L),H_2({\tilde{\mathrm{A}}}_1^L)\right)$$

and

$${\stackrel{\approx }{\mathrm{A}}}_2=({\tilde{\mathrm{A}}}_2^U,{\tilde{\mathrm{A}}}_2^L)=\left(a_{21}^U,a_{22}^U,a_{23}^U,a_{24}^U;H_1({\tilde{\mathrm{A}}}_2^U),H_2({\tilde{\mathrm{A}}}_2^U)\right),\left(a_{21}^L,a_{22}^L,a_{23}^L,a_{24}^L;H_1({\tilde{\mathrm{A}}}_2^L),H_2({\tilde{\mathrm{A}}}_2^L)\right)$$

The addition operation between the two trapezoidal IT2F is defined as follows:

$$\begin{array}{c}{\stackrel{\approx }{A}}_1\oplus {\stackrel{\approx }{A}}_1=({\tilde{A}}_1^U,{\tilde{A}}_1^L)\oplus ({\tilde{A}}_2^U,{\tilde{A}}_2^L)=[a_{11}^U+a_{21}^U,a_{12}^U+a_{22}^U,a_{13}^U+a_{23}^U,a_{14}^U+\\ a_{24}^U;min(H_1({\tilde{A}}_1^U),H_1({\tilde{A}}_2^U)),min(H_2({\tilde{A}}_1^U),H_2({\tilde{A}}_2^U))],[a_{11}^L+a_{21}^L,a_{12}^L+a_{22}^L,a_{13}^L+a_{23}^L,a_{14}^L+\\ a_{24}^L;min(H_1({\tilde{A}}_1^L),H_1({\tilde{A}}_2^L)),min(H_2({\tilde{A}}_1^L),H_2({\tilde{A}}_2^L))]\end{array}$$

Definition 4. 

Subtraction of Trapezoidal Interval Type-2 Fuzzy Sets.

Similar to the addition operation, the subtraction operation between trapezoidal IT2F sets can be defined as follows:

$$\begin{array}{c}{\stackrel{\approx }{A}}_1\ominus {\stackrel{\approx }{A}}_1=({\tilde{A}}_1^U,{\tilde{A}}_1^L)\ominus ({\tilde{A}}_2^U,{\tilde{A}}_2^L)=[a_{11}^U-a_{21}^U,a_{12}^U-a_{22}^U,a_{13}^U-a_{23}^U,a_{14}^U-\\ a_{24}^U;min(H_1({\tilde{A}}_1^U),H_1({\tilde{A}}_2^U)),min(H_2({\tilde{A}}_1^U),H_2({\tilde{A}}_2^U))],[a_{11}^L-a_{21}^L,a_{12}^L-a_{22}^L,a_{13}^L-a_{23}^L,a_{14}^L-\\ a_{24}^L;min(H_1({\tilde{A}}_1^L),H_1({\tilde{A}}_2^L)),min(H_2({\tilde{A}}_1^L),H_2({\tilde{A}}_2^L))]\end{array}$$

Definition 5. 

Multiplication of Trapezoidal Interval Type-2 Fuzzy Sets.

The multiplication operation between trapezoidal IT2F sets can be defined as follows:

$$\begin{array}{c}{\stackrel{\approx }{A}}_1\otimes {\stackrel{\approx }{A}}_1=({\tilde{A}}_1^U,{\tilde{A}}_1^L)\otimes ({\tilde{A}}_2^U,{\tilde{A}}_2^L)=[a_{11}^U\times a_{21}^U,a_{12}^U\times a_{22}^U,a_{13}^U\times a_{23}^U,a_{14}^U\times \\ a_{24}^U;min(H_1({\tilde{A}}_1^U),H_1({\tilde{A}}_2^U)),min(H_2({\tilde{A}}_1^U),H_2({\tilde{A}}_2^U))],[a_{11}^L\times a_{21}^L,a_{12}^L\times a_{22}^L,a_{13}^L\times a_{23}^L,a_{14}^L\times \\ a_{24}^L;min(H_1({\tilde{A}}_1^L),H_1({\tilde{A}}_2^L)),min(H_2({\tilde{A}}_1^L),H_2({\tilde{A}}_2^L))]\end{array}$$

Definition 6. 

Arithmetic operation between scalar k and a trapezoidal IT2F set ${\stackrel{\approx }{A}}_1$.

$$k{\stackrel{\approx }{\mathrm{A}}}_i=(k\times a_{i1}^U,k\times a_{i2}^U,k\times a_{i3}^U,k\times a_{i4}^U;H_1({\tilde{\mathrm{A}}}_i^U),H_2({\tilde{\mathrm{A}}}_i^U)),\left(k\times a_{i1}^L,k\times a_{i2}^L,k\times a_{i3}^L,k\times a_{i4}^L;H_1({\tilde{\mathrm{A}}}_i^L),H_2({\tilde{\mathrm{A}}}_i^L)\right)$$

$${\tilde{\tilde{A}}}_i/k=\left(\frac{a_{i1}^U}{k},\frac{a_{i2}^U}{k},\frac{a_{i3}^U}{k},\frac{a_{i4}^U}{k};H_1({\tilde{A}}_i^U),H_2({\tilde{A}}_i^U)\right),\left(\frac{a_{i1}^L}{k},\frac{a_{i2}^L}{k},\frac{a_{i3}^L}{k},\frac{a_{i4}^L}{k};H_1({\tilde{A}}_i^L),H_2({\tilde{A}}_i^L)\right)$$

This paper employs a methodological approach employing fuzzy AHP with interval type-2 fuzzy sets as described by Kahraman et al. [71] to deal with ambiguity in participant responses. A five-level Likert scale using specific linguistic terms was used to gather participant responses in the questionnaire. IT2F sets representing the linguistic terms used in this study are tabulated in

Table 3. Mathematical representation of the linguistic response scale used in the study.

Linguistic Variables	Trapezoidal Interval Type-2 Fuzzy Scales
Absolutely Strong (AS)	((7, 8, 9, 9; 1, 1), (7.2, 8.2, 8.8, 9; 0.8, 0.8))
Very Strong (VS)	((5, 6, 8, 9; 1, 1), (5.2, 6.2, 7.8, 8.8; 0.8, 0.8))
Fairly Strong (FS)	((3, 4, 6, 7; 1, 1), (3.2, 4.2, 5.8, 6.8; 0.8, 0.8))
Slightly Strong (SS)	((1, 2, 4, 5; 1, 1), (1.2, 2.2, 3.8, 4.8; 0.8, 0.8))
Exactly Equal (E)	((1, 1, 1, 1; 1, 1), (1, 1, 1, 1; 1, 1))
If factor i has one of the above linguistic variables assigned to it when compared with factor j, then j has the reciprocal value compared with i.	Reciprocals of above

and graphically presented in Figure 3.

3.2. Criteria Selection

Selection of appropriate criteria based on the criteria of construction safety training development—namely, applicability, demonstrability, and visualization—was the first stage of the study catering to research question 1. A cohort of 8 identified experts with expertise in construction safety training and implementation were engaged for the structured interview to identify the criteria for this study. Four experts had a background in administering training to construction workers and trainees. In contrast, two of the experts on the panel had substantial experience with designing of immersive safety training, while two of the experts had backgrounds in researching construction safety. Based on consultation with the experts, ARCS model of motivation was chosen as the appropriate model for defining criteria to understand the difference in the effectiveness of training methodologies. A standard survey, IMMS questionnaire, based on the ARCS model has been developed by Keller to measure the motivation of participants in an instructional study. This questionnaire has 36 questions—12 questions related to attention, 9 questions related to relevance, nine questions related to confidence and 6 questions related to satisfaction. Keywords defining key sub-criteria for each of the 36 questions were detailed out to create a pool of sub-criteria under each of the four main criteria—attention, relevance, confidence, and satisfaction, as shown in

Table 4. Sub-criteria based on IMMS questionnaire.

Attention
IMMS Factor	Attention Related Questions	Keyword (Sub Criterion)	Unique/ Duplicate
02A01	Use of extended reality application for this exercise got my attention	Use	Unique
08A02	XR app made the lesson more engaging	Engagement	Unique
11A03	The user interface in XR app allowed me to maintain attention	User Interface	Unique
12A04	The activity was so abstract that it was hard to keep my attention on it. (−)	Detail	Unique
15A05	The interface of this app makes it dry and appealing. (−)	User Interface	Duplicate
17A06	Presentation of content in virtual environment helped me maintain attention.	Presentation	Unique
20A07	The XR app elements stimulated my curiosity while performing the exercise.	Stimulation of Curiosity	Unique
22A08	The amount of repetition in this activity caused me to get bored sometimes. (−)	Repetition	Duplicate
24A09	I learned some things that were surprising or unexpected.	Newness	Duplicate
28A10	The variety of content helped keep my attention on the XR app.	Variety of Content	Unique
29A11	The presentation of holograms, 3D models, texts and images is boring. (−)	Presentation	Duplicate
31A12	The presentation of content in XR app is overlapping and irritating. (−)	Presentation	Duplicate
Relevance
IMMS Factor	Relevance related Questions	Keyword (Sub criterion)	Unique/ Duplicate
06R01	My knowledge of construction safety was relevant in performing this exercise using XR app.	Knowledge	Unique
09R02	The visuals in XR app showed me how this material could be important to students studying construction safety	Visual content	Unique
10R03	Completing this lesson successfully was important to me.	Successful Completion	Unique
16R04	The content of material presented is relevant to my interest in learning construction safety.	Updated Content	Unique
18R05	The exercises performed using XR app represent practical situations	Practicality	Unique
23R06	Content shown in the virtual environment with the XR app conveys the impression that it is worth lerarning construction safety.	Practicality	Duplicate
26R07	This exercise was not relevant to my needs because I already knew most of it. (−)	Knowledge	Duplicate
30R08	I could relate the content of exercises in XR app to things I have seen, done, or thought about in my activities outside this class.	Practicality	Duplicate
33R09	The content of this lesson will be useful to me.	Usability	Unique
Confidence
IMMS Factor	Confidence-related Questions	Keyword (Sub criterion)	Unique/ Duplicate
01C01	When I first saw the XR application, I had the impression that it would be easy for me to understand	Ease of Understanding	Unique
03C02	Information presented through XR application was more difficult to understand than I would like for it to be. (−)	Ease of Understanding	Duplicate
04C03	After the initial introduction, I felt confident that I knew what I was supposed to learn from this lesson	Practicality	Duplicate
07C04	The activities in XR app had so much information that it was hard to pick out and remember the important points. (−)	Conciseness	Unique
13C05	While performing exercises in the XR app, I was confident I could learn the content.	Ease of Understanding	Duplicate
19C06	The exercises performed using XR app were too difficult. (−)	Ease of Understanding	Duplicate
25C07	After working on this exercise in XR app for a while, I was confident that I would be able to pass a test on it.	Repetition	Unique
34C08	I could not understand a significant amount of the material in this lesson. (−)	Ease of Understanding	Duplicate
35C09	The proper organization of the visualizations helped me be confident that I would learn this material.	Visualization	Unique
Satisfaction
IMMS Factor	Satisfaction related Questions	Keyword(Sub criterion)	Unique/Duplicate
05S01	Completing the exercises in this lesson gave me a satisfying feeling of accomplishment.	Completion of Activity	Unique
14S02	I enjoyed the exercise so much that I would like to know more about construction safety	Additional exploration	Unique
21S03	I enjoyed learning about construction safety with XR app	Enjoyment	Unique
27S04	The feedback received after each exercise helped me feel rewarded for my effort.	Feedback	Unique
32S05	It felt good to complete given exercises in XR app.	Pleasure	Unique
36S06	It was a pleasure to work on such a well-designed lesson.	Activity Design	Unique

. After purging duplicates, the total number of sub-criteria was refined from 36 to 23 unique sub-criteria.

3.3. Interval Type-2 Fuzzy Delphi (IT2FD)

The selection of appropriate sub-criteria under each of the main criteria is important in further refining the answer to the first research question. For this, the study employs the interval type-2 fuzzy Delphi (IT2FD) technique to investigate the importance of each unique sub-factor identified in Table 4. The fuzzy Delphi method (FDM) is a modified form of the classical Delphi technique that adopts fuzzy sets for processing information obtained from subject matter expert (SME) responses to select criteria. Owing to the clarity of outcomes obtained from the use of IT2F sets, IT2FD technique was used to gather responses from the same cohort of 8 experts as described earlier in Section 3.2.

In this stage of the study, the authors engaged the panel of experts and gathered their opinions and responses for the selection of appropriate sub-criteria. Using the selected criteria and sub-criteria, a hierarchy was created to evaluate knowledge retention among trainees. A total of three rounds of structured interviews were conducted with the expert panel and their feedback was analysed to arrive at the final set of sub-criteria. The steps followed for the analysis of responses from experts are detailed below:

• Step 1: Significance value of sth sub-criteria as given by ith expert from the cohort of n experts

Let ${\stackrel{\approx }{\mathrm{A}}}_s$ be the IT2F set representing the significant value of the sth sub-criteria as represented in the following equation:

$${\stackrel{\approx }{\mathrm{A}}}_s=({\tilde{\mathrm{A}}}_s^U,{\tilde{\mathrm{A}}}_s^L)=\left(a_{s1}^U,a_{s2}^U,a_{s3}^U,a_{s4}^U;H_1({\tilde{\mathrm{A}}}_s^U),H_2({\tilde{\mathrm{A}}}_s^U)\right),\left(a_{s1}^L,a_{s2}^L,a_{s3}^L,a_{s4}^L;H_1({\tilde{\mathrm{A}}}_s^L),H_2({\tilde{\mathrm{A}}}_s^L)\right)$$

where, $a_{s1}^U=\min \left\{a_{is1}^U\right\}$, $a_{s2}^U=\min \left\{a_{is2}^U\right\}$, $a_{s3}^U=\mathrm{Max}\left\{a_{is3}^U\right\}$, $a_{s4}^U=\mathrm{Max}\left\{a_{is4}^U\right\}$, $a_{s1}^L=\min \left\{a_{is1}^L\right\}$, $a_{s2}^L=\min \left\{a_{is2}^L\right\}$, $a_{s3}^L=\mathrm{Max}\left\{a_{is3}^L\right\}$, $a_{s4}^L=\mathrm{Max}\left\{a_{is4}^L\right\}$, $H_1({\tilde{\mathrm{A}}}_s^U)=\frac{1}{n}{\displaystyle \sum }_{i=1}^n{H}_1({\tilde{\mathrm{A}}}_{is}^U)$, $H_2({\tilde{\mathrm{A}}}_s^U)=\frac{1}{n}{\displaystyle \sum }_{i=1}^n{H}_2({\tilde{\mathrm{A}}}_{is}^U)$, $H_1({\tilde{\mathrm{A}}}_s^L)=\frac{1}{n}{\displaystyle \sum }_{i=1}^n{H}_1({\tilde{\mathrm{A}}}_{is}^L)$, $H_2({\tilde{\mathrm{A}}}_s^L)=\frac{1}{n}{\displaystyle \sum }_{i=1}^n{H}_2({\tilde{\mathrm{A}}}_{is}^L)$.

• Step 2: Defuzzification

To defuzzify the fuzzy weight of sth sub-criteria, the center of area (CoA) method for defuzzification of trapezoidal type-2 fuzzy sets (DTraT) as developed by Kahraman et al. [71] is described as follows:

$$W_s=\frac{\frac{\left(a_{s4}^U-a_{s1}^U\right)+\left(H_1\left({\tilde{\mathrm{A}}}_s^U\right)·a_{s2}^U-a_{s1}^U\right)+\left(H_2\left({\tilde{\mathrm{A}}}_s^U\right)·a_{s3}^U-a_{s1}^U\right)}{4}+a_{s1}^U+\frac{\left(a_{s4}^L-a_{s1}^L\right)+\left(H_1\left({\tilde{\mathrm{A}}}_s^L\right)·a_{s2}^L-a_{s1}^L\right)+\left(H_2\left({\tilde{\mathrm{A}}}_s^L\right)·a_{s3}^L-a_{s1}^L\right)}{4}+a_{s1}^L}{2}$$

where W_s is the defuzzified weight of sth sub-factor.

• Step 3: Threshold Value

For the selection of an acceptable sub-criteria, a threshold value (α) needs to be derived based on the mean of all defuzzified weights of sub-criteria.

If W_s ≥ α, then the sth sub-factor is considered to be acceptable.

If W_s ≤ α, then the sth sub-factor considered is rejected.

Based on the IT2FD outcome, a total of 17 sub-criteria were selected. Weights of the sub-criteria are presented in Figure 4, along with the threshold value of 0.4847. All sub-criteria with weights greater than 0.4847 were accepted.

The finalized sub-criteria were used to prepare a questionnaire for the construction safety training. A hierarchy was created for analysing participant responses to the questionnaire after the training using interval type-2 fuzzy analytical hierarchy process (IT2F–AHP). The hierarchy structure is presented in Figure 5.

3.4. Training Administration and Data Collection

Construction safety training delivery was prepared based on a code of practice—Preventing Falls in Housing Construction [69], an enforceable code of practice developed by Work Safe Australia and aimed at preventing falls in the construction industry. The training was developed on three distinct visualization platforms—traditional safety training presentation, VR-based safety training, and AR-based safety training. The safety training aimed to educate the trainees on the different fall risks in housing construction by demonstrating different fall risks and fall protections that can be employed for managing those risks.

The traditional safety training was prepared using Microsoft PowerPoint, and all the visualizations for safety risks and fall protections were consistent with those presented in the code of practice. However, to prepare immersive visualizations for XR-based training (AR and VR), the authors used Unity3D. Two different construction models were used—one representing the construction of a free-standing house and the other representing a low-rise housing building. Figure 6 shows the two models as seen in VR and AR media. For visualization of safety training in XR realm, two head-mounted displays (HMD) were used—one each for VR and AR visualizations. Samsung Odyssey VR headset was used for VR training while Microsoft Hololens 2 was used for AR training. Fall risks and their protections were modelled as per the code of practice. Animations were created to demonstrate how the fall protection devices are expected to be used and how they can protect workers from falling—i.e., if they protect workers before or after a fall has been triggered. Some of the active protection devices are shown in Figure 7 while Figure 8 shows a visualization of passive protection.

A cohort of 60 trainees enrolled in construction safety and risk management at Monash University were employed for the construction safety training. The cohort had 30 males and 30 females with ages ranging from 21 years to 40 years with 25 years being the median age of the group. All the trainees were well experienced with XR technology given the popularity of VR headsets in video game applications. All of the students were final-year undergraduate or graduate students and had some familiarity with construction operations since all of the students had at least part-time work experience on construction sites and were aware of site safety practices. All post-graduate students had prior work experience working with major construction contractors on building construction sites.

For administration of training, the cohort was randomly divided into 3 groups of 20 trainees each and were subjected to different modes of training—one at a time with a gap of two weeks between two training sessions. This gap between training sessions was kept so as to avoid influence of learning from previous presentations since most students would have forgotten about 80% of the content in a week’s time [72].

Before the first construction safety training session, all of the participants were made aware of the difference between the modes of safety training and explained the experimental setup. After the introduction, each group of trainees was subjected to one the three different training modes in a classroom setting. The traditional mode of training was presented using an overhead projector and the session was made interactive by encouraging the trainees to ask questions regarding the types of fall protection and how each one could be used to mitigate fall risks. A Samsung Odyssey VR headset was used to demonstrate the VR-based safety training. The visualizations were shown in real-time to the cohort of students on the projector screen. Similarly, for demonstrating the interactive AR-based training, Microsoft Hololens2 was used, and the interactive animations were demonstrated using a projector screen in real time. A total of three training sessions were held to demonstrate each mode of training to the entire cohort of students.

During the demonstration of the three modes of training, students were asked to provide feedback on an online questionnaire. Their responses to the questionnaire were directly related to knowledge retention with each type of visualization medium. A total of 60 individual responses were collected after the three construction safety training sessions.

3.5. Interval Type-2 Fuzzy AHP (IT2F–AHP)

To answer research question 2, the evaluation of knowledge retention from the three different training modes was done by analysing participant responses to the ARCS questionnaire prepared as part of the study. IT2F–AHP was used as the method of analysis as described by Kahraman et al. and pictorially presented in Figure 1.

Step 1: Construction of pairwise comparison matrices and consistency check

In this step, comparison matrices for criteria, sub-criteria, and alternatives are constructed from participant responses. The linguistic terms and their corresponding fuzzy sets, as defined in Table 3, are used to construct pairwise matrices. Each pairwise comparison matrix is then checked for consistency. For checking consistency, each of the elements in a comparison matrix is defuzzified using the DTraT method described in step 2 of Section 3.3 and the consistency index of the matrix is computed. Based on consistency, responses from 32 participants out of 60 were considered for the IT2F–AHP analysis.

Step 2: Aggregate pairwise comparison matrices and calculate geometric means

To create an aggregate pairwise comparison matrices for criteria, sub-criteria, and alternatives, response matrices for all participants are aggregated. Aggregation for each element is done by taking the geometric mean for the respective element. The formula for calculating the geometric mean of each row is given below:

$${\tilde{r}}_i={\left[{\tilde{a}}_{i1}\otimes {\tilde{a}}_{i2}\otimes {\tilde{a}}_{i3}\otimes \dots \otimes {\tilde{a}}_{in}\right]}^{\frac{1}{n}}$$

where ${\tilde{r}}_i$ is the geometric mean of the ith row and ${\tilde{a}}_{in}$ is the element of the matrix in ith row and nth column.

The aggregate pairwise comparison matrix for criteria is presented in

Table 5. IT2F Pairwise comparison matrix for the criteria.

	Attention	Relevance	Confidence	Satisfaction
Attention	(1, 1, 1, 1; 1, 1) (1, 1, 1, 1; 0.8, 0.8)	(0.77, 0.98, 1.47, 1.87; 1, 1) (0.81, 1.02, 1.4, 1.76; 0.8, 0.8)	(0.58, 0.79, 1.23, 1.56; 1, 1) (0.63, 0.82, 1.18, 1.48; 0.8, 0.8)	(1.21, 1.77, 2.76, 3.3; 1, 1) (1.33, 1.86, 2.65, 3.18; 0.8, 0.8)
Relevance	(0.53, 0.69, 1.03, 1.29; 1, 1) (0.57, 0.71, 0.98, 1.22; 0.8, 0.8)	(1, 1, 1, 1; 1, 1) (1, 1, 1, 1; 0.8, 0.8)	(1.21, 1.61, 2.4, 2.91; 1, 1) (1.3, 1.67, 2.3, 2.79; 0.8, 0.8)	(0.96, 1.25, 1.89, 2.44; 1, 1) (1.02, 1.3, 1.81, 2.29; 0.8, 0.8)
Confidence	(0.64, 0.82, 1.27, 1.7; 1, 1) (0.68, 0.85, 1.21, 1.58; 0.8, 0.8)	(0.34, 0.42, 0.63, 0.82; 1, 1) (0.36, 0.43, 0.59, 0.77; 0.8, 0.8)	(1, 1, 1, 1; 1, 1) (1, 1, 1, 1; 0.8, 0.8)	(1.36, 1.77, 2.69, 3.41; 1, 1) (1.45, 1.85, 2.58, 3.23; 0.8, 0.8)
Satisfaction	(0.3, 0.37, 0.57, 0.82; 1, 1) (0.32, 0.37, 0.53, 0.75; 0.8, 0.8)	(0.41, 0.53, 0.81, 1.04; 1, 1) (0.43, 0.55, 0.77, 0.98; 0.8, 0.8)	(0.29, 0.38, 0.57, 0.73; 1, 1) (0.31, 0.39, 0.54, 0.69; 0.8, 0.8)	(1, 1, 1, 1; 1, 1) (1, 1, 1, 1; 0.8, 0.8)

.

Step 3: Calculate IT2F and crisp global fuzzy weights

Following the creation of aggregated pairwise comparison matrices, the global weights are computed by normalization using the following formula:

$${\tilde{w}}_i=r\otimes {\left[{\tilde{r}}_1\oplus {\tilde{r}}_2\dots \oplus {\tilde{r}}_i\dots \oplus {\tilde{r}}_n\right]}^{-1}$$

where ${\tilde{w}}_i$ is the priority weight for ith sub-criterion and $r$ is the geometric mean of the ith row.

Using this equation, fuzzy weights of all the criteria, sub-criteria, and alternatives are then calculated and used for ranking the alternatives concerning each criterion. For ease of comparison and ranking, the fuzzy weights are converted to crisp weights using the DTraT equation and these crisp weights are then further normalized. The fuzzy, crisp, and global weights for criteria, sub-criteria, and alternatives are presented in

Table 6. Fuzzy, crisp, and normalized weights for criteria.

Criteria	IT2F Weights	Crisp Weights	Normalized Weights
Attention	(0.15, 0.23, 0.41, 0.57; 1, 1) (0.15, 0.23, 0.4, 0.58; 0.8, 0.8)	0.33	0.302
Relevance	(0.16, 0.23, 0.4, 0.57; 1, 1) (0.16, 0.23, 0.4, 0.57; 0.8, 0.8)	0.32	0.300
Confidence	(0.13, 0.19, 0.33, 0.48; 1, 1) (0.13, 0.19, 0.33, 0.48; 0.8, 0.8)	0.27	0.250
Satisfaction	(0.08, 0.11, 0.2, 0.29; 1, 1) (0.08, 0.11, 0.19, 0.29; 0.8, 0.8)	0.16	0.148

and

Table 7. Fuzzy, crisp, and global weights of sub-factors.

Criteria	IT2F Weight	Normalized Weight	Global Weight	Ranking
Practicality	(0.2, 0.3, 0.49, 0.65; 1, 1) (0.2, 0.3, 0.49, 0.67; 0.8, 0.8)	0.37	0.112	1
Visual Content	(0.18, 0.25, 0.42, 0.57; 1, 1) (0.18, 0.25, 0.41, 0.58; 0.8, 0.8)	0.32	0.096	2
Ease Understanding	(0.17, 0.27, 0.5, 0.73; 1, 1) (0.18, 0.27, 0.5, 0.74; 0.8, 0.8)	0.37	0.091	3
Visualization	(0.19, 0.28, 0.49, 0.71; 1, 1) (0.19, 0.28, 0.49, 0.72; 0.8, 0.8)	0.36	0.091	4
User Interface	(0.12, 0.19, 0.33, 0.48; 1, 1) (0.12, 0.18, 0.33, 0.48; 0.8, 0.8)	0.24	0.073	5
Clarity	(0.11, 0.17, 0.31, 0.44; 1, 1) (0.12, 0.17, 0.31, 0.45; 0.8, 0.8)	0.23	0.069	6
Engagement	(0.11, 0.17, 0.3, 0.43; 1, 1) (0.11, 0.16, 0.3, 0.44; 0.8, 0.8)	0.22	0.067	7
Design	(0.21, 0.3, 0.51, 0.7; 1, 1) (0.21, 0.3, 0.51, 0.71; 0.8, 0.8)	0.39	0.058	8
Updated content	(0.11, 0.15, 0.24, 0.35; 1, 1) (0.11, 0.14, 0.24, 0.35; 0.8, 0.8)	0.19	0.057	9
Curiosity	(0.09, 0.13, 0.24, 0.37; 1, 1) (0.09, 0.13, 0.23, 0.37; 0.8, 0.8)	0.18	0.054	10
Conciseness	(0.08, 0.12, 0.21, 0.32; 1, 1) (0.08, 0.12, 0.21, 0.32; 0.8, 0.8)	0.16	0.040	11
Variety	(0.07, 0.1, 0.17, 0.26; 1, 1) (0.07, 0.1, 0.17, 0.26; 0.8, 0.8)	0.13	0.039	12
Enjoyment	(0.13, 0.19, 0.32, 0.45; 1, 1) (0.13, 0.19, 0.31, 0.45; 0.8, 0.8)	0.24	0.036	13
Knowledge	(0.06, 0.08, 0.14, 0.22; 1, 1) (0.06, 0.08, 0.14, 0.22; 0.8, 0.8)	0.12	0.035	14
Activity Completion	(0.12, 0.17, 0.27, 0.38; 1, 1) (0.12, 0.17, 0.27, 0.38; 0.8, 0.8)	0.21	0.032	15
Repetition	(0.06, 0.08, 0.14, 0.23; 1, 1) (0.06, 0.08, 0.14, 0.22; 0.8, 0.8)	0.11	0.028	16
Feedback	(0.08, 0.12, 0.19, 0.28; 1, 1) (0.08, 0.11, 0.19, 0.28; 0.8, 0.8)	0.15	0.023	17

and

Table A1. Fuzzy, crisp, and global weights of alternatives with respect to sub-criteria.

Criteria	IT2F Weight	Crisp Weight	Normalized Weight	Global Weight
Engagement
AR	(0.25, 0.33, 0.5, 0.66; 1, 1) (0.25, 0.33, 0.49, 0.67; 0.8, 0.8)	0.41	0.41	0.027
VR	(0.3, 0.42, 0.64, 0.82; 1, 1) (0.31, 0.41, 0.64, 0.83; 0.8, 0.8)	0.52	0.51	0.034
Conventional	(0.06, 0.07, 0.1, 0.14; 1, 1) (0.06, 0.07, 0.1, 0.14; 0.8, 0.8)	0.09	0.09	0.006
User Interface
AR	(0.29, 0.37, 0.52, 0.65; 1, 1) (0.3, 0.37, 0.51, 0.65; 0.8, 0.8)	0.44	0.44	0.032
VR	(0.32, 0.41, 0.55, 0.67; 1, 1) (0.33, 0.41, 0.55, 0.68; 0.8, 0.8)	0.47	0.47	0.035
Conventional	(0.06, 0.08, 0.11, 0.14; 1, 1) (0.06, 0.08, 0.1, 0.14; 0.8, 0.8)	0.09	0.09	0.007
Curiosity
AR	(0.34, 0.47, 0.74, 0.96; 1, 1) (0.35, 0.47, 0.74, 0.98; 0.8, 0.8)	0.60	0.58	0.032
VR	(0.2, 0.26, 0.42, 0.57; 1, 1) (0.2, 0.26, 0.41, 0.57; 0.8, 0.8)	0.35	0.34	0.018
Conventional	(0.05, 0.07, 0.1, 0.13; 1, 1) (0.05, 0.06, 0.1, 0.13; 0.8, 0.8)	0.08	0.08	0.004
Variety
AR	(0.35, 0.47, 0.71, 0.91; 1, 1) (0.35, 0.47, 0.71, 0.92; 0.8, 0.8)	0.58	0.57	0.022
VR	(0.19, 0.25, 0.39, 0.53; 1, 1) (0.19, 0.25, 0.39, 0.53; 0.8, 0.8)	0.33	0.32	0.013
Conventional	(0.07, 0.09, 0.13, 0.18; 1, 1) (0.07, 0.09, 0.13, 0.18; 0.8, 0.8)	0.11	0.11	0.004
Clarity
AR	(0.27, 0.38, 0.61, 0.8; 1, 1) (0.27, 0.38, 0.61, 0.82; 0.8, 0.8)	0.49	0.48	0.033
VR	(0.23, 0.31, 0.49, 0.67; 1, 1) (0.24, 0.31, 0.49, 0.67; 0.8, 0.8)	0.41	0.39	0.027
Conventional	(0.08, 0.11, 0.16, 0.22; 1, 1) (0.08, 0.1, 0.16, 0.22; 0.8, 0.8)	0.14	0.13	0.009
Knowledge
AR	(0.27, 0.38, 0.59, 0.76; 1, 1) (0.28, 0.38, 0.59, 0.78; 0.8, 0.8)	0.48	0.47	0.016
VR	(0.19, 0.25, 0.38, 0.52; 1, 1) (0.19, 0.24, 0.38, 0.52; 0.8, 0.8)	0.32	0.31	0.011
Conventional	(0.14, 0.18, 0.27, 0.37; 1, 1) (0.14, 0.18, 0.27, 0.37; 0.8, 0.8)	0.23	0.22	0.008
Visual Content
AR	(0.25, 0.37, 0.63, 0.86; 1, 1) (0.26, 0.38, 0.63, 0.88; 0.8, 0.8)	0.51	0.48	0.046
VR	(0.21, 0.29, 0.48, 0.69; 1, 1) (0.21, 0.29, 0.48, 0.7; 0.8, 0.8)	0.40	0.38	0.037
Conventional	(0.08, 0.11, 0.18, 0.26; 1, 1) (0.08, 0.11, 0.17, 0.26; 0.8, 0.8)	0.15	0.14	0.014
Updated content
AR	(0.19, 0.26, 0.42, 0.58; 1, 1) (0.19, 0.26, 0.42, 0.58; 0.8, 0.8)	0.35	0.34	0.019
VR	(0.14, 0.18, 0.28, 0.39; 1, 1) (0.14, 0.18, 0.28, 0.39; 0.8, 0.8)	0.24	0.23	0.013
Conventional	(0.25, 0.35, 0.56, 0.73; 1, 1) (0.25, 0.35, 0.55, 0.74; 0.8, 0.8)	0.45	0.44	0.025
Practicality
AR	(0.28, 0.41, 0.66, 0.89; 1, 1) (0.29, 0.41, 0.66, 0.91; 0.8, 0.8)	0.54	0.51	0.057
VR	(0.21, 0.29, 0.47, 0.65; 1, 1) (0.21, 0.29, 0.47, 0.66; 0.8, 0.8)	0.39	0.37	0.041
Conventional	(0.07, 0.09, 0.15, 0.21; 1, 1) (0.07, 0.09, 0.14, 0.21; 0.8, 0.8)	0.12	0.12	0.013
Ease Understanding
AR	(0.35, 0.48, 0.74, 0.95; 1, 1) (0.36, 0.48, 0.74, 0.96; 0.8, 0.8)	0.60	0.59	0.054
VR	(0.19, 0.25, 0.38, 0.52; 1, 1) (0.19, 0.24, 0.37, 0.52; 0.8, 0.8)	0.32	0.31	0.028
Conventional	(0.07, 0.08, 0.12, 0.16; 1, 1) (0.07, 0.08, 0.12, 0.16; 0.8, 0.8)	0.10	0.10	0.009
Conciseness
AR	(0.34, 0.44, 0.65, 0.81; 1, 1) (0.34, 0.44, 0.64, 0.82; 0.8, 0.8)	0.53	0.53	0.021
VR	(0.2, 0.26, 0.38, 0.5; 1, 1) (0.21, 0.26, 0.38, 0.5; 0.8, 0.8)	0.32	0.32	0.013
Conventional	(0.1, 0.13, 0.18, 0.23; 1, 1) (0.1, 0.12, 0.18, 0.23; 0.8, 0.8)	0.15	0.15	0.006
Visualization
AR	(0.34, 0.46, 0.71, 0.91; 1, 1) (0.34, 0.46, 0.71, 0.93; 0.8, 0.8)	0.58	0.57	0.052
VR	(0.21, 0.28, 0.43, 0.58; 1, 1) (0.21, 0.28, 0.43, 0.58; 0.8, 0.8)	0.36	0.35	0.032
Conventional	(0.06, 0.07, 0.1, 0.14; 1, 1) (0.06, 0.07, 0.1, 0.14; 0.8, 0.8)	0.09	0.08	0.008
Repetition
AR	(0.29, 0.39, 0.58, 0.73; 1, 1) (0.3, 0.39, 0.57, 0.74; 0.8, 0.8)	0.47	0.47	0.013
VR	(0.23, 0.3, 0.43, 0.55; 1, 1) (0.23, 0.29, 0.43, 0.55; 0.8, 0.8)	0.36	0.36	0.010
Conventional	(0.12, 0.14, 0.21, 0.26; 1, 1) (0.12, 0.14, 0.2, 0.27; 0.8, 0.8)	0.17	0.17	0.005
Activity Completion
AR	(0.3, 0.41, 0.6, 0.75; 1, 1) (0.31, 0.41, 0.6, 0.76; 0.8, 0.8)	0.49	0.49	0.015
VR	(0.25, 0.32, 0.47, 0.63; 1, 1) (0.25, 0.31, 0.47, 0.62; 0.8, 0.8)	0.40	0.39	0.012
Conventional	(0.08, 0.1, 0.15, 0.19; 1, 1) (0.08, 0.1, 0.14, 0.19; 0.8, 0.8)	0.12	0.12	0.004
Enjoyment
AR	(0.34, 0.47, 0.71, 0.91; 1, 1) (0.35, 0.47, 0.71, 0.92; 0.8, 0.8)	0.58	0.57	0.021
VR	(0.22, 0.28, 0.43, 0.59; 1, 1) (0.22, 0.28, 0.43, 0.59; 0.8, 0.8)	0.36	0.36	0.013
Conventional	(0.05, 0.06, 0.09, 0.12; 1, 1) (0.05, 0.06, 0.09, 0.12; 0.8, 0.8)	0.08	0.08	0.003
Feedback
AR	(0.34, 0.45, 0.65, 0.8; 1, 1) (0.35, 0.45, 0.65, 0.81; 0.8, 0.8)	0.54	0.54	0.012
VR	(0.21, 0.26, 0.38, 0.49; 1, 1) (0.21, 0.26, 0.37, 0.49; 0.8, 0.8)	0.32	0.32	0.007
Conventional	(0.11, 0.13, 0.17, 0.22; 1, 1) (0.11, 0.12, 0.17, 0.22; 0.8, 0.8)	0.15	0.15	0.003
Design
AR	(0.32, 0.43, 0.65, 0.83; 1, 1) (0.32, 0.43, 0.65, 0.84; 0.8, 0.8)	0.53	0.52	0.030
VR	(0.24, 0.31, 0.46, 0.61; 1, 1) (0.24, 0.31, 0.46, 0.61; 0.8, 0.8)	0.39	0.38	0.022
Conventional	(0.07, 0.08, 0.11, 0.14; 1, 1) (0.07, 0.08, 0.11, 0.15; 0.8, 0.8)	0.09	0.09	0.005

(Appendix A) respectively.

Step 4: Criteria ranking

Based on the defuzzified global weights of criteria, sub-criteria, and alternatives, the most important criteria determining the effectiveness of safety training are determined.

3.6. Sensitivity Analysis

To assess the robustness of the IT2F–AHP methodology, sensitivity analysis was performed by varying the priority weights of criteria. A sensitivity analysis was performed to address the third research question exploring the robustness of the analysis methodology adopted for the study. A total of eight cases were defined by first varying priority weights of ARCS criteria while keeping the priority weights of sub-criteria unchanged and then varying the weights of ARCS criteria along with the highest-weighted sub-criteria within each criterion. The definitions of 9 cases, including the default case, are provided in

Table 8. Case definitions for sensitivity analysis.

Cases	Description
Case 1	Current Case
Case 2	High weight of Attention, Low for Relevance, Confidence and Satisfaction
Case 3	High weight of Relevance, Low for Attention, Confidence and Satisfaction
Case 4	High weight of Confidence, Low for Attention, Relevance and Satisfaction
Case 5	High weight of Satisfaction, Low for Attention, Relevance and Confidence
Case 6	Case 2 + High weight of User Interface, Low for other sub-criteria
Case 7	Case 3 + High weight of Practicality, Low for other sub-criteria
Case 8	Case 4 + High weight of Ease of Understanding, Low for other sub-criteria
Case 9	Case 5 + High weight of Design, Low for other sub-criteria

. The outcomes of the sensitivity analysis are presented in the next section.

4. Results

An IT2F–AHP was used for the interpretation of participant feedback on the ARCS questionnaire, as described in Section 3.2. The fuzzy comparison matrix derived from the IT2F–AHP analysis is shown in Table 5 and the calculated fuzzy, crisp, and normalized weights are tabulated in Table 6. The normalized priority weights of the main criteria presented in Figure 9 reveal attention to be the most critical factor that affects knowledge retention among the trainees. However, we can observe from the normalized weights of the main criteria that relevance is almost as important as attention since the normalized weight of relevance is less than that of attention only by 0.2 percent.

The global weights of sub-criteria provide a much deeper insight into the importance of sub-criteria in influencing knowledge retention. From Table 7 and Figure 10, we can see that the two highest-ranked sub-criteria—Practicality and Visual Content—fall under “Relevance”. At the same time, Ease of Understanding, User Interface, and Design are the most critical sub-criteria within “Attention”, “Confidence”, and “Satisfaction” respectively.

As a part of the IT2F–AHP analysis, the three different training modes were analysed concerning their impacts on the different criteria and sub-criteria. The outcomes of the analysis are provided in Table A1 and graphically presented in Figure 11 with the red, green, and blue columns representing the VR, conventional (overhead projection), and AR visualization methods, respectively.

We can observe that AR was preferred as the most effective method for the majority of sub-criteria and allowed trainees to acquire the maximum amount of information through the interactive visualization of common fall hazards and applicable protection devices for preventing and/or arresting falls. The conventional method of training, on the other hand, was the most ineffective for knowledge acquisition in the majority of sub-criteria.

For the validation of the results and to analyse the robustness of the IT2F–AHP, a sensitivity analysis was performed. The output of the sensitivity analysis was aimed at answering the third research question. The results of the analysis are presented in Figure 12.

5. Discussion

This study was conducted to answer the three research questions in Section 1. This section discusses findings related to the three research questions.

Research Question 1. What are the most influential criteria for evaluating a trainee’s knowledge acquisition for applying safety measures in their workplace? This study explores the most important criteria that influence knowledge acquisition from safety training. Important aspects based on the motivational theory were analysed, and it was observed that the factors that allowed trainees to maintain their attention on the concepts being delivered throughout the training were the most important for the effective transfer of knowledge. However, based on the priority weights of the criteria, it can be argued that the “Relevance” factors were almost as important, since the difference between weights of the “Attention” and “Relevance” criteria was less than 2%. Further, in analysing the importance of the sub-criteria for effective knowledge acquisition by the trainees, “Practicality” was the highest-ranked sub-criterion while “Visual Content” was ranked second. The IT2F–AHP analysis of sub-criteria revealed that the top two ranked sub-criteria belonged to the “Relevance” criterion, which represents the relatability of the safety training to the actual task being performed on site. This result shows that designing customized training relevant to specific tasks being performed on-site is important in ensuring efficient knowledge acquisition by trainees.

The lowest-ranked subcriterion was “Feedback”, which demonstrates that the trainees were able to provide sufficient feedback to the trainers regarding their knowledge acquisition. The lowest ranked sub-criteria amongst each group of ARCS criteria were “Variety of content”, “Knowledge”, “Repetition”, and “Feedback” respectively. “Variety of content” being the least important in its group of sub-criteria shows that multiple demonstrations and visualizations are not necessary for knowledge acquisition when the training has been customized to represent the task being performed. Similarly, “Knowledge” getting the lowest priority weight in “Relevance” demonstrates that having prior knowledge of the subject is not a must for understanding the safety concepts when the training is customized to represent the actual task. Additionally, with customized training sessions, the necessity of repetitions for ensuring effective knowledge acquisition is reduced, as demonstrated by the low global weight of “Repetition” which was ranked 16th, as shown in Table 7.

As seen from the outcome of the IT2F–AHP analysis, we can conclude that knowledge acquired from customized virtual safety training sessions can be translated into safe work practices on construction sites, as demonstrated by Park et al. [33]. To develop an effective safety training session, it must be relatable—i.e., customized to the applicable work area as demonstrated by “Practicality” and ensure the training interface allows trainees to maintain their attention during the training administration as shown by the global weight of “Visual Content”.

Research Question 2. To what extent do Extended Reality (XR) training methods impact criteria important for knowledge acquisition among trainees compared to the traditional training method? With the advancement in visualization technology, VR and AR devices are much more easily accessible. Paired with immersive visualizations that can be generated using BIMs, these devices are now used for various studies, reviews, and training sessions in the construction industry [73]. As a part of this study, an IT2F–AHP analysis of pairwise comparisons between the sub-criteria and alternatives (modes of training) was carried out. The IT2F–AHP analysis revealed that AR safety training is the most effective method for effective knowledge transfer using safety training. Fourteen of the pairwise comparisons revealed the greatest priority weights for AR safety training followed by VR safety training, while the conventional method had the lowest priority weights for these pairwise comparisons. Interestingly, the pairwise comparisons for two of the sub-criteria—“User Interface” and “Engagement” —had the highest priority weights for VR training sessions followed by AR and conventional training sessions, respectively. A plausible explanation for this result can be the difference in one’s ability to interact with VR visualizations vs AR visualizations. Since the VR visualizations were created in a virtual space and the interactions also occurred in a virtual space, the participants could seamlessly move around the models and interact with them in the immersive environment using keyboard inputs.

On the other hand, the virtual visualizations were super-imposed onto a physical space and moving around them in a physical space was cumbersome for the participants. This led to VR getting a higher weightage than AR for “User Interface” and “Engagement”. Another important observation was the result from the pairwise comparison under the “Updated Content” criterion. Since the participants felt it was easier to provide feedback and implement changes to the content with the conventional training presentation, it has the highest priority weight, followed by the AR and VR training methods, respectively. It can be inferred that the XR training methods allowed maximum knowledge acquisition by improving the visualization of content and allowing the trainees to maintain their attention on the subject matter when compared to the traditional method of training. Knowledge acquisition increases with improvements in the visualization of the task at hand with greater clarity [74]. Among the XR training methods, the AR-based training session was the most effective owing to the ease of interaction and visualization offered by this technology.

Research Question 3. How does the decision of modelling criteria weight priorities change the outcome of IT2F–AHP? This study employs IT2F–AHP in order to analyse participant responses. A sensitivity analysis was carried out to understand the effect of modelling on criteria weight properties and to check the methodology’s robustness. Cases for the sensitivity analysis were defined to maximise the impact on the outcome of the IT2F–AHP. The case definitions are detailed in Table 8 and the analysis results are presented in Figure 12. Case 1 is the base case with its weights obtained from the IT2F–AHP analysis. The simulated weights in cases 2 to 9 were compared against the base case to test the robustness of the current method of analysis. In cases 2 to 5, a very high weight was assigned to a single criterion while minimizing the priority weights of the other criteria. This allowed us to model extreme cases in which the participant responses were heavily in favour of a particular criterion. Further, to increase the skew of weights, extremely high-priority weights were assigned one-by-one to the highest-weighted sub-criteria under each main criterion in cases 6 to 9. The resulting global weights of the alternatives (modes of training) are plotted in Figure 12. From the graph, it is observed that the results from only two out of the eight simulated cases had any departure in the selection of optimum mode of training when compared to the base case (case 1) due to the introduced skew of priority weights in the criteria and sub-criteria.

The VR training method had a slightly higher-priority weight than the AR training method and traditional training method in Case 6, in which had a high-priority weight for the “User Interface” sub-criterion along with its main criterion, “Attention”. Similarly, the conventional training method had a higher-priority weight than the AR training method and VR training method in Case 7, in which “Practicality” had a high-priority weight along with the main criterion—“Relevance”. These cases demonstrate that outcomes will change only in extreme cases in which the weights are highly skewed with the 90% weight assigned to certain sub-criteria along with their main criteria. The outcome of the sensitivity analysis demonstrates the robustness of the IT2F–AHP, and the results reinforce that “Attention” and “Relevance” are essential criteria for the effective administration of safety training and that AR visualizations can facilitate maximum knowledge acquisition among trainees.

6. Conclusions

In this study, essential factors for effective knowledge acquisition in construction safety training were analysed. A cohort of experts in the construction safety training field were consulted to identify the key criteria. Using the interval type-2 fuzzy Delphi (IT2FD) technique, the number of identified criteria was further refined, and a total of 17 sub-criteria were identified. For data collection, a group of 60 trainees enrolled in construction safety and risk management at Monash University were subjected to three different modes of construction training. Each training method employed a different method of visualization—conventional overhead projection, VR visualization, and AR visualization. The feedback from the trainees was collected by asking them to respond to an online survey. An interval type-2 fuzzy AHP (IT2F–AHP) was used to analyse the participant responses. This analysis was followed by a sensitivity analysis that tested the robustness of the employed method.

By observing the results from the IT2F–AHP and based on the outcome of the sensitivity analysis, we can conclude that “Attention” is the most critical criterion. At the same time, “Relevance” is also an important criterion, based on the highest ranking of its sub-criteria “Practicality” and “Visual Content”. The role of AR visualizations for effective construction safety training is highlighted by a pairwise comparison between the training methods and sub-criteria. These results from the IT2F–AHP analysis answer research questions 1 and 2 by revealing the most important criteria for ensuring the effectiveness of virtual construction safety training along with the most effective method of visualization for the training.

The sensitivity analysis outcome demonstrated the robustness of the IT2F–AHP in analysing participant feedback. Assigning high-priority weights across seven out of nine cases did not change the preference of visualization technology. This conclusively answers the third research question by demonstrating that XR technology allows greater knowledge acquisition by trainees compared to the conventional training method.

The results from this study contribute to the body of knowledge in construction safety by revealing the most influential criteria for evaluating a trainee’s knowledge acquisition for safety measures in their workplace. The results are of practical importance since they allow for the design of effective virtual safety training that is customized to the task at hand and virtual environments relatable to the actual construction site. Further, the results show visualisation technology is important, as this study reveals XR-based visualizations, especially AR visualizations, allow greater knowledge retention by trainees, ultimately leading to higher safety performance on construction sites. For future studies, the findings of this research can be applied towards the development of solutions to safety problems using visualization tools, and the performance, applicability, comprehensiveness, and duration of these solutions can be analysed.