Navigating the Healthcare Metaverse: Immersive Technologies and Future Perspectives

Abstract

The year is 2030. The internet has evolved into the metaverse. People navigate through advanced avatars, shop in digital marketplaces, and connect with others through extended reality social media platforms. Three-dimensional patient scans, multidisciplinary tele-collaborations, digital twins and metaverse health records are part of clinical practices. Younger generations regularly immerse themselves in virtual worlds, playing games and attending social events in the metaverse. This sounds like a sci-fi movie, but as the world embraces immersive technologies post-COVID-19, this future is not too far off. This article aims to provide a foundational background to immersive technologies and their applications and discuss their potential for transforming healthcare and education. Moreover, this article will introduce the metaverse ecosystem and characteristics, and its potential for health prevention, treatment, education, and research. Finally, this article will explore the synergy between generative artificial intelligence and the metaverse. As younger generations of healthcare professionals embrace this digital frontier, the metaverse’s potential in healthcare is definitely attractive. Mainstream adoption may take time, but it is imperative that healthcare professionals be equipped with interdisciplinary skills to navigate the plethora of immersive technologies in the future of healthcare.

1. Introduction

Imagine living in a futuristic world ten years from now… The internet has evolved into a new version—the metaverse. In this upgraded digital realm, people navigate and explore this virtual world using a more advanced form of their avatars—known as digital humans. Online shopping is carried out at virtual marketplaces and connecting with family, friends and colleagues is conducted through an enhanced form of social media that integrates Zoom and Microsoft Teams with mixed reality (MR) within the metaverse. As a healthcare professional (HCP), you can access three-dimensional (3D) visualizations of patient’s scans, collaborate remotely with multidisciplinary teams, and discuss patient diagnoses and treatment plans using digital twins stored in their metaverse health records, all within your office in the virtual hospital. For parents of Generation Zs (Gen Zs) and Generation Alphas, imagine your children playing games and attending social/entertainment events in digital worlds like Fortnite and Roblox.

This scenario may seem too far-fetched, perhaps even impossible to imagine it happening during our lifetime… But as the world continues to adapt to digital transformations post-COVID-19, this future may not be as distant as it seems. This article is designed to introduce the foundational concepts of immersive technologies to readers, such as extended reality (XR) and the metaverse, so that they can better understand how these current trends are shaping healthcare practices and education.

2. Immersive Technologies—A Spectrum of Realistic Possibilities

HCPs are already experimenting with various immersive technologies for clinical practice and healthcare education. But what distinguishes augmented reality (AR), virtual reality (VR), MR and XR? While these immersive technologies share a key feature of creating a sense of “presence” or “being there”, understanding the subtle differences between them can potentially enhance the communication among HCPs, game designers, technical developers, as well as technology vendors.

A straightforward way to understand these differences is to examine the reality–virtuality (RV) continuum (Figure 1) [1]. On one end of the continuum lies reality, the real-world or physical environment, where users interact and engage with tangible objects. For example, in physical escape room games, players collaborate together to find clues and solve puzzles within a real-world setting to “escape” [2]. On the opposite end of the continuum is VR, which immerses users in a fully simulated environment, where they also interact with virtual elements.

There are two main types of VR that most people are familiar with—immersive VR and non-immersive VR (Figure 2). The main differences between these two types of VR are their abilities to provide interactivity, presence and narrative engagement [3,4]. Examples of non-immersive VR are the use of a laptop or desktop to watch a VR video or play a video game. In these cases, user interactions are usually limited, and the user does not get a sense of presence (or “being there” in the virtual environment). In certain situations, the feeling of narrative engagement (i.e., immersed in the story) is also limited (e.g., playing a Pac-man game), but in certain games with an adventure storyline or in first-party shooter games, players may become immersed in the storyline as they take on the role of the protagonist in the game story. On the other hand, immersive VR is able to provide users with presence, but the immersivity level depends on the level of fidelity of the technology, which provides a measure of the realism of the simulation. In general, high-fidelity equipment, such as high-performance computers, stereo/surround sound, hand controllers, body-tracking sensors and head-mounted displays, can provide users with a more immersive experience.

The original RV continuum was mainly concerned with visual displays [5], but recent updates further classified VR based on their virtual environments [1]. The first kind is an external virtual environment (EVE), whereby users interact with virtual objects with a head-mounted device while being embedded in a virtual environment. Since these devices are actually physical items situated in a physical environment, the user’s experience is not fully immersive. On the other hand, the second kind of environment—a “Matrix-like” virtual environment (MVE)—following the popular Matrix film series, provides a greater sense of immersivity. Although both EVE and MVE are able to stimulate the user’s exteroceptive senses (e.g., hearing, sight, taste, touch, smell), only MVE is able to stimulate the user’s interoceptive (e.g., proprioception) senses; thus, not only does it provide a highly immersive experience, but the technological system also has a high extent of world knowledge—a phenomenon that is often referred to as a “perfect” digital twin, where the system is fully aware of its real-world surroundings and can react to any changes.

In between the physical world/reality and MVE is a continuum that integrates both virtual and real-world elements that allows users to experience them simultaneously—known as MR. In essence, its degree of variation between the virtual and real-world components can be categorized into AR and augmented virtuality (AV) (Figure 1). In AR, users have limited interactions with digital/virtual objects that are superimposed onto the physical/real-world environment. In general, AR can be categorized into either a marker-based/marked-based system, or a markerless/markless system (Figure 3) [4,6]. The marker-based system usually requires some sort of marker or identifier to trigger the activation of the AR technology. A common example is the scanning of two-dimensional (2D) QR barcodes using a smartphone’s camera, which activates the AR technology to display some sort of digital or virtual content on top of the barcode. On the other hand, the markerless system is more versatile as it allows the user to decide where to put the virtual content. In this latter category, there are four sub-types of markerless AR [4,6]. Location-based AR uses a location-based functionality, like global positioning systems (GPS) to add the virtual content. A famous example is the popular Pokémon Go game, in which players catch and battle virtual creatures (known as Pokémon), which appear in their surroundings. Superimposition/overlay AR creates virtual content based on the recognition of a real-world item in the physical environment. An example is the Ikea app that allows users to visualize how their homes will be like with virtual furniture from the store [7]. Projection-based AR projects light onto a surface to create the virtual content in the form of a hologram to create an interactive experience for users. The last sub-type of markerless AR is an outlining/contour-based technology that uses the silhouettes of objects, such as boundaries and lines, to help users navigate their environments, for instance, in driverless cars.

In contrast, AV is a less familiar concept that involves the incorporation of physical objects into a virtual environment. An example is a user manipulating a physical 3D-printed replica of an object while seeing its original version displayed in the virtual setting [8]. The main difference between AV and AR boils down to the type of user interactions. AR is when the user interacts with virtual objects in a physical space, but AV is when the user interacts with a physical object in a virtual environment [9,10]. Meanwhile, XR is an umbrella concept that covers all types of environments and interactions that combine virtual and real elements, facilitated by computer technologies and wearables, excluding the physical world/reality. In XR systems, the user typically performs tasks and interacts with both the physical and virtual components through multiple sensors and input devices. These data are then processed by a data engine and stored in a data warehouse, which can potentially be in the cloud. The data can then be analyzed through various artificial intelligence (AI) algorithms, and the feedback provided back to the user through output devices.

3. Application Trends of Immersive Technologies in Healthcare

This section introduces a non-exhaustive list of application trends of immersive technologies in healthcare. The content described in this section is purely for educational purposes and should not be considered a replacement or substitute for professional medical advice or consultations with healthcare providers.

3.1. VR for Rehabilitation

VR is being increasingly utilized for physical and neurorehabilitation as a way to increase engagement, motivation and adherence to conventional therapies, especially for home-based rehabilitation. This virtual approach has demonstrated to be effective for some conditions, such as cerebral palsy and stroke, and has shown positive outcomes in areas like pain management [11,12,13]. In a randomized trial of 300 participants with walking difficulties in Australia, it was reported that the intervention group (149 participants using VR applications in addition to their usual rehabilitation) had clinically important improvements in mobility (e.g., walking, standing up, balance) after 3 weeks and 6 months [14]. In another small-scaled randomized-controlled study of 41 patients with cerebral palsy in Turkey, one-hour based VR therapy sessions administered 3 days a week for 4 weeks in conjunction with conventional treatment and neurophysiological and occupational therapy showed a significant improvement in motor function and mobility in the intervention group [15]. Many studies used game-based VR technologies since these could easily adapt to individual user needs as an adjunct to conventional rehabilitation therapy. For example, serious-games-based VR was used to aid in the neurorehabilitation of specific neurocognitive disorders, such as for retrograde amnesia after brain stroke and spatial extrapersonal neglect due to cerebral lesions [16,17]. While some advantages of using VR therapy include improvements in motor, psychological and cognitive functions, greater community participation, as well as reducing the work burden on therapists [12,13], a challenge is the need for specialist technical skills to train clinicians and patients in the proper use of VR applications as useful rehabilitation tools, and in accordance with evidence-based guidelines [12].

3.2. XR Telehealth

The popularity of telehealth in many countries during COVID-19 led to some organizations exploring a combination of XR with telehealth as a solution to enhance public health safety during the COVID-19 restrictions. By June 2020, telehealth visits in the USA increased by over 2000%, while the shift in telehealth for mental health services more than tripled this percentage [18]. A US-based company, XRHealth (Brookline, MA, USA, https://www.xr.health/, accessed on 3 September 2024), introduced a VR Telehealth Virtual Clinic using a combination of both modalities [19]. Patients would receive a personalized care plan, play gamified exercises using a VR headset in their homes, and connect with their therapist through video calls and a mobile health (mHealth) app. Besides physical and occupational therapy, the clinic’s services have since expanded to include management of stress, anxiety, depression, memory and cognitive training, pain management and post-COVID-19 rehabilitation, among others. Other applications that surfaced during the pandemic included AR systems for remote consultations and ventilator management for COVID-19-positive patients, VR systems for training in trauma and emergency medicine, and VR tele-mental health [18]. The advantage, besides providing immersive and accessible care during the pandemic, was that XR telehealth also afforded an avenue to combat social isolation and aid in long-distance health-promotion by connecting people virtually [18].

3.3. Virtual Reality Exposure Therapy (VRET)

VRET has been used as a technology-based treatment method that uses VR to conduct graded exposure therapy for mental health conditions, such as specific phobias and post-traumatic stress disorder (PTSD). This form of therapy typically involves immersing the person in a computer-generated environment through a head-mounted display or in a computer-controlled room filled with simulated images. The environment can be tailored to help the person confront their phobia or trauma triggers in a safe and gradual manner, thus growing their confidence before dealing with them in real-life. VRET has shown effectiveness as an intervention for conditions like acrophobia (fear of heights), agoraphobia (fear of enclosed spaces), dental phobia, fear of flying, and panic disorder [20]. Similarly, VRET has also shown promise for improving PTSD symptoms [21]. This form of therapy is able to create an interactive and emotionally engaging VR environment that provides multi-sensory stimuli to enhance the visual, auditory, haptic and olfactory experiences for patients during exposure therapy. In general, VRET seemed to be comparable to in vivo exposure and gold-standard cognitive behavioral therapy.

3.4. VR in Oncology

VR has been used to help reduce anxiety and pain in cancer patients through distraction in a variety of situations, including during chemotherapy, palliative and supportive care, and for patients undergoing painful medical procedures [22]. It was reported in a systematic review that pain scores in children and adolescents were 53% lower with the use of VR compared to standard care [3]. In fact, one-third of studies had reported statistically significant decreases in pain experienced by cancer participants undergoing the VR intervention [3]. Interestingly, VR seemed to be more effective in reducing pain when compared to a single management approach (e.g., local anesthetic, non-medical conversation, or no intervention) and with studies that used a multi-modal approach (e.g., with topical anesthetics, opioid, sedative and parental support). A higher fidelity of VR technology (e.g., head-mounted displays with headtracking, stereo-sound and controllers) was better for pain relief, as these features could enhance distraction by allowing users to explore and interact with the virtual environment in a more immersive way. In addition, by engaging multiple senses (e.g., sight, sound, movement) and increasing the level of “presence” and “narrative engagement” through the virtual environment and a captivating storyline, a higher level of immersion was achieved, which better engaged the user in terms of fun and interactivity, thereby leading to more positive results in pain management.

3.5. XR-Based Surgery

Technological advancements and decreasing costs have contributed to a growing acceptance and exploration of XR technologies in the field of surgery. Studies have classified XR applications for surgery into three main categories: (i) planning before surgery and intervention, (ii) guidance during surgery and intervention, and (iii) training and education for surgical and interventional procedures [23,24]. The use of XR in surgery has several advantages, including a better understanding of critical anatomical areas, enhanced visualizations of structural anomalies that are hidden or overlooked on a CT scan, improved surgical planning and execution, and superior depth perception of intricate structures. These advantages can lead to increased surgical precision, reduced operation times and shorter hospital stays. Additionally, XR technologies can enhance clinician-patient communication, improve patients’ understanding of their conditions, reduce patient anxiety, facilitate real-time interactions and collaborations, boost surgical confidence and ease the learning curve in surgical education and training. Consequently, many XR-based companies have entered this sector to develop innovations for surgery [25].

3.6. AR-Based Digital Operations

AR technologies can ease the operational aspects of healthcare. For example, Augmedix (San Francisco, CA, USA, https://augmedix.com/, accessed on 3 September 2024) harnesses the power of Google Glass, AR, natural language processing, and more recently, generative AI, to create a digital scribe that helps clinicians with the documentation of medical notes in real-time during ward rounds and patient consultations so that it cuts down on their administrative workload, and in-turn, increases productivity and decreased burnout [26,27,28,29,30]. Another example is AccuVein (New York, NY, USA, https://www.accuvein.com/, accessed on 3 September 2024), a Medtech startup that employs projection AR technology to help clinicians/nurses in visualizing patients’ veins so that it is easier to carry out blood draws. Through a hand-held laser-based scanner, this AR technology projects over the skin and translates the heat signature of veins into an image that is superimposed on the skin, thus improving the chances of finding a vein and the success rate of drawing blood [31].

3.7. XR in Healthcare Education

XR has also become popular in the field of healthcare education. For example, VR has been used to train soft skills like empathic responding and health communication [32,33]. The Royal College of Surgeons in Ireland has also created a VR training simulation for clinicians in the emergency department to learn how to manage emergency situations and operate on patients suffering from road traffic accidents [34]. Similarly, VR approaches have also been used for surgical training. Studies have shown that VR outperformed traditional training methods in terms of surgeon confidence [24], time to complete procedures, post-intervention checklist scores, and increased accuracy of implant placements [35]. In Singapore, healthcare institutions have also embarked on VR to train clinical and healthcare staff in areas such as septic shock management, cardiopulmonary resuscitation, intravenous cannula insertion and medication safety [36,37]. Furthermore, during COVID-19, some institutions in the US and Singapore also embarked on MR technologies to teach anatomy and train certain procedural skills [38,39].

4. From XR to the Metaverse

The term “metaverse” gained significant attention after the CEO of Meta, Mark Zuckerberg, unveiled his vision in 2021 [40]. Numerous technological companies and organizations have since embraced this concept as a revolutionary way to enhance user immersion by blending the physical and digital realms. Gartner has even predicted that within the next five years, approximately a quarter of the population will spend at least one hour daily for common activities in the metaverse, such as work, education, shopping and entertainment [41]. But what exactly is the metaverse?

Interestingly, there is no single definition of the metaverse that has been universally agreed upon. Broadly speaking, it is a shared virtual space where the physical world merges with the digital world to provide users with an immersive experience. The literature classifies the metaverse into four main characteristics (Figure 4) [42]:

• Augmented reality (AR): this technology involves the overlaying of virtual objects onto the physical world, thereby enhancing the user’s environment (e.g., Pokémon Go).
• Lifelogging: this involves the capturing, storing and sharing of users’ daily activities and experiences (e.g., Facebook, Instagram, Fitbit).
• Mirror worlds: these are digital reflections of the real world that also incorporate external environmental data (e.g., Google Maps, Zoom).
• Virtual worlds: this is the VR technology that we are familiar with, whereby users explore virtual environments and interact with virtual objects through avatars, often using VR headsets.

Ultimately, it is envisioned that many VR, AR and MR activities that are currently standalone in healthcare practice and education will eventually be transitioned into the metaverse!

5. Opportunities of the Metaverse in Healthcare

A recent scoping review highlighted that the opportunities are still limited for integrating the metaverse into healthcare, with its potential being described mostly in reviews and editorials [43]. Despite this, it is anticipated that the metaverse can have significant applications across various healthcare areas, such as health prevention, treatment, education, training, and healthcare research. Health 4.0 is an emerging concept that integrates advanced technologies like XR, Internet of Medical Things (IoMT), cloud computing, cyber–physical systems, big data analytics, AI and blockchain into healthcare practices [43]. Envisioned scenarios include virtual care models where HCPs use avatars to provide consultations to patients in virtual and telehealth clinics, and conduct tele-monitoring through wearable sensors, such as smart watches and mobile apps [44]. In the domain of healthcare education, the metaverse could serve as a new and novel social communication platform that offers more creative freedom and immersive experiences for sharing. For instance, Seoul National University Bundang Hospital launched a metaverse-based training program on lung cancer surgery in 2021 [45]. This program utilized a smart operating room equipped with high-resolution 360-degree cameras to broadcast surgeries live to online conference participants, where they could view lectures and engage in real-time discussions about the surgical procedures. Similarly, in Singapore, the metaverse was also explored for continuing the education of pharmacy staff [46,47]. Participants navigated an educational metaverse that was created in the form of a virtual art gallery, where they could explore portraits, watch mini-lectures and interact with 3D artifacts related to telehealth and virtual care. In addition, a virtual escape room was developed in the metaverse to raise public awareness among healthcare students in Singapore on the “War Against Salt” [48]. The study’s participants, who belonged to Gen Zs, not only showed an improvement in knowledge, but were also actively engaged in gameplay.

The metaverse’s opportunities in healthcare lie in its integration with a range of other technologies, including 5G/6G networks, IoMT, haptics, holography, digital twins, computational security, and medical devices and facilities, on a greater socio-economic scale [49]. For example, beyond immersive visual rendering, multimodal interactions, such as sound and haptics, also play a critical role in creating a more realistic and engaging environment. Sound synthesis, propagation, and rendering can simulate acoustic environments in the hospital setting, thereby enhancing the sense of presence and immersivity by replicating real-world audio cues [50,51]. The advantages of incorporating sound are that it increases one’s sense of realism in the virtual world and is not just one-directional. The user can experience the sound effects regardless of where the person turns his head (or headset). On the other hand, haptic technologies are able to simulate the sense of touch through feedback on tactile devices, which, in addition to visual and auditory cues, can provide an added layer of immersivity in the virtual environment [51]. Studies using haptic feedback in surgery and surgical training have shown benefits in terms of higher accuracies, reductions in completion times and improvements in success rates of surgical tasks [52,53]. Furthermore, the in-house developed Virtual Aseptic Compounding training program at our hospital [54] afforded participants with the ability to touch, hold and manipulate objects (e.g., 3D-printed needles and syringes), which, when augmented with a virtual reality laboratory environment, enabled them to experience an enhanced degree of immersivity that would otherwise not have been possible with just the virtual reality headset. Thus, by integrating XR with other technologies, it can enhance the realism and immersivity of applications in the healthcare metaverse in both clinical and educational settings. However, to ensure seamless interaction between users and the virtual environment, there needs to be a robust infrastructure to integrate these technologies successfully. But with the hype about generative AI, is the metaverse only a fad or will it have a place in healthcare?

6. Generative AI + Metaverse: To Infinity and Beyond

With rapid advancements in AI technologies, AI is expected to play a more crucial role in enhancing the metaverse and boosting user engagement [55]. Generative AI is able to create dynamic new and original content (e.g., text, images, videos, music, 3D objects/assets, personalized avatars, etc.) so as to provide a unique and immersive experience for users. In fact, an intelligent metaverse for healthcare incorporating a multiverse of immersive technologies beyond XR has been envisioned for medical imaging-guided diagnosis and therapy [49]. Social connections and collaborations in the metaverse that are driven by multiple large language models (LLMs) will be accelerated in the future [56]. Generative AI’s ability to create dynamic video content in multiple styles, develop virtual actors, synthesize videos, animate avatars, and enable empathy-driven LLMs and multi-sensory interactions suggests that user avatars can potentially become more interactive, realistic, and can converse with AI-powered non-player characters (NPCs) in a more lifelike manner [55,56]. Furthermore, other technological enablers such as digital twins of individual patient avatars, ubiquitous access and sharing of secure medical data, an enhanced regulatory framework, and virtualized medical interventions based on patient-specific simulations in the metaverse can potentially revolutionize healthcare practices [49].

Research has shown that younger generations are increasingly integrating digital platforms, such as social media, gaming and the metaverse into their daily lives [57,58]. In fact, this generation that has grown and lived in both the physical and digital (i.e., “phygital”) realities has been coined “Generation Meta” [59]. A recent Nielson study predicted that these consumers will spend 4 to 5 h daily in the metaverse within the next 5 years [60]. Furthermore, McKinsey reported that in the next 5 years, the younger generations of consumers will perform activities such as immersive shopping, telehealth appointments, education, travel and socializing in XR in the metaverse [61]. A profound 38-fold increase in telehealth usage during COVID-19 indicates an increasing trend towards virtual healthcare [62]. Pioneering efforts in exploring the metaverse for healthcare [45,46,47,63] also suggest that the healthcare metaverse will eventually become a reality. In a healthcare education study that used an in-house developed Metaverse Art Gallery of Image Chronicles (MAGIC) to provide relatable insights into patients’ lived experiences of their chronic medications, generative AI and metaverse technologies were harnessed to create a virtual space where pharmacy learners could explore patient’s descriptions of their medication journeys through animated superhero–villain portraits [64]. The study’s results indicated that among their cohort of Gen Z learners, perceived playfulness, perceived compatibility to their learning styles, and social norms/influence had strong associations with their intention to use MAGIC for education. Hence, even though the metaverse is still at its beginning stages in healthcare, its opportunities in clinical practice, education and training, especially among the younger generations of HCPs, are highly promising.

7. Considerations When Designing and Developing Healthcare Innovations Using Immersive Technologies

When designing and developing healthcare innovations using immersive technologies, there are several factors that should be considered (

Table 1. Consideration factors when using immersive technologies to design and develop healthcare applications.

Consideration Factors	Immersive Technologies Spanning the Reality-Virtuality Spectrum
	Reality (Physical World)	Augmented Reality (AR)	Augmented Virtuality (AV)	Virtual Reality (VR)	Mixed Reality (MR)	Metaverse
Description	No digital augmentation.	Virtual objects overlaid in a physical environment.	Real-world objects in a virtual environment.	Fully immersive virtual environment, includes EVE and MVE.	A combined use of AR and VR in which virtual and physical objects and environments exist together and can interact with each other in real-time.	A combined virtual environment that is shared, and where the physical and virtual worlds are integrated and enhanced, covering the entire XR spectrum.
Technical requirements	None, except for physical tools and resources.	Examples include smartphones, iPads/tablets, or AR glasses/headset; with AR software.	VR setup with additional real-world object integration (e.g., haptic devices).	VR headset with or without controllers; may include haptic devices; requires powerful computing software.	MR headset, usually with sensors and integration software.	Advanced XR setups; users navigate virtual worlds; may involve cryptocurrencies and blockchain technologies.
Technical cost and scalability	Low cost; easily scalable.	Moderate cost to users; potentially scalable with devices that are widely available.	High cost; less scalable due to specialized equipment needs.	High cost; need for advanced hardware may limit scalability.	High cost; scalability depends on availability and compatibility of equipment and devices.	Very high cost; scalability depends on technological infrastructure and user adoption.
User experience	Users are familiar, no learning curve.	Device interfaces are usually familiar, generally easy to use.	High immersivity but can be disorienting to user.	Highly immersive, but some users may experience motion sickness.	Highly interactive and engaging, but users may need to adapt to the mixed environment.	Highly engaging and immersive, but can be overwhelming to users who are not familiar with immersive technologies.
Learning curve and adoption	No learning curve; universally understood.	Low to moderate learning curve; familiar interfaces but requires initial installation and setup.	High learning curve; requires familiarity with VR and physical interactions.	Steep learning curve due to specialized equipment, software and hardware.	Moderate to high learning curve; users may need to adapt to new ways of interacting with the mixed environment.	Steep learning curve due to the complexity of the metaverse environment and its applications.
Advantages	Completely realistic, no technological barriers.	Enhances real-world tasks with additional virtual information; generally accessible.	High level of immersivity with real-world relevance.	High to full immersivity experienced in a controlled environment; can be used for creating high fidelity, vivid, authentic and life-like scenarios and/or complex scenarios; improves patient safety as it allows learners to commit mistakes in a safe space, yet without experiencing negative consequences that may happen in clinical practices.	Versatile in applications as it combines the best aspects of AR and VR.	Advanced and comprehensive integration of virtual and physical worlds; vast potential for various applications when combined with other advanced technologies.
Disadvantages	Limited digital enhancements and interactivity, bound by physical space.	High costs of development and implementation; Limited immersivity depending on device capability; may lead to safety risks if users are distracted.	High cost, complex setup, may cause motion sickness in users.	High cost, complex setup, lack of face-to-face interaction; may cause motion sickness, disorientation or discomfort in users.	High cost, complex setup, may cause disorientation in users.	Still in early development; high costs of metaverse technologies may impede healthcare adoption; challenges with compatibility, interoperability and computational power of metaverse healthcare technologies.
Privacy concerns	Privacy concerns are related to the use of the physical space and/or data from wearable sensors.	Privacy concerns are in relation to capturing and usage of data (e.g., photos) in public places.	Privacy concerns are around the data and simulated environments; physical safety concerns during use.	Privacy and security concerns are around the collected data, especially within MVE scenarios.	Privacy and security concerns are around the collected data, potential safety issues during interactions with both the virtual and real-world environments.	Significant ethical and cyberprivacy concerns surrounding digital identities, virtual properties, data privacy and digital rights.
Future trends and potential	Limited to physical interactions. However, advancements in wearable sensors can potentially augment physical parameters.	Advancements in AR technologies may enable more seamless integration with daily activities.	Integration with real-world objects provides potential for more realistic simulations.	Highly immersive and realistic; potential for users to have full-body VR experiences; VR equipment may become more accessible in future.	Advancements in wearable technologies and sensors have the potential to make MR more seamless and integrated with user tasks and activities.	Evolving rapidly with uptake by various brands, societies and organizations; potential for large-scale adoptions through events in virtual worlds, integration with other advanced technologies, and adoption of virtual economies.

). Each technology has unique characteristics, different technical requirements and potential challenges that may impact its suitability for different healthcare applications and target audiences. One of the important considerations is the technical and infrastructural requirements, costs and scalability of the healthcare innovation. For example, AR can be implemented with devices that are readily available (e.g., smartphones, iPads/tablets, AR glasses/headset) and relatively low-cost to users, compared to VR and MR applications that require more advanced hardware, such as VR/MR headsets, haptic devices or sensors, which are more costly, and potentially less scalable. However, with the high-fidelity technology in VR, detailed and vivid, authentic and life-like scenarios can be created to increase engagement and interest in healthcare education, which also facilitates information retention in the learner [65]. In healthcare, perhaps one of the greatest benefits of using VR is that it allows learners to make mistakes (e.g., medical or medication-related errors) in a safe and controlled environment without facing negative real-world consequences that may cause patient safety issues [66]. On the other hand, even though the metaverse can provide a range of XR experiences, it does require a more highly complex technological infrastructure, which may include enhanced network capabilities, as well as processing speeds and performances of computers and servers to handle the huge amounts of structured and unstructured healthcare data that is exchanged. There will also be an additional layer of complexity when integrating healthcare technologies with the metaverse, as current technologies and devices are still not fully compatible and interoperable with healthcare ecosystems [67,68].

Another important consideration is the target audience, their user experience and learning curve, as these factors impact their ease of adoption of the healthcare innovation. For instance, innovations that make use of AR tap on the familiarity of users with smartphones and tablets, making it easier to adopt into healthcare practices. AR applications can also increase social interactions among its users [69]. For example, the well-known Pokémon Go game helped bring people together as players had to combine forces to fight and win battles in Pokémon gyms. In fact, the social nature of this game helped individuals with mild depression improve their mental health symptoms [70]. However, it has been suggested that such AR apps may also cause distraction to users, which may pose safety risks in certain hazardous situations, increasing the likelihood of accidents [71]. In contrast, VR, MR and metaverse technologies usually require more specialized equipment and training; hence, there is a steep learning curve for users of such healthcare innovations. Furthermore, some immersive environments may be disorienting to users, and they may require some time to adapt to the technology and virtual environment.

Last but not least, privacy and ethical concerns is another critical aspect of implementing immersive technologies due to the collection and potential use of personal and health-related data. For example, AR technologies may raise concerns about the capture and use of data (e.g., photos) in public places [72]. Privacy and cybersecurity concerns around the data collected in VR and MR environments may also occur [73,74]. The metaverse, however, has the most significant ethical and privacy concerns, especially in relation to digital identities, virtual properties, data privacy and digital rights [75,76]. As the healthcare innovation landscape continues to evolve, healthcare innovators, designers and developers are encouraged to be cognizant and follow ethical guidelines, and incorporate robust data protection measures so as to ensure sustainability when integrating these technologies into healthcare.

8. Readiness and Sustainability of Immersive Technologies in Healthcare

Integrating immersive technologies into daily hospital practices presents a challenge due to the lack of hospital infrastructures, as well as readiness of healthcare professionals, patients and caregivers to adopt and utilize these technologies for clinical care. There is often a steep learning curve in learning how to use these immersive technologies for patient care and education. Clinicians not only have to learn and become proficient with these technologies on top of their daily work, but also incorporate them into their workflows without compromising patient safety or care quality. On the other hand, patients and caregivers may also need training and support to understand and navigate these new technologies for their care delivery. Therefore, successful integration of immersive technologies into healthcare systems requires a purposeful investment in terms of effort, resources and support for all stakeholders.

Additionally, the future impact of immersive technologies in public healthcare depends on the preparedness of both the healthcare environment, and patients and caregivers. With the global phenomenon of a growing “silver tsunami” and the transition toward a “hospital-in-a-home”, there are currently still many infrastructural gaps in many homes, such as the lack of high-speed internet connections, monitoring devices and computers that support high-resolution and immersive simulation technologies. These gaps can lead to accessibility challenges for patients suffering from chronic medical conditions who may require constant use of immersive technologies for their care. Successful implementations of digital solutions in healthcare often rely on the digital literacy of both patients and caregivers [77]. However, with families becoming smaller units and an increasing aging population, the traditional model of caregiving, where children and grandchildren are often caregivers of their parents and grandparents, is under strain. The reduction in the number of younger individuals with greater digital and technological proficiency who can assist their less technologically savvy older parents and/or relatives may exacerbate the caregiver burden, which can include both physical and mental strain [77]. Consequently, this strain can result in reduced care provision and a lower quality of life for caregivers.

In order for immersive technologies to be a standard part of healthcare, efforts are needed to upskill healthcare professionals, patients and caregivers so that they are not only comfortable using these technologies, but are also able to leverage them to enhance their care. In addition, equitable access to immersive technologies must be considered so that patients and caregivers can benefit from these innovations regardless of socioeconomic status or geographic location. In summary, having an inclusive and digitally literate healthcare environment is essential for a successful and sustainable adoption of immersive technologies in healthcare settings. Creating an AI-enabled healthcare metaverse ecosystem will also require infrastructure that is capable of integrating the diversity of clinicians, patients, researchers, devices, and data on a massive social scale.

9. Conclusions

This article has explored the impacts of immersive technologies, including XR applications and the metaverse, in healthcare practice and education. Many of these applications are already transforming clinical practices. The applications described in this article only represent a fraction of what is possible. Furthermore, by integrating generative AI with the metaverse, dynamic and immersive experiences can be created for patients, healthcare professionals and healthcare students alike. While it may take several years before the healthcare metaverse becomes more widely accepted, ongoing advancements in immersive technologies suggest that we will see more of these healthcare innovations in the near future. To fully harness these opportunities, we need a new generation of interdisciplinary healthcare practitioners who are not only passionate but also possess the clinical expertise and technological skills to navigate the multiverse of immersive technologies in a more structured manner.