Emerging Technologies in Augmented Reality (AR) and Virtual Reality (VR) for Manufacturing Applications: A Comprehensive Review

Abstract

As manufacturing processes evolve towards greater automation and efficiency, the integration of augmented reality (AR) and virtual reality (VR) technologies has emerged as a transformative approach that offers innovative solutions to various challenges in manufacturing applications. This comprehensive review explores the recent technological advancements and applications of AR and VR within the context of manufacturing. This review also encompasses the utilization of AR and VR technologies across different stages of the manufacturing process, including design, prototyping, assembly, training, maintenance, and quality control. Furthermore, this review highlights the recent developments in hardware and software components that have facilitated the adoption of AR and VR in manufacturing environments. This comprehensive literature review identifies the emerging technologies that are driving AR and VR technology toward technological maturity for implementation in manufacturing applications. Finally, this review discusses the major difficulties in implementing AR and VR technologies in the manufacturing sectors.

1. Introduction

Industry 4.0 is transforming the current industrial environment through digitizing production processes. This is accomplished by implementing full network integration in all phases of product life cycle management and real-time data exchange, moving the current industry toward digitally enabled smart factories [1]. Industry 4.0 is driven by the nine fundamental technologies listed in Figure 1. Immersive technologies such as augmented reality (AR) and virtual reality (VR) are among those nine transformative technologies that are enabling digital transformation towards the factory of the future [2]. VR is defined as “the use of real-time digital computers and other specialized hardware and software to construct a simulation of an alternate world or environment, which is believable as real or true by the users” [3]. AR is a technique for “augmenting” the real world with digital things [4]. The characteristics of AR systems as being able to geometrically match the virtual and real worlds and run interactively in real-time. AR must have three main characteristics: combining real and virtual worlds, real-time interaction, and registration in the 3D environment [5]. The combination of a real environment scene and a virtual environment is known as Mixed Reality (MR). The term “Mixed Reality” refers to applications where “real world and virtual world items are exhibited simultaneously within a single display, that is, anywhere between the extrema of the virtuality continuum [4].” While AR takes place in a physical setting with virtual information added, MR combines physical reality with virtual reality, allowing interaction between the physical and virtual world [4]. For VR systems, the real and digital worlds are blended to fully immerse the user in a computer-generated environment. This includes a fully immersive display, often in the form of a headset, sensors to track user movement and orientation, controllers or gloves for user interaction within the virtual environment, and a computational unit, often a PC or a dedicated system which renders the virtual world in real-time [6,7]. The Reality–Virtuality (RV) Continuum is depicted in Figure 2, which illustrates the distinction between AR, VR, and MR [4].

In recent years, there has been a substantial and rapid evolution of AR and VR technologies. These advancements have been recognized for their potential to enhance learning methodologies and improve task execution, particularly within contexts characterized by dynamic and adaptable requirements [8]. AR and VR are leveraging the advantages of state-of-the-art technologies, like the Industrial Internet of Things (IIoT), Cyber–Physical Systems (CPSs), and cloud computing making manufacturing processes smarter and more efficient [9]. People can now interact with the manufacturing process efficiently, reducing the downtime of the manufacturing processes and leading to positive economic growth [10].

The rapid evolution of manufacturing processes towards automation and efficiency, alongside the increasing integration of AR and VR technologies, serves as the primary motivation for this comprehensive review. Despite growing recognition of the potential benefits of AR and VR in manufacturing, there exists a notable research gap in synthesizing and critically evaluating recent technological advancements and applications within this context. This review aims to address this gap by systematically examining the current applications, state-of-the-art technologies, and challenges hindering widespread adaptation of AR and VR in manufacturing. By offering insights into emerging trends, this review aims to contribute to the advancement of AR and VR technologies in manufacturing applications.

This paper is organized into four sections. The proposal is introduced in Section 1. Section 2 explains the methodology for this literature review. Section 3 discusses the findings of the literature review by answering the research questions (RQs). It highlights the detailed applications of AR and VR in the manufacturing industry, summarizes the key findings and emerging technologies providing the statistical analysis to find the research trends, and the challenges of implementing AR and VR technologies in manufacturing applications. In Section 4, the literature review is concluded.

2. Review Methodology

This paper utilizes a systematic approach to focus on cutting-edge research regarding AR and VR applications in manufacturing, spanning the period of 2015 to 2022. The primary goal was to collect key insights from relevant papers found in peer-reviewed journals and conferences. Through a thorough evaluation, the study aims to identify challenges in various aspects of AR and VR implementations for manufacturing and identify research trends. To conduct this literature review, the following steps were undertaken: planning the review methodology, defining the scope, conducting article searches, evaluating the identified articles, synthesizing relevant information, and analyzing the results [11]. The steps of the review methodology are illustrated in Figure 3.

2.1. Planning

The initial step involved formulating research questions to guide the research methodology, with a focus on identifying current technological trends and challenges of AR and VR technologies for manufacturing applications. The research questions are outlined in

Table 1. Research questions and the objectives of the research questions.

Research Question	Objective
RQ1: What are the current applications of AR/VR in manufacturing?	This research question aims to identify and understand the current trends in the applications and implementations of augmented reality (AR) and virtual reality (VR) technologies in manufacturing sectors.
RQ2: What are the state-of-the-art technologies used in AR/VR applications in manufacturing?	The main objective of this research question is to gather the latest hardware and software used in AR/VR technologies for manufacturing applications.
RQ3: What are the emerging technologies in the field of AR/VR applications in manufacturing?	This research question seeks to identify the emerging technologies that are driving innovation in the field of AR/VR application within manufacturing. The goal is to understand the technological trends in modern manufacturing practices.
RQ4: What are the challenges for the adaptation of AR/VR applications in manufacturing?	The main idea of this research question is to address the challenges and difficulties of implementing AR/VR-based applications in the context of the manufacturing industry.

. Subsequently, the Web of Science, www.webofscience.com (accessed between 1 September 2022 and 21 October 2024) database and Google Scholar (www.scholar.google.com, accessed between 1 September 2022 and 21 October 2024) search platform were used for gathering the peer-reviewed research articles as they are recognized for their reliability in providing peer-reviewed research articles.

2.2. Article Search

To collect research articles, a combination of search strings was used, which are listed in

Table 2. List of search strings.

#	Search String	Search Platform
1	“AR” AND “Manufacturing”	Web of Science Google Scholar
2	“Augmented Reality” AND “Manufacturing”
3	“Virtual Reality” AND “Manufacturing”
4	“VR” AND “Manufacturing”
5	“Augmented Reality” AND “Industry 4.0”
6	“AR” AND “Industry 4.0”
7	“Virtual Reality” AND “Industry 4.0”
8	“VR” AND “Industry 4.0”
9	“Augmented Reality” AND “Factory”
10	“AR” AND “Factory”
11	“Virtual Reality” AND “Factory”
12	“VR” AND “Factory”

. For the initial search, the research articles that were newer than 2014 were included. Studies older than 2015 have not been included as these studies lose their relevance with the current technological advancement of AR and VR technologies. The statistics of the collected articles are listed in

Table 3. Statistics of collected articles.

Search Platform	Search Statistics
Web of Science, Google Scholar	Papers Retrieved	297
	Irrelevant Papers	50
	Duplicate Entries	36

. Only articles in English were included and commercial and repeated studies were excluded.

2.3. Initial Screening

In this step, titles and abstracts were screened to find the relevant papers that support the research questions. Articles related only to the AR and VR manufacturing applications were included. Relevant papers were also gathered by screening the references of the collected articles.

2.4. Quality Screening

To check the quality of the paper, the introduction, methodology, results, discussion, and conclusion of the collected papers were looked at to ensure the quality of the research and determine relevant research articles. The goal was to have a wide range of papers that would cover key topics of AR and VR. This would ensure that the information extracted from these articles would give an overview of all aspects of manufacturing.

2.5. Data Extraction

In this step, key elements from the research articles were extracted and stored for further quantitative and qualitative analysis of the research.

2.6. Reporting

In this final stage, the extracted data was reviewed and is reported in Section 3. Different categories of papers and specific data were analyzed to obtain a detailed overview of the AR- and VR-based manufacturing applications.

3. Results and Discussion

This section highlights key findings and insights derived from the reviewed literature. After a careful review of the collected articles, the answers to the research questions were compiled.

3.1. RQ1: What Are the Current Applications of AR/VR in Manufacturing?

To begin addressing Research Question 1 (RQ1) on the current applications of AR and VR in manufacturing, the literature review identifies the common applications where AR and VR are used within the manufacturing sector.

3.1.1. Maintenance Applications

Maintenance, a critical facet of manufacturing, is important for keeping production lines working smoothly and ensuring product quality. A large portion of research focuses on this area as it can increase costs significantly and impact safety [12]. AR and VR have emerged as potent tools in the domain of maintenance, offering substantial potential for cost reduction by streamlining troubleshooting procedures, expediting access to documentation, and pinpointing malfunction locations. A study documented in [2] underscores the advantages of graphical representations over text-based information, as they facilitate quicker and more effective information processing. Using AR instructions for maintenance has been proven to be effective, with real-world evidence showing it can cut maintenance time for industrial machinery by up to 30% [10]. An AR-based maintenance operation in which symbols/arrows are used to show screw locations during repair is depicted in Figure 4. Many of the papers found in maintenance applications showed the use of Head-Mounted Displays (HMDs) as they allowed both hands to be free during repair. Some applications, however, use Handheld Devices (HHDs) for some troubleshooting and diagnostic scenarios. This could involve looking into machines or walls where equipment and parts are not necessarily visible.

3.1.2. Assembly Applications

AR and VR in the field of assembly applications are one of the most widely used. Studies show that AR and VR solutions, when compared to paper-based instructions, are superior in assembly tasks, demonstrating a decrease in both time to complete and error rate [13]. Moreover, they exhibit a larger decrease in these aspects as the tasks become more complex. Participants in the study who were not experienced in analyzing complex isometric drawings were able to understand them for their assembly task. The learning curve also steepened compared to paper-based instructions, showing a decrease in time for users to become familiar with a reoccurring task. Studies also showed that when comparing experts and untrained workers with assembly tasks, both categories of workers increased in their efficiency and decreased in errors. An example of an AR-based assembly application is shown in Figure 5.

3.1.3. Operation Applications

Shop floor operators show a lot of movement on the floor of a manufacturing plant, moving to different tasks, to gather materials, or to obtain information. Operation applications focus on decreasing the time of movement of operators around the plant and customizing routes to be the most efficient [2]. They also describe the planning method in manufacturing as well as the setup of machines during production change [14]. In Figure 6, machine data is sent to the headset where the errors are determined and displayed to the user. In this image, basic lab data is shown to the user and tells them the location of tasks needed to be conducted. The study makes an example of using material restocking as one of the operator tasks. Sensors under the material tray station determine if material is out of stock. The headset then displays this error to the user, telling them the material type and location to stock the material.

3.1.4. Training Applications

Training systems play a pivotal role in modern workplaces, alleviating the time constraints faced by trainers and enhancing the overall learning experience. AR and VR have gained prominence in these contexts due to their ability to engage users effectively, thereby amplifying learning performance and motivation [15]. It is essential to note that the term “training” encompasses the concepts discussed in the preceding sections, all of which contribute to the comprehensive training of new employees. Training applications not only aim to familiarize users with AR and VR interfaces but also strive to foster self-sufficiency to the point where AR and VR become unnecessary. Additionally, these applications facilitate the seamless integration of tasks regardless of the worker’s skill level. In manufacturing industries, where onboarding and training processes for new employees can span several weeks, such systems become indispensable. By reducing the time required by already trained employees, they allow these individuals to focus on their core responsibilities.

3.1.5. Product Design Applications

Product design and prototyping are integral parts of manufacturing processes. The success of any design project depends on effective prototyping [16,17]. Designers have been predominantly using Computer-Aided Design (CAD) tools since the 1990s for the design, development, prototyping, and simulation of new products [18]. With the technological maturity of AR and VR hardware and software, designers are adopting AR and VR technologies as extensions of the CAD tools for better visualization and realization of the product design. Using these technologies, product designs are overlaid onto the physical setup or users can interact with the product design and assemblies in an immersive environment using AR and VR devices, making the design process more realistic and collaborative. AR and VR can be incorporated into every stage of the product design life cycle [18] (Figure 7). AR and VR can effectively be used for remote design collaboration by bringing different remote users into one immersive platform. In one study, authors utilized a model-based definition approach and AR to develop an industrial product inspection using 3D CAD software and product manufacturing information (PMI) data transfer methods [19]. Despite the potential significant benefits of utilizing AR and VR in design applications, AR and VR technologies are not adopted in a wide range of design applications. The authors proposed a Double Diamond design model to address this issue by utilizing systematic AR case identification, a criteria catalog for appropriate AR device selection, and the recommendation of modular software architecture for efficient customization and data integration within enterprise IT frameworks [20].

3.1.6. Quality Control Applications

Continuous monitoring of the production process and the evaluation of product quality are an essential part of the manufacturing process. Quality control helps manufacturers identify and address quality-based issues in real-time and helps to enhance the consistency of the process, reduce product defects, and improve overall production efficiency. Computer vision technology, sensor-based defect detection, and non-destructive testing are general technologies that are predominantly used for defect detection. AR and VR technologies are seen to be emerging in the area of quality control and visualization. Several research studies have been conducted to incorporate AR and VR technologies into manufacturing. The authors of [21] identified the three main areas of AR and VR application, depicted in Figure 8, for quality control in manufacturing: immersive lean tool, metrological inspection, and in-line quality assessment. The authors of [22] utilized 3DS MAX and Unreal Engine 4 to develop a virtual simulation, shown in Figure 9, for teaching machine learning-based beer bottle defect detection, encompassing bottom, mouth, and body defects. The system was evaluated with traditional experiments, where 40 junior students preferred the immersive and interactive virtual experiment, highlighting its effectiveness in the learning process. In one study, researchers employed AR technology for real-time monitoring of polished surface quality in a SYMPLEXITY robotic cell with an ABB robot, visualizing metrological data onto the parts to check if the quality reached the required specifications [23]. Some AR- and VR-based quality control applications in manufacturing are listed in Table 4.

In summary, the major applications of AR and VR in manufacturing encompass maintenance, assembly, operations, training, product design, and quality control, each demonstrating substantial benefits in improving efficiency, reducing errors, and enhancing overall productivity. However, the potential of these technologies extends beyond these core areas to include inventory management, supply chain optimization, customer engagement, and more.

3.2. RQ2: What Are the State-of-the-Art Technologies Used in AR and VR Applications in Manufacturing?

Addressing Research Question 2 (RQ2) regarding the state-of-the-art technologies used in AR and VR applications within manufacturing requires a comprehensive exploration of cutting-edge tools and platforms utilized in the industry. This investigation delves into key components, hardware, and software that are vital for the implementation of AR and VR technologies across various manufacturing processes.

There are several aspects of AR and VR applications and devices that have been recognized by most papers to be necessary for creating a system that incorporates smart manufacturing paradigms. Several reoccurring ideal features depicted in Figure 10 can be seen for AR and VR systems: interoperability, virtualization, decentralization, real-time capabilities, and modularity [24,25]. Interoperability is one of the largest contributors to smart manufacturing, allowing for communication between the user and the machines in the manufacturing environment through the Internet of Things (IoT) and other industry standards. Standardization of these devices is also crucial for the system to be robust throughout multiple scenes [10]. Virtualization is necessary for calibrating devices and creating the digital environment and simulation models using Cyber–Physical Systems (CPSs). CPSs can use this information to monitor processes that are happening in real-time and notify the user of changes in the scene. Decentralization enables easy access to diverse work instructions, providing users with specific information and procedures without manual searching, saving valuable time [24,25]. Real-time capabilities are especially important as it is crucial for the users to have up-to-date information on how the status of machines and operations are changing as well as updating technical machine data to keep plant records current. This also applies to the use of the application, as tracking technologies are needed to process data in real-time. Finally, the modularity of the application keeps it flexible to changes in the environment and allows the system to adapt to specific scenarios in the scene. This means that documentation on new processes and procedures needs to be up to date and modularized to specific use cases [24].

Table 4. Quality control applications in manufacturing using AR and VR.

Research Group	Application	Type of Quality Control
[22]	Virtual defect detection	In-line process monitoring
[23]	Quality assessment of polished surface	Metrological inspection
[26]	Quality inspection	In-line process monitoring
[27]	PCBA Inspection	In-line process monitoring
[28]	Predictive Maintenance	Lean operation
[29]	Car panel alignment	Metrological inspection
[30]	Automatic aviation connectors inspection	In-line process monitoring
[31,32]	Quality control of car body	In-line process monitoring
[33]	Non-destructive condition monitoring of IGBT wafer	In-line process monitoring
[34]	Identification of bottlenecks of a production line	Lean operation

Table 4. Quality control applications in manufacturing using AR and VR.

Research Group	Application	Type of Quality Control
[22]	Virtual defect detection	In-line process monitoring
[23]	Quality assessment of polished surface	Metrological inspection
[26]	Quality inspection	In-line process monitoring
[27]	PCBA Inspection	In-line process monitoring
[28]	Predictive Maintenance	Lean operation
[29]	Car panel alignment	Metrological inspection
[30]	Automatic aviation connectors inspection	In-line process monitoring
[31,32]	Quality control of car body	In-line process monitoring
[33]	Non-destructive condition monitoring of IGBT wafer	In-line process monitoring
[34]	Identification of bottlenecks of a production line	Lean operation

There are certain key components of AR and VR technologies that are required for the implementation of AR- and VR-based applications in manufacturing. In this section, the key components, specifically hardware and software, utilized in AR- and VR-based manufacturing applications are reviewed and discussed at length.

3.2.1. Hardware Used in AR Applications

The components of an AR hardware system include tracking devices and haptic and force feedback rendering devices, displays, and sensors [35]. AR display/visualization devices can be divided into three types: HMD, HHD, and Spatial Display (SD).

An HMD is a display device that is worn on the head or as a component of a helmet, and it has a display optic in front of one (monocular HMD) or both eyes (binocular HMD). In AR applications, HMDs are frequently used because the eye-level display enables users to experience the AR scene hands-free. HMD devices are the preferred choice for AR applications in manufacturing. They offer hands-free operation, which enables the workers to interact with manufacturing systems and access virtual overlayed information. Additionally, for AR systems, HMDs overlay real-time data directly onto the physical world, optimizing the efficiency and accuracy of the manufacturing operation. There are many commercially available AR visualization devices in the market.

HHDs enable users to visualize the AR environment when the display is held in reference to a specific environment [36]. These can include tablets and cellphones that have screens and cameras available to receive and display data.

SDs are mostly projection-based displays that do not require users to wear Head-Mounted Displays and instead project information into the actual surroundings [35]. SDs in manufacturing can project assembly instructions and quality inspection details directly onto parts. Additionally, they enhance training and safety by overlaying crucial data and guidelines directly onto the shop floor or machinery [37].

3.2.2. Tracking Systems for AR Applications

In manufacturing applications, various tracking systems are employed, including marker-based, model-based, and feature-based tracking. Among these, marker-based tracking has widespread popularity due to its robustness and ease of setup. Model- and feature-based tracking methods are also employed, often in conjunction with marker-based tracking. These tracking systems enable the AR headset to precisely determine the location of objects within the environment and establish the spatial positioning of the AR system itself [10]. Notably, these tracking systems can be integrated in a hybrid fashion, using multiple tracking methods at once, allowing for more versatile and accurate tracking solutions.

Marker-based tracking involves the utilization of external visual aids, such as ArUco markers or infrared markers, to precisely determine the spatial position of objects. However, these tracking systems come with distinct advantages and disadvantages. Marker-based tracking stands out as a highly reliable solution due to its minimal external processing requirements, resulting in low-latency performance. Additionally, these markers remain stationary, mitigating the need for the AR application to rely on other reference points or external data sources. For marker-based tracking to function effectively, the markers must remain within the field of view of the camera, ensuring the viability and accuracy of the tracking system [38]

Model-based tracking utilizes CAD models to locate components in the environment, leveraging prior knowledge of object shapes and appearances for AR interpretation. By placing markers on CAD models during setup, the system can accurately track and display virtual information, and when combined with digital twins, it provides a highly accurate representation of the work environment [2]. These models can be used with a deep learning program to evaluate different forms of the object, which is especially useful when interacting with objects that move like robots.

Feature-based tracking brings objects into the digital domain by considering their points, using the object’s features as markers, similar to marker-based tracking. It relies on a point cloud model combined with CAD models to identify and track objects, providing a way to determine an object’s identity and follow its movements [39].

3.2.3. Hardware Used in VR Applications

VR applications require a range of hardware to deliver immersive experiences. Various types of visualization displays are used for VR applications such as HMDs like Meta Quest, HHDs like tablets or PCs, and some Personal Computers are also used for simulation and digital twin applications [35]. Different tracking sensors such as camera, motion, and gaze are used to track the headset, controllers, and human actions. Haptic systems are also an important part of the VR hardware system. Input devices range from motion controllers to eye-tracking modules. Positional tracking encompasses sensors and treadmills, while haptic feedback offers tactile sensations. Lastly, the VR experience can be powered by VR-ready PCs or wearable backpack computers, complemented by spatial audio hardware for sound immersion [40].

3.2.4. Software Used in AR and VR Applications

The field of AR and VR is continuously evolving with new software tools and platforms. Game engines are often used to develop AR and VR applications as they use extensive computation to overlay virtually generated information into 3D spaces. Unity 3D and Unreal Engine are two of the most popular game engines for developing AR and VR applications [41]. Among those two, Unity 3D is mostly adopted by developers to build AR and VR applications due to its cross-platform capabilities. Along with these game engines, developers also use different AR and VR libraries for application development. AR libraries such as Vuforia, ARToolKit, ARCore, ARKit, Mixed Reality Tool Kit (MRTK), and Wikitude are widely implemented. Vuforia is one of the widely used AR libraries for AR application development, ARKit is used for iOS-based AR application development, and ARCore is used for developing AR applications on Android devices. Wikitude is a cross-platform AR library. MRTK is necessary for the development of AR applications using HoloLens. It has also become widely used in the field as recently it has become an open source for other glass types to use. For VR development, there are different Software Development Kits (SDKs) available in the market. OpenVR is an SDK that enables programs to use different VR devices without needing to know the specifics of VR devices [42]. Other popular VR SDKs are Oculus SDK, SteamVR SDK, and Google VR SDK [43]. Developers use VR platforms such as Steam VR, Oculus Home, and Windows Mixed Reality, which offer comprehensive ecosystems to share VR content. Developers leverage these technologies to develop interactive, easy-to-share VR applications for manufacturing applications [44].

3.2.5. Overview of Hardware and Software Used in AR/VR Applications in Manufacturing

The review articles were scrutinized to determine the trend of software and hardware used in AR and VR applications, as illustrated in

Table 5. Overview of AR and VR hardware and software used for manufacturing applications.

Application Type	Research Group	Mounting Type	AR/VR Glass Type	Tracking Type	Software	AR/VR
Maintenance	[45]	HMD	HTC Vive Pro	N/A	Unity, Virtual Reality Toolkit, Azure Speech SDK	VR
	[28]	HMD	HoloLens	Marker-based and feature-based	Unity, Vuforia, Python 3.7 for fault prediction	AR
	[46]	HMD	HoloLens	Feature-based	Unity	AR
	[47]	HHD	Mobile	Marker-based	Unity, Vuforia	AR
	[8]	HHD	Ipad Air	Feature-based	iOS, Metaio SDK 5.5	AR
	[48]	HHD	Tablet	Not Mentioned	Unity, Vuforia	AR
	[49]	Desktop/Laptop, HMD, HHD	Goggles, laptop PC, mobile device, Vuzix star 1200xl	Marker-based	Unity, Vuforia	AR
	[24]	Projector	DLP projector Benq W1080ST+	Marker-based	Unity, ARToolkit	AR
	[50]	HHD	Android Device	Model Target	Unity, Vuforia	AR
	[51]	HHD	Not Mentioned	Not Mentioned	Unity, Vuforia	AR
	[52]	HMD	Not Mentioned	Head–gaze	Unity, Vuforia	AR
	[53]	HHD	Samsung Tablet	Model Target	Unity, Vuforia	AR
	[54]	Desktop/Laptop	Not Mentioned	N/A	Unity, AvatarSDK, Virtual Reality Toolkit, Photon Unity Networking	VR
	[55]	HHD	Samsung Galaxy S10, iPhone 11	Feature-based	YOLOv5, Roboflow, VoTT, PyTorch, Ffmpeg	AR
	[56]	Not mentioned	Not Mentioned	Not Mentioned	FreeCAD	AR
	[57]	HMD	HoloLens	Not Mentioned	Not Mentioned	AR
	[58]	HMD, Desktop/Laptop	HMD, Desktop/Laptop	Marker-based	Unity, Vuforia	AR
	[57]	HHD	Tablet	Model Target Tacking	Unity	AR
	[59]	HMD	Headset	Not Mentioned	Unity, Vuforia	AR
	[60]	HMD	Headset	N/A	N/A	AR
	[61]	HMD	Headset	Not Mentioned	Unity	AR
	[62]	Desktop/Laptop	Desktop	Not Mentioned	Unity	AR
	[63]	HMD	Headset	Feature-based	Unity	AR
Design	[64]	Projector	IR-RGB Dual-Input Projector	Not Mentioned	Not Mentioned	AR
	[65]	HMD	HoloLens	Not Mentioned	Unity	AR
	[66]	Projector	Three-walled immersive projection environment called METaL	N/A	Siemens PLM Software	VR
	[20]	N/A	N/A	N/A	Double Diamond design process model	AR
	[67]	HMD, Desktop/Laptop, HHD	HoloLens, PC, Andriod Tablet	Not Mentioned	Unity, Mixed Reality Toolkit, Vuforia	AR
	[19]	HMD	HoloLens	Model Target	Unity, Vuforia, PiXYZ Unity Plugin	AR
	[12]	HMD	HoloLens	Feature-based	Unity, Vuforia, Mixed Reality Toolkit	AR
	[68]	HHD	Android Mobile	Maker	ANSYS, Vuforia Android Application	AR
	[69]	HMD	HTC Vive VR Headset	N/A	Unity, PUN2 Library	VR
	[70]	HMD	Oculus Quest VR Headset	N/A	Unreal Engine	VR
	[71]	HMD	Oculus Rift S VR Headset	N/A	Flexsim	VR
	[72]	HMD	Not Mentioned	Feature-based	Vuforia	AR/VR
	[73]	HMD	HTC Vive	N/A	Not Mentioned	VR
	[74]	HMD	Not Mentioned	N/A	Unity	VR
Quality Control	[75]	HMD	Vuzix Wrap 920AR	Marker-based	OpenSceneGraph, ARToolkit	AR
	[76]	HHD	Tablet PC	Marker-based	Unity, Vuforia	AR
	[77]	Not Mentioned	Not Mentioned	N/A	Unity	VR
	[78]	HHD	Lenovo Phab 2 Pro Smartphone	Marker-based	Unity, Google Project Tango Development Kit	AR
	[23]	HMD	HoloLens	Feature-based	Unity, Mixed Reality Toolkit	AR
	[79]	HMD	HoloLens	Marker-based	Unity	AR
	[27]	HMD	HoloLens	Feature-based	Unity, Mixed Reality Toolkit	AR
	[80]	HMD	HP Reverb VR Pro	N/A	Unity	VR
	[26]	HHD	Samsung Galaxy Tab S4	Marker-based	Unity, Google ARCore SDK	AR
	[81]	HMD	HoloLens	Marker-based	Unity, Mixed Reality Toolkit	AR
	[38]	HMD, HHD	HoloLens, Andriod Mobile Device, Moverio BT300	Marker-based	Unity, ARCore, Google Tango, ARToolKit	AR
	[82]	HHD	Phone/Tablet	Feature-based	Mask-RNN Algorithm	AR
	[83]	HMD, HHD	HoloLens, Samsung S7 and Samsung Galaxy Tab	Location-based	Unity	AR
	[84]	HMD	HoloLens 2	Marker-based	Unity, MobileNet-v2	AR
	[31]	HMD	HoloLens 2	Feature-based	Blender, Siemens NX	AR
	[29]	HMD, HHD	Not Mentioned	Marker-based	Unity, Vuforia	AR
	[32]	HMD	HoloLens	Marker-based	Unity, Mixed Reality Toolkit	AR
	[30]	Desktop/Laptop	Not Mentioned	Feature-based	YoloV3	AR
	[33]	Desktop/Laptop	Desktop/Laptop	Feature-based	Not Mentioned	AR
	[34]	HHD	Apple Smartphones and Tablets	Feature-based	iOS, Apple ARKit	AR
	[26]	HHD	Samsung Galaxy Tab S4	Marker-based	Unity, ARCore	AR
	[22]	Desktop/Laptop	PC	N/A	Unreal Engine 4	VR
	[85]	HHD	Mobile	Feature-based	Unity, Microsoft Azure, ARKit	AR
	[86]	HHD/HMD	Not Mentioned	Feature-Based	Unity, Arraycast	AR
Training	[87]	HMD	HTC Vive	N/A	Unity, Steam VR	AR
	[15]	Desktop/Laptop	Personal Computer	Marker-based	ARToolkit, Solid edge for CAD Modeling, Optical Flow	AR/VR
	[88]	HMD	HoloLens	Feature-based	Windows 10 system, Visual Studios 2017 Community	AR
	[89]	HHD	Leap Motion Controller	Marker-based	Unity, Vuforia, NX	AR/VR
	[39]	HHD	Mobile	Marker-based	Unity, Vuforia, SolidWorks, 3DSMax	AR
	[75]	HMD	Vuzix Wrap 920AR	Marker-based	OpenSceneGraph, ARToolkit	AR
	[8]	HHD	Ipad Air	Feature-based	iOS, Metaio SDK 5.5	AR
	[48]	HHD	Tablet	Not Mentioned	Unity, Vuforia	AR
	[50]	HHD	Android Device	Model Target	Unity, Vuforia	AR
	[90]	HMD	Headset, Smart Glass	Marker-based	Unity, Vuforia	AR/VR
	[76]	HHD	Tablet PC	Marker-based	Unity, Vuforia	AR
	[51]	HHD	Not Mentioned	Not Mentioned	Unity, Vuforia	AR
	[91]	HMD	HoloLens	Not Mentioned	Not Mentioned	AR
	[92]	HHD	Mobile	Model Target Tacking	OpenCV, C++	AR
	[93]	HMD	HTC Vive	Not Mentioned	Unity, OpenCV	AR
	[39]	HHD	Mobile	Marker-based	Unity, Vuforia	AR
	[94]	HMD	HoloLens	Gaze marker-based	Unity, Mixed Reality Toolkit	AR
	[95]	Projector	VPL-DX271 Projector	Feature-based	Not Mentioned	AR
	[96]	Desktop/Laptop	Monitor	Feature-based	Not Mentioned	AR
	[97]	HHD	Mobile	Marker-based	Vuforia	AR
	[98]	HMD	HoloLens	Not Mentioned	Unity	AR
	[99]	HMD, HHD	Android device, HoloLens	Marker-based, Feature-based	Unity, Vuforia, Mixed Reality ToolKit	AR
	[100]	HMD	HoloLens	Not Mentioned	Unity, Mixed Reality Toolkit	AR
	[101]	HMD	ACER Windows Mixed Reality HMD	N/A	Unity	VR
	[102]	HMD	HTC Vive Pro	N/A	Unity, SteamVR	VR
	[103]	HMD	HoloLens 2	Model Target Tracking	Unity, Vuforia version 8.3.8 object tracking	AR
	[104]	HMD	HoloLens 2	Marker-based	Unity, Mixed Reality Toolkit	AR
	[105]	HMD	HTC Vive VR headset	N/A	Unity	VR
	[106]	HMD	HoloLens 2	Marker-based	Unity	AR/VR
	[107]	HHD	iPhone XS	Marker-based	ARKit	AR
	[108]	HMD, Desktop/Laptop	Samsung Odyssey+ VR Headset	N/A	Unity	VR
	[109]	HMD	HTC Vive VR Headset	N/A	Unity, Tobii eye-tracking technology	VR
	[110]	HMD	Techlens T2	Feature-based	Unity, EasyAR, OpenCV	AR
	[111]	HMD	HoloLens	Feature-based	Unity, Vuforia	AR
	[112]	HMD	HoloLens 2	Feature-based	Unity	AR
	[113]	HHD	Samsung Galaxy Tab 3	Marker-based	Unity, Vuforia	AR
	[114]	HMD	HTC Vive	N/A	Unity	VR
	[115]	HMD	HTC Vive Cosmos	N/A	Unity	VR
	[74]	HMD	Not Mentioned	N/A	Not Mentioned	VR
Assembly	[116]	HHD	Android Smartwatch	Marker-based	Unity, Vuforia	AR
	[13]	HHD	LCD Screen	Marker-based	ARToolkitplus	AR
	[117]	HMD	HoloLens	Not Mentioned	Not Mentioned	AR
	[24]	Projector	DLP projector Benq W1080ST+	Marker-based	Unity, ARToolkit	AR
	[118]	HMD, HHD	HoloLens, Tablet	Marker-based	Unity, Mixed Reality Toolkit	AR
	[15]	Desktop/Laptop	Personal Computer	Marker-based	ARToolkit	AR
	[119]	Desktop/Laptop	Desktop PC	Feature-based	Unity	AR
	[120]	HMD, HHD	Mobile, Head-Mounted Display	Marker-based	Unity, Vuforia	AR
	[121]	Not Mentioned	Not Mentioned	Feature-based	Not Mentioned	AR
	[122]	Desktop/Laptop	Personal Computer	Marker-based	Unity, Vuforia	AR/VR
	[123]	HHD	Mobile	Marker-based	Unity, ARToolKit	AR
	[124]	HMD	Not Mentioned	Marker-based	Unity	AR/VR
	[125]	HMD	HoloLens	Not Mentioned	Unity, Holotoolkit	AR
	[126]	Desktop/Laptop	Laptop	Marker-based	Unity, Vuforia	AR
	[127]	Desktop/Laptop	Monitor	Marker-based	Unity	AR
	[128]	HHD	Mobile Device	Model Target Tacking	Unity, Vuforia	AR
	[129]	HMD	HoloLens	Marker-based	Unity, Vuforia, Mixed Reality Toolkit	AR
	[130]	Projector	VPL-DX271 projector	Feature-based	Not Mentioned	AR
	[131]	Projector	VPL-DX271 projector	Feature-based	Not Mentioned	AR
	[132]	HMD	Sony Smart Eyeglass Sed-E1	Marker-based	Not Mentioned	AR
	[133]	Projector	Not Mentioned	N/A	Unity	AR
	[134]	HMD	HoloLens 2	Feature-based	Unity, Vuforia	AR
	[135]	HHD	Not Mentioned	Marker-based	Unity, Vuforia	AR
	[136]	Desktop/Laptop	Emulator	N/A	Unity, ViCor	VR
	[137]	HMD	Oculus Rift	N/A	Not Mentioned	VR
	[138]	HMD	HTC Vive	N/A	Unity	VR
Operation	[78]	HHD	Lenovo Phab 2 Pro smartphone	Marker-based	Google Project Tango Development Kit	AR
	[139]	HHD	iPhone 7	Marker-based	Unity, Vuforia, ARKit	AR
	[140]	Projector	Not Mentioned	Feature-based	Not Mentioned	AR
	[141]	HMD	HoloLens	Marker-based	Unity	AR
	[142]	HMD	HoloLens	Marker-based	Unity, Vuforia	AR
	[143]	HHD	Tablet	Not Mentioned	No Mentioned	AR
	[144]	HHD	iPhone	Marker-based	Not Mentioned	AR
	[145]	HMD	HoloLens 2	Feature-based	Unity, Vuforia	AR
	[146]	HMD	HoloLens 2	Not Mentioned	Unity	AR
	[147]	HHD	Not Mentioned	Marker-based	Unity, Vuforia	AR
	[148]	HMD	HoloLens 2	Not Mentioned	Not Mentioned	AR

. The table offers a snapshot of various AR and VR applications, highlighting the diverse range of hardware and software technologies employed across different research groups. It encompasses different types of display devices, tracking technologies, software platforms, and application domains.

Across the surveyed studies, Unity emerges as the most dominant software platform, widely adopted for both AR and VR application development. In AR-specific implementations, Unity is often paired with the Vuforia SDK, particularly for marker-based and model target tracking applications. Other software tools frequently employed include ARToolkit, Mixed Reality Toolkit, Unreal Engine, Flexism and several Artificial Intelligence frameworks such as YOLOv5, PyTorch, and Mask-RNN, especially in advanced AR applications for quality inspection and object recognition.

HMDs are the most commonly used hardware, with Microsoft HoloLens leading among AR systems. HTC Vive, Oculus Rift, and Oculus Quest are widely used in VR applications. These VR headsets utilize advanced tracking technologies like SteamVR and Tobii eye-tracking. HMDs are favored in use cases requiring immersive interaction, hands-free operation, or precise spatial visualization.

In contrast, HHDs such as tablets and smartphones are commonly adopted in AR applications involving mobile or field-based operations, particularly for maintenance and inspection tasks. A smaller number of studies utilize projector-based systems or desktop computers, typically in collaborative AR environments or VR simulation setups.

In terms of tracking technologies, marker-based tracking is the most widely used method because of its reliability and ease of implementation in structured environments. Feature-based tracking is also prevalent, which offers flexibility in markerless tracking. A few studies employ alternative tracking methods such as head–gaze tracking or location-based tracking for specific use cases.

3.3. RQ3: What Are the Emerging Technologies in the Field of AR and VR Applications in Manufacturing?

This research question aims to identify and explore the emerging technologies that are used in AR and VR applications for manufacturing. By investigating these advancements, the objective is to identify the technological trends of current manufacturing practices through the integration of AR and VR technologies.

During the initial phase of AR and VR technology development, manufacturing applications were primarily image- or video-based, lacking interactive capabilities and closed-loop mechanisms, thus providing limited feedback from user interactions. However, with substantial research in the AR and VR domain, advancements in key technologies have propelled these applications towards increased interactivity and closed-loop features, significantly enhancing the overall user experience and system responsiveness.

After a careful review of the articles, emerging technologies that are driving AR/VR technology towards technological maturity for implementing closed-loop manufacturing applications were identified. Artificial Intelligence (AI), digital twin, teleportation, and edge technologies are among those emerging technologies, as listed in Figure 11. Human interaction is a vital part of any closed-loop manufacturing system. Due to the advancement of industrial robotics, interaction with the system is becoming more and more challenging. For this reason, human–robot collaboration is becoming one of the emerging fields of research, and AR and VR are becoming integral parts of applications involving human–robot collaboration. In the subsequent sections, the key findings of these technological trends and their applications in the AR and VR domain are discussed in detail while proposing generalized architectures for the implementation of these technologies in the aspect of manufacturing industries.

3.3.1. Edge Applications

AR and VR technologies play a vital role in manufacturing process monitoring, providing real-time and interactive process information, quality control, maintenance, and training applications. One of the key technologies that are making this possible is the IoT and in a broad sense, edge devices. When using AR and VR in conjunction with the IoT, machine diagnostics are being overlayed onto the machine and production environment, making manufacturing operations and troubleshooting more interactive and efficient. By leveraging edge devices, AR and VR have significantly enhanced the manufacturing processes by obtaining the data from the physical systems in real-time and creating a deeper and more immersive experience. Obtaining the physical systems’ data such as manufacturing data from PLC, the robot controller’s data, wireless sensor data, and wired sensor data, which are local as well as distributed, needs to be transferred from the equipment layer in real-time to the edge layer. The edge layer processes the data using edge devices and central processing servers to transfer it to the immersive visualization layer for AR- and VR-based visualization and interaction. In one study, the authors demonstrated an AR-based alert and early warning system for real-time machine health monitoring, which was deployed on Android devices and Microsoft HoloLens; data was collected from the physical system using TCP/IP protocol [83]. The solution intends to notify the operators via notifications whenever an alarm arises on a machine and provide alarm information.

AR and VR with edge intelligence can also be used for monitoring the machining operation. The authors of [149] proposed a VR system with a robot, replacing the traditional human–machine interface (HMI) console to visualize real-time robot trajectories for predicting errors and accident prevention and replaying previously executed trajectories from a database for post-operation error analysis. Here the data has been transferred using the Modbus TCP protocol. Communication protocols are a vital part of implementing AR- and VR-based manufacturing applications that use edge devices. Communication protocols vary based on the use cases and the hardware used for the applications lists some communication technologies used in AR- and VR-based applications using edge devices in manufacturing. The proposed generalized architecture for implementing AR and VR applications using edge devices is shown in Figure 12. The architecture can be more complex based on the use case.

Table 6. Communication technologies used in AR- and VR-based manufacturing using edge devices.

Research Group	Application	Communication Technology
[23]	Quality assessment operation	Wireless TCP/IP
[27]	Maintenance application	Ultra-Wide Band (UWB)
[38]	Human–robot collaboration	ROSbridge (websocket)
[46]	Maintenance application	WebRTC protocol
[83]	Machine operation	TCP/IP Protocol
[85]	Digital twin	Wireless TCP/IP
[149]	Human–robot collaboration	Modbus TCP
[150]	Machining operation	Wireless TCP/IP
[151]	Machine operation	Websocket
[152]	SCADA system	Wireless TCP/IP
[153]	Human–robot collaboration	ROSbridge (wifi)

lists some communication technologies used in AR and VR-based manufacturing using edge devices.

3.3.2. AI-Based Applications

AR/manufacturers are increasingly utilizing Artificial Intelligence (AI) and machine learning (ML) techniques to enhance their processes with the goal of improving overall operational efficiency. One study conducted comprehensive research regarding the applications of classical machine learning and deep learning methods and the authors identified that there is a growing trend of implementing machine learning and AI algorithms in manufacturing operations [111]. AI joined with AR and VR in manufacturing can unlock advanced capabilities and future innovations. AI-based applications are one of the most promising fields of future research in the AR and VR domain. AI is predominantly used in image processing and computer vision, making it a potential tool for AR and VR systems as image recognition, object detection, and pose estimation are crucial parts of AR/VR systems. An immersive system with AI features involves several key steps to integrate real and virtual environments seamlessly. These steps start with calibrating the virtual camera with the real camera to ensure an accurate depiction of the real world. Then, dynamic objects are detected and tracked in real-time, and the real camera’s pose estimation is carried out. This information is used to create and render virtual objects that are aligned with the real environment, enhancing the user’s AR experience [154]. Several research works have been performed to implement and improve AI-based AR and VR applications. The deep learning-based Convolutional Neural Network (CNN) is widely used for identifying objects in images. One of the prominent methods in this area is the Region-based CNN (R-CNN) [111,155,156]. The authors of [127] used multimodal AR instructions in conjunction with a tool detector built with a Faster R-CNN model trained on a synthetic tool dataset to increase worker performance during spindle motor assembly testing of a CNC carving machine. The system reduces assembly completion time and mistakes by 33.2% and 32.4%, respectively, compared to the manual assembly process. Many researchers also used YOLOv5, a state-of-the-art object detection model, for real-time and accurate object detection in AR and VR applications [157]. Pose estimation of industrial robots is vital in AR and VR manufacturing applications as it enables the precise alignment of virtual objects with real-world objects or locations and AI can play a vital role in implementing almost real-time post-estimation. In one study, the authors proposed a markerless pose estimation method using CNN to reference AR devices and robot coordinate systems [158]. The architecture of the proposed system used both RGB-based and depth-image-based approaches, which offer automatic spatial calibration.

Generative AI, one of the newest fields in AI, has transformative potential in AR and VR applications in manufacturing by enhancing situational awareness and enabling more efficient maintenance and design processes [159,160]. By integrating large language models with augmented and virtual reality, manufacturers can provide real-time, context-aware assistance to workers, enabling hands-free operation and in situ knowledge access [48]. These technologies can support tele-maintenance and consultation, allowing experts to guide on-site technicians remotely with immersive, interactive experiences [161]. Furthermore, VR-enabled chatbots facilitate the customization of complex products by offering intelligent, natural language interactions that streamline the design and manufacturing processes [162].

Table 7. AR- and VR-assisted AI-based manufacturing applications.

Research Group	Application	AI Method	AI Results
[28]	Predictive maintenance	CNN-LSTM	Mean Absolute Error (MAE) of 0.4257% and Root Mean Square Error (RMSE) of 0.4505% and the running time of 293.3658 s
[46]	Maintenance application	PSO-CNN	Validation accuracy reaches 97.63%
[55]	Task assistance	YOLOv5	High precision and real-time performance achieving prediction times of approximately 0.007 s
[82]	Machine inspection	Mask R-CNN	Mask R-CNN achieved 70% marker detection accuracy, with 100% accuracy for untrained machines
[84]	Pose estimation	MobileNetv2	Not mentioned
[85]	Process monitoring	One-Class SVM	Confidence score between 0 and 1 for each data point
[106]	Human–robot collaboration	Mask R-CNN	Not mentioned
[111]	Task assistance	Mask R-CNN	Significantly outperforms the AR marker-based approach
[127]	Work instruction	Tuned R-CNN	Achieved a mean Average Precision of 84.7%
[151]	Quick Development Toolkit for AR	YOLOv5	Only retains the objects with more than 70% detection confidence
[158]	Markerless pose estimation	CNN (VoteNet)	Not mentioned (accuracy depending on sensor systems and data quality)
[163]	Human–robot collaboration	RL	Success rate of reaching tasks is approximately 98.7%

lists some AR- and VR-assisted AI-based manufacturing applications. The selection of the algorithm, the quality of the data, computational speed, and data transfer speed are the key aspects of AI-based manufacturing applications using AR and VR. The proposed generalized system architecture for implementing AI-based AR- and VR-assisted manufacturing applications is shown in Figure 13.

3.3.3. Digital Twin Applications

Digital twin applications using AR and VR are one of the transformative tools in manufacturing for digital transformation providing smart decision support [127]. Different authors represented the definition of the term “digital twin” in different ways. The authors of [164] provided a generalized and consolidated definition of “digital twin”, which is the most suitable definition for this literature review: “digital twin is a virtual representation of a physical system (and its associated environment and processes) that is updated through the exchange of information between the physical and virtual systems.” The main features of a digital twin application are illustrated in Figure 14 [164]. AR and VR are the key technologies to implement digital twin-based applications in manufacturing such as product design and development, simulation, training, machine and robot calibration, and fault diagnosis. Research has been carried out to implement digital twin-based AR- and VR-based applications in manufacturing. The authors of [165] proposed an AR-assisted robotic welding path programming system. The authors overlaid the virtual robot on the actual robot, and a handheld pointer was tracked using an Optitrack Flex 3 camera motion capture system to directly interact with the workspace and to guide the virtual robot’s movements and path definitions.

Traditional HMIs sometimes have limited capabilities in interacting with the physical world. By using digital twin features, AR and VR systems are becoming a preferred choice for replacing traditional HMIs. Consideration needs to be made while choosing the level of virtualization for digital twin applications. Digital twin-based AR and VR systems can be immersive and non-immersive. Conventional non-immersive virtual systems engage sight and hearing through visualization of the computer screen while keeping users connected to the physical world. On the other side, immersive virtual systems isolate users from reality and facilitate interaction within the virtual environment using specialized haptic devices. There are different levels of virtualization in virtual systems, as shown in Figure 15 [166]. In one study, the authors developed a VR system with a robot to replace the traditional HMI console to visualize real-time robot trajectories for predicting errors and preventing accidents. The system can replay previously executed trajectories from a database for post-operation error analysis [149].

Table 8. Digital twin application in manufacturing using AR and VR.

Research Group	Application	Type of Visualization
[69]	Design review application	Immersive
[79]	CNC milling machine operation	Immersive
[80]	Remote human–robot collaboration	Immersive
[85]	Process monitoring	Non-immersive
[104]	Assembly operation	Immersive
[106]	Human–robot collaboration	Immersive
[108]	human–robot collaboration	Immersive
[149]	Simulation and training	Immersive
[153]	Robot programming	Immersive
[163]	Remote human–robot collaboration	Immersive
[165]	Robot welding programming	Immersive
[167]	Human–robot collaboration	Immersive
[168]	Robot programming	Immersive
[169]	Robot programming	Immersive

shows DT applications and their corresponding types of visualization, stating whether they immersive or non-immersive. Immersive visualization seemed to be the more popular style.

VR systems were seen to be used more often in the sense of digital twin applications. They were mostly used off-site where AR would be less useful. When being used for AR, they were used as a way of creating content for the application, which is less immersive. Digital twins were often used for training scenarios in a more immersive format. This gave trainees the option to simulate being on a shop floor without the dangers of being inexperienced with equipment.

In another study, a virtual robot model was overlaid on a physical robot system, and users could interact with the system using Microsoft HoloLens. The virtual layer converts the user-defined trajectories into robot commands, and the Robot Operating System (ROS) layer processes these commands to communicate with the industrial robot controller [166]. Digital twin features can be used for remote operations also. The authors used HTC Vive headsets and the Unity plugin PUN2 to synchronize users and handle the network for multiuser communication, identifies some general manufacturing applications of AR and VR using digital twin technology. The proposed generalized system architecture for implementing digital twin-based AR- and VR-assisted manufacturing applications is shown in Figure 16.

3.3.4. Teleportation and Remote Collaboration Applications

Remote collaboration enables the experts to share their knowledge and guide engineers, operators, and technicians with less domain knowledge to troubleshoot manufacturing operations. In addition, remote collaboration enables designers and product developers to collaborate from different locations, which can significantly reduce the product or process development time. Traditionally remote collaboration is performed using video- and audio-based communication [170]. However, it has limited interaction capabilities. With the recent advancement of immersive technologies (AR and VR), remote collaboration and teleportation applications are becoming more interactive and realistic by providing haptic feedback and sensing capabilities. A growing amount of research has been carried out in the field of AR- and VR-based teleportation and remote collaboration. The authors of [171] demonstrated the process of improving the effectiveness of teleoperated robotic arms for manipulation and grasping activities using Augmented Visual Cues (AVCs). A case study with 36 participants was conducted to validate the system’s usefulness, and the authors also employed NASA Raw-TLX (RTLX) to quantify task performance [172]. In another study, the authors proposed a situation-dependent remote AR collaboration system that offers two functionalities: image-based AR collaboration for limited network or device capabilities and live video-based AR collaboration with a synchronized VR system. For image-based collaboration, annotations are added directly onto 2D images or a 3D perspective map is generated. For the synchronized VR mode, annotations are attached to a virtual object to eliminate inconsistencies while changing the viewpoints during collaboration [173]. In remote collaboration, the remote users need to have dimensional knowledge about the location they are interacting with. Often the virtual environment of the physical system is developed to obtain knowledge about the physical system remotely. The development of the virtual system is challenging, and serval research studies have been conducted on developing virtual systems for teleportation and remote collaboration. The authors of [174] introduced a methodology for creating a point cloud-based virtual environment incorporating regional haptic constraints and dynamic guidance constraints for teleportation operation. The system integrates a virtual robot model with real-time point cloud data and haptic feedback, allowing operators to control the virtual robot using a haptic device. The authors of [163] proposed an AR-based multi-robot communication system using digital twins and a reinforcement learning (RL) algorithm for teleportation operations. The digital twin of the industrial robots is mapped onto the physical robots and visualized using AR glasses. The RL algorithm is used for robot motion planning, which replaces the traditional kinematics-based movement with target positions.

Table 9. AR- and VR-assisted teleportation and remote collaboration for manufacturing applications.

Research Group	Application	Research Focus
[45]	Maintenance application	Asynchronous communication for maintenance method development and documentation creation
[69]	Product design	Feasibility study of developing VR system to reduce environmental impact
[70]	Human–robot collaboration	Virtual commissioning of production line
[80]	Quality control	Remote metrological inspection using haptic feedback of manufacturing process
[153]	Human–robot collaboration	Development of remote collaboration platform
[163]	Human–robot collaboration	Collaborative manufacturing using digital twin, RL algorithm, and AR
[171]	Human–robot collaboration	Improvement of the performance in manipulation and grasping tasks
[173]	Work instruction	Visual annotation for image- and video-based remote collaboration
[174]	Human–robot collaboration	Haptic constraints in a point cloud augmented virtual reality environment

identifies some general manufacturing applications of AR and VR for teleportation and remote collaboration. The proposed generalized system architecture for implementing AR- and VR-based teleportation applications in manufacturing is shown in Figure 17.

Teleportation applications were also more often found to be used in VR applications over AR due to the format being more immersive, allowing the user to simulate being near equipment. This however does not mean that AR cannot be used for teleportation applications.

3.3.5. Human–Robot Collaboration Applications

In the development of any manufacturing application, human interaction is an integral component. While robotics and automation have significantly advanced manufacturing processes, human interaction remains an essential part of the successful operation of manufacturing operations. For this reason, human–robot collaboration has become a prominent field of research. AR and VR are among the leading technologies that have been used to make human–robot collaboration more interactive. In one study, the authors proposed a multi-robot collaborative manufacturing system with AR assistance where humans use AR glasses to determine the position of the physical robot’s end-effector, and an RL-based motion planning algorithm calculates the joint value solution that is viable [163]. In another study, the authors proposed an HRC system using VR for human–robot simulation. The authors used Tecnomatix Process Simulate (TPS), Universal Robots, and an HTC Vive VR headset to perform an event-driven human–robot collaboration simulation [166].

The authors of [167] proposed an AR-based human–robot collaboration system with a closed compensation mechanism. The proposed system used four layers: the AR layer, the virtual environment, the Robot Operating System (ROS), and the KUKA Robot Controller layer. Microsoft HoloLens was used to interact with users, allowing them to control a virtual robot model overlaid on an actual robot using gestures, and user-defined trajectories were converted into robot commands.

The authors of [76] leveraged an MR system, deep learning, and digital twin technology to develop a human–robot collaboration (HRC) system by calculating real-time minimum safety distances. The system also provides task assistance to users using Microsoft HoloLens 2.

Table 10. AR- and VR-assisted human–robot collaboration for manufacturing applications.

Research Group	Technology	Research Focus
[80]	AR/VR	Teleportation application using AR, VR, and haptic feedback
[104]	AR	Robot programming, safety, and task assistance
[106]	AR	Deep learning- and digital twin-based safety-aware system for human–robot collaboration
[111]	AR	Deep learning-based task assistance
[158]	AR	Distributed automation architecture and markerless pose estimation for robot
[163]	AR	RL- and digital twin-based collaborative manufacturing application
[166]	VR	Human–robot simulation with virtual chatbot
[167]	AR	Closed-loop HRC with a compensation mechanism
[175]	VR	Multisensory constrained and contactless coordinated motion task

identifies some general manufacturing applications of AR and VR for human–robot collaboration. The proposed generalized system architecture for implementing human–robot collaboration using AR and VR is shown in Figure 18.

The statistical analysis of research papers focusing on applications and emerging technologies is illustrated in Figure 19. The red scale within the figure represents applications, while the gray scale denotes the different technologies utilized. The findings reveal a notable surge in interest towards AR and VR applications for manufacturing across various domains in recent years.

Figure 20 presents detailed statistics on the utilization of AR and VR devices within the reviewed papers. It was observed that Head-Mounted Displays (HMDs) emerge as the preferred choice for immersive visualization among these devices. This preference may be attributed to the development of new Software Development Kits (SDKs), resulting in more accessible and stable application development compared to previous years. Additionally, the rise in emerging hardware has facilitated the creation of more affordable and user-friendly AR and VR applications.

Furthermore, Figure 21 illustrates the statistics on tracking technologies employed in AR and VR applications. Mark-based tracking technologies remain popular due to their ease of implementation. However, there has been a growing trend towards feature-based tracking over the years, driven by its ability to track immersive content without the need for physical markers. However, the 2023 to 2024 papers show little to no use of model-based tracking. This could be due to the newer ease of creating feature-based tracking models. More and more introductory technologies come with features of making these models without the need of external sources. Many smart phones come with lidar sensors built in. Most of the papers that use HHDs as their viewing device often use marker tracking methods. These are often preliminary applications and prevent the need for investing into expensive hardware.

Figure 19, Figure 20 and Figure 21 collectively show a noticeable surge in research activity and technology adoption around the year 2021. This peak is primarily attributed to the release of more affordable HMDs, such as the Oculus Quest 2 in late 2020, which significantly lowered the barrier to entry for researchers; the introduction of more stable and robust SDKs, enabling easier and more reliable application development; and the broader availability of AR and VR hardware, which made experimentation and implementation more accessible during this period. After 2021, the surge continues, likely due to consolidation around a few dominant technologies and a shift toward integrating AR and VR into specific industrial use cases.

3.4. RQ4: What Are the Challenges for the Adaptation of AR and VR Applications in Manufacturing?

There are many challenges for applying AR and VR applications in manufacturing environments. These challenges are categorized as technological challenges, organizational challenges, and environmental challenges.

3.4.1. Technological Challenges

In this section, different technological challenges of AR and VR technologies in manufacturing applications are discussed in detail.

• Tracking and registration

Tracking and registration are the most vital of any AR and VR application. Real-time tracking is preferred for AR- and VR-based manufacturing applications. Different tracking technologies are discussed in Section 3.2.2. Among those tracking technologies, marker-based tracking is the most widely used and easier choice for AR- and VR-based manufacturing applications and for this reason, many researchers used marker-based tracking for manufacturing applications [26,48,58,76,84]. Marker-based tracking systems face challenges from maker occlusion, making a part or entire marker not viewable using the tracking system [176]. As the manufacturing system is complex in nature, marker occlusion can easily happen to any movable parts which need to be tracked. The alternate approach to solve this problem is the markerless tracking system. Simultaneous localization and mapping (SLAM) is commonly used in markerless tracking [35]. In one study, the authors developed a quick development toolkit using SLAM and QR code for augmented reality visualization of a factory [151]. AI-based deep learning techniques can also be used for markerless tracking. However, there are many challenges in applying these technologies with immersive technologies for markerless tracking in the manufacturing environment. In the manufacturing environment, there are products with limited variations such as nuts and bolts of different sizes but with the same shapes. These products are harder to differentiate, making it challenging for object detection algorithms to detect the desired objects. Again, training these algorithms is sometimes resource-intensive and time-consuming, which makes it harder to implement it in AR and VR systems [35]. Additionally, collecting data from the manufacturing process is challenging and there are also data privacy issues in manufacturing industries. Another challenging part of implementing these algorithms in AR and VR systems is real-time data transmission. Modular AR and VR devices are resource-constraint devices and have less computation power and in most cases cannot run AI algorithms. In this scenario, object detection algorithms need to run on the remote server and decisions have to be transferred to the AR and VR systems using network devices in real-time, which is very difficult to implement [154]. The challenges of implementing markerless tracking with AR and VR systems in manufacturing are listed in Figure 22. Registration of the virtual component in the immersive environment depends on the successful tracking of the object [176]. The accuracy of the tracking and the correct alignment of virtual components in the real environment are challenging to implement in complex manufacturing settings.

• Evaluation of AR and VR devices

There are many AR and VR devices available for implementing AR and VR applications but selecting the proper AR and VR devices for a specific manufacturing application can be challenging. The selection of the AR and VR devices depends on various types of factors such as the price, battery life, desired field of view, optics technology, camera type, open API support, audio technology, sensor technologies used, haptic controls, processor, memory and storage, data connectivity options, operating system, ingress protection (IP rating), appearance, among other factors. There is no standardized benchmarking tool for the evaluation of AR and VR devices for manufacturing applications [177]. In one study, the authors developed a VR benchmark for assembly assessment but generalizing the benchmark for any manufacturing environment is complex [178].

• Development Challenges

The advancement of AR and VR in the industrial sector faces a multitude of challenges that hinder its progression. Primarily, the absence of universally applicable methods for designing applications is a notable impediment. AR development typically operates on a bespoke, case-by-case basis, tailored to the unique requirements of individual systems. This approach can prove to be cost-prohibitive, particularly when considering the need to design and implement distinct applications for various sectors within a manufacturing plant [10]. The absence of a “one-size-fits-all” solution also extends to the compatibility of AR and VR software with different hardware platforms. When developing an application, the intricacies of setup procedures are intricately tied to the specific software and hardware combination for which the application is intended. For instance, an application designed for the HoloLens platform cannot be seamlessly transferred to another Head-Mounted Display (HMD) device [2]. These developmental challenges are further exacerbated by the demand for job-specific solutions. Diverse job roles necessitate distinct hardware and software requirements, influenced by factors such as environmental conditions, machinery configurations, and the availability of specific models. Achieving a harmonious approach to application development across different areas within a singular environment remains a formidable task. Currently, the industry lacks convenient “drag-and-drop” solutions, which could potentially reduce employment costs and standardize application development practices.

• Cybersecurity threat

The cybersecurity threat is among the top concerns for implementing AR- and VR-based manufacturing applications. Confidentiality, integrity and availability (CIA), data theft, observation attacks for graphical PIN in 2D, security, privacy and safety (SPS), granular authentication and authorization, and latency problems are the most frequent cyber security concerns in AR and VR systems [179]. Therefore, careful development needs to be conducted for AR- and VR-based manufacturing. If the system gets exposed, attackers can easily obtain secret information that might jeopardize any manufacturing company with substantial financial loss.

3.4.2. Organizational Challenges

• User acceptance

In the field of manufacturing, AR and VR are very new technologies that have huge transformative potential. Given the novelty of the technologies, acceptance and adoption by end-users are imperative for the successful integration of these technologies. Consequently, many scholarly research articles focused on understanding the user acceptance and the usability of AR and VR applications within the context of manufacturing contexts. Various researchers have adopted different systematic approaches to adopt AR and VR systems tailored to manufacturing environments and emphasized the critical need for professional and strategic implementation of these technologies. The authors of [21,180] identified an architecture framework for the systematic development of AR applications. This framework can also be adopted for developing VR applications. The framework consists of five stages: the concept and theory stage, implementation stage, evaluation stage, industry adaptation stage, and intelligent application solution stage [21,180]. In each phase of the architecture framework, careful considerations need to be taken and user evaluation needs to be conducted so that the applications can be well accepted in the manufacturing setting, which are very challenging tasks. For this reason, AR and VR technologies are facing challenges to being adopted in the manufacturing industries.

Many of the papers found conducted field studies to examine how their application improved different metrics and how the users reacted to use. These studies ranged from 5 to 50 users at a time. While some of these studies were conducted in large groups, they did not last for an extent of time that could give significant findings. Large-scale field research with long usage periods must be studied to find out how they would fair in the manufacturing industry. This could improve how they are adopted not only by users but by companies as well.

• Return on investment

AR and VR can prove to bring large returns on investment, especially with large overturn rates involved. However, this is dependent on accuracy, ease of use, and speed at which the application can be developed. The accuracy and ease of use of the applications can have an overall effect on the knowledge retention of the user. This cannot be evaluated at a higher level as there have not been studies with large enough field groups to determine if the applications are effective. Currently, most studies are performed based on the time to conduct tasks. While this is a good metric, there is more that can be done when determining the effectiveness of AR and VR. At the moment, there is also a lack of common knowledge on the use of AR and VR as it is an emerging technology in the public. This does not mean that people do not know what it is, but there is a lack of knowledge of common practices or maneuvers when using applications, which causes a learning curve when implementing one of these systems.

3.4.3. Environmental Challenges

The environment has many different factors that can prove to be difficult for AR and VR systems to handle. This can include factors such as lighting, atmosphere, moving bodies, etc. Of these, occlusion seems to be one that receives a lot of attention. Occlusion can involve having to exclude objects in the field of view of the user like other people, the users’ hands, and other structures in the environment. As the AR system is the last layer of vision for the user, without occlusion handling, the virtual aspects can be displayed on top of objects that it should not. This can disrupt the flow of the application and the user experience [88]. Occlusion systems also cause larger processing strains on the system, which reduces the realism of the simulation and in some cases causes discomfort to the user. There have been many proposed solutions that involve the occlusion of specific types of objects in the field of view but an all-inclusive solution remains to be determined [35]. The use of CAD-based tracking has also been seen to improve partial occlusions and rapid motion, but this relies on the availability of CAD models [10].

The identified emerging technologies and problems with AR and VR can be seen to correlate directly with each other. While many of the technologies are being researched individually, they should be looked at alongside each other. This would allow for different scenarios to be looked at with multiple problems being faced. For example, many of the papers chose a single tracking type when developing the application but this might not be the most optimal solution. Evaluating different challenges while the technologies are being developed and tested during reporting would allow for newer studies to supersede these problems. The challenges being faced by AR and VR are often discussed apart from each other. This paper proves that they often face the same problems. When conducting literature reviews in the future, researchers can look into both realms to find challenges and solutions.

4. Conclusions

In this paper, a detailed examination of the applications of AR and VR within the context of the manufacturing industry was conducted. Both software and hardware components utilized in these applications were meticulously categorized and identified.

Research Question 1 (RQ1) systemically identified the current key applications of AR and VR in manufacturing by reviewing and analyzing various studies and examples. AR and VR applications enhance manufacturing processes, including maintenance, assembly, operations, training, product design, and quality control. The potential of AR and VR technologies extends beyond the identified core areas, which include inventory management, supply chain optimization, and customer engagement. These extended applications are not extensively covered in the reviewed literature, which will be explored in future research.

Research Question 2 (RQ2) identifies the state-of-the-art technologies in AR and VR applications for manufacturing, offering a comprehensive overview of key components, hardware, and software utilized in this domain. Cutting-edge technologies were examined, which include tracking systems, display devices, game engines, AR libraries, and VR development kits.

Research Question 3 (RQ3) highlights emerging technologies in AR and VR applications for manufacturing, providing a detailed exploration of cutting-edge advancements such as Edge applications, AI-based applications, digital twin applications, teleportation, remote collaboration applications, and human–robot collaboration applications. By synthesizing insights from various studies, the review identifies key technological trends of AR and VR technologies in modern manufacturing practices. However, challenges exist in implementing these technologies. There is a lack of standardized architectures for their implementation, and challenges related to interoperability and scalability persist. More industry-specific case studies and applications are needed for the large-scale adoption of these technologies.

Research Question 4 (RQ4) identifies challenges across technological, organizational, and environmental domains for the implementation of AR- and VR-based applications in manufacturing. The section highlights the key technological hurdles like tracking and registration complexities, device selection and evaluation, development challenges, and cybersecurity threats that impact AR and VR implementation in manufacturing. Additionally, it highlights organizational obstacles such as user acceptance and return on investment, underscoring the importance of strategic frameworks for technology adoption. Research gaps remain in standardizing architectures and benchmarking tools for AR and VR devices in manufacturing, as well as in developing industry-specific case studies to facilitate broader technology adoption. Further research is needed to address environmental challenges like occlusion handling and other factors influencing AR and VR usability in manufacturing settings.

In conclusion, this paper provides a comprehensive overview of the applications, technologies, and challenges associated with implementing AR and VR in the manufacturing sector. Key findings highlight the diverse range of applications within manufacturing, from enhancing operations to exploring emerging technologies. Despite technological advancements, challenges such as tracking complexities and user acceptance barriers underscore the need for further research and standardized frameworks to maximize the potential of AR and VR in manufacturing.