Augmented Reality, Virtual Reality and Artiﬁcial Intelligence in Orthopedic Surgery: A Systematic Review

: Background: The application of virtual and augmented reality technologies to orthopaedic surgery training and practice aims to increase the safety and accuracy of procedures and reducing complications and costs. The purpose of this systematic review is to summarise the present literature on this topic while providing a detailed analysis of current ﬂaws and beneﬁts. Methods: A comprehensive search on the PubMed, Cochrane, CINAHL, and Embase database was conducted from inception to February 2021. The Preferred Reporting Items for Systematic reviews and Meta-Analyses (PRISMA) guidelines were used to improve the reporting of the review. The Cochrane Risk of Bias Tool and the Methodological Index for Non-Randomized Studies (MINORS) was used to assess the quality and potential bias of the included randomized and non-randomized control trials, respectively. Results: Virtual reality has been proven revolutionary for both resident training and preoperative planning. Thanks to augmented reality, orthopaedic surgeons could carry out procedures faster and more accurately, improving overall safety. Artiﬁcial intelligence (AI) is a promising technology with limitless potential, but, nowadays, its use in orthopaedic surgery is limited to preoperative diagnosis. Conclusions: Extended reality technologies have the potential to reform orthopaedic training and practice, providing an opportunity for unidirectional growth towards a patient-centred approach.


Introduction
Nowadays, Virtual Reality (VR), Augmented Reality (AR), and Artificial Intelligence (AI) are commonly adopted as operative and training tools for surgery. However, the real advantages and fields of application of these technologies are still new and in constant evolution. VR and AR, also defined as "extended reality technologies" [1], are similar, but specific differences allow surgeons to use them in various contexts. With VR, the surgeon can interact with a virtual environment that is completely generated by a computer. In this setting, the surgeon can prepare preoperative planning or perform a surgical simulation [2][3][4]. However, the entire simulation is virtual and does not allow the surgeon to interact with the patient. Conversely, with AR, it is possible to visualize the real world [1]. A superimposed computer-generated image is directly projected on the surgeon's field of view, overcoming VR limitations. In fact, the user cannot interact with the external environment, including the patient, the operating room and surgical tools, and cannot see the digital content and reality at the same time. Both VR and AR represent a step forward in surgical training since they simulate multiple surgical cases [5][6][7].
Orthopaedic surgery significantly benefits from these technologies, as they have been adopted for preoperative planning, intraoperative navigation and postoperative rehabilitation [8,9]. Extended reality technologies have already been used to improve the performance of young surgeons and residents in arthroscopic knee surgery [10] and hip arthroplasty [11]. Furthermore, AR is commonly used to assist surgeons during specific procedures, such as minimally invasive surgery, spine surgery, and robot-assisted surgery.
In some articles, authors also describe mixed reality (MR). MR may be defined as a hybrid between VR and AR. This technology is able to generate a virtual environment while also giving the user the possibility to interact with their surroundings. However, the functional and technical similarities in MR and AR applications make these two almost interchangeable [12].
AI is adopted in orthopaedic surgery as well, with particular applications in imaging analysis and surgical training. The mechanism behind AI employs algorithms that recognize patterns in complex medical data received by the computer [13][14][15]. This technology could also aid surgeons by improving diagnostic accuracy and preventing human error [16].
In conclusion, the following systematic review aims to assess the role of VR, AR and AI in orthopaedic surgery and their possible future value.

Materials and Methods
The present paper focused on studies concerning the use of VR, AR and AI in orthopedic surgery. The Preferred Reporting Items for Systematic reviews and Meta-Analyses (PRISMA) guidelines were used to improve the reporting of the review (Figure 1).

Eligibility Criteria and Search Strategy
The research question was formulated using a PICOS approach-Patient (P); Intervention (I); Comparison (C); Outcome (O), and Study design (S). This study selects those articles that described orthopaedic surgical procedures (P) assisted by VR, AR and AI (I). The aim was to assess the role of these technologies in orthopedic surgery and their potential future applications (O). For this purpose, randomized studies (RCTs) and nonrandomized controlled studies (NRCTs), as prospective, retrospective, cross-sectional, observational studies, case-series, pilot and case-control studies, were included (S).
A comprehensive search on the databases PubMed, Medline, Cochrane, CINAHL and Embase databases was conducted from inception to February 2021. The following keywords were used isolated and combined-Virtual Reality; Augmented Reality; Artificial Intelligence; Orthopedic Surgery; Reality; preoperative planning; training; intraoperative; and mixed reality. All the keywords were searched isolated and combined with their MeSH terms. More studies were searched among the reference lists of the selected papers. The exclusion criteria included reviews, books, case reports, technical notes, letters to editors, instructional courses, in vitro and cadaver studies.

Study Selection and Data Collection
This systematic review was carried out in February 2021. Only English and Italian publications were included. The initial search of the article was conducted by two authors (SF and LG) using the search protocol previously described. The following research order was adopted-titles were screened first, then abstracts and full papers. A paper was considered potentially relevant and its full text reviewed if, following a discussion between the two independent reviewers, it could not be excluded based on its title and abstract. The screening process was performed using CADIMA software [18]. The number of articles excluded or included was registered and reported in a PRISMA flowchart ( Figure 1). For designing the PRISMA, the rules by Liberati et al. were followed [17].

Quality Assessment
The Cochrane Risk of Bias Tool was used to assess the quality and potential bias of the included Randomized Control Trials [19]. The Cochrane Risk of Bias Tool classifies evidence using seven different domains-sequence generation, allocation concealment, blinding of participants and personnel, blinding of outcome assessment, incomplete outcome data, selective outcome reporting and other potential sources of bias. A quantitative score was assigned to each judgement as follows-Low risk of bias = 0; Uncertain risk of bias = 1; High risk of bias = 2. An overall quality score was calculated for each RCT study by summing the values of the different items using the following scale: Overall score ≤1 High Quality; Overall score ≤3 Moderate Quality; and Overall score >3 Low Quality.
The Methodological Index for Non-Randomized Studies (MINORS) was used for quality assessment of non-randomized studies [20]. This score consists of 12 items-clearly stated aim; inclusion of consecutive patients; prospective data collection; endpoints appropriate to study aim; unbiased assessment of study endpoint; follow-up period appropriate to study aim; <5% lost to follow-up; prospective calculation of study size; adequate control group; contemporary groups; and baseline equivalence of groups and adequate statistical analyses. The reviewers individually evaluated all these items. The MINORS items were scored 0 if not reported, 1 when reported but inadequate, and 2 when reported and adequate. The ideal global score was 20 for NRCTs. The simplicity of MINORS comprising only 12 items makes this item readily usable by both readers and researchers. The reliability of this score has already been demonstrated [20].
Two reviewers independently evaluated (SF/LG) the potential risk of bias of the studies. The consensus was reached by the two reviewers (MGDM/VD) when there was a difference in opinion on an item. If no consensus was reached, the independent opinion of a third reviewer was decisive (SDS).

Data Synthesis and Analysis
Data were extracted and synthesized through Microsoft Excel. General study characteristics extracted were-author and year, type of study, levels of evidence, sample size, country, purpose (preoperative planning, intraoperative use and surgical training), hardware, surgical procedure and conclusions. Due to the heterogeneity of the study, only qualitative characteristics were described. Considering the heterogeneity of the included studies, it was not possible to perform a meta-analysis.

Results
According to the PRISMA protocol, a flow-chart diagram showing the selection process of the studies was reported (Figure 1). A total of 1452 studies were found (no additional studies were found in the grey literature, and no unpublished studies were retrieved). A total of 1211 studies after duplicate removal were maintained. Of that, 974 were excluded from the study through title and abstract screening because they were not in line with our objective (n = 572), non-orthopaedic topics (n = 488) or were reviews (n = 56). Then, 95 full-text articles were screened. Of these studies, 73 were excluded (not inherent to the study = 65; no full-text available = 7). After this process, 21 articles were eligible for this study.

Study Selection and Patient Characteristics
All of the studies included, excluding four articles, reported the patient sample size.

Quality Assessment
The Cochrane Risk of Bias Tool was used to assess RCT quality. Concerning the RCTs, two studies with an overall "moderate" quality [5,21] and one with an overall "high" quality [22] were found. The MINORS tool was used to assess the risk of bias in NRCTs. Among these studies, fifteen studies (83.3%) [11,24,25,27,[29][30][31][32][33][34][35][36][37][38][39] had a low risk of bias, and three studies (16.7%) [23,26,28] had a high risk of bias. The Cochrane Risk of Bias Tool and MINORS were reported in Tables 1 and 2.   Three studies were included (one prospective comparative, one prospective casecontrol, one retrospective cohort) [27,32,33]. The authors of these studies reported that VR and AR were both excellent tools for preoperative planning. These two technologies allowed the surgeon to act in a fast, precise manner, providing an additional degree of safety when approaching the surgical phase [2,3]. Specifically, AR was useful in the visualization of preoperative planning images directly overlaying them on the patient. The most frequently utilized hardware were head-mounted displays (HMDs), including Magic Leap One, backed by Google and Microsoft HoloLens.
Two articles were found regarding AI use in preoperative planning. The former depicted its use in lumbar spinal stenosis detection [33]. The purpose of this study was to classify central lumbar spinal stenosis in four grades-none, mild, moderate and severe. This classification was compared with the independent readings of a spinal surgeon. The system can learn how to grade the severity of lumbar stenosis with outstanding results, providing many possible uses in epidemiological studies. The second article described the accuracy of AI in predicting, diagnosing, and classifying patients affected by cervical spondylotic myelopathy (CSM) [27]. The purpose of this study was mainly diagnostic, as it aimed to understand how neural networks can be used to predict CSM severity. Two different neural network models were employed to complete these tasks. Both models showed high accuracy percentages. According to MINORS, the overall quality of evidence in these studies was assessed in the range between "low" and "high". All the results are reported in Table 3.

Outcome: Intraoperative Use
Fourteen studies were included (five pilot, one retrospective cohort, two retrospective comparative, two prospective comparative, one prospective case-control, two prospective cohort, one prospective observational) [11,[23][24][25][26][28][29][30][31]34,35,[37][38][39]. The authors of these studies reported that AR efficiently aided surgeons in performing surgical operations. Moreover, it supported the surgeon in a broad range of aspects, enabling them to understand the 3-D anatomy, providing extremely accurate guidance and reducing surgery time and radiation exposure at the same time. AR has been demonstrated to be successful in different surgical procedures, including K-wire placement, vertebroplasty and pedicle screw placement [2]. Only one study reported no differences in the results achieved by the AR group compared to the traditional group during total hip arthroplasty [34]. The hardware used in these procedures included head-mounted displays, such as HoloLens or Xvision (Augmedics). One of the articles mentioned MR [28] rather than AR. Different AR registration techniques were used in these studies. These include preoperative CT scans, also used for intraoperative capturing [3], or intraoperative computed tomography (iCT) imaging [25], which ensures high navigational accuracy. In other cases, an optically tracked reference plate was strapped to the patient's limb [28] or a tracked tool was used to perform registration of preoperative CT data directly on the anatomical landmarks of the patient [30]. Lastly, CT scan data were integrated with ultrasound imaging [23], and markerless AR was used to perform real-time registration and to update imaging data during surgery [9].
According to MINORS, the overall quality of evidence in these studies was assessed in the range between "low" and "high". All the results are reported in Table 3.

Outcome: Surgical Training
Four studies were included (three RCTs, one prospective observational) [5,21,22,36]. The authors of these studies reported that VR showed promising results, significantly improving trainee and resident skills. These studies showed that VR-trained residents performed surgery faster and with fewer errors compared to those who were traditionally trained. Moreover, VR-trained residents have proven more skillful in performing surgical tasks rather than other control groups, even with minimal guidance, resulting in overall better performance. All the mentioned studies showed high construct validity, as the levels of skill of trainees were successfully measured and improved. Moreover, one of the studies [5] showed evidence of criterion-based validity, because it compared a group of trainees using a simulator with one using conventional methods. The overall quality of evidence in the NRCT was assessed as "high", according to MINORS. According to the Cochrane Risk of Bias Tool, one RCT had an overall score of zero, indicating high quality [22] and the other two had an overall score of two, indicating moderate quality [5,21]. All the results are reported in Table 3.

Discussion
Extended reality technologies are beginning to assume a central role in health care, assisting and improving the practice of both trainees and experienced surgeons. Nonetheless, these advancements come with inherent limitations. These technologies are active participants in reducing the learning curve for trainees. However, better and more cost-efficient ways of utilizing them still have to be theorized and employed.
VR reproduces the patient's anatomy and surgical conditions in a virtual environment, in which the user is immersed through a wearable headset [42]. Furthermore, it can be enhanced by sounds and other inputs [1], providing a fully immersive and detailed experience for simulated surgical procedures. To reinforce its training capability, this technology also offers the possibility to implement features for recording and performance analysis. Simulators, devised to promote and upgrade learning prospects, are indeed being validated for both training and surgical practice. Contextually, it has been shown that VR-trained residents execute surgical tasks faster than those who were trained traditionally. Nonetheless, simulators are still not available in equal numbers in the different subfields of orthopaedics. As per Vaughan et al. [40], while numerous advanced simulators are available for being employed in hip trauma and drilling practices, other procedures seem neglected in comparison [40]. Observing how spinal and hip replacement surgery simulators are still at an embryological state, it is possible to assume that not enough research has been carried out regarding reality technology application in orthopaedic surgery during the last decades [40].
Nowadays, the possibility that orthopaedics will become one of the central focuses of reality technology-related research is feasible. This would, of course, be represented by a uniform development of VR, AR and AI in their applications to different subfields and procedures. Specifically, while VR is already widely accredited to create simulators for surgical training and preoperative planning, AR is more promising for intraoperative purposes [25]. This technology could aid the surgeon in understanding and visualizing the surgical site, including the topographical relationship between anatomical landmarks, orthopaedic implants and operating tools [43]. While offering a limited user-machine interaction potential, AR can create virtual images, project them onto real-life surroundings and superimpose them on data obtained by imaging, thereby providing a guided, minimally invasive approach [2][3][4]. However, in orthopaedic trauma surgery, AR use was not already validated [9]. As shown in numerous experimental studies [24,[44][45][46][47][48][49][50], such as the one conducted by Cho et al. [51], AR's potential practical applications appear countless. Some of these are still under development, while others have been proven successful in animals and still need to be redesigned for human application. Carl et al. [25] described several types of surgeries performed using AR. Indeed, it has been shown that combining iCT imaging with AR can greatly facilitate spine surgery by ensuring meticulous navigational accuracy and a deep understanding of the surrounding anatomy. In the study conducted by Ogawa et al. [34], forty-six patients underwent acetabular cup placement during total hip arthroplasty. In one group, AR navigation was used, while the other group received traditional surgery. No differences were found between the groups in the end, meaning that surgery was just as successful in the AR group as it was in the traditional one. Even though this result shows that AR does not yield benefits in all cases, it also shows that it does not have adverse effects.
Moreover, minimally invasive procedures represent an increasingly popular trend in the orthopedic field, delineating the value of augmented reality technologies even more clearly [52]. With its visualization potential, AR also has a lot to offer as a training tool, having been proven to reduce the learning curve for trainees.
Unanimity does not appear to have been reached regarding the drawbacks of these technologies due to most of them being improvable in the foreseeable future. Limitations of extended reality technologies include visual fatigue, inaccuracy of these devices, and high costs.
AI, a newly rising technology, achieved remarkable improvements in the medical field. Despite its almost flawless prediction accuracy in femur fracture classification cases [53] and general fracture detection [54], AI has been used the most for other purposes. Specifically, it has been used to grade the severity of lumbar spinal stenosis in epidemiological studies (31). These studies involved a considerable volume of highly consistent MRI spine data computed by the system thanks to its algorithm elaboration capabilities. Additionally, this technology has been used to predict the diagnosis and severity of cervical spondylotic myelopathy [27], further demonstrating its great accuracy. In the aforementioned studies, machine-learning, a subset of AI, was employed. This branch of computer science has the potential to revolutionize epidemiologic sciences [55]. The algorithms utilized by this technology can be used in the prediction, classification and clustering of several conditions. Hopkins et al. [27] utilized deep learning, which is defined as a subfield of machine-learning, which employs neural networks (algorithms inspired by the function and structure of the human brain). Neural networks have gained attention in the radiology field, as they can analyze various areas of the human body with great accuracy [56]. Even though AI has the incredible potential to revolutionize orthopaedics, not enough literature discusses its applications in orthopaedic surgery.
When applied to surgery, VR, AR and AI allow the surgeon to operate more consciously and safely. Physicians will work side-by-side with their machine counterparts and completely shift their focus towards more patient-centred health care, yielding unprecedented benefits. It appears clear that extended reality technologies are subject to continuous development, which will likely push them to become the object of a revolution in the medical field.
The limitations of this paper were the high heterogeneity between studies and the lack of data such as sample size in four studies or mean follow up. Moreover, due to the high heterogeneity of the data, it was impossible to perform a meta-analysis. Only English and Italian articles were included, constituting a limitation in the search string. Lastly, the quality of evidence of the studies included was low; therefore, it was impossible to obtain significant conclusions.

Conclusions
Extended reality technologies assist the surgeon in visualizing patient data and anatomical landmarks in a 3D setting. Moreover, they can significantly enhance preoperative planning and increase surgical accuracy. Additionally, these technologies allow trainees to experience surgical practice in a cost-and time-efficient manner. The high costs and availability issues, combined with technical limitations of these devices, are bound to be overcome by the growing interest in technology observable nowadays.
Conversely, further studies are needed on AI. Even considering the boundless computing potential and AI applications, not enough articles connecting it to the field of orthopaedic surgery were found in the literature.
With ongoing research and ulterior theoretical applications needing only to be put in practice, extended reality technologies may be considered as an exciting new milestone for orthopaedic surgical practice.

Data Availability Statement:
The data presented in this study are available on request from the corresponding author.