Assessment of the Learning Curve for Virtual Surgical Planning in Orbital Fractures

Abstract

Virtual surgical planning (VSP) is becoming more widely used in maxillofacial reconstruction and can be surgeon-based or industry-based. Surgeon-based models require software training but allow surgeon autonomy. We evaluate the learning curve for VSP through a prospective cohort study in which planning times and accuracy of 7 otolaryngology residents with no prior VSP experience were compared to that of a proficient user after a single training protocol and 6 planning sessions for orbital fractures. The average planning time for the first session was 21 min 41 s ± 6 min 11 s with an average maximum deviation of 2.5 ± 0.8 mm in the lateral orbit and 2.3 ± 0.6 mm in the superior orbit. The average planning time for the last session was 13 min 5 s ± 10 min and 7 s with an average maximum deviation of 1.4 ± 0.5 mm in the lateral orbit and 1.3 ± 0.4 mm in the superior orbit. Novice users reduced planning time by 40% and decreased maximum deviation of plans by 44% and 43% in the lateral and superior orbits, respectively, approaching that of the proficient user. Virtual surgical planning has a quick learning curve and may be incorporated into surgical training.

Introduction

Computer-aided surgery (CAS) is gaining greater acceptance in maxillofacial reconstruction. Computer-aided surgery can be divided into 3 steps: (1) planning: also known as virtual surgical planning (VSP), (2) execution: carrying out the surgical plan (i.e., models, guides, and patient specific implants), and (3) verification: confirming that the proposed plan has been achieved (i.e., intraoperative navigation or computed tomography [CT]). This study focuses on VSP; the technique of using detailed CT imaging and planning software to generate a preoperative plan in a virtual environment. [1,2,3] Virtual surgical planning modalities may be surgeon-based (i.e., performed by the surgeon) or industry-based (i.e., performed by the surgeon and an engineer via teleconferencing). While the surgeon-based model requires an upfront financial investment in planning software and surgeon training, it allows the individual surgeon to autonomously generate the plan. The industry-based model does not require a financial investment or surgeon training. However, engineer costs, prolonged planning time, and diminished surgeon input are limitations in this model. [4,5,6] The industry-based model has gained a strong foothold, however as planning software becomes more user friendly, it is likely that surgeon-based workflows will become more common and gain greater acceptance. [6] The aim of this study is to evaluate the learning curve for novice VSP software users. The authors hypothesize that VSP software can be learned quickly and effectively without the need for extensive training, making it a feasible tool to incorporate as part of resident surgical training.

Materials and Methods

Study Design

Institutional review board approval was obtained for this prospective study. Study participants included 7 otolaryngology residents (2 postgraduate year 1 [PGY-1], 3 PGY-2, 1 PGY-3, 1 PGY-5) at a single academic institution. The participants had no experience with VSP. The study was broken into 2 parts, a single training protocol where participants were educated on the use of the VSP tool (iPlan—Brainlab AG) and 6 planning sessions where the participants’ virtual surgical plans were evaluated for duration and accuracy. Only the planning sessions were used for data collection/analysis. The senior author (EBS), having previously created >500 plans, acted as a proficient user/control.

Training protocol. The training protocol was developed to educate inexperienced participants in a uniform fashion. It was completed individually and proctored by the same investigator (TYH). Each participant went through the training protocol at their own pace (range 30-60 minutes). The training protocol was broken into 2 parts: user interface training and case training.

User interface training. The user interface training involved viewing an online training manual and basic interaction with the user interface options on a workstation.

Case training. Case training involved planning a single orbital reconstruction. Four tasks were included: (1) placement of fiducial markers to be used for intraoperative registration, (2) segmentation of the uninjured right orbit, (3) mirroring of the uninjured right orbit across the midline plane, and (4) precise alignment of the uninjured mirrored orbit onto the contralateral injured orbit (Figure 1A).

Planning sessions. Each participant independently completed 6 planning sessions. Each planning session required the participant to complete the same 4 tasks demonstrated in the training protocol. One case was provided to each participant every other day. Each case represented a similar but unique orbital fracture amenable to VSP. The order, setting, and timing of presentation were uniform and completed over an 11-day period. Recorded data points included (1) total time of each planning session, (2) accuracy of the mirroring task, and (3) the number of times each participant referred to the training manual per session. Accuracy was determined by comparing the largest deviation (in mm) of the mirrored orbit to the injured orbit. The measurements were taken along the orbital roof and lateral orbital wall, where there were no fractures (Figure 1B). These measurements were performed independently and averaged between 2 authors (TYH and RD) who were blinded to the participant and trial number.

Data Analysis

Ordinary least squares regression analysis with exponential fitting using RStudio (RStudio) was used for data analysis. To assess for proficiency and accuracy in the learning curve of VSP, the participants’ planning times and maximum deviation in the orbital roof and lateral orbital wall were compared with a proficient user’s data (EBS). A P value less than .05 was considered statistically significant. The data were plotted using Microsoft Excel.

Results

All participants completed the entire 6 case series. None of the participants referred to the BrainLab user manual during their planning session. The average planning time for the first case was 21 minutes 41 seconds ± 6 minutes 11 seconds with an average maximum deviation of 2.5 ± 0.8 mm in the lateral orbit and 2.3 ± 0.6 mm in the superior orbit (

Table 1. The Planning Time and Maximum Deviation in the Lateral and Superior Orbits in Plans Created by Each Novice User Throughout the Study.

Abbreviation: SD, standard deviation.

and Figure 2, Figure 3 and Figure 4). With the completion of subsequent planning cases, regression analysis showed an overall downward trend in the planning time (P = .0037, R² = 0.19; Figure 2) and in the maximum deviation in the lateral orbit (P = .0149, R² = 0.14; Figure 3) and the superior orbit (P = .0301, R² = 0.11; Figure 4). The average planning time for the last case was 13 minutes 5 seconds ± 10 minutes and 7 seconds with an average maximum deviation of 1.4 ± 0.5 mm in the lateral orbit and 1.3 ± 0.4 mm in the superior orbit (Table 1 and Figure 2, Figure 3 and Figure 4). Overall, there was a 40% decrease in total planning time with a decrease of 44% and 43% in the maximum deviation of plans in the lateral and superior orbits, respectively.

Discussion

Virtual surgical planning is an evolving tool in maxillofacial reconstruction that facilitates surgical repair and may optimize patient outcomes. [7,8,9] Bly et al. demonstrated that mirror image overlay in unilateral orbital fractures had a 5-fold decrease in revision surgeries and a 2-fold improvement in postoperative diplopia. [7] The use of intraoperative surgical planning has been proposed to improve intraoperative accuracy and reduce the incidence of revision procedures. [8] While this technology continues to grow, there is very little known about the learning curve for use of these tools. This study evaluated the learning curve for the use of iPlan software in orbital reconstruction by novice users.

The results of this study demonstrated a marked reduction in planning time from 21 minutes 41 seconds (± 6 minutes 11 seconds) to 13 minutes 5 seconds (±10 minutes and 7 seconds) after completion of 6 cases; a 40% reduction in time. Accuracy improved from 2.5 ± 0.8 mm to 1.4 ± 0.5 mm (lateral orbit) and 2.3 ± 0.6 mm to 1.3 ± 0.4 mm (superior orbit) after 6 cases; resulting in a 44% and 43% reduction, respectively. The average participant planning times and the maximum deviations in the lateral and superior orbits began to approach those of a proficient user (Figure 2, Figure 3 and Figure 4). It was noted that lateral orbit wall accuracy was improved more than the superior orbit. This difference may be reflective of the learning curve required to recognize facial asymmetry and geometric variability between the 2 orbits. [10] Given the rapid improvements in total time and accuracy, as well as regression analysis with P values <.05, we conclude that the data demonstrate both statistically significant decreases and practical improvements in planning time and planning accuracy. While improvement is expected with repetition, our results suggest that novice users can approach the proficiency of a competent user after a relatively short timeframe with only 1 training session and 6 planning sessions. Following the study, participating residents were ultimately able to independently use VSP software to develop preoperative surgical plans.

A potential limitation with VSP is cost. However, Rodby et al. demonstrated that VSP can decrease operative times while improving the accuracy of reconstruction; thus the overall cost savings may offset the technological cost. [11] Resnick et al. reported that VSP can significantly decrease planning time and overall planning cost associated with orthognathic surgery. [12] Finally, Mendez et al. demonstrated that surgeon-based VSP could potentially eliminate the costs and time associated with industrybased VSP. [13]

Limitations of this study include (1) use of a single medical center, (2) small sample size—likely resulting in low R² values which may influence the fit of the data in the regression analysis, and (3) use of single VSP software.

Conclusion

There is a quick learning curve for VSP with iPlan software. After 1 training session and the completion of 6 cases, novice users reduced their planning time by 40% (from 21 minutes 41 seconds to 13 minutes 5 seconds). Accuracy was improved by 44% for the lateral orbit (2.5 ± 0.8 mm to 1.4 ± 0.5 mm) and 43% for the superior orbit (2.3 ± 0.6 mm to 1.3 ± 0.4 mm). Novice users were able to approach the proficiency of a competent user relatively rapidly, suggesting VSP software and application may be a valuable addition to resident surgical training.