Information Recall in Pre-Operative Consultation for Glioma Surgery Using Actual Size Three-Dimensional Models

Three-dimensional (3D) technologies are being used for patient education. For glioma, a personalized 3D model can show the patient specific tumor and eloquent areas. We aim to compare the amount of information that is understood and can be recalled after a pre-operative consult using a 3D model (physically printed or in Augmented Reality (AR)) versus two-dimensional (2D) MR images. In this explorative study, healthy individuals were eligible to participate. Sixty-one participants were enrolled and assigned to either the 2D (MRI/fMRI), 3D (physical 3D model) or AR groups. After undergoing a mock pre-operative consultation for low-grade glioma surgery, participants completed two assessments (one week apart) testing information recall using a standardized questionnaire. The 3D group obtained the highest recall scores on both assessments (Cohen’s d = 1.76 and Cohen’s d = 0.94, respectively, compared to 2D), followed by AR and 2D, respectively. Thus, real-size 3D models appear to improve information recall as compared to MR images in a pre-operative consultation for glioma cases. Future clinical studies should measure the efficacy of using real-size 3D models in actual neurosurgery patients.


Introduction
Patient education plays an essential role in patient-centered care. Effective patient education enables patients to become active participants in their health care, which is the principle of shared decision making. Their participation in decision making leads to higher patient empowerment (defined as "a process through which people gain greater control over decisions and actions affecting their health" (World Health Organization) [1]) and high-quality health care [2,3]. Patients themselves also express the desire to receive such information [4] and good communication has resulted in higher patient satisfaction and better comprehension of medical information [5]. Effective communication may also enhance recall. This would be desirable, as about 40-80% of the information patients receive from healthcare professionals is immediately forgotten [6] and retention can be as low as 13% after one month [7].
In order to achieve informed consent, the patient has to understand the given information and be able to recall it correctly. The form in which information is given to the patient affects the information recall [6,8,9]. Thus, visual tools that have the ability to improve recall, might contribute to achieving informed consent. New 3D technologies (physical 3D models, Augmented Reality (AR), Virtual Reality (VR)), are increasingly being used in medicine. The literature about the use of 3D models in patient education is limited, but it has been shown to improve several aspects of information recall [10][11][12].
Especially in oncological cases, it is important to visualize the tumor, as the tumor size and location determine the treatment strategy. Particularly for glioma, the use of 3D technologies may positively influence communication with the patient. That is, the main treatment option in glioma is a resection as complete as possible, without damaging healthy brain tissue. The complex relation of the tumor to healthy brain tissue is different in each patient and three-dimensional (3D) in nature. Consequently, current two-dimensional visual tools lack the ability to properly display these complex 3D relations. A personalized 3D model, however, enables visualization of the patient-specific tumor and eloquent areas.
In a previous qualitative study about the use of 3D models in pre-operative glioma care, patients indicated that the model helped in understanding the surgical procedure and its risks [13]. Better recall was also mentioned by some patients. However, effects on recall of treatment-related information using 3D models in glioma patients have never been quantified.
In this study, we compare the amount of information recall after a mock pre-operative consult using a 3D model (physically printed or in AR) versus MR images in healthy individuals. In order to create an as optimal design as possible, our study group consists of carefully selected healthy participants with similar baseline characteristics as those of patients with a glioma. We hypothesize the percentage of information recall to be higher when using the 3D model or AR, than when using the MR images.

Participants
We performed an explorative study in which healthy, native Dutch speaking individuals between the age of 18 and 60 were eligible to participate. Exclusion criteria were as follows: 1. A history of neuro-oncological disease, 2. Cognitive impairment or subjective cognitive complaints due to a neurological or psychiatric disorder and 3. Brain anatomy knowledge at a high level (i.e., at the level of a neurologist, neurosurgeon, neuroscientist). To recruit individuals, a call was put out on two local platforms (Nextdoor application, Version 2.103 and the online board of the Radboud University). All participants were assigned to either the 2D (MRI/fMRI), 3D (physical 3D model) or AR group, stratified for age, sex, education and pre-existing anatomical knowledge. The research ethics committee of the Radboud University Nijmegen Medical Centre approved the study (registration number: 2019-5550).

Study Procedure
The experiment was set up to reflect a real-life scenario of a pre-operative consultation for low-grade glioma surgery. For this experiment, three different clinical cases of low-grade gliomas were used. Each case was linked to a specific neurosurgical resident who conducted all the consultations based on a pre-developed script (Appendix A) as to ensure a consistent content of the discussed information. Each resident conducted consultations using all three different visualization methods (2D images, 3D model and AR).
The experiment was conducted on a one-to-one basis in order to make the encounter as similar as possible to a real clinical consultation. Participants received a short scenario and were instructed to act like when they would visit the doctor themselves. Residents were instructed to act like a regular patient was visiting and to limit the consultation to a maximum of thirty minutes. After informed consent provided by the participant, consultations were audio recorded and later transcribed and analyzed for content.
Immediately after the consultation, participants filled out Questionnaire 1 for immediate recall (Appendix B), consisting of 26 multiple choice and one open question. Questions addressed four domains of factual knowledge: 1. Tumor characteristics, 2. Diagnosis/imaging, 3. Treatment options, 4. Risks of treatment. One week after the first assessment, participants received the same questionnaire, Questionnaire 2 for 1-week follow-up. All questionnaires were sent out through Castor (Ciwit B.V., Amsterdam, The Netherlands).

Visualization Tools
Within the 2D group, participants were shown axial, coronal and sagittal images of the tumor and its surrounding eloquent brain areas. T1 images with and without contrast, T2/FLAIR images and functional MR images were shown. These images were obtained from actual clinical cases and were fully anonymized.
Within the 3D group, participants were shown a physical, real-size 3D model of the tumor and the eloquent brain areas as well as the (f)MRI images on the screen. To produce the 3D models, the tumor and eloquent brain areas were segmented and, together with the tractography based on diffusion-weighted images, were visualized as 3D models. These were exported as a Digital Imaging and Communications in Medicine (DICOM) image sequence. The 3D model reconstructions were made using MeVisLab (MeVis MedicalSolutions AG, Bremen, Germany). Finally, the 3D model reconstructions were exported as STL files, made ready for printing using 3DSMax 2015 (Autodesk Inc., San Rafael, CA, USA) and imported in Cura version 2.1.2 64-bit to be sent to the Ultimaker 3D printer (Ultimaker BV, Geldermalsen, The Netherlands).
Within the AR group, an iPad Pro 11 (Apple Inc. Cupertino, CA, USA) was used to show the 3D image. The AR models were created by editing the STL files (mentioned above) in Unity (Unity Technologies, San Francisco, CA, USA). These were imported into in-house developed software, GreyMapp (© 2020, Nijmegen, the Netherlands).
An illustration of the used visual tools can be found in Figure 1.

Analysis
Scoring of questionnaires was done with an answer form per clinical case. Questionnaire outcomes are presented as percentages of correct answers (total sum of correct answers divided by the maximum score × 100). The maximum possible score was adjusted on a case-by-case basis after analysis of the audio transcripts when certain information had been omitted by the resident during the consultation. Differences between groups were analyzed using a repeated-measure general linear model (GLM) analysis with assessment (immediate vs 1 week) as within-subject factor and group (3D vs. 2D, AR vs. 2D and 3D vs. AR) as between-subject factor. Post hoc comparisons were performed using Fisher's Least Significant Difference. To determine effect sizes, we computed Cohen's d (d = 0.2 being a "small" effect, 0.5 a "medium" effect and 0.8 a "large" effect). The effect sizes for the 4 domains in the questionnaires were presented for descriptive purposes. All analyses were done using SPSS software version 26 (IBM Inc., Armonk, NY, USA).

Results
Between November 2019 and March 2020, sixty-three participants were enrolled in the study. However, as a result of the COVID-19 measures, only sixty-one participants participated. All patients completed both assessment 1 and 2, with a mean duration of 8.7 days between both assessments. There was no difference in age, level of education or pre-existing anatomical knowledge between groups (Table 1). There was no difference in age, level of education or pre-existing anatomical knowledge between groups. 2D = two-dimensional group (MRI), 3D = three-dimensional group (3D model), AR = Augmented Reality group. * = other than neurologist/neurosurgeon.
A summary of the questionnaire outcomes can be found in Table 2. Figure 1 shows the outcomes of both questionnaires for each group and the spread, the latter being largest in the 2D group for both questionnaires. The questionnaire outcomes did not differ significantly across the three cases, belonging to a specific resident (immediate recall; F(2, 58) = 0.10 ; p = 0.909 and 1-week follow-up; F(2, 58) = 0.38 ; p = 0.686). There was an overall higher score at immediate recall (p = 0.001). Both 3D (p < 0.0005) and AR (p = 0.025) group scored significantly higher than the 2D group. 2D = two-dimensional group (MRI), 3D = three-dimensional group (3D model), AR = Augmented Reality group.
The effects were large for the 3D model group compared to the 2D group (immediate recall: d = 1.76; 1-week follow-up: d = 0.94). Comparing the AR to the 2D group showed a large difference for immediate recall (d = 1.09), but a small difference after the 1-week follow-up (d = 0.29). Thus, the effect sizes of 3D as compared to 2D are higher than the effect of AR as compared to 2D. Figure 2 shows effect sizes per domain for immediate recall and the 1-week follow-up recall. Per-domain analysis showed that the effect on recall ranges for each group; thus, improvement in recall is not limited to one specific domain.

Discussion
The main purpose of this study was to compare the amount of information that is recalled after a mock pre-operative consult using either a 3D model, AR models or 2D MR images. Our outcome shows that recall was highest for the 3D printed model group, followed by the AR group, with the lowest recall obtained in the 2D MRI group.
Our findings, that both 3D methods (either physically printed or presented using augmented reality) result in superior recall than using a 2D MR images, are in line with the few previous studies about the use of 3D models in patient education. In a qualitative study, patients indicated that the use of a 3D model made it easier for them to understand their surgeon and to ask questions regarding their situation [13]. Another, quantitative study [12] described a significant larger improvement in comprehension and satisfaction when using 3D printed models (Likert score of 1.7 with SD = 0.67 to 4.7 with SD = 0.67) versus computed tomography angiography (CTA) images (1.5 with SD = 0.53 to 2.5 with SD = 0.53) for explaining cerebral aneurysm clipping. For lumbar degeneration, a prospective study on the use of 3D models and 3D reconstructions (on a 2D screen) in a pre-operative consultation for lumbar degeneration showed a better understanding in the 3D model group (15.9% compared to CT/MRI, 8.5% compared to 3D reconstruction) as well as the highest satisfaction in the 3D model group, followed by 3D reconstruction and CT/MRI (91.3%, 66.83%, and 32.8%) [10].
Studies on non-neurosurgical patients found similar results [11,[14][15][16]. Patients verbally stated an improvement in understanding of anatomical details and surgical intervention in patients with renal malignancies [15]. Another study [16] used 3D models to explain the anatomy, disease state and treatment options in nasal sinus defects and found statistically significant improvements in patient reported (anatomical) understanding.
Extending previous studies, we also examined subdomains of the information provided separately (Tumor characteristics, Imaging and diagnosis, Treatment options and Risks of treatment). This is relevant, as for instance the treatment options and risks are the most important aspects of a pre-operative consultation for the patient to understand, since they determine the prognosis. We found a medium to large effect size for all four domains in the 3D group, suggesting that 3D models improve recall on all information dimensions.
When comparing AR to 3D models we found that the 3D model scored consistently better on both questionnaires, but this difference was not significant. The effect size of 3D models both on immediate and 1-week-follow-up recall was higher than for the AR models (Cohen's d = 1.76 and 0.94 (3D) vs. 1.09 and 0.29 (AR)) as compared to 2D. As shown in Figure 1, this seems especially to be caused by low performance of the AR models in the tumor characteristics domain. This finding is similar to a previous study investigating the role of 3D and AR models in renal and prostate malignancies [11,14,15]. This possibly can be explained by inexperience of patients in using AR, difficulty in translating AR images to "real life" and the added value of being able to touch and feel a physical 3D model. Considering these superior effect sizes, we recommend using physical models over AR models.
The first step in creating both 3D and AR models, is to visualize the tumor and the eloquent brain areas as a 3D model using Brainlab technology. At our institute, this step is part of the standard pre-operative workflow of the neurosurgeon and is not considered as additional work. However, the following step to translate these files into physical 3D models or AR images for the application, takes one to two hours of technician time per model. The production price of the physical 3D models depends on the used printer and materials; at our institute, materials for one model cost around 5 EUR. Considering the benefits we have found, we think these costs are acceptable.
This experimental study is the first to compare the information recall after using a 3D model or AR models. A strength of our study is the set-up of the experiment with three different clinical cases. Another strength is the use of two assessments, that is, immediate recall and a 1-week follow-up assessment, and the use of audio recordings, which enabled us to analyze the consultations of individual participants in a reliable way.
A limitation of our study could be the inclusion of healthy individuals rather than patients, which limits the external validity of our findings. Inclusion of healthy individuals without any cognitive impairments and who are not emotionally involved in the discussed subject might also explain the relatively high overall recall. However, we selected a study population based on similar baseline characteristics as those of patients with a glioma. Furthermore, this study provides important information for setting up a subsequent study in glioma patients. Such a study would also require detailed assessment of cognitive dysfunction (which may be the case in any brain tumor patient), and take factors such as attentional narrowing [17] and anxiety [18] into account. In healthy individuals, these factors do not play an important role and thus do not act as confounding factors. Finally, a subsequent study in glioma patients might show a different (possibly lower) percentage of information recall.

Conclusions
Real-size 3D models, either as 3D prints or as AR models, improve information recall compared to using 2D MR images in a pre-operative consultation for glioma cases. To further investigate the effectiveness of 3D models as a visual aid for patient education, clinical studies should be set up to quantitively measure patient experiences. Acknowledgments: The authors would like to thank the three neurosurgical residents involved in this study, Martine van Bilsen, Ilaria Viozzi and Hugo F. den Boogert, for their time and effort.

Conflicts of Interest:
The authors declare no conflict of interest.

Appendix A. Pre-Developed Script
General Information (All Groups) Explain the following: • Lesion found on imaging, suspected of low-grade glioma.

•
There are four different grades and the lesion is suspected to be a grade 2 glioma but inevitably grows into a higher grade. • Surgery provides better prognosis than a wait and see approach. This is why we will discuss the options for surgery and its risks.

•
The tumor pathology will give more insight into whether post-operative treatment (chemotherapy, radiotherapy) are indicated.

•
To assess the possible risks, we made a functional MRI (fMRI) to map the functional areas and tracts related to the tumor. During the fMRI, they had to tap their fingers, move their toes, think of different words and, finally, say different verbs out loud.

Group 1
Explain and show the following while sliding through the (f)MR images: • Left and right are reversed. • T2 FLAIR: The tumor size (mention the maximum diameter, further specified in the script for each case) and location (brain lobe). • fMRI: activation of functional areas (e.g., hand-, leg-, lip-, tongue function and speech), tractography (e.g., corticospinal and superior longitudinal fasciculus) and their function. Their relation to the tumor. The enhancing areas that display the artifacts.

Group 2 and 3
Explain and show the following while showing the (f)MR images and 3D model or AR model: • Left and right are reversed. • T2 FLAIR: The tumor size (mention the maximum diameter, further specified in the script for each case) and location (brain lobe).

Treatment Options and Risks (All Groups)
Explain the following treatment options and risks: • There are four treatment options: watchful waiting, biopsy, partial resection (debulking) and complete resection.

•
Watchful waiting means following the tumor through scanning after a set number of months and checking for tumor growth. This eliminates all risks related to surgery, but is not the preferred option, since the tumor will grow and develop into a higher grade.

•
Biopsy means taking multiple small pieces of the tumor through a small burr hole in the skull. This gives us more information about the diagnosis, but with less risks than extensive surgery. Risks (chances are approximately 1% of happening) include bleeding followed by neurological deficit (paralysis or speech deficit), epilepsy and infection. Also, the amount of tumor that is removed during surgery is minimal, thus the advantages for prognosis seen in resection do not apply for biopsy. The tumor can be heterogenous, so there can be a risk of sampling error (meaning not getting the exact diagnosis). Finally, there is a small risk that the biopsy is insufficient after which a second biopsy is needed.

•
Show the angle in which the biopsy will be taken (depending on the group either on MRI, 3D model of AR model).

•
Partial resection (debulking) means taking a piece of tumor that is safely possible. The opening is larger than in a biopsy. This way, we keep away from the relevant brain areas and tracts. While the chances are lower, there is still a risk for neurological deficit (paralysis or speech deficit) if bleeding occurs. This lower risk is an advantage (compared to complete resection), but the disadvantage is the limited resection and thus it does not carry the advantages for progression in complete resection.

•
Show the part of the tumor that will be removed in partial resection (debulking) surgery and its relation to the eloquent brain areas and tracts (depending on the group either on (f)MRI, 3D model of AR model).

•
Complete resection means that the aim is to remove as much tumor as safely possible. To this end, the functions will be monitored during surgery. For speech, the patient needs to be awake for some time during the surgery. Complete resection means that a large part of the tumor cannot evolve into a higher grade. However, it is impossible to resect one hundred percent, since the infiltrative growth. With the large amount of tumor that is resected, the right diagnosis will be made. It is known that this surgery provides the best prognosis. The disadvantage is the highest risk chance: bleeding (2%), permanent neurological deficit (10%), temporary neurological deficit (50%), epilepsy and infection.

•
Show the functional areas and mention that these functions will be monitored during surgery (depending on the group either on (f)MRI, 3D model of AR model).

•
Explain the details of an awake procedure: pre-operative preparing through psychological tests and exercises (similar to the exercises during surgery), peri-operative procedure (sleeping during surgery preparation i.e., catheter and head clamp placement, opening of skin and skull, awake during testing and sleeping during closing).

Appendix B. Questionnaire
Subdomain 1: Tumor Characteristics • What type of tumor is shown?