Intraoperative Use of Mixed Reality Technology in Median Neck and Branchial Cyst Excision
Abstract
:1. Introduction
- Medical Data Visualization Systems;
- Navigation Technologies through the Combination of Medical Data with the Patient during the Operation.
2. Materials and Methods
- The main aspects of application for this approach: The markers themselves are made of PLA plastic, while the image-texture coating is applied by means of UV print, which allows for the sterilization of the markers before their application in the operating room (Patent [19]);
- We transformed the CT into a 3D model using a segmentation approach with respect to the set density threshold or the area of interest in the 3D Slicer. This allowed us to isolate all key anatomical structures of the neck as a series of separate three-dimensional models. After that, resulting 3D models were optimized and cleaned up using the procedural toolkit in Houdini (up to 75–95% polygon reduction); we then transformed the MSCT into a 3D model using a segmentation approach with respect to the set density threshold or the area of interest. This allowed us to isolate all key anatomical structures of the neck separately as a series of three-dimensional models;
- We visualized 3D models using mixed reality smartglasses and the developed software, which has preset parameters for referencing the markers to the three-dimensional model. It has an integrated basic interface [20], which allows the user to display the required anatomical elements as well as customize the parameters of the marker tracking system;
- The 3D models were transferred to the HoloLens glasses using TCP via the Wi-Fi network, which alleviated time-consuming app compilations for each procedure.
3. Results
3.1. Case 1. Fixation Mask
Developed Approach
3.2. Case 2. Adjustable Navigation Frame
Operating Principle and Conclusion
3.3. Case 3. Adjustable Navigation Frame
Operating Principle and Conclusion
4. Conclusions
- Marker-based 3D visual localization [17]. Translation error from 1 m distance is about 2–3 mm;
- Manufacturing accuracy of fabricated marker holder. For FDM technology, this is about 0.2 mm;
- CT data slice thickness. In our case, all datasets have 1 mm thickness;
- Marker placement.
- The MR technology can be applied in the operating room as an additional auxiliary technique to assess the adequacy of surgical exposure as well as for intraoperative navigation;
- The bloodstream visualization eliminates the risk of great arterial and venous trunks intersection. Additional ligation is almost completely excluded;
- At the current stage of technological development, there is a competition between the brightness of the operating room lighting and the brightness of the hologram, which makes their simultaneous use impossible. Therefore, the MR technology can be implemented for periodic navigation of the wound with the lights dimmed as well as for postoperative follow-up of cysts and fistula excision;
- The direct use of mixed reality technology during surgical manipulations based on the problem of “focal rivalry” [25] is difficult. Therefore, glasses were used only in the process of planning and marking access, as well as in checking the results with the preoperative state.
- Despite the rigid fixation of the patient in Case 1, Cases 2 and 3, which involved no rigid immobilization of the patient relative to the operating table, demonstrated the best accuracy for hologram referencing. We attribute the obtained results to the variants of the reference marker’s attachment to the patients’ body parts and its relation to the surgery area. The closer they were to each other, the more accurately the hologram could reference to the real landmarks.
Author Contributions
Funding
Institutional Review Board Statement
Informed Consent Statement
Data Availability Statement
Conflicts of Interest
References
- Pilipiuk, N.V.; Gobzhelianova, T.A.; Chumakov, A.N.; Pilipiuk, D.N. Diagnostics and treatment of innate cysts and fistulae of neck. Visnyk Stomatol. 2011, 2, 44–50. [Google Scholar]
- Parkhimovich, N.P.; Lenkova, I.I. Diagnostic tactics for congenital cysts of the neck. In Dentistry Yesterday, Today, Tomorrow, Proceedings of the Anniversary Scientific and Practical Conference with International Participation Devoted to the 60th Anniversary of the Dentistry Department, Minsk, Belarus, 2–3 April 2020; Belarusian State Medical University: Minsk, Belarus, 2020; pp. 410–415. [Google Scholar]
- Rokhsaritalemi, S.; Sadeghi-Niaraki, A.; Choi, S.M. A review on mixed reality: Current trends, challenges and prospects. Appl. Sci. 2020, 10, 636. [Google Scholar] [CrossRef] [Green Version]
- Proniewska, K.; Dołęga-Dołęgowski, D.; Pręgowska, A.; Walecki, P.; Dudek, D. Holography as a progressive revolution in medicine. In Simulations in Medicine; De Gruyter: Berlin, Germany, 2020; pp. 103–116. [Google Scholar]
- Bin, S.; Masood, S.; Jung, Y. Chapter twenty—Virtual and augmented reality in medicine. In Biomedical Engineering; Feng, D., Ed.; Academic Press Elsevier: Cambridge, MA, USA, 2020; pp. 673–686. [Google Scholar]
- De Paolis, L.T. An augmented reality platform for preoperative surgical planning. J. Interdiscip. Res. App. Med. 2019, 3, 19–24. [Google Scholar]
- Ruthberg, J.S.; Quereshy, H.A.; Ahmadmehrabi, S.; Trudeau, S.; Chaudry, E.; Hair, B.; Kominsky, A.; Otteson, T.D.; Bryson, P.C.; Mowry, S.E. A multimodal multi-institutional solution to remote medical student education for otolaryngology during COVID-19. Otolaryngol. Neck Surg. 2020, 163, 707–709. [Google Scholar] [CrossRef] [PubMed]
- Proniewska, K.; Pręgowska, A.; Walecki, P.; Dołęga-Dołęgowski, D.; Ferrari, R.; Dudek, D. Overview of the holographic-guided cardiovascular interventions and training—A perspective. Bio-Algorithms Med-Syst. 2020, 16, 20200043. [Google Scholar] [CrossRef]
- Tepper, O.M.; Rudy, H.L.; Lefkowitz, A.; Weimer, K.A.; Marks, S.M.; Stern, C.S.; Garfein, E.S. Mixed reality with HoloLens: Where virtual reality meets augmented reality in the operating room. Plast. Reconstr. Surg. 2018, 140, 1066–1070. [Google Scholar] [CrossRef] [PubMed]
- Cartucho, J.; Shapira, D.; Ashrafian, H.; Giannarou, S. Multimodal mixed reality visualisation for intraoperative surgical guidance. Int. J. Comput. Assist. Radiol. Surg. 2020, 15, 819–826. [Google Scholar] [CrossRef] [PubMed]
- García-Vázquez, V.; von Haxthausen, F.; Jäckle, S.; Schumann, C.; Kuhlemann, I.; Bouchagiar, J.; Höfer, A.; Matysiak, F.; Hüttmann, G.; Goltz, J.; et al. Navigation and visualisation with HoloLens in endovascular aortic repair. Innov. Surg. Sci. 2018, 3, 167–177. [Google Scholar] [CrossRef] [PubMed]
- Saito, Y.; Sugimoto, M.; Imura, S.; Morine, Y.; Ikemoto, T.; Iwahashi, S.; Yamada, S.; Shimada, M. Intraoperative 3D hologram support with mixed reality techniques in liver surgery. Ann. Surg. 2020, 271, e4–e7. [Google Scholar] [CrossRef] [PubMed]
- Rose, A.S.; Kim, H.; Fuchs, H.; Frahm, J.-M. Development of augmented-reality applications in otolaryngology-head and neck surgery: Augmented Reality Applications. Laryngoscope 2019, 129, S1–S11. [Google Scholar] [CrossRef] [PubMed]
- Heinrich, F.; Schwenderling, L.; Becker, M.; Skalej, M.; Hansen, C. HoloInjection: Augmented reality support for CT-guided spinal needle injections. Healthc. Technol. Lett. 2019, 6, 165–171. [Google Scholar] [CrossRef] [PubMed]
- Sun, Q.; Mai, Y.; Yang, R.; Ji, T.; Jiang, X.; Chen, X. Fast and accurate online calibration of optical see-through head-mounted display for AR-based surgical navigation using Microsoft HoloLens. Int. J. Comput. Assist. Radiol. Surg. 2020, 15, 1907–1919. [Google Scholar] [CrossRef]
- Pepe, A.; Trotta, G.F.; Mohr-Ziak, P.; Gsaxner, C.; Wallner, J.; Bevilacqua, V.; Egger, J. A marker-less registration approach for mixed reality-aided maxillofacial surgery: A pilot evaluation. J. Digit. Imaging 2019, 32, 1008–1018. [Google Scholar] [CrossRef]
- Pentenrieder, K.; Meier, P.; Klinker, G.; Gmbh, M. Analysis of tracking accuracy for singlecamera square-marker-based tracking. In Proceedings of the Dritter Workshop Virtuelle und Erweiterte Realitat der GI Fachgruppe VR/AR, Koblenz, Germany, 25–26 September 2006. [Google Scholar]
- Ivanov, V.; Krivtsov, A.; Strelkov, S.; Gulyaev, D.; Godanyuk, D.; Kalakutsky, N.; Pavlov, A.; Petropavloskaya, M.; Smirnov, A.; Yaremenko, A. Surgical Navigation Systems Based on Augmented Reality Technologies. Available online: https://arxiv.org/ftp/arxiv/papers/2106/2106.00727.pdf (accessed on 20 December 2020).
- Ivanov, V.M.; Klygach, A.S.; Strelkov, S.V. Marker Holder Used for Head Surgery Based on Mixed Reality. RF Patent No. 202367, 2021. [Google Scholar]
- Shen, B.; Tan, W.; Guo, J.; Cai, H.; Wang, B.; Zhuo, S. A study on design requirement development and satisfaction for future virtual world systems. Future Internet 2020, 12, 112. [Google Scholar] [CrossRef]
- Hwang, L.; Lee, J.; Hafeez, J.; Kang, J.; Lee, S.; Kwon, S. A study on optimized mapping environment for real-time spatial mapping of HoloLens. Int. J. Internet Broadcasting Commun. 2017, 9, 1–8. [Google Scholar]
- Bartz, D.; Meiner, M. Voxels versus polygons: A comparative approach for volume graphics. In Volume Graphics; Chen, M., Kaufman, A.E., Yagel, R., Eds.; Springer: London, UK, 2000. [Google Scholar] [CrossRef] [Green Version]
- Ivanov, V.M.; Strelkov, S.V.; Smirnov, A.Y. Certificate of State Registration for Software No. 2021613930. Software for Anatomical Structures Visualization in a Form of Holograms and Their Space Positioning. Available online: http://fips.ru/EGD/7088ca20-1d0b-4095-97ba-b1375978825c (accessed on 16 March 2021).
- Yamamoto, S.; Taniike, N.; Takenobu, T. Application of an open position splint integrated with a reference frame and registration markers for mandibular navigation surgery. Int. J. Oral Maxillofac. Surg. 2020, 49, 686–690. [Google Scholar] [CrossRef] [PubMed]
- Condino, S.; Carbone, M.; Piazza, R.; Ferrari, M.; Ferrari, V. Perceptual limits of optical see-through visors for augmented reality guidance of manual tasks. IEEE Trans. Biomed. Eng. 2019, 67, 411–419. [Google Scholar] [CrossRef] [PubMed]
Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations. |
© 2021 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).
Share and Cite
Ivanov, V.M.; Krivtsov, A.M.; Strelkov, S.V.; Kalakutskiy, N.V.; Yaremenko, A.I.; Petropavlovskaya, M.Y.; Portnova, M.N.; Lukina, O.V.; Litvinov, A.P. Intraoperative Use of Mixed Reality Technology in Median Neck and Branchial Cyst Excision. Future Internet 2021, 13, 214. https://doi.org/10.3390/fi13080214
Ivanov VM, Krivtsov AM, Strelkov SV, Kalakutskiy NV, Yaremenko AI, Petropavlovskaya MY, Portnova MN, Lukina OV, Litvinov AP. Intraoperative Use of Mixed Reality Technology in Median Neck and Branchial Cyst Excision. Future Internet. 2021; 13(8):214. https://doi.org/10.3390/fi13080214
Chicago/Turabian StyleIvanov, Vladimir M., Anton M. Krivtsov, Sergey V. Strelkov, Nikolay V. Kalakutskiy, Andrey I. Yaremenko, Marina Yu. Petropavlovskaya, Maria N. Portnova, Olga V. Lukina, and Andrey P. Litvinov. 2021. "Intraoperative Use of Mixed Reality Technology in Median Neck and Branchial Cyst Excision" Future Internet 13, no. 8: 214. https://doi.org/10.3390/fi13080214