Next Article in Journal
Application for Decision-Making on Mild Cognitive Impairments
Previous Article in Journal
Monitoring of Older Adults’ Daily Activity and Sleep with Xiaomi Mi Band 2
 
 
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Proceeding Paper

Mixed Reality in an Operating Room Using Hololens 2—The Use of the Remote Assistance from Manufacturers Techinicians during the Surgeries †

by
Rita Veloso
1,‡,
Renato Magalhães
1,‡,
António Marques
1,‡,
Paulo Veloso Gomes
1,*,‡ and
Javier Pereira
2,‡
1
LabRP-CIR, Psychosocial Rehabilitation Laboratory, Center for Rehabilitation Research, School of Health, Polytechnic Institute of Porto, 4200-374 Porto, Portugal
2
CITIC, Research Center of Information and Communication Technologies, Talionis Research Group, Universidade da Coruña, 15071 A Coruña, Spain
*
Author to whom correspondence should be addressed.
Presented at the 4th XoveTIC Conference, A Coruña, Spain, 7–8 October 2021.
These authors contributed equally to this work.
Eng. Proc. 2021, 7(1), 54; https://doi.org/10.3390/engproc2021007054
Published: 27 October 2021
(This article belongs to the Proceedings of The 4th XoveTIC Conference)

Abstract

:
The aim of this work is that the participants, using HoloLens 2 and Dynamics 365 Remote Assistance, can receive all the training and information necessary for the correct application of prosthesis and medical devices remotely, from a support center of the manufacturers, avoiding the displacement and presence of these technicians during surgeries. After implementing this method, an analysis will be made on its impact, avoiding displacement and the presence of technicians during surgery, in terms of increasing satisfaction and improving the experience of the participants, reduction of various risks (including the risk of infection) and on reduction of some economic and environmental costs.

1. Introduction

Mixed reality (MR) was first mentioned in 1994 by Paul Milgram, and is a blend of physical and digital worlds, unlocking the links between human, computer, and environment interaction, based on advancements in computer vision, graphical processing power, display technology, and input systems [1]. By using holographic devices, such as Microsoft HoloLens 2, the participants could take advantage of the ability to place digital content in the real world as if it were there. This technology allows participants to see through display and see the physical environment while wearing the headset and allows a full six-degrees-of-freedom movement, both rotation and translation. The participants can hold “hands-free” and “heads-up” teams video calls with experts anywhere in the world, with all of benefits in this kind of experience [1,2].
Recent research has shown that with the use of advanced technological solutions such as MR, the spaces of the operating rooms tend to decrease and the number of professionals present during a surgery too [3], improving the efficiency in the use of resources in a hospital, whether they are human, of space, or technicians and materials. MR is a concept which provide an “ideal virtual space with [sufficient] reality essential for communication” [4]. The combination of computer processing, human input, and environmental input sets the stage for creating true MR experiences. Movement through the physical world translates to movement in the digital world and improves the experience and better outcomes of the participants and tasks [5], once it did not blind doctors’ original view of the real world, showing a new vision with mutual correction function, which improved the safety of surgery [6].
Currently, surgeons and nurses focus their attention on operating room, both on the patient’s body and on surgical monitors, to get all the information needed [7]. To help them in their procedures, MR starting been in operating rooms all over the world [8]. This kind of technology is also a powerful for better training and improving education on surgical tactics and methods [9]. During the last few decades, changes in surgical materials are constantly being developed and incremental to improve better outcomes [10], most of them needs accomplishment by the manufacturer’s technicians during the surgeries. Their presence in the operating room increases the risk of infection [11,12], the delays in surgeries, and the logistic costs of the prosthesis, particularly, in orthopedic field.
Every year in Portugal, orthopedic surgeons perform around 5000 arthroplasties (surgical replacements of necrotic or fractured with a prosthesis), mostly hip and knees arthroplasties [13]. Through displaying specific images in the HoloLens 2 and using in the Dynamics 365 Remote Assistance application, all the assistance could be virtual with a better accuracy of the patient instrumentation, scaling up the expert support all over the world with less costs. There are numerous complex surgeries that require the physical presence of technicians from the prosthesis manufacturers during the surgery to help the participants with their correct application. These technicians, despite being trained for the surgical environment, increase the number of professionals within the operating room, with all the associated risks, namely, the risk of infection.
The objectives of this study are: Test the effectiveness use of MR technology, in the surgical environment as a pilot example for future projects, as a tool for aiding participants during arthroscopies (position and collocation of the prosthesis) without the presence of the manufacturer’s technicians, firstly from the identification of the main requirements for this process: analyzing its impact of avoiding displacement and the presence of technicians during surgery, reducing the risk of infection, increasing satisfaction and improving the experience of the participants, the eventual reduction of surgery time, increasing flexibility in scheduling surgeries, increasing the profitability of the technicians’ time to perform assistance to a greater number of surgeries and finally, with no physical visits to hospitals, reducing the CO 2 emissions.

2. Methods

Preliminary systematic review, document analysis, analysis of similar previous cases in the industry, in particular the automotive industry using HoloLens 2 and Dynamics 365 Remote Assistance [14], identification of requirements to implement with professionals involved in surgeries with prosthesis, and initial exploratory interviews with orthopaedic surgeons.

3. Results

The development of a system that allows the use of HoloLens 2 as a tool for aiding participants during arthroscopies, and analyze its impact on several factors such as reducing the risk of infection; increasing satisfaction and improve the experience of the participants, using the video recording feature for future training; improve patient experience, sharing some previous videos of the procedure collected from the HoloLens 2; reducing surgery time increasing flexibility in scheduling surgeries; increasing the profitability of the technicians’ time to perform assistance to a greater number of surgeries; reducing the CO 2 emissions; reducing of the global costs of the prosthesis.

4. Conclusions

Using HoloLens 2 to support the surgical team in the operating rooms during the arthroplasty’s surgeries, participants will be able to share, in real-time, their vision of the patient’s specific location where the prosthesis will be applied. While communicating with the manufacturer’s specialist technician via Dynamics 365 Remote Assistance, to monitor the correct sequence of application of the parts, maintaining constant visual and voice interaction with technicians, they can be watched continuing its activity in a concentrated and focused way with the patient and the rest of the team. The collected data have shown that the use of HoloLens 2, while an immersive technology of mixed reality, could be useful to eliminate the presence of the manufacturer’s technicians during these surgeries, improving participants and patient experience, reducing the risk of infection and the duration of the surgeries while reducing the price of the prosthesis in a future negotiation with the manufacturers.

Author Contributions

Conceptualization, R.V., R.M.; methodology, R.V., R.M., A.M.; validation, R.V., R.M., P.V.G.; investigation R.V., R.M.; writing—original draft preparation, R.V., R.M.; writing—review and editing, R.V., R.M., P.V.G.; visualization, R.V., R.M.; supervision, A.M., J.P.; project administration, R.V., R.M. All authors have read and agreed to the published version of the manuscript.

Funding

This research received no external funding.

Institutional Review Board Statement

Not applicable.

Informed Consent Statement

Not applicable.

Acknowledgments

This research was carried out and used the equipment of the Psychosocial Rehabilitation Laboratory (LabRp) of the Research Center in Rehabilitation of the School of Allied Health Technologies, Polytechnic Institute of Porto.

Conflicts of Interest

The authors declare no conflict of interest.

References

  1. Microsoft. “What is Mixed Reality?”. 2021. Available online: https://docs.microsoft.com/en-us/windows/mixed-reality/discover/mixed-reality (accessed on 3 August 2021).
  2. Gallagher, J.A.L. Mixed-Reality Headsets in Hospitals Help Protect Doctors and Reduce Need for PPE. 2020. Available online: https://www.imperial.ac.uk/news/197617/mixed-reality-headsets-hospitals-help-protect-doctors/ (accessed on 3 August 2021).
  3. Joseph, A.; Allison, D. Designing A Safer OR. 2018. Available online: https://healthcaredesignmagazine.com/trends/research-theory/designing-a-safer-or/ (accessed on 3 August 2021).
  4. Milgram, P.; Kishino, F. Taxonomy of mixed reality visual displays. IEICE Trans. Inf. Syst. 1994, 77, 1321–1329. [Google Scholar]
  5. Carlos, F.; Sergio, I.-S.; Carlos, O. The impact of virtual, augmented and mixed reality technologies on the customer experience. J. Bus. Res. 2019, 100, 547–560. [Google Scholar]
  6. Hu, H.Z.; Feng, X.B.; Shao, Z.W.; Xie, M.; Xu, S.; Wu, X.H.; Ye, Z.W. Application and Prospect of Mixed Reality Technology in Medical Field. Curr. Med. Sci. 2019, 39, 1–6. [Google Scholar] [CrossRef] [PubMed]
  7. Galati, R.; Simone, M.; Barile, G.; Luca, R.D.; Cartanese, C.; Grassi, G. Experimental Setup Employed in the Operating Room Based on Virtual and Mixed Reality: Analysis of Pros and Cons in Open Abdomen Surgery. J. Healthc. Eng. 2020, 2020, 8851964. [Google Scholar] [CrossRef] [PubMed]
  8. Ferrari, V.; Klinker, G.; Cutolo, F. Augmented reality in healthcare. J. Healthc. Eng. 2019. [Google Scholar] [CrossRef] [PubMed]
  9. Zhu, E.; Hadadgar, A.; Masiello, I.; Zary, N. Augmented reality in healthcare education: An integrative review. PeerJ 2014, 2, e469. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  10. Keeney, J.A. Innovations in total knee arthroplasty: Improved technical precision, but unclear clinical benefits. Orthopedics 2016, 39, 217–220. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  11. Rebelo, S. Segurança do Doente no Bloco Operatório. Dissertação (Mestre em Enfermagem Médico-Cirúrgica), Escola Superior de Enfermagem de Coimbra, Coimbra, Portugal, 2013. [Google Scholar]
  12. Cristina, M.L.; Sartini, M.; Schinca, E.; Ottria, G.; Spagnolo, A.M. Operating room environment and surgical site infections in arthroplasty procedures. J. Prev. Med. Hyg. 2016, 57, E142. [Google Scholar] [CrossRef] [PubMed]
  13. Portuguese Arthroplasty Register (RPA). Available online: http://www.rpa.spot.pt/Quick-Links/Home.aspx?lang=en-GB (accessed on 3 August 2021).
  14. Mercedes-Benz Has Begun Using HoloLens 2 To Provide Maintenance Support. 2021. Available online: https://www.vrfocus.com/2020/09/mercedes-benz-has-begun-using-hololens-2-to-provide-maintenance-support/ (accessed on 3 August 2021).
Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Share and Cite

MDPI and ACS Style

Veloso, R.; Magalhães, R.; Marques, A.; Gomes, P.V.; Pereira, J. Mixed Reality in an Operating Room Using Hololens 2—The Use of the Remote Assistance from Manufacturers Techinicians during the Surgeries. Eng. Proc. 2021, 7, 54. https://doi.org/10.3390/engproc2021007054

AMA Style

Veloso R, Magalhães R, Marques A, Gomes PV, Pereira J. Mixed Reality in an Operating Room Using Hololens 2—The Use of the Remote Assistance from Manufacturers Techinicians during the Surgeries. Engineering Proceedings. 2021; 7(1):54. https://doi.org/10.3390/engproc2021007054

Chicago/Turabian Style

Veloso, Rita, Renato Magalhães, António Marques, Paulo Veloso Gomes, and Javier Pereira. 2021. "Mixed Reality in an Operating Room Using Hololens 2—The Use of the Remote Assistance from Manufacturers Techinicians during the Surgeries" Engineering Proceedings 7, no. 1: 54. https://doi.org/10.3390/engproc2021007054

APA Style

Veloso, R., Magalhães, R., Marques, A., Gomes, P. V., & Pereira, J. (2021). Mixed Reality in an Operating Room Using Hololens 2—The Use of the Remote Assistance from Manufacturers Techinicians during the Surgeries. Engineering Proceedings, 7(1), 54. https://doi.org/10.3390/engproc2021007054

Article Metrics

Back to TopTop