Next Article in Journal
Hyperpigmented Scleroderma-like Lesions under Combined Pembrolizumab and Pemetrexed Treatment of Non-Small Lung Cancer
Previous Article in Journal
The Case of a Patient with Limited Systemic Sclerosis and Interstitial Lung Disease Overlapping with Systemic Lupus Erythematosus
Order Article Reprints
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:

Augmented and Virtual Reality in Dermatology—Where Do We Stand and What Comes Next?

Sensors and Measuring Systems Group, Institute of Computational Physics, School of Engineering, Zurich University of Applied Sciences ZHAW, 8400 Winterthur, Switzerland
Department of Dermatology, University Hospital Zurich USZ, 8091 Zurich, Switzerland
Department of Dermatology, University Hospital Basel USB, 4031 Basel, Switzerland
Author to whom correspondence should be addressed.
Dermato 2022, 2(1), 1-7;
Received: 10 November 2021 / Revised: 13 January 2022 / Accepted: 21 January 2022 / Published: 25 January 2022
(This article belongs to the Collection Artificial Intelligence in Dermatology)


As the skin is an accessible organ and many dermatological diagnostics still rely on the visual examination and palpation of the lesions, dermatology could be dramatically impacted by augmented and virtual reality technologies. If the emergence of such tools raised enormous interest in the dermatological community, we must admit that augmented and virtual reality have not experienced the same breakthrough in dermatology as they have in surgery. In this article, we investigate the status of such technologies in dermatology and review their current use in education, diagnostics, and dermatologic surgery; additionally, we try to predict how it might evolve in the near future.

1. Introduction

Augmented reality (AR) is a component of virtual reality (VR) or virtual environments (VE) where the user is completely immersed into a synthetic environment [1]. While in VR, the user does not see the real world. Instead, AR allows us to superimpose virtual objects onto real ones. The number of AR/VR applications has undeniably exploded over the last decades. While both techniques have been extensively used in various medical fields, such as in plastic surgery [2], little investigation has been performed so far in dermatology [3,4,5]. This is surprising as (i) the skin is an accessible organ of the body particularly suited for AR applications, (ii) dermatology is a medical field where visualization is essential, and (iii) the sector is worth billions of dollars in the US alone, leading to numerous business opportunities. The main applications of AR/VR in dermatology can be categorized into three main groups: education [6,7,8], dermatologic surgery [9,10] and diagnostics [11]. In a paper published in early 2000, Gladstone and coworkers identified the twenty-first century dermatological VR applications [10]. Detailed in their visionary manuscript, they predicted that dermatologists will be soon able to train the night prior to performing complex Mohs surgery using patient specific skin and skeletal data. Students will not only be able to visualize a lesion in 3D but will also be capable of physically touching it using haptic feedback systems. Thanks to such tools, telemedicine could be revolutionized. Finally, VR technology will enable “moving” inside a lesion and observing it at different angles to improve diagnostics. In this following article, we review the status of AR and VR and give insight into potential future developments.

2. Education

Improving dermatologic teaching is a challenge, as visual memory is difficult to maintain without constant exposure. As a result, it has been reported that the diagnostic accuracy of common skin diseases is poor by primary care physicians [12]. The use of 3D silicon-made phantoms greatly improves diagnostics skills and knowledge retention in comparison to standard 2D images [13]. Therefore, it is expected that AR should exhibit similar effects [8]. Nonetheless, recent studies have demonstrated ambiguous results with minor differences in knowledge gain and slightly better knowledge retention for the group using AR-based tools in comparison to textbooks [7]. This outcome could potentially be improved by coupling AR with haptic feedback, as is the case in most surgical training tools.
A potential application that could prove to be beneficial in training the next generation of medical experts would include realistic-looking 3D models displaying skin diseases and irregularities (see Figure 1). These models would contain high resolution clinical textures that are anonymized on newly generated 3D models with anatomical proportions, all of which could be run on smartphones/tablets or VR goggles. The environmental conditions (like lighting conditions) could be adapted for a more realistic experience. In addition, such models could be expanded by using synthetic clinical images, to reduce data scarcity and poor training regarding rare dermatological diseases and certain Fitzpatrick skin types. For example, incidence of melanoma is relatively rare among black populations, where survival rates lag behind the rates for non-Hispanic white populations in the US [14]. Such issues have recently been highlighted in the press [15]. The goal of the DermGAN project is to improve diversity in clinical skin images [16]. In this project, a model generates skin images that exhibit the characteristics of a given pre-specified skin condition, location, and underlying skin color that could be recreated into a 3D visualization.
Aside from health professionals, some studies have investigated AR/VR as an educational tool for patients or the public to increase their awareness of skin disease prevention. The group Elke Hacker in Australia has developed a virtual reality (VR) game containing preventative skin cancer messages for youth and young adults [17]. This was based on the fact that despite successful public campaigns in the 1980’s, high rates of sunburn continue to be reported among the young population.
In another application of VR in dermatology, we created the VR app Virtual Derm, which places students in a virtual practice for dermatology [18]. The app runs on standard Android smartphones, making it accessible to most students. A total of 100 patient vignettes can be solved by students, making it a unique learning opportunity (see Figure 2).

3. Dermatologic Surgery

Dermatologic surgery is the practice of dermatology that specializes in surgical procedures to repair or improve the function and appearance of the skin [19]. While plastic surgery greatly benefited from AR/VR technology [20], few applications have been reported in dermatologic surgery. Aside from surgical procedure training [21] (not discussed in the paragraph above dedicated to education), AR and VR have been implemented in Mohs surgery to reduce patient anxiety [22] or as a surgical assistance tool [23].
For patients, VR has been used to reduce pre- and intraoperative anxiety. The tools employed in this study were unspecific VR sequences which could be used to reduce anxiety under any circumstance. It has been shown that almost all anxiety-related parameters could be improved in the context of surgery [22]. More specifically, VR could be used as tool for patient education, as many patients do not have a clear understanding what a surgical procedure entails, such as using a flap for the reconstruction of a surgical defect. However, this has not been investigated and may also induce anxiety in some patients where too much detail about a surgical procedure is given.
For physicians, VR has been used as a training tool in many surgical specialties and has great potential in dermatologic surgery. The more realistic appearance of a VR image, combined with haptic sensations, can be used to train dermatologic surgical interventions in the same way as in other surgical fields, notably plastic surgery. AR has been successfully used by one group to improve the communication between surgeons and pathologists in cases of Mohs surgery, where the pathology slides were not necessarily read by the surgeon themselves [23]. In addition, AR has great potential for other aspects of Mohs surgery and dermatologic surgery. After acquiring the theoretical background and training with artificial models, a surgical procedure is best learnt by performing the procedure on a real patient. In most training programs, this is performed under the direct supervision of a senior faculty member who is present at the operating table. However, due to shortcomings in staff resources and planning, this is not always possible and junior surgeons sometimes find themselves performing an operation that is above their level of confidence alone. AR could allow a senior faculty member to follow an operation remotely and give very precise and practical instructions to the junior surgeon at the table, with AR images superimposed on the surgical site to guide the surgery. In larger institutions, this would also allow senior staff resources to be used more efficiently, as one experienced surgeon could follow several surgeries at the same time.

4. Diagnostics

Artificial intelligence (AI) has been productively utilized for several decades in dermatology, particularly for skin cancer detection. It has been reported that convolutional neural networks (CNNs) achieve dermatologist-level classification of cutaneous lesions using both dermoscopic and non-dermoscopic images [24]. AI has proven very useful when combined with VR reconstruction of the body surface. In a prominent example, the Canfield® Vectra WB360 device combines 2D photographs of 46 stereo-vision pods, resulting in a full-body VR reconstruction that can be turned and manipulated at will. The machine further analyzes all photographs with ML-based segmentation and classification, looking for naevi and other cutaneous lesions. All naevi can be analyzed by artificial intelligence for their probability of being malignant. The information on these lesions is saved, with the position of each lesion on the body map, and upon repeated patient use of the Vectra device, changes are tracked over time. By utilizing this technology, newly developing melanomas are actively being found. We are currently performing a study (Melanoma Detection in Switzerland With VECTRA–MELVEC, Identifier: NCT04605822) which compares dermatologists’ diagnostic accuracy with and without support by artificial intelligence. Patients receive their VR reconstruction and their results to peruse within the comfort of their home. To date, no other VR approach has been made available in dermatology and has generated much enthusiastic feedback from those in the field.
AI-based smartphone applications, or “apps”, are being successfully developed as practical tools to improve diagnostics [25,26,27,28], despite some limitations in terms of accuracy for melanoma screening [27]. In a recent study [25], a mobile augmented reality approach was used to support real-time diagnostics for melanoma lesions using deep learning. Parameters such as lesion diameter, color, or asymmetry are displayed in real time in the camera view (see Figure 3). The preliminary evaluation of the app was very encouraging [25].
The use of AR/VR in dermatology could strongly benefit from haptic feedback. Haptic feedback, or “haptics”, refers to manual interaction with the environment, and could be used to relay force and tactile information to the user [29]. Indeed, sensory information such as skin temperature, roughness or stiffness would provide valuable additional information to support diagnostic evaluation [30]. In this direction, Kim and co-workers developed a method to convert a single image to a 3D haptic surface and render the generated haptic surface in real-time [31,32].

5. Methodology

For this review we performed a literature search using the Web of Science Core Collection and MEDLINE® databases. The descriptors used were “virtual environments”, “virtual reality”, “augmented reality”, “dermatology”, and “dermatological science”. We excluded the descriptor “skin” on purpose as we were not interested in haptics systems, but wanted to focus on the applications of augmented and virtual reality in dermatology only. Using these descriptors, the search was performed among “title”, “abstract”, and “author keywords”. We considered all publication years and document types. We did not exclude any “research areas”. We did not exclude any MeSH Qualifiers or MeSH Headings. We restricted our search to documents in the English language only. With such parameters, 64 articles were found, with 35 documents published in the last 5 years and 51 published in the research area “Dermatology” (full search report available as Supplementary document). We manually went through the articles, selecting relevant ones via the title and abstract.

6. Limitations

In this paragraph we shortly review the limitations of augmented and virtual reality in dermatology, focusing on applications, and without investigating potential adverse health effects on the user resulting from the use of AR/VR.
The use of AR/VR in dermatologic teaching suffers from similar limitations as in general medical education. Those have been recently extensively studied, potentially due to the restriction associated with COVID-19 pandemic [33,34,35]. Parsons and MacCallum investigated the affordances and limitations of augmented reality in medical education [33]. Selecting five potential affordances (A1: reducing negative impacts (risk, cost), A2: visualizing the otherwise invisible, A3: developing practical skills in a spatial context, A4: device portability across locations, A5: situated learning in context), the authors explored how these affordances were highlighted in a systematic literature review. They pointed out the versatility of the use of AR/VR in medical education, but suggested that proper selection of the affordances when designing a new system for medical education may provide benefits [33].
In dermatologic surgery and diagnostics, augmented and virtual reality technologies exhibit several limitations that are shared with the applications of AR/VR in general surgery [20]. If technological limitations such as image quality, battery life, usability (e.g., size, cables) can hamper the method, they most likely will decrease with technological evolution. As with all patient electronic data, confidentiality should be guaranteed and all legal requirements should be met [20].

7. Conclusions

AR and VR have brought significant advances for dermatology in education—both for medical students and for patients and general public—in dermatological surgery, particularly for complex surgical operations such as Mohs surgery, and in diagnosis, particularly for naevi screening. Even though the technology has not revolutionized standard practice and is still rarely used in the field, the dramatic increase of the research performed during the last 5 years allows for optimism. There is more to come as the technology progresses and is democratized. However, we believe that VR will be replaced successively with AR approaches that allow users to directly interact with the clinical environment via dedicated haptics.

Supplementary Materials

The following supporting information can be downloaded at: Supplementary document: Search report from Web of Science.

Author Contributions

Conceptualization, M.B.; methodology, M.B.; formal Analysis, M.B.; writing—original draft preparation, M.B.; writing—review and editing, S.L. and A.N.; Funding Acquisition, M.B. All authors have read and agreed to the published version of the manuscript.


Mathias Bonmarin is a DIZH Fellow and acknowledges financial support from ZHAW digital.

Institutional Review Board Statement

Not applicable.

Informed Consent Statement

Not applicable.

Data Availability Statement

Not applicable.


The authors would like to thank Valery Metry for implementing the 3D full body visualization App presented in this paper, and Madeleine DeBrosse for editing the English of the manuscript.

Conflicts of Interest

The Virtual Derm application has been developed by A.N.


  1. Azuma, R.T. A survey of augmented reality. Presence-Virtual Augment. Real. 1997, 6, 355–385. [Google Scholar] [CrossRef]
  2. Reznick, R.K.; MacRae, H. Medical education-Teaching surgical skills-Changes in the wind. N. Engl. J. Med. 2006, 355, 2664–2669. [Google Scholar] [CrossRef] [PubMed][Green Version]
  3. Obagi, Z.; Rundle, C.; Dellavalle, R. Widening the scope of virtual reality and augmented reality in dermatology. Dermatol. Online J. 2020, 26, 13030. [Google Scholar] [CrossRef] [PubMed]
  4. Sharma, P.; Vleugels, R.A.; Nambudiri, V.E. Augmented reality in dermatology: Are we ready for AR? J. Am. Acad. Dermatol. 2019, 81, 1216–1222. [Google Scholar] [CrossRef]
  5. Prado, G.; Kovarik, C. Cutting Edge Technology in Dermatology: Virtual Reality and Artificial Intelligence. Cutis 2018, 101, 236–237. [Google Scholar]
  6. Noll, C.; Haussermann, B.; von Jan, U.; Raap, U.; Albrecht, U.V. Mobile Augmented Reality in Dermatology. Biomed. Eng.-Biomed. Tech. 2014, 59, S1216. [Google Scholar] [CrossRef]
  7. Noll, C.; von Jan, U.; Raap, U.; Albrecht, U.-V. Mobile Augmented Reality as a Feature for Self-Oriented, Blended Learning in Medicine: Randomized Controlled Trial. JMIR mHealth uHealth 2017, 5, e7943. [Google Scholar] [CrossRef][Green Version]
  8. Aldridge, R.B.; Li, X.A.; Ballerini, L.; Fisher, R.B.; Rees, J.L. Teaching Dermatology Using 3-Dimensional Virtual Reality. Arch. Dermatol. 2010, 146, 1184–1185. [Google Scholar] [CrossRef]
  9. Kantor, J. Application of Google Glass to Mohs Micrographic Surgery: A Pilot Study in 120 Patients. Dermatol. Surg. 2015, 41, 288–289. [Google Scholar] [CrossRef]
  10. Gladstone, H.B.; Raugi, G.J.; Berg, D.; Berkley, J.; Weghorst, S.; Ganter, M. Virtual reality for dermatologic surgery: Virtually a reality in the 21st century. J. Am. Acad. Dermatol. 2000, 42, 106–112. [Google Scholar] [CrossRef]
  11. Zhang, S.; Blalock, T.W. Measuring Cutaneous Lesions: Trends in Clinical Practice. Dermatol. Surg. 2018, 44, 383–387. [Google Scholar] [CrossRef] [PubMed]
  12. Federman, D.G.; Kirsner, R.S. The abilities of primary care physicians in dermatology: Implications for quality of care. Am. J. Manag. Care 1997, 3, 1487–1492. [Google Scholar] [PubMed]
  13. Garg, A.; Haley, H.-L.; Hatem, D. Modern Moulage Evaluating the Use of 3-Dimensional Prosthetic Mimics in a Dermatology Teaching Program for Second-Year Medical Students. Arch. Dermatol. 2010, 146, 143–146. [Google Scholar] [CrossRef] [PubMed][Green Version]
  14. Culp, M.B.; Lunsford, N.B. Melanoma among Non-Hispanic Black Americans. Prev. Chronic Dis. 2019, 16. [Google Scholar] [CrossRef] [PubMed][Green Version]
  15. Caryn Rabin, R. Dermatology’s Skin Color Problem. The New York Times, 8 September 2020; 1. [Google Scholar]
  16. Ghorbani, A.; Natarajan, V.; Coz, D.; Liu, Y. DermGAN: Synthetic Generation of Clinical Skin Images with Pathology. In Proceedings of the Machine Learning for Health NeurIPS Workshop. Proc. Mach. Learn. Res. 2020, 116, 155–170. [Google Scholar]
  17. Horsham, C.; Dutton-Regester, K.; Antrobus, J.; Goldston, A.; Price, H.; Ford, H.; Hacker, E. A Virtual Reality Game to Change Sun Protection Behavior and Prevent Cancer: User-Centered Design Approach. JMIR Serious Games 2021, 9, e24652. [Google Scholar] [CrossRef]
  18. Virtual Derm Is a Health Education app, Meant to Provide a Training Platform to Help Medical Students and Dermatologists Training Their Observational, Diagnostic and Treatment/Care Skills for a Better Patient Care in Dermatology. Version 2.0 Available for Download on Google Play. Available online: (accessed on 20 December 2021).
  19. Hale, E. Handbook of Dermatologic Surgery; Springer: New York, NY, USA, 2014. [Google Scholar]
  20. Khor, W.S.; Baker, B.; Amin, K.; Chan, A.; Patel, K.; Wong, J. Augmented and virtual reality in surgery-the digital surgical environment: Applications, limitations and legal pitfalls. Ann. Transl. Med. 2016, 4. [Google Scholar] [CrossRef][Green Version]
  21. Berg, D.; Raugi, G.; Gladstone, H.; Berkley, J.; Weghorst, S.; Ganter, M.; Turkiyyah, G. Virtual reality simulators for dermatologic surgery: Measuring their validity as a teaching tool. Dermatol. Surg. 2001, 27, 370–374. [Google Scholar] [CrossRef]
  22. Higgins, S.; Feinstein, S.; Hawkins, M.; Cockburn, M.; Wysong, A. Virtual Reality to Improve the Experience of the Mohs Patient-A Prospective Interventional Study. Dermatol. Surg. 2019, 45, 1009–1018. [Google Scholar] [CrossRef]
  23. Rodriguez-Jimenez, P.; Ruiz-Rodriguez, R. Augmented reality in Mohs micrographic surgery. Int. J. Dermatol. 2020, 59, E22–E23. [Google Scholar] [CrossRef]
  24. Young, A.T.; Xiong, M.L.; Pfau, J.; Keiser, M.J.; Wei, M.L. Artificial Intelligence in Dermatology: A Primer. J. Investig. Dermatol. 2020, 140, 1504–1512. [Google Scholar] [CrossRef] [PubMed]
  25. Francese, R.; Frasca, M.; Risi, M.; Tortora, G. A mobile augmented reality application for supporting real-time skin lesion analysis based on deep learning. J. Real-Time Image Process. 2021, 18, 1247–1259. [Google Scholar] [CrossRef]
  26. Freeman, K.; Dinnes, J.; Chuchu, N.; Takwoingi, Y.; Bayliss, S.E.; Matin, R.N.; Jain, A.; Walter, F.M.; Williams, H.C.; Deeks, J.J. Algorithm based smartphone apps to assess risk of skin cancer in adults: Systematic review of diagnostic accuracy studies. Bmj-Br. Med. J. 2020, 368. [Google Scholar] [CrossRef] [PubMed][Green Version]
  27. Sun, M.D.; Kentley, J.; Mehta, P.; Duzsa, S.; Halpern, A.C.; Rotemberg, V. Accuracy of commercially available smartphone applications for the detection of melanoma. Br. J. Dermatol. 2021. [Google Scholar] [CrossRef]
  28. Chuchu, N.; Takwoingi, Y.; Dinnes, J.; Matin, R.N.; Bassett, O.; Fmoreau, J.; Bayliss, S.E.; Davenport, C.; Godfrey, K.; O’Connell, S.; et al. Smartphone applications for triaging adults with skin lesions that are suspicious for melanoma. Cochrane Database Syst. Rev. 2018, 12. [Google Scholar] [CrossRef]
  29. Srinivasan, M.A.; Basdogan, C. Haptics in virtual environments: Taxonomy, research status, and challenges. Comput. Graph. 1997, 21, 393–404. [Google Scholar] [CrossRef]
  30. Waldron, K.J.; Enedah, C.; Gladstone, H. Stiffness and texture perception for teledermatology. Stud. Health Technol. Inform. 2005, 111, 579–585. [Google Scholar]
  31. Kim, K.; Lee, S. Perception-based 3D tactile rendering from a single image for human skin examinations by dynamic touch. Ski. Res. Technol. 2015, 21, 164–174. [Google Scholar] [CrossRef]
  32. Kim, K. Roughness based perceptual analysis towards digital skin imaging system with haptic feedback. Ski. Res. Technol. 2016, 22, 334–340. [Google Scholar] [CrossRef]
  33. Parsons, D.; MacCallum, K. Current Perspectives on Augmented Reality in Medical Education: Applications, Affordances and Limitations. Adv. Med. Educ. Pract. 2021, 12, 77–91. [Google Scholar] [CrossRef]
  34. Xu, X.; Mangina, E.; Campbell, A.G. HMD-Based Virtual and Augmented Reality in Medical Education: A Systematic Review. Front. Virtual Real. 2021, 2. [Google Scholar] [CrossRef]
  35. Kassutto, S.M.; Baston, C.; Clancy, C. Virtual, Augmented, and Alternate Reality in Medical Education: Socially Distanced but Fully Immersed. ATS Sch. 2021, 2, 651–664. [Google Scholar] [CrossRef]
Figure 1. A 3D human, full body (middle) and with closeup to chest (left) and back (right). Photos are screenshots from a smartphone application.
Figure 1. A 3D human, full body (middle) and with closeup to chest (left) and back (right). Photos are screenshots from a smartphone application.
Dermato 02 00001 g001
Figure 2. Extracted pictures from the Virtual Derm app [18]. Dermatitis herpetiformis Duhring on the forearm of male adult homunculus (top). Melanoma on left flank of female homunculus (bottom).
Figure 2. Extracted pictures from the Virtual Derm app [18]. Dermatitis herpetiformis Duhring on the forearm of male adult homunculus (top). Melanoma on left flank of female homunculus (bottom).
Dermato 02 00001 g002
Figure 3. Sketch of proposed layout of the augmented reality app for diagnostics of melanoma (Reprinted from [25]).
Figure 3. Sketch of proposed layout of the augmented reality app for diagnostics of melanoma (Reprinted from [25]).
Dermato 02 00001 g003
Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Share and Cite

MDPI and ACS Style

Bonmarin, M.; Läuchli, S.; Navarini, A. Augmented and Virtual Reality in Dermatology—Where Do We Stand and What Comes Next? Dermato 2022, 2, 1-7.

AMA Style

Bonmarin M, Läuchli S, Navarini A. Augmented and Virtual Reality in Dermatology—Where Do We Stand and What Comes Next? Dermato. 2022; 2(1):1-7.

Chicago/Turabian Style

Bonmarin, Mathias, Severin Läuchli, and Alexander Navarini. 2022. "Augmented and Virtual Reality in Dermatology—Where Do We Stand and What Comes Next?" Dermato 2, no. 1: 1-7.

Article Metrics

Back to TopTop