Next Article in Journal
Time-Frequency Characteristics of In-Home Radio Channels Influenced by Activities of the Home Occupant
Previous Article in Journal
A Photoluminescent Colorimetric Probe of Bovine Serum Albumin-Stabilized Gold Nanoclusters for New Psychoactive Substances: Cathinone Drugs in Seized Street Samples
Previous Article in Special Issue
The Fabrication and Characterization of InAlAs/InGaAs APDs Based on a Mesa-Structure with Polyimide Passivation
Open AccessArticle

HoloLens-Based AR System with a Robust Point Set Registration Algorithm

Degree Program of Digital Space and Product Design, Kainan University, Taoyuan 33857, Taiwan
Department of Electrical Engineering, Chang Gung University, Taoyuan 333, Taiwan
Department of Neurosurgery, Chang Gung Memorial Hospital at LinKou, Taoyuan 333, Taiwan
Department of Electrical Engineering, Ming Chi University of Technology, New Taipei City 24301, Taiwan
Author to whom correspondence should be addressed.
Sensors 2019, 19(16), 3555;
Received: 15 June 2019 / Revised: 28 July 2019 / Accepted: 13 August 2019 / Published: 15 August 2019
(This article belongs to the Special Issue Selected Papers from TIKI IEEE ICASI 2019)
PDF [2796 KB, uploaded 15 August 2019]


By the standard of today’s image-guided surgery (IGS) technology, in order to check and verify the progress of the surgery, the surgeons still require divert their attention from the patients occasionally to check against the display. In this paper, a mixed-reality system for medical use is proposed that combines an Intel RealSense sensor with Microsoft’s Hololens head-mounted display system, for superimposing medical data onto the physical surface of a patient, so the surgeons do not need to divert their attention from their patients. The main idea of our proposed system is to display the 3D medical images of the patients on the actual patients themselves by placing the medical images and the patients in the same coordinate space. However, the virtual medical data may contain noises and outliers, so the transformation mapping function must be able to handle these problems. The transform function in our system is performed by the use of our proposed Denoised-Resampled-Weighted-and-Perturbed-Iterative Closest Points (DRWP-ICP) algorithm, which performs denoising and removal of outliers before aligning the pre-operative medical image data points to the patient’s physical surface position before displaying the result using the Microsoft HoloLens display system. The experimental results shows that our proposed mixed-reality system using DRWP-ICP is capable of performing accurate and robust mapping despite the presence of noise and outliers. View Full-Text
Keywords: stereo vision; ICP; Helmet Mounted Display; DRWP-ICP stereo vision; ICP; Helmet Mounted Display; DRWP-ICP

Figure 1

This is an open access article distributed under the Creative Commons Attribution License which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited (CC BY 4.0).

Share & Cite This Article

MDPI and ACS Style

Chien, J.-C.; Tsai, Y.-R.; Wu, C.-T.; Lee, J.-D. HoloLens-Based AR System with a Robust Point Set Registration Algorithm. Sensors 2019, 19, 3555.

Show more citation formats Show less citations formats

Note that from the first issue of 2016, MDPI journals use article numbers instead of page numbers. See further details here.

Related Articles

Article Metrics

Article Access Statistics



[Return to top]
Sensors EISSN 1424-8220 Published by MDPI AG, Basel, Switzerland RSS E-Mail Table of Contents Alert
Back to Top