An Augmented Reality System Using Improved-Iterative Closest Point Algorithm for On-Patient Medical Image Visualization

In many surgery assistance systems, cumbersome equipment or complicated algorithms are often introduced to build the whole system. To build a system without cumbersome equipment or complicated algorithms, and to provide physicians the ability to observe the location of the lesion in the course of surgery, an augmented reality approach using an improved alignment method to image-guided surgery (IGS) is proposed. The system uses RGB-Depth sensor in conjunction with the Point Cloud Library (PCL) to build and establish the patient’s head surface information, and, through the use of the improved alignment algorithm proposed in this study, the preoperative medical imaging information obtained can be placed in the same world-coordinates system as the patient’s head surface information. The traditional alignment method, Iterative Closest Point (ICP), has the disadvantage that an ill-chosen starting position will result only in a locally optimal solution. The proposed improved para-alignment algorithm, named improved-ICP (I-ICP), uses a stochastic perturbation technique to escape from locally optimal solutions and reach the globally optimal solution. After the alignment, the results will be merged and displayed using Microsoft’s HoloLens Head-Mounted Display (HMD), and allows the surgeon to view the patient’s head at the same time as the patient’s medical images. In this study, experiments were performed using spatial reference points with known positions. The experimental results show that the proposed improved alignment algorithm has errors bounded within 3 mm, which is highly accurate.


Introduction
With the rapid development of computer and image processing technologies, Augmented Reality (AR) has become more widely used in many different areas, such as education [1], entertainment [2], medicine [3], etc., and it also adds the feelings of reality for the user compared to Virtual Reality (VR) [4]. In AR applications, images of the real world are captured with a camera and virtual objects are drawn on top of them according in the user's designated locations. This can be useful in medical applications such as Image-Guided Surgery (IGS). Nowadays, the use of IGS system [5] plays a very significant role in the healthcare industry. The motivation for introducing IGS is to reduce invasiveness and to implement non-contact alignment. IGS will improve surgical safety and help to avoid contact with the patient with the physical alignment instrument, which would pose a risk of infection. In 2015,

Modeling from Medical Images
In our experiment, a dummy model was used to simulate the head of a patient. First, a CT scan of the dummy model was performed and the resulting stack of images were processed using image processing methods [20] in order to capture the regions of interest (ROI) and a threshold was set on the image stack to obtain the 3D modeling data, as shown in Figure 1.

Modeling from Medical Images
In our experiment, a dummy model was used to simulate the head of a patient. First, a CT scan of the dummy model was performed and the resulting stack of images were processed using image processing methods [20] in order to capture the regions of interest (ROI) and a threshold was set on the image stack to obtain the 3D modeling data, as shown in Figure 1.

Surface Data Alignment
Currently, the most widely used surface alignment algorithm for three-dimensional objects is the Point Cloud Library-Iteration Closest Point (PCL-ICP) algorithm [21]. This algorithm aligns two sets of point cloud data and its processes are divided into four main steps: (1) sample from the original point cloud data; (2) determine the initial correspondence point data; (3) remove the erroneous corresponding points; and (4) derive the coordinate transformation. First, it assumes two corresponding point sets P and Q, or reference and floating, are determined, and the number of corresponding points is N. The optimal coordinate transformation between P and Q is iteratively calculated by the least square method to obtain a rotation matrix R and a displacement matrix T, set in the same coordinate system. The main method aligns the floating point group to the reference point group by means of iteration.
In this study, a RGB-D sensor [22] and Point Cloud Library (PCL) [23] are used to reconstruct 3D information of the surface of real objects. The traditional ICP alignment algorithm is not considered for our system. It is because there are two distinct disadvantages in the traditional ICP algorithm: The first disadvantage is that an arbitrarily different starting position may cause the algorithm to ends up in a locally optimal solution instead of the globally optimal solution. The second disadvantage is that grossly different points or noise in the non-overlapped area may affect the result of the alignment. Thus, to solve these two problems, a perturbation mechanism is added in each iteration to facilitate escaping from locally optimum solutions. For further improvements, the improved alignment algorithm, i.e., I-ICP, adds the concept of weights into the strategy: According to the distance between the two points of the correspondence in each group, different weight values are assigned to each point to reduce the error caused by the matching wrong points. This weighting scheme is so designed because, when a point in the floating point group searches for its corresponding point in the reference point group, it may be possible that multiple floating points find correspondences in the same reference point. Therefore, the weighting values are assigned to the floating point group to prevent the accumulation of multiple alignment errors. The median value is used in order to avoid choosing the extreme values thus avoiding misalignments.
The procedure of the I-ICP alignment algorithm can be divided into four parts: data input, ICP registration, the stop condition and the perturbation mechanism. The flowchart of the algorithm is shown in Figure 2.

Surface Data Alignment
Currently, the most widely used surface alignment algorithm for three-dimensional objects is the Point Cloud Library-Iteration Closest Point (PCL-ICP) algorithm [21]. This algorithm aligns two sets of point cloud data and its processes are divided into four main steps: (1) sample from the original point cloud data; (2) determine the initial correspondence point data; (3) remove the erroneous corresponding points; and (4) derive the coordinate transformation. First, it assumes two corresponding point sets P and Q, or reference and floating, are determined, and the number of corresponding points is N. The optimal coordinate transformation between P and Q is iteratively calculated by the least square method to obtain a rotation matrix R and a displacement matrix T, set in the same coordinate system. The main method aligns the floating point group to the reference point group by means of iteration.
In this study, a RGB-D sensor [22] and Point Cloud Library (PCL) [23] are used to reconstruct 3D information of the surface of real objects. The traditional ICP alignment algorithm is not considered for our system. It is because there are two distinct disadvantages in the traditional ICP algorithm: The first disadvantage is that an arbitrarily different starting position may cause the algorithm to ends up in a locally optimal solution instead of the globally optimal solution. The second disadvantage is that grossly different points or noise in the non-overlapped area may affect the result of the alignment. Thus, to solve these two problems, a perturbation mechanism is added in each iteration to facilitate escaping from locally optimum solutions. For further improvements, the improved alignment algorithm, i.e., I-ICP, adds the concept of weights into the strategy: According to the distance between the two points of the correspondence in each group, different weight values are assigned to each point to reduce the error caused by the matching wrong points. This weighting scheme is so designed because, when a point in the floating point group searches for its corresponding point in the reference point group, it may be possible that multiple floating points find correspondences in the same reference point. Therefore, the weighting values are assigned to the floating point group to prevent the accumulation of multiple alignment errors. The median value is used in order to avoid choosing the extreme values thus avoiding misalignments.
The procedure of the I-ICP alignment algorithm can be divided into four parts: data input, ICP registration, the stop condition and the perturbation mechanism. The flowchart of the algorithm is shown in Figure 2. The I-ICP Procedure

•
Step 1: Data Input Assuming that the surface of the preoperative CT image is taken as a set of reference points R, then = ( , , ), 1 , and the facial feature points captured by the RGB-D senor and calculated with the PCL library and the KAZE [24] algorithm are to be taken as a set of floating points F, then = ( , , ), 1 .

•
Step 2: ICP Alignment For any given point f in the set floating data set F, find its nearest point in the reference data set R, by calculating the minimum distance between f and each , as shown in Equation (1): where d is the minimum distance calculated by the corresponding points, and median is the middle number in each group after the corresponding points are sorted from the smallest to the largest as in Equation (2): Then, according to the distance calculated by each corresponding point, different weight values are given in Equation (3): where = , , , … . < < … . etc , and NF is the total number of points in the set F.
After the assignment of the weight values of the corresponding points in each group is completed, an objective function, RMS, is calculated as in Equation (4): The I-ICP Procedure

• Step 1: Data Input
Assuming that the surface of the preoperative CT image is taken as a set of reference points R, then R = r j (x, y, z), 1 ≤ j ≤ N R , and the facial feature points captured by the RGB-D senor and calculated with the PCL library and the KAZE [24] algorithm are to be taken as a set of floating points F, then F = { f i (x, y, z), 1 ≤ i ≤ N f }.

• Step 2: ICP Alignment
For any given point f in the set floating data set F, find its nearest point r j in the reference data set R, by calculating the minimum distance between f and each r j , as shown in Equation (1): where d is the minimum distance calculated by the corresponding points, and median is the middle number in each group after the corresponding points are sorted from the smallest to the largest as in Equation (2): Then, according to the distance calculated by each corresponding point, different weight values are given in Equation (3): . . etc. , and N F is the total number of points in the set F. After the assignment of the weight values of the corresponding points in each group is completed, an objective function, RMS, is calculated as in Equation (4): A set of transformation matrix T is obtained by the ICP calculation, and the last calculated RMS is used as the evaluation value [25]. If this value is less than the evaluation value calculated in the previous recursion, then T is used to replace the currently optimal transformation matrix T, and the disturbance mechanism stage, Step 4, is entered. Otherwise, it should be determined whether the stop condition has been reached in Step 3.

• Step 3: Stop Condition
If the RMS value is less than the pre-set threshold or the number of recursions, k, is greater than the pre-set maximum number of recursions, then the execution stops and ends.

• Step 4: Perturbation Mechanism
The ICP method uses the value calculated from the objective function as the evaluation value in each recursion to obtain the minimum evaluation value. Therefore, in each iteration, the calculated evaluation value would be smaller than that estimated in the previous recursion. This type of evaluation method is similar to the gradient descent method shown in Figure 3.
A set of transformation matrix T′ is obtained by the ICP calculation, and the last calculated RMS is used as the evaluation value [25]. If this value is less than the evaluation value calculated in the previous recursion, then T′ is used to replace the currently optimal transformation matrix T, and the disturbance mechanism stage, Step 4, is entered. Otherwise, it should be determined whether the stop condition has been reached in Step 3.

•
Step 3: Stop Condition If the RMS value is less than the pre-set threshold or the number of recursions, k, is greater than the pre-set maximum number of recursions, then the execution stops and ends.

•
Step 4: Perturbation Mechanism The ICP method uses the value calculated from the objective function as the evaluation value in each recursion to obtain the minimum evaluation value. Therefore, in each iteration, the calculated evaluation value would be smaller than that estimated in the previous recursion. This type of evaluation method is similar to the gradient descent method shown in Figure 3. To speed up the search, the golden section search [26] is used: As illustrated in Figure 4, if the function's interval is between − and , then the golden ratio ϕ can be chosen, as shown in Equation (5), and is then set according to the ratio, as shown in Equation (6).  To speed up the search, the golden section search [26] is used: As illustrated in Figure 4, if the function's interval is between −αγ and αγ, then the golden ratio ϕ can be chosen, as shown in Equation (5), and αγ is then set according to the ratio, as shown in Equation (6).
Sensors 2018, 18, x FOR PEER REVIEW 5 of 15 A set of transformation matrix T′ is obtained by the ICP calculation, and the last calculated RMS is used as the evaluation value [25]. If this value is less than the evaluation value calculated in the previous recursion, then T′ is used to replace the currently optimal transformation matrix T, and the disturbance mechanism stage, Step 4, is entered. Otherwise, it should be determined whether the stop condition has been reached in Step 3.

•
Step 3: Stop Condition If the RMS value is less than the pre-set threshold or the number of recursions, k, is greater than the pre-set maximum number of recursions, then the execution stops and ends.

•
Step 4: Perturbation Mechanism The ICP method uses the value calculated from the objective function as the evaluation value in each recursion to obtain the minimum evaluation value. Therefore, in each iteration, the calculated evaluation value would be smaller than that estimated in the previous recursion. This type of evaluation method is similar to the gradient descent method shown in Figure 3. To speed up the search, the golden section search [26] is used: As illustrated in Figure 4, if the function's interval is between − and , then the golden ratio ϕ can be chosen, as shown in Equation (5), and is then set according to the ratio, as shown in Equation (6).  If RMS T temp > RMS(T init ), then a local minimum falls between n = (T temp + αγ) and T init . On the other hand, if RMS T temp < RMS(T init ), then a local minimum falls between T init and m = (T temp − αγ), and the gray area shown in the figure may be ignored during the search. A minimum value can be found through by searching endlessly, however, the result may not necessarily be the globally best solution, and in this case, we call the solution a local solution. To overcome this problem, we propose to use a perturbation method to make the search jump out of the local solution and restart the search for the best solution.
If we let then use the value to calculate the differences in rotation in the X-axis, Y-axis, and Z-axis. Given a parabolic probability function, it can be used to assign probability to a variable y, representing its probability of being selected, as given in Equation (8): If the defined solution space is not sufficiently large, it may happen that search cannot escape from the local solution. Therefore, it is necessary to expand the scope of the solution space using a scale factor α, where α is a coefficient to expand the perturbation range until the stop condition is satisfied.

Detection of the Marker Board
To prepare pre-operative spatial positioning, a preset marker plate is placed next to the dummy head model during surgery to establish an unchanging relationship between the two. When the camera detects the marker plate, the virtual image is superimposed on the dummy head model. The mixed reality system uses the Vuforia's image tracking SDK [27], which can identify and track feature points on a 2D planar images as well as 3D images. Compared to AR Toolkit [28], Vuforia SDK can track the feature points and identify the image even if the image is partially occluded. This study uses a self-made QR code marker, shown in Figure 5, as features to be tracked. and the gray area shown in the figure may be ignored during the search. A minimum value can be found through by searching endlessly, however, the result may not necessarily be the globally best solution, and in this case, we call the solution a local solution. To overcome this problem, we propose to use a perturbation method to make the search jump out of the local solution and restart the search for the best solution.
If we let then use the value to calculate the differences in rotation in the X-axis, Y-axis, and Z-axis. Given a parabolic probability function, it can be used to assign probability to a variable y, representing its probability of being selected, as given in Equation (8): If the defined solution space is not sufficiently large, it may happen that search cannot escape from the local solution. Therefore, it is necessary to expand the scope of the solution space using a scale factor α, where α is a coefficient to expand the perturbation range until the stop condition is satisfied.

Detection of the Marker Board
To prepare pre-operative spatial positioning, a preset marker plate is placed next to the dummy head model during surgery to establish an unchanging relationship between the two. When the camera detects the marker plate, the virtual image is superimposed on the dummy head model. The mixed reality system uses the Vuforia's image tracking SDK [27], which can identify and track feature points on a 2D planar images as well as 3D images. Compared to AR Toolkit [28], Vuforia SDK can track the feature points and identify the image even if the image is partially occluded. This study uses a self-made QR code marker, shown in Figure 5, as features to be tracked. Because the HoloLens device has its own coordinate system [29], to observe pre-operative medical image overlays on the target, the coordinate systems need to be integrated first. First, the I-ICP transforms the coordinate system, , of the medical image data to the RGB-D sensor coordinate system , using . Then when the QR-Code marker is placed in the field of the RGB-D sensor, and the conversion transformation, , to the world coordinate system can then be calculated. Finally, the conversion transform, , of the HoloLens' coordinate system, , to the QR-Code marker's coordinate system, , completes the conversion. This process is illustrated in Figure 6. Because the HoloLens device has its own coordinate system [29], to observe pre-operative medical image overlays on the target, the coordinate systems need to be integrated first. First, the I-ICP transforms the coordinate system, C I MG , of the medical image data to the RGB-D sensor coordinate system C DEP , using T DEP I MG . Then when the QR-Code marker is placed in the field of the RGB-D sensor, and the conversion transformation, T DEP MAR , to the world coordinate system can then be calculated. Finally, the conversion transform, T MAR CAM , of the HoloLens' coordinate system, C CAM , to the QR-Code marker's coordinate system, C MAR , completes the conversion. This process is illustrated in Figure 6. The hardware devices used in this study consisted of a desktop computer with Windows 10 OS, a 64-bit Intel(R) Core(TM) i5-4460 CPU, a RGB-D sensor, and Microsoft HoloLens [30]. The HoloLens system uses Windows 10 OS, includes a CPU and a GPU, and has a display resolution of 1280 × 720. In addition, a holographic processing unit (HPU) is also designed to handle spatial mapping using SLAM (Simultaneous Localization and Mapping) [31,32] to process spatial projections [33], and allows the user to access the user interface in the augmented reality environment through voice and gestures. This system was developed with Unity [34].

Experimental Results
In this section, the results of comparisons between the proposed I-ICP algorithm and other ICPbased alignment algorithms using non-medical and medical datasets are presented.

Alignment Tests Using a Non-Medical Dataset
The Stanford Bunny [35], shared by Grek Turk and Marc Levoy in 1994, was used as test data. The number of data points of the Bunny is 40,256, so first the number data points is reduced to 100, then rotated 50 degrees along the X-, Y-, and Z-axis before a translation of 200 mm is performed. The processed data points are then treated as the floating point set, and ICP-based alignment tests are performed with the original data points. The visual results of the alignment tests using proposed I-ICP, PCL-ICP, Go-ICP, CPD-ICP (rigid), CPD-ICP (affine), and CloudCompare-ICP [36] are shown in Figure 7, where the red cloud points are the reference points, the yellow dot are the five marker points in the reference point cloud, the white cloud points are the floating points, and the green dots are the five markers in the floating point cloud. The Root-Mean-Square (RMS) error is used as an evaluation, as shown in Table 1. To achieve the demands of accuracy, five reference points are first obtained for verification of the errors, as shown in Table 2, and the unit of measurement is mm.
The TRE values are presented in Figure 8. In general, the Target Registration Error (TRE) [17] is a measurement standard to evaluate error in aligning image to the physical space. Assume that is the validation point, shown in yellow in Figure 7, and is the floating point after the ICP transform, shown in green in Figure 7. Then, the TRE can be calculated using Equation The hardware devices used in this study consisted of a desktop computer with Windows 10 OS, a 64-bit Intel(R) Core(TM) i5-4460 CPU, a RGB-D sensor, and Microsoft HoloLens [30]. The HoloLens system uses Windows 10 OS, includes a CPU and a GPU, and has a display resolution of 1280 × 720. In addition, a holographic processing unit (HPU) is also designed to handle spatial mapping using SLAM (Simultaneous Localization and Mapping) [31,32] to process spatial projections [33], and allows the user to access the user interface in the augmented reality environment through voice and gestures. This system was developed with Unity [34].

Experimental Results
In this section, the results of comparisons between the proposed I-ICP algorithm and other ICP-based alignment algorithms using non-medical and medical datasets are presented.

Alignment Tests Using a Non-Medical Dataset
The Stanford Bunny [35], shared by Grek Turk and Marc Levoy in 1994, was used as test data. The number of data points of the Bunny is 40,256, so first the number data points is reduced to 100, then rotated 50 degrees along the X-, Y-, and Z-axis before a translation of 200 mm is performed. The processed data points are then treated as the floating point set, and ICP-based alignment tests are performed with the original data points. The visual results of the alignment tests using proposed I-ICP, PCL-ICP, Go-ICP, CPD-ICP (rigid), CPD-ICP (affine), and CloudCompare-ICP [36] are shown in Figure 7, where the red cloud points are the reference points, the yellow dot are the five marker points in the reference point cloud, the white cloud points are the floating points, and the green dots are the five markers in the floating point cloud. The Root-Mean-Square (RMS) error is used as an evaluation, as shown in Table 1. To achieve the demands of accuracy, five reference points are first obtained for verification of the errors, as shown in Table 2, and the unit of measurement is mm.
The TRE values are presented in Figure 8. In general, the Target Registration Error (TRE) [17] is a measurement standard to evaluate error in aligning image to the physical space. Assume that p valid i is the validation point, shown in yellow in Figure 7, and p f i is the floating point after the ICP transform, shown in green in Figure 7. Then, the TRE can be calculated using Equation (9):       Comparing the results in Tables 1 and 2 and Figure 8, we found that the RMS errors of all ICPbased methods used in the comparison are very small, with CC-ICP appearing to be the best performer. However, upon comparing the average error of the reference points, we found that the results of most of the ICP-based methods, except for the proposed I-ICP and Go-ICP, are over 3 mm.

Dummy Head Alignment Test
In this test, a marker plate is placed next to the dummy head while keeping their coordinate relationship unchanged, so when the HoloLens camera detects the QR Code feature point, the medical image is displayed on the screen and then overlays the dummy head in the correct position, as shown in Figure 9.
To facilitate the doctor to observe more detailed information, the design of the user interface in the augmented reality environment allows the user to change the rotation angle and the scale of the virtual object through the use of gestures [37], as shown in Figure 10. Comparing the results in Tables 1 and 2 and Figure 8, we found that the RMS errors of all ICP-based methods used in the comparison are very small, with CC-ICP appearing to be the best performer. However, upon comparing the average error of the reference points, we found that the results of most of the ICP-based methods, except for the proposed I-ICP and Go-ICP, are over 3 mm.

Dummy Head Alignment Test
In this test, a marker plate is placed next to the dummy head while keeping their coordinate relationship unchanged, so when the HoloLens camera detects the QR Code feature point, the medical image is displayed on the screen and then overlays the dummy head in the correct position, as shown in Figure 9.
To facilitate the doctor to observe more detailed information, the design of the user interface in the augmented reality environment allows the user to change the rotation angle and the scale of the virtual object through the use of gestures [37], as shown in Figure 10. Comparing the results in Tables 1 and 2 and Figure 8, we found that the RMS errors of all ICPbased methods used in the comparison are very small, with CC-ICP appearing to be the best performer. However, upon comparing the average error of the reference points, we found that the results of most of the ICP-based methods, except for the proposed I-ICP and Go-ICP, are over 3 mm.

Dummy Head Alignment Test
In this test, a marker plate is placed next to the dummy head while keeping their coordinate relationship unchanged, so when the HoloLens camera detects the QR Code feature point, the medical image is displayed on the screen and then overlays the dummy head in the correct position, as shown in Figure 9.
To facilitate the doctor to observe more detailed information, the design of the user interface in the augmented reality environment allows the user to change the rotation angle and the scale of the virtual object through the use of gestures [37], as shown in Figure 10.  To compare I-ICP's accuracy with the other ICP-based alignment algorithms, five reference points were obtained using the MicroScribe G2X Digitizer coordinate tracking device. They are used as the standard for verification of errors [38], and taken in the digitizer's own coordinate system, . A checkerboard is used in the reference coordinates, , to align the reference points to the world coordinate system using the transform . This process is illustrated in Figure 11. The experimental results are the results of taking the averages of the results of ten experiments in comparing the alignment results of I-ICP with CloudCompare-ICP, PCL-ICP, Go-ICP, CPD-ICP (rigid), and CPD-ICP (affine). The experiments were performed without using a good initial starting point, and using mm as the unit of measure for error. The time used for alignment is also displayed. The results of the comparison for Dummy Head #1 in Figure 10 is displayed in Table 3 and their TREs are shown in Figure 12.  To compare I-ICP's accuracy with the other ICP-based alignment algorithms, five reference points were obtained using the MicroScribe G2X Digitizer coordinate tracking device. They are used as the standard for verification of errors [38], and taken in the digitizer's own coordinate system, C TRA . A checkerboard is used in the reference coordinates, C REF , to align the reference points to the world coordinate system using the transform T REF DEP . This process is illustrated in Figure 11. The experimental results are the results of taking the averages of the results of ten experiments in comparing the alignment results of I-ICP with CloudCompare-ICP, PCL-ICP, Go-ICP, CPD-ICP (rigid), and CPD-ICP (affine). The experiments were performed without using a good initial starting point, and using mm as the unit of measure for error. The time used for alignment is also displayed. The results of the comparison for Dummy Head #1 in Figure 10 is displayed in Table 3 and their TREs are shown in Figure 12. To compare I-ICP's accuracy with the other ICP-based alignment algorithms, five reference points were obtained using the MicroScribe G2X Digitizer coordinate tracking device. They are used as the standard for verification of errors [38], and taken in the digitizer's own coordinate system, . A checkerboard is used in the reference coordinates, , to align the reference points to the world coordinate system using the transform . This process is illustrated in Figure 11. The experimental results are the results of taking the averages of the results of ten experiments in comparing the alignment results of I-ICP with CloudCompare-ICP, PCL-ICP, Go-ICP, CPD-ICP (rigid), and CPD-ICP (affine). The experiments were performed without using a good initial starting point, and using mm as the unit of measure for error. The time used for alignment is also displayed. The results of the comparison for Dummy Head #1 in Figure 10 is displayed in Table 3 and their TREs are shown in Figure 12.     The visual results of the I-ICP alignment are shown in Figure 13, where the red point cloud is the set of floating points, and the blue point cloud is the set of reference points.  The visual results of the I-ICP alignment are shown in Figure 13, where the red point cloud is the set of floating points, and the blue point cloud is the set of reference points.   The visual results of the I-ICP alignment are shown in Figure 13, where the red point cloud is the set of floating points, and the blue point cloud is the set of reference points.  The second experiment included performing an initial coarse alignment, making the floating point cloud closer to the reference point cloud. A greedy rough alignment method uses the rotation-invariant property, so it is possible to obtain only a locally optimum solution. Therefore, the Sampling Consensus Initial Alignment (SAC-IA) [39] method was used to do the coarse alignment. It first performs a Fast Point Feature Histograms (PFFH) to obtain the sampling features, then searches for their corresponding points in the reference point cloud, followed by calculating the transformation matrix between these corresponding points. The time takes for rough alignment is about 370 s for the given dataset. From the results in Table 4, it can be seen that the alignment errors for I-ICP, PCL-ICP and CPD-ICP (affine) methods are all about 3 mm. In addition, the results show that most of ICP-based methods can be improved by performing a coarse alignment first.

Discussion
There are several requirements for many of the ICP-based alignment algorithms published in the literature [14][15][16][17][18][19] for them to converge to acceptable solutions. These requirements include that a good starting position must be given, and that the number of points in the two sets of original point clouds to be aligned must be approximately equal. To avoid these requirements, in the I-ICP algorithm, we added the golden section search, perturbations and weighting mechanisms. These mechanisms are added to increase the speed for searching for the smallest value, avoid falling into locally optimal solution during search as well as to avoid the adverse influence of outlier points, respectively. According to the results in the Stanford Bunny alignment experiment, it is found though the RMS values all fell within 1.0, but the reference points' errors show that there are gaps between the PCL-ICP [21] and the CPD-ICP [18] alignment results and the actual reference points' positions. This observation shows using the RMS errors alone as the measure of comparison is not sufficient to describe the accuracies of different alignment methods. This is because the RMS errors only describe the overall correspondence accuracy, so a small RMS error value would not guarantee that individual target alignment error would also be small. Therefore, TRE (Target Registration Error) is also used in addition to the RMS error as the final evaluation standard.
In other words, if the RMS errors of two alignment methods are similarly small, then the TRE value can be referenced to differentiate their performances. We found that, although the CC-ICP [36] was faster than the I-ICP method proposed in this study, from the average TRE measurements, it is obvious that the I-ICP has better registration performance. In the experiment using the dummy head, shown in Tables 3 and 4, we used the TRE measurements [40] to validate alignment accuracy, and five markers on the dummy's head were used to evaluate alignment errors. The results, shown in Table 3, were the performance comparisons of using multiple ICP-based alignment methods, where a good starting position was deliberately withheld. It can be seen from the results that the GO-ICP algorithm failed to converge. This is in spite of the fact that the designers of the GO-ICP algorithm claimed that it does not need a good starting position to converge to the global solution. Our experiment showed that this claim is untrue when the two sets of cloud point data are far apart: its maximum error on the y-axis at the first marker point was 126.17 mm. In the same experiment, the CPD-ICP (rigid) algorithm had the greatest amount of error amongst all methods compared; its errors on the y-axis of marker points 1, 2, and 3 were as high as 176.52 mm. On the other hand, the proposed I-ICP can converge to the global optimal solution, with or without a good starting position. If two sets of cloud points are different in size, it would be necessary to consider a scaling factor while aligning. Because the CPD-ICP (affine) algorithm does not consider the scaling factor, its alignment result may cause the phenomenon of point cloud deformation. In that same experiment, the results showed that its registration errors on the y-axis of marker points 1, 2, and 3 were also very large, while the I-ICP algorithm proposed by this study had the best performance overall; its errors fell mostly within 3 mm, while spent only 7 s in computation time.
This study investigated the problem of surface reconstruction by combining a RGB-D sensor and the PCL and KAZE algorithms to calculate feature points. However, the current setup may not find sufficient feature information in bright light environment and overly smooth surfaces.

Conclusions
This study proposed an augmented reality system for on-patient medical display. The patient's CT images are taken before the surgery and then accurately positioned on the patient's face through the use of the Improved-Iterative Closest Point alignment algorithm proposed in this study. With this system, surgeons no longer need to look up and watch the LCD as they used to do. This system can assist in operations by superimposing virtual images on real images and display the results on the head-mounted display screen. The proposed system eliminates the need for physical contact with the patient, which is one of its design goals, and thus adds many conveniences. The experimental results show that the proposed method has better efficiency and accuracy. Its average TRE measurement is less than 3 mm, and within the range acceptable by doctors, which is the consensus opinion of consulted professional neural surgeons.
In the future, our methods can be further improved by using the structure light equipment to acquire the surface information when sufficient feature points on smooth surface cannot be extracted under normal lighting. In addition, the current setup still uses marker-based localization to complete space coordinate integration; it is expected that this will not be necessary and that Hololens itself will provide depth information to its users in the future, thus reducing the need for excessive coordinate conversions.