Stylus Tip Center Position Self-Calibration Based on Invariable Distances in Light-Pen Systems

The light-pen coordinate measuring machine (LPCMM for short) is portable and flexible to measure features including invisible ones in-situ. Since different styluses are needed to measure different features and even during the process of measuring a single workpiece with complicated configurations, to improve the system measurement accuracy it is beneficial to calibrate the stylus tip center position after it is mounted to the light-pen before measurement in an industrial field. A novel and simple method aiming at self-calibrating the position of the tip center based on invariable distances is presented. The distinguishing feature of the proposed method is that the center position of the tip can be calibrated by using a kinematic seat with an inverted cone hole without any external reference and auxiliary devices. Calibration is based on that the distance between the tip center and that of any LED is invariable when the light-pen is swung smoothly with its spherical tip firmly touching the fixed cone seat. To ensure the repeatability of the algorithm some error constraint parameters are given. Based on invariable distances, the tip center position in the light-pen coordinate system can be obtained. Experiment results show that the self-calibration method has the advantage of good repeatability, with standard deviations 0.027, 0.023 and 0.014 mm in U, V and W directions, respectively. Experimental results of measuring a circle and a gauge block indirectly demonstrate the accuracy of the proposed self-calibration method.


Introduction
As a kind of mature and high-precision measuring instrument, coordinate measuring machines (CMMs for short) have gained wide acceptance in many fields, such as mechanical manufacturing, aviation, reverse engineering, etc. Different types of physical quantities, such as length, diameter, roundness and many others, can be obtained with CMMs by measuring points on the surface of workpieces. However, the most widely used Cartesian CMMs are not portable and cannot meet the requirements of in-situ measurement because of their large size and heavy weight [1,2]. During the past few decades, to meet the need of in-situ measurement in the field and especially in large-scale metrology [3], there has been a focus on portable and flexible measuring techniques, even at the cost of sacrificing accuracy to a certain degree. Many types of novel portable coordinate measuring machines (PCMMs for short) such as articulated arm CMM, laser tracker, indoor GPS, mobile spatial CMM, photogrammetric system, have seen broad and rapid development and brought great convenience to in-situ and large-scale measurement tasks [4][5][6][7][8].
The light-pen coordinate measuring machine (LPCMM for short) uses a light-pen as its detecting element. It mainly consists of a light-pen, one or two industrial cameras, a laptop computer and associated software. Some other auxiliary devices may be attached to enhance its function. The light-pen is a kind of handheld detection device with several mounted LEDs providing control points and a replaceable stylus. During the measurement, the stylus tip of the light-pen touches the workpiece surface firmly and the position of the measured point is determined by the images of control points, which are called feature points, and their coordinates are calculated through image processing algorithms. Since not the stylus tip but control points are imaged, even hidden features, such as small and deep holes, invisible points or corners inside the measured piece, which cannot be sensed by optical PCMMs, can be measured by properly selecting the stylus [9][10][11][12].
Since only the positions of the control points are sensed by the camera, it is essential to know the relative position of the measured point with reference to the positions of control points. For this both the diameter of the spherical tip and the center position of the tip in the light-pen coordinate system, as well as the camera system, should be accurately calibrated. During the past several decades there have been a large number of studies on the topic of camera calibration, such as Tsai's two-stage approach [13], Zhang's method using 1D, 2D or 3D reference objects [14,15], the bundle adjustment method [16], the direct linear transformation method [17] and many others [18,19]. By contrast, very few publications about the calibration of the light-pen could be found. The tip diameter is fixed for a stylus and will not be changed with remounting so it can be calibrated in advance by using a reference ball or a gauge block in the lab and recalled while knowing which stylus is mounted on the light-pen. Since it is a common practice [1], we will simply discuss it and will concentrate our discussion on the tip center position calibration.
System calibration, including the camera calibration, center position calibration of control points and the tip, should be completed before measurement. The center position calibration of control points can be completed in a temperature controlled laboratory environment with high-precision equipment, such as CMMs, since for a well-designed light-pen their positions will not be changed during transportation, mounting and use. However, the center position of the stylus tip center cannot be the same when different kinds of stylus are mounted to the light-pen. In some cases, several different styluses may be applied to complete a single in-situ measurement. After a stylus is replaced by another one, the tip center position is changed and an in-situ calibration is required before further measurement can be performed. Furthermore, to protect the styluses from damage, they should be placed in a special box when not in use, and mounted to the light-pen before usage, so the tip center position needs to be calibrated in-situ rather than pre-calibrated in a lab [20]. The approach introduced in [21] needs to take two groups of images and consists of three calibration steps. More-over, the standard deviations of its repeatability test reach (0.178, 0.188, 0.221) (mm) in x, y and z directions, respectively and cannot meet the requirement of accurate measurement. The methodology introduced in [22] has an advantage of the relatively high repeatability with standard deviations (0.033, 0.030, 0.043) (mm) in x, y and z directions, respectively. However, its algorithm has the big problem that it does not converge sometimes.
To calibrate the tip center position in-situ, the method should be as simple as possible. It is preferred not to rely on a high accurate reference. A novel method for self-calibration of the tip center position with high accuracy based on invariable distances is presented in the paper. In Section 2, the system structure and model of the LPCMM is presented. The self-calibration model and calibration steps of the tip center position are discussed in Section 3. Repeatability tests and several measurement experiments are presented in turn in Section 4. Some conclusions are given in Section 5. Figure 1 shows the whole system structure of the LPCMM. This system is mainly composed of a laptop computer, a camera and a light-pen. Because more information can be obtained from images of the light-pen in the dual-camera LPCMM, theoretically, it has a higher measurement accuracy than the single-camera one. However, the system will be more complicated and the calibration for the relative position between two cameras as well as the limited measured area make the dual-camera LPCMM not so flexible in an industrial field. Only the single-camera system will be discussed in this paper.

System Structure of the LPCMM
As a type of handheld detection device, the light-pen is presented in Figure 1. The body of the light-pen, made of carbon fiber, is rigid, of light weight and almost free from temperature influences. To meet different measurement requirements, such as measuring different shapes and sizes of deep holes, the stylus of the light-pen should be replaceable. Several types of styluses are shown in Figure 2. In order to capture stable images with suitable brightness of feature points, high power infrared LEDs are chosen as control points. They can emit the light with a wavelength of about 870 nm. As a suitable light filter is mounted to the camera lens, the stray background light will have little impact on the measurement. To further improve the measurement repeatability and accuracy, not only each control point could be switched on or off separately, but also the lightening time spans and brightness of the LEDs can be dynamically controlled in real time, according to environmental changes [23]. That means with this dynamic adjustment algorithm, the repeatability of feature point center locations will not be influenced by the distance between the camera and the light-pen to certain extent. Thirteen LEDs are arranged as shown in Figure 1: Nine of them are distributed uniformly in the front plane (plane F), and other four are arrayed evenly in a line in the back plane (plane B), which is parallel to the front plane at a distance of 100 mm. LPCMM not so flexible in an industrial field. Only the single-camera system will be discussed in this paper.
As a type of handheld detection device, the light-pen is presented in Figure 1. The body of the light-pen, made of carbon fiber, is rigid, of light weight and almost free from temperature influences. To meet different measurement requirements, such as measuring different shapes and sizes of deep holes, the stylus of the light-pen should be replaceable. Several types of styluses are shown in Figure 2. In order to capture stable images with suitable brightness of feature points, high power infrared LEDs are chosen as control points. They can emit the light with a wavelength of about 870 nm. As a suitable light filter is mounted to the camera lens, the stray background light will have little impact on the measurement. To further improve the measurement repeatability and accuracy, not only each control point could be switched on or off separately, but also the lightening time spans and brightness of the LEDs can be dynamically controlled in real time, according to environmental changes [23]. That means with this dynamic adjustment algorithm, the repeatability of feature point center locations will not be influenced by the distance between the camera and the light-pen to certain extent. Thirteen LEDs are arranged as shown in Figure 1: Nine of them are distributed uniformly in the front plane (plane F), and other four are arrayed evenly in a line in the back plane (plane B), which is parallel to the front plane at a distance of 100 mm.   LPCMM not so flexible in an industrial field. Only the single-camera system will be discussed in this paper.
As a type of handheld detection device, the light-pen is presented in Figure 1. The body of the light-pen, made of carbon fiber, is rigid, of light weight and almost free from temperature influences. To meet different measurement requirements, such as measuring different shapes and sizes of deep holes, the stylus of the light-pen should be replaceable. Several types of styluses are shown in Figure 2. In order to capture stable images with suitable brightness of feature points, high power infrared LEDs are chosen as control points. They can emit the light with a wavelength of about 870 nm. As a suitable light filter is mounted to the camera lens, the stray background light will have little impact on the measurement. To further improve the measurement repeatability and accuracy, not only each control point could be switched on or off separately, but also the lightening time spans and brightness of the LEDs can be dynamically controlled in real time, according to environmental changes [23]. That means with this dynamic adjustment algorithm, the repeatability of feature point center locations will not be influenced by the distance between the camera and the light-pen to certain extent. Thirteen LEDs are arranged as shown in Figure 1: Nine of them are distributed uniformly in the front plane (plane F), and other four are arrayed evenly in a line in the back plane (plane B), which is parallel to the front plane at a distance of 100 mm.

Coordinate System Establishment of the LPCMM
There are four coordinate systems in the LPCMM, as shown in Figure 3: (a) The light-pen coordinate system O L -UVW. The origin O L is set at the center of the LED 1 (marked in Figure 1). The axis U is parallel to the line connecting LEDs 8 and 11, and its positive direction is towards LED 11. The axis V, perpendicular to U, is parallel to the line linking LEDs 1 and 4, and its positive direction is towards LED 4. The axis W is set up according to the right-hand rule. (b) The pixel coordinate system O 1 -x 1 y 1 . The origin is placed at the up-right corner of the image plane.
x 1 and y 1 are parallel to the horizontal and vertical pixel arrays, respectively. The orientations of

Coordinate System Establishment of the LPCMM
There are four coordinate systems in the LPCMM, as shown in Figure 3: (a) The light-pen coordinate system OL-UVW. The origin OL is set at the center of the LED 1 (marked in Figure 1). The axis U is parallel to the line connecting LEDs 8 and 11, and its positive direction is towards LED 11. The axis V, perpendicular to U, is parallel to the line linking LEDs 1 and 4, and its positive direction is towards LED 4. The axis W is set up according to the right-hand rule.

System Model of the LPCMM
During the measurement, the stylus tip of the light-pen should touch the workpiece surface steadily and keep the light-pen's body vertically with LEDs facing the camera. Photos of the lightpen are taken by the camera and make sure that all 13 LEDs are well imaged to feature points. With algorithm processing, these coordinates of the center of feature points in O1-x1y1, (x1i, y1i) (i = 1~13), are obtained: In Equation (1), (x2i, y2i) are corresponding coordinates of (x1i, y1i) in O2-x2y2. (δx, δy, dx, dy, cx, cy) are the intrinsic parameters of the camera, and can be obtained by the sub-regional camera calibration method given in [17]. (δx, δy) are distortion values of the camera lens in x1 and y1 directions respectively. (dx, dy) are the sizes of the pixel in in x1 and y1 directions separately, while (cx, cy) are the position of the point O2 in O1-x1y1:

System Model of the LPCMM
During the measurement, the stylus tip of the light-pen should touch the workpiece surface steadily and keep the light-pen's body vertically with LEDs facing the camera. Photos of the light-pen are taken by the camera and make sure that all 13 LEDs are well imaged to feature points. With algorithm processing, these coordinates of the center of feature points in O 1 -x 1 y 1 , (x 1i , y 1i ) (i = 1~13), are obtained: In Equation (1), (x 2i , y 2i ) are corresponding coordinates of (x 1i , y 1i ) in O 2 -x 2 y 2 . (δ x , δ y , d x , d y , c x , c y ) are the intrinsic parameters of the camera, and can be obtained by the sub-regional camera calibration method given in [17].
where f is the focus length of the camera. (x i , y i , z i ) are coordinates of feature point centers in O C -XYZ. From Equations (1) and (2), (x i , y i , z i ) can be calculated: where (x i ', y i ', z i ') are coordinates of the LED centers in O C -XYZ.
Since the coordinates of the LED centers in O L -UVW, (u i , v i , w i ) have been pre-calibrated by a high-accurate CMM in the lab, the rotation matrix R given in Equation (5) or (6) and the translation vector T (t x , t y , t z ) could be computed from Equations (3), (4) and (6) by using the nonlinear least square generalized inverse method given in [24]: where r i (i = 1~9) and (t x , t y , t z ) are the parameters of the rotation and translation matrices between the O L -UVW and O C -XYZ, respectively. r i in Equation (4) are functions of rotated angles α, β and γ as shown in Equation (5): cos β cos γ sin α sin β cos γ − cos α sin γ cos α sin β cos γ + sin α sin γ cos β sin γ sin α sin β sin γ + cos α cos γ cos α sin β sin γ − sin α cos γ − sin β sin α cos β cos α cos β    where α, β and γ are the rotation angles around U, V and W axes, respectively. The rotations are in the order of α, β and γ. When the light-pen is almost vertical, (α, β, γ) equal to (π, 0, 0) approximately, R could be simplified as shown in Equation (6): When coordinates of the tip center in system O L -UVW, (u 0 , v 0 , w 0 ), are known, corresponding coordinates in system O C -XYZ, (x 0 , y 0 , z 0 ), can be calculated from Equation (7): The coordinates of the measured point in O C -XYZ can be obtained with tip radius compensation. As shown in Figure 4, the enveloping surface method is always used to compensate the tip radius.  The coordinates of the measured point in OC-XYZ can be obtained with tip radius compensation. As shown in Figure 4, the enveloping surface method is always used to compensate the tip radius. The fitted curved surface could be obtained with center positions of the stylus tip. Then the measured point's position in OC-XYZ can be obtained by the corresponding stylus tip center position in OC-XYZ, its normal direction in OC-XYZ and the tip radius. The tip center position varies when the stylus is changed and mounted, even when the same stylus is remounted. Self-calibration of (u0, v0, w0) in-situ is essential.

Establishment of the Self-Calibration Model
As shown in Figure 5, the stylus tip of the light-pen should be kept closely and firmly in touch with the fixed kinematic seat with an inverted cone hole during self-calibration. The camera is fixed at a suitable position so all LEDs always appear in the camera's field of view. Then we swing the light-pen smoothly and slowly with all LEDs facing the camera.  The tip center position varies when the stylus is changed and mounted, even when the same stylus is remounted. Self-calibration of (u 0 , v 0 , w 0 ) in-situ is essential.

Establishment of the Self-Calibration Model
As shown in Figure 5, the stylus tip of the light-pen should be kept closely and firmly in touch with the fixed kinematic seat with an inverted cone hole during self-calibration. The camera is fixed at a suitable position so all LEDs always appear in the camera's field of view. Then we swing the light-pen smoothly and slowly with all LEDs facing the camera.
The coordinates of the measured point in OC-XYZ can be obtained with tip radius compensation. As shown in Figure 4, the enveloping surface method is always used to compensate the tip radius. The fitted curved surface could be obtained with center positions of the stylus tip. Then the measured point's position in OC-XYZ can be obtained by the corresponding stylus tip center position in OC-XYZ, its normal direction in OC-XYZ and the tip radius. The tip center position varies when the stylus is changed and mounted, even when the same stylus is remounted. Self-calibration of (u0, v0, w0) in-situ is essential.

Establishment of the Self-Calibration Model
As shown in Figure 5, the stylus tip of the light-pen should be kept closely and firmly in touch with the fixed kinematic seat with an inverted cone hole during self-calibration. The camera is fixed at a suitable position so all LEDs always appear in the camera's field of view. Then we swing the light-pen smoothly and slowly with all LEDs facing the camera.  As known from above, the coordinates of the tip center are (u 0 , v 0 , w 0 ) in O L -UVW which needs to be calibrated. The light-pen can be treated as a rigid body, (u 0 , v 0 , w 0 ) are unchanged when the light-pen swings. Its coordinates, (x 0 , y 0 , z 0 ), in O C -XYZ are constants as well, because the relative position between the kinematic seat and the camera is invariable and the spherical tip touches the cone hole firmly during calibration.
(a) After all parameters of the matrices R and T are determined in accordance with the equations given in Section 2.3. Several images of control points are taken and determined R and T are verified by calculating the reprojection error [25][26][27] as follows: The re-projected feature points (x 2i ', y 2i ') (i = 1~13) are obtained from pre-calibrated (u i , v i , w i ) and the calculated matrices R, T. (x ij ", y ij ", z ij ") are defined from Equation (9) given below. j is the serial number of image. For assuring the required accuracy of calibration it is suggested to take at least seven images. It is known that the center positions of feature points can be determined more accurately when the light-pen is vertical than it is slant. When the pitch angle of the light-pen is small during the calibration, ∆p j will be small. However, the variations of (x ij ", y ij ", z ij ") will be small as well, and it will lead to large errors in solving the equations. Conversely, when the pitch angle of the light-pen is big, ∆p j will be big too. So two threshold values, Q 1 < Q 2 , should be given for ∆p j to make the pitch angle of the light-pen within a suitable range and to obtain eligible parameters of the matrices R, T. (b) In O C -XYZ, the distance between the center of each LED and the tip center d i (i = 1~13), and (x 0 , y 0 , z 0 ) can be determined by solving the following equation using the nonlinear least square generalized inverse method: where (x ij ", y ij ", z ij ") are coordinates of the center of the i-th LED in O C -XYZ system in the j-th image. (c) Because of unavoidable errors in calibration, the distance between the center of the i-th LED and the center of the probe tip cannot be the same as d i calculated from Equation (9). ∆d j of the j-th image is the sum of the absolute values of the differences between these two distances in the j-th image. Two threshold values Q 3 , Q 4 are given for ∆d max , the maximum of all ∆d j , and variation of ∆d max defined in Equation (10): In this way, the stylus tip center position of the light-pen in O L -UVW, (u 0 , v 0 , w 0 ), can be self-calibrated.

Calibration Steps
(a) To process one of the image and calculate the matrices R j , T j from Equations (1)-(6) and then to determine (x 2i ', y 2i ') and (x ij ", y ij ", z ij ") (i = 1~13, j ≥ 7) from Equations (2)-(4). To compute ∆p j in Equation (8) and save the parameters of R j , T j if Q 1 < ∆p j < Q 2 . Otherwise ignore this photo.
(b) After at least seven eligible images are obtained, 13 distances d i (i = 1~13) and (x 0 , y 0 , z 0 ) can be determined from Equation (9). (c) To calculate ∆d max from Equation (10) and remove the image in which ∆d j is the biggest, and then to capture one more image and go back to step (a). After that, a new ∆d max is obtained. If this new ∆d max < Q 3 and the difference between two ∆d max obtained from adjacent seven eligible photos is less than Q 4 , then distances d i (i = 1~13) are considered as valid ones. Otherwise, similarly, to give up the image in which ∆d j is the biggest, and to capture one more image and go back to step (a).
(d) To determine the coordinates of the tip center in O L -UVW, (u 0 , v 0 , w 0 ) from Equation (11) based on eligible d i .

Experiments
Repeatability tests of tip center position self-calibration at different distances and experiments for measuring a circle and a gauge block have been carried out in a lab environment with the temperature about 20 • C.

Repeatability Tests of Tip Center Position Self-Calibration
By using the method mentioned above, the tip center positon of the light-pen has been self-calibrated ten times to test the repeatability of the method proposed in this paper. The test results are given in Table 1. As shown in Table 1, the average value (Ave for short), the standard deviation (Std for short) and the range (the difference between the maximum and minimum value) of the calibrated tip center positon, as well as the total number of photos needed for particular calibration, are presented. It can be seen that the dispersion range is (0.085, 0.068, 0.045) (mm) with the standard deviation (0.027, 0.023, 0.014) (mm) in the U, V and W directions, respectively. It shows the repeatability of the self-calibration method based on invariable distances could meet the needs of measurement systems with submillimeter accuracy. The total number of photos cannot be the same during each calibration, for that the number will be affected by many factors, such as the proficiency of experiment operator and some others. However, less than 300 photos are needed and it takes less than 2 min for each calibration, which shows that the proposed method of self-calibration has the advantage of high efficiency.
During the calibration, the distance between the camera and the kinematic seat or the light-pen should be suitable. If it is too short, brightness of feature points is too high to obtain their center positions accurately, even with the dynamic adjustment algorithm of feature points, whereas if the distance is too long, the solution accuracy for the LPCMM system will decrease. More images are needed for the self-calibration described above with the increase of the distance. In the repeatability experiment above, the camera keeps about 1.4 m away from the kinematic seat. However, this distance cannot be the same in each practical self-calibration. Table 2 shows the results of self-calibration at different distances. The average values, the standard deviations and the ranges of ten tests for each distance are listed. In Table 2, the dispersion ranges of the average u 0 , v 0 , w 0 at different distances are (0.084, 0.108, 0.094) (mm) with the standard deviations (0.030, 0.035, 0.042) (mm) in U, V and W directions, respectively, which is acceptable. From the table, it can be seen that the absolute values of u 0 and v 0 become bigger and bigger as the distance increases, while the value of w 0 has no significant Sensors 2017, 17, 131 9 of 12 changes. It might be caused by the error in camera focal length calibration. Some deep studies are needed. Table 2. Results of self-calibration for (u 0 , v 0 , w 0 ) at different distances (mm).

Distance
Ave Std Range In order to test the accuracy of the self-calibration method based on invariable distances, several measurement experiments are performed. A circle shown in Figure 6 and a gauge block were measured with the self-calibrated values of (u 0 , v 0 , w 0 ) equal to (−1.988, −70.663, −16.636) (mm) in the U, V, W directions, respectively. distance cannot be the same in each practical self-calibration. Table 2 shows the results of selfcalibration at different distances. The average values, the standard deviations and the ranges of ten tests for each distance are listed. In Table 2, the dispersion ranges of the average u0, v0, w0 at different distances are (0.084, 0.108, 0.094) (mm) with the standard deviations (0.030, 0.035, 0.042) (mm) in U, V and W directions, respectively, which is acceptable. From the table, it can be seen that the absolute values of u0 and v0 become bigger and bigger as the distance increases, while the value of w0 has no significant changes. It might be caused by the error in camera focal length calibration. Some deep studies are needed.

Measurement Experiments of the System with the (U0, V0, W0) Self-Calibrated
In order to test the accuracy of the self-calibration method based on invariable distances, several measurement experiments are performed. A circle shown in Figure 6 and a gauge block were measured with the self-calibrated values of (u0, v0, w0) equal to (−1.988, −70.663, −16.636) (mm) in the U, V, W directions, respectively.  The diameter of the circle measured by the CMM equals to 63.521 mm, taken as its true value. The maximum permissible error (MPE for short) of the CMM is 2.3 + 3.3 L/1000 (µm) where L is the measured length in mm. The circle was measured at five different positions and ten times at each position while the distance between the camera and the light-pen changes from 1.5 m to 9 m. The test results are given is Table 3. The standard deviation and the dispersion range increase with the increase of distance. The maximum absolute error (Abs for short) is 0.083 mm, which indirectly proves the validity of the self-calibration method.
Similar experiments have been performed by measuring a gauge block of 500 mm with uncertainty ±2.20 µm. The experimental results are shown in Table 4. The largest error in its absolute value is −0.119 mm measured at distance of 1.5 m. The length of the gauge block is obtained by measuring several points on its two end surfaces. The images of these points are near the boundary of the image plane, where the distortion errors are quite large even after corrections. However, the maximum absolute error is still acceptable, that indirectly verifies the validity of the self-calibration method again.

Conclusions
A new convenient method for stylus tip center position self-calibration based on invariable distances in the LPCMM is presented in this paper. Only a kinematic seat with an inverted cone hole is used and several images of control points are required to take during the self-calibration.
The system structure and model of the whole system are given. The model and steps of self-calibration are discussed in detail. To verify the effectiveness of the proposed method repeatability tests at different distances and experiments by measuring a circle and a gage block have been carried out. Test results are satisfactory and show that the LPCMM system with the calibrated light-pen could realize submillimeter in-situ measurements. Further studies will be carried out in the real industrial field to apply it in practice.