Self-Calibration Method and Pose Domain Determination of a Light-Pen in a 3D Vision Coordinate Measurement System

Light pens for 3D vision coordinate measurement systems are increasingly widely used due to their advantages, such as their small size, convenience of being carried, and widespread applicability. The posture of the light pen is an important factor that affects accuracy. The pose domain of the pen needs to be given so that the measurement system has a suitable measurement range to obtain more qualified parameters. The advantage of the self-calibration method is that the entire self-calibration process can be completed at the measurement site with no auxiliary equipment. After the system camera calibration was completed, we took several pictures of the same measurement point with different poses to obtain the conversion matrix of the picture and subsequently used spherical fitting, the generalized inverse method of least squares, and the principle of position invariance in the pose domain range. The combined stylus tip center self-calibration method calculates the actual position of the light pen probe. The experimental results verify the effectiveness of the method; the measurement accuracy of the system can satisfy basic industrial measurement requirements.


Introduction
With the ongoing advancement of human society, manufacturing continues to flourish, and industrial measurement has become an extremely important link in the industrial development process. With the development of the automotive industry and aerospace industry, which require high-precision real-time measurement, there are increasingly more applications and requirements for workpiece measurement, particularly in terms of accuracy. At present, mainstream measurement technology cannot satisfy the ever-increasing measurement requirements. The traditional coordinate measuring machine (CMM) [1] cannot satisfy the online measurement requirements of some industrial sites due to its large size and limited measurement range. Indoor GPS, white light scanners [2,3], optical laser trackers [4][5][6], and other common measurement equipment [7,8] also have many shortcomings such as low measurement efficiency, high cost, complex configuration, and poor portability. The shortcomings of modern measurement technology have limited the development of industry. Therefore, the development of new measurement technology has become a common expectation. Vision measurement systems [9] are mainly based on computer vision, including measurement technology, electronic technology, image processing technology, etc. to create a new measurement structure frame, reduce measurement limitations, increase flexibility and convenience, and enable workpiece measurement from multiple angles. Its small size has met with a favorable response in modern manufacturing fields such as aerospace and automobiles. The light-pen-type 3D vision measurement system [10] based on machine vision is increasingly used because of its small size, convenient portability, flexible assembly according to on-site measurement requirements, and good applicability; and related products have been put into production [11]. Of course, many scholars have unique insights into visual measurement systems and have achieved notable results [12][13][14][15][16][17].
The actual coordinates of the center of the light pen stylus, in the coordinate system, directly affect the measurement accuracy of the three-coordinate measuring system. In practical applications, the center of the light pen stylus is usually taken as a theoretical value when the light pen is used for processing. Due to the processing error of the light pen, the center value directly affects the measurement accuracy. When measuring various measurement objects such as deep holes and grooves, it is necessary to replace the light pen probe with other angles or add an extension rod for measurement. The difference in force and loss of multiple uses of the light pen also cause small changes in the coordinates, so the self-calibration of the stylus tip center is particularly important. Liu optimized an objective function based on the principle of position invariance and generalized an inverse method of the least squares solution of the nonlinear equations and obtained the position of the stylus tip center in the coordinate system [18,19]. Zheng proposed a two-step calibration method for the center of a plane target probe [20]. According to the principle of position invariance, the objective optimization function is established, and the optimized correction value of the stylus tip center in the X and Y directions in the target coordinate system is solved by the Levenberg-Marquardt algorithm [21]. Then, the error of the stylus tip center in the Z direction is solved. Zhang also adopted the principle of position invariance, upon which he provided the error constraint parameters [22].
The posture of the light pen is an important factor that affects the accuracy. The pose domain of the light pen needs to be given so that the measurement system has a suitable measurement range to obtain more qualified parameters. Therefore, a threshold measurement device was designed according to the measurement system in the experiment. The main body was composed of a light pen, connecting rod, base, motor seat, coupling, and other parts. The transmission device was composed of a Siemens servo motor and the corresponding drive. The experiment was completed on the built measurement system [23]. After the motor parameters and program settings were completed, the experiment was driven by the motor to drive the device of the mechanical structure to make precise angle changes. For each rotation of 1 • , the corresponding experimental image was collected to obtain the measurement error at this angle, and the data were sorted to obtain the pose that satisfied the requirement area.
A new self-calibration method for the center of the optical pen probe was proposed. The actual position of the light pen probe was obtained by combining methods such as spherical fitting, the generalized inverse method of the least squares method, and the principle of position invariance [19]. Regardless of the changes in pose of the stylus, the position of the center of the stylus remained the same, as shown in Figure 1. After the camera calibration was completed [24][25][26][27], the light pen probe collected multiple sets of images in different poses that satisfied the threshold measurement requirements under the CCD camera and used the conversion relationship of each image to fit the same feature point on a sphere with the center of the probe as the center of the sphere, as shown in Figure 2. Thus, the actual position of the center of the probe was obtained.

Light-Pen-Type 3D Vision Measurement System
The light-pen-type vision measurement system is mainly composed of three parts: a CCD industrial camera capable of imaging infrared rays, a computer, and a light pen, as shown in Figure 3. The light pen consists of eight infrared LEDs and a detachable ruby ball probe. Figure 4 shows several types of probes. A and B have different core diameters, and the B-E measuring rods have different lengths. During the measurement, the light pen image is taken by the CCD industrial camera and transmitted to the computer for related calculations; finally, the measurement result is obtained.

Light-Pen-Type 3D Vision Measurement System
The light-pen-type vision measurement system is mainly composed of three parts: a CCD industrial camera capable of imaging infrared rays, a computer, and a light pen, as shown in Figure 3. The light pen consists of eight infrared LEDs and a detachable ruby ball probe. Figure 4 shows several types of probes. A and B have different core diameters, and the B-E measuring rods have different lengths. During the measurement, the light pen image is taken by the CCD industrial camera and transmitted to the computer for related calculations; finally, the measurement result is obtained.

Light-Pen-Type 3D Vision Measurement System
The light-pen-type vision measurement system is mainly composed of three parts: a CCD industrial camera capable of imaging infrared rays, a computer, and a light pen, as shown in Figure 3. The light pen consists of eight infrared LEDs and a detachable ruby ball probe. Figure 4 shows several types of probes. A and B have different core diameters, and the B-E measuring rods have different lengths. During the measurement, the light pen image is taken by the CCD industrial camera and transmitted to the computer for related calculations; finally, the measurement result is obtained.
The measurement process of the measurement system is as follows: After the camera's internal parameters have been calibrated [28], the user places the measured object and light pen in the range that the CCD camera can shoot, the user presses the shooting button, the CCD camera takes a set of pictures, and the world coordinates of the measured object are obtained. For the coordinates in the system, the distance between two points can be similarly measured by placing the light pen on another point.  The measurement process of the measurement system is as follows: After the camera's internal parameters have been calibrated [28], the user places the measured object and light pen in the range that the CCD camera can shoot, the user presses the shooting button, the CCD camera takes a set of pictures, and the world coordinates of the measured object are obtained. For the coordinates in the system, the distance between two points can be similarly measured by placing the light pen on another point.

Establishment of the Coordinate System
The visual coordinate measurement system contains three coordinate systems: 1.
Image coordinate system (O0-XY): The coordinate system is established on the imaging surface of the CCD camera with the center of the imaging surface as the coordinate origin. The x-axis and y-axis are parallel to the horizontal and vertical imaging directions of the CCD camera, respectively, and parallel to the O1UV plane (unit: pixel). 2.
Camera coordinate system (O1-uvw): The camera coordinate system is established on the industrial camera, and the origin of the camera coordinate system is the optical center of the industrial camera. The plane formed by the camera coordinate system is also parallel to the O1UV plane (unit: mm).

3.
Light pen coordinate system (O2-xyz):   The measurement process of the measurement system is as follows: After the camera's internal parameters have been calibrated [28], the user places the measured object and light pen in the range that the CCD camera can shoot, the user presses the shooting button, the CCD camera takes a set of pictures, and the world coordinates of the measured object are obtained. For the coordinates in the system, the distance between two points can be similarly measured by placing the light pen on another point.

Establishment of the Coordinate System
The visual coordinate measurement system contains three coordinate systems: 1.
Image coordinate system (O0-XY): The coordinate system is established on the imaging surface of the CCD camera with the center of the imaging surface as the coordinate origin. The x-axis and y-axis are parallel to the horizontal and vertical imaging directions of the CCD camera, respectively, and parallel to the O1UV plane (unit: pixel). 2.
Camera coordinate system (O1-uvw): The camera coordinate system is established on the industrial camera, and the origin of the camera coordinate system is the optical center of the industrial camera. The plane formed by the camera coordinate system is also parallel to the O1UV plane (unit: mm).

Establishment of the Coordinate System
The visual coordinate measurement system contains three coordinate systems: 1.
Image coordinate system (O 0 -XY): The coordinate system is established on the imaging surface of the CCD camera with the center of the imaging surface as the coordinate origin. The X-axis and Y-axis are parallel to the horizontal and vertical imaging directions of the CCD camera, respectively, and parallel to the O 1 UV plane (unit: pixel).

2.
Camera coordinate system (O 1 -uvw): The camera coordinate system is established on the industrial camera, and the origin of the camera coordinate system is the optical center of the industrial camera. The plane formed by the camera coordinate system is also parallel to the O 1 UV plane (unit: mm).

3.
Light pen coordinate system (O 2 -xyz): The theoretical value of the center of the light pen stylus is the coordinate origin, and the direction of the light pen shaft is the Y-axis; the lower plane perpendicular to the six characteristic points of the light pen is the Z-axis, and the X-axis is determined by the right-hand rule (unit: mm).

Solve Equations
Before measurement, the stylus tip center in the light pen coordinate system should be calibrated. During the measurement, the images of the LED targets are captured by a CCD camera, and the computer processes the images to obtain the center coordinates of the center position of the target as a known quantity to calculate the rotation matrix R and translation vector T [23]. The coordinates of the stylus tip center that correspond to the measured position in the camera coordinate system can be obtained by (1). The camera coordinate system is transformed into the light pen coordinate system as follows: We establish the relationship between the camera coordinate system and pixel coordinate system: The conversion between the camera coordinate system and pixel coordinate system is carried out using Equation (2); f is the focal length, X = f w u, Y = f w v; We establish the linear equation set of the j-th feature point in the image: let t z   a 1 a 2 a 3 a 4 a 5 a 6 a 7 a 8 a 9 a 10 a 11 1 r 1 r 2 r 3 t x r 4 r 5 r 6 t y r 7 r 8 r 9 t z   s j = r 7 x j + r 8 y j + r 9 z j + t z = t z (a 9 x j + a 10 y j + a 11 z j + 1) = t z ζ j Equation (3) is equivalent to: a 1 a 2 a 3 a 4 a 5 a 6 a 7 a 8 a 9 a 10 a 11 1 After solving the linear equations, the parameters are separated to obtain the initial values of R and T.
In the ideal perspective imaging process, the solution obtained by the above formula is feasible because there is no error in the coordinate values of the object point and image point in each coordinate system, and the calculated R satisfies the orthogonal system constraint. However, in the actual environment, due to various factors such as camera parameter calibration error [29], image plane position extraction error of the control point center, and coordinate value calibration error of the control point center in the coordinate system, the R matrix does not satisfy the orthogonal constraint relationship. Consequently, matrix T also has a large error, so the initial values of R and T can be obtained by the linear equation solving method. Then, the optimal solution of R and T can be obtained by the Newton-Gaussian iteration method [23].

Self-Calibration Method of the Stylus Tip Center
After R and T have been determined, because the distance between the characteristic point of the light pen and the actual position of the stylus tip center coordinate is a fixed value, according to the generalized inverse method of the nonlinear least square method, we obtain: where i represents the i-th image, and j represents the j-th feature point. The coordinate positions of the pictures in the coordinate system are different, but the distance from the actual position of the center coordinate of the probe is the same. Hence, the same feature point of n groups of pictures in the same spatial coordinate system (based on the probe and actual position of the center coordinate) is the center of the sphere, and the constant distance between the characteristic point and the stylus tip center is the radius of the sphere. The same feature point of all pictures is fitted to a sphere, and eight feature points are fitted to eight spheres. The spherical center coordinates of the fitted spheres are the actual position of the center coordinates of the stylus.
We obtain the residual formula: The corresponding partial derivatives are We insert the coordinate values, set the partial derivative equal to zero, construct a system of equations, and solve it to obtain the actual position of the center coordinate of the stylus.

Self-Calibration Steps
The self-calibration method is realized on the built visual coordinate measurement system. Briefly, we outline the steps to obtain the actual position of the stylus center coordinates using the self-calibration method.

1.
After the measurement system has been built, use the spatial coordinates of the characteristic points of the light pen measured by CMM as the initial value.

2.
Use the principle of position invariance to shoot images of different poses. Considering the particularity of the self-calibration method, the image pose should be changed as much as possible in the threshold range to make the result more accurate.

3.
After obtaining at least eight sets of pictures within the threshold range, calculate R and T for each image using Equations (1)-(5).

5.
Obtain the actual position of the stylus tip center coordinate in the light pen coordinate system.

Pose Domain Experiment
The rotation angle of the light pen in each direction affects the measurement accuracy. Considering the rotation angles of the three rotation axes of the light pen, the threshold measurement experiment was carried out according to the above method.
We performed the threshold measurement experiment according to the above method. When the pen was rotated around the X-axis, the angle between the light pen and the positive half axis of the Y-axis (pitch) was 41 •~1 59 • , and there was a clear and recognizable image.
When the pen was rotated around the Y-axis, the angle between the light pen and the positive half axis of the X-axis (yaw) was 0 •~1 80 • , and there was a clear and recognizable image.
When the pen was rotated around the Z-axis, the angle between the light pen and the positive half axis of the X-axis (roll) was 0 •~4 5 • and 136 •~1 80 • , and there was a clear and recognizable image.
Due to the processing error of the mechanical device [30][31][32] and measurement error of the measuring equipment, we used 1 • as the unit of the data to ensure that the error did not affect the experimental results. The motor drove the light pen to the measurement position, took a stable image, calculated the measurement accuracy, and rotated 1 degree to take the image. Every time it rotated by 1 degree, the camera collected 15 groups of images within the angle, analyzed and processed the data, and compared the obtained measurement results with the real values to determine differences. The graphs are shown in Figures 5-7.      When the pitch angle range is 80~97°, the yaw angle is 80~101°, and the tilt angle is 0~2° and 177~180°, a certain measurement accuracy can be guaranteed to satisfy the measurement requirements. Satisfactory experimental results were obtained. When the pitch angle range is 80 •~9 7 • , the yaw angle is 80 •~1 01 • , and the tilt angle is 0 •~2• and 177 •~1 80 • , a certain measurement accuracy can be guaranteed to satisfy the measurement requirements. Satisfactory experimental results were obtained.

Self-Calibration Measurement Experiment
We carried out the self-calibration experiment according to the method steps in this article. The self-calibration method was incorporated into the light pen measurement system for measurement comparison experiments, and each light pen performed ten measurement experiments. The measured value was compared with the real value (30.0317 mm) to obtain the absolute error and relative error. The measurement results are shown in Table 1. The measured values of the five light pen styluses are displayed in lines 2-11. The last three lines show the average, absolute error, and relative error. The measurement results of each stylus in the ten images are shown in lines 1-10. As shown in Table 1, the absolute error of the probe of each light pen is stabilized below 0.0737 mm, and the relative error is stabilized below 0.0025 mm. The experimental results satisfy the basic industrial measurement requirements, which verifies the effectiveness of this method.

Single-Point Repeatability Experiment
We put the light pen stylus in a standard cone. To maintain the position of the light pen stylus unchanged, we rotated the position of the light pen to perform a single-point repeatability experiment.
The experimental results of five light pen styluses are shown in Table 2. The coordinates of the center point of each light pen stylus in the ten images are shown in columns 1-10. The single-point repeatability is expressed by the standard deviation of ten center point coordinates. As shown in the previous column, the probe repeatability of each light pen does not exceed 0.041, which verifies the effectiveness of the self-calibration method.

Conclusions
A self-calibration method for the center of a light pen stylus based on spherical fitting is proposed. Multiple sets of light pen images with different poses were taken, and the actual position of the light pen probe was obtained by combining methods such as spherical fitting, the generalized inverse method of least squares, and the position invariance principle. The model and steps of the self-calibration method are given. To verify the effectiveness of the method, five different light-pen probes were used to conduct experiments, and a singlepoint repeatability experiment was carried out. The test results show that the measurement accuracy after using the method is obviously better than that before the method is used, and it satisfies the basic industrial measurement requirements.
The pose of the light pen is an important factor that affects the measurement accuracy of the light pen. Combined with the measurement system, the measurement experiment of the light pen pose field was carried out. In the experiment, three angles were measured, and the pose domain of the light pen was obtained, which standardized the pose of the light pen and ensured its measurement accuracy. The next step of research will involve actual industrial applications, and it will be better applied to actual measurements.

Conflicts of Interest:
The authors declare no conflict of interest.