3.1. A Simulation Experiment of Two Cameras
In this paper, the above theory and calibration accuracy are analyzed by simulation experiment and physical experiments respectively. First, a two-camera calibration model is simulated and tested. For all experiments, we use an internally calibrated camera with a focal length of 16 mm and pixel size of 4.8 um×4.8 um. The image resolution is 1280 × 1024, the principal point is in the ideal optical center. The target contains 12 × 12 grid of feature points and the interval between points is 30 mm in both directions. We compare the accuracy of the linear solution and the optimization of re-projection error algorithm solution. In the calculation process, the different levels of noise are added to pixel coordinates of the corner points, which are extracted from images. The noise is divided into 10 levels from 0 pixels to 2 pixels, to verify the robustness of the proposed algorithm. The noise values are Gaussian distributed with a standard deviation of 0–2 pixels in 10 steps.
In the simulation experiment, the true value of the three relative rotation angles between the two cameras are R = [0.5233 rad, 0.6977 rad, 0.6977 rad], the true value of the translational vector is T = [106 mm, −5 mm, 2 mm]. During the simulation the cameras moved ten times. The pose relationship between the camera and target is calculated using the EPNP [
26] algorithm. For each noise level, 100 tests are carried out. The RMS errors of the three rotation angles and the three translations at different noise levels is shown in
Figure 3. The ordering of the Euler angles in experiment is Roll, Yaw, and Pitch.
Experimental results show that the linear algorithm is easily affected by noise. With the increase in noise level, the error of rotation angle increases exponentially, and the error of translation oscillates greatly. When the re-projection error optimization method and the orthogonal constraint are taken into account, the rotation angle error is obviously suppressed, and the improvement of the yaw angle accuracy is most obvious. At the same time, the translation error has also been suppressed to a certain extent.
3.2. Simulation Experiment of Multi-Camera Extrinsic Parameters Calibration with the Global Optimization Method
After calculating the extrinsic parameter matrix between any two cameras, the global optimization method is used to calculate the extrinsic parameter matrices of all cameras relative to the reference coordinate system. We analyzed the error variation of the extrinsic parameters before and after global optimization.
The simulation experiment takes a five-camera system as an example. Using the method in
Section 2.1 to calculate the extrinsic matrix between any two cameras, and we can obtain 10 sets of extrinsic parameters. Ten levels of noise are added to the 10 sets of extrinsic parameters. The noise added to the rotation angles range from 0.001 rad to 0.01 rad, and the translation noise ranges from 0.05 mm to 0.5 mm. For each noise level, 100 tests are carried out. The extrinsic parameters of the five cameras relative to the reference coordinate system are shown in
Table 1. Without loss of generality, we take camera 3 as an example, before and after optimization, the changes in the RMS errors of the extrinsic parameters are shown in
Figure 4.
The experimental results show that the global optimization method can effectively suppress the random noise of single calibration. Because the noise of this simulation experiment is directly added to three Euler angles and three translations of the extrinsic parameters, so the error has a linear relationship with the noise level before optimization. It can be seen from the graph that the rotation angle error is obviously suppressed after optimization. From the figure we see within a certain noise level, the relationship between the rotation angle RMS error and noise level is linear. From
Section 2.2.2 we can see that since the global optimization of the translation vector depends on the rotation angle, the noise of the rotation angle has a great influence on the translation vector. The experimental results show that when the noise level is higher and the rotation angle error increases, the error of the translation vector increases more intense.
To illustrate the applicability of the algorithm to all cameras, we give the extrinsic parameters errors of the remaining cameras relative to the reference coordinate system (camera coordinate system of the camera 1) at different noise levels (level 2 and level 5 which are consistent with
Figure 4). The errors before and after optimization are shown in
Table 2.
From the
Table 2, we can see that the optimization results of the remaining cameras are consistent with the results of the camera 3 as described in
Figure 4. The calibration accuracy of all the cameras have improved after global optimization.
3.3. Two Camera Experiment with Real Image
The above simulation experiment tests the robustness of the algorithm. In order to verify the validity of the algorithm, the following physical experiments are designed to verify the algorithm described in the article. First, we verify the calibration method of the binocular system. The method described herein can also be applicable to calibration cameras with a common field of view. In order to obtain the true value of the extrinsic parameters, a binocular camera system with partially overlapping FOVs is used to verify the method. The intrinsic parameters of the two cameras are shown in
Table 3. Because the FOVs of the cameras are partially overlap, the Calibration Toolbox for Matlab [
30] method is used to calibrate the extrinsic parameters of the two cameras as true values. The true value of the rotation angles is [−0.2024 rad, 0.1583 rad, 0.0191 rad], and the true value of the translation is [−215.709 mm, −1.769 mm, 13.586 mm]. To obtain accurate results in the experiment, we have evaluated the accuracy of the Calibration Toolbox, the reprojection error of the checkerboard corners is less than 0.3 pixels. At the same time, the Calibration Toolbox can calculate the confidence interval for the calibration results. The confidence interval of the rotation angles is ±[0.00067 rad, 0.00086 rad, 0.00010 rad], the confidence interval of the translation vector is ±[0.008215 mm, 0.01422 mm, 0.04773 mm]. The above results show that the calibration results of the toolbox have high precision and the results can be used as a standard to test the proposed algorithm. Then, 15 experiments are performed using the method proposed in this article, as shown in
Figure 5. The error between 15 experimental values and true values is shown in
Figure 6.
The experimental setup is shown in
Figure 5, we keep the target position stationary and the camera is moved 10 positions, and then the extrinsic matrix between the two cameras can be solved.
The experimental results show that the error of the three rotation angles is within 0.002 rad, where the RMS error of the roll angle is 0.0041, the RMS error of the yaw
angle is 0.0039, and the RMS error of the pitch angle is 0.0056. The difference between the mean value and true value of the roll angle is 9.023 × 10−4
, the difference between the mean value and the true value of the yaw angle is 9.417 × 10−4, and the difference between the mean value and the
true value of the pitch angle is 0.0014. The error of the three translational is within 0.08 mm, where the RMS error in the x-direction displacement is 0.1816, the RMS
error in the y-direction displacement is 0.1102, and the RMS error in the z-direction is 0.2206. The difference between the mean value and the true value
in the x direction is 0.0424, the difference between the mean value and the true value in the y direction is 0.0073, and the difference between the mean
value and the true value in the z direction is 0.0531.
3.4. Real Data Experiment of Global Optimization Calibration for Multi-Cameras
The previous section has verified the two camera calibration method, in order to verify the global optimization process of multi-camera systems, a 4-camera system without common field of view is built to verify the accuracy of the calibration algorithm based on the reprojection error and global optimization. To verify the accuracy of the calibration algorithm under real conditions, cameras in this experiment have no overlapping FOVs. Since the extrinsic parameters’ true value of cameras without a common field of view cannot be directly calculated, we verify the accuracy of extrinsic parameters by indirect methods with a coordinate measuring machine. The experimental device is shown in
Figure 7.
In this experiment, 4 checkerboard targets are used to calibrate 4 cameras. First, using the method described in
Section 2.1 to calculate the extrinsic parameter matrix between any two cameras, a total of 6 combinations can be obtained. Then, using the global optimization method to calculate the extrinsic parameters of each camera relative to the reference coordinate system through 6 sets of extrinsic parameter data. Since the position of the camera relative to the corresponding checkerboard can be calculated by EPNP algorithm [
26], we can use the extrinsic parameters between the checkerboards to obtain the extrinsic parameters between the corresponding cameras, and we used it as ground truth.
Without loss of generality, we use the extrinsic parameters of camera 1 and camera 2 as an example to verify the algorithm. The extrinsic parameters between the corresponding checkerboard targets are obtained using a coordinate measuring machine. The measurement method of the extrinsic parameters between the checkerboard targets is as follows:
(1) Select 8 corner points on target 1, and measure the coordinates of the 8 points using the coordinate measuring machine. The position of the eight points on target 1 is shown in
Figure 7.
(2) Measure the 8 points on target 2 at the same position as target 1.
(3) Establish constraint equations with Absolute Directional Algorithm [
31], then we can obtain the extrinsic parameters between the two targets using the coordinates of the 8 points. The equation is shown in Equation (13):
where
denotes the coordinates of corner point on target 1 and
denotes the coordinates of corner point on target 2.
The spatial coordinates of the measured points on target 1 and target 2 in the coordinate system of the three coordinates are shown in
Figure 8 and the coordinate values of the 8 points are shown in
Table 4.
The coordinate measuring machine we used is the Romer model of the Hexagon company. The measurement accuracy of the coordinate measuring machine is 5μm. The accuracy of the coordinate measuring machine is much higher than the vision measurement system we used. Therefore, we can use the coordinate measuring machine to evaluate the accuracy of the visual system in the experiment. We have evaluated the measurement results of the coordinate measuring machine, and calculated the residuals of Equation (13) using the measured data. Using the data in
Table 4, the residual of Equation (13) is 0.2365 mm (the sum of the residuals of the 8 corner points). The smaller residual value indicates that the coordinate measuring machine has higher measurement accuracy and can be used to verify the accuracy of the proposed algorithm.
A total of 15 experiments were carried out, the experiment results are shown in
Table 5, and the errors between calculated results and the ground truth data measured using a coordinate measuring machine are shown in
Figure 9.
It can be seen from the above experimental results that the three rotation angle errors are within 0.001rad, the RMS error of the roll angle is 0.0021, the RMS error of the yaw angle is 0.0020, and the RMS error of the pitch angle is 0.00065. The difference between the mean value and the true value is −2.323 × 10−5, the difference between the mean value and the true value of the yaw angle is −7.779 × 10−5, and the difference between the mean value and the true value of the pitch angle is −2.632 × 10−5. The errors of the three translational values are within 0.08 mm, the RMS value in the x direction is 0.0655, the RMS value in the y direction is 0.0877, and the RMS value in the z direction is 0.1611. The difference between the average value and the true value of the x-direction displacement is 4.734 × 10−3. The difference between the mean value and the true value of the y-direction displacement is −6.176 × 10−3, and the difference between average value and the true value the z-direction displacement is 3.111 × 10−2.
To illustrate the applicability of the algorithm to all cameras, we present the extrinsic parameters errors of the remaining cameras relative to the reference coordinate system (camera coordinate system of the camera 1). The maximum values of the errors in 15 experiments are shown in
Table 6.
From the
Table 6, we can see that the optimization results of the remaining cameras are consistent with the results of camera 2 as shown in
Figure 9. The three rotation angle errors of these cameras are within 0.001rad, and the three translational errors of these cameras are within 0.08 mm. The accuracy of the extrinsic parameters is improved after optimization. Compared with the two-camera experiment in
Section 3.3, after global optimization, the calibration accuracy has been obviously improved. The main performance is: the three angles of the extrinsic parameters and the average of the three translations are all close to the true value, and the RMS error has also been suppressed to a certain extent.