Camera calibration is a major issue in computer vision since it is related to many vision problems such as neurovision, remote sensing, photogrammetry, visual odometry, medical imaging
, and shape from motion/silhouette/shading/stereo
. Metric information within images can be supplied only by the calibrated cameras [1
]. The 3D computer vision problem is mathematically determined only if the optical parameters (i.e., parameters of intrinsic orientation
) and geometrical parameters (i.e., parameters of extrinsic orientation
) of the camera system are precisely known. The camera calibration methods can be classified according to the determination methods of optical
parameters of the imaging system [1
]. The number of camera calibration parameters (i.e., rotation angles, translations, coordinates of principal points, scale factors, skewness between image axes, radial lens distortion coefficients, affine-image parameters
, and lens-decentering parameters
) depends on the mathematical model of the camera used [2
In the literature, many camera calibration methods have been introduced. A self-calibration method to estimate the optic
parameters of a camera from vertical line segments of the same height is examined in [3
]. Extrinsic calibration of multiple cameras is very important for 3D metric information extraction from images. Computation of relative orientation parameters between multiple photo/video cameras is still one of the active research fields in the computational vision [4
]. Using geometric constraints within the images, such as lines and angles, enables performing 3D scene reconstruction tasks with fewer images [6
Plane-based camera calibration is an active area in computational vision because of its flexibility [7
]. A planar calibration grid-pattern has some important advantages with respect to 3D calibration objects such as simple design, simple structure, easy scaling and easy construction. Therefore, planar calibration objects are preferred in computer vision applications [8
]. Planar calibration objects and projective constraints can be used for calibration of parametric and nonparametric distortions of a camera system [9
]. The camera calibration problem for planar robotic manipulators through visual servoing under a fixed-camera configuration has been investigated in [10
Dual images of spheres and the dual image of the absolute conic have been used for solving the problem of camera calibration from spheres in [11
]. The mirror-symmetric objects have been used for camera calibration in [12
]. An accurate calibration procedure has been introduced for fish-eye lenses in [13
]. The calibration of a projector-camera system by estimating the homography has been investigated in [14
]. Online calibration methods have been used in virtual reality applications in [15
]. A dynamic calibration method for multiple cameras has been investigated in [16
]. Due to the noise-influenced image coordinates, most of the existing camera calibration techniques are unsuccessful aspects of robustness and accuracy.
The artificial neural networks (ANNs) can mimic the transformation between the image plane and the global coordinate system. By using ANNs, it becomes unnecessary to know both the physical parameters and the geometrical parameters of the imaging systems for 3D perception of objects from their 2D images. ANNs have been intensively used for camera calibration in some recently introduced methods [17
]. A planar pattern has been observed at different rotations for setting training and test data sets of the ANN used. The rotation value of the planar pattern has been acquired by using an Xsens MTi-9 inertial sensor [20
]. With the proposed method, the 3D global coordinates of object points have been predicted from their 2D corresponding image coordinates.
The Xsens MTi-9 sensor is a miniaturized, gyro-based Attitude and Heading Reference System whose internal signal processor provides drift-error free 3D acceleration, 3D orientation, and 3D earth-magnetic field data. The drift-error growing nature of inertial systems limits the accuracy of inertial measurement devices. Inertial sensors can supply reliable measurements only for small time intervals. The inertial sensors have been used in some recent research for stabilization and control of digital cameras, calibration patterns and other equipment [22
The Modified Direct Linear Transformation (MDLT) is one of the commonly used camera calibration methods in computational vision applications for 2D and 3D object reconstruction [24
]. The success of the proposed method has been evaluated by comparing the test results of the proposed method and MDLT method.
The camera calibration methods have been classified into two main classes in the literature: explicit and implicit camera calibration methods. The explicit camera calibration means the process of computing the physical parameters of a camera. The proposed method is classified as an implicit camera calibration method and implicit camera calibration methods do not require physical parameters of cameras for back-projection.
The rest of the paper is organized as follows: Artificial Neural Networks are explained in Section 2. Proposed Method and Experiments are given in Section 3 and Section 4, respectively. Finally, Results and Discussion are given in Section 5.
In this paper, a set of real images
have been used in the experiments. The proposed method has been implemented by using the image processing toolbox of Matlab, and SDK of Canon Camera Control. The images of calibration pattern have been captured by using two static, computer-controlled
Canon SX110IS 9MP cameras. Therefore, the neural structure used in the proposed method has only four inputs. All the captured images were 1,600 × 1,200 pixels sized and 24 bits/pixel. One precalibrated 3D object has been used for computing the parameters of MDLT. The interior parameters (including distortion coefficients
) of the test cameras have been computed by using the camera calibration toolbox given in [2
]. Before performing the MDLT, geometric distortion corrections have been applied to the image coordinates, in order to increase the success of the MDLT. On the other hand, no distortion corrections have been applied to the image coordinates for the proposed method.
The performance of the proposed method has been examined by scanning both a 2D test object, a 3D test object and the face of the author. The experimental results of the proposed method have been compared with the experimental results of MDLT. All the measurements have been denoised by using the FastRBF toolbox [32
] before performance analysis of backprojection of 2D and 3D test objects, in order to analyze effectiveness of smoothing of FastRBF.
and depth reconstruction
accuracies of the mentioned methods have been evaluated in Mean-Squared-Error (MSE) as seen at Table 3
The experimental results verify the success of the proposed method and MDLT. All of the errors have been measured with respect to checkerboard-test object. The test points have been marked at the corners of the test object. The checkerboard-test pattern has been designed in Matlab and printed with a 9,600 DPI professional plotter and attached onto a flat board. Since 373 points/mm have been defined on test pattern for 9,600 DPI, 0.002mm (1/373mm) has been accepted as the ground truth of test object.
Totally 1,127 3D points have been captured over the 2D test pattern and totally 1,460 3D points have been captured over the author's face. All of the measurements in the global coordinates have been performed in centimeters. The solid model of the author's face obtained by using the proposed method has been illustrated in Figure 2(e)
. Extensive simulations show that the results of the proposed method are close to MDLT for 2D test object but they are better in both planimetric and depth perception.
The 3D backprojection tests have been realized on a 3D test object, which is illustrated in Figure 3
. The FastRBF toolbox based denoising phase has not been employed in 3D backprojection tests of 3D object. The related 3D test object has been located inside of the calibration volume, and its images were captured by using the cameras. The distances of 686 backprojected 3D points to the computed-planes (Figure 3
) have been analyzed. The mean (μ
) and standard deviation (σ
) values of related distances have been computed. For the MDLT, μ
= 2.487 mm and σ
= 0.868 mm have been computed. For the proposed method, μ
= 2.128 mm and σ
= 0.793 mm have been computed. The edge lengths, illustrated in the Figure 3
, have been computed by using the mentioned methods and results have been compared with the mean of the manually measured values. The manual measurements have been realized by using a vernier-caliper at the resolution of 0.01 mm. All the manual measurements of vernier-caliper have been repeated 20 times, in order to avoid reading errors of user. In Table 4
, the test results on the 3D test object have been given.
5. Results and Discussion
In this paper, an Xsens MTi-9 inertial sensor and an RBF have been used together for 3D information recovery from images. The obtained results have been compared with the results obtained from a traditional camera calibration method, MDLT.
The main advantages of the proposed method are as follows: It does not require the knowledge of complex mathematical models of view-geometry and an initial estimation of camera calibration, it can be used with various cameras by producing correct outputs, and it can be used in dynamical systems to recognize the position of the camera after training the ANN structure. Therefore, the proposed method is more flexible and straightforward than many of the methods introduced in the literature.
The advantages of the proposed method may be summarized as follows:
The proposed method introduces a novel implicit camera calibration method based on inertial sensors (Implicit camera calibration techniques are not interested in the physical parameters of the cameras).
The results of the proposed method are close to MDLT but they are better, therefore it can be used in robotic vision as MDLT.
The computational-burden of the proposed method is less than MDLT.
The required time for preparation and scaling of the 2D calibration object of the proposed method is less than the time of preparation and scaling of the 3D calibration object of MDLT.
It offers high accuracy both in planimetric (x,y) and in depth (z).
It is simple to apply and fast after training.
The image distortion and the physical parameters of the cameras have been covered by the neural network model of the proposed method.
No image distortion model is required.
It does not use physical parameters of cameras.
An approximated solution for initial step of camera calibration is not employed.
Optimization algorithms are not employed during 3D reconstruction in contrary to some of the well-known 3D acquisition methods.