This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution license (http://creativecommons.org/licenses/by/3.0/).

There are increasing applications that require precise calibration of cameras to perform accurate measurements on objects located within images, and an automatic algorithm would reduce this time consuming calibration procedure. The method proposed in this article uses a pattern similar to that of a chess board, which is found automatically in each image, when no information regarding the number of rows or columns is supplied to aid its detection. This is carried out by means of a combined analysis of two Hough transforms, image corners and invariant properties of the perspective transformation. Comparative analysis with more commonly used algorithms demonstrate the viability of the algorithm proposed, as a valuable tool for camera calibration.

When a Computer Vision system is designed, a series of parameters must always be considered, which are decisive to the best possible appearance of objects in images with respect to posterior analysis algorithms. The focal length,

The values that are to be obtained are the horizontal and vertical focal distances, the coordinates of the optical center, the distortion of the lens, and the position and orientation of the camera with respect to the global coordinate system.

This article is laid out in the following way: the next section deals with a description of the advantages of having an automatic calibration system, and the state-of-the-art for automatic detection of calibration points is presented. This is followed by a description of the algorithm that is proposed in this article. Subsequent to this, the next section describes the localization methods for both the main corners and the lines of a chess board pattern separately; after, a description of the combination of this data to determine the points of the pattern is presented. Finally, a series of comparative results using the proposed method compared to more commonly used algorithms is made available, where these results clearly demonstrate the viability of the algorithm proposed.

In general, calibration techniques involve the analysis in images of the projection of a series of characteristic points, which are inherent features of an object whose three-dimensional coordinates are known with a high level of precision. To achieve this, a sequence of images on which the calibration pattern has been captured is made available. Within this pattern, a series of characteristic points are located, which give vital information on the appearance and the distance between the points to be defined. Once these points and their two-dimensional coordinates have been extracted from the sequence of images, the relation between each point of the pattern is found for the series in all the sequences. These are taken as the input for the calibration algorithm, which is used to provide both intrinsic and extrinsic parameters of the camera.

Over the past years, many different types of calibration patterns have been used. The first patterns were of a three-dimensional nature [

The above-mentioned calibration method [

The majority of research work in the area of calibration has been centered on the identification of the parameters using robust algorithms, however less attention has been paid to determining the characteristic points that are used as the inputs of the algorithms. The user is left with the more routine and mechanical task of locating the coordinates of the points in the images. The aforementioned task, which was acceptable when no effective calibration algorithms were available, has at the present time a series of disadvantages. These are:

Since the process is mechanical, users commonly do not pay sufficient attention to the process, and as a result it is subject to errors that are difficult to detect.

The number of applications is constantly growing and there are many fields of research that use a large number of cameras, thus emphasizing the need for more automatic calibration processes.

Although the calibration of the camera intrinsic parameters is only carried out once, this is not the case for the extrinsic parameters, which may change on various occasions until the definitive configuration is obtained.

Up until now, the steps required for the correct calibration process have been carried out by trained and experienced personnel who are familiar with the selection of adequate positions on the patterns, but this situation has to change if such applications are to be performed by users with no specific training in Computer Vision.

Recently, there has been much more research in this area of automatic determination of the calibration points that arise in patterns from a sequence of images taken at different positions. One option which has been considered is that each element in the pattern has a unique code, which helps in the detection and classification process. This method has been used by [

Within the group of algorithms where all the elements of the pattern are similar, work has been carried out by [

The rest of the methods use the chess board pattern, which was first presented by [

The recognition of a chessboard using only the detection of its corners is possible if the number of rows and columns is known as well as the area where the chessboard is located as the referenced toolboxes do, but it is very complex if this information is not known and the area is not marked, due to the huge amount of corners an image can have. The recognition using only the lines is possible if the images do not have geometrical distortion and line detection was perfect. The two points with more line intersections would be the clue to find the two families of lines of the chessboard. But images have a distortion, and because of this the edges between the files and columns of the patter are not straight lines and the Hough transform is discrete. Therefore, not all the lines can be detected perfectly. Because of that, the intersection of the lines is not at only two points and this consequence is stronger when the pattern is almost perpendicular to the camera. For these reasons, according to the method proposed in this article, the set of lines are determined from the periodic pattern, which appears in the Hough transform. Since many optical systems have significant geometric distortion, the lines of the pattern are curved in the image and the intersection of these lines does not coincide with the points of the pattern. With this in mind, the previously described analysis is combined with corner detection, on one hand, to validate the lines that are part of the pattern with respect to rest of the objects, and on the other, to determine with sub-pixel precision the true position of the points.

The Harris corner detector is widely used to find these features in images. This technique may also be used to find the initial points required for the Hough transform, as its response is positive for corners, almost nil for the uniform zones and is negative for the edges. Once the corner detector has been applied (

Further information used is the intersection between the lines of the image. To do this the Hough transform is used based on the points that were detected as edges from the Harris image (

The results obtained from the two images shown in

In order to obtain the sets of vertical and horizontal lines of the chessboard a series of precautions must be taken into consideration. The first is to decide whether to take a fixed or variable number of maximums, whereby these will depend on the values obtained from the transform. In both cases it involves choosing all lines that form the pattern, but without taking an excessive amount of lines that are not part of the calibration pattern; this will reduce the complexity of the posterior analysis, as can be seen below.

From

To find these points the “top hat” morphological transform is used. This operation is carried out in the following way. A first opening morphological transform is performed on the image and the result is subtracted from the original image. Observing

The amount of the detected points which follow the line distribution is observed in _{1}, _{1}) points as the inputs.

As can be seen in

Up to a maximum of three sets of similar lines are sought in the second Hough transform. For each one of these sets, the maximum value is found and compared to the threshold value; if it is greater than this value the set of lines is thus considered to be part of the pattern. In addition, it must be considered that the values of _{1},_{1}) to the set of lines (_{2}, _{2})

The results using this analysis can be observed in

These errors occur as the periodic nature of the Hough transform has not been taken into account in the previous analysis (_{1} ∈ [0, π], where the results are similar but shifted to the right. Thus, a series of lines will be taken (_{1}, _{1}), which are aligned, but given the interval where _{1} is defined, they are not found to be represented by the same values of (_{2}, _{2}) but by (−_{2}, _{2}). As can be observed in

It may be concluded from this analysis that the complete set of lines forming the pattern is composed of both parallel and convergent lines. In _{1} and will also have the values obtained from the second Hough transform (_{2}, _{2}) and (−_{2}, _{2}) associated with them. When considering the line _{1} but is negative, thus the lines parallel to it will be closer to the values (_{2}, −_{2}) and (−_{2}, −_{2}). To determine the complete set of lines, four different distances are calculated; if for any of the lines they are less than a pre-set value these lines are then associated to the set of lines (

To locate the pattern in the image, both the horizontal and vertical sets of lines must be determined. To do this the three previous sets of lines are taken and the one with the smallest number of lines associated with it is disregarded. From the two remaining sets, one will correspond to the horizontal lines and the other to the vertical lines. If the average value of the angle is calculated as:

The lines belonging to the chess board pattern must comply with the fact that the crossing points between the horizontal and vertical lines are close to one of the corners, which were found with the Harris corner detector (

The following step involves two different processes. The first process removes the lines with a small number of crossover points, as these correspond to remote lines, which do not form part of the pattern but are located in the background. The number of valid crossing points is again calculated and followed by the same process of elimination of the lines containing few crossover points.

When a line is eliminated this changes the common crossover points, therefore these two processes are performed iteratively until no more lines are eliminated.

Once the previously described processes reach the stage where no lines are removed, the second process detects the set of lines having the same number of crossover points. These lines correspond to the chess board pattern. Then, the average value of the crossover points is calculated for the horizontal and vertical lines. The lines, both horizontal and vertical, where the amount of crossover points is less than the average value, are eliminated; the process is repeated until none of the horizontal or vertical lines are eliminated.

The two groups of lines remaining are thus the lines of the chess board pattern. Different results using this elimination process are presented in

Once a series of lines has been obtained, the next task is to determine which of those lines actually correspond to the board pattern, as the situation may arise where a line is parallel to the axis of the board but does not belong to the pattern. To eliminate such lines the chess board pattern is used, whose number of rows and columns is unknown, however its squares have known dimensions; using this information all the lines that do not comply with this cross-ratio can be removed, since this value is invariant in the projection.

The cross-ratio of four aligned points, A, B, C and D is expressed as:

Since the points have the same vertical coordinate, it may be assumed that the four points are consecutive and belong to a horizontal line:

The lines are ordered from left to right and from top to bottom. The double ratio for the first four vertical lines on the left hand side of the pattern is calculated, if the result is within the 5% error margin of the value obtained in (6) it is considered as valid. If not, the first line is rejected and the fifth line is added and the value is recalculated. This process is continued until the first four lines are obtained that satisfy the criteria, providing the lines on the left hand side of the pattern. In a similar way, the last four vertical lines on the right side of the pattern are obtained where the double ratio is again calculated and the lines are accepted or rejected depending on the result. Following the same method the horizontal lines are also obtained. Once this process is finished the outline of the complete chess board pattern is obtained.

The last step of the algorithm is to eliminate the external lines of the board, so that the system only detects the interior area of the pattern. This process is carried out for two reasons:

The external lines that outline the board consist of lines where only half of the squares provide points for the Hough transform, these lines are not always detected. If a robust detection system is required involving different positions and conditions an element whose detection is uncertain should not be added. Even though this problem can be solved the following reason advises against its use.

The detection of the board serves for the detection of corners, the edge of this pattern is where the corners are most poorly defined; this is because there are no alternating black and white squares in both directions, which is the case for the rest of the corners. For this reason they are not considered as good points to be used in the calibration process.

To confirm if these lines should be removed from the detection algorithm, the value obtained from the Hough transform is observed and compared to the width of the pattern obtained. If it does not exceed the preset threshold level this indicates that a discontinuous line has been detected. This process is repeated for the four edges of the pattern. Results from the final detection process may be observed in

Finally, due to the geometric distortion, the crossover points between the lines will not coincide with the real location of the corners of the chessboard. For this reason, the corners are searched within the crossover points’ surroundings, and are used in the final calibration. (

The system is calibrated by the implementation of Zhang’s algorithm, which has been developed within OpenCV. It is a function that has as inputs the corners detected in the sequence of images and as output the intrinsic parameters of the camera.

Once these values have been obtained the translation and orientation respect to the chessboard for every image of the sequence can be calculated, this will help to refine the intrinsic parameters. To perform this process, the points of the pattern are projected onto the image. The error between the real value for the point projection and the theoretical is calculated (_{e}_{e}_{e}_{e}_{e}_{e}

The system described does not serve only for the calibration of the intrinsic parameters, it also allows calculation of the position and orientation of the camera with respect to a global reference. As can be observed in

To demonstrate the feasibility of the results obtained from the proposed system, they were compared to detection techniques that are already well established. This analysis was carried out for both the detection of the board pattern and the calibration values.

The OpenCV method first performs a thresholding, then followed by an erosion process, which is used to break the connectivity of the squares; after this it finds their number and position in the image. Since information in regards to the number of rows and columns is supplied, the erosion process is repeated as many times as is necessary to break the connectivity and find the corresponding numbers. Next, the squares are brought back together to form the board pattern. The number of points must be passed through each calibration and the pattern must be completely visible in the image, as is opposite to the technique proposed in this article.

The results of the calibration with the proposed method were compared to those obtained with the Bouguet Matlab Toolbox for three sequences, which are presented in

Concerning the extrinsic parameters, an image is presented in

The algorithm has been tested on two different equipments with images of 1,024 × 768 pixels and takes 1.9 seconds in an Intel Core2 Duo Mobile Processor at 1.66 GHz with 2 GB of RAM and 1.1 seconds in an Intel Core2 Duo at 3.00 GHz with 3.25 GB of RAM.

In this article, an automatic camera calibration method has been presented. This is achieved by using an algorithm proposed in this article to detect an artificial pattern located within an image. The only information available in the calibration pattern used is that it is composed of a chess board configuration. The algorithm is capable of determining the number of rows and columns of the pattern. Using different examples, it has been shown that the algorithm is a valuable tool for camera calibration and that its use will make this task less tedious, allowing the process to be easily performed by non-specialized personnel.

The authors would like to thank the anonymous reviewers for their help in improving the contents and style of the manuscript. This work has been carried out thanks to partial support from the Spanish Ministry for Science and Innovation

Distribution of cameras in the VISION project.

Harris corner detector (a) Original images (b) Harris operator (c) corner detection (d) edges.

Hough transform of the edge image.

Results from taking a fixed number of lines. (a) Original image (b) edge image (c) Hough transform with its 36 maximums (d) detected lines.

Results from taking a variable number of lines. (a) Original image (b) edge image (c) Hough transform with the maximums (d) detected lines

Lines detected using the local maximum search. (a) Local maximums (b) Detected Lines.

Detection of the set of lines. Maximums of the Hough transform (left column), images with the lines detected (middle column) and the second Hough transform (right column).

Initial detection of the set of lines.

(a-c-e) sets of lines (b-d-f) lines whose distance is small according to (3).

(a) Periodic character of the Hough transform (b) Convergent lines case.

Detection of sets of lines (a) correct detection (b-c) erronous detection (d) correct detections.

(a) Pairing up of the crossover points, corners and (b) detected lines.

Final line filter.

Input points of the calibration algorithm.

Errors between the theoretical projection and the real projection of the points of the pattern in the image for (a) the initial calibration (b) refined results.

(a) original image (b) Harris operator (c) detected corners (d) edges of the objects (e) Hough transform (f) Hough transform of the principle lines (g) principle line (h) sets of lines (i) lines which are coherent with the corners detected (j) final lines (k) board.

Drawing from two cameras of the outline of the room.

Comparison of the calibration values between the Toolbox of Matlab of Bouguet and this method.

| |||||||||||
---|---|---|---|---|---|---|---|---|---|---|---|

^{−3} |
^{−3} |
^{−3} |
^{−3} | ||||||||

S1 | Bouguet | 821.1 | 812.8 | 327.4 | 234.6 | −248.5 | 214.9 | 2.5 | −5.1 | 0.50 | 0.36 |

Our method | 820.6 | 812.4 | 327.4 | 234.7 | −247.0 | 210.9 | 2.5 | −5.1 | 0.39 | 0.40 | |

S2 | Bouguet | 514.3 | 515.9 | 322.1 | 243.0 | 23.9 | −157.5 | 3.1 | 1.5 | 0.30 | 0.17 |

Our method | 517.3 | 518.8 | 320.6 | 239.8 | 3.1 | −54.4 | 1.4 | 0.7 | 0.24 | 0.23 | |

S3 | Bouguet | 2371.6 | 2421.8 | 326.3 | 389.8 | −35.7 | 684.7 | −3.96 | −57.5 | 1.17 | 0.31 |

Our method | 2440.0 | 2493.0 | 328.2 | 387.9 | 28.0 | 334.9 | −4.2 | −59.5 | 0.93 | 0.39 |