1. Introduction
With the rapid development of image processing technology, image mosaic has become a hot topic in the field of image processing, which refers to combining multiple images with overlapping parts into a panoramic image that mainly consists of image registration and image fusion. In the study of image registration, Hines et al. [
1] proposed a method of phase correlation in 1975 using Fourier transformation to effectively register images with translation. However, when the image gets larger, the time of the Fourier transformation increases exponentially, which seriously reduces the efficiency of the algorithm. In 1988, Harris et al. [
2] proposed a feature-based registration algorithm that improved the matching speed and accuracy compared with the method of phase correlation, but the calculation of the Harris response function was related to the empirical value and lacked stability. In 2004, Lowe et al. [
3] proposed a new image registration algorithm based on scale-invariant feature transform (SIFT). This algorithm applies to images with translation, rotation, and distortion, and its stability is greatly improved. However, it is difficult to avoid many false matching points, and the execution speed of the algorithm is slow. In 2006, Edward et al. [
4] proposed an algorithm called features from accelerated segment test (FAST), which significantly improved the detection speed but lacked a description of the feature points. In the same year, Bay et al. [
5] proposed the speed-up robust features (SURF) algorithm, which solved the disadvantages of SIFT, but its stability and number of feature points were inferior to SIFT. Then, Michael et al. [
6] proposed an improved algorithm based on FAST, named oriented FAST, and Rotated BRIEF (ORB). Moreover, the advantage of this algorithm was that it had a significant increase in noise resistance and speed, but it lacked scale-invariance. Accuracy, rapidity, and stability are three important evaluation indexes in the study of image registration [
7].
In terms of the purification of matching point pairs, Liu et al. [
8] eliminated some mismatching points by modifying the threshold values of Euclidean distance and integral images, and realized the effective utilization of feature points, but the registration accuracy was still not high. Zhu et al. [
9] proposed an improved SIFT algorithm based on the idea of block matching, which had a good registration effect for remote sensing images. To solve the problem of algorithm stability, Yi et al. [
10] proposed an iterative SIFT algorithm based on adaptive non-maximum suppression, which realized the high robustness of UAV cognitive navigation but also increased its complexity. Aimed at the problem of poor registration effect caused by simply improving SIFT, scholars started to combine SIFT and the random sample consensus (RANSAC) algorithm for image registration, which was famous for its robustness. Even if there are a few wrong data points, it can still obtain an ideal model parameter.
Rahul et al. [
11] presented a comprehensive overview of existing research in RANSAC-based robust estimation, and provided a universal framework for RANSAC (USAC) algorithm. Shi et al. [
12] presented an improved RANSAC (I-RANSAC) algorithm for SIFT feature point matching. Li et al. [
13] proposed an algorithm based on a double-column histogram, which has a good application in remote sensing images. Zhao et al. [
14] proposed an algorithm based on the consistency principle of matching distance and realized high matching accuracy. Eduardo et al. [
15] proposed a dynamic distributed robust RANSAC algorithm, which improved the stability index. Gao et al. [
16] proposed a rotating average pipeline based on layered RANSAC to handle relative rotational outliers, and improve the accuracy index. Lati et al. [
17] proposed an image mosaic algorithm based on effective blur technology, which has good recognition performance for motion blur images. In terms of image fusion, classical fusion algorithms include direct fusion, weighted smooth fusion, PCA fusion, wavelet transformation fusion, and multi-scale transformation fusion [
18]. The traditional RANSAC method has obvious disadvantages, because it only produces reasonable results under certain probability, and improves the accuracy through multiple iterations of the optimal solution of model parameters, which increases the complexity of the algorithm.
In addition, in terms of underwater image mosaic, Chen et al. [
19] scombined SIFT and wavelet transformation, but the image registration speed was slow. Xie et al. [
20] applied the SURF algorithm to underwater images and obtained a good mosaic effect, but only for images of shallow water with good illumination. Rahul et al. [
21] proposed a seamless underwater image mosaic technology based on alpha clipping. Due to degradation problems such as low contrast and color deviation of underwater images, traditional image mosaic algorithms cannot be directly applied to underwater image mosaic.
In the above-mentioned studies, there exist three problems. Firstly, the algorithms have high computational complexity. Secondly, the algorithms focus on improving the effectiveness of a single index and less on improving the performance of two or three indicators. Thirdly, they are seldom used for low contrast and low definition in image processing. Based on these problems, in this paper, we proposed an improved image registration (IIR) algorithm to realize the rapidity of calculation and accuracy of image processing, by optimizing the solution of the Homography matrix according to the number of inner points. In addition, the proposed algorithm was tested on the images from the Oxford Buildings Dataset [
22] to verify that it has high feasibility since this dataset is recognized by the academic community for image processing research and is highly representative, containing thousands of images. Finally, the IIR algorithm was implemented in underwater image processing to improve the quality of underwater image mosaic.
3. Results and Discussion
In this study, image mosaic refers to combining two original images with overlapping regions into one image, which is completed through three steps including image enhancement, image registration, and image fusion. The two original images are respectively defined as the reference image and the be-registered image (the image to be registered). The former can be regarded as a standard reference image without any change, and the latter will be mapped to the former via the spatial transformation with dynamic change. The image processing platform was unified as one laptop, and the detailed information of which was as follows: Windows 10 system, Intel(R) Core(TM) i5-7200u CPU @2.50 GHz 2.71 GHz, and 8 GB of memory.
Firstly, two-group images (four images) with low contrast and low definition were randomly selected as experimental objects from the Oxford Buildings Dataset, as shown in
Figure 3 and
Figure 4. The two-group images are named T1 and T2; and the original images of each group included a reference image and a be-registered image that corresponded to Sub-figure (a) and Sub-figure (b), respectively.
After mixed method and sigma filtering,
Figure 5 and
Figure 6 are the enhanced results of the two-group images. To evaluate the enhancement effect quantitatively, three indexes, including the mean square error
, the peak signal-to-noise ratio
, and one-dimensional image entropy
, were used to evaluate the image quality before and after enhancement.
Table 1 lists the specific information of the three indexes.
The improvement of the image contrast and definition can be visually seen in
Figure 5 and
Figure 6. Through a quantitative analysis of the data in
Table 1, the average mean square error for the reference image and be-registered image were 7.95 and 8.37, respectively; the average peak signal-to-noise ratio for the reference image and be-registered image were 49.42 and 48.86, respectively; and the average one-dimensional image entropy for the reference image and be-registered image were 0.08 and 0.09, respectively. Therefore, the three indexes all meet the conditions of image non-distortion that the mean square error was smaller, the peak signal-to-noise ratio was larger, and the difference of one-dimensional entropy was also smaller. Thus, there was no artificial distortion between the enhanced images and the original images, and subsequent processes of the image registration and image fusion can be carried out.
To prove that the proposed algorithm in this paper can effectively eliminate many mismatching points,
Figure 7 and
Figure 8 respectively show the image registration effect of two-group images processed by the I-RANSAC algorithm [
12] and the IIR algorithm.
Compared to the I-RANSAC algorithm, in
Figure 7 and
Figure 8, the number of lines for matching points in the registration images by the IIR algorithm was greatly reduced, effectively avoiding some mismatching points. To quantitatively prove that the IIR algorithm can simultaneously improve the accuracy and speed in the process of image registration,
Table 2 shows the specific information of feature points for the two-group images in image registration, and
Table 3 shows a comparison of the matching accuracy and matching time among five algorithms, i.e., ORB, FAST, SURF, I-RANSAC, and IIR.
Compared with the I-RANSAC algorithm, the IIR algorithm reduced the number of feature points and the number of consistent matching pairs, as shown in
Table 2. The reason for this is that the improved algorithm is more rigorous in the selection of feature points and more accurate in the solution of the Homography matrix, which effectively eliminates the mismatching pairs. Moreover, accuracy and rapidity are two important indexes for image registration, which will be displayed as matching accuracy and matching time. According to
Table 3, when compared with the I-RANSAC algorithm alone, the IIR algorithm can greatly improve the matching accuracy and matching time. Regarding the average matching accuracy and average matching time of the two-group images, the IIR algorithm improved the average matching accuracy by 3.54% and reduced the average matching time by 4.30%. When compared with the existing four algorithms of ORB, FAST, SURF, and I-RANSAC, the IIR algorithm was close to the four existing algorithms in matching time but it significantly improved the matching accuracy. The average matching accuracy was 92.12%, which was the best among the five algorithms.
Considering the randomness of the two groups of experimental images randomly selected, the experimental results may be unconvincing. Therefore, this paper conducts multiple groups of experiments to verify the effectiveness of the IIR algorithm by obtaining the mean (
) and standard deviation (
).
Table 4 shows the specific matching accurate information of the experimental images of different groups.
As shown in
Table 4, with the increases of the number of groups, in IIR algorithm, the average of matching accuracy takes the first place, while at the same time, the standard deviation of matching accuracy is the smallest, among the five algorithms. So, it is obvious that the IIR algorithm is effective and stable in image registration.
To further prove that the IIR algorithm proposed in this paper can also partly improve the image fusion effect,
Figure 9 and
Figure 10 show the fusion effect of two-group images by the method of weighted smoothing after image registration through the I-RANSAC algorithm [
12] and IIR algorithm.
As shown in
Figure 9 and
Figure 10, the same method of weighted smoothing was adopted to image fusion, but the final results were different. Compared with the I-RANSAC algorithm, the image fusion effect by the IIR algorithm is better because image seams disappear in the fusion images. The reason for this is that the IIR algorithm can eliminate more mismatching points in image registration, which improves the percentage of accurate matching pairs. Therefore, the fusion effect has also been improved. According to the above experimental results, the following conclusion can be drawn: the proposed IIR algorithm can simultaneously improve the matching accuracy and matching time.
Due to the sharp increase of the research cruises, together with the large amounts of images and video recordings for underwater topography, it is a great demand for image mosaic and processing. We have demonstrated the effectiveness of the IIR algorithm in the Oxford Buildings Dataset and then applied it in underwater scenarios. The proposed algorithm has been used to study underwater image mosaic. Two-group images (four images) with low contrast and low definition were randomly selected as experimental objects from the image dataset of manganese nodules, as shown in
Figure 11 and
Figure 12. Specifically, four images were segmented from the two original images. The purpose of that is to obtain the experimental images with similar sizes and attributes and make the appropriate overlap area between the reference image and be-registered image. Then, two-group images were named T3 and T4, respectively; and the original images of each group similarly included the reference image and be-registered image, which corresponded to Sub-figure (a) and Sub-figure (b).
Similar to the process of T1 and T2,
Figure 13 and
Figure 14 show the enhancement effect of the two-group underwater images. Also, the information of three image quality evaluation indexes corresponding to each group before and after the process of image enhancement is given in
Table 5.
Compared with
Figure 11 and
Figure 12, the image contrast and definition in
Figure 13 and
Figure 14 are significantly improved, which can be intuitively sensed by the naked eye. A quantitative analysis of the data in
Table 5 shows that the average mean square error for the reference image and the be-registered image were 0.56 and 0.47, respectively; the average peak signal to noise ratio for the reference image and be-registered image were 56.33 and 56.71, respectively; and the average one-dimensional image entropy for the reference image and the be-registered image both were 0.07. Therefore, the three indexes better meet the conditions of image non-distortion that the mean square error was smaller, the peak signal-to-noise ratio was larger, and the difference of one-dimensional entropy was smaller as well. Thus, the subsequent processes of image registration and image fusion can also be carried out.
To prove that the proposed algorithm in this paper can also effectively eliminate many mismatching points on underwater images,
Figure 15 and
Figure 16 respectively show the image registration effect of two-group images processed by the I-RANSAC algorithm [
12] and the IIR algorithm.
As shown in
Figure 15 and
Figure 16, compared with the I-RANSAC algorithm, the number of lines for matching points in the registration images by the IIR algorithm was reduced, effectively avoiding some mismatching points. To quantitatively prove that the IIR algorithm can simultaneously improve the accuracy and speed in the process of image registration,
Table 6 shows the specific information of the feature points for the two-group images in image registration.
Table 7 shows a comparison of the matching accuracy and matching time among the ORB, FAST, SURF, I-RANSAC, and IIR algorithms.
Compared with the I-RANSAC algorithm, the IIR algorithm reduced the number of feature points and the number of consistent matching pairs, as shown in
Table 6. The reason for this is that the improved algorithm is more rigorous in the selection of feature points and more accurate in the solution of the homology matrix, which effectively eliminates the mismatching pairs. Moreover, according to
Table 7, when compared to the I-RANSAC algorithm alone, the IIR algorithm can improve the matching accuracy and matching time. Regarding the average matching accuracy and average matching time of the two-group images, the IIR algorithm improved the average matching accuracy by 3.54% and reduced the average matching time by 4.30%. Compared to the existing four algorithms (ORB, FAST, SURF, and I-RANSAC), the IIR algorithm is close to the four existing algorithms in matching time but significantly improved the matching accuracy. The average matching accuracy was 92.12%, which was the best among the five algorithms.
To further prove that the IIR algorithm can also partly improve the image fusion effect,
Figure 17 and
Figure 18 show the fusion effect of two-group images by the method of weighted smoothing after image registration through the I-RANSAC algorithm [
12] and IIR algorithm.
The same method of weighted smoothing was adopted for image fusion, but the final results are different, as shown in
Figure 17 and
Figure 18. Compared to the I-RANSAC algorithm, the image fusion effect by the IIR algorithm was better because the image seams disappeared in the fusion images. The reason for this is that the IIR algorithm can eliminate more mismatching points in image registration, which improves the percentage of accurate matching pairs. Therefore, the fusion effect is also improved. Therefore, the algorithm in this paper also has a good application in underwater image mosaic, which cannot only improve the accuracy and rapidity but also improve the image mosaic quality.