6.2. Accuracy Assessment
Assessment of the mosaicking results can be done in two aspects: spatial and spectral accuracy. Higher spatial accuracy means the mosaicking leads to less geometric distortion while higher spectral accuracy means the mosaicking operation can keep better the spectral information from original images at each pixel location.
A SPOT 5 satellite image of Beitun city was used as a reference for assessing spatial accuracy. The satellite image is the 2.5 m resolution SPOT full-color image, captured on 21 November 2014. The matching points are selected manually and used to calculate the spatial root mean square error (RMSE). The matching points are illustrated in
Figure 15.
The comparison in
Table 5 shows that the proposed method has a smaller RMS error than the Pix4Dmapper, even if the latter should theoretically have higher spatial accuracy. This is because the 3D reconstruction model has more DOFs. Thus, it is more sensitive to matching noise. In local areas, the reconstruction process may be unstable due to the reasons like wrong matches or the rolling shutter effect. Since it has experienced simplification, our proposed model is more stable with the quality variation of the data. For the dataset-popcorn, despite the fact that the Pix4Dmapper generated a visually satisfactory result, the point cloud of the result has considerable distortion (
Figure 16) and the calibrated camera positions also have major distortion because the 3D reconstruction algorithm is more sensitive to image data quality. Without additional information such as the Ground Control Points (GCPs), the error will accumulate and finally form the arc of the point cloud shown in
Figure 17, especially with datasets that have low texture such as the dataset-popcorn. The lack of matched point pairs will reduce the chance to correct such errors in the 3D-reconstruction process. On the contrary, the proposed method is more robust in such circumstances because of the hypothesis of planar ground and flying trajectory.
To evaluate the spectral accuracies of the image mosaic results, a comparative method based on feature matching is adopted. Because it is difficult to pick up sufficient points manually to perform the spectral comparison. Thus, we just chose several original images from the datasets and matched them with the two mosaicking results. The average RMS error of all the bands for one pixel is calculated based on the following equation:
where
is the total number of the image bands,
and
is the gray value of the
band at a matching point, and the
are the total number of the image bands. In this case, the
is equal to six.
To keep the same point set used to make the comparison, the point pairs that correspond to the same points on the original images were used to perform the comparison, i.e., the point pairs linked by solid lines in
Figure 18. The positions of the original images in the mosaicking results are shown in
Figure 19:
Table 6 shows the RMS errors of spectra at the seven areas on the mosaic images generated by the proposed method and the Pix4Dmapper. Z-Tests are performed to estimate the significance of the diversity of the two sets of the RMS errors. The result indicates that the proposed method has a smaller RMS error than the Pix4Dmapper in vegetated areas such as farmlands. In the bare soil area (site No.6) the two results have similar performances (
). Moreover, in the urban area (site No.3), the Pix4Dmapper performed better than the proposed method, but the diversity is also not significant. In sites No.1, 2, 4, and 7, the proposed method performs better than the Pix4Dmapper, and the diversity has major significance (
). In site No.5, the performance of the Pix4Dmapper is significantly better than the proposed method. Considering the proposed method mainly aims to be used in the farming areas, the result is acceptable. The spectral distortion of the mosaicking results is mainly caused by the resample process and the blending process. For the proposed method, an image will be resampled using the same translation and rotation parameters. On the contrary, the Pix4Dmapper will resample the pixels according to the 3D reconstruction results, indicating the resampling parameters will vary in different parts of the image, and this process will bring more spectral distortion. Considering the agricultural areas always have a smaller elevation variation, the simplified projection model of the proposed method may have great potential in agriculture applications.
For most 3D reconstruction methods, the feature points are used as reference points, which will have a lower precision loss in the resampling process. Meanwhile, the proposed method applies the same rotation parameter to one single image, which means such a comparison is not in favor of the proposed method. Even in such a comparison, the proposed method performs better than the state-of-the-art commercial software in agricultural lands. We can conclude that the proposed method can realize a lower precision loss in the spectral properties of the target.
The spectral distortion reduced by the proposed method can be discussed from two aspects. From one aspect, the proposed method completed the mosaicking of the two datasets in a rather shorter time than the Pix4Dmapper. This indicates that our proposed method may be a more efficient solution to conduct UAV image mosaicking in some emergency response applications such as disaster monitoring. From the other aspect, the spectral distortion brought by the blending process is not adequately studied in this research. Thus, a blending algorithm with less spectral distortion needs to be studied in the future. The lower spectral distortion with a conventional blending algorithm also indicates that the modified projection model is necessary to reduce the spectral distortion during the mosaicking process; thus, with a better blending algorithm, the spectral distortion may be further reduced.