An Optimal Image–Selection Algorithm for Large-Scale Stereoscopic Mapping of UAV Images

Abstract

Recently, the mapping industry has been focusing on the possibility of large-scale mapping from unmanned aerial vehicles (UAVs) owing to advantages such as easy operation and cost reduction. In order to produce large-scale maps from UAV images, it is important to obtain precise orientation parameters as well as analyzing the sharpness of they themselves measured through image analysis. For this, various techniques have been developed and are included in most of the commercial UAV image processing software. For mapping, it is equally important to select images that can cover a region of interest (ROI) with the fewest possible images. Otherwise, to map the ROI, one may have to handle too many images, and commercial software does not provide information needed to select images, nor does it explicitly explain how to select images for mapping. For these reasons, stereo mapping of UAV images in particular is time consuming and costly. In order to solve these problems, this study proposes a method to select images intelligently. We can select a minimum number of image pairs to cover the ROI with the fewest possible images. We can also select optimal image pairs to cover the ROI with the most accurate stereo pairs. We group images by strips and generate the initial image pairs. We then apply an intelligent scheme to iteratively select optimal image pairs from the start to the end of an image strip. According to the results of the experiment, the number of images selected is greatly reduced by applying the proposed optimal image–composition algorithm. The selected image pairs produce a dense 3D point cloud over the ROI without any holes. For stereoscopic plotting, the selected image pairs were map the ROI successfully on a digital photogrammetric workstation (DPW) and a digital map covering the ROI is generated. The proposed method should contribute to time and cost reductions in UAV mapping.

1. Introduction

Unmanned aerial vehicles (UAVs) were initially developed for military uses, such as reconnaissance and surveillance. Their utilization has been expanded to industrial and civilian purposes, such as precision agriculture, change detection and facility management. UAVs are equipped with various sensors, such as optical and infrared sensors. Optical sensors installed in UAVs are small non-metric digital cameras due to their limited weight and power. UAVs usually capture images at higher resolutions than manned aerial vehicles due to their low flight altitudes. They also acquire images rapidly. Due to these advantages, the mapping industry has developed a great interest in large-scale map production using UAVs [1,2,3].

For large-scale mapping, it is important to have precise orientation parameters (OPs) for UAV images. OPs can be classified into interior orientation parameters (IOPs) and exterior orientation parameters (EOPs). IOPs are related to imaging sensors mounted on UAVs, such as focal length and lens distortion. They can be estimated through camera calibration processes [4,5,6]. EOPs are related to the pose of the UAVs during image acquisition, such as position and orientation. Initial EOPs are provided by navigation sensors installed in the UAV. They are refined through bundle adjustment processes [7,8]. These techniques are also included in most commercial UAV image processing software.

For large-scale mapping, it is important to select optimal images to be used. UAVs need a large number of images to cover a region of interest (ROI), often with a high overlap ratio. One needs to cover the ROI with the fewest possible images. Otherwise, to map the ROI, one may have to handle too many UAV images. For this reason, stereo plotting of UAV images in particular is time consuming and costly.

Previous research on large-scale mapping using UAVs dealt with mapping accuracy analysis [9], accuracy in 3D model extraction [10,11,12] and image mosaicking [13,14,15,16]. On the other hand, studies on reducing the number of images for mapping or stereoscopic plotting were difficult to find. We believe there is a strong need for research on how to select optimal UAV images for stereoscopic mapping in particular.

This study proposes an intelligent algorithm to select a minimum number of image pairs that can cover the ROI, or to select a minimum number of the most accurate image pairs to cover the ROI. UAV images are grouped into strips and the initial image pairs are formed from among images within the same strip. We then apply an intelligent scheme to iteratively select the optimal image pairs from the start to the end of an image strip. Through the proposed image selection method, the ROI can be covered by either the minimum number of image pairs or the most accurate image pairs. The selected image pairs produce a dense 3D point cloud over the ROI without any holes. For stereoscopic plotting, the selected image pairs successfully mapped the ROI on a digital photogrammetric workstation (DPW), and a digital map covering the ROI was generated.

2. Proposed Method

The steps of the proposed algorithm are shown in Figure 1. Images are first grouped into strips. In this paper, we assume that images with bundle adjustment are taken in multiple linear strips. We need this process to confine pair selection and stereoscopic mapping to images within the same strip. Secondly, overlaps among images within the same strip are calculated. Any two images with overlapping regions are defined as initial image pairs. For initial pair composition, one may impose geometric constraints such minimum overlap ratio or Y-parallax limits. The Y-parallax is a vertical coordinate difference between left and right tie points mapped from an origin image to an epipolar image through epipolar resampling. The closer the Y-parallax is to zero, the higher the accuracy [17]. Next, optimal image pairs are determined by an iterative pair-selection process. Among images paired with the first image within a strip, a first optimal pair is selected by using prioritized geometric constraints. Images paired with subsequent images within the strip are checked against the pair selected first. All image pairs overlapping the selected pair are candidates for the next optimal pair. We select the next pair according to specific criteria. All other pairs are considered redundant and are removed from further processing. Selection is repeated with the new optimal pair until reaching the end of the image strip.

2.1. Image Grouping into Strips

For optimal image-pair selection, images are first grouped into strips. In this paper, we develop a scheme for image grouping based on linear strips. This scheme needs to be adapted to other types, such as circular strips.

Images are grouped into strips by comparing the EOPs among successive images in a sequence. Figure 2 shows an example of image grouping and its result. The first image within a sequence is defined as a starting image. An image strip is formed by the first and second images of the sequence. Next, azimuth $A_s^i$ is calculated between the starting image and the i-th image, which is the last image of the strip. Similarly, the azimuth between the i-th image and image i + 1 ($A_i^{i+1}$) is calculated. The formulas for $A_s^i$ and $A_i^{i+1}$ are

$$A_s^i={\tan }^{-1}\left(\frac{Y_i-Y_s}{X_i-X_s}\right)$$

(1)

$$A_i^{i+1}={\tan }^{-1}\left(\frac{Y_{i+1}-Y_i}{X_{i+1}-X_i}\right)$$

(2)

In the above equations, $X_s$ and $Y_s$ are the coordinates of the starting image, $X_{i }$ and $Y_i$ are coordinates for the i-th image and $X_{i+1 }$ and $Y_{i+1}$ are for image i + 1. If the difference between $A_s^i$ and $A_i^{i+1}$ is less than a set threshold, we include image i + 1 in the same group as the starting image. In this case, the image-grouping process continues with the starting image, image i + 1 and image i + 2. When the angle difference is more than the threshold, we check the angle between the i-th image and image i + 2 ( $A_s^{i+2}$ ). This is to avoid undesirable effects from accidental drift by the UAV. If the difference between $A_s^i$ and $A_i^{i+2}$ is also more than the threshold, the i-th image is defined as the final image of the current image group. In this case, image i + 1 is defined as a new image of a new image group. The image grouping process restarts with the new starting image.

After the image grouping process is completed for the entire image sequence, image groups with a small number of images are assumed to be outside the ROI and can be removed from further processing. The bottom image in Figure 2 shows the grouping result. In this example, the second group will be removed.

2.2. Initial Image Pair Composition

Once images are grouped into strips, overlaps between adjacent images within the same strip are estimated. First, the ground coverage of individual images is calculated by projecting image corner points onto the ground reference plane though the collinear equations in (3) and (4) below.

$$\left[\begin{array}{c}{X}_P-X_O\\ Y_P-Y_O\\ Z_P-Z_O\end{array}\right]=\lambda \left[\begin{array}{ccc}{r}_{11}& r_{12}& r_{13}\\ r_{21}& r_{22}& r_{23}\\ r_{31}& r_{32}& r_{33}\end{array}\right]\left[\begin{array}{c}{x}_p\\ y_p\\ -f\end{array}\right]$$

(3)

In the collinear equations, X_p, Y_p and Z_p are the object coordinates of point P on the ground, while X_O, Y_O and Z_O denote the position of the camera projection center; $\lambda$ is a scale constant; x_p and y_p are the coordinates of p in the image, f is the focal length and r₁₁ to r₃₃ are rotation matrix parameters calculated from orientation angles. Z_p is the approximated height of the ground reference plane. It can be calculated by the average height of ground control points (GCPs), if any, or by the average height of tie points.

$$X_P=\frac{r_{11}{x}_p+r_{12}{y}_p-r_{13}f}{r_{31}{x}_p+r_{32}{y}_p-r_{33}f}\left(Z_P-Z_o\right)+X_o$$

$$Y_P=\frac{r_{21}{x}_p+r_{22}{y}_p-r_{23}f}{r_{31}{x}_p+r_{32}{y}_p-r_{33}f}\left(Z_P-Z_o\right)+Y_o$$

(4)

$$Z_P= \mathrm{Height} \mathrm{of} \mathrm{Ground} \mathrm{Reference} \mathrm{Plane}$$

Once individual image coverages are calculated, an overlapping area between two images is calculated based on the ground reference plane (Figure 3) [18,19,20]. As shown in Figure 3, the coverage of the reference image and that of the target image define the reference polygon and target polygon, respectively. Vertices of overlap polygon are defined by the corner points of corresponding images and the intersection of the reference polygon and target polygon. Once the overlap included in the vertices is calculated, the polygon is defined atop the overlap for further processing.

If there is any overlap between two images, the two images are defined as an initial image pair. In this process, one may apply a minimum overlap ratio (20%, for example) as a limit for initial image-pair selection. One may also impose stereoscopic quality measures for initial image-pair selection. Y-parallax errors among image pairs are good candidates for measurement [11,18,19], and can help explain how well two images will align after epipolar rectification. Convergence angles are also good candidates. When they are too small or too large, two images are likely to form weak stereo geometry and bring accuracy degradation [20,21]. Additional parameters, such as asymmetric and bisector elevation angles [20,21], can be considered, although they may not be dominant factors for stereoscopic quality in UAV images.

Figure 4 shows an example of initial image pairs in a strip with six images. The ground coverage of the first image (1) is checked with that of subsequent images. Image 1 overlaps images 2, 3 and 4. It does not overlap 5 or 6 at a ratio of more than 20%. An image-pair set for image 1 (Pair-Set 1) is defined comprising image pairs 1–2, 1–3 and 1–4. Similarly, image-pair sets are defined for images 2, 3, 4 and 5. There are 12 image pairs in total for the image strip. A simple selection of image pairs from among the 12 defined pairs could be done by choosing one image pair per Pair-Set. For example, one can select pairs with neighboring images within the image sequence, and five image pairs will be selected. However, this selection may contain redundancies, and may not offer the best accuracy. We need a more intelligent strategy for pair selection.

2.3. Optimal Pair Selection

From the initial image pairs, optimal image pairs can be selected to cover the ROI according to the desired selection criterion. In this paper, we describe two such criteria: minimum pair selection and most accurate pair selection. One may apply other criteria according to individual requirements.

The process starts by selecting a reference pair from among image pairs in the Pair-Set of the starting image of a strip (e.g., image 1 in Figure 4). One may select the pair based on stereoscopic quality measures. For example, one can select the pair with the minimum Y-parallax error and with a convergence angle within an acceptable range. Figure 5 shows an example of reference pair selection and the forthcoming optimal pair–selection process. In the Figure, image pair 1-2 is selected as the reference pair from Pair-Set 1.

Next, image pairs that overlap 1–2 are sought from the next Pair-Set (Pair-Set 2) to the last Pair-Set (Pair-Set 5). Note that, in this case, overlaps are sought not from the coverage of individual images but from pairs (Figure 6). The overlapping region of the reference pair and an image pair within the Pair-Sets define the reference pair polygon and target pair polygon, respectively.

Whether or not the overlap of two pair polygons is calculated for covering the ROI without leaving any inner holes through cross points by applying a point-in polygon algorithm [22]. As shown in Figure 6, the coverage of the reference pair and that of the target pair define the reference polygon and target pair polygon, respectively. Vertices of each polygon have been defined via the initial image-pair selection process, as illustrated in Figure 3. If two polygons overlap, one of the vertex vectors of the target pair polygon must cross over the line of the reference pair polygon. By using the vertex vectors, we generate a cross point.

Figure 5 shows the results of the overlapping-pair search based on the reference pair (Pair 1–2). Pair 1–2 overlaps 2–3, 2–4 and 3–4. These pairs are candidates for the next reference pair. One can impose selection criteria for the next reference pair selection. If one prefers to select the minimum number of pairs, all pairs in Pair-Set 2 are redundant because a pair in Pair-Set 3 overlaps the reference pair. Alternatively, one may select the pair with optimal geometric measures, such as the minimum Y-parallax error, from among all candidate pairs. As such, in Figure 5, Pair 3–4 is selected as the next reference pair.

Once the next reference pair is selected, the overlapping-pair search starts with the new reference pair. This process repeats until the last image within a strip is included as the next reference pair. Figure 7 shows the result of an overlapping-pair search for Pair 3–4. Pair 5–6 is selected as the next reference pair. The iteration is completed, since image 6 is the last image of the strip, and there are no remaining image pairs for an overlap search.

3. Experiment Results and Discussion

3.1. Study Area and Data

The study area selected was downtown Incheon, Korea, which has various roads, buildings and houses. A total of 362 UAV images were collected using a Firefly6 Pro. The number of strips created was 11, and the flight altitude of the UAV was 214 m. The ground sampling distance (GSD) for the images was 5.0 cm. For a precise bundle adjustment and accuracy check, five GCPs and four checkpoints (CPs) were obtained through a GPS/RTK survey.

3.2. Image Grouping and Initial-Pair Composition

The image grouping process was applied to the 362 images based on an angular threshold of 30 degrees. If the number of images within a strip was less than five, the images were removed from further processing.

Table 1. The results of image grouping into strips.

Strip	Start Index of Image	End Index of Image	Number of Images	Number of Pair-Sets	Number of Pairs
1	1	27	27	26	86
2	34	59	26	25	79
3	67	92	26	25	77
4	99	124	26	25	77
5	133	158	26	25	77
6	164	192	29	28	82
7	199	226	28	27	81
8	234	259	26	25	76
9	268	295	28	27	80
10	299	328	30	29	87
11	334	362	29	28	82
Total Number			301	290	884

shows the results of image grouping into image sequences. Start index and end index mean the starting and ending images of each strip, with 11 strips classified from 301 of the 362 images used for final composing of the strips. On average, 27 images were grouped within a strip, after 61 images were removed because they were on the boundaries of valid strips.

Figure 8 and Figure 9 show the results from total of flight path and image grouping. Compared to Figure 8, Figure 9a shows that images on the boundaries of strips were removed successfully. Removed images were mostly oblique, because the UAV changed pose for the next strip, as shown in Figure 9b–e.

Overlaps between images within an image strip were calculated, and initial image pairs were defined. We used 5 degrees and 45 degrees as the lower and upper limits for a valid range of convergence angles, an average magnitude of two pixels was the upper limit for allowable Y-parallax, and a ratio of 20% was the lower limit for overlaps between two images. The number of pair sets and initial-image pairs for all strips are shown in Table 1.

3.3. Optimal Image-Pair Selection

We separately applied optimal-pair selection based on two selection criteria. The first criterion was to select the minimum number of pairs covering the ROI. The second was to select image pairs with the best accuracy.

Table 2. Initial-image pairs and the results of optimal-image selection for the first strip (red font: the minimum pairs; underline: the best accuracy pairs). Each cell shows an image ID with overlap ratio and magnitude of Y-parallax error in parentheses.

Pair-Set 1	Pair-Set 2	Pair-Set 3	Pair-Set 4	Pair-Set 5	Pair-Set 6	Pair-Set 7
1–2 (66, 0.6)	2–3 (84, 1.0)	3–4 (85, 0.7)	4–5 (84, 0.4)	5–6 (72, 1.0)	6–7 (76, 0.6)	7–8 (73, 0.8)
1–3 (51, 0.7)	2–4 (68, 1.0)	3–5 (73, 0.7)	4–6 (58, 1.1)	5–7 (51, 0.7)	6–8 (50, 0.9)	7–9 (43, 0.8)
1–4 (38, 1.1)	2–5 (57, 1.1)	3–6 (46, 1.2)	4–7 (39, 0.6)	5–8 (30, 1.6)	6–9 (22, 1.0)	7–10 (22, 1.0)
1–5 (28, 3.2)	2–6 (36, 2.4)	3–7 (26, 0.8)
Pair-Set 8	Pair-Set 9	Pair-Set 10	Pair-Set 11	Pair-Set 12	Pair-Set 13	Pair-Set 14
8–9 (69, 0.8)	9–10 (77, 0.7)	10–11 (74, 3.8)	11–12 (72, 0.5)	12–13 (81, 0.4)	13–14 (76, 0.4)	14–15 (78, 0.7)
8–10 (47, 1.0)	9–11 (54, 2.9)	10–12 (49, 3.7)	11–13 (58, 0.5)	12–14 (59, 0.5)	13–15 (57, 0.7)	14–16 (51, 0.8)
8–11 (27, 0.8)	9–12 (30, 2.8)	10–13 (35, 3.3)	11–14 (39, 0.7)	12–15 (40, 0.8)	13–16 (33, 0.8)	14–17 (29, 1.0)
Pair-Set 15	Pair-Set 16	Pair-Set 17	Pair-Set 18	Pair-Set 19	Pair-Set 20	Pair-Set 21
15–16 (72, 0.4)	16–17 (76, 0.5)	17–18 (76, 0.7)	18–19 (79, 0.6)	19–20 (72, 0.5)	20–21 (78, 0.6)	21–22 (75, 0.8)
15–17 (48, 0.6)	16–18 (53, 0.7)	17–19 (58, 1.1)	18–20 (53, 1.1)	19–21 (54, 0.7)	20–22 (63, 0.8)	21–23 (56, 1.5)
15–18 (26, 0.6)	16–19 (35, 0.9)	17–20 (32, 1.9)	18–21 (36, 1.1)	19–22 (38, 0.9)	20–23 (36, 1.6)	21–24 (37, 1.1)
			18–22 (20, 2.2)		20–24 (25, 0.7)
Pair-Set 22	Pair-Set 23	Pair-Set 24	Pair-Set 25	Pair-Set 26
22–23 (69, 1.3)	23–24 (75, 1.0)	24–25 (71, 0.7)	25–26 (70, 1.7)	26–27 (74, 1.1)
22–24 (61, 0.7)	23–25 (48, 0.9)	24–26 (48, 1.7)	25–27 (48, 1.2)
22–25 (32, 0.8)	23–26 (33, 2.0)	24–27 (25, 1.8)

shows the results of the initial image–pair composition for Strip 1 in detail. Twenty-six pair sets were defined, and a total of 80 initial image pairs were formed. The table also shows the overlap ratio and Y-parallax error for each image pair. In Pair-Set 1, for example, image 1 had an overlap of 66% with image 2. The magnitude of the Y-parallax between images 1 and 2 was 0.6 pixels.

The optimal image–pair selection process was tested based on the first selection criterion. The starting pair was 1–2. The next pair selected was the pair overlapping 1–2 at the far-most pair set. The pairs selected based on this criterion were 1–2, 5–6, 7–8, 10–11, 13–14, 16–17, 19–20, 23–24 and 26–27 (shown in red). The selected optimal pairs are shown in the table in bold. Out of 26 pair sets, nine pairs were sufficient to cover the ROI of the strip.

Next, the optimal image–pair selection process was tested based on the second selection criterion. The magnitude of the Y-parallax between stereo images was used as the accuracy measure. The pairs selected under this criterion were 1–2, 4–5, 6–7, 9–10, 12–13, 13–14, 15–16, 16–17, 19–20, 20–21, 22–24, 24–25 and 26–27. They are underlined in the table. Out of 26 pair sets, 13 were selected as the most accurate.

The other 10 strips were processed separately using the same two selection criteria. In each strip, initial-image pairs were formed, and optimal pairs were selected.

Table 3. Results of optimal adjacent-, minimum-, and geometric-pair composition for each strip.

Strip	Case 1. Adjacent—Pair Selection		Case 2. Minimum—Pair Selection		Case 3. Most Accurate—Pair Selection
	Number of Pairs	Y-Parallax (Pixels)	Number of Pairs	Y-Parallax (Pixels)	Number of Pairs	Y-Parallax (Pixels)
1	26	0.80	9	0.90	13	0.61
2	25	0.61	9	0.56	12	0.51
3	25	0.70	8	0.88	13	0.55
4	25	0.61	11	0.76	14	0.53
5	25	0.77	7	0.80	12	0.61
6	28	0.70	10	0.60	12	0.50
7	27	0.74	9	0.65	14	0.58
8	25	0.75	9	0.72	15	0.64
9	27	0.75	10	0.80	15	0.66
10	29	0.74	10	0.55	15	0.51
11	28	0.71	9	0.84	12	0.63
Avg	26	0.71	9	0.73	13	0.58
Total	290	-	101	-	148	-

summarizes the results of optimal-pair selection. In reference to the proposed optimal selections, we include selecting all adjacent image pairs.

Table 3 shows the results of adjacent-pair selection, minimum-pair selection and most accurate–pair selection per strip. The total number of pairs under adjacent pair selection was 290 for all 11 image strips. This indicates that 290 pairs would be processed from among the original 362 images without any intelligent pair-selection process. The average magnitude for Y-parallax for this case was 0.71 pixels.

The total number of pairs from the minimum-pair selection was 101 for all strips. Compared to adjacent-pair selection, the number of pairs was reduced greatly. A significant amount of time can be saved in stereoscopic processing of the dataset under this selection criterion. The average magnitude for Y-parallax in this case was 0.73, slightly less than the first case.

The total number of pairs from the most accurate–pair selection was 148 for all strips. Compared to minimum-pair selection, the number of pairs increased. However, this is still a great reduction, compared to adjacent-pair selection. It is notable that the average Y-parallax in this case was 0.58 pixels. Stereoscopic processing in this case should generate the most accurate results with a reduced number of pairs.

3.4. Performance of Stereoscopic Processing

Digital surface models (DSMs) and ortho-mosaic images were generated from all three cases to evaluate the performance of the proposed optimal-pair selections. For DSM generation, a stereo-matching algorithm developed in-house was used [22,23]. Stereo matching was applied to all stereo pairs in each case, and 3D points from stereo matching were allocated to DSM grids. Points in each DSM grid were averaged. Ortho-mosaics were generated using the DSMs from the three cases. For each ortho-mosaic grid, a height value was obtained from the DSM. This 3D point was then back-projected to multiple original UAV images. We chose the back-projected image pixel that was closest to the image center as the corresponding image pixel to the ortho grid.

Figure 10 shows the results of DSM and ortho-mosaic generation. In it, parts (a) and (b) were generated via adjacent-pair selection, (c) and (d) by minimum-pair selection and (e) and (f) with most accurate–pair selection. All three datasets covered the ROI without leaving any inner holes. Slight shrinking of the processed areas was observed for the second and third datasets. However, shrunken regions were due to removed image pairs, which were outside the ROI.

Table 4. Resulting processing times for all types of pair selection.

Type of Product	Case 1. Adjacent—Pair Selection	Case 2. Minimum—Pair Selection	Case 3. Most Accurate—Pair Selection
Stereo matching	100 min	38 min	57 min
DSM and Ortho-mosaic	15 min	7 min	9 min
Total processing time	115 min	45 min	66 min

shows the processing times for all types of pair selection. Case 1, which had the largest number of pairs, took the most time. On the other hand, Case 2 with the smallest number of pairs took the least time. As the number of pairs was reduced, processing time was also reduced.

For accuracy analysis, we overlapped the GCPs which were used during bundle adjustment, and the CPs, which were not used during bundle adjustment, onto the ortho-mosaic maps and the DSMs of the three datasets. Horizontal and vertical accuracy were analyzed, and

Table 5. Accuracy analysis results of DSMs and ortho-mosaic images.

Accuracy	Case 1. Adjacent Pair Selection		Case 2. Minimum Pair Selection		Case 3. Most Accurate Pair Selection
	Horizontal (cm)	Vertical (cm)	Horizontal (cm)	Vertical (cm)	Horizontal (cm)	Vertical (cm)
Model	8.4	25.2	5.5	22.8	6.1	7.4
Check	8.2	18.3	7.2	24.2	5.9	7.5

shows the results, in which the row labeled Model shows the accuracy of the GCPs, whereas the row labeled Check shows the accuracy of the CPs.

The results in Table 5 support the image-selection strategy proposed here. Note that all three cases used the same EOPs, processed with Pix4D software. Model accuracy and Check accuracy were in a similar range for all three cases. This indicates that the modeling process by Pix4D did not clear any bias. The results in Case 1 and Case 2 are in a similar range. This indicates that the minimum pair–selection criterion could successfully replace adjacent-pair selection. It is promising that the accuracy in Case 3 was lower than the others. The proposed criteria for most accurate–pair selection have been proven influential.

A digital map at 1:1000 scale was generated through stereoscopic plotting with the dataset from most accurate–pair selection [24]. As shown in Figure 11, large-scale maps could be generated successfully with a reduced number of images.

Table 6. Most accurate pair composition result for each region.

Study Region	Type of UAV	Number of Strip	Number of Input Images	Number of Most Accurate Pairs	Y-Parallax RMSE (Pixel)
Agricultural	Fixed-wings	6	59	18	0.84
Costal	Rotary-wings	6	95	26	0.88
Agricultural	Rotary-wings	6	51	15	0.55

shows the result of most accuracy pairs for agricultural and costal region by using fixed and rotary wings. It was confirmed that the number of most accurate pairs in all regions was reduced by about one-third compared to the input pairs. In addition, the RMSE of Y-parallax was measured to be less than 1.0 pixel, which makes it possible to generate stable stereoscopic plotting.

4. Conclusions

In this study, we proposed an intelligent pair-selection algorithm for stereoscopic processing of UAV images. We checked overlaps between stereo pairs and Y-parallax errors in each stereo pair, and we selected image pairs according to predefined selection criteria. In this paper, we defined two criteria: the minimum number of pairs and the most accurate pairs, because it is difficult to find other methods such as optimal pair composition paper. Other selection criteria are open, depending on users’ needs.

Both criteria tested could reduce the number of image pairs to cover the ROI, and hence, reduce processing time greatly. This reduction did not lead to any image loss of the processed area. Through accuracy analysis using ground control points, we verified the validity of the proposed criteria. The minimum-selection criterion showed accuracy similar to conventional adjacent-pair selection with a large redundancy. This shows that the ROI can be successfully covered with a minimum number of pairs, and that the minimum-selection criterion could replace adjacent-pair selection. Notably, the criterion proposed for most accurate–pair selection greatly reduced horizontal and vertical errors, compared to both adjacent- and minimum-pair selection. This shows that the proposed criterion could indeed be used for accurate and cost-effective stereoscopic mapping of UAV images.

Due to data availability, we used images taken in linear strips. We also assumed that the image sequence was ordered according to the image acquisition time. One may adapt our image grouping method to handle other types of strips or unordered images. In this paper, we considered image-pair reduction only within the same strip. An interesting research topic would be to extend the image-selection criteria to pair images from different strips.