An Effective Approach for Automatic River Features Extraction Using High-Resolution UAV Imagery

Abstract

The effects of climate change are causing an increase in the frequency and extent of natural disasters. Because of their morphological characteristics, rivers can cause major flooding events. Indeed, they can be subjected to variations in discharge in response to heavy rainfall and riverbank failures. Among the emerging methodologies that address the monitoring of river flooding, those that include the combination of Unmanned Aerial Vehicle (UAV) and photogrammetric techniques (i.e., Structure from Motion-SfM) ensure the high-frequency acquisition of high-resolution spatial data over wide areas and so the generation of orthomosaics, useful for automatic feature extraction. Trainable Weka Segmentation (TWS) is an automatic feature extraction open-source tool. It was developed to primarily fulfill supervised classification purposes of biological microscope images, but its usefulness has been demonstrated in several image pipelines. At the same time, there is a significant lack of published studies on the applicability of TWS with the identification of a universal and efficient combination of machine learning classifiers and segmentation approach, in particular with respect to classifying UAV images of riverine environments. In this perspective, we present a study comparing the accuracy of nine combinations, classifier plus image segmentation filter, using TWS, also with respect to human photo-interpretation, in order to identify an effective supervised approach for automatic river features extraction from UAV multi-temporal orthomosaics. The results, which are very close to human interpretation, indicate that the proposed approach could prove to be a valuable tool to support and improve the hydro-geomorphological and flooding hazard assessments in riverine environments.

1. Introduction

The effects of climate change, such as sea level rise, more intense precipitation, and higher river discharge, as well as more intensive urbanization in flood-prone areas, are causing an increase in the frequency, areal extent and magnitude of flooding events [1].

Flooding is one of the most disruptive natural disasters, generating harmful impacts on human lives and economic activities [2,3]. In fact, in the past decade, about 1 billion people have been directly or indirectly affected by floods and about 100,000 have lost their lives [4].

From this point of view, rivers are included among the most critical factors, as they frequently alter their morphological characteristics leading to catastrophic flooding due to overflow caused both by variations in discharge as a response to heavy rainfall and riverbank failures [5,6,7].

The monitoring of river flood events represents a crucial task that is currently addressed through the application of different methodologies ranging from topographical field surveys to monitor river conditions [8,9], to the use of satellite imagery to study river dynamics and generate forecast maps [10,11,12,13] or to detect and monitor flood-prone areas [14,15,16], to the development of hydraulic and hydrodynamic models and numerical simulations [17,18,19,20].

The leading emerging methodologies for monitoring river conditions include the combination of Unmanned Aerial Vehicles (UAV) and photogrammetric techniques (i.e., Structure from Motion-SfM), which overcomes the intrinsic limitations of satellite- and airborne-based imagery and in situ traditional surveys [21], ensuring the on-demand acquisition of very high-resolution spatial data over wide areas and the generation of orthomosaics used for automatic hydro-geomorphological feature extraction [22,23,24,25,26].

The automatic extraction of river features from UAV images proves to be a key aspect in the management of flood-prone environments especially in emergency scenarios, as it ensures a reduction of time in the image analysis and assures a rapid identification of flood-affected areas immediately after the calamitous event [27,28]. Furthermore, rapid and accurate damage assessment facilitates the intervention of military corps and civil protection and enables the implementation of recovery plans and insurance estimates.

The most widely used automatic image features extraction methodologies include Machine-Learning classification algorithms, that are (1) generally able to model complex class signatures, (2) accept a variety of predictive input data, and (3) do not make assumptions about data distribution [29]. Several studies address the automatic feature extraction, from images of different sources, in alluvial environments, such as water, terrain, vegetation, buildings, etc., using supervised and unsupervised classification techniques. Several classifiers were applied, such as Support Vector Machines (SVMs), Artificial Neural Networks (ANNs), Fully Convolutional Networks (FCN), Random Forests (RFs), K-Means, k-Nearest Neighbor (kNN), and edge detection filters (Gaussian, Sobel), on satellite and UAV images (i.e., [30,31,32,33,34,35,36,37,38,39]).

In order to extract features of interest, several open-source image processing tools provide full accessibility to the source code and the ability to implement specific (pixel-based or object-based) classification and segmentation workflows (i.e., [40,41,42,43]).

Among these open-source tools, Trainable Weka Segmentation (TWS) [44] includes an extensive collection of machine learning and image processing algorithms for data mining tasks (training a classifier and segmenting the remaining data automatically). These algorithms can be used interactively as plug-ins of the user-friendly interface, Fiji—ImageJ, which provides a simple macro language, and the ability to record GUI commands and quickly develop custom-made codes [45]. TWS was developed primarily to fulfill supervised classification purposes of biological microscope images, and its usefulness has been demonstrated in several image pipelines that involve different segmentation tasks, such as monitoring nests of bees [46], analyzing wing photomicrographs [47], visualizing myocardial blood flow [48], extracting urban road features [49,50], building health monitoring [51], and analysis of grasslands species composition [52]. At the same time, to the best of our knowledge, there is a lack of studies in the literature on the applicability of TWS for classifying and segmenting UAV images, in particular in riverine environments, with the aim of extracting the water class and improving the monitoring activities.

The wide range of Machine Learning and image-processing algorithms of TWS requires detailed analysis, in order to devise the most efficient combination of machine learning classifier and segmentation approach for the specific task of depicting the features of riverine environments.

Here, a study is presented that compares, by means of TWS, the accuracy (overall accuracy, misclassification rate, precision, recall, F-measure and kappa statistic) of different combinations of classifiers (Random Forest—RF, BayesNet—BN, k—Nearest Neighbors—kNN) and image segmentation filters (Sobel Filter-SF and Difference Of Gaussians-DoG) compared also to human photo-interpretation. The goal is to identify an effective open-source supervised approach for automatic water-class extraction from UAV multitemporal orthomosaics of wide areas of riverine environments and improve the flooding hazard assessment.

2. Data and Methods

2.1. Case Study

The study area is located along the hilly plain of the Basento river near the industrial area of Ferrandina (MT), in the Basilicata region, southern Italy (Figure 1). The river, locally referred to as “Fossa Cupa”, rises at Monte Arioso (1709 m ASL), in the southern Apennine Mountains and occupies a catchment of about 1546 km² with an NW–SE trending valley of 150 km length. It flows into the Ionian Sea crossing the Metaponto Coastal Plain (MCP) perpendicularly to the shoreline [53], at about 30 km SW of Taranto, near Metaponto.

The hydrological regime of the river follows the seasonal trend of the rainfall (mean annual rainfall about 766 mm with major events recorded between October and March), which shows the relationship between rainfall and river flow: maximum values between November and January, and minimum in July or August, with an average annual flow rate of 12.2 m³/s [54].

Specifically, the area under examination is subject to hydro-geomorphological high-risk: significant morphological changes have been recorded in the past [55], leading to several failures including the collapse of a road along its riverbank.

2.2. Data

In order to identify a universal and efficient automatic river features extraction approach from multitemporal aerial images, UAV image sequences of a river reach were collected in different seasons (June 2019, July 2020, October 2021 and March 2022), and with different weather conditions (sunny and cloudy). In this way, we considered a wide range of conditions that might affect the image quality (e.g., the reflectivity of water bodies, in sunny condition and the presence of shadows) and the river’s seasonal changes (e.g., flow rate and sediment load carried, which might affect water-color shades). In June 2019 and July 2020, river conditions in lean period were recorded, while in October 2021 and March 2022, river conditions during a flood period were recorded (following the hydrological regime of the river shown in Section 2.1).

The UAV images acquisition was carried out by using a DJI Inspire 2 (specifications at [56]) equipped with optical sensor DJI Zenmuse X5S (specifications at [57]), with a flying height of 50 m AGL of take-off location, nadir camera angle, images forward and side overlap of 85% and 65%, respectively. The UAV flight missions were set up with a single grid path using the “follow topography” option of Litchi (UAV flight mission planning app) to ensure a constant Ground Sampling Distance (GSD) of the images. Each flight mission consists of 9 strips (perpendicular to the main river path) with a length of about 160 m. The images were taken as RGB with a GSD of 1.09 cm/pix. In 2019 and 2021, about 3000 images were collected, in 2020 and 2022 about 4000 images.

The acquired datasets were processed using the Pix4D software v.4.7.5 (which follows the principles of SfM, as described in [58]), for generating RGB orthomosaics with an original resolution of 1.3 cm/pix (file size of about 6 GB) and positional accuracy ranging from 1 to 2 cm of RMSE values (using Manual Tie Points in addition to Ground Control Points uniformly distributed over the entire investigated area taking into account the geomorphological features). To analyze the resulting orthomosaics with TWS, due to the limited computing resources of our workstation (HP Z2 SFF G9 with 12 core Intel^® Core™ i7 12,700 12th Generation, RAM 32 GB DDR5 – HP Store, Italy), a down-sampling of the resolution to 17 cm/pix (file size of about 60 MB) was performed (Figure 2), as a result of a trial-and-error method to find the resolution needed to run the entire classification workflow without complications.

2.3. Methods

The methodology presented in this paper consists of a sequence of steps. First, UAV multitemporal RGB orthomosaics are generated (see Section 2.2). Then, the classes to be extracted are defined: water, terrain, dense vegetation and background. Next, some training data for each class (set of pixels marked manually by the user exclusively over the orthomosaic 2021, because of the high variability of weather conditions and color hues of the classes, see Figure 2) are selected, specifically 188 pixels, as input for the TWS pixel-based segmentation algorithm.

TWS was set up with nine specific combinations of classifiers and image filters (representing training features, which are the key to the machine learning procedure). Each classifier was combined with the filters: BN + SF, BN + DoG, BN + SF + DoG; RF + SF, RF + DoG, RF + SF + DoG; kNN + SF, kNN + DoG, kNN + SF + DoG.

SF is a gradient-based edge detection filter. By applying two kernels, it computes an approximation of the gradient of the image intensity at each point, finding the direction along which there is the maximum increase from light to dark, the rate at which the change occurs along this direction and the probability of that part of image to represent an edge [59].

DoG is an image feature enhancement filter used for increasing the visibility of the edges. The method consists of subtracting two Gaussian images, where a kernel has a standard deviation smaller than the previous one. The convolution between the subtraction of kernels and the input image results in the edge detection of the image [60].

RF is an ensemble classifier, which uses a randomly selected subset of training variables and samples to generate several decision trees. Each tree casts a unique vote for the most popular class to classify input data [61].

BN is a classifier network, which computes a probability that indicates how likely the object belongs to a class. Each node of the network (direct acyclic graph) has a conditional probability distribution, and the object is assigned to the class in which the highest probability was calculated. Because the probability is calculated in terms of all features, each feature is able to play a role in the classification process [62].

Moreover, kNN is a nonparametric supervised learning classifier, which uses proximity to make classification or prediction about the clustering of a single data point. It identifies the nearest neighbor of a given point by determining the Euclidean distance between the query point and other data points [34].

Once the classifiers have been trained (on the original image and on its filtered version) and saved for each combination, they are used to classify the rest of the input pixels for the 2021 orthomosaic and all image pixels for 2019, 2020 and 2022 orthomosaics, using the same training data.

The outputs resulting from the application of each combination are a classified image (showing all the classes identified) and probability maps (gray-scale images, containing the probability that each pixel belongs to each class). The probability maps containing exclusively the water class (the class of greatest interest in the river flood monitoring perspective) are then transformed into binary ones by using the automatic binary segmentation tool of Fiji. This tool analyzes the histogram of the entire image and divides it into foreground objects and background, by taking an initial threshold and then calculating the averages of the pixels below and above the threshold. The averages of those two values are computed, the threshold is incremented, and the process is repeated until the threshold is larger than the composite average: threshold = (average background + average objects)/2. In this way, the area of the segmented class can be derived from the binary image, by applying a threshold in Fiji to select foreground pixels (with 255 value), and compared to the area obtained by the user manual digitization of the class in GIS environment (assumed as ground truth). Specifically, the comparison of the water class area, derived from automatic classification and user digitization, is made by considering the water class area strictly covering the riverbed. Then the misclassified water class area, over the whole image of the probability maps, is determined. All the steps listed above are shown in Figure 3.

2.4. Accuracy Estimation

To estimate the accuracy of the applied method, (a) a measure of the performance of the classifier for each combination and, (b) a measure of the accuracy between the classified and digitized water class (orthomosaic human interpretation), were carried out.

(a)

In this accuracy estimation, a confusion matrix was used to determine the metrics for each combination: accuracy (ACC; TP and TN indicate True Positives and True Negatives; FP and FN indicate False Positives and False Negatives) expressed in % (1), misclassification rate (ERR) expressed in % (2), precision (PR) expressed in weighted average (3), recall expressed in weighted average (4), F-measure (FM) weighted average (5) and kappa statistic (kappa) with expected accuracy (Exp ACC) (6).

$$ACC=\frac{TP+TN}{TP+TN+FP+FN}$$

(1)

$$ERR=\frac{FP+FN}{TP+TN+FP+FN}$$

(2)

$$PR=\frac{TP}{TP+FP}$$

(3)

$$Recall=\frac{TP}{TP+FN}$$

(4)

$$FM=\frac{2\ast Recall\ast PR}{Recall+Precision}$$

(5)

$$kappa=\frac{ACC-Exp ACC }{1-Exp ACC}Exp ACC=\frac{\frac{{{\displaystyle \sum }}_{class=1}^n\frac{\left(TP+FP\right)\ast \left(TP+FN\right)}{TP+TN+FP+FN}}{TP+TN+FP+FN}}{TP+TN+FP+FN}$$

(6)

(b)

The percentage error (7) between classified and digitized water class area covering the riverbed was determined for each combination, to measure the accuracy of True Positive (TP) reconstruction.

$$Error=\frac{classified water class-digitized water class}{digitized water class}\times 100$$

(7)

Furthermore, the misclassified water class area over the whole probability maps (subtracted by the area covering the riverbed, TP) was calculated for each combination, to measure the accuracy considering the False Positives (FP). In order to compare the values for each orthomosaic due to their different size, the percentage ratio (pMWA) of the misclassified water class area to the total image area (i.e., all pixels of the orthomosaics with not null values) was determined (8).

$$pMWA=\frac{misclassified water class area}{image total area}\times 100$$

(8)

3. Results

In order to judge the performance of each combination (classifier plus image filter), with respect to the training data of each class marked exclusively on orthomosaic 2021, the metrics (see Section 2.4) were determined and compared, as shown in

Table 1. Performance metrics values for each combination (classifier plus image filter).

.	RandomForest			BayesNet			k-Nearest Neighbor
Performance Measure	SF	DoG	SF + DoG	SF	DoG	SF + DoG	SF	DoG	SF + DoG
Accuracy (%)	96.8085	97.8723	96.8085	93.6170	94.1489	95.2128	97.8723	97.8723	96.2766
Misclassification rate (%)	3.1915	2.1277	3.1915	6.3830	5.8511	4.7872	2.1277	2.1277	3.7234
Precision (weighted avg.)	0.9690	0.9790	0.9690	0.9440	0.9410	0.9570	0.9790	0.9790	0.9620
Recall (weighted avg.)	0.9680	0.9790	0.9680	0.9360	0.9410	0.9520	0.9790	0.9790	0.9630
F-measure (weighted avg.)	0.9680	0.9790	0.9680	0.9380	0.9400	0.9530	0.9780	0.9780	0.9620
Kappa statistic	0.9365	0.9572	0.9368	0.8780	0.8818	0.9070	0.9569	0.9567	0.9241

. Considering the values of all metrics, the combinations that showed the best performance were RF + DoG and kNN + DoG. Each combination was applied to the UAV multitemporal orthomosaics (see Figure 2) for generating the classified images including all classes (water, terrain, dense vegetation, background) and the probability maps of water class (the class of greatest interest in river flood monitoring perspective), as shown in Figure 4 and Figure 5, respectively. Based on preliminary visual analysis focusing mainly on the classification of water class, the best results were obtained with RF + SF and RF + SF + DoG.

Based on a quantitative analysis of the accuracy of the water class area obtained by automatic classification compared to manual digitization (Figure 6), in some cases the best results were quite different with respect to the preliminary visual analysis, considering both (a) the water class area covering the riverbed (TP) and (b) the misclassified water class area identified (FP) (following the methodologies reported in Section 2.4).

(a)

In this analysis, the best results were obtained by using RF + SF + DoG for the 2019 and 2022 orthomosaics, while kNN + DoG for the 2020 and 2021 orthomosaics, with percentage error values ranging from 12% to 20% (

Table 2. Water area in riverbed (TP) and percentage error values between classified and digitized water class area covering the riverbed were calculated for each combination and the different orthomosaics.

Orthomosaic Basento 2019
	Random Forest			Bayes Net			k-Nearest Neighbors			Manual Digitization
	SF	DoG	SF + DoG	SF	DoG	SF + DoG	SF	DoG	SF + DoG
Water area in riverbed (m2)	35,147.46	35,407.47	37,730.88	31,253.84	33,203.96	29,387.79	34,980.80	34,532.09	32,808.49	47,135.98
Error (%)	25.43	24.88	19.95	33.69	29.56	37.65	25.79	26.74	30.40
Orthomosaic Basento 2020
	Random Forest			Bayes Net			k-Nearest Neighbors			Manual digitization
	SF	DoG	SF + DoG	SF	DoG	SF + DoG	SF	DoG	SF + DoG
Water area in riverbed (m2)	67,255.33	78,939.70	60,753.58	46,231.95	70,553.69	34,042.26	72,887.94	80,355.13	63,664.58	91,828.33
Error (%)	26.76	14.04	33.84	49.65	23.17	62.93	20.63	12.49	30.67
Orthomosaic Basento 2021
	Random Forest			Bayes Net			k-Nearest Neighbors			Manual digitization
	SF	DoG	SF + DoG	SF	DoG	SF + DoG	SF	DoG	SF + DoG
Water area in riverbed (m2)	64,300.23	61,433.31	58,294.81	52,521.63	61,071.15	39,037.59	65,805.72	66,903.31	64,274.03	81,073.76
Error (%)	20.69	24.23	28.10	35.22	24.67	51.85	18.83	17.48	20.72
Orthomosaic Basento 2022
	Random Forest			Bayes Net			k-Nearest Neighbors			Manual digitization
	SF	DoG	SF + DoG	SF	DoG	SF + DoG	SF	DoG	SF + DoG
Water area in riverbed (m2)	75,437.53	44,895.41	103,124.78	80,544.60	38,530.83	70,273.95	92,609.33	82,991.06	95,667.82	130,058.57
Error (%)	42.00	65.48	20.71	38.07	70.37	45.97	28.79	36.19	26.44

). The lowest percentage error, 12.49%, was recorded with kNN + DoG for the 2020 orthomosaic, and the highest, 70.37%, with BN + DoG for the 2022 orthomosaic (see Table 2 and Figure 7).

(b)

The best results were obtained by using BN + SF + DoG for the 2019 and 2022 orthomosaics, and BN + SF for the 2020 and 2021 orthomosaics, with percentage values ranging from 0.70% to 2.13% (

Table 3. Misclassified water class area values (FP) and percentage ratio values between FP water area and total image area were calculated for each combination (classifier plus image filter) and the different orthomosaics.

Orthomosaic Basento 2019
	Random Forest			Bayes Net			k-Nearest Neighbors
	SF	DoG	SF + DoG	SF	DoG	SF + DoG	SF	DoG	SF + DoG
FP water area (m2)	38,528.79	157,206.29	33,924.63	15,876.12	61,354.49	9810.49	56,098.31	113,647.05	34,401.36
% FP water area wrt total image area	8.37	34.16	7.37	3.45	13.33	2.13	12.19	24.69	7.47
Orthomosaic Basento 2020
	Random Forest			Bayes Net			k-Nearest Neighbors
	SF	DoG	SF + DoG	SF	DoG	SF + DoG	SF	DoG	SF + DoG
FP water area (m2)	24,793.50	170,571.48	18,854.61	6314.43	79,321.43	6983.49	49,214.38	231,246.00	37,700.85
% FP water area wrt total image area	3.42	23.51	2.60	0.87	10.93	0.96	6.78	31.87	5.20
Orthomosaic Basento 2021
	Random Forest			Bayes Net			k-Nearest Neighbors
	SF	DoG	SF + DoG	SF	DoG	SF + DoG	SF	DoG	SF + DoG
FP water area (m2)	22,894.14	75,227.14	13,123.71	3499.83	43,787.27	9965.12	28,516.41	103,959.12	29,340.77
% FP water area wrt total image area	4.55	14.95	2.61	0.70	8.70	1.98	5.67	20.66	5.83
Orthomosaic Basento 2022
	Random Forest			Bayes Net			k-Nearest Neighbors
	SF	DoG	SF + DoG	SF	DoG	SF + DoG	SF	DoG	SF + DoG
FP water area (m2)	65,749.60	207,434.32	31,361.01	15,625.17	46,707.95	15,170.74	53,096.96	137,507.32	34,822.72
% FP water area wrt total image area	6.44	20.33	3.07	1.53	4.58	1.49	5.20	13.48	3.41

). The lowest value, 0.70%, was recorded with BN + SF for the 2021 orthomosaic, and the highest, 34.16%, with RF + DoG for the 2019 orthomosaic (see Table 3 and Figure 8).

4. Discussion

Analyses of the water-class accuracy classification revealed that the best results are not consistent in all cases with the performance values of the combinations of classifiers plus image filters (see Section 3). In fact, despite the fact that the best-performing combinations were RF + DoG and kNN + DoG, considering the riverbed water class area accuracy analysis, for the 2019 and 2020 orthomosaics the best results were obtained by Random Forest, with different filter combinations RF + SF + DoG, while for the 2020 and 2021 orthomosaics, the results were consistent with the performance values. This situation was corroborated by the results of the misclassified water-class area accuracy analysis: the best results, indeed, were obtained by BN + SF for the 2020 and 2021 orthomosaics, and by BN + SF + DoG for the 2019 and 2022 orthomosaics. This behavior is mainly related to the training dataset used for the classification (a small set of pixels extracted exclusively from the 2021 orthomosaic, which was also used to classify 2019, 2020, and 2022 orthomosaics), on test data with different conditions. Water-class color hues due to both different weather conditions and the date of UAV image acquisition (lean and high discharge periods of the river, affecting water flow and sediment load), indeed, may lead to slightly different results.

Based on the results of the riverbed water-class area accuracy analysis (TP accuracy analysis), the combination that led to better classification was RF + SF + DoG for the 2019 and 2022 orthomosaics, and kNN + DoG for the 2020 and 2021 orthomosaics. This behavior was probably due to the different amount and color hue of water in the riverbed (ongoing study).

The results of the misclassified water-class area accuracy analysis (FP accuracy analysis) showed the same behavior for the 2020 and 2021 orthomosaics, with the best results with BN + SF, and for the 2019 and 2022 orthomosaics, the best results were with BN + SF + DoG. This behavior resulted in the inference that BN + SF works well in the presence of less vegetation (percentage values of 0.87% and 0.70%, for the 2020 and 2021 orthomosaics, respectively), while BN + SF + DoG works well in the presence of greater vegetation (percentage values of 2.13% and 1.49%, for the 2019 and 2022 orthomosaics, respectively). This result was validated by an analysis of the amount of vegetation present in the different orthomosaics, by determining the ENDVI (vis) index (assumed as ground truth for the identification of vegetation [63]) and the ratio of the resulting vegetation area to the total image area for each orthomosaic (

Table 4. Amount of vegetation determined by the ENDVI(vis) index, calculated for the different orthomosaics. The percentage values represent the ratio of the vegetation area to the total image area.

Orthomosaic	Vegetation Area (ENDVI(vis))/Total Image Area
2019	5.8%
2020	1.9%
2021	1.3%
2022	2.8%

). This index turns out to be an adaptation of the classical ENDVI index (developed by LDP LLC, Carlstadt, NJ, USA [64]) to the bands of the visible. The results confirmed a higher presence of vegetation for the 2019 and 2022 orthomosaics with values of 5.8% and 2.8%, respectively (see Table 4). Vegetation is the mostly misclassified class as the water class (as also stated in [39]) because it can show greater variability in color hues according to the different seasons.

In order to indicate a universal combination for water-class identification, according to the different characteristics of the orthomosaics (2019, 2020, 2021, 2022), RF + SF + DoG could represent the best combination for water segmentation (more TP and less FP) considering all the values of both TP and FP reconstruction accuracy (Table 2 and Table 3, respectively). Considering the results obtained, the use of high-resolution, low-altitude and multi-temporal UAV imagery in combination with the supervised classification approach proved to be an efficient, cost-effective and prompt solution for the identification of water in riverine environments. Specifically, the advantage of acquiring on-demand images, recording the different quality conditions and the river’s seasonal changes, results in an improvement in the water classification process.

Despite the optimal results, the proposed approach presents some constraints that have partially affected the results and that could be overcome in a future perspective to improve the classification process. In particular, a limit of our approach could have been the use of a training dataset consisting of a small set of user-identified pixels on a specific orthomosaic; in this view, the use of more populated and variable training datasets could greatly improve the classification process. The use of multi and hyper-spectral UAV images in addition to RGB could also improve the classification process, by recognizing ground objects that may have very similar features (we have seen how using RGB imagery the vegetation is often misclassified as water, as it may present similar color hues). The assumption of a manually digitized water class as ground truth may represent a limit, indeed the operator may run into a misclassification error as in the case of identifying wetlands. Another limit of this approach consists in the analysis of UAV orthomosaics at original resolution in Fiji with TWS: the large size (order of magnitude of one GB) and the low computational resources of a traditional workstation, result in the need to down-scale the original data (in this case from 1.3 cm/pix with a file size about 6 GB, to 17 cm/pix with a file size about 60 MB). The down-scaling could be overcome by using the high-performance computing (HPC) paradigm. Taking advantage of the near-unlimited computing resources of computing cluster by parallel computing, for example, it is possible to divide the very high-resolution orthomosaic into tiles and to apply the classification workflow for each of them in parallel on a single server, eventually merging the results obtained by each server. This could result in an exponentially reduced processing time (crucial in emergency scenarios) and in the ability to analyze increasingly large river reaches with unprecedented levels of detail.

Finally, a combination of methods using all the outputs generated by UAV surveys (dense point cloud, digital terrain model and digital surface model) or Lidar sensor data could improve the classification of both terrain and vegetation classes, which could help to identify the water class accurately.

Notwithstanding the above limitations, our approach for automatic river features extraction represents an efficient tool that provides results very close to the human interpretation and, therefore, could prove to be a key element for improving the hydro-geomorphological and flooding hazard assessment in riverine environments. The proposed approach represents an important contribution to the management activities of flood-prone environments, especially in view of sudden climate changes, as it ensures a reduction of time in the image analysis and accurate and rapid identification of flood-affected areas immediately after the calamitous event. Therefore, this approach enhances the post-disaster damage estimation, which facilitates the intervention of military corps and civil protection, and enables the implementation of recovery plans, also for insurance purposes.

5. Conclusions

The present work compared the accuracy of different combinations (classifier plus image filter) using TWS, with respect to human photo-interpretation, in order to identify an effective open-source supervised approach for automatic water-class extraction from UAV multitemporal orthomosaics of wide areas of riverine environments.

TWS was set up with 9 specific combinations of classifiers and image filters: RF + SF, RF + DoG, RF + SF + DoG; BN + SF, BN + DoG, BN + SF + DoG; kNN + SF, kNN + DoG, kNN + SF + DoG.

Performance metrics values for each combination were determined with respect to the training data of each class (water, terrain, dense vegetation and background) extracted from the 2021 orthomosaic. The accuracy values recorded ranged from about 93% to about 97%, while the misclassification rate values ranged from about 2% to about 6%.

The accuracy of the water class was determined by (a) comparing the classified and digitized water-class area covering the riverbed, and by (b) determining the misclassified water-class area over the whole probability maps. In (a), the best results were obtained by using RF + SF + DoG for the 2019 and 2022 orthomosaics, while kNN + DoG for the 2020 and 2021 orthomosaics, with percentage errors of the comparison between classified and digitized riverbed water-class area, ranging from 12% to 20%. In (b) the best results were obtained by using BN + SF + DoG for the 2019 and 2022 orthomosaics, and BN + SF for the 2020 and 2021 orthomosaics, with percentage values of the ratio between misclassified water area and total image area, ranging from 0.70% to 2.13%.

According to the different characteristics of the analyzed orthomosaics and the results obtained from the water class accuracy analyses, RF + SF + DoG could represent the best combination for water segmentation (more TP and less FP).

The approach proposed in this work for automatic river water-class extraction represents an efficient tool that provides results very close to the human interpretation and, therefore, it could prove to be a valuable tool to support and improve the hydro-geomorphological and flooding hazard assessments in riverine environments.