Recent Advancements and Perspectives in UAS-Based Image Velocimetry

Abstract

Videos acquired from Unmanned Aerial Systems (UAS) allow for monitoring river systems at high spatial and temporal resolutions providing unprecedented datasets for hydrological and hydraulic applications. The cost-effectiveness of these measurement methods stimulated the diffusion of image-based frameworks and approaches at scientific and operational levels. Moreover, their application in different environmental contexts gives us the opportunity to explore their reliability, potentialities and limitations, and future perspectives and developments. This paper analyses the recent progress on this topic, with a special focus on the main challenges to foster future research studies.

1. River Monitoring from UAS

In the eighties of the XX century, image velocimetry techniques started to emerge from laboratory and industrial applications and later on were also adapted for fluvial monitoring [1]. Optical methods for river flow monitoring are nowadays popular among researchers and are starting to be considered at operational levels due to the following reasons: traditional contact measurements require expert operators and, therefore, they are time-consuming and rather expensive.

The increasing popularity of optical measurements for river monitoring is motivated by the growing availability of low-cost technology and the introduction of dedicated smartphones apps [2,3]. Moreover, the growing number of applications based on Unmanned Aerial Systems (UAS) further favoured the development of such techniques, making accessible any location of a river system [4,5,6].

The effort to develop image-based techniques is evidenced by the increasing number of research studies published in the last years. The evolution of research activities has been depicted in Figure 1, which provides the trend of publications on river monitoring adopting different techniques over the last 40 years. The graph has been obtained using the Scopus database, adopting the following keywords: “river” and “velocity” with “image velocimetry” or “current meter” or “radar”. This allows for observing how the use of different techniques has changed over time. It is possible to observe that image velocimetry techniques were applied to river monitoring for the first time in 1997 and have grown steadily over time, reaching 43% of the publications on river monitoring. It is also worth mentioning that radar use has increased over time, making surface velocity measurements the most relevant topic, representing about 75% of the total number of publications in river monitoring.

The increasing interest in image-based techniques is also testified by the common intent to make available datasets and codes. For instance, Perks et al. [7] collected 13 field experiences (five UAS-based) conducted in six different countries, providing all survey data, including pre-processed frames and benchmark measurements. The recent work of Bandini et al. [8], which collects 27 UAS-based case studies in Denmark, promotes the use of UAS for river monitoring. These studies represent a great opportunity to test and validate methodologies or highlight the main sources of errors in outdoor settings.

Surface velocity observations can be used to derive river discharge estimations by combining depth-integrated water velocity profiles and cross-sectional areas. On the one hand, the mean flow velocity along each profile can be derived by exploiting the linear relationship with the maximum and surface flow velocity [9,10]. On the other hand, 3D river cross-section reconstructions can be performed by Structure from Motion (SfM) algorithms [11]. Moreover, this data can also be exploited to measure water level elevation using Machine Learning Algorithms (ML) used for automatic segmentation of water surfaces [12]. However, difficult lighting conditions or water conditions (e.g., vegetation, turbidity) can severely affect automatic water line detection. Alternative methods use drone-based eco-sounder [13], onboard radar altimetry [8] and LIDAR (Light Detection And Ranging) systems [14] for the same purposes in extended morphological river conditions (e.g., turbid waters, obstructed river view). Figure 2 provides an overview of the operational flexibility of UAS platforms for river discharge estimations (Q) by combining cross-sectional areas (A) and water level (D) observations, as well as surface velocity (V_s) and mean velocity (V_m) observations.

The use of UAS for optical flow measurements in unfavourable flow conditions highlights the possibility to monitor floods, ungauged or inaccessible areas [15]. Moreover, the possibility to capture videos from different flight heights and with nadir or oblique camera angles allows one to observe large and dynamic rivers [16], as well as identify the water surface patterns from different points of view [17]. The payload flexibility—for instance, many choices ranging from RGB to TIR sensors—allows to cover and regulate different spectral ranges based on specific fields and flow conditions [18]. UAS remote-controlled systems allow the real-time definition of acquisition frequency based on water flow velocities [19]. The possibility of using filters, polarisers, and changing the flight height based on imposed resolution allows for adapting footage acquisition based on environmental settings (e.g., sunlight, reflections, shadows) to maximise the caption of different patterns in time [20]. Despite the commonly recognised UAS strengths, UAS are subject to several limitations. The most important ones are related to (1) the UAS maximum payload that limits the possibility to adopt multiple sensors and communication hardware; (2) the restriction imposed by national flight regulations that limit the use of UAS, especially in urban areas; (3) the need of continuous power supply for frequent flight missions and the impossibility for flight under extreme meteorological conditions. Moreover, the main limitations regarding the use of UAS for image velocimetry concern other technical limitations such as the wind speed and the local favourable light conditions during the acquisition.

2. Recent Research Progress on UAS-Based Image Velocimetry

In the last decades, numerical and field-based studies have been carried out to evaluate image-based solutions for velocity measurements. These studies generally adopt deterministic [21,22] or statistical [23] approaches for studying the error sources in outdoor applications. The main objectives concern the definition of parameter settings (e.g., camera frame rate, the size of regular sub-regions for cross-correlation approaches) to obtain accurate surface flow velocity estimations.

Different studies enhanced the influence of some sources of errors on image velocimetry in outdoor applications strictly related to mobile platforms (e.g., UAS or handheld cameras). Detert [24], for instance, evidenced that the stabilisation issues and neglected—or poorly executed—camera calibration during field measurement could potentially induce significant errors on a frame-by-frame displacement calculation. According to these findings, Ljubičić et al. [25] explored different commercial and ad-hoc tools for image stabilisation algorithms, highlighting the influence of stabilisation errors on image velocimetry performances and the beneficial effects of stabilisation algorithms for these purposes. For calibration and geo-rectification purposes, optical data features with known coordinates (Ground Control Points, GCPs) are usually taken into consideration. To overcome the need for GCPs, recent approaches use onboard radars or lasers to convert image units (pixels) into metric units [8,18].

Another commonly recognised challenge is related to the influence of environmental noise on the velocity signal under challenging weather conditions, poor illumination, sunlight reflections, glare and shadows on the flow surface, river colour background, and riverine flora movements, among others [24,26]. Additionally, extremely scarce illumination or rapid illumination changes may introduce several problems in recovering long and reliable features trajectories [22,27].

Field studies are frequently affected by the difficulties in acquiring reference surface velocity measurements. Generally, contact (e.g., current meters or ADCPs) and non-contact (e.g., radars) instruments are indifferently adopted for this purpose [7]. These instruments are usually affected by different approximations related to surface velocity estimations. For instance, current meters and ADCPs cannot measure the surface velocity, which is usually extrapolated assuming a velocity depth profile. For these reasons, several authors are adopting computer vision techniques for reproducing natural environments and adopting an imposed reference velocity (Figure 3). Different synthetic image generators are commonly used in particle image/tracking velocimetry research studies to create realistic particle images within a reasonable computational time. Starting from laboratory PIV generators, recent advancements consider more realistic configurations such as real river background and homogeneous [19] and heterogeneous [28] distribution of particles, with colour and shape noises and surface velocity distribution along the transect [29]. 3D computer graphics and flow simulation tools have been recently adopted to generate videos reproducing realistic river flow, including turbulence, sunlight effects, and camera settings [30,31].

Along with these issues, the lack of surface tracking features or homogeneously distributed materials across the cross-section represents the more recognised challenges for outdoor applications. In natural conditions, flows can present low seeding densities or locally distributed tracer clusters. These conditions can introduce a high variance and underestimate the flow velocity field, especially near the riverbanks. In this regard, Dal Sasso et al. [21] and Pizarro et al. [28] introduced three metrics for quantification of spatial and temporal characteristics of seeding during the video acquisition period. These metrics are based on the calculation of the (1) seeding density, (2) index of dispersion of tracers, and (3) coefficient of variation of tracer dimension demonstrating their statistical significance on image-based performances. Furthermore, Pizarro et al. [28,32] recently proposed the Seeding Distribution Index (SDI) as a dimensionless parameter that synthesises the seeding conditions in the field, merging seeding and spatial distribution characteristics. This index was formulated using numerical experiments and tested in some field case studies for describing the heterogeneous spatial distribution of tracers and the tendency to form clusters. Remarkable is the strong positive correlation between SDI values and image-velocimetry errors (the lower SDI, the lower the errors), providing a useful tool for practical applications. Recently, Dal Sasso et al. [33] explored the possibility of applying the SDI index at different spatial scales along the cross-section, dividing the region of interest into sub-sectors to better capture the variability of tracers in space and time. We showed a significant reduction of errors (between 20% to 39%) using the proposed SDI-based criterion with respect to the use of the total number of frames available (classical approach).

From a practical point of view, most of the literature experiments have been artificially seeded to simplify the identification of moving patterns on the water surface [7]. However, the recurrence of artificial tracer deployments is not practical and safe because operators need to access the area. Thus, the current research is moving in the direction to maximise the information of the water movement, identifying flow structures such as ripples, differences of colour intensity due to suspended sediments or illumination, and turbulence structures. Several computer vision techniques that are features-detector-based, such as Feature Tracking Velocimetry (FTV, [34]), Optical Tracking Velocimetry (OTV, [27]), Space-Time Image Velocimetry (STIV, [35]), Surface Structure Image Velocimetry (SSIV, Leitão et al. [2]), and Kanade–Lucas Tomasi Image Velocimetry (KLT-IV, [36]), have been implemented for this purpose. The efficiency of these new emerging methods is promising, considering the possibility of monitoring flows without visible objects. However, moving from the recent works of Koutalakis et al. [37] and Pearce et al. [22], more research efforts are needed to test algorithms in different environmental conditions and to discriminate the main differences of approaches. In turn, the seeding limitation can be partially compensated using high-visibility tracers [38] or thermal sensors that are less affected by water surface reflections and illumination conditions than RGB imagery [18]. Thermal sensors allow for monitoring in daylight and nighttime conditions [39], but their current spatial resolution, low contrast, and price present a limitation for monitoring larger rivers or when a high level of detail is required. For this reason, pre-processing techniques based on image enhancement are needed to increase image velocimetry performances and obtain realistic trajectories in rivers [22,27].

Moreover, a considerable limitation affecting optical methods is the lack of information on the velocity profile along the vertical to estimate river discharge. In this context, it is a common practice to use a conversion factor between the surface and depth-averaged velocities (usually known as the alpha value). This factor is site-specific, depending on river hydraulic and geometric characteristics, and its calibration requires intense field campaigns in different flow conditions. Generally, it is influenced by the presence of vegetation on the riverbed or secondary currents affecting the shape of the velocity profile [9,10]. In this regard, new methods based only on UAS measurements of surface flow velocities and water surface slope for the parametrisation of this coefficient as a function of the Gauckler-Manning-Strickler coefficient have been explored by Bandini et al. [8]. All these efforts should allow for the development of clear operational guidelines specific to each environmental condition [40].

3. Final Remarks and Future Prospects

UAS are fascinating platforms for fluvial monitoring that ensure (1) high spatial resolution, (2) high accuracy, (3) high flexibility, and (4) low costs. UAS observations are promising for their applicability on surface velocity estimations, morphology reconstruction, and river discharge monitoring in a wide range of water and river morphology conditions. These hydrological variables are essential for sediment transport analysis, flow dynamics simulations, inundation process reconstruction, flood and droughts forecasting, and pollution dispersion monitoring.

The scientific community is moving towards the direction to maximise the performances of image-velocimetry techniques for river flow monitoring by assessing new features detection algorithms and frameworks, tracking methods, and defining guidelines for their correct application. Surface velocity data are essential for studying flow patterns, erosion dynamics, and instream habitats.

A greater effort is necessary to identify a set of strategies for automatic discrimination between tracking features and water reflections and environmental noise in natural settings. In addition, research studies should focus on implementing an automatic workflow to enhance tracking features in particularly challenging conditions, such as intense sunlight and shadows, or to filter the influence of wind and river turbulence on surface velocity estimations. Recent advances in automatic features detection and machine learning algorithms (ML) can identify static and dynamic patterns based on their characteristics (e.g., shapes, colour, texture). Furthermore, exploring the spectral characteristics of the water surface represents a matter of great interest to obtain a classification of floating material.

New research advancements are needed to integrate surface flow velocity field and water depth data for flow discharge estimation. In this regard, a combined approach to detect and classify surface and background features is needed to obtain a smart river discharge estimation from cameras exportable in different river contexts. Moreover, new models and algorithms should be tested in the future to maximise the surface velocity information, reducing the variability of depth-averaged velocity estimates and limiting the dependence on river geometry and flow conditions.

Technical advances and miniaturisation can greatly improve flight performances and multisensor applications overcoming the recognised UAS limitations, especially for river bathymetry and water depth estimations. These advancements combined with real-time data transferring to a cloud system will allow faster data processing. Moreover, considerable work is needed on the goal of optical systems automation in natural environments. An extended phase of learning should be necessary to identify the common river hydro-morphological characteristics and develop a modular framework able to describe all river flow processes using image-based methods.