A fundamental question driving ecological research is finding explanations that lead to species interactions and their spatial distributions, from global down to local scale [1
Today, various methods of remote sensing (such as radio-telemetry, harmonic radar or LIDAR) are used for detecting and recording the movements of organisms in their natural environments [2
]. However, these methods often require the attachment of devices on every single individual every single individual, and are time and/or cost consuming, especially for investigations of larger insect populations. Alternatively, the use of fluorescent powder dyes was successfully applied as a non-invasive method for vertebrates [7
] and invertebrates [11
]. It proved to be an affordable method while allowing the detection of many individuals in parallel [12
]. In principle, the fluorescent tracer dye emits signals when illuminated by a UV-light source, which can be detected optically or perceived visually. Thus, the method allows the detection of animal interactions (e.g., flower visits of pollinators [13
]) or tracking movements indirectly, e.g., by following tracks until full signal decay [7
]. However, manually searching for fluorescent powder in the field, such as residues on flowers using UV radiation flashlights, is very labor-intensive and time-consuming.
Automated image processing and object-based analysis techniques are by now well-established in the remote sensing community. Moreover, recent technical trends, including minimization of high-quality sensors, have led to new possibilities in the use of unmanned aerial systems (UAS) in various fields and applications [20
]. In particular, UAS are now used in ecology and agriculture in a spectrum of research topics [20
], such as plant identification [22
] and weed management [23
], or pest control and insect monitoring [25
With the advantage of being able to fly at low altitudes, very high-resolution data can be generated flexibly in time and space and thus according to the requirements of the user [20
]. Therefore, remote sensing based on using UAS can provide new capabilities to capture fluorescent traces on plants over larger spatial scales. Recent studies already used UAS to detect locusts combined with strobe-based optical methods [31
]. However, to our knowledge neither any study tested the detection of fluorescent powder dye by UAS derived image data so far, nor used an automated classification and object identification approach to provide tracking information on insect movement from such detections.
Here we aimed to combine these methods and scrutinized if such a detection, automated identification and classification of fluorescent tracer dyes by UAS and post-processing is principally possible and feasible. To answer these questions, we developed a standardized experimental protocol under simple, low-altitude field-conditions as a first proof-of-principles. We tested three parameters in our set-up, which we determined to be highly influential on the efficiency of fluorescent tracer detection: (a) the distance between the UAS camera sensor and a fluorescent tracer, (b) the distance between the projection center on the ground of a UV radiation source spotlight and the fluorescent tracer (i.e., this measure determines the excitation intensity of the fluorescent tracer) and (c) the size of the fluorescent tracer.
We hypothesize that:
detection probability decreases with increasing distance between the UAS camera sensor and the fluorescent tracer (DTS),
detection probability decreases with increasing distance between the projection center on the ground of a UV radiation source spotlight and the fluorescent tracer (DTL),
detection probability increases with increasing size of the fluorescent tracer.
In general, both, the accuracy of the classification scheme and the results of the statistical analysis, suggest that an automated, UAS-based imaging, subsequent classification, and the identification of fluorescent dyes traces to record movement patterns of organisms (for example pollinators), is technically possible (Table 1
, Table 2
and Table 3
, Figure 4
With the featured classification optimization and sampling, the SVM achieved high prediction values for the classification of the used fluorescent dyes leading to almost a complete identification of visually discernible fluorescent traces in the recordings (Table 1
). Nevertheless, we account the application of the method to be independent of the program or method used for object classification and identification. Based on the results of the statistical analysis an identification probability of more than 97% can be assumed under optimal conditions, i.e., strong fluorescence signal, traces of a size of around 4 mm2
and a flight height of 2.2 m to 2.7 m (Figure 4
In sum, all tested main parameters, the distance between the camera sensor and the object level (DTS), the size of the fluorescent trace (size), and the distance between a trace and the center of the UV-light cone at the object level (DTL), significantly affected the identification probability:
Confirming our hypothesis, (i) identification probability decreased with increasing DTS (Figure 4
, Table 2
). The increased distance (DTS 2; 3 m to 3.65 m) between camera sensor and fluorescent traces led to a significantly lower identification probability of the traces, which presumably occurred due to the lower spatial resolution (0.94–1.13 mm per pixel edge compared to 0.67–0.83 mm per pixel edge at DTS 1) and the lower fluorescent radiation energy that reaches the camera sensor (Table 3
, Figure 4
In this study, the realized flight scenarios, resulting in DTS 1 and DTS 2, were both conducted at very low altitude. In real surveys, when tracers are to be detected in vegetation, we also assume comparably low DTS, relative to altitudes where UAS normally are flown when surveying for pests [53
] or breeding birds [54
]. However, under these conditions also other methods are applicable, such as camera stands, tripod-based or crane-based options (i.e., a dolly), which may allow for taking images in a more controlled way, without adding the problems derived from UAS-mounted moving cameras. In our approach, we refrained from using such methods, since ecological studies often require more surveyed ground space than a few local spots, which often requires UAS-based surveys [54
]. Thus, we tested by directly using an UAS-based approach, to ensure the inclusion of unstable flight conditions. However, we did not test the application in a field survey in a full flight campaign, which needs to be tested in future studies.
In our study, we did not include a higher difficulty of UV trace-detection induced by complex vegetation structure. This might affect the detection success of the UV tracers, whereby more stable conditions during image taking (as derived by the alternative methods) may ensure high detectability and thus may limit the use of a UAS.
Regarding camera settings, we used a large aperture and relatively high film speed (ISO 1600) to allow the use of short exposure times of 1/200 s. This was done to reduce low image qualities due to movements. However, it has to be considered that larger apertures potentially lead to more pronounced optical aberrations of the optical system and therefore may negatively influence image quality, especially at the edges of the images. In contrast, lower apertures in combination with slightly higher exposure times potentially enhance the potential of the presented technique. Therefore, we suggest future investigations consider in depth how image sharpness can be improved by settings of the optical system.
An approach for increasing the DTS, i.e. higher flight levels, while keeping the high spatial resolution constant, is the usage of a professional aerial camera with a larger image sensor size (e.g., medium format). For example, using the PhaseOne iXU 100MP camera sensor would increase the flight height to about 8 m with a comparable GSD of 0.7 mm. However, such professional aerial camera systems are of a heavy weight and thus require more powerful UAS. In this study, the influence of the image sensor size on the classification accuracy was not investigated.
With (ii) increasing DTL, the identification probability decreased significantly (Figure 4
, Table 2
). This decrease was stronger, the smaller the trace size and the larger the DTS was. This observation can most likely be attributed to the lower intensity of the fluorescence stimulating UV radiation at larger distances to the radiation source. To compensate for this negative effect, we recommend future studies to upregulate light intensity with increasing flight altitude, thus ensuring a constant level of light intensity. Moreover, it is desirable to increase the power of the UV radiation source in general, because higher excitation energies increase the fluorescence of the tracers and therefore the contrast. With this, it could even be possible to compensate for lower resolutions at higher flight heights. Another option could be to also use a light source in addition to the UV radiation source in order to enhance image sharpness. This may support lower film speeds and apertures and therefore could yield a higher signal to noise ratio and fewer aberrations. However, it has to be tested if such a setting would complicate image classification due to lit features that are no fluorescent traces.
We think that all these parameters are worth testing since all suggestions may potentially increase detection success. We suggest analyzing the modulation transfer function (MTF) on test charts using fluorescent tracers as it provides detailed information on image contrast in dependency of both object parameters and parameters of the optical system [55
]. By this, it may be possible to offer a procedure on how to choose parameters under different conditions (e.g., size of the traces in different applications or flight height). However, this requires a more detailed and strictly standardized experimental setting.
Our results demonstrate (iii) that identification probability increased with increasing size of the fluorescence tracer (Figure 4
, Table 2
and Table 3
). The size of the fluorescent traces in relation to the identification probability also interacts significantly with the DTL. This interaction may be caused by a more intense fluorescence and higher detectability of larger traces, even under less intense UV radiation at higher DTS (Figure 4
, Table 2
and Table 3
). Therefore, it can be assumed that the size of the fluorescent traces is critical for the applicability of the method if the intensity of the UV radiation is low. In this case, differences in the identification success are visible even for small changes in the intensity of the UV radiation. In more realistic flight scenarios, UV-light source and UAS will not be decoupled, but attached to each other, contrary to our approach using a hand-held UV light spot yielding a fixed height to the object level. Thus, when the UV-light spot is attached to a UAS, the DTS and the resulting DTL will be interdependent, both varying with flight altitude. In consequence, we still emphasize to consider interdependency between all three main parameters in future studies.
In this study, we provide a first proof-of-principle for the UAS-based detection and identification of small fluorescent dye traits (size range between 1 and 4 mm2) by means of a simple experiment under outdoor light conditions.
Based on the results of the statistical analysis, the highest identification probability in our study is given under optimal conditions of a strong fluorescence signal (low DTL), trace sizes of around 4 mm2
and a low flight height (DTS of 2.2–2.7 m; Figure 4
). Since these parameters proved to be interdependent, there is no single parameter to focus on in future studies. We discussed several improvements to be considered, including the promising modulation transfer function (MTF). Since tracer size derived by organisms may vary and cannot be controlled in a standardized way, we also suggest testing more complex set-ups to overcome the limitations of this study.
In summary, we think the proposed method represents the first step to enable automated identification of fluorescent traces and thus facilitate many applications in ecology, such as monitoring of biodiversity, especially of insects, exploring patterns of animal movement, spatial distributions and plant-pollinator interactions.