Modern airborne imaging technology based on unmanned airborne vehicles (UAVs) offers unprecedented possibilities for measuring our environment. For many applications, UAV-based airborne methods offer the possibility for cost-efficient data collection with the desired spatial and temporal resolutions. An important advantage of UAV-based technology is that the remote sensing data can be collected even under poor imaging conditions, that is, under cloud cover, which makes it truly operational in a wide range of environmental measuring applications.
We focus here on lightweight systems, which is one of the most rapidly growing fields in UAV technology. The systems are quite competitive in local area applications and especially if repetitive data collection or a rapid response is needed.
An appropriate sensor is a fundamental component of a UAV imaging system. The first operational, civil, lightweight UAV imaging systems typically used commercial video cameras or customer still cameras operating in selected three wide-bandwidth bands in red, green, blue and/or near-infrared spectral regions [1
]. The recent sensor developments tailored for operation from UAVs offer enhanced possibilities for remote sensing applications in terms of better image quality, multi-spectral, hyper-spectral and thermal imaging [4
] and laser scanning [11
One interesting new sensor is a lightweight spectral camera developed by the VTT Technical Research Center of Finland (VTT). The camera is based on a piezo-actuated, Fabry-Perot interferometer (FPI) with an adjustable air gap [6
]. This technology makes it possible to manufacture a lightweight spectral imager that can provide flexibly selectable spectral bands in a wavelength range of 400–1,000 nm. Furthermore, because the sensor produces images in a frame format, 3D information can be extracted if the images are collected with stereoscopic overlaps. In comparison to pushbroom imaging [7
], the advantages of frame imaging include the possibility to collect image blocks with stereoscopic overlaps and the geometric and radiometric constraints provided by the rigid rectangular image geometry and multiple overlapping images. We think that this is important in particular for UAV applications, which typically utilize images collected under dynamic, vibrating and turbulent conditions.
Conventional photogrammetric and remote sensing processing methods are not directly applicable for typical, small-format UAV imagery, because they have been developed for more stable data and images with a much larger spatial extent than what can be obtained with typical UAV imaging systems. With UAV-based, small-format frame imaging, a large number—hundreds or even thousands—of overlapping images are needed to cover the desired object area. Systems are often operated under suboptimal conditions, such as below full or partial cloud cover. Despite the challenging conditions, the images must be processed accurately so that object characteristics can be interpreted on a quantitative geometric and radiometric basis using the data.
Precision agriculture is one of the potential applications for hyperspectral UAV imaging [1
]. In precision agriculture, the major objectives are to enable efficient use of resources, protection of the environment and documentation of applied management treatments by applying machine guidance and site-specific seeding, fertilization and plant protection. The expectation is that UAVs might provide an efficient remote sensing tool for these tasks [18
]. A review by Zhang and Kovacs [16
] showed that research is needed on many topics in order to develop efficient UAV-based methods for precision agriculture. In this study, we will demonstrate the use of the FPI spectral camera in a biomass estimation process for wheat crops; biomass is one of the central biophysical parameters to be estimated in precision agriculture [17
The objectives of this investigation were to investigate a complete processing methodology for the FPI spectral imagery, as well as to demonstrate its potential in a biomass estimation process for precision agriculture. We depict a method for FPI image data processing in Section 2. We describe the test setup used for the empirical investigation in Section 3. We present the empirical results in Section 4 and discuss them in more detail in Section 5.
Developing quantitative, lightweight UAV remote sensing applications is becoming ever more important, because this technology is increasingly needed in various environmental measurement and monitoring applications. In this study, we presented a complete processing chain for a novel, lightweight, spectrometric imaging technology based on a Fabry-Perot interferometer (FPI) in an agricultural application.
In our previous investigations [20
], we performed the first set of analyses with the FPI spectral camera 2011 prototype using five selected spectral bands. In this investigation, we processed the data using the improved 2012 prototype sensor. First, we developed a method to process all of the bands. We investigated the orientation process and calculated the digital surface models (DSM) by automatic image matching using FPI spectral camera data. We also evaluated the impacts of different radiometric correction approaches during a supervised biomass estimation process. It was important to carry out the entire processing chain in order to identify major bottlenecks and develop the methods further. The results were quite promising; they indicated that the current sensor is already operational and that the processing can be carried out quantitatively and also be highly automated. All sensor-specific processing steps could be implemented as independent steps in our existing processing environment based on commercial photogrammetric and remote sensing software; this is an important issue for companies planning to use the FPI spectral camera in their operational work.
The challenging part of processing the FPI data is that the bands in the spectral data cube are collected with a small time delay. Our approach was to select a few reference bands and determine the exterior orientations for them. We transformed the rest of the bands to match the geometry of the reference bands and, then, applied the orientation parameters of the reference bands to these bands. While the method for band matching proved to be operational, it can still be further improved. In the dataset, the estimated error was on the level of 1–2 pixels (15–30 cm). This level of accuracy is sufficient for most remote sensing applications, and we expect further improvements in the future because of improving sensors and processing methods. Difficulties or reduced accuracy are to be expected for objects with extensive height differences if large spatial differences exist between the bands (fast vehicle) and in cases where the objects are homogeneous (water areas); carefully designed flight parameters and band matching processes are needed to obtain good accuracy. The most optimal approach for producing the best accuracy could be to georeference the individual bands separately. This is a software issue: the software used in this investigation was not ideal for this approach. The FPI sensor provides many alternative ways for processing the data, but in this study, we concentrated on methods that could easily be integrated with our existing photogrammetric and remote sensing environment.
Geometric processing of the frame imaging sensors is a quite mature technology, even though methods are being improved constantly, for instance, to improve the reliability of processing very small format sensors operating in a highly dynamic environment, such as UAV imaging. Our processing required a certain amount of interaction during the block initialization phase; approaches for improving this include a better direct georeferencing solution [11
] or applying some recently presented ordering methods to determine the approximate orientations of the images, such as the structure from the motion technique (e.g., [39
]). Because rigorous integrated global navigation satellite and inertial measurement unit (GNSS/IMU) orientation systems for direct georeferencing are still quite expensive and heavy for light and low-cost systems, photogrammetry-based methods should be developed, so that they can operate at an optimum level. For image matching, SNR is a critical image quality indicator. Accurate GNSS data will also improve the georeferencing accuracy and eliminate or reduce the need for GCPs, which has been demonstrated in previous investigations [26
]. While our geometric accuracy results were quite good in comparison to recent results obtained using hyperspectral sensors [7
], they were not as good as those obtained using higher spatial resolution, wide-bandwidth sensors [26
The quality of the point clouds extracted from the FPI spectral camera imagery was poorer than what many recently published results have indicated [20
]. In these studies, wide-bandwidth, high dynamic range, high spatial resolution sensors were used. Because the spatial resolution of the spectral data is expected to be lower than what can be obtained with commercial wide-bandwidth small-format cameras, a functional approach would be to integrate a high spatial resolution sensor with an FPI spectral camera in order to obtain high-quality 3D information, as suggested in our previous study [20
]. However, the lower quality DSM provided by the FPI spectral camera is also useful when processing and analyzing the data.
In the case of UAV imaging, radiometric processing is a relatively unexplored topic. Radiometric sensor correction is needed for a quantitative remote sensing processing line, and we also applied these methods to our processing line [7
]. In this study, we considered the available methods to be accurate, but in future studies, reliable quality criteria should be developed for the sensor pre-processing phase. Traditional atmospheric correction methods based on radiative transfer have been developed for pushbroom imaging systems [31
], and similar approaches have also been applied to UAV-based hyperspectral imaging systems [7
]. Recently, approaches have been established for making radiometric block adjustments and for generating reflectance images for block data with rectangular images collected using stable, large-format digital photogrammetric cameras [42
]. For UAV remote sensing applications using rectangular images, simple balancing approaches are typically used [3
]; and empirical line-based approaches are popular [4
]. Our objective is to develop a physically-based method for the atmospheric correction of frame images, one that includes a radiometric block adjustment utilizing radiometric tie points and utilizes in situ
irradiance measurements in UAV and/or on the ground, but we are still applying many simplifications to the method [20
]. While the radiometric processing proved to be quite complicated, due to the variability in the illumination conditions, we also found that both radiometric block adjustment and in situ
irradiance measurement-based methods greatly improved the data quality. In the future, it will be of interest to integrate these methods [33
]. The investigated and developed processing methods are useful for airborne UAV frame format imagery in general. Further investigations are still needed in order to develop accurate radiometric correction methods for high-resolution, multi-overlap frame image data collected under variable conditions. In the future, there will be a need to thoroughly consider the reflectance output products resulting from UAV remote sensing [28
]. The quantitative radiometry is expected to improve the performance of the remote sensing application in general and, furthermore, will enable the use of rigorous radiative transfer modeling-based methods in the analysis of object characteristics; this would be advantageous for UAV-based precision agriculture, as the need for site-specific training data would be eliminated; the importance of the accurate radiometric processing and atmospheric correction is highlighted also in agricultural applications with global and regional focus [49
Recently, researchers have conducted experiments with UAV imaging systems with hyperspectral scanners using the pushbroom principle [7
]. In comparison to those systems, the FPI spectral camera collects less spectral bands that are not as narrow (10–40 nm in comparison to 1–10 nm). The advantages of the FPI spectral camera include its light weight and the fact that a direct orientation solution requiring expensive GNSS/IMU equipment is not needed, as well as the fact that it offers the possibility to conduct stereoscopic measurements and multi-angular reflectance measurements. All innovations developed for frame geometry images can be directly utilized when processing FPI spectral camera images; these techniques are expected to develop further due to the invasion of computer vision technologies in personal mobile equipment. We expect that the FPI spectral camera concept could provide more robust and cost-efficient applications than systems based on the pushbroom principle and that data collection can be optimized by using carefully selected spectral bands for each application. For many applications, we expect that the FPI spectral camera will be used as one component in an integrated sensor system; in the agricultural application, the important sensors that need to be integrated are a high spatial resolution, wide-bandwidth camera that provides more accurate DSMs, as well as a thermal camera [4
We demonstrated the use of FPI data for estimating crop biomass in order to validate the data processing phase. Our results when using radiometrically corrected data and a supervised classification method provided at best a 15.5% normalized root-mean-square-error (NRMSE) during the biomass estimation process, which is in line with the results presented in the existing literature [15
]; the NRMSE was 26.3% for the radiometrically uncorrected data. We assume that the results can be improved upon in many ways, such as if we were to use spectral band selection, spectral indices or multivariate statistics for the feature extraction [15
]. The results when using the new sensor data and the entire data cube were better than the results from the previous year, which were obtained using the 2011 prototype sensor [20
]. We will emphasize the optimization of the estimation process in our future investigations [4
]. Integrating the vegetation heights into the estimation process is also an interesting option [21
In the future, it will be important to develop an operational concept for precision agriculture using UAV technology [6
]. In this operational concept, one of the crucial steps will be to quantify the geometric and radiometric properties required for the UAV remote sensing data, which has also been emphasized by Zhang and Kovacs [16
]. Further legislation also needs to be developed; this is an important factor influencing the way in which UAV technology is used in practical applications, as discussed by Watts et al.
Rapidly developing lightweight unmanned airborne vehicle (UAV) sensor technology provides new possibilities for environmental measurement and monitoring applications. We investigated the processing and performance of a new Fabry-Perot interferometer (FPI)-based spectral camera weighing less than 700 g that can be operated from lightweight UAV platforms. By collecting frame-format images in a block structure, spectrometric, stereoscopic data can be obtained. We developed and assessed an end-to-end processing chain for the FPI spectral camera data, together with image preprocessing; spectral data cube generation, image orientation, digital surface model (DSM) extraction, radiometric correction and supervised biomass estimation.
Our results provided new knowledge about high-resolution, passive UAV remote sensing. The pre-processing provided consistent results, and the orientations of the images could be calculated using self-calibrating bundle block adjustment using regular photogrammetric software. The quality of the DSM provided by automatic image matching was not as high as what was obtained with a wider spectral bandwidth, higher spatial resolution camera. The estimated root-mean-square-error was 40 cm in height and 20 cm in horizontal coordinates, for output image mosaics, and a DSM with ground sample distances of 20 cm, for image data collected using a flying altitude of 140 m. The varying illumination conditions caused great radiometric differences between the images; our radiometric correction methods reduced the variation of grey values in overlapping images from 14%–18% to 6%–8%. The supervised estimation of biomass provided a normalized root-mean-square-error of 15.5% at best. Data quality is an important factor influencing the performance of a remote sensing application; our results showed that signal-to-noise-ratio (SNR) and the radiometric uniformity amongst individual images forming the image mosaics impacted the biomass estimation quality. These results proved that a lightweight imaging sensor that is based on the sequential exposure of different bands can provide spectrometric, stereoscopic data. Furthermore, the results validated that useful spectrometric, stereoscopic data can be collected using lightweight sensors under highly variable illumination conditions, with fluctuating cloudiness, which is a typical operating environment for these systems.
The results showed that all FPI technology-related processing steps (image preprocessing and spectral data cube generation) can be taken care of in separate steps and that the rest of the processing can be carried out using regular photogrammetric and remote sensing software. The fact that images can be processed using regular software is an important aspect for users integrating the FPI spectral camera into their operational workflows. For radiometric processing, we have developed new quantitative methods that are suited for frame format images collected in variable illumination and atmospheric conditions.
This was the first quantitative experiment with FPI camera-type technology covering the entire remote sensing processing chain. Our emphasis in developing analysis tools for extremely challenging illumination conditions represents a new approach in hyperspectral remote sensing. Our results confirmed the operability of the FPI camera in UAV remote sensing and the high potential of lightweight UAV remote sensing in general.
We identified many aspects that should be improved in our processing line. These are also recommendations for method development universally. In general, there is a fundamental need to develop reliable methods for the geometric and radiometric processing of huge numbers of small, overlapping images. Another important conclusion is that it will be crucial to develop all-weather processing technology in order to take full advantage of this new technology and to make this technology operational in practical applications. We expect that there will be a great demand for these methods in the near future.
We note that in the future, it will be of great importance to develop reliable error propagation for all phases of the process to enable quantitative applications for these data. Numerical tolerances and criteria will be required by the user community as soon as the UAV-based remote sensing business increases. Further development of the quality indicators that were presented in this investigation is necessary.