Vicarious Radiometric Calibration of a Multispectral Camera on Board an Unmanned Aerial System

Combinations of unmanned aerial platforms and multispectral sensors are considered low-cost tools for detailed spatial and temporal studies addressing spectral signatures, opening a broad range of applications in remote sensing. Thus, a key step in this process is knowledge of multi-spectral sensor calibration parameters in order to identify the physical variables collected by the sensor. This paper discusses the radiometric calibration process by means of a vicarious method applied to a high-spatial resolution unmanned flight using low-cost artificial and natural covers as control and check surfaces, respectively.


Introduction
Unmanned aerial systems (UASs) are gaining ground in the field of remote sensing as a new and versatile tool for data acquisition.In this sense, the interest of the international scientific community in them is steadily increasing.NASA has been a pioneer in the use of UASs, an example being agricultural resource monitoring, such as coffee crops [1,2], or the analysis of vineyard crop vigor variables [3], among others.
In comparison with manned aircraft or satellite platforms, UASs provide unique advantages in the data captured: their low operating height enables the generation of data at a very high spatial resolution in small areas [4], up to 1 cm per pixel [5,6].Furthermore, UAS platforms allow short revisit periods, in contrast to satellite platforms, with their unfavorable orbital coverage patterns [7].In addition, this high temporal resolution in data capturing [8] and increased maneuverability allow remote data acquisition in small inaccessible areas or in hazardous environments [9].For these reasons, together with their low operational costs, UASs are becoming a key tool to meet the requirements of satellite imagery and aerial photography users.
The progress of microelectronics in the field of digital sensors, navigation equipment (GNSS/IMU (Global Navigation Satellite System/inertial measurement unit)), along with the design of small aircraft and light-weight materials, has reduced the cost of the fundamental components of UASs [10].Several authors have published works in which, using cameras on board small planes or radio-controlled helicopters, they have demonstrated the viability of such airborne vehicles as image acquisition platforms for scientific purposes [11][12][13][14][15][16].With the increasing availability of commercial, low-cost components, research groups now have the option to develop their own projects based on UASs.Accordingly, they have the possibility of loading sensors with adequate spectral and radiometric resolution to satisfy their own research requirements.
The possibility of working with multispectral cameras on these platforms allows radiometric studies to be carried out.To this end, sensors must undergo a calibration that analyzes the radiometric behavior of each pixel in the different regions of the spectrum in which information has been recorded.This behavior depends on the weather conditions and the characteristics of the sensor [17].Analyzing and comparing these magnitudes to other field measurements, a vicarious calibration model is achieved [18] following the empirical line approach [19].As a result, vicarious calibration allows physical quantities to be known in units of radiance (W• m −2 • sr −1 • nm −1 ) for any pixel from a single image in a particular camera channel.The basis of this behavior is that each body has its own, different reflected/emitted energy pattern that sets it apart from other material when electromagnetic energy impinges on it [20].
This study aims to obtain the calibration parameters of a multispectral camera onboard a UAS using low-cost targets.To achieve this, different natural and artificial surfaces were used to determine radiance accurately at the sensor level through the use of a calibrated radiometer [21].As result, it was possible to extract quantitative data from the multispectral imaging.Additionally, with the determination of the radiometric calibration parameters, several sensor corrections were applied to improve the data quality [22].This workflow highlights the advantages, limitations and problems associated with radiometric capture using multispectral remote sensing onboard UASs.
The present work has the following structure and organization.First, the instruments employed are described, together with the flight planning for data gathering (Section 2) and the radiometric and geometric corrections made to the camera (Section 3).We then discuss the proposed calculation process of the radiometric calibration (Section 4).Thirdly, the field campaign of the case study is explained, and the results achieved are analyzed and validated (Section 5).Finally, we outline the conclusions and future work (Section 6).

Materials
The instruments employed included a multispectral camera, an aerial platform and the spectroradiometer, which will set the ground truth in the form of radiances over artificial control surfaces and natural check surfaces.In the case of the UAS, the flight planning needs to be considered to optimize the data gathering step.

Instruments
A Mini-MCA camera with 6 channels was used as the multispectral sensor [23] (Figure 1); its low weight suggested that it was suitable for loading on a UAS.The specifications of the multispectral camera are listed in the following table (     Each of the six channels of the camera is constituted by a CMOS (complementary metal-oxide-semiconductor) sensor and a filter with a pre-set performance against the spectral range.Such filters are characterized by a central wavelength in the range of 531 to 801 nm.
The spectral response of the CMOS is not uniform, due to quantum efficiency and sensitivity.In turn, filters do not have the same transmittance.The combination of CMOS and the 6 filters is reflected by a reduction in the radiance captured by the camera.These responses are defined in the following graphic (Figure 2), according to the manufacturer's data (Andover Corporation; Salem, NH, USA and Tetracam Inc.; Chatsworth, CA, USA).
Figure 2 shows the spectral range covered by the camera (green, red and near-infrared).The exposure time of each filter is different for the same capture and has the following relationship (Table 2):  The unmanned aerial system was an eight-rotor Oktokopter [24] (Figure 3), which has a gimbal stabilized with two degrees of freedom.This multi-rotor has an IMU system with 10 degrees of freedom and a GNSS, thanks to which scheduled flight paths can be established.The most relevant characteristics are specified in Table 3.The spectroradiometer used to carry out the calibration was the FieldSpec 3 ASD (Analytical Spectral Devices) spectroradiometer.This is a general-purpose spectroradiometer used in different areas of application that require reflectance, transmittance, radiance and irradiance measures, and it is especially designed to acquire spectral measurements in the visible to short-wave infrared range.
The spectroradiometer is a compact, portable instrument that allows one to capture spectral data in the region from 350 nm to 2500 nm, with a spectral resolution of 1 nm.The spectroradiometer is configured by three detectors, separated by appropriate filters to eliminate the light of lower orders.The electromagnetic radiation projected onto a holographic diffraction grating is captured through an optical fiber.This grid separates and reflects wavelength components, to be measured independently by detectors.The visible/near-infrared (350-1000 nm) portion of the spectrum is measured by a 512-channel silicon photodiode array overlaid with an order separation filter.The short-wave infrared (SWIR) portion of the spectrum is acquired with two scanning spectrometers: for wavelength ranges of 1000-1830 nm and 1830-2500 nm.Each SWIR spectrometer consists of a concave holographic grating and a single thermo-electrically cooled indium gallium arsenide (InGaAs) detector with a 2-nm sampling interval.
The incoming light to the device is captured through a 3-m optical fiber, whose field of view (FOV) is modified by various foreoptics.

Flight Planning
Proper planning of UAS flights is an important aspect in order to ensure that the data capture fits the theoretical parameters and user requirements pursued and optimizes the available resources.Furthermore, risks to humans are avoided, and higher quality images can be obtained.
This planning takes into account all the limitations and restrictions that are required by the final images themselves to meet the objectives of the study, acting as a guarantee in the photo capture process.The values that can be specified include the position and attitude of the camera, the flight path, the design of the different image blocks, the determination of the overlaps between the different images, the required camera angles, the scale (through the choice of the pixel size on the ground (GSD (Ground sampling distance)) and control of the time of flight, among others.The theoretical GSD value, which sets the geometric resolution of the study, is defined as: where h is the flight height, S the pixel size and f the camera focal length.One of the most important factors is the overlap between images, since this will ensure greater robustness of the geometry captured, determining the image orientations and the reconstruction of the object with greater accuracy and reliability [25].A UAS flight without the proper flight planning will merely lead to a waste of resources, since the local topography will modify the theoretical flight parameters (GSD, forward and side overlap, etc.), causing them to move away from optimal values.A local increase in height in the study area will lead to a higher spatial resolution (a decrease in h), but also decrease in image overlap, and gaps may appear between the strips.
For this study, the planned flight was carried out (Figure 4) with a flight height of 30 m and a GSD of 16 mm, allowing the radiometric calibration of the camera to be resolved correctly.The flight path was calculated with the UFLIP (UAS Flight Planning) software (developed by the Tidop research group), which allows the above photogrammetric flight planning parameters to be taken into account.

Multispectral Camera Correction
The use of a multispectral sensor requires a series of corrections prior to the radiometric calibration process: background error and vignetting.Furthermore, an additional geometric correction (geometric calibration) necessary for correct channel fusion is considered.All these corrections are determined in a single laboratory analysis and only need to be checked periodically to ensure their stability or when the camera is modified.

Background Error Correction
Image noise sources can be classified as signal-dependent noise (photon shot noise) and signal-independent noise (dark current, amplifier noise, quantization error) [26].Some of these noise sources, such as the quantization error, may be negligible, as long as the noise does not exceed the quantization interval of the ADC (Analog to Digital Converter).However, a multispectral camera may be affected by non-random errors [27], which will degrade the final image quality.
This study analyzed the background error recorded by the camera, whose bimodal behavior was different for each channel and more pronounced on high-reflectance surfaces and is not related to the random noise caused by the sensor electronics (dark current).The systematic error has two configurations: on the one hand, a series of periodic horizontal bands, due to the blockage of the diaphragm; and on the other hand, a pseudo-texture in the distribution of digital levels.This systematic error is assessable in a completely dark room in the absence of light, where only the random noise component is to be expected.
To eliminate both effects, a laboratory analysis was undertaken in the absence of light, evaluating the average response of the camera per channel under different exposure times.The maximum background error for this study involved a 0.49% increment in the digital level value.

Vignetting
The term vignetting refers to the effect of the brightness attenuation of an image as we depart from its principal point radially.This phenomenon occurs due to the effective size of the camera lens aperture.Vignetting is decreased proportionally to lens aperture (or inversely to the f-number).Furthermore, vignetting is related to the focal length, since the angle of the light incidence on the sensor causes a dimming, such that wider-angle lenses are more affected by this phenomenon.
Since this condition affects the image radiometry, it was corrected to ensure that each pixel would contain the correct digital level.The study was conducted in a laboratory, with uniform illumination, acquiring a series of photographs of a white pattern with low-specular reflection [27,28] (Figure 5).

Geometric Distortion
Geometric distortion caused by the camera lens can be considered as a supplementary aspect to the radiometric calibration process.Moreover, the processing of geometric distortion involves an alteration of digital levels, due to the resampling process, and hence, its correction (direct or reverse) should be carried out in the final stage.
The goal is to determine the geometric (principal point coordinates, x p , y p , and principal distance, f) and physical (radial and tangential distortion) parameters that define the internal orientation of the camera, using a laboratory calibration.
This aim can be achieved thanks to a protocol in which image shots are convergent to a pattern or grid of known dimensions and by applying the collinearity, which relates image points with ground points.In particular, an open source tool, Bouguet [29], was used.More specifically, a set of images with a planar checkerboard pattern were acquired under different roll and pitch angles.The images ensured that the pattern covered the largest area of the image in order to model the geometric distortions without extrapolations.
Table 4 shows the results of the 6-sensor camera (Tetracam Mini-MCA) calibration, expressed in the balanced model [30].This distortion model fits the effect of radial distortion (∆r) through the coefficients, a 0 , a 1 and a 2 , whereas the coefficients, P 1 and P 2 , model the tangential component (∆t), according to the mathematical model of Equation ( 2): r a r a r a r t P r 2 x x 2P x x y y t P r 2 y y 2P x x y y where r′ stands for the radial distance of the real image (in contrast to the radial image of the ideal or undistorted image).The coefficients, a 0 , a 1 and a 2 , are functions of the radial distance from the principal point of symmetry.Additional information about the geometric calibration can be found in [31].The differences in construction between the sensors are also shown in Figure 6, where the maximum discrepancy reaches 18 pixels, illustrating the relevance of this geometric correction for individual image fusion.
Since the multispectral camera has six non-collinear objectives, the image fusion has to take into account, not only the calculated intrinsic camera parameters (specific for each sensor), but also the extrinsic parameters of the sensors; the three-axis orientation and spatial position.The distance, or baseline, among the optical centers of the sensors will cause a parallax [32] in the image fusion.This effect can usually be neglected in real applications (due to the height of the flight).However, for laboratory experiments or very low flights, this parallax can be considered by resampling the images according to the coefficients of the fundamental matrix [33].

Calibration Method
Analyses derived from data captured by multispectral cameras require previous knowledge of the radiometric calibration parameters of each channel.According to Dianguirard and Slater [34], radiometric calibration processes can be classified as:  Laboratory calibration before the flight (preflight calibration).This procedure involves a rigorous calibration of sensors. Satellite or airborne calibration (onboard calibration), implementing checks during image acquisition.Lamps or solar-diffuser panels are used in this kind of calibration. Calibration through in situ measurement campaigns (vicarious calibration).This entails an absolute radiometric calibration in flight conditions other than those found in the laboratory.Within this modality, the absolute method based on radiance or reflectance is included.
The radiance-based method is theoretically more accurate, and its uncertainty is approximately 2.8% versus 4.9% for the reflectance-based method [35].This low value arises from the calibration and stability of the spectroradiometer required for calibration [34].
Among the different calibration methodologies, we chose a vicarious calibration based on the absolute radiance method (Figure 7), considering that the digital level that defines each pixel has a direct relationship with the radiance detected by the sensor [27,36].
Thus, for each spectral channel of the camera, a linear model is established that relates the digital level to the radiance captured by the sensor.
Radiometric calibration processes require homogeneous and Lambertian surfaces.Among the possible materials that could function as control surfaces, we chose low-cost elements: a canvas with 6 different tones of grey and 6 PVC (polyvinyl chloride) vinyl sheets with different colors.For this calibration workflow, artificial targets were chosen instead of pseudo-invariant features, since they have proven to be more appropriate [18,37,38].The critical factor for this selection is the requirement of uniform reflectivity with respect to the viewing direction and wavelength [38].In the case of pseudo-invariant objects, these are not suitable, because their radiometric properties change over time [39,40].Pseudo-invariant features were only employed as check surfaces.
Digital levels (DL) of artificial targets are extracted from the aerial images to calculate the relationship between them and the radiance of the surfaces (obtained with the spectroradiometer).The simplified radiative transfer model is defined according to the following equation: Since several images are involved in the calibration adjustment, a luminance homogenization factor between photos was taken into account.This factor absorbs exposure differences (due to changes in lighting between different shots) and the inherent shutter time of each channel.
where c 0 and c 1 , offset and gain, are the calibration coefficients of each camera channel.The variable, F h , is the homogenization factor of digital levels, defined as follows, where F eq is the exposure factor and F v the shutter opening time factor.Furthermore, because the images are affected by different types of radiometric distortion generated by the sensor (see Section 3), these corrections were taken into account in adjustment Equation ( 5), obtaining the final calibration model: where R is the systematic background error correction and V the vignetting correction; both variables are functions of the pixel position in the image.
In classical aerial photogrammetry, aiming at the determination of physical parameters at the surface level and not at the sensor level, the 6S atmospheric model [41] has been applied.The modeling of the influence of the atmosphere on the propagation of radiation for a height of 1 m (spectroradiometer data captured) and 30 m (UAS flight height) shows no discrepancy.More specifically, the difference has an order of magnitude of <1 × 10 −9 W• cm −2 • sr −1 • nm −1 .Therefore, it could be suggested that in UAS photogrammetry, the influence of the surface to sensor component of the atmosphere is minimal, since radiation passes through a very small atmospheric column.Due to its reduced value, the relative atmospheric correction can be neglected in the adjustment model, as reported in [42].
Finally, the results of the radiometric calibration process were validated by checking the surface: natural covers, such as vegetation, soil-covered land and bare soil.

Fitting Model
From multiple artificial targets collected in several images, a least squares adjustment was applied.A robust estimation was chosen instead of an ordinary least squares (OLS) method, since OLS is highly sensitive to outliers, because real measurements of error distributions have larger tails than the Gaussian distribution [43].In our case, we chose the Danish Method proposed by Krarup [44], which, applied iteratively, gives a series of weights according to the residual values of the previous iteration.
In the first iteration, the weight matrix, W, is set as the identity matrix: where x is the vector of calibration coefficients, A is the design matrix (digital levels) and K is the matrix of independent terms (radiance).The residual vector v is: whose a posteriori variance is: where m is the number of equations and n the total number of unknowns.
From the first adjustment of residuals, new weights are calculated individually for each equation, based on the following weight function of the Danish estimator: where c is a constant that varies between 2 and 3, depending on the redundancy of the adjustment and data quality.
The convergence selection criterion of the iterative process is established based on the fulfillment of one of the following conditions:  Standard deviation estimator < 0.001;  Change in variance < 0.01;  When there are more than 20 iterations.
In the adjustment, an additional unknown was added per image to the x vector to absorb the heterogeneity regarding the possible variations in irradiance between images; this is more likely to occur in unstable weather conditions.

Radiometric Campaign
The study area is located in Gotarrendura, a village close to Avila (Castilla y León, Spain).Data collection was carried out on 27 July 2012, on a pine plot of 2.52 ha, which was overflown at a height of 30 m.The pine species was Pinus pinaster, with a density of 1330 trees per hectare and a height between 1.5 and 2.1 m.
As control surfaces, a 5 m × 1 m greyscale canvas (GS) and six 0.55 m × 0.35 m vinyl sheets of different colors (red, gray, white, black, blue and green) were selected, similar to [42].These artificial surface sizes guaranteed at least 21 pixels (up to 61 pixels), which exceeds the minimum of three times the GSD to rule out neighbor effects.The check surfaces, corresponding to natural covers (pseudo-invariant features), are highlighted with the yellow, orange and red circles in Figure 8.The low-cost colored artificial targets provide a transportable test field as an alternative to a permanent radiometric calibration field.They also avoid the problems of painted targets associated with permanent test fields, caused by environmental conditions [45].In the radiometric study, calibration surfaces were characterized using the spectroradiometer as a detector of the radiant flux that is reflected from such covers.During data acquisition, it was necessary to take into account that the incidence angle that the spectroradiometer gun formed with the surfaces was as orthogonal as possible, taking two spectral measurements per cover.Prior to each sample measurement, the calibrated white reference (Labsphere, Inc.Spectralon™, North Sutton, NH, USA) was measured.The spectra were measured in absolute radiance mode.Each spectral measurement is the average result of 120 individual spectra, following the protocol shown in [42].
In parallel, a planned UAS flight was conducted over the study area, capturing multispectral images (Figure 9) and choosing those in which the maximum numbers of control and check surfaces were visible.The selected radiance control surfaces were obtained from the spectroradiometer.Figure 10 shows the spectral signatures of the vinyl sheets used in the radiometric calibration.The reflectance of these surfaces was obtained as the ratio between the reflected radiance of each cover and the radiance of a white reference target (Spectralon 99%), both measured with the spectroradiometer.To compare the radiometric measurements with digital levels, it is important to note that the radiance obtained with the radiometer lies between the 350 and 2500 nm spectral range with a 1-nm resolution, whereas the Mini-MCA is capable of recording digital levels in its six channels, each one characterized by a particular response (Figure 2), due to the differential responses of the filter and CMOS at each wavelength.Therefore, it was necessary to adapt and standardize the radiometric measurements to the spectral resolution of the camera, together with the camera spectral response (R C ).The spectral camera response includes the CMOS response, as well as the filter transmission function.Equation (11) shows the integration process for the measured reflectivity (ρ) of a target (t) with a white reference panel: The equation was also applied to obtain the target radiance values involved in the calibration model (Equation ( 6)).

Analysis and Validation of Results
The Radiometric Calibration was resolved with the support of software developed for this purpose in MATLAB.
The control surfaces used have a typical radiance response for each of the multispectral camera channels, and based on these, the vicarious calibration relationship was established.The following figure (Figure 11) shows this feature for each of the control surfaces used in the calibration process.Regarding the radiometric calibration parameters for an altitude of 30 m for each of the six channels of the Mini-MCA onboard the UAS, Table 5 shows the final results of this study.The R 2 fitting coefficient was 0.9833 for a simultaneous block adjustment of six channels.Table 5 also shows the coefficients for the same fitting model, but with an individual channel adjustment.In this second case, the results are fairly similar, as is the determination coefficient, with no significant discrepancies between the two fitting methodologies.It should be noted that the C 0 value (intercept) is very small (compatible with zero), such that it could be excluded from the calculation.
In this sense, the statistical test used for the validation of the results was the average of the errors in radiance, expressed as percentages (with respect to 10 bits) per control surfaces and per channel (Figure 12).In the above figure, it may be seen that the control surfaces with the greatest error in their radiance estimation are those with the highest reflectance, i.e., the white vinyl and the second lightest color, the greyscale canvas.Furthermore, this error is noteworthy in channel 1 (530 nm), which has the lowest performance, due to the low CMOS response (Figure 2).However, this maximum error means an error of 8%, which can be considered acceptable, since it is an isolated value, as shown in Figure 13, the average residues (2.5%) remaining within the range of error estimated for this calibration mode.
In order to validate the radiometric calibration process, calibration coefficients were applied to the digital levels of natural surfaces to obtain the radiance.The following figure (Figure 13) shows the setting of the radiance measurements considered as -ground truth‖ (spectroradiometer) for the case of a pine and their corresponding values obtained after applying calibration coefficients to the pine digital levels.In this case, a strong correlation can be seen between the calculated radiances and in situ measurements, it being possible to calculate the pine radiance for each of the six camera channels with a relative error of only 1.8%.

Conclusions
This study shows the validity of a vicarious radiance-based calibration for the Mini-MCA onboard a UAS through the use of low-cost covers as control surfaces.The correlation of 0.98 between ground radiance and that derived from the digital level shows the degree of consistency achieved.Furthermore, despite the complexity of the data, the average error of 2.5% is very encouraging.
In addition, after several laboratory and field studies, the validity of using low-cost surfaces for the calibration process was confirmed.Moreover, low-cost covers show an invariant reflectivity for a certain period of time in which they remain unaffected by deterioration.
Another important contribution of high-spatial resolution remote sensing at low flight heights, as provided by this study, is that the relative surface-sensor atmospheric effects on UAS versus ground truth measurements are negligible, thus simplifying the workflow.
Finally, in view of the high spatial, spectral and temporal resolution achieved for UAS remote sensors, these platforms can generate high value products at reduced costs as compared to satellite or manned aerial platforms.UAS remote sensing is proving to be a valuable non-invasive technique for the recognition and analysis of different types of strata, crops and rocks, among others.Furthermore, not only qualitative results are obtained, physically relevant quantitative results derived from the digital levels of UAS images can be obtained, as well, and are most relevant.

Figure 2 .
Figure 2. Complementary metal-oxide-semiconductor (CMOS) and filter spectral performance of the Mini-MCA multispectral camera.

Figure 4 .
Figure 4. Photogrammetric flight planning using an orthoimage of the study area.

Figure 6 .
Figure 6.Graphic representation of geometric distortion of the six channels of the MCA.

Figure 7 .
Figure 7. Workflow of the radiometric calibration process.

Figure 8 .
Figure 8. Aerial image of the control and check surfaces.

Figure 10 .
Figure 10.The spectral signature of the control surfaces (vinyl sheets) used in the radiometric calibration process.

Figure 13 .
Figure 13.Pine radiance from the spectroradiometer and from the calibrated Mini MCA.

Table 1 .
Characteristics of the Mini-MCA multispectral camera.

Table 2 .
Characteristics of the six channels of the camera and their corresponding exposures times.

Table 4 .
Radial and tangential distortion parameters of the six MCA channels.