Development of a Mobile Multispectral Imaging Platform for Precise Field Phenotyping

Phenotyping in field experiments is challenging due to interactions between plants and effects from biotic and abiotic factors which increase complexity in plant development. In such environments, visual or destructive measurements are considered the limiting factor and novel approaches are necessary. Remote multispectral imaging is a powerful method that has shown significant potential to estimate crop physiology. However, precise measurements of phenotypic differences between crop varieties in field experiments require exclusion of the disturbances caused by wind and varying sunlight. A mobile and closed multispectral imaging system was developed to study canopies in field experiments. This system shuts out wind and sunlight to ensure the highest possible precision and accuracy. Multispectral images were acquired in an experiment with four different wheat varieties, two different nitrogen levels, replicated on two different soil types at four different dates from 15 May (BBCH 13) to 18 June (BBCH 41 to 57). The images were analyzed and derived vegetation coverage and Normalized Difference Vegetation index (NDVI) were used to assess varietal differences. The results showed potentials for differentiating between the varieties using both vegetation coverage and NDVI, especially at the early growth stages. The perspectives of high-precision and high-throughput imaging for field phenotyping are discussed including the potentials of measuring varietal differences via spectral imaging in comparison to other simpler technologies such as spectral reflectance and RGB imaging.


Introduction
Plant scientists have gained vast knowledge about the genetics of various plant species during recent years.It has become increasingly easier and more inexpensive to sequence genomes and measure plant characteristics with non-destructive sensors [1][2][3].However, the vast majority of information on genome and phenome expression comes from experiments with single plants under controlled environments in green houses or climate chambers.Growing a population of plants under varying climate and soil conditions creates very different phenotypes in comparison with single plants grown under controlled environments.A major reason for this is that increased plant density changes the architecture and physiology of plants significantly as a result of competition for light, nutrients, and water [4,5].Another reason is the dynamic response of genotypes towards varying environment and management practices [6].Field phenotyping of crops should therefore be based on frequent and non-destructive measurements at the canopy level, e.g., measuring light interception, total area of leaves, growth rate and biomass.Based on [2] we refer to phenotyping as a set of methods to measure the dynamic development of a plant based on growth, architecture and physiological responses both visually and via remote sensing.
Currently, field phenotyping is considered a limiting factor as it often depends on destructive measurement or manual assessment during a growth period which is laborious and time-consuming [7,8].Therefore, different crop sensing and imaging systems have been applied in order to identify both strengths and weaknesses in the different research-and breeding systems [7].The main goal of field phenotyping is to enable scientists and breeders to detect and evaluate dynamic traits or physiological responses which so far are difficult or even impossible to measure destructively or manually.[3].An additional perspective of field phenotyping platforms should be to increase precision of parameter estimates in order to identify subtle varietal differences.
Absorption of specific wavelengths is recognized to be associated with distinct morphological and physiological plant characteristics reflecting crop growth [9,10] and light interception [11].There are a vast amount of studies demonstrating, how multispectral reflectance can be used in different context when investigating plant physiology and growth [12].A more detailed approach is multispectral imaging that combines multispectral reflectance with spatial image information.Each pixel in a multispectral image contains the signature of the electromagnetic spectra being measured [13].From multivariate image analysis it is possible to group pixels according to their spectral signature and thereby divide such images into classes of objects.The analyses can be used to distinguish plant and soil objects, diseases or physiological and biochemical traits [14].Currently, several platforms based on remote sensing, spectroradiometry and spectral imaging are being developed for field plot screening [3,15,16].
The objective of this article is to show the potentials of field phenotyping using a mobile multispectral imaging platform that excludes the impact of wind and varying spectral irradiation.

A Novel Multispectral Imaging Platform-PhenoField
A mobile multispectral imaging platform named PhenoField (PF) (Figure 1A) was developed to measure spectral characteristics of the canopy and geometric characteristics like vegetation indices and canopy texture.This system was developed by the company Videometer (Videometer A/S Dk-2970 Hoersholm, Denmark) and was based on a commercial hard-and software platform the company has developed for the industry, medical care and laboratories.The platform consists of a two-box construction (Figure 1B) with a vertically movable outer box that can be lowered to the ground and close tightly to the soil surface to exclude influence from wind and sunlight.A top centered CCD camera (5 MPix monochrome) was mounted inside the box in a position two meter above the canopy surface.The imaged area was 1.0 m × 1.0 m.A LED light panel was mounted inside the box to produce nine spectral wavelengths (465, 500, 525, 590, 615, 625, 660, 740 and 850 nm) as the active light source for multispectral images.The inside of the inner box was painted white to obtain diffuse illumination of the canopy.An image session was carried out when series of LED flashes were strobed in all nine spectra within a second.Each strobe was synchronized with one image acquisition, i.e., an image session consisted of nine images.These images were then stacked into a multispectral image and stored in a HIPS format with a 2056 × 2056 pixels resolution.With an imaged area of 1.0 m × 1.0 m it gave a spatial resolution of 0.4280 mm/pixel.The commercial VideometerLab software package (v.2.12.8 (2850), Videometer ® ) controlled the two-box system, LED camera and imaging process.
The image session was carried out via as a series of automatized step in the VideometerLab software: (1) Lowering the outer box to the soil surface; (2) Automated stop when the outer box touched the soil surface; (3) After 10 s the LED flashes and image series were taken within a second; (4) Outer box was raised above the canopy to the top position; (5) The image session was displayed on the laptop to check if sunlight or wind had affected the image session.If so, the image session was repeated.The two-box construction was moveable in X-Y-Z dimension via a jib telescopic arm on a rebuild hydraulic stable lifter mounted on a tractor (Figure 1A).The images were acquired at the same height over the plots two meter above and perpendicular to the upper surface of canopy.
The acquisition of images included procedures that coped with illumination geometry, camera orientation, spectral and illumination redundancy, and optics.The acquisition also included a calibration procedure, trend correction and standardization of the images.The platform was calibrated between every fourth imaging campaign to adjust pixel geometry and basic spectral light setup as these settings would slightly drift due to movement of the box in the field and changing outdoor temperatures.Three calibration discs were used for the instrument pre-calibration; one uniform bright, one uniform dark, and one geometric disc which were white with a symmetric grid of black dots to adjust pixel settings in an entire image.At the startup of each imaging campaign in a specific experiment the light settings were calibrated based on a reference training plot.This procedure adjusted the intensity of the strobe time of each illumination type to suit the conditions of the plant population being measured.All calibration procedures were carried out by an intuitive step-wise procedure in the VideometerLab software.The capacity of the platform in the field experiment was approximately 80 images per hour in plot designs.

Image Analysis and Processing-Supervised Classification
The multispectral images were analyzed via a supervised classification method.This type of classification was a two-step process.First a classifier was subjectively trained on two pre-known objects (classes) located in the multispectral image.One class being the object with Region Of Interest (ROI) (e.g., green leaf) and the other class being objects to remove from overall segmentation (e.g., soil) also defined as noise.A simple painting tool in the software was used to mark classes in different colors.After classification the image was transformed using normalized Canonical Discriminant Analysis (nCDA).The classifier was then used to divide unknown objects in each pixel into one of the two classes [17].Classification and transformations was performed in VideometerLab (v.2.12.8 (2850), Videometer ® ).

Classification Tree
A classification tree (Figure 2) was used to stepwise separate the multispectral images into ROI, starting with foliage material (ROI 1), and soil background (Noise).The classification tree can be extended to separate other regions of interest (ROI 2, ROI 3, etc.), e.g., chlorotic foliage versus versus green foliage.One reference image for supervised classification was chosen randomly from an image campaign in a specific field experiment (Figure 3A).Via the supervised classification and the nCDA, the reference image was transformed into 2D image, in which each pixel had a normalized index value ranging between −2 and 2, with class of interest (ROI) having index 0 to 2 and class to be disregarded (Noise) index −2 to 0 (Figure 3B).Subsequently this 2D index image was segmented into a binary image based on an index threshold of 0 used on the nCDA image.Based on this procedure a binary image was generated so that it contained ROI class as foreground (white pixels) and the noise class as background (black pixels) (Figure 3C).The selected transformation and segmentation algorithms based on the reference image were subsequently applied to all the images from the same imaging campaign.

Parameters and Vegetation Indices
Figure 3 illustrates where in the image processing parameters were calculated.The Normalized Difference Vegetation Index (NDVI) was determined with an integrated session in the VideometerLab software based on the original image (Figure 3A).The Equation: NDVI = (660 nm − 850 nm)/(660 nm + 850 nm) (1) was used to calculate the NDVI of images.The vegetation coverage was calculated from segmented transformed images (Figure 3C) with two categories of pixel values either soil or non-soil pixels.An integrated session in VideometerLab was used for this: % vegetation coverage = number of white pixels/2056 × 2056 pixels (2)

Vegetation Coverage and NDVI in Spring Wheat
Two field experiments were conducted in 2013 on a sandy loam and clay loam soil respectively at the university farm Hoejbakkegaard, Taastrup, Denmark (WGS 84: 55°40′11.0″N 12°18′21.8″E).Four spring wheat varieties (A, B, C, D) with differences in earliness and height were sown on 17 April.Two different nitrogen regimes (100 kg N ha −1 and 200 kg N ha −1 ) were applied on 23 April.The experiment was a randomized block design with four replicates and each plot was 1.5 m × 12.0 m.An imaging campaign was carried out on four dates with the PF system.Within each plot images were taken at three different locations of one square meter each.Images were acquired on 15 May (BBCH 13), 30 May (BBCH 21 to 22), 12 June (BBCH 33) and 18 June (BBCH 40 to 57) The three locations per plot were geo-referenced via RTK GPS (Trimble FmX auto guidance system) via GPS antenna centered on top of the PF system to ensure that images were acquired from exactly the same position on each date.An Analysis of variance was applied to test the differences and interactions between varieties, soil type and nutrient regimes using the Equation: where Y is the estimated parameter (vegetation coverage and NDVI), i is the plot unit, V the variety, S the soil type and N the nitrogen level.Block was included as an additive factor to include block variation.The statistical analysis was carried out with open R software (R-studio v. 0.97.551).Table 1 summaries the statistical analyses of the results of applying Equation (3) to the parameters derived from the image campaign.There was a significant effect of variety on vegetation coverage on all 4 dates, whereas variety only had significant effect on NDVI on the first two dates.Soil type had a significant effect on vegetation coverage on the three first dates whereas the effect on NDVI was significant on all four dates.Except for 15 May, there were a significant effect of nitrogen regime on both vegetation coverage and NDVI.The test also revealed specific significant interactions of the three factors on specific dates.Figure 4 shows the vegetation coverage and NDVI of experimental factors on the four dates.The significant effect of soil type on both vegetation coverage and NDVI was generally highest on sandy soil on 15 and 30 May.Especially the difference in vegetation coverage on the two soil types was high on the first two dates.The difference between soil types remained similar for the vegetation coverage on 12 June and 18 June (18 June not significant), whereas it became the opposite for NDVI that was higher on clay loam compared to sandy loam on 12 June and 18 June.Generally the accelerated nitrogen regime of 200 kg ha −1 had a positive effect on both vegetation coverage and NDVI compared to 100 kg N ha −1 .Variety A and C generally had the lowest vegetation coverage and NDVI on 15 and 30 May opposite to variety B and D. On 12 and 18 June vegetation coverage remained lowest for variety C and slightly higher for variety.Varieties A and B had the same level of vegetation coverage and NDVI.

Date Vegetation Coverage (%) NDVI
Figure 5 shows the correlation between NDVI and vegetation coverage index.Generally, there is a close linear correlation between the two indices within the dates and soil types; however, the relationships are not the same on the two soil types.The correlation decreases along with the increase in vegetation coverage and NDVI, especially on the sandy loam.

Discussion
Multispectral imaging in general and PF in particular has great potentials as a high-throughput phenotyping platform in field plots.By multispectral images and multivariate image analysis it is possible to calculate several phenotypic parameters that all can be useful in the context of field phenotyping.However these parameters must be interpreted with care and selected thoroughly according to target investigation and crop development.The two simple parameters Vegetation coverage and NDVI were tested in the described field experiment.
By testing the PF platform under field conditions it has so far proven reliable and robust in regards of hardware, software, image quality, image analysis and parameter estimates.The results of vegetation coverage and NDVI (Figure 4) also showed robust estimates of varietal differences.It was possible to detect significant effects between variety, nitrogen regime and soil type when compared to vegetation coverage and NDVI in a multi-way analysis of variance (Table 1) which underlines that PF can become a valuable tool for field phenotyping.
Vegetation coverage is a simple method to quantify development of plant growth, also defined as vigor [18].According to [19] estimation of early vigor is important in cereal breeding, as an indicator of production of both biomass and final yield.Digital RGB images of wheat plants at early growth stages were found to be applicable for quantifying early vigor estimated by green pixel fraction.Even though it was stressed that the image acquisition and analysis process was an onerous factor [19] algorithms have been developed which in a reliable and simple way calculates green coverage in RGB images [20].This low cost approach could be an argument against using more sophisticated and high cost imaging systems like the PF.One limitation to open RGB systems is the effect of varying illumination during an imaging campaign.[15] Suggested to solve this problem with a flash, however, it required frequent calibration of the image colors.Another important problem with the RGB images was the blurring of the spatial information in the images caused by wind movements [15].Testing of the PF platform so far showed that the sunlight and wind was efficiently excluded from the measurements.
The high spatial and spectral resolution of multispectral images in contrast to broad band RGB images and measurements of only spectral reflectance, holds the potential to diagnose and quantify symptoms and epidemic development of diseases and nutrient deficiency [14,21].The applied perspective could be valuable for plant breeding that currently is based mainly on visual ratings [14].Due to high resolution and a controlled environment for multispectral imaging, PF is expected to be a very powerful tool for training classifiers to distinguish between different leaf tissue (Figures 2 and 3).
Due to its simplicity NDVI is a fast and simple index to estimate dynamics in biomass development.NDVI have proven valuable to relate biomass development with yield, light interception, evaluation of scenecense, nitrogen uptake and physiological responses related to chlorophyll contents [1,[22][23][24].NDVI is to a great extend influenced by canopy spatial structure, growth stage, vegetation density, sensor/camera angle, soil reflectance properties and illumination [22,23,[25][26][27] which may induce much redundant variation.Opposite to multispectral images and NDVI different low-cost RGB cameras have been tested to estimate nitrogen content in leaves based on visible color properties, e.g., calculating the Dark Green Color Index [28].This index correlated well with the chlorophyll measurements on maize leaves photographed under different outdoor lighting conditions (R 2 > 0.85) and leaf nitrogen content (R 2 = 0.8-0.89).The high correlation relied on a pre-calibration of the camera with green and yellow discs [28].The experiment was based on images from single leaf samples from corn plant.Therefore it remains to be investigated, what the efficacy of low-cost RGB images versus multispectral images is in more complex canopies.NDVI may be used to detect more subtle changes due to changes in canopy structure and biochemical components [15], and should therefore be more advantageous in complex canopies.This assumption was questioned by [22], where NDVI images indicated that only upper visible leaves in a canopy has a certain impact on NDVI.In this context it will be interesting to test the nCDA index (Figure 3B).So far this index has only been used for multivariate image analysis, yet as it is a linear transformation of spectral information in all nine bands, it may hold valuable biological information about leaf biochemistry and structure.This remains to be tested in 2014.
PF operates with a closed environment with standardized and diffuse illumination to avoid shading and effects of changing solar angels on vegetation indices which is expected to eliminate much redundant variation within these parameters.Earlier studies have shown that differences in soil color induce much variation on NDVI [29].It is expected that a uniform and diffuse illumination will minimize this influence.In this context another promising feature of the VideometerLab software is the possibility to set soil reflectance to zero in the multispectral images via transformation and segmentation procedures.Thereby the average reflectance in the image only reflects the canopy properties.This procedure will be tested in field experiments in 2014 in comparison to other reflectance sensors.
Vegetation coverage and NDVI are influenced by the crop growth stage (Figures 4 and 5).Hence it would be beneficial to include both phenotyping techniques under the same measuring conditions, so that data may complement each other.In Figure 4 it is clear that NDVI on clay loam followed the same pattern as vegetation coverage, whereas on the sandy loam with lower soil water capacity compared to the clay loam, NDVI develops opposite compared to vegetation coverage, both on 12 June and 18 June.Precipitation was very low in this period which visually affected the plants on the sandy loam.NDVI is effected by both structural changes as well as changes in pigments [30] whereas vegetation coverage in this article is based only on canopy structure observed perpendicular, which might explain why NDVI was more affected by water limitations than vegetation coverage.It must be mentioned though, that ears had emerged on variety B on 18 June on both soil types.This will have influenced NDVI in another way than for varieties A, C and D. Studies have proven a good correlation between NDVI and vegetation coverage index and this correlation decreases as vegetation coverage and Leaf Area Index (LAI) increases [15,28].A strong correlation between NDVI and vegetation coverage was calculated via the PF images at early crop growth stage (Figure 5).Calculated NDVI and vegetation coverage correlated strongly on both soil types on 15 May in an open crop, and correlation remained high on clay soil on 30 May whereas it decreased extensively on sandy loam.The high correlation compared to correlations made by [28], is an indication of PF being a powerful instrument for estimating both vegetation coverage and NDVI at early growth stages.It is however important to note, that weeds may contribute to vegetation coverage and NDVI at these early growth stages.Therefore, weeds should be strictly controlled in such experiments [18].
The novelty in the PF platform is the focus on bringing high precision into field phenotyping, by introducing uniform environments for imaging resembling those found in greenhouse facilities.Most research now focuses on high-throughput phenotyping and sensor fusion [3,8,15,16].Yet, as underlined in this article, much variance due to abiotic factors and hardware can influence the precision of the measurements.In comparison with [3,8,15,16] PF cannot directly be defined as high-throughput.Yet in comparison with a system such as the one tested by [15], where measurements were conducted in a narrow timespan around solar noon, the flexibility of the PF system, as it is independent of optimal conditions of natural illumination, will allow for more working hours available for imaging.Therefore it may be argued, that definitions of high-throughput phenotyping should also include possible operational time during the day as well as precision of estimates.
The PF system has limitations and disadvantages that have to undergo further investigation.First of all damage of the canopy structure due to lowering the outer box into the canopy especially at the late growth stages have to be investigated.Secondly, the system has a limitation to the possible throughput of plots per day due to the size of the platform and the time consumption with movement of the outer box during image acquisition.Thirdly, the system is based on multispectral imaging in only nine spectral regions, which may compromise the details of specific physiological changes that otherwise may be detected using hyperspectral imaging.Therefore, the PF system has to be considered a prototype with the aim to improve the setup of hardware and software as well as the experimental design and protocols.Finally, the platform is applicable on different plant species, but so far the system will have certain limitations such as to measure on crops on ridges due to the challenges with excluding external light on such soil surface.Furthermore, plants exceeding a height of 1.3-1.4m will be problematic to handle for the PF two-box system.
The implementation of multispectral imaging in general and PF in particular includes two major aspects in the 2014 work at the University of Copenhagen.One aspect is the entire infrastructure to enable robust phenotyping with the employed system.This is data security, statistical data analysis, data presentation and storage, automatic transfer of data, metadata and algorithms, detailed protocols on imaging and parameter interpretation.Also included are experimental designs and strategies for timing of imaging sessions.Such infrastructure is a crucial prerequisite to support efficient, dynamic, and intuitive phenotyping in field experiments.
The second aspect includes several tests of PF in a range of different experimental setups, which will include a combination of different crop species with genotypes, management, and environmental conditions.This is to test the robustness of the platform, the accuracy (ground truthing) compared to destructive sampling, and the precision based on repeated imaging in the same plots.Also, the precision and reliability in automatized image analysis based on few reference images requires additional testing and evaluation.So far the PF system shows great potential for this.Furthermore, it is the intention to make an approach towards sensor fusion to understand phenotypic development in even more details.This was clearly demonstrated by [16] and their mobile setup of sensors to measure canopy height, temperature and reflectance.The first step would be the fusion of multispectral imaging with a 3D laser system, which is a powerful tool to describe the structure of plants when influenced by a range of biotic and abiotic factors, especially in combination with spectral imaging to relate physiological responses to plant architecture [31].Further combination with thermal cameras should also improve crop phenotyping with respect to drought resistance, water use and stomatal behavior [7,32,33].Therefore, this technology will eventually be implemented as part of the PF system.

Conclusions
Vegetation indices and crop parameters comprises valuable phenotypic information that can be used in future breeding programs and crop management.A mobile and closed multispectral imaging platform applied in field experiments, showed a high degree of precision and accuracy due to the combination of spectral and spatial information.The software developed for this platform included a range of calibration procedures and industrialized tools for multivariate image analysis.This multispectral imaging platform eliminates several factors of noise, such as the effect of varying wind, sun and soil conditions, and therefore has the potential to estimate small phenotypic differences, e.g., between varieties.Two important phenotypic parameters are the vegetation coverage and the NDVI index which can be readily estimated via multispectral images.Results from this platform in a field experimental setup of different wheat varieties under two different nitrogen levels and two different soil types revealed significant differences in crop vigor and growth dynamics.Along with these traits the future development of this multispectral imaging platform should include sensor fusion of 3D laser scanners and thermal cameras to estimate canopy structure and other physiological parameters.

Figure 1 .
Figure 1.(A) Multispectral imaging box (1.3 m × 1.6 m × 2.5 m) mounted on a jib crane and in position above the canopy to initiate a multispectral imaging session.Box is movable in X-Y-Z dimensions; (B) Outer box (two-box system) is automatically lowered to soil surface before image acquisition in order to exclude sunlight and wind from the image area.Bottom frame is open to allow imaging of one m 2 .Top centered is a 5Mpix CCD camera two meter above crop surface, and the nine spectral LED panel is placed in the middle of the inner-box.

Figure 2 .
Figure 2. Classification tree.For each step in the classification tree, the procedure is to divide the image pixels into two classes of interest (supervised classification).Subsequently, the normalized Canonical Discriminant Analysis (nCDA) transformation is commenced.ROI (Region Of Interest) is the object of the image to be kept for further analysis and noise are objects in the image to be disregarded such as soil.

Figure 3 .
Figure 3. Images and parameters.(A) Original Multispectral Image from which spectral information and Normalized difference vegetation index (NDVI) can be derived; (B) Linear transformed and indexed (−2; 2) image via normalized Canonical Discriminant Analysis (nCDA); (C) Binary image from transformation and segmentation of the original image.Vegetation coverage can be derived from this binary image.Entire analysis is made by VideometerLab (v.2.12.8 (2850), Videometer ® ).

Figure 4 .Figure 4 .
Figure 4. Vegetation coverage (%) and NDVI of the varieties A, B, C and D under 2 different nitrogen levels (100 kg N ha −1 and 200 kg N ha −1 ) on two different soil types (Clay loam and Sandy loam) based on image campaigns performed on 15 May, 30 May, 12 June and 18 June.The vegetation coverage and NDVI values are the mean of three image series per plot in four blocks.Error bars indicate the standard deviations.

Figure 5 .
Figure 5.The relationship between the vegetation coverage and NDVI measured with the PF platform.

Table 1 .
Analysis of variance of vegetation coverage and NDVI measured with the PhenoField (PF) platform in the field experiment.