Legume-grass mixtures with red clover (Trifolium pratense
L.) and lucerne (Medicago sativa
L.) in combination with Italian ryegrass (Lolium multiflorum
Lam.), grown for 1–3 years, play an important role in crop rotations, particularly in organic farming in temperate European climates. Such crops serve as green manure by fixing nitrogen or as feedstock for animals and biogas plants. These intensive grasslands are harvested three to five times per year and, therefore, an early and precise yield estimation is mandatory for management decisions and economic optimizations at the field and farm level [1
]. An emerging and promising technology in grassland faming is remote sensing, and especially the use of different sensors on unmanned aerial vehicles (UAV) show great potential for improving agricultural use of grasslands [2
Destructive biomass sampling is considered to be the most accurate yield estimation method but can also be considered as the most labor-intensive method [4
]. Another approach for estimating biomass in grasslands is the assessment of canopy height (CH), which was frequently found to be positively correlated with crop biomass [5
]. Traditional height measurements in grassland are often conducted with a rising plate meter, determining the compressed sward height, or with a ruler stick [7
]. Furthermore, several portable technical devices for non-destructive biomass estimation were developed in the recent years, which so far were not widely distributed in agricultural practice, e.g., leaf area meter to asses leaf area index (LAI) [9
], electronic capacitance meter, which measures the difference of capacitance between air and biomass [1
] and a reflectometer, which measures intensity of spectral reflectance by light emitting diodes (LED) [10
]. Biomass sampling, manual height measurement and the above mentioned technical devices need a substantial number of repetitions in combination with a spatially uniform distributions of the measurements to generate a reliable yield estimation [11
]. Therefore, much time and effort is required to receive reliable data especially on large areas.
Sensors attached on UAVs are useful non-destructive tools for obtaining spatial information from large and remote areas. There exist different sensor systems, such as LiDAR, ultrasound and RGB (red, green, blue) imaging to collect spatial data for a rapid quantification of aboveground biomass [3
]. A UAV in combination with a consumer-grade digital camera for RGB imaging represents a low-cost approach for estimating yield, which may be affordable und workable for farmers. By photogrammetric structure from motion (SfM) processing of the UAV-based images into point clouds, 3D spatial data can be easily generated. Forsmoo et al. [14
] calculated in a grassland sward, that for a plot of 8 m2
a CH assessment with a ruler needed 550 single height measurements to receive the same accuracy as a SfM-based CH assessment based on UAV RGB imaging. Therefore, the high spatial resolution makes UAV RGB imagery in combination with an SfM approach an interesting tool for yield estimation in practical grassland farming.
SfM derived height measurement based on UAV RGB imaging was successfully used in forestry [15
], and, to a lower degree also in agricultural crops, such as wheat [6
], barley [17
], maize [18
] and vegetable crops [20
]. All these studies found strong relationships between biomass and RGB imaging in homogeneous crops. Contrary, grassland represents a mixed crop containing legumes and grasses of several species. Additionally, species contribution and yield changes in the field throughout the growing season and is affected by many factors, such as cutting intensity, soil features, fertilization and climate conditions [21
]. So far, only a few studies exist using SfM based on RGB imaging in grassland. Cooper et al. [23
] and Wallace et al. [13
] compared height measurements in permanent grasslands based on terrestrial SfM and terrestrial LiDAR and observed similar relationships between these two methods and the grassland biomass. Van Iersel et al. [24
] showed the potential of modeling temporal dynamics of CH in a floodplain grassland and of classifying different vegetation types (pixel size = 5 cm) with a consumer-grade camera attached on a UAV. Viljanen et al. [25
] worked in an intensive grassland with four harvest cuts and different nitrogen applications rates. The latter used a high-resolution camera (pixel size = 1 cm) mounted on a UAV and combined CH with spectral vegetation indices (VI), resulting in highly significant correlations with biomass. VIs from multi- or hyperspectral sensors also show promising potential for qualitative and quantitative biomass estimation; however extensive spectral calibration work is necessary for these technique [3
To understand the spatial variability and dynamics in grasslands over the entire growing season multi-temporal studies are needed. So far, no study using yield prediction models by SfM considered different proportions of legumes (including pure legume and grass stands as well as legume-grass mixtures), which frequently occur in practical grassland farming. The aim of this study is the development of estimation models for dry matter yield (DMY) in mixed grassland by CH generated by UAV RGB imaging over a whole growing season. The study compares this novel remote sensing technique with ruler height measurements, an established conventional method and uses destructive biomass sampling as a reference data. All measurements were conducted in two different legume-grass mixtures. To cover the wide range of legume contribution in practical grassland, pure stands of legumes (100% legumes) and grasses (0% legumes) were investigated as well as their mixtures. The specific objectives of this study were: (1) the assessment of CH by RGB imaging in two legumes-grass mixtures over the whole growing season; (2) the comparison of prediction models based on an SfM approach with a conventional measurement method; (3) the development of a yield prediction model containing also pure stands of legumes and grasses; (4) assessment of the prediction accuracy for the total annual DMY based on SfM including all harvest cuts during one complete growing season.
2. Materials and Methods
2.1. Experimental Site and Design
The study was carried out at the experimental farm in Neu-Eichenberg of the Universität Kassel (51°23’ N, 9°54’ E, 227 m above sea level) in northern Hesse, Germany. The soil is a silty clay loam with 3.6% sand, 73% silt, 23.4% clay and 2% humus. Summer barley was cultivated as a preceding crop. The mean annual precipitation and daily temperature of the site is 728 mm and 8 °C, respectively (Figure 1
Field plots (1.5 m × 10 m) of clover-grass (CG) and lucerne-grass (LG) in mixtures as well as of pure stands of legumes (L; 100% legumes) and grasses (G; 0% legumes) (Table 1
) were sown in autumn of 2016 in four randomized replicate blocks, giving a total of 24 plots (Figure 1
C). All treatments were sown with a total seed rate of 35 kg ha−1
. CG contained 60% Lolium multiflorum
, 30% Trifolium pratense
, 5% Trifolium hybridum
L. and 5% Trifolium repens
L., whereas LG included 40% Medicago sativa
, 20% Festuca pratensis
Huds., 15% Lolium perenne
L., 10% Lolium multiflorum
, 10% Trifolium pratense
and 5% Phleum pretense
L. Consequently, the treatments formed a set of heterogenous vegetation which differed largely in their morphological and optical characteristics. As the experimental farm is managed organically, no fertilizer or pesticides were applied.
The total amount of rainfall (762 mm in 2017) was higher than the mean annual precipitation (728 mm), though early summer was characterized by a severe drought. Precipitation for the first cut (17 May 2017) revealed 66 mm compared to long-term average of 91 mm and for the second cut (26 June 2017) 47 mm compared to long-term average of 95 mm. This drought caused an accelerated maturation of the grasses with pronounced stem formation and little leaf growth especially in the second cut.
2.2. RGB Remote Sensing and Data Acquisition
RGB images were taken with a low-cost quadrocopter (DJI Phantom 3 Advanced; Shenzhen, China). The flight plan was done with autopilot by means of Pix4Dcapture software (App version 4.4.0, Pix4D SA, Lausanne, Switzerland). All missions were carried out in the morning to ensure equal sun position (8:00–12:00 a.m.). RGB images were taken one day before every harvest (4 flights) and during growth every second week (6 flights). A standard digital camera (DJI FC300S, DJI, Shenzhen, China) was mounted on a gimble and had a f/2.8 lens with a 94° field of view and 12 megapixels. For each mission, images with a forward and side overlap of 80% were taken in a grid pattern at a constant flying height of 20 m, resulting in 300 to 400 individual images with a spatial ground resolution between 7 and 8 mm per pixel. Seven wooden targets, painted black and white (10 × 10 cm, cross-centered) and mounted on small tripods, were used as portable ground control points (GCPs) for georeferencing the generated point clouds at each flight. The GCPs were evenly distributed and set up into balance on the pathways between the experimental plots. The GPS (global positioning system) coordinates were measured using a Leica RTK DGPS with a horizontal and vertical precision of 2 cm. Additionally, the whole experimental field was georeferenced once by recording the coordinates of the corners of each plot.
Subsequent to each flight mission, manual height measurements (CHR
) and destructive biomass samples were taken. The mean grassland height per plot resulted from 50 randomly distributed measurements, which were conducted with a ruler at a precision of 0.01 m. The height was defined as the vertical distance from the soil surface to the highest point of the plant which touched the ruler [26
]. Plots were harvested four times (17 May 2017, 26 June 2017, 8 August 2017, 9 October 2017) with a Haldrup forage plot harvester at a stubble height of 5 cm and a cutting width of 1.50 m. Prior to harvest two biomass samples for fresh and dry matter yield were taken manually in every plot on an area of 50 × 50 cm each. To enrich the calibration database with data from less mature sward conditions sub-samples were taken every second week during sward growth between 17 May 2017 and 9 October 2017 on an area of 25 × 25 cm in every plot (Table A1
). To avoid sampling effects, sub-samples were taken in the first 1.50 m of the plots, whereas the remaining plot area remained untouched during growth. All biomass samples were dried at 105 °C to constant weight (~ 48 h), to determine dry matter content and to calculate dry matter yield (DMY).
2.3. Data Processing and Analysis
Three-dimensional (3D) point clouds were generated from the RGB images for each dataset using the SfM approach. The software Agisoft PhotoScan Professional (Agisoft LLC, St. Petersburg, Russia) was used to calculate the digital surface model (DSM) from the RGB images. The SfM algorithm obtains 3D information from 2D images and converts the images automatically into a DSM. All processing steps were executed separately for a better control, using uniform settings on a high-performance computer (Figure 2
After importing a dataset, the overlapping images were aligned by internal image matching techniques and algorithms. Location and orientation of the individual images of the moving camera were automatically determined. The alignment was done with the accuracy setting “medium” and a key point limit of 40,000 and tie point limit of 4000. The output was a sparse point cloud containing a certain amount of paired multidimensional points of all images, which were linked together by identical features.
For the optimization step, the GPS coordinates of the GCPs were imported using the coordinate system WSG 84. The sparse point cloud was georeferenced manually by determining three pictures for each of the seven GCPs and placing the renumbered GCP marker on the cross center of the targets. After that, the software estimated the positions of the GCP marker for the other images automatically and, when needed, the markers were adjusted and placed manually. The spatial error of the GCPs varied between 1 and 2 cm. Additional optimizing of the sparse cloud was done by enhancing the camera lens parameters. With these optimized settings, the image alignment, the location and orientation of images and, eventually, the sparse point cloud was updated and corrected.
In the next step the georeferenced sparse point cloud was converted into a dense point cloud. The software computed the depth information by the image alignment for all points of the images. For that, medium quality settings were used to keep the processing time at an acceptable level and depth filtering was set to “aggressive” to sort out outlier points due to noise or inaccurate focusing. These steps resulted in one single point cloud, which was much denser and more detailed.
In the last step the dense point cloud was exported in the form of a DSM as a TIFF file with a resolution between 1 and 3 cm per pixel. The DSM represented the recorded surface as a raster data.
Further processing was done with Quantum Geographical Information System (QGIS 2.18.14, QGIS Development Team, Raleigh, NC, USA) software to provide a digital elevation model (DEM). After importing the raster image, ground points in the pathways next to the plots were selected from the DEM, which were interpolated by an inverse interpolation with a power of 3 to provide a continuous ground surface model over the whole field. CH was calculated by subtracting DEM from DSM. The coordinates of the plot corners were used to delimit each plot area. The mean height value for each plot from the drone-based RGB imaging (CHD) was extracted by zonal statistics. These processes were done for every flight mission separately. From the whole dataset seven CHD values had to be removed due to unrealistic negative height values.
2.4. Statistical Analysis
Statistical analysis was performed using R programming language version 3.5.1 (R Foundation for Statistical Computing, Vienna, Austria). DMY was tested for normal distribution and its residuals for homoscedasticity. As these assumptions were not fulfilled, DMY was square root transformed. ANOVA was used to detect differences among yields of the four cuts in each treatment.
For creating estimation models for the whole dataset (including the subsamples), CH and DMY were used for linear regression models. One main assumption for an ordinary least square (OLS) regression (Type-I regression model) is the allocation of y
as dependent variable and x
as independent variable. Another assumption for OLS is the measurement of the independent variable x
without error. In our Study it was not feasible to distinguish between x
for CH and DMY and as x
(i.e., is CH) was measured both manually and by RGB imaging, the height measurements cannot be considered as error-free. In this case, a reduced major axis (RMA) regression, which is a Type-II regression model, was suggested by Cohen et al. [27
] for analysis in the field of remote sensing. Though slope, intercept and root mean square error (RMSE) are calculated differently, RMA shows the same coefficient of determination (R2
) as OLS. In the current study, RMA was used to predict DMY by CHR
for the whole dataset and for the treatments separately. For a better understanding of the graphical presentation of the yield estimation model, the equations were re-transformed to the original scale.
For an assessment of the model accuracy, cross-validation was carried out. The dataset was split into a calibration (training) and validation dataset. To generate an even distribution of the randomly chosen validation dataset, one value was withdrawn from each treatment and sampling date, resulting in a 75% calibration and 25% validation dataset. The calibration dataset was used to generate the corresponding models and the coefficient of determination (R2cal
), the root mean square error (RMSEcal
) were calculated. The validation dataset was used as an independent dataset to verify the calibrated models by linear regression between measured and predicted DMY, represented by coefficient of determination (R2val
) and root mean square error (RMSEval
) as well as relative RMSEval
). Additionally, to determine the accuracy of the validation between measured and predicted DMY Willmott’s refined index of agreement (d
) was generated. Willmott’s refined index of agreement is a dimensionless value between 0 and 1 indicating no agreement and total agreement, respectively [28
The primary aim of this study was to develop and evaluate a prediction model for DMY of heterogeneous temperate grasslands by means of SfM using UAV-based RGB imaging. CH was successfully predicted both by manual and SfM height measurement. SfM-based CH was on average 4 cm lower than the manually measured values. The same tendency was found in previous studies with barley with a difference of 10 cm [17
] and 19 cm [29
], respectively, though both investigations used a higher resolution of 1 cm per pixel. In our study, manual measurements represented 50 single points from the ground soil to the highest point touching the ruler. In contrast, SfM datasets covered the whole area of interest, scanning the complete visible canopy surface and not only the top of single plants [17
]. Furthermore, the nadir position of the camera with a resolution of 2 cm per pixel may not have captured every single grass tiller, especially at windy conditions, and the strong depth filtering during the generation of dense point clouds may have already caused an exclusion of single outlying points. All this led to a generally lower average CH compared to the manual measurements and is also supported by the finding that the correlation between SfM-based and manual height measurement improved, when the data of the extreme mature grasses of the second cut were excluded. Cunliffe et al. [30
] showed that in dryland vegetation ultra-fine resolution of less than 1 cm of the height model was able to depict single grass stems.
The prediction models for DMY based on the calibration dataset of CHR
showed similar accuracies across all treatments with R2
= 0.58–0.80 and R2
= 0.62–0.81, respectively. Most other multi-temporal studies reporting on SfM-based prediction models by means of UAV RGB imaging were conducted with arable crops. Correlation results were comparable for barley with an R2
= 0.82 [17
], for winter wheat (R2
= 0.68–0.95) [16
] and for maize and sorghum (R2
= 0.68–0.78) [31
]. Moeckel et al. [20
] achieved substantially higher R
² values with stands of tomato, eggplant and cabbage (R2
= 0.89–0.97), which represent more heterogeneous crops. Roth and Streit [32
] examined different cover crops, including also two clover species, and achieved an R2
of 0.58. When plants, which were growing close to the ground or even lodging, were excluded from the regression model, R2
increased to 0.74. In a permanent timothy-dominated grassland Viljanen et al. [25
] obtained a pearson correlation coefficient between 0.77 and 0.97. They showed that the quality of the estimation model depended on plant density and growth stage of the sward. Bendig et al. [33
] achieved the best results in summer barley prior to heading of the crop. Malambo et al. [31
] and Grenzdörffer [34
] showed that the SfM approach worked better for uniform crops than for crops with a heterogeneous canopy surface. Compared to other arable crops, which are usually grown in monocultures (e.g., cereals, maize), clover- and lucerne-grass mixtures form a rather heterogeneous canopy with a wide range of coverage of the two components, which above all, have a very different stature.
Several studies indicated that plant density influences the estimation of plant height [6
]. In our study, the second cut was markedly affected by drought, which resulted particularly for grasses in a low tiller density and less leaf biomass. The exclusion of the data of the pure grass stands of the second cut improved the correlations for both methods. A camera with a higher resolution to capture more details of the canopy surface [30
] and the integration of plant density as additional information may further improve the prediction accuracy in heterogeneous crops. In a study of Schut et al. [37
] the combination of spectral indices with remotely sensed CH information showed promising results at small-scaled farm level. In other studies fusion of 3D LiDAR and spectral data substantially improved biomass prediction in extensively manage permanent grasslands compared to the use of single sensors [5
Accuracy of biomass estimation by RGB imaging may be reduced due to errors and uncertainties during the image and point cloud generation. The seven negative CHD values, which were excluded from the present analysis, were located in an area of the experimental site, where a slight slope occurred. In our study, the DEM was generated by the interpolation of z-values from points located in the pathways between the plots. As the biomass at the erased points were extremely low, negative height values may, thus, have been caused by an inadequate representation of the true ground surface in the interior of the plots. With a small and rather flat area like our experimental field the error due to insufficient interpolation is relatively low, whereas for practical and possibly uneven fields of several hectares explicitly generated DEM point clouds may be necessary for a sound calculation of canopy heights. This can e.g., be done during periods of bare soil, e.g., before sowing or after harvest of the crop. However, the production of additional pre-sowing or post-harvest point clouds requires extra flight missions and a considerable effort of data analysis. Tests which were conducted prior to the experiment indicated that extremely diligent georeferencing of the point clouds with an adequate number of georeferenced control points is necessary to avoid errors and uncertainties in the CH data.
In practical forage production strategic decisions (e.g., for adjusting the number of farm animals to the amount of available roughage) are usually taken based on data from one year or growing season. Thus, generating reliable data on the expected total annual biomass produced from single fields would substantially support farmers’ decisions. As a first approach models were developed to predict total annual yield separately for the clover- and lucerne-grass stands. As the legume contribution of legume-grass mixtures varies greatly in farming practice [38
], the corresponding pure stands of legumes and grasses were also included in both models. The fact, that model accuracy was different for clover- (R2
= 0.75) and lucerne-grass (R2
= 0.64) data indicates that, morphological characteristics (e.g., leaf position, vertical or horizontal distribution) of clover canopies allow a better prediction of biomass than for lucerne-grass mixtures, and that these crop-specific models cannot be easily transferred to other grassland types. The reasons behind this finding cannot be determined with the present study and, thus, further investigations (also considering model evaluation on the field scale) are needed. However, it is encouraging that averaged across all treatments, the accuracy between measured yield and UAV-based yield assessment was similar to manual height measurement.
Data acquisition by UAV can be done in a relative short time, whereas data processing by SfM is a computationally intensive process, depending on computer performance and number of images. In our study, data acquisition for each sampling date by manual height measurement, which was a rather high number of measurements (50 measurements per plot ~ 3 h) and UAV-based RGB imaging including data processing yielded in a similar amount of time (~ 3–4 h). So far, a practical implementation does not seem feasible, but as flight time of UAVs, computer performance and automation of data processing steadily increases, this method is seen to have great potential. To summarize, the results indicate that UAV-based RGB imaging may serve as a suitable estimator for total annual yield of heterogeneous legume-grass mixtures, which future studies should evaluate.