Time Series Field Estimation of Rice Canopy Height Using an Unmanned Aerial Vehicle-Based RGB/Multispectral Platform

Abstract

Crop plant height is a critical parameter for assessing crop physiological properties, such as above-ground biomass and grain yield and crop health. Current dominant plant height estimation methods are based on digital surface model (DSM) and vegetation indexes (VIs). However, DSM-based methods usually estimate plant height by growth stages, which would result in some discontinuity between growth stages due to different fitting curves. Additionally, there has been limited research on the application of VI-based plant height estimation for multiple crop species. Thus, this study investigated the validity and challenges associated with these methods for estimating canopy heights of multi-variety rice throughout the entire growing season. A total of 474 rice varieties were cultivated in a single season, and RGB images including red, green, and blue bands, DSMs, multispectral images including near infrared and red edge bands, and manually measured plant heights were collected in 2022. DSMs and 26 commonly used VIs were employed to estimate rice canopy heights during the growing season. The plant height estimation using DSMs was performed using different quantiles (50th, 75th, and 95th), while two-stage linear regression (TLR) models based on each VI were developed. The DSM-based method at the 95th quantile showed high accuracy, with an R² value of 0.94 and an RMSE value of 0.06 m. However, the plant height estimation at the early growth stage showed lower effectiveness, with an R² < 0. For the VIs, height estimation with MTCI yielded the best results, with an R² of 0.704. The first stage of the TLR model (maximum R² = 0.664) was significantly better than the second stage (maximum R² = 0.133), which indicated that the VIs were more suitable for estimating canopy height at the early growth stage. By grouping the 474 varieties into 15 clusters, the R² values of the VI-based TLR models exhibited inconsistencies across clusters (maximum R² = 0.984; minimum R² = 0.042), which meant that the VIs were suitable for estimating canopy height in the cultivation of similar or specific rice varieties. However, the DSM-based method showed little difference in performance among the varieties, which meant that the DSM-based method was suitable for multi-variety rice breeding. But for specific clusters, the VI-based methods were better than the DSM-based methods for plant height estimation. In conclusion, the DSM-based method at the 95th quantile was suitable for plant height estimation in the multi-variety rice breeding process, and we recommend using DSMs for plant height estimation after 26 DAT. Furthermore, the MTCI-based TLR model was suitable for plant height estimation in monoculture planting or as a correction for DSM-based plant height estimation in the pre-growth period of rice.

1. Introduction

Crop canopy height is a crucial growth parameter that can capture variations in yield, biomass, and growth stages of different crop varieties [1,2,3]. The height variability of crops provides important information on plant health, growth, and response to environmental effects [4]. Currently, traditional manual plant height measurements are taken in the field using a meterstick, which is time-consuming and labor-intensive [5]. This method fails to meet the demand for high-throughput monitoring of crop traits. Therefore, it is necessary to compare the existing methods of canopy height estimation to analyze their stability and applicability and develop standardized approaches.

Due to the lower costs and flexible working methods, an increasing number of researchers are employing unmanned aerial vehicles (UAVs) to gather multispectral data for estimating crop growth parameters [6,7,8]. The spectral features of crop canopies are closely related to visible and near-infrared vegetation indexes (VIs), which are significantly influenced by plant height [9]. For a decade, RGB-based and multispectral-based VIs have been commonly utilized for monitoring plant height. O. Payero, M. U. Neale, and L. Wright [10] discovered that the normalized difference vegetation index (NDVI), infrared percent vegetation index (IPVI), and transformed vegetation index (TVI) exhibited sensitivity to changes in alfalfa plant height during the early growth stage. Papadavid, Hadjimitsis, Toulios, and Michaelides [11] compared manual measurement results with the estimation results of potato plant height by NDVI, soil-adjusted vegetation index (SAVI), and weighted difference vegetation index (WDVI) and found that the linear model based on WDVI had the best potato plant height estimation, with an R² value of 0.89. However, variations in VIs could be different depending on crop variety, growth stage, and environmental conditions [12,13]. In addition, most of the plant height estimation methods based on VIs focused on similar or fewer varieties. For example, Zhen, Yunsheng, Moses, Rui, Li, and Jun [14] selected the tested wheat variety “Sumai 188” as an experimental material; Guo, Xiao, Li, Hao, Zhang, Sun, de Beurs, Fu, and He [15] selected the maize variety “Zhengdan 958” as an experimental material; and the VI-based plant height estimation for multiple species of rice has not been reported. Therefore, there is a need to evaluate the applicability of VIs in estimating the canopy height of multiple rice varieties.

On the other hand, the use of digital surface models (DSMs) obtained by UAVs for estimating crop canopy height has been widely examined [16]. DSMs include information about the height of ground surfaces, vegetation, and other objects. Demir, Sönmez, Akar, and Ünal [17] utilized a DSM of wheat to estimate canopy height, obtaining a t value of 1.82 for a statistical t test between the field measurements and plant heights retrieved from the DSM. Similarly, Zang, Wang, Yang, He, Zhou, Zheng, and Li [18] extracted wheat plant height using a DSM, and the R² and RMSE of the predicted plant height and the measured values were 0.96 and 6.32 cm, respectively. These results suggest that utilizing DSMs for extracting canopy height is an effective approach [19]. The current research method based on DSMs divided the plant height program into different time intervals, which could lead to some discontinuities between intervals due to different fitting curves [16]. Therefore, we hoped to give a uniform plant height estimation method based on DSM over the whole growth period of multiple rice varieties.

Additionally, no comprehensive comparison of plant height estimation methods utilizing DSM, RGB-based Vis, and multispectral-based VIs throughout the growth period has been reported. In light of these considerations, we utilized UAVs to collect RGB images and multispectral images throughout the entire growth period of multiple varieties of rice, analyzed the accuracy and consistency of the DSM-based method and the VI-based method for estimating rice heights throughout the entire growth stage, and investigated the effects of varieties and growth period on their performance.

2. Materials and Methods

2.1. Experimental Field

The experiment was conducted in 2022 at the experimental farm of the China National Rice Research Institute, located in Fuyang district, Hangzhou city, Zhejiang Province (119°55′ E, 30°04′ N). The experiment field is characterized by a subtropical monsoon climate. The physical and chemical properties of the soil before sowing were as follows: alkaline soluble nitrogen content of 115.37 mg kg⁻¹, effective phosphorus content of 14.20 mg kg⁻¹, fast-acting potassium content of 140.19 mg kg⁻¹, organic matter content of 30.48 g kg⁻¹, cation exchange capacity of 18.37 mg kg⁻², and pH value of 5.83. In this study, 550 plots were established, including 474 breeding lines, of which 4 varieties of 20 replicates were randomly placed in the fields. The rice seeds were sown in a nursery bed on 12 June 2022, and seedlings with three leaves were transplanted in mid-June with a spacing of 20 × 20 cm, with one seedling per hill. A total of 10 rows and 12 columns were planted in each plot with 40 cm spacing between plots (Figure 1). Field irrigation, pest control, and other management practices followed local high-yielding cultivation methods. From the final experimental records, selected breeding lines differed widely in growth stage characteristics and yield characteristics (Figure 2).

2.2. Field Data Collection

Image data collection using a DJI P4M quadrotor UAV (DJI Technology, Shenzhen, China) was conducted once a week before 10 a.m. to prevent excessive sunlight. The UAV platform was equipped with six 1/2.9-inch CMOS image sensors, including one RGB sensor for conventional visible light imaging and five monochrome sensors for multispectral imaging, including blue (B), green (G), red (R), red edge (RE), and near-infrared light (NIR). Radiometric calibration was performed using standard diffuse reflectance panels with 30% and 50% reflectance. The UAV flew at a height of 10 m and maintained a speed of 1 m per second during the flight. A heading overlap rate of 70% and a side overlap rate of 52% ensured sufficient coverage and overlap between consecutive images and adjacent flight lines, respectively. The camera captured images every 2 s along the flight direction, storing one RGB image and five spectral narrow-band images at 1600 × 1300 pixel resolution in JPG format on an SD card. The camera settings were configured to Auto mode, allowing for automatic control of color balance and exposure when capturing images. In total, 11 collection sessions were conducted under weather conditions that were clear and calm. However, on the 41st and 76th days after transplanting (DAT), data collection faced challenges due to heavy fog and continuous rainfall.

Rice plant height was measured once a week from 10 days after transplanting. Rice plant height was obtained as follows: five representative plants were randomly selected in each plot and the height from the top to the ground of each plant in the natural state was measured by a ruler. The average value of 5 plants was taken as the ground-measured plant height for each plot.

2.3. Image Preprocessing

The raw canopy images were stitched using DJI SmartMap software (V3.4, DJI Technology, Shenzhen, China) to create orthophoto maps, including RGB images, DSMs, and five single-band spectral images (R, G, B, NIR, and RE). Each single-band image had a size of 16,056 × 14,530 pixels and the DSM had a size of 8028 × 7265 pixels. To facilitate further analysis, the orthophoto maps covering the entire growth period of the rice varieties were divided into single-plot images. The Otsu threshold method [20] was used to separate the rice plants from the background. In this study, the digital values of the five single-band spectral images were utilized to calculate 14 RGB-based VIs and 12 multispectral-based VIs (Table S1) [15,21,22,23,24,25,26,27,28,29,30,31,32,33,34,35,36,37,38,39]. The mean values of the VIs were computed for each rice plot.

2.4. Plant Height Estimation Methods

In this study, the rice plant height was estimated using digital values in the DSMs and regression based on RGB-based VIs and multispectral-based VIs, as illustrated in Figure 3.

For the DSMs, we calculated the difference between the DSMs acquired during rice growth stages and the DSM acquired during the pre-planting (soil surface) period to estimate the plant height. Using masks on single-band images for DSMs would result in more errors due to the difference in spatial resolution between DSMs and other single-band images. According to the manual plant height records, the surface was considered the background when the relative bare soil height was less than or equal to 15 cm. We tested different quantiles (the 50th quantile, the 75th quantile, and the 95th quantile) to assess the stability of the method based on the DSM.

For VIs, we performed a one-way regression using VIs that are frequently used to represent changes in surface vegetation coverage and plant growth (Table S1). For canopy height estimation, we utilized a linear regression model as the base model. However, the relationship between the vegetation index and plant height varies in the early and late stages of growth [40]. To address this phenomenon, we constructed a 2-stage linear threshold regression (LTR) model to segment the entire growth period [41].

First, we considered the most straightforward linear model:

$$y=X\beta +\epsilon$$

(1)

where $X$ is the observation VI of rice lines/varieties over the target period, $y$ is the predicted plant height, $\beta$ is the parameter vector, and $\epsilon$ is the error vector. The least squares method estimates $\beta$. We sampled each plot from $t_1$ to $t_n$ and assumed the existence of a threshold $t_k$, turning Equation (2) into a child model:

$$y=\left\{\begin{array}{l}X{\beta }_1+{\epsilon }_1,i\le k\\ X{\beta }_2+{\epsilon }_2,otherwise\end{array}\right.$$

(2)

The general linear F test [42] can be used to assess the significance of adding a threshold to a model. The null hypothesis of the linear F test, H₀, states that there exists no significant nonlinearity in the training instances; thus, the parent model is sufficient to model the instances. The alternative hypothesis, H₁, states that there exists a significant nonlinearity in the training instances, and therefore, a threshold, $t_k$, needs to be set to split the model. The constructed F statistic is shown in Equation (3), where SSE₁ represents the sum of squared errors for Equation (1), SSE₂ represents the sum of squared errors for Equation (2), df₁ represents the degrees of freedom for Equation (1), and df₂ represents the degrees of freedom for Equation (2). The p value of the F test is determined by comparing the value of the F statistic to the F distribution. If the p value is less than the considered significance level (α = 0.05), then H₁ is accepted, the model is further split into the child models, and the optimal threshold ($t_k$) is the time at which the p value is the lowest.

$$F=\frac{(SS{E}_1-SS{E}_2)d{f}_2}{(d{f}_1-d{f}_2)SS{E}_2}$$

(3)

2.5. Data Analysis and Model Evaluation

The DSM-based method and VI-based method were performed using the Scikit-Learn and Seaborn packages in Python (version 3.7.0), and we compared the performance of support vector machines (SVMs), random forest (RF), and linear regression and TLR models. The models’ performance and generalization ability were evaluated using fivefold cross-validation [43]. The coefficient of determination (R²) and the root-mean-squared error (RMSE) were used to evaluate the plant height estimation model performance, and the R² and RMSE were calculated from Equations (4) and (5), respectively:

$$R^2=1-\frac{{\displaystyle \sum _{i=1}^n{(x_i-{\widehat{x}}_i)}^2}}{{\displaystyle \sum _{i=1}^n{(x_i-{\overline{x}}_i)}^2}}$$

(4)

$$RMSE=\sqrt{\frac{{\displaystyle \sum _{i=1}^n{(x_i-{\widehat{x}}_i)}^2}}{n}}$$

(5)

where $n$ is the total number of samples; $x_i$ and ${\widehat{x}}_i$ are the measured and estimated values of plant height, respectively; and ${\overline{x}}_i$ is the mean value of all samples.

3. Results

3.1. Rice Canopy Height Estimation Based on DSMs

The estimation results of rice canopy height based on DSMs are shown in Figure 4. The estimations at different quantiles produced similar overall effects, with R² values ranging from 0.88 to 0.94 and RMSE values ranging from 0.06 to 0.08 (Figure 4a–c). Among them, the best performance was achieved by estimating canopy height using the 95th quantile, with an R² of 0.94 and an RMSE of 0.06 m. At 13 DAT, the DSM-based canopy height estimation was consistently lower than the ground-measured plant height. The average RMSE for the three quantiles at 13 DAT was 0.10 m, with R² values less than 0, indicating a poor extraction performance (Figure 4d). The estimation effect at 22 DAT was initially poor but improved significantly with increasing quantiles. After 26 DAT, the differences in plant height estimation results at three quantiles started to narrow.

Figure 5 illustrates the distribution of the digital values in DSMs for the same plot between 13 and 82 DAT. At 13 DAT, the difference between the 50th and 95th quantiles was 20 cm (Figure 5a); at 26 DAT, the difference between the 50th and 95th quantiles was 30 cm (Figure 5b). The digital values in DSM gradually transitioned from a left-skewed distribution to a normal distribution as growth progressed, with the difference between the 50th and 95th quantiles being less than 5 cm (Figure 5c–f). By combining the DSMs-based estimation and the data distribution characteristics, it was recommended to use the DSM for estimating rice canopy height after 26 DAT.

3.2. Rice Canopy Height Estimation Using VIs

To explain the phase transition of the vegetation indexes and rice canopy height, we took a sample of 100 randomly selected plots and chose temporal trends of the OSAVI, IKAW, and measured height as examples (Figure 6). Before 26 DAT, the OSAVI exhibited a relatively fast growth rate and displayed a positive correlation with measured rice height. However, after 26 days, the growth rate of the OSAVI slowed. Before 40 DAT, the IKAW experienced a rapid decrease and showed a negative correlation with measured rice height. After 40 DAT, the overall change in the IKAW was minimal, with notable fluctuations observed only at 56 DAT. However, the growth trend of measured height started to slow only after 56 DAT, which contrasted with the earlier stages when the OSAVI and IKAW showed transitions. These observations highlighted the presence of stage differences in the correspondence between canopy height and vegetation indexes.

The plant height estimation results of different models based on each VI are shown in Figure 7. The results showed that the LTR model (mean R² = 0.686) performed significantly better than the linear model (mean R² = 0.381), SVM (mean R² = 0.364), and RF (mean R² = 0.552). Among the vegetation indexes used in the LTR model, the best-performing index was MTCI, with an R² value of 0.704, while the worst-performing index was IPVI, with an R² value of 0.655. This suggested that the MTCI strongly correlated with the actual measured height.

Figure 8 provides a comparison of the two stages of the LTR model based on R². The results indicated that the first stage of the model (mean R² = 0.518) performed significantly better than the second stage (mean R² = 0.099). The MGRVI-based LTR model had the best results in the first stage (R² = 0.664), but the R² in the second stage was only 0.104. This indicated that the VIs were more effective in estimating canopy height at the early stage than at the second stage during growth.

To analyze the role of variety on the regression effect, we first proposed a two-stage regression model for each variety. Next, the k-means clustering algorithm was used to group similar varieties into 15 clusters based on their regression characteristics (β₁, β₂ and t_k). After clustering, the LTR model was constructed for each cluster of varieties. The R² values of the VI-based TLR models in different clusters are depicted in Figure 9. The R² values of the VI-based TLR models exhibited inconsistencies across groups, with the most significant fluctuation observed in the R² values of the IPVI-based TLR model, ranging from 0.042 to 0.909. Notably, the R² value of the EXG-based TLR model for cluster 14 reached a maximum of 0.984, which was markedly higher than the R² value of the model developed for all species (R² = 0.701). And there was always a vegetation index under each clustering cluster that was better than the DSM-based plant height estimation. This indicated that the regression model based on the VI was more suitable for estimating the plant height of an individual or a group of closely related rice varieties.

4. Discussion

As a fundamental growth parameter for crops, the high-throughput measurement of plant height contributes to optimizing crop breeding and management methods. However, current plant height estimation methods have limitations regarding applicable crop varieties [15,44] and growth stages [16]. Therefore, we conducted an analysis by collecting entire growth period data for 474 rice varieties to analyze the applicability and limitations of DSM-based methods and regression methods based on VIs for estimating canopy height.

The DSM-based method, specifically using the 95th quantile, demonstrated high accuracy, with an R² value of 0.94 and an RMSE of 0.06 m. The overall effects of estimating plant height using DSMs at different quantiles were consistent (Figure 4). However, it should be noted that the current plant height estimation methods mainly focused on the period after tillering [16,45] and did not consider the influence of this stage on plant height estimation. Our study found that before 26 DAT, the estimation effect varied more among different quantiles (Figure 4d), which could be attributed to the growth process of rice. During the early stages of growth, rice underwent leaf extension, resulting in a small proportion of complete leaves in the canopy image. As the tillering stage began, rice plants produced tillers from leaf axils [46], leading to an increase in the proportion of complete leaves in the image. Consequently, the estimation effects of canopy height at different quantiles gradually stabilized. Therefore, we recommended using the DSM-based method for rice plant height estimation after 26 DAT. Notable changes occurred at 41 DAT and 76 DAT due to specific environmental conditions during image acquisition. At 41 DAT, there was rainfall after manual measurement, causing the leaves to sag when the UAV was flying. At 76 DAT, the presence of fog affected the signal propagation. Consequently, the accuracy of the reflected signals was compromised, leading to a decrease in their reliability. These observations highlighted the impact of environmental factors on the accuracy and reliability of DSM-based canopy height estimation [47]. Furthermore, the DSM-based estimation method showed approximate results among the clustered clusters of different varieties, indicating that the DSM-based method was less affected by the varieties (Figure 9). And it was worth noting that the mean estimated plant height for all samples after 62 DAT was lower than the actual measured plant height (Figure 4). In Figure S1, we present the plant height variation curve based on DSM estimation for selected samples, which showed that the peak plant height occurs around the full heading stage, suggesting that the underestimated values might be due to canopy aging during the grain filling stage.

In the VI-based regression method, the two-stage linear threshold regression model showed better performance compared to the traditional linear regression model [40]. The mean R² value of the two-stage model was 0.686, outperforming the mean R² value of 0.381 for the traditional linear regression model. The two-stage model effectively captured the two-stage relationship between vegetation indices and plant height (Figure S2). However, the effects of the models after clustering were not consistent across clusters (Figure 7), which highlighted the importance of varietal differences, e.g., the rice varieties and canopy optical characteristics [48], in applying the regression effect of VIs in estimating plant height. Therefore, the VI-based regression model might appropriate specifically for estimating canopy height for a single variety or a group of rice varieties with similar traits. It is important to note that the clustering method used in this study might not provide accurate results for unknown varieties and may not be appropriate for classifying multiple species based on plant height. The two-stage regression model based on the VIs performed the best on MTCI, with an R² value of 0.704. The MTCI index was specifically designed to be sensitive to changes in chlorophyll content [21], which was strongly correlated with vegetation growth and development. As plant height increased, chlorophyll content typically increased as well. By focusing on this sensitive parameter, the MTCI might be able to provide more accurate estimations of plant height. Furthermore, the regression effect of the first stage was better than that of the second stage, with mean R² values of 0.616 and 0.104, respectively, which could be that vegetation indices tended to saturate earlier than plant height (Figure 6). This result indicated that the VI-based regression model would possibly be more suitable than the DSM-based method for estimating plant height in the pre-growth period.

5. Conclusions

This study aimed to assess the effectiveness of DSM-based and VI-based methods for estimating the heights of multiple rice varieties throughout the entire growth stage. The results demonstrated that the DSM-based method, particularly using the DSMs at the 95th quantile, achieved high accuracy in estimating rice plant height. However, the DSM-based method showed relatively poorer estimation results during the early growth stage. Consequently, considering the trend of digital values in DSMs, we recommend utilizing DSMs for plant height estimation after 26 DAT. Additionally, it should be acknowledged that adverse weather conditions might impair the accuracy of DSM measurements. The two-stage linear threshold regression methods based on VIs were found to be suitable for estimating plant height in the early growth stage, especially for a single species or a group of similar species. But, in the context of significant varietal differences, the estimation results were not as precise. Furthermore, for specific clusters, the VI-based methods outperformed the DSM-based methods in plant height estimation. In conclusion, the VI-based TLR model was recommended for estimating plant height in monoculture plantings, and the DSM-based model using the 95th percentile was recommended for estimating plant height in multi-variety crops in further research.