Next Article in Journal
Diverse Geological Evolution of Impact Basins on the Moon
Previous Article in Journal
Application of Resistivity and Seismic Refraction Tomography for Landslide Stability Assessment in Vallcebre, Spanish Pyrenees
 
 
Search for Articles:
Title / Keyword
Author / Affiliation / Email
Journal
Article Type
 
 
Advanced Search
 
Section
Special Issue
Volume
Issue
Number
Page
 
Journals
Remote Sensing
Volume 14
Issue 24
10.3390/rs14246334
remotesensing-logo
Submit to this Journal Review for this Journal Propose a Special Issue
► Article Menu

Article Menu

share Share announcement Help format_quote Cite question_answer Discuss in SciProfiles thumb_up
...
Endorse textsms
...
Comment
first_page
settings
Order Article Reprints
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

Predicting Wheat Leaf Nitrogen Content by Combining Deep Multitask Learning and a Mechanistic Model Using UAV Hyperspectral Images

by
Xiao Ma
1,2,
Pengfei Chen
1,3,* and
Xiuliang Jin
4
1
State Key Laboratory of Resources and Environment Information System, Institute of Geographic Sciences and Natural Resources Research, Chinese Academy of Sciences, Beijing 100101, China
2
University of Chinese Academy of Sciences, Beijing 100049, China
3
National Earth System Science Data Center, National Science and Technology Infrastructure of China, Beijing 100101, China
4
Institute of Crop Sciences, Chinese Academy of Agricultural Sciences, Beijing 100081, China
*
Author to whom correspondence should be addressed.
Remote Sens. 2022, 14(24), 6334; https://doi.org/10.3390/rs14246334
Submission received: 1 November 2022 / Revised: 9 December 2022 / Accepted: 10 December 2022 / Published: 14 December 2022
(This article belongs to the Topic Applications of Big Data and Machine Learning in Smart Agriculture)
Browse Figures
Versions Notes

Abstract

:
Predicting leaf nitrogen content (LNC) using unmanned aerial vehicle (UAV) images is of great significance. Traditional LNC prediction methods based on empirical and mechanistic models have limitations. This study aimed to propose a new LNC prediction method based on combining deep learning methods and mechanistic models. Wheat field experiments were conducted to make plants with different LNC values. The LNC and UAV hyperspectral images were collected during the critical growth stages of wheat. Based on these data, a method combining the deep multitask learning method and the N-based PROSAIL model was proposed and compared with traditional LNC prediction methods, including spectral index (SI), partial least squares regression (PLSR) and artificial neural network (ANN) methods. The results show that the new proposed method obtained the best LNC prediction results, with R2, RMSE and RMSE% values of 0.79, 20.86 μg/cm2 and 18.63%, respectively, during calibration and 0.82, 18.40 μg/cm2 and 16.92%, respectively, during validation. The other methods obtained R2, RMSE and RMSE% values between 0.29 and 0.68, 25.71 and 38.52 μg/cm2 and 22.95 and 34.39%, respectively, during calibration and between 0.43 and 0.74, 22.79 and 33.55 μg/cm2 and 20.96 and 30.86%, respectively, during validation. Thus, this study provides an accurate LNC prediction tool for precise nitrogen (N) management in the field.

1. Introduction

Nitrogen (N) is an important nutrient element that is necessary for wheat growth and development. N plays an important role in improving crop photosynthetic capacity and assimilation products. Due to the large spatial variations in soil N content in fields, farmers tend to use excess N fertilizer to ensure wheat yield. However, excessive N in fields not only reduces wheat yield and quality [1], but also negatively affects the global ecosystem [2]. Therefore, obtaining the N nutrition status of wheat over time and applying N fertilizer reasonably according to plant needs is of great significance.
Leaf nitrogen content (LNC) is an important indicator for determining wheat N nutrition status. The traditional LNC detection methods depend on destructive sampling and labor-intensive analyses in the laboratory, which are time-consuming and expensive [3]. Previous studies have shown that crop biophysical and biochemical parameters can be detected by remote sensing technology [4,5]. In recent years, the rapid development of unmanned aerial vehicle (UAV) remote sensing technology has provided an opportunity to obtain high spatial and time-resolution images in a flexible manner [6], which is extremely suitable for precisely managing fields.
At present, the prediction methods for crop biophysical and biochemical parameter inversion include empirical methods, mechanistic methods and hybrid methods. Considering LNC prediction, empirical methods apply quantitative regression models between the canopy spectral reflectance and LNC using calibration datasets to predict the LNC in the target area. Zhang et al. used UAV hyperspectral images to compare the performance of partial least square regression (PLSR), generic algorithm integrated with the PLSR (GA–PLSR), random forest (RF) and extreme gradient boosting (XGBoost) methods for LNC prediction in alpine meadow ecosystems [7]. Based on field measured hyperspectral data, Jay et al. used the spectral index (SI) method to predict the LNC of sugar beet [8]. Empirical methods are easy to apply but lack mechanistic explanations and require large quantities of data for model training. The designed models may have errors when applied in places with no training data [9]. Using vegetation biophysical and biochemical parameters as input variables, mechanistic models describe the radiative transmission process of electromagnetic waves in the canopy. The reverse process of the mechanistic models can be used to predict the desired input parameters [10]. According to the relationship between chlorophyll and nitrogen, Yang et al. used the equivalent N absorption coefficient to replace the chlorophyll-specific absorption coefficient in the PROSPECT model and designed a nitrogen-based PROSPECT (N-PROSPECT) model, which can be used to combine the canopy scale model named SAIL (scattering by arbitrarily inclined leaves) to invert LNC values using canopy spectra data [11]. According to previous studies, the model that combined the N-PROSPECT model and the SAIL model can be called N-PROSAIL model for short. Using the N-PROSAIL model, Li et al. successively predicted wheat LNC by using the look-up table method [12]. Mechanistic methods have clear mechanistic explanations. However, they require many input parameters that are not easy to obtain in practice [13]. When the input parameters are uncertain, problems may be encountered during the model inversion process [14]. Additionally, due to the complexity of the electromagnetic transmission process, existing mechanism models cannot fully describe the transmission process, which causes the simulated data generated by the mechanism models to not be exactly consistent with the actual measured data, resulting in prediction errors. Hybrid methods first generate a large quantity of simulated data based on the mechanistic models, then use the simulated data to help train the empirical model, and finally apply the trained model to predict the biophysical and biochemical parameters in the target area [15]. The purpose of hybrid methods is to try to combine the advantages of empirical and mechanistic methods. Until recently, few studies have been conducted using hybrid wheat LNC prediction methods and UAV images. However, related work has been performed on the inversion of chlorophyll, which can provide a reference for LNC inversion based on the hybrid methods. Xu et al. simulated a rice dataset based on the PROSAIL model, and then used the dataset to train a Bayesian network model, and finally applied the trained model to predict rice canopy chlorophyll content using multispectral UAV images obtained in the target area [16]. Zhang et al. used the dataset generated by PROSAIL to pretrain a deep neural network (DNN), and then fine-tuned the parameters of the pretrained model using field-measured data, and finally used the model to predict wheat leaf chlorophyll content [17]. The above studies represent existing strategies used in hybrid methods. However, as stated before, the simulated spectra may differ from the actual measured spectra; thus, these approaches are not always efficient. Consistent with existing research [18,19], our initial study showed that both models trained using only simulated data and models pretrained using simulated data and fine-tuned with measured data do not achieve good inversion results in LNC prediction, and these models face overfitting problems.
Deep multitask learning is an inductive transfer learning method. The model trains the network based on multiple tasks. By sharing information between interrelated tasks, the robustness of the model is effectively improved by taking advantage of the similarities and differences between different tasks [20,21,22]. Due to the complexity of the vegetation canopy structure, the simulated spectra data may differ from the actual measured spectra data, but they have the common features that follow the absorption properties of biochemical parameters. In other words, when the measured data are limited, the simulated data can help to train the model for LNC prediction, but the simulated data cannot be used as the measured data directly during model training. Thus, based on the idea of multitask deep learning, we can set two tasks: one is to train the model based on the simulated data and use it to predict simulated LNC, and another is to train the model based on actual measured data and use it to predict measured LNC. The two tasks can share part of the network structure to learn their common features, while using their separate network structures to learn their unique features. The model learns information from the simulated and measured data simultaneously and transfers the information learned from the simulated data to improve the training process of the measured data, ensuring that the model focuses on important features. In addition, the noise in the simulated data with different patterns helps to improve the model robustness, reducing the overfitting problem [23]. Therefore, we can combine deep multitask learning methods and mechanistic models to develop a hybrid method for LNC prediction; however, no such approach has been established.
In this study, a field experiment was conducted to obtain LNC and hyperspectral image data of winter wheat under different growth conditions. To provide technical support for high-precision LNC prediction, based on these data, the following objectives were set: (i) to propose a new robust hybrid method for LNC prediction by combining a deep multitask learning method and a mechanistic model and (ii) to compare the proposed method with traditional methods (SI, PLSR, artificial neural network (ANN)) to recommend the best model for LNC prediction.

2. Materials and Methods

2.1. Field Experiment

This study was conducted at the Yucheng Integrated Experiment Station (116°34′13″E, 36°50′00″N) of the Chinese Academy of Sciences during 2020–2021. The winter wheat cultivar “Jimai 22” was sown with 20 cm row spacing. The experiment included two irrigation treatments and five N treatments in 32 plots. Each plot was 10 m × 5 m in size. The two irrigation levels were 60% and 80% of field water capacity and arranged with a split-plot design. The five N levels were 0 kg N/ha (N1), 70 kg N/ha (N2), 140 kg N/ha (N3), 210 kg N/ha (N4) and 280 kg N/ha (N5) and were arranged in a randomized block design, with N1-N4 having three replicates and N5 having four replicates. Except for the N fertilizer, the other management measures were the same in each plot, and the experimental design is shown in Figure 1.

2.2. Field Data Acquisition

Field campaigns were conducted to obtain UAV hyperspectral images and ground-measured LNC data at critical wheat growth stages. These wheat growth stages are Feekes 4–5 (14 April 2021), Feekes 10.2 (7 May 2021) and Feekes 11.1 (17 May 2021).

2.2.1. UAV Image Data

The hyperspectral images were acquired by an M600 pro six-rotor UAV (SZ DJI Technology Co., Guangzhou, China) equipped with an S185 imager (Cubert, Ulm, Germany). S185 is a new type of snapshot hyperspectral sensor characterized by short exposure and integration times. The sensor can capture wavelengths from the visible to near-infrared spectra (450–950 nm) at a 4 nm spectral resolution. In one shot, it can obtain a panchromatic image with a pixel resolution of 1000 × 1000 and a hyperspectral image with a pixel resolution of 50 × 50. Based on the software provided by the sensor manufacturer, these two image types can be fused to make hyperspectral images having same resolution of panchromatic image. To minimize the influence of changes in the incident angle of the sun on the images, the flight time was selected between 10:00 and 12:00. The flight height was set to 30 m, and the forward and side overlaps were both set to 80%. A white panel image was obtained prior to UAV take-off and used to convert the image data from digital number (DN) values to reflection values in the subsequent processing step. In addition, to geometrically correct the obtained UAV images, high-accuracy ground control points (GCPs) were measured with a global navigation satellite system (GNSS) receiver GEO7X handheld global positioning system (GPS) device (Trimble, CA, USA) in network real-time kinematic (NRTK) mode. The NRTK service provided by Qianxun Company (Shanghai, China) and the GCPs measured under these conditions had errors below 1 cm. In the UAV image preprocessing step, first, Curbert Utils Touch (Cubert, Ulm, Germany) software was used to fuse hyperspectral and panchromatic images and export the fused images in .tiff format with approximately 0.84 cm of spatial resolution. Second, the Agisoft Photoscan (Agisoft LLC, St. Petersburg, Russia) software program was used to mosaic the fused images and convert the image DN values to reflectance values. Third, the sampled high-accuracy GCPs were used to geometrically correct the mosaicked images taken at the Feekes 4–5 growth stage, and the mosaicked images from other growth stages were then geo-rectified based on the above geo-corrected image. Finally, the images were resized and masked to retain only the experimental zones.

2.2.2. Field Sampled Data

Field sampling was carried out immediately after obtaining the UAV images. For each plot, representative areas with relatively uniform wheat growth status were selected as sampling sites. First, LAI-2200 (LI-COR Inc., Lincoln, NE, USA) was used to measure the leaf area index (LAI). Then, two rows of wheat plants with lengths of 30 cm were sampled at each selected site and taken to the laboratory, where the plants were divided into leaves and stems (including spikes) and dried in an oven until a constant weight was obtained. The N concentration values in each plant part were measured using the Dumas combustion method with a vario-MACRO cube analyzer (Elementar, Hanau, Germany). Finally, according to the LAI, leaf dry weight and leaf nitrogen concentration, LNC (μg/cm2) values were calculated for each sampling site using Formula (1). Notably, the distances between the sampling points and the boundaries were recorded to allow for each sampling point to be accurately located in the geo-corrected images.
LNC = W × N % LAI
where LNC represents the leaf nitrogen content (μg/cm2) based on leaf area, W represents the leaf dry weight per unit ground area (g/m2), LAI represents the leaf area index value and N% represents the leaf nitrogen concentration (%).

2.3. A New LNC Prediction Method

According to the logic framework stated in the Introduction Section, this study proposes a new LNC prediction method that combines the deep multitask learning method and the N-PROSAIL mechanism model. It is called the multitask learning-based hybrid model (ML-HM) in this study. As simulated spectrum and actual measured spectrum have both similarities and differences, the model adopts hard parameter sharing to construct a deep multitask learning network [24]. It includes two subtasks: training the network model based on the simulated data of the N-PROSAIL model to predict simulated LNC and training the network model based on the measured data to predict measured LNC. By sharing part of the network parameters among different tasks, information can be shared among multiple tasks. Therefore, the task of inverting the simulated LNC based on the model trained by the simulated data is used as an auxiliary task to improve the inversion accuracy of inverting the actual LNC based on the measured data. The training flowchart of the model is shown in Figure 2. The model proposed in this study has three key modules: a shared layer network, a subtask layer network and multitask optimization. The fundamental goal and implementation details of each module are described below.

2.3.1. Shared Layer

This part of the network is shared by the major and auxiliary tasks. This component extracts the common features among the simulated and measured data. The input variables of the major and auxiliary tasks are the measured spectra data and simulated spectra data. The output variables are the common features used in the subsequent analyses for each task. Using this network structure reduces the risk of overfitting the network when training the major tasks based on limited measured data. The shared layer consists of six layers, with 20, 20, 20, 10, 10 and 10 neurons in sequence. A rectified linear unit (ReLU) is used as the activation function in each layer.

2.3.2. Subtask Layer

The subtask layers of the major and auxiliary tasks learn the unique characteristics of the simulated and measured data, respectively. They use the output variables of the shared layer as input variables, and the LNCs from the simulated and measured data are the output variables. Both subtask layers have three layers, with 10 neurons in each layer. The first layer uses ReLU as the activation function, and the latter two layers use traditional hyperbolic tangent (tanh) as the activation function. The subtask layers allow for the network to focus not only on the common features of the two tasks but also on the unique features of each task.

2.3.3. Multitask Optimization

This module integrates the LNC prediction error of the two tasks to determine the model parameters [25]. As each task impacts the network and different tasks have various convergence rates during network training, if each task has the same weight, the model is dominated by certain tasks. Thus, according to Liu et al. [26], we calculate the dynamic weighting error (DWA) according to the two tasks. The cost function is calculated using Formulas (2)–(5). To measure the prediction error, the mean square error (MSE) is calculated according to the predicted LNC and actual LNC for each task. For the entire process of this module, the deep multitask learning model runs according to the input data, and the corresponding cost function value is calculated; then, with the goal of minimizing the cost function, the backpropagation algorithm is run to optimize the model parameters. This process loops until the terminal condition is reached.
M S E a l l t = α k t × M S E k t
α k t = W t ∑ i = 1 N W i t
W k t = N   exp r k t − 1 / T ∑ i = 1 N exp r i t − 1 / T
r k t − 1 = M S E k t − 1 M S E k t − 2
In the above formulas, MSEall(t) represents the total LNC prediction error at training time t, MSEk(t) represents the mean square error of task k at training time t, αk(t) represents the normalized weight of task k at training time t, Wk(t) represents the unnormalized weight of task k at training time t, N represents the total number of tasks and is set to 2 in this study, rk(t − 1) represents the training speed of task k at training time t − 1, T is a constant and is set to 0.5 in this study and MSEk(t − 1) and MSEk (t − 2) represent the mean square error of task k at training times t − 1 and t − 2, respectively.

2.4. Data Analysis Method

To verify the performance of the ML-HM method, we compared the proposed approach with traditional methods (SI, PLSR and ANN) for LNC prediction. During this process, using collected UAV images, the averaged spectrum of all the pixels in the circle with the measured position of sampling points as center and two-times row spacing (40 cm) as the radius was used to correspond LNC values in the sampling point. It should be noted that as the image of S185 imager is the fused data from a high-resolution pan image and coarse resolution hyperspectral image, the pixels of the image are still the mixed pixels corresponding to the resolution of original coarse image and removing soil pixels cannot obtain a high crop N prediction accuracy [27]. Thus, we did not remove soil pixels. A total of 96 pairs of sampling data points were collected in this study. Three of these data points were found to have errors and were discarded. Thus, a total of 93 sampling points were retained. Based on the method of systematic random sampling, 75% of the samples were selected as the calibration dataset, and the remaining 25% were used as the validation dataset. The calibration dataset was used for model training, and the validation dataset was used for model validation. The R2, RMSE and RMSE% (RMSE/mean × 100%) values were used to evaluate the performance of each model. It should be noted that a large amount of data was also simulated using the N-PROSAIL model, and these data were used as the simulated data in the training of the ML-HM method. The main flow chart is shown in Figure 3 and the detailed description of each method is as follows.

2.4.1. SI Method

The spectral index uses the band combination method to eliminate background noise and improves the sensitivity to target parameters. It is the most commonly used method for LNC prediction. In this study, based on previous studies, the commonly used spectral indices for estimating LNC were selected and are shown in Table 1. For each index, four types of models (linear, exponential, power and logarithmic models) were used to design LNC prediction models based on calibration dataset, and the best model with the highest R2 and lowest RMSE and RMSE% values was selected and validated using validation dataset.

2.4.2. PLSR Method

PLSR is a commonly used method for addressing variable collinearity [45]. When establishing the PLSR-based LNC prediction models, this study selected the first five principal components (PCs) as model input data, as using more PCs did not improve model performance. Notably, as the spectral bands above 850 nm in the images obtained by the hyperspectral sensor S185 have poor quality according to Chen et al. [46], only the spectral bands between 450 nm and 850 nm were used as input variables. In this study, the PLSR models were calibrated using a calibration dataset and validated using a validation dataset. MATLAB 2020a code was written and used to perform the above process.

2.4.3. ANN Method

The ANN is a mathematical model inspired by the biological brain that simulates the complex information processing of the brain–nervous system [47] and has been widely used in many fields [48]. Compared with other ANN models, backpropagation (BP) ANN is one of the most widely used neural network models due to its good performance and ability to handle any linear or nonlinear relationship between the input and output variables [49]. In addition, Homik [50] et al. documented that a three-layer BP network could perform all N-dimensional to M-dimensional transformations. Therefore, a three-layer BP neural network with an input layer, a hidden layer and an output layer was used in this study. The hidden layer had three neurons, the activation function from the input layer to the hidden layer was the log-sigmoid function, and the activation function from the hidden layer to the output layer was the tan-sigmoid function. The maximum number of model iterations was set to 10,000, and the learning rate was set to 0.01. To reduce the number of input variables, principal component analysis (PCA) was performed on the band reflectance values, and the five first PCs were selected as the model input data, similar to the PLSR methods. The ANN model was trained based on all calibration samples and validated based on all validation samples. The above process was also performed using MATLAB 2020a code.

2.4.4. ML-HM Method

As stated previously, when the measured data are limited due to time and cost constraints, the training of deep learning models can be assisted by simulated data. The ML-HM method needs simulated data to help train the model. In this study, the N-PROSPECT model proposed by Li et al. was used to generate simulated data [51]. Based on the distribution characteristics of the field-measured data and related references, the ranges and sampling methods for the input parameters were determined and are shown in Table 2. These parameters were used to generate representative datasets under different conditions. Finally, a total of 20,000 data were generated.
Similar to the ANN method, when designing the LNC prediction model based on the ML-HM method, the bands between 450 and 850 nm were selected, PCA was used to reduce the number of input variables and the five first principal components were taken as input variables. As mentioned in Section 2.3, the two tasks of the ML-HM model included an auxiliary task, namely, training a network based on the simulated data, and a major task, namely, training a network based on the measured data. The two tasks shared part of the network. During the model calibration process, all 20,000 simulated data and all 70 calibration samples of the measured data were used to train the network. For major task model validation, all 23 validation samples of the measured data were used to validate the model performance. Considering the training process, the adaptive moment estimation (Adam) algorithm was used as the optimizer, as it requires less memory and converges more rapidly than other optimizers. In addition, the learning rate was set to 0.001, the weight decay coefficient was set to 0.0016, the learning rate decay strategy was set to reduce the learning rate to 0.2 times the current value every 20 cycles and the number of training cycles was set to 100. Due to the fact that the quantity of training data for the two tasks significantly differed, to avoid the shared network being mainly influenced by simulated data, the method used in Narayannan’s study was used [52] and described as follows. Each training cycle contained 1000 iterations to continuously train the network. In each iteration, 20 samples from the simulated dataset and 8 samples from the measured data were randomly selected to train the model. The above process was called as mini-batch gradient descent training process [53]. All the above was performed using Python 3.8 code. To note that, as our objective is to obtain a model for predicting actual LNC, we did not using simulated data to validate the performance of auxiliary task for simulated LNC prediction.
Table
Table 2. Ranges and sampling methods of the input parameters for the N-PROSAIL model.
Table 2. Ranges and sampling methods of the input parameters for the N-PROSAIL model.
VariableMinMaxAVGSDSampling MethodReference
LNC (Leaf nitrogen content, μg/cm2)2022011045GaussMeasured dataset
Cbrown (Brown pigment content, μg/cm2)00--Fixed[54]
Cw (Equivalent water thickness, cm)0.0040.04--Uniform[12]
Cdm (Dry matter content, g/m2)0.0010.02--Uniform[55]
Nstructer (Leaf structure)1.21.81.50.3Gauss[55]
LID (Leaf inclination distribution, deg)30806030Gauss[56]
LAI (Leaf area index, m2/m2)0.193.91.6GaussMeasured dataset
SL (Hot spot parameter)0.10.50.20.5Gauss[57]
θs (Solar zenith angle, deg)2045--UniformMeasured dataset
Rsoil (Soil brightness parameter)0.20.90.40.4Gauss[58]

3. Results

3.1. LNCs in the Field

Analysis of variance (ANOVA) and Duncan’s new multiple range test (MRT) analyses were performed on the LNC values of wheat under different irrigation and N treatment levels during various growth stages and are shown in Table 3. There were significant differences (p < 0.05) in LNC among the different N levels in all wheat growth stages. The LNC values in the experimental year varied between 20.45 and 216.38 μg/cm2, covering a wide LNC range. Thus, our dataset is a good dataset for LNC model design.

3.2. Simulated and Measured Spectral Reflectance

All the measured and some simulated spectra data are shown in Figure 4a. It can be seen that the change range of the measured spectrum is within the change range of the simulated spectrum and that they have a similar spectral structure, with absorption valleys at the red and blue bands and peaks at the near infrared band. However, when considering the correlation coefficients between each band and LNC, they not only have similarities between bands 450 nm and 734 nm, both having significant (p < 0.05) correlation with LNC, but also have differences between bands 738 nm and 766 nm, with simulated spectra having a significant correlation with LNC and measured data having no significant correlation with LNC. In summary, the data demonstrated our theory according to which the simulated data has similarities with the measured data, but it also has differences with the measured data and cannot be used to train the model directly for actual LNC prediction.

3.3. LNC Prediction Results by the SI Method

The LNC prediction results for the models designed using different spectral indices are shown in Table 4 and ranked from best to worst performance according to the calibration and validation results. Among them, MCART/MTVI2 and TCARI/OSAVI achieved the best results, with R2 values of 0.68 and 0.65, RMSE values of 29.86 and 31.26 μg/cm2, and RMSE% values of 26.66% and 27.91%, respectively, during calibration and R2 values of 0.68 and 0.68, RMSE values of 25.25 and 25.69 μg/cm2 and RMSE% values of 23.23% and 23.63%, respectively, during validation. Compared with the other vegetation indices, MSAVI and TVI had the worst accuracy, with R2 values of 0.33 and 0.29, RMSE values of 37.41 and 38.52 μg/cm2 and RMSE% values of 33.40% and 34.39%, respectively, during calibration and R2 values of 0.48 and 0.43, RMSE values of 32.20 and 33.55 μg/cm2 and RMSE% values of 29.62% and 30.86%, respectively, during validation. The remaining spectral indices achieved moderate LNC prediction results. During the calibration stage, the R2 values varied between 0.35 and 0.57, the RMSE values varied between 29.98 and 36.85 μg/cm2 and the RMSE% values varied between 26.77% and 32.90%. During the validation stage, the R2 values varied between 0.49 and 0.70, the RMSE values varied between 23.86 and 31.60 μg/cm2 and the RMSE% values varied between 21.95% and 29.07%.

3.4. LNC Prediction Results by the PLSR Method

The LNC prediction results of the PLSR method are shown in Figure 5. During calibration, the PLSR method achieved an R2 value of 0.68, an RMSE value of 25.71 μg/cm2 and an RMSE% value of 22.95%. Moreover, during the validation stage, the model had an R2 value of 0.74, an RMSE value of 22.79 μg/cm2 and an RMSE% value of 20.96%. In addition, although most of the points are clustered near the 1:1 line, some points deviate from the 1:1 line. The PLSR method may underestimate samples with higher LNC values and overestimate samples with lower LNC values.

3.5. LNC Prediction Results by the ANN Method

The LNC prediction results of the ANN method are shown in Figure 6. The ANN model shows a moderate LNC prediction performance. During calibration, the ANN method obtained an R2 value of 0.56, an RMSE value of 33.62 μg/cm2 and an RMSE% value of 30.01%. During validation, the model had an R2 value of 0.62, an RMSE value of 30.42 μg/cm2 and an RMSE% value of 27.98%. In addition, many points deviate from the 1:1 line. The ANN method underestimates samples with higher LNC values and overestimates samples with lower LNC values.

3.6. LNC Prediction Results by the ML-HM Method

For the ML-HM method, based on measured data, it obtained an R2 value of 0.79, an RMSE value of 20.86 μg/cm2 and an RMSE% value of 18.63% during calibration as well as an R2 value of 0.82, an RMSE value of 18.40 μg/cm2 and an RMSE% value of 16.92% during validation (Figure 7). In addition, all points were near the 1:1 line. Thus, the ML-HM method showed very good LNC prediction performance.

4. Discussion

4.1. Comparison with Previous Studies

In this study, the newly proposed ML-HM method achieved the best LNC prediction results, with an R2 value of 0.82 during validation. The corresponding R2 values of the SI, PLSR and ANN methods were between 0.43 and 0.74. Using field-measured hyperspectral data, Jia et al. designed various spectral indices for wheat LNC prediction and obtained the maximum R2 value of 0.66 [59]. Moreover, based on UAV hyperspectral images, Zhang et al. used different methods (PLSR, GA-PLSR, RF and XGBoost) for LNC prediction, and the maximum R2 value of the four methods is 0.55 [7]. Compared with previous studies, the results of the traditional methods (SI, PLSR and ANN methods) in our study are within reasonable ranges, and the novel ML-HM method performed better than the traditional methods.

4.2. Best Structure for the Hybrid Method

Previous studies on hybrid methods that combined deep learning methods and mechanistic models used single-task schemes to design the model. To verify the effectiveness of the multitask learning scheme proposed in this study, several single-task schemes were designed according to previous studies and compared with the multitask scheme proposed in this study. These single-task schemes include the following: (a) the simulated data generated in Section 2.4.4 and the calibration dataset of the measured data were used to train the same network for LNC prediction, and the validation dataset of the measured data was used to validate the model. For comparison, the network used the same structure as the major task network in the ML-HM method and is denoted as the single-task method I. (b) Only the simulated data were used to train the network for LNC prediction, and the validation dataset of the measured data was used to validate the model. The network used the same structure as the major task network in the ML-HM method and is denoted as the single-task method II. (c) Only the calibration dataset of the measured data was used to train the network, and the validation dataset of the measured data was used to validate the model. The network used the same structure as the major task network in the ML-HM method and is denoted as the single-task method III.
The results of the different LNC prediction schemes are shown in Table 5. As the ML-HM method, the single-task method I used the simulated data and part of the measured data to train the network. However, this scheme considered the simulated data to be the same as the measured data. In fact, due to the complexity of the electromagnetic transmission process, existing mechanistic models cannot fully describe the transmission process; thus, the simulated data differ from the measured data. Thus, although single-task method I performs better during model calibration, its performance decreases significantly during model validation. Single-task method II used only the simulated data to train the model. Similar to the prior discussion, this scheme also has an overfitting problem, with good performance during calibration and poor performance during validation. Single-task method III used part of the measured data to train the network and the remaining data to validate the network. Compared with the above two single-task schemes, this scheme has a stable performance during model calibration and validation. However, its LNC prediction results are worse than those of the multitask method proposed in this study. This result may have occurred due to the following reasons: i) the measured data have a relatively small number of calibration samples and thus cannot represent data in different situations, and ii) although the simulated data differ from the measured data, they represent data in different situations. In our ML-HM model, we effectively extracted the common features shared by the simulated and measured data for LNC prediction through the network in the shared layer and used these shared features to assist in training the network on the measured data, which effectively improved the LNC prediction accuracy.

4.3. Optimal LNC Prediction Method

Many commonly used methods were selected and compared with the ML-HM method proposed in this study. The ML-HM method performed the best among the comparison methods. This good performance may be due to the following reasons: (i) multiple factors (e.g., LAI, chlorophyll, leaf mesophyll structure, leaf water content, leaf angle distribution function, background) influence canopy reflectance, and the combined influence of these factors on the reflectance spectrum is not purely linear. The PLSR method is a linear regression method that cannot effectively determine the nonlinear relationship between the spectral reflectance and LNC. (ii) The SI model is based on multispectral information and utilizes only limited spectral features. (iii) Due to the limited quantity of measured data, the ANN model training was not sufficient. Thus, it could not express the relationship between spectral reflectance and LNC. In the ML-HM method, we used useful information in the simulated data to help train the network on the measured data, resulting in an improved LNC prediction accuracy.

4.4. Application Potential and Limitations of This Study

In this study, an ML-HM LNC prediction method was designed. In this method, the simulated data from the mechanistic model were used as an auxiliary task to help train the LNC prediction model, thereby improving the robustness of the model and reducing the number of measured samples required by the model. For field applications, UAV images can be obtained; then, the ML-HM method can be used to predict LNC in the field. Next, an N application map can be produced by considering the LNC in the field and the law of wheat N demand during its life cycle. Finally, precise N management can be conducted according to the N application map.
Although the method proposed in this paper achieved good results, some limitations remain. In this study, during mini-batch gradient descent training process, the quantity of simulated and measured data was determined by experience with no theoretical support. An excessive amount of simulated data can significantly increase the calculation time. However, a small amount of simulated data are insufficient for helping the network learn enough information to assist the training of the main task network. Therefore, determining a reasonable ratio of simulated to measured data can improve model training efficiency and model accuracy, which will be investigated in future work.

5. Conclusions

In this study, wheat field experiments under different irrigation and N treatments were conducted, and LNC and UAV hyperspectral images were collected during critical growth stages. Based on these data, a new hybrid method combining the deep multitask learning method and the N-PROSAIL model was proposed and compared with traditional LNC prediction methods (SI, PLSR and ANN methods). Additionally, we compared the proposed approach with traditional hybrid methods using single-task schemes. The results show that the proposed ML-HM method has better LNC prediction performance than the SI, PLSR and ANN methods. Moreover, compared with the hybrid method using a single-task scheme, the ML-HM method more effectively extracts useful information from the simulated data to help train the network on the measured data and discard confusing information, increasing the robustness of the model. Thus, this study provides an accurate LNC prediction tool for precise N management in the field.

Author Contributions

Data analysis, writing original draft, X.M.; Conceptualization, review and editing, supervision, P.C.; Review and editing, X.J. All authors have read and agreed to the published version of the manuscript.

Funding

This research was supported by The National Natural Science Foundation of China (Grant Numbers 41871344), the Strategic Priority Research Program of the Chinese Academy of Sciences (Grant Number XDA28040502).

Data Availability Statement

The data presented in this study are available upon request from the corresponding author. The data are not publicly available due to their use in subsequent studies.

Acknowledgments

We thank the Yucheng Integrated Experiment Station of the Chinese Academy of Sciences and Jing Li for providing the experimental field used for this study.

Conflicts of Interest

The authors declare no conflict of interest.

References

  1. Albornoz, F. Crop responses to nitrogen overfertilization: A review. Sci. Hortic. 2016, 205, 79–83. [Google Scholar] [CrossRef]
  2. Gruber, N.; Galloway, J.N. An earth-system perspective of the global nitrogen cycle. Nature 2008, 451, 293–296. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  3. Panda, S.S.; Hoogenboom, G.; Paz, J.O. Remote sensing and geospatial technological applications for sits-specific management of fruit and nut crop: A review. Remote Sens. 2010, 2, 1973–1997. [Google Scholar] [CrossRef] [Green Version]
  4. Mondal, P.; Basu, M. Adoption of precision agriculture technologies in India and in some developing countries: Scope, present status and strategies. Prog. Nat. Sci. 2009, 19, 659–666. [Google Scholar] [CrossRef]
  5. Zhang, L.; Niu, Y.; Zhang, H.; Han, W.; Li, G.; Tang, J.; Peng, X. Maize canopy temperature extracted from UAV thermaland RGB imagery and its application in water stress monitoring. Front. Plant Sci. 2019, 10, 1270. [Google Scholar] [CrossRef]
  6. Shafian, S.; Rajan, N.; Schnell, R.; Bagavathiannan, M.; Valasek, J.; Shi, Y.; Olsenholler, J. Unmanned aerial systems-based remote sensing for monitoring sorghum growth and development. PLoS ONE 2018, 13, e0196605. [Google Scholar] [CrossRef] [Green Version]
  7. Zhang, Y.; Wang, T.; Guo, Y.; Skidmore, A.; Zhang, Z.; Tang, R.; Song, S.; Tang, Z. Estimating community-level plant functional traits in a species-rich alpine meadow using UAV image spectroscopy. Remote Sens. 2022, 14, 3399. [Google Scholar] [CrossRef]
  8. Jay, S.; Maupas, F.; Bendoula, R.; Gorretta, N. Retrieving LAI, chlorophyll and nitrogen contents in sugar beet crops from multi-angular optical remote sensing: Comparison of vegetation indices and PROSAIL inversion for field phenotyping. Field Crop. Res. 2017, 210, 33–46. [Google Scholar] [CrossRef] [Green Version]
  9. Croft, H.; Arbian, J.; Chen, J.M.; Shang, J.; Liu, J. Mapping within-field leaf chlorophyll content in agricultural crops for nitrogen management using Landsat-8 imagery. Precis. Agric. 2020, 21, 856–880. [Google Scholar] [CrossRef] [Green Version]
  10. Li, Z.; Jin, X.; Wang, J.; Yang, G.; Nie, C.; Xu, X.; Feng, H. Estimating winter wheat (Triticum aestivum) LAI and leaf chlorophyll content from canopy reflectance data by integrating agronomic prior knowledge with the PROSAIL model. Int. J. Remote Sens. 2015, 36, 2634–2653. [Google Scholar] [CrossRef]
  11. Yang, G.; Zhao, C.; Pu, R.; Feng, H.; Li, Z.; Li, H.L.; Sun, C. Leaf nitrogen spectral reflectance model of winter wheat (Triticum aestivum) based on PROSPECT: Simulation and inversion. J. Appl. Remote Sens. 2015, 9, 095976. [Google Scholar] [CrossRef]
  12. Li, Z.; Jin, X.; Yang, G.; Drummond, J.; Yang, H.; Clark, B.; Li, Z.; Zhao, C. Remote sensing of leaf and canopy nitrogen status in winter wheat (Triticum aestivum L.) based on N-PROSAIL model. Remote Sens. 2018, 10, 1463. [Google Scholar] [CrossRef] [Green Version]
  13. Darvishzadeh, R.; Atzberger, C.; Skidmore, A.; Schlerf, M. Mapping grassland leaf area index with airborne hyperspectral imagery: A comparison study of statistical approaches and inversion of radiative transfer models. ISPRS-J. Photogramm. Remote Sens. 2011, 66, 894–906. [Google Scholar] [CrossRef]
  14. Durbha, S.S.; King, R.L.; Younan, N.H. Support vector machines regression for retrieval of leaf area index from multiangle imaging spectroradiometer. Remote Sens. Environ. 2007, 107, 348–361. [Google Scholar] [CrossRef]
  15. Annala, L.; Honkavarra, E.; Tuominen, S.; Pölönen, I. Chlorophyll concentration retrieval by training convolutional neural network for stochastic model of leaf optical properties (SLOP) inversion. Remote Sens. 2020, 12, 283. [Google Scholar] [CrossRef] [Green Version]
  16. Xu, X.; Lu, J.; Zhang, N.; Yang, T.; He, J.; Yao, J.; Cheng, T.; Zhu, Y.; Cao, W.; Tian, Y. Inversion of rice canopy chlorophyll content and leaf area index based on coupling of radiative transfer and Bayesian network models. ISPRS-J. Photogramm. Remote Sens. 2019, 150, 185–196. [Google Scholar] [CrossRef]
  17. Zhang, Y.; Hui, J.; Qin, Q.; Sun, Y.; Zhang, T.; Hong, S.; Li, M. Transfer-learning-based approach for leaf chlorophyll content estimation of winter wheat from hyperspectral data. Remote Sens. Environ. 2021, 267, 112724. [Google Scholar] [CrossRef]
  18. Jiao, Q.; Sun, Q.; Zhang, B.; Huang, W.; Ye, H.; Zhang, Z.; Zhang, X.; Qiang, B. A random forest algorithm for retrieving canopy chlorophyll content of wheat and soybean trained with PROSAIL simulation using adjusted average leaf angle. Remote Sens. 2022, 14, 98. [Google Scholar] [CrossRef]
  19. Svendsen, D.H.; Martino, L.; Campos-Taberner, M.; Garcia-Haro, F.J.; Camps-Valls, G. Joint gaussian processes for biophysical parameter retrieval. IEEE Trans. Geosci. Remote Sens. 2018, 56, 1718–1727. [Google Scholar] [CrossRef] [Green Version]
  20. Baxter, J. A model of inductive bias learning. J. Artif. Intell. Res. 2000, 12, 149–198. [Google Scholar] [CrossRef] [Green Version]
  21. Thrun, S. Is Learning the n-th Thing Any Easier than Learning the First? In Proceedings of the Advances in Neural Information Processing Systems 8, NIPS, Denver, CO, USA, 27–30 November 1995; Touretzky, D.S., Mozer, M., Hasselmo, M.E., Eds.; MIT Press: Cambridge, MA, USA, 1995; pp. 640–646. [Google Scholar]
  22. Caruana, R. Multi task learning. Mach. Learn. 1997, 28, 41–75. [Google Scholar] [CrossRef]
  23. Zhang, Y.; Yang, Q. An overview of multi-task learning. Natl. Sci. Rev. 2018, 5, 30–43. [Google Scholar] [CrossRef] [Green Version]
  24. Baxter, J. A bayesian/information theoretic model of learning to learn via multiple task sampling. Mach. Learn. 1997, 28, 7–39. [Google Scholar] [CrossRef]
  25. Zhang, Q.; Wu, S.; Wang, X.; Sun, B.; Liu, H.M. A PM2.5 concentration prediction model based on multi-task deep learning for intensive air quality monitoring stations. J. Clean. Prod. 2020, 275, 122722. [Google Scholar] [CrossRef]
  26. Liu, S.; Johns, E.; Davison, A.J. End-to-End multi-task learning with attention. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Long Beach, CA, USA, 15–20 June 2019; pp. 1871–1880. [Google Scholar]
  27. Chen, P.; Wang, F. Effect of crop spectra purification on plant nitrogen concentration estimations performed using high-spatial-resolution images obtained with unmanned aerial vehicles. Field Crop Res. 2022, 288, 108708. [Google Scholar] [CrossRef]
  28. Rouse, J.; Haas, R.H.; Deering, D.; Schell, J.A.; Harlan, J. Monitoring the vernal advancement and retrogradation (Green wave effect) of natural vegetation. Environ. Sci. 1974, 1974, 1–37. [Google Scholar]
  29. Gitelson, A.A.; Kaufman, Y.J.; Merzlyak, M.N. Use of a green channel in remote sensing of global vegetation from EOS-MODIS. Remote Sens. Environ. 1996, 58, 289–298. [Google Scholar] [CrossRef]
  30. Qi, J.; Chehbouni, A.; Huete, A.R.; Kerr, Y.H.; Sorooshian, S. A modified soil adjusted vegetation index. Remote Sens. Environ. 1994, 48, 119–126. [Google Scholar] [CrossRef]
  31. Rondeaux, G.; Steven, M.; Baret, F. Optimization of soil-adjusted vegetation indices. Remote Sens. Environ. 1996, 55, 95–107. [Google Scholar] [CrossRef]
  32. Huete, A.; Justice, C.; Liu, H. Development of vegetation and soil indices for MODIS-EOS. Remote Sens. Environ. 1994, 49, 224–234. [Google Scholar] [CrossRef]
  33. Broge, N.H.; Leblanc, E. Comparing prediction power and stability of broadband and hyperspectral vegetation indices for estimation of green leaf area index and canopy chlorophyll density. Remote Sens. Environ. 2002, 76, 156–172. [Google Scholar] [CrossRef]
  34. Haboudans, D.; Miller, J.R.; Pattey, E.; Zarco-Tejada, P.J.; Strachan, I.B. Hyperspectral vegetation indices and novel algorithms for predicting green LAI of crop canopies: Modeling and validation in the context of precision agriculture. Remote Sens. Environ. 2004, 90, 337–352. [Google Scholar] [CrossRef]
  35. Xue, L.; Cao, W.; Luo, W.; Dai, T.; Zhu, Y. Monitoring leaf nitrogen status in rice with canopy spectral reflectance. Agron. J. 2004, 96, 135–142. [Google Scholar] [CrossRef]
  36. Fitzgerald, G.J.; Rodriguez, D.; Christensen, L.K.; Belford, R.; Sadras, V.O.; Clarke, T.R. Spectral and thermal sensing for nitrogen and water status in rainfed and irrigated wheat environments. Precis. Agric. 2006, 7, 233–248. [Google Scholar] [CrossRef]
  37. Reyniers, M.; Walvoort, D.J.J.; Baardemaaker, D.J. A linear model to predict with a multi-spectral radiometer the amount of nitrogen in winter wheat. Int. J. Remote Sens. 2006, 27, 4159–4179. [Google Scholar] [CrossRef]
  38. Chen, P.; Haboudance, D.; Tremblay, N.; Wang, J.; Vigneault, P.; Li, B. New spectral indicator assessing the efficiency of crop nitrogen treatment in corn and wheat. Remote Sens. Environ. 2010, 114, 1987–1997. [Google Scholar] [CrossRef]
  39. Eitel, J.U.H.; Long, D.S.; Gessler, P.E.; Smith, A.M.S. Using in-situ measurements to evaluate the new RapideEyeTM satellite series for prediction of wheat nitrogen status. Int. J. Remote Sens. 2007, 28, 4183–4190. [Google Scholar] [CrossRef]
  40. Dash, J.; Curran, P.J. The MERIS terrestrial chlorophyll index. Int. J. Remote Sens. 2004, 25, 5403–5413. [Google Scholar] [CrossRef]
  41. Haboudane, D.; Miller, J.R.; Tremblay, N.; Zaro-Tejada, P.J.; Dextraze, L. Integrated narrow-band vegetation indices for prediction of crop chlorophyll content for application to precision agriculture. Remote Sens. Environ. 2002, 81, 416–426. [Google Scholar] [CrossRef]
  42. Cho, M.A.; Skidmore, A.K. A new technique for extracting the red edge position from hyperspectral data: The linear extrapolation method. Remote Sens. Environ. 2006, 101, 181–193. [Google Scholar] [CrossRef]
  43. Gitrlson, A.A.; Viña, A.; Cogand, V.; Rundquist, D.C.; Arkebauer, T.J. Remote estimation of canopy chlorophyll content in crops. Geophys. Res. Lett. 2005, 32, L08403. [Google Scholar]
  44. Chen, P.; Tremblay, N.; Wang, J.; Vigneault, P.; Huang, W.; Li, B. New index of crop canopy fresh biomass estimation. Spectrosc. Spectr. Anal. 2010, 30, 512–517. (In Chinese) [Google Scholar]
  45. Yang, B.; Wang, M.; Sha, Z.; Wang, B.; Chen, J.; Yao, X.; Cheng, T.; Cao, W. Evaluation of aboveground nitrogen content of winter wheat using digital imagery of unmanned aerial vehicles. Sensors 2019, 19, 4416. [Google Scholar] [CrossRef] [PubMed]
  46. Chen, P.; Li, G.; Shi, Y.; Xu, Z.; Yang, F.; Cao, Q. Validation of an unmanned aerial vehicle hyperspectral sensor and its application in maize leaf area index estimation. Sci. Agric. Sin. 2018, 51, 1464–1474. (In Chinese) [Google Scholar]
  47. McCulloch, W.S.; Pitts, W. A logical calculus of the ideas immanent in nervous activity. Bull. Math. Biophys. 1943, 5, 115–133. [Google Scholar] [CrossRef]
  48. LeCun, Y.; Bengio, Y.; Hinton, G. Deep learning. Nature 2015, 512, 436–444. [Google Scholar] [CrossRef]
  49. Chen, P.; Wang, J.; Huang, W.; Tremblay, N.; Ou, Y.; Zhang, Q. Critical nitrogen curve and remote detection of nitrogen nutrition index for corn in the northwestern plain of shandong province, China. IEEE J. Sel. Top. Appl. Earth Obs. Remote Sens. 2013, 6, 682–689. [Google Scholar] [CrossRef]
  50. Hornik, K.; Stinchcombe, M.; White, H. Universal approximation of an unknown mapping and its derivatives using multilayer feedforward networks. Neural Netw. 1990, 3, 551–560. [Google Scholar] [CrossRef]
  51. Li, Z.; Li, Z.; Fairbairn, D.; Li, N.; Xu, B.; Feng, H.; Yang, G. Multi-LUTs method for canopy nitrogen density estimation in winter wheat by field and UAV hyperspectral. Comput. Electron. Agric. 2019, 162, 174–182. [Google Scholar] [CrossRef]
  52. Narayanan, S.; Mannam, K.; Achan, P.; Ramesh, M.V.; Rangan, V.; Rajan, S. A contextual multi-task neural approach to medication and adverse events identification from clinical test. J. Biomed. Inform. 2022, 125, 103960. [Google Scholar] [CrossRef]
  53. Senthil, T.; Chinnasamy, R.; Jeevanandham, D. An improved optimization technique using deep neural networks for digit recognition. Soft Comput. 2021, 25, 1647–1658. [Google Scholar] [CrossRef]
  54. Wang, L.; Chen, S.; Peng, Z.; Huang, J.; Wang, C.; Jiang, H.; Zheng, Q.; Li, D. Phenology effects on physically based estimation of paddy rice canopy traits from UAV hyperspectral imagery. Remote Sens. 2021, 13, 1792. [Google Scholar] [CrossRef]
  55. Upreti, D.; Huang, W.; Kong, W.; Pascucci, S.; Pignatti, S.; Zhou, X.; Ye, H.; Casa, R. A Comparison of hybrid machine learning algorithms for the retrieval of wheat biophysical variables from Sentinel-2. Remote Sens. 2019, 11, 481. [Google Scholar] [CrossRef] [Green Version]
  56. Li, H.; Liu, G.; Liu, Q.; Chen, Z.; Huang, C. Retrieval of winter wheat leaf area index from Chinese GF-1 satellite data using the PROSAIL model. Sensors 2018, 18, 1120. [Google Scholar] [CrossRef] [PubMed]
  57. Delloye, C.; Weiss, M.; Defourny, P. Retrieval of the canopy chlorophyll content from Sentinel-2 spectral bands to estimate nitrogen uptake in intensive winter wheat cropping systems. Remote Sens. Environ. 2018, 216, 245–261. [Google Scholar] [CrossRef]
  58. Liang, L.; Geng, D.; Yan, J.; Qiu, S.; Di, L.; Wang, S.; Xu, L.; Wang, L.; Kang, J.; Li, L. Estimating crop LAI using spectral feature extraction and the hybrid inversion method. Remote Sens. 2020, 12, 3534. [Google Scholar] [CrossRef]
  59. Jia, M.; Colombo, R.; Rossini, M.; Celesti, M.; Zhu, J.; Cogliati, S.; Cheng, T.; Tian, Y.; Zhu, Y.; Cao, W.; et al. Estimation of leaf nitrogen content and photosynthetic nitrogen use efficiency in wheat using sun-induced chlorophyll fluorescence at the leaf and canopy scales. Eur. J. Agron. 2021, 122, 126192. [Google Scholar] [CrossRef]
Remotesensing 14 06334 g001 550
Figure 1. A schematic diagram of the water and nitrogen coupling experiment (W1: 80% field capacity; W2: 60% field capacity; N1: no fertilizer; N2: 70 kg N/ha; N3: 140 kg N/ha; N4: 210 kg N/ha; N5: 280 kg N/ha).
Figure 1. A schematic diagram of the water and nitrogen coupling experiment (W1: 80% field capacity; W2: 60% field capacity; N1: no fertilizer; N2: 70 kg N/ha; N3: 140 kg N/ha; N4: 210 kg N/ha; N5: 280 kg N/ha).
Remotesensing 14 06334 g001
Remotesensing 14 06334 g002 550
Figure 2. Main structure of the proposed model.
Figure 2. Main structure of the proposed model.
Remotesensing 14 06334 g002
Remotesensing 14 06334 g003 550
Figure 3. Main flow chart of the LNC prediction models designed based on SI, PLSR, ANN and ML-HM methods.
Figure 3. Main flow chart of the LNC prediction models designed based on SI, PLSR, ANN and ML-HM methods.
Remotesensing 14 06334 g003
Remotesensing 14 06334 g004 550
Figure 4. Simulated and measured spectral reflectance curves (a) and the correlation coefficients between each band in simulated spectrum and LNC and between each band in measured spectrum and LNC (b).
Figure 4. Simulated and measured spectral reflectance curves (a) and the correlation coefficients between each band in simulated spectrum and LNC and between each band in measured spectrum and LNC (b).
Remotesensing 14 06334 g004
Remotesensing 14 06334 g005 550
Figure 5. PLSR model results. (a) Calibration; (b) Validation.
Figure 5. PLSR model results. (a) Calibration; (b) Validation.
Remotesensing 14 06334 g005
Remotesensing 14 06334 g006 550
Figure 6. ANN model results. (a) Calibration; (b) Validation.
Figure 6. ANN model results. (a) Calibration; (b) Validation.
Remotesensing 14 06334 g006
Remotesensing 14 06334 g007 550
Figure 7. ML-HM model results. (a) Calibration; (b) Validation.
Figure 7. ML-HM model results. (a) Calibration; (b) Validation.
Remotesensing 14 06334 g007
Table
Table 1. Spectral indices used in this study.
Table 1. Spectral indices used in this study.
IndexNameFormulaDeveloped by
NDVINormalized difference vegetation index(R800 − R670)/(R800 + R670)[28]
GNDVIGreen normalized difference vegetation index(R800 − R550)/(R800 + R550)[29]
MSAVIModified soil adjusted vegetation index(2R800 + 1 − sqrt((2R800 + 1) − 8(R800 − R670)))/2[30]
OSAVIOptimized adjusted vegetation index1.16(R800 − R670)/(R800 + R670 + 0.16)[31]
EVIEnhanced vegetation index2.5(R800 − R670)/(R800 + 6R670 − 7.5R490 + 1)[32]
TVITriangular vegetation index0.5(120(R750 − R550) − 200(R670 − R550))[33]
MTVI2Modified triangular vegetation index 21.5(1.2(R800 − R550) − 2.5(R670 − R550))/sqrt((2R800 + 1)2 − (6R800 − 5sqrt(R670)) − 0.5)[34]
RVIRatio vegetation indexR810/R560[35]
NDRENormalized difference red-edge index(R790 − R720)/(R790 + R720)[36]
VIoptOptimal vegetation index(1 + 0.45)((R800)2 + 1)/(R670 + 0.45)[37]
DNCIDouble peak canopy nitrogen index(R720 − R700)/(R700 − R670)/(R720 − R670 + 0.03)[38]
MCARI/MTVI2Combined index I †MCARI/MTVI2[39]
MCARI: (R700 − R670 − 0.2(R700 − R550))(R700/R670)
MTVI2: 1.5(1.2(R800 − R550) − 2.5(R670 − R550))/sqrt((2R800 + 1)2 − (6R800 − 5sqrt(R670)) − 0.5)
MTCIMERIS terrestrial chlorophyll index(R750 − R710)/(R710 − R680)[40]
TCARI/OSAVICombined index II †TCARI: 3((R700 − R670) − 0.2(R700 − R550)(R700/R670)) OSAVI: 1.16(R800 − R670)/(R800 + R670 + 0.16)[41]
REPRed-edge position700 + 40(Rre − R700)/(R740 − R700)
Rre: (R670 + R780)/2		[42]
R-MRed modelR750/R720 − 1[43]
RTVIRed-edge triangular vegetation index(100(R750 − R730) − 10(R750 − R550))sqrt(R700/R670)[44]
† indicates named in this study.
Table
Table 3. Mean LNC (μg/cm2) in wheat under different water and N treatments during different growth stages.
Table 3. Mean LNC (μg/cm2) in wheat under different water and N treatments during different growth stages.
Growth StageIrrigation TreatmentN Fertilizer Treatment *
N1N2N3N4N5
Feekes 4–5W128.65 a81.09 a,b116.40 b138.01 b137.66 b
W245.26 a98.47 b131.78 b,c122.22 b,c144.31 c
Feekes 10.2W178.86 a114.45 a,b164.65 b,c142.07 b,c173.68 c
W251.54 a70.66 a124.96 b140.28 b154.72 b
Feekes 11.1W159.05 a64.85 a,b115.29 a,b153.08 b119.18 a,b
W228.21 a63.49 b114.81 c141.14 c118.35 c
*: Numbers indicate the average LNC in the corresponding N and irrigation treatments. Within each row, the different letters indicate significant differences at the 0.05 level (p < 0.05); W1: 80% field capacity; W2: 60% field capacity; N1: 0 kg N/ha−1; N2: 70 kg N/ha−1; N3: 140 kg N/ha−1; N4: 210 kg N/ha−1; N5: 280 kg N/ha−1.
Table
Table 4. LNC prediction results for the models designed using different spectral indices. Linear, logarithmic, exponential and power models were used for fitting each index. The model type with the best results is expressed in the table.
Table 4. LNC prediction results for the models designed using different spectral indices. Linear, logarithmic, exponential and power models were used for fitting each index. The model type with the best results is expressed in the table.
SIModel TypeCalibrationValidation
R2RMSE (μg/cm2)RMSE%R2RMSE
(μg/cm2)		RMSE%
MCARI/MTVI2Exponential0.6829.8626.66%0.6825.2523.23%
TCARI/OSAVIExponential0.6531.2627.91%0.6825.6923.63%
MTCILogarithmic0.5630.2026.96%0.7023.8621.95%
REPLinear0.5729.9826.77%0.6825.0623.05%
RMLogarithmic0.5231.5328.15%0.6625.3723.34%
NDRELogarithmic0.5331.4428.07%0.6525.7523.68%
GNDVILogarithmic0.5131.9128.49%0.6526.0223.93%
DCNILogarithmic0.5131.9428.52%0.6426.4924.37%
NDVILogarithmic0.4833.0929.54%0.6127.5025.29%
VIoptLogarithmic0.4633.6430.03%0.6028.0925.84%
RTVIPower0.5435.2631.48%0.4928.3326.06%
OSAVILogarithmic0.4334.3530.67%0.5629.1526.81%
RVILogarithmic0.4035.4531.65%0.5729.1026.77%
MTVI2Logarithmic0.3736.2232.34%0.5130.9028.42%
EVILogarithmic0.3536.8532.90%0.5031.6029.07%
MSAVILogarithmic0.3337.4133.40%0.4832.2029.62%
TVILogarithmic0.2938.5234.39%0.4333.5530.86%
Table
Table 5. LNC prediction results using different structures to design the hybrid model.
Table 5. LNC prediction results using different structures to design the hybrid model.
Model TypeCalibrationValidation
R2RMSE
(μg/cm2)		RMSE%R2RMSE
(μg/cm2)		RMSE%
Single-task method I0.8913.2611.90%0.6824.5622.59%
Single-task method II0.8814.3412.88%0.2238.5835.49%
Single-task method III0.7024.9318.65%0.7123.7721.86%
ML-HM method
(this study)		0.7920.8618.63%0.8218.4016.92%
Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Share and Cite

MDPI and ACS Style

Ma, X.; Chen, P.; Jin, X. Predicting Wheat Leaf Nitrogen Content by Combining Deep Multitask Learning and a Mechanistic Model Using UAV Hyperspectral Images. Remote Sens. 2022, 14, 6334. https://doi.org/10.3390/rs14246334

AMA Style

Ma X, Chen P, Jin X. Predicting Wheat Leaf Nitrogen Content by Combining Deep Multitask Learning and a Mechanistic Model Using UAV Hyperspectral Images. Remote Sensing. 2022; 14(24):6334. https://doi.org/10.3390/rs14246334

Chicago/Turabian Style

Ma, Xiao, Pengfei Chen, and Xiuliang Jin. 2022. "Predicting Wheat Leaf Nitrogen Content by Combining Deep Multitask Learning and a Mechanistic Model Using UAV Hyperspectral Images" Remote Sensing 14, no. 24: 6334. https://doi.org/10.3390/rs14246334

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop