Retrieval of Live Fuel Moisture Content Based on Multi-Source Remote Sensing Data and Ensemble Deep Learning Model

Abstract

Live fuel moisture content (LFMC) is an important index used to evaluate the wildfire risk and fire spread rate. In order to further improve the retrieval accuracy, two ensemble models combining deep learning models were proposed. One is a stacking ensemble model based on LSTM, TCN and LSTM-TCN models, and the other is an Adaboost ensemble model based on the LSTM-TCN model. Measured LFMC data, MODIS, Landsat-8, Sentinel-1 remote sensing data and auxiliary data such as canopy height and land cover of the forest-fire-prone areas in the Western United States, were selected for our study, and the retrieval results of different models with different groups of remote sensing data were compared. The results show that using multi-source data can integrate the advantages of different types of remote sensing data, resulting in higher accuracy of LFMC retrieval than that of single-source remote sensing data. The ensemble models can better extract the nonlinear relationship between LFMC and remote sensing data, and the stacking ensemble model with all the MODIS, Landsat-8 and Sentinel-1 remote sensing data achieved the best LFMC retrieval results, with $R^2$ = 0.85, RMSE = 18.88 and ubRMSE = 17.99. The proposed stacking ensemble model is more suitable for LFMC retrieval than the existing method.

1. Introduction

Live fuel moisture content (LFMC) is the ratio of vegetation water content to its dry weight [1]. The research shows that there is a clear correlation between the probability of fire and LFMC [2,3], which is an important index affecting the occurrence probability and propagation rate of forest wildfire. To put it another way, accurate and dynamic retrieval of LFMC is extremely valuable to realize the fire risk assessment and spatial modeling of fire behavior [4]. Remote sensing satellite can provide large-scale, multi-band and near-real-time image data, which makes remote sensing technology one of the main methods to estimate LFMC on a large scale [5]. The method of estimating LFMC based on optical remote sensing data is the most widely studied [6,7,8]. MODIS optical remote sensing data are commonly used in the early stage. Myoung et al. [9] developed an empirical model function of LFMC using an aqua-enhanced vegetation index based on MODIS satellite data for wildfire risk assessment in Southern California. Carmine et al. [10] developed a new spectral index, the perpendicular moisture index (PMI), which is sensitive to LFMC based on MODIS satellite data. The experimental results show that PMI had a linear relationship with LFMC, and the highest R² was 0.87. Landsat-8 can provide higher spatial resolution than MODIS, which has been introduced to estimate LFMC in recent years. Considering the complexity of upper tree canopy and lower grass canopy, Quan et al. [11] predicted the forest FMC of a two-layer canopy structure in Southwest China by coupling a radiative transfer model and a Landsat-8 product. Mbulisi et al. [5] used Landsat-5 and Landsat-8 data to quantitatively retrieve vegetation canopy FMC in six study areas based on PROSAIL and PROGeoSAIL radiative transfer models. These methods based on optical remote sensing depend on the absorption characteristics of leaf water at near-infrared (NIR) or short-wave infrared (SWIR) wavelengths [12]. Optical and infrared reflectance are highly sensitive to vegetation characteristics such as canopy structure [13] and leaf area index [14,15], and so these models are often only applicable for very specific sites, and the generalization ability of different regions is limited [16,17].

The wavelength of microwave remote sensing is longer than that of optical remote sensing by four orders of magnitude. Microwaves can penetrate the clouds and enter the vegetation canopy, which enables microwave remote sensing to acquire the dynamic changes in vegetation moisture better than optical remote sensing [18,19,20]. In recent years, the prediction ability of active microwave remote sensing technology represented by synthetic aperture radar (SAR) for fire-related variables has been verified [21,22]. Wang et al. [20] coupled the soil backscatter linear model with the vegetation backscatter water cloud model, achieving forest FMC retrieval based on Sentinel-1 SAR data and a better performance than that obtained using Landsat-8 data and empirical methods.

Different remote sensing data have different sensitivities to vegetation water and biomass, and the effect of single-source remote sensing data retrieval of LFMC is limited. Using multi-source remote sensing data to estimate LFMC can avoid the limitations of single-source remote sensing data and provide more comprehensive data for extracting the parameters required for LFMC retrieval [23]. Deep learning can approach the complex nonlinear relationship between various biological, geophysical parameters and remote sensing data through multi-layer learning [24,25], which provides a data-driven alternative for large-scale LFMC retrieval. Rao et al. [19] performed LFMC retrieval based on a long short-term memory (LSTM) network with fused data, i.e., Landsat-8 data, SAR data, terrain, slope and other auxiliary variables. The retrieval of fused data achieved R² = 0.63, RMSE = 25%, which is better than that of single-source remote sensing data (R² = 0.44, RMSE = 31.8%). Zhu et al. [26] proposed the LFMC retrieval architecture TempCNN-LFMC based on temporal convolutional networks (TCNs). With MODIS, altitude, slope and other auxiliary data as the input fused data, the retrieval achieved R² = 0.64, RMSE = 22.74%. The above research shows that the fused data are helpful in improving the performance of LFMC retrieval.

A single model cannot completely extract the features of remote sensing variables in LFMC. To improve the accuracy of LFMC retrieval, it is worthwhile to combine multiple models to extract the features of multi-source remote sensing in time and space dimensions at the same time [25]. Therefore, based on deep learning and ensemble learning methods, this study discusses the LFMC retrieval performance using multi-source remote sensing data. The contents of this study include the following aspects:

(1)

We explore the advantages of LFMC retrieval utilizing multi-source remote sensing data obtained from combing MODIS, Landsat-8, Sentinel-1 and auxiliary data such as canopy height and land cover as data sources, which can provide more comprehensive data and avoid the limitations of single-source remote sensing data.

(2)

We propose a LFMC retrieval model integrating the LSTM and TCN, which exploits the long-time memory capability of LSTM and the superior feature extraction capability of TCN, and finally performs better than LSTM and the TCN alone.

(3)

Based on LSTM, TCN and TCN-LSTM models, two ensemble models (the stacking and Adaboost ensemble models) are designed, and the advantages of stacking ensemble model are confirmed by comparative experiments.

2. Data and Methods

2.1. Study Area

The Western United States was selected as the study area (shown in Figure 1), where wildfires occurred frequently. This area covers more than 3.7 million square kilometers, containing different climates and terrains. The vegetation types are abundant, including broadleaf deciduous forests, needleleaf evergreen forests, shrublands, grasslands and sparse vegetation areas, which made it an ideal area for studying LFMC prediction methods. Considering the integrity and generality of the data, the selected study period was from 1 January 2015 to 31 December 2018.

2.2. Research Data

2.2.1. LFMC Data

The National Fuel Moisture Database (NFMD) [27] is a web-based query system. There have been over 200,000 actual measurements of fuel moisture data since 1977. The database regularly updates monitored fuel moisture data, covering 976 samples mainly located in the Western United States, each covering an area of 5 acres. The measurements were taken in the mid-afternoon and on dry days with no dew or precipitation. In this paper, 133 representative samples were selected, and the specific location is marked by circular points in Figure 1. During the study period, the value of LFMC varied from 16% to 320%, which covers the common water state of live fuels.

2.2.2. MODIS Data

The MODIS data came from MODIS Terra and Aqua joint observation of the MCD43A4 product [28]. The product was the nadir bidirectional reflectance distribution function (BRDF)-adjusted reflectance (NBAR) data, the spatial resolution of which is 500 m. BRDF was fitted using 16-day Terra and Aqua MODIS data and applied to the original reflectance to obtain NBAR. In this study, Band1–Band7 of NBAR reflectivity data were selected as the model input.

In addition, snow cover will lead to abnormal reflectivity. Thus, the snow pixels need to be deleted. The MODIS snow product (MOD10A2-V6) [29] was used to determine whether there was snow. MOD10A2 is a snow cover product synthesized every eight days from the first day of each year. In MOD10A2, if a pixel is classified as snow on any day of the eight days, the pixel is identified as snow.

2.2.3. Landsat Data

Landsat data came from the 16-day surface reflectance data of Landsat-8 [30], which are Level 1T products with a spatial resolution of 30 m. There is a strong correlation between the normalized difference water index (NDWI) and LFMC [31]. Considering that water mainly absorbs the energy of near-infrared (NIR) and short-wave infrared (SWIR) spectral regions, the original band reflectances of red, green, blue, near-infrared and short-wave infrared channels were selected to directly reflect the change in water [32]. The normalized difference vegetation index (NDVI) is a simple, effective and empirical measurement of surface vegetation, and it is also a key factor affecting the prediction of LFMC [33]. The near-infrared vegetation index (NIRV) is an indicator of vegetation biomass level because it is related to carbon assimilation of photosynthesis, so it may help to separate the effects of biomass and LFMC on Sentinel-1 backscattering [34]. To sum up, three vegetation indexes, NDWI, NDVI and NIRV, and the original band reflections of red, green, blue, near-infrared and short-wave infrared channels were selected from Landsat-8 data.

2.2.4. Sentinel-1 Data

Sentinel-1 is a 5.4 GHz C-band synthetic aperture radar (SAR) with a 12-day revisit cycle in the Western United States. The Sentinel-1 data used in this study were derived from the ground-range detector (GRD) data of Sentinel-1, and the data were collected in the wide-strip mode of interferometric measurement with vertical–vertical (VV) and vertical–horizontal (VH) polarization on land [35]. Since the microwave signal has a longer wavelength, is less sensitive to atmospheric conditions, is not susceptible to cloud pollution and can detect deeper vegetation canopy, the microwave remote sensing data can provide more continuous global observation [36]. At the same time, the absorption and scattering of the microwave signal by the surface (including vegetation and soil) is mainly determined by the microwave backscatter σ [37], and microwave backscatter is mainly affected by the moisture content, so the microwave signal is sensitive to vegetation water content [38]. Therefore, in this paper, ${\sigma }_{VV}$, ${\sigma }_{VH}$ and ${\sigma }_{VV}-{\sigma }_{VH}$ were used as the microwave input for the model.

2.2.5. Auxiliary Data

Seven kinds of static auxiliary data were chosen to help the model learn the radiative transfer process between time-varying input and LFMC. The specific data can be divided into the following three categories:

The first category is soil data, including silt, sand and clay content, which was used to control the sensitivity of microwave backscattering to soil moisture, so that the retrieval model could separate vegetation-related information from microwave backscattering. The soil data come from the North American soil map of Liu et al. [39].

Vegetation canopy water content has a certain sensitivity in the near-infrared and short-wave infrared bands [5], and the sensitivity of different vegetations to remote sensing data is also different. The canopy height measured by the Global Laser Altimeter System lidar [40] and the land cover information of 300 m spatial resolution obtained by GLOBCOVER [41] were selected as the second auxiliary data.

The third category is terrain data; considering that the local incidence angle will affect the parameterization of backward scattering on vegetation water [37], it was necessary to use the elevation and slope of the National Elevation Dataset [42] to help the model calibrate the local terrain.

Table 1. Input variables of LFMC retrieval model.

MODIS	Landsat-8	Sentinel-1	Auxiliary Variables
Band1	red	${\sigma }_{VV}$	Silt content
Band2	green	${\sigma }_{VH}$	Sand content
Band3	blue	${\sigma }_{VV}-{\sigma }_{VH}$	Clay content
Band4	NIR		Canopy height (m)
Band5	SWIR		Land cover
Band6	NDWI		Altitude (m)
Band7	NDVI		Slope (°)
	NIRV

summarizes all the inputs used in the model.

2.3. Data Process

Given the presence of numerous vegetation species at certain sample points and the lack of information on the abundance of these species, directly averaging the LFMCs of different species will result in significant inaccuracies. We adopted the same strategy as [19], excluding sampling points with multiple species unless the LFMCs of multiple species were similar during the research period (Pearson r between any two species ≥ 0.5). Thus, 2934 samples from 133 sampling points were included in total (shown in Figure 1).

Because the spatial and temporal resolutions of MODIS, Landsat-8 and Sentinel-1 are not the same, spatial and temporal consistency processing was needed. The remote sensing variable data of the sample points were extracted by the Google Earth Engine (GEE) according to the latitude and longitude coordinates. According to the latitude and longitude coordinates, the spatial synchronization of ground data and remote sensing data could be realized. The remote sensing data were unified to the resolution of 250 m using bilinear interpolation.

The sampling period of live fuel samples in each location was roughly one month, so the time series input was linearly interpolated to the end of each month to ensure that the data had the same time phase. The maximum changes in MODIS, Landsat-8 and Sentinel-1 data were only 2.3%, 6.7% and 3.0%, respectively. It can be considered that the interpolation operation had little effect on the input data.

2.4. Dataset

In this work, three-fold cross-validation was used to test the model. To ensure that the performance of the model was tested on samples that were completely different from the training sample points by separating data by sample points, the data were first stratified randomly sampled into training and test sets by a ratio of 2:1 to ensure that the distribution of land cover types in the training and test sets remained the same. This implies that the training set was made up of data from two-thirds of the locations (89 sample points), while the test set was made up of data from the remaining one-third (44 sample points). In addition, the training set was divided into three folds, two for training and one for validation. Finally, the results presented in the paper were calculated based on the estimated value of the test set.

2.5. LFMC Retrieval Models

2.5.1. TCN-LSTM Model

LSTM can effectively deal with the dynamic dependence of complex long-term time series. The TCN has simple structure and strong feature extraction ability. Combined with the ability of TCN feature extraction and LSTM long-time series memory, the TCN-LSTM network is designed to predict the LFMC. The structure of the TCN-LSTM network is shown in Figure 2.

The retrieval process is as follows:

(1)

Firstly, the LFMC data and selected input variables (x1, …, xn) are fed into the TCN. The features of remote sensing variables and LFMC are extracted through the causal convolution layer contained in the TCN.

(2)

Then, multiple LSTM layers combined with the dropout mechanism are used for prediction, which can prevent over fitting.

(3)

Through the flatten layer, the output matrix is compressed into one dimension to facilitate the connection of the later dense layer.

(4)

The nonlinear relationship is mapped to the output space through the dense layer to achieve the LFMC prediction results.

2.5.2. Stacking Ensemble Model

In order to further improve the performance of LFMC retrieval, a two-layer stacking ensemble model integrating LSTM, TCN and TCN-LSTM was further proposed. The model structure is shown in Figure 3.

The first layer extracts the features from the original split dataset through three basic learners. The basic learners of the stacking model should be “accurate but different”, that is, the prediction accuracy of each basic learner is required to be high, and the types of basic learners should also be diverse. So, LSTM, the TCN and TCN-LSTM were introduced as the base learners. In order to avoid over fitting, a simple linear regression (LR) was selected as the meta-learner of the second layer.

2.5.3. Adaboost Ensemble Model

Unlike stacking ensemble, Adaboost ensemble trains several weak learners based on different training subsets randomly selected from the original training dataset. Adaboost ensemble is based on homogeneous integration, which is composed of the same type of basic learners. In this work, the TCN-LSTM model was selected as the weak learner to construct the Adaboost ensemble model. Figure 4 shows the structure of the Adaboost ensemble model.

In each training process, the initial weights are assigned to the samples at first, and the weights are updated after each iteration. The samples with a high error rate obtain higher weights, which makes the algorithm focus on the samples that are more difficult to learn. The sample weight is adjusted to Dn, and passed to the next weak learner Gn for better prediction. Therefore, the features extracted by G1 are transmitted to G2, and then the features estimated with high error can be corrected in the transmission process, which is helpful to improve the prediction accuracy. At last, the weighted average method is utilized to obtain the strong learner HM, the output of which is the final prediction result. Considering the computational efficiency, the number of TCN-LSTM, that is, the number of iterations t, was set to 3.

2.5.4. Model Settings

There are three basic models, LSTM, the TCN and TCN-LSTM.

Table 2. The architectures of the used basic prediction models.

LSTM		TCN		TCN-LSTM
Layer	Output Shape	Layer	Output Shape	Layer	Output Shape
LSTM	(32,4,10)	Conv1D	(32,365,64)	Conv1D	(32,4,32)
LSTM	(32,4,10)	AvgPool	(32,182,64)	Conv1D	(32,4,32)
LSTM	(32,10)	Conv1D	(32,182,64)	MaxPool	(32,2,32)
Dense	(32,1)	AvgPool	(32,60,64)	Flatten	(32,64)
		Conv1D	(32,60,64)	RepeatVector	(32,1941,64)
		MaxPool	(32,15,64)	LSTM	(32,1941,10)
		Flatten	(32,960)	LSTM	(32,1941,10)
		Dense	(32,256)	LSTM	(32,10)
		Dense	(32,1)	Dense	(32,1)

lists the architectures of these models. In the ensemble models, the same architectures were used. All the proposed models estimated LFMC for each month using input variables of three previous months. Although predicting one month averaged LFMC value can be error-prone due to the variations in LFMC, we were constrained by the temporal resolution of the remote sensing data.

3. Experiments and Results

3.1. Experimental Setup

The hardware environment of the experiments was: CPU: Intel (R) Core (TM) i7-8565U, Memory: 8 GB. The software environment was: Windows 10 64 operating system, deep learning framework Tensorflow2.3.0 and python 3.7. Adam optimizer was used, and the parameters were the default values. The batch size was 32, the learning rate was 0.01, and the epoch was 300. In order to avoid over fitting, early stopping based on the loss of the validation set was used [43], and patience was 30.

3.2. Evaluating Indicator

Bias, determination coefficient $R^2$, root mean square error (RMSE) and unbiased root mean square error (ubRMSE) between estimated and measured LFMCs were chosen to quantitatively evaluate the performance of the models. When R² was closer to 1 and the RMSE value was lower, the model accuracy was higher and the model was more accurate. The calculations of RMSE and ubRMSE are shown in Formulas (1) and (2):

$$RMSE=\sqrt{\frac{1}{N}{{\displaystyle \sum }}_i^N{(LFM{C}_{i,m}-LFM{C}_{i,e})}^2}$$

(1)

$$ubRMSE=\sqrt{\frac{1}{N}{\displaystyle \sum }_i^N{(LFM{C}_{i,m}-LFM{C}_{i,e}-\left(\overline{LFM{C}_m}-\overline{LFM{C}_e}\right))}^2}$$

(2)

where N is the number of measurements; $LFM{C}_{i,m}$ and $LFM{C}_{i,e}$ are the ith measured and estimated LFMC, respectively; $\overline{LFM{C}_m}$ and $\overline{LFM{C}_e}$ are the averages of measured and estimated LFMC, respectively.

3.3. Comparison of Different Deep Learning Models

The performances of three different models, LSTM, the TCN and TCN-LSTM, with different remote sensing data were compared. The retrieval results are shown in Figure 5.

By comparing and analyzing the results in Figure 5 and

Table 3. Evaluation results of LFMC retrieval results based on LSTM, TCN and TCN-LSTM.

Data	Model	Bias (%)	R2	RMSE (%)	ubRMSE (%)
S	LSTM	−9.39	0.38	37.07	35.86
	TCN	−7.42	0.42	35.97	35.2
	TCN-LSTM	−5.93	0.44	34.81	34.3
L	LSTM	−3.93	0.51	32.67	32.43
	TCN	−3.8	0.54	31.58	31.35
	TCN-LSTM	−3.12	0.60	25.83	25.64
M	LSTM	−7.39	0.52	33.01	32.17
	TCN	−5.04	0.55	31.39	30.98
	TCN-LSTM	−4.74	0.62	25.11	24.66
L+S	LSTM	−7.06	0.60	26.32	25.35
	TCN	−4.83	0.67	23.05	22.54
	TCN-LSTM	−4.81	0.72	22.78	22.31
M+S	LSTM	−4.76	0.62	25.33	24.88
	TCN	−4.53	0.68	22.43	21.97
	TCN-LSTM	−4.05	0.75	22.21	21.71
M+L+S	LSTM	−6.53	0.73	24.39	23.5
	TCN	−5.03	0.76	23.03	22.48
	TCN-LSTM	−4.66	0.81	21.73	19.93

, it can be concluded that:

(1)

The bias of all the three models was negative, indicating that all the models underestimated LFMC as a whole. The TCN-LSTM model had the lowest bias among all the models on the same dataset. The bias of Sentinel-1 was the largest, and that of Landsat-8 was the lowest. Although microwave remote sensing (Sentinel-1) is more penetrating due to its high sensitivity to surface moisture, it is difficult to distinguish between vegetation and bare soil backscatter only using microwave remote sensing data, which leads to higher bias. The multi-source remote sensing data fuse the microwave remote sensing and optical remote sensing together, which can be essentially seen as the integration of the microwave backscattering characteristics and optical characteristic. Therefore, the retrieval performances of multi-source remote sensing data were higher than those of the single-source remote sensing data.

(2)

The $R^2$, RMSE and ubRMSE of the TCN-LSTM model were also better than those of the LSTM and TCN models. The retrieval accuracy of the TCN-LSTM model with all three kinds of remote sensing data was the highest at $R^2$ = 0.81, RMSE = 21.73 and ubRMSE = 19.93, which means that TCN-LSTM can incorporate the advantages of LSTM and the TCN and effectively extract the features of multi-source remote sensing.

A comparison with the retrieval results of references is shown in

Table 4. Comparison of different LFMC retrieval methods.

Method	${\mathit{R}}^2$	RMSE (%)
LSTM (Landsat+SAR) [19]	0.63	25
TempCNN-LFMC (MODIS+Auxiliary data) [20]	0.64	22.74
TCN-LSTM model	0.81	21.73

; the TCN-LSTM model with multi-source remote sensing achieved the best results for LFMC retrieval. Compared with the best results of the existing method [20], $R^2$and RMSE were improved by 26.56% and 4.44%, respectively.

3.4. Comparison of Different Ensemble Learning Models

Table 5. Performances of different ensemble learning models.

Data	Stacking				Adaboost
	Bias (%)	${\mathit{R}}^2$	RMSE (%)	ubRMSE (%)	Bias (%)	${\mathit{R}}^2$	RMSE (%)	ubRMSE (%)
S	−5.75	0.53	31.87	31.35	−4.59	0.53	31.62	31.29
L	−3.35	0.7	23.26	23.21	−1.65	0.65	23.6	23.54
M	−4.55	0.74	23.82	23.39	−4.16	0.68	22.53	22.14
L+S	−1.55	0.81	19.96	19.95	−2.61	0.76	22	21.31
M+S	−1.43	0.81	19.86	19.81	−2.7	0.8	20.5	20.32
M+L+S	−0.542	0.85	18.88	17.99	−0.563	0.83	19.7	18.8

shows the performance comparison of different ensemble learning models with different remote sensing data. It can be seen that the performances of two ensemble learning models with all three kinds of remote sensing data were better than with other data. The retrieval results for LFMC based on the stacking ensemble model with MODIS, Landsat-8 and Sentinel-1 are the best. This is mainly due to the integration of the advantages of three different models. While the Adaboost ensemble model only uses one kind of basic learner, its performance was poorer than that of the stacking ensemble model.

Together with Table 3, it was found that with different groups of remote sensing data, the trend in the performances of the single model and the ensemble model was almost the same. The more data used, the better the performance. Additionally, based on multi-source data (M+L+S), compared with the TCN-LSTM model, the $R^2$, RMSE and ubRMSE of the stacking ensemble model realized an improvement of 4.9%, 15.1% and 10.8%, respectively.

4. Discussion

4.1. Explanation of Estimated LFMC Value

In general, when using Landsat-8, Sentinel-1 and MODIS for LFMC retrieval, the estimated value is higher than the measured value, and the linear fitting is good when the LFMC is low. With the increase in LFMC, the estimated value is lower than the actual value, the points are discrete, and the overall correlation is high [44]. Figure 6 shows the LFMC retrieval results and measured values based on two ensemble learning models combined with MODIS, Landsat-8 and Sentinel-1. We can see that two proposed ensemble learning models underestimated high LFMC values (>120%), and there was a systematic bias for phenological periods with high LFMC values. This can be partly explained by the limited sensitivity of the optical sensing data to wet vegetation and the tendency of the proposed method to globally optimize the solution at the cost of underestimation at high values. Similar underestimations have been observed in other studies using physical or data-driven methods [45]. However, such underestimation is not significant when considering the cause of the fire hazard or behavior [46]. Experience has shown that when LFMC is high (>120%), the probability of fire occurrence is comparatively low, or fire movement through this area is limited, so this has less of an impact on fire managers, who might use this model to assess LFMC.

The proposed models also overestimated low values (<30%), which may have been due to the presence of dead combustibles, such as grass fuel [47] and leaf litter. Nevertheless, the magnitude of the positive bias was very small (as shown in Figure 6). Moreover, when the LFMC value is lower than 60%, the likelihood of fire occurrence increasing dramatically [48]. So, if the LFMC value is less than 30%, fire managers will be more aggressive with the estimated results. The impact introduced by the minor error on fire managers who may use the model is limited to the extent that this is overestimated in the range (<30%).

4.2. Advantages of Multi-Source Remote Sensing Data and Ensemble Learning

Due to the different shortcomings of different remote sensing data, as expected, the LFMC retrieval results with all the Landsat-8, Sentinel-1 and MODIS remote sensing data are much higher than those of other data when using the same model, which can be attributed to the fact that multi-source remote sensing data can reduce the uncertainty of single-source data and provide more valuable features derived from the complementarity of different data.

Furthermore, the ensemble learning method comprises several basic learners together to obtain better performance. The Adaboost ensemble model is a sequential ensemble technique, in which the final prediction is based on the weighted average results of three weak learners (TCN-LSTM) trained on different training subsets, while the stacking ensemble model combines three parallel basic learners (LSTM, TCN and TCN-LSTM) in the first layer to extract abundant features, and then concatenates straightforward logistic regression as the second learner. Three different basic learners combined with sequential concatenation operation produced better features and an improved retrieval performance over the Adaboost ensemble model.

To summarize, the combination of multi-source data fusion and ensemble learning can significantly improve retrieval performance and provide considerable potential for accurate LFMC estimation.

4.3. Limitations of the Proposed Method with Processed Data

As we all know, the estimation of LFMC using remote sensing data (such as optical and microwave data) has the same issue since remote sensing data are dependent not only on LFMC but also on other bio- and geophysical characteristics [26]. Previous studies tried to find the empirical relationships or physical models between LFMC and other factors. Despite the satisfactory results of these methods, carefully handcrafted input variables chosen based on our understanding of radiative transfer processes must be selected; additionally, corresponding field data are needed, making these models challenging to generalize and operationalize on large-scale sites.

In our work, we introduced deep learning to capture the complicated nonlinear relationship among the LFMC and the remote sensing data, hoping to avoid the selection of carefully handcrafted input variables and the collection of corresponding field data, making it easier to realize large-scale LFMC estimation. The results demonstrate that this method performs admirably in large-scale sites (133 sampling points) with diverse vegetations, while during the data processing, considering the time resolution of remote sensing data and the frequency of measured LFMC, we interpolated the data to the end of each month, which means the time resolution was one month, resulting in the misrepresentation of daily or weekly fluctuations in LFMC. However, this limitation would be solved by gathering data with a smaller resolution.

Furthermore, we are all aware that the kind of vegetation has a direct impact on various remote sensing data [19,26]. Here, we simply delegated the task of classifying vegetation types implicitly to the deep learning model. Figure 7 presents the RMSEs of LFMC retrieval results of different vegetations, demonstrating that the RMSEs of four single-vegetations are often lower than that of mixed vegetations. In particular, the worst predictions were made for mixed shrub–grassland. This suggests that our previous strategy of selecting sample points makes sense, and that using the selected samples to train the model is beneficial in improving the accuracy of the predictions. Nevertheless, a fully data-driven model would very likely result in mistakes if detailed vegetation distribution data were not included. In practical application, a feasible option is to collect more data and then create more advanced models.

Finally, while it is widely acknowledged that deep learning is a data-driven nonlinear model with high automated learning and generalization capabilities that have the potential to be applied to other regions, the efficacy of its application in other locations requires more data for validation.

5. Conclusions

In this study, a TCN-LSTM model was firstly designed to improve the effect of feature extraction, and further, two ensemble models were proposed based on the TCN-LSTM model to achieve more accurate retrieval of LFMC. Considering the different shortcomings of separate Landsat-8, Sentinel-1 and MODIS remote sensing data, all the three data were utilized together to obtain higher performance. The results of the experiments on the LFMC data from the Western United States show that the stacking ensemble model with all three remote sensing data achieved the best performance. The proposed stacking ensemble model was trained on historical data, which can automatically extract the nonlinear correlation between remote sensing data and LFMC. This enabled the proposed model’s good generalization ability. Our model is data-driven, which means it has the potential to realize significant accuracy in LFMC estimation for other locations with appropriate training data. Meanwhile, our results reveal that our proposed models had a mixture of predictions with low and high amounts of bias. We believe this is because different vegetations are not explicitly considered. We will study and improve our model on more available data in the future.