A Combined Strategy of Improved Variable Selection and Ensemble Algorithm to Map the Growing Stem Volume of Planted Coniferous Forest

: Remote sensing technology is becoming mainstream for mapping the growing stem volume (GSV) and overcoming the shortage of traditional labor-consumed approaches. Naturally, the GSV estimation accuracy utilizing remote sensing imagery is highly related to the variable selection methods and algorithms. Thus, to reduce the uncertainty caused by variables and models, this paper proposes a combined strategy involving improved variable selection with the collinearity test and the secondary ensemble algorithm to obtain the optimally combined variables and extract a reliable GSV from several base models. Our study extracted four types of alternative variables from the Sentinel-1A and Sentinel-2A image datasets, including vegetation indices, spectral reﬂectance variables, backscattering coefﬁcients, and texture features. Then, an improved variable selection criterion with the collinearity test was developed and evaluated based on machine learning algorithms (classiﬁcation and regression trees (CART), k-nearest neighbors (KNN), support vector regression (SVR), and artiﬁcial neural network (ANN)) considering the correlation between variables and GSV (with random forest (RF), distance correlation coefﬁcient (DC), maximal information coefﬁcient (MIC), and Pearson correlation coefﬁcient (PCC) as evaluation metrics), and the collinearity among the variables. Additionally, we proposed a secondary ensemble with an improved weighted average approach (IWA) to estimate the reliable forest GSV using the ﬁrst ensemble models constructed by Bagging and AdaBoost. The experimental results demonstrated that the proposed variable selection criterion efﬁciently obtained the optimal combined variable set without affecting the forest GSV mapping accuracy. Speciﬁcally, considering the ﬁrst ensemble, the relative root mean square error (rRMSE) values ranged from 21.91% to 30.28% for Bagging and 23.33% to 31.49% for AdaBoost, respectively. After the secondary ensemble involving the IWA, the rRMSE values ranged from 18.89% to 21.34%. Furthermore, the variance of the GSV mapped by the secondary ensemble with various ranking methods was signiﬁcantly reduced. The results prove that the proposed combined strategy has great potential to reduce the GSV mapping uncertainty imposed by current variable selection approaches and algorithms.


Introduction
With the aggravation of global warming, resources and environmental sustainability have become the most concerning problem [1][2][3]. Given the decrease in natural forests, planted forests are regarded as the most critical ecological system on land, playing an indispensable role in reducing carbon dioxide concentration [4,5]. The forest growing stock presents a great capability to reduce the uncertainty caused by the employed base models and ultimately provide a reliable forest GSV.
Spurred by the deficiencies of current methods, this study proposed a combined strategy of improved variable selection and ensemble algorithm for mapping forest GSV. The suggested strategy can efficiently obtain the optimally combined variables and reduce the model uncertainty. Based on Sentinel-1A and Sentinel2-A data, we developed a criterion to connect machine learning models (CART, KNN, SVR, ANN) with standard methods for evaluating the variable importance (RF, DC, MIC, and PCC). After constructing the first ensemble models by Bagging and AdaBoost, the secondary ones involved an IWA capable of estimating a reliable forest GSV.
The remainder of this paper is as follows. Section 2 presents the materials and methods considered in this paper and the proposed variable selection criterion and IWA. Section 3 presents the experimental results, while Section 4 presents the discussion. Finally, Section 5 concludes this work.

Study Area
This work considered the Wangyedian experimental forest farm (total area of 25,958 ha), located in the southwest of Chifeng City, Inner Mongolia (118 • 09 -118 • 30 E, 41 • 21 -41 • 39 N), which as shown in Figure 1. The mid-temperate continental monsoon climate characterizes this region, and the mean annual sunshine duration is up to 2913.3 h. The topography of the forest farm is mainly hilly, with an altitude of 800 m~1890 m. The percentage of forest cover in the study area was about 93% by the end of 2016, with a total stock volume of 1.527 million m 3 . The area of the planted coniferous forest is about 49.78%, and the main planted tree species are larch (Larix principis-rupprechtii and Larixolgensis) and Chinese pine (Pinus tabuliformis).
Remote Sens. 2021, 13, x FOR PEER REVIEW 3 of 18 times with Bagging or AdaBoost. On the other hand, combining the advantages of different types of base models has also been proposed, with the results usually being derived from base models utilizing the Voting or Stacking methods. Hence, the ensemble algorithm presents a great capability to reduce the uncertainty caused by the employed base models and ultimately provide a reliable forest GSV. Spurred by the deficiencies of current methods, this study proposed a combined strategy of improved variable selection and ensemble algorithm for mapping forest GSV. The suggested strategy can efficiently obtain the optimally combined variables and reduce the model uncertainty. Based on Sentinel-1A and Sentinel2-A data, we developed a criterion to connect machine learning models (CART, KNN, SVR, ANN) with standard methods for evaluating the variable importance (RF, DC, MIC, and PCC). After constructing the first ensemble models by Bagging and AdaBoost, the secondary ones involved an IWA capable of estimating a reliable forest GSV.
The remainder of this paper is as follows. Section 2 presents the materials and methods considered in this paper and the proposed variable selection criterion and IWA. Section 3 presents the experimental results, while Section 4 presents the discussion. Finally, Section 5 concludes this work.

Study Area
This work considered the Wangyedian experimental forest farm (total area of 25,958 ha), located in the southwest of Chifeng City, Inner Mongolia (118°09′-118°30′E, 41°21′-41°39′N), which as shown in Figure 1. The mid-temperate continental monsoon climate characterizes this region, and the mean annual sunshine duration is up to 2913.3 h. The topography of the forest farm is mainly hilly, with an altitude of 800 m~1890 m. The percentage of forest cover in the study area was about 93% by the end of 2016, with a total stock volume of 1.527 million m 3 . The area of the planted coniferous forest is about 49.78%, and the main planted tree species are larch (Larix principis-rupprechtii and Larixolgensis) and Chinese pine (Pinus tabuliformis).

Framework of This Research
We reduced the influence of the utilized variable selection method and model by suggesting a combined strategy that involved several variable selection criteria and ensemble algorithms in enhancing the GSV estimation. Figure 2 illustrates the combined strategy framework. Our study was divided into three parts: data preparation and variable extraction, variable selection, and constructing the secondary ensemble models.

Framework of This Research
We reduced the influence of the utilized variable selection method and model by suggesting a combined strategy that involved several variable selection criteria and ensemble algorithms in enhancing the GSV estimation. Figure 2 illustrates the combined strategy framework. Our study was divided into three parts: data preparation and variable extraction, variable selection, and constructing the secondary ensemble models.

Ground Data Collection and Processing
Based on the age and spatial distribution of forest GSV, 81 ground samples were measured in September 2017 utilizing random stratification sampling in the study area ( Figure 1). The sample size was 25 m × 25 m, with the position of the four corners and the central point measured through a Global Positioning System (GPS). The height and DBH of each tree were measured, and the volume of each tree was derived from the equations related to the measured height and DBH of the samples. The volumetric equations for the Wangyedian forest farm were defined as follows: Chinese pine: where H, D, and V denote the measured height, DBH, and volume per tree.

Remote Sensing Images and Pre-Processing
This study considered two cloud-free images of Sentinel-1A and Sentinel-2A acquired on September 19 and 22 September 2017, respectively, which are matched with the corresponding ground measurements. Table 2 illustrates the spectral band (Band2-Band8A) from Sentinel-2A and the polarizations (VH and VV) from Sentinel-1A used in this paper, matched for spatial resolution (25 m × 25 m). By employing the SNAP 8.0 software, the radiometric calibration and the Lee sigma filter (size of 7 × 7) were initially processed to reduce the speckle noise of the Sentinel-1A imagery. Then, the Range-Doppler terrain correction technology was used to correct the distance distortion caused by the scene terrain change and satellite sensor tilt. Regarding the Sentinel-2A imagery, the radiation correction, geometric correction, and atmospheric correction were applied to reduce the errors caused by the influence of interference factors.

Extraction of Variables
After pre-processing the images, four types of alternative variables, including vegetation indices, the values of spectral reflection, backscattering coefficients, and texture features, were extracted from the Sentinel-1A and Sentinel-2A imagery (Table 3). The spectral reflection values were directly derived from band 2 to band 8A for Sentinel-2A, and six commonly used vegetation indices were obtained through the mathematical operation of related bands, including Enhanced Vegetation Index (EVI) [36], Enhanced Vegetation Index-2 (EVI-2) [37], Normalized Difference Vegetation Index (NDVI) [38], Ratio Vegetation Index (RVI) [39], Spectral Vegetation Index (SVI) [40] and Soil Adjusted Vegetation Index Remote Sens. 2021, 13, 4631 6 of 18 (SAVI) [41]. Moreover, backscattering coefficients of VH and VV polarization were extracted from Sentinel-1A, and the ratio of VH to VV was also calculated. Additionally, using the Gray Level Co-occurrence Matrix (GLCM), eight texture features (mean, variance, uniformity, contrast, dissimilarity, entropy, second moment, and correlation) were calculated with a size of 3 × 3 in the spectral reflection values, and backscattering coefficients.

Proposed Variable Selection Criterion
Typically, the number and types of the selected variables are highly dependent on the variable selection approaches. It is necessary to select an optimal variables combination for obtaining a reliable forest GSV and construct the models. However, the filter method selects variables by specific quantified criteria without considering the model's accuracy, such as RF, DC, MIC and PCC. On the other hand, the wrappers that focus on the model's accuracy will be time-consuming. For example, the forward selection method, which only considers accuracy, is denoted by FORW in this paper. This method iterates through all the variables and selects variables with the greatest gain until all the added variables fail to increase the model's accuracy. To overcome these drawbacks, the autocorrelation and interaction of independent variables were considered to reduce the operating time. Thus, we proposed an improved variable selection criterion involving a collinearity test that combined machine learning models with standard methods that evaluated the variable's importance. The proposed variable selection criterion involves the following steps: Step 1: All alternative variables are ranked by selecting a single criterion (calculated by RF, DC, MIC, and PCC).
Step 2: The ranked first variable was initially selected as the most critical variable, and the initial rRMSE 0 was calculated by the employed model and the most critical variable.
Step 3: Then, the most critical variable was again selected among the remaining alternative variables and was tested for collinearity (VIF) with each selected variable. If VIF > 10, proceed to step 4. Otherwise, the variable is excluded, and step 3 is repeated for the remaining alternative variables.
Step 4: The selected variable that satisfies the collinearity test is added to the selected variable set, and the rRMSE 1 is calculated using the updated combined variables set. If rRMSE 1 < rRMSE 0, the variable is defined as valid. Otherwise, this variable is excluded.
Step 5: Repeat step 3 and step 4. The variables whose contribution decreased the rRMSE values are added to the combined variables set.
In the proposed criterion, the combined variables set were efficiently selected using the collinearity test. The capability of the proposed criterion regarding variable selection for estimating GSV depended on the single criterion for ranking and the algorithms. Therefore, four ranking criteria (RF, DC, MIC, and PCC) and four machine learning algorithms (CART, KNN, SVR, and ANN) were employed during processing. Additionally, we selected the FORW method as the reference method during the experiments.

Secondary Ensemble Algorithm
The estimated GSV accuracy is highly related to the employed models. To reduce this dependence, four machine learning models (CART, KNN, SVR, and ANN) were utilized as base learners, and then, we constructed the first ensemble models utilizing Bagging and AdaBoost. Therefore, we extracted four ensemble models using Bagging and four ensemble models using AdaBoost by the optimal combined variables sets.
After the first ensemble, the validated rRMSE values of the eight models were regarded as weight coefficients. The estimated GSV uncertainty imposed by the different models was reduced through the proposed secondary ensemble involving an IWA, defined as: whereŶ i andŷ i are the estimated GSV from the secondary and first ensemble models, respectively, rRMSE i is the value of rRMSE from the first ensemble models, and n is the number of selected first ensemble models. Specifically, the selected first ensemble model that did not decrease the rRMSE values for forest GSV was neglected. The first ensembles were substituted into Equation (3) according to its validated rRMSE value from smallest to largest. All n values (1-8) are traversed to minimize the rRMSE value of the secondary ensemble model. The construction process of the secondary ensemble model is illustrated in Figure 3.
Remote Sens. 2021, 13, x FOR PEER REVIEW 7 of 18 Step 5: Repeat step 3 and step 4. The variables whose contribution decreased the rRMSE values are added to the combined variables set.
In the proposed criterion, the combined variables set were efficiently selected using the collinearity test. The capability of the proposed criterion regarding variable selection for estimating GSV depended on the single criterion for ranking and the algorithms. Therefore, four ranking criteria (RF, DC, MIC, and PCC) and four machine learning algorithms (CART, KNN, SVR, and ANN) were employed during processing. Additionally, we selected the FORW method as the reference method during the experiments.

Secondary Ensemble Algorithm
The estimated GSV accuracy is highly related to the employed models. To reduce this dependence, four machine learning models (CART, KNN, SVR, and ANN) were utilized as base learners, and then, we constructed the first ensemble models utilizing Bagging and AdaBoost. Therefore, we extracted four ensemble models using Bagging and four ensemble models using AdaBoost by the optimal combined variables sets.
After the first ensemble, the validated rRMSE values of the eight models were regarded as weight coefficients. The estimated GSV uncertainty imposed by the different models was reduced through the proposed secondary ensemble involving an IWA, defined as: where and are the estimated GSV from the secondary and first ensemble models, respectively, is the value of rRMSE from the first ensemble models, and n is the number of selected first ensemble models. Specifically, the selected first ensemble model that did not decrease the rRMSE values for forest GSV was neglected. The first ensembles were substituted into Equation (3) according to its validated rRMSE value from smallest to largest. All n values (1-8) are traversed to minimize the rRMSE value of the secondary ensemble model. The construction process of the secondary ensemble model is illustrated in Figure 3.

Accuracy Evaluation
To evaluate the estimated GSV of the various approaches, all ground measured samples were divided into three sets: training (36 samples), validation (18 samples), and testing (27 samples). We randomly repeated 100 times, selecting the training and validation sets to reduce the uncertainty for forest GSV estimation. Then, the rRMSE and determination coefficient (R 2 ) were calculated using the test set by averaging the results of each employed model. These indexes were calculated as follows:

The Results of Variables Selection
In our study, 99 variables were extracted from Sentinel-1A and Sentinel-2A, and four standard methods of variable importance evaluation were employed, including RF, DC, MIC, and PCC, to rank all variables, such as vegetation indices, the spectral reflection values, backscattering coefficients, and texture features. The rRMSE values varied with the number of selected variables but determining the number of variables for estimating forest GSV was controversial. Figure 4 illustrates that for the single criterion case, the accuracy of the estimated GSV varied with the number of combined variables.

Accuracy Evaluation
To evaluate the estimated GSV of the various approaches, all ground measured samples were divided into three sets: training (36 samples), validation (18 samples), and testing (27 samples). We randomly repeated 100 times, selecting the training and validation sets to reduce the uncertainty for forest GSV estimation. Then, the rRMSE and determination coefficient (R 2 ) were calculated using the test set by averaging the results of each employed model. These indexes were calculated as follows:

The Results of Variables Selection
In our study, 99 variables were extracted from Sentinel-1A and Sentinel-2A, and four standard methods of variable importance evaluation were employed, including RF, DC, MIC, and PCC, to rank all variables, such as vegetation indices, the spectral reflection values, backscattering coefficients, and texture features. The rRMSE values varied with the number of selected variables but determining the number of variables for estimating forest GSV was controversial. Figure 4 illustrates that for the single criterion case, the accuracy of the estimated GSV varied with the number of combined variables.  To construct the proposed variable selection criterion and establish the optimally combined variables, four variable selection methods (RF, DC, MIC, and PCC) and four machine learning models (CART, KNN, SVR, and ANN) were employed. Additionally, the proposed strategy was challenged against the FORW method. The results of the combined variables are listed in Table 4. The first selected variable per approach depended on the ranking method, while the numbers of operations and the selected variables were related to the employed models for estimating GSV. Using the suggested variable selection criteria with various single ranking methods, the selected variables ranged from 4 to 9. By using the FORW, the corresponding number varied from 2 to 10 and the number of operations from 295 to 1034. To construct the proposed variable selection criterion and establish the optimally combined variables, four variable selection methods (RF, DC, MIC, and PCC) and four machine learning models (CART, KNN, SVR, and ANN) were employed. Additionally, the proposed strategy was challenged against the FORW method. The results of the combined variables are listed in Table 4. The first selected variable per approach depended on the ranking method, while the numbers of operations and the selected variables were related to the employed models for estimating GSV. Using the suggested variable selection criteria with various single ranking methods, the selected variables ranged from 4 to 9. By using the FORW, the corresponding number varied from 2 to 10 and the number of operations from 295 to 1034. The proposed variable selection criteria show great potential for promoting efficiency. Indeed, the number of operations required by our strategy ranged from 20 to 95, significantly less than FORW (Table 4), with Figure 5 illustrating the variable selection processing utilizing various approaches. By employing FORW's criterion, the number of operations significantly increased (Figure 5a,b). Filtering the alternative variables through the collinearity test increased the efficiency of determining the number of optimally combined variables for all four ranking methods (Figure 5c-f).

Method of Ranking Models
The First Selected Variable The proposed variable selection criteria show great potential for promoting efficiency. Indeed, the number of operations required by our strategy ranged from 20 to 95, significantly less than FORW (Table 4), with Figure 5 illustrating the variable selection processing utilizing various approaches. By employing FORW's criterion, the number of operations significantly increased (Figure 5a,b). Filtering the alternative variables through the collinearity test increased the efficiency of determining the number of optimally combined variables for all four ranking methods (Figure 5c-f). Furthermore, the accuracy of estimating the forest GSV is also a critical factor in evaluating the proposed variable selection criterion. Figure 6 presents the rRMSE and the R 2 histogram extracted from different variable selection approaches. The values of rRMSE ranged from 33.32% to 38.20% for FORW and from 30.40% to 37.75% for the proposed criterion utilizing four methods (RF, DC, MIC, and PCC), respectively. Considering the FORW's R 2 values, these were slightly lower than those of the proposed criterion. Therefore, the proposed criterion efficiently obtained the optimally combined variables without decreasing the accuracy of mapping forest GSV. Furthermore, the accuracy of estimating the forest GSV is also a critical factor in evaluating the proposed variable selection criterion. Figure 6 presents the rRMSE and the R 2 histogram extracted from different variable selection approaches. The values of rRMSE ranged from 33.32% to 38.20% for FORW and from 30.40% to 37.75% for the proposed criterion utilizing four methods (RF, DC, MIC, and PCC), respectively. Considering the FORW's R 2 values, these were slightly lower than those of the proposed criterion. Therefore, the proposed criterion efficiently obtained the optimally combined variables without decreasing the accuracy of mapping forest GSV.

The Result of the Secondary Ensemble
As already noted, all ground measured samples were divided into three sets: training (36 samples), validation (18 samples), and testing (27 samples). For each base learner model and ranking method, the approaches of Bagging and AdaBoost were applied to reduce the uncertainty of constructing the model based on the selected training and validation set. Table 5 and Figure 7 illustrate the results of the first ensemble of forest GSV.
Considering the results from four base learners (Figure 6), the rRMSE values exceeded 30%, and the R 2 was less than 0.5. After the first ensemble exploiting Bagging and AdaBoost (Table 5), the accuracy of results was obviously improved, and the rRMSE and R 2 values ranged from 21.93% to 31.49% and from 0.43 to 0.72, respectively. Furthermore, the range variance of the precision index was relatively small for Bagging and AdaBoost. However, the accuracy of estimating GSV is still highly related to the employed models.
We reduced the performance variance when various models and combined variable sets were applied by involving a secondary ensemble that integrated the results from different base learners. This study regarded as base learners four machine learning models (CART, KNN, SVR, and ANN) and employed the IWA to construct the model of the secondary ensemble. For each ranking method, eight results were extracted from the four base learners through Bagging and AdaBoost. The first ensemble results were joined to

The Result of the Secondary Ensemble
As already noted, all ground measured samples were divided into three sets: training (36 samples), validation (18 samples), and testing (27 samples). For each base learner model and ranking method, the approaches of Bagging and AdaBoost were applied to reduce the uncertainty of constructing the model based on the selected training and validation set. Table 5 and Figure 7 illustrate the results of the first ensemble of forest GSV.  Unlike the first ensemble models, the secondary ensemble models have the advantages of base learners, and the estimated forest GSV is more reliable than the results from the first ensemble. Table 5 illustrates that the rRMSE values from the secondary ensemble models (ranged from 18.89% to 21.34%) were less than those from the first ensemble models (ranged from 21.93% to 31.49%), and the variance of the accuracy was narrowed to within 3% (Figure 7). It is found that the secondary ensemble decreased the variance between base models and ranking methods.
To further analyze the results from various approaches, the scatterplots between predicted and measured GSV with various models are shown in Figure 8. For the base learners without integration (Figure 8a,b), the accuracies of the estimated forest GSV were highly related to the models and the variable selection methods, and overestimated results often appeared. The errors between the predicted and measured GSV were reduced using the first ensemble (Figure 8c,d). After the secondary ensemble, the variance between the Considering the results from four base learners (Figure 6), the rRMSE values exceeded 30%, and the R 2 was less than 0.5. After the first ensemble exploiting Bagging and AdaBoost (Table 5), the accuracy of results was obviously improved, and the rRMSE and R 2 values ranged from 21.93% to 31.49% and from 0.43 to 0.72, respectively. Furthermore, the range variance of the precision index was relatively small for Bagging and AdaBoost. However, the accuracy of estimating GSV is still highly related to the employed models.
We reduced the performance variance when various models and combined variable sets were applied by involving a secondary ensemble that integrated the results from different base learners. This study regarded as base learners four machine learning models (CART, KNN, SVR, and ANN) and employed the IWA to construct the model of the secondary ensemble. For each ranking method, eight results were extracted from the four base learners through Bagging and AdaBoost. The first ensemble results were joined to the secondary ensemble model one by one, and the contribution to decreasing the rRMSE values was regarded as a threshold to determine whether the model should be removed. Finally, after exploiting the secondary ensemble, the corresponding results from four ranking methods are presented in Table 5 and Figure 7.
Unlike the first ensemble models, the secondary ensemble models have the advantages of base learners, and the estimated forest GSV is more reliable than the results from the first ensemble. Table 5 illustrates that the rRMSE values from the secondary ensemble models (ranged from 18.89% to 21.34%) were less than those from the first ensemble models (ranged from 21.93% to 31.49%), and the variance of the accuracy was narrowed to within 3% (Figure 7). It is found that the secondary ensemble decreased the variance between base models and ranking methods.
To further analyze the results from various approaches, the scatterplots between predicted and measured GSV with various models are shown in Figure 8. For the base learners without integration (Figure 8a,b), the accuracies of the estimated forest GSV were highly related to the models and the variable selection methods, and overestimated results often appeared. The errors between the predicted and measured GSV were reduced using the first ensemble (Figure 8c,d). After the secondary ensemble, the variance between the base models decreased, especially for the samples with high GSV. The experimental results proved that the ensemble between various base learners had improved the mapping forest GSV accuracy.
Remote Sens. 2021, 13, x FOR PEER REVIEW 12 of 18 base models decreased, especially for the samples with high GSV. The experimental results proved that the ensemble between various base learners had improved the mapping forest GSV accuracy.

Mapping the Forest GSV
To map the forest GSV, the results of the secondary ensemble utilizing four ranking methods were derived from the first ensemble models through the IWA (Figure 9). The latter figure illustrates that the estimated GSV ranged from 50 m 3 /ha to 250 m 3 /ha. To further analyze the capability of the ensemble, the variance between the ensemble models using various ranking methods was extracted from the first and secondary ensemble models, and the histogram of the variance between the ensemble models is shown in Figure 10. Five groups of variance GSV from the first ensemble (Bagging and AdaBoost) and secondary ensemble (IWA) were extracted from the mapped forest GSV. For the results of the secondary ensemble, the mapped GSV extracted from the IWA with PCC was regarded as a reference, and the variance of GSV between PCC and other methods was calculated. Regarding the results obtained from Bagging and AdaBoost, the mapped GSV utilizing the CART model with four ranking methods were employed, and the results extracted from the first ensemble with RF were regarded as a reference. Additionally, the Bagging and AdaBoost results with the variable ranking of PCC and the mapped GSV extracted from the first ensemble (Bagging and AdaBoost) with CART were also regarded as a reference.
The variance between the two mapped GSV was divided into three parts: less than 50 m 3 /ha, 50 to 100 m 3 /ha, and greater than 100 m 3 /ha. For the same ranking method variables (PCC), the estimated GSV varied with the employed base model and ensemble algorithms, and the percentage of statistic variance GSV (greater than 100 m 3 /ha) was up to 34.16%. For the same base learner with different ensemble algorithms, the mapped GSV uncertainty still hindered obtaining reliable forest GSV because of the different capabilities of the various ensemble.

Mapping the Forest GSV
To map the forest GSV, the results of the secondary ensemble utilizing four ranking methods were derived from the first ensemble models through the IWA (Figure 9). The latter figure illustrates that the estimated GSV ranged from 50 m 3 /ha to 250 m 3 /ha. To further analyze the capability of the ensemble, the variance between the ensemble models using various ranking methods was extracted from the first and secondary ensemble models, and the histogram of the variance between the ensemble models is shown in Figure 10. Five groups of variance GSV from the first ensemble (Bagging and AdaBoost) and secondary ensemble (IWA) were extracted from the mapped forest GSV. For the results of the secondary ensemble, the mapped GSV extracted from the IWA with PCC was regarded as a reference, and the variance of GSV between PCC and other methods was calculated. Regarding the results obtained from Bagging and AdaBoost, the mapped GSV utilizing the CART model with four ranking methods were employed, and the results extracted from the first ensemble with RF were regarded as a reference. Additionally, the Bagging and AdaBoost results with the variable ranking of PCC and the mapped GSV extracted from the first ensemble (Bagging and AdaBoost) with CART were also regarded as a reference. Remote Sens. 2021, 13, x FOR PEER REVIEW 13 of 18  After applying the secondary ensemble, the percentage of statistic variance GSV (less than 50 m 3 /ha) ranged from 82.01% to 90.59%, and the percentage from 0.43% to 2.54% for variance exceeding 100 m 3 /ha. The mapped GSV uncertainty was significantly reduced  After applying the secondary ensemble, the percentage of statistic variance GSV (less than 50 m 3 /ha) ranged from 82.01% to 90.59%, and the percentage from 0.43% to 2.54% for variance exceeding 100 m 3 /ha. The mapped GSV uncertainty was significantly reduced The variance between the two mapped GSV was divided into three parts: less than 50 m 3 /ha, 50 to 100 m 3 /ha, and greater than 100 m 3 /ha. For the same ranking method variables (PCC), the estimated GSV varied with the employed base model and ensemble algorithms, and the percentage of statistic variance GSV (greater than 100 m 3 /ha) was up to 34.16%. For the same base learner with different ensemble algorithms, the mapped GSV uncertainty still hindered obtaining reliable forest GSV because of the different capabilities of the various ensemble.
After applying the secondary ensemble, the percentage of statistic variance GSV (less than 50 m 3 /ha) ranged from 82.01% to 90.59%, and the percentage from 0.43% to 2.54% for variance exceeding 100 m 3 /ha. The mapped GSV uncertainty was significantly reduced regardless of the four variables ranking methods. Therefore, it is proved that the secondary ensemble exploiting the IWA afforded a reliably mapped forest GSV by reducing the variance between the selected models.

Variable Selection
The accuracy of estimating GSV is strongly related to the variable selection methods [8,[42][43][44]. However, it is still controversial to determine the combination of the optimal variable when mapping the forest GSV employing imagery from various sensors. It is well known that the standard variable selection methods consider only the relationship between variables and GSV through a quantitative evaluation process, such as the PCC correlation coefficient. In previous studies, these approaches were often used to assess the sensitivity and set thresholds directly [16,[45][46][47][48]. However, the GSV estimation accuracy depending on the thresholds was not stable (Figure 4). The FORW method combined variable selection methods with the estimated models, with its higher accuracy obtained at the expense of computational complexity [49,50].
Our study, in the proposed variable selection criterion, considered the variables' combination effect and collinearity. Based on our experimental results, the number of operations required by the proposed approach was significantly less (ranged from 21 to 95) than these of FORW (ranged from 295 to 1034), and the time for searching the optimal combination of variables was significantly reduced. Furthermore, the estimated GSV accuracy using the proposed variables selection method was slightly higher than the FORW. Therefore, the proposed variable selection criteria can efficiently obtain the optimally combined variables without decreasing the accuracy of GSV estimation (Table 4 and Figure 5).

Ensemble Model
After selecting the optimal variables combination, the accuracy and reliability of the estimated GSV are highly related to the capability of the employed models. Generally, machine learning approaches have been widely used to solve complex and multi-variable problems [51][52][53]. These approaches were also employed to map forest GSV to combine the advantages of multi-source remote sensing data. Indeed, the machine learning methods achieved higher estimated GSV accuracy compared to parameter models. However, the estimated GSV uncertainty was induced when a single machine learning model was selected [33][34][35]. Similar results were also obtained in our study, with the rRMSE and R 2 values obtained by a single machine learning model ranging from 30.40% to 37.75% and from 0.18 to 0.47, respectively. Thus, the mapped GSV uncertainty was generated by the capability of the employed models [7,34,35].
To reduce the impact of uncertainty, various ensemble algorithms were employed, combining several base models. The ensemble algorithms had a better generalization capability and presented a more robust performance than the single model. In this study, the estimated GSV accuracy greatly improved with the rRMSE values ranging from 21.91% to 31.49% after the first ensemble with Bagging or AdaBoost. Figure 11 illustrates the relationship between the residuals and ground measured GSV. The residuals derived from the first ensemble (Figure 11c,d) were smaller than those derived from the single machine learning model (Figure 11a,b). To reduce the various model uncertainty, we proposed a secondary ensemble to obtain the merits of each selected base model. The methods involved in the first ensemble reduce the model uncertainty caused by the samples, while the uncertainty from the different models themselves is omitted. Exploiting the secondary ensemble decreased the variance between various models, and the proposed IWA further enhanced the forest GSV reliability. The percentage of statistic variance GSV (smaller than 50 m 3 /ha) ranged from 82.01% to 90.59% for the IWA. The results of existing studies exploiting ensemble algorithms for learners with different [7,34] or the same base [19,54,55] learners have also shown an appealing estimation accuracy. Therefore, it is confirmed that the proposed combination strategy of variable selection and ensemble algorithm has a great capability to obtain reliable forest GSV.
The distribution of GSV in this study area was also mapped in previous studies [7,56]. In this paper, we obtained a similar GSV estimation accuracy, but the remote sensing datasets used, the variable selection methods and the ensemble algorithms were different, and our objectives were not the same. Multiple variable selection methods and ensemble algorithms were compared and used in this study to reduce uncertainty and obtain more accurate and stable estimation results.

Conclusions
This study proposed a combined strategy of improved variable selection and ensemble algorithm to map the reliable GSV of planted coniferous forests. Considering the variables' combination effects and collinearity, an improved variable selection criterion was initially applied to efficiently select the combination of the optimal variable extracted from Sentinel-1A and Sentinel-2A datasets. Four machine learners were regarded as base learners, and the combined strategy was constructed to reduce the uncertainty of the mapped GSV caused by selecting samples and models. It is proved that the proposed variable selection criterion could efficiently obtain the optimal combination of variables without reducing the accuracy of mapping forest GSV. The results also confirmed that the first ensemble models using Bagging and AdaBoost were more stable and accurate than a single machine learner.
Furthermore, the uncertainty of various models is significantly decreased by utilizing the secondary ensemble with the IWA. The percentage of statistic variance GSV (less than To reduce the various model uncertainty, we proposed a secondary ensemble to obtain the merits of each selected base model. The methods involved in the first ensemble reduce the model uncertainty caused by the samples, while the uncertainty from the different models themselves is omitted. Exploiting the secondary ensemble decreased the variance between various models, and the proposed IWA further enhanced the forest GSV reliability. The percentage of statistic variance GSV (smaller than 50 m 3 /ha) ranged from 82.01% to 90.59% for the IWA. The results of existing studies exploiting ensemble algorithms for learners with different [7,34] or the same base [19,54,55] learners have also shown an appealing estimation accuracy. Therefore, it is confirmed that the proposed combination strategy of variable selection and ensemble algorithm has a great capability to obtain reliable forest GSV.
The distribution of GSV in this study area was also mapped in previous studies [7,56]. In this paper, we obtained a similar GSV estimation accuracy, but the remote sensing datasets used, the variable selection methods and the ensemble algorithms were different, and our objectives were not the same. Multiple variable selection methods and ensemble algorithms were compared and used in this study to reduce uncertainty and obtain more accurate and stable estimation results.

Conclusions
This study proposed a combined strategy of improved variable selection and ensemble algorithm to map the reliable GSV of planted coniferous forests. Considering the variables' combination effects and collinearity, an improved variable selection criterion was initially applied to efficiently select the combination of the optimal variable extracted from Sentinel-1A and Sentinel-2A datasets. Four machine learners were regarded as base learners, and the combined strategy was constructed to reduce the uncertainty of the mapped GSV caused by selecting samples and models. It is proved that the proposed variable selection criterion could efficiently obtain the optimal combination of variables without reducing the accuracy of mapping forest GSV. The results also confirmed that the first ensemble models using Bagging and AdaBoost were more stable and accurate than a single machine learner.
Furthermore, the uncertainty of various models is significantly decreased by utilizing the secondary ensemble with the IWA. The percentage of statistic variance GSV (less than 50 m 3 /ha) ranged from 82.01% to 90.59% for the IWA. It is proved that the proposed combination strategy has great potential to reduce the variance of estimated GSV between various variable selection approaches and employed base models. However, the