Evaluation of Water Use Efficiency Derived from MODIS Products against Eddy Variance Measurements in China

Abstract

Water use efficiency (WUE) is a useful indicator to illustrate the interaction of carbon and water cycles in terrestrial ecosystems. MODIS gross primary production (GPP) and evapotranspiration (ET) products have been used to analyze the spatial and temporal patterns of WUE and their relationships with environmental factors at regional and global scales. Although MODIS GPP and ET products have been evaluated using eddy covariance flux measurements, the accuracy of WUE estimated from MODIS products has not been well quantified. In this paper, we evaluated WUE estimated from MODIS GPP and ET products against eddy covariance measurements of GPP and ET during 2003–2008 at eight sites of the Chinese flux observation and research network (ChinaFLUX) and conducted sensitivity analysis to investigate the possible key contributors to the bias of MODIS products. Results show that MODIS products underestimate eight-day water use efficiency in four forest ecosystems and one cropland ecosystem with the bias from −0.36–−2.28 g·C·kg⁻¹ H₂O, while overestimating it in three grassland ecosystems with the bias from 0.26–1.11 g·C·kg⁻¹ H₂O. Mean annual WUE was underestimated by 14%–54% at four forest sites, 45% at one cropland site and 7% at an alpine grassland site, but overestimated by 66% and 9% at a temperate grassland site and an alpine meadow site, respectively. The underestimation of WUE by MODIS data results from underestimated GPP and overestimated ET at four forest sites, while MODIS WUE values are significantly overvalued mainly due to underestimated ET in the three grassland ecosystems. The maximum light use efficiency and fraction of photosynthetically-active radiation (FPAR) were the two most sensitive factors to the estimation of WUE derived from the MODIS GPP and ET algorithms. The error in meteorological data partly caused the overestimation of ET and accordingly underestimation in WUE in subtropical and tropical forests. The bias of MODIS-produced WUE was also derived from the uncertainties in eddy flux data due to gap-filling processes and unbalanced surface energy issue. Their contributions to the uncertainty in estimated WUE at both eight-day and annual scales still need to be further quantified.

1. Introduction

Water use efficiency (WUE), the ratio of carbon assimilation or productivity to water loss, is a valuable and commonly-used index to investigate the relationship between terrestrial carbon and water cycles [1,2,3]. Improving WUE is an effective way of saving plant water consumption and optimizing water management. Therefore, it has been extensively studied at different scales (leaf scale, canopy scale and regional scale) [4,5,6]. At the ecosystem level, WUE can be calculated by the ratio of gross primary productivity (GPP) to evapotranspiration (ET). Eddy covariance techniques provide an effective way to quantify the magnitudes, patterns and changes in WUE at the ecosystem level over varied vegetation types [7,8,9,10,11,12]. However, it is still difficult to obtain dynamic WUE maps over a large region or even at the global level based on eddy flux data measured at a limited number of tower sites.

Satellite-based remote sensing techniques play an important role in assessing the impacts of global change on terrestrial ecosystems for their ability to provide land surface information on regional and global scales. Great progress has been made in the field of remote sensing for Earth observation in recent years, which brings us more remote sensing products with higher spatial, temporal and spectral resolution [13]. At present, the information on the spatiotemporal patterns of ecosystem WUE could be acquired from remote sensing methods. With the availability of satellite GPP and ET products, such as MODIS products (Moderate-Resolution Imaging Spectroradiometer, MOD17 for GPP and MOD16 for ET), it provides a direct way of retrieving WUE at the regional scale. MODIS GPP and ET products have been used to investigate the regional and global WUE of vegetation [3,14]. However, the accuracy of WUE estimated from MODIS products has not been well quantified.

Eddy covariance measurements have been regarded as an effective way to evaluate the performance of MODIS GPP and ET products, due to their representing fluxes at the ecosystem level coving the area between one hundred meters and several kilometers. Previous studies show that MODIS GPP tends to underestimate at high productivity sites (especially for croplands) and to overestimate at low productivity sites [15,16,17], and MODIS ET is generally overestimated in forest and underestimated in grassland and cropland [18,19,20,21]. Although the accuracy of GPP and ET may reflect the accuracy of WUE to some degree, because WUE is estimated from them, there is still a difference between individual evaluation for GPP and ET and synthetic evaluation for WUE due to WUE being the ratio of GPP to ET. The errors in WUE from GPP and ET may be canceled out or be magnified. Heinsch et al. [22] found that meteorological data contributes an error of 28% to MODIS annual GPP. In contrast, Liu et al. [21] argued that the impact of meteorological data on GPP estimation is little, while the parameter maximum light use efficiency and remote sensing leaf area index/fraction of photosynthetically-active radiation (LAI/FPAR) have greater effects on GPP estimations. LAI/FPAR also have a great impact on ET estimation [21]. However, it is still unclear how input meteorological and remote sensing variables and model parameters influence the estimation of ET and, thus, WUE. Therefore, the objectives of this paper are to evaluate WUE estimated from MODIS GPP and ET products against eddy covariance measurements of GPP and ET at eight sites of the Chinese flux observation and research network (ChinaFLUX) and to the identify key factors that influence the accuracy of WUE derived from MODIS data.

2. Data and Methods

2.1. Study Sites

Observed WUE was calculated by the ratio of GPP and ET, which was collected at eight ChinaFLUX eddy covariance flux sites from 2003–2008. Detailed information for each flux site is listed in Table 1. These eight flux sites cover the main dominant vegetation types in China, including Changbaishan temperate mixed forest (CBS), Qianyanzhou evergreen needle leaf forest (QYZ), Dinghushan evergreen mixed forest (DHS), Xishuangbanna evergreen broadleaf forest (XSBN), Inner Mongolia temperate steppe (NM), Haibei alpine shrub-meadow (HBGC), Damxung alpine steppe-meadow (DX) and Yucheng cropland (YC).

Table 1. Site descriptions (Changbaishan temperate mixed forest (CBS), Qianyanzhou evergreen needle leaf forest (QYZ), Dinghushan evergreen mixed forest (DHS), Xishuangbanna evergreen broadleaf forest (XSBN), Inner Mongolia temperate steppe (NM), Haibei alpine shrub-meadow (HBGC), Damxung alpine steppe-meadow (DX) and Yucheng cropland (YC)).

Sites	Location and Altitude	Mean Annual Temperature (°C)	Mean Annual Precipitation (mm)	Soil Type	Predominant Species
CBS	42.40°N, 128.10°E; 738 m	3.6	713	Upland dark brown forest soil	Pinus koraiensis, Tilia amurensis, Quercus mongolica and Fraxinus mandshurica
QYZ	26.73°N, 115.05°E; 102 m	17.9	1542	Typical red earth	Pinus elliottii Engelm, Pinus massoniana Lamb and Cunninghamia lanceolate Hook
DHS	23.15°N, 112.50°E; 300 m	20.9	1956	Lateritic red-earth, yellow-earth and mountain shrubby-meadow soil	Schima superba, Castanopsis chinensis, Pinus massoniana
XSBN	21.90°N, 101.27°E; 756 m	21.8	1493	Lateritic and red lateratic soil	Pometia tomentosa, Terminalia myriocarpa
NM	44.13°N, 116.30°E; 1189 m	0.4	350.9	Chernozem soil	Leymus chinensis, Stipagrandis, Koeleria cristata, Agropyron cristatum
HBGC	37.67°, 101.33°E; 3327m	−1.7	580	Alpine meadow soil, alpine scrubby meadow soil and swamp soil	Potentilla fruticosa L., Stipaaliena, Kobresia capillifolia, Kobresia humilis
DX	30°51′N, 91.07°E; 4333 m	−10.4	476. 8	Meadow soil with sandy loam	Blysmus sinocompressus, K. microglochin, K. littledalei K. parva K. humilis and Stipa purpurea
YC	36.96°N, 116.64°E; 28 m	13.1	610	Alluvial deposit of the Yellow River	Winter wheat and summer maize

Table 1. Site descriptions (Changbaishan temperate mixed forest (CBS), Qianyanzhou evergreen needle leaf forest (QYZ), Dinghushan evergreen mixed forest (DHS), Xishuangbanna evergreen broadleaf forest (XSBN), Inner Mongolia temperate steppe (NM), Haibei alpine shrub-meadow (HBGC), Damxung alpine steppe-meadow (DX) and Yucheng cropland (YC)).

Sites	Location and Altitude	Mean Annual Temperature (°C)	Mean Annual Precipitation (mm)	Soil Type	Predominant Species
CBS	42.40°N, 128.10°E; 738 m	3.6	713	Upland dark brown forest soil	Pinus koraiensis, Tilia amurensis, Quercus mongolica and Fraxinus mandshurica
QYZ	26.73°N, 115.05°E; 102 m	17.9	1542	Typical red earth	Pinus elliottii Engelm, Pinus massoniana Lamb and Cunninghamia lanceolate Hook
DHS	23.15°N, 112.50°E; 300 m	20.9	1956	Lateritic red-earth, yellow-earth and mountain shrubby-meadow soil	Schima superba, Castanopsis chinensis, Pinus massoniana
XSBN	21.90°N, 101.27°E; 756 m	21.8	1493	Lateritic and red lateratic soil	Pometia tomentosa, Terminalia myriocarpa
NM	44.13°N, 116.30°E; 1189 m	0.4	350.9	Chernozem soil	Leymus chinensis, Stipagrandis, Koeleria cristata, Agropyron cristatum
HBGC	37.67°, 101.33°E; 3327m	−1.7	580	Alpine meadow soil, alpine scrubby meadow soil and swamp soil	Potentilla fruticosa L., Stipaaliena, Kobresia capillifolia, Kobresia humilis
DX	30°51′N, 91.07°E; 4333 m	−10.4	476. 8	Meadow soil with sandy loam	Blysmus sinocompressus, K. microglochin, K. littledalei K. parva K. humilis and Stipa purpurea
YC	36.96°N, 116.64°E; 28 m	13.1	610	Alluvial deposit of the Yellow River	Winter wheat and summer maize

2.2. Flux Data

Carbon and water fluxes were measured with eddy covariance systems, which consist of an open-path CO₂/H₂O gas analyzer (model LI-7500, Licor Inc., Lincoln, NE, USA) and a 3D sonic anemometer/thermometer (model CSAT3, Campbell Scientific Inc., Logan, UT, USA) [23]. These flux data were originally sampled at 10 Hz, and the values averaged over 30 minutes were obtained from the flux stations. At these eight sites, GPP and ET were aggregated to an eight-day step from half-hourly CO₂ and H₂O flux data measured by open-path eddy covariance techniques. Outlier data were rejected, and unreasonable nighttime values were excluded according to friction velocity. Gaps in half-hourly CO₂ and water flux data were filled using a nonlinear regression algorithm [24]. GPP was estimated from the observed Net Ecosystem Exchange (NEE) and total ecosystem respiration (RE), which is the sum of nighttime and daytime ecosystem respiration. The daytime ecosystem respiration was calculated from the relationship between nighttime ecosystem respiration and soil temperature and moisture. Detailed steps in processing eddy covariance datasets were described in Zhang et al. [25] and Yu et al. [10].

2.3. MODIS Data

We used the MODIS GPP (MOD17) and ET (MOD16) datasets obtained from the University of Oklahoma (http://www.eomf.ou.edu). MODIS GPP was calculated based on the light use efficiency model using satellite-derived land cover and fractional photosynthetically-active radiation (FPAR) [26].

MODIS ET was produced by the Penman-Monteith (PM) equation (Equation (1)) driven by daily meteorology (i.e., temperature, actual water vapor pressure, relative humidity and downward shortwave radiation) and remotely-sensed albedo, leaf area index (LAI) and FPAR [27].

$$ET=\frac{\text{s}\times \text{A}+\text{ρ}\times {\text{C}}_{\text{p}}\times \left({\text{e}}_{\text{sat}}-\text{e}\right)/{\text{r}}_{\text{a}}}{\text{s}+\text{γ}\times \left(1+{\text{r}}_{\text{s}}/{\text{r}}_{\text{a}}\right)}$$

(1)

where s is the slope of the curve relating saturated water vapor pressure (e_sat) to temperature; A is available energy partitioned between sensible heat, latent heat and soil heat fluxes on the land surface; ρ is air density; C_p is the specific heat capacity of air; γ is the psychrometric constant; r_a is the aerodynamic resistance; r_s is the surface resistance; e is the actual water vapor pressure; and the difference between e_sat and e is the water vapor deficit (VPD).

MODIS eight-day GPP and ET data in 3 × 3 homogenous pixels (3 × 3 km) around the center of eight ChinaFLUX flux towers were extracted from and MOD17 and MOD16 products from January 2003–December 2008.

2.4. LAI Data

Observed LAI data were collected at three forest sites (i.e., CBS, DHS and XSBN) in 2005 and 2010 and at one cropland site (YC) during 2005–2011 from the Chinese Ecosystem Research Network (http://www.cerndata.ac.cn). LAI was measured by a leaf area scanner or plant canopy analyzer monthly every five years at forest sites, and several times each year according to the development stages for crops at the cropland site.

2.5. Analysis

We compared seasonal and annual WUE derived from MODIS data (WUE_MODIS) with eddy flux data (WUE_EC) at eight ChinaFLUX sites. In this study, WUE was defined as the ratio between GPP and ET (Equation (2)).

$$\text{WUE}=\text{GPP}/\text{ET}$$

(2)

Three agreement measures, including mean bias, root mean square error (RMSE) and Pearson’s correlation coefficient (R), between MODIS- and flux-derived WUE were used to evaluate the performance of MODIS-derived WUE over different vegetation types in China. The mean bias, RMSE and R were computed as the following equations:

$$\text{bias}=\frac{{\displaystyle \sum }\left(\text{WUE}\_\text{MODIS}-\text{WUE}\_\text{EC}\right)}{\text{N}}$$

(3)

$$\text{RMSE}=\sqrt{\frac{{\displaystyle \sum }{\left(\text{WUE}\_\text{MODIS}-\text{WUE}\_\text{EC}\right)}^2}{\text{N}}}$$

(4)

$$\text{R}=\frac{{\displaystyle \sum }\left(\text{WUE}\_\text{EC}-\overline{\text{WUE}\_\text{EC}}\right)\left(\text{WUE}\_\text{MODIS}-\overline{\text{WUE}\_\text{MODIS}}\right)}{\left(\sqrt{{\displaystyle \sum }{\left(\text{WUE}\_\text{EC}-\overline{\text{WUE}\_\text{EC}}\right)}^2}\right)\left(\sqrt{{\displaystyle \sum }{\left(\text{WUE}\_\text{MODIS}-\overline{\text{WUE}\_\text{MODIS}}\right)}^2}\right)}$$

(5)

2.6. Sensitivity Analysis

We applied the one-at-a-time sensitivity method [28] to compute the sensitivity of annual GPP, ET and WUE simulated by the MODIS GPP and ET algorithms to input daily meteorological variables, remotely-sensed variables and model parameters (Table 2) at the eight ChinaFLUX sites. We computed GPP, ET and WUE using the MODIS algorithms in the year 2005, given that each of the input variables or model parameters increases or decreases 10%, 20% and 30%, while the other parameters are fixed. The model run (run_+10% or run_−10%) had +10% or −10% variations for a chosen variable or parameter and the reference run (run_base) used default values. The sensitivity of modeled variables to the selected parameter or driving variables was calculated as the following equation [29,30]:

$$\text{S}=\text{max}\left(\frac{\left|{\text{run}}_{+10\text{\%}}-{\text{run}}_{\text{base}}\right|}{{\text{run}}_{\text{base}}},\frac{\left|{\text{run}}_{-10\text{\%}}-{\text{run}}_{\text{base}}\right|}{{\text{run}}_{\text{base}}}\right)\times 100$$

(6)

Table 2. Input driving variables and model parameters in the MODIS gross primary production (GPP) and evapotranspiration (ET) algorithms.

ID	Symbol	Definition	Units			Used for GPP or ET
Driving Variable
1	Tmin	minimum air temperature	oC			GPP, ET
2	Tmean	mean air temperature	oC			ET
3	Tday	daytime air temperature	oC			ET
4	Tnight	nighttime air temperature	oC			ET
5	FPAR	fraction of absorbed photosynthetically-active radiation	-			GPP, ET
6	RH	relative humidity	%			ET
7	VPD	water vapor deficit	Pa			GPP, ET
8	LAI	leaf area index	m2·m−2			ET
9	α	albedo	-			ET
10	Rs↓	downward shortwave radiation	J·m−2·d−1			GPP, ET
Model Parameter
11	LUEmax	maximum light use efficiency	kg·C·m−2·d−1·MJ−1			GPP
12	Tmin_close	threshold value below which the stomatal close completely	oC			GPP, ET
13	Tmin_open	threshold value upon which there will be no temperate stress	oC			GPP, ET
14	VPD_open	threshold value below which the stomatal close completely	Pa			GPP, ET
15	VPD_close	the threshold value upon which there will be no water stress	Pa			GPP, ET
16	gl_sh	leaf conductance to sensible heat per unit LAI	m·s−1			ET
17	gl_e_wv	leaf conductance to evaporated water vapor per unit LAI	m·s−1			ET
18	CL	mean potential stomatal conductance per unit leaf area	m·s−1			ET
19	rbl_min	minimum value of totalaerodynamic resistance to vapor transport	s·m−1			ET
20	rbl_max	maximum value of total aerodynamic resistance to vapor transport	s·m−1			ET

Table 2. Input driving variables and model parameters in the MODIS gross primary production (GPP) and evapotranspiration (ET) algorithms.

ID	Symbol	Definition	Units			Used for GPP or ET
Driving Variable
1	Tmin	minimum air temperature	oC			GPP, ET
2	Tmean	mean air temperature	oC			ET
3	Tday	daytime air temperature	oC			ET
4	Tnight	nighttime air temperature	oC			ET
5	FPAR	fraction of absorbed photosynthetically-active radiation	-			GPP, ET
6	RH	relative humidity	%			ET
7	VPD	water vapor deficit	Pa			GPP, ET
8	LAI	leaf area index	m2·m−2			ET
9	α	albedo	-			ET
10	Rs↓	downward shortwave radiation	J·m−2·d−1			GPP, ET
Model Parameter
11	LUEmax	maximum light use efficiency	kg·C·m−2·d−1·MJ−1			GPP
12	Tmin_close	threshold value below which the stomatal close completely	oC			GPP, ET
13	Tmin_open	threshold value upon which there will be no temperate stress	oC			GPP, ET
14	VPD_open	threshold value below which the stomatal close completely	Pa			GPP, ET
15	VPD_close	the threshold value upon which there will be no water stress	Pa			GPP, ET
16	gl_sh	leaf conductance to sensible heat per unit LAI	m·s−1			ET
17	gl_e_wv	leaf conductance to evaporated water vapor per unit LAI	m·s−1			ET
18	CL	mean potential stomatal conductance per unit leaf area	m·s−1			ET
19	rbl_min	minimum value of totalaerodynamic resistance to vapor transport	s·m−1			ET
20	rbl_max	maximum value of total aerodynamic resistance to vapor transport	s·m−1			ET

MODIS eight-day data of albedo, LAI and FPAR in 3 × 3 homogenous pixels (3 × 3 km) around the center of eight flux towers were downloaded from the University of Oklahoma (http://www.eomf.ou.edu). To remove the effect of cloud cover, we used the harmonic analysis of time series (HANTS) algorithm to obtain the temporal interpolation of LAI and FPAR at a daily time step. Gap-filled half-hourly temperature, relative humidity, VPD and incoming solar radiation measured at the flux towers were aggregated to a daily step.

3. Results

3.1. Evaluation of Seasonal MODIS-Derived WUE

Figure 1 shows the seasonal dynamics of WUE calculated based on eight-day GPP and ET estimated from MODIS data and eddy covariance data. Due to WUE having a similar seasonal pattern over different years, we calculated the annual mean values and standard deviations of eight-day WUE. For the deciduous forest (CBS) and grassland ecosystems, we only compared WUE in the growing season, because GPP in the non-growing season was assumed to be zero. WUE presents different seasonal patterns among eight ecosystems. Compared to WUE estimated from eddy covariance observations, WUE derived from MODIS GPP and ET product has different behaviors at these eight sites (Table 3). It underestimates WUE in four forest ecosystems and one cropland ecosystem with a bias from −0.36–−2.28 g·C·kg⁻¹ H₂O, while overestimating WUE in three grassland ecosystems with the bias from 0.26–1.11 g·C·kg⁻¹ H₂O. Among the eight ChinaFLUX sites, MODIS WUE shows the best performance in the Haibei alpine grassland ecosystem (HBGC) with the highest R of 0.96 and the low RMSE of 0.42 g·C·kg⁻¹ H₂O.

The MODIS product has a better performance in estimating WUE in the CBS temperate mixed forest and the DHS subtropical evergreen forest than the QYZ subtropical evergreen forest and the XSBN tropical evergreen forest. It has a higher R and lower RMSE for CBS (R = 0.85, RMSE= 0.97 g·C·kg⁻¹ H₂O) and DHS (R = 0.71, RMSE = 0.45 g·C·kg⁻¹ H₂O) than for QYZ (R = 0.64, RMSE = 1.18 g·C·kg⁻¹ H₂O). There is no significant linear relationship between MODIS- and flux-derived WUE in the XSBN tropical evergreen forest. The RMSE of WUE at XSBN is up to 2.53 g·C·kg⁻¹ H₂O. Although MODIS WUE has a good correlation with flux-derived WUE in CBS, it underestimated the high water use efficiency during DOY 145–257 by 39% on average. The bias of WUE in summer (from June–August) is −1.62 g·C·kg⁻¹ H₂O in CBS. The underestimation of WUE mainly resulted from the lower estimates in MODIS GPP in CBS in comparison to mean annual values observed in 2003–2008 (Figure 2). Systematic underestimates in MODIS WUE were found for QYZ with a bias of −1.13 g·C·kg⁻¹ H₂O and at XSBN with a bias of −2.28 g·C·kg⁻¹ H₂O, which are basically caused by undervalued GPP and high ET estimates (Figure 2). For example, in the summer season (from June–August) for XSBN, GPP was underestimated by 27%–36% and ET was overestimated by 62%–127%, which led to the underestimated WUE by 53%–74%. In the DHS subtropical evergreen forest, MODIS GPP and ET were higher than those derived from eddy covariance measurements by 55% and 77% (Figure 2) and, thus, led to an underestimate of WUE by 16% on average.

Figure 1. Comparison of MODIS- and eddy covariance-derived eight-day Water Use Efficiency (WUE) at eight sites of the Chinese Flux Observation and Research Network (ChinaFLUX). (a) CBS; (b) DHS; (c) QYZ; (d) XSBN; (e) YC; (f) HBGC; (g) NM; (h) DX.

Figure 1. Comparison of MODIS- and eddy covariance-derived eight-day Water Use Efficiency (WUE) at eight sites of the Chinese Flux Observation and Research Network (ChinaFLUX). (a) CBS; (b) DHS; (c) QYZ; (d) XSBN; (e) YC; (f) HBGC; (g) NM; (h) DX.

Table 3. Relationships (y = A + B × X) of eight-day WUE between MODIS (y) and eddy covariance observation (x) with correlation coefficient (R), significance (p), root mean square error (RMSE) and bias.

Site	A	B	R	p	RMSE (g·C·kg−1 H2O)	Bias (g·C·kg−1 H2O)
CBS	0.25	0.58	0.85	<0.0001	0.97	−0.45
DHS	0.46	0.63	0.71	<0.0001	0.45	−0.36
QYZ	0.43	0.45	0.64	<0.0001	1.18	−1.13
XSBN	2.13	−0.01	−0.03	0.83	2.53	−2.28
YC	1.04	0.25	0.52	0.0002	1.70	−0.51
HBGC	0.10	1.30	0.96	<0.0001	0.42	0.26
NM	0.45	2.88	0.80	<0.0001	1.63	1.11
DX	0.00	1.87	0.82	<0.0001	0.34	0.14

Table 3. Relationships (y = A + B × X) of eight-day WUE between MODIS (y) and eddy covariance observation (x) with correlation coefficient (R), significance (p), root mean square error (RMSE) and bias.

Site	A	B	R	p	RMSE (g·C·kg−1 H2O)	Bias (g·C·kg−1 H2O)
CBS	0.25	0.58	0.85	<0.0001	0.97	−0.45
DHS	0.46	0.63	0.71	<0.0001	0.45	−0.36
QYZ	0.43	0.45	0.64	<0.0001	1.18	−1.13
XSBN	2.13	−0.01	−0.03	0.83	2.53	−2.28
YC	1.04	0.25	0.52	0.0002	1.70	−0.51
HBGC	0.10	1.30	0.96	<0.0001	0.42	0.26
NM	0.45	2.88	0.80	<0.0001	1.63	1.11
DX	0.00	1.87	0.82	<0.0001	0.34	0.14

MODIS WUE had a worse performance at the NM temperate grassland site than the other two alpine grassland sites. The bias of WUE derived from MODIS data at the NM temperate grassland site is 4.3- and 7.7-times larger than that at the HBGC alpine grassland site and the DX alpine grassland site, respectively. MODIS WUE also has a higher RMSE for NM (1.63 g·C·kg⁻¹ H₂O) than HBGC (0.42 g·C·kg⁻¹ H₂O) and DX (0.34 g·C·kg⁻¹ H₂O). The overestimation of WUE in the three grassland ecosystems was due to different reasons. The significant high MODIS WUE for NM was generated by overvalued GPP by 57% and underestimated ET by 40% in the growing season (Figure 2). For HBGC, MODIS GPP was rather consistent with the observed one; however, ET was underestimated, and that caused the overvalued WUE in summer (Figure 2). In contrast, the overestimation of WUE for DX was mainly derived from the remarkable underestimation of ET in the growing season, with a bias of 47%, on average (Figure 2).

Figure 2. Comparison of MODIS- and eddy covariance-derived eight-day gross primary production (GPP) and evapotranspiration (ET) at eight ChinaFLUX sites. (a) CBS; (b) DHS; (c) QYZ; (d) XSBN; (e) YC; (f) HBGC; (g) NM; (h) DX.

Figure 2. Comparison of MODIS- and eddy covariance-derived eight-day gross primary production (GPP) and evapotranspiration (ET) at eight ChinaFLUX sites. (a) CBS; (b) DHS; (c) QYZ; (d) XSBN; (e) YC; (f) HBGC; (g) NM; (h) DX.

The MODIS product has a significant low estimate of WUE for summer maize at the YC cropland site, compared to the observed WUE values estimated from eddy covariance systems. The bias of WUE is up to −2.16 g·C·kg⁻¹ H₂O during the growing season of summer maize (from June–September). Both undervalued GPP and overvalued ET contributed to the big negative bias. The GPP of summer maize was 35%–66% undervalued, which played a more important role in the underestimation of WUE. Although the seasonal pattern of MODIS WUE was more consistent with the eddy covariance measurements for winter wheat than summer maize, we cannot ignore the big bias of GPP (28%–69%) and ET (33%–70%) for winter wheat (Figure 2).

As important input remotely-sensed data in the algorithm of the MODIS ET product, MODIS LAI data are used to upscale the stomatal conductance at the leaf level to canopy conductance [31], the quality of which directly influences MODIS ET data. Therefore, we further compared the difference of LAI between MODIS LAI and observed one at three forest sites (i.e., CBS, DHS, and XSBN) and one cropland site, as shown in Figure 3 and Figure 4. Due to missing and unreliable eight-day MODIS LAI data, these had been filled before using them to produce the MODIS ET dataset; here, we excluded eight-day MODIS LAI data with bad quality control values. For the CBS temperate deciduous forest, the MODIS product underestimated LAI in spring and autumn by 57%–67% in 2005 and 53%–66% in 2010 and overestimated the maximum value of LAI by 20% in 2005 and 19% in 2010. For the DHS subtropical evergreen forest, LAI was 56% and 22% overvalued by MODIS in 2005 and 2010. For the XSBN tropical evergreen forest, MODIS overestimated LAI by 21% in 2005 and 9% in 2010, on average. For the YC cropland ecosystem, LAI in growing seasons was significantly underestimated by the MODIS product (Figure 4). The mean bias of the LAI of winter wheat and summer maize was −1.47 and −0.74 m²·m⁻²in 2005–2011. Since ET has a good linear relationship with LAI [21], high ET in forest and low ET in cropland were supposed to be associated with the overestimation of LAI for forest and the underestimation of LAI for cropland.

Figure 3. Comparison of MODIS and observed leaf area index (LAI) at the (a,b) Changbaishan (CBS), (c,d) Dinghushan (DHS) and (e,f) Xishuangbanna (XSBN) forest sites.

Figure 3. Comparison of MODIS and observed leaf area index (LAI) at the (a,b) Changbaishan (CBS), (c,d) Dinghushan (DHS) and (e,f) Xishuangbanna (XSBN) forest sites.

Figure 4. Comparison of MODIS and observed leaf area index (LAI) at the Yucheng (YC) cropland site.

Figure 4. Comparison of MODIS and observed leaf area index (LAI) at the Yucheng (YC) cropland site.

3.2. Evaluation of Annual MODIS-Derived WUE

Annual WUE derived from eddy flux data varied with ecosystems, which ranged from 0.37 ± 0.08–4.32 ± 0.63 g·C·kg⁻¹ H₂O across the eight flux sites (Figure 5c). Generally, forest ecosystems had higher annual WUE than grassland ecosystems. Compared to the observed WUE, MODIS WUE had a smaller variation across the eight ecosystems, with a range of 0.34 ± 0.05–1.98 ± 0.07 g·C·kg⁻¹ H₂O. Mean annual WUE was underestimated by 14%–54% at four forest sites, 45% at one cropland site and 7% at DX alpine grassland site, but overestimated by 66% and 9% at the NM temperate grassland site and the HBGC alpine grassland site, respectively. The underestimation of annual WUE in CBS, QYZ and XSBN forests was caused by low annual GPP and high annual ET (Figure 5a,b). In the DHS subtropical forest, annual WUE was undervalued, because there was a larger overestimation in annual ET (72%) than annual GPP (48%). In contrast, the low annual WUE at the YC cropland site mainly resulted from the underestimation of annual GPP (54%).

Figure 5. Comparison of MODIS- and eddy covariance-derived annual (a) Gross Primary Production (GPP), (b) Evapotranspiration (ET) and (c) Water Use Efficiency (WUE) at eight ChinaFLUX sites.

Figure 5. Comparison of MODIS- and eddy covariance-derived annual (a) Gross Primary Production (GPP), (b) Evapotranspiration (ET) and (c) Water Use Efficiency (WUE) at eight ChinaFLUX sites.

MODIS WUE also had a smaller interannual variation during 2003–2008 across the eight ecosystems, in comparison to the observed WUE. WUE derived from eddy flux data had a small interannual variability at the seven ecosystems with coefficients of variation (CVs) of 6%–22%, except for the NM temperate grassland ecosystem with a CV of 46%. In contrast, the CVs of annual WUE from MODIS data at the eight ecosystems varied from 3%–14%.

3.3. Sensitivities of MODIS GPP, ET and WUE to Driving Variables and Model Parameters

The sensitivities of GPP, ET and WUE almost linearly varied when each of the selected driving variables or model parameters was changed by 10%, 20% and 30%, while other parameters are fixed. Therefore, we only presented the result of sensitivity analysis when we changed the individual driving variable or model parameter by 10%, as illustrated in Figure 6. Overall, annual WUE derived from MODIS GPP and ET was most sensitive to LUE_max and FPAR at all selected sites. Increasing LUE_max by 10% led to a 10% increase in both annual GPP and WUE due to the linear relationship between GPP and LUE_max. The ten percent increase in FPAR resulted in an increase of 4.6%–10.2% in WUE at the eight sites, except for the YC cropland site, where FPAR had a similar effect on both GPP and ET. Besides, RH had a great effect of 5.9%–14.2% on the estimation of WUE at the YC cropland site and the three grassland sites, especially for the DX grassland site. Incoming shortwave radiation (R_s↓) greatly affected GPP and ET with similar sensitivities, so it had little effect on the estimation of WUE.

Figure 6. Sensitivities of annual GPP, ET and WUE to driving variables and model parameters with their ID numbers listed in Table 1. (a) CBS; (b) DHS; (c) QYZ; (d) XSBN; (e) YC; (f) HBGC; (g) NM; (h) DX.

Figure 6. Sensitivities of annual GPP, ET and WUE to driving variables and model parameters with their ID numbers listed in Table 1. (a) CBS; (b) DHS; (c) QYZ; (d) XSBN; (e) YC; (f) HBGC; (g) NM; (h) DX.

Considering the big differences in LAI between MODIS and observed data at the DHS, XSBN and YC sites, we further conducted the sensitivity analysis by decreasing LAI at DSH and XSBN and increasing LAI at YC to examine how the modeled ET and WUE respond to those changes. The results show that the decrease in LAI by 50% for DHS and XSBN led to a reduction of 0.2% in ET and an increase of 0.2% in WUE, as well as a 200% increase in LAI for YC only caused by a 0.4% increase in ET and a 0.4% decrease in WUE. It indicated that the large error in MODIS LAI has little effect on the estimation of ET and WUE due to its small sensitivity.

Compared to the original MODIS ET product in 2005, the relative errors of ET in modeled subtropical and tropical forests by the MODIS ET algorithm and site-level meteorological data were reduced from 56% down to 6% for DHS, from 91% down to 48% at QYZ and from 128% down to 98% for XSBN (Table 4).

Table 4. Comparison between flux-derived ET (ET_EC), the original MODIS-derived ET (ET_MODIS_o) and modeled ET using the MODIS ET algorithm and meteorological data measured in subtropical and tropical forests (ET_MODIS_m).

Site	ET_EC	ET_MODISo	ET_MODISm
DHS	675	1051	715
QYZ	510	976	755
XSBN	564	1287	1117

Unit: kg·H₂O·m⁻²·yr⁻¹.

Table 4. Comparison between flux-derived ET (ET_EC), the original MODIS-derived ET (ET_MODIS_o) and modeled ET using the MODIS ET algorithm and meteorological data measured in subtropical and tropical forests (ET_MODIS_m).

Site	ET_EC	ET_MODISo	ET_MODISm
DHS	675	1051	715
QYZ	510	976	755
XSBN	564	1287	1117

Unit: kg·H₂O·m⁻²·yr⁻¹.

Therefore, we inferred that the overestimation of ET and accordingly the underestimation of WUE for QYZ, DHS and XSBN were partly caused by the bias in incoming shortwave radiation (R_s↓), relative humidity and VPD. The underestimation of GPP for CBS, QYZ, XSBN and YC may result from undervalued LUE_max for forest and cropland, which also contributed to the underestimation of WUE. Further investigation of the comparison in meteorological variables between the Data Assimilation Office dataset used by the MODIS algorithms and local observations is necessary to identify the dominant error contributors.

4. Discussion

This study evaluated eight-day and annual WUE calculated from MODIS GPP and ET products in China using eddy covariance-measured GPP and ET at eight ChinaFLUX sites. Both MODIS- and eddy covariance-derived WUE values were within the range of previous studies (0.36–6.07 g·C·kg⁻¹ H₂O) [9,32]. They also show higher values in forest ecosystems than grassland ecosystems, which were consistent with previous studies of the variation of WUE across different ecosystem types [7,8,33]. Because we defined WUE as the ratio of GPP and ET, the performance of MODIS WUE depends on the accuracy of both GPP and ET estimation. Previous studies of the evaluation of MODIS GPP and ET products reported that MODIS GPP generally underestimates GPP and MODIS ET overestimates ET using eddy flux measurement from 2003–2005 at the same eight ChinaFLUX sites [17,21]. This indicated that WUE tends to be underestimated by MODIS data when compared to WUE calculated from eddy flux measurements. That is the case for the four forests and one cropland site, as presented in this study. However, for the three grassland ecosystems, MODIS WUE values are significant overvalued, mainly due to underestimated ET.

The evaluation of the MODIS global ET product based on FLUXNET data showed that monthly ET is overestimated in forest with a mean bias error of 6 mm per month and underestimated in grassland and cropland with mean bias errors of −8 and −10 mm per month, respectively [18]. Many studies on the validation of MODIS LAI also demonstrated that the MODIS LAI product often overestimated LAI in forest, but underestimated it in grassland and cropland ecosystems [34,35,36,37,38,39]. However, the bias in LAI might not influence the estimation of ET and WUE much, because neither ET nor WUE is sensitive to the change in LAI.

Similar to the results in this study and in [21] based on the eddy flux data from ChinaFLUX, an assessment of MODIS GPP against eddy covariance measured GPP at 10 flux towers in northern China pointed out that MODIS underestimated GPP at nine sites, especially for all cropland sites [16]. MODIS GPP was also undervalued in an evergreen coniferous forest in Britain [40]. However, the MODIS product was found to have the tendency to generally overestimate GPP for low productivity and to underestimate it for high productivity in North America, compared to ecosystem-level measurements of GPP at eddy covariance flux towers from AmeriFlux and Fluxnet-Canada [15,41,42]. Moreover, MODIS GPP overestimates GPP in a deciduous forest in Korea [43], three subtropical forest ecosystems [41] and non-forest areas in North America [44]. The underestimation in GPP was associated with the low estimation of light use efficiency, especially for cropland [15,21,45,46]. On the contrary, the overestimation in GPP might result from high FPAR [15]. Meteorological data also have a significant contribution to the uncertainty in MODIS GPP [22]. Besides, some bias of GPP may be derived from an imperfect model structure in the current MODIS GPP algorithm, which did not consider the contribution of shade leaf to canopy photosynthesis [41].

Although the eddy flux measurement has a larger footprint compared to the field measurements at certain points [47], it remains uncertain due to random measurement errors [48], gap-filling errors [49], the surface energy balance disclosure issue [50,51] and the variation in the footprint [52]. Besides, remote sensing data and eddy flux data have different observation frequencies, so there are unavoidable errors in estimating daily GPP and ET from both transient remote sensing data and half-hourly eddy flux data. For example, random measurement errors in CO₂ and water fluxes at the eight ChinaFLUX sites were 0.29–5.0 μmol·m⁻²·s^–1 and 9.2–49.8 W·m⁻², respectively. The average daily missing rates of half-hourly GPP and ET data after quality control were 53%–68% and 39%–59%. The energy balance ratios ranged from 0.58–1.00 with an average of 0.87 over the eight ChinaFLUX sites [53], which caused the true ET to be underestimated by the eddy covariance measurements to some degree. Their contributions to the uncertainty in estimated WUE at both eight-day and annual scales still need to be further quantified.

5. Conclusions

We evaluated eight-day and annual WUE values estimated from MODIS GPP and ET products against eddy covariance measurements of GPP and ET during 2003–2008 at eight ChinaFLUX sites. MODIS products underestimated WUE in four forest ecosystems and one cropland ecosystem, while overestimating it in three grassland ecosystems. At the four forest sites, MODIS WUE was underestimated because of undervalued GPP and overvalued ET. In contrast, MODIS WUE was significantly overvalued mainly due to underestimated ET in the three grassland ecosystems. Sensitivity analysis shows that annual WUE derived from MODIS GPP and ET was most sensitive to the maximum light use efficiency (LUE_max) and FPAR. The underestimation of GPP at forest and cropland sites may result from undervalued LUE_max. The overestimation of ET and, accordingly, underestimation of WUE at three subtropical and tropical forests were partly derived from the bias of meteorological data used in the MODIS algorithms. The bias in LAI may not influence the estimation of ET and WUE much, due to its small sensitivity. Uncertainties in eddy flux data due to gap-filling processes and the unbalanced surface energy issue also accounted for the bias of MODIS-produced WUE. Further improvements of the algorithms of the MODIS products (including LAI/FPAR, GPP and ET) and the driving meteorological data, as well as deeper investigation to identify the contributions of the main error sources to the uncertainty in estimated WUE are required in the future.