Estimating Global Wheat Yields at 4 km Resolution during 1982–2020 by a Spatiotemporal Transferable Method

Abstract

Reliable and spatially explicit information on global crop yield has paramount implications for food security and agricultural sustainability. However, most previous yield estimates are either coarse-resolution in both space and time or are based on limited studied areas. Here, we developed a transferable approach to estimate 4 km global wheat yields and provide the related product from 1982 to 2020 (GlobalWheatYield4km). A spectra–phenology integration method was firstly proposed to identify spatial distributions of spring and winter wheat, followed by choosing the optimal yield prediction model at 4 km grid scale, with openly accessible data, including subnational-level census data covering ~11,000 political units. Finally, the optimal models were transferred at both spatial and temporal scales to obtain a consistent yield dataset product. The results showed that GlobalWheatYield4km captured 82% of yield variations with an RMSE of 619.8 kg/ha, indicating good temporal consistency (r and nRMSE ranging from 0.4 to 0.8 and 13.7% to 37.9%) with the observed yields across all subnational regions covering 40 years. In addition, our dataset generally had a higher accuracy (R² = 0.71) as compared with the Spatial Production Allocation Model (SPAM) (R² = 0.49). The method proposed for the global yield estimate would be applicable to other crops and other areas during other years, and our GlobalWheatYield4km dataset will play important roles in agro-ecosystem modeling and climate impact and adaptation assessment over larger spatial extents.

1. Introduction

Approximately 800 million people worldwide suffered from undernourishment in 2020 [1]. Sustainable Development Goal (SDG) 2 is dedicated to eradicating hunger and all forms of malnutrition by 2030 and achieving food security [2]. However, the goal of eliminating hunger might remain elusive even by 2050 due to climate variability, extreme weather events, and global crises such as the COVID-19 pandemic and the current Russia–Ukraine war [3]. Climate change is projected to force an additional 72 million people to face hunger risks in 2050, and the COVID-19 pandemic is estimated to have led to 83–132 million more undernourished people in 2020 [3,4]. In these contexts, global food production needs to increase by at least 70% to feed the unprecedented population growth of up to 10 billion by 2050 [5,6]. To better inform a series of agricultural resource allocation and food security decisions, timely and accurate information on crop yield at a global scale is of paramount significance [7,8,9,10].

There are two mainstream methods of crop yield prediction, that is, process-based crop models and statistical methods [11,12,13,14,15,16]. Crop models can dynamically simulate crop physiological processes, including development, growth, and grain formation processes [17,18,19,20,21]. Despite incorporating diverse physiological mechanisms, crop models are highly dependent on substantial data inputs and massive computations [22]. On the other hand, statistical models often relate crop yields to diverse predictor variables (e.g., vegetation indices and climatic variables) and calibrate the empirical relationships based on measurements [23]. The superiority of statistical models lies in their simplicity and reduced requirement for extensive inputs; however, they are particularly vulnerable to co-linearity problems and noise of inputs [24]. Fortunately, machine learning (ML) provides an innovative alternative to statistical modeling and can address the nonlinear relationships between the predictor variables and crop yield, demonstrating superior performance in many applications [25,26,27,28]. For instance, Kang et al. (2020) proved that more advanced ML models achieved better accuracy in estimating county-level maize yield [29]. Emerging breakthroughs in algorithms such as deep learning (DL) approaches have accomplished more accurate crop yield estimation [30,31]. For example, the long short-term memory (LSTM) model adopts a recurrent neural network structure that can recognize sequential information for long time periods and capture sophisticated nonlinear relationships. Jiang et al. (2020) found that LSTM outperformed the RF model in estimating county-level corn yields in the United States [32]. The superior performance of LSTM over two ML approaches was further proved during the prediction of wheat yield in the Guanzhong Plain by Tian et al. (2021) (e.g., support vector machines) [33].

Previous studies using ML and DL methods focused on very limited areas rather than global scales. It is well recognized that a global spatially explicit crop yield dataset has important implications for large-scale agricultural system modeling and climate change impact assessments [34,35,36]. Although a few studies have filled such data gaps, there is still room for significant development. For example, a 10 km global dataset of harvested area and yield was firstly generated for 175 crops circa 2000 [37], followed by the Global Agro-ecological Zones (GAEZ) datasets in 2000 and 2010 [38], Spatial Production Allocation Model (SPAM) at 5-arcmin resolution for three years (2000, 2005, and 2010) [39], and the latest data proposed by Grogan et al. (2022) with a resolution of 5 min for 2015 [40]. However, these four publicly accessible products only cover 1~3 years, hampering related studies on investigating the long-term impacts of climate change on yields [41,42]. Iizumi et al. (2020) developed a global dataset of historical yields (GDHY) for staple crops at a spatial resolution of 0.5° by integrating agricultural census data and remote sensing [43]. GDHY covers a longer period, but its spatial resolution is relatively coarse. Moreover, these yield datasets were established based on crop distribution maps generated by a downscaling method rather than accurate satellite-derived maps, which might result in misestimated yield and inaccurate assessments of climate change impacts [44]. Therefore, it is urgent to acquire the global gridded yield dataset with a higher resolution and a longer time span based on the accurate spatial distribution of harvesting areas.

In this study, by integrating multi-source data (e.g., remote sensing, climate, soil data, and subnational-level census data) and data-driven methods, we aim to (1) propose a transferable method to accurately estimate global crop yield; (2) compare the performance of two ML and DL models in predicting gridded yields; and (3) choose the optimal models to generate global wheat yield datasets. The resultant dataset with 4 km spatial resolution will benefit the investigation of spatiotemporal patterns of crop production, assessment of climate change impacts and modeling of crop growth processes over large spatial extents.

2. Materials and Methods

2.1. Study Area

The study area contains the main wheat-planting countries (54) in the world, covering ~92% of the total harvested area and ~93% of the total production (Figure 1) [45]. We compiled the abundant subnational-level census data in the countries, with diverse climatic conditions and cropping systems. Winter wheat dominates the majority (>75%) of global wheat harvesting area, while areas of spring wheat <25% (primarily in the Northern Hemisphere high latitude areas such as the United States, the Russian Federation, and Canada) [46,47].

2.2. Data

2.2.1. Remote Sensing Data

We acquired the global daily 0.05° Normalized Difference Vegetation Index (NDVI) data during 1981–2021 derived from the Advanced Very High-Resolution Radiometer (AVHRR) sensor on the Google Earth Engine (GEE) platform. The data were generated using eight NOAA polar orbiting satellites (i.e., NOAA-7, -9, -11, -14, -16, -17, -18, and -19) and VIIRS for two time periods before and after 2014. The main strength of AVHRR NDVI lies in its longest time coverage, which can be used to derive predictors for yield prediction [48]. In addition, the 8 d composite Global Land Surface Satellite (GLASS) Leaf Area Index (LAI) at 1 km spatial resolution from 2005 to 2015 was used to capture phenological information on different crops, and the Global Food Security-support Analysis Data (GFSAD) 1 km Crop Mask product (GFSAD1KCM) was utilized as a cropland mask. The GLASS LAI was retrieved using general regression neural networks with multiple inputs (http://glass-product.bnu.edu.cn/?pid=3&c=1, accessed on 19 June 2024), with the specific advantages of being spatiotemporally continuous without gaps and having higher accuracy than other datasets [49,50]. GFSAD1KCM provides the global cropland extent for the nominal year 2010 and is produced based on four inputs with the highest accuracy of 85% [51]. Moreover, the annual dataset of the 1 km wheat harvesting area (named ChinaCropArea1km) in China during 2000–2015 was used [52].

2.2.2. Wheat Harvesting Area and Yield

The subnational-level census data on harvested area (unit: ha), production (unit: ton), and yield (unit: kg/ha) were collected from ~11,000 administrative units in the 54 countries, with the longest time coverage spanning from 1981 to 2020. Yield is calculated as production divided by harvested area. Overall, 97% of data came from administrative unit level 2 (ADM2) and 3 (ADM3). For the European Union, the data were collected at NUTS-2 level. The temporal coverage differs across the study area (Table S1). We eliminated outliers of census data with values +/−2 standard deviations from the average.

2.2.3. Environmental Data

Meteorological information was obtained from high-spatial resolution (1/24°, ~4 km) monthly TerraClimate datasets [53]. The climate variables used for this analysis were maximum temperature (Tmin), minimum temperatures (Tmax), precipitation (Pre), vapor pressure (Vap), vapor pressure deficit (Vpd), reference evapotranspiration (Petref), soil moisture (Soil), Palmer drought severity index (Pdsi), and downward surface shortwave radiation (Srad) from 1981 to 2021. In addition, soil properties were derived from the Harmonized World Soil Database (HWSD) at 0.00833° (~1 km), involving the bulk density, organic carbon content, pH, gravel, clay, and sand and silt fraction for the topsoil (0–30 cm) [54] (see

Table 1. Summarization for information on each country across the study area.

Data Type	Data Product Name	Spatial Resolution	Temporal Resolution	Purposes	Reference
Satellite data	NOAA CDR AVHRR NDVI	0.05°	1981–2021	Extracting predictor variable NDVI	[48]
	GLASS LAI	1 km	2005–2015	Identifying phenological characteristics of wheat	http://glass-product.bnu.edu.cn/?pid=3&c=1, accessed on 19 June 2024
	GFSAD1KCM	1 km	2010	Deriving cropland mask	[51]
Wheat harvesting area and yield	Agricultural census data	-	1981–2020	Training and validating yield estimation model	See Table S1
	ChinaCropArea1km	1 km	2000–2015	Extracting wheat growing areas in China	[52]
Environmental data	TerraClimate	4 km	1981–2021	Extracting predictor variables including Tmin, Tmax, Pre, Vap, Vpd, Pet, Soil, Pdsi, Srad	[53]
	HWSD	0.00833°	-	Extracting predictor variables including bulk density, organic carbon, pH, gravel, clay, sand and silt fraction	[54]

).

2.3. Methods

We applied the Global Wheat Production Mapping System (GWPMS) framework developed by Luo et al. (2022) with two aspects of improvement (Figure 2) [44]. We conducted the study according to the following: (1) mapping the harvesting area of spring and winter wheat by a spectra–phenology integration algorithm; (2) comparing the performances of two ML and DL approaches in predicting gridded yield, (3) generating the GlobalWheatYield4km dataset using the optimal model, and (4) evaluating the accuracy and uncertainty of the dataset.

2.3.1. Identifying the Spatial Distribution of Wheat

Phenology information plays a paramount role in large-scale crop mapping [52,55,56]. The phenological dates of winter wheat are earlier than summer crops and spring wheat and are later than some winter crops such as winter barley [44,57]. For example, the sowing date of winter wheat often occurred in autumn, while that of summer crops and spring wheat was concentrated in spring, leading to earlier timing of heading and maturity. In addition, the duration of the growth period of winter wheat is generally longer. Spring wheat can also be differentiated from other summer crops as its phenological phases occur earlier. Therefore, we developed a wheat detection algorithm that formalized these features in rules to automatically detect the harvest areas of spring and winter wheat [44]. Here, we modified the algorithm and mapped the spatial distribution of wheat as following steps.

First, we compared the cropland map derived from the GFSAD1KCM with census data to determine whether to use it as a cropland mask; that is, the mask was utilized only when the GFSAD1KCM-derived areas matched with (or were larger than) census data.

Second, we used the Savitzky–Golay (S-G) filter method to remove the noise from the GLASS LAI composites for each pixel. This method has shown good performance for smoothing time series [58,59,60].

Lastly, we utilized the algorithm to detect annual spatial distribution of spring and winter wheat during 2006–2014. Generally, three specific characteristics should be recognized concurrently, namely, the green-up of winter wheat or emergence of spring wheat, heading, and crop senescence. Specifically, the green-up/emergence of winter/spring wheat was regarded as an inflection point after which the first derivative increased for three adjacent images. The heading date was identified as the maximum point within a restricted time window obtained from existing work regarding wheat phenology, with the LAI value exceeding a certain threshold (LAI_max). Moreover, crop senescence signal was detected as a sharp decline in the LAI (LAI decreased by more than LAI_dec%) during a 40-day time period. In addition, we modified the algorithm when applying it to some regions where winter wheat was not a dominant crop (e.g., Mexico, Bolivia, and Peru) or grown in rotation with other crops (e.g., Argentina and Brazil in South America, India and Pakistan in Southeast Asia, and China). For example, the rule for the heading and senescence phase was loosened or even eliminated when the signal was weak due to the mixed pixel issues or the short duration of the interval between the maturity date of winter wheat and the planting date of the second crop (Table S2).

2.3.2. Estimating Gridded Yield Using Data-Driven Models

We first compared the predictive performance of two commonly used ML and DL approaches, i.e., the Random Forest (RF) and LSTM models. The RF model combines a set of decision trees that are constructed from a random subset of data [61]. Each tree is trained separately on these samples, and the remaining data are called out of bag (OOB) samples and can be used to validate the RF model. In this study, we used the Python scikit-learn library to develop the RF regression model. The number of decision trees (n_estimators), the minimum number of samples required to be at a leaf node (min_samples_leaf), and the number of features (max_features) were selected for tuning. The LSTM network consists of a framework of a recurrent neural network (RNN) and memory gate structure, demonstrating superior performance in coping with sequential data and capturing the nonlinear and cumulative relationships between crop yield and meteorological factors [32,62]. The model consists of an input layer, one or more LSTM layers, and an output layer. The LSTM layers are composed of LSTM cells, in which information is forgotten or outputted decided by three gates. Batch normalization was firstly implemented for all the input data. The transient data (i.e., NDVI and climate data) were dealt with via two LSTM layers that have 200 hidden units, whereas the non-sequential data (i.e., soil properties) were appended to the final LSTM layer and then fully connected to the output layer. In addition, a rectified linear unit (ReLU) activation function was used for all the layers. The model was run for 2000 maximum iterations with a mini-batch size of 500, and RMSprop was used to optimize hyperparameters with a learning rate of 0.001. The LSTM network for estimating gridded yield was performed on TensorFlow (GPU version 2.0). Keras, a deep learning library, was applied for developing the LSTM model.

Here, we first resampled the gridded input data (i.e., NDVI, climate, and soil data) into 4 km and unified NDVI and climate data into monthly time steps by the maximum value synthesis and monthly mean method, individually. The time series of monthly NDVI composites were further gap-filled by a moving median method [63], which replaced the missing data with the median composite of three adjacent values (i.e., preceding, current, and subsequent values). Then, we derived an integrated wheat map to represent reliable spatial distribution over a long-term period on the basis of the grids with cultivation for at least 5 years during 2006–2014. Finally, all input data were averaged on the subnational scale after being masked by wheat cultivation pixels. These processes were performed on the Google Earth Engine (GEE) platform.

We implemented the “leave-one-year-out” method to examine performance of the ML and DL models, that is, one-year data were used for testing and the data of the remaining years for training. More specifically, each model was first trained separately by excluding one year in the data. For instance, the temporal extent of county-level statistics for the United States was from 1982 to 2016. If two models were trained with the data for the years 1982–1999 and 2001–2016, then they were validated using the data of 2000. The best hyperparameters were determined with the ten-fold cross-validated coefficient of determination (R²). Then, the optimized models were used to estimate gridded yield for the excluded year. Finally, the resultant yields were aggregated to the corresponding ADM level and were compared with census data for the excluded year. The R² and root mean square error (RMSE) were calculated to validate estimates’ accuracy. The whole process was repeated 20 times, and the mean R² and RMSE were used to compare the performance of the two data-driven models. Note that the evaluation metric of the RF model performance was the R² and RMSE of the OOB validation (i.e., OOB R² and RMSE).

The RF and LSTM models were generally constructed for each country; however, their predicted performance was poor in some countries (e.g., Kazakhstan) due to the limited statistical data. To improve the accuracy of the yield dataset and lengthen its time coverage, we first combined the census data of some countries together to train the model and spatially transferred it to estimate gridded yields. For example, we only collected observed yields of Kazakhstan for the years 2014–2020. Since the growing season of spring wheat was identical in the Russian Federation and Kazakhstan, their data were integrated to feed into the model, and the yield maps were ultimately generated from 1995 to 2020. Similar treatment was conducted for all European countries, as well as Afghanistan and Iran (Table S3). In addition, we applied the pre-trained model to other years where observed yields are unavailable, aiming at generating a spatiotemporally continuous yield dataset.

2.3.3. Uncertainty Analysis

To provide the uncertainty of GlobalWheatYield4km, we spatialized the normalized RMSE (nRMSE) to depict the spatial patterns of uncertainty. More specifically, we first calculated the nRMSE of the yield between the GlobalWheatYield4km-derived estimates and the observed data in each subnational unit. Then, the nRMSE value was allocated to the centroid of each subnational unit, and the kriging interpolation method was used to map the spatial distribution of uncertainty, which was masked by wheat cultivation pixels.

2.3.4. Comparison with Other Global Yield Datasets

We compared our gridded yield estimates with a prevalent product (i.e., SPAM) using census data to demonstrate the reliability of our dataset. These two datasets could be directly compared as they were both generated using census data. More specifically, we calculated the R² and RMSE between the observed yield and the estimates of SPAM or GlobalWheatYield4km in 2000, 2005, and 2010. Since the crop yield of SPAM was the nominal value for three adjacent years centered on 2000, 2005, and 2010, the averages of observed yields in the corresponding years (e.g., the averages of 1999, 2000, and 2001 match SPAM 2000) were used.

3. Results

3.1. Assessing Accuracy of Wheat Distribution Maps

To illustrate the reliability of the wheat distribution maps, we validated them with the subnational-level area. The estimated areas generally matched well with the observed area, with an R² ranging from 0.65 to 0.89 (average: 0.8) and nRMSE ranging from 31.4% to 54.7% (average: 41.1%) (Figure 3, Table S4). The mapped areas were overestimated in the Russian Federation, Kazakhstan, Australia, Canada, and the United States, while they were underestimated in South America. A possible reason for the overestimation could be the difficulty of distinguishing spring wheat from other spring cereals such as spring barley because of their similar phenology. In addition, the wheat distribution maps showed the lowest accuracy in South America, with an R² ranging from 0.65 to 0.82 and nRMSE ranging from 38.3% to 48.3%, which was ascribed to the mixed pixels and the larger uncertainties from remote sensing products. Overall, the comparisons showed the high consistency between the resultant maps and the census data, demonstrating that the derived maps were reliable for further yield prediction.

Besides the above comparison of annual planting areas in the globe, we further selected the U.S. as an example to compare national spatial distribution in detail because of the popular CDL (Cropland Data Layer) products (Figure S1). The spatial distribution of our model matched well with CDL, with an R² (RMSE) of 0.82 (6.2 Kha) and 0.81 (21.4 Kha) for winter and spring wheat, respectively. The higher accuracy (R² > 0.80) together with lower RMSE further indicates the reliability of our maps derived from the spectra–phenology integration method.

3.2. Selecting the Optimal Model of Wheat Yield Estimates

The performance of the RF and LSTM models in gridded yield prediction during 2006–2014 for each region/country is shown in Figure 4. Generally, the LSTM model outperformed the RF model, with an average R² (nRMSE) of 0.72 (13.1%) and 0.64 (16.2%), respectively. More specifically, LSTM achieved the highest accuracy in the United States, Europe, China, India, and Pakistan (R² > 0.8, nRMSE < 20%) while the RF model showed comparable performance (R² of 0.7~0.82, nRMSE of 21%~29%), which was ascribed to the abundant training samples (>3000) (Table S3). However, compared to the better predictive performance of winter wheat in China and the United States, both the LSTM and RF model could capture less than 62% of yield variations in Brazil despite the sufficient statistical data. One possible reason was the many mixed pixels in Brazil with highly heterogeneous land cover types, consequently resulting in errors in aggregated predictor variables. The other was the larger uncertainties from remote sensing products, especially more frequent cloud contamination in Brazil. In addition, the RF model showed similar performance in Nepal (R² = 0.69, nRMSE = 20%) as compared with LSTM (R² = 0.68, nRMSE = 19.5%). A possible reason was that the training samples for Nepal were scarce, and the spatial variability of predictor variables and yield was relatively lower. Moreover, the LSTM models improved (decreased) R² (nRMSE) by around 15% as compared with the RF model, especially in the Russian Federation, Ukraine, Bangladesh, Japan, Brazil, Peru, and Bolivia, with more improvements in R² (nRMSE) ranging from 14% to 50%. The superior performance of LSTM was attributed to its powerful temporal learning capabilities that can capture nonlinear and cumulative relationships between yield and meteorological factors over long time periods.

Therefore, the optimal LSTM model was implemented to predict global wheat yield at the grid scale. The out-of-sample performance was evaluated over the subnational level, and the time period is same as that of observed yields (Table S1). More specifically, the model was recursively trained using all data after leaving one year for testing, and the gridded-yield estimates were aggregated to the subnational level and validated by the remaining year. Overall, the predicted yield agreed well with the census data as they were closely and consistently distributed around the 1:1 line, with an R² of 0.56~0.86, RMSE of 123.2~911.3 kg/ha, and nRMSE of 13.8~33.8% (Figure 5 and Figure S2). The overall R² of GlobalWheatYield4km was 0.82 across all subnational regions and years, with the RMSE and nRMSE values of 619.8 kg/ha and 23.5%, respectively. The highest R² was found in Bangladesh (R² = 0.86, nRMSE = 14.9%) and Europe (R² = 0.86, nRMSE = 17.3%), followed by China, Chile, Pakistan, India, Canada, and the United States (R² of 0.77~0.82). By contrast, the lowest R² was found in Japan (R² = 0.56, nRMSE = 20.6%), Afghanistan, and Iran (R² = 0.58, nRMSE = 33.8%), which might be caused by the lower wheat cultivation or insufficient observed yields.

The spatial distributions of GlobalWheatYield4km were consistent with the observed yields in 2010 (Figure 6), with a large variability from 130 to 11,546 kg/ha. We further summarized the gridded yield by country. The average yields were the highest in Europe (e.g., Belgium: 8457 kg/ha; Netherlands: 8011 kg/ha), followed by Chile (5201 kg/ha) and China (4658 kg/ha). By contrast, Kazakhstan, Bangladesh, and Bolivia were indicated to have the lowest average yield (<1000 kg/ha). However, the spatial comparisons between predictions and statistical records indicated many blank values located in the prediction map, especially for two provinces (Newfoundland and Labrador) in Canada and most areas in Northern and Eastern Russia. We uncovered the missing data caused by their relatively sparce planting areas (Figure 6a) since our baseline map was derived by the criterion above 50% identification rate (at least 4 years) when identifying annual area dynamics during the period of 2006~2014.

Besides the spatial distribution assessment above, we further assessed annual variation accuracy. The correlation coefficients between the time series of predicted and the observed yields for each subnational-level unit were summarized separately by main wheat planting regions in the world (Figure S3). We simultaneously found good temporal consistencies between the time series of GlobalWheatYield4km and observed yield. The average correlation coefficients (r values) ranged from 0.4 to 0.8, implying a strong temporal learning ability of our GWPMS method. More interestingly, GlobalWheatYield4km clearly indicated the impacts on yield from extreme events (Figure S5). For example, a drought that occurred in 2006 hit wheat production in Australia, and the predicted yields distinctly showed yield reductions in all regions across Australia, suggesting our GlobalWheatYield4km captures the spatiotemporal variability of wheat yields well at the subnational scale.

The nRMSE values in most areas (88.9% of grids) were below 30%, suggesting a higher accuracy. However, 2.9% of grids were indicated to have a higher nRMSE > 40%. Moreover, the regions with higher uncertainty are mainly located in Southern India, Western Afghanistan and Iran, Southern South America, Northeastern China, and Central Mexico, possibly due to the sparse distributions of wheat or short period of census data available there (Figure 7). Similarly, the lower errors, indicated by the nRMSE averages ranging from 13.7% to 37.9%, further evidence a strong temporal learning ability of our GWPMS method (Figure S5).

3.3. Comparing GlobalWheatYield4km with SPAM

We aggregated gridded-yield estimates of GlobalWheatYield4km in 2000, 2005, and 2010 and the average of SPAM for three adjacent years to administrative units, and then compared them with census yields, respectively. Overall, the yield estimates of GlobalWheatYield4km showed higher consistencies with census yields as they were closer to the 1:1 line than SPAM, with an average R² (RMSE) of 0.84 (670.2 kg/ha) and 0.7 (932.3 kg/ha), respectively (Figure 8). In addition, GlobalWheatYield4km exhibited higher and more robust accuracies than SPAM in all three years and regions (Figure S6 and Table S5). The R² (RMSE) of GlobalWheatYield4km was improved (reduced) by an average of 42.2% (22.6%) as compared with SPAM, especially in Argentina, Australia, Iran, Pakistan, and the United States (improvements over 23% for R² and RMSE). We ascribed such improvement to the consequent high-quality input data at more consistent and finer resolution. In contrast, the methodology and input data of SPAM were improved stepwise.

4. Discussion

4.1. Advantages of GlobalWheatYield4km

GlobalWheatYield4km outperforms other global crop yield products (e.g., SPAM, GAEZ) with the following advantages: (a) the highest spatial resolution (4 km) among all yield datasets presently available; (b) higher accuracy as compared with SPAM; (c) clear subdivision of spring and winter wheat; and (d) clear characterization of the spatiotemporal dynamics of global wheat yields over 40 years. We compared two typical ML and DL models that are commonly used for yield prediction, determined the optimal model, and transferred them to generate gridded-yield estimates, which could improve the accuracy of our dataset. We found that LSTM consistently outperformed the RF model regardless of year and region, which was also confirmed by many previous studies [30,33,44,64]. The LSTM model, characterized by its recurrent neural network structure, has been widely proved to capture successfully cumulative and complex nonlinear relationships between crop yields and climatic factors [31,32,65,66].

Due to census data being unavailable, especially for the long term, it was very hard for us to collect finer-scale census data in some countries such as Kazakhstan and Afghanistan as well as Africa and Central Asia. Consequently, even compiling the detailed census data from the largest subnational units (~11,000), some data gaps are still left in some areas, similarly to the inconsistency among temporal periods and spatial distributions. Interestingly, the spatiotemporal transferable method we developed makes up for the above limitations by accurately estimating yields to some degree. Accordingly, we transferred the optimal model spatially and temporally to other areas and years with unavailable census data (Table S3) and provided the global wheat yield data with the highest resolution of 4 km and covering 40 years.

4.2. Uncertainties

Despite the higher accuracy of GlobalWheatYield4km, there were still some limitations. First, the largest constraint was the uncertainties of remote sensing data. For example, cloud and snow contaminations could cause noise in GLASS LAI products and consequently dampen the wheat detection signal [50]. The other uncertainty was from GFSAD1KCM, with a coarser resolution than GFSAD30 m FSAD products in accurately capturing the spatial distributions of cropland with respect to medium and small agriculture field sizes in some regions such as South Asia [67]. Algorithms do impact prediction accuracy, as evidenced by the comparison between the RF and LSTM models (Figure 4). With the rapid development in algorithms, many state-of-the-art deep learning techniques have been proposed. Among these, the attention model is one such hotspot proven to be more efficient especially for inference or prediction than LSTM [68,69]. Nevertheless, we should note that the accuracy of predictions is depended strongly on the quantity and quality of observed samples (Figure 4 and Figure 7).

Moreover, the spatial resolution of 1 km could result in mixed pixel issues, potentially lowering our dataset accuracy, especially in areas where wheat was sparsely cultivated (e.g., South America). Nevertheless, two approaches can offset these impacts to some degree. First, the cultivation pattern is complicated in the main wheat-planting areas (e.g., the North China Plain in China, Saskatchewan in Canada, North Dakota in the United States, and Northern India). These areas are cultivated with complicated patterns in small fields generally behaving like “large fields” and consequently weakening the impacts of mixed pixel issues [52]. In addition, to avoid the misclassification of pixels, we integrated the annual 1 km map during 2006–2014 to generate a baseline map with the grids with wheat-planting areas for at least 4 years, which consequently purified pixels to some degree. Therefore, it should be noted that potential users should mask our products with explicitly annual wheat planting maps to obtain accurate yield data, especially for the annual growing areas changing significantly over time. We will try to map the spatial distribution of wheat using remote sensing images with finer spatial resolutions in the future and further improve yield estimates [70,71].

5. Conclusions

Here, we proposed a spatiotemporal transferable method to estimate global wheat yield at a 4 km resolution over four decades using data-driven models (available at https://doi.org/10.6084/m9.figshare.10025006). We firstly identified the spatial distribution of wheat harvesting areas using a spectra–phenology integration method then developed ML and DL models and transferred the optimal models to estimate yield at the grid scale. The distribution map showed a high accuracy, with an R² of 0.8. The optimal models (LSTM, R² = 0.72, nRMSE = 13.1%) were transferred into global wheat planting areas across 40 years and obtained the first global yield estimate product with a higher resolution. We comprehensively assessed the accuracy at both spatial and temporal scales. Our GlobalWheatYield4km indicates high spatial consistency with observed yields, with an R² (RMSE) of 0.82 (619.8 kg/ha). Good temporal consistency is further evidenced by r values (0.4~0.8) and nRMSE (13.7~37.9%) across all subnational regions covering 40 years. As compared with another public product, GlobalWheatYield4km showed 45% higher accuracy (R²~0.71) than SPAM (R²~0.49). The resultant dataset can benefit many studies, including agricultural system modeling and climate change impact assessments over larger regions.