Comparing the Ability of Burned Area Products to Detect Crop Residue Burning in China

: Burning crop residues is a common way to remove them during the final stages of crop ripening in China. To conduct research effectively, it is critical to reliably and quantitatively estimate the extent and location of a burned area. Here, we investigated three publicly available burned area products—MCD64A1, FireCCI 5.1, and the Copernicus Burnt Area—and evaluated their relative performance at estimating total burned areas for cropland regions in China between 2015 and 2019. We compared these burned area products at a fine spatial and temporal scale using a grid system comprised of three-dimensional cells spanning both space and time. In general, the Copernicus Burnt Area product detected the largest annual average burned area (37,095.1 km 2 ), followed by MCD64A1 (21,631.4 km 2 ) and FireCCI 5.1 (12,547.99 km 2 ). The Copernicus Burnt Area product showed a consistent pattern of monthly burned areas during the study period, whereas MCD64A1 and FireCCI 5.1 showed frequent changes in monthly burned area peaks. The greatest spatial differences between all three products occurred in Northeast and North China, where cultivated land is concentrated. The burned area detected by Copernicus in Xinjiang Province was larger than that detected by the other two products. In conclusion, we found that all three products underestimated the amount of crop residues present in a burned area. This limits the ability of end users to understand fire-related impacts and burned area characteristics, and hinders them in making an informed choice of which product is most appropriate for their application.


Introduction
Biomass burning is an important driver of climate change, and impacts both the carbon cycle and plant succession processes. It is essential to accurately constrain a fire's location and extent in order to conduct reliable fire-related research, and several methods have been developed to detect and monitor fires using remote sensing. Several multiannual burned areas have been developed worldwide since the late 1990s and are freely available for study [1]. These were developed for different purposes, occur at different scales, and offer different spatial resolutions, such that they vary considerably in distribution, size, and frequency of mapped fires [2]. Significant discrepancies in areal estimates and spatial patterns of fire damage have also been observed among them [3].
To support the case for a systematic investigation of the differences among burned area products, numerous studies have validated the products using independent reference data, such as ground-truthing data [4,5] or higher spatial resolution images that have minimal errors (e.g., Landsat or Sentinel) [6][7][8][9][10]; however, such efforts are expensive and time consuming. Comparison between burned area products is therefore a necessary step and practical tool for characterizing each product according to their performance, especially when field-based validation data are not available [11]. This should be recognized as a complementary assessment to product validation. Furthermore, since no product can perfectly represent burned area characteristics, as all have limitations, the choice of which product to use should consider the advantages and disadvantages of each. A key consideration in such a choice is a study's research objective and intended application of any acquired data, making the regional performance of each product an important criterion. Pessôa et al. [11] compared four fire products in the Amazon region and analyzed their individual impacts on a forest's carbon emissions. A similar study was performed by Valencia et al. [12], where five forested burned area products were compared with each other. However, there is still remarkable uncertainty within the community regarding the worldwide burned area extent in cropland areas. Currently, there are very few studies that have provided a global or regional comparison of burned area products for crop residues that have burned areas smaller than 100 ha, which results in large detection errors due to the relatively coarse spatial resolution of these data [13]. Therefore, additional analysis of crop residue burning is needed to provide new insights into the relative performance of burned area products.
Fires are often used as a tool to eliminate crop residues during the final stages of crop ripening in China, as there are no alternative methods for treating them [14]. Knowledge about the current state of fire dynamics in China relies heavily on satellite-derived analyses of burned areas. Nevertheless, very few studies have compared the consistency of available satellite products detailing the location and extent of burned areas in this region. Given the importance of this assessment to improve fire products in China, it is critical to evaluate the relative performances of the most-used global burned area products in cropland regions, which can provide clear insight into their limitations. Here, we performed a comparison of three global burned area products by considering the total cropland burned area detected in China from 2015 to 2019. The specific objectives of this study were as follows: (i) evaluate the differences and similarities among the MCD64A1, FireCCI 5.1, and the Copernicus Burnt Area products, specifically regarding the total burned area detected over cropland regions; (ii) compare the temporal differences and similarities between the products, with MCD64A1 used as a reference; and (iii) compare the spatial differences and similarities between the products, with MCD64A1 used as a reference.

Study Area
China is one of the largest agricultural nations in the world and produces more agricultural crop residues than any other country [15]. Regions of China that are used largely for agricultural purposes include plains situated in Northeast and North China, and the Sichuan Basin in Southwest China [16]. Slash-and-burn cultivation and crop residue burning are traditional agricultural practices [17], and a large amount of crop residue is burned in China [14] during the summer/autumn harvest season in order to eliminate agricultural straw [16]. Northeast and North China each have two main fire seasons: spring (February to May, with a peak in March) and autumn (late October to early November) in the former, and summer (late May to early June) and autumn (September to October) in the latter. The main characteristics of burning crop residues in China are: (i) a large number of fire spots that are widely distributed; (ii) random occurrence times; and (iii) small burned areas [18]. These characteristics make it difficult to accurately detect all of the residues that occur in a burned area with coarse-resolution satellite data, leading to large errors in the detection results. This study considered the entire land surface of China (Figure 1).

Global Burned Area Products
Most satellite-derived global fire data sets fall into two broad categories: burned area products and active fire products [19]. Burned area products demarcate the location of a burned area, its spatial extent, and its approximate data of occurrence, which are all needed to assess crop residue burning activities. In this study, three global burned area products were selected for analysis and comparison: the Copernicus Burnt Area product, FireCCI 5.1 burned area product, and MODIS MCD64A1 burned area product. Brief descriptions of these products are provided in Table 1 and their algorithms are described below. April 2014 to December 2019 represents a common period of time in which these products have data available; therefore, the five-year period between 2015 and 2019 was considered in this study.
The MCD64A1 based on a hybrid approach is generated from the active-fire and MODIS cloud-free surface reflectance imagery. First, a daily burn-sensitive vegetation index (VI) derived from MODIS time series using the red, near-infrared and short-wave infrared channels. Then, the algorithm calculates descriptive statistics based on the VI data time series and defines a measure of temporal separability through dynamic thresholds, yielding composites images of these parameters. Finally, cumulative active fire maps are used to generate regional probability density functions for the classification of burned and unburned training samples that will guide the final determination of burned and unburned pixels [20,21]. The main inputs to derive FireCCI 5.1 product based on a two-phase approach are the daily MODIS near infrared reflectance and thermal anomalies data. Firstly, only hotspots that have a high probability of being burned are selected as seed pixels. Then, a contextual growing algorithm is applied from the seed pixels to reduce omission errors by detecting the whole burned patch. Both phases use detection thresholds better adapted to local fuel and fire conditions [22].
The algorithm of Copernicus is based on a breakpoint analysis that identifies sudden spectral changes related to burns on temporal series of reflectance observations. Active fires location and timing are used as seeds in a spatio-temporal regional growing algorithm. The region (burn patch) grows through the detections of sudden burn-like spectral changes, across the spatial and temporal domains. It uses fixed thresholds to identifies the sudden spectral changes [23]. Land cover information was extracted from operational products made available through the Copernicus Land Monitoring Service. Corine Land Cover (CLC) versions with a 100-m pixel resolution and spanning the 2015-2019 study period was used to obtain cropland distribution information.

Pre-Processing of Burned Area Products
The HDF files of MCD64A1 were converted into GeoTIFF and reprojected from sinusoidal to geographic coordinates to calculate the burned areas. This product provides the approximate burn date, burn date uncertainty, quality assurance (QA), first day and last day layers. The data type of FireCCI 5.1 used in this study is pixel-version with 250 m spatial resolution. FireCCI 5.1 provides a monthly GeoTIFF dataset with three layers, estimated first day of burn (Julian-date), confidence level, and land cover type of a detected burn pixel. All the pixels of MCD64A1 and FireCCI 5.1 regardless the confidence level was used for this analysis. Copernicus has two 10-day products. One contains the pixel that have been flagged as being burned during the 10-day period. The second contains a file value indicating the date of detection expressed as a Julian date for those pixels that were detected as being burned in the 10-day period. These 10-day products were composited into monthly dataset in order to be consistent with the other two products. Before the three products were calculated with land cover data to obtain the cropland distribution area, all these three products were resampled to 100 m to keep their spatial resolution consistent with the land cover product.

Comparison of Burned Area Products
This study compared products by using a grid of fine-resolution temporal and spatial data comprised of voxels [1], which were defined spatially by Landsat Thiessen scene areas (TSAs) and temporally defined by calendar month. These TSA polygons have been used as one of the validation bases for global burned area products in previous studies [1, 6,27]. Together, the entire land surface of China is covered by 531 TSAs and the entire cropland surface of China is covered by 447 TSAs. Here, the cropland area refers to that present in 2019.
Comparisons among products were made with respect to MCD64A1, as it is the product most commonly used to monitor burned areas at regional and global scales [8], and was also the most recently updated product at the time of analysis [1]. Thus, burned area differences between products were calculated as MCD64A1 minus FireCCI 5.1 and MCD64A1 minus Copernicus.
Our methodology involved six steps performed at both temporal and spatial scales. At the temporal scale, analyses were conducted in three steps: (1) comparing the total burned areas on both annual and monthly scales for each product; (2) analyzing the consistency between FireCCI 5.1, Copernicus, and MCD64A1 data, which involved using monthly burned area data to fit linear regressions and calculate goodness-of-fit values in a least-squares model (R 2 ); and (3) comparing the monthly burned area differences between FireCCI 5.1, Copernicus, and MCD64A1. At the spatial scale, analyses were conducted in three steps: (1) comparing the spatial distribution of burned area difference in each TSA; (2) fit linear regressions and consider goodness-of-fit values in a least-squares model (R 2 ) between FireCCI 5.1, Copernicus, and MCD64A1, which was achieved by using a TSA as a sample to analyze spatial consistency; and (3) comparing the ability of all three products by using typical TSAs as examples.

Total Burned Area
Annual total burned areas in cropland regions in China detected by each global burned area product are reported in Figure 2 2 in each year of the study period, respectively. In general, Copernicus detected the largest annual totals, except with data for 2015, when the MCD64A1 product detected the largest. Overall, Copernicus detected the largest amount of burned area throughout the study period, exceeding the totals detected by MCD64A1 and FireCCI 5.1 by approximately 71.5% and 195.6%, respectively. These results are consistent with the study by Pessôa et al. [11], which assessed differences among burned area products in a forested region of the Amazon and reported that MCD64A1 detected a larger burned area than FireCCI 5.1, and that Copernicus detected a larger burned area in Asia than MCD64 did in 2019 [28]. Older versions of Copernicus and MCD64A1 (i.e., L3JRC and MCD45A1) gave the same results, whereby MCD45A1 burned areas were relatively larger than areas calculated using L3JRC [4]. However, in a global study that did not distinguish land cover type [1], results showed that Copernicus detected the smallest total burned area when compared to MCD64A1 and FireCCI 5.1, which shows the opposite pattern to our results.
The biggest discrepancy between MCD64A1 and FireCCI 5.1 occurred in data for 2017, where the former product detected 20,591.94 km 2 more burned area than the latter, while the least burned area difference between these two products occurred in 2016 data, where the former product detected 113.81 km 2 more burned area than the latter. When MCD64A1 and Copernicus are compared, the greatest difference in burned area also occurred in 2019, with a disparity of 77,318.51 km 2 , while the smallest difference in burned area occurred in 2015, where MCD64A1 detected 2144.3 km 2 more than Copernicus.  Curves showing MCD64A1 and FireCCI 5.1 data show stronger fluctuations than those of Copernicus, they have significant interannual variability, and their peak values are lower than those of Copernicus. There is a significant discrepancy between all three products during the burning season. During the Northeast China burning season (March to April), Copernicus detected more burned areas than MCD64A1 and FireCCI 5.1, while in the North Plain China burning season (September to October), the burned areas identified by these three products are comparable in size.    , with relatively little underestimation. However, the correlation between Copernicus and MCD64A1 burned areas is poorer (R 2 = 0.151) and has a slight bias (slope = 1.0295), showing that the relationship between these two products is not significant and that the former makes an overestimation compared with the latter. It is notable that there are few anomalous points that lie away from the regression line between FireCCI 5.1 and MCD64A1, although many points lie away from the regression line between Copernicus and MCD64A1. This indicates that the consistency between Copernicus and MCD64A1 monthly burned data is low. Furthermore, as the values of these abnormal points are very high, an analyst must be very cautious when using these data to estimate fire-related emissions. To further illustrate the variation in monthly burned area detected by all three products, absolute differences were determined by subtracting FireCCI 5.1 data from MCD64A1 data, and subtracting Copernicus data from MCD64A1 data, and their temporal variation patterns were plotted ( Figure 5).  According to Figure 5, burned area differences determined between FireCCI 5.1 and MCD64A1, and between Copernicus and MCD64A1 have very different temporal patterns. The peak values of difference changes between FireCCI 5.1 and MCD64A1 are irregular and show a degree of randomness; however, the peak values of negative differences between Copernicus and MCD64A1 systematically appear in May in every year of the study period, and positive peak values appear in March in every year. The negative peaks in May can be explained by the summer fire season in the North China Plain, while the positive peaks in March can be explained by the spring fire season in Northeast China. The largest difference of monthly burned area between MCD64A1 and FireCCI 5.1 occurred in April 2017, with the former product detecting 13,624.56 km 2 more burned area than the latter, while the smallest difference occurred in July 2019, with only 0.375 km 2 less burned area detected by former than the latter. For Copernicus and MCD64A1, the maximum difference was in April 2019, with the former detecting 23,454.02 km 2 more than the latter, and the minimum difference was in August 2017, with the latter detecting 8.06 km 2 less than the former.

Spatial Comparison
Spatial comparison involved considering each TSA as a unit and calculating the difference between their burned areas, as determined by FireCCI 5.1, Copernicus, and MCD64A1. These TSA spatial distribution patterns show the same characteristics from 2015 to 2019; as such, only the magnitude of the differences determined for 2016 are shown in Figure 6. Further, Figure 7 shows the results of taking each TSA as a sample, plotting burned area scatter plots between FireCCI 5.1, Copernicus, and MCD64A1 in the same year, and fitting a linear regression to the data.    Figure 4 that show changes through time, meaning that burned area calculations derived by FireCCI 5.1 and MCD64A1 are similar at both spatial and temporal scales. One reason for these similarities between MCD64A1 and FireCCI 5.1 may be that they are both MODIS-based products. However, the correlation between Copernicus and MCD64A1 burned areas is less clear (R 2 = 0.7694) and has a slight bias (slope = 2.3298), showing that the relationship between Copernicus and MCD64A1 is significant and the former makes overestimations compared with the latter. This result is very different from that shown in Figure 4, such that the discrepancy between Copernicus and MCD64A1 is more significant at the spatial scale than at the temporal scale. Thus, it is essential for users to understand these characteristics in order to make informed choices of which product is most appropriate for their intended spatial analysis and comparison.
Notably, there are several outliers that deviate far away from the regression line between FireCCI 5.1 and MCD64A1, meaning that there were some analyzed areas where the results obtained by these two products were very different. However, there are fewer outliers situated away from the regression line between Copernicus and MCD64A1; therefore, although the difference between these two products is large, the calculated values have the same spatial characteristics. The results shown in Figure 7 are opposite to the results shown in Figure 4, revealing that differences between FireCCI 5.1, Copernicus, and MCD64A1 are inconsistent in both space and time. In summary, when choosing a burned area product for analyzing fire emission studies and ecosystem management, the user must consider whether to focus on a spatial or a temporal characteristic. Figure 7 also shows that some of the outliers situated away from the best-fit curve represent the largest differences in burned area detected between FireCCI 5.1, Copernicus, and MCD64A1. To investigate this, we selected the TSAs that represent areas of MCD64A1 larger than 25.0 km 2 and those that represent areas of Copernicus larger than 80.0 km 2 . As these TSAs are all located in the Songnen Plain and Sanjiang Plain in Northeast China, we analyzed the similarities and differences of cropland burned areas detected by all three products by choosing a single TSA that has Landsat images coverage of both regions. Figures 8 and 9 illustrate the differences among product in two representative sample TSAs. The TSA shown in Figure 8 is located in Songnen Plain, where spring burning season is from March to April. The TSA shown in Figure 9 is located in Xinjiang Province, and shows evidence of a long fire season by the occurrence of small and fragmented scars.  As shown in Figure 8, the burned area presented by three products are shown in red patchs, which means the burned area products have been filtered to only show the burning before April 2nd. All three products underestimate the burned area and have frequent omission errors, with the FireCCI 5.1 product having the most. Meanwhile, commission errors are common; for example, the bottom parts of each image show large areas of nonburning cropland, although both MCD64A1 and Copernicus products identify these as burned areas.
As shown in Figure 9, the burned area obtained by Copernicus are shown with green symbol, which means the burned area product has been filtered to only show the burning before September 26th. Copernicus identified some bare farmland areas as burned areas, whereas MCD64A1 and FireCCI 5.1 both identified them as non-burning areas, which resulted in large commission errors.

Discussion
A systematic comparison of three global burned area products was performed in this study, and has identified spatial and temporal similarities and differences in cropland burned area detections in China from 2015 to 2019. On the whole, the Copernicus product identified the largest total burned area and the FireCCI 5.1 product detected the smallest total burned area. The MCD64A1 and FireCCI 5.1 products underestimated the burned areas in China's cropland region, whereas the Copernicus product overestimated the burned areas, especially in Xinjiang Province. The findings of this study match those of study [11], which reported that MCD64A1 detected a larger burned area than FireCCI 5.1 in an Amazonian forest region and study [29] which reported that MCD64A1 failed to identify cropland burned area in Eastern China where dominated by very large numbers of individually small fires. However, the result reported here differs from study [2], which reported that the FireCCI 5.1 product's annual burned area average in 2001-2018 was 11% higher than that for MCD64A1. Other research [4,7,30,31] has also pointed out that MCD64A1 underestimates the extent of burned areas in a region. Underestimation by the MCD64A1 and FireCCI 5.1 products may be associated with large unmapped areas in a studied region, which are often caused by cloud cover and missing data. By contrast, overestimation by the Copernicus product is mostly related to excessive detection of cropland burned areas during periods outside of fire seasons.
In general, differences in burned areas detected by different products may be caused by variations in burned area algorithms and spectral and spatial domains. For example, the MCD64A1 product uses information about hotspots detected by MODIS instrumentation, and combines a burn-sensitive vegetation index based on short-wave infrared (SWIR) channels with a measure of temporal texture [30]. The smallest burned area detected by FireCCI 5.1 may therefore be a function of the resolution of a single NIR reflectance band that does not consider SWIR, which is a very sensitive tool to discriminate burned areas. Second, the underestimation of FireCCI 5.1 areas may be due to (i) the absence of an unmapped class, which likely increased the possibility of unclear pixels being labeled as unburned [32], or (ii) a result of the short duration when residues were burning. All algorithms use a thermal infrared band to capture active fires in order to reduce omission errors, although these algorithms are incapable of identifying short-duration incidences of residue burning.
This study focused on comparing the spatial and temporal correlations between three products for detecting burning crop residues, but without analyzing their accuracy, as the accuracy of the products is directly affected by land cover type. The accuracy of the MCD64A1 product at a global scale, as reported by the developers, is that it has omission errors (OE) exceeding >35% and commission errors (CE) exceeding >23% [24]. By contrast, the FireCCI 5.1 product has a reported OE count of 54.5% and CE count of 25.7% [25]. However, the accuracy of these three products is also different for different types of land cover. Santana et al. [8] reported that Brazilian biomes containing savanna and grassy areas have an OE count of 64.05% and a CE count of 45.85%. Hall et al. [10] found that MCD64A1 and FireCCI 5.1 suffer from very high OE counts of 71-76% and 63-99%, respectively, and CE counts of 62-81% and 49-93%, respectively, in cropland regions. Chang et al. [4] noted a characteristic delay between burning in an area and fire-related emissions, which is significantly influenced by the combustion efficiency of different land cover types. Alonso-Canas et al. [31] showed that the signs of burning in an area can be removed quickly due to climatic conditions and the rate of vegetation regeneration depends on the time that has elapsed after the fire has been extinguished. Therefore, it is very important to specifically compare the burned areas of crop residues.
We have compared the capability of three major products used to detect crop residue burning events in abnormal TSAs in China, and report large different between them. In particular, we found that their OE data are consistently large, most likely due to the small and discrete nature of burned patch residues, although the level of agreement for different burned area products increases proportionally with the size of the fires [5]. Zhu et al. [33] reported that the ability for a product to detect a burned area changes according to vegetation type, and is particularly low in cropland regions that have small fires. Ramo et al. [9] showed that Sentinel-2 images can detect 80% more burned area than when using a MODIS product, with this difference interpreted to be predominately related to small fires (<100 ha). Campagnolo et al. [34] pointed out that the accuracy of fire products is lower than 36% for fires that are smaller than 250 ha. Moreover, the large omission of burned area detection is known to be related to the short duration of fires. Olive et al. [35] stated explicitly limitations associated with the traditional burned area product include long revisit time of the satellite data. McCarty et al. [36] reported that some burned area commission errors are likely due to the practice of tilling directly after burning and stated that active fire with higher temporal resolution are useful for this situation. Liu et al. [37] also concluded that the temporal resolutions of MODIS and Landsat are both too coarse to accurately classify cropland burned area because of the quick harvest to sowing turnover period in India. Whereas, study [33] used the sampling interval on temporal scale is less than 20 days for cropland burned area estimation in Eastern Europe, and considered this interval is the best attempt of this kind of study. These studies can fully demonstrate that the duration of crop residues burned area is a region-specific phenomenon. For example, the characteristics of cropland burning in different regions of China are quite different. In the Songnen Plain in this study, farmers do not plow immediately after burning because this region dominated by one year one ripe area and there is a longer time between burning and sowing next season crop. Whereas, in the South China, farmers need plowing immediately after burning in order to plant next season crops because the crops could be harvested for two times per year, or three times every two years here. Thus, it is reasonable to conclude that the studied global burned area products are based on coarse spatial resolution sensors (>250 m) and are unsuitable to detect small fires that burn only within a fraction of a pixel in satellite imagery. Additionally, the spatial and temporal resolutions of the data for cropland burned area estimation are region specific. It is thus important to develop a product to detect crop residue burned areas that is based on higher spatial resolution satellite data and high revisit frequency.

Conclusions and Perspectives
In this study, we systematically compared three global burned area products and their ability to identify burned areas of cropland in mainland China between 2015 and 2019. We focused on both spatial and temporal correlations between products. In that period, the Copernicus product detected the largest annual average burned area (37,095.1 km 2 ) in the study regions, followed by MCD64A1 and FireCCI 5.1, with 21,631.4 km 2 and 12,547.99 km 2 , respectively. At the temporal scale, interannual variation of Copernicus monthly burned areas showed a consistent pattern from 2015 to 2019, whereas the monthly burned area peaks according to MCD64A1 and FireCCI 5.1 often changed during this period. At the spatial scale, the primary differences between the three products occur in Xinjiang province, especially in Northeast and North China, where the cropland burned areas are most concentrated.
The comparison of burned area products in this study only considered their correlations in space and time. Whereas, the accuracy and associated uncertainty are much important for fire emission study, end users might want to use a range of burned area for their emission estimates (i.e., error bars) and not simply choose which burned area product best suits their needs. Further work is needed to evaluate the accuracy of the burned area products and validate the uncertainty associated with the products, especially as the same burned area may not be identified in some instances when analyses are performed at different spatial resolutions [12]. Whether a small fire can be detected successfully during such an investigation is an important factor affecting the accuracy of calculated burned area products; therefore, further research should be conducted to improve the ability to detect small fires. This can be readily achieved by developing new burned area algorithms that use medium-resolution sensors and consider the morphologies of fire patches, which themselves influence burned area distributions.
Author Contributions: S.Z. and H.Z. conceived and designed the study; Z.W. and L.T. analyzed the data; S.Z. wrote the paper. All authors have read and agreed to the published version of the manuscript.
Funding: This work is financially supported by the National Natural Science Foundation of China (No. 42101414, 41771504) and the National Natural Science Foundation of Jilin Province (No.20200201214JC).

Institutional Review Board Statement:
The study did not involve humans or animals.

Informed Consent Statement:
The study did not involve humans.
Data Availability Statement: Not applicable.