Progress and Limitations in the Satellite-Based Estimate of Burnt Areas

Abstract

The detection of burnt areas from satellite imagery is one of the most straightforward and useful applications of satellite remote sensing. In general, the approach relies on a change detection analysis applied on pre- and post-event images. This change detection analysis usually is carried out by comparing the values of specific spectral indices such as: NBR (normalised burn ratio), BAI (burn area index), MIRBI (mid-infrared burn index). However, some potential sources of error arise, particularly when near-real-time automated approaches are adopted. An automated approach is mandatory when the burnt area monitoring should operate systematically on a given area of large size (country). Potential sources of errors include but are not limited to clouds on the pre- or post-event images, clouds or topographic shadows, agricultural practices, image pixel size, level of damage, etc. Some authors have already noted differences between global databases of burnt areas based on satellite images. Sources of errors could be related to the spatial resolution of the images used, the land-cover mask adopted to avoid false alarms, and the quality of the cloud and shadow masks. This paper aims to compare different burnt areas datasets (EFFIS, ESACCI, Copernicus, FIRMS, etc.) with the objective to analyse their differences. The comparison is restricted to the Italian territory. Furthermore, the paper aims to identify the degree of approximation of these satellite-based datasets by relying on ground survey data as ground truth. To do so, ground survey data provided by CUFA (Comando Unità Forestali, Ambientali e Agroalimentari Carabinieri) and CFVA (Corpo Forestale e Vigilanza Ambientale Sardegna) were used. The results confirm the existence of significant differences between the datasets. The subsequent comparison with the ground surveys, which was conducted while also taking into account their own approximations, allowed us to identify the accuracy of the satellite-based datasets.

1. Introduction

Several studies on the possibility of using satellite images for estimating burnt areas (BA) and, more remarkably, fire damages (FD), i.e., the damages to the vegetation caused by fire [1,2,3,4,5], have been carried out in the past three decades. These studies and the associated products are based on coarse- (250 m–1 km), medium- (100–250 m), and high- (10–100 m) spatial resolution satellite imagery. The provided information is of vital importance to fire-related disciplines [6]. The long-time maps of BA allow the assessment of trends in fire regimes that could be associated to changes in human behaviour, landscape, climate, etc. In addition, information on BA allows the estimation of the impact of this phenomenon in determining atmospheric conditions through atmospheric emissions (CO₂, CH₄, NOx, PM, etc.) [6,7] and the vegetation depletion/degradation (vegetation CO₂ sink effect reduction). Moreover, long time series and wide area views can be used to calibrate and validate fire models and fire danger indices [8]. The accuracy and reliability of all deductions and conclusions based on BA information inevitably suffer from the uncertainty associated with the data retrieved using remotely sensed imagery. Usually, these BA detection procedures are based on the definition of spectral indices, which, by exploiting the availability of pre-event and post-event images, measure the impact of fire on vegetation and allow to determine the extent of the damage (Figure 1). A variety of methods have been employed for BA detection [9], including spectral indices, surface temperature inversion, principal component analysis (PCA), image classification [10], neural networks [11], and spectral mixture analysis [12].

Spectral indices are generally more intuitive and simpler to implement, and these qualities allow them to obtain high degrees of accuracy, making them very popular in the related literature [9]. Since forest fires cause vegetation changes, the distinction between burnt and unburnt areas can be achieved by using classical vegetation indices, such as the normalised difference vegetation index (NDVI), enhanced vegetation index (EVI), and normalised difference water index (NDWI), among others [13]. However, several specialised spectral indices have been developed for detecting BA on remotely sensed optical images, some of which are listed in Table 1. They were designed to take into account the main effects of fire on the spectral characteristics of vegetation. Indeed, fire leads to a rapid increase in SWIR and decrease in NIR reflectance. Based on this, the normalised burn ratio (NBR) [14], the normalised difference SWIR (NDSWIR) index [15], and the burnt area index (BAI) [16] were specifically designed to detect BA. Some researchers found that the combination of two SWIR bands showed a strong spectral separation on burnt and unburnt areas. Therefore, the Mid-InfraRed Bi-spectral Index (MIRBI) [17] was designed using these bands. Ref. [18] observed that BA could appear to be 5–6 °C hotter than in unburnt areas within 1 month after burning; thus, the thermal infrared (TIR) bands can also be used to detect the BA during the fire season. The NBR-thermal (NBRT) index [18] introduces the brightness temperature (BT) measured in the TIR band to improve the accuracy of fire detection. Of course, this requires the availability of a sensor capable of detecting radiation in thermal channels, ruling out, for instance, platforms such as Sentinel-2. Therefore, this index could be classified among the sensor-specialised indices, that is to say, within the category of those indices designed to exploit the characteristics of a specific satellite sensor.

The currently used spectral indices reflect BA changes due to the loss of vegetation between the pre-fire and post-fire images. Nevertheless, other events might cause similar spectral behaviours impacting land cover, namely clouds, topographic shadows, agricultural practices, etc., which may lead to commission errors. A multitemporal image approach is useful and effective for automatic BA detection without relying on ground reference data. An analysis of the impact of different sources of error on the accuracy of BA products was performed by [19]. Their conclusions are also discussed in the section of this paper devoted to discussion.

It is well known that climate impacts fire regimes [20]. In addition, biomass burning is a critical source of aerosols and greenhouse gases, which directly affect atmospheric composition [21,22,23,24]. On the other hand, fires have important ecological implications, as they play an important role in determining biodiversity [25,26,27]. Finally, wildfires have significant societal impacts, particularly when extreme events occur. Heavy damage to people’s lives, health, and infrastructure were observed in recent catastrophic fire seasons in Southern Europe and the western USA (Montiferru, Italy, 2021; Tenerife, Spain, 2023; Alexandroupolis, Greece, 2023; Maui, Hawaii, 2023).

As a demonstration of its relevance in characterising the Earth’s climate system, fire disturbance was included by the Global Climate Observing System (GCOS) among the essential climate variables (ECV) [28]. Therefore, the main objective of satellite-based global BA datasets is to develop a long-term archive of observations aiming at assessing trends in wildfire phenomena and their potential impact on the near future. A historical analysis of the trends in BA mapping from the early 1980s to the beginning of this century is given in [29]: Initial attempts to generate global BA products were described, and an analysis of global and regional datasets was performed. As mentioned in that paper [29], a first accuracy estimate of global products based on higher-spatial-resolution satellite sensors (Landsat/TM versus MODIS) became available a few years ago [30]. The commission and omission error rates of the most accurate product were 42% and 68%, respectively [30]. In a more recent study on sub-Saharan Africa [31], better accuracy values of the global BA products were observed since the standard NASA BA product (MCD64A1 c6) showed a commission error of 20.4% and an omission error of 59.6%. Some recent papers were also devoted to characterizing the sources of error that impact the accuracy of the global BA products [19,31,32,33].

The main objective of this paper is to compare some of the most widely used BA datasets (among which are the EFFIS (European Forest Fire Information System) and FIRMS (Fire Information for Resource Management)) and to assess their accuracy by using ground data surveys and high-resolution imagery (Sentinel-2). The comparison analysis was carried out on the entire Italian territory, whereas the accuracy assessment was restricted to the three Italian regions of Calabria, Sicily, and Sardinia. The paper is organised as follows: Section 2 describes the data and methods used, Section 3 is devoted to showing the results obtained and to discussing them, and finally, Section 4 is reserved for conclusions.

Table 1. List of spectral indices commonly used to detect burnt areas by using images from satellite optical sensors.

Index	Formula	Reference
Burnt area index (BAI)	$\frac{1}{{\left(0.06-R_{NIR}\right)}^2+{\left(0.1-R_{red}\right)}^2}$	[34]
Normalised burn ratio (NBR)	$\frac{\left(R_{NIR}-R_{SWIR2}\right)}{\left(R_{NIR}+R_{SWIR2}\right)}$	[35]
Normalised burn ratio 2 (NBR2)	$\frac{\left(R_{SWIR1}-R_{SWIR2}\right)}{\left(R_{SWIR1}+R_{SWIR2}\right)}$	[35]
Mid-infrared bi-spectral index (MIRBI)	$10\times R_{SWIR2}-9.8\times R_{SWIR1}+2$	[17]
Normalised difference SWIR (NDSWIR)	$\frac{\left(R_{NIR}-R_{SWIR1}\right)}{\left(R_{NIR}+R_{SWIR1}\right)}$	[16]
Normalised burn ratio plus (NBR+)	$\frac{\left(R_{SWIR2}-R_{NIR}-R_{GREEN}-R_{BLUE}\right)}{\left(R_{SWIR2}+R_{NIR}+R_{GREEN}+R_{BLUE}\right)}$	[36]
Burnt area index for Sentinel-2 (BAI2)	$\left(1-\sqrt{\frac{R_6\times R_7\times R_{8A}}{R_4}}\right)\times \left(\frac{R_{12}-R_{8a}}{\sqrt{R_{12}+R_{8a}}}+1\right)$	[37]
NBR thermal (NBRT) *	$\frac{\left(R_{NIR}-R_{SWIR2}\times S_{TIR}\right)}{\left(R_{NIR}+R_{SWIR2}\times S_{TIR}\right)}$	[18]

* S_TIR denotes scaled brightness temperature, that is, TIR value divided by 10.000.

Table 1. List of spectral indices commonly used to detect burnt areas by using images from satellite optical sensors.

Index	Formula	Reference
Burnt area index (BAI)	$\frac{1}{{\left(0.06-R_{NIR}\right)}^2+{\left(0.1-R_{red}\right)}^2}$	[34]
Normalised burn ratio (NBR)	$\frac{\left(R_{NIR}-R_{SWIR2}\right)}{\left(R_{NIR}+R_{SWIR2}\right)}$	[35]
Normalised burn ratio 2 (NBR2)	$\frac{\left(R_{SWIR1}-R_{SWIR2}\right)}{\left(R_{SWIR1}+R_{SWIR2}\right)}$	[35]
Mid-infrared bi-spectral index (MIRBI)	$10\times R_{SWIR2}-9.8\times R_{SWIR1}+2$	[17]
Normalised difference SWIR (NDSWIR)	$\frac{\left(R_{NIR}-R_{SWIR1}\right)}{\left(R_{NIR}+R_{SWIR1}\right)}$	[16]
Normalised burn ratio plus (NBR+)	$\frac{\left(R_{SWIR2}-R_{NIR}-R_{GREEN}-R_{BLUE}\right)}{\left(R_{SWIR2}+R_{NIR}+R_{GREEN}+R_{BLUE}\right)}$	[36]
Burnt area index for Sentinel-2 (BAI2)	$\left(1-\sqrt{\frac{R_6\times R_7\times R_{8A}}{R_4}}\right)\times \left(\frac{R_{12}-R_{8a}}{\sqrt{R_{12}+R_{8a}}}+1\right)$	[37]
NBR thermal (NBRT) *	$\frac{\left(R_{NIR}-R_{SWIR2}\times S_{TIR}\right)}{\left(R_{NIR}+R_{SWIR2}\times S_{TIR}\right)}$	[18]

* S_TIR denotes scaled brightness temperature, that is, TIR value divided by 10.000.

2. Materials and Methods

2.1. Materials

The study is based on the following:

• The global data were provided in the framework of international initiatives:
• EFFIS data, downloaded from EFFIS website [38];
• FIRMS data, downloaded from FIRMS website [39];
• Copernicus Global Land Service (CGLS) burnt areas [40];
• ESA Climate Change Initiative (ESACCI) [41];
• Copernicus Climate Change Service (C3S) [21].
• The local data used are as follows:
• That made available by CFVA (Corpo Forestale e di Vigilanza Ambientale) through the Sardinia Geoportal “https://www.sardegnageoportale.it (accessed on 17 December 2023)”;
• That made available in the framework of an agreement between the School of Aerospace Engineering (SIA) and the CUFA (Comando Unità per la tutela Forestale, Ambientale e Agroalimentare), a unit of the Italian Carabinieri specialised in the protection of forests and in the prevention and repression of environmental and agri-food related crimes;
• Sentinel-2-based BA dataset generated by SIA.

Since 1998, EFFIS (European Forest Fire Information System, https://effis.jrc.ec.europa.eu/ (accessed on 17 December 2023)) has supported the services in charge of the protection of forests against fires in the EU and neighbouring countries and provides the European Commission and the European Parliament with updated and reliable information on wildland fires in the territory of the Union.

Since 2000, NASA’s FIRMS (Fire Information for Resource Management System, “https://firms.modaps.eosdis.nasa.gov/download/ (accessed on 17 December 2023)”) has provided active fire (and BA) data acquired from the moderate-resolution imaging spectroradiometer (MODIS) aboard the Aqua and Terra satellites and, more recently, from the visible infrared imaging radiometer suite (VIIRS) aboard S-NPP and NOAA 20 (formerly known as JPSS-1). These data are available globally within 3 h of the satellite observation.

The ESA CCI (Climate Change Initiative) project aims to improve the consistency of the BA, using better algorithms for both pre-processing and BA detection while incorporating error characterization in their product.

The CGLS (Copernicus Global Land Service, “https://land.copernicus.eu/global/products/ba (accessed on 10 September 2023)”) provides BA at 300 m spatial resolution since 2014.

The area of interest is restricted to Italy and three of its regions most affected by wildfires (Figure 2): Calabria (28,482 ha burnt in 2021), Sicily (57,420 ha burnt in 2021), and Sardinia (25,840 ha burnt in 2021).

2.2. Methods

All data were rasterised and resampled to the same pixel size of 250 m in ENVI using the nearest neighbour option. (In the case of the datasets that were resampled from 300 m to 250 m, namely CGLS and C3S, this choice could introduce errors in the estimation of the burnt area in terms of its shape and size but not an overestimation of the number of fire events). At the Italian country level, the comparison was made by constructing a table like the one shown in

Table 2. Example of the “confusion matrix” adopted to compare different available databases. TP, number of true positives.

	EFFIS	FIRMS	CGLS	CCI	Common Burnt Areas	Total Burnt Areas
EFFIS	-	TP	TP	TP	Percentage of BAs common to EFFIS and at least two other datasets	Total EFFIS BA
FIRMS	TP	-	TP	TP	Percentage of BAs common to FIRMS and at least two other datasets	Total FIRMS BA
CGLS	TP	TP	-	TP	Percentage of BAs common to CGLS and at least two other datasets	Total CGLS BA
CCI	TP	TP	TP	-	Percentage of BAs common to CCI and at least two other datasets	Total CCI BA

. The table shows the following for each dataset:

• The correspondence between a dataset and each of the others, in terms of percentage of common events;
• The events that the dataset has in common with at least two of the other datasets.

In order to allow a comparison of the results shown here with those of other studies [19,42,43], we have to emphasise that the analysis does not aim to distinguish between related or unrelated errors (see [44]), which could be useful for assessing the effect of spatial resolution in the BA detection. However, we do analyse, for each BA of each one of the datasets, whether or not it has at least one pixel in common with any of the others. Therefore, our only objective is to determine whether a given BA was detected or not, while we did not aim to assess BA over- or under-estimation, which could be related to the spatial resolution of the sensor. Consequently, a burnt pixel of a dataset was considered a true positive when the corresponding pixel in another dataset was also burnt. A detailed analysis of the sources of error that affect the accuracy of current coarse-resolution BA products is given in [19].

Further, a detailed (local) analysis of the accuracy of each one of these databases was carried out by considering the events that occurred in some of the Italian regions most affected by forest fires (Sardinia, Calabria, and Sicily) in the years 2019, 2020, and 2021, for which ground data are available.

For the year 2019, CFVA provided ground data for the territory of Sardinia.

For the year 2020, CUFA provided ground data for Calabria and CFVA provided ground data for Sardinia.

For the year 2021, CFVA provided ground data for Sardinia, and the regional authorities of Sicily provided ground data for Sicily.

These data were used to assess the quality of the databases built on satellite imagery.

The comparison between databases built on satellite imagery was made on a pixel-by-pixel basis, which requires resampling the BA products to the same pixel size. For this study, this was not a big issue since almost all datasets were built by using sensors with similar spatial resolution [45,46]. Datasets provided in vectorial format (EFFIS and FIRMS) were rasterised. All the low/medium-resolution maps were co-registered and resampled at the same spatial resolution (250 m). For each dataset, we assessed the pixel-to-pixel correspondence with the others.

In addition, the burnt pixels were grouped in “events”, defined as groups of burnt pixels corresponding to the same fire, using functions available in the Image Processing Toolbox of MATLAB, such as edge, regionprops, etc. In this way, it was possible to estimate the differences in the number of events recorded in the different datasets.

The correspondence between satellite-based datasets and ground-based observations in the Sardinia, Calabria, and Sicily regions for years 2019, 2020, and 2021 was analysed by comparing satellite and ground BA. In this case, the raster CCI dataset was vectorised. In this “local” analysis, we left out the CGLS dataset since it seems significantly less accurate than others (see following results).

Finally, we dedicated a detailed analysis to the most damaging fire event of the Italian fire season of 2021, which occurred in Sardinia, with the main purpose to show the relevance of the land-cover map in the estimate of the BA.

Table 3. List of the datasets used to carry out the analysis.

Dataset	Source	Spatial Resolution	Timespan	Algorithm/Method Used
ESA CCI	MODIS	250 m	2001–present	NIR time series + active fires
CGLS	VGT (PROBA-V)	300 m	2014–present	Reflectance change
EFFIS	MODIS	250 m	2000–present	Combination of bands
FIRMS/MCD64	MODIS	250 m *	2000–present	NBR2 + active fire
C3S	OLCI	300 m	2017–2020	NIR time series + active fires
CUFA	GPS	-	2020–present	Ground survey
CFVA	GPS	-	2005–present	Ground survey
SIA	Sentinel-2	20 m	2020–present	NBR

* Product based on MODIS 500 m spectral channels resampled to 250 m by using ENVI resampling tool, nearest neighbour algorithm.

provides some details on each one of the datasets used in the analysis.

SIA BA, based on Sentinel-2 images, are computed systematically for the whole Italian territory at a frequency of 5 days (84 S2 tiles are needed to cover the whole territory). The procedure is based on the use of the classical dNBR (dNBR = NBR − NBR) index. Additional data layers, such as the SCL (scene classification layer) map available in the Sentinel-2 L2A image, the 2018 Corine Land Cover, and the hotspots acquired in near-real-time from a NASA website (https://nrt4.modaps.eosdis.nasa.gov/archive, accessed on 17 December 2023), are included in the processing pipeline with the objective to reduce omissions and false detections.

3. Results and Discussion

This section reports the results of the comparison between different BA datasets. The results are categorised in two parts:

• The comparison between global datasets performed at the Italian country level considering seven years’ worth of data;
• The comparison at the local level considering some of the Italian regions most affected by forest fires and both satellite and in situ data in specific years for which these in situ data were made available.

3.1. Results and Discussion of the Long-Time Comparison

This section is devoted to reporting the results of the long-time comparison between global datasets considering the whole Italian country as a region of interest. All data were rasterised and resampled to the same spatial resolution of 250 m.

Table 4. Comparison between four databases at the Italian country level on seven years of data: 2014–2020.

	EFFIS	FIRMS	CGLS	CCI	Common Burnt Areas in 7 Years in %	Total Burnt Areas in 7 Years (km2)
EFFIS	-	32.3%	22.26%	50.14%	32.7%	4068.0
FIRMS	23.9%	-	15.3%	40.30%	23.8%	5496.0
CGLS	4.1%	4.25%	-	9.1%	4.3%	22,313.0
CCI	14.7%	15.93	14.6%	-	10.8%	13,895.0

and

Table 5. Comparison in terms of number of wildfire events between the four datasets at the Italian country level on seven years of data: 2014–2020.

	EFFIS	FIRMS	CGLS	CCI	Total Number of Events in 7 Years
EFFIS	-	44.7%	24.21%	40.31%	2714
FIRMS	26.71%	-	46.5%	49.12%	2445
CGLS	2.21%	2.28%	-	6.21%	73,547
CCI	20.38%	21.97	49.75%	-	6099

synthetically show the results of the comparison between the four datasets (EFFIS, CCI, CGLS, and FIRMS) that cover the time interval from 2014 to 2020. Reading along a row of Table 4, one can find the correspondence between the BA of a given dataset (e.g., EFFIS) and the other datasets (CCI, CGLS, and FIRMS). The comparison was performed on a pixel-by-pixel basis. For instance, 32.3% of the EFFIS BA has a corresponding BA in the FIRMS dataset, whereas the correspondence with CGLS and CCI datasets amounts to 22.26% and 50.14%, respectively. The CGLS dataset shows the largest disagreement with the other datasets, reaching a correspondence of around 9% (9.1% with CCI) in the best case. The total BA is also the largest among all considered datasets, totalling 22,313 km² in the 7 years considered. For perspective, this value is five times larger than the total BA reported by EFFIS.

If we consider the number of events computed as described above, the CGLS dataset still returns a much higher count than the other datasets. In summary, EFFIS provides the smallest amount of BA (4068.0 km²), whereas FIRMS provides the lowest number of events (2445). CCI contains more than three times the smallest area (13,895.0 km²) and 2.5 times the lowest number of events reported by other datasets. The number of events registered by CGLS is significantly higher than the others, almost 30 times the minimum registered by FIRMS. This number, combined with the value of the total BA, could suggest that many of the BA detected by CGLS correspond to small groups of a few pixels. This is, in some ways, confirmed by Figure 3, which shows the number of events as a function of the BA. The figure shows that for BA smaller than 125 ha, the number of events recorded by CGLS is much larger than other datasets.

The annual distribution of BA and frequency of fire events are shown in Figure 4 and Figure 5, respectively. It is immediately apparent that the CGLS dataset, differently from the other three, does not exhibit the BA minimum that occurred in 2018. The same happens in terms of number of recorded events (Figure 5). On the other hand, the other three datasets (EFFIS, FIRMS, and CCI) show similar trends, even though the reported values remain significantly different. However, it is interesting to note that the EFFIS dataset referring to year 2020 does not show a decreasing trend in terms of BA and fire events, which is present in the CCI and FIRMS datasets. To this end, it is important to note that in 2018, EFFIS started to use both MODIS and Sentinel-2 imagery for BA detection. This fact may explain the increase in the number of detected events.

3.2. Results and Discussion of the Local Comparison

Following the “long-time” (7 years) country-level (Italy) comparison, the analysis of the quality of the satellite-based BA datasets was carried out by considering the data collected in situ by CUFA and CFVA. In this case, the BA are made available in vectorial format. CFVA has its own protocol to delineate BA that is mostly based on ground survey by personnel equipped with GPS. The personnel walk along the perimeter of the BA, recording their GPS position every 3 to 5 s. For larger fires, drone and aerial imagery are used.

3.2.1. 2019 Local Comparison: Sardinia

We started the local analysis by comparing the year 2019 BA polygons provided by CFVA for the Sardinia region (Italy) with the EFFIS, FIRMS, CCI, and C3S datasets. The C3S dataset was not included in the long-term comparison since it does not cover the entire seven-year period considered (see Table 3).

The results are summarised in

Table 6. Results of the comparison between different burnt areas datasets for the year 2019 on the Sardinia region.

Database 2019	Tot. Burnt Areas (ha)	Number of Areas	Correspondence CFVA vs. Other	Correspondence Other vs. CFVA
CFVA	6758.0	1428
EFFIS	4343.0	25	2.9%	72.0%
FIRMS	2800.0	53	1.0%	37.7%
CCI	1810.7	23	1.7%	61.0%
C3S	2673.0	13	1.8%	76.9%

, which contains the correspondence between the CFVA ground dataset and the satellite-based datasets, and

Table 7. Comparison between different datasets in terms of the types of land cover affected (year 2019, Sardinia region).

Database 2019	CORINE Classes Agricultural (from 211 to 242) (ha)	CORINE Classes Forest and Seminatural (from 243 to 333) (ha)	Other Classes (ha)	Difference in Forest Area (%) (Omission)
CFVA	4215.0	2479.0	64.0	-
EFFIS	2330.0	1959.0	53.0	21.0%
FIRMS	1811.0	902.0	87.0	63.6%
CCI	463.0	1317.0	30.8	46.9%
C3S	1571.0	1101.6	0.0	55.6%

, which contains the amounts of BA associated with different land-cover types for each dataset.

As can be observed from Table 6, there exist significant differences between the number of BA in the satellite- and ground-based datasets. In terms of the extent of the BA, the differences range from 35.7% to 73%. The correspondence between the CFVA BA and the satellite-based datasets is very low, but more interestingly, the correspondence between EFFIS and C3S BA and those available in the CFVA database is rather high: 72% and 76.9%, respectively. This means that, if we treat the CFVA dataset as a ground truth, the rate of false alarms (false detection of BA) for these two datasets is relatively low, while it is higher for FIRMS and CCI. In general, the omission error is very large. As stated elsewhere [19], the highest omission error, particularly in croplands, could reach values >85%.

All algorithms devoted to detecting BA from satellite images use land-cover maps for masking non-burnable and agricultural areas in order to minimize the false detections due to industrial activities and agricultural practices. Therefore, differences between datasets can also result from the use of different land-cover maps. In order to verify this hypothesis, we analysed the BA datasets in terms of land cover by using the 2018 Corine Land Cover (CLC) map “https://land.copernicus.eu/pan-european/corine-land-cover/clc2018 (accessed on 17 December 2023)”.

The analysis of the land-cover type of the BA was performed by distinguishing three main classes: agricultural (CLC classes: 211–242), forest and semi-natural (243–333), and other. The differences in the extent of the burnt forest area ranges from 21% (EFFIS) to 63.6% (FIRMS). FIRMS shows the largest difference. It is interesting to note that FIRMS is the only non-European dataset analysed. Therefore, the use of a land-cover map other than Corine to mask non-burnable areas (e.g., MODIS MCD12Q1, MODIS/Terra + Aqua yearly land-cover type) could significantly contribute to this large mismatch.

The results of the comparison in terms of distribution of the events according to BA sizes are shown in Figure 6. It is interesting to observe that, as expected, the larger the size of the BA, the smaller the difference in number of recorded events between CFVA and the other datasets.

Looking more deeply into the EFFIS dataset, we note that the smallest size of the BA detected is 21 ha and that the BA compared with those measured on the ground are usually overestimated: Figure 7 shows the histogram of the size differences between the in situ surveyed BA and the corresponding EFFIS estimates. To check if the accessibility of the area affected by the fire could impact the accuracy of the field data, an analysis of the imperviousness (in terms of the slope of the terrain) of the affected areas was conducted. No relationship was found between the EFFIS—CFVA BA differences and the imperviousness of the area. In other words, no correlation was found between the differences in the BA provided by CVFA and EFFIS and the averaged slope of the areas affected by the fire.

It is worthwhile to observe that 1379 out of the 1429 BA registered by CFVA in 2019 cover an area smaller than 20 ha, which is far below the detection threshold declared by EFFIS.

3.2.2. 2020 Local Comparison: Sardinia

A similar analysis was carried out for year 2020. The results are summarised in

Table 8. Results of the comparison between different burnt areas datasets for the year 2020 in the Sardinia region.

Database	Tot. Burnt Areas (ha)	Number of Areas	Correspondence CFVA vs. Other	Correspondence Other vs. CFVA
CFVA	7985.0	932	-	-
EFFIS	4536.0	34	3.8%	89.0%
FIRMS	2549.0	30	3.5%	53.0%
CCI	5299.0	31	1.7%	45.0%
C3S	4358.0	10	1.3%	100.0%

and

Table 9. Comparison between different datasets in terms of the types of land cover affected for the year 2020 in the Sardinia region.

Database	CORINE Classes Agricultural (from 211 to 242) (ha)	CORINE Classes Forest and Seminatural (from 243 to 333) (ha)	Other Classes (ha)	Difference on Forest Area (%) (Omission)
CFVA	4554.0	3364.0	67.0	-
EFFIS *	1935.0	2785.0	0.0	13.4%
FIRMS	1224.4	1312.0	13.0	59.0%
CCI	2665.0	2620.0	13.7	18.4%
C3S	2285.0	2064.0	8.0	35.7%

* Note that the small difference in the sum of the values along the row, if compared to Table 8, is a consequence of the rasterization of the polygons of the BA.

.

Again, as shown in Table 8, the number of BA is significantly different between those available in the satellite-based datasets and those collected on the ground. In terms of the extent of the BA, the differences range from 43.0% to 68%. In general, the correspondence between CFVA BA and the other datasets is very low. However, it is interesting to note that the correspondence between EFFIS and C3S BA and those available in CFVA database is rather high: 89% and 100%, respectively. This means that the rate of false alarms (false detection of BA) is very low for these two datasets. The results mark a significant improvement with respect to year 2019. For FIRMS and CCI, the rate of false detections is higher. Concerns the characteristics of the BA, the results are reported in Table 9. It can be seen that most of the BA registered by CFVA belong to the following classes: 211–213 (arable land), 221–223 (permanent crops), 231 (pasture), 241 (annual crops associated with permanent crops), and 242 (complex cultivation patterns). Inversely, in the EFFIS case, 40% of the BA correspond to agricultural areas, whereas for FIRMS, CCI, and C3S the value is around 50%. If we focus our attention on the forest and semi-natural areas (CLC classes: 243–333), we see that the differences between the BA registered by CFVA and that reported by other datasets decreases from a minimum of 17.2% (EFFIS) to a maximum of 61.0% (FIRMS). Again, the highest difference occurs with respect the FIRMS dataset, the only one provided by a non-European institution (Maryland University), which, as mentioned above, uses a land-cover map other than Corine.

To assess the impact of the land cover in the estimate of the BA, we compared the results shown in Table 8 with the corresponding values provided by CFVA with the shapefiles of the BA.

Table 10. Comparison between land-cover distribution of the burnt areas based on CLC 2018 or as reported in the EFFIS and CFVA BA datasets for year 2020.

Database	CORINE Classes Agricultural (from 211 to 242) (ha)	Agricultural from Polygons (ha)	CORINE Classes Forest and Seminatural (from 243 to 333) (ha)	Forest + Seminatural from Polygons (ha)
CFVA	4554.0	4713.0	3364.0	3272.0
EFFIS	1935.0	2435.0	2785.0	2083.0

contains the land-cover classes of the BA determined using CLC 2018 compared with the cover classes present in the CFVA and EFFIS BA datasets. As can be observed, in the case of EFFIS, the differences are significant. It should also be remarked that in the case of CFVA, only three classes are distinguished: forest, pasture, and other. Therefore, it is not trivial to decide to which group should be assigned the pasture class because it could be considered an agricultural (CLC n. 231: pasture) or natural (CLC n. 331: natural grassland) class at the same time. In Table 10, we assume that class as natural. EFFIS identifies more classes (broadleaf, coniferous, sclerophyllous, mixed, other natural, artificial, agricultural, etc.).

The results show that the difference in the ascription of the land-cover classes of the BA runs from 3% for CVFA to a maximum of 33% in the case of EFFIS. Then, clearly, two different land-cover maps have been used to classify the BA and, probably, in the case of EFFIS, to mask the non-burnable areas.

The results of the comparison, in terms of distribution of the BA sizes, are shown in Figure 8. As before, the difference in the number of events registered between CFVA and other datasets decreases as the size of the BA increases.

A deeper analysis of the 2020 EFFIS dataset shows that the smallest of the BA detected is 2 ha, a surprisingly small value considering the sensor used. This represents a significant improvement with respect the previous year (2019), when the smallest area registered was 21 ha. Probably, as stated on the EFFIS website, in some cases, Sentinel-2 images are used to enhance the results. However, only 350 of 932 BA registered by CFVA cover more than 2 ha. A further indication of the improvement in the 2020 EFFIS dataset is that not all the BA are overestimated if compared with those measured on the ground. To this end, Figure 9 shows the histogram of the differences between the in situ measurements and EFFIS data.

3.2.3. 2020 Local Comparison: Calabria

A further analysis was carried out for year 2020 by using the data collected in situ by CUFA in the Calabria region. Before proceeding with the description of the results, it should be emphasised that the data collected by CUFA are not complete since they are mainly concerned with fires of a certain relevance that have affected forest areas.

The results are summarised in

Table 11. Results of the comparison between different burnt areas datasets for the year 2020 in the Calabria region.

Database	Tot. Burnt Areas (ha) Forest (All Cover Types)	Number of Areas	Correspondence CUFA vs. Other	Correspondence Other vs. CUFA
CUFA	3254.5	599	-	-
EFFIS	3789.0 (9185.0) *	278	21.7%	38.5%
FIRMS	753.0 (2066.0)	40	1.5%	37.5%
CCI	2466.0 (7635.6)	89	5.0%	29.0%
C3S	3329.0 (8192.0)	79	7.3%	36.7%

* The total burnt areas as estimated by EFFIS, FIRMS, CCI, and C3S are given in parentheses.

, which contains the correspondence between the CUFA dataset and the satellite-based datasets.

As a consequence of the specific interests of the CUFA, the total extension of the BA shows differences ranging from −2.3% (C3S) to 76.9% (FIRMS). This means that the differences are smaller than in the Sardinian case. The number of events is also closer, and the correspondence among events is higher. The FIRMS dataset represents an exception in this general performance improvement. Likely, the reason behind this can still be explained in terms of the different land-cover map used by FIRMS to mask non-burnable or agricultural areas.

Figure 10 shows the difference between ground measurements and satellite-based datasets when the sizes of the BA are considered. As expected, larger differences occur at smaller sizes.

3.2.4. 2021 Local Comparison: Sicily and Sardinia

In Italy, the regional authorities are in charge of the surveying of BA. For this reason, the surveying procedures may vary from region to region. To check the potential impact of the different data collection procedures, a similar analysis as the previous ones was carried out for the Sicily region for year 2021. The results, referring to regional and EFFIS data, are shown in

Table 12. Results of the comparison between in situ and EFFIS BA datasets for the year 2021 in the Sicily region.

Database	Tot. Burnt Areas (ha)	Number of Areas	Correspondence Reg. vs. EFFIS	Correspondence EFFIS vs. Reg
Reg. auth.	57,420.0	1000	-	-
EFFIS	62,302.0	621	33.7%	32.85%

.

The data concerning this region allows us to make some considerations:

• First of all, despite the number of recorded events being lower, the EFFIS dataset shows a larger amount of BA;
• The correspondence between the BA contained in the two datasets amounts to almost the same value. This could mean that, even if EFFIS tends to detect the larger events, the false-positive rate is still rather high or that a different approach of the regional authority is adopted in the selection of the fires to be delineated. Figure 11, which shows a comparison between the EFFIS polygons (in red) and the ones provided by the Sicilian regional authority (in yellow), seems to corroborate the latter hypothesis. The EFFIS 2867.0 ha BA polygon has no match in the regional dataset apart from two small, yellow polygons located in the upper right and lower left. The BA polygons are super-imposed on a Sentinel-2 false colour composite image (R = band 4, G = band 8, B = band 2) acquired on the 10th of August 2021. The reason for the absence of this large burnt area in the regional BA dataset could be due to the fact that the area is mostly agricultural. The presence of many hotspots (yellow circles), as provided by the FIRMS Webmapper in the period 29 July–1 August, demonstrates the presence of several fires. However, their scattered appearance may also lead us to hypothesise their association with the typical agricultural practice of burning after harvesting. This could explain the absence of the polygon in the regional dataset.

It is also interesting to note that in this case, the numbers of BA present in the two datasets are comparable to each other (1000 vs. 621). In particular, the number of BA smaller than 20 ha in the EFFIS dataset is 216 (34% of the total). The difference between the two datasets decreases as the percentage of BA covered by forest increases, as shown in Figure 12.

Particularly instructive is the analysis of the existing relationship between hotspots and BA, for which the results are given in

Table 13. Comparison between burnt areas and FIRMS hotspots on Sicily, year 2021.

Dataset	Number of BA	Correspondence HS and BA	BA and HS
EFFIS BA	621	12,649 (50.0%)	545 (82.4%)
Sicily region BA	940	23,553 (93.2%)	462 (49.1%)

, which pertain to the year 2021 in the region of Sicily. FIRMS (based on MODIS and VIIRS) registered 25,271 hotspots. On one hand, it can be seen that in 93% of the cases, there is a correspondence between the BA reported in the regional dataset and the hotspots. The correspondence is much lower in the case of EFFIS. On the other hand, more EFFIS BA (82.4%) than regional BA (49.1%) have at least one associated hotspot. This can be attributed to the fact that in EFFIS algorithm for the identification of BA, similarly as other algorithms, uses a combination of reflectance change detection and hotspot presence. This is clearly not the case for the regional ground surveys. However, it should be noted that the 91% of the BA covering a surface lower than 5 ha and involving some forest areas do not correspond to any hotspot. Therefore, they could refer to fires too small and too short-lived to be detectable by polar orbiting satellites.

3.3. High-Resolution Burnt Area Maps

SIA has been collaborating with CUFA since 2020, providing them with high-resolution (20 m) BA maps at a 5-day frequency by exploiting Sentinel-2 imagery and using the dNBR index. Initially, the experimentation of the service covered a few provinces of the Italian country. In 2021, the mapping was extended to the whole Italian territory. In 2022, in order to reduce the omission errors due to the presence of clouds in the pre- or post-event data, the mapping procedure (the procedure is automatic) started to include hotspots provided by FIRMS, which are based on detections from the MODIS and VIIRS sensors.

Table 14. Results of the comparison between different burnt areas datasets for the year 2021 in the Sardinia region.

Database	Tot. Burnt Areas (ha)	Number of Areas	Correspondence CFVA vs. Other	Correspondence Other vs. CFVA
CFVA	25,840.0	1108 (609) 1	-	-
EFFIS	18,283.0	42	4.4%	83.3%
SIA	26,250.0	544 (2100) 2	22.0%	57.9%

¹ In parentheses is the number of events with BA smaller than 2 ha. ² In parentheses is the total number of burnt areas detected by SIA, including fires that occurred in agricultural areas only.

and Figure 13 show the results of the comparison between CFVA (note that in this case, CUFA is the customer, while CFVA is the producer of the in situ data), EFFIS, and SIA data for the year 2021 on Sardinia. No resampling was carried out in this case because all datasets were in vectorial format.

The extension of the total BA shows differences ranging from 1.6% (SIA) to 29.0% (EFFIS). The number of events registered by SIA is comparable with that provided by CFVA and much larger than the number of events registered by EFFIS. However, the correspondence between areas is still low (22.0%). It should be remarked that, for reasons dictated by their duties, CUFA requested not to neglect agricultural fires in order to give them the possibility of determining whether the potential causes of forest fires may be related to improper agricultural practices, including surface changes related to agricultural areas, which is problematic since it could cause many false alarms due to the detection of changes in crop areas resulting from harvesting. In 2022, the procedure was extended to overcome this issue by introducing a new index (SBI, soil brightness index) that measures the brightness of the soil in the area of the potential fire. The rationale of this index is that a burned area is expected to be darker than a harvested area. Therefore, the area is considered burnt if, in addition to an appropriate dNBR index value, it is also darker than the same in the pre-event image:

SBI = 0.406 × R3 + 0.6 × R4 + 0.645 × R6 + 0.243 × R8,

where R3, R4, R6, and R8 refer to the Sentinel-2-corresponding spectral channel reflectances.

In the SIA procedure, the type of land cover affected by fires is assessed by referring to the 2018 Corine Land Cover. According to the type of land cover affected by the fire, three levels of warning are defined: low (only agricultural area has burnt), medium (some natural area has burnt), and high (mainly forest area has burnt). Further, the SIA procedure considers only BA larger than 2 ha. In Table 14, 544 is the number of BA detected by SIA using Sentinel-2 imagery, which concern, at least partially, natural areas.

Figure 13 compares the distribution of the BA as a function of their sizes. As expected, as the size of the BA increases, the difference in the number of detections decreases. It is clear how the SIA dataset closely follows the distribution of the events as a function of the BA given by the ground-based dataset.

The comparison of Table 13 and Table 14 allows us to highlight the large difference in the number of events registered for year 2021 in the EFFIS dataset for the two regions. According to Corine Land Cover 2018, Sicily has a forest area 2.4 times larger than Sardinia. Of course, this fact alone is not sufficient to explain the large difference in the events registered by EFFIS (621 vs. 42).

Finally, Figure 14 shows the spatial distribution of the BA recorded by EFFIS, SIA, and CFVA for the year 2021. Such spatial distribution is superimposed to a simplified version of the 2018 CLC map, in which we reduced the original classification to a few basic classes (artificial, agricultural, pasture, forest, wetlands, and water bodies).

Of course, EFFIS is able to detect the larger and more significant events even if we have to mention that their dataset reports three BA smaller than 2 ha. More interestingly, many of the CFVA polygons, according to CLC 2018, refer to agricultural areas. A magnification of the zone of higher BA concentration is shown in Figure 15. This could explain why the correspondence between the CFVA- and SIA-recorded events reaches a value of “only” 22%. Indeed, that value refers to the 544 SIA BA, which did not only affect agricultural areas. Agricultural areas are the main source of error (both omission and commission) due to the low sensitivity of coarse-resolution sensors to cropland BA [19] (omission) or due to the effect of harvested fields or post-harvest burning practices (commission) [47].

3.4. Large Fire Event Analysis: The Montiferru Megafire

To assess the relevance of the land cover in driving the estimation of the BA, we analysed the case of the large fire that occurred in western Sardinia (Montiferru area) in the middle of the summer (25–30 July) of 2021. This wildfire burned more than 10,000 hectares of vegetation, including 4000 hectares of forest, making it the most destructive wildfire to have occurred in Sardinia in the past two decades [47]. Figure 16 shows the polygons delimiting the BA as estimated by EFFIS, SIA, and CFVA. The three datasets show very similar results in terms of the extent of the total burnt area. Sentinel-2 images allow the identification of small patches in the area that are either not burnt or for which the dNBR index was below the established threshold. However, if we look at the amount of the forest area burnt as reported in the shapefile of the event, the results are quite different, as shown in

Table 15. Comparison between the burnt area aggregates as provided by the three datasets.

Database	Tot. Burnt Area (ha)	Forest Area Burnt (ha)	Difference
CFVA	12,555.0	4177.1	-
EFFIS	13,278.0	6535.5	61.8%
SIA	10,907.0	4044.3	−3.2%

. The total BA values, as clearly shown in Figure 16, are very similar, whereas the difference between EFFIS and CVFA in the estimation of the burnt forest area is very high (61.8%). On the other hand, the difference between SIA and CFVA is very low. The SIA algorithm uses Corine Land Cover 2018 to identify the type of burnt land cover.

The smaller BA estimated by using Sentinel-2 images is justified by its higher spatial resolution, which allows the identification of smaller non-burnt or non-vegetated patches.

4. Conclusions

The paper carries out two types of analyses:

• A “long-time” (seven-year) analysis, restricted to the Italian country, of the most commonly used BA datasets in Europe;
• A comparison, restricted to some of the Italian regions most affected by summer fires, of BA surveyed in situ with some of the global/continental datasets based on satellite images.

From the first of the two analyses, we learned that significant differences exist between different datasets that cannot be explained simply by appealing to the different sensors or algorithms used, and the land-cover map adopted by each of the product providers to remove non-burnable areas from the computation should be considered as a more likely culprit. Indeed, the FIRMS dataset, the only non-European dataset among the analysed ones and the only one to use a land-cover map other than Corine, is the one that showed the lowest correspondence with the in situ data (Sardinia, 2019 and 2020; Table 7 and Table 9) in terms of the land-cover type of the BA.

The comparison between global/continental datasets shows that the BA product provided by Copernicus Global Land Service (CGLS) presents the largest overestimation both in terms of BA extent and in terms of number of events detected (Table 4 and Table 5) with respect to the other datasets.

The comparison at the local level shows that most of the events (more than 95%) appear to be undetected, mainly as consequence of the insufficient resolution of the satellite images used. As can be observed by looking at the extent of the total BA, the omission of most of the fires has a relatively low impact on the estimation of the BA appertaining to forests (less than 20%). However, it could have a very relevant impact on the following:

• The development of fire hazard indices that use fires distribution to identify the areas more susceptible to burn;
• The assessment of the annual fire trend;
• The assessment of the extinguishing efficiency;
• The assessment of the efficiency of the fire prevention activity;
• The assessment of the effectiveness of the land management practices.

Therefore, in order to be useful for the above-listed objectives, BA mapping, at the country or region level, needs to be carried out by using high-resolution images like Sentinel-2.

Another important point is the need to adopt a common land-cover map, which will allow a product capable of satisfying the different needs of the end users.

Finally, the land-cover map used to mask non-burnable areas should be more explicitly mentioned in the datasets. This is particularly necessary for BA data collected through ground surveys and could ultimately allow to better identify the possible sources of the differences between datasets.