Remote Sens. 2014, 6(3), 2050-2068; doi:10.3390/rs6032050
Article

Assessing the Temporal Stability of the Accuracy of a Time Series of Burned Area Products

1,* email, 2email, 3email and 1email
Received: 22 October 2013; in revised form: 14 February 2014 / Accepted: 27 February 2014 / Published: 6 March 2014
(This article belongs to the Special Issue Quantifying the Environmental Impact of Forest Fires)
This is an open access article distributed under the Creative Commons Attribution License which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.
Abstract: Temporal stability, defined as the change of accuracy through time, is one of the validation aspects required by the Committee on Earth Observation Satellites’ Land Product Validation Subgroup. Temporal stability was evaluated for three burned area products: MCD64, Globcarbon, and fire_cci. Traditional accuracy measures, such as overall accuracy and omission and commission error ratios, were computed from reference data for seven years (2001–2007) in seven study sites, located in Angola, Australia, Brazil, Canada, Colombia, Portugal, and South Africa. These accuracy measures served as the basis for the evaluation of temporal stability of each product. Nonparametric tests were constructed to assess different departures from temporal stability, specifically a monotonic trend in accuracy over time (Wilcoxon test for trend), and differences in median accuracy among years (Friedman test). When applied to the three burned area products, these tests did not detect a statistically significant temporal trend or significant differences among years, thus, based on the small sample size of seven sites, there was insufficient evidence to claim these products had temporal instability. Pairwise Wilcoxon tests comparing yearly accuracies provided a measure of the proportion of year-pairs with significant differences and these proportions of significant pairwise differences were in turn used to compare temporal stability between BA products. The proportion of year-pairs with different accuracy (at the 0.05 significance level) ranged from 0% (MCD64) to 14% (fire_cci), computed from the 21 year-pairs available. In addition to the analysis of the three real burned area products, the analyses were applied to the accuracy measures computed for four hypothetical burned area products to illustrate the properties of the temporal stability analysis for different hypothetical scenarios of change in accuracy over time. The nonparametric tests were generally successful at detecting the different types of temporal instability designed into the hypothetical scenarios. The current work presents for the first time methods to quantify the temporal stability of BA product accuracies and to alert product end-users that statistically significant temporal instabilities exist. These methods represent diagnostic tools that allow product users to recognize the potential confounding effect of temporal instability on analysis of fire trends and allow map producers to identify anomalies in accuracy over time that may lead to insights for improving fire products. Additionally, we suggest temporal instabilities that could hypothetically appear, caused by for example by failures or changes in sensor data or classification algorithms.
Keywords: validation; global products; error matrix; fire disturbance
PDF Full-text Download PDF Full-Text [826 KB, uploaded 19 June 2014 02:07 CEST]

Export to BibTeX |
EndNote


MDPI and ACS Style

Padilla, M.; Stehman, S.V.; Litago, J.; Chuvieco, E. Assessing the Temporal Stability of the Accuracy of a Time Series of Burned Area Products. Remote Sens. 2014, 6, 2050-2068.

AMA Style

Padilla M, Stehman SV, Litago J, Chuvieco E. Assessing the Temporal Stability of the Accuracy of a Time Series of Burned Area Products. Remote Sensing. 2014; 6(3):2050-2068.

Chicago/Turabian Style

Padilla, Marc; Stehman, Stephen V.; Litago, Javier; Chuvieco, Emilio. 2014. "Assessing the Temporal Stability of the Accuracy of a Time Series of Burned Area Products." Remote Sens. 6, no. 3: 2050-2068.


Remote Sens. EISSN 2072-4292 Published by MDPI AG, Basel, Switzerland RSS E-Mail Table of Contents Alert