Next Article in Journal
Mapping 30 m Fractional Forest Cover over China’s Three-North Region from Landsat-8 Data Using Ensemble Machine Learning Methods
Next Article in Special Issue
Change Detection from SAR Images Based on Convolutional Neural Networks Guided by Saliency Enhancement
Previous Article in Journal
Estimating Tree Diameters from an Autonomous Below-Canopy UAV with Mounted LiDAR
Previous Article in Special Issue
Accuracy Assessment in Convolutional Neural Network-Based Deep Learning Remote Sensing Studies—Part 1: Literature Review
Review

Accuracy Assessment in Convolutional Neural Network-Based Deep Learning Remote Sensing Studies—Part 2: Recommendations and Best Practices

Department of Geology and Geography, West Virginia University, Morgantown, WV 26505, USA
*
Author to whom correspondence should be addressed.
These authors contributed equally to this work.
Academic Editors: Kuo-Chin Fan and Ying-Nong Chen
Remote Sens. 2021, 13(13), 2591; https://doi.org/10.3390/rs13132591
Received: 1 June 2021 / Revised: 28 June 2021 / Accepted: 1 July 2021 / Published: 2 July 2021
Convolutional neural network (CNN)-based deep learning (DL) has a wide variety of applications in the geospatial and remote sensing (RS) sciences, and consequently has been a focus of many recent studies. However, a review of accuracy assessment methods used in recently published RS DL studies, focusing on scene classification, object detection, semantic segmentation, and instance segmentation, indicates that RS DL papers appear to follow an accuracy assessment approach that diverges from that of traditional RS studies. Papers reporting on RS DL studies have largely abandoned traditional RS accuracy assessment terminology; they rarely reported a complete confusion matrix; and sampling designs and analysis protocols generally did not provide a population-based confusion matrix, in which the table entries are estimates of the probabilities of occurrence of the mapped landscape. These issues indicate the need for the RS community to develop guidance on best practices for accuracy assessment for CNN-based DL thematic mapping and object detection. As a first step in that process, we explore key issues, including the observation that accuracy assessments should not be biased by the CNN-based training and inference processes that rely on image chips. Furthermore, accuracy assessments should be consistent with prior recommendations and standards in the field, should support the estimation of a population confusion matrix, and should allow for assessment of model generalization. This paper draws from our review of the RS DL literature and the rich record of traditional remote sensing accuracy assessment research while considering the unique nature of CNN-based deep learning to propose accuracy assessment best practices that use appropriate sampling methods, training and validation data partitioning, assessment metrics, and reporting standards. View Full-Text
Keywords: accuracy assessment; thematic mapping; feature extraction; object detection; semantic segmentation; instance segmentation; deep learning accuracy assessment; thematic mapping; feature extraction; object detection; semantic segmentation; instance segmentation; deep learning
Show Figures

Graphical abstract

MDPI and ACS Style

Maxwell, A.E.; Warner, T.A.; Guillén, L.A. Accuracy Assessment in Convolutional Neural Network-Based Deep Learning Remote Sensing Studies—Part 2: Recommendations and Best Practices. Remote Sens. 2021, 13, 2591. https://doi.org/10.3390/rs13132591

AMA Style

Maxwell AE, Warner TA, Guillén LA. Accuracy Assessment in Convolutional Neural Network-Based Deep Learning Remote Sensing Studies—Part 2: Recommendations and Best Practices. Remote Sensing. 2021; 13(13):2591. https://doi.org/10.3390/rs13132591

Chicago/Turabian Style

Maxwell, Aaron E., Timothy A. Warner, and Luis A. Guillén 2021. "Accuracy Assessment in Convolutional Neural Network-Based Deep Learning Remote Sensing Studies—Part 2: Recommendations and Best Practices" Remote Sensing 13, no. 13: 2591. https://doi.org/10.3390/rs13132591

Find Other Styles
Note that from the first issue of 2016, MDPI journals use article numbers instead of page numbers. See further details here.

Article Access Map by Country/Region

1
Back to TopTop