Next Article in Journal
A 3D Cascaded Spectral–Spatial Element Attention Network for Hyperspectral Image Classification
Next Article in Special Issue
Accuracy Assessment in Convolutional Neural Network-Based Deep Learning Remote Sensing Studies—Part 2: Recommendations and Best Practices
Previous Article in Journal
Insights from the P Wave Travel Time Tomography in the Upper Mantle Beneath the Central Philippines
Review

Accuracy Assessment in Convolutional Neural Network-Based Deep Learning Remote Sensing Studies—Part 1: Literature Review

Department of Geology and Geography, West Virginia University, Morgantown, WV 26505, USA
*
Author to whom correspondence should be addressed.
These authors contributed equally to this work.
Academic Editors: Kuo-Chin Fan and Ying-Nong Chen
Remote Sens. 2021, 13(13), 2450; https://doi.org/10.3390/rs13132450
Received: 1 June 2021 / Revised: 18 June 2021 / Accepted: 21 June 2021 / Published: 23 June 2021
Convolutional neural network (CNN)-based deep learning (DL) is a powerful, recently developed image classification approach. With origins in the computer vision and image processing communities, the accuracy assessment methods developed for CNN-based DL use a wide range of metrics that may be unfamiliar to the remote sensing (RS) community. To explore the differences between traditional RS and DL RS methods, we surveyed a random selection of 100 papers from the RS DL literature. The results show that RS DL studies have largely abandoned traditional RS accuracy assessment terminology, though some of the accuracy measures typically used in DL papers, most notably precision and recall, have direct equivalents in traditional RS terminology. Some of the DL accuracy terms have multiple names, or are equivalent to another measure. In our sample, DL studies only rarely reported a complete confusion matrix, and when they did so, it was even more rare that the confusion matrix estimated population properties. On the other hand, some DL studies are increasingly paying attention to the role of class prevalence in designing accuracy assessment approaches. DL studies that evaluate the decision boundary threshold over a range of values tend to use the precision-recall (P-R) curve, the associated area under the curve (AUC) measures of average precision (AP) and mean average precision (mAP), rather than the traditional receiver operating characteristic (ROC) curve and its AUC. DL studies are also notable for testing the generalization of their models on entirely new datasets, including data from new areas, new acquisition times, or even new sensors. View Full-Text
Keywords: accuracy assessment; thematic mapping; feature extraction; object detection; semantic segmentation; instance segmentation; deep learning accuracy assessment; thematic mapping; feature extraction; object detection; semantic segmentation; instance segmentation; deep learning
Show Figures

Graphical abstract

MDPI and ACS Style

Maxwell, A.E.; Warner, T.A.; Guillén, L.A. Accuracy Assessment in Convolutional Neural Network-Based Deep Learning Remote Sensing Studies—Part 1: Literature Review. Remote Sens. 2021, 13, 2450. https://doi.org/10.3390/rs13132450

AMA Style

Maxwell AE, Warner TA, Guillén LA. Accuracy Assessment in Convolutional Neural Network-Based Deep Learning Remote Sensing Studies—Part 1: Literature Review. Remote Sensing. 2021; 13(13):2450. https://doi.org/10.3390/rs13132450

Chicago/Turabian Style

Maxwell, Aaron E., Timothy A. Warner, and Luis A. Guillén 2021. "Accuracy Assessment in Convolutional Neural Network-Based Deep Learning Remote Sensing Studies—Part 1: Literature Review" Remote Sensing 13, no. 13: 2450. https://doi.org/10.3390/rs13132450

Find Other Styles
Note that from the first issue of 2016, MDPI journals use article numbers instead of page numbers. See further details here.

Article Access Map by Country/Region

1
Back to TopTop