Next Article in Journal
Reversible Data Hiding for AMBTC Compressed Images Based on Matrix and Hamming Coding
Next Article in Special Issue
A Survey on Deep Learning Based Methods and Datasets for Monocular 3D Object Detection
Previous Article in Journal
A New Approach for Power Losses Evaluation of IGBT/Diode Module
Previous Article in Special Issue
Layer-Wise Network Compression Using Gaussian Mixture Model
Article

A Comparative Analysis of Object Detection Metrics with a Companion Open-Source Toolkit

Electrical Engineering Program/Alberto Luiz Coimbra Institute for Post-Graduation and Research in Engineering (PEE/COPPE), PO Box 68504, Rio de Janeiro 21941-972, RJ, Brazil
*
Author to whom correspondence should be addressed.
Academic Editor: Tomasz Trzcinski
Electronics 2021, 10(3), 279; https://doi.org/10.3390/electronics10030279
Received: 25 December 2020 / Revised: 15 January 2021 / Accepted: 20 January 2021 / Published: 25 January 2021
(This article belongs to the Special Issue Deep Learning Based Object Detection)
Recent outstanding results of supervised object detection in competitions and challenges are often associated with specific metrics and datasets. The evaluation of such methods applied in different contexts have increased the demand for annotated datasets. Annotation tools represent the location and size of objects in distinct formats, leading to a lack of consensus on the representation. Such a scenario often complicates the comparison of object detection methods. This work alleviates this problem along the following lines: (i) It provides an overview of the most relevant evaluation methods used in object detection competitions, highlighting their peculiarities, differences, and advantages; (ii) it examines the most used annotation formats, showing how different implementations may influence the assessment results; and (iii) it provides a novel open-source toolkit supporting different annotation formats and 15 performance metrics, making it easy for researchers to evaluate the performance of their detection algorithms in most known datasets. In addition, this work proposes a new metric, also included in the toolkit, for evaluating object detection in videos that is based on the spatio-temporal overlap between the ground-truth and detected bounding boxes. View Full-Text
Keywords: object-detection metrics; precision; recall; evaluation; automatic assessment; bounding boxes object-detection metrics; precision; recall; evaluation; automatic assessment; bounding boxes
Show Figures

Figure 1

MDPI and ACS Style

Padilla, R.; Passos, W.L.; Dias, T.L.B.; Netto, S.L.; da Silva, E.A.B. A Comparative Analysis of Object Detection Metrics with a Companion Open-Source Toolkit. Electronics 2021, 10, 279. https://doi.org/10.3390/electronics10030279

AMA Style

Padilla R, Passos WL, Dias TLB, Netto SL, da Silva EAB. A Comparative Analysis of Object Detection Metrics with a Companion Open-Source Toolkit. Electronics. 2021; 10(3):279. https://doi.org/10.3390/electronics10030279

Chicago/Turabian Style

Padilla, Rafael, Wesley L. Passos, Thadeu L.B. Dias, Sergio L. Netto, and Eduardo A.B. da Silva. 2021. "A Comparative Analysis of Object Detection Metrics with a Companion Open-Source Toolkit" Electronics 10, no. 3: 279. https://doi.org/10.3390/electronics10030279

Find Other Styles
Note that from the first issue of 2016, MDPI journals use article numbers instead of page numbers. See further details here.

Article Access Map by Country/Region

1
Back to TopTop