Fully Automated DCNN-Based Thermal Images Annotation Using Neural Network Pretrained on RGB Data
1
Robotics and AI Research Group, Faculty of Electrical Engineering, Brno University of Technology, 61600 Brno, Czech Republic
2
Cybernetics and Robotics Research Group, Central European Institute of Technology, Brno University of Technology, 61600 Brno, Czech Republic
3
Artificial Intelligence and Vision Research Group, Department of Computer Science, Tampere University, 33101 Tampere, Finland
*
Author to whom correspondence should be addressed.
Academic Editor: Frantisek Duchon
Sensors 2021, 21(4), 1552; https://doi.org/10.3390/s21041552
Received: 22 January 2021 / Revised: 17 February 2021 / Accepted: 19 February 2021 / Published: 23 February 2021
(This article belongs to the Special Issue Sensors and Data Processing in Robotics)
One of the biggest challenges of training deep neural network is the need for massive data annotation. To train the neural network for object detection, millions of annotated training images are required. However, currently, there are no large-scale thermal image datasets that could be used to train the state of the art neural networks, while voluminous RGB image datasets are available. This paper presents a method that allows to create hundreds of thousands of annotated thermal images using the RGB pre-trained object detector. A dataset created in this way can be used to train object detectors with improved performance. The main gain of this work is the novel method for fully automatic thermal image labeling. The proposed system uses the RGB camera, thermal camera, 3D LiDAR, and the pre-trained neural network that detects objects in the RGB domain. Using this setup, it is possible to run the fully automated process that annotates the thermal images and creates the automatically annotated thermal training dataset. As the result, we created a dataset containing hundreds of thousands of annotated objects. This approach allows to train deep learning models with similar performance as the common human-annotation-based methods do. This paper also proposes several improvements to fine-tune the results with minimal human intervention. Finally, the evaluation of the proposed solution shows that the method gives significantly better results than training the neural network with standard small-scale hand-annotated thermal image datasets.
View Full-Text
Keywords:
deep convolutional neural networks; transfer learning; YOLO; RGB; IR; thermal; data annotation; object detector
▼
Show Figures
This is an open access article distributed under the Creative Commons Attribution License which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited
MDPI and ACS Style
Ligocki, A.; Jelinek, A.; Zalud, L.; Rahtu, E. Fully Automated DCNN-Based Thermal Images Annotation Using Neural Network Pretrained on RGB Data. Sensors 2021, 21, 1552. https://doi.org/10.3390/s21041552
AMA Style
Ligocki A, Jelinek A, Zalud L, Rahtu E. Fully Automated DCNN-Based Thermal Images Annotation Using Neural Network Pretrained on RGB Data. Sensors. 2021; 21(4):1552. https://doi.org/10.3390/s21041552
Chicago/Turabian StyleLigocki, Adam; Jelinek, Ales; Zalud, Ludek; Rahtu, Esa. 2021. "Fully Automated DCNN-Based Thermal Images Annotation Using Neural Network Pretrained on RGB Data" Sensors 21, no. 4: 1552. https://doi.org/10.3390/s21041552
Find Other Styles
Note that from the first issue of 2016, MDPI journals use article numbers instead of page numbers. See further details here.
Search more from Scilit