Next Article in Journal
The Calibration of Displacement Sensors
Next Article in Special Issue
Identification of Road-Surface Type Using Deep Neural Networks for Friction Coefficient Estimation
Previous Article in Journal
Chemical Gas Sensors Studied at SENSOR Lab, Brescia (Italy): From Conventional to Energy-Efficient and Biocompatible Composite Structures
Previous Article in Special Issue
Finger Gesture Spotting from Long Sequences Based on Multi-Stream Recurrent Neural Networks
Open AccessArticle

Recognizing New Classes with Synthetic Data in the Loop: Application to Traffic Sign Recognition

1
Computer Vision Center (CVC), Universitat Autònoma de Barcelona (UAB), 08193 Bellaterra, Spain
2
Computer Science Department, Universitat Autònoma de Barcelona (UAB), 08193 Bellaterra, Spain
*
Author to whom correspondence should be addressed.
Sensors 2020, 20(3), 583; https://doi.org/10.3390/s20030583
Received: 20 December 2019 / Revised: 10 January 2020 / Accepted: 16 January 2020 / Published: 21 January 2020
(This article belongs to the Special Issue Advance in Sensors and Sensing Systems for Driving and Transport)
On-board vision systems may need to increase the number of classes that can be recognized in a relatively short period. For instance, a traffic sign recognition system may suddenly be required to recognize new signs. Since collecting and annotating samples of such new classes may need more time than we wish, especially for uncommon signs, we propose a method to generate these samples by combining synthetic images and Generative Adversarial Network (GAN) technology. In particular, the GAN is trained on synthetic and real-world samples from known classes to perform synthetic-to-real domain adaptation, but applied to synthetic samples of the new classes. Using the Tsinghua dataset with a synthetic counterpart, SYNTHIA-TS, we have run an extensive set of experiments. The results show that the proposed method is indeed effective, provided that we use a proper Convolutional Neural Network (CNN) to perform the traffic sign recognition (classification) task as well as a proper GAN to transform the synthetic images. Here, a ResNet101-based classifier and domain adaptation based on CycleGAN performed extremely well for a ratio 1 / 4 for new/known classes; even for more challenging ratios such as 4 / 1 , the results are also very positive. View Full-Text
Keywords: CNNs; training with synthetic data; traffic sign recognition CNNs; training with synthetic data; traffic sign recognition
Show Figures

Figure 1

MDPI and ACS Style

Villalonga, G.; Van de Weijer, J.; López, A.M. Recognizing New Classes with Synthetic Data in the Loop: Application to Traffic Sign Recognition. Sensors 2020, 20, 583.

Show more citation formats Show less citations formats
Note that from the first issue of 2016, MDPI journals use article numbers instead of page numbers. See further details here.

Article Access Map by Country/Region

1
Back to TopTop