Next Article in Journal
CF-CNN: Coarse-to-Fine Convolutional Neural Network
Next Article in Special Issue
Driving Behavior Classification and Sharing System Using CNN-LSTM Approaches and V2X Communication
Previous Article in Journal
Sustained Antibacterial Effect and Wear Behavior of Quaternary Ammonium Contact-Killing Dental Polymers after One-Year of Hydrolytic Degradation
Previous Article in Special Issue
Path Planning Based on Obstacle-Dependent Gaussian Model Predictive Control for Autonomous Driving
Article

Adversarial Optimization-Based Knowledge Transfer of Layer-Wise Dense Flow for Image Classification

by 1, 2,* and 3,*
1
KSB Convergence Research Department, Electronics and Telecommunications Research Institute, Daejeon 34129, Korea
2
Department of Human Intelligence and Robot Engineering, Sangmyung University, Cheonan 03016, Korea
3
Department of AI and Big Data Engineering, Daegu Catholic University, Gyeongsan-si 38430, Korea
*
Authors to whom correspondence should be addressed.
Academic Editor: Myo-Taeg Lim
Appl. Sci. 2021, 11(8), 3720; https://doi.org/10.3390/app11083720
Received: 21 March 2021 / Revised: 16 April 2021 / Accepted: 19 April 2021 / Published: 20 April 2021
A deep-learning technology for knowledge transfer is necessary to advance and optimize efficient knowledge distillation. Here, we aim to develop a new adversarial optimization-based knowledge transfer method involved with a layer-wise dense flow that is distilled from a pre-trained deep neural network (DNN). Knowledge distillation transferred to another target DNN based on adversarial loss functions has multiple flow-based knowledge items that are densely extracted by overlapping them from a pre-trained DNN to enhance the existing knowledge. We propose a semi-supervised learning-based knowledge transfer with multiple items of dense flow-based knowledge extracted from the pre-trained DNN. The proposed loss function would comprise a supervised cross-entropy loss for a typical classification, an adversarial training loss for the target DNN and discriminators, and Euclidean distance-based loss in terms of dense flow. For both pre-trained and target DNNs considered in this study, we adopt a residual network (ResNet) architecture. We propose methods of (1) the adversarial-based knowledge optimization, (2) the extended and flow-based knowledge transfer scheme, and (3) the combined layer-wise dense flow in an adversarial network. The results show that it provides higher accuracy performance in the improved target ResNet compared to the prior knowledge transfer methods. View Full-Text
Keywords: adversarial optimization; layer-wise dense flow; knowledge transfer; image classification adversarial optimization; layer-wise dense flow; knowledge transfer; image classification
Show Figures

Figure 1

MDPI and ACS Style

Yeo, D.; Kim, M.-S.; Bae, J.-H. Adversarial Optimization-Based Knowledge Transfer of Layer-Wise Dense Flow for Image Classification. Appl. Sci. 2021, 11, 3720. https://doi.org/10.3390/app11083720

AMA Style

Yeo D, Kim M-S, Bae J-H. Adversarial Optimization-Based Knowledge Transfer of Layer-Wise Dense Flow for Image Classification. Applied Sciences. 2021; 11(8):3720. https://doi.org/10.3390/app11083720

Chicago/Turabian Style

Yeo, Doyeob, Min-Suk Kim, and Ji-Hoon Bae. 2021. "Adversarial Optimization-Based Knowledge Transfer of Layer-Wise Dense Flow for Image Classification" Applied Sciences 11, no. 8: 3720. https://doi.org/10.3390/app11083720

Find Other Styles
Note that from the first issue of 2016, MDPI journals use article numbers instead of page numbers. See further details here.

Article Access Map by Country/Region

1
Back to TopTop