Next Article in Journal
Effects of Equivalent-Oxide-Thickness and Fin-Width Scaling on In0.53Ga0.47As Tri-Gate Metal-Oxide-Semiconductor-Field-Effect-Transistors with Al2O3/HfO2 for Low-Power Logic Applications
Next Article in Special Issue
An Effective Edge-Assisted Data Collection Approach for Critical Events in the SDWSN-Based Agricultural Internet of Things
Previous Article in Journal
Bulk Bias as an Analog Single-Event Transient Mitigation Technique with Negligible Penalty
Previous Article in Special Issue
Collaborative Computation Offloading and Resource Allocation in Cache-Aided Hierarchical Edge-Cloud Systems
Article

Hardware Resource Analysis in Distributed Training with Edge Devices

1
Department of Computer Science and Engineering, Chungnam National University, Daejeon 34134, Korea
2
Future Computing Research Division, Artificial Intelligence Research Laboratory, Electronics and Telecommunications Research Institute (ETRI), Daejeon 34129, Korea
3
The Division of Computer Convergence, Chungnam National University, Daejeon 34134, Korea
*
Author to whom correspondence should be addressed.
Electronics 2020, 9(1), 28; https://doi.org/10.3390/electronics9010028
Received: 2 December 2019 / Revised: 20 December 2019 / Accepted: 23 December 2019 / Published: 26 December 2019
(This article belongs to the Special Issue Edge Computing in IoT)
When training a deep learning model with distributed training, the hardware resource utilization of each device depends on the model structure and the number of devices used for training. Distributed training has recently been applied to edge computing. Since edge devices have hardware resource limitations such as memory, there is a need for training methods that use hardware resources efficiently. Previous research focused on reducing training time by optimizing the synchronization process between edge devices or by compressing the models. In this paper, we monitored hardware resource usage based on the number of layers and the batch size of the model during distributed training with edge devices. We analyzed memory usage and training time variability as the batch size and number of layers increased. Experimental results demonstrated that, the larger the batch size, the fewer synchronizations between devices, resulting in less accurate training. In the shallow model, training time increased as the number of devices used for training increased because the synchronization between devices took more time than the computation time of training. This paper finds that efficient use of hardware resources for distributed training requires selecting devices in the context of model complexity and that fewer layers and smaller batches are required for efficient hardware use. View Full-Text
Keywords: deep learning; distributed training; edge computing; Internet of Things; performance monitoring deep learning; distributed training; edge computing; Internet of Things; performance monitoring
Show Figures

Figure 1

MDPI and ACS Style

Park, S.; Lee, J.; Kim, H. Hardware Resource Analysis in Distributed Training with Edge Devices. Electronics 2020, 9, 28. https://doi.org/10.3390/electronics9010028

AMA Style

Park S, Lee J, Kim H. Hardware Resource Analysis in Distributed Training with Edge Devices. Electronics. 2020; 9(1):28. https://doi.org/10.3390/electronics9010028

Chicago/Turabian Style

Park, Sihyeong, Jemin Lee, and Hyungshin Kim. 2020. "Hardware Resource Analysis in Distributed Training with Edge Devices" Electronics 9, no. 1: 28. https://doi.org/10.3390/electronics9010028

Find Other Styles
Note that from the first issue of 2016, MDPI journals use article numbers instead of page numbers. See further details here.

Article Access Map by Country/Region

1
Back to TopTop