Next Article in Journal
MEDUSA: A Low-Cost, 16-Channel Neuromodulation Platform with Arbitrary Waveform Generation
Next Article in Special Issue
An Advanced CNN-LSTM Model for Cryptocurrency Forecasting
Previous Article in Journal
SR-SYBA: A Scale and Rotation Invariant Synthetic Basis Feature Descriptor with Low Memory Usage
Previous Article in Special Issue
sDeepFM: Multi-Scale Stacking Feature Interactions for Click-Through Rate Prediction
Article

An Advanced Pruning Method in the Architecture of Extreme Learning Machines Using L1-Regularization and Bootstrapping

1
JKU-Department of Knowledge-Based Mathematical Systems, Johannes Kepler University Linz, 4203 Linz, Austria
2
UFOP-Department of Computing and Systems, Federal University of Ouro Preto, Rua 36, 115-Loanda, João Monlevade, Minas Geras 30360-740, Brazil
3
UFMG-Av. Antônio Carlos 6627, Federal University of Minas Gerais, Belo Horizonte, Minas Gerais 31270-901, Brazil
*
Author to whom correspondence should be addressed.
Electronics 2020, 9(5), 811; https://doi.org/10.3390/electronics9050811
Received: 15 April 2020 / Revised: 8 May 2020 / Accepted: 8 May 2020 / Published: 15 May 2020
(This article belongs to the Special Issue Regularization Techniques for Machine Learning and Their Applications)
Extreme learning machines (ELMs) are efficient for classification, regression, and time series prediction, as well as being a clear solution to backpropagation structures to determine values in intermediate layers of the learning model. One of the problems that an ELM may face is due to a large number of neurons in the hidden layer, making the expert model a specific data set. With a large number of neurons in the hidden layer, overfitting is more likely and thus unnecessary information can deterioriate the performance of the neural network. To solve this problem, a pruning method is proposed, called Pruning ELM Using Bootstrapped Lasso BR-ELM, which is based on regularization and resampling techniques, to select the most representative neurons for the model response. This method is based on an ensembled variant of Lasso (achieved through bootstrap replications) and aims to shrink the output weight parameters of the neurons to 0 as many and as much as possible. According to a subset of candidate regressors having significant coefficient values (greater than 0), it is possible to select the best neurons in the hidden layer of the ELM. Finally, pattern classification tests and benchmark regression tests of complex real-world problems are performed by comparing the proposed approach to other pruning models for ELMs. It can be seen that statistically BR-ELM can outperform several related state-of-the-art methods in terms of classification accuracies and model errors (while performing equally to Pruning-ELM P-ELM), and this with a significantly reduced number of finally selected neurons. View Full-Text
Keywords: extreme learning machine; lasso with bootstrapping; pruning of neurons; least angle regression extreme learning machine; lasso with bootstrapping; pruning of neurons; least angle regression
Show Figures

Figure 1

MDPI and ACS Style

de Campos Souza, P.V.; Bambirra Torres, L.C.; Lacerda Silva, G.R.; Braga, A.d.P.; Lughofer, E. An Advanced Pruning Method in the Architecture of Extreme Learning Machines Using L1-Regularization and Bootstrapping. Electronics 2020, 9, 811. https://doi.org/10.3390/electronics9050811

AMA Style

de Campos Souza PV, Bambirra Torres LC, Lacerda Silva GR, Braga AdP, Lughofer E. An Advanced Pruning Method in the Architecture of Extreme Learning Machines Using L1-Regularization and Bootstrapping. Electronics. 2020; 9(5):811. https://doi.org/10.3390/electronics9050811

Chicago/Turabian Style

de Campos Souza, Paulo V., Luiz C. Bambirra Torres, Gustavo R. Lacerda Silva, Antonio d.P. Braga, and Edwin Lughofer. 2020. "An Advanced Pruning Method in the Architecture of Extreme Learning Machines Using L1-Regularization and Bootstrapping" Electronics 9, no. 5: 811. https://doi.org/10.3390/electronics9050811

Find Other Styles
Note that from the first issue of 2016, MDPI journals use article numbers instead of page numbers. See further details here.

Article Access Map by Country/Region

1
Back to TopTop