Search for Articles:
Title / Keyword
Author / Affiliation / Email
Journal
Article Type
 
 
Advanced Search
 
Section
Special Issue
Volume
Issue
Number
Page
 
4.7
2.9
Journals
Electronics
Special Issues
Important Features Selection in Deep Neural Networks
share announcement

Important Features Selection in Deep Neural Networks

A special issue of Electronics (ISSN 2079-9292). This special issue belongs to the section "Computer Science & Engineering".

Deadline for manuscript submissions: closed (31 October 2022) | Viewed by 5771

Special Issue Editors

Dr. Matei Mancas

E-Mail Website
Guest Editor
Numediart Institute of Creative Technologies, University of Mons (UMONS), 7000 Mons, Belgium
Interests: prediction, analysis and trigger of human attention
Special Issues, Collections and Topics in MDPI journals
Dr. Jean-Benoit Delbrouck

E-Mail Website
Guest Editor
Department of Biomedical Data Science Stanford University, Stanford, CA 94305, USA
Interests: attention models within complex DNN architectures
Prof. Dr. Sidi Ahmed Mahmoudi

E-Mail Website
Guest Editor
Faculty of Engineering, University of Mons, 7000 Mons, Belgium
Interests: artificial intelligence; explainable artificial intelligence; machine and deep learning; edge artificial intelligence; multimedia processing; high-performance computing; cloud and edge computing
Special Issues, Collections and Topics in MDPI journals

Special Issue Information

Dear Colleagues,

The development of deep learning architectures continues to open up new fields of applications every day. However, several drawbacks of deep learning models are now well-known, including 1) the fact that the results are not easy to explain and 2) the large memory size needed for the models, which usually implies high computational costs for the results’ inference.

For this Special Issue, we invite submissions from researchers addressing feature selection within DNN architectures with techniques such as pruning or attention modules, with a focus on the data that are the most relevant for the ongoing task. We encourage authors to submit papers within different domains, such as NLP, computer vision, or multimedia frameworks. We also invite submissions from researchers studying DNN model compression to obtain lighter and faster models for edge computing while maintaining performance, in addition to researchers studying DNN attention models to better understand and explain the models’ decisions or to obtain more accurate results by focusing the models on the most interesting data for a given application.

This Special Issue aims to cover recent advances in DNN architecture compression, attention-based optimization, and understanding the model results. Reviews and surveys of the state-of-the-art DNN compression and attention models are also welcomed.

The topics of interest for this Special Issue include:

  • DNN compression;
  • DNN pruning;
  • Attention-based modules;
  • Transformers;
  • Saliency-based explainability;
  • Attention visualization;
  • NLP and speech processing;
  • Computer vision;
  • Edge computing;
  • Deployment and migration of AI applications on cloud/edge platforms;
  • Explainable artificial intelligence, "XAI";
  • Distributed and embedded deep learning.

However, please do not feel limited by these topics; we will consider submissions in any area of feature selection in DNN architectures. The Special Issue is linked to the TRAIL Institute for AI, Belgium, but is open to any submission.

Dr. Matei Mancas
Dr. Jean-Benoit Delbrouck
Dr. Sidi Ahmed Mahmoudi
Guest Editors

Manuscript Submission Information

Manuscripts should be submitted online at www.mdpi.com by registering and logging in to this website. Once you are registered, click here to go to the submission form. Manuscripts can be submitted until the deadline. All submissions that pass pre-check are peer-reviewed. Accepted papers will be published continuously in the journal (as soon as accepted) and will be listed together on the special issue website. Research articles, review articles as well as short communications are invited. For planned papers, a title and short abstract (about 100 words) can be sent to the Editorial Office for announcement on this website.

Submitted manuscripts should not have been published previously, nor be under consideration for publication elsewhere (except conference proceedings papers). All manuscripts are thoroughly refereed through a single-blind peer-review process. A guide for authors and other relevant information for submission of manuscripts is available on the Instructions for Authors page. Electronics is an international peer-reviewed open access semimonthly journal published by MDPI.

Please visit the Instructions for Authors page before submitting a manuscript. The Article Processing Charge (APC) for publication in this open access journal is 2400 CHF (Swiss Francs). Submitted papers should be well formatted and use good English. Authors may use MDPI's English editing service prior to publication or during author revisions.

Keywords

  • deep-learning
  • DNNs
  • attention modules
  • deep architecture pruning
  • attention modules
  • saliency
  • explainability
  • deep model compression

Published Papers (3 papers)

Download All Papers
Order results
Result details
Show export options expand_more Show export options expand_less
Select all
Export citation of selected articles as:

Research

19 pages, 3401 KiB  
Article
FasterAI: A Lightweight Library for Neural Networks Compression
by Nathan Hubens, Matei Mancas, Bernard Gosselin, Marius Preda and Titus Zaharia
Electronics 2022, 11(22), 3789; https://doi.org/10.3390/electronics11223789 - 18 Nov 2022
Viewed by 1050
Abstract
FasterAI is a PyTorch-based library, aiming to facilitate the use of deep neural network compression techniques, such as sparsification, pruning, knowledge distillation, or regularization. The library is built with the purpose of enabling quick implementation and experimentation. More particularly, compression techniques are leveraging [...] Read more.
FasterAI is a PyTorch-based library, aiming to facilitate the use of deep neural network compression techniques, such as sparsification, pruning, knowledge distillation, or regularization. The library is built with the purpose of enabling quick implementation and experimentation. More particularly, compression techniques are leveraging callback systems of libraries, such as fastai and Pytorch Lightning to propose a user-friendly and high-level API. The main asset of FasterAI is its lightweight, yet powerful, simplicity of use. Indeed, because it has been developed in a very granular way, users can create thousands of unique experiments by using different combinations of parameters, with only a single line of additional code. This allows FasterAI to be suited for practical usage, as it contains the most common compression techniques available out-of-the-box, but also for research, as implementing a new compression technique usually boils down to writing a single line of code. In this paper, we propose an in-depth presentation of the different compression techniques available in FasterAI. As a proof of concept and to better grasp how the library is used, we present results achieved by applying each technique on a ResNet-18 architecture, trained on CALTECH-101. Full article
(This article belongs to the Special Issue Important Features Selection in Deep Neural Networks)
►▼ Show Figures

Figure 1

15 pages, 11097 KiB  
Article
Document-Level Sentiment Analysis Using Attention-Based Bi-Directional Long Short-Term Memory Network and Two-Dimensional Convolutional Neural Network
by Yanying Mao, Yu Zhang, Liudan Jiao and Heshan Zhang
Electronics 2022, 11(12), 1906; https://doi.org/10.3390/electronics11121906 - 18 Jun 2022
Cited by 7 | Viewed by 2364
Abstract
Due to outstanding feature extraction ability, neural networks have recently achieved great success in sentiment analysis. However, one of the remaining challenges of sentiment analysis is to model long texts to consider the intrinsic relations between two sentences in the semantic meaning of [...] Read more.
Due to outstanding feature extraction ability, neural networks have recently achieved great success in sentiment analysis. However, one of the remaining challenges of sentiment analysis is to model long texts to consider the intrinsic relations between two sentences in the semantic meaning of a document. Moreover, most existing methods are not powerful enough to differentiate the importance of different document features. To address these problems, this paper proposes a new neural network model: AttBiLSTM-2DCNN, which entails two perspectives. First, a two-layer, bidirectional long short-term memory (BiLSTM) network is utilized to obtain the sentiment semantics of a document. The first BiLSTM layer learns the sentiment semantic representation from both directions of a sentence, and the second BiLSTM layer is used to encode the intrinsic relations of sentences into the document matrix representation with a feature dimension and a time-step dimension. Second, a two-dimensional convolutional neural network (2DCNN) is employed to obtain more sentiment dependencies between two sentences. Third, we utilize a two-layer attention mechanism to distinguish the importance of words and sentences in the document. Last, to validate the model, we perform an experiment on two public review datasets that are derived from Yelp2015 and IMDB. Accuracy, F1-Measure, and MSE are used as evaluation metrics. The experimental results show that our model can not only capture sentimental relations but also outperform certain state-of-the-art models. Full article
(This article belongs to the Special Issue Important Features Selection in Deep Neural Networks)
►▼ Show Figures

Figure 1

23 pages, 5201 KiB  
Article
DeepRare: Generic Unsupervised Visual Attention Models
by Phutphalla Kong, Matei Mancas, Bernard Gosselin and Kimtho Po
Electronics 2022, 11(11), 1696; https://doi.org/10.3390/electronics11111696 - 26 May 2022
Viewed by 1508
Abstract
Visual attention selects data considered as “interesting” by humans, and it is modeled in the field of engineering by feature-engineered methods finding contrasted/surprising/unusual image data. Deep learning drastically improved the models efficiency on the main benchmark datasets. However, Deep Neural Networks-based (DNN-based) models [...] Read more.
Visual attention selects data considered as “interesting” by humans, and it is modeled in the field of engineering by feature-engineered methods finding contrasted/surprising/unusual image data. Deep learning drastically improved the models efficiency on the main benchmark datasets. However, Deep Neural Networks-based (DNN-based) models are counterintuitive: surprising or unusual data are by definition difficult to learn because of their low occurrence probability. In reality, DNN-based models mainly learn top-down features such as faces, text, people, or animals which usually attract human attention, but they have low efficiency in extracting surprising or unusual data in the images. In this article, we propose a new family of visual attention models called DeepRare and especially DeepRare2021 (DR21), which uses the power of DNNs’ feature extraction and the genericity of feature-engineered algorithms. This algorithm is an evolution of a previous version called DeepRare2019 (DR19) based on this common framework. DR21 (1) does not need any additional training other than the default ImageNet training, (2) is fast even on CPU, (3) is tested on four very different eye-tracking datasets showing that DR21 is generic and is always within the top models on all datasets and metrics while no other model exhibits such a regularity and genericity. Finally, DR21 (4) is tested with several network architectures such as VGG16 (V16), VGG19 (V19), and MobileNetV2 (MN2), and (5) it provides explanation and transparency on which parts of the image are the most surprising at different levels despite the use of a DNN-based feature extractor. Full article
(This article belongs to the Special Issue Important Features Selection in Deep Neural Networks)
►▼ Show Figures

Figure 1

Show export options expand_more Show export options expand_less
Select all
Export citation of selected articles as:
 
Displaying articles 1-3
Back to TopTop