Innovative Topologies and Algorithms for Neural Networks

A special issue of Future Internet (ISSN 1999-5903).

Deadline for manuscript submissions: closed (30 November 2019) | Viewed by 65370

Printed Edition Available!
A printed edition of this Special Issue is available here.

Special Issue Editors

Department of Electrical, Electronics and Computer Engineering (DIEEI), University of Catania, Viale Andrea Doria 6, 95125 Catania, Italy
Interests: artificial intelligence; neural networks; soft sensors; ionic polymeric transducers; sensor modelling and characterization; mechanical sensors; energy harvesting; smart materials; smart sensing systems
Special Issues, Collections and Topics in MDPI journals

Special Issue Information

Dear Colleagues,

The introduction of deep neural networks and new training procedures and topologies has solicited a renewed interest in the field of neural computation. Consequently, the use of deep structures has significantly improved the state-of-the-art in many application fields, such as computer vision, speech, language processing, soft sensors, and IoT, just to mention a few.

The chances of success of a neural network are largely linked to the adequate choice of the network architecture and the training algorithm. As a matter of fact, much of the recent interest on neural networks has been devoted to propose and study novel architectures, including solutions tailored to specific problems. Nevertheless, the choice of the structure, the network sizing, and hyperparameters selection, still are the subject of a vivid research interest.

The aim of this Special Issue is to bring together developments in the field of innovative topologies and algorithms for neural networks, along with applications relevant, but not limited, to industry and the IoT. Papers addressing the wide range of aspects of this technology are sought, including, but not limited to:

  • Deep Belief Networks
  • Autoencoders
  • Long- Short-Term Memory networks
  • Convolution Neural Networks
  • Hierarchical-Deep models
  • Other innovative structures
  • Supervised learning
  • Semi-supervised learning
  • Unsupervised learning
  • Ensemble learning
  • Network structure selection and validation
  • Neural networks for Data Fusion
  • Computational intelligence methods for industrial applications
  • Soft Sensors
  • Neural Networks for the IoT and IIoT
Prof. Maria Gabriella Xibilia
Prof. Salvatore Graziani
Guest Editors

Manuscript Submission Information

Manuscripts should be submitted online at www.mdpi.com by registering and logging in to this website. Once you are registered, click here to go to the submission form. Manuscripts can be submitted until the deadline. All submissions that pass pre-check are peer-reviewed. Accepted papers will be published continuously in the journal (as soon as accepted) and will be listed together on the special issue website. Research articles, review articles as well as short communications are invited. For planned papers, a title and short abstract (about 100 words) can be sent to the Editorial Office for announcement on this website.

Submitted manuscripts should not have been published previously, nor be under consideration for publication elsewhere (except conference proceedings papers). All manuscripts are thoroughly refereed through a single-blind peer-review process. A guide for authors and other relevant information for submission of manuscripts is available on the Instructions for Authors page. Future Internet is an international peer-reviewed open access monthly journal published by MDPI.

Please visit the Instructions for Authors page before submitting a manuscript. The Article Processing Charge (APC) for publication in this open access journal is 1600 CHF (Swiss Francs). Submitted papers should be well formatted and use good English. Authors may use MDPI's English editing service prior to publication or during author revisions.

Keywords

  • neural Networks
  • network topology
  • network structure
  • network learning
  • learning algorithms
  • IoT
  • IIoT
  • network selection
  • industrial applications
  • Soft Sensors
  • Data Fusion

Published Papers (13 papers)

Order results
Result details
Select all
Export citation of selected articles as:

Editorial

Jump to: Research, Other

4 pages, 163 KiB  
Editorial
Innovative Topologies and Algorithms for Neural Networks
by Salvatore Graziani and Maria Gabriella Xibilia
Future Internet 2020, 12(7), 117; https://doi.org/10.3390/fi12070117 - 11 Jul 2020
Cited by 2 | Viewed by 2421
Abstract
The introduction of new topologies and training procedures to deep neural networks has solicited a renewed interest in the field of neural computation. The use of deep structures has significantly improved the state of the art in many applications, such as computer vision, [...] Read more.
The introduction of new topologies and training procedures to deep neural networks has solicited a renewed interest in the field of neural computation. The use of deep structures has significantly improved the state of the art in many applications, such as computer vision, speech and text processing, medical applications, and IoT (Internet of Things). The probability of a successful outcome from a neural network is linked to selection of an appropriate network architecture and training algorithm. Accordingly, much of the recent research on neural networks is devoted to the study and proposal of novel architectures, including solutions tailored to specific problems. The papers of this Special Issue make significant contributions to the above-mentioned fields by merging theoretical aspects and relevant applications. Twelve papers are collected in the issue, addressing many relevant aspects of the topic. Full article
(This article belongs to the Special Issue Innovative Topologies and Algorithms for Neural Networks)

Research

Jump to: Editorial, Other

13 pages, 1770 KiB  
Article
Pedestrian Attribute Recognition with Graph Convolutional Network in Surveillance Scenarios
by Xiangpeng Song, Hongbin Yang and Congcong Zhou
Future Internet 2019, 11(11), 245; https://doi.org/10.3390/fi11110245 - 19 Nov 2019
Cited by 9 | Viewed by 3797
Abstract
Pedestrian attribute recognition is to predict a set of attribute labels of the pedestrian from surveillance scenarios, which is a very challenging task for computer vision due to poor image quality, continual appearance variations, as well as diverse spatial distribution of imbalanced attributes. [...] Read more.
Pedestrian attribute recognition is to predict a set of attribute labels of the pedestrian from surveillance scenarios, which is a very challenging task for computer vision due to poor image quality, continual appearance variations, as well as diverse spatial distribution of imbalanced attributes. It is desirable to model the label dependencies between different attributes to improve the recognition performance as each pedestrian normally possesses many attributes. In this paper, we treat pedestrian attribute recognition as multi-label classification and propose a novel model based on the graph convolutional network (GCN). The model is mainly divided into two parts, we first use convolutional neural network (CNN) to extract pedestrian feature, which is a normal operation processing image in deep learning, then we transfer attribute labels to word embedding and construct a correlation matrix between labels to help GCN propagate information between nodes. This paper applies the object classifiers learned by GCN to the image representation extracted by CNN to enable the model to have the ability to be end-to-end trainable. Experiments on pedestrian attribute recognition dataset show that the approach obviously outperforms other existing state-of-the-art methods. Full article
(This article belongs to the Special Issue Innovative Topologies and Algorithms for Neural Networks)
Show Figures

Figure 1

30 pages, 1070 KiB  
Article
Partitioning Convolutional Neural Networks to Maximize the Inference Rate on Constrained IoT Devices
by Fabíola Martins Campos de Oliveira and Edson Borin
Future Internet 2019, 11(10), 209; https://doi.org/10.3390/fi11100209 - 29 Sep 2019
Cited by 18 | Viewed by 4850
Abstract
Billions of devices will compose the IoT system in the next few years, generating a huge amount of data. We can use fog computing to process these data, considering that there is the possibility of overloading the network towards the cloud. In this [...] Read more.
Billions of devices will compose the IoT system in the next few years, generating a huge amount of data. We can use fog computing to process these data, considering that there is the possibility of overloading the network towards the cloud. In this context, deep learning can treat these data, but the memory requirements of deep neural networks may prevent them from executing on a single resource-constrained device. Furthermore, their computational requirements may yield an unfeasible execution time. In this work, we propose Deep Neural Networks Partitioning for Constrained IoT Devices, a new algorithm to partition neural networks for efficient distributed execution. Our algorithm can optimize the neural network inference rate or the number of communications among devices. Additionally, our algorithm accounts appropriately for the shared parameters and biases of Convolutional Neural Networks. We investigate the inference rate maximization for the LeNet model in constrained setups. We show that the partitionings offered by popular machine learning frameworks such as TensorFlow or by the general-purpose framework METIS may produce invalid partitionings for very constrained setups. The results show that our algorithm can partition LeNet for all the proposed setups, yielding up to 38% more inferences per second than METIS. Full article
(This article belongs to the Special Issue Innovative Topologies and Algorithms for Neural Networks)
Show Figures

Figure 1

15 pages, 1675 KiB  
Article
An Improved Approach for Text Sentiment Classification Based on a Deep Neural Network via a Sentiment Attention Mechanism
by Wenkuan Li, Peiyu Liu, Qiuyue Zhang and Wenfeng Liu
Future Internet 2019, 11(4), 96; https://doi.org/10.3390/fi11040096 - 11 Apr 2019
Cited by 26 | Viewed by 4903
Abstract
Text sentiment analysis is an important but challenging task. Remarkable success has been achieved along with the wide application of deep learning methods, but deep learning methods dealing with text sentiment classification tasks cannot fully exploit sentiment linguistic knowledge, which hinders the development [...] Read more.
Text sentiment analysis is an important but challenging task. Remarkable success has been achieved along with the wide application of deep learning methods, but deep learning methods dealing with text sentiment classification tasks cannot fully exploit sentiment linguistic knowledge, which hinders the development of text sentiment analysis. In this paper, we propose a sentiment-feature-enhanced deep neural network (SDNN) to address the problem by integrating sentiment linguistic knowledge into a deep neural network via a sentiment attention mechanism. Specifically, first we introduce a novel sentiment attention mechanism to help select the crucial sentiment-word-relevant context words by leveraging the sentiment lexicon in an attention mechanism, which bridges the gap between traditional sentiment linguistic knowledge and current popular deep learning methods. Second, we develop an improved deep neural network to extract sequential correlation information and text local features by combining bidirectional gated recurrent units with a convolutional neural network, which further enhances the ability of comprehensive text representation learning. With this design, the SDNN model can generate a powerful semantic representation of text to improve the performance of text sentiment classification tasks. Extensive experiments were conducted to evaluate the effectiveness of the proposed SDNN model on two real-world datasets with a binary-sentiment-label and a multi-sentiment-label. The experimental results demonstrated that the SDNN achieved substantially better performance than the strong competitors for text sentiment classification tasks. Full article
(This article belongs to the Special Issue Innovative Topologies and Algorithms for Neural Networks)
Show Figures

Figure 1

11 pages, 791 KiB  
Article
Dynamic Gesture Recognition Based on MEMP Network
by Xinyu Zhang and Xiaoqiang Li
Future Internet 2019, 11(4), 91; https://doi.org/10.3390/fi11040091 - 03 Apr 2019
Cited by 23 | Viewed by 4803
Abstract
In recent years, gesture recognition has been used in many fields, such as games, robotics and sign language recognition. Human computer interaction (HCI) has been significantly improved by the development of gesture recognition, and now gesture recognition in video is an important research [...] Read more.
In recent years, gesture recognition has been used in many fields, such as games, robotics and sign language recognition. Human computer interaction (HCI) has been significantly improved by the development of gesture recognition, and now gesture recognition in video is an important research direction. Because each kind of neural network structure has its limitation, we proposed a neural network with alternate fusion of 3D CNN and ConvLSTM, which we called the Multiple extraction and Multiple prediction (MEMP) network. The main feature of the MEMP network is to extract and predict the temporal and spatial feature information of gesture video multiple times, which enables us to obtain a high accuracy rate. In the experimental part, three data sets (LSA64, SKIG and Chalearn 2016) are used to verify the performance of network. Our approach achieved high accuracy on those data sets. In the LSA64, the network achieved an identification rate of 99.063%. In SKIG, this network obtained the recognition rates of 97.01% and 99.02% in the RGB part and the rgb-depth part. In Chalearn 2016, the network achieved 74.57% and 78.85% recognition rates in RGB part and rgb-depth part respectively. Full article
(This article belongs to the Special Issue Innovative Topologies and Algorithms for Neural Networks)
Show Figures

Figure 1

12 pages, 1807 KiB  
Article
Embedded Deep Learning for Ship Detection and Recognition
by Hongwei Zhao, Weishan Zhang, Haoyun Sun and Bing Xue
Future Internet 2019, 11(2), 53; https://doi.org/10.3390/fi11020053 - 21 Feb 2019
Cited by 33 | Viewed by 5553
Abstract
Ship detection and recognition are important for smart monitoring of ships in order to manage port resources effectively. However, this is challenging due to complex ship profiles, ship background, object occlusion, variations of weather and light conditions, and other issues. It is also [...] Read more.
Ship detection and recognition are important for smart monitoring of ships in order to manage port resources effectively. However, this is challenging due to complex ship profiles, ship background, object occlusion, variations of weather and light conditions, and other issues. It is also expensive to transmit monitoring video in a whole, especially if the port is not in a rural area. In this paper, we propose an on-site processing approach, which is called Embedded Ship Detection and Recognition using Deep Learning (ESDR-DL). In ESDR-DL, the video stream is processed using embedded devices, and we design a two-stage neural network named DCNet, which is composed of a DNet for ship detection and a CNet for ship recognition, running on embedded devices. We have extensively evaluated ESDR-DL, including performance of accuracy and efficiency. The ESDR-DL is deployed at the Dongying port of China, which has been running for over a year and demonstrates that it can work reliably for practical usage. Full article
(This article belongs to the Special Issue Innovative Topologies and Algorithms for Neural Networks)
Show Figures

Figure 1

12 pages, 6161 KiB  
Article
Tooth-Marked Tongue Recognition Using Gradient-Weighted Class Activation Maps
by Yue Sun, Songmin Dai, Jide Li, Yin Zhang and Xiaoqiang Li
Future Internet 2019, 11(2), 45; https://doi.org/10.3390/fi11020045 - 15 Feb 2019
Cited by 18 | Viewed by 5268
Abstract
The tooth-marked tongue is an important indicator in traditional Chinese medicinal diagnosis. However, the clinical competence of tongue diagnosis is determined by the experience and knowledge of the practitioners. Due to the characteristics of different tongues, having many variations such as different colors [...] Read more.
The tooth-marked tongue is an important indicator in traditional Chinese medicinal diagnosis. However, the clinical competence of tongue diagnosis is determined by the experience and knowledge of the practitioners. Due to the characteristics of different tongues, having many variations such as different colors and shapes, tooth-marked tongue recognition is challenging. Most existing methods focus on partial concave features and use specific threshold values to classify the tooth-marked tongue. They lose the overall tongue information and lack the ability to be generalized and interpretable. In this paper, we try to solve these problems by proposing a visual explanation method which takes the entire tongue image as an input and uses a convolutional neural network to extract features (instead of setting a fixed threshold artificially) then classifies the tongue and produces a coarse localization map highlighting tooth-marked regions using Gradient-weighted Class Activation Mapping. Experimental results demonstrate the effectiveness of the proposed method. Full article
(This article belongs to the Special Issue Innovative Topologies and Algorithms for Neural Networks)
Show Figures

Figure 1

17 pages, 3479 KiB  
Article
3D-CNN-Based Fused Feature Maps with LSTM Applied to Action Recognition
by Sheeraz Arif, Jing Wang, Tehseen Ul Hassan and Zesong Fei
Future Internet 2019, 11(2), 42; https://doi.org/10.3390/fi11020042 - 13 Feb 2019
Cited by 36 | Viewed by 6361
Abstract
Human activity recognition is an active field of research in computer vision with numerous applications. Recently, deep convolutional networks and recurrent neural networks (RNN) have received increasing attention in multimedia studies, and have yielded state-of-the-art results. In this research work, we propose a [...] Read more.
Human activity recognition is an active field of research in computer vision with numerous applications. Recently, deep convolutional networks and recurrent neural networks (RNN) have received increasing attention in multimedia studies, and have yielded state-of-the-art results. In this research work, we propose a new framework which intelligently combines 3D-CNN and LSTM networks. First, we integrate discriminative information from a video into a map called a ‘motion map’ by using a deep 3-dimensional convolutional network (C3D). A motion map and the next video frame can be integrated into a new motion map, and this technique can be trained by increasing the training video length iteratively; then, the final acquired network can be used for generating the motion map of the whole video. Next, a linear weighted fusion scheme is used to fuse the network feature maps into spatio-temporal features. Finally, we use a Long-Short-Term-Memory (LSTM) encoder-decoder for final predictions. This method is simple to implement and retains discriminative and dynamic information. The improved results on benchmark public datasets prove the effectiveness and practicability of the proposed method. Full article
(This article belongs to the Special Issue Innovative Topologies and Algorithms for Neural Networks)
Show Figures

Figure 1

16 pages, 2133 KiB  
Article
Forward-Looking Element Recognition Based on the LSTM-CRF Model with the Integrity Algorithm
by Dong Xu, Ruping Ge and Zhihua Niu
Future Internet 2019, 11(1), 17; https://doi.org/10.3390/fi11010017 - 14 Jan 2019
Cited by 2 | Viewed by 3981
Abstract
A state-of-the-art entity recognition system relies on deep learning under data-driven conditions. In this paper, we combine deep learning with linguistic features and propose the long short-term memory-conditional random field model (LSTM-CRF model) with the integrity algorithm. This approach is primarily based on [...] Read more.
A state-of-the-art entity recognition system relies on deep learning under data-driven conditions. In this paper, we combine deep learning with linguistic features and propose the long short-term memory-conditional random field model (LSTM-CRF model) with the integrity algorithm. This approach is primarily based on the use of part-of-speech (POS) syntactic rules to correct the boundaries of LSTM-CRF model annotations and improve its performance by raising the integrity of the elements. The method incorporates the advantages of the data-driven method and dependency syntax, and improves the precision rate of the elements without losing recall rate. Experiments show that the integrity algorithm is not only easy to combine with the other neural network model, but the overall effect is better than several advanced methods. In addition, we conducted cross-domain experiments based on a multi-industry corpus in the financial field. The results indicate that the method can be applied to other industries. Full article
(This article belongs to the Special Issue Innovative Topologies and Algorithms for Neural Networks)
Show Figures

Figure 1

14 pages, 2421 KiB  
Article
Object Detection Network Based on Feature Fusion and Attention Mechanism
by Ying Zhang, Yimin Chen, Chen Huang and Mingke Gao
Future Internet 2019, 11(1), 9; https://doi.org/10.3390/fi11010009 - 02 Jan 2019
Cited by 21 | Viewed by 5658
Abstract
In recent years, almost all of the current top-performing object detection networks use CNN (convolutional neural networks) features. State-of-the-art object detection networks depend on CNN features. In this work, we add feature fusion in the object detection network to obtain a better CNN [...] Read more.
In recent years, almost all of the current top-performing object detection networks use CNN (convolutional neural networks) features. State-of-the-art object detection networks depend on CNN features. In this work, we add feature fusion in the object detection network to obtain a better CNN feature, which incorporates well deep, but semantic, and shallow, but high-resolution, CNN features, thus improving the performance of a small object. Also, the attention mechanism was applied to our object detection network, AF R-CNN (attention mechanism and convolution feature fusion based object detection), to enhance the impact of significant features and weaken background interference. Our AF R-CNN is a single end to end network. We choose the pre-trained network, VGG-16, to extract CNN features. Our detection network is trained on the dataset, PASCAL VOC 2007 and 2012. Empirical evaluation of the PASCAL VOC 2007 dataset demonstrates the effectiveness and improvement of our approach. Our AF R-CNN achieves an object detection accuracy of 75.9% on PASCAL VOC 2007, six points higher than Faster R-CNN. Full article
(This article belongs to the Special Issue Innovative Topologies and Algorithms for Neural Networks)
Show Figures

Figure 1

12 pages, 3528 KiB  
Article
Bidirectional Recurrent Neural Network Approach for Arabic Named Entity Recognition
by Mohammed N. A. Ali, Guanzheng Tan and Aamir Hussain
Future Internet 2018, 10(12), 123; https://doi.org/10.3390/fi10120123 - 13 Dec 2018
Cited by 29 | Viewed by 5392
Abstract
Recurrent neural network (RNN) has achieved remarkable success in sequence labeling tasks with memory requirement. RNN can remember previous information of a sequence and can thus be used to solve natural language processing (NLP) tasks. Named entity recognition (NER) is a common task [...] Read more.
Recurrent neural network (RNN) has achieved remarkable success in sequence labeling tasks with memory requirement. RNN can remember previous information of a sequence and can thus be used to solve natural language processing (NLP) tasks. Named entity recognition (NER) is a common task of NLP and can be considered a classification problem. We propose a bidirectional long short-term memory (LSTM) model for this entity recognition task of the Arabic text. The LSTM network can process sequences and relate to each part of it, which makes it useful for the NER task. Moreover, we use pre-trained word embedding to train the inputs that are fed into the LSTM network. The proposed model is evaluated on a popular dataset called “ANERcorp.” Experimental results show that the model with word embedding achieves a high F-score measure of approximately 88.01%. Full article
(This article belongs to the Special Issue Innovative Topologies and Algorithms for Neural Networks)
Show Figures

Figure 1

13 pages, 2241 KiB  
Article
Neurologist Standard Classification of Facial Nerve Paralysis with Deep Neural Networks
by Anping Song, Zuoyu Wu, Xuehai Ding, Qian Hu and Xinyi Di
Future Internet 2018, 10(11), 111; https://doi.org/10.3390/fi10110111 - 16 Nov 2018
Cited by 22 | Viewed by 4356
Abstract
Facial nerve paralysis (FNP) is the most common form of facial nerve damage, which leads to significant physical pain and abnormal function in patients. Traditional FNP detection methods are based on visual diagnosis, which relies solely on the physician’s assessment. The use of [...] Read more.
Facial nerve paralysis (FNP) is the most common form of facial nerve damage, which leads to significant physical pain and abnormal function in patients. Traditional FNP detection methods are based on visual diagnosis, which relies solely on the physician’s assessment. The use of objective measurements can reduce the frequency of errors which are caused by subjective methods. Hence, a fast, accurate, and objective computer method for FNP classification is proposed that uses a single Convolutional neural network (CNN), trained end-to-end directly from images, with only pixels and disease labels as inputs. We trained the CNN using a dataset of 1049 clinical images and divided the dataset into 7 categories based on classification standards with the help of neurologists. We tested its performance against the neurologists’ ground truth, and our results matched the neurologists’ level with 97.5% accuracy. Full article
(This article belongs to the Special Issue Innovative Topologies and Algorithms for Neural Networks)
Show Figures

Figure 1

Other

Jump to: Editorial, Research

12 pages, 491 KiB  
Essay
Chinese Text Classification Model Based on Deep Learning
by Yue Li, Xutao Wang and Pengjian Xu
Future Internet 2018, 10(11), 113; https://doi.org/10.3390/fi10110113 - 20 Nov 2018
Cited by 65 | Viewed by 7124
Abstract
Text classification is of importance in natural language processing, as the massive text information containing huge amounts of value needs to be classified into different categories for further use. In order to better classify text, our paper tries to build a deep learning [...] Read more.
Text classification is of importance in natural language processing, as the massive text information containing huge amounts of value needs to be classified into different categories for further use. In order to better classify text, our paper tries to build a deep learning model which achieves better classification results in Chinese text than those of other researchers’ models. After comparing different methods, long short-term memory (LSTM) and convolutional neural network (CNN) methods were selected as deep learning methods to classify Chinese text. LSTM is a special kind of recurrent neural network (RNN), which is capable of processing serialized information through its recurrent structure. By contrast, CNN has shown its ability to extract features from visual imagery. Therefore, two layers of LSTM and one layer of CNN were integrated to our new model: the BLSTM-C model (BLSTM stands for bi-directional long short-term memory while C stands for CNN.) LSTM was responsible for obtaining a sequence output based on past and future contexts, which was then input to the convolutional layer for extracting features. In our experiments, the proposed BLSTM-C model was evaluated in several ways. In the results, the model exhibited remarkable performance in text classification, especially in Chinese texts. Full article
(This article belongs to the Special Issue Innovative Topologies and Algorithms for Neural Networks)
Show Figures

Figure 1

Back to TopTop