Submit to Mathematics Review for Mathematics Propose a Special Issue

Journal Browser

Recent Advances in Deep Learning

Print Special Issue Flyer
Special Issue Editors
Special Issue Information
Keywords
Published Papers

A special issue of Mathematics (ISSN 2227-7390). This special issue belongs to the section "Computational and Applied Mathematics".

Deadline for manuscript submissions: closed (30 June 2020) | Viewed by 61918

Share This Special Issue

Special Issue Editor

Dr. Catalin Stoean

E-Mail Website
Guest Editor

Human Language Technology Research Center, University of Bucharest, 010014 Bucharest, Romania
Interests: machine learning; deep learning; computer vision; data mining; classification; evolutionary computation
Special Issues, Collections and Topics in MDPI journals

Special Issue Information

Dear Colleagues,

Deep learning represents a fresh wave that caught the attention of a high number of researchers from various fields in the last decade due to its outstanding performance in solving different problems. As most of the scientists dealing with a branch of machine learning got to use deep learning to solve their tasks, the field received a great opportunity to grow, develop, and flourish in many directions. The achievements of the deep learning architectures did not stop at only reaching and surpassing the results of other machine learning algorithms: Its accomplishments were generally similar and sometimes even went beyond the human results for tasks like image recognition or game playing, thus exceeding the expectations of the experts.

The purpose of this Special Issue is to collect articles where the latest challenges in deep learning are tackled. Papers could include means of reducing the computation time, understanding the insights of the network, interpretation of the intermediary outcomes during training, observation of a failing training process from the early stages, as well as ways to overcome overfitting. The use of concepts from other emerging fields, like evolutionary computation, in deep learning with the goal of overcoming certain issues is also of high interest. Applications to different domains like medicine, chemistry, natural language processing, game playing, economy, speech recognition, to name but a few, are encouraged.

Dr. Catalin Stoean
Guest Editor

Manuscript Submission Information

Manuscripts should be submitted online at www.mdpi.com by registering and logging in to this website. Once you are registered, click here to go to the submission form. Manuscripts can be submitted until the deadline. All submissions that pass pre-check are peer-reviewed. Accepted papers will be published continuously in the journal (as soon as accepted) and will be listed together on the special issue website. Research articles, review articles as well as short communications are invited. For planned papers, a title and short abstract (about 100 words) can be sent to the Editorial Office for announcement on this website.

Submitted manuscripts should not have been published previously, nor be under consideration for publication elsewhere (except conference proceedings papers). All manuscripts are thoroughly refereed through a single-blind peer-review process. A guide for authors and other relevant information for submission of manuscripts is available on the Instructions for Authors page. Mathematics is an international peer-reviewed open access semimonthly journal published by MDPI.

Please visit the Instructions for Authors page before submitting a manuscript. The Article Processing Charge (APC) for publication in this open access journal is 2600 CHF (Swiss Francs). Submitted papers should be well formatted and use good English. Authors may use MDPI's English editing service prior to publication or during author revisions.

Keywords

Convolutional neural networks
Long short-term memory
Generative adversarial networks
Autoencoders
Supervised learning
Classification
Reinforcement learning
Deep learning applications

Published Papers (14 papers)

Download All Papers

Research

Jump to: Review

23 pages, 1493 KiB

Open AccessArticle

Comparing Deep-Learning Architectures and Traditional Machine-Learning Approaches for Satire Identification in Spanish Tweets

by Óscar Apolinario-Arzube, José Antonio García-Díaz, José Medina-Moreira, Harry Luna-Aveiga and Rafael Valencia-García

Mathematics 2020, 8(11), 2075; https://doi.org/10.3390/math8112075 - 20 Nov 2020

Cited by 8 | Viewed by 2590

Abstract

Automatic satire identification can help to identify texts in which the intended meaning differs from the literal meaning, improving tasks such as sentiment analysis, fake news detection or natural-language user interfaces. Typically, satire identification is performed by training a supervised classifier for finding linguistic clues that can determine whether a text is satirical or not. For this, the state-of-the-art relies on neural networks fed with word embeddings that are capable of learning interesting characteristics regarding the way humans communicate. However, as far as our knowledge goes, there are no comprehensive studies that evaluate these techniques in Spanish in the satire identification domain. Consequently, in this work we evaluate several deep-learning architectures with Spanish pre-trained word-embeddings and compare the results with strong baselines based on term-counting features. This evaluation is performed with two datasets that contain satirical and non-satirical tweets written in two Spanish variants: European Spanish and Mexican Spanish. Our experimentation revealed that term-counting features achieved similar results to deep-learning approaches based on word-embeddings, both outperforming previous results based on linguistic features. Our results suggest that term-counting features and traditional machine learning models provide competitive results regarding automatic satire identification, slightly outperforming state-of-the-art models. Full article

(This article belongs to the Special Issue Recent Advances in Deep Learning)

► Show Figures

Figure 1

18 pages, 566 KiB

Open AccessArticle

AlgoLabel: A Large Dataset for Multi-Label Classification of Algorithmic Challenges

by Radu Cristian Alexandru Iacob, Vlad Cristian Monea, Dan Rădulescu, Andrei-Florin Ceapă, Traian Rebedea and Ștefan Trăușan-Matu

Mathematics 2020, 8(11), 1995; https://doi.org/10.3390/math8111995 - 09 Nov 2020

Cited by 2 | Viewed by 2606

Abstract

While semantic parsing has been an important problem in natural language processing for decades, recent years have seen a wide interest in automatic generation of code from text. We propose an alternative problem to code generation: labelling the algorithmic solution for programming challenges. While this may seem an easier task, we highlight that current deep learning techniques are still far from offering a reliable solution. The contributions of the paper are twofold. First, we propose a large multi-modal dataset of text and code pairs consisting of algorithmic challenges and their solutions, called AlgoLabel. Second, we show that vanilla deep learning solutions need to be greatly improved to solve this task and we propose a dual text-code neural model for detecting the algorithmic solution type for a programming challenge. While the proposed text-code model increases the performance of using the text or code alone, the improvement is rather small highlighting that we require better methods to combine text and code features. Full article

(This article belongs to the Special Issue Recent Advances in Deep Learning)

► Show Figures

Figure 1

18 pages, 750 KiB

Open AccessArticle

Author Identification Using Chaos Game Representation and Deep Learning

by Catalin Stoean and Daniel Lichtblau

Mathematics 2020, 8(11), 1933; https://doi.org/10.3390/math8111933 - 02 Nov 2020

Cited by 6 | Viewed by 2399

Abstract

An author unconsciously encodes in the written text a certain style that is often difficult to recognize. Still, there are many computational means developed for this purpose that take into account various features, from lexical and character-based attributes to syntactic or semantic ones. We propose an approach that starts from the character level and uses chaos game representation to illustrate documents like images which are subsequently classified by a deep learning algorithm. The experiments are made on three data sets and the outputs are comparable to the results from the literature. The study also verifies the suitability of the method for small data sets and whether image augmentation can improve the classification efficiency. Full article

(This article belongs to the Special Issue Recent Advances in Deep Learning)

► Show Figures

Figure 1

18 pages, 1439 KiB

Open AccessArticle

Towards Mapping Images to Text Using Deep-Learning Architectures

by Daniela Onita, Adriana Birlutiu and Liviu P. Dinu

Mathematics 2020, 8(9), 1606; https://doi.org/10.3390/math8091606 - 18 Sep 2020

Cited by 8 | Viewed by 3039

Abstract

Images and text represent types of content that are used together for conveying a message. The process of mapping images to text can provide very useful information and can be included in many applications from the medical domain, applications for blind people, social networking, etc. In this paper, we investigate an approach for mapping images to text using a Kernel Ridge Regression model. We considered two types of features: simple RGB pixel-value features and image features extracted with deep-learning approaches. We investigated several neural network architectures for image feature extraction: VGG16, Inception V3, ResNet50, Xception. The experimental evaluation was performed on three data sets from different domains. The texts associated with images represent objective descriptions for two of the three data sets and subjective descriptions for the other data set. The experimental results show that the more complex deep-learning approaches that were used for feature extraction perform better than simple RGB pixel-value approaches. Moreover, the ResNet50 network architecture performs best in comparison to the other three deep network architectures considered for extracting image features. The model error obtained using the ResNet50 network is less by approx. 0.30 than other neural network architectures. We extracted natural language descriptors of images and we made a comparison between original and generated descriptive words. Furthermore, we investigated if there is a difference in performance between the type of text associated with the images: subjective or objective. The proposed model generated more similar descriptions to the original ones for the data set containing objective descriptions whose vocabulary is simpler, bigger and clearer. Full article

(This article belongs to the Special Issue Recent Advances in Deep Learning)

► Show Figures

Figure 1

16 pages, 13096 KiB

Open AccessArticle

Semantic Segmentation for Aerial Mapping

by Gabriel Martinez-Soltero, Alma Y. Alanis, Nancy Arana-Daniel and Carlos Lopez-Franco

Mathematics 2020, 8(9), 1456; https://doi.org/10.3390/math8091456 - 30 Aug 2020

Cited by 5 | Viewed by 2594

Abstract

Mobile robots commonly have to traverse rough terrains. One way to find the easiest traversable path is by determining the types of terrains in the environment. The result of this process can be used by the path planning algorithms to find the best traversable path. In this work, we present an approach for terrain classification from aerial images while using a Convolutional Neural Networks at the pixel level. The segmented images can be used in robot mapping and navigation tasks. The performance of two different Convolutional Neural Networks is analyzed in order to choose the best architecture. Full article

(This article belongs to the Special Issue Recent Advances in Deep Learning)

► Show Figures

Figure 1

13 pages, 948 KiB

Open AccessArticle

Uncertainty Quantification through Dropout in Time Series Prediction by Echo State Networks

by Miguel Atencia, Ruxandra Stoean and Gonzalo Joya

Mathematics 2020, 8(8), 1374; https://doi.org/10.3390/math8081374 - 17 Aug 2020

Cited by 6 | Viewed by 2465

Abstract

The application of echo state networks to time series prediction has provided notable results, favored by their reduced computational cost, since the connection weights require no learning. However, there is a need for general methods that guide the choice of parameters (particularly the reservoir size and ridge regression coefficient), improve the prediction accuracy, and provide an assessment of the uncertainty of the estimates. In this paper we propose such a mechanism for uncertainty quantification based on Monte Carlo dropout, where the output of a subset of reservoir units is zeroed before the computation of the output. Dropout is only performed at the test stage, since the immediate goal is only the computation of a measure of the goodness of the prediction. Results show that the proposal is a promising method for uncertainty quantification, providing a value that is either strongly correlated with the prediction error or reflects the prediction of qualitative features of the time series. This mechanism could eventually be included into the learning algorithm in order to obtain performance enhancements and alleviate the burden of parameter choice. Full article

(This article belongs to the Special Issue Recent Advances in Deep Learning)

► Show Figures

Figure 1

19 pages, 2019 KiB

Open AccessArticle

Eigenloss: Combined PCA-Based Loss Function for Polyp Segmentation

by Luisa F. Sánchez-Peralta, Artzai Picón, Juan Antonio Antequera-Barroso, Juan Francisco Ortega-Morán, Francisco M. Sánchez-Margallo and J. Blas Pagador

Mathematics 2020, 8(8), 1316; https://doi.org/10.3390/math8081316 - 07 Aug 2020

Cited by 12 | Viewed by 3354

Abstract

Colorectal cancer is one of the leading cancer death causes worldwide, but its early diagnosis highly improves the survival rates. The success of deep learning has also benefited this clinical field. When training a deep learning model, it is optimized based on the selected loss function. In this work, we consider two networks (U-Net and LinkNet) and two backbones (VGG-16 and Densnet121). We analyzed the influence of seven loss functions and used a principal component analysis (PCA) to determine whether the PCA-based decomposition allows for the defining of the coefficients of a non-redundant primal loss function that can outperform the individual loss functions and different linear combinations. The eigenloss is defined as a linear combination of the individual losses using the elements of the eigenvector as coefficients. Empirical results show that the proposed eigenloss improves the general performance of individual loss functions and outperforms other linear combinations when Linknet is used, showing potential for its application in polyp segmentation problems. Full article

(This article belongs to the Special Issue Recent Advances in Deep Learning)

► Show Figures

Graphical abstract

13 pages, 2403 KiB

Open AccessArticle

A Highly Efficient Neural Network Solution for Automated Detection of Pointer Meters with Different Analog Scales Operating in Different Conditions

by Alexey Alexeev, Georgy Kukharev, Yuri Matveev and Anton Matveev

Mathematics 2020, 8(7), 1104; https://doi.org/10.3390/math8071104 - 05 Jul 2020

Cited by 13 | Viewed by 4522

Abstract

We investigate a neural network–based solution for the Automatic Meter Reading detection problem, applied to analog dial gauges. We employ a convolutional neural network with a non-linear Network in Network kernel. Presently, there is a significant interest in systems for automatic detection of analog dial gauges, particularly in the energy and household sectors, but the problem is not yet sufficiently addressed in research. Our method is a universal three-level model that takes an image as an input and outputs circular bounding areas, object classes, grids of reference points for all symbols on the front panel of the device and positions of display pointers. Since all analog pointer meters have a common nature, this multi-cascade model can serve various types of devices if its capacity is sufficient. The model is using global regression for locations of symbols, which provides resilient results even for low image quality and overlapping symbols. In this work, we do not focus on the pointer location detection since it heavily depends on the shape of the pointer. We prepare training data and benchmark the algorithm with our own framework a3net, not relying on third-party neural network solutions. The experimental results demonstrate the versatility of the proposed methods, high accuracy, and resilience of reference points detection. Full article

(This article belongs to the Special Issue Recent Advances in Deep Learning)

► Show Figures

Figure 1

17 pages, 876 KiB

Open AccessArticle

Ranking Information Extracted from Uncertainty Quantification of the Prediction of a Deep Learning Model on Medical Time Series Data

by Ruxandra Stoean, Catalin Stoean, Miguel Atencia, Roberto Rodríguez-Labrada and Gonzalo Joya

Mathematics 2020, 8(7), 1078; https://doi.org/10.3390/math8071078 - 02 Jul 2020

Cited by 13 | Viewed by 2266

Abstract

Uncertainty quantification in deep learning models is especially important for the medical applications of this complex and successful type of neural architectures. One popular technique is Monte Carlo dropout that gives a sample output for a record, which can be measured statistically in terms of average probability and variance for each diagnostic class of the problem. The current paper puts forward a convolutional–long short-term memory network model with a Monte Carlo dropout layer for obtaining information regarding the model uncertainty for saccadic records of all patients. These are next used in assessing the uncertainty of the learning model at the higher level of sets of multiple records (i.e., registers) that are gathered for one patient case by the examining physician towards an accurate diagnosis. Means and standard deviations are additionally calculated for the Monte Carlo uncertainty estimates of groups of predictions. These serve as a new collection where a random forest model can perform both classification and ranking of variable importance. The approach is validated on a real-world problem of classifying electrooculography time series for an early detection of spinocerebellar ataxia 2 and reaches an accuracy of 88.59% in distinguishing between the three classes of patients. Full article

(This article belongs to the Special Issue Recent Advances in Deep Learning)

► Show Figures

Figure 1

33 pages, 1310 KiB

Open AccessArticle

Monarch Butterfly Optimization Based Convolutional Neural Network Design

by Nebojsa Bacanin, Timea Bezdan, Eva Tuba, Ivana Strumberger and Milan Tuba

Mathematics 2020, 8(6), 936; https://doi.org/10.3390/math8060936 - 08 Jun 2020

Cited by 63 | Viewed by 5264

Abstract

Convolutional neural networks have a broad spectrum of practical applications in computer vision. Currently, much of the data come from images, and it is crucial to have an efficient technique for processing these large amounts of data. Convolutional neural networks have proven to be very successful in tackling image processing tasks. However, the design of a network structure for a given problem entails a fine-tuning of the hyperparameters in order to achieve better accuracy. This process takes much time and requires effort and expertise from the domain. Designing convolutional neural networks’ architecture represents a typical NP-hard optimization problem, and some frameworks for generating network structures for a specific image classification tasks have been proposed. To address this issue, in this paper, we propose the hybridized monarch butterfly optimization algorithm. Based on the observed deficiencies of the original monarch butterfly optimization approach, we performed hybridization with two other state-of-the-art swarm intelligence algorithms. The proposed hybrid algorithm was firstly tested on a set of standard unconstrained benchmark instances, and later on, it was adapted for a convolutional neural network design problem. Comparative analysis with other state-of-the-art methods and algorithms, as well as with the original monarch butterfly optimization implementation was performed for both groups of simulations. Experimental results proved that our proposed method managed to obtain higher classification accuracy than other approaches, the results of which were published in the modern computer science literature. Full article

(This article belongs to the Special Issue Recent Advances in Deep Learning)

► Show Figures

Figure 1

21 pages, 5004 KiB

Open AccessArticle

Enhancement of Deep Learning in Image Classification Performance Using Xception with the Swish Activation Function for Colorectal Polyp Preliminary Screening

by Natinai Jinsakul, Cheng-Fa Tsai, Chia-En Tsai and Pensee Wu

Mathematics 2019, 7(12), 1170; https://doi.org/10.3390/math7121170 - 03 Dec 2019

Cited by 37 | Viewed by 5978

Abstract

One of the leading forms of cancer is colorectal cancer (CRC), which is responsible for increasing mortality in young people. The aim of this paper is to provide an experimental modification of deep learning of Xception with Swish and assess the possibility of developing a preliminary colorectal polyp screening system by training the proposed model with a colorectal topogram dataset in two and three classes. The results indicate that the proposed model can enhance the original convolutional neural network model with evaluation classification performance by achieving accuracy of up to 98.99% for classifying into two classes and 91.48% for three classes. For testing of the model with another external image, the proposed method can also improve the prediction compared to the traditional method, with 99.63% accuracy for true prediction of two classes and 80.95% accuracy for true prediction of three classes. Full article

(This article belongs to the Special Issue Recent Advances in Deep Learning)

► Show Figures

Figure 1

17 pages, 2652 KiB

Open AccessArticle

Towards Repayment Prediction in Peer-to-Peer Social Lending Using Deep Learning

by Ji-Yoon Kim and Sung-Bae Cho

Mathematics 2019, 7(11), 1041; https://doi.org/10.3390/math7111041 - 03 Nov 2019

Cited by 17 | Viewed by 3191

Abstract

Peer-to-Peer (P2P) lending transactions take place by the lenders choosing a borrower and lending money. It is important to predict whether a borrower can repay because the lenders must bear the credit risk when the borrower defaults, but it is difficult to design feature extractors with very complex information about borrowers and loan products. In this paper, we present an architecture of deep convolutional neural network (CNN) for predicting the repayment in P2P social lending to extract features automatically and improve the performance. CNN is a deep learning model for classifying complex data, which extracts discriminative features automatically by convolution operation on lending data. We classify the borrower’s loan status by capturing the robust features and learning the patterns. Experimental results with 5-fold cross-validation show that our method automatically extracts complex features and is effective in repayment prediction on Lending Club data. In comparison with other machine learning methods, the standard CNN has achieved the highest performance with 75.86%. Exploiting various CNN models such as Inception, ResNet, and Inception-ResNet results in the state-of-the-art performance of 77.78%. We also demonstrate that the features extracted by our model are better performed by projecting the samples into the feature space. Full article

(This article belongs to the Special Issue Recent Advances in Deep Learning)

► Show Figures

Figure 1

13 pages, 2522 KiB

Open AccessArticle

Automatic Melody Composition Using Enhanced GAN

by Shuyu Li, Sejun Jang and Yunsick Sung

Mathematics 2019, 7(10), 883; https://doi.org/10.3390/math7100883 - 23 Sep 2019

Cited by 21 | Viewed by 6315

Abstract

In traditional music composition, the composer has a special knowledge of music and combines emotion and creative experience to create music. As computer technology has evolved, various music-related technologies have been developed. To create new music, a considerable amount of time is required. Therefore, a system is required that can automatically compose music from input music. This study proposes a novel melody composition method that enhanced the original generative adversarial network (GAN) model based on individual bars. Two discriminators were used to form the enhanced GAN model: one was a long short-term memory (LSTM) model that was used to ensure correlation between the bars, and the other was a convolutional neural network (CNN) model that was used to ensure rationality of the bar structure. Experiments were conducted using bar encoding and the enhanced GAN model to compose a new melody and evaluate the quality of the composition melody. In the evaluation method, the TFIDF algorithm was also used to calculate the structural differences between four types of musical instrument digital interface (MIDI) file (i.e., randomly composed melody, melody composed by the original GAN, melody composed by the proposed method, and the real melody). Using the TFIDF algorithm, the structures of the melody composed were compared by the proposed method with the real melody and the structure of the traditional melody was compared with the structure of the real melody. The experimental results showed that the melody composed by the proposed method had more similarity with real melody structure with a difference of only 8% than that of the traditional melody structure. Full article

(This article belongs to the Special Issue Recent Advances in Deep Learning)

► Show Figures

Figure 1

Review

Jump to: Research

42 pages, 14445 KiB

Open AccessReview

Comprehensive Review of Deep Reinforcement Learning Methods and Applications in Economics

by Amirhosein Mosavi, Yaser Faghan, Pedram Ghamisi, Puhong Duan, Sina Faizollahzadeh Ardabili, Ely Salwana and Shahab S. Band

Mathematics 2020, 8(10), 1640; https://doi.org/10.3390/math8101640 - 23 Sep 2020

Cited by 104 | Viewed by 14082

Abstract

The popularity of deep reinforcement learning (DRL) applications in economics has increased exponentially. DRL, through a wide range of capabilities from reinforcement learning (RL) to deep learning (DL), offers vast opportunities for handling sophisticated dynamic economics systems. DRL is characterized by scalability with the potential to be applied to high-dimensional problems in conjunction with noisy and nonlinear patterns of economic data. In this paper, we initially consider a brief review of DL, RL, and deep RL methods in diverse applications in economics, providing an in-depth insight into the state-of-the-art. Furthermore, the architecture of DRL applied to economic applications is investigated in order to highlight the complexity, robustness, accuracy, performance, computational tasks, risk constraints, and profitability. The survey results indicate that DRL can provide better performance and higher efficiency as compared to the traditional algorithms while facing real economic problems in the presence of risk parameters and the ever-increasing uncertainties. Full article

(This article belongs to the Special Issue Recent Advances in Deep Learning)

► Show Figures

Figure 1

Journal Menu

Journal Browser

Recent Advances in Deep Learning

Share This Special Issue

Special Issue Editor

Special Issue Information

Keywords

Published Papers (14 papers)

Research

Review

Further Information

Guidelines

MDPI Initiatives

Follow MDPI