Search for Articles:
Title / Keyword
Author / Affiliation / Email
Journal
Article Type
 
 
Advanced Search
 
Section
Special Issue
Volume
Issue
Number
Page
 
3.5
2.4
Journals
Mathematics
Special Issues
Deep Learning for Natural Language Processing: Advances and Challenges
share announcement

Deep Learning for Natural Language Processing: Advances and Challenges

A special issue of Mathematics (ISSN 2227-7390). This special issue belongs to the section "Mathematics and Computer Science".

Deadline for manuscript submissions: closed (29 February 2024) | Viewed by 9077

Special Issue Editors

Dr. Víctor Manuel Darriba Bilbao

E-Mail Website
Guest Editor
Computer Science Department, Computer Science Engineering High School, University of Vigo, Vigo, Spain
Interests: natural language processing; information retrieval; text classification; automata theory and formal languages
Prof. Dr. Alexander Gelbukh

E-Mail Website
Guest Editor
Natural Language Processing Laboratory, Computer Research Center, National Institute of Technology, Mexico City, Mexico
Interests: computational linguistics; natural language processing
Dr. Alvaro Rodrigo

E-Mail Website
Guest Editor
Department of Computer Systems and Languages, National University of Distance Education (UNED), Madrid, Spain
Interests: question answering; answer validation; machine reading

Special Issue Information

Dear Colleagues,

The advances in information and communication technologies in recent decades have caused a vast expansion in the volume and availability of data in human languages, both text and speech, along with the need to manage them adequately. The answer to this need lies in natural language processing (NLP), a field encompassing a wide variety of tasks related to the computational processing and understanding of human languages. 

Initial NLP approaches, based on symbolic techniques and explicit linguistic knowledge, have been widely superseded in many NLP tasks by machine learning (ML) models capable of generalization from suitable training databases. More recently, advances in computational power and parallelization with graphical processing units have greatly increased the popularity of deep learning (DL) models, based on artificial neural network architectures. 

Today, DL approaches are at the forefront of technology based on ML, and in many NLP applications. In this context, this Special Issue is focused on the application of DL techniques for solving NLP tasks, both for specific applications and more general language modeling. We also welcome solutions which address the known challenges in the application of DL: identification of appropriate network structures, hyperparameter optimization, integration with other linguistic resources, efficient representations capable of capturing long-term dependencies, prevention of overfitting, etc.

Dr. Víctor Manuel Darriba Bilbao
Prof. Dr. Alexander Gelbukh
Dr. Alvaro Rodrigo
Guest Editors

Manuscript Submission Information

Manuscripts should be submitted online at www.mdpi.com by registering and logging in to this website. Once you are registered, click here to go to the submission form. Manuscripts can be submitted until the deadline. All submissions that pass pre-check are peer-reviewed. Accepted papers will be published continuously in the journal (as soon as accepted) and will be listed together on the special issue website. Research articles, review articles as well as short communications are invited. For planned papers, a title and short abstract (about 100 words) can be sent to the Editorial Office for announcement on this website.

Submitted manuscripts should not have been published previously, nor be under consideration for publication elsewhere (except conference proceedings papers). All manuscripts are thoroughly refereed through a single-blind peer-review process. A guide for authors and other relevant information for submission of manuscripts is available on the Instructions for Authors page. Mathematics is an international peer-reviewed open access semimonthly journal published by MDPI.

Please visit the Instructions for Authors page before submitting a manuscript. The Article Processing Charge (APC) for publication in this open access journal is 2600 CHF (Swiss Francs). Submitted papers should be well formatted and use good English. Authors may use MDPI's English editing service prior to publication or during author revisions.

Keywords

  • DL-based techniques and tools for NLP
  • embeddings and language models
  • domain-specific resources
  • low-resource languages
  • transfer learning
  • early stopping and prevention of overfitting
  • reasoning with large contexts and multiple documents
  • graph-based DL
  • zero-shot and few-shot learning

Published Papers (3 papers)

Download All Papers
Order results
Result details
Show export options expand_more Show export options expand_less
Select all
Export citation of selected articles as:

Research

22 pages, 1456 KiB  
Article
Natural Language Understanding for Navigation of Service Robots in Low-Resource Domains and Languages: Scenarios in Spanish and Nahuatl
by Amadeo Hernández, Rosa María Ortega-Mendoza, Esaú Villatoro-Tello, César Joel Camacho-Bello and Obed Pérez-Cortés
Mathematics 2024, 12(8), 1136; https://doi.org/10.3390/math12081136 - 10 Apr 2024
Viewed by 354
Abstract
Human–robot interaction is becoming increasingly common to perform useful tasks in everyday life. From the human–machine communication perspective, achieving effective interaction in natural language is one challenge. To address it, natural language processing strategies have recently been used, commonly following a supervised machine [...] Read more.
Human–robot interaction is becoming increasingly common to perform useful tasks in everyday life. From the human–machine communication perspective, achieving effective interaction in natural language is one challenge. To address it, natural language processing strategies have recently been used, commonly following a supervised machine learning framework. In this context, most approaches rely on the use of linguistic resources (e.g., taggers or embeddings), including training corpora. Unfortunately, such resources are scarce for some languages in specific domains, increasing the complexity of solution approaches. Motivated by these challenges, this paper explores deep learning methods for understanding natural language commands emitted to service robots that guide their movements in low-resource scenarios, defined by the use of Spanish and Nahuatl languages, for which linguistic resources are scarcely unavailable for this specific task. Particularly, we applied natural language understanding (NLU) techniques using deep neural networks and transformers-based models. As part of the research methodology, we introduced a labeled dataset of movement commands in the mentioned languages. The results show that models based on transformers work well to recognize commands (intent classification task) and their parameters (e.g., quantities and movement units) in Spanish, achieving a performance of 98.70% (accuracy) and 96.96% (F1) for the intent classification and slot-filling tasks, respectively). In Nahuatl, the best performance obtained was 93.5% (accuracy) and 88.57% (F1) in these tasks, respectively. In general, this study shows that robot movements can be guided in natural language through machine learning models using neural models and cross-lingual transfer strategies, even in low-resource scenarios. Full article
(This article belongs to the Special Issue Deep Learning for Natural Language Processing: Advances and Challenges)
►▼ Show Figures

Figure 1

17 pages, 7048 KiB  
Article
Low-Resource Language Processing Using Improved Deep Learning with Hunter–Prey Optimization Algorithm
by Fahd N. Al-Wesabi, Hala J. Alshahrani, Azza Elneil Osman and Elmouez Samir Abd Elhameed
Mathematics 2023, 11(21), 4493; https://doi.org/10.3390/math11214493 - 30 Oct 2023
Viewed by 1299
Abstract
Low-resource language (LRL) processing refers to the development of natural language processing (NLP) techniques and tools for languages with limited linguistic resources and data. These languages often lack well-annotated datasets and pre-training methods, making traditional approaches less effective. Sentiment analysis (SA), which involves [...] Read more.
Low-resource language (LRL) processing refers to the development of natural language processing (NLP) techniques and tools for languages with limited linguistic resources and data. These languages often lack well-annotated datasets and pre-training methods, making traditional approaches less effective. Sentiment analysis (SA), which involves identifying the emotional tone or sentiment expressed in text, poses unique challenges for LRLs due to the scarcity of labelled sentiment data and linguistic intricacies. NLP tasks like SA, powered by machine learning (ML) techniques, can generalize effectively when trained on suitable datasets. Recent advancements in computational power and parallelized graphical processing units have significantly increased the popularity of deep learning (DL) approaches built on artificial neural network (ANN) architectures. With this in mind, this manuscript describes the design of an LRL Processing technique that makes use of Improved Deep Learning with Hunter–Prey Optimization (LRLP-IDLHPO). The LRLP-IDLHPO technique enables the detection and classification of different kinds of sentiments present in LRL data. To accomplish this, the presented LRLP-IDLHPO technique initially pre-processes these data to improve their usability. Subsequently, the LRLP-IDLHPO approach applies the SentiBERT approach for word embedding purposes. For the sentiment classification process, the Element-Wise–Attention GRU network (EWAG-GRU) algorithm is used, which is an enhanced version of the recurrent neural network. The EWAG-GRU model is capable of processing temporal features and includes an attention strategy. Finally, the performance of the EWAG-GRU model can be boosted by adding the HPO algorithm for use in the hyperparameter tuning process. A widespread simulation analysis was performed to validate the superior results derived from using the LRLP-IDLHPO approach. The extensive results indicate the significant superiority of the performance of the LRLP-IDLHPO technique compared to the state-of-the-art approaches described in the literature. Full article
(This article belongs to the Special Issue Deep Learning for Natural Language Processing: Advances and Challenges)
►▼ Show Figures

Figure 1

17 pages, 436 KiB  
Article
A Mathematical Investigation of Hallucination and Creativity in GPT Models
by Minhyeok Lee
Mathematics 2023, 11(10), 2320; https://doi.org/10.3390/math11102320 - 16 May 2023
Cited by 11 | Viewed by 6400
Abstract
In this paper, we present a comprehensive mathematical analysis of the hallucination phenomenon in generative pretrained transformer (GPT) models. We rigorously define and measure hallucination and creativity using concepts from probability theory and information theory. By introducing a parametric family of GPT models, [...] Read more.
In this paper, we present a comprehensive mathematical analysis of the hallucination phenomenon in generative pretrained transformer (GPT) models. We rigorously define and measure hallucination and creativity using concepts from probability theory and information theory. By introducing a parametric family of GPT models, we characterize the trade-off between hallucination and creativity and identify an optimal balance that maximizes model performance across various tasks. Our work offers a novel mathematical framework for understanding the origins and implications of hallucination in GPT models and paves the way for future research and development in the field of large language models (LLMs). Full article
(This article belongs to the Special Issue Deep Learning for Natural Language Processing: Advances and Challenges)
►▼ Show Figures

Figure 1

Show export options expand_more Show export options expand_less
Select all
Export citation of selected articles as:
 
Displaying articles 1-3
Back to TopTop