Special Issue "Natural Language Processing (NLP) and Applications"

A special issue of Applied Sciences (ISSN 2076-3417). This special issue belongs to the section "Computing and Artificial Intelligence".

Deadline for manuscript submissions: 31 March 2023 | Viewed by 5523

Special Issue Editors

Prof. Dr. Gui-Lin Qi
E-Mail Website
Guest Editor
School of Computer Science and Engineering, Southeast University, Nanjing 210000, China
Interests: natural language processing; knowledge graph; multimodal learning
Dr. Tong Xu
E-Mail Website
Guest Editor
School of Data Science, University of Science and Technology of China, Hefei 230027, China
Interests: natural language processing; social media analysis; multimodal intelligence
Dr. Meng Wang
E-Mail Website
Guest Editor
School of Computer Science and Engineering, Southeast University, Nanjing 210000, China
Interests: natural language processing; knowledge graph; multimodal learning

Special Issue Information

Dear Colleagues,

Natural Language Processing (NLP) is a key technology of artificial intelligence. In recent years, many highly recognized efforts in NLP have mushroomed, such as BERT and GPT-3. They already power a wide range of applications that we experience on a daily basis, such as question answering, machine translation, and smart assistants. They are also crucial to a wide range of other research topics, for biomedical information processing, knowledge graph, and multimodal intelligence. However, numerous relevant unsolved theoretical and technological problems await further research. This special issue aims at addressing the aforementioned questions by inviting scholarly contributions covering recent advances in NLP and applications. We welcome original research articles reporting the development of NLP novel models and algorithms, as well as NLP applications papers with novel ideas.

Prof. Dr. Gui-Lin Qi
Dr. Tong Xu
Dr. Meng Wang
Guest Editors

Manuscript Submission Information

Manuscripts should be submitted online at www.mdpi.com by registering and logging in to this website. Once you are registered, click here to go to the submission form. Manuscripts can be submitted until the deadline. All submissions that pass pre-check are peer-reviewed. Accepted papers will be published continuously in the journal (as soon as accepted) and will be listed together on the special issue website. Research articles, review articles as well as short communications are invited. For planned papers, a title and short abstract (about 100 words) can be sent to the Editorial Office for announcement on this website.

Submitted manuscripts should not have been published previously, nor be under consideration for publication elsewhere (except conference proceedings papers). All manuscripts are thoroughly refereed through a single-blind peer-review process. A guide for authors and other relevant information for submission of manuscripts is available on the Instructions for Authors page. Applied Sciences is an international peer-reviewed open access semimonthly journal published by MDPI.

Please visit the Instructions for Authors page before submitting a manuscript. The Article Processing Charge (APC) for publication in this open access journal is 2300 CHF (Swiss Francs). Submitted papers should be well formatted and use good English. Authors may use MDPI's English editing service prior to publication or during author revisions.

Keywords

  • natural language understanding
  • natural language generation
  • question answering
  • machine translation
  • knowledge graph
  • NLP for knowledge extraction
  • NLP for multimodal intelligence
  • NLP applications in specific domains, like life sciences, health, and medicine
  • eGovernment and public administration
  • news and social media

Published Papers (13 papers)

Order results
Result details
Select all
Export citation of selected articles as:

Research

Article
Automatic Classification of Eyewitness Messages for Disaster Events Using Linguistic Rules and ML/AI Approaches
Appl. Sci. 2022, 12(19), 9953; https://doi.org/10.3390/app12199953 (registering DOI) - 03 Oct 2022
Viewed by 161
Abstract
Emergency response systems require precise and accurate information about an incident to respond accordingly. An eyewitness report is one of the sources of such information. The research community has proposed diverse techniques to identify eyewitness messages from social media platforms. In our previous [...] Read more.
Emergency response systems require precise and accurate information about an incident to respond accordingly. An eyewitness report is one of the sources of such information. The research community has proposed diverse techniques to identify eyewitness messages from social media platforms. In our previous work, we created grammar rules by exploiting the language structure, linguistics, and word relations to automatically extract feature words to classify eyewitness messages for different disaster types. Our previous work adopted a manual classification technique and secured the maximum F-Score of 0.81, far less than the static dictionary-based approach with an F-Score of 0.92. In this work, we enhanced our work by adding more features and fine-tuning the Linguistic Rules to identify feature words related to Twitter Eyewitness messages for Disaster events, named as LR-TED approach. We used linguistic characteristics and labeled datasets to train several machine learning and deep learning classifiers for classifying eyewitness messages and secured a maximum F-score of 0.93. The proposed LR-TED can process millions of tweets in real-time and is scalable to diverse events and unseen content. In contrast, the static dictionary-based approaches require domain experts to create dictionaries of related words for all the identified features and disaster types. Additionally, LR-TED can be evaluated on different social media platforms to identify eyewitness reports for various disaster types in the future. Full article
(This article belongs to the Special Issue Natural Language Processing (NLP) and Applications)
Article
Long Text Truncation Algorithm Based on Label Embedding in Text Classification
Appl. Sci. 2022, 12(19), 9874; https://doi.org/10.3390/app12199874 (registering DOI) - 30 Sep 2022
Viewed by 145
Abstract
The long text classification task has become a hot research topic in the field of text classification due to its long length and redundant information. At present, the common processing methods for long text data, such as the truncation method and pooling method, [...] Read more.
The long text classification task has become a hot research topic in the field of text classification due to its long length and redundant information. At present, the common processing methods for long text data, such as the truncation method and pooling method, are prone to the problem of too many sentences or loss of contextual semantic information. To deal with these issues, we present LTTA-LE (Long Text Truncation Algorithm Based on Label Embedding in Text Classification), which consists of three key steps. Firstly, we build a pretraining prefix template and a label word mapping prefix template to obtain the label word embedding, and we realize the joint training of long text and label words. Secondly, we calculate the cosine similarity between the label word embedding and the long text embedding, and we filter the redundant information of the long text to reduce the text length. Finally, a three-stage model training architecture is introduced to effectively improve the classification performance and generalization ability of the model. We conduct comparative experiments on three public long text datasets, and the results show that LTTA-LE has an average F1 improvement of 1.0518% over other algorithms, which proves that our method can achieve satisfactory performance. Full article
(This article belongs to the Special Issue Natural Language Processing (NLP) and Applications)
Article
Research on Chinese Medical Entity Relation Extraction Based on Syntactic Dependency Structure Information
Appl. Sci. 2022, 12(19), 9781; https://doi.org/10.3390/app12199781 - 28 Sep 2022
Viewed by 166
Abstract
Extracting entity relations from unstructured medical texts is a fundamental task in the field of medical information extraction. In relation extraction, dependency trees contain rich structural information that helps capture the long-range relations between entities. However, many models cannot effectively use dependency information [...] Read more.
Extracting entity relations from unstructured medical texts is a fundamental task in the field of medical information extraction. In relation extraction, dependency trees contain rich structural information that helps capture the long-range relations between entities. However, many models cannot effectively use dependency information or learn sentence information adequately. In this paper, we propose a relation extraction model based on syntactic dependency structure information. First, the model learns sentence sequence information by Bi-LSTM. Then, the model learns syntactic dependency structure information through graph convolutional networks. Meanwhile, in order to remove irrelevant information from the dependencies, the model adopts a new pruning strategy. Finally, the model adds a multi-head attention mechanism to focus on the entity information in the sentence from multiple aspects. We evaluate the proposed model on a Chinese medical entity relation extraction dataset. Experimental results show that our model can learn dependency relation information better and has higher performance than other baseline models. Full article
(This article belongs to the Special Issue Natural Language Processing (NLP) and Applications)
Article
SupMPN: Supervised Multiple Positives and Negatives Contrastive Learning Model for Semantic Textual Similarity
Appl. Sci. 2022, 12(19), 9659; https://doi.org/10.3390/app12199659 - 26 Sep 2022
Viewed by 221
Abstract
Semantic Textual Similarity (STS) is an important task in the area of Natural Language Processing (NLP) that measures the similarity of the underlying semantics of two texts. Although pre-trained contextual embedding models such as Bidirectional Encoder Representations from Transformers (BERT) have achieved state-of-the-art [...] Read more.
Semantic Textual Similarity (STS) is an important task in the area of Natural Language Processing (NLP) that measures the similarity of the underlying semantics of two texts. Although pre-trained contextual embedding models such as Bidirectional Encoder Representations from Transformers (BERT) have achieved state-of-the-art performance on several NLP tasks, BERT-derived sentence embeddings have been proven to collapse in some way, i.e., sentence embeddings generated by BERT depend on the frequency of words. Therefore, almost all BERT-derived sentence embeddings are mapped into a small area and have a high cosine similarity. Hence, sentence embeddings generated by BERT are not so robust in the STS task as they cannot capture the full semantic meaning of the sentences. In this paper, we propose SupMPN: A Supervised Multiple Positives and Negatives Contrastive Learning Model, which accepts multiple hard-positive sentences and multiple hard-negative sentences simultaneously and then tries to bring hard-positive sentences closer, while pushing hard-negative sentences away from them. In other words, SupMPN brings similar sentences closer together in the representation space by discrimination among multiple similar and dissimilar sentences. In this way, SupMPN can learn the semantic meanings of sentences by contrasting among multiple similar and dissimilar sentences and can generate sentence embeddings based on the semantic meaning instead of the frequency of the words. We evaluate our model on standard STS and transfer-learning tasks. The results reveal that SupMPN outperforms state-of-the-art SimCSE and all other previous supervised and unsupervised models. Full article
(This article belongs to the Special Issue Natural Language Processing (NLP) and Applications)
Show Figures

Figure 1

Article
Multigranularity Syntax Guidance with Graph Structure for Machine Reading Comprehension
Appl. Sci. 2022, 12(19), 9525; https://doi.org/10.3390/app12199525 - 22 Sep 2022
Viewed by 252
Abstract
In recent years, pre-trained language models, represented by the bidirectional encoder representations from transformers (BERT) model, have achieved remarkable success in machine reading comprehension (MRC). However, limited by the structure of BERT-based MRC models (for example, restrictions on word count), such models cannot [...] Read more.
In recent years, pre-trained language models, represented by the bidirectional encoder representations from transformers (BERT) model, have achieved remarkable success in machine reading comprehension (MRC). However, limited by the structure of BERT-based MRC models (for example, restrictions on word count), such models cannot effectively integrate significant features, such as syntax relations, semantic connections, and long-distance semantics between sentences, leading to the inability of the available models to better understand the intrinsic connections between text and questions to be answered based on it. In this paper, a multi-granularity syntax guidance (MgSG) module that consists of a “graph with dependence” module and a “graph with entity” module is proposed. MgSG selects both sentence and word granularities to guide the text model to decipher the text. In particular, syntactic constraints are used to guide the text model while exploiting the global nature of graph neural networks to enhance the model’s ability to construct long-range semantics. Simultaneously, named entities play an important role in text and answers and focusing on entities can improve the model’s understanding of the text’s major idea. Ultimately, fusing multiple embedding representations to form a representation yields the semantics of the context and the questions. Experiments demonstrate that the performance of the proposed method on the Stanford Question Answering Dataset is better when compared with the traditional BERT baseline model. The experimental results illustrate that our proposed “MgSG” module effectively utilizes the graph structure to learn the internal features of sentences, solve the problem of long-distance semantics, while effectively improving the performance of PrLM in machine reading comprehension. Full article
(This article belongs to the Special Issue Natural Language Processing (NLP) and Applications)
Show Figures

Figure 1

Article
Knowledge Graph Alignment Network with Node-Level Strong Fusion
Appl. Sci. 2022, 12(19), 9434; https://doi.org/10.3390/app12199434 - 20 Sep 2022
Viewed by 338
Abstract
Entity alignment refers to the process of discovering entities representing the same object in different knowledge graphs (KG). Recently, some studies have learned other information about entities, but they are aspect-level simple information associations, and thus only rough entity representations can be obtained, [...] Read more.
Entity alignment refers to the process of discovering entities representing the same object in different knowledge graphs (KG). Recently, some studies have learned other information about entities, but they are aspect-level simple information associations, and thus only rough entity representations can be obtained, and the advantage of multi-faceted information is lost. In this paper, a novel node-level information strong fusion framework (SFEA) is proposed, based on four aspects: structure, attribute, relation and names. The attribute information and name information are learned first, then structure information is learned based on these two aspects of information through graph convolutional network (GCN), the alignment signals from attribute and name are already carried at the beginning of the learning structure. In the process of continuous propagation of multi-hop neighborhoods, the effect of strong fusion of structure, attribute and name information is achieved and the more meticulous entity representations are obtained. Additionally, through the continuous interaction between sub-alignment tasks, the effect of entity alignment is enhanced. An iterative framework is designed to improve performance while reducing the impact on pre-aligned seed pairs. Furthermore, extensive experiments demonstrate that the model improves the accuracy of entity alignment and significantly outperforms 13 previous state-of-the-art methods. Full article
(This article belongs to the Special Issue Natural Language Processing (NLP) and Applications)
Show Figures

Figure 1

Article
An End-to-End Mutually Interactive Emotion–Cause Pair Extractor via Soft Sharing
Appl. Sci. 2022, 12(18), 8998; https://doi.org/10.3390/app12188998 - 07 Sep 2022
Viewed by 334
Abstract
Emotion–cause pair extraction (ECPE), i.e., extracting pairs of emotions and corresponding causes from text, has recently attracted a lot of research interest. However, current ECPE models face two problems: (1) The common two-stage pipeline causes the error to be accumulated. (2) Ignoring the [...] Read more.
Emotion–cause pair extraction (ECPE), i.e., extracting pairs of emotions and corresponding causes from text, has recently attracted a lot of research interest. However, current ECPE models face two problems: (1) The common two-stage pipeline causes the error to be accumulated. (2) Ignoring the mutual connection between the extraction and pairing of emotion and cause limits the performance. In this paper, we propose a novel end-to-end mutually interactive emotion–cause pair extractor (Emiece) that is able to effectively extract emotion–cause pairs from all potential clause pairs. Specifically, we design two soft-shared clause-level encoders in an end-to-end deep model to measure the weighted probability of being a potential emotion–cause pair. Experiments on standard ECPE datasets show that Emiece achieves drastic improvements over the original two-step ECPE model and other end-to-end models in the extraction of major emotional cause pairs. The effectiveness of soft sharing and the applicability of the Emiece framework are further demonstrated by ablation experiments. Full article
(This article belongs to the Special Issue Natural Language Processing (NLP) and Applications)
Show Figures

Figure 1

Article
Zero-Shot Emotion Detection for Semi-Supervised Sentiment Analysis Using Sentence Transformers and Ensemble Learning
Appl. Sci. 2022, 12(17), 8662; https://doi.org/10.3390/app12178662 - 29 Aug 2022
Viewed by 352
Abstract
We live in a digitized era where our daily life depends on using online resources. Businesses consider the opinions of their customers, while people rely on the reviews/comments of other users before buying specific products or services. These reviews/comments are usually provided in [...] Read more.
We live in a digitized era where our daily life depends on using online resources. Businesses consider the opinions of their customers, while people rely on the reviews/comments of other users before buying specific products or services. These reviews/comments are usually provided in the non-normative natural language within different contexts and domains (in social media, forums, news, blogs, etc.). Sentiment classification plays an important role in analyzing such texts collected from users by assigning positive, negative, and sometimes neutral sentiment values to each of them. Moreover, these texts typically contain many expressed or hidden emotions (such as happiness, sadness, etc.) that could contribute significantly to identifying sentiments. We address the emotion detection problem as part of the sentiment analysis task and propose a two-stage emotion detection methodology. The first stage is the unsupervised zero-shot learning model based on a sentence transformer returning the probabilities for subsets of 34 emotions (anger, sadness, disgust, fear, joy, happiness, admiration, affection, anguish, caution, confusion, desire, disappointment, attraction, envy, excitement, grief, hope, horror, joy, love, loneliness, pleasure, fear, generosity, rage, relief, satisfaction, sorrow, wonder, sympathy, shame, terror, and panic). The output of the zero-shot model is used as an input for the second stage, which trains the machine learning classifier on the sentiment labels in a supervised manner using ensemble learning. The proposed hybrid semi-supervised method achieves the highest accuracy of 87.3% on the English SemEval 2017 dataset. Full article
(This article belongs to the Special Issue Natural Language Processing (NLP) and Applications)
Show Figures

Figure 1

Article
Identifying Irregular Financial Operations Using Accountant Comments and Natural Language Processing Techniques
Appl. Sci. 2022, 12(17), 8558; https://doi.org/10.3390/app12178558 - 26 Aug 2022
Viewed by 283
Abstract
Finding not typical financial operations is a complicated task. The difficulties arise not only due to the sophisticated actions of fraudsters but also because of the large number of financial operations performed by business companies. This is especially true for large companies. It [...] Read more.
Finding not typical financial operations is a complicated task. The difficulties arise not only due to the sophisticated actions of fraudsters but also because of the large number of financial operations performed by business companies. This is especially true for large companies. It is highly desirable to have a tool to reduce the number of potentially irregular operations significantly. This paper presents an implementation of NLP-based algorithms to identify irregular financial operations using comments left by accountants. The comments are freely written and usually very short remarks used by accountants for personal information. Implementation of content analysis using cosine similarity showed that identification of the type of operation using the comments of accountants is very likely. Further comment content analysis and financial data analysis showed that it could be expected to reduce the number of potentially suspicious operations significantly: analysis of more than half a million financial records of Dutch companies enabled the identification of 0.3% operations that may be potentially suspicious. This could make human financial auditing easier and more robust task. Full article
(This article belongs to the Special Issue Natural Language Processing (NLP) and Applications)
Article
Research on Chinese Medical Entity Recognition Based on Multi-Neural Network Fusion and Improved Tri-Training Algorithm
Appl. Sci. 2022, 12(17), 8539; https://doi.org/10.3390/app12178539 - 26 Aug 2022
Viewed by 396
Abstract
Chinese medical texts contain a large number of medically named entities. Automatic recognition of these medical entities from medical texts is the key to developing medical informatics. In the field of Chinese medical information extraction, annotated Chinese medical text data are very few. [...] Read more.
Chinese medical texts contain a large number of medically named entities. Automatic recognition of these medical entities from medical texts is the key to developing medical informatics. In the field of Chinese medical information extraction, annotated Chinese medical text data are very few. In the named entity recognition task, there is insufficient labeled data, which leads to low model recognition performance. Therefore, this paper proposes a Chinese medical entity recognition model based on multi-neural network fusion and the improved Tri-Training algorithm. The model performs semi-supervised learning by improving the Tri-Training algorithm. According to the characteristics of the medical entity recognition task and medical data, the method in this paper is improved in terms of the division of the initial sub-training set, the construction of the base classifier, and the integration of the learning voting method. In addition, this paper also proposes a multi-neural network fusion entity recognition model for base classifier construction. The model learns feature information jointly by combining Iterated Dilated Convolutional Neural Network (IDCNN) and BiLSTM. Through experimental verification, the model proposed in this paper outperforms other models and improves the performance of the Chinese medical entity recognition model by incorporating and improving the semi-supervised learning algorithm. Full article
(This article belongs to the Special Issue Natural Language Processing (NLP) and Applications)
Show Figures

Figure 1

Article
KGNER: Improving Chinese Named Entity Recognition by BERT Infused with the Knowledge Graph
Appl. Sci. 2022, 12(15), 7702; https://doi.org/10.3390/app12157702 - 30 Jul 2022
Viewed by 521
Abstract
Recently, the lexicon method has been proven to be effective for named entity recognition (NER). However, most existing lexicon-based methods cannot fully utilize common-sense knowledge in the knowledge graph. For example, the word embeddings pretrained by Word2vector or Glove lack better contextual semantic [...] Read more.
Recently, the lexicon method has been proven to be effective for named entity recognition (NER). However, most existing lexicon-based methods cannot fully utilize common-sense knowledge in the knowledge graph. For example, the word embeddings pretrained by Word2vector or Glove lack better contextual semantic information usage. Hence, how to make the best of knowledge for the NER task has become a challenging and hot research topic. We propose a knowledge graph-inspired named-entity recognition (KGNER) featuring a masking and encoding method to incorporate common sense into bidirectional encoder representations from transformers (BERT). The proposed method not only preserves the original sentence semantic information but also takes advantage of the knowledge information in a more reasonable way. Subsequently, we model the temporal dependencies by taking the conditional random field (CRF) as the backend, and improve the overall performance. Experiments on four dominant datasets demonstrate that the KGNER outperforms other lexicon-based models in terms of performance. Full article
(This article belongs to the Special Issue Natural Language Processing (NLP) and Applications)
Show Figures

Figure 1

Article
Boosting the Transformer with the BERT Supervision in Low-Resource Machine Translation
Appl. Sci. 2022, 12(14), 7195; https://doi.org/10.3390/app12147195 - 17 Jul 2022
Viewed by 417
Abstract
Previous works trained the Transformer and its variants end-to-end and achieved remarkable translation performance when there are huge parallel sentences available. However, these models suffer from the data scarcity problem in low-resource machine translation tasks. To deal with the mismatch problem between the [...] Read more.
Previous works trained the Transformer and its variants end-to-end and achieved remarkable translation performance when there are huge parallel sentences available. However, these models suffer from the data scarcity problem in low-resource machine translation tasks. To deal with the mismatch problem between the big model capacity of the Transformer and the small parallel training data set, this paper adds the BERT supervision on the latent representation between the encoder and the decoder of the Transformer and designs a multi-step training algorithm to boost the Transformer on such a basis. The algorithm includes three stages: (1) encoder training, (2) decoder training, and (3) joint optimization. We introduce the BERT of the target language in the encoder and the decoder training and alleviate the data starvation problem of the Transformer. After the training stage, the BERT will not further attend the inference section explicitly. Another merit of our training algorithm is that it can further enhance the Transformer in the task where there are limited parallel sentence pairs but large amounts of monolingual corpus of the target language. The evaluation results on six low-resource translation tasks suggest that the Transformer trained by our algorithm significantly outperforms the baselines which were trained end-to-end in previous works. Full article
(This article belongs to the Special Issue Natural Language Processing (NLP) and Applications)
Show Figures

Figure 1

Article
Empirical Analysis of Parallel Corpora and In-Depth Analysis Using LIWC
Appl. Sci. 2022, 12(11), 5545; https://doi.org/10.3390/app12115545 - 30 May 2022
Viewed by 456
Abstract
The machine translation system aims to translate source language into target language. Recent studies on MT systems mainly focus on neural machine translation. One factor that significantly affects the performance of NMT is the availability of high-quality parallel corpora. However, high-quality parallel corpora [...] Read more.
The machine translation system aims to translate source language into target language. Recent studies on MT systems mainly focus on neural machine translation. One factor that significantly affects the performance of NMT is the availability of high-quality parallel corpora. However, high-quality parallel corpora concerning Korean are relatively scarce compared to those associated with other high-resource languages, such as German or Italian. To address this problem, AI Hub recently released seven types of parallel corpora for Korean. In this study, we conduct an in-depth verification of the quality of corresponding parallel corpora through Linguistic Inquiry and Word Count (LIWC) and several relevant experiments. LIWC is a word-counting software program that can analyze corpora in multiple ways and extract linguistic features as a dictionary base. To the best of our knowledge, this study is the first to use LIWC to analyze parallel corpora in the field of NMT. Our findings suggest the direction of further research toward obtaining the improved quality parallel corpora through our correlation analysis in LIWC and NMT performance. Full article
(This article belongs to the Special Issue Natural Language Processing (NLP) and Applications)
Show Figures

Figure 1

Planned Papers

The below list represents only planned manuscripts. Some of these manuscripts have not been received by the Editorial Office yet. Papers submitted to MDPI journals are subject to peer-review.

Title: KEMMRL: Knowledge Extraction Model for Morphologically Rich Languages
Author:
Highlights: - Deep neural models for morphologically rich languages - Rule based approach for knowledge representation using syntectic and semantic information - State of the art models for Croatian language based on BERT architecture

Back to TopTop