Search for Articles:
Title / Keyword
Author / Affiliation / Email
Journal
Article Type
 
 
Advanced Search
 
Section
Special Issue
Volume
Issue
Number
Page
 
5.3
2.5
Journals
Applied Sciences
Special Issues
Natural Language Processing (NLP) and Applications—2nd Edition
applsci-logo
Submit to Special Issue Submit Abstract to Special Issue Review for Applied Sciences Propose a Special Issue

Journal Menu

► Journal Menu

Journal Browser

► Journal Browser
share announcement

Natural Language Processing (NLP) and Applications—2nd Edition

A special issue of Applied Sciences (ISSN 2076-3417). This special issue belongs to the section "Computing and Artificial Intelligence".

Deadline for manuscript submissions: 30 July 2025 | Viewed by 36715

Special Issue Editors

Prof. Dr. Guilin Qi

E-Mail Website
Guest Editor
School of Computer Science and Engineering, Southeast University, Nanjing 211189, China
Interests: natural language processing; knowledge graph; multimodal learning
Special Issues, Collections and Topics in MDPI journals
Prof. Dr. Tong Xu

E-Mail Website
Guest Editor
Lab of Big Data Analysis and Application, University of Science and Technology of China, Hefei 230027, China
Interests: natural language processing; social media analysis; multimodal intelligence
Special Issues, Collections and Topics in MDPI journals
Dr. Meng Wang

E-Mail Website
Guest Editor
School of Computer Science and Engineering, Southeast University, Nanjing 211189, China
Interests: natural language processing; knowledge graph; multimodal learning
Special Issues, Collections and Topics in MDPI journals

Special Issue Information

Dear Colleagues,

Due to the success of the first edition of the Special Issue of Applied Science, entitled “Natural Language Processing (NLP) and Applications”, we have launched a second edition.

This Special Issue will showcase advances in NLP and its applications, including significant advances in sentiment analysis, machine translation, semantic understanding, and more. Large-scale pre-trained models such as BERT and GPT-3 have revolutionized NLP and provided a solid foundation for future advancement. The transformer design enhances cross-language and multi-modal intelligence. However, NLP still faces challenges such as unsupervised learning, model generalization, and linguistic diversity. Factors such as background, language and culture should be considered in real applications. This SI invites experts and scholars from around the world to share their latest research results and technological advances in order to provide more inspiration and ideas for the future development of NLP.

Prof. Dr. Guilin Qi
Prof. Dr. Tong Xu
Dr. Meng Wang
Guest Editors

Manuscript Submission Information

Manuscripts should be submitted online at www.mdpi.com by registering and logging in to this website. Once you are registered, click here to go to the submission form. Manuscripts can be submitted until the deadline. All submissions that pass pre-check are peer-reviewed. Accepted papers will be published continuously in the journal (as soon as accepted) and will be listed together on the special issue website. Research articles, review articles as well as short communications are invited. For planned papers, a title and short abstract (about 100 words) can be sent to the Editorial Office for announcement on this website.

Submitted manuscripts should not have been published previously, nor be under consideration for publication elsewhere (except conference proceedings papers). All manuscripts are thoroughly refereed through a single-blind peer-review process. A guide for authors and other relevant information for submission of manuscripts is available on the Instructions for Authors page. Applied Sciences is an international peer-reviewed open access semimonthly journal published by MDPI.

Please visit the Instructions for Authors page before submitting a manuscript. The Article Processing Charge (APC) for publication in this open access journal is 2400 CHF (Swiss Francs). Submitted papers should be well formatted and use good English. Authors may use MDPI's English editing service prior to publication or during author revisions.

Keywords

  • natural language understanding
  • natural language generation
  • machine translation
  • knowledge graph
  • NLP for knowledge extraction
  • NLP for multimodal intelligence
  • NLP applications in specific domains, like life sciences, health, and medicine
  • eGovernment and public administration

Benefits of Publishing in a Special Issue

  • Ease of navigation: Grouping papers by topic helps scholars navigate broad scope journals more efficiently.
  • Greater discoverability: Special Issues support the reach and impact of scientific research. Articles in Special Issues are more discoverable and cited more frequently.
  • Expansion of research network: Special Issues facilitate connections among authors, fostering scientific collaborations.
  • External promotion: Articles in Special Issues are often promoted through the journal's social media, increasing their visibility.
  • e-Book format: Special Issues with more than 10 articles can be published as dedicated e-books, ensuring wide and rapid dissemination.

Further information on MDPI's Special Issue policies can be found here.

Published Papers (22 papers)

Download All Papers
Order results
Result details
Show export options expand_more Show export options expand_less
Select all
Export citation of selected articles as:

Research

23 pages, 10500 KiB  
Article
Advanced Default Risk Prediction in Small and Medum-Sized Enterprises Using Large Language Models
by Haonan Huang, Jing Li, Chundan Zheng, Sikang Chen, Xuanyin Wang and Xingyan Chen
Appl. Sci. 2025, 15(5), 2733; https://doi.org/10.3390/app15052733 - 4 Mar 2025
Viewed by 778
Abstract
Predicting default risk in commercial bills for small and medium-sized enterprises (SMEs) is crucial, as these enterprises represent one of the largest components of a nation’s economic structure, and their financial stability can impact systemic financial risk. However, data on the commercial bills [...] Read more.
Predicting default risk in commercial bills for small and medium-sized enterprises (SMEs) is crucial, as these enterprises represent one of the largest components of a nation’s economic structure, and their financial stability can impact systemic financial risk. However, data on the commercial bills of SMEs are scarce and challenging to gather, which has impeded research on risk prediction for these businesses. This study aims to address this gap by leveraging 38 multi-dimensional, non-financial features collected from 1972 real SMEs in China to predict bill default risk. We identified the most influential factors among these 38 features and introduced a novel prompt-based learning framework using large language models for risk assessment, benchmarking against seven mainstream machine learning algorithms. In the experiments, the XGBoost algorithm achieved the best performance on the Z-Score standardized dataset, with an accuracy of 81.42% and an F1 score of 80%. Additionally, we tested both the standard and fine-tuned versions of the large language model, which yielded accuracies of 75% and 82.1%, respectively. These results indicate that the proposed framework has significant potential for predicting risks in SMEs and offers new insights for related research. Full article
(This article belongs to the Special Issue Natural Language Processing (NLP) and Applications—2nd Edition)
►▼ Show Figures

Figure 1

17 pages, 2395 KiB  
Article
Automated Dataset-Creation and Evaluation Pipeline for NER in Russian Literary Heritage
by Kenan Kassab, Nikolay Teslya and Ekaterina Vozhik
Appl. Sci. 2025, 15(4), 2072; https://doi.org/10.3390/app15042072 - 16 Feb 2025
Viewed by 627
Abstract
Developing robust and reliable models for Named Entity Recognition (NER) in the Russian language presents significant challenges due to the linguistic complexity of Russian and the limited availability of suitable training datasets. This study introduces a semi-automated methodology for building a customized Russian [...] Read more.
Developing robust and reliable models for Named Entity Recognition (NER) in the Russian language presents significant challenges due to the linguistic complexity of Russian and the limited availability of suitable training datasets. This study introduces a semi-automated methodology for building a customized Russian dataset for NER specifically designed for literary purposes. The paper provides a detailed description of the methodology employed for collecting and proofreading the dataset, outlining the pipeline used for processing and annotating its contents. A comprehensive analysis highlights the dataset’s richness and diversity. Central to the proposed approach is the use of a voting system to facilitate the efficient elicitation of entities, enabling significant time and cost savings compared to traditional methods of constructing NER datasets. The voting system is described theoretically and mathematically to highlight its impact on enhancing the annotation process. The results of testing the voting system with various thresholds show its impact in increasing the overall precision by 28% compared to using only the state-of-the-art model for auto-annotating. The dataset is meticulously annotated and thoroughly proofread, ensuring its value as a high-quality resource for training and evaluating NER models. Empirical evaluations using multiple NER models underscore the dataset’s importance and its potential to enhance the robustness and reliability of NER models in the Russian language. Full article
(This article belongs to the Special Issue Natural Language Processing (NLP) and Applications—2nd Edition)
►▼ Show Figures

Figure 1

16 pages, 3369 KiB  
Article
Authorship Detection on Classical Chinese Text Using Deep Learning
by Lingmei Zhao, Jianjun Shi, Chenkai Zhang and Zhixiang Liu
Appl. Sci. 2025, 15(4), 1677; https://doi.org/10.3390/app15041677 - 7 Feb 2025
Viewed by 916
Abstract
Authorship detection has played an important role in social information science. In this study, we propose a support vector machine (SVM)-based authorship detection model for classical Chinese texts. Term frequency-inverse document frequency (TF-IDF) feature extraction technique is combined with the SVM-based method. The [...] Read more.
Authorship detection has played an important role in social information science. In this study, we propose a support vector machine (SVM)-based authorship detection model for classical Chinese texts. Term frequency-inverse document frequency (TF-IDF) feature extraction technique is combined with the SVM-based method. The linguistic features used in this model are based on TF-DIF calculations of different function words, including literary Chinese words, end-function words, vernacular function words, and transitional function words. Furthermore, a bidirectional long short-term memory (BiLSTM)-based authorship model is introduced to detect authorship in classical Chinese texts. The BiLSTM model incorporates an attention mechanism to better capture the meaning and weight of the words. We conduct a comparative analysis between the SVM-based and BiLSTM-based models in the context of authorship detection in Chinese classical literature. The applicability of the two authorship detection models for classical Chinese texts is examined. Results indicate varying authorship between different sections of the texts, with the SVM model outperforming the BiLSTM model. Notably, these classification outcomes are consistent with findings from prior studies in classical Chinese literary analysis. The proposed SVM-based authorship detection model is especially suited for automatic literary analysis, which underscores its potential for broader literary studies. Full article
(This article belongs to the Special Issue Natural Language Processing (NLP) and Applications—2nd Edition)
►▼ Show Figures

Figure 1

16 pages, 2375 KiB  
Article
Research on Co-Interactive Model Based on Knowledge Graph for Intent Detection and Slot Filling
by Wenwen Zhang, Yanfang Gao, Zifan Xu, Lin Wang, Shengxu Ji, Xiaohui Zhang and Guanyu Yuan
Appl. Sci. 2025, 15(2), 547; https://doi.org/10.3390/app15020547 - 8 Jan 2025
Viewed by 872
Abstract
Intent detection and slot filling tasks share common semantic features and are interdependent. The abundance of professional terminology in specific domains, which poses difficulties for entity recognition, subsequently impacts the performance of intent detection. To address this issue, this paper proposes a co-interactive [...] Read more.
Intent detection and slot filling tasks share common semantic features and are interdependent. The abundance of professional terminology in specific domains, which poses difficulties for entity recognition, subsequently impacts the performance of intent detection. To address this issue, this paper proposes a co-interactive model based on a knowledge graph (CIMKG) for intent detection and slot filling. The CIMKG model comprises three key components: (1) a knowledge graph-based shared encoder module that injects domain-specific expertise to enhance its semantic representation and solve the problem of entity recognition difficulties caused by professional terminology and then encodes short utterances; (2) a co-interactive module that explicitly establishes the relationship between intent detection and slot filling to address the inter-dependency of these processes; (3) two decoders that decode the intent detection and slot filling. The proposed CIMKG model has been validated using question–answer corpora from both the medical and architectural safety fields. The experimental results demonstrate that the proposed CIMKG model outperforms benchmark models. Full article
(This article belongs to the Special Issue Natural Language Processing (NLP) and Applications—2nd Edition)
►▼ Show Figures

Figure 1

37 pages, 1179 KiB  
Article
A Rule-Based Parser in Comparison with Statistical Neuronal Approaches in Terms of Grammar Competence
by Simon M. Strübbe, Alexander T. D. Grünwald, Irina Sidorenko and Renée Lampe
Appl. Sci. 2025, 15(1), 87; https://doi.org/10.3390/app15010087 - 26 Dec 2024
Cited by 2 | Viewed by 899
Abstract
The “Easy Language” standard was created to help individuals with cognitive disabilities understand texts more easily. Typically, text simplification is performed by language experts and is available for limited materials. We introduce a new software tool designed to analyze and simplify any text [...] Read more.
The “Easy Language” standard was created to help individuals with cognitive disabilities understand texts more easily. Typically, text simplification is performed by language experts and is available for limited materials. We introduce a new software tool designed to analyze and simplify any text according to the “Easy Language” rules. This tool uses a rule-based system, conducting a full grammatical analysis of each sentence and then simplifying it into a grammatically correct form. Unlike neuronal approaches, which are based on statistics and are very popular today, our rule-based approach explicitly addresses language ambiguities by examining all possible interpretations and eliminating the incorrect ones. The purpose of the present study is to compare the performance of our rule-base parser with two state-of-the-art statistical parsers, one based on dependencies between words (SpaCy parser) and the other based on linguistic constituents (Stanford parser). Although large language models (LLMs), which are the technical basis of the software ChatGPT, were not designed specifically for grammatical parsing, because of their popularity, users, especially language learners, often ask them grammatical questions as well. Therefore, we use LLMs as supplementary models for comparison. LMMs produce grammatically correct text on any topic; however, their grammar knowledge is implicit within the trained weights. To evaluate how well state-of-the-art methods can perform a grammatical analysis, we parse ten sentences with our tool, the statistical parsers from SpaCy and Stanford, and ask two LLMs equivalent grammar questions. The results show that our rule-based method provides a more informative and reliable grammatical analysis compared to these two parsers and outperforms LLMs in that specific task. Full article
(This article belongs to the Special Issue Natural Language Processing (NLP) and Applications—2nd Edition)
►▼ Show Figures

Figure 1

15 pages, 1333 KiB  
Article
GoalBERT: A Lightweight Named-Entity Recognition Model Based on Multiple Fusion
by Yingjie Xu, Xiaobo Tan, Mengxuan Wang and Wenbo Zhang
Appl. Sci. 2024, 14(23), 11003; https://doi.org/10.3390/app142311003 - 26 Nov 2024
Viewed by 808
Abstract
Named-Entity Recognition (NER) as a core task in Natural Language Processing (NLP) aims to automatically identify and classify specific types of entities from unstructured text. In recent years, the introduction of Transformer architecture and its derivative BERT model has pushed the performance of [...] Read more.
Named-Entity Recognition (NER) as a core task in Natural Language Processing (NLP) aims to automatically identify and classify specific types of entities from unstructured text. In recent years, the introduction of Transformer architecture and its derivative BERT model has pushed the performance of NER to unprecedented heights. However, these models often have high requirements for computational power and memory resources, making it difficult to train and deploy them on small computing platforms. Although ALBERT as a lightweight model uses parameter sharing and matrix decomposition strategies to reduce memory consumption to some extent consumption, it does not effectively reduce the computational load of the model. Additionally, due to its internal sharing mechanism, the model’s understanding ability of text is reduced leading to poor performance in named-entity recognition tasks. To address these challenges, this manuscript proposes an efficient lightweight model called GoalBERT. The model adopts multiple fusion technologies by integrating a lightweight and efficient BiGRU that excels at handling context into part of the Transformer’s self-attention layers. This reduces the high computational demand caused by stacking multiple self-attention layers while enhancing the model’s ability to process context information. To solve the problem of gradient disappearance and explosion during training, residual connections are added between core layers for more stable training and steady performance improvement. Experimental results show that GoalBERT demonstrates recognition accuracy comparable to standard models with accuracy surpassing ALBERT by 10% in multi-entity type scenarios. Furthermore, compared to standard models, GoalBERT reduces memory requirements by 200% and improves training speed by nearly 230%. Experimental results indicate that GoalBERT is a high-quality lightweight model. Full article
(This article belongs to the Special Issue Natural Language Processing (NLP) and Applications—2nd Edition)
►▼ Show Figures

Figure 1

16 pages, 432 KiB  
Article
NAS-CRE: Neural Architecture Search for Context-Based Relation Extraction
by Rongen Yan, Dongmei Li, Yan Wu, Depeng Dang, Ye Tao and Shaofei Wang
Appl. Sci. 2024, 14(23), 10960; https://doi.org/10.3390/app142310960 - 26 Nov 2024
Viewed by 748
Abstract
Relation extraction, a crucial task in natural language processing (NLP) for constructing knowledge graphs, entails extracting relational semantics between pairs of entities within a sentence. Given the intricacy of language, a single sentence often encompasses multiple entities that mutually influence one another. Recently, [...] Read more.
Relation extraction, a crucial task in natural language processing (NLP) for constructing knowledge graphs, entails extracting relational semantics between pairs of entities within a sentence. Given the intricacy of language, a single sentence often encompasses multiple entities that mutually influence one another. Recently, various iterations of recurrent neural networks (RNNs) have been introduced into relation extraction tasks, where the efficacy of neural network structures directly influences task performance. However, many neural networks necessitate manual determination of optimal parameters and network architectures, resulting in limited generalization capabilities for specific tasks. In this paper, we formally define the context-based relation extraction problem and propose a solution utilizing neural architecture search (NAS) to optimize RNN. Specifically, NAS employs an RNN controller to delineate an RNN cell, yielding an optimal structure to represent all relationships, thereby aiding in extracting relationships between target entities. Additionally, to enhance relation extraction performance, we leverage the XLNet pretrained model to comprehensively capture the semantic features of the sentence. Extensive experiments conducted on a real-world dataset containing words with multiple relationships demonstrate that our proposed method significantly enhances micro-F1 scores compared to state-of-the-art baselines. Full article
(This article belongs to the Special Issue Natural Language Processing (NLP) and Applications—2nd Edition)
►▼ Show Figures

Figure 1

14 pages, 563 KiB  
Article
An NLP-Based Perfume Note Estimation Based on Descriptive Sentences
by Jooyoung Kim, Kangrok Oh and Beom-Seok Oh
Appl. Sci. 2024, 14(20), 9293; https://doi.org/10.3390/app14209293 - 12 Oct 2024
Viewed by 1872
Abstract
The perfume industry is a suitable candidate for applying advanced natural language processing techniques, yet most existing studies focus on developing fragrance design systems based on artificial intelligence advances. To meet the increasing demand for analyzing and exploiting descriptive sentences for the fragrance [...] Read more.
The perfume industry is a suitable candidate for applying advanced natural language processing techniques, yet most existing studies focus on developing fragrance design systems based on artificial intelligence advances. To meet the increasing demand for analyzing and exploiting descriptive sentences for the fragrance market, we investigate the relationship between descriptive sentences of perfumes and their notes in this paper. Our purpose for this investigation is to build a core idea for a perfume recommendation system of descriptive sentences. To accomplish this, we propose a system for perfume note estimation of descriptive sentences based on several sentence transformer models. In our leave-one-out cross-validation tests using our dataset containing 62 perfumes and 255 perfume notes, we achieved significant performance improvements (from a 37.1∼41.1% to 72.6∼79.0% hit rate with the top five items, and from a 22.1∼31.9% to a 57.3∼63.2% mean reciprocal rank) for perfume note estimation via our fine-tuning process. In addition, some qualitative examples, including query descriptions, estimated perfume notes, and the ground truth perfume notes, are presented. The proposed system improves the perfume note estimation performances using a fine-tuning process on a newly constructed dataset containing descriptive sentences of perfumes and their notes. Full article
(This article belongs to the Special Issue Natural Language Processing (NLP) and Applications—2nd Edition)
►▼ Show Figures

Figure 1

15 pages, 885 KiB  
Article
A Character-Word Information Interaction Framework for Natural Language Understanding in Chinese Medical Dialogue Domain
by Pei Cao, Zhongtao Yang, Xinlu Li and Yu Li
Appl. Sci. 2024, 14(19), 8926; https://doi.org/10.3390/app14198926 - 3 Oct 2024
Cited by 1 | Viewed by 1057
Abstract
Natural language understanding is a foundational task in medical dialogue systems. However, there are still two key problems to be solved: (1) Multiple meanings of a word lead to ambiguity of intent; (2) character errors make slot entity extraction difficult. To solve the [...] Read more.
Natural language understanding is a foundational task in medical dialogue systems. However, there are still two key problems to be solved: (1) Multiple meanings of a word lead to ambiguity of intent; (2) character errors make slot entity extraction difficult. To solve the above problems, this paper proposes a character-word information interaction framework (CWIIF) for natural language understanding in the Chinese medical dialogue domain. The CWIIF framework contains an intent information adapter to solve the problem of intent ambiguity caused by multiple meanings of words in the intent detection task and a slot label extractor to solve the problem of difficulty in yellowslot entity extraction due to character errors in the slot filling task. The proposed framework is validated on two publicly available datasets, the Intelligent Medical Consultation System (IMCS-21) and Chinese Artificial Intelligence Speakers (CAIS). Experimental results from both datasets demonstrate that the proposed framework outperforms other baseline methods in handling Chinese medical dialogues. Notably, on the IMCS-21 dataset, precision improved by 2.42%, recall by 3.01%, and the F1 score by 2.4%. Full article
(This article belongs to the Special Issue Natural Language Processing (NLP) and Applications—2nd Edition)
►▼ Show Figures

Figure 1

16 pages, 1928 KiB  
Article
A New Chinese Named Entity Recognition Method for Pig Disease Domain Based on Lexicon-Enhanced BERT and Contrastive Learning
by Cheng Peng, Xiajun Wang, Qifeng Li, Qinyang Yu, Ruixiang Jiang, Weihong Ma, Wenbiao Wu, Rui Meng, Haiyan Li, Heju Huai, Shuyan Wang and Longjuan He
Appl. Sci. 2024, 14(16), 6944; https://doi.org/10.3390/app14166944 - 8 Aug 2024
Viewed by 1261
Abstract
Named Entity Recognition (NER) is a fundamental and pivotal stage in the development of various knowledge-based support systems, including knowledge retrieval and question-answering systems. In the domain of pig diseases, Chinese NER models encounter several challenges, such as the scarcity of annotated data, [...] Read more.
Named Entity Recognition (NER) is a fundamental and pivotal stage in the development of various knowledge-based support systems, including knowledge retrieval and question-answering systems. In the domain of pig diseases, Chinese NER models encounter several challenges, such as the scarcity of annotated data, domain-specific vocabulary, diverse entity categories, and ambiguous entity boundaries. To address these challenges, we propose PDCNER, a Pig Disease Chinese Named Entity Recognition method leveraging lexicon-enhanced BERT and contrastive learning. Firstly, we construct a domain-specific lexicon and pre-train word embeddings in the pig disease domain. Secondly, we integrate lexicon information of pig diseases into the lower layers of BERT using a Lexicon Adapter layer, which employs char–word pair sequences. Thirdly, to enhance feature representation, we propose a lexicon-enhanced contrastive loss layer on top of BERT. Finally, a Conditional Random Field (CRF) layer is employed as the model’s decoder. Experimental results show that our proposed model demonstrates superior performance over several mainstream models, achieving a precision of 87.76%, a recall of 86.97%, and an F1-score of 87.36%. The proposed model outperforms BERT-BiLSTM-CRF and LEBERT by 14.05% and 6.8%, respectively, with only 10% of the samples available, showcasing its robustness in data scarcity scenarios. Furthermore, the model exhibits generalizability across publicly available datasets. Our work provides reliable technical support for the information extraction of pig diseases in Chinese and can be easily extended to other domains, thereby facilitating seamless adaptation for named entity identification across diverse contexts. Full article
(This article belongs to the Special Issue Natural Language Processing (NLP) and Applications—2nd Edition)
►▼ Show Figures

Figure 1

15 pages, 925 KiB  
Article
Entity-Alignment Interaction Model Based on Chinese RoBERTa
by Ping Feng, Boning Zhang, Lin Yang and Shiyu Feng
Appl. Sci. 2024, 14(14), 6162; https://doi.org/10.3390/app14146162 - 15 Jul 2024
Viewed by 1166
Abstract
Entity alignment aims to match entities with the same semantics from different knowledge graphs. Most existing studies use neural networks to combine graph-structure information and additional entity information (such as names, descriptions, images, and attributes) to achieve entity alignment. However, due to the [...] Read more.
Entity alignment aims to match entities with the same semantics from different knowledge graphs. Most existing studies use neural networks to combine graph-structure information and additional entity information (such as names, descriptions, images, and attributes) to achieve entity alignment. However, due to the heterogeneity of knowledge graphs, aligned entities often do not have the same neighbors, which makes it difficult to utilize the structural information from knowledge graphs and results in a decrease in alignment accuracy. Therefore, in this paper, we propose an interaction model that exploits only the additional information on entities. Our model utilizes names, attributes, and neighbors of entities for interaction and introduces attention interaction to extract features to further evaluate the matching scores between entities. Our model is applicable to Chinese datasets, and experimental results show that it has achieved good results on the Chinese medical datasets denoted MED-BBK-9K. Full article
(This article belongs to the Special Issue Natural Language Processing (NLP) and Applications—2nd Edition)
►▼ Show Figures

Figure 1

21 pages, 409 KiB  
Article
Transferring Sentiment Cross-Lingually within and across Same-Family Languages
by Gaurish Thakkar, Nives Mikelić Preradović and Marko Tadić
Appl. Sci. 2024, 14(13), 5652; https://doi.org/10.3390/app14135652 - 28 Jun 2024
Viewed by 1142
Abstract
Natural language processing for languages with limited resources is hampered by a lack of data. Using English as a hub language for such languages, cross-lingual sentiment analysis has been developed. The sheer quantity of English language resources raises questions about its status as [...] Read more.
Natural language processing for languages with limited resources is hampered by a lack of data. Using English as a hub language for such languages, cross-lingual sentiment analysis has been developed. The sheer quantity of English language resources raises questions about its status as the primary resource. This research aims to examine the impact on sentiment analysis of adding data from same-family versus distant-family languages. We analyze the performance using low-resource and high-resource data from the same language family (Slavic), investigate the effect of using a distant-family language (English) and report the results for both settings. Quantitative experiments using multi-task learning demonstrate that adding a large quantity of data from related and distant-family languages is advantageous for cross-lingual sentiment transfer. Full article
(This article belongs to the Special Issue Natural Language Processing (NLP) and Applications—2nd Edition)
►▼ Show Figures

Figure 1

13 pages, 1591 KiB  
Article
Evaluating Neural Networks’ Ability to Generalize against Adversarial Attacks in Cross-Lingual Settings
by Vidhu Mathur, Tanvi Dadu and Swati Aggarwal
Appl. Sci. 2024, 14(13), 5440; https://doi.org/10.3390/app14135440 - 23 Jun 2024
Cited by 1 | Viewed by 1229
Abstract
Cross-lingual transfer learning using multilingual models has shown promise for improving performance on natural language processing tasks with limited training data. However, translation can introduce superficial patterns that negatively impact model generalization. This paper evaluates two state-of-the-art multilingual models, Cross-Lingual Model-Robustly Optimized BERT [...] Read more.
Cross-lingual transfer learning using multilingual models has shown promise for improving performance on natural language processing tasks with limited training data. However, translation can introduce superficial patterns that negatively impact model generalization. This paper evaluates two state-of-the-art multilingual models, Cross-Lingual Model-Robustly Optimized BERT Pretraining Approach (XLM-Roberta) and Multilingual Bi-directional Auto-Regressive Transformer (mBART), on the cross-lingual natural language inference (XNLI) natural language inference task using both original and machine-translated evaluation sets. Our analysis demonstrates that translation can facilitate cross-lingual transfer learning, but maintaining linguistic patterns is critical. The results provide insights into the strengths and limitations of state-of-the-art multilingual natural language processing architectures for cross-lingual understanding. Full article
(This article belongs to the Special Issue Natural Language Processing (NLP) and Applications—2nd Edition)
►▼ Show Figures

Figure 1

16 pages, 410 KiB  
Article
STOD: Towards Scalable Task-Oriented Dialogue System on MultiWOZ-API
by Hengtong Lu, Caixia Yuan and Xiaojie Wang
Appl. Sci. 2024, 14(12), 5303; https://doi.org/10.3390/app14125303 - 19 Jun 2024
Viewed by 1166
Abstract
Task-oriented dialogue systems (TODs) enable users to complete specific goals and are widely used in practice. Although existing models have achieved delightful performance for single-domain dialogues, scalability to new domains is far from well explored. Traditional dialogue systems rely on domain-specific information like [...] Read more.
Task-oriented dialogue systems (TODs) enable users to complete specific goals and are widely used in practice. Although existing models have achieved delightful performance for single-domain dialogues, scalability to new domains is far from well explored. Traditional dialogue systems rely on domain-specific information like dialogue state and database (DB), which limits the scalability of such systems. In this paper, we propose a Scalable Task-Oriented Dialogue modeling framework (STOD). Instead of labeling multiple dialogue components, which have been adopted by previous work, we only predict structured API queries to interact with DB and generate responses based on the complete DB results. Further, we construct a new API-schema-based TOD dataset MultiWOZ-API with API query and DB result annotation based on MultiWOZ 2.1. We then propose MSTOD and CSTOD for multi-domain and cross-domain TOD systems, respectively. We perform extensive qualitative experiments to verify the effectiveness of our proposed framework. We find the following. (1) Scalability across multiple domains: MSTOD achieves 2% improvements than the previous state-of-the-art in the multi-domain TOD. (2) Scalability to new domains: our framework enables satisfying generalization capability to new domains, a significant margin of 10% to existing baselines. Full article
(This article belongs to the Special Issue Natural Language Processing (NLP) and Applications—2nd Edition)
►▼ Show Figures

Figure 1

22 pages, 877 KiB  
Article
Towards Media Monitoring: Detecting Known and Emerging Topics through Multilingual and Crosslingual Text Classification
by Jurgita Kapočiūtė-Dzikienė and Arūnas Ungulaitis
Appl. Sci. 2024, 14(10), 4320; https://doi.org/10.3390/app14104320 - 20 May 2024
Cited by 1 | Viewed by 1728
Abstract
This study aims to address challenges in media monitoring by enhancing closed-set topic classification in multilingual contexts (where both training and testing occur in several languages) and crosslingual contexts (where training is in English and testing spans all languages). To achieve this goal, [...] Read more.
This study aims to address challenges in media monitoring by enhancing closed-set topic classification in multilingual contexts (where both training and testing occur in several languages) and crosslingual contexts (where training is in English and testing spans all languages). To achieve this goal, we utilized a dataset from the European Media Monitoring webpage, which includes approximately 15,000 article titles across 18 topics in 58 different languages spanning a period of nine months from May 2022 to March 2023. Our research conducted comprehensive comparative analyses of nine approaches, encompassing a spectrum of embedding techniques (word, sentence, and contextual representations) and classifiers (trainable/fine-tunable, memory-based, and generative). Our findings reveal that the LaBSE+FFNN approach achieved the best performance, reaching macro-averaged F1-scores of 0.944 ± 0.015 and 0.946 ± 0.019 in both multilingual and crosslingual scenarios. LaBSE+FFNN’s similar performance in multilingual and crosslingual scenarios eliminates the need for machine translation into English. We also tackled the open-set topic classification problem by training a binary classifier capable of distinguishing between known and new topics with the average loss of ∼0.0017 ± 0.0002. Various feature types were investigated, reaffirming the robustness of LaBSE vectorization. The experiments demonstrate that, depending on the topic, new topics can be identified with accuracies above ∼0.796 and of ∼0.9 on average. Both closed-set and open-set topic classification modules, along with additional mechanisms for clustering new topics to organize and label them, are integrated into our media monitoring system, which is now used by our real client. Full article
(This article belongs to the Special Issue Natural Language Processing (NLP) and Applications—2nd Edition)
►▼ Show Figures

Figure 1

18 pages, 2271 KiB  
Article
Document Retrieval System for Biomedical Question Answering
by Harun Bolat and Baha Şen
Appl. Sci. 2024, 14(6), 2613; https://doi.org/10.3390/app14062613 - 20 Mar 2024
Cited by 1 | Viewed by 2366
Abstract
In this paper, we describe our biomedical document retrieval system and answers extraction module, which is part of the biomedical question answering system. Approximately 26.5 million PubMed articles are indexed as a corpus with the Apache Lucene text search engine. Our proposed system [...] Read more.
In this paper, we describe our biomedical document retrieval system and answers extraction module, which is part of the biomedical question answering system. Approximately 26.5 million PubMed articles are indexed as a corpus with the Apache Lucene text search engine. Our proposed system consists of three parts. The first part is the question analysis module, which analyzes the question and enriches it with biomedical concepts related to its wording. The second part of the system is the document retrieval module. In this step, the proposed system is tested using different information retrieval models, like the Vector Space Model, Okapi BM25, and Query Likelihood. The third part is the document re-ranking module, which is responsible for re-arranging the documents retrieved in the previous step. For this study, we tested our proposed system with 6B training questions from the BioASQ challenge task. We obtained the best MAP score on the document retrieval phase when we used Query Likelihood with the Dirichlet Smoothing model. We used the sequential dependence model at the re-rank phase, but this model produced a worse MAP score than the previous phase. In similarity calculation, we included the Named Entity Recognition (NER), UMLS Concept Unique Identifiers (CUI), and UMLS Semantic Types of the words in the question to find the sentences containing the answer. Using this approach, we observed a performance enhancement of roughly 25% for the top 20 outcomes, surpassing another method employed in this study, which relies solely on textual similarity. Full article
(This article belongs to the Special Issue Natural Language Processing (NLP) and Applications—2nd Edition)
►▼ Show Figures

Figure 1

14 pages, 1216 KiB  
Article
Margin and Shared Proxies: Advanced Proxy Anchor Loss for Out-of-Domain Intent Classification
by Junhyeong Park, Byeonghun Kim, Sangkwon Han, Seungbin Ji and Jongtae Rhee
Appl. Sci. 2024, 14(6), 2312; https://doi.org/10.3390/app14062312 - 9 Mar 2024
Viewed by 1464
Abstract
Out-of-Domain (OOD) intent classification is an important task for a dialog system, as it allows for appropriate responses to be generated. Previous studies aiming to solve the OOD intent classification task have generally adopted metric learning methods to generate decision boundaries in the [...] Read more.
Out-of-Domain (OOD) intent classification is an important task for a dialog system, as it allows for appropriate responses to be generated. Previous studies aiming to solve the OOD intent classification task have generally adopted metric learning methods to generate decision boundaries in the embedding space. However, these existing methods struggle to capture the high-dimensional semantic features of data, as they learn decision boundary using scalar distances. They also use generated OOD samples for learning. However, such OOD samples are biased, and they cannot include all real-world OOD intents, thus representing a limitation. In the current paper, we attempt to overcome these challenges by using Advanced Proxy-Anchor loss, which introduces a margin proxy and shared proxy. First, to generate a decision boundary that has the high-dimensional semantic features of training data, we use a margin proxy for learnable embedding vectors. Next, the shared proxy, which is shared by all In-Domain (IND) samples, is introduced to make it possible to learn the discriminative feature between IND intents and OOD intent, ultimately leading to the improved classification of OOD samples. We conduct evaluations of the proposed method using three benchmark datasets. The experimental results demonstrate that our method achieved an improved performance compared to the methods described in previous studies. Full article
(This article belongs to the Special Issue Natural Language Processing (NLP) and Applications—2nd Edition)
►▼ Show Figures

Figure 1

17 pages, 1401 KiB  
Article
A Study on the Emotional Tendency of Aquatic Product Quality and Safety Texts Based on Emotional Dictionaries and Deep Learning
by Xingxing Tong, Ming Chen and Guofu Feng
Appl. Sci. 2024, 14(5), 2119; https://doi.org/10.3390/app14052119 - 4 Mar 2024
Cited by 1 | Viewed by 1377
Abstract
The issue of aquatic product quality and safety has gradually become a focal point of societal concern. Analyzing textual comments from people about aquatic products aids in promptly understanding the current sentiment landscape regarding the quality and safety of aquatic products. To address [...] Read more.
The issue of aquatic product quality and safety has gradually become a focal point of societal concern. Analyzing textual comments from people about aquatic products aids in promptly understanding the current sentiment landscape regarding the quality and safety of aquatic products. To address the challenge of the polysemy of modern network buzzwords in word vector representation, we construct a custom sentiment lexicon and employ the Roberta-wwm-ext model to extract semantic feature representations from comment texts. Subsequently, the obtained semantic features of words are put into a bidirectional LSTM model for sentiment classification. This paper validates the effectiveness of the proposed model in the sentiment analysis of aquatic product quality and safety texts by constructing two datasets, one for salmon and one for shrimp, sourced from comments on JD.com. Multiple comparative experiments were conducted to assess the performance of the model on these datasets. The experimental results demonstrate significant achievements using the proposed model, achieving a classification accuracy of 95.49%. This represents a notable improvement of 6.42 percentage points compared to using Word2Vec and a 2.06 percentage point improvement compared to using BERT as the word embedding model. Furthermore, it outperforms LSTM by 2.22 percentage points and textCNN by 2.86 percentage points in terms of semantic extraction models. The outstanding effectiveness of the proposed method is strongly validated by these results. It provides more accurate technical support for calculating the concentration of negative emotions using a risk assessment system in public opinion related to quality and safety. Full article
(This article belongs to the Special Issue Natural Language Processing (NLP) and Applications—2nd Edition)
►▼ Show Figures

Figure 1

25 pages, 545 KiB  
Article
Applying Named Entity Recognition and Graph Networks to Extract Common Interests from Thematic Subfora on Reddit
by Jan Sawicki, Maria Ganzha, Marcin Paprzycki and Yutaka Watanobe
Appl. Sci. 2024, 14(5), 1696; https://doi.org/10.3390/app14051696 - 20 Feb 2024
Viewed by 2340
Abstract
Reddit is the largest topically structured social network. Existing literature, reporting results of Reddit-related research, considers different phenomena, from social and political studies to recommender systems. The most common techniques used in these works, include natural language processing, e.g., named entity recognition, as [...] Read more.
Reddit is the largest topically structured social network. Existing literature, reporting results of Reddit-related research, considers different phenomena, from social and political studies to recommender systems. The most common techniques used in these works, include natural language processing, e.g., named entity recognition, as well as graph networks representing online social networks. However, large-scale studies that take into account Reddit’s unique structure are scarce. In this contribution, similarity between subreddits is explored. Specifically, subreddit posts (from 3189 subreddits, spanning the year 2022) are processed using NER to build graph networks which are further mined for relations between subreddits. The evaluation of obtained results follows the state-of-the-art approaches used for a similar problem, i.e., recommender system metrics, and applies recall and AUC. Overall, the use of Reddit crossposts discloses previously unknown relations between subreddits. Interestingly, the proposed approach may allow for researchers to better connect their study topics with particular subreddits and shows promise for subreddit similarity mining. Full article
(This article belongs to the Special Issue Natural Language Processing (NLP) and Applications—2nd Edition)
►▼ Show Figures

Figure 1

24 pages, 4014 KiB  
Article
A Modular Framework for Domain-Specific Conversational Systems Powered by Never-Ending Learning
by Felipe Coelho de Abreu Pinna, Victor Takashi Hayashi, João Carlos Néto, Rosangela de Fátima Pereira Marquesone, Maísa Cristina Duarte, Rodrigo Suzuki Okada and Wilson Vicente Ruggiero
Appl. Sci. 2024, 14(4), 1585; https://doi.org/10.3390/app14041585 - 16 Feb 2024
Viewed by 2086
Abstract
Complex and long interactions (e.g., a change of topic during a conversation) justify the use of dialog systems to develop task-oriented chatbots and intelligent virtual assistants. The development of dialog systems requires considerable effort and takes more time to deliver when compared to [...] Read more.
Complex and long interactions (e.g., a change of topic during a conversation) justify the use of dialog systems to develop task-oriented chatbots and intelligent virtual assistants. The development of dialog systems requires considerable effort and takes more time to deliver when compared to regular BotBuilder tools because of time-consuming tasks such as training machine learning models and low module reusability. We propose a framework for building scalable dialog systems for specific domains using the semi-automatic methods of corpus, ontology, and code development. By separating the dialog application logic from domain knowledge in the form of an ontology, we were able to create a dialog system for the banking domain in the Portuguese language and quickly change the domain of the conversation by changing the ontology. Moreover, by using the principles of never-ending learning, unsupported operations or unanswered questions create triggers for system knowledge demand that can be gathered from external sources and added to the ontology, augmenting the system’s ability to respond to more questions over time. Full article
(This article belongs to the Special Issue Natural Language Processing (NLP) and Applications—2nd Edition)
►▼ Show Figures

Figure 1

20 pages, 8097 KiB  
Article
VL-Few: Vision Language Alignment for Multimodal Few-Shot Meta Learning
by Han Ma, Baoyu Fan, Benjamin K. Ng and Chan-Tong Lam
Appl. Sci. 2024, 14(3), 1169; https://doi.org/10.3390/app14031169 - 30 Jan 2024
Cited by 1 | Viewed by 2444
Abstract
Complex tasks in the real world involve different modal models, such as visual question answering (VQA). However, traditional multimodal learning requires a large amount of aligned data, such as image text pairs, and constructing a large amount of training data is a challenge [...] Read more.
Complex tasks in the real world involve different modal models, such as visual question answering (VQA). However, traditional multimodal learning requires a large amount of aligned data, such as image text pairs, and constructing a large amount of training data is a challenge for multimodal learning. Therefore, we propose VL-Few, which is a simple and effective method to solve the multimodal few-shot problem. VL-Few (1) proposes the modal alignment, which aligns visual features into language space through a lightweight model network and improves the multimodal understanding ability of the model; (2) adopts few-shot meta learning in the multimodal problem, which constructs a few-shot meta task pool to improve the generalization ability of the model; (3) proposes semantic alignment to enhance the semantic understanding ability of the model for the task, context, and demonstration; (4) proposes task alignment that constructs training data into the target task form and improves the task understanding ability of the model; (5) proposes generation alignment, which adopts the token-level training and multitask fusion loss to improve the generation ability of the model. Our experimental results show the effectiveness of VL-Few for multimodal few-shot problems. Full article
(This article belongs to the Special Issue Natural Language Processing (NLP) and Applications—2nd Edition)
►▼ Show Figures

Figure 1

20 pages, 2767 KiB  
Article
A Robust Chinese Named Entity Recognition Method Based on Integrating Dual-Layer Features and CSBERT
by Yingjie Xu, Xiaobo Tan, Xin Tong and Wenbo Zhang
Appl. Sci. 2024, 14(3), 1060; https://doi.org/10.3390/app14031060 - 26 Jan 2024
Cited by 5 | Viewed by 1763
Abstract
In the rapidly evolving field of cybersecurity, the integration of multi-source, heterogeneous, and fragmented data into a coherent knowledge graph has garnered considerable attention. Such a graph elucidates semantic interconnections, thereby facilitating sophisticated analytical decision support. Central to the construction of a cybersecurity [...] Read more.
In the rapidly evolving field of cybersecurity, the integration of multi-source, heterogeneous, and fragmented data into a coherent knowledge graph has garnered considerable attention. Such a graph elucidates semantic interconnections, thereby facilitating sophisticated analytical decision support. Central to the construction of a cybersecurity knowledge graph is Named Entity Recognition (NER), a critical technology that converts unstructured text into structured data. The efficacy of NER is pivotal, as it directly influences the integrity of the knowledge graph. The task of NER in cybersecurity, particularly within the Chinese linguistic context, presents distinct challenges. Chinese text lacks explicit space delimiters and features complex contextual dependencies, exacerbating the difficulty in discerning and categorizing named entities. These linguistic characteristics contribute to errors in word segmentation and semantic ambiguities, impeding NER accuracy. This paper introduces a novel NER methodology tailored for the Chinese cybersecurity corpus, termed CSBERT-IDCNN-BiLSTM-CRF. This approach harnesses Iterative Dilated Convolutional Neural Networks (IDCNN) for extracting local features, and Bi-directional Long Short-Term Memory networks (BiLSTM) for contextual understanding. It incorporates CSBERT, a pre-trained model adept at processing few-shot data, to derive input feature representations. The process culminates with Conditional Random Fields (CRF) for precise sequence labeling. To compensate for the scarcity of publicly accessible Chinese cybersecurity datasets, this paper synthesizes a bespoke dataset, authenticated by data from the China National Vulnerability Database, processed via the YEDDA annotation tool. Empirical analysis affirms that the proposed CSBERT-IDCNN-BiLSTM-CRF model surpasses existing Chinese NER frameworks, with an F1-score of 87.30% and a precision rate of 85.89%. This marks a significant advancement in the accurate identification of cybersecurity entities in Chinese text, reflecting the model’s robust capability to address the unique challenges presented by the language’s structural intricacies. Full article
(This article belongs to the Special Issue Natural Language Processing (NLP) and Applications—2nd Edition)
►▼ Show Figures

Figure 1

Show export options expand_more Show export options expand_less
Select all
Export citation of selected articles as:
 
Displaying articles 1-22
Back to TopTop