Neural Network Technologies in Natural Language Processing and Data Mining

1. Introduction

The rise of large language models (LLMs) has marked a significant leap in natural language processing (NLP) capabilities, enabling machines to understand, generate, and interact with human language at a sophisticated level [1]. This progress has been largely driven by the Transformer architecture [2], which effectively captures long-range dependencies and rich contextual information and now underpins most state-of-the-art and widely adopted LLMs. Transformer-based LLMs can be grouped into three main categories: encoder-only (e.g., BERT [3]), decoder-only (e.g., GPT-4 [4]), and encoder-decoder models (e.g., T5 [5], MASS [6], BART [7]). Multimodal large language models (MLLMs), exemplified by GPT-4 [4], have recently emerged as a major research focus, leveraging powerful LLMs as a central reasoning core to process and perform tasks across multiple modalities [8].

LLMs face limitations such as a tendency to hallucinate, i.e., producing factually incorrect or nonsensical outputs, and a dependence on static training data that can quickly become outdated. Beyond these technical constraints, LLMs also raise a broad range of ethical concerns, encompassing issues shared with other AI systems, such as privacy, data security, and bias, as well as challenges that are amplified or unique in the LLM context, including transparency, accountability, misinformation, and intellectual property risks [9].

To address both the limitations and the rapid evolution of large language models, recent research has explored complementary strategies that enhance accuracy, adaptability, and scope. Retrieval-Augmented Generation (RAG) [10] has emerged as an effective approach to mitigating the aforementioned hallucination phenomenon by grounding LLM outputs in externally retrieved, factual information. When coupled with advanced data preprocessing and indexing strategies, RAG architectures can substantially enhance the factual accuracy and contextual relevance of generated responses compared with conventional LLM-only approaches. Fine-tuning, by contrast, tailors the representations acquired during pre-training to downstream tasks, improving model performance while requiring significantly less task-specific data [11].

2. An Overview of the Published Articles

The contributions in this Special Issue demonstrate how contemporary Transformer-based LLMs can be effectively adapted and extended to address both methodological and application-driven challenges in NLP. While several studies focus on language- and domain-specific challenges, showing how tailored datasets and architectures are essential for robust performance in less-resourced or linguistically complex settings, others extend NLP toward insight generation, including retrieval-augmented generation for reliable knowledge creation, temporal modeling for mental health forecasting, and scalable NLP-based evaluation of LLM-generated educational personalization, or present a methodological comparison contrasting fine-tuned transformers with prompt-based LLM approaches.

The role of fine-tuning in transferring pre-trained language representations to specialized tasks is highlighted in several papers. In particular, Transformer-based approaches to authorship attribution leverage fine-tuning on labeled datasets to capture distinctive writing styles, enabling automatic feature learning and reducing dependence on manual feature engineering. This adaptation is further strengthened through hybrid strategies that combine handcrafted linguistic features with contextualized embeddings from BERT, leading to improved predictive performance [12].

Complementing the focus on fine-tuning, some other contributions address the challenge of factual reliability in LLM outputs by investigating RAG architectures that ground model responses in external knowledge sources. In particular, ref. [13] provides a comprehensive review of both naïve and advanced RAG models, highlighting key techniques such as optimized indexing, query refinement, metadata utilization, and the integration of autonomous AI agents within agentic RAG systems. It further examines effective data preprocessing methods, semantic-aware chunking, and advanced retrieval strategies to mitigate issues such as irrelevant retrieval and semantic fragmentation.

Together, these works reflect a broader shift in NLP research—from developing increasingly large models toward designing robust, accurate, and adaptable systems that combine pre-trained linguistic knowledge with task-specific learning and external information sources.

3. Conclusions

Although LLMs have driven major advances in NLP, they have also introduced significant challenges. The contributions in this Special Issue illustrate the rapid maturation and diversification of NLP research, unified by the increasing dominance of transformer-based LLMs across languages, domains, and application contexts. The findings collectively suggest that further performance gains can be achieved through targeted fine-tuning on task-specific datasets, integration with complementary machine learning techniques, and the development of hybrid modeling approaches. Given the largely black-box nature of LLMs, the application of interpretability methods remains essential for improving transparency, trust, and responsible deployment. In parallel, RAG architectures demonstrate clear potential for enhancing the factual consistency and contextual relevance of model outputs compared with standard LLM approaches. In particular, emerging agentic RAG systems [14], which employ autonomous agents for iterative reasoning and context-aware retrieval, represent a promising direction for more adaptive, reliable, and robust information synthesis.