Sign in to use this feature.

Years

Between: -

Subjects

remove_circle_outline
remove_circle_outline
remove_circle_outline
remove_circle_outline
remove_circle_outline
remove_circle_outline
remove_circle_outline
remove_circle_outline
remove_circle_outline

Journals

remove_circle_outline
remove_circle_outline
remove_circle_outline
remove_circle_outline
remove_circle_outline

Article Types

Countries / Regions

Search Results (148)

Search Parameters:
Keywords = generative pre-trained transformers (GPTs)

Order results
Result details
Results per page
Select all
Export citation of selected articles as:
25 pages, 1751 KiB  
Review
Large Language Models for Adverse Drug Events: A Clinical Perspective
by Md Muntasir Zitu, Dwight Owen, Ashish Manne, Ping Wei and Lang Li
J. Clin. Med. 2025, 14(15), 5490; https://doi.org/10.3390/jcm14155490 - 4 Aug 2025
Abstract
Adverse drug events (ADEs) significantly impact patient safety and health outcomes. Manual ADE detection from clinical narratives is time-consuming, labor-intensive, and costly. Recent advancements in large language models (LLMs), including transformer-based architectures such as Bidirectional Encoder Representations from Transformers (BERT) and Generative Pretrained [...] Read more.
Adverse drug events (ADEs) significantly impact patient safety and health outcomes. Manual ADE detection from clinical narratives is time-consuming, labor-intensive, and costly. Recent advancements in large language models (LLMs), including transformer-based architectures such as Bidirectional Encoder Representations from Transformers (BERT) and Generative Pretrained Transformer (GPT) series, offer promising methods for automating ADE extraction from clinical data. These models have been applied to various aspects of pharmacovigilance and clinical decision support, demonstrating potential in extracting ADE-related information from real-world clinical data. Additionally, chatbot-assisted systems have been explored as tools in clinical management, aiding in medication adherence, patient engagement, and symptom monitoring. This narrative review synthesizes the current state of LLMs in ADE detection from a clinical perspective, organizing studies into categories such as human-facing decision support tools, immune-related ADE detection, cancer-related and non-cancer-related ADE surveillance, and personalized decision support systems. In total, 39 articles were included in this review. Across domains, LLM-driven methods have demonstrated promising performances, often outperforming traditional approaches. However, critical limitations persist, such as domain-specific variability in model performance, interpretability challenges, data quality and privacy concerns, and infrastructure requirements. By addressing these challenges, LLM-based ADE detection could enhance pharmacovigilance practices, improve patient safety outcomes, and optimize clinical workflows. Full article
(This article belongs to the Section Pharmacology)
Show Figures

Figure 1

10 pages, 426 KiB  
Proceeding Paper
Guiding or Misleading: Challenges of Artificial Intelligence-Generated Content in Heuristic Teaching: ChatGPT
by Ping-Kuo A. Chen
Eng. Proc. 2025, 103(1), 1; https://doi.org/10.3390/engproc2025103001 - 4 Aug 2025
Viewed by 3
Abstract
Artificial intelligence (AI)-generated content (AIGC) is an innovative technology that utilizes machine learning, AI models, reward modeling, and natural language processing (NLP) to create diverse digital content such as videos, images, and text. It has the potential to support various human activities with [...] Read more.
Artificial intelligence (AI)-generated content (AIGC) is an innovative technology that utilizes machine learning, AI models, reward modeling, and natural language processing (NLP) to create diverse digital content such as videos, images, and text. It has the potential to support various human activities with significant implications in teaching and learning, facilitating heuristic teaching for educators. By using AIGC, teachers can create extensive knowledge content and effectively design instructional strategies to guide students, aligning with heuristic teaching. However, incorporating AIGC into heuristic teaching has controversies and concerns, which potentially mislead outcomes. Nevertheless, leveraging AIGC greatly benefits teachers in enhancing heuristic teaching. When integrating AIGC to support heuristic teaching, challenges and risks must be acknowledged and addressed. These challenges include the need for users to possess sufficient knowledge reserves to identify incorrect information and content generated by AIGC, the importance of avoiding excessive reliance on AIGC, ensuring users maintain control over their actions rather than being driven by AIGC, and the necessity of scrutinizing and verifying the accuracy of information and knowledge generated by AIGC to preserve its effectiveness. Full article
Show Figures

Figure 1

8 pages, 192 KiB  
Brief Report
Accuracy and Safety of ChatGPT-3.5 in Assessing Over-the-Counter Medication Use During Pregnancy: A Descriptive Comparative Study
by Bernadette Cornelison, David R. Axon, Bryan Abbott, Carter Bishop, Cindy Jebara, Anjali Kumar and Kristen A. Root
Pharmacy 2025, 13(4), 104; https://doi.org/10.3390/pharmacy13040104 - 30 Jul 2025
Viewed by 495
Abstract
As artificial intelligence (AI) becomes increasingly utilized to perform tasks requiring human intelligence, patients who are pregnant may turn to AI for advice on over-the-counter (OTC) medications. However, medications used in pregnancy may pose profound safety concerns limited by data availability. This study [...] Read more.
As artificial intelligence (AI) becomes increasingly utilized to perform tasks requiring human intelligence, patients who are pregnant may turn to AI for advice on over-the-counter (OTC) medications. However, medications used in pregnancy may pose profound safety concerns limited by data availability. This study focuses on a chatbot’s ability to accurately provide information regarding OTC medications as it relates to patients that are pregnant. A prospective, descriptive design was used to compare the responses generated by the Chat Generative Pre-Trained Transformer 3.5 (ChatGPT-3.5) to the information provided by UpToDate®. Eighty-seven of the top pharmacist-recommended OTC drugs in the United States (U.S.) as identified by Pharmacy Times were assessed for safe use in pregnancy using ChatGPT-3.5. A piloted, standard prompt was input into ChatGPT-3.5, and the responses were recorded. Two groups independently rated the responses compared to UpToDate on their correctness, completeness, and safety using a 5-point Likert scale. After independent evaluations, the groups discussed the findings to reach a consensus, with a third independent investigator giving final ratings. For correctness, the median score was 5 (interquartile range [IQR]: 5–5). For completeness, the median score was 4 (IQR: 4–5). For safety, the median score was 5 (IQR: 5–5). Despite high overall scores, the safety errors in 9% of the evaluations (n = 8), including omissions that pose a risk of serious complications, currently renders the chatbot an unsafe standalone resource for this purpose. Full article
(This article belongs to the Special Issue AI Use in Pharmacy and Pharmacy Education)
17 pages, 609 KiB  
Article
GPT-Based Text-to-SQL for Spatial Databases
by Hui Wang, Li Guo, Yubin Liang, Le Liu and Jiajin Huang
ISPRS Int. J. Geo-Inf. 2025, 14(8), 288; https://doi.org/10.3390/ijgi14080288 - 24 Jul 2025
Viewed by 251
Abstract
Text-to-SQL for spatial databases enables the translation of natural language questions into corresponding SQL queries, allowing non-experts to easily access spatial data, which has gained increasing attention from researchers. Previous research has primarily focused on rule-based methods. However, these methods have limitations when [...] Read more.
Text-to-SQL for spatial databases enables the translation of natural language questions into corresponding SQL queries, allowing non-experts to easily access spatial data, which has gained increasing attention from researchers. Previous research has primarily focused on rule-based methods. However, these methods have limitations when dealing with complicated or unknown natural language questions. While advanced machine learning models can be trained, they typically require large labeled training datasets, which are severely lacking for spatial databases. Recently, Generative Pre-Trained Transformer (GPT) models have emerged as a promising paradigm for Text-to-SQL tasks in relational databases, driven by carefully designed prompts. In response to the severe lack of datasets for spatial databases, we have created a publicly available dataset that supports both English and Chinese. Furthermore, we propose a GPT-based method to construct prompts for spatial databases, which incorporates geographic and spatial database knowledge into the prompts and requires only a small number of training samples, such as 1, 3, or 5 examples. Extensive experiments demonstrate that incorporating geographic and spatial database knowledge into prompts improves the accuracy of Text-to-SQL tasks for spatial databases. Our proposed method can help non-experts access spatial databases more easily and conveniently. Full article
Show Figures

Figure 1

22 pages, 1805 KiB  
Article
A Hybrid Semantic and Multi-Attention Mechanism Approach for Detecting Vulnerabilities in Smart Contract Code
by Zhenxiang He, Yanling Liu and Xiaohui Sun
Symmetry 2025, 17(7), 1161; https://doi.org/10.3390/sym17071161 - 21 Jul 2025
Viewed by 355
Abstract
Driven by blockchain technology, numerous industries are increasingly adopting smart contracts to enhance efficiency, reduce costs, and improve transparency. As a result, ensuring the security of smart contracts has become critical. Traditional detection methods often suffer from low efficiency, are prone to missing [...] Read more.
Driven by blockchain technology, numerous industries are increasingly adopting smart contracts to enhance efficiency, reduce costs, and improve transparency. As a result, ensuring the security of smart contracts has become critical. Traditional detection methods often suffer from low efficiency, are prone to missing complex vulnerabilities, and have limited accuracy. Although deep learning approaches address some of these challenges, issues with both accuracy and efficiency remain in current solutions. To overcome these limitations, this paper proposes a symmetry-inspired solution that harmonizes bidirectional and generative semantic patterns. First, we generate distinct feature extraction segments for different vulnerabilities. We then use the Bidirectional Encoder Representations from Transformers (BERT) module to extract original semantic features from these segments and the Generative Pre-trained Transformer (GPT) module to extract generative semantic features. Finally, the two sets of semantic features are fused using a multi-attention mechanism and input into a classifier for result prediction. Our method was tested on three datasets, achieving F1 scores of 93.33%, 93.65%, and 92.31%, respectively. The results demonstrate that our approach outperforms most existing methods in smart contract detection. Full article
(This article belongs to the Section Computer)
Show Figures

Figure 1

26 pages, 15354 KiB  
Article
Transforming Physics Teacher Training Through ChatGPT: A Study on Usability and Impact
by Marcos Guerrero-Zambrano, Leonor Sanchez-Alvarado, Bryan Valarezo-Chamba and Erick Lamilla-Rubio
Educ. Sci. 2025, 15(7), 887; https://doi.org/10.3390/educsci15070887 - 11 Jul 2025
Viewed by 697
Abstract
Teacher training in Physics often faces challenges related to engaging students and conveying abstract concepts effectively. Generative AI tools, such as ChatGPT, present transformative opportunities for designing innovative and tailored educational activities. This study investigates the impact of ChatGPT on pre-service Physics teacher [...] Read more.
Teacher training in Physics often faces challenges related to engaging students and conveying abstract concepts effectively. Generative AI tools, such as ChatGPT, present transformative opportunities for designing innovative and tailored educational activities. This study investigates the impact of ChatGPT on pre-service Physics teacher training, focusing on its usability, effectiveness, and influence on participant satisfaction. Utilizing a quantitative research approach, two Likert-scale surveys were administered to 24 prospective Physics teachers in Ecuador, both before and after an intervention workshop. The workshop introduced participants to ChatGPT’s features and its applications in designing playful, Physics-focused learning activities. Results indicated a significant increase in familiarity with AI tools, enhanced activity design quality, and high satisfaction rates. Notably, 79% of participants highlighted ChatGPT’s utility in adapting activities to diverse learning levels, and 83% acknowledged its efficiency in reducing preparation time. These findings underscore ChatGPT’s potential to revolutionize Physics education by facilitating the creation of personalized and engaging learning resources. Future research should explore larger sample sizes and longitudinal impacts to fully realize the implications of AI-driven tools in educational contexts. Full article
(This article belongs to the Topic Artificial Intelligence in Early Childhood Education)
Show Figures

Figure 1

20 pages, 1535 KiB  
Article
Multi-Agentic LLMs for Personalizing STEM Texts
by Michael Vaccaro, Mikayla Friday and Arash Zaghi
Appl. Sci. 2025, 15(13), 7579; https://doi.org/10.3390/app15137579 - 6 Jul 2025
Viewed by 518
Abstract
Multi-agent large language models promise flexible, modular architectures for delivering personalized educational content. Drawing on a pilot randomized controlled trial with middle school students (n = 23), we introduce a two-agent GPT-4 framework in which a Profiler agent infers learner-specific preferences and [...] Read more.
Multi-agent large language models promise flexible, modular architectures for delivering personalized educational content. Drawing on a pilot randomized controlled trial with middle school students (n = 23), we introduce a two-agent GPT-4 framework in which a Profiler agent infers learner-specific preferences and a Rewrite agent dynamically adapts science passages via an explicit message-passing protocol. We implement structured system and user prompts as inter-agent communication schemas to enable real-time content adaptation. The results of an ordinal logistic regression analysis hinted that students may be more likely to prefer texts aligned with their profile, demonstrating the feasibility of multi-agent system-driven personalization and highlighting the need for additional work to build upon this pilot study. Beyond empirical validation, we present a modular multi-agent architecture detailing agent roles, communication interfaces, and scalability considerations. We discuss design best practices, ethical safeguards, and pathways for extending this framework to collaborative agent networks—such as feedback-analysis agents—in K-12 settings. These results advance both our theoretical and applied understanding of multi-agent LLM systems for personalized learning. Full article
Show Figures

Figure 1

13 pages, 972 KiB  
Article
Assessing ChatGPT-v4 for Guideline-Concordant Inflammatory Bowel Disease: Accuracy, Completeness, and Temporal Drift
by Oguz Ozturk, Mucahit Ergul, Yavuz Cagir, Ali Atay, Kadir Can Acun, Orhan Coskun, Ilyas Tenlik, Muhammed Bahaddin Durak and Ilhami Yuksel
J. Clin. Med. 2025, 14(13), 4599; https://doi.org/10.3390/jcm14134599 - 29 Jun 2025
Viewed by 561
Abstract
Background/Objectives: Chat Generative Pretrained Transformer (ChatGPT) is a useful resource for individuals working in the healthcare field. This paper will include descriptions of several ways in which ChatGPT-4 can achieve greater accuracy in its diagnosis and treatment plans for ulcerative colitis (UC) and [...] Read more.
Background/Objectives: Chat Generative Pretrained Transformer (ChatGPT) is a useful resource for individuals working in the healthcare field. This paper will include descriptions of several ways in which ChatGPT-4 can achieve greater accuracy in its diagnosis and treatment plans for ulcerative colitis (UC) and Crohn’s disease (CD) by following the guidelines set out by the European Crohn’s and Colitis Organization (ECCO). Methods: The survey, which comprised 102 questions, was developed to assess the precision and consistency of respondents’ responses regarding the UC and CD. The questionnaire incorporated true/false and multiple-choice questions, with the objective of simulating real-life scenarios and adhering to the ECCO guidelines. We employed Likert scales to assess the responses. The inquiries were put to ChatGPT-4 on the initial day, the 15th day, and the 180th day. Results: The 51 true or false items demonstrated stability over a six-month period, with an initial accuracy of 92.8% at baseline, 92.8% on the 15th day, and peaked to 98.0% on the 180th day. This finding suggests a negligible effect size. The accuracy of the multiple-choice questions was initially 90.2% on Day 1, reached its highest point at 92.2% on Day 15, and then decreased to 84.3% on Day 180. However, the reliability of the data was found to be suboptimal, and the impact was deemed negligible. A modest, transient increase in performance was observed at 15 days, which subsequently diminished by 180 days, resulting in negligible effect sizes. Conclusions: ChatGPT-4 demonstrates potential as a clinical decision support system for UC and CD, but its assessment is marked by temporal variability and the inconsistent execution of various tasks. Essential initiatives that should be carried out before involving artificial intelligence (AI) technology in IBD trials are routine revalidation, multi-rater comparisons, prompt standardization, and the cultivation of a comprehensive understanding of the model’s limitations. Full article
(This article belongs to the Section Gastroenterology & Hepatopancreatobiliary Medicine)
Show Figures

Figure 1

20 pages, 2223 KiB  
Article
ChatGPT-Based Model for Controlling Active Assistive Devices Using Non-Invasive EEG Signals
by Tais da Silva Mota, Saket Sarkar, Rakshith Poojary and Redwan Alqasemi
Electronics 2025, 14(12), 2481; https://doi.org/10.3390/electronics14122481 - 18 Jun 2025
Viewed by 603
Abstract
With an anticipated 3.6 million Americans who will be living with limb loss by 2050, the demand for active assistive devices is rapidly increasing. This study investigates the feasibility of leveraging a ChatGPT-based (Version 4o) model to predict motion based on input electroencephalogram [...] Read more.
With an anticipated 3.6 million Americans who will be living with limb loss by 2050, the demand for active assistive devices is rapidly increasing. This study investigates the feasibility of leveraging a ChatGPT-based (Version 4o) model to predict motion based on input electroencephalogram (EEG) signals, enabling the non-invasive control of active assistive devices. To achieve this goal, three objectives were set. First, the model’s capability to derive accurate mathematical relationships from numerical datasets was validated to establish a foundational level of computational accuracy. Next, synchronized arm motion videos and EEG signals were introduced, which allowed the model to filter, normalize, and classify EEG data in relation to distinct text-based arm motions. Finally, the integration of marker-based motion capture data provided motion information, which is essential for inverse kinematics applications in robotic control. The combined findings highlight the potential of ChatGPT-generated machine learning systems to effectively correlate multimodal data streams and serve as a robust foundation for the intuitive, non-invasive control of assistive technologies using EEG signals. Future work will focus on applying the model to real-time control applications while expanding the dataset’s diversity to enhance the accuracy and performance of the model, with the ultimate aim of improving the independence and quality of life of individuals who rely on active assistive devices. Full article
(This article belongs to the Special Issue Advances in Intelligent Control Systems)
Show Figures

Figure 1

8 pages, 1398 KiB  
Proceeding Paper
Analysis of Three-Stage Visit Behavior of Tourists Using ChatGPT: Agenda for Future Study
by Pahrudin Pahrudin, Li-Wei Liu, Anfitri Kristin Sihombing and Idrus Jamalulel
Eng. Proc. 2025, 98(1), 15; https://doi.org/10.3390/engproc2025098015 - 18 Jun 2025
Viewed by 393
Abstract
Chat generative pre-trained transformer (ChatGPT) is an artificial intelligence (AI) engine. Research on tourism using ChatGPT has gained traction from scholars all over the world. However, limited studies on ChatGPT and the tourism industry have been conducted using an analysis of three-stage visit [...] Read more.
Chat generative pre-trained transformer (ChatGPT) is an artificial intelligence (AI) engine. Research on tourism using ChatGPT has gained traction from scholars all over the world. However, limited studies on ChatGPT and the tourism industry have been conducted using an analysis of three-stage visit behavior. We analyzed the current trend in tourism research using ChatGPT with a bibliometric analysis based on the Scopus database. A total of 110 documents were used in this study for document review, and R studio Version 2022.12.0+353 was used to analyze the documents. The results present indicators for a systematic review of the documents, such as the number of publications and co-word analysis. A theoretical system was developed in this study to explore travelers’ behavior using ChatGPT in the pre-, during, and post-travel periods. The study results contribute to the development of the tourism industry to understand tourist behavior using ChatGPT. Full article
Show Figures

Figure 1

13 pages, 895 KiB  
Article
Perspectives on Generative Sound Design: A Generative Soundscapes Showcase
by Grzegorz Samson
Arts 2025, 14(3), 67; https://doi.org/10.3390/arts14030067 - 12 Jun 2025
Viewed by 696
Abstract
Recent advancements in generative neural networks, particularly transformer-based models, have introduced novel possibilities for sound design. This study explores the use of generative pre-trained transformers (GPT) to create complex, multilayered soundscapes from textual and visual prompts. A custom pipeline is proposed, featuring modules [...] Read more.
Recent advancements in generative neural networks, particularly transformer-based models, have introduced novel possibilities for sound design. This study explores the use of generative pre-trained transformers (GPT) to create complex, multilayered soundscapes from textual and visual prompts. A custom pipeline is proposed, featuring modules for converting the source input into structured sound descriptions and subsequently generating cohesive auditory outputs. As a complementary solution, a granular synthesizer prototype was developed to enhance the usability of generative audio samples by enabling their recombination into seamless and non-repetitive soundscapes. The integration of GPT models with granular synthesis demonstrates significant potential for innovative audio production, paving the way for advancements in professional sound-design workflows and immersive audio applications. Full article
(This article belongs to the Special Issue Sound, Space, and Creativity in Performing Arts)
Show Figures

Graphical abstract

19 pages, 1662 KiB  
Article
Scoring German Alternate Uses Items Applying Large Language Models
by Janika Saretzki, Thomas Knopf, Boris Forthmann, Benjamin Goecke, Ann-Kathrin Jaggy, Mathias Benedek and Selina Weiss
J. Intell. 2025, 13(6), 64; https://doi.org/10.3390/jintelligence13060064 - 29 May 2025
Viewed by 716
Abstract
The alternate uses task (AUT) is the most popular measure when it comes to the assessment of creative potential. Since their implementation, AUT responses have been rated by humans, which is a laborious task and requires considerable resources. Large language models (LLMs) have [...] Read more.
The alternate uses task (AUT) is the most popular measure when it comes to the assessment of creative potential. Since their implementation, AUT responses have been rated by humans, which is a laborious task and requires considerable resources. Large language models (LLMs) have shown promising performance in automatically scoring AUT responses in English as well as in other languages, but it is not clear which method works best for German data. Therefore, we investigated the performance of different LLMs for the automated scoring of German AUT responses. We compiled German data across five research groups including ~50,000 responses for 15 different alternate uses objects from eight lab and online survey studies (including ~2300 participants) to examine generalizability across datasets and assessment conditions. Following a pre-registered analysis plan, we compared the performance of two fine-tuned, multilingual LLM-based approaches [Cross-Lingual Alternate Uses Scoring (CLAUS) and the Open Creativity Scoring with Artificial Intelligence (OCSAI)] with the Generative Pre-trained Transformer (GPT-4) in scoring (a) the original German AUT responses and (b) the responses translated to English. We found that the LLM-based scorings were substantially correlated with human ratings, with higher relationships for OCSAI followed by GPT-4 and CLAUS. Response translation, however, had no consistent positive effect. We discuss the generalizability of the results across different items and studies and derive recommendations and future directions. Full article
(This article belongs to the Special Issue Generative AI: Reflections on Intelligence and Creativity)
Show Figures

Figure 1

20 pages, 2354 KiB  
Article
ChatGPT vs. Human Journalists: Analyzing News Summaries Through BERTScore and Moderation Standards
by Hui-Sang Kim, Ji-Won Kang and Sun-Yong Choi
Electronics 2025, 14(11), 2115; https://doi.org/10.3390/electronics14112115 - 22 May 2025
Viewed by 1152
Abstract
Recent advances in natural language processing (NLP) have enabled the development of powerful language models such as Generative Pre-trained Transformers (GPTs). This study evaluates the performance of ChatGPT in generating news summaries by comparing them with summaries written by professional journalists at The [...] Read more.
Recent advances in natural language processing (NLP) have enabled the development of powerful language models such as Generative Pre-trained Transformers (GPTs). This study evaluates the performance of ChatGPT in generating news summaries by comparing them with summaries written by professional journalists at The New York Times. Using BERTScore as the primary metric, we assessed the semantic similarity between ChatGPT-generated and human-authored summaries. We further employed OpenAI’s moderation API to examine the extent to which each set of summaries contained potentially biased, inflammatory, or violent language. The results indicate that ChatGPT-generated summaries exhibit a high degree of contextual alignment with human-written summaries, achieving a BERTScore F1-score above 0.87. Moreover, ChatGPT outputs consistently omit language flagged as problematic by moderation algorithms, producing summaries that are less likely to include harmful or polarizing content—a feature we define as moderation-friendly summarization. These findings suggest that ChatGPT can serve as a valuable tool for automated news summarization, offering content that is both contextually accurate and aligned with content moderation standards, thereby supporting more objective and responsible news dissemination. Full article
(This article belongs to the Section Artificial Intelligence)
Show Figures

Figure 1

38 pages, 2033 KiB  
Article
DCAT: A Novel Transformer-Based Approach for Dynamic Context-Aware Image Captioning in the Tamil Language
by Jothi Prakash Venugopal, Arul Antran Vijay Subramanian, Manikandan Murugan, Gopikrishnan Sundaram, Marco Rivera and Patrick Wheeler
Appl. Sci. 2025, 15(9), 4909; https://doi.org/10.3390/app15094909 - 28 Apr 2025
Viewed by 601
Abstract
The task of image captioning in low-resource languages like Tamil is fraught with challenges due to limited linguistic resources and complex semantic structures. This paper addresses the problem of generating contextually and linguistically coherent captions in Tamil. We introduce the Dynamic Context-Aware Transformer [...] Read more.
The task of image captioning in low-resource languages like Tamil is fraught with challenges due to limited linguistic resources and complex semantic structures. This paper addresses the problem of generating contextually and linguistically coherent captions in Tamil. We introduce the Dynamic Context-Aware Transformer (DCAT), a novel approach that synergizes the Vision Transformer (ViT) with the Generative Pre-trained Transformer (GPT-3), reinforced by a unique Context Embedding Layer. The DCAT model, tailored for Tamil, innovatively employs dynamic attention mechanisms during its Initialization, Training, and Inference phases to focus on pertinent visual and textual elements. Our method distinctively leverages the nuances of Tamil syntax and semantics, a novelty in the realm of low-resource language image captioning. Comparative evaluations against established models on datasets like Flickr8k, Flickr30k, and MSCOCO reveal DCAT’s superiority, with a notable 12% increase in BLEU score (0.7425) and a 15% enhancement in METEOR score (0.4391) over leading models. Despite its computational demands, DCAT sets a new benchmark for image captioning in Tamil, demonstrating potential applicability to other similar languages. Full article
Show Figures

Figure 1

14 pages, 1375 KiB  
Article
Instance-Level Weighted Contrast Learning for Text Classification
by Xinhui Liu, Jifa Chen and Qiubo Huang
Appl. Sci. 2025, 15(8), 4236; https://doi.org/10.3390/app15084236 - 11 Apr 2025
Viewed by 500
Abstract
With the explosion of information, the amount of text data has increased significantly, making text categorization a central area of research in natural language processing (NLP). Traditional machine learning methods are effective, but deep learning models excel in processing semantic information. Models such [...] Read more.
With the explosion of information, the amount of text data has increased significantly, making text categorization a central area of research in natural language processing (NLP). Traditional machine learning methods are effective, but deep learning models excel in processing semantic information. Models such as CNN, RNN, LSTM, and GRU have emerged as powerful tools for text classification. Pre-trained models such as BERT and GPT have further advanced text categorization techniques. Contrastive learning has become a key research focus aimed at improving classification performance by learning the similarities and differences between samples using models. However, existing contrastive learning methods have notable shortcomings, primarily concerning insufficient data utilization. This study focuses on data enhancement techniques to expand the text data through symbol insertion, affirmative auxiliary verbs, double negation, and punctuation repetition, aiming to improve the generalization and robustness of the pre-trained model. Two data enhancement strategies, affirmative enhancement and negative transformation, are introduced to deepen the data’s meaning and increase the volume of training data. To address the introduction of false data, an instance weighting method is employed to penalize false negative samples, while complementary models generate sample weights to mitigate the impact of sampling bias. Finally, the effectiveness of the proposed method is demonstrated through several experiments. Full article
Show Figures

Figure 1

Back to TopTop