Search Results (21)

Background: The widespread adoption of digital systems in healthcare has produced large volumes of unstructured text data, including outpatient messages sent through electronic medical record (EMR) portals. Efficient classification of these messages is essential for improving workflow automation and enabling timely clinical responses. Methods: This study investigates the use of large language models (LLMs) for classifying real-world outpatient messages collected from a healthcare system in central Illinois. We compare general-purpose (GPT-4o) and domain-specific (BioBERT and ClinicalBERT) models, evaluating both fine-tuned and few-shot configurations against a TF-IDF + Logistic Regression baseline. Experiments were performed under a HIPAA-compliant environment using de-identified and physician-labeled data. Results and Conclusions: Fine-tuned GPT-4o achieved 97.5% accuracy in urgency detection and 97.8% in full message classification, outperforming BioBERT and ClinicalBERT. These results demonstrate the feasibility and validity of applying modern LLMs to outpatient communication triage while ensuring both interpretability and privacy compliance. Full article

Traditional methods for patient cohort identification from Electronic Health Records (EHRs) are often slow, labor-intensive, and fail to capture the rich semantic nuance embedded in unstructured clinical narratives. This paper introduces a scalable, end-to-end framework that creates a synergy between deep medical embeddings and Approximate Nearest Neighbor Search (ANNs) to overcome these limitations. We detail a complete pipeline that begins with preprocessing multi-modal EHR data and creating holistic patient representations using a domain-specific language model combined with an intelligent gated fusion mechanism. These high-dimensional embeddings are then indexed using an ANN method to enable near real-time retrieval. A comprehensive experimental evaluation was conducted on the MIMIC-III and MIMIC-IV datasets, comparing the retrieval performance of ClinicalBERT against BioBERT across several ANN algorithms. The results demonstrate that the combination of ClinicalBERT and HNSW consistently achieves the highest retrieval accuracy, with F1-Scores exceeding 0.78, and query latencies under 10 ms. This framework enables a paradigm shift towards high-speed, semantic patient similarity search, with significant implications for accelerating clinical trial recruitment, augmenting clinical decision support, and paving the way for a new era in data-driven precision medicine. Full article

Background: Malnutrition is a serious health concern among older adults in residential aged care (RAC), and timely identification is critical for effective intervention. Recent advancements in transformer-based large language models (LLMs), such as RoBERTa, provide context-aware embeddings that improve predictive performance in clinical tasks. Fine-tuning these models on domain-specific corpora, like nursing progress notes, can further enhance their applicability in healthcare. Methodology: We developed a RAC domain-specific LLM by training RoBERTa on 500,000 nursing progress notes from RAC electronic health records (EHRs). The model’s embeddings were used for two downstream tasks: malnutrition note identification and malnutrition prediction. Long sequences were truncated and processed in segments of up to 1536 tokens to fit RoBERTa’s 512-token input limit. Performance was compared against Bag of Words, GloVe, baseline RoBERTa, BlueBERT, ClinicalBERT, BioClinicalBERT, and PubMed models. Results: Using 5-fold cross-validation, the RAC domain-specific LLM outperformed other models. For malnutrition note identification, it achieved an F1-score of 0.966, and for malnutrition prediction, it achieved an F1-score of 0.687. Conclusions: This approach demonstrates the feasibility of developing specialised LLMs for identifying and predicting malnutrition among older adults in RAC. Future work includes further optimisation of prediction performance and integration with clinical workflows to support early intervention. Full article

Accurate prediction of the impact of genetic variants on human health is of paramount importance to clinical genetics and precision medicine. Recent machine learning (ML) studies have tried to predict variant pathogenicity with different levels of success. However, most missense variants identified on a clinical basis are still classified as variants of uncertain significance (VUS). Our approach allows for the interpretation of a variant for a specific disease and, thus, for the integration of disease-specific domain knowledge. We utilize a comprehensive knowledge graph, with 11 types of interconnected biomedical entities at diverse biomolecular and clinical levels, to classify missense variants from ClinVar. We use BioBERT to generate embeddings of biomedical features for each node in the graph, as well as DNA language models to embed variant features directly from genomic sequence. Next, we train a two-stage architecture consisting of a graph convolutional neural network to encode biological relationships. A neural network is then used as the classifier to predict disease-specific pathogenicity of variants, essentially predicting edges between variant and disease nodes. We compare performance across different versions of our model, obtaining prediction-balanced accuracies as high as 85.6% (sensitivity: 90.5%; NPV: 89.8%) and discuss how our work can inform future studies in this area. Full article

Background: Drug–drug interactions (DDIs) involve pharmacokinetic or pharmacodynamic changes that occur when multiple drugs are co-administered, potentially leading to reduced efficacy or adverse effects. As polypharmacy becomes more prevalent, especially among patients with chronic diseases, scalable and accurate DDI prediction has become increasingly important. Although numerous computational approaches have been proposed to predict DDIs using various modalities such as chemical structure and biological networks, the intrinsic heterogeneity of these data complicates unified modeling; Methods: We address this challenge with a multimodal deep learning framework that integrates three complementary, heterogeneous modalities: (i) chemical structure, (ii) BioBERT-derived semantic embeddings (a domain-specific large language model, LLM), and (iii) pharmacological mechanisms through the CTET proteins. To incorporate indirect biological pathways within the PPI network, we apply a random walk with restart (RWR) algorithm. Results: Across features combinations, fusing structural feature with BioBERT embedding achieved the highest classification accuracy (0.9655), highlighting the value of readily available data and the capacity of domain-specific language models to encode pharmacological semantics from unstructured texts. Conclusions: BioBERT embeddings were particularly informative, capturing subtle pharmacological relationships between drugs and improving prediction of potential DDIs. Beyond predictive performance, the framework is readily applicable to real-world clinical workflows, providing rapid DDI references to support the polypharmacy decision-making. Full article

Clinical notes often contain unstructured text filled with abbreviations, non-standard terminology, and inconsistent phrasing, which pose significant challenges for automated medical information extraction. Named Entity Recognition (NER) plays a crucial role in structuring this data by identifying and categorizing key clinical entities such as symptoms, medications, and diagnoses. However, traditional and even transformer-based NER models often struggle with ambiguity and fail to produce clinically interpretable outputs. In this study, we present a hybrid two-stage framework that enhances medical NER by integrating a fine-tuned BERT model for initial entity extraction with a Dictionary-Infused Retrieval-Augmented Generation (DiRAG) module for terminology normalization. Our approach addresses two critical limitations in current clinical NER systems: lack of contextual clarity and inconsistent standardization of medical terms. The DiRAG module combines semantic retrieval from a UMLS-based vector database with lexical matching and prompt-based generation using a large language model, ensuring precise and explainable normalization of ambiguous entities. The fine-tuned BERT model achieved an F1 score of 0.708 on the MACCROBAT dataset, outperforming several domain-specific baselines, including BioBERT and ClinicalBERT. The integration of the DiRAG module further improved the interpretability and clinical relevance of the extracted entities. Through qualitative case studies, we demonstrate that our framework not only enhances clarity but also mitigates common issues such as abbreviation ambiguity and terminology inconsistency. Full article

Electrocardiogram reports are an important component of cardiovascular diagnostics, routinely generated in hospitals and clinical settings to monitor cardiac activity and guide medical decision-making. ECG reports often consist of structured waveform data accompanied by free-text interpretations written by clinicians. Although the waveform data can be analyzed using signal processing techniques, the unstructured text component contains rich, contextual insights into diagnoses, conditions, and patient-specific observations that are not easily captured by conventional methods. Extracting meaningful patterns from clinical narratives poses significant challenges. In this work, we present an unsupervised framework for exploring and analyzing ECG diagnostic reports using transformer-based language modeling and clustering techniques. We use the domain-specific language model BioBERT to encode text-based ECG reports into dense vector representations that capture the semantics of medical language. These embeddings are subsequently standardized and subjected to a series of clustering algorithms, including KMeans, hierarchical clustering, DBSCAN, and K-Medoids, to uncover latent groupings within the data. Full article

Large language models (LLMs) are increasingly being applied to clinical tasks, but it remains unclear whether medical-specific models consistently outperform smaller, generalpurpose ones. This study investigates that assumption in the context of cardiovascular disease (CVD) risk assessment. We fine-tuned eight LLMs—both general-purpose and medical-specific—using textualized data from the Behavioral Risk Factor Surveillance System (BRFSS) to classify individuals as “High Risk” or “Low Risk”. To provide actionable insights, we integrated a Retrieval-Augmented Generation (RAG) framework for personalized recommendation generation and deployed the system within an interactive chatbot interface. Notably, Gemma2, a compact 2B-parameter general-purpose model, achieved a high recall (0.907) and F1-score (0.770), performing on par with larger or medical-specialized models such as Med42 and BioBERT. These findings challenge the common assumption that larger or specialized models always yield superior results, and highlight the potential of lightweight, efficiently fine-tuned LLMs for clinical decision support—especially in resource-constrained settings. Overall, our results demonstrate that general-purpose models, when fine-tuned appropriately, can offer interpretable, high-performing, and accessible solutions for CVD risk assessment and personalized healthcare delivery. Full article

This study investigates the ability of six pre-trained sentence transformers to organize medical knowledge by performing unsupervised clustering on 70 high-level Medical Subject Headings (MeSH) terms across seven medical specialties. We evaluated models from different pre-training paradigms: general-purpose, domain-adapted, and from-scratch domain-specific. The results reveal a clear performance hierarchy. A top tier of models, including the general-purpose MPNet and the domain-adapted BioBERT and RoBERTa, produced highly coherent, specialty-aligned clusters (Adjusted Rand Index > 0.80). Conversely, models pre-trained from scratch on specialized corpora, such as PubMedBERT and BioClinicalBERT, performed poorly (Adjusted Rand Index < 0.51), with BioClinicalBERT yielding a disorganized clustering. These findings challenge the assumption that domain-specific pre-training guarantees superior performance for all semantic tasks. We conclude that model architecture, alignment between the pre-training objective and the downstream task, and the nature of the training data are more critical determinants of success for creating semantically coherent embedding spaces for medical concepts. Full article

Background: In the medical field, various deep learning (DL) algorithms have been effectively used to extract valuable information from unstructured clinical text data, potentially leading to more effective outcomes. This study utilized clinical text data to classify clinical case reports into tuberculosis (TB) and non-tuberculosis (non-TB) groups using natural language processing (NLP), a pre-processing technique, and DL models. Methods: This study used 1743 open-source respiratory disease clinical text data, labeled via fuzzy matching with ICD-10 codes to create a labeled dataset. Two tokenization methods preprocessed the clinical text data, and three models were evaluated: the existing Text-CNN, the proposed Text-CNN with t-test, and Bio_ClinicalBERT. Performance was assessed using multiple metrics and validated on 228 baseline screening clinical case text data collected from ICMR–NIRT to demonstrate effective TB classification. Results: The proposed model achieved the best results in both the test and validation datasets. On the test dataset, it attained a precision of 88.19%, a recall of 90.71%, an F1-score of 89.44%, and an AUC of 0.91. Similarly, on the validation dataset, it achieved 100% precision, 98.85% recall, 99.42% F1-score, and an AUC of 0.982, demonstrating its effectiveness in TB classification. Conclusions: This study highlights the effectiveness of DL models in classifying TB cases from clinical notes. The proposed model outperformed the other two models. The TF-IDF and t-test showed statistically significant feature selection and enhanced model interpretability and efficiency, demonstrating the potential of NLP and DL in automating TB diagnosis in clinical decision settings. Full article

Despite remarkable advances in neural language models, a substantial gap remains in precisely interpreting the complex semantics of Electronic Medical Records (EMR). We propose Contrastive Representations Pre-Training (CRPT) to address this gap, replacing the conventional Next Sentence Prediction task’s cross-entropy loss with contrastive loss and incorporating whole-word masking to capture multi-token domain-specific terms better. We also introduce a carefully designed negative sampling strategy that balances intra-document and cross-document sentences, enhancing the model’s discriminative power. Implemented atop a BERT-based architecture and evaluated on the Biomedical Language Understanding Evaluation (BLUE) benchmark, our Discharge Summary CRPT model achieves significant performance gains, including a natural language inference precision of 0.825 and a sentence similarity score of 0.775. We further extend our approach through Bio+Discharge Summary CRPT, combining biomedical and clinical corpora to boost downstream performance across tasks. Our framework demonstrates robust interpretive capacity in clinical texts by emphasizing sentence-level semantics and domain-aware masking. These findings underscore CRPT’s potential for advancing semantic accuracy in healthcare applications and open new avenues for integrating larger negative sample sets, domain-specific masking techniques, and multi-task learning paradigms. Full article

This work introduces TrialSieve, a novel framework for biomedical information extraction that enhances clinical meta-analysis and drug repurposing. By extending traditional PICO (Patient, Intervention, Comparison, Outcome) methodologies, TrialSieve incorporates hierarchical, treatment group-based graphs, enabling more comprehensive and quantitative comparisons of clinical outcomes. TrialSieve was used to annotate 1609 PubMed abstracts, 170,557 annotations, and 52,638 final spans, incorporating 20 unique annotation categories that capture a diverse range of biomedical entities relevant to systematic reviews and meta-analyses. The performance (accuracy, precision, recall, F1-score) of four natural-language processing (NLP) models (BioLinkBERT, BioBERT, KRISSBERT, PubMedBERT) and the large language model (LLM), GPT-4o, was evaluated using the human-annotated TrialSieve dataset. BioLinkBERT had the best accuracy (0.875) and recall (0.679) for biomedical entity labeling, whereas PubMedBERT had the best precision (0.614) and F1-score (0.639). Error analysis showed that NLP models trained on noisy, human-annotated data can match or, in most cases, surpass human performance. This finding highlights the feasibility of fully automating biomedical information extraction, even when relying on imperfectly annotated datasets. An annotator user study (n = 39) revealed significant (p < 0.05) gains in efficiency and human annotation accuracy with the unique TrialSieve tree-based annotation approach. In summary, TrialSieve provides a foundation to improve automated biomedical information extraction for frontend clinical research. Full article

Background/Objectives: Glaucoma is the second-leading cause of irreversible blindness globally. Retinal images such as color fundus photography have been widely used to detect glaucoma. However, little is known about the effectiveness of using raw clinical notes generated by glaucoma specialists in detecting glaucoma. This study aims to investigate the capability of deep learning approaches to detect glaucoma from clinical notes based on a real-world dataset including 10,000 patients. Different popular models are explored to predict the binary glaucomatous status defined from a comprehensive vision function assessment. Methods: We compared multiple deep learning architectures, including Long Short-Term Memory (LSTM) networks, Convolutional Neural Networks (CNNs), and transformer-based models BERT and BioBERT. LSTM exploits temporal feature dependencies within the clinical notes, while CNNs focus on extracting local textual features, and transformer-based models leverage self-attention to capture rich contextual information and feature correlations. We also investigated the group disparities of deep learning for glaucoma detection in various demographic groups. Results: The experimental results indicate that the CNN model achieved an Overall AUC of 0.80, slightly outperforming LSTM by 0.01. Both models showed disparities and biases in performance across different racial groups. However, the CNN showed reduced group disparities compared to LSTM across Asian, Black, and White groups, meaning it has the advantage of achieving more equitable outcomes. Conclusions: This study demonstrates the potential of deep learning models to detect glaucoma from clinical notes and highlights the need for fairness-aware modeling to address health disparities Full article

Objective: Electronic health records (EHRs) are becoming increasingly important in both academic research and business applications. Recent studies indicate that predictive tasks, such as heart failure detection, perform better when the geometric structure of EHR data, including the relationships between diagnoses and treatments, is considered. However, many EHRs lack essential structural information. This study aims to improve predictive accuracy in healthcare by constructing a Patient Knowledge Graph Ensemble Framework (PKGNN) to analyse ICU patient cohorts and predict mortality and hospital readmission outcomes. Methods: This study utilises a cohort of 42,671 patients from the MIMIC-IV dataset to build the PKGNN framework, which consists of three main components: (1) medical note extraction, (2) patient graph construction, and (3) prediction tasks. Advanced Natural Language Processing (NLP) models, including Clinical BERT, BioBERT, and BlueBERT, extract and integrate semantic representations from discharge summaries into a patient knowledge graph. This structured representation is then used to enhance predictive tasks. Results: Performance evaluations on the MIMIC-IV dataset indicate that the PKGNN framework outperforms state-of-the-art baseline models in predicting mortality and 30-day hospital readmission. A thorough framework analysis reveals that incorporating patient graph structures improves prediction accuracy. Furthermore, an ensemble model enhances risk prediction performance and identifies crucial clinical indicators. Conclusions: This study highlights the importance of leveraging structured knowledge graphs in EHR analysis to improve predictive modelling for critical healthcare outcomes. The PKGNN framework enhances the accuracy of mortality and readmission predictions by integrating advanced NLP techniques with patient graph structures. This work contributes to the literature by advancing knowledge graph-based EHR analysis strategies, ultimately supporting better clinical decision-making and risk assessment. Full article

The extraction of Adverse Drug Reactions from biomedical text is a critical task in the field of healthcare and pharmacovigilance. It serves as a cornerstone for improving patient safety by enabling the early identification and mitigation of potential risks associated with pharmaceutical treatments. This process not only helps in detecting harmful side effects that may not have been evident during clinical trials but also contributes to the broader understanding of drug safety in real-world settings, ultimately guiding regulatory actions and informing clinical practices. In this study, we conducted a comprehensive evaluation of eleven transformer-based models for ADR extraction, focusing on two widely used datasets: CADEC and SMM4H. The task was approached as a sequence labeling problem, where each token in the text is classified as part of an ADR or not. Various transformer architectures, including BioBERT, PubMedBERT, and SpanBERT, were fine-tuned and evaluated on these datasets. BioBERT demonstrated superior performance on the CADEC dataset, achieving an impressive F1 score of 86.13%, indicating its strong capability in recognizing ADRs within patient narratives. On the other hand, SpanBERT emerged as the top performer on the SMM4H dataset, with an F1 score of 84.29%, showcasing its effectiveness in processing the more diverse and challenging social media data. These results highlight the importance of selecting appropriate models based on the specific characteristics such as text formality, domain-specific language, and task complexity to achieve optimal ADR extraction performance. Full article