Search Results (34)

Breast cancer remains one of the leading causes of cancer-related mortality among women worldwide, where early and accurate diagnosis is critical for effective treatment. Although recent advances in deep learning have enabled automated radiology report generation from breast ultrasound images, most existing approaches function as black-box systems, limiting clinical trust and interpretability. This study proposes a trustworthy and explainable framework for automated breast ultrasound report generation that combines Vision-Language Modelling (VLM) with multi-level Explainable Artificial Intelligence (XAI). The proposed architecture integrates a Swin Transformer for visual feature extraction, BioBERT/ClinicalBERT for clinical text representation, and a GPT-2-based decoder for report generation through a dual cross-attention fusion mechanism. The framework is evaluated on benchmark breast ultrasound datasets paired with expert-annotated radiology reports using standard natural language generation metrics, including BLEU, ROUGE-L, METEOR, and CIDEr. Experimental results demonstrate that the multimodal architecture significantly improves report quality, clinical consistency, and semantic accuracy compared with conventional image-only and single-modal baselines. To address transparency and trustworthiness, the framework provides dual-level explanations through Grad-CAM visual heatmaps and LIME/SHAP-based token attribution analysis, enabling clinicians to understand both image regions and textual features influencing generated reports. Qualitative assessment further indicates strong alignment between model explanations and radiologist-identified diagnostic findings. Full article

Modern grid digitalization connects communication networks, monitoring terminals, service platforms, security devices, and operational data sources. Intrusion detection in this setting requires correlating heterogeneous security data with grid-side contextual evidence. To address class imbalance and cross-domain heterogeneity, this study proposes a heterogeneous feature group integration framework for intrusion detection with grid cybersecurity data. Four semantic feature subspaces are constructed symmetrically: network behaviour, power operation context, zone-derived communication/event topology, and system operation state, ensuring equal structural footing for subsequent modality-specific encoding. Transformer-based encoders model temporal dependencies in network, physical, and system state modalities, while a graph neural network encodes topology-related structural information. The resulting embeddings are integrated by a late fusion classifier for multiclass attack identification; the fusion process treats each feature group symmetrically at the decision level, without imposing a priori dominance among modalities. In the main run, the full model achieves an accuracy of 0.944, a macro F1 score of 0.891, a weighted F1 score of 0.937, a macro precision of 0.929, and a macro recall of 0.878. The corresponding balanced accuracy is 0.878, and the multiclass MCC is 0.924. Class-wise results show reliable performance on Benign, Scan, WebAtk, DDoS, DoS, and Backdoor classes, while Ransomware remains difficult and is frequently confused with WebAtk. Specifically, the Ransomware recall is 0.27, with most errors assigned to WebAtk. Modality analysis further indicates that modality contribution is class dependent: some feature groups have limited standalone discriminative power but provide complementary evidence after fusion. This finding highlights an inherent asymmetry in class-wise utility, which we counterbalance by employing both macro and weighted metrics, offering a symmetric evaluation lens that accounts for both minority and majority classes. These results show that grid-oriented intrusion detection benefits from decision-level integration of heterogeneous feature groups and imbalance-aware evaluation, where symmetric treatment of feature subspaces and evaluation perspectives jointly enhances robustness. Full article

Accurate, automated analysis of medical images is indispensable for effective diagnosis and treatment planning, particularly for complex multiclass diseases. This paper presents a system that combines a cascaded dual-stage U-Net with texture-based deep learning techniques to improve segmentation and classification precision. The cascaded dual-stage U-Net architecture comprises two parallel encoding-decoding pathways optimized for deep semantic feature extraction. This dual-path design enables the network to recognize lesion edges and intricate structural variations across imaging modalities. To enhance diagnostic performance, texture features are extracted using the Color Co-occurrence Matrix (CCM), which preserves local texture patterns and color relationships, providing helpful context for deep feature extraction. We feed this enriched data into a convolutional neural network (CNN) classifier, which categorizes the images into disease groups. Extensive evaluation on benchmark medical image datasets (MRI, CT, endoscopic images) demonstrates the framework’s superior performance in segmentation accuracy, classification precision, and robustness to noise and distortions. Integrating segmentation and classification in a coherent pipeline increases the reliability and interpretability of the diagnostic process. This technique represents an important step toward the clinical utility of intelligent, automated medical image processing. Full article

Accurate kidney tumor segmentation from abdominal CT is essential for quantitative assessment and treatment planning. However, indistinct tumor boundaries and substantial inter-patient shape variability render traditional hand-crafted feature-based methods unreliable for precise delineation. Although deep learning has advanced this task, these methods still struggle with multi-scale tumor characteristics, complex morphological variations, and background noise in medical images. To address these challenges, we propose MDA-Net, an end-to-end segmentation method based on enhanced multi-scale feature extraction and attention refinement. Specifically, we introduce a Multi-Scale Feature Extraction (MSFE) module into encoder–decoder skip connections to aggregate dilated features across multiple receptive fields and learn branch-wise weights for adaptive refinement and fusion, thereby enhancing boundary details and semantic cues to reduce tumor-tissue ambiguity. At the bottleneck, a Deformable Pyramid Feature Refinement (DPFR) module combines deformable sampling with pyramid contextual modeling, thereby improving adaptability to variations in tumor shape and scale while preserving feature resolution. Moreover, a Channel and Spatial Attention (CASA) module is embedded in the decoder to suppress background interference and enhance boundary-sensitive structures during upsampling via coordinated channel and spatial reweighting, thereby improving the reconstruction of fine-grained tumor morphology and contours. Experiments on both KiTS19 and KiTS21 show that MDA-Net consistently improves tumor boundary delineation, lesion localization, and mask reconstruction, demonstrating stronger robustness and cross-dataset generalizability than representative baseline methods. Ablation studies further confirm the complementary effects of MSFE, DPFR, and CASA. In addition, Grad-CAM visualizations improve the clinical transparency and interpretability of the model. Overall, this method advances deep learning for medical image analysis and supports precise diagnosis and treatment of renal tumors. Full article

The global incidence of skin cancer is rising, making it an increasingly critical public health issue. Malignant skin tumors such as melanoma originate from pathological alterations in skin cells, and their accurate early-stage segmentation is crucial for quantitative analysis, early diagnosis, and effective treatment. However, achieving precise and efficient segmentation remains a major challenge, as existing methods often struggle to capture complex lesion characteristics. To address this challenge, we propose a novel deep learning framework that integrates the PVT v2 backbone with two key modules: the Spatial-Aware Feature Enhancement (SAFE) module and the Multiscale Dual Cross-attention Fusion (MDCF) module. The SAFE module enhances multi-scale encoder features through a dual-branch architecture, which adaptively extracts offset information to integrate fine-grained shallow details with deep semantic information, thereby bridging the feature gap across network depths. The MDCF module establishes bidirectional cross-attention between decoder and encoder features, followed by multi-scale deformable convolutions that capture lesion boundaries and small fragments across heterogeneous receptive fields, thereby enriching semantic details while suppressing background interference. The proposed model was evaluated on two public benchmark datasets (ISIC 2016 and ISIC 2018), achieving Intersection over Union (IoU) scores of 87.33% and 83.67%, respectively. These results demonstrate superior performance compared to current state-of-the-art methods and indicate that our framework significantly enhances skin lesion image analysis, offering a promising tool for improving early detection of skin cancer. Full article

Deep learning approaches for egocentric video understanding often lack a principled theoretical treatment of stability, particularly when dealing with the sparse, noisy, and temporally ambiguous observations characteristic of first-person imaging. In this work, we frame egocentric video question answering not merely as a classification task, but as an ill-posed inverse problem aimed at reconstructing latent semantic intent from stochastically perturbed visual signals. To address the instability inherent in standard dual-encoder architectures, we present a framework with a mathematical interpretation that incorporates gated cross-modal interaction within the transformer backbone. Formally, the video-side update analyzed in this work is defined as a learnable convex combination of unimodal feature representations and cross-modal attention residuals; the full implementation applies analogous gated cross-modal updates bidirectionally. From a regularization perspective, the gating mechanism can be interpreted as an adaptive parameter that balances data fidelity against language-conditioned structural constraints during feature reconstruction. We provide the Bounded Update Property (Lemma 1) and an analytical layer-wise sensitivity bound and empirically demonstrate that the proposed framework achieves measurable improvements in both accuracy and stability on the EgoTaskQA and MSR-VTT benchmarks. On EgoTaskQA, our model improves accuracy from 27.0% to 31.7% (+4.7 pp) and reduces the accuracy drop under 50% frame drop from 3.93 pp to 0.94 pp. On MSR-VTT, our model improves accuracy by 13.0 pp over the dual-encoder baseline. Under severe perturbation (50% frame drop) on MSR-VTT, our model retains 97.7% of its clean performance, whereas the baseline exhibits near-zero drop accompanied by majority-class behavior. These results provide empirical evidence that the proposed interaction induces stable behavior under perturbations in an ill-posed multimodal inference setting, mitigating sensitivity to sampling variability while preserving query-relevant temporal structure. Furthermore, an entropy-based analysis indicates that the gating mechanism prevents excessive diffusion of attention, promoting coherent temporal reasoning. Overall, this work offers a mathematically informed perspective on designing interaction mechanisms for stable multimodal systems, with a focus on robust reasoning under temporal ambiguity. Full article

Multiple Sclerosis (MS) and Myelitis are serious inflammatory spinal cord disorders with overlapping clinical symptoms and radiological characteristics, making accurate differentiation challenging yet clinically essential. Early and precise diagnosis is critical for guiding treatment strategies and improving patient outcomes. In this study, we propose KhayyamNet, a novel hybrid deep learning architecture designed to fuse complementary local and global representations for the accurate diagnosis of MS and Myelitis using spinal MRI. To improve robustness and generalization capability, a comprehensive preprocessing strategy including data augmentation and intensity normalization is also applied to reduce noise and address data variability. The proposed architecture combines three complementary deep learning models for feature extraction composed of Xception for high-level semantic features, Convolutional Neural Networks (CNNs) for fine-grained local patterns, and Vision Transformers (ViTs) for global contextual representations via attention mechanisms. Extracted features are then fused and refined using the Minimum Redundancy Maximum Relevance (MRMR) algorithm to eliminate redundancy and retain the most informative signals. Finally, a Random Forest (RF) classifier utilizes the optimized feature set to achieve accurate and robust differentiation between MS, Myelitis, and control spinal MRIs. Experimental results demonstrate that KhayyamNet outperforms existing methods by achieving an average classification accuracy of $98.15\pm 0.80\%$. This framework demonstrates promising performance for the automated analysis of spinal MRIs and shows potential to assist in the differentiation of MS and Myelitis. While these findings highlight the potential of KhayyamNet for automated MRI interpretation, its evaluation is limited to a single-center dataset, and further validation on external multi-center data is required. Full article

Background: Hearing loss affects over 466 million individuals globally and is recognized as a major risk factor for Alzheimer’s disease, yet treatment personalization remains limited due to the complexity and diversity of underlying causes. Current diagnostic and therapeutic approaches lack standardized methods to accurately predict the most appropriate intervention for individual patients. The integration of medical ontologies with machine learning offers a promising solution for enhancing diagnostic accuracy and treatment personalization. Aim: Our study aimed to (i) develop a Systematized Nomenclature of Medicine—Clinical Terms (SNOMED CT)-aligned clinical ontology for hearing loss using Semantic Web Rule Language for automated reasoning; (ii) implement a Random Forest classifier trained on ontology-enriched patient data to classify hearing loss types (conductive, sensorineural, mixed, or normal); and (iii) predict optimal personalized treatments based on laterality, severity, audiometric thresholds, and medical history using real-world patient data. Methods: We developed a task ontology using Protégé 5.6.3 with Web Ontology Language (OWL), integrated SNOMED CT terminology alignment, and implemented Semantic Web Rule Language rules executed by the Pellet 2.2.0 reasoner. The framework was trained and evaluated on 3723 adult patients from the 2015–2016 National Health and Nutrition Examination Survey (NHANES) dataset with complete audiometric and clinical data. Random Forest models were developed using an 80–20 train-test split with stratified sampling and five-fold cross-validation. Performance was compared between K-Means clustering-based labeling and ontology-based semantic inference using accuracy, precision, recall, F1-score, and log loss metrics. Results: The ontology successfully generated semantic labels for all 3723 patients, enabling precise classification of hearing loss types, severity levels, and laterality. The Random Forest model with K-Means clustering achieved a test accuracy of 90.2% with a log loss of 0.2766 and a cross-validation mean accuracy of 91.22% (standard deviation 1.2%). Integration of ontology-based semantic enrichment significantly improved performance, achieving a test accuracy of 92.48% with a cross-validation mean accuracy of 92.80% (standard deviation 0.9%). F1-scores improved across all classes, with mixed hearing loss showing a notable increase from 0.86 to 0.92. Feature importance analysis identified audiometric thresholds, ontology-derived severity labels, and medical history as top predictors, enhancing clinical interpretability. Conclusions: This study demonstrates that combining SNOMED CT-aligned ontology with Random Forest classification achieves superior diagnostic accuracy and enables personalized treatment recommendations for hearing loss. The hybrid framework provides clinically interpretable decision support while ensuring semantic interoperability with electronic health records. Multi-institutional validation studies are necessary to assess generalizability across diverse populations before clinical deployment. Full article

The management of non-small cell lung cancer (NSCLC), including lung adenocarcinomas (LUAD), has been revolutionized with the advent of precision oncology. While advanced cancers often carry poor prognosis, those harboring specific molecular alterations sensitive to targeted therapy (notably tyrosine kinase inhibitor [TKI]) have experienced improved response to treatment and survival outcomes. Consequently, detecting these alterations through molecular screening panel has become standard in several countries, although this necessitates high-quality tissue sampling to inform optimal therapeutic decisions. Oncologic imaging occupies a pivotal role in the routine care of patients, in particular at diagnosis, with a wealth of information gathered but underutilized, as medical imaging reflects the disease in its entirety at a given time point. Moreover, recent advancements in imaging quantitative analysis, including radiomics and artificial intelligence, could aid in better integration and understanding of this information that has been overlooked for years. Several radiological phenotypes (or radiophenotypes) have been linked to tumor genomic alterations, both in standard radiology relying on semantic features and metastatic patterns, and in radiomics. Ultimately, understanding the relationships between imaging and targetable genomic alterations via accurate imaging biomarkers could complement ambiguous tumor or liquid biopsy, detect emerging new alterations, and even substitute biopsy through ‘virtual biopsy’. During the past decade, there has been a surge in research focused on radiogenomic assessment of NSCLC and especially LUAD. However, due to the low prevalence of many oncogenic drivers, the scientific literature may lack clarity or present conflicting findings. This comprehensive review aims to provide a summary of the current state of this research, offering insights into the complex interplay between imaging and genomic alterations in lung adenocarcinoma. Full article

Background/Objectives: Primary Progressive Aphasia (PPA) is a neurodegenerative disorder characterized by gradual and progressive deterioration of speech and language abilities. Speech and language therapy is considered an important intervention to slow decline and support the recovery of linguistic functions in individuals with PPA. This study aims to examine the effectiveness of an elaborated Semantic Feature Analysis (SFA) approach in enhancing naming abilities and semantic networks in individuals with the logopenic and semantic variants of PPA. Methods: Fourteen participants were recruited, including seven individuals with logopenic PPA and seven with semantic PPA. All participants received an elaborated SFA intervention twice weekly for four weeks. The Aphasia Language Assessment Test (ADD), the Turkish Picture Naming Test (T-RAT), and the SAQOL-39 were conducted at the following three time points: prior to treatment (pre-test), immediately after treatment (post-test), and one month post-treatment (follow-up). Results: Significant improvements were observed in ADD, T-RAT, and SAQOL-39 scores in both logopenic and semantic PPA groups following treatment (p < 0.05). Although follow-up scores declined compared to posttest performance (p < 0.05), several follow-up scores remained higher than pretest levels. Between-group comparisons indicated no significant difference in ADD scores; however, logopenic PPA participants demonstrated higher T-RAT scores (p < 0.05), while semantic PPA participants showed higher SAQOL-39 scores, except at follow-up (p < 0.05). Conclusions: Preliminary results suggest that the elaborated SFA intervention is effective in improving naming skills, language functioning, and quality of life in both logopenic and semantic variants of PPA. Although treatment gains partially decreased after one month, many improvements were maintained above baseline, supporting the clinical value of SFA in managing language decline in PPA. Full article

Extracting and organizing knowledge from the agricultural crop disease research literature are challenging tasks because of the heterogeneous terminologies, complicated symptom descriptions, and unstructured nature of scientific documents. In this study, we developed a multi-stage natural language processing (NLP) pipeline to automate knowledge extraction, organization, and integration from the agricultural research literature into a domain-consistent crop disease knowledge graph. The model combines transformer-based sentence embeddings with variational deep clustering to extract topics, which are further refined via facet-aware relevance scoring for sentence selection to be included in the summary. Lexicon-guided named entity recognition helps in the precise identification and normalization of terms for crops, diseases, symptoms, etc. Relation extraction based on a combination of lexical, semantic, and contextual features leads to the meaningful generation of triplets for the knowledge graph. The experimental results show that the method yielded consistently good results at each stage of the knowledge extraction process. Among the combinations of embedding and deep clustering methods, SciBERT + VaDE achieved the best clustering results. The extraction of representative sentences for disease symptoms, control/treatment, and prevention obtained high F1-scores of around 0.8. The resulting knowledge graph has high node coverage and high relation completeness, as well as high precision and recall in triplet generation. The multi-stage NLP pipeline effectively converts unstructured agricultural research texts into a coherent and semantically rich knowledge graph, providing a basis for further research in crop disease analysis, knowledge retrieval, and data-driven decision support in agricultural informatics. Full article

Semantic segmentation plays a critical role in medical image analysis, offering indispensable information for the diagnosis and treatment planning of liver diseases. However, due to the complex anatomical structure of the liver and significant inter-patient variability, the current methods exhibit notable limitations in feature extraction and fusion, which pose a major challenge to achieving accurate liver segmentation. To address these challenges, this study proposes an improved U-Net-based liver semantic segmentation method that enhances segmentation performance through optimized feature extraction and fusion mechanisms. Firstly, a multi-scale input strategy is employed to account for the variability in liver features at different scales. A multi-scale convolutional attention (MSCA) mechanism is integrated into the encoder to aggregate multi-scale information and improve feature representation. Secondly, an atrous spatial pyramid pooling (ASPP) module is incorporated into the bottleneck layer to capture features at various receptive fields using dilated convolutions, while global pooling is applied to enhance the acquisition of contextual information and ensure efficient feature transmission. Furthermore, a Channel Transformer module replaces the traditional skip connections to strengthen the interaction and fusion between encoder and decoder features, thereby reducing the semantic gap. The effectiveness of this method was validated on integrated public datasets, achieving an Intersection over Union (IoU) of 0.9315 for liver segmentation tasks, outperforming other mainstream approaches. This provides a novel solution for precise liver image segmentation and holds significant clinical value for liver disease diagnosis and treatment. Full article

Background: Prostate cancer is a major cause of cancer-related mortality among men, with approximately 15% of newly diagnosed patients having pelvic lymph node metastasis (PLNM). For this reason, PLNM identification before localized PCa treatment would significantly impact treatment planning, clinical judgment, and patient outcome prediction. Radiomics has gained popularity for its ability to predict tumor behavior and prognosis without invasive procedures. Magnetic resonance imaging (MRI) is widely used in radiomic workups, particularly for prostate cancer. This study aims to predict lymph node invasion in prostate cancer patients using clinical information and mp-MRI radiomics features extracted from the suspicious nodule, prostate gland, and periprostatic adipose tissue (PPAT). Methods: A retrospective review of 85 patients who underwent mp-MRI at our radiology department between 2016 and 2022 was conducted. This study included patients who underwent prostatectomy and lymphadenectomy with complete histological examination and previous staging mp-MRI and were divided into two groups based on lymph node status (positive/negative). Data were collected from each patient, including clinical information, radiomics, and semantic data (such as tumor MRI characteristics, histological tumor details, and lymph node status (LNS)). MRI exams were conducted using a 1.5-T system and were used to study the prostate gland. A three-year resident manually segmented the prostate nodule, prostatic gland, and periprostatic tissue using an open-source segmentation program. A random forest (RF) machine learning model was developed and tested using Chat-GPT version 4.0 software. The model’s performance in predicting LNS was assessed using accuracy, precision, recall, F1 score, and area under the curve (AUC) receiver operating characteristic (ROC), with sensitivity and specificity evaluated using DeLong’s test. Results: Random forest demonstrated the best performance in prediction considering features extracted from DWI nodules (67% of accuracy, 0.83 AUC), from T2 fat (78% of accuracy, 0.86 AUC), and from T2 glands (78% of accuracy, 0.97 AUC). The combination of the three sequences in the nodule evaluation was more accurate compared with the single sequences (88%). Combining all the nodule features with gland and PPAT features, an accuracy of 89% with AUC near 1 was obtained. Compared with the analysis of the nodule and the PPAT, the whole-gland evaluation had the best performance (p ≤ 0.05) in predicting LNS when compared with the nodule. Conclusions: Precise nodal staging is essential for PCa patients’ prognosis and therapeutic strategy. When compared with a radiologist’s assessment, radiomics models enhance the diagnostic accuracy of lymph node staging for prostate cancer. Although data are still lacking, deep learning models may be able to further improve on this. Full article

This paper extends our previous work on cleft lip and/or palate (CL/P) classification, which employed vision transformers (ViTs) and Siamese neural networks. We now integrate SigLIP 2, a state-of-the-art multilingual vision–language model, for feature extraction, replacing the previously utilized BiomedCLIP. SigLIP 2 offers enhanced semantic understanding, improved localization capabilities, and multilingual support, potentially leading to more robust feature representations for CL/P classification. We hypothesize that SigLIP 2’s superior feature extraction will improve the classification accuracy of CL/P types (bilateral, unilateral, and palate-only) from the UltraSuite CLEFT dataset, a collection of ultrasound video sequences capturing tongue movements during speech with synchronized audio recordings. A comparative analysis is conducted, evaluating the performance of our original ViT-Siamese network model (using BiomedCLIP) against a new model leveraging SigLIP 2 for feature extraction. Performance is assessed using accuracy, precision, recall, and F1 score, demonstrating the impact of SigLIP 2 on CL/P classification. The new model achieves statistically significant improvements in overall accuracy (86.6% vs. 82.76%) and F1 scores for all cleft types. We discuss the computational efficiency and practical implications of employing SigLIP 2 in a clinical setting, highlighting its potential for earlier and more accurate diagnosis, personalized treatment planning, and broader applicability across diverse populations. The results demonstrate the significant potential of advanced vision–language models, such as SigLIP 2, to enhance AI-powered medical diagnostics. Full article

Medical image analysis is critical for diagnosing and planning treatments, particularly in addressing heart disease, a leading cause of mortality worldwide. Precise segmentation of the left atrium, a key structure in cardiac imaging, is essential for detecting conditions such as atrial fibrillation, heart failure, and stroke. However, its complex anatomy, subtle boundaries, and inter-patient variations make accurate segmentation challenging for traditional methods. Recent advancements in deep learning, especially semantic segmentation, have shown promise in addressing these limitations by enabling detailed, pixel-wise classification. This study proposes a novel segmentation framework Adaptive Multiscale U-Net (AMU-Net) combining Convolutional Neural Networks (CNNs) and transformer-based encoder–decoder architectures. The framework introduces a Contextual Dynamic Encoder (CDE) for extracting multi-scale features and capturing long-range dependencies. An Adaptive Feature Decoder Block (AFDB), leveraging an Adaptive Feature Attention Block (AFAB) improves boundary delineation. Additionally, a Spectral Synthesis Fusion Head (SFFH) synthesizes spectral and spatial features, enhancing segmentation performance in low-contrast regions. To ensure robustness, data augmentation techniques such as rotation, scaling, and flipping are applied. Laplacian approximation is employed for uncertainty estimation, enabling interpretability and identifying regions of low confidence. Our proposed model achieves a Dice score of 93.35, a Precision of 94.12, and a Recall of 92.78, outperforming existing methods. Full article