Next Issue
Volume 8, June
Previous Issue
Volume 8, April
 
 

Mach. Learn. Knowl. Extr., Volume 8, Issue 5 (May 2026) – 28 articles

Cover Story (view full-size image): Self-Supervised Learning (S-SL) is a recent line of research that could represent the next step to understanding human intuition. By blending the strengths of unsupervised and supervised learning paradigms, S-SL endows Deep Learning models with stronger generalization capabilities. Although better known for its applications in Computer Vision and Natural Language Processing, S-SL has also proved its value in other fields, such as Cybersecurity. In this work, we review the current progress and future trends in S-SL for the two most relevant problems discussed in the Cybersecurity literature: network intrusion and malware detection. We highlight several promising research directions, such as creating S-SL models for tabular data and studying the transferability of vulnerabilities. View this paper
  • Issues are regarded as officially published after their release is announced to the table of contents alert mailing list.
  • You may sign up for e-mail alerts to receive table of contents of newly released issues.
  • PDF is the official format for papers published in both, html and pdf forms. To view the papers in pdf format, click on the "PDF Full-text" link, and use the free Adobe Reader to open them.
Order results
Result details
Section
Select all
Export citation of selected articles as:
19 pages, 1853 KB  
Article
A Directional Semantic Enhancement Approach with Gated Fusion for Multimodal Arabic Sentiment Analysis
by Ayoub Ben Cheikhi, El Habib Nfaoui and Oumayma Elbiach
Mach. Learn. Knowl. Extr. 2026, 8(5), 139; https://doi.org/10.3390/make8050139 - 21 May 2026
Viewed by 557
Abstract
Multimodal Arabic sentiment analysis has gained increasing attention due to the growing volume of user-generated multimedia content. However, effectively integrating textual, acoustic, and visual modalities remains challenging because of modality imbalance and weak cross-modal alignment. This study proposes a Directional Semantic Enhancement approach [...] Read more.
Multimodal Arabic sentiment analysis has gained increasing attention due to the growing volume of user-generated multimedia content. However, effectively integrating textual, acoustic, and visual modalities remains challenging because of modality imbalance and weak cross-modal alignment. This study proposes a Directional Semantic Enhancement approach with Gated Fusion to address these limitations. The objective is to explicitly model similarity-guided semantic transfer between modalities while adaptively regulating information flow during fusion. The proposed architecture consists of four main stages: modality encoding, directional semantic enhancement, gated fusion, and classification. Directional semantic interactions enable structured cross-modal knowledge exchange, while adaptive gating mechanisms balance original and enhanced representations to mitigate modality-specific noise. Extensive experiments are conducted on the Ar-MuSA benchmark dataset, which contains 8700 multimodal samples. The proposed approach achieves 89.89% accuracy and an F1-score of 0.8989 with a latent dimension of 1024, outperforming early fusion, late fusion, and recent state-of-the-art methods. The study highlights the importance of controlled cross-modal alignment and provides a scalable approach for robust multimodal sentiment understanding in Arabic multimedia environments. Full article
(This article belongs to the Section Learning)
Show Figures

Figure 1

23 pages, 4335 KB  
Article
Evaluating the Impact of Adapter-Based Fine-Tuning on Structured Parsing Performance in Large Language Models
by Ratomir Karlović, Luka Sever, Sandi Baressi Šegota, Vedran Mrzljak and Ivan Lorencin
Mach. Learn. Knowl. Extr. 2026, 8(5), 138; https://doi.org/10.3390/make8050138 - 21 May 2026
Viewed by 640
Abstract
Recent advances in large language models (LLMs) highlight two dominant strategies for performance improvement: prompt engineering and fine-tuning. While prompt design can significantly influence model output, it remains uncertain whether lightweight fine-tuning methods, such as adapter-based training, offer meaningful advantages for structured, domain-specific [...] Read more.
Recent advances in large language models (LLMs) highlight two dominant strategies for performance improvement: prompt engineering and fine-tuning. While prompt design can significantly influence model output, it remains uncertain whether lightweight fine-tuning methods, such as adapter-based training, offer meaningful advantages for structured, domain-specific tasks. This study builds on prior research comparing three prompting strategies for natural-language command parsing into JSON schemas. Expanding that framework, the current work investigates how adapter-based fine-tuning, where most model parameters are frozen and only small adapter modules are trained, affects model accuracy and consistency. The experiment uses the same controlled shopping-cart parsing task and dataset of 12,000 synthetic commands to ensure direct comparability. Results quantify the trade-off between computational cost and performance gains, offering evidence-based insights into whether fine-tuning is a justified investment compared to advanced prompt engineering. The contribution of this study is a clear, empirical framework for deciding when fine-tuning meaningfully enhances LLM utility in applied natural-language understanding. Full article
(This article belongs to the Section Learning)
Show Figures

Graphical abstract

28 pages, 3750 KB  
Article
Generating Multiple-Choice Knowledge Questions with Interpretable Difficulty Estimation Using Knowledge Graphs and Large Language Models
by Mehmet Can Şakiroğlu, Halil Altay Güvenir and Kamer Kaya
Mach. Learn. Knowl. Extr. 2026, 8(5), 137; https://doi.org/10.3390/make8050137 - 20 May 2026
Viewed by 388
Abstract
Generating multiple-choice questions (MCQs) with difficulty estimation remains challenging in automated MCQ-generation systems used in adaptive, AI-assisted education. This study proposes a novel methodology for generating MCQs with difficulty estimation from input documents by utilizing knowledge graphs (KGs) and large language models (LLMs). [...] Read more.
Generating multiple-choice questions (MCQs) with difficulty estimation remains challenging in automated MCQ-generation systems used in adaptive, AI-assisted education. This study proposes a novel methodology for generating MCQs with difficulty estimation from input documents by utilizing knowledge graphs (KGs) and large language models (LLMs). Our approach uses an LLM to construct a KG from input documents, from which MCQs are then systematically generated. Each MCQ is generated by selecting a node from the KG as the key, sampling a related triple or quintuple—optionally augmented with an extra triple—and prompting an LLM to generate a corresponding stem from these graph components. Distractors are then selected from the KG. For each MCQ, nine difficulty signals are computed and combined into a unified difficulty score using a data-driven approach. Within a 150-MCQ, proof-of-concept dataset from Wikipedia, the proposed signals show interpretable associations with empirical incorrect-answer rates aligning with human responses/performance. The results support the feasibility of the proposed pipeline, yet a larger-scale human study may be required to establish deployment-scale validity. Our approach improves automated MCQ generation by integrating structured knowledge representations with LLMs and a data-driven difficulty estimation model. Full article
(This article belongs to the Section Learning)
Show Figures

Figure 1

21 pages, 3948 KB  
Article
Demonstrating Data-to-Knowledge Pipelines for Connecting Production Sites in the World Wide Lab
by Leon Gorissen, Jan-Niklas Schneider, Mohamed Behery, Philipp Brauner, Moritz Lennartz, David Kötter, Thomas Kaster, Oliver Petrovic, Christian Hinke, Thomas Gries, Gerhard Lakemeyer, Martina Ziefle, Christian Brecher and Constantin Häfner
Mach. Learn. Knowl. Extr. 2026, 8(5), 136; https://doi.org/10.3390/make8050136 - 20 May 2026
Cited by 1 | Viewed by 590
Abstract
The digital transformation of production requires methods for integrating, storing, and operationalizing data across organizational boundaries, yet most existing approaches remain siloed and unidirectional, lacking a systematic loop from raw data to actionable knowledge and back. We introduce Data-to-Knowledge (D2K) and Knowledge-to-Data (K2D) [...] Read more.
The digital transformation of production requires methods for integrating, storing, and operationalizing data across organizational boundaries, yet most existing approaches remain siloed and unidirectional, lacking a systematic loop from raw data to actionable knowledge and back. We introduce Data-to-Knowledge (D2K) and Knowledge-to-Data (K2D) pipelines as a universal production concept built on networks of Digital Shadows. The Data-to-Knowledge (D2K) pipeline is realized as a cross-organizational proof of concept that captures and semantically annotates robotic trajectory data from three independent research institutes and uses those data to train an inverse-dynamics foundation model for robot control. Centralized aggregation via an existing FAIR-compliant research data repository was chosen deliberately over federated alternatives to maximize semantic interoperability and reuse of shared infrastructure; federated and privacy-preserving extensions are identified as a promising future direction. Fine-tuning the cross-organizationally trained foundation model reduces training time by approximately 85% relative to end-to-end training from scratch, while achieving comparable accuracy on a standardized inverse-dynamics benchmark. These gains are attributable to the combination of cross-site data aggregation and transfer learning; isolating the contribution of semantic annotation alone remains a topic for future ablation work. The implementation demonstrates that semantically enriched, cross-organizational D2K pipelines can accelerate model development and reduce redundant data collection within a constrained but practically relevant class of robotics tasks. We further discuss limitations, governance challenges, and how these pipelines can contribute to a broader World Wide Lab for collaborative production research. Full article
(This article belongs to the Section Learning)
Show Figures

Graphical abstract

30 pages, 26440 KB  
Article
HybridHiT-UNet: Multi-Scale Temporal U-Net with Hierarchical Shot-Aware Transformers for Video Summarization
by Saadman Sakib, Tanjim Mahmud, Karl Andersson and Kaushik Deb
Mach. Learn. Knowl. Extr. 2026, 8(5), 135; https://doi.org/10.3390/make8050135 - 20 May 2026
Viewed by 587
Abstract
Video summarization aims to produce a short yet informative summary of a long video while reducing the amount of redundancy. Most transformer-based methods are single-temporal scale or are unconcerned with shot-level structure, limiting temporal coherence and cross-dataset generalization. To fill these gaps, we [...] Read more.
Video summarization aims to produce a short yet informative summary of a long video while reducing the amount of redundancy. Most transformer-based methods are single-temporal scale or are unconcerned with shot-level structure, limiting temporal coherence and cross-dataset generalization. To fill these gaps, we present HybridHiT-UNet, a supervised framework that combines three complementary parts: a pretrained Vision Transformer encoder to provide spatially rich frame representations, a multi-scale 1D Temporal U-Net backbone to provide hierarchical temporal modeling of frame representations, and a shot-aware hierarchical transformer scoring head to provide inter-shot context to importance prediction. Frame-level scores are summed into shot-level utilities and optimized with a knapsack selection on a fixed-length budget, and a weighted focal loss is used to address extreme class imbalance. Wide experiments using four benchmarks (SumMe, TVSum, OVP, and YouTube) under canonical, augmented, and transfer protocols demonstrate that HybridHiT-UNet achieves F1-scores of 65.8% on SumMe and 79.92% on TVSum, which is higher than the existing methods, which still achieve diversity scores of 64.98% and 48.68%, respectively. A systematic study further demonstrates that a 20% summary budget would yield a consistently superior coverage-diversity trade-off than the traditional 15% one, which provides useful evidence-based advice on the selection of summary length. Full article
(This article belongs to the Section Learning)
Show Figures

Graphical abstract

38 pages, 1431 KB  
Systematic Review
Explainable Artificial Intelligence (XAI) for Cancer Classification in Medical Imaging: A Systematic Review
by Khairil Imran Ghauth and Yanche Ari Kustiawan
Mach. Learn. Knowl. Extr. 2026, 8(5), 134; https://doi.org/10.3390/make8050134 - 20 May 2026
Viewed by 1170
Abstract
Our study examines the growing role of Explainable Artificial Intelligence (XAI) in cancer medical imaging, where transparency and interpretability are essential for trustworthy clinical decision making. Using a PRISMA-guided systematic literature review, 926 records published between 2020 and 2026 were identified from major [...] Read more.
Our study examines the growing role of Explainable Artificial Intelligence (XAI) in cancer medical imaging, where transparency and interpretability are essential for trustworthy clinical decision making. Using a PRISMA-guided systematic literature review, 926 records published between 2020 and 2026 were identified from major databases, with 46 studies meeting the inclusion criteria after screening and quality assessment. The review systematically analyzes XAI techniques, model architectures, evaluation approaches, interpretability mechanisms, challenges, and future research directions. The findings show that gradient-based methods, particularly Grad-CAM, dominate the field due to their ease of integration with convolutional neural networks. At the same time, complementary approaches such as LIME, SHAP, and Integrated Gradients provide additional attribution insights. Evaluation practices remain heterogeneous, with a strong reliance on qualitative visual inspection and limited standardized quantitative frameworks. XAI contributes to interpretability primarily through spatial localization, feature attribution, and clinical decision support; however, challenges persist, including instability in explanations, coarse localization, high computational cost, and limited compatibility with transformer-based models. Overall, while XAI enhances transparency in cancer imaging, its clinical reliability remains constrained by methodological and technical limitations. Future work should focus on standardized evaluation, clinician-centered validation, and the development of robust, multimodal, and architecture-aware explainability frameworks. Full article
Show Figures

Graphical abstract

29 pages, 1270 KB  
Systematic Review
Reactive to Predictive Mobility Management: A Systematic Review of ML-Driven Handover Optimization in 5G and Beyond
by Teresia Ankome and Eisuke Hanada
Mach. Learn. Knowl. Extr. 2026, 8(5), 133; https://doi.org/10.3390/make8050133 - 18 May 2026
Viewed by 724
Abstract
Handover optimization is essential for seamless connectivity in 5G and beyond networks. Existing approaches present fundamental challenges of centralized solutions achieving coordination and accuracy but creating privacy risks under the General Data Protection Regulation (GDPR), while distributed privacy-preserving approaches protect user data but [...] Read more.
Handover optimization is essential for seamless connectivity in 5G and beyond networks. Existing approaches present fundamental challenges of centralized solutions achieving coordination and accuracy but creating privacy risks under the General Data Protection Regulation (GDPR), while distributed privacy-preserving approaches protect user data but lack the network-wide visibility necessary for optimal mobility decisions. This systematic review synthesizes 49 peer-reviewed studies published between 2010 and 2025, identified through a PRISMA-compliant search across IEEE Xplore, ScienceDirect, SpringerLink, MDPI, ACM Digital Library, and Google Scholar. Eligible studies addressed cellular handover or mobility management using traditional signal-based, Machine Learning, Federated Learning, Software-Defined Networking strategies, and reported quantitative performance metrics. A structured quality assessment evaluated methodological rigor, dataset validation, benchmarking practices, handover-specific metrics, and scalability. Synthesis evidence shows that existing approaches do not simultaneously satisfy critical requirements for next-generation mobility management of accuracy, privacy, scalability, and real-time network-wide coordination. Machine learning achieves high accuracy (up to 97%) but depends on centralized data; Reinforcement Learning supports real-time adaptation but incurs high computational costs; federated learning preserve privacy but suffers from limited global coordination; and software-defined networking enables centralized control but requires continuous transmission of raw data. Evidence quality is further limited to simulation-based assessments and limited real-world datasets. Overall, the reviews identify a clear evolution from reactive threshold-based methods towards proactive prediction and highlights the need for unified, privacy-preserving and globally coordinated handover frameworks. The findings point toward integrating federated learning with Software-Defined Mobile Networking as promising architectural direction for 6G mobility management. Full article
Show Figures

Figure 1

26 pages, 2758 KB  
Article
A Quantum-Probability-Inspired Complex-Valued Model for Multilingual Stance Detection
by Muhammad Ebrahim Ahmadi and Monireh Hosseini
Mach. Learn. Knowl. Extr. 2026, 8(5), 132; https://doi.org/10.3390/make8050132 - 18 May 2026
Viewed by 410
Abstract
In this study, we propose a quantum-probability-inspired complex-valued model for multilingual stance detection. The model brings together ideas from granular computing and quantum theory to better capture semantic meaning across different languages. The proposed model combines contextual embeddings, graph convolutional networks, and a [...] Read more.
In this study, we propose a quantum-probability-inspired complex-valued model for multilingual stance detection. The model brings together ideas from granular computing and quantum theory to better capture semantic meaning across different languages. The proposed model combines contextual embeddings, graph convolutional networks, and a quantum-inspired feature interaction module (QFIM) to capture complex, high-order, and non-linear relationships in multilingual data. The QFIM models quantum amplitude-like interactions to represent overlapping semantic patterns in the latent space. This design helps the system better distinguish subtle differences in stance expressions. To strengthen the representation, a granulation mechanism based on contextual embedding similarity is employed to extract semantically coherent text granules. These granules expand the feature space and help align different elements of the stance structure more accurately. Experimental results on standard benchmark datasets show that the proposed model consistently achieves better performance than existing state-of-the-art methods. Full article
(This article belongs to the Section Learning)
Show Figures

Figure 1

64 pages, 2805 KB  
Systematic Review
State of the Art: Analysis of Deep Learning Techniques in Images Acquired in an Aquatic Environment
by Vanesa Lopez-Vazquez, Geovanny Satama-Bermeo, Hasan Issa Raheem and Jose Manuel Lopez-Guede
Mach. Learn. Knowl. Extr. 2026, 8(5), 131; https://doi.org/10.3390/make8050131 - 14 May 2026
Viewed by 411
Abstract
The oceans and other marine ecosystems are indispensable to life, so the understanding and knowledge of their biodiversity is crucial to the use of their resources and exploration. These environments are complex and difficult to access, so different types of remote sensing technologies [...] Read more.
The oceans and other marine ecosystems are indispensable to life, so the understanding and knowledge of their biodiversity is crucial to the use of their resources and exploration. These environments are complex and difficult to access, so different types of remote sensing technologies are used to study them. These intelligent sensors can collect a massive amount of data, which, once reviewed and analyzed, can help to draw conclusions and increase knowledge of these underwater environments. Manually reviewing and organizing through this large amount of information is both time-consuming and costly. Therefore, it is advisable to employ automated techniques from machine learning and deep learning fields. In recent years, these methods have proven to be efficient and have obtained very good results in solving different problems applied to the marine world: image enhancement, image classification, segmentation and object detection. This paper presents a systematic review, conducted in accordance with the PRISMA 2020 guidelines, aimed at summarizing the methods used to address underwater problems and their reported results. Full article
(This article belongs to the Section Thematic Reviews)
Show Figures

Figure 1

28 pages, 3333 KB  
Article
Automatic Index Tuning via Quantum Deep Reinforcement Learning
by Jorge Duarte, Le Gruenwald, Laurent D’Orazio and Jorge Bernardino
Mach. Learn. Knowl. Extr. 2026, 8(5), 130; https://doi.org/10.3390/make8050130 - 13 May 2026
Viewed by 598
Abstract
The Index Selection Problem (ISP) refers to the task of automatically identifying the most appropriate set of indexes for a given database workload that can minimize execution costs. However, ISP is a fundamental yet complex challenge in database management systems. In the era [...] Read more.
The Index Selection Problem (ISP) refers to the task of automatically identifying the most appropriate set of indexes for a given database workload that can minimize execution costs. However, ISP is a fundamental yet complex challenge in database management systems. In the era of data-intensive applications, efficient index strategies are increasingly necessary to maintain scalability and responsiveness. This paper presents a novel automated index selection algorithm for centralized databases that employs a Double Deep Q-Network (DDQN) as the classical learning backbone and extends it with quantum-enhanced variants. Two hybrid quantum variants were proposed: Quantum Double Deep Q-Network Mixed (QDDQNM), which incorporates a residual classical pathway, and Quantum Double Deep Q-Network Boosted (QDDQNB), a boosted model without residuals. All variants were systematically evaluated using the TPC-H benchmark at two small scale factors, 10 MB and 100 MB. Experimental results show that the evaluated Deep Reinforcement Learning (DRL)-based methods improve on the SMARTIX baseline within this proof-of-concept setting. The quantum-enhanced models achieved higher best-run accumulated rewards in the reported experiments, but they also incurred substantially higher simulation cost. The results therefore suggest interesting hybrid learning behavior under the tested conditions, while also highlighting that practical scalability and cost-performance trade-offs remain important limitations for future work. Full article
(This article belongs to the Section Learning)
Show Figures

Figure 1

47 pages, 8799 KB  
Article
An Interpretable and Uncertainty-Aware Deep Learning Framework for Early Sepsis Prediction Using SHAP-Enhanced Attention and Continuous-Time Neural Networks
by Rekha R. Nair, Tina Babu, Balamurugan Balusamy, Wee How Khoh, Alaa M. Momani and Basem Abu Zneid
Mach. Learn. Knowl. Extr. 2026, 8(5), 129; https://doi.org/10.3390/make8050129 - 13 May 2026
Viewed by 782
Abstract
Sepsis is a prominent cause of death in intensive care units, and delayed diagnosis greatly worsens fatal outcomes due to the complex, irregular, and uneven character of clinical time-series data. Hence we proposed an interpretable and uncertainty-aware deep learning architecture that solves data [...] Read more.
Sepsis is a prominent cause of death in intensive care units, and delayed diagnosis greatly worsens fatal outcomes due to the complex, irregular, and uneven character of clinical time-series data. Hence we proposed an interpretable and uncertainty-aware deep learning architecture that solves data quality, temporal irregularity, and clinical explainability restrictions, which are frequently addressed separately by existing models. The suggested method combines Bidirectional Recurrent Imputation for Time Series (BRITS)-based imputation, hybrid Conditional Tabular Generative Adversarial Network-Synthetic Minority Over-sampling Technique (CTGAN-SMOTE) data augmentation, a Temporal Convolutional Networks (TCN)-Attention architecture, and continuous-time neural Ordinary Differential Equations (ODEs), along with SHapley Additive exPlanations (SHAP)-based feature attribution and uncertainty quantification. The experimental evaluation on a large ICU dataset reveals greater predictive accuracy, with an AUROC of 0.926 and accurate early warnings up to six hours before clinical onset, all while maintaining strong interpretability and calibration. The proposed framework demonstrates strong predictive performance and provides early warnings up to six hours before clinical onset, while maintaining interpretability and calibration. While the results are promising, further validation across multiple clinical settings is required to confirm its generalisability and real-world applicability. Full article
(This article belongs to the Section Learning)
Show Figures

Graphical abstract

28 pages, 981 KB  
Article
Web Search-Enhanced Small Language Models: A Case Study for a Kazakh-Centric Language Model
by Akylbek Maxutov, Nūrali Medeu and Huseyin Atakan Varol
Mach. Learn. Knowl. Extr. 2026, 8(5), 128; https://doi.org/10.3390/make8050128 - 12 May 2026
Viewed by 400
Abstract
Small language models (SLMs) are valued for their computational efficiency and suitability for edge deployment, but often underperform in localized linguistic and cultural contexts due to their limited parameter size. This study explores integrating real-time web search into Qolda, a 4B-parameter Kazakh-centric SLM, [...] Read more.
Small language models (SLMs) are valued for their computational efficiency and suitability for edge deployment, but often underperform in localized linguistic and cultural contexts due to their limited parameter size. This study explores integrating real-time web search into Qolda, a 4B-parameter Kazakh-centric SLM, to close the performance gap with larger models. We assess two strategies: Naïve Retrieval-Augmented Generation (RAG), which uses raw benchmark questions as search queries, and Query-Refined RAG, which applies various refiner models, including a supervised distillation-tuned Qolda, to optimize queries. On the KazCulture and KazMMLU benchmarks, the Naïve RAG approach in reasoning-enabled mode achieved an average accuracy of 76.00%, improving on the Zero-Shot evaluation result of 60.37%, and, in this system-level comparison, exceeding the Zero-Shot accuracy of larger open-source models such as Qwen3-32B (64.72%) and Gemma-3-27b-it (60.24%), which were evaluated without retrieval augmentation. Query refinement improved the accuracy by about 3%, from 76.00% to 79.46%, but nearly doubled the computational cost. Inference time analysis shows that Naïve RAG adds approximately 1 s of retrieval latency per question. Query refiners introduce up to 4 s of additional overhead. However, the retrieved context reduces the time required for model reasoning in think mode. The most notable gains were observed in localized cultural knowledge, where web search integration correctly answered 32.9% of KazCulture questions that the Zero-Shot baseline failed on, while losing only 16.9% in return. These results suggest that retrieval-augmented SLMs can offer a cost-effective and competitive alternative to larger models for tasks in the domains of Kazakh language and Kazakh culture. Full article
Show Figures

Graphical abstract

27 pages, 12506 KB  
Article
An Integrated RAG and Agent-Based Architecture for Automated Assessment in Moodle
by Anastasia Vangelova and Adelina Aleksieva-Petrova
Mach. Learn. Knowl. Extr. 2026, 8(5), 127; https://doi.org/10.3390/make8050127 - 12 May 2026
Viewed by 971
Abstract
The growing adoption of Generative AI in education has created opportunities to automate complex pedagogical tasks, yet reliably and scalably assessing open-ended responses remains a challenge. This study proposes and evaluates an architectural solution for integrating a Large Language Model (LLM) into Moodle, [...] Read more.
The growing adoption of Generative AI in education has created opportunities to automate complex pedagogical tasks, yet reliably and scalably assessing open-ended responses remains a challenge. This study proposes and evaluates an architectural solution for integrating a Large Language Model (LLM) into Moodle, combining Retrieval-Augmented Generation (RAG) and AI agent mechanisms to enable automated grading of open-ended student responses. A Moodle instance was deployed for experimental purposes, with 32 students across Bulgarian- and English-language sections, yielding data at the student (N = 32) and task (N = 160) levels, including AI-generated and instructor-assigned scores and system processing logs. The results demonstrate that the proposed system achieves substantial reductions in grading time while maintaining high agreement with expert assessments. Bias analysis revealed minimal systematic deviation across both language groups, indicating that the system preserves assessment objectivity without consistent over- or underestimation based on language. These findings suggest that a combined RAG and agentic LLM architecture can deliver efficient, accurate, and linguistically robust automated assessment within an LMS environment, offering practical design guidelines applicable to other educational platforms and similar systems. Full article
(This article belongs to the Section Learning)
Show Figures

Graphical abstract

39 pages, 2528 KB  
Review
Knowledge Graphs in Autonomous Driving: Construction, Integration, and Real-Time Reasoning
by Patrik Viktor and Gábor Kiss
Mach. Learn. Knowl. Extr. 2026, 8(5), 126; https://doi.org/10.3390/make8050126 - 11 May 2026
Viewed by 588
Abstract
Autonomous driving systems require the integration of heterogeneous sensor data, distributed V2X communication, and safety-critical decision-making into coherent and interpretable world models. This review provides a systematic analysis of knowledge graph (KG)-based approaches in autonomous driving between 2015 and 2025, following a PRISMA-aligned [...] Read more.
Autonomous driving systems require the integration of heterogeneous sensor data, distributed V2X communication, and safety-critical decision-making into coherent and interpretable world models. This review provides a systematic analysis of knowledge graph (KG)-based approaches in autonomous driving between 2015 and 2025, following a PRISMA-aligned methodology. The literature is organised along a perception → representation → reasoning → decision taxonomy, covering traffic ontologies, V2X knowledge integration, dynamic KG updates, real-time reasoning architectures, and benchmark datasets. A clear shift from static representational ontologies toward predictive and, in a smaller subset, closed-loop validated neuro-symbolic architectures. Knowledge graphs emerge as semantic integration layers that improve contextual reasoning, explainability, and rule compliance in safety-critical environments. Key challenges include scalable real-time reasoning, standardised evaluation frameworks, and safety-aligned integration of learning-based components. Full article
Show Figures

Figure 1

31 pages, 1936 KB  
Article
Controlled Agentic AI Systems: A Governance-Driven Architecture for Auditable and Reproducible Decision Pipelines
by Tymoteusz Miller
Mach. Learn. Knowl. Extr. 2026, 8(5), 125; https://doi.org/10.3390/make8050125 - 8 May 2026
Viewed by 1009
Abstract
Artificial intelligence systems deployed in safety-critical and regulated environments require not only predictive performance, but also strict adherence to operational constraints, auditability, and reproducibility. However, in most contemporary architectures, governance is treated as an external or post hoc mechanism, limiting the ability to [...] Read more.
Artificial intelligence systems deployed in safety-critical and regulated environments require not only predictive performance, but also strict adherence to operational constraints, auditability, and reproducibility. However, in most contemporary architectures, governance is treated as an external or post hoc mechanism, limiting the ability to ensure consistent and verifiable decision execution. This paper introduces Controlled Agentic AI Systems (CAIS), a formal architectural framework in which governance is embedded directly into the decision pipeline as a deterministic operator. The proposed formulation integrates a decision model, a constraint specification, and a governance operator that transforms proposed actions into admissible executed actions. The framework further defines audit trace semantics and replayability conditions, enabling deterministic reconstruction of decision trajectories. Theoretical analysis demonstrates that, under standard regularity assumptions, governance can be modeled as a non-expansive projection that enforces constraint-aware decision transformation while inducing bounded decision drift. This provides formal guarantees that governance does not destabilize system dynamics under perturbations. To evaluate these properties, we implement a reference CAIS architecture and conduct controlled experiments in multi-agent and federated simulation environments. The results show that embedding governance significantly reduces the frequency and severity of constraint violations across a range of scenarios. Projection-based repair consistently outperforms approval-only strategies, achieving near-complete compliance in structured regimes while maintaining bounded intervention costs. Importantly, governance does not degrade stability or convergence in federated settings and, in some cases, reduces action-level variance induced by distributed training. While strict feasibility cannot be guaranteed in all practical settings due to approximation and solver limitations, the empirical findings confirm that governance acts as a stabilizing transformation that consistently improves compliance without introducing destabilizing effects. The CAIS framework establishes governance as a first-class architectural component of agentic AI systems, providing a unified foundation for designing constraint-aware, auditable, and reproducible decision pipelines in regulated environments. Full article
(This article belongs to the Section Safety, Security, Privacy, and Cyber Resilience)
Show Figures

Figure 1

22 pages, 9152 KB  
Article
Selective Knowledge Reuse and Adaptive Retraining for Efficient Resource Management in Autonomous Networks
by Fatemeh Tabatabaei and Hamzeh Khalili
Mach. Learn. Knowl. Extr. 2026, 8(5), 124; https://doi.org/10.3390/make8050124 - 5 May 2026
Viewed by 627
Abstract
This paper presents a predictive analytics framework for dynamic resource allocation in next-generation networks, with a focus on 5G and 6G scenarios. The proposed approach uses Long Short-Term Memory (LSTM) models to predict traffic patterns and proactively support automated resource management decisions in [...] Read more.
This paper presents a predictive analytics framework for dynamic resource allocation in next-generation networks, with a focus on 5G and 6G scenarios. The proposed approach uses Long Short-Term Memory (LSTM) models to predict traffic patterns and proactively support automated resource management decisions in network environments. Building on our previous work, we introduce a selective knowledge transfer mechanism, termed “cognitive transfer,” which allows for the reuse of relevant components from previously trained models. This method consolidates multiple models into a compact, generalized form and transfers only the most relevant segments to new traffic scenarios, significantly reducing the need for training from scratch, especially when local data is limited. This maintains an initial consolidated (base) model that is adapted to new traffic scenarios by selecting the corresponding segment from the most relevant parts of the model, identified via a scoring metric. To complement this, a decision-based model retraining strategy is integrated to monitor prediction accuracy, triggering updates only when performance degrades to reduce computational overhead. The framework is evaluated using the Abilene network topology and traffic dataset. Results demonstrate that the approach maintains high prediction accuracy while minimizing under-provisioning, which is critical for avoiding packet loss and ensuring service continuity in high-reliability applications. Across four network links, our method reduced the average number of re-trainings by 1.8× under relaxed thresholds compared to full model transfer, while the prediction error increased by a negligible 0.0008. Full article
(This article belongs to the Section Network)
Show Figures

Figure 1

34 pages, 11866 KB  
Article
Boolean Particle Swarm Optimization with 0-Mutation
by Zaharias D. Zaharis, Emmanouil Georgios Nikopolitidis, Pavlos I. Lazaridis, Panagiotis Sarigiannidis and Sotirios K. Goudos
Mach. Learn. Knowl. Extr. 2026, 8(5), 123; https://doi.org/10.3390/make8050123 - 3 May 2026
Viewed by 588
Abstract
First introduced in 1995, the Particle Swarm Optimization (PSO) algorithm offers a reliable and efficient solution to real-valued optimization problems. However, extending it to binary-valued problems proved challenging. This paper proposes a new Boolean version of the PSO technique based on a novel [...] Read more.
First introduced in 1995, the Particle Swarm Optimization (PSO) algorithm offers a reliable and efficient solution to real-valued optimization problems. However, extending it to binary-valued problems proved challenging. This paper proposes a new Boolean version of the PSO technique based on a novel mutation strategy. By employing an innovative mutation mechanism in the velocity bitstring, the method enforces a minimum perturbation level, reduces the risk of premature convergence, and promotes broader global search. Several variations of the algorithm and parameter combinations are evaluated using 47 benchmark functions to derive the best-performing configuration, which is then compared with other population-based methods to demonstrate the effectiveness of the proposed algorithm. Finally, the technique is applied to an antenna array thinning problem for the design of a planar antenna array with certain specifications. Full article
(This article belongs to the Section Learning)
Show Figures

Figure 1

25 pages, 33743 KB  
Article
CTCF: A Three-Level Coarse-to-Fine Cascade for Unsupervised Deformable Medical Image Registration
by Daniil Pasenko and Roman Davydov
Mach. Learn. Knowl. Extr. 2026, 8(5), 122; https://doi.org/10.3390/make8050122 - 2 May 2026
Viewed by 559
Abstract
Deformable medical image registration aims to spatially align anatomical structures across volumetric scans. Recent transformer-based methods achieve high overlap accuracy but often produce deformation fields with topological violations. We propose CTCF, a Cascade Transformer for Coarse-to-Fine registration that wraps a lightweight coarse-and-refined envelope [...] Read more.
Deformable medical image registration aims to spatially align anatomical structures across volumetric scans. Recent transformer-based methods achieve high overlap accuracy but often produce deformation fields with topological violations. We propose CTCF, a Cascade Transformer for Coarse-to-Fine registration that wraps a lightweight coarse-and-refined envelope around a core registration module. Level 1 provides a coarse displacement estimate at quarter resolution, Level 2 performs the main registration via a Swin Transformer encoder with deformable cross-attention and a learned super-resolution decoder, and Level 3 applies error-driven flow refinement at half resolution. The two outer levels add only 3.0% parameter overhead yet improve registration accuracy while maintaining competitive deformation regularity relative to external baselines. The model is trained end-to-end with a composite unsupervised loss combining local normalized cross-correlation, diffusion regularization, inverse-consistency, and Jacobian-based topology preservation. On the OASIS brain MRI benchmark, CTCF achieves the highest Dice score of 0.8208 among the compared unsupervised methods while maintaining competitive SDlogJ, with all Dice improvements statistically significant at p<0.001 by the Wilcoxon signed-rank test. On IXI, CTCF also achieves the best Dice, HD95, SDlogJ, and fold percentage among the compared methods. A five-round ablation study validates each component: cascade decomposition isolates each level’s contribution, and resolution scaling experiments confirm the framework’s scalability, yielding further accuracy gains with zero parameter overhead. Full article
Show Figures

Graphical abstract

32 pages, 2468 KB  
Review
A Survey on Self-Supervised Learning in Cybersecurity: Network Intrusion and Malware Detection
by Josue Genaro Almaraz-Rivera, Jose Antonio Cantoral-Ceballos and Juan Felipe Botero
Mach. Learn. Knowl. Extr. 2026, 8(5), 121; https://doi.org/10.3390/make8050121 - 1 May 2026
Viewed by 695
Abstract
Self-Supervised Learning (S-SL) is a recent line of research that could represent the next step to understanding human intuition. By blending the strengths of unsupervised and supervised learning paradigms, S-SL endows Deep Learning models with stronger generalization capabilities. Although better known for its [...] Read more.
Self-Supervised Learning (S-SL) is a recent line of research that could represent the next step to understanding human intuition. By blending the strengths of unsupervised and supervised learning paradigms, S-SL endows Deep Learning models with stronger generalization capabilities. Although better known for its applications in Computer Vision and Natural Language Processing, S-SL has also proved its value in other fields, such as cybersecurity. In this work, we review the current progress and future trends in S-SL for the two most relevant problems discussed in the cybersecurity literature: network intrusion and malware detection. The scope of this survey spans from 2019 to 2025. From an initial analysis of over 200 documents, we distill the 50 most relevant papers. We also highlight opportunity areas, such as attack detection over encrypted network traffic, RAM-based analysis of obfuscated malware, creating S-SL models for tabular data and resource-constrained devices, as well as the research on backdooring, encoder extraction, the transferability of vulnerabilities, and data memorization in S-SL. To the best of our knowledge, this is the first comprehensive survey regarding the application of Self-Supervised Learning in cybersecurity, benchmarking contrastive learning vs auxiliary pretext tasks and presenting the data requirements for implementing S-SL solutions in this field. We hope this paper provides a firm ground for further exploration. Full article
(This article belongs to the Section Thematic Reviews)
Show Figures

Figure 1

33 pages, 4403 KB  
Article
Individual Indicators of the Learning Process for Identifying Critical Thinking in Students in Adaptive Learning
by Vassiliy Serbin, Mateus Mendes, Aray Kassenkhan, Akbayan Bekarystankyzy, Gulnur Ibragim and Azamat Tolegenov
Mach. Learn. Knowl. Extr. 2026, 8(5), 120; https://doi.org/10.3390/make8050120 - 1 May 2026
Viewed by 952
Abstract
The rapid digitalization of higher education has intensified the need for reliable methods to assess higher-order cognitive skills, particularly critical thinking, in adaptive learning environments. However, most existing assessment approaches rely primarily on test outcomes and academic performance indicators, which do not adequately [...] Read more.
The rapid digitalization of higher education has intensified the need for reliable methods to assess higher-order cognitive skills, particularly critical thinking, in adaptive learning environments. However, most existing assessment approaches rely primarily on test outcomes and academic performance indicators, which do not adequately capture the multidimensional and process-based nature of critical thinking. This study proposes a multi-criteria hierarchical model for identifying and quantitatively assessing students’ critical thinking based on individual process indicators of learning activity in an intelligent educational environment. The model integrates cognitive, metacognitive, and behavioral indicators, including knowledge dynamics, task complexity, time characteristics, learning activity intensity, error rate, level of doubt, user interaction patterns, and system operating modes. These indicators are aggregated into a three-component structure representing metacognitive awareness, analytical depth, and strategic learning activity. The proposed model was empirically validated through a quasi-experimental longitudinal study involving 500 university students divided into control and experimental groups. The results demonstrate a statistically significant increase in all latent components of critical thinking and in the integral indicator within the experimental group. The model shows satisfactory internal consistency (Cronbach’s α0.77) and acceptable construct validity confirmed by confirmatory factor analysis. The findings indicate that the proposed model can serve as a practical analytical tool for monitoring critical thinking development and supporting personalized learning trajectories in adaptive digital educational systems. Full article
(This article belongs to the Section Learning)
Show Figures

Graphical abstract

26 pages, 20102 KB  
Article
Morphology-Aware Multi-Scale Deep Representation Learning for Interpretable Knowledge Extraction in Brain Tumor MRI
by Helala AlShehri and Mariam Busaleh
Mach. Learn. Knowl. Extr. 2026, 8(5), 119; https://doi.org/10.3390/make8050119 - 1 May 2026
Viewed by 528
Abstract
Robust brain tumor classification from magnetic resonance imaging (MRI) remains challenging due to complex structural heterogeneity and subtle inter-class variability. Beyond predictive accuracy, conventional convolutional neural networks predominantly rely on texture-dominant features and fixed receptive fields, which may limit the extraction of clinically [...] Read more.
Robust brain tumor classification from magnetic resonance imaging (MRI) remains challenging due to complex structural heterogeneity and subtle inter-class variability. Beyond predictive accuracy, conventional convolutional neural networks predominantly rely on texture-dominant features and fixed receptive fields, which may limit the extraction of clinically meaningful structural information. This study proposes a morphology-aware multi-scale deep representation learning framework that embeds morphological inductive bias directly within hierarchical feature extraction. The proposed architecture synergistically integrates trainable morphological operations with multi-scale convolutional feature learning inside a unified residual framework, supported by an in-block morphological refinement mechanism and a morphology-aware downsampling module. Unlike prior approaches that treat morphological operators as preprocessing or auxiliary branches, the proposed design incorporates differentiable dilation and erosion into the core feature hierarchy to guide structure-aware representation formation. The model was evaluated using five-fold cross-validation and an independent test set, achieving an overall test accuracy of 99.31% with consistently high macro-averaged precision, recall, F1-score, and AUC values. Grad-CAM analysis further demonstrates that the learned representations emphasize clinically relevant tumor regions, supporting interpretable structural knowledge extraction. Ablation studies confirm that performance improvements arise from the synergistic integration of multi-scale learning and morphology-aware refinement. Overall, embedding structural inductive bias within multi-scale deep representation learning enhances robustness, stability, and interpretable knowledge extraction for brain tumor MRI analysis. Full article
(This article belongs to the Section Learning)
Show Figures

Figure 1

32 pages, 2549 KB  
Article
Supervised Machine Learning for Technical Debt in Python: Analysis and Prediction
by Elif Fırıncı and Mohanad Alayedi
Mach. Learn. Knowl. Extr. 2026, 8(5), 118; https://doi.org/10.3390/make8050118 - 1 May 2026
Viewed by 708
Abstract
The appearance of technical debt (TD) becomes a critical problem, posing challenges related to software maintainability and its quality within the context of fast modern software development. The presented research focuses on the issue of TD appearance in the context of Python software [...] Read more.
The appearance of technical debt (TD) becomes a critical problem, posing challenges related to software maintainability and its quality within the context of fast modern software development. The presented research focuses on the issue of TD appearance in the context of Python software development by employing a hybrid approach involving perception and predictive approaches. Within the scope of research, the perceptions of 86 IT practitioners and developers have been studied with regard to their reactions, adaptation, and prioritization of different types of TDs. According to the qualitative results, cyclic architectural debt stems from low test coverage, documentation deficiency, and complicated code structure. Based on the aforementioned information, the research team developed a dataset consisting of 130 Python codes in real conditions with the following characteristics: code complexity, comments-to-code ratio, code smells, and software maintainability indexes being used. Thereafter, the application of DT, LR, NB, SVM, KNN, and RF predictive models allowed detecting TDs. The presented results reveal the possibility of predicting TDs with the use of machine learning methods, with optimal performance provided by random forest and optimized logistic regression models. Full article
(This article belongs to the Section Data)
Show Figures

Figure 1

19 pages, 3273 KB  
Article
COMPAS: Compose Actions and Slots in Object-Centric World Models
by Vitaliy Vorobyov, Leonid Ugadiarov, Vladimir Frolov, Alexey Kovalev and Aleksandr Panov
Mach. Learn. Knowl. Extr. 2026, 8(5), 117; https://doi.org/10.3390/make8050117 - 29 Apr 2026
Viewed by 846
Abstract
In this paper, we propose a novel approach, COMPAS (COMPose Actions and Slots), which leverages the strengths of state-of-the-art object-centric approaches for modeling the dynamics of an environment. Our method encodes the environment’s state into symbol-like, object-centric representations, known [...] Read more.
In this paper, we propose a novel approach, COMPAS (COMPose Actions and Slots), which leverages the strengths of state-of-the-art object-centric approaches for modeling the dynamics of an environment. Our method encodes the environment’s state into symbol-like, object-centric representations, known as slots, where each slot corresponds to an individual object. This approach offers a structured and interpretable way to model complex environments by combining slots with action representations for accurate next-state prediction. The primary contribution of our work is an efficient world model with a dynamics predictor capable of predicting accurate trajectories in action-dependent environments. Additionally, our slot extractor module enhances the predictive capabilities by extracting deterministic slots that remain consistent both within a single trajectory and across episodes. Unlike slots sampled from a trainable distribution, deterministic slots are generated from a single trainable parameter together with slot positional embeddings. This design improves the consistency across episodes, which in turn leads to more accurate dynamics prediction. We present a comprehensive evaluation of our approach in various environments, demonstrating that our proposed method outperforms competing models in environments with discrete and continuous action spaces. Full article
(This article belongs to the Section Learning)
Show Figures

Figure 1

44 pages, 2726 KB  
Article
A Tiny Vision-Based Model for Real-Time Student Attention Detection in Online Classes
by Chaymae Yahyati, Ismail Lamaakal, Yassine Maleh, Khalid El Makkaoui and Ibrahim Ouahbi
Mach. Learn. Knowl. Extr. 2026, 8(5), 116; https://doi.org/10.3390/make8050116 - 28 Apr 2026
Cited by 1 | Viewed by 1071
Abstract
Online and blended classrooms widen access but remove the in-person cues instructors use to gauge attention. Prior work typically relies on heavy, cloud-bound or multimodal models that are hard to deploy on commodity laptops, treats attention as an unordered label without calibrated probabilities, [...] Read more.
Online and blended classrooms widen access but remove the in-person cues instructors use to gauge attention. Prior work typically relies on heavy, cloud-bound or multimodal models that are hard to deploy on commodity laptops, treats attention as an unordered label without calibrated probabilities, and evaluates on subject-overlapping splits with limited robustness analysis. This creates a gap in Tiny, deployable, calibration-aware methods validated under realistic protocols. We address this gap with a TinyML, vision-only pipeline that estimates four attention levels: (Very Low, low, high, Very High ) from short webcam clips under strict on-device budgets. Each clip of T=30 frames at 224×224 is processed by a compact hybrid encoder: a CNN extracts per frame spatial features, a BiLSTM models temporal context, and a lightweight GRU refines dynamics; three parallel branches with staggered widths encourage feature diversity before fusion. We apply structured pruning of convolutional channels and recurrent units, post-training INT8 quantization, and temperature scaling for calibrated probabilities; models are exported as ONNX. On DAiSEE with subject-independent splits, the baseline attains 99.86% accuracy and 0.998 macro-F1, with strong ordinal agreement (QWK = 0.998, ordinal MAE = 0.03). The compressed model preserves reliability (macro-F1 = 0.995, QWK = 0.995), remains robust to low light, partial occlusion, and head yaw, and yields ∼4× smaller size and ∼2.3× CPU speedups. These results indicate a deployable, privacy-preserving approach to fine-grained, on-device attention analytics. Full article
Show Figures

Figure 1

22 pages, 3221 KB  
Article
Mapping Dialectal Landscape: A Sequence-to-Sequence Approach to Japanese Dialect-to-Standard Normalization
by Kinga Lasek, Michal Ptaszynski, Fumito Masui, Mujahid Khalifah and Juuso Eronen
Mach. Learn. Knowl. Extr. 2026, 8(5), 115; https://doi.org/10.3390/make8050115 - 26 Apr 2026
Viewed by 826
Abstract
Despite the progressing standardization of the Japanese language, regional dialects persist, particularly among older generations, causing communication gaps, which results in problems especially in healthcare and emergency contexts. This study proposes a text-to-text normalization method to convert eight Japanese dialects into standard Japanese [...] Read more.
Despite the progressing standardization of the Japanese language, regional dialects persist, particularly among older generations, causing communication gaps, which results in problems especially in healthcare and emergency contexts. This study proposes a text-to-text normalization method to convert eight Japanese dialects into standard Japanese using a fine-tuned mT5-small architecture. We evaluate the impact of learning rate schedulers, training duration, and data preprocessing on model performance. Our results demonstrate that the CharacTER (Character Translation Edit Rate) metric provides a more accurate evaluation than BLEU, which is practically ill-suited for the unsegmented nature of Japanese text. The optimal configuration minimizes character error rates by aligning input data with natural, unspaced Japanese orthography. Furthermore, we observe a statistically significant correlation between the model’s conversion error rate and the physical distance of the source dialect from Tokyo. This finding suggests that the model’s performance effectively serves as a proxy for measuring linguistic distance between dialectal variations and the standard language. Full article
(This article belongs to the Section Learning)
Show Figures

Figure 1

22 pages, 19401 KB  
Article
Explainable Combined Spatial Representations for ECG Arrhythmia Classification
by Iulia Onică and Iulian B. Ciocoiu
Mach. Learn. Knowl. Extr. 2026, 8(5), 114; https://doi.org/10.3390/make8050114 - 25 Apr 2026
Viewed by 751
Abstract
The paper addresses ECG arrhythmia classification using a novel input fusion strategy that combines spatial representations of ECG time series recordings. Four distinct time series-to-image transformations are considered, namely classical spectrograms, Gramian Angular Field (GAF), Recursive Plot (RP), and the S-Transform (ST). Classification [...] Read more.
The paper addresses ECG arrhythmia classification using a novel input fusion strategy that combines spatial representations of ECG time series recordings. Four distinct time series-to-image transformations are considered, namely classical spectrograms, Gramian Angular Field (GAF), Recursive Plot (RP), and the S-Transform (ST). Classification of combined 2 × 2 images generated from single-lead ECG recordings is performed using both custom and ResNet-50 deep learning architectures. Finally, several distinct explainability algorithms are used to identify the relevant regions in the input images that mainly influence the classification decisions. Experiments performed on the MIT-BIH and Chapman–Shaoxing arrhythmia datasets revealed performance comparable to more sophisticated approaches in terms of accuracy (99%), F1-score (98.6%), and AUC (0.999) values. Full article
(This article belongs to the Section Data)
Show Figures

Figure 1

17 pages, 2716 KB  
Article
DPA-HiVQA: Enhancing Structured Radiology Reporting with Dual-Path Cross-Attention
by Ngoc Tuyen Do, Minh Nguyen Quang and Hai Van Pham
Mach. Learn. Knowl. Extr. 2026, 8(5), 113; https://doi.org/10.3390/make8050113 - 24 Apr 2026
Viewed by 666
Abstract
Structured radiology reporting can improve clinical decision support by standardizing clinical findings into hierarchical formats. However, thousands of questions in structured report templates about clinical findings are prohibitively time-consuming, which can limit clinical adoption. Furthermore, early medical VQA datasets primarily focused on free-text [...] Read more.
Structured radiology reporting can improve clinical decision support by standardizing clinical findings into hierarchical formats. However, thousands of questions in structured report templates about clinical findings are prohibitively time-consuming, which can limit clinical adoption. Furthermore, early medical VQA datasets primarily focused on free-text and independent question–answer pairs while a recent dataset, Rad-ReStruct, introduced a hierarchical VQA, but the accompanying model still relies heavily on flattened embedding representations and single-path text–image fusion mechanisms that inadequately handle complex hierarchical dependencies in responses. In this paper, we propose DPA-HiVQA (Dual-Path Cross-Attention for Hierarchical VQA), addressing these limitations through two key contributions: (1) multi-scale image embedding representing global semantic embeddings with patch-level spatial features from domain-specific BioViL encoder; (2) dual-path cross-attention mechanism enabling simultaneous holistic semantic understanding and fine-grained spatial reasoning. Evaluated on the Rad-ReStruct benchmark, the model substantially outperforms the established benchmark baseline with an overall F1-score and Level 3 F1-score improvement by 21.2% and 31.9%, respectively. The proposed model demonstrates that dual-path cross-attention architectures can effectively connect holistic semantic understanding and fine-grained spatial detail, paving the way for practical AI-assisted structured reporting systems that reduce radiologist burden while maintaining diagnostic accuracy. Full article
Show Figures

Graphical abstract

16 pages, 707 KB  
Article
Scaling Laws in the Tiny Regime: How Small Models Change Their Mistakes
by Mohammed Alnemari, Rizwan Qureshi and Nader Bagherzadeh
Mach. Learn. Knowl. Extr. 2026, 8(5), 112; https://doi.org/10.3390/make8050112 - 24 Apr 2026
Viewed by 1229
Abstract
Neural scaling laws describe how model performance improves as a power law with size, but existing work has focused almost entirely on models above 100 M parameters. The regime below 20 million parameters, where TinyML and edge AI systems operate, remains largely unexamined. [...] Read more.
Neural scaling laws describe how model performance improves as a power law with size, but existing work has focused almost entirely on models above 100 M parameters. The regime below 20 million parameters, where TinyML and edge AI systems operate, remains largely unexamined. We train 90 models spanning 22 K to 19.8 M parameters across two architecture families (a plain ConvNet and MobileNetV2) on CIFAR-100, varying width while holding depth and training protocol fixed. Both architectures follow approximate power laws, with exponents of α=0.156 (ScaleCNN) and α=0.106 (MobileNetV2). However, the power law does not hold uniformly: local exponents decay with scale, and MobileNetV2 saturates at 19.8 M parameters (αlocal=0.006), hitting a data wall. The structure of errors also changes with scale. The Jaccard overlap between error sets of the smallest and largest ScaleCNN models is only 0.35; compression changes which inputs are misclassified, not merely how many. Small models develop a triage strategy, concentrating capacity on easy classes (Gini of per-class accuracy: 0.26 at 22 K params vs. 0.09 at 4.7 M) while effectively abandoning the hardest ones (bottom-5 class accuracy: 10% vs. 53%). The smallest models achieve the lowest ECE values (0.013 vs. peak 0.110 at mid-size), reversing the typical overconfidence–capacity relationship, though this partly reflects a global-mean matching artifact rather than well-calibrated per-bin confidence. On CIFAR-100, aggregate accuracy alone is therefore a misleading basis for edge deployment decisions; validation must happen at the target model size. All findings in this study are based on CIFAR-100 (32 × 32, 100 classes); their generalizability to other datasets, resolutions, and architectures remains to be verified. Full article
Show Figures

Figure 1

Previous Issue
Next Issue
Back to TopTop