Opportunities and Challenges in Explainable Artificial Intelligence (XAI)

Topic Information

Dear Colleagues,

Artificial intelligence has seen a shift in focus toward designing and deploying intelligent systems that are interpretable and explainable with the rise of a new field: explainable artificial intelligence (XAI). This has been echoed in the research literature and the press, attracting scholars worldwide and a lay audience. Initially devoted to the design of post hoc methods for explainability, essentially combining machine and deep learning models with explanations, it is now expanding its boundaries to ante hoc methods for producing self-interpretable models. Along with this, neuro-symbolic approaches for reasoning have been employed in conjunction with machine learning to improve modeling accuracy and precision by incorporating self-explainability and justifiability. Scholars have also shifted the focus onto the structure of explanations, since the ultimate users of interactive technologies are humans, linking artificial intelligence and computer sciences to psychology, human–computer interaction, philosophy, and sociology.

This multi- and interdisciplinary topic brings together academics and scholars from different disciplines, including computer science, psychology, philosophy, and social science, to mention a few, as well as industry practitioners interested in the technical, practical, social, and ethical aspects of the explanation of the models emerging from the discipline of artificial intelligence (AI). In particular, XAI can help to solve some of the problems of AI, as highlighted in the Regulation of the European Parliament and The Council (AI ACT), laying down harmonized rules on artificial intelligence and amending certain union legislative acts.

Explainable artificial intelligence is certainly gaining momentum. Therefore, this Special Issue calls for contributions to this new fascinating area of research, seeking articles that are devoted to the theoretical foundation of XAI, its historical perspectives, and the design of explanations and interactive human-centered intelligent systems with knowledge representation principles and automated learning capabilities, not only for experts but for a lay audience as well. Invited topics include, but are not limited to, the following:

Technical methods for XAI:

• Action influence graphs;
• Agent-based explainable systems;
• Ante hoc approaches for interpretability;
• Argumentative-based approaches for explanations;
• Argumentation theory for explainable AI;
• Attention mechanisms for XAI;
• Automata for explaining recurrent neural network models;
• Auto-encoders and explainability of latent spaces;
• Bayesian modeling for interpretable models;
• Black boxes vs. white boxes;
• Case-based explanations for AI systems;
• Causal inference and explanations;
• Constraint-based explanations;
• Decomposition of neural-network-based models for XAI;
• Deep learning and XAI methods;
• Defeasible reasoning for explainability;
• Evaluation approaches for XAI-based systems;
• Explainable methods for edge computing;
• Expert systems for explainability;
• Explainability and the semantic web;
• Explainability of signal processing methods;
• Finite state machines for enabling explainability;
• Fuzzy systems and logic for explainability;
• Graph neural networks for explainability;
• Hybrid and transparent black box modeling;
• Interpreting and explaining convolutional neural networks;
• Interpretable representational learning;
• Methods for latent space interpretations;
• Model-specific vs. model-agnostic methods for XAI;
• Neuro-symbolic reasoning for XAI;
• Natural language processing for explanations;
• Ontologies and taxonomies for supporting XAI;
• Pruning methods with XAI;
• Post hoc methods for explainability;
• Reinforcement learning for enhancing XAI systems;
• Reasoning under uncertainty for explanation;
• Rule-based XAI systems;
• Robotics and explainability;
• Sample-centric and dataset-centric explanations;
• Self-explainable methods for XAI;
• Sentence embeddings to explainable semantic features;
• Transparent and explainable learning methods;
• User interfaces for explainability;
• Visual methods for representational learning;
• XAI benchmarking;
• XAI methods for neuroimaging and neural signals;
• XAI and reservoir computing.

Ethical considerations for XAI:

• Accountability and responsibility in XAI-based technologies;
• Addressing user-centric requirements for XAI systems;
• Assessment of model accuracy and interpretability trade-off;
• Explainable bias and fairness of XAI-based systems;
• Explainability for discovering, improving, controlling, and justifying;
• Explainability as a prerequisite for responsible AI systems;
• Explainability and data fusion;
• Explainability and responsibility in policy guidelines;
• Explainability pitfalls and dark patterns in XAI;
• Historical foundations of XAI;
• Moral principles and dilemma for XAI-based systems;
• Multimodal XAI approaches;
• Philosophical consideration of synthetic explanations;
• Prevention and detection of deceptive AI explanations;
• Social implications of automatically generated explanations;
• Theoretical foundations of XAI;
• Trust and explainable AI;
• The logic of scientific explanation within AI;
• The epistemic and moral goods expected from explaining AI;
• XAI for fairness checking;
• XAI for time-series-based approaches;
• XAI for transparency and unbiased decision making.

Psychological notions and concepts for XAI:

• Algorithmic transparency and actionability;
• Cognitive approaches and architectures for explanations;
• Cognitive relief in explanations;
• Contrastive nature of explanations;
• Comprehensibility vs. interpretability vs. explainability;
• Counterfactual explanations;
• Designing new explanation styles;
• Explanations for correctability;
• Faithfulness and intelligibility of explanations;
• Interpretability vs. traceability;
• Interestingness and informativeness of explanations;
• Irrelevance of probabilities to explanations;
• Iterative dialog explanations;
• Justification and explanations in AI-based systems;
• Local vs. global interpretability and explainability;
• Methods for assessing the quality of explanations;
• Non-technical explanations in AI-based systems;
• Notions and metrics of/for explainability;
• Persuasiveness and robustness of explanations;
• Psychometrics of human explanations;
• Qualitative approaches for explainability;
• Questionnaires and surveys for explainability;
• Scrutability and diagnosis of XAI methods;
• Soundness and stability of XAI methods;
• Theories of explanation.

Social examinations of XAI:

• Adaptive explainable systems;
• Backward- and forward-looking responsibility forms of XAI;
• Data provenance and explainability;
• Explainability for reputation;
• Epistemic and non-epistemic values for XAI;
• Human-centric explainable AI;
• Person-specific XAI systems;
• Presentation and personalization of AI explanations for target groups;
• Social nature of explanations.

Legal and administrative considerations within XAI:

• Black box model auditing and explanation;
• Explainability in regulatory compliance;
• Human rights for explanations in AI systems;
• Policy-based systems of explanations;
• The potential harm of explainability in AI;
• Trustworthiness of explanations for clinicians and patients;
• XAI methods for model governance;
• XAI in policy development;
• XAI to increase situational awareness and compliance behavior.

Safety and security approaches for XAI:

• Adversarial attack explanations;
• Explanations for risk assessment;
• Explainability of federated learning;
• Explainable IoT malware detection;
• Privacy and agency of explanations;
• XAI for privacy-preserving systems;
• XAI techniques of stealing, attack, and defense;
• XAI for human–AI cooperation;
• XAI and model output confidence estimation.

Applications of XAI-based systems:

• Application of XAI in cognitive computing;
• Dialog systems for enhancing explainability;
• Explainable methods for medical diagnosis;
• Business and marketing applications of XAI;
• Biomedical knowledge discovery and explainability;
• Explainable methods for human–computer interaction;
• Explainability in decision support systems;
• Explainable recommender systems;
• Explainable methods for finance and automatic trading systems;
• Explainability in agricultural AI-based methods;
• Explainability in transportation systems;
• Explainability for unmanned aerial vehicles (UAVs);
• Explainability in brain–computer interface systems;
• Interactive applications for XAI;
• Manufacturing chains and application of XAI systems;
• Models of explanations in criminology, cybersecurity, and defense;
• XAI approaches in Industry 4.0;
• XAI systems for health-care;
• XAI technologies for autonomous driving;
• XAI methods for bioinformatics;
• XAI methods for linguistics and machine translation;
• XAI methods for neuroscience;
• XAI models and applications for IoT;
• XAI methods for XAI for terrestrial, atmospheric, and ocean remote sensing;
• XAI in sustainable finance and climate finance;
• XAI in bio-signal analysis.

Dr. Luca Longo
Dr. Mario Brcic
Dr. Sebastian Lapuschkin
Topic Editors

Keywords

Participating Journals

Journal Name	Impact Factor	CiteScore	Launched Year	First Decision (median)	APC
Applied Sciences applsci	2.5	5.3	2011	18.4 Days	CHF 2400	Submit
Computers computers	2.6	5.4	2012	15.5 Days	CHF 1800	Submit
Entropy entropy	2.1	4.9	1999	22.3 Days	CHF 2600	Submit
Information information	2.4	6.9	2010	16.4 Days	CHF 1600	Submit
Machine Learning and Knowledge Extraction make	4.0	6.3	2019	20.8 Days	CHF 1800	Submit
Systems systems	2.3	2.8	2013	19.6 Days	CHF 2400	Submit

Preprints.org is a multidisciplinary platform offering a preprint service designed to facilitate the early sharing of your research. It supports and empowers your research journey from the very beginning.

MDPI Topics is collaborating with Preprints.org and has established a direct connection between MDPI journals and the platform. Authors are encouraged to take advantage of this opportunity by posting their preprints at Preprints.org prior to publication:

1. Share your research immediately: disseminate your ideas prior to publication and establish priority for your work.
2. Safeguard your intellectual contribution: Protect your ideas with a time-stamped preprint that serves as proof of your research timeline.
3. Boost visibility and impact: Increase the reach and influence of your research by making it accessible to a global audience.
4. Gain early feedback: Receive valuable input and insights from peers before submitting to a journal.
5. Ensure broad indexing: Web of Science (Preprint Citation Index), Google Scholar, Crossref, SHARE, PrePubMed, Scilit and Europe PMC.

Published Papers (4 papers)

Order results

Content type Publication Date

Result details

Normal Extended Compact

Journals

All Applied Sciences Computers Entropy Information MAKE Systems

Show export options expand_more Show export options expand_less

Select all

Export citation of selected articles as:

Plain Text BibTeX BibTeX (without abstracts) Endnote Endnote (without abstracts) Tab-delimited RIS

Error

Oops... you haven't selected anything for export.

Ok

clear

31 pages, 10098 KiB  

Open AccessArticle

CARAG: A Context-Aware Retrieval Framework for Fact Verification, Integrating Local and Global Perspectives of Explainable AI

by Manju Vallayil, Parma Nand, Wei Qi Yan, Héctor Allende-Cid and Thamilini Vamathevan

Appl. Sci. 2025, 15(4), 1970; https://doi.org/10.3390/app15041970 - 13 Feb 2025

Viewed by 896

Abstract

This study introduces an explainable framework for Automated Fact Verification (AFV) systems, integrating a novel Context-Aware Retrieval and Explanation Generation (CARAG) methodology. CARAG enhances evidence retrieval by leveraging thematic embeddings derived from a Subset of Interest (SOI, a focused subset of the fact-verification [...] Read more.

This study introduces an explainable framework for Automated Fact Verification (AFV) systems, integrating a novel Context-Aware Retrieval and Explanation Generation (CARAG) methodology. CARAG enhances evidence retrieval by leveraging thematic embeddings derived from a Subset of Interest (SOI, a focused subset of the fact-verification dataset) to integrate local and global perspectives. The retrieval process combines these thematic embeddings with claim-specific vectors to refine evidence selection. Retrieved evidence is integrated into an explanation-generation pipeline employing a Large Language Model (LLM) in a zero-shot paradigm, ensuring alignment with topic-based thematic contexts. The SOI and its derived thematic embeddings, supported by a visualized SOI graph, provide transparency into the retrieval process and promote explainability in AI by outlining evidence-selection rationale. CARAG is evaluated using FactVer, a novel explanation-focused dataset curated to enhance AFV transparency. Comparative analysis with standard Retrieval-Augmented Generation (RAG) demonstrates CARAG’s effectiveness in generating contextually aligned explanations, underscoring its potential to advance explainable AFV frameworks. Full article

(This article belongs to the Topic Opportunities and Challenges in Explainable Artificial Intelligence (XAI))

►▼ Show Figures

Figure 1

20 pages, 12647 KiB  

Open AccessArticle

Decoding Mental States in Social Cognition: Insights from Explainable Artificial Intelligence on HCP fMRI Data

by José Diogo Marques dos Santos, Luís Paulo Reis and José Paulo Marques dos Santos

Mach. Learn. Knowl. Extr. 2025, 7(1), 17; https://doi.org/10.3390/make7010017 - 13 Feb 2025

Viewed by 1073

Abstract

Artificial neural networks (ANNs) have been used for classification tasks involving functional magnetic resonance imaging (fMRI), though typically focusing only on fractions of the brain in the analysis. Recent work combined shallow neural networks (SNNs) with explainable artificial intelligence (xAI) techniques to extract [...] Read more.

Artificial neural networks (ANNs) have been used for classification tasks involving functional magnetic resonance imaging (fMRI), though typically focusing only on fractions of the brain in the analysis. Recent work combined shallow neural networks (SNNs) with explainable artificial intelligence (xAI) techniques to extract insights into brain processes. While earlier studies validated this approach using motor task fMRI data, the present study applies it to Theory of Mind (ToM) cognitive tasks, using data from the Human Connectome Project’s (HCP) Young Adult database. Cognitive tasks are more challenging due to the brain’s non-linear functions. The HCP multimodal parcellation brain atlas segments the brain, guiding the training, pruning, and retraining of an SNN. Shapley values then explain the retrained network, with results compared to General Linear Model (GLM) analysis for validation. The initial network achieved 88.2% accuracy, dropped to 80.0% after pruning, and recovered to 84.7% post-retraining. SHAP explanations aligned with GLM findings and known ToM-related brain regions. This fMRI analysis successfully addressed a cognitively complex paradigm, demonstrating the potential of explainability techniques for understanding non-linear brain processes. The findings suggest that xAI, and knowledge extraction in particular, is valuable for advancing mental health research and brain state decoding. Full article

(This article belongs to the Topic Opportunities and Challenges in Explainable Artificial Intelligence (XAI))

►▼ Show Figures

Figure 1

28 pages, 4223 KiB  

Open AccessArticle

Analyzing the Impact of Data Augmentation on the Explainability of Deep Learning-Based Medical Image Classification

by Xinyu (Freddie) Liu, Gizem Karagoz and Nirvana Meratnia

Mach. Learn. Knowl. Extr. 2025, 7(1), 1; https://doi.org/10.3390/make7010001 - 25 Dec 2024

Viewed by 1820

Abstract

Deep learning models are widely used for medical image analysis and require large datasets, while sufficient high-quality medical data for training are scarce. Data augmentation has been used to improve the performance of these models. The lack of transparency of complex deep-learning models [...] Read more.

Deep learning models are widely used for medical image analysis and require large datasets, while sufficient high-quality medical data for training are scarce. Data augmentation has been used to improve the performance of these models. The lack of transparency of complex deep-learning models raises ethical and judicial concerns inducing a lack of trust by both medical experts and patients. In this paper, we focus on evaluating the impact of different data augmentation methods on the explainability of deep learning models used for medical image classification. We investigated the performance of different traditional, mixing-based, and search-based data augmentation techniques with DenseNet121 trained on chest X-ray datasets. We evaluated how the explainability of the model through correctness and coherence can be impacted by these data augmentation techniques. Local Interpretable Model-agnostic Explanations (LIME) and SHapley Additive exPlanations (SHAP) methods were used. Sanity checks and overlapping scores were applied to confirm the correctness and coherence of explainability. The results indicate that both LIME and SHAP passed the sanity check regardless of the type of data augmentation method used. Overall, TrivialAugment performs the best on completeness and coherence. Flipping + cropping performs better on coherence using LIME. Generally, the overlapping scores for SHAP were lower than those for LIME, indicating that LIME has a better performance in terms of coherence. Full article

(This article belongs to the Topic Opportunities and Challenges in Explainable Artificial Intelligence (XAI))

►▼ Show Figures

Figure 1

16 pages, 538 KiB  

Open AccessArticle

What ChatGPT Has to Say About Its Topological Structure: The Anyon Hypothesis

by Michel Planat and Marcelo Amaral

Mach. Learn. Knowl. Extr. 2024, 6(4), 2876-2891; https://doi.org/10.3390/make6040137 - 15 Dec 2024

Cited by 1 | Viewed by 1427

Abstract

Large language models (LLMs) achieve remarkable predictive capabilities but remain opaque in their internal reasoning, creating a pressing need for more interpretable artificial intelligence. Here, we propose bridging this explanatory gap by drawing on concepts from topological quantum computing (TQC), specifically the anyonic [...] Read more.

Large language models (LLMs) achieve remarkable predictive capabilities but remain opaque in their internal reasoning, creating a pressing need for more interpretable artificial intelligence. Here, we propose bridging this explanatory gap by drawing on concepts from topological quantum computing (TQC), specifically the anyonic frameworks arising from $SU{\left(2\right)}_k$ theories. Anyons interpolate between fermions and bosons, offering a mathematical language that may illuminate the latent structure and decision-making processes within LLMs. By examining how these topological constructs relate to token interactions and contextual dependencies in neural architectures, we aim to provide a fresh perspective on how meaning and coherence emerge. After eliciting insights from ChatGPT and exploring low-level cases of $SU{\left(2\right)}_k$ models, we argue that the machinery of modular tensor categories and topological phases could inform more transparent, stable, and robust AI systems. This interdisciplinary approach suggests that quantum-theoretic principles may underpin a novel understanding of explainable AI. Full article

(This article belongs to the Topic Opportunities and Challenges in Explainable Artificial Intelligence (XAI))

Show export options expand_more Show export options expand_less

Select all

Export citation of selected articles as:

Plain Text BibTeX BibTeX (without abstracts) Endnote Endnote (without abstracts) Tab-delimited RIS

 

Error

Oops... you haven't selected anything for export.

Ok

clear

Displaying articles 1-4