A Multi-Criteria Decision-Making Approach for the Selection of Explainable AI Methods

Abstract

The growing trend of using artificial intelligence models in many areas increases the need for a proper understanding of their functioning and decision-making. Although these models achieve high predictive accuracy, their lack of transparency poses major obstacles to trust. Explainable artificial intelligence (XAI) has emerged as a key discipline that offers a wide range of methods to explain the decisions of models. Selecting the most appropriate XAI method for a given application is a non-trivial problem that requires careful consideration of the nature of the method and other aspects. This paper proposes a systematic approach to solving this problem using multi-criteria decision-making (MCDM) techniques: ARAS, CODAS, EDAS, MABAC, MARCOS, PROMETHEE II, TOPSIS, VIKOR, WASPAS, and WSM. The resulting score is an aggregation of the results of these methods using Borda Count. We present a framework that integrates objective and subjective criteria for selecting XAI methods. The proposed methodology includes two main phases. In the first phase, methods that meet the specified parameters are filtered, and in the second phase, the most suitable alternative is selected based on the weights using multi-criteria decision-making and sensitivity analysis. Metric weights can be entered directly, using pairwise comparisons, or calculated objectively using the CRITIC method. The framework is demonstrated on concrete use cases where we compare several popular XAI methods on tasks in different domains. The results show that the proposed approach provides a transparent and robust mechanism for objectively selecting the most appropriate XAI method, thereby helping researchers and practitioners make more informed decisions when deploying explainable AI systems. Sensitivity analysis confirmed the robustness of our XAI method selection: LIME dominated 98.5% of tests in the first use case, and Tree SHAP dominated 94.3% in the second.

1. Introduction

In an era of increasingly widespread use of artificial intelligence in practice, it is important to ensure that its use is safe, fair, and non-discriminatory. The danger lies in generating and using results that are not justified, legitimate, or that do not allow for detailed explanations of their behavior [1]. While the very first artificial intelligence systems were easy to interpret, in recent years, more complex models have been on the rise, which are becoming opaque. There is a need to better understand the functioning of such models and their results.

The solution is provided by explainable artificial intelligence (XAI), which offers explanations of model predictions. Data scientists are and will increasingly be confronted with stakeholder demands for explanations beyond the traditional focus on the predictive performance of models.

The terms interpretability and explainability are often used interchangeably by researchers [2]. These concepts are very closely related, and some works identify their differences and distinguish between the two. It is difficult to define explainability and interpretability mathematically. Miller [3] offers a non-mathematical definition of interpretability, where interpretability is the degree to which a human can understand the reason for a decision. Another definition states that interpretability is the degree to which a human can consistently predict the outcome of a model [4]. One of the most popular definitions of interpretability is that of Doshi-Velez and Kim, who in their work [5] define it as “the ability to explain or present to a human in terms that are understandable”. These authors consider interpretability to be a broader concept than explainability. Gilpin et al. [6] describe interpretability as a necessary condition for explainability. According to them, the interpretability of an explanation describes how understandable the explanation is to humans. In our opinion, the clearest definition is that of the authors of [7], according to which interpretability refers to the passive characteristic of a model, which refers to the level at which the model makes sense to humans. This property is also expressed as transparency. Explainability can be considered as an active characteristic of a model, denoting any action or procedure that a model performs with the intention of explaining or detailing its internal workings.

The main advantages of using explainable artificial intelligence methods include solving the “black box” problem of modern complex models (such as deep neural networks), making their predictions understandable, transparent, and trustworthy. Research has so far revealed various goals that can be achieved by creating an explainable model. The primary goal of XAI is to obtain human-interpretable models because domain experts require assistance in solving issues more efficiently, but they also want to receive meaningful output so they can comprehend and trust those solutions. Additionally, researchers use XAI to achieve a variety of goals, such as enhanced justification, control, improvement, and discovery [8]. Other authors have specialized these goals to the following [9]:

• Empower individuals to combat any negative consequences of automated decision-making.
• Help individuals make more informed decisions.
• Detect and prevent security vulnerabilities.
• Integrate algorithms with human values.
• Improve industry standards for the development of AI-based products, thereby increasing consumer and business trust.
• Promote a Right of Explanation policy.

Arrieta et al. [7] identified nine main goals: trustworthiness, causality, portability, informativeness, reliability, fairness, accessibility, interactivity, and privacy awareness. They assigned these goals to their main target group, as the area of explainability is closely linked to users and their desiderata.

As is evident, the primary role of XAI goes beyond the technical interpretation of models; XAI serves as a crucial bridge for ensuring human understanding and trust in increasingly complex automated decision-making systems. The broad set of identified objectives—from improving control and knowledge discovery to ensuring compliance with the right to explanation—clearly demonstrates that XAI is a cornerstone for responsible, ethical, and acceptable AI development. Ultimately, integrating causality, reliability, and fairness into AI through XAI is not just a technical matter but a strategic imperative for its successful and safe deployment in critical domains.

In recent years, a number of methods have been proposed to explain machine learning models and their decisions. Explainability methods differ in many criteria and are used in a wide range of tasks [10]. The main division of methods is offered by a basic taxonomy that focuses on different characteristics. The question is the right choice of method for a specific use case. As the number of methods increases, it becomes increasingly difficult for stakeholders to choose the right explainability method for a specific use scenario. This carries the risk of failing to achieve reliable interpretability of the model [11]. Our solution to this problem is the proposed framework for selecting the right method that takes into account all important aspects.

The structure of this work is designed to systematically move from theoretical foundation to applied solution: Section 2 presents a basic taxonomy of XAI methods, while Section 3 critically analyzes related works in the field of XAI and MCDM methodology, including XAI benchmarks and metrics. Subsequently, Section 4 defines the methods used. These tools are integrated in Section 5, which presents a proposed solution for objective weighting and ranking of XAI methods. The whole work is concluded in Section 6, Discussion, where the key findings, their contribution, and implications for the responsible deployment of artificial intelligence are interpreted.

2. Basic Taxonomy of XAI Methods

Various and significantly different methods have been developed in the field of explainability [2,12,13]. As explainable methods have become more popular, the number of such methods has also increased, making it more difficult to navigate among them. There are different perspectives on dividing the wide range of explainability methods. The basic pillar is the taxonomy described in the following section. This basic taxonomy is shown in Figure 1.

2.1. Intrinsic and Post hoc Methods

The first criterion is the phase in which the explanation is generated. Accordingly, we divide methods into those in which the explanation is generated during the creation of the model (intrinsic) and those in which the explanation is generated later (post hoc) [13]. Simple models that are interpretable due to their uncomplicated structure, such as decision trees or linear models, belong to the intrinsic group. This form of interpretability is also defined as model transparency and describes how the model works [14]. Post hoc explainability refers to explanatory methods that are applied after a model has been trained and include techniques that allow the conversion of an uninterpretable model into an explainable one.

2.2. Model-Specific and Model-Agnostic Methods

Post hoc methods are further divided into model-specific and model-agnostic methods based on whether they are applicable to specific or arbitrary models. The first group of methods is based on the intrinsic properties of a particular type of model, which limits their applicability. Model-agnostic methods are applicable to any machine learning model after it has been trained. These methods analyze the input and output values and do not have access to the inner workings of the models. Typical representatives of model-specific methods include Grad-CAM [15], LRP [16], and DeepLift [17]. On the other hand, there are model-agnostic methods such as LIME [18], SHAP [19], and Anchors [20].

2.3. Local and Global Methods

If a method explains an individual prediction, it is a local method. It usually tries to approximate the behavior of the model around the instance that the user wants to explain in order to gain information about how the model works. Global methods describe the average behavior of the model as a whole and are often expressed as expected values based on the data distribution. Local methods include LIME [18], Saliency maps [21], and IntGrad [22]. Global methods include ICE [23], PDP [24], and ALE [25].

2.4. Methods by Input Data

Along with the previous criteria, several explainability studies [9,26,27] have also mentioned input and output data as essential factors. Explainable methods differ based on the type of input data, such as images, texts, or tabular data. Each of the input modalities requires different procedures for creating an explanation.

2.5. Methods by Explanation Type

The output of explainability methods is explanations, which can have different formats, such as numerical data, textual data, visualizations, rules, or a combination of the aforementioned [28]. Arrieta et al. [7] describe different types of explanations that differ in their characteristics. They mention textual and visual explanations, but other types are explanation by example and explanation by simplification.

3. Related Work

The aim of this section is to analyze the existing research that deals with the key areas of this study: the selection of the right Explainable Artificial Intelligence method and the application of multi-criteria decision-making. A sequential literature review of these two domains will allow us to comprehensively identify the current state of knowledge and, at the same time, point out the gaps. Although there are many works dedicated to the development of new XAI methods, only a few of them deal with the correct selection of XAI according to the criteria. Therefore, this section serves as a basis that justifies the need for the proposed solution that combines the selection of XAI methods with MCDM, thereby providing a transparent and robust framework for the selection of the optimal XAI method.

3.1. Selecting the XAI Method

An interesting approach to developing a methodology for the correct method selection was introduced by Vermeire et al. [11]. In their research, they argue that a methodology is needed to bridge the gap between stakeholder needs and explainability methods. They designed a card that provides a detailed overview of stakeholder needs. To identify these needs, they created a questionnaire designed to capture stakeholder requirements.

Typical categorizations of stakeholders are based on their role in the organization, their experience with machine learning, or a combination of both. Different methods are implemented to collect needs from stakeholders in a particular use case. Most of them use an approach based on information systems research and software development, where user requirements gathering is a well-known and studied problem. Sometimes, data scientists may have difficulty discussing XAI solutions directly with stakeholders, and therefore, they suggest focusing more on what the stakeholder wants to achieve with explainability [29,30]. They try to understand the background, capabilities, and goals of the stakeholder well. In this way, they create textual and/or visual scenarios that describe where and when explainability is needed.

XAI methods differ significantly in terms of the output/explanation and the way in which these explanations are generated. Therefore, they may be more or less suitable for a particular use case and/or stakeholder. There is therefore a risk that explainability will not be used appropriately. This problem has been noted in other works [31,32], but the existing literature lacks a specific methodology. On the other hand, various studies are being produced devoted to the characterization of XAI methods [33]. These allow us to create a generalized view of explainability methods, which can serve as documentation and a means of comparing them. Several experts [34,35] have proposed a framework of XAI method characteristics that can be supplemented for a specific explainability method. Hall et al. [36] proposed an approach that consists of characteristics that are divided into dimensions: efficiency, versatility, limitations, types and categories of XAI methods, explanation properties, and personal considerations. Sokol and Flach [34] created a framework with five dimensions of the so-called XAI method requirements. Functional requirements determine whether it is practically possible to use the explanation method for a specific use case. Operational requirements concern the interaction of users with the system. Usability requirements consist of properties that are important for the recipient of the explanations. Security and validation requirements focus on aspects such as privacy, security, and validation of the explanation method. It is clear that the dimensions in both studies not only cover general properties of explainability methods but also, to some extent, the needs of stakeholders. According to the authors of [11], there is a lack of research in the professional literature on how to perform a mapping between explainability needs and the properties of the explanation method.

To address the challenge of choosing the right method, a novel framework, AutoXAI [35], was conceptualized and developed. The primary objective of AutoXAI was to automate these complex tasks, thereby assisting data scientists in selecting the optimal XAI solutions based on their specific context. This context was defined as a set of critical elements, including the dataset characteristics, the machine learning model architecture, and the user’s specific XAI needs and constraints. The design of AutoXAI drew upon two distinct methodological domains. Firstly, it leveraged strategies from context-aware recommender systems [37]. This approach allowed for the integration of the user’s context across three distinct phases: contextual prefiltering (to select an initial candidate subset), contextual modeling (to embed context into the recommendation process), and contextual postfiltering (to adjust the final recommendation). A significant preliminary challenge encountered was the lack of unified formalization for XAI elements, which necessitated a rigorous formal definition of the context elements within the framework itself.

Secondly, AutoXAI adapted optimization and evaluation strategies from the field of automated machine learning (AutoML) [38]. This inspiration was crucial because suggesting a reliable explanation required verifying multiple properties of interest simultaneously, which was achieved by optimizing corresponding XAI evaluation metrics. Given the computational expense inherent in hyperparameter optimization using multiple metrics, AutoML techniques were adopted to implement time-saving strategies for efficient evaluation. This combined approach ensured that the recommended XAI solution was both adapted to the user’s context and validated against desired properties. The advantage of this solution is the optimization of hyperparameters of the methods, but the disadvantage is the small number of implemented methods (only LIME, SHAP, and Protodash).

In response to this methodological gap, eXplego [39] was developed and introduced as an interactive, tree-structured tool designed to streamline the XAI method selection process. It operates as a decision tree toolkit that provides developers and practitioners with guided, interactive assistance. Through a number of useful requirements that users must take into account when choosing XAI techniques, eXplego offers navigation to a variety of XAI techniques. The tool recommends an XAI approach that is in line with the user’s explainability requirements after two to five questions. These questions focused on the desired type of explanation, the scope of the explanation, and the nature of the data/model. The final leaf of the decision tree recommended the XAI method deemed most appropriate for the stated use case. It contains 13 different methods that are designed for tabular data. The advantage of this solution is the descriptions of each method, along with sources referring to articles and software implementations. Beyond mere recommendation, the tool provided detailed supporting information, including simplified descriptions of the method’s functionality, practical examples, key points to consider during implementation, and links to the original academic paper and software resources. The disadvantage is its sole focus on tabular input data and the relatively low number of recommended methods.

In [33], Arya et al. presented the AI Explainability 360 software tool, along with a taxonomy of explainable AI methods. The authors proposed a taxonomy of explanations that helps users navigate different approaches and decide which method is appropriate for their context. They also created a guidance taxonomy tree for better orientation. The project aims to address the growing demands of various stakeholders (regulators, users, developers, domain experts) for explanations of AI system decisions. The tool includes twenty-one different explanation methods and two metrics for evaluating explanations. Supports tabular, text, images, and time series data and has an extensible architecture so that new methods and metrics can be added.

In the study by Nauta et al. [40], the authors systematically reviewed more than 600 scientific papers published between 2014 and 2020 that deal with explainable artificial intelligence methods, 312 of which were original XAI approaches. They developed a filtering tool that serves to sort and select relevant scientific papers according to predefined criteria. These criteria include input data type, model type, method type, explanation type, problem type, and task type. Additional criteria include publication source, year of publication, and text search for title, authors, or abstract. The result is a list of relevant studies that can be exported as a list in JSON format. The tool is useful for exploring different methods, but does not offer any further evaluation.

In summary, the research to date has yielded a rich range of theoretical taxonomies and diverse approaches to the characterization and selection of XAI methods. Although these works have significantly contributed to the transparency of AI models, all of the proposed frameworks for selecting the optimal XAI method have their shortcomings. As a result, the decision-making process for selecting an XAI technique often remains incomplete and lacks the required transparency. This is where multi-criteria decision-making intervenes, which has the potential to fill this gap.

Identical predictions can have conflicting and diverse explanations from many explainable methods [41]. To improve explanation quality, some methods aggregate multiple explanations to create a more robust explanation [42]. In [43], the authors proposed a method combining MCDM and XAI. They proposed a method to aggregate multiple explanation models, aiming to enhance the overall robustness of explanations. To maintain purely rank-based procedures, they developed rank-based versions of existing XAI metrics for complexity, stability, and faithfulness. Once the explanations from the component models were evaluated against these metrics, an MCDM algorithm was used to quantify the performance of each component using scalar weights, which were later combined by a rank aggregation algorithm to form a single explanation. Eight MCDM algorithms were considered: EDAS [44], TOPSIS [45], COPRAS [46], PROMETHEE II [47], ARAS [48], COCOSO [49], CODAS [50], and MABAC [51]. Experiments comparing MCDM and rank aggregation algorithms revealed TOPSIS and the weighted sum method to be the best candidates for this use case. A comprehensive experimental analysis across five datasets demonstrated the technique’s effectiveness in enhancing explanation robustness. The study combined the results of several XAI methods and compared several MCDM methods. However, the result is not the selection of the best method, but the ranking of individual features.

Another study that combines XAI methods and MCDM is [52]. The study deals with the development of a sophisticated clinical decision support system (CDSS) that addresses the critical problem of trust in artificial intelligence models in complex domains such as medicine. The authors designed and implemented the CDSS-EQCM framework, which uses an axiomatic integration of explainable AI and multi-criteria decision-making methods to achieve this goal. Specifically, the system generated transparent explanations of models using the model-agnostic techniques LIME and SHAP, and these models were subsequently evaluated according to a set of multiple criteria. To aggregate this multi-criteria evaluation and determine the final ranking of the models, an MCDM approach with TOPSIS and Borda count [53] methods was used. This approach allowed the system to effectively recommend not only the most accurate, but also the most reliable and understandable model to the end user, thereby increasing transparency and the likelihood of AI adoption in clinical practice. The benefit of the work is on-site manual acceptance testing, but the disadvantage is the small number of compared XAI methods and low variability in the evaluation results.

With its ability to systematically process and objectively weigh multiple conflicting criteria and derive a ranking of alternatives, MCDM offers a robust and transparent framework for selecting the optimal XAI method. The aforementioned studies combine MCDM and XAI methods, but they are either focused on evaluating the results of explanations or include only a small number of methods. We propose a solution that will be suitable for directly selecting the most appropriate XAI method in a given context and can be used to compare a larger number of methods.

3.2. Evaluation of XAI Methods

Evaluating explainable artificial intelligence methods requires defining key features that allow us to assess the quality, reliability, and practical utility of the generated explanations. These features provide a foundation for designing metrics to evaluate the extent to which explanations improve model understanding, increase user trust, and facilitate transparent decision-making.

Nauta et al. [40] proposed a scheme of 12 conceptual features (Co-12) to serve as a comprehensive catalogue of criteria for evaluating the quality of an explanation, going beyond fidelity and robustness. These features provide researchers with specific criteria that should be quantified. The authors divided the Co-12 explanation quality properties into three groups according to their focus:

• Content: Correctness, Completeness, Consistency, Continuity, Contrastivity, Covariate complexity;
• Presentation: Compactness, Composition, Confidence;
• User: Context, Coherence, Controllability.

The study critically analyzes evaluation practices in over 300 papers and reveals that a high percentage (approximately one in three papers) rely exclusively on anecdotal evidence to evaluate XAI methods, i.e., a subjective selection of “good” examples of explanations that seem plausible. The paper calls for a paradigm shift toward objective, quantifiable, and comparable evaluation methods.

A systematic approach to the assessment of explainability was introduced by Doshi-Velez and Kim [5], who presented a multi-level view of the XAI assessment that integrates objective and subjective aspects of explainability. This framework still forms the theoretical basis for most recent XAI evaluations. They divided the assessment into three main categories: application-grounded evaluation, human-grounded evaluation, and functionally grounded evaluation. This division clearly shows that no type of assessment is universally sufficient—to achieve reliable explainability, it is necessary to combine functional metrics with user studies and validate explanations in specific application contexts.

• Application-grounded evaluation: This involves implementing models and testing them on a real-world task by running experiments with end users. The best way to show that a model works is to evaluate it on the task for which it was created.
• Human-grounded evaluation: This level is also tested in practice, the difference being that these experiments are not performed with domain experts, but with laypeople. Since no domain experts are needed, the experiments are cheaper, and it is easier to find more testers.
• Functionally grounded evaluation: This level does not require human evaluation. Instead, it uses a formal definition of explainability as a model of the quality of explanation. This evaluation is most appropriate when we have models that have already been validated, e.g., through human-based experiments.

We can combine application-grounded evaluation and human-grounded evaluation into one group of human-centered evaluations. These evaluations are inherently subjective because the results of the methods depend on the selected sample of people.

Functionally grounded assessment cannot replace human-centered assessment, but it is objective and can help guide the selection of explanations that experts can use in a user study.

3.2.1. Objective XAI Metric

As highlighted in the previous section, part of the responsible deployment of XAI is the use of objective metrics that can independently measure the quality and practical feasibility of explanations. These metrics translate abstract properties into measurable numerical values. The aim of this subsection is to introduce some basic objective metrics.

One of the basic properties is fidelity (also called faithfulness, correctness or truthfulness) [40,54,55,56], which determines the extent to which the explanation faithfully reflects the internal behavior of the model. An explanation with high fidelity should be consistent with the model—that is, if the model changes its output, the explanation should respond appropriately. This property is often considered the most important technical characteristic of XAI methods, as it determines their credibility in terms of correctly representing the decision process.

Another key property is stability (also called robustness, continuity, or sensitivity) [28,57,58], which describes how consistent the explanations are with small changes in the input data. If two similar instances lead to significantly different explanations, the method loses credibility. Stability is therefore essential for the practical deployment of XAI in sensitive areas such as healthcare, finance, or autonomous systems.

Consistency [40] is closely related to stability, but it evaluates whether identical inputs have identical explanations. In practice, this property addresses the extent to which the explanation method is deterministic. Low consistency indicates that the explainer is sensitive to noise, and its outputs may not be trustworthy for the decision maker in practice.

Computation complexity [59] is one of the basic technical metrics for evaluating XAI methods, which assesses the practical feasibility of explainability in real systems. This metric determines how much computational resources (time, memory, number of model calls) are required to generate an explanation for a given input. In the context of explainable AI, it is important because many popular methods are conceptually simple but computationally intensive, which limits their use in systems that require fast response or processing of large data.

A related metric is speed [60], which represents one of the most practically important, but less formalized in the literature, criteria for evaluating methods (XAI). It expresses the time required to generate an explanation. In contrast to computational complexity, which theoretically describes the behavior of an algorithm, the speed metric focuses on empirical performance in a real environment, i.e., how quickly an XAI method can provide an interpretation in practice.

3.2.2. Subjective XAI Metric

On the user side, the so-called human-centered properties are evaluated, such as interpretability, understandability, trustworthiness, usefulness, and user satisfaction. These properties measure whether the explanation is understandable to a human, whether it helps to better understand the behavior of the model, and whether it supports rational decision-making in the context of the task.

Interpretability [61] measures the extent to which the user understands the principles or reasons for the model’s decision. It is assessed, for example, using questionnaires, comprehension tests, or qualitative interviews.

Understandability (Comprehensibility) [13] measures how easily and quickly a user can understand the logic and content of the explanation. Another definition describes it as the capacity of a method to make a model understandable [28].

Trustworthiness [7] determines how the explanation affects the user’s trust in the model. This metric is often measured by a subjective scale or through decision-making experiments, in which it is observed whether the user prefers to rely on the model or on their own judgment.

Usefulness [54] assesses whether the explanation helps the user to make an effective decision or improve performance on a given task. This metric tests the practical value of the explanation in real-world scenarios.

User Satisfaction [62] determines the subjective satisfaction of users with the form and content of the explanation. It can be supplemented with qualitative responses about the clarity or visual appropriateness of the explanation. This is the overall emotional and cognitive acceptance of the explanation.

3.2.3. Benchmarks and Libraries Focused on Evaluating XAI Methods

In recent years, several comprehensive benchmarks and libraries have emerged aimed at systematically evaluating and comparing explainable artificial intelligence methods. Together, these cover a wide range of approaches to XAI evaluation. Their comparison is summarized in

Table 1. Comparison of benchmarks and libraries aimed at evaluating XAI methods.

Benchmark	XAI Methods	Metrics	Data Type	Supported Models	Extensible
M4 [63]	LIME	Faithfulness	IMG	ResNets	Yes
	Integrated Gradient		TXT	MobileNets
	SmoothGrad			VGG
	GradCAM			ViT
	Generic Attribution			MAE-ViT-base
	Bidirectional Explanations			BERTs
				DistilBERT
				ERNIE-2.0-base
				RoBERTa
Quantus [64]	GradientShap	Faithfulness	IMG	NN	Yes
	IntegratedGradients	Robustness	TAB
	DeepLift	Localisation	TS
	DeepLiftShap	Complexity
	InputXGradient	Randomisation (Sensitivity)
	Saliency	Axiomatic
	Feature Ablation
	Deconvolution
	Feature Permutation
	LIME
	Kernel SHAP
	LRP
	Gradient
	Occlusion
	Layer GradCam
	Guided GradCam
	Layer Conductance
	Layer Activation
	Internal Influence
	Layer GradientXActivation
	Control Var. Sobel Filter
	Control Var. Constant
	Control Var. Random Uniform
	Vanilla Gradients
	Gradients Input
	Occlusion Sensitivity
	GradCAM
	SmoothGrad
BEExAI [65]	Feature Ablation	Sensitivity	TAB	Linear Regression	Yes
	LIME	Infidelity		Logistic Regression
	Shapley Value Sampling	Comprehensiveness		Random Forest
	Kernel SHAP	Sufficiency		Decision Tree
	Integrated Gradients	Faithfulness Correlation		Gradient Boosting
	Saliency	AUC-TP		XGBoost
	DeepLift	Monotonicity		Dense Neural Network
	InputXGradient	Complexity
		Sparseness
FUNCXAI-11 [66]	Not defined	Representativeness	TAB	Not defined	Yes
		Structure	IMG
		Selectivity	TXT
		Contrastivity
		Interactivity
		Fidelity
		Faithfulness
		Truthfulness
		Stability
		(Un)certainty
		Speed
XAI UNITS [67]	DeepLIFT	Infidelity	TAB	MLP	Yes
	Shapley Value Sampling	Sensitivity	IMG	CNN
	InputXGradient	MSE	TXT	ViT
	IntegratedGradients	Mask Error		LLM
	LIME	Mask Proportion Image
	Kernel SHAP	Mask Proportion Text
	Feature Ablation
	Gradient SHAP
	DeepLIFT SHAP
	Saliency
	Deconvolution
	Guided Backpropagation
	Guided GradCAM
	Feature Permutation
	Occlusion
Compare-xAI [68]	Exact Shapley Values	Comprehensibility:	TAB	Not defined	Yes
	Kernel SHAP	• Fidelity
	LIME	• Fragility
	MAPLE	• Stability
	Partition	• Simplicity
	Permutation	• Stress tests
	Permutation Partition	Portability
	Saabas	Average execution time
	SAGE
	SHAP Interaction
	Shapley Taylor Interaction
	Tree SHAP
	Tree SHAP Approximation
XAIB [69]	Constant	Model Randomization Check (MRC)	TAB	SVC	Yes
	LIME	Small Noise Check (SNC)		MLP
	SHAP	Label Difference (LD)		KNN
	KNN	Different Methods Agreement (DMA)
		Sparsity (SP)
		Covariate Regularity (CVR)
		Target Discriminativeness (TGD)
		Same Class Check (SCC)
XAI-Bench [70]	LIME	Faithfulness	TAB	Linear Regression	No
	SHAP	Monotonicity		Decision Tree
	MAPLE	ROAR		MLP
	SHAPR	GT-Shapley
	BF-SHAP	Infidelity
	L2X
	BreakDown
OpenXAI [71]	LIME	Faithfulness:	TAB	NN	Yes
	SHAP	• Feature Agreement (FA)		Logistic regression
	Vanilla Gradients	• Rank Agreement (RA)
	InputXGradient	• Sign Agreement (SA)
	SmoothGrad	• Signed Rank Agreement (SRA)
	Integrated Gradients	• Rank Correlation (RC)
		• Pairwise Rank Agreement (PRA)
		• Prediction Gap on Important feature perturbation (PGI)
		• Prediction Gap on Unimportant feature perturbation (PGU)
		Stability:
		• Relative Input Stability (RIS)
		• Relative Representation Stability (RRS)
		• Relative Output Stability (ROS)
		Fairness
Saliency bench [72]	Grad CAM	mean Intersection over Union (mIoU)	IMG	CNN	Yes
	GradCAM++	Pointing Game (PG)		ViT
	Integrated Gradients	Insertion (iAUC)
	InputXGradient	Precision
	Occlusion	Recall
	RISE
OpenHEXAI [73]	LIME	Accuracy	TAB	NN	Partially
	SHAP	F1		Logistic regression
	Vanilla Gradients	AVG Time
	InputXGradient	Over-Reliance
	SmoothGrad	Under-Reliance
	Integrated Gradients	Average Absolute Odds Difference (AAOD)
		Equal Opportunity Difference (EOD)

TAB: tabular, TXT: text, IMG: image, TS: time series, NN: neural network, CNN: convolutional neural network, Vit: vision transformer, LLM: large language model.

. Extensibility may mean the ability to add new models, methods, or metrics.

The M4 benchmark [63] provides a unified framework for comparing faithfulness across different modalities (text, image, tabular data). Quantus [64] is similarly multimodal, providing an open library of over thirty quantitative metrics grouped into six categories: faithfulness, robustness, localization, randomization (sensitivity), complexity, and axiomatic. It also supports a large number of methods from the Captum, Zennit, and tf.explain libraries. The BEExAI benchmark [65] focuses on faithfulness, robustness, and complexity metrics. They leverage 50 popular datasets covering regression and classification tasks to provide a rigorous and repeatable benchmark. Newer approaches, such as FUNCXAI-11 and XAI-Units, extend the assessment to include functional and cognitive dimensions. The functionally grounded benchmark framework developed by the developers of FUNCXAI-11 [66] is hierarchically structured and offers a clear and layered organization of properties (organizing 24 properties into 11 separate property categories). The metrics examined include speed, stability, truthfulness, fidelity, and interactivity. The XAI-Units benchmark [67] is an open benchmark framework designed to objectively and reliably evaluate feature attribution (FA) methods. The main driving force is the fact that several FA techniques frequently yield contradictory significance scores for the same model, and there is no trustworthy “ground truth” to confirm which approach is accurate. By offering a collection of procedurally produced models and synthetic datasets where the internal mechanisms and behavior of the model are understood, XAI-Units tackles this issue in a manner akin to software engineering unit testing. Compare-xAI [68] is a benchmark designed to uniformly quantitatively evaluate and compare different post-hoc methods. It uses a set of functional tests to assess specific aspects of explanation and compresses the results into a single multidimensional score, helping users choose the most appropriate XAI algorithm for their needs. Furthermore, XAIB [69] offers an open and extensible benchmark with an evaluation ontology taken from Co-12, thus contributing to the standardization of XAI evaluation in both academic and industrial contexts. Not all of the 12 properties for each explainer type were addressed during the implementation of metrics; some may be inapplicable, while others have not yet been covered. Correctness, continuity, contrastivity, covariate complexity, compactness, and coherence cases are included. XAI-Bench [70] uses synthetic data with a known ground truth, which allows for efficient and reliable evaluation of the quality of explanations using metrics such as fidelity and monotonicity. OpenXAI [71] is an open source framework designed for systematic, reproducible, and transparent benchmarking of post hoc explainable AI methods, primarily for feature attribution. The library provides a comprehensive collection of real and synthetic datasets, pre-trained models, and quantitative metrics for assessing the faithfulness, stability, and fairness of explanations. Saliency-Bench [72] is specifically designed for visual models. The benchmark includes a standardized and unified evaluation pipeline for assessing faithfulness and alignment of the visual explanation, providing a holistic visual explanation performance assessment. On the other hand, OpenHEXAI [73] represents a human-centered approach that measures factors such as trustworthiness, understandability, and user satisfaction through user experiments, reflecting the growing interest in subjective evaluation of explainability. The authors based their work on OpenXAI [71], and in total, they prepared 16 subjective survey questions. The research by the authors of [74] was also subjectively focused, using a questionnaire to evaluate XAI methods and asking about the metrics of understandability, usefulness, trustworthiness, informativeness, and satisfaction through simple questions and a Likert scale.

Together, these benchmarks and libraries reflect a shift from isolated tests to multidimensional and reproducible evaluation of explainability that combines technical, cognitive, and computational aspects into a unified framework.

4. Methods

4.1. Multi-Criteria Decision-Making

Multi-criteria decision-making, often referred to as multi-criteria decision analysis (MCDA), represents a significant advance in the field of decision-making [75,76]. Instead of relying on a single factor, MCDM immediately considers several different factors—whether these are easily measurable quantitative criteria or qualitative criteria [77]. Finding the optimal solution is therefore inherently dependent on achieving a compromise guided by the explicit preferences of the decision maker. This approach necessitates a clear process for evaluating criteria, which is typically achieved through expert groups providing weighting schemes to reflect the relative importance of each factor in the specific case under study. A wide range of MCDM methods have been developed over recent decades. Each of these methods has its own strengths and weaknesses and differs primarily in several aspects, including the complexity of their algorithms, the specific weighting methodologies they employ, their approach to representing preferences, their capacity to handle uncertain data, and their final data aggregation techniques [78]. The extensive family of MCDM methods is generally divided into two main categories: multi-objective decision-making (MODM) and multi-attribute decision-making (MADM) [79]. Specifically, MODM deals with scenarios where alternatives are not predetermined; instead, it seeks to optimize a set of objective functions subject to constraints, aiming for the most satisfactory and efficient solution where no single objective can be improved without degrading another. In contrast, MADM is applied when a small, predetermined number of alternatives must be evaluated against a predefined set of attributes, which are often challenging to quantify, with the goal of selecting the single best alternative based on comparative analysis and a final compromise. Each of the aforementioned categories contains multiple approaches. For a variety of issues, priority-based, outranking, distance-based, and mixed methods are also used. Every technique has unique properties, and they can be divided into three categories: fuzzy, stochastic, and deterministic, or may be combined. The techniques can be categorized as either single or group decision-making procedures, depending on the number of decision makers.

The practical applications of MCDM span a broad spectrum of fields, demonstrating its versatility and effectiveness in optimizing complex choices, from the evaluation of technology investments [80] to applications in the healthcare [81] and energy [82] industries.

4.1.1. Analytical Hierarchy Process (AHP)

The analytical hierarchy process, conceptualized by Thomas L. Saaty [83,84], stands as a seminal methodology within MCDM, fundamentally designed to structure and solve complex decision problems. At its core, AHP employs hierarchical decomposition, where the overall objective is placed at the top, followed by criteria and sub-criteria, concluding with the available decision alternatives at the lowest level. The methodology mandates a process of pairwise comparison among the elements at any given hierarchy level, assessing their relative importance concerning an element at the immediately superior level. This comparison is quantified using Saaty’s foundational scale of 1–9, where odd values denote discrete levels of preference intensity (1 for equal, 3 for moderately more, 5 for strongly more, 7 for very strongly, and 9 for extremely more importance), and even numbers are utilized for compromise values. The elegance of AHP lies in its use of both ratio scales and verbal assessments, allowing for the effective weighting of both quantifiable and qualitative factors.

Subsequently, the method leverages matrix algebra to compute and aggregate the eigenvectors derived from these comparison matrices. This mathematical procedure culminates in the composite final vector of weight coefficients for all alternatives. The entries within this final vector represent the intrinsic value or relative priority of each alternative relative to the primary goal at the hierarchy’s summit. The decision maker can then utilize this vector, often multiplying it with higher-level weight coefficients in an upward cascading process throughout the hierarchy, to derive the overall weight coefficient for each alternative relative to the ultimate goal. Conventionally, the alternative associated with the maximum final weight coefficient is identified as the optimal choice.

A distinguishing and critical feature of AHP is its capacity to calculate the Inconsistency Index. This metric is generated as the ratio between the decision maker’s derived inconsistency and a corresponding randomly generated index (Random Index). The Inconsistency Index serves as a vital diagnostic tool, providing the decision maker with assurance regarding the internal consistency of their judgments. It is generally accepted that this index should not exceed a threshold of 0.10. While exceeding this limit typically necessitates a re-evaluation of the pairwise comparisons, in certain specific contexts, a non-compliant decision can still be deemed acceptable, although this should be cautiously considered.

4.1.2. Criteria Importance Through Intercriteria Correlation (CRITIC)

The CRITIC [85] method is an objective multi-criteria decision-making method used to determine the weights of individual criteria. Unlike subjective methods (e.g., AHP) that rely on expert judgment, CRITIC derives the weights exclusively from the inherent structure of the data. The key advantage of this method is that it takes into account not only the contrast (variability) of the data within a single criterion (standard deviation), but also the conflict (correlation) between the criteria. A criterion receives a higher weight if it has high variability between alternatives and low correlation with other criteria. This ensures that criteria that provide unique and non-overlapping information receive higher importance in the final decision-making, which makes CRITIC an extremely suitable tool for objectively weighing criteria in complex systems such as the evaluation of XAI methods.

4.1.3. Additive Ratio Assessment (ARAS)

The ARAS method [48] is an MCDM technique that evaluates alternatives based on their relative utility. Its core principle is that an alternative’s overall efficiency is directly proportional to the weighted influence of all criteria used in the assessment. ARAS operates by first normalizing the decision matrix so that all criteria become comparable, eliminating issues arising from differing units or scales. Each normalized value is then multiplied by a criterion weight, ensuring that the relative importance of each metric is fully incorporated into the evaluation.

For every alternative, these weighted normalized values are summed to produce a single complex utility score representing its aggregated performance. The method then introduces an optimal hypothetical alternative, defined as the best possible performance attainable on each criterion. The final utility coefficient for each real alternative is calculated as the ratio between its utility score and that of the optimal solution. These coefficients range between zero and one, with higher values indicating better relative utility and determining the final ranking.

4.1.4. Borda Count

The Borda count method [53] is a classical approach for aggregating rankings of alternatives in multi-criteria decision-making and preference-based systems. Originally developed by Jean-Charles de Borda in the 18th century, the method provides a systematic means of integrating ordinal rankings across multiple criteria or decision-makers into a single, comprehensive ranking.

The fundamental principle of the Borda method involves assigning numerical scores to alternatives based on their relative position within each criterion. Specifically, in a set of $n$ alternatives, the highest-ranked alternative receives $n-1$ points, the second-ranked $n-2$ points, and so forth, down to the lowest-ranked alternative receiving zero points. These scores are subsequently summed across all criteria to produce a total Borda count for each alternative. The alternative with the highest cumulative score is then identified as the most preferred.

A notable advantage of the Borda method is its capacity to capture consensus among criteria or evaluators. By considering the complete ordering rather than only top-ranked preferences, it rewards alternatives that consistently perform well across multiple dimensions, providing a more balanced and representative assessment.

4.1.5. Combinative Distance-Based Assessment (CODAS)

The CODAS method [50] is a modern MCDM approach designed to provide a more robust evaluation of alternatives by combining two distance measures. Its core idea is straightforward: an alternative is considered better if it lies farther from the negative ideal solution (NIS), which represents the worst possible performance on each criterion.

The method begins by identifying the NIS and then measuring how far each alternative is from this point. CODAS uses two complementary distance metrics. The primary one is the Euclidean distance, which captures the overall deviation from the NIS across all criteria. It gives a general sense of how well an alternative performs relative to the worst-case scenario. The second metric is the Taxicab (or Manhattan) distance, which sums absolute deviations and provides more sensitivity to individual criterion differences. This second measure becomes important when alternatives are very close to each other in terms of Euclidean distance.

A threshold value determines whether the Taxicab distance should be used. If the Euclidean distances of two alternatives differ enough, the ranking is based directly on that measure. If not, the Taxicab distance serves as a tie-breaker, ensuring fine-grained discrimination.

4.1.6. Evaluation Based on Distance from Average Solution (EDAS)

The EDAS method [44] evaluates decision alternatives by comparing them to the average performance across all criteria, rather than to ideal or worst-case reference points. This makes it different from methods such as TOPSIS or VIKOR, which rely on ideal and anti-ideal solutions.

The process begins by calculating the average solution (AS), which is simply the mean value of all alternatives for each criterion. Each alternative is then assessed in terms of how far it lies above or below this average. Two distances are computed for every criterion: the positive distance (PD), which reflects how much an alternative exceeds the average in benefit criteria (or falls below it in cost criteria), and the negative distance (ND), which shows the degree to which it performs worse than the average in benefit criteria (or better in cost criteria).

These distances are then weighted and summed, producing aggregated PD and ND values for each alternative. The final appraisal score is obtained by combining these weighted distances, and alternatives are ranked according to this score, with higher values indicating better overall performance.

4.1.7. Multi-Attributive Border Approximation Area Comparison (MABAC)

The MABAC method [51] is an intuitive MCDM approach that evaluates alternatives based on how far they lie from a defined reference boundary. Its central idea is the creation of a border approximation area (BAA), which acts as a threshold separating stronger from weaker performance on each criterion.

The BAA is usually derived from the dataset, often calculated as the geometric mean between the best and worst values for each criterion. This produces a reference point that represents a balanced boundary. With this boundary in place, each criterion is conceptually divided into three zones: the boundary area itself, an upper area where performance is better than the BAA, and a lower area where performance is worse.

Once the BAA is defined, each alternative is evaluated by calculating its distance from this boundary. Distances are positive if an alternative lies in the upper (preferred) zone, negative if it falls in the lower zone, and close to zero if it sits near the boundary. These distances are then weighted according to the importance of each criterion. Summing the weighted distances across all criteria produces a final score for each alternative, with higher scores indicating more consistent performance above the boundary.

4.1.8. Measurement of Alternatives and Ranking According to Compromise Solution (MARCOS)

The MARCOS method [86] is a recent and powerful MCDM approach designed to provide a stable and reliable ranking by evaluating alternatives against both ideal and anti-ideal reference points. This dual-reference framework ensures a comprehensive assessment of each option’s relative performance.

MARCOS begins by adding the ideal solution (IS) and anti-ideal solution (AIS) to the decision matrix. Each alternative is then evaluated relative to these references, producing two utility functions: one measuring proximity to the ideal outcome, and the other measuring distance from the worst outcome. These two measures are combined into a single utility degree, which captures both how close an alternative is to the best solution and how far it is from the worst.

The utility degree is then incorporated into a weighting function that aggregates the alternative’s performance across all criteria. This ensures that the final score reflects both absolute performance and strategic positioning within the decision space. Alternatives are ranked in descending order of their final score, with the highest value indicating the preferred choice.

4.1.9. Preference Ranking Organization Method for Enrichment of Evaluations II (PROMETHEE II)

The PROMETHEE II method [47] is a well-established outranking technique within MCDM that differs fundamentally from utility-based or distance-based models. Instead of computing a single aggregated value for each alternative, it builds the ranking through structured pairwise comparisons guided by preference functions.

For each criterion, the decision-maker selects a function—chosen from several standard types—that translates the difference between two alternatives into a degree of preference ranging from zero (no preference) to one (strong preference). This allows PROMETHEE to incorporate meaningful thresholds, such as when small differences should be ignored or when larger gaps reflect clear superiority.

Using these functions, the method calculates two flows for every alternative. The positive flow measures how much an alternative is preferred over all others across the criteria, while the negative flow measures how much it is dominated by the rest. PROMETHEE II then combines these into a net outranking flow, obtained by subtracting the negative flow from the positive one. This net flow produces a complete ranking: the higher the net flow, the stronger the alternative overall.

4.1.10. The Technique for Order Preference by Similarity to Ideal Solutions (TOPSIS)

The technique for order preference by similarity to ideal solution is a powerful and intuitive MCDM method [45]. The fundamental idea behind TOPSIS is simple: the chosen optimal alternative should be geographically closest to the ideal solution (the “best case”) and simultaneously farthest from the negative-ideal solution (the “worst case”) in a multi-dimensional criterion space.

The process begins by formulating a decision matrix containing M alternatives evaluated against N criteria. This matrix is then subjected to normalization and the application of criteria weights to create a weighted decision matrix. Crucially, TOPSIS identifies two theoretical benchmarks: the ideal solution, which represents the maximum performance across all benefit criteria and minimum performance across all cost criteria, and the negative-ideal solution, which represents the inverse. The method then uses Euclidean distance to calculate the separation measure of each real alternative from both the ideal solution and the negative-ideal solution.

Finally, the relative closeness to the Ideal Solution is calculated for every alternative. The alternative achieving the highest closeness score is designated as the best option. TOPSIS is highly valued for its straightforward, distance-based ranking mechanism, making it easily applicable even in common spreadsheet software. The advantage of this method is robustness and reliability [87].

4.1.11. Višekriterijumska Optimizacija I Kompromisno Rešenje (VIKOR)

The VIKOR method [88] is a well-known MCDM technique designed to identify a compromise solution—one that comes closest to the ideal performance while avoiding excessive deviation on any single criterion. Rather than relying solely on distance to the ideal or anti-ideal solution, VIKOR explicitly balances two competing perspectives: overall group satisfaction and individual regret.

At the core of the method are two measures. The first is the group utility measure (S), which reflects the overall performance of an alternative across all criteria. Lower values indicate better collective performance. The second is the individual regret measure (R), which captures the worst deviation of an alternative from the ideal on any single criterion. A smaller R value means the alternative avoids large individual shortcomings.

These two measures are combined into the Compromise Index (Q), which determines the final ranking. A strategy weight governs how much emphasis is placed on group utility versus individual regret: values near 1 favor the majority’s overall benefit, while values near 0 prioritize minimizing the worst-case outcome. A common default is 0.5, representing a balanced compromise. Alternatives are ranked by increasing Q, with the lowest score indicating the recommended compromise solution.

4.1.12. Weighted Aggregated Sum Product Assessment (WASPAS)

The WASPAS method [89] is a modern MCDM technique that combines the strengths of two classical approaches: the Weighted sum model (WSM) and the weighted product model (WPM). This hybrid structure improves ranking accuracy and reliability compared to using either method alone.

WASPAS evaluates alternatives using two complementary aggregation strategies. The WSM component calculates the additive utility by summing weighted normalized scores across all criteria, offering simplicity and a compensatory view of performance. The WPM component calculates the multiplicative utility by taking the weighted product of normalized scores, which emphasizes sensitivity to poor performance in any single criterion. By combining these approaches, WASPAS balances overall utility with attention to individual weaknesses.

The method generates a single combined score, the Joint Generalized Criterion (J), which merges the WSM and WPM results using a generalization parameter. This parameter controls the relative influence of the additive versus multiplicative components: values near 1 favor the WSM, emphasizing overall utility, while values near 0 favor the WPM, emphasizing sensitivity to individual criteria. Alternatives are then ranked by descending J values, with higher scores representing better options.

4.1.13. Weighted Sum Model (WSM)

The weighted sum model [90] is one of the oldest and most widely used MCDM techniques. Its main advantage lies in its simplicity, as it evaluates alternatives by directly aggregating weighted performance scores across all criteria.

WSM is based on the principle that an alternative’s overall performance can be represented as the sum of its weighted scores. It assumes that all criteria are fully compensatory, meaning that a low score on one criterion can be offset by a high score on another.

The method involves two main steps. First, the raw performance scores are normalized to ensure comparability across criteria with different units and scales. Second, each normalized score is multiplied by its criterion weight, and these weighted values are summed to produce a single utility score for each alternative. Alternatives are then ranked in descending order of this total score, with the highest-scoring option considered the best choice.

5. Proposed Tool for Selection of Explainable AI Methods

The following section will describe the proposed solution for selecting explainability methods in a specific case. The tool and all necessary materials are available on the website https://github.com/m-matejova/XAI_selection (accessed on 17 November 2025). The current XAI field faces a diversity problem, and it is extremely challenging for developers and domain experts to objectively and systematically determine which method provides not only technically accurate, but also contextually relevant and credible explanations that meet the specific requirements of a given use case.

Creating a tool that integrates quantitative data from benchmarks and user studies with user preferences and aggregates them through a robust multi-criteria decision-making method directly addresses this challenge. The tool thus serves as a bridge between the technical complexity of XAI and the practical requirements of the end user, thereby increasing trust, transparency, and efficiency in the deployment of AI systems in critical applications. The tool has two parts that can be used separately. The first part is focused on filtering methods. The second phase is focused on selecting the best method based on multi-criteria decision-making.

5.1. Filtering Methods

The taxonomy of explainable artificial intelligence methods provides a systematic framework that helps to navigate the wide range of existing approaches. By combining criteria—according to the scope of explanation, approach to the model, data type, task type, and output form—it is possible to determine which XAI method or methods are most suitable for a given problem.

As a basic list, we used the work of the authors [66], who compiled a list of 249 post-hoc XAI methods. Of these, we kept only 113 methods (see

Table 2. List of XAI methods.

Method	Year	Portability	Model	Scope	Data Type	Problem	Output
LIME [18]	2016	MA		L	TAB, IMG, TXT	C, R	N, V
SHAP [19]	2017	MA		L, G	TAB, IMG, TXT	C, R	N, V
Shapley values [91]	2014	MA		L	TAB	C, R	N
LRP [16]	2015	MS	DNN	L	IMG, TXT	C	N, V
Saliency Maps [21]	2013	MS	DNN	L	IMG, TXT	C	V
Grad-CAM [15]	2019	MS	CNN	L	IMG	C	V
IntGrad [22]	2017	MS	DNN	L	IMG, TXT	C	N, V
Anchors [20]	2018	MA		L	TAB, TXT	C	RU
DeepLIFT [17]	2017	MS	DNN	L	IMG, TXT	C	N, V
Influence Functions [92]	2017	MA		L	IMG	C	N, V
TCAV [93]	2017	MA		G	IMG	C	N
ICE [23]	2015	MA		G	TAB	C, R	V
DTD [94]	2017	MS	DNN	L	IMG	C	N, V
DeconvNet [95]	2013	MS	CNN	L	IMG	C	N, V
SmoothGrad [96]	2017	MS	DNN	L	IMG	C	N, V
PDP [24]	2001	MA		G	TAB	C, R	V
PatternAttribution [97]	2017	MS	DNN	L	IMG	C	N, V
Guided BackProp [98]	2014	MS	DNN	L	IMG	C	N, V
Meaningful Perturbation [99]	2017	MS	DNN	L	IMG	C	N, V
PatternNet [97]	2017	MS	DNN	L	IMG	C	N, V
Show, Attend and Tell [100]	2015	MS	CNN	L	IMG	C	T, V
Activation Maximization [101]	2010	MS	DNN	L	IMG	C	V
CAM [102]	2015	MS	CNN	L	IMG	C	V
Rationales [103]	2016	MS	NLP model	L	TXT	C	T
LORE [104]	2018	MA		L	TAB	C	RU
PDA [105]	2017	MS	DNN	L	IMG	C	V
RISE [106]	2018	MA		L	IMG	C	N, V
CEM [107]	2018	MA		L	IMG, TAB, TXT, GRH, TS, VID	C	N, T
Guided Proto [108]	2019	MA		L	IMG, TAB	C	V, T
DICE [109]	2020	MA		L	TAB	C	N, T
ACE [110]	2019	MA		G	IMG	C	N, T
ConceptSHAP [111]	2020	MA		G	IMG	C	N, V
Soft DT [112]	2017	MS	Decision Tree	G	IMG	C	RU
ALE [25]	2016	MA		G	TAB	C, R	V
CIU [113]	1995	MA		L, G	IMG, TAB	C, R	N
Regularisation [114]	2015	MS		L	IMG	C	N
GFA [115]	2016	MA		G	TAB	C, R	N, V
GAN Dissection [116]	2018	MS	GAN	L	IMG	C	V
FACE [117]	2019	MA		L	IMG, TAB, TXT, GRH, TS, VID	C	N, T
NAM [118]	2020	MS		L	TAB	C, R	V
RuleMatrix [119]	2018	MA		G	TAB	C	RU, V
Grad-CAM++ [120]	2018	MS	CNN	L	IMG	C	V
L2X [121]	2018	MA		L	IMG, TXT	C	N
MAPLE [122]	2018	MA		L	TAB	C, R	N, RU
PIMP [123]	2010	MA		G	TAB	C	N
BreakDown [124]	2018	MA		L	TAB	C, R	N, T
ABELE [125]	2020	MA		L	IMG	C	N, T
MOC [126]	2020	MA		L	TAB	C, R	N, T
SEDC [127]	2014	MA		L	TXT	C	V, RU
LIVE [124]	2018	MA		L	TAB	C, R	V
GraphLIME [128]	2020	MA		L	GRH	C	N, V
GLocalX [129]	2020	MA		L, G	TAB	C	RU
Privacy-Preserving Explanations [130]	2020	MA		L	TAB	C	N
PI, ICI [131]	2018	MA		G	TAB	C, R	N
QII [132]	2016	MA		G	TAB	C	N
FOCUS [133]	2021	MA		G	TAB	C, R	N
EBCF [134]	2020	MS	Recommender Systems	L	TAB	C	V, T
DCE [135]	2019	MA		L	TAB	C	N, T
Actionable Recourse [136]	2019	MA		L	TAB	C	N, T
DACE [137]	2020	MA		L	TAB	C	N, T
MACE [138]	2020	MA		L	TAB	C, R	N, T
C-CHVAE [139]	2020	MA		L	TAB	C	N, T
SYNTH [140]	2020	MA		L	TAB	C	V
DECE [141]	2020	MA		L, G	TAB	C	N, T
ALG-REC [142]	2020	MA		L	TAB	C	N, T
Growing Spheres [143]	2017	MA		L	TAB	C	N, T
CERTIFAI [144]	2020	MA		L	TAB, IMG, TXT	C, R	N
ViCE [145]	2020	MA		L	TXT	C	V, T
MC-BRP [146]	2020	MA		L	TAB	C, R	RU
LIME-C/SHAP-C [147]	2020	MA		L	TAB	C	N, V
CLEAR [148]	2019	MA		L	TAB	C, R	T, RU
GNNExplainer [149]	2019	MA		L	GRH	C	N, V
Shapley Flow [150]	2020	MA		L, G	TAB	C, R	N, V
FBT [151]	2020	MS	DT Ensambles	G	TAB	C	N
CHIRPS [152]	2020	MS	Random Forest	L	TAB	C	N, V
LoRMIka [153]	2019	MA		L	TAB	C	RU, V
Color-based monogram [154]	2016	MA		G	TAB	C	V
SR map [155]	2019	MS	CNN	L	IMG	C	RU, V
DLIME [156]	2019	MA		L	IMG, TAB, TXT, GRH, TS, VID	C, R	N
LioNets [157]	2019	MS	Intrinsically Interpretable DNN	L	TXT	C	RU
SkopeR	2020	MA		L, G	TAB	C	RU
XRAI [158]	2019	MS	CNN	L	IMG	C	V
XSPELLS [159]	2020	MA		L	TXT	C	RU
exBERT [160]	2019	MS	Transformer model	L	TXT	C	V
Slot Activation Vectors [161]	2018	MS	Slot Attention Model	L	TXT	C	N
Peak Response [162]	2018	MS	CNN	L	IMG	C	V
Autofocus-Layer [163]	2018	MS	CNN	L	IMG	C	N
NNKX [164]	2017	MS	DNN	G	TAB	C	N, V
Hypothesis Testing [165]	2019	MA		L	IMG, TXT	C	N
GAM [166]	2019	MA		G	IMG, TAB	C	V
Important Neurons and Patches [167]	2017	MS	CNN	G	IMG	C	N, V
InterpNET [168]	2017	MA		L	IMG	C	V
ACE [169]	2019	MS	DNN	G	IMG, TAB, TXT, GRH, TS, VID	C	N, T
DoctorXAI [170]	2020	MA		L	TAB	C	N, T
ORDCE [171]	2021	MA		L	TAB	C	N, T
PIECE [172]	2021	MS	CNN	L	IMG	C	V
POLYJUICE [173]	2021	MA		L	TXT	C, R	T
GeCO [174]	2021	MA		L	TAB	C, R	N, T
RF-OCSE [175]	2020	MA		L	TAB	C, R	RU
What-If [176]	2020	MA		G	TAB, IMG, TXT	C, R	V, N
PRINCE [177]	2020	MA		L	GRH	C	V, T
Gradient Boosted CFs [178]	2020	MS	DT Ensembles	L	TAB	C	N, T
Score-CAM [179]	2019	MS	CNN	L	IMG	C	V
ACV [180]	2022	MA		L	TAB	C, R	N
SoundLIME [181]	2017	MA		L	TS	C	N, V
Forest Floor [182]	2016	MS	DT Ensembles	G	TAB	C, R	V
ChemHeatmap [183]	2011	MA		L	GRH	C	V
Cocox [184]	2020	MA		L	IMG	C	N, T
NAFER [185]	2024	MA		L, G	TAB	C	N
Improved LRP [186]	2022	MS	DNN	L	IMG, TXT, GRH	C	N, V
CIE [187]	2021	MA		L	TXT, TAB	C	N
LeGrad [188]	2024	MA		L	IMG	C	V
SyntaxShap [189]	2024	MS	NLP model	L	TAB	C	N, T

MA: model-agnostic, MS: model-specific, L: local, G: global, TAB: tabular, TXT: text, IMG: image, GRH: graph, TS: time series, VID: video, C: classification, R: regression, N: numerical, T: textual, V: visual, RU: rules.

) that have source code for reproducing the explanations. This list was created based on a survey of relevant scientific articles in the field of XAI from 2015 to 2024. The first column is the name of the method, followed by the year of publication and properties according to the basic taxonomy. If the method is model-specific, the model column specifies the type of model. Another important column is the type of problem being solved, whether it is classification or regression. The methods are ranked according to the original article, where they were ranked by popularity, calculated as a percentage of the reviewed surveys in which each method is mentioned. We supplemented this list with a column describing the output format based on further studies [26,40].

As we can see in Figure 2, first of all, the user enters information about the use case for which they are looking for a suitable XAI method/methods. It is known objective information that results from the basic taxonomy of methods. The tool also includes help texts that describe the filter selection options. These texts briefly explain the differences and describe the selection options with specific examples. (For example, “Select Global if you are interested in understanding the overall system behavior—how the model makes decisions across all possible data points.”) By adding help texts, we ensure that users quickly understand the impact and meaning of each choice. Based on this information, methods from the list of 113 methods are filtered. In this context, a user can be a developer, a domain expert, or even a newcomer to the XAI field. The output is a list of matching methods that the user can view and download in CSV format. The filtered set of methods shall constitute the cornerstone for the subsequent MCDM phase.

Ideally, we aim to secure results for all methods included in this refined list. These results can be derived from existing benchmark studies (Table 1), internally conducted experimental analyses, or dedicated user studies. These data must then be uploaded to the MCDM section in a standardized format, specifically as CSV or XLSX files, to construct the decision matrix (Figure 2). The structure of this input file is crucial:

• The first column must strictly contain the names of the XAI methods (serving as the alternatives).
• The subsequent columns must specify the names of the evaluation metrics (serving as the criteria) and the corresponding performance results for each method against these metrics.

Upon successful generation of the decision matrix, the process is prepared to proceed to the second phase, involving the application of MCDM methods for the aggregation and selection of the optimal XAI technique.

5.2. Choosing a Method Using Multi-Criteria Decision-Making

Because it can be difficult or impossible to obtain ground-truth labels and explanations, evaluating an XAI approach is not as easy as analysing the performance of an ML model. The fact that an explanation’s quality is somewhat subjective, depending on how each user interprets it, makes it much more difficult. No XAI method is universally the best.

The use of multi-criteria decision-making methods for selecting a suitable explainable artificial intelligence method represents a significant advance towards the systematization of the decision-making process, which is often complex, subjective and dependent on multiple, interacting factors. The selection of the optimal XAI method rarely depends on just one property, such as accuracy or robustness, but requires taking into account multiple dimensions.

MCDM methods allow for the quantification and weighting of individual criteria according to their importance, thus providing a transparent and reproducible framework for comparing alternative XAI methods. The main advantage of their use is that they can integrate both objective (e.g., fidelity, stability, speed) and subjective metrics (e.g., interpretability, trustworthiness, usability) into a single decision-making model. This overcomes the problem where individual methods excel only in partial areas, but there is no unified way of evaluating them overall.

Another advantage of MCDM approaches is their flexibility and adaptability. Researchers can adapt the weights of the criteria to a specific context—for example, in medicine, the priority may be the understandability and credibility of the explanation, while in industrial applications the speed of calculation dominates. In addition, MCDM techniques support sensitivity analysis, which allows examining how changing the weights will affect the choice of method, which significantly increases the transparency of the decision.

Currently, however, there is no perfect metric or benchmark that compares all existing methods. It is therefore essential that the user uploads a list of methods together with their results from the selected metrics. Next, the user determines the weights of individual metrics or criteria (see Figure 2). There are three options to choose from:

• Direct Rating—The first option is to enter the criteria weights directly for each of them as a number on a scale from 1 (unimportant) to 10 (very important). This option is suitable when the user has a clear idea of the importance of the metrics.
• Pairwise comparison—The second option is a pairwise comparison of metrics (based on AHP). This method is not suitable for a higher number of metrics, because in that case, it is necessary to make many comparisons, which can be cognitively demanding and confusing. The recommended number is a maximum of 9 [83], ideally 3 to 7.
• CRITIC—The third option is the use of the CRITIC method, which serves to objectively determine weights based on variability and relationships between criteria. This method is also suitable for a larger number of metrics.

The metric weights are used as input to the Decision matrix (see Figure 2), which is the input for calculating preferences using MCDM methods: ARAS, CODAS, EDAS, MABAC, MARCOS, PROMETHEE II, TOPSIS, VIKOR, WASPAS, and WSM. The individual preferences/rankings are aggregated using the Borda count method. This process generates a final, comprehensive list of XAI methods, ordered from best to worst based on their cumulative Borda count.

5.3. Sensitivity Analysis

To assess the impact of criterion weights on the final ranking of alternatives, we executed a comprehensive sensitivity analysis. The methodology employed for this procedure draws inspiration from the approach detailed in the study [190]. Sensitivity analysis is also part of the proposed tool.

We generated a total of twenty weight variations for each MCDM method utilized. The weights were systematically varied by adding a value, $\delta$, which ranged between −0.40 and +0.40. The new weight of the criterion $W_i^{\ast }$ is computed as defined below (Equation (1)). Here, $W_i$ refers to the weight used originally in MCDM methods, $\delta$ is as defined above, and $n$ is the number of decision criteria (metrics).

$$W_i^{\ast }=\left|W_i+\delta \right|, for 1\le i\le n$$

(1)

Following this adjustment, the modified weights must be normalized (using Equation (2)) to ensure that the sum of the new weights collectively equals one.

$$W_i^{\prime }={\displaystyle \frac{W_i^{\ast }}{\sum W_i^{\ast }}}, for 1 \le i\le n$$

(2)

Across the twenty experiments performed for each MCDM method (one for each value $\delta$), we recalculated the preferences and derived the new ranking of alternatives. This extensive re-calculation allowed us to thoroughly examine how alterations in the criteria’s weights influence the final ranking.

5.4. Use Cases

The following section describes in detail a practical workflow that has been designed for users of our tool to systematically, objectively, and transparently select the most appropriate explainable artificial intelligence method.

1.

Define Context, Metrics, and XAI Methods

The user must first define the problem space, as this determines the necessary trade-offs.

• Identify XAI candidates: Filter the set of XAI methods to be evaluated (e.g., the 3–5 methods relevant to the user’s specific ML model).
• Define evaluation metrics: Determine the key performance indicators that the XAI method must satisfy. These should include standard measures and domain-specific needs.
• Determine metric type: Classify each metric as either a Benefit (higher value is better) or a Cost (lower value is better).
• Input performance data: The user must run each candidate XAI method and input the performance data into the tool.

2.

Establish Weights

Metric weights can be obtained in several ways. The user must choose one of the options: direct rating, pairwise comparison (AHP), or CRITIC.

3.

Obtaining the Borda count based on MCDM methods results

The tool automatically processes the decision matrix using a suite of diverse MCDM techniques to ensure the quality of the solution.

• Initial ranking: Review the initial preference scores and the resulting ranking from each method.
• Borda count: The Borda count provides a balanced overall rank for each XAI method.

4.

Validate Robustness (Sensitivity Analysis)

This is the most critical step for ensuring the chosen XAI method is stable and defensible.

• Sensitivity tests: Observe how the resulting order of methods changes when changing the weights by a factor of $\delta$.
• Analyze rank stability: Review the table that shows how often each XAI method retained Rank 1 across all weight perturbations and MCDM methods.
• Review rank change plots: Examine the visual plots to identify the following:

  ○

  Stable Leaders: Lines that stay consistently high.

  ○

  Crossover Points: Areas where lines intersect, indicating high instability and a change in the rank when the weights shift marginally.

5.

Final Selection and Documentation

The user makes the final choice based on both initial performance and validated stability.

• Document Decision: Use the generated MCDM ranking tables and the sensitivity analysis plots as rigorous evidence to document the decision, thereby providing a clear audit trail for compliance and quality control.

To demonstrate the practical use of the proposed solution, we will describe its application in three different use cases. The presented cases demonstrate the flexibility of the proposed approach, which confirms its broad applicability across various domains and tasks. Crucially, the method effectively integrates both objective and subjective metrics and an approach to their weighting.

5.4.1. Choosing the XAI Method in the Field of Hate Speech Detection

As a practical demonstration of the use of the proposed framework for selecting a suitable XAI method, we used the results of an experiment we conducted in the field of hate speech content detection on social networks. The goal was to compare the quality of explanations generated by three popular XAI methods—LIME, SHAP, and Grad-CAM—applied to the ResNet-50 model trained for the image modality on the Hateful Memes dataset [191]. The model classified images containing potentially hateful content, while the explanations of individual methods were subsequently assessed in terms of their comprehensibility and practical value for the user.

The subjective metrics understandability, usefulness, trustworthiness, informativeness, and satisfaction were used to assess the quality of the explanations [74]. The evaluation was carried out using questionnaires and a Likert scale (1–7), where 1 represented the lowest and 7 the highest level of agreement with the explanation statement.

The experiment involved 70 respondents aged 18–22 with varying levels of knowledge in the field of artificial intelligence and visual interpretation. Participants were shown the outputs of all three XAI methods in sequence, and after each viewing, they filled out a short questionnaire evaluating the above metrics.

The questionnaire was structured into distinct sections, with each dedicated to a specific explainability method. To ensure participant comprehension, every section began with accompanying textual instruction and an example explanation. This setup clarified the method’s purpose and established the context for its application. Importantly, all sections utilized the identical set of evaluative statements (metrics noted parenthetically):

• The explanation contains information that is essential for me to understand the model’s decision. (Understandability)
• The explanation is useful to me for making better decisions or performing an action. (Usefulness)
• Based on the explanation, I have more confidence in the model’s decision. (Trustworthiness)
• The explanation provides sufficient information to explain how the system makes decisions. (Informativeness)
• I have a satisfied attitude towards the explanation of the model. (Satisfaction)

Users answered to what extent they agreed with the given statements on a scale from 1 to 7. From the responses, we calculated metrics as the average score of the responses.

Imagine a team of researchers working on a project that aims to automatically detect hate speech on a social network—images that spread insults or incite violence. The model they use is a modern ResNet-50 neural network, capable of accurately recognizing visual elements, but its decision-making remains a “black box” for the average user. When the system labels an image as hateful, the social network’s moderators ask themselves: Why this one? What did the model see in it? How do we explain it to users?

To get answers, the team used our tool to filter which methods were suitable for use in this task (model-agnostic, local, image input data, classification, visual output) and selected three popular XAI methods—LIME, SHAP, and Grad-CAM. Each offered a different perspective on how the model thinks. However, they can only deploy one of them.

To verify which method is most understandable and trustworthy for people, the researchers used the research results shown in

Table 3. Average results for subjective metrics of evaluation of XAI methods in the field of hate speech obtained from questionnaires based on a Likert scale (1–7).

Method	Understandability	Usefulness	Trustworthiness	Informativeness	Satisfaction
LIME	5.46	5.41	4.99	5.33	5.47
SHAP	5.46	5.39	4.97	5.23	5.60
Grad-CAM	4.83	4.76	4.49	4.80	4.81

.

The ratings thus obtained can then be processed in the proposed decision tool, which allows determining the most appropriate method. After loading the survey results, it is necessary to determine the weights of the individual criteria. With 5 metrics, the team chose the pairwise comparison method, which allows them to compare individual pairs of metrics against each other and thus obtain their weights. The team of researchers created the following Saaty’s matrix by comparing the importance of the metrics (

Table 4. Saaty’s matrix created by researchers. Inconsistency index is 0.068, which is acceptable.

	Un	Us	Tr	I	S
Un	1	3	1/3	3	3
Us	1/3	1	1/3	3	3
Tr	3	3	1	7	5
I	1/3	1/3	1/7	1	1/3
S	1/3	1/3	1/5	3	1

Un: understandability, Us: usefulness, Tr: trustworthiness, I: informativeness, S: satisfaction.

):

Trustworthiness (weight calculated based on Saaty matrix 0.476) is of the utmost importance in this area, as users need to trust that the explanation is not misleading. Model failure or bias can have serious social or legal consequences. If the system can transparently and accurately explain why certain content has been removed, it increases trust in the platform. Understandability (0.234) and usefulness (0.151) are of moderate importance—people want to understand the output and be able to use it. Informativeness (0.053) and satisfaction (0.088) are complementary but less crucial.

After applying all MCDM methods, the research team obtained preferential results (

Table 5. Preferences of LIME, SHAP, and Grad-CAM according to all MCDM methods for the first use case.

Method	LIME	SHAP	Grad-CAM
ARAS	0.9979	0.9966	0.8897
CODAS	0.1684	0.1629	−0.3313
EDAS	1	0.9811	0
MABAC	0.4111	0.3919	−0.5745
MARCOS	0.7033	0.7023	0.627
PROMETHEE II	0.7956	0.2044	−1
TOPSIS	0.9748	0.9611	0
VIKOR	0	0.0147	1
WASPAS	0.998	0.9965	0.8897
WSM	0.346	0.3455	0.3085

) and the Borda count. The LIME method obtained a score of 20, the SHAP method 10, and the Grad-CAM method 0. The team therefore decided to implement the LIME method. This use case demonstrates the possible combination of subjective metrics and pairwise weighting.

The ranking for XAI methods across all MCDM methods is shown in Figure 3. The entire ranking is constant.

5.4.2. Choosing the XAI Method in the Field of Medicine

Imagine a young doctor who works in a diabetes clinic at a university hospital. Every day, she sees dozens of patients who are suspected of having diabetes. In recent years, the hospital has begun experimenting with artificial intelligence systems that can predict the risk of developing the disease based on various data. A model based on a Random Forest showed very promising results, the ability to identify hidden patterns in data that the human eye cannot see.

Nevertheless, the doctor had doubts. If the AI recommends that a patient is at high risk of diabetes, what does it base this conclusion on? What factors were decisive? Was it glucose levels, age, or perhaps BMI? In order for the doctor to trust the decision, it was necessary to understand why the model produced a particular prediction.

In her research, she came across a study [66] that compares 3 methods to explain a Random Forest in a similar problem, but she could not determine which of them would be the best choice for her. Therefore, she will use the proposed framework, where she will load the results (

Table 6. Results of 11 metrics for LIME, Kernel SHAP, and Tree SHAP methods [66].

Method	Re	St	Se	Co	In	Fi	Fa	Tr	Sta	Unc	Sp
LIME	11	7.7	1	1.1	3	0.7	1.4	2	0.4	2	3
Kernel SHAP	10	6	1	1.2	3	0.6	2.5	2	1.2	2	2
Tree SHAP	10	7.3	1	1.2	3	2	2.5	3	1.2	2	4

Re: representativeness, St: structure, Se: selectivity, Co: contrastivity, In: interactivity, Fi: fidelity, Fa: faithfulness, Tr: truthfulness, Sta: stability, Unc: uncertainty, Sp: speed.

) from the study.

In the next step, it is necessary to determine the weights for each of the metrics. Due to a lack of experience and a large number of metrics, the doctor decides to use an objective calculation of weights using the CRITIC metric. Since this method calculates variability and some criteria had zero variability (standard deviation 0), the result is their weight of 0 (e.g., selectivity, interactivity, uncertainty). This means that these criteria do not contribute to distinguishing between alternatives in the given matrix. The calculated weights are given in

Table 7. Weights of criteria according to the CRITIC method.

Criterion	Weight
Representativeness	0.2266
Structure	0.1397
Selectivity	0.0000
Contrastivity	0.1267
Interactivity	0.0000
Fidelity	0.0825
Faithfulness	0.1267
Truthfulness	0.0849
Stability	0.1267
Uncertainty	0.0000
Speed	0.0862

.

Using the proposed tool, the physician obtained the following Borda count for the methods: LIME 5, Kernel SHAP 6, and Tree SHAP 19. The preference values are given in

Table 8. Preferences of LIME, Kernel SHAP, and Tree SHAP according to all MCDM methods.

Method	LIME	Kernel SHAP	Tree SHAP
ARAS	0.7251	0.7855	0.9674
CODAS	−0.4043	−0.0843	0.4886
EDAS	0.0989	0.3129	0.9726
MABAC	−0.0239	−0.0591	0.3013
MARCOS	0.5928	0.6391	0.767
PROMETHEE II	−0.0563	−0.274	0.3303
TOPSIS	0.5178	0.4194	0.5542
VIKOR	0.4512	1	0.5
WASPAS	0.7184	0.7838	0.9644
WSM	0.2908	0.3154	0.3938

. Figure 4 gives us a look at the ranking of XAI methods in the MCDM results, where we can see how the ranking varied. Three SHAP maintained the first place among the nine methods. The doctor decided to implement Tree SHAP to help explain the prognosis to patients. This use case demonstrates the possible combination of objective metrics and weighting.

5.4.3. Choosing the XAI Method in Finance

The authors of the OpenHEXAI benchmark [73] focused their research on creating a framework for human-centered evaluation of XAI methods. They used LIME, SHAP, SmoothGrad, and Integrated Gradients methods that they implemented on a neural network model. When testing the proposed solution, they examined the impact of XAI methods on improving results. The German Credit Dataset [192], a widely used dataset in the fields of machine learning, data science, and financial modeling, was used. Its main goal is to classify loan applicants based on various attributes. Evaluation included both objective (e.g., accuracy) and subjective metrics (answers to various questions on a scale from 1 to 5). They examined 23 metrics in total. With such a large number of metrics, it may not be easy to determine which method is the best. We therefore suggest using our tool, which allows you to find a compromise between the four tested alternatives. After loading the data, it is necessary to determine the weights of the metrics; we recommend using the objective CRITIC metric, which can easily handle a larger number of metrics. The weights obtained in this way are shown in

Table 9. Criterion weights according to the CRITIC method for the German Credit Dataset.

Criterion	Weight
Accuracy	0.105
F1	0.098
AVG Time	0.0441
Over-Reliance	0.0378
Under-Reliance	0.0551
Average Absolute Odds Difference (AAOD)	0.0306
Equal Opportunity Difference (EOD)	0.0602
Q1	0.0305
Q2	0.0288
Q3	0.0301
Q4	0.0289
Q5	0.0419
Q6	0.0265
Q7	0.0305
Q8	0.0257
Q9	0.0378
Q10	0.029
Q11	0.0338
Q12	0.0268
Q13	0.0333
Q14	0.085
Q15	0.0495
Q16	0.0309

. Using all MCDM methods, we obtained the following Borda count: LIME: 20, SHAP: 29, SmoothGrad: 9, Integrated Gradients: 2.

Table 10. Preferences of LIME, SHAP, SmoothGrad, and Integrated Gradients according to all MCDM methods.

Method	LIME	SHAP	SmoothGrad	Integrated Gradients
ARAS	0.8842	0.9055	0.865	0.8118
EDAS	0.6029	0.7655	0.5161	0.149
CODAS	−0.0138	0.1328	0.0077	−0.1267
MABAC	0.1619	0.2	0.0151	−0.1644
PROMETHEE II	0.2616	0.2223	−0.0601	−0.4238
VIKOR	0.3132	0.0317	0.7538	0.5
TOPSIS	0.5333	0.6839	0.412	0.5099
WASPAS	0.8883	0.8993	0.8657	0.8152
MARCOS	0.6803	0.6939	0.6649	0.6264
WSM	0.255	0.2616	0.2495	0.2339

displays the computed preference scores derived from all MCDM methods. The best choice for implementation in this case is SHAP. This use case demonstrates the possible combination of objective and subjective metrics and objective weighting.

The proposed tool also allows for a simple display of positions in rankings according to MCDM methods for each of the XAI methods (Figure 5). This allows for analysis of changes in positions according to individual MCDM methods. Another useful display is the correlation matrix.

The tool also allows for deeper analysis of the results and the examination of correlations between the results of MCDM methods. The correlation matrix for this third use case is shown in Figure 6. The correlation matrix displays the degree of concordance between the rankings generated by the ten different MCDM methods. The values may range from 0.00 (no agreement) to 1.00 (perfect agreement). Most MCDM methods show a very high degree of correlation (0.80 to 1.00), indicating that for this specific dataset, they largely agree on the final ranking of the alternatives.

In the next section, we will describe the sensitivity analysis for the third use case. According to the analysis design, preferences and rankings were calculated for the data from this use case for various weights with $\delta$ increment for all MCDM methods, a total of 200 calculations. The table of weight changes according to the $\delta$ value is in Appendix A due to its large scope. The analysis is focused on investigating, in particular, the selection of the best method.

Table 11. Percentage of 1st Rank across sensitivity analysis experiments.

Method	LIME	SHAP	SmoothGrad	Integrated Gradients
ARAS	85.00%	15.00%	0.00%	0.00%
CODAS	65.00%	35.00%	0.00%	0.00%
EDAS	80.00%	20.00%	0.00%	0.00%
MABAC	95.00%	5.00%	0.00%	0.00%
MARCOS	90.00%	10.00%	0.00%	0.00%
PROMETHEE II	95.00%	5.00%	0.00%	0.00%
TOPSIS	85.00%	15.00%	0.00%	0.00%
VIKOR	45.00%	20.00%	35.00%	0.00%
WASPAS	90.00%	10.00%	0.00%	0.00%
WSM	85.00%	15.00%	0.00%	0.00%

presents the percentage of methods in first place according to the ranking of MCDM methods. At first glance, it is obvious that the LIME method dominated over all MCDM methods. However, this is in contrast to the results for the original weights, where the SHAP method dominated.

The least robust in this case is the VIKOR method, which replaced 3 methods in first place when changing the weights. A more detailed overview of the ranking change based on the weight change is shown for this method in Figure 7.

The pie chart (Figure 8) shows that among all calculations performed, the preference in this use case was dominated by the LIME method (81.5%); SHAP was in first place only in 15% of cases.

Although SHAP offered a marginally better preference score in the initial setup, the results from the extensive sensitivity analysis clearly establish LIME as the more robust and reliable choice. This suggests that LIME’s performance is stable even when the importance of the XAI metrics is adjusted, making it a more dependable method for application.

Sensitivity analysis for the first and second use cases demonstrated the robustness of the selection of the best XAI method. In the first case, the LIME method dominated in 98.5% of all tests, and in the second case, the Tree SHAP method dominated in 94.3%.

6. Discussion

In this section, we discuss the proposed approach to selecting an appropriate XAI method using MCDM. The complexity and diversity of existing XAI techniques require a systematic framework that would allow users to navigate and select the most appropriate solution based on their specific needs and constraints.

To simplify the selection process and increase its usability, we have created a useful list of XAI methods, which is structured and allows users to filter methods according to key features, such as local vs. global explanation, model-agnostic vs. model-specific, or type of input data (e.g., image, text). To support the evaluation of XAI methods, we have compiled an overview of relevant benchmarks and software libraries that are focused on a quantitative evaluation of XAI methods. The challenge for the future is to add other important aspects that will be more focused on specific domains and user skills, and experiences.

The use of multi-criteria decision-making methods in the context of Explainable Artificial Intelligence is extremely useful and provides solutions to many problems. Compared to previous studies (e.g., AutoXAI or eXplego), we use MCDM to select the right method given the metrics and specifics of the application. When choosing an XAI method, several, often conflicting, requirements need to be considered. For example, a method that is extremely faithful to the model prediction may also be computationally demanding. MCDM allows these different criteria to be weighed, and an optimal balance to be found based on the user’s needs. The proposed tool allows the user to define weights for each metric (e.g., someone will give more weight to Robustness, while another will give more weight to Speed). This makes it a very flexible tool adaptable to specific domain requirements, going beyond the framework of fixed decision trees such as eXplego. Furthermore, the list of XAI methods is easily expandable, which can be more difficult with a tree structure.

While traditional benchmarks often focus only on technical metrics, MCDM provides a structured framework for incorporating the subjective preferences of domain experts. Methods such as AHP or CRITIC allow experts to assign importance (weights) to different metrics, ensuring that the selected XAI method is relevant to their specific work context and requirements. Compared to AutoXAI, our solution does not focus on hyperparameter tuning. Its advantage is that it does not restrict users to predefined metrics; instead, they can select metrics according to their specific use, whereas AutoXAI is limited to the metrics and hyperparameters it supports.

MCDM transforms the complex selection of an XAI method from a manual, intuitive process into a transparent, mathematically justified, and reproducible decision. The proposed solution aggregates the results of ten MCDM methods using the Borda count, thus ensuring the robustness of the result.

MCDM is crucial because the problem of selecting the best XAI method is not a single-criterion problem, but a complex multi-criteria problem requiring a weighted synthesis of objective performance and subjective usability.

The proposed approach includes the possibility of using a wide range of criteria that cover the desired properties of explanations (e.g., fidelity, robustness) as well as technical aspects (e.g., computational complexity, speed). The key is the implementation of a flexible mechanism for calculating the weights of the criteria. Users can enter the weights directly (direct rating method), which is simple and intuitive if they have a clear idea of the importance of the criteria. Alternatively, a pairwise comparison method (based on AHP) is available that takes into account subjective preferences. To ensure objectivity and minimize subjective bias, we integrated the CRITIC method, which determines weights solely based on the internal structure of the data and the contrasting strength of individual criteria. The ability to combine subjective and objective weights represents a significant benefit that ensures robust selection of the most appropriate XAI method.

The presented use cases confirm the broad domain and task applicability of the proposed approach. At the same time, they illustrate the ability of this approach to flexibly obtain weight coefficients using objective and subjective methods and to take into account a wide range of metrics.

Legislative changes pose a challenge for the near future use of artificial intelligence systems. The European Union’s AI Act represents a pioneering regulatory framework designed to ensure that artificial intelligence systems deployed within the EU adhere to safety, transparency, and fundamental rights standards. It utilizes a risk-based approach, classifying AI applications into four tiers—minimal, limited, high, and unacceptable risk—with compliance obligations scaled accordingly. This framework builds upon existing European data protection legislation, notably the General Data Protection Regulation (GDPR), which already mandates strict requirements for the processing of personal data, including the principles of data minimization and the right to explanation for automated decisions.

The EU AI Act inherently relies on explainable AI (XAI) to ensure compliance for systems categorized as High-Risk. The legislation mandates specific outcomes—namely, transparency, human oversight, and verifiable data governance—that necessitate XAI tools. Without effective interpretability techniques, operators cannot adequately understand complex model behavior, audit for bias in training data, or fulfill the requirement for meaningful human oversight. Thus, XAI serves as the crucial technical mechanism for meeting the legal and ethical obligations imposed by the Act.

Since the choice of XAI method involves complex trade-offs, our MCDM tool is crucial for risk mitigation. By systematically evaluating XAI candidates against relevant metrics and employing robust methods to find the optimal compromise, our tool provides a defensible, objective, and stable ranking. Several XAI methods also offer Counterfactual Explanations, which can help with objections.