Integration Modes Between MCDM Methods and Machine Learning Algorithms: A Structured Approach for Framework Development

Abstract

Decision-making is increasingly guided by the integration of Multi-Criteria Decision-Making (MCDM) and Machine Learning (ML) approaches. Despite their complementary strengths, the literature lacks clarity on which forms of integration exist, what contributions they offer, and how to determine the most effective form for a given decision problem. This study systematically investigates integration modes through a methodology that combines a literature review, expert judgment, and statistical analyses. It develops a novel categorization of integration modes based on methodological characteristics, resulting in five distinct modes: sequential approaches (ML → MCDM and MCDM → ML), hybrid integration (MCDM + ML), and performance comparison approaches, including ML vs. MCDM and ML vs. ML evaluated through MCDM. In addition, new evaluation criteria are introduced to ensure rigor, comparability, and reliability in assessing integration forms. By applying correspondence, cluster, and discriminant analyses, the study reveals distinctive patterns, relationships, and gaps across integration modes. The primary outcome is a novel evidence-based framework designed to guide researchers and practitioners in selecting the appropriate integration modes based on problem characteristics, methodological requirements, and application context. The findings reveal that sequential approaches (ML → MCDM and MCDM → ML) are most appropriate when efficiency, structured decision workflows, bias reduction, minimal human intervention, and the management of complex multi-variable decision problems are key objectives. Hybrid integration (MCDM + ML) is better suited to dynamic and data-rich environments that require flexibility, continuous adaptation, and a high level of automation. Performance comparison approaches are most appropriate for validation-oriented studies that evaluate outputs (MCDM[ML vs. ML]) and benchmark alternative methods (ML vs. MCDM), thereby supporting reliable method selection. Furthermore, the study underscores the predominance of integration modes that combine value-based MCDM methods with classification-based ML algorithms, particularly for enhancing interpretability. Environmental science and healthcare emerge as leading domains of adoption, primarily due to their high data complexity and the need to balance diverse, multi-criteria stakeholder requirements.

1. Introduction

Decision-making processes in the academic literature and practical applications have been addressed through a variety of methodological approaches that differ in how they represent, process, and synthesize inputs for evaluating alternatives. Among these, Multi-Criteria Decision-Making (MCDM) and Machine Learning (ML) methods have emerged as prominent and complementary approaches. MCDM, as a prescriptive approach grounded in normative decision theory, provides structured frameworks for evaluating and prioritizing alternatives across multiple, often conflicting, criteria. Notably, MCDM is not exclusively expert-driven. While early applications frequently relied on subjective judgments to elicit criteria weights and preference structures, the methodological scope of MCDM has progressively expanded to incorporate objective, data-driven inputs as well. Contemporary MCDM techniques can integrate subjective judgments, objective measurements, or hybrid combinations thereof, making them applicable to both judgmental and non-judgmental decision environments. In this context, the defining characteristic of MCDM lies in its explicit modeling of preferences, trade-offs, and aggregation mechanisms that guide decision-oriented outcomes (e.g., ranking, sorting, or selection), rather than in the mere origin of the inputs as expert judgments or empirical datasets. Objective weighting schemes have been widely employed in empirical settings where expert judgments are unavailable, impractical, or intentionally excluded. These approaches preserve the normative structure of MCDM while leveraging observed data to infer relative importance among criteria. ML, on the other hand, encompasses a range of algorithms capable of learning patterns and relationships from large and complex datasets to support prediction, classification, and optimization tasks, offering high adaptability and computational efficiency [1].

Advancements in data analytics and intelligent systems have increasingly blurred the boundaries between these methodological streams, fostering the development of integrated decision-making frameworks. This study focuses on the integration of MCDM and ML methods, enabling decision-makers to harness their complementary strengths—the interpretability, structured multi-criteria evaluation, and transparency of MCDM, together with the adaptability, learning capability, and predictive power of ML algorithms—to enhance both the effectiveness and quality of decision-making.

Although the number of studies combining both approaches has recently increased [2], only a few studies [3,4,5,6,7] have examined how researchers use ML and MCDM together and the directions in which these methods are evolving, particularly through structured literature reviews and bibliometric analyses. However, none of these studies have investigated how the two approaches can be effectively integrated, nor have they systematically compared the advantages of different integration forms or their applicability across decision contexts. As a result, the literature offers little practical guidance on selecting appropriate integration modes for specific decision contexts, and there are no established criteria to inform the choice of ML–MCDM integration modes. In the absence of structured categorization and evaluation criteria, the choice of integration modes is often arbitrary, driven by convenience or precedent rather than specific problem requirements. This lack of clarity constitutes the core research problem addressed in this study, as it may lead to underutilization or inappropriate application of ML–MCDM integration, and limit its potential benefits for complex decision-making problems.

Therefore, to gain a deeper understanding of the integration of these approaches, this study aims to address the following questions:

• Which forms of integration between ML algorithms and MCDM methods exist for addressing decision problems?
• What criteria should be considered when selecting the most appropriate form of integration?
• What specific contributions are delivered by these integration forms?
• What are the current trends in research on the joint use of MCDM and ML methods?
• Which application areas have adopted integrated approaches most extensively?
• What are promising future research directions?

To answer these questions, this study employs a structured methodological approach that involves a systematic literature review, categorization, and expert evaluation of different integration forms, and the development of a framework. Based on the review of the current literature, the various ways in which MCDM methods and ML algorithms have been integrated are identified and categorized, while evaluation criteria are established drawing on prior research and expert input to benchmark these integration forms. The relevant studies are then systematically mapped to both categories and criteria. The use of expert judgments in this process ensures a reliable and comprehensive view of how integration forms are employed and what contributions they deliver. These relationships are further examined through correspondence analysis (CA) and cluster analyses, which uncover patterns within and between integration modes and evaluation dimensions. The findings offer an evidence-based framework for selecting a suitable integration approach based on the inherent features, structure, and requirements of the decision problem. The validity of the proposed framework is then assessed through discriminant and Procrustes analyses. Finally, key insights from the analysis, such as the most commonly used integration modes, common method–algorithm pairings, and dominant application areas, are presented, along with limitations and suggestions for future research to advance the integration of MCDM methods and ML algorithms.

Consequently, the theoretical and practical contributions of the study can be summarized as follows:

• Follows a methodology that synthesizes insights from the literature with expert opinions, thereby ensuring both theoretical depth and practical relevance.
• Establishes a new systematic categorization of integration modes between ML algorithms and MCDM methods, grounded in their methodological characteristics and application contexts.
• Introduces, for the first time, a structured set of criteria that enable standardized and reliable evaluation of integration modes, ensuring comparability and rigor.
• Demonstrates how the complementary strengths of ML and MCDM enhance decision-making, overcome their individual limitations, and generate added value when combined.
• Employs CA and cluster analysis correspondence analysis to visually map the relationships between integration modes and evaluation criteria, revealing clusters, patterns, and associations that highlight dominant integration modes, trends, and potential gaps in the literature.
• Proposes a new evidence-based framework that consolidates theoretical insights with practical considerations. It provides researchers and practitioners with clear guidance on selecting the appropriate integration mode, ensuring effective alignment between methodological choices and problem-specific requirements. The proposed framework is designed to address a wide range of complex decision problems where data is available, enhancing the effectiveness and efficiency of the decision process.
• Validates the spatial associations and cluster structure revealed by CA and clustering using discriminant analysis and further confirms the robustness of the proposed framework through Procrustes and discriminant analyses conducted on a holdout sample.
• Finally, outlines future research directions to address current gaps, leverage emerging trends, and further advance ML–MCDM integration.

The rest of this paper is organized as follows: Section 2 provides an overview of MCDM and ML methods. Section 3 describes the research methodology, including the systematic literature review, expert evaluation, mapping, framework development, and validation. Section 4 provides an overview of the current state of research, emerging trends in the joint use of MCDM and ML methods, and key findings. Section 5 provides future research directions. Finally, Section 6 presents the concluding remarks and limitations of the study.

2. Overview of MCDM Methods and ML Algorithms

2.1. MCDM Methods and Their Strengths and Weaknesses

MCDM is a branch of operations research and decision-making methods that provide a powerful framework for analyzing and evaluating complex decision problems. MCDM methods can be categorized into several methodological families. Elementary methods rely on simple mathematical aggregation, with Simple Additive Weighting (SAW) being a representative example [8]. Value-based methods, such as the Analytic Hierarchy Process (AHP), Technique for Order Preference by Similarity to Ideal Solution (TOPSIS), and Grey Relational Analysis (GRA), construct value or utility functions to evaluate alternatives. Outranking methods—including Preference Ranking Organization Method for Enrichment Evaluation (PROMETHEE) and Elimination and Et Choice Translating Reality (ELECTRE)—use pairwise dominance relations to identify preferred alternatives without requiring full compensability among criteria. Other notable families include goal programming and reference point methods, as well as interactive approaches that involve the decision-maker iteratively in the search for the most preferred solution.

MCDM methods offer several advantages, particularly in complex decision-making scenarios where multiple, often conflicting criteria must be considered. By accommodating multiple criteria simultaneously, these methods allow decision-makers to achieve a more comprehensive understanding of the trade-offs and interactions among different factors [9,10,11,12,13], especially in complex decision problems [14,15,16]. These methods improve decision-making by incorporating quantitative data such as cost, time, and efficiency, along with qualitative judgments such as expert opinions and stakeholder preferences [17,18], while also effectively integrating historical data and expert judgments to quantify subjective opinions [19].

While MCDM methods offer numerous advantages, they also have weaknesses and limitations. Some MCDM methods can be computationally intensive, requiring substantial processing power and time, especially when dealing with many criteria and alternatives [20]. As a result, their applicability may be restricted in real-time or resource-constrained scenarios [21,22]. Furthermore, humans’ ability to process and analyze large volumes of data is inherently limited [23]. This limitation often necessitates the use of advanced analytical tools and techniques to aid decision-making.

MCDM methods also depend heavily on comprehensive and high-quality data for all criteria considered. When data are incomplete, uncertain, or unreliable, the resulting evaluations may become misleading, potentially compromising the quality of decisions. Finally, relying on expert judgment to define criteria and assign weights can yield valuable insights to mitigate these issues; however, it is essential to remain aware of potential biases and inconsistencies that may arise [10,13,15,24,25,26].

2.2. ML Algorithms and Their Strengths and Weaknesses

ML is a subfield of artificial intelligence (AI) that involves developing algorithms and statistical models. These models enable computers to learn from data and make predictions and decisions without requiring specific programming instructions. ML algorithms are classified into three categories: supervised learning, unsupervised learning, and reinforcement learning [27]. Supervised learning algorithms learn from a labeled dataset. In this type of learning, each training example is associated with an output label. The main objective is to enable the model to understand and create a relationship between inputs and outputs to accurately predict new data points’ labels. Some common algorithms in this category are Linear Regression, Logistic Regression, Decision Trees, and Support Vector Machines (SVMs). Unsupervised learning, on the other hand, involves training algorithms on unlabeled data to uncover hidden patterns or structures within the dataset [28]. These algorithms include K-means clustering, Hierarchical clustering, and Principal Component Analysis (PCA). Finally, Reinforcement learning focuses on the interaction between the environment and the agent. The agent learns from the outcomes of its actions rather than receiving direct instructions [27].

Machine learning integrates various elements of computer science, applied mathematics, and statistics, supported by a robust theoretical foundation, making it successful in producing more accurate results [19,29]. These algorithms can process and analyze large volumes of data that would be impractical for humans to handle manually. Their ability to recognize complex patterns and relationships within large datasets provides deeper insights for decision-makers, making them invaluable tools [9,30,31,32]. ML algorithms improve decision-making quality by transforming data into information without human intervention, thus reducing human errors [17,26,33].

Even though ML algorithms are powerful and versatile, they have several disadvantages and challenges that can hinder their practical implementation and reliability. One key challenge is their dependency on high-quality data; machine learning models require large, clean, and representative datasets to perform well, and poor-quality or imbalanced data can lead to inaccurate or biased outcomes [34]. Complexity and interpretability issues are also significant, as many advanced models—particularly deep learning architectures—are often treated as “black boxes”, making it difficult for users to understand how decisions are made, which reduces transparency and trust in critical applications [35]. Machine learning models can also be resource-intensive, requiring substantial computational power, memory, and storage—especially during training—which can limit their accessibility and scalability for smaller organizations or edge devices [36]. Additionally, machine learning projects often rely on domain expertise for feature engineering, problem formulation, and interpretation of results, which may not be readily available in all contexts [37]. Lastly, scalability issues can arise when deploying models in real-world settings with vast and dynamic data streams, requiring robust infrastructure and architecture to support real-time processing and updates [19,23].

The strengths and limitations of ML algorithms and MCDM methods are discussed separately in this section, not as independent evaluations, but to emphasize their complementary nature. Specifically, the strengths and limitations of each approach, when used independently, highlight the need for integrating ML and MCDM methods.

3. Methodology

This study suggests a structured methodological approach to examine the integration modes between MCDM methods and ML algorithms and to provide insight to researchers and practitioners. The process begins with the formulation of research questions (presented in the Introduction), which serve as the foundation for the study’s overall design. To address these questions, two primary sources of input are utilized: expert opinion and a comprehensive review of the literature. Expert insights draw on practical experience and contextual knowledge, while the literature review ensures theoretical depth and coverage of the state of the art.

Building on these inputs, two parallel tasks are undertaken. First, the categorization of integration modes identifies and classifies the different ways in which MCDM methods and ML algorithms are combined in the existing body of research. Second, the evaluation criteria are determined, drawing on expert input and prior studies to establish a set of benchmarks for assessing these integration modes.

Once the categories and criteria are established, articles are associated with relevant categories to capture the integration modes employed in each study and to analyze how different methods are combined or applied systematically. Experts against the criteria then evaluate each article to highlight the contributions and outcomes reported in prior studies. To ensure consistency in judgments, inter-rater reliability is assessed to verify the level of agreement among the experts. These associations and evaluations are then synthesized to reveal the relationships between categories and criteria, providing a structured view of how different forms of integration perform against the established evaluation criteria.

To analyze these associations in depth, CA and cluster analysis are employed. These techniques enable the identification of patterns and connections between integration modes and evaluation criteria, offering an evidence-based foundation for further interpretation. The results of this analysis inform the development of a structured framework to identify the appropriate integration form based on the nature of the decision problem. The framework is then validated by confirming the spatial associations and cluster structure through discriminant analysis, and by further demonstrating its robustness using Procrustes and discriminant analyses on an independent holdout sample.

The methodology concludes with the presentation of key findings from the analysis, followed by suggestions for future research that highlight promising avenues to advance the integration of MCDM methods and ML algorithms. Figure 1 illustrates the structured methodological approach employed in this study, while the following sections explain each step in detail.

3.1. Literature Review

The literature review for this study was conducted using Elsevier’s Scopus and Web of Science databases, selected for their extensive coverage of high-quality publications in engineering, computer science, and decision science—ensuring alignment with the research focus. To capture the evolution and development of the concept, we included publications spanning 1999 to 2026. Only peer-reviewed journal articles were considered to ensure fully validated research, prioritizing methodological rigor, reliability, and completeness.

Table 1. Search criteria for literature review.

Search Criteria	Details
Databases	Elsevier’s Scopus
Search Strings	(KEY (“Machine Learning”) AND KEY (“multi criteria”) OR KEY (“multi attribute”) OR KEY (“multiple criteria”) OR KEY (“multiple attribute”) OR KEY (“MADM”) OR KEY (“MCDM”) OR KEY (“ANP”) OR KEY (“VIKOR”) OR KEY (“ELECTRE”) OR KEY (“PROMETHEE”) OR KEY (“TOPSIS”) OR KEY (“AHP”)) AND PUBYEAR > 1999 AND PUBYEAR < 2026 AND (LIMIT-TO (DOCTYPE, “ar”)) AND (LIMIT-TO (LANGUAGE, “English”)) AND (LIMIT-TO (SRCTYPE, “j”))
Time Frame	From: 1999, To: 2026
Document Type	Article
Language	English
Source Type	Journal
Document Results	763

summarizes the key details of the search process for Scopus, and a similar process was followed for Web of Science.

As shown in Figure 2, the systematic selection process began by retrieving 1146 records from the Scopus and Web of Science databases. After removing duplicate studies that appeared in both databases, the remaining 816 records were further examined for relevance, resulting in the exclusion of 254 papers that were deemed out of scope based on their keywords. After this initial filtering, 562 papers were selected for further evaluation through an additional title and abstract screening. The titles and abstracts of these papers were then reviewed, resulting in the exclusion of 246 additional papers, bringing the total to 316 papers that underwent full-text screening. A more detailed full-text screening was then conducted, during which 86 papers were discarded for reasons such as lack of original content, insufficient detail, or irrelevance to the study’s objectives. Ultimately, 230 studies were deemed relevant, of which 208 were included in the final review, ensuring a refined and focused set of studies for the literature review. 22 (approximately 10% of the relevant studies) were reserved for validation.

3.2. Categorization

The literature recently focused on combining MCDM and ML approaches to make more informed decisions. To enhance the outcomes of these approaches, an effective combination of structured decision-making processes, human judgment, analytics (machines), and relevant data is essential for achieving more informed and high-quality decisions. This integrated perspective is clearly stated in the DECAS model proposed by Elgendy et al. [38], where decision-making effectiveness relies on the structured interaction of five key components: decision-making process, decision-maker, decision, data, and analytics [38]. This comprehensive approach emphasizes integrating these fundamental components in decision-making contexts.

Growing interest in combining expert-based and data-driven decision-making approaches underscores the need for a deeper understanding of how to achieve effective human–machine integration. To this end, Ransbotham et al. [39] identified five primary modes of interaction, each shaped by the decision context and type:

• Machines make decisions jointly with humans and implement them together.
• Machines decide, while humans are responsible for implementation.
• Machines provide recommendations, and humans make the final decision.
• Machines generate insights to inform the human decision-making process.
• Humans create options and hypothetical scenarios, which machines then evaluate.

The discussion in these studies predominantly frames the distinction as expert-based versus data-driven, implicitly suggesting that the choice of integration is determined solely by the nature of the data and the sequence of methods. As noted in the Introduction, numerous studies on MCDM employ also objective schemes for non-judgmental or data-driven settings.

In another study, conducting a bibliometric analysis, Düzen et al. [6] suggested purpose-oriented models to describe different types of combinations of ML and MCDM methods, with particular emphasis on the sequence of methods and the comparison of methodological performances. Unlike studies that categorize integration types, Liao et al. [4] review a particular type of integration by outlining how machine learning technologies contribute to MCDM. Specifically, they conduct an in-depth examination of ML-supported MCDM through four key procedures: criteria extraction, criteria interaction, parameter determination, and decision support system development. The study by Reyes-Norambuena et al. [7] examines how MCDM and ML are combined to enhance decision-making in pedestrian dynamics. Rather than proposing a systematic categorization of integration types, the review paper provides examples from the literature. The findings indicate that hybrid MCDM algorithms can effectively assess the performance of ML models.

Building on these insights, selecting and implementing the suitable and effective mode of integration is not straightforward. To better understand the integration dynamics, the review of the literature enabled us to examine prior research to identify and group the different ways MCDM methods and ML algorithms have been integrated into broad categories based on common characteristics. As a result, five primary categories of integration between these two approaches were identified to more effectively represent the diversity and scope of recent studies. The integration modes are presented below with their respective mathematical formulations and explanations. The formulations are provided in a generalized form, with article-specific details omitted for clarity and conciseness. For consistency, the notation used for each mode is specified as follows. For further information, please refer to Supplementary Materials.

Notation

The notation used in the mathematical formulations of the integration modes is presented below.

$\mathrm{A}$: set of decision alternatives, where $\mathrm{A} = \left\{A_1, A_2, \dots , A_m\right\}$

$\mathrm{C}$: set of evaluation criteria, where $\mathrm{C}= \left\{C_1, C_2, \dots , C_n\right\}$

$x_{ij}$: value of alternative i under criterion j

${\mathrm{X}}_i$: feature vector for alternative A_i, where ${\mathrm{X}}_i = \left[x_{i1}, x_{i2}, \dots , x_{\mathrm{in}}\right]$

$p_{ij}$: observed the performance of ML algorithm $A_i$ on performance criterion $C_j$

$N\left(·\right)$: normalization operator

$r_{ij}$: normalized or scaled criterion value

$I_j$: importance of feature j derived from ML model

$W^{\mathrm{M}\mathrm{C}\mathrm{D}\mathrm{M}}$: MCDM weighting method

$w_j^{\mathrm{M}\mathrm{L}}$: weight of criterion j derived by normalizing $I_j$

$w_j^{\mathrm{e}\mathrm{x}\mathrm{p}}$: weight of criterion j derived from expert judgment

$w_j$: final weight of criterion j used in MCDM

$W$: weight-fusion operator combining multiple weight sources

$D\left(A_i\right)$: overall decision score or ranking value for alternative I

$\theta$: parameters of ML model

$f\left(·;\theta \right)$: ML function algorithm parameterized by θ

$\Lambda$: parameters of the MCDM aggregation method

$A_{\Lambda }\left(·\right)$: MCDM aggregation function

$Y$: observed target or response values

$\widehat{y}$: predicted outputs from ML model(s), e.g., predictions, probabilities, class labels, cluster assignments, feature importances, or learned weights

${\widehat{y}}_i$ = predicted output from ML model for alternative $A_i$

$L\left(\widehat{y}, Y\right)$: loss or objective function used to optimize ML parameters

$\eta$: learning rate

$\lambda$: trade-off parameter controlling MCDM influence

${\mathrm{E}}^{\mathrm{M}\mathrm{L}}$: evaluation metrics for ML results

${\mathrm{E}}^{\mathrm{M}\mathrm{C}\mathrm{D}\mathrm{M}}$: evaluation metrics for MCDM results

$\rho \left(·,·\right)$: performance comparison or correlation function

$g\left(·\right)$: function for estimating feature importance in the ML model

$\Phi \left(·\right)$: transformation or concatenation operator

$\Omega \left(·\right)$: synthesis or fusion operator

$\delta \left(·\right)$: selection/decision function that chooses the preferred algorithm(s) given scores and additional constraints.

Category 1: Using ML Outputs as Inputs to MCDM Methods (ML → MCDM)

This category involves integrating outputs from ML algorithms or data analytics techniques as inputs to MCDM methods. For instance, predictions, scores, extracted features, or feature importances derived from large datasets can serve as criterion values or weights in the decision-making process. This integration enables decision-makers to leverage data-driven insights, improve the objectivity of criteria evaluation, and enhance the overall quality and robustness of multi-criteria decisions. Several studies have demonstrated this integration, such as Pirouz et al. [40], who utilized a Bayesian network–supported AHP method for COVID-19 analysis, while Yadegaridehkordi et al. [16] applied segmentation-based TOPSIS to select green hotels. Similarly, Elomiya et al. [41] integrated Random Forest with MCDM methods such as Analytic Hierarchy Process (AHP), fuzzy AHP, and Stepwise Weight Assessment Ratio Analysis (SWARA) to select optimal sites for electric vehicle charging stations. The general formulation of the integration mode ML → MCDM can be written concisely as:

$$A^{*}={argmax}_{\mathrm{i}} A_{\Lambda }\left(\left\{N_{\mathrm{j}}\left(f_{\mathrm{j}}\left(A_i; {\theta }_{\mathrm{j}}\right)\right), W\left(w_j^{ml}, w_{\mathrm{j}}^{\exp }\right)\right\} for all j\right),$$

(1)

where $f_{\mathrm{j}}$ denotes the ML model that produces usable data for MCDM, $N_j$ denotes normalizing or representing the data (numerical, probabilistic, or fuzzy), $W$ denotes the operator combining ML-derived and expert-derived weights, and $A_{\Lambda }$ denotes aggregating all weighted and normalized criterion values into a final decision score. See

Table 2. Summary of the stages of integration mode ML → MCDM.

Stage	Mathematical Representation	Equation	Purpose
Generate ML outputs	${\widehat{y}}_{ij}=f_j\left(A_i; \theta \right)$	(2)	Produce predictive ML outputs (e.g., scores, probabilities, class labels, or estimates) for alternative $A_i$ under criterion $C_j.$
Derive feature importance	$I_j=g\left(f_j, X\right)$	(3)	Derive feature importance or weight indicators from ML models (e.g., SHAP, Gini importance, coefficients).
Construct input for MCDM	$x_{ij}={\widehat{y}}_{ij}$	(4)	Use ML outputs as criterion values ($x_{ij}$) and/or as weights ($w_j$) in the MCDM framework. If needed, ML-derived weights can be combined with expert-derived weights.
	$w_j^{\mathrm{ML}}=normalize\left(I_j\right)$	(5)
	$w_j=W\left(w_j^{\mathrm{ML}}, w_j^{\exp }; \alpha \right)$	(6)
Normalize (Optional)	$r_{ij}=N_{\mathrm{j}}\left(x_{ij}\right)$	(7)	Normalize ML-derived data to make them comparable across criteria or scales.
Aggregate normalized data using MCDM	$D\left(A_i\right)=A_{\Lambda }\left(\left\{r_{ij}, w_j\right\}\right)$	(8)	Aggregate the ML-based inputs through the MCDM method (e.g., Simple Additive Weighting, TOPSIS, AHP) to obtain overall decision scores for each alternative.
Rank or select alternatives	$rank\left(A_i\right)=\mathrm{p}\mathrm{o}\mathrm{s}\mathrm{i}\mathrm{t}\mathrm{i}\mathrm{o}\mathrm{n} \mathrm{o}\mathrm{f} A_i \mathrm{w}\mathrm{h}\mathrm{e}\mathrm{n} \mathrm{s}\mathrm{o}\mathrm{r}\mathrm{t}\mathrm{i}\mathrm{n}\mathrm{g} \mathrm{b}\mathrm{y} D\left(A_i\right)$	(9)	Rank alternatives or select the best one based on the integrated ML–MCDM evaluation results.
	$A^{*}={argmax}_{\mathrm{i}} D\left(A_i\right)$	(10)
Validate	$E=Eval\left(D\left(A_i\right), GroundTruth\right)$	(11)	Validate and refine the decision model by comparing MCDM outcomes with observed or expert-evaluated results.

for a summary of the stages of the integration mode.

Category 2: Using MCDM Results as Input in ML Algorithms (MCDM → ML)

This category integrates results obtained with MCDM methods as input to ML algorithms. The criteria weights or alternative rankings/sortings derived from MCDM can be used to train machine learning models or to improve decision-making. This integration enables ML algorithms to incorporate structured expert knowledge and multi-criteria preferences, enhancing their accuracy and practical relevance. Notable examples of this approach include Oliveira de Sousa et al. [42], who used AHP-based multi-criteria analysis for flood risk assessment with machine learning, and Saleh et al. [43] who applied TOPSIS followed by machine learning classifiers to prioritize maintenance tasks in healthcare facilities. Additionally, Sotiropoulou et al. [44] integrated PROMETHEE II with machine learning regression methods, such as k-Nearest Neighbors and Support Vector Machines, for land use suitability analysis. The general formulation of the integration mode MCDM → ML can be written concisely as:

$${\widehat{y}}_i= f\left(\Phi \left(X_i, A_{\Lambda }\left(\left\{N_j\left(x_{ij}\right), w_j\right\}\right), \left\{w_j\right\}\right);\theta \right),$$

(12)

where $A_{\Lambda }(·)$ produces aggregated decision results from multi-criteria data, $\Phi (·)$ fuses those structured results into the ML input space, and $f\left(·; \theta \right)$ learns mappings from enriched inputs to outcomes, guided by loss $L$. See

Table 3. Summary of the stages of integration mode MCDM → ML.

Stage	Mathematical Representation	Equation	Purpose
Compute MCDM results	$D\left(A_i\right)=A_{\Lambda }\left(\left\{r_{ij}, w_{\mathrm{j}}\right\}\right)$	See (8)	Compute overall decision scores, ranks, or classes for each alternative from normalized multi-criteria data ($r_{ij}$) and weights ($w_j$).
Construct MCDM-informed feature vectors	$Z_i=\Phi \left(X_i, D\left(A_i\right), \left\{w_j\right\}\right)$	(13)	Combine the MCDM-derived outputs with other features or contextual data to form the enriched feature vector for ML.
Train ML model using enriched data	${\widehat{y}}_i=f\left(Z_i; \theta \right)$	(14)	Train the ML model to learn data-driven patterns from MCDM-enhanced features.
	$\min over \theta : L\left(\hat{\mathrm{y}}, Y\right)=\Sigma \left(L\left(f\left(Z_i; \theta \right), Y_i\right)\right)$	(15)
Model prediction or inference	${\hat{y}}^{*}=f\left(\Phi \right(X^{*}, D\left(A^{*}\right), w_1,\dots , w_n); \theta )$	(16)	Once trained, employ MCDM-informed features within the model to predict outputs for new alternatives ($A^{*}$).

for a summary of the stages of the integration mode.

Category 3: Combining ML and MCDM Methods (MCDM + ML)

This category involves jointly applying ML and MCDM methods within the same decision-making process. By integrating the data-driven analytical and predictive capabilities of ML with the multi-criteria evaluation and weighting strengths of MCDM, this hybrid approach supports comprehensive and consistent decisions that account for both quantitative data and qualitative judgments. Such combinations allow decision-makers to exploit the complementary advantages of the two methods, leading to more accurate, balanced, and practically relevant outcomes. Several studies have demonstrated this integration, such as Guerrero-Gómez-Olmedo et al. [45] who enhanced the interpretability of deep neural networks (DNNs) using multi-criteria analysis. Rafiei-Sardooi et al. [46] integrated state-of-the-art machine learning methods (support vector machine, random forest, and boosted regression tree) with TOPSIS for urban flood vulnerability analysis, and Parishani et al. [47] developed a new weighting method for multi-class classification in disease diagnosis by combining the confusion matrix and MCDM methods. The general formulation of the integration mode MCDM + ML can be concisely expressed as:

$$S\left(A_i\right)= \Omega \left(A_{\Lambda }\left(\left\{N_{\mathrm{j}}\left(x_{ij}, {\widehat{y}}_{ij}\right), w_j\right\}\right),{\widehat{y}}_i\right),$$

(17)

where the ML model $f\left(X; \theta \right)$ produces insights or predictions that inform the multi-criteria aggregation process $\left(A_{\Lambda }\right)$, MCDM aggregation yields structured decision outcomes $D\left(A_i\right)= A_{\Lambda }\left(\left\{N_j\left(x_{ij}, {\widehat{y}}_{ij}\right), w_j\right\} \mathrm{f}\mathrm{o}\mathrm{r} \mathrm{a}\mathrm{l}\mathrm{l} j\right)$ that can also regulate or be fused with ML predictions, and the synthesis function $\Omega \left(·\right)$ combines both perspectives to produce a comprehensive decision score $\mathrm{S}\left(A_i\right) = \Omega \left(D\left(A_i\right), {\widehat{y}}_i\right)$. See

Table 4. Summary of the stages of integration mode MCDM + ML.

Stage	Mathematical Representation	Equation	Purpose
Generate ML outputs	$\widehat{y}=f\left(X; \theta \right)$	(18)	Extract patterns, predictions, or importances from data that informs or supports the multi-criteria evaluation process.
Perform multi-criteria evaluation	$D\left(A_i\right)=A_{\Lambda }\left(\left\{N_{\mathrm{j}}\left(x_{ij}, {\widehat{y}}_{ij}\right), w_j\right\}\right)$	(19)	Provide structured multi-criteria evaluation of alternatives considering normalized data and ML-derived insights.
Implement coupled optimization	$w_j\left(\mathrm{n}\mathrm{e}\mathrm{w}\right)=w_j\left(\mathrm{o}\mathrm{l}\mathrm{d}\right)+\eta \times {\displaystyle \frac{\partial L}{\partial w_j}}$	(20)	Implement coupled optimization to update each other iteratively: (a) Criterion weights are adjusted based on ML optimization; (b) MCDM results regularize or constrain ML training by embedding decision preferences into learning.
	$\mathrm{m}\mathrm{i}\mathrm{n}\mathrm{i}\mathrm{m}\mathrm{i}\mathrm{z}\mathrm{e} \mathrm{o}\mathrm{v}\mathrm{e}\mathrm{r} \theta : \left[L\left(\widehat{y}, Y\right)+\lambda \times P\left(D\left(A_i\right)\right)\right]$	(21)
Synthesize	$S\left(A_i\right)=\Omega \left(D\left(A_i\right), {\widehat{y}}_i\right)$	(22)	Integrate both MCDM and ML outputs into one final score or decision.
Conduct iterative refinement (optional)	$\mathrm{u}\mathrm{p}\mathrm{d}\mathrm{a}\mathrm{t}\mathrm{e} \left(\theta , w_j\right)\leftarrow \mathrm{a}\mathrm{r}\mathrm{g}\mathrm{m}\mathrm{i}\mathrm{n}\left[L\left(\widehat{y}, Y\right)+\lambda \times \left(1-\rho \left(D, \widehat{y}\right)\right)\right]$	(23)	Find both ML parameters and MCDM weights that jointly minimize prediction error while maximizing alignment between ML outputs and MCDM-based evaluations. This enables mutual adaptation and balance.

for a summary of the stages of the integration mode.

Category 4: Comparing the performances of ML algorithms and MCDM methods (ML vs. MCDM)

This category involves evaluating and comparing the performance of ML algorithms and MCDM methods across different decision-making contexts. Given their distinct strengths—ML in handling large-scale data and uncovering patterns, and MCDM in structuring complex problems and incorporating expert judgment—such comparisons aim to determine which approach is more effective for a given problem setting. Fernández et al. [48] compared support vector regression (SVR) and VIKOR combined with entropy weighting for multi-material extrusion processes, while Almansi et al. [49] compared multilayer perceptron (MLP) and AHP for hospital site suitability mapping in Malacca, Malaysia. The general formulation of the integration mode ML vs. MCDM can be concisely expressed as:

If observed target values or true labels are available,

$$M^{*}=argmax \Omega \left(Eva{l}^{\mathrm{M}\mathrm{C}\mathrm{D}\mathrm{M}}\left(A_{\Lambda }\left(\left\{N_j\left(X\right), w_j\right\}\right), Y\right), Eva{l}^{\mathrm{ML}}\left(f\left(X; \theta \right), Y\right), \rho \left(D\left(A\right),\widehat{y}\right)\right)$$

(24)

If observed target values or true labels are not available,

$$M^{*}=argmax \Omega \left(Eva{l}^{\mathrm{M}\mathrm{C}\mathrm{D}\mathrm{M}}\left(A_{\Lambda }\left(\left\{N_j\left(X\right), w_j\right\}\right)\right), Eva{l}^{\mathrm{ML}}\left(f\left(X; \theta \right)\right), \rho \left(D\left(A\right),\widehat{y}\right)\right)$$

(25)

This formulation shows:

• Both methods are applied independently on the same data.
• Each is evaluated separately via its own metric.
• Their results are compared through performance, consistency, or correlation measures.
• The best-performing or most consistent approach is selected via the synthesis function $\mathit{\Omega }$

See

Table 5. Summary of the stages of integration mode ML vs. MCDM.

Stage	Mathematical Representation	Equation	Purpose
Compute MCDM results	$D\left(A_i\right)=A_{\Lambda }\left(\left\{N_j\left(x_{ij}\right), w_j\right\}\right)$	See (8)	Compute overall decision scores, ranks, or classes for each alternative from normalized multi-criteria data ($r_{ij}$) and weights ($w_j$).
Generate ML outputs	${\widehat{y}}_i=f\left(X_i; \theta \right)$	See (18)	Produce predictive ML outputs (e.g., scores, probabilities, class labels, or estimates) for alternative Ai.
Evaluate performance of both models separately	$E^{\mathrm{M}\mathrm{C}\mathrm{D}\mathrm{M}}=\{\begin{array}{cc}{Eval}^{\mathrm{M}\mathrm{C}\mathrm{D}\mathrm{M}}\left(D\left(A\right),Y\right),& \mathrm{i}\mathrm{f} Y \mathrm{a}\mathrm{v}\mathrm{a}\mathrm{i}\mathrm{l}\mathrm{a}\mathrm{b}\mathrm{l}\mathrm{e} \\ {Eval}^{\mathrm{M}\mathrm{C}\mathrm{D}\mathrm{M}}\left(D\left(A\right)\right),& \mathrm{o}\mathrm{t}\mathrm{h}\mathrm{e}\mathrm{r}\mathrm{w}\mathrm{i}\mathrm{s}\mathrm{e}\end{array}$	(26)	Compute performance indicators for each approach separately using appropriate evaluation functions (e.g., R2, RMSE, accuracy, consistency, correlation).
	$E^{\mathrm{M}\mathrm{L}}=\{\begin{array}{cc}{Eval}^{\mathrm{M}\mathrm{L}}\left(\widehat{y},Y\right),& \mathrm{i}\mathrm{f} Y \mathrm{a}\mathrm{v}\mathrm{a}\mathrm{i}\mathrm{l}\mathrm{a}\mathrm{b}\mathrm{l}\mathrm{e} \\ {Eval}^{\mathrm{M}\mathrm{L}}\left(\widehat{y}\right),& \mathrm{o}\mathrm{t}\mathrm{h}\mathrm{e}\mathrm{r}\mathrm{w}\mathrm{i}\mathrm{s}\mathrm{e}\end{array}$	(27)
Compare results through performance or correlation measures	$\Delta =\rho \left(D\left(A\right),\widehat{y}\right)$	(28)	Assess alignment between ML and MCDM results—or between either and the ground truth— using comparison or agreement metrics such as ranking consistency, value proximity, correlation or predictive accuracy.
Establish comparative evaluation framework	$\Pi =\left\{\left(E^{\mathrm{M}\mathrm{C}\mathrm{D}\mathrm{M}}, E^{\mathrm{M}\mathrm{L}}, \rho \left(D\left(A\right),\widehat{y}\right)\right)\right\}$	(29)	Construct a performance comparison framework to identify which approach yields higher accuracy, the degree of decision agreement, and each method’s contextual suitability.
Select the superior approach	$M^{*}=argmax \Omega \left(E^{\mathrm{M}\mathrm{L}}, E^{\mathrm{M}\mathrm{C}\mathrm{D}\mathrm{M}}, \rho \right)$	(30)	Identify the best-performing or most consistent approach via the synthesis function.

for a summary of the stages of the integration mode

Category 5: Comparison of ML algorithm performances using MCDM methods (MCDM[ML vs. ML])

This category compares the performance of ML algorithms using MCDM methods. This approach allows ML algorithms to be evaluated and ranked according to multiple performance criteria. Various criteria, such as accuracy, precision, recall, F1 score, and processing time, are usually taken into account when evaluating the performance of ML algorithms. When there are trade-offs among these criteria, MCDM methods enable a comprehensive comparison of algorithms. Tashakkori et al. [50] applied the TOPSIS method to compare various ML algorithms such as Support Vector Machine (SVM), Decision Tree (DT), Gaussian Process (GP), Multilayer Perceptron (MLP), ensemble models Random Forest (RF) and Bagging, for neonatal mortality prediction, while Mekouar [51] proposed AHP-based MCDM framework for selecting the most optimal classifier from algorithms like Random Forest, Stochastic Gradient Descent (SGD), Multilayer Perceptron (MLP), Decision tree (DT), Naive Bayes (NB), Logistic regression (LG), K nearest neighbors (KNN). The general formulation of the integration mode MCDM[ML vs. ML] can be concisely expressed as

$$M^{*}=argma{x}_i \delta \left(rank\left(A_{\Lambda }\left(\left\{N_j\left(p_{ij}\right), W^{\mathrm{M}\mathrm{C}\mathrm{D}\mathrm{M}}\left(\left\{r_{ij}\right\} \mathrm{o}\mathrm{r} \mathrm{e}\mathrm{x}\mathrm{p}\mathrm{e}\mathrm{r}\mathrm{t} \mathrm{i}\mathrm{n}\mathrm{p}\mathrm{u}\mathrm{t}\mathrm{s}\right)\right\}\right)\right), \mathrm{c}\mathrm{o}\mathrm{n}\mathrm{s}\mathrm{t}\mathrm{r}\mathrm{a}\mathrm{i}\mathrm{n}\mathrm{t}\mathrm{s}\right)$$

(31)

where $M^{*}$ represents the optimal ML model selected from a set of candidates $\left\{A_i\right\}$, $\delta \left(\cdot \right)$ is the decision or selection operator that identifies the most preferred alternative according to a specified rule, and the rank operator orders alternatives based on their aggregated evaluation scores $A_{\Lambda }$, and $W^{\mathrm{M}\mathrm{C}\mathrm{D}\mathrm{M}}\left(\cdot \right)$ provides the criterion importance weights, derived either from expert judgments $r_{ij}$ or other multi-criteria weighting procedures. See

Table 6. Summary of the stages of integration mode MCDM[ML vs. ML].

Stage	Mathematical Representation	Equation	Purpose
Construct performance evaluation matrix	$E=\left[p_{ij}\right]for i=1, \dots , m, j=1, \dots , n$	(32)	Capture each ML algorithm’s $\left(A_i\right)$ quantitative performance $p_{ij}$ under multiple criteria $\left\{C_j\right\}.$
Normalize or scale the performance values	$r_{ij}=N_j\left(p_{ij}\right)$	(33)	Normalize the $p_{ij}$ values to ensure comparability across different performance scales and units (i.e., criteria).
Determine criterion weights	$w_j=W^{\mathrm{M}\mathrm{C}\mathrm{D}\mathrm{M}}\left(\left\{r_{ij}\right\} \mathrm{o}\mathrm{r} \mathrm{e}\mathrm{x}\mathrm{p}\mathrm{e}\mathrm{r}\mathrm{t} \mathrm{i}\mathrm{n}\mathrm{p}\mathrm{u}\mathrm{t}\mathrm{s}\right); {\Sigma }_{\mathrm{j}} w_j=1 \mathrm{a}\mathrm{n}\mathrm{d} w_j \ge 0$	(34)	Compute or obtain the relative importance of each performance criterion (e.g., accuracy vs. computation time) using a weighting method such as expert-based AHP, data-driven entropy measure, or a hybrid approach.
Aggregate normalized values using an MCDM operator	$D\left(A_{\mathrm{i}}\right)=A_{\Lambda }\left(\left\{r_{ij} , w_j \right\}\right)$	See (8)	Combine normalized performance values and weights to compute an overall score for each ML algorithm.
Rank the ML algorithms	$rank{(A}_i)=\mathrm{p}\mathrm{o}\mathrm{s}\mathrm{i}\mathrm{t}\mathrm{i}\mathrm{o}\mathrm{n} \mathrm{o}\mathrm{f} A_i \mathrm{w}\mathrm{h}\mathrm{e}\mathrm{n} \mathrm{s}\mathrm{o}\mathrm{r}\mathrm{t}\mathrm{i}\mathrm{n}\mathrm{g} \mathrm{b}\mathrm{y} D\left(A_i\right)$	See (9)	Order ML algorithms from best to worst according to their aggregated scores.
Select the most suitable ML algorithm	$M^{*}=argma{x}_i \delta \left(\left\{rank\left(A_i\right),\mathrm{c}\mathrm{o}\mathrm{n}\mathrm{s}\mathrm{t}\mathrm{r}\mathrm{a}\mathrm{i}\mathrm{n}\mathrm{t}\mathrm{s}\right\}\right)$	(35)	Identify the most appropriate ML algorithm given multi-criteria performance trade-offs. $\delta$ may choose top-k, those meeting thresholds, or Pareto-optimal subset.
Perform sensitivity, robustness, and statistical validation (optional)	Weight sensitivity: ${\displaystyle \frac{\partial rank}{\partial w}}$	(36)	Evaluate stability and robustness of results by varying weights, propagating performance uncertainty, bootstrapping confidence intervals for scores and ranks, and applying statistical tests (e.g., t-test, Wilcoxon, Friedman) to verify significant performance differences.
	if $p_{ij}\sim U$, ${\mu }_i= E\left[D\left(A_i\right)\right], CI\left(D\left(A_i\right)\right)$	(37)
	Bootstrap: $CI\left(D^{\left(b\right)}\left(A_i\right)\right)$	(38)

for a summary of the stages of the integration mode.

In response to Research Question 1, the analysis identified five distinct modes of integration between ML algorithms and MCDM methods, distinguished by purpose and methodological approach. This categorization provides a comprehensive framework for understanding how MCDM methods and ML algorithms can be integrated in decision-making. By combining these approaches, more effective decision-making and prediction processes can be achieved, both theoretically and practically.

3.3. Criteria Identification

To address Research Question 2, a set of fourteen criteria was introduced to evaluate integration modes across diverse decision-making problems, thereby establishing benchmarks for systematic assessment. By applying these criteria to the reviewed studies, we aim to develop a more informed and structured framework that generates valuable insights into the applications of ML–MCDM integration. Moreover, this framework is intended to guide future research by providing concrete recommendations and identifying promising directions for further exploration.

To achieve this, the criteria were derived from an extensive literature review and refined through expert input, with their definitions carefully determined to ensure conceptual clarity. The selection process was deliberately designed to align with the most current and relevant research, thereby providing a rigorous and well-founded methodological foundation. The extracted criteria then underwent a meticulous content analysis, which involved a detailed examination of their definitions and meanings. During this process, closely related criteria were identified based on their conceptual overlap and thematic similarities. For example, criteria such as “Adaptability,” “Scalability,” “Responsiveness,” “Adjustability,” “Expandability,” and “Compatibility” were found to share a conceptual relationship with “Flexibility.” In this way, conceptually related items were consolidated under common headings wherever possible. Nevertheless, complete independence among the 14 identified criteria cannot be fully ensured, and some degree of conceptual overlap may remain. This does not present a methodological concern for the subsequent analyses, as the correspondence analysis employed here is specifically designed to capture and represent associations among categories within a low-dimensional space. In other words, CA does not assume independence among (row or column) categories, and it is acceptable for integration modes or criteria to exhibit internal correlations. The complete set of criteria, their closely related criteria, and the corresponding source citations are provided in Appendix A (

Table A1. Grouping of related criteria under representative headings.

Criteria	Closely Related Criteria
Novelty [53,54]	Creativity [52,53]	Innovative [130]	Authenticity [53,131]	Specificity [131]	Originality [132]
Complexity [55,130,133,134]	Simplicity [53]
Validation [56]
Subjectivity [57]	Reasonable [135]	Rationality [136]
Knowledge Base [58,137]
Effectiveness [59,138]	Accuracy [130,139,140,141]
Efficiency [3,60,133]	Computational time [139]
Applicability [61,142]	Generalizability [143,144]
Flexibility [55]	Adaptability [145,146,147]	Scalability [36,139,148]	Responsiveness [149,150,151]	Adjustability [152]	Expandability [134]	Compatibility [153]
Consistency [62,154]	Stability [133]	Reliability [141,143]	Maintainability [155]	Robustness [130,156]
Automation [63,145,157]	Automated [139,141,148]	Autonomous [137]
Sequential Processing
Dynamic Nature [64]
Explainability [65,66]	Interpretability [66,133,139]	Understandability [133]	Comprehensibility [158]

).

The evaluation criteria and their detailed descriptions are presented below.

Novelty (C1): An approach is considered “novel” if it introduces a new and original idea, method, or feature. Novel approaches typically differ from existing models or methods and provide innovative ways to address specific problems. Accordingly, the novelty of an integration mode is closely linked to its degree of innovation and originality [52,53,54]. This study examines the extent to which the integration of MCDM methods and ML algorithms contributes to novel methodological developments.

Complexity (C2): A decision problem is considered complex when it involves multiple components, interdependencies, a large number of alternatives, and/or conflicting objectives, and can manifest at different levels depending on the context [55]. In this study, complexity refers to the challenges of understanding, analyzing, and managing a problem or system arising from numerous variables, interdependencies, operational steps, and data heterogeneity. The research examines whether integrating MCDM and ML approaches can help manage this complexity.

Validation (C3): Validation is the process of assessing the extent to which a method produces accurate, reliable, and meaningful results in relation to its intended purpose [56]. In this study, the validation criterion is defined as the systematic process of assessing and comparing the performance and outcomes of MCDM and/or ML approaches.

Subjectivity (C4): In decision-making, subjectivity refers to the variability in judgments that arises from individual perspectives, interpretations, and biases, rather than from objective data or knowledge gaps [57]. In other words, subjectivity highlights the human element of decision-making. It reflects the personal influence of decision-makers’ backgrounds, expertise, and values on the evaluation process. This study examines whether integrating MCDM methods with ML algorithms reduces subjectivity in decision-making.

Knowledge Base (C5): Knowledge-based approaches typically rely on the insights, principles, and intuitions accumulated by experts or professionals over time to make decisions or solve problems [58]. In this study, this criterion indicates the extent to which integrating MCDM methods and ML algorithms requires expert knowledge.

Effectiveness (C6): Effectiveness denotes the capacity of a method to generate accurate predictions, classifications, or decisions from input data [59]. In this study, the effectiveness criterion refers to the ability of integrating MCDM and ML approaches to successfully and accurately perform the specified task.

Efficiency (C7): Efficiency refers to completing a task or process using resources as effectively as possible. In this study, the efficiency criterion evaluates how well the integration of MCDM and ML approaches performs regarding resource utilization, speed, and computational cost while achieving its objectives [3,60].

Applicability (C8): This criterion essentially captures whether a method is suitable, practical, and relevant for addressing a particular decision problem, given the characteristics of the data, criteria, decision context, and objectives. In this study, the applicability of an integration approach reflects its practical usability and the extent to which it can effectively address real-world decision problems [61].

Flexibility (C9): Flexibility refers to a method’s or model’s ability to adapt or adjust under various conditions, data types, or tasks. A flexible method can accommodate variations or different requirements and function effectively across diverse datasets or contexts [55]. Flexibility, in this study, refers to the capacity of a model or method to be modified or adapted when needed and to respond effectively to unexpected changes.

Consistency (C10): Consistency refers to a method’s ability to produce results of similar quality and performance under different conditions, datasets, or timeframes. A consistent method is essential for obtaining repeatable outcomes and making reliable decisions or predictions [62]. In this study, consistency reflects the extent to which the integration of MCDM methods and ML algorithms yields results of comparable quality and repeatability across varying conditions, datasets, or decision contexts.

Automation (C11): Automation refers to the full or partial automation of a method’s operation or functionality. It enables the method to perform specific tasks or processes without human intervention [63]. In this study, automation reflects the extent to which integrating MCDM methods and ML algorithms facilitates the automatic execution of decision-related tasks.

Sequential Processing (C12): Sequential Processing refers to the stepwise use of MCDM methods and ML algorithms, where one method’s output serves as the input or basis for the next in solving a decision problem. This allows for maintaining methodological coherence and enhancing traceability of the decision-making. In this study, the sequential processing criterion examines whether the methods/algorithms are applied in a defined sequence rather than simultaneously, allowing each to contribute at different stages of the decision-making process and thereby complement one another.

Dynamic Nature (C13): Being dynamic refers to a method’s ability to handle evolving conditions, data, or requirements over time. Dynamic models are designed to handle situations where key factors or parameters are not fixed and may change, allowing them to adapt and model these time-dependent variations effectively [64]. This study examines whether the integration of MCDM methods and ML algorithms successfully accounts for the dynamic nature of the decision problem being modeled.

Explainability (C14): Explainability reduces a model’s black-box nature by making its internal workings and decisions more transparent and understandable [65,66]. Enhancing a model’s explainability helps users understand the reasoning behind its predictions or decisions, thereby fostering greater trust in the model. In this study, the explainability criterion examines whether integrating MCDM methods and ML algorithms enhances the interpretability and transparency of the decision-making process.

A brief description and a guiding question for each criterion are provided in Appendix A to support consistent expert evaluation of ML–MCDM integration modes.

3.4. Evaluation

After defining the categories and evaluation criteria, the next step is to determine how well each category (i.e., integration mode) meets the specified criteria. This evaluation provides a structured understanding of the distinctive characteristics and comparative strengths of different integration modes. However, evaluating categories directly against criteria without an in-depth review of the source articles may lead to oversimplified or even misleading associations, as contextual nuances and methodological details would likely be overlooked. Therefore, a thorough examination of articles that apply different integration modes is necessary to provide essential evidence for evaluation. This approach establishes a clear link between the conceptual framework (categories and criteria) and empirical evidence (the reviewed articles), thereby enhancing methodological rigor and credibility. To ensure this, a three-stage procedure was employed, with its details elaborated in the following paragraphs:

• Mapping: Drawing on expert input, 208 relevant articles were carefully reviewed and systematically mapped to the corresponding categories. This step ensured that every category was grounded in representative studies.
• Scoring: The same set of articles was evaluated by experts according to the extent to which the approach proposed in each article satisfies each evaluation criterion. This stage translated qualitative insights from the literature into structured, comparable scores.
• Synthesis: The outputs of the mapping and scoring stages were combined through matrix multiplication to determine how well each category satisfies the evaluation criteria, ensuring that the findings are firmly grounded in evidence derived from prior research.

Six qualified experts, including the authors, recognized for their substantial experience and knowledge in decision-making and ML, participated in the evaluation process. In decision-making studies, there is no strict, universally accepted number of experts; emphasis is placed on the quality of judgments rather than their quantity [67,68]. Accordingly, a sample of six experts is considered adequate and reasonable for this study.

Table 7. Expert profiles.

Expert	Affiliation Type	Field of Specialization	Years of Experience	Current Position/Title
E1	Academic	Multi-Criteria Decision-Making, Marketing Analytics	25	Associate Professor
E2	Academic	Multi-Criteria Decision-Making, Human Resources Analytics	25	Associate Professor
E3	Academic	Industrial Engineering, Multi-Criteria Decision-Making	22	Associate Professor
E4	Academic	Industrial Engineering, Decision Sciences, Machine Learning	9	Researcher
E5	Industry Professional	Artificial Intelligence, Deep Learning, Machine Learning	8	Research and Development Center Manager
E6	Industry Professional	Strategic Decision-Making, Artificial Intelligence	15	Technology Consultant

presents detailed information about their fields of specialization and professional experience, demonstrating the diversity and relevance of expertise represented in the study. This diversity helped mitigate individual biases and subjectivity.

Given the large number of articles to be evaluated, the task was challenging and potentially tedious, increasing the risk of errors in assessment. To improve both judgment accuracy and efficiency, each expert was assigned a subset of articles, consistent with approaches in previous research [67,69]. All experts were assumed to have equal credibility, and consistency was ensured by assigning each article to three experts.

Accordingly, the reviewed articles were first classified into one of five distinct categories based on the integration mode employed, and experts were provided with a brief description (see Section 3.2) to ensure consistent understanding and application.

Table 8. Mapping of articles to categories.

Author(s)	Year	ML → MCDM	MCDM → ML	MCDM + ML	ML vs. MCDM	MCDM[ML vs. ML]
[41]	2024	✓
[50]	2024					✓
[70]	2025				✓
[71]	2025	✓
[72]	2024		✓
[73]	2024				✓
[74]	2023			✓
[75]	2023					✓
[76]	2023		✓
[77]	2022				✓
[78]	2021	✓
[79]	2021		✓
[80]	2020					✓
[81]	2019			✓
[82]	2018			✓
…	…	…	…	…	…	…

presents the results for a subset of the 208 articles, while the complete classification is provided in Appendix B.

Subsequently, experts examined the extent to which the identified criteria were addressed in the reviewed articles to highlight their contributions and outcomes.

To ensure consistency in expert evaluations, a standardized form was provided to all experts (see Appendix C). The form includes each article’s title, a brief description of the relevant criterion, the evaluation question, and brief guidance on which parts of the article experts could focus on during their review. Experts evaluated each article using the following scale in response to the evaluation questions, which are provided in Appendix A (

Table A2. Criteria, descriptions, and questions for expert evaluation.

Criterion	Description	Question
Novelty	The integration mode introduces a new or original idea, method, or feature beyond what the MCDM method, the ML algorithm, or even both can achieve individually.	To what extent does the integration mode proposed in the article satisfy the criterion of Novelty as described?
Complexity	The integration mode improves the ability to understand, analyze, and manage complex decision problems—characterized by numerous variables, interdependencies, operational steps, and data heterogeneity—beyond what the MCDM method, the ML algorithm, or even both can achieve individually.	To what extent does the integration mode proposed in the article satisfy the criterion of Complexity as described?
Validation	The integration mode proposes a systematic approach for assessing and comparing performance and outcomes—such as the accuracy, reliability, and meaningfulness of results—beyond what the MCDM method, the ML algorithm, or even both can achieve individually.	To what extent does the integration mode proposed in the article satisfy the criterion of Validation as described?
Subjectivity	The integration mode helps reduce the influence of individual perspectives, interpretations, and biases in decision-making—enhancing objectivity beyond what the MCDM method, ML algorithm, or even both can achieve individually.	To what extent does the integration mode proposed in the article satisfy the criterion of Subjectivity as described?
Knowledge Base	The integration mode enhances the contribution of expert knowledge, experience, and domain insights to the decision-making process beyond what the MCDM method, the ML algorithm, or even both can achieve individually.	To what extent does the integration mode proposed in the article satisfy the criterion of Knowledge Base as described?
Effectiveness	The integration mode enhances the ability to generate accurate, reliable, and successful outcomes—such as predictions, classifications, or decisions—beyond what the MCDM method, ML algorithm, or even both can achieve individually.	To what extent does the integration mode proposed in the article satisfy the criterion of Effectiveness as described?
Efficiency	The integration mode enhances resource utilization and computational speed while reducing computational cost beyond what the MCDM method, ML algorithm, or even both can achieve individually.	To what extent does the integration mode proposed in the article satisfy the criterion of Efficiency as described?
Applicability	The integration mode introduces a practical and context-sensitive approach that appropriately addresses real-world decision problems beyond what the MCDM method, the ML algorithm, or even both can achieve individually.	To what extent does the integration mode proposed in the article satisfy the criterion of Applicability as described?
Flexibility	The integration mode introduces an approach that can adapt and scale to diverse conditions, data types, and decision tasks, effectively managing large-scale challenges, variations, and unexpected changes beyond what the MCDM method, the ML algorithm, or even both individually can achieve.	To what extent does the integration mode proposed in the article satisfy the criterion of Flexibility as described?
Consistency	The integration mode introduces a robust approach that produces results that are repeatable and of comparable quality across varying conditions, datasets, and decision contexts, beyond what the MCDM method, the ML algorithm, or even both individually can achieve.	To what extent does the integration mode proposed in the article satisfy the criterion of Consistency as described?
Automation	The integration mode introduces an approach that allows decision-related tasks to be executed partially or fully without human intervention—beyond what the MCDM method, the ML algorithm, or even both individually can achieve.	To what extent does the integration mode proposed in the article satisfy the criterion of Automation as described?
Sequential Processing	The integration mode introduces a stepwise approach in which the output of one method serves as the input for the next, maintaining methodological coherence, enhancing traceability, and addressing the limitations of individual methods that go beyond what the MCDM method, the ML algorithm, or even both individually can achieve.	To what extent does the integration mode proposed in the article satisfy the criterion of Sequential Processing as described?
Dynamic Nature	The integration mode introduces an approach capable of handling evolving conditions, data, and requirements over time—allowing adaptation to time-dependent variations and changes beyond what the MCDM method, the ML algorithm, or even both individually can achieve.	To what extent does the integration mode proposed in the article satisfy the criterion of Dynamic Nature as described?
Explainability	The integration mode enhances the interpretability and transparency of the decision-making process—making the model’s reasoning behind its predictions or decisions more understandable beyond what the MCDM method, the ML algorithm, or even both individually can achieve.	To what extent does the integration mode proposed in the article satisfy the criterion of Explainability as described?

):

0: Does not satisfy the criterion at all

1: Partially satisfies the criterion

2: Fully satisfies the criterion

Experts were also required to provide supporting evidence by citing keywords, sentences, or sections, accompanied by a brief descriptive note justifying the score. They were given three months to complete their evaluations. An example evaluation is provided in Appendix C (

Table A5. Example evaluation.

Article No. 114	A weighted Bonferroni-OWA operator-based cumulative belief degree approach to personnel selection based on automated video interview assessment data.
C12. Sequential Processing	The integration mode introduces a stepwise approach in which the output of one method serves as the input for the next, maintaining methodological coherence, enhancing traceability, and addressing the limitations of individual methods that go beyond what the MCDM method, the ML algorithm, or even both individually can achieve.
Q12.	To what extent does the approach proposed in the article satisfy the criterion of Sequential Processing as described above?
Explanation:	Please pay particular attention to the sections that most directly describe the integration approach, including its underlying rationale, procedural structure, implementation details, contributions, limitations, and resulting outcomes.
Score:	0 ☐     1 ☐     2 ☒
Evidence:	Abstract: “In order to address these issues, an effective and practical approach is proposed that is able to transform, weight, combine, and rank automated AVI assessments obtained through AI technologies and machine learning.” Introduction: “The proposed approach is a prime example of the integration of learning-based techniques with MCDM techniques. This is the first study that transforms and aggregates automated AVI assessments in a multi-criteria environment for personnel selection.” Section 4. Proposed Approach: Steps of the proposed approach and Figure 1.

).

After completing the evaluation process, scores were compared across experts to assess consistency. The standard deviations among the experts’ scores were first examined. In cases of high deviations, experts whose ratings differed from the group were asked to review and, if necessary, revise their evaluations. After the revisions, the standard deviations ranged between 0 and 1.155. Among the 2912 comparisons, only 15 exhibited the maximum deviation value (1.155), indicating an acceptable level of consistency.

Next, interrater reliability was measured using the Intraclass Correlation Coefficient (ICC). ICC was chosen for its suitability in assessing agreement among more than two raters for both continuous and ordinal data, and for accounting for variance components [83]. A two-way mixed model with absolute agreement was employed for the calculations.

The average-measure ICC results, along with their 95% confidence intervals, were reported separately for the two expert groups and are presented in Appendix D. Except for the Effectiveness criterion, all average-measure ICC values were 0.815 or higher for Group 1 and 0.805 or higher for Group 2. According to Koo and Li [84], ICC values below 0.5, between 0.5–0.75, 0.75–0.9, and above 0.9 indicate poor, moderate, good, and excellent reliability, respectively. Accordingly, the reliability levels observed in this study range from good to excellent. Overall, interrater reliability was high across all criteria, except Effectiveness, which showed lower reliability (0.629 in Group 1 and 0.651 in Group 2). This was due to the frequent use of the maximum score (“2”), which made the criterion sensitive to small differences.

The final evaluation for each criterion was determined by taking the median of the revised scores from all experts, resulting in a reliable and representative set of scores for the assessed criteria.

Table 9. Evaluation of articles against criteria.

Author(s)	Year	C1	C2	C3	C4	C5	C6	C7	C8	C9	C10	C11	C12	C13	C14
[41]	2024	0	2	0	2	1	2	1	1	1	0	1	2	0	2
[50]	2024	0	1	2	0	1	2	0	1	1	0	1	0	0	1
[70]	2025	0	1	2	0	1	2	1	1	0	0	0	0	0	0
[71]	2025	2	2	0	0	1	2	0	2	2	1	0	2	2	1
[72]	2024	1	2	0	2	2	2	0	2	2	1	1	2	0	1
[73]	2024	0	0	2	2	2	2	1	2	1	0	0	0	0	0
[74]	2023	1	2	0	2	1	2	0	1	2	1	1	0	0	0
[75]	2023	0	1	2	0	1	1	0	1	1	0	1	0	0	1
[76]	2023	0	2	0	0	1	2	1	2	2	1	1	2	1	1
[77]	2022	0	0	2	1	1	2	0	2	0	0	0	0	0	0
[78]	2021	0	2	1	0	2	2	2	2	1	2	1	2	1	1
[79]	2021	0	0	0	0	2	2	0	1	1	1	1	2	0	1
[80]	2020	0	1	2	0	1	2	1	2	1	1	1	0	0	1
[81]	2019	0	0	1	2	2	2	0	2	2	1	1	1	2	1
[82]	2018	2	1	0	0	2	2	0	1	2	2	1	0	0	0
…	…	…	…	…	…	…	…	…	…	…	…	…	…	…	…

presents the results for a subset of the 208 articles.

Following the mapping and scoring stages, the results presented in Table 8 and Table 9 were synthesized to uncover the relationships between categories and criteria, providing a structured view of how different forms of integration perform against the established evaluation criteria. Since the list of articles is common to both tables (article × category and article × criteria), a matrix multiplication approach was employed to link categories with criteria (see Figure 3). Before this, the article–category table, which classifies articles into five integration modes, was transposed.

The resulting category–criteria table is presented in

Table 10. Evaluations of categories against criteria.

Integration Mode	C1	C2	C3	C4	C5	C6	C7	C8	C9	C10	C11	C12	C13	C14	# of Articles
ML → MCDM	27	99	13	62	86	127	64	105	67	59	68	126	23	50	64
MCDM → ML	13	78	11	43	87	110	35	100	72	46	63	109	13	39	56
MCDM + ML	18	18	3	16	27	36	10	29	33	12	24	8	11	11	18
ML vs. MCDM	5	23	71	34	50	80	22	76	14	19	15	12	3	7	40
MCDM[ML vs. ML]	6	61	77	14	52	76	24	62	49	40	27	3	3	38	39

, offering a comprehensive overview of how each category satisfies the established evaluation criteria. The final column indicates the total number of articles classified under each category.

It is worth noting that categories with more articles may appear disproportionately dominant with respect to the criteria, which could lead to misleading interpretations. To ensure a more reliable comparison and improve the interpretability of the categories with respect to the criteria, normalization was applied to the results in Table 10. The normalization formula is defined as follows:

$$X^{\prime }={\displaystyle \frac{X}{\sum X_{category}}}\times 100$$

(39)

where $X$ is the original value for a given category-criterion pair, $\sum X_{category}$ represents the total number of articles within that category, $X^{\prime }$ is the normalized value.

For example, for Novelty (C1) in the ML → MCDM category, the original value 27 was divided by the total number of articles in that category (refer to the last column of Table 10), yielding $27/64\times 100=42.2$. This normalization procedure was applied to all values in the category-criteria table, with the results presented in

Table 11. Normalized evaluations of categories against criteria.

Integration Mode	C1	C2	C3	C4	C5	C6	C7	C8	C9	C10	C11	C12	C13	C14
ML → MCDM	42.2	154.7	20.3	96.9	134.4	198.4	100.0	164.1	104.7	92.2	106.3	196.9	35.9	78.1
MCDM → ML	23.2	139.3	19.6	76.8	155.4	196.4	62.5	178.6	128.6	82.1	112.5	194.6	23.2	69.6
MCDM + ML	100.0	100.0	16.7	88.9	150.0	200.0	55.6	161.1	183.3	66.7	133.3	44.4	61.1	61.1
ML vs. MCDM	12.5	57.5	177.5	85.0	125.0	200.0	55.0	190.0	35.0	47.5	37.5	30.0	7.5	17.5
MCDM[ML vs. ML]	15.4	156.4	197.4	35.9	133.3	194.9	61.5	159.0	125.6	102.6	69.2	7.7	7.7	97.4

. Each value in the table represents the relative contribution of a category with respect to a specific criterion, allowing for a more systematic comparison.

3.5. Framework Development

3.5.1. Correspondence Analysis

To reveal underlying patterns and aid in interpreting the distinguishing characteristics of different integration modes, Correspondence Analysis (CA) was applied as a dimensionality reduction and visualization technique. CA is a multivariate statistical technique used to explore relationships in categorical variables. It reduces complex data into a low-dimensional map, where rows and columns of a contingency table are represented as profile points. The profiles are obtained by dividing each cell frequency by its corresponding marginal total, forming conditional frequency distributions [85]. The relative positions and distances of these profile points, computed using chi-square distances, reveal similarities, associations, and underlying patterns in the data [86]. More precisely, points that are close to each other on the map indicate similar profiles, reflecting a strong associative pattern between the corresponding row (or column) categories.

In the joint map, the position of every point in one set is determined by all the points in the other set, and vice versa. Therefore, distances are most meaningful when comparing points within the same set (rows with rows or columns with columns), while proximity between row and column points only provides an approximate indication of association [87]. Usually, however, points from different sets tend to be positioned close together when the observed frequency is higher than expected, and farther apart when it is lower than expected [88]. For example, if a specific integration mode lies close to an evaluation criterion on the map (compared with the other modes), it suggests that the articles within that category exhibit similar patterns of association with that criterion or strongly satisfy it. Conversely, categories that are far from the criterion suggest a weaker relationship with it.

Each axis represents a latent dimension extracted through singular value decomposition, capturing the maximum possible variation (inertia) in the associations among categories [89]. The contribution of each point to a given dimension (principal axis) indicates its relative importance in defining that dimension. A point contributes strongly to the inertia of a principal axis either because it lies far from the origin (even with a small mass) or because it has a large mass despite being relatively close to the center [90]. Here, the mass of a row (or column) is its total in the correspondence matrix (see Appendix E), representing its relative weight in the data. Conversely, points that contribute little to the inertia of an axis are nearly identical to the average profile and therefore appear near the center of the map. This two-dimensional representation allows a visual identification of which integration modes align with which evaluation criteria, while the axes and clusters further highlight the most significant underlying associations in the data. Further details about the computational procedure and interpretation are provided in Appendix E.

The rest of this section provides the results of the CA and their interpretation. First, a test of independence between the rows (integration modes) and the columns (criteria) is performed. The chi-square statistic (${\chi }^2$ = 1284.369; df = 52; p < 0.001) confirms that nonrandom associations exist among the categories—i.e., there are statistically significant relationships in the contingency structure.

Table 12. Eigenvalues and percentages of inertia.

	F1	F2	F3	F4
Eigenvalue	0.117	0.044	0.023	0.003
Inertia (%)	62.307	23.581	12.292	1.820
Cumulative %	62.307	85.888	98.180	100

presents the eigenvalues, along with the explained and cumulative percentages of variance for each of the first four dimensions. The first two dimensions, F1 (62.31%) and F2 (23.58%), together account for 85.89% of the variance, capturing most of the variance in the data structure, whereas F3 and F4 explain less than 15%. A two-dimensional solution was deemed adequate for further analysis to compare the additional variance explained in relation to the increased complexity in interpreting the results.

Table 13. Overview column points (for symmetrical normalization).

Criterion	Score in Dimension		Contribution of
			Point to Inertia of Dimension		Dimension to Inertia of Point
	1	2	1	2	1	2	Total
Novelty	−0.415	−0.647	0.042	0.267	0.273	0.664	0.937
Complexity	−0.032	0.077	0.001	0.012	0.018	0.104	0.122
Validation	1.073	0.056	0.620	0.004	0.997	0.003	1.000
Subjectivity	−0.062	0.020	0.002	0.000	0.042	0.004	0.047
Knowledge Base	0.035	−0.018	0.001	0.001	0.154	0.041	0.195
Effectiveness	0.097	−0.002	0.012	0.000	0.582	0.000	0.582
Efficiency	−0.021	0.117	0.000	0.015	0.012	0.351	0.363
Applicability	0.122	0.035	0.016	0.003	0.440	0.036	0.476
Flexibility	−0.169	−0.264	0.021	0.133	0.224	0.546	0.770
Consistency	0.040	0.044	0.001	0.003	0.044	0.055	0.099
Automation	−0.258	−0.132	0.038	0.026	0.769	0.202	0.971
Sequential Process	−0.591	0.534	0.206	0.447	0.548	0.449	0.997
Dynamic Nature	−0.489	−0.445	0.040	0.088	0.508	0.421	0.929
Explainability	−0.035	−0.016	0.000	0.000	0.010	0.002	0.012

summarizes how each criterion (column points) contributes to and is represented by the two principal dimensions of the CA, using symmetrical normalization. The contribution of point to inertia of dimension indicates how much a criterion defines each axis, which is important for axis interpretation. Strong contributors to F1 include Validation (0.620), Sequential Processing (0.206), Dynamic Nature (0.040), and Novelty (0.042), while strong contributors to F2 include Sequential Processing (0.447), Novelty (0.267), Flexibility (0.133), and Dynamic Nature (0.088). Moreover, with respect to the principal coordinates, F1 mainly contrasts criteria, including Validation (high positive score), with Sequential Processing, Dynamic Nature, Novelty, and Automation (high negative scores), while F2 mainly contrasts Sequential Processing with Novelty, Dynamic Nature, and Flexibility. This suggests that F1 may represent a continuum from developmental aspects to evaluative characteristics, while F2 may represent a continuum from complementary aspects to evolutionary characteristics. The last column in Table 13 (total contribution of the dimensions to inertia of each point) indicates that Validation, Sequential Processing, Automation, Novelty, Flexibility, and Dynamic Nature are most influential and best represented in the two-dimensional solution.

The symmetric plot obtained from the CA is presented in Figure 4. In the plot, blue points represent modes (rows), whereas red points represent criteria (columns). According to the plot, F1 clearly separates ML → MCDM and MCDM → ML modes from the ML vs. MCDM and MCDM[ML vs. ML] modes, while F2 distinguishes MCDM + ML from the ML → MCDM and MCDM → ML modes.

3.5.2. Cluster Analysis

To complement and objectively confirm the spatial associations revealed by the CA, a subsequent clustering analysis was performed using the CA principal coordinates. While CA effectively maps the relative proximities between row and column profiles in a low-dimensional space, it does not formally define groups. Clustering is a necessary algorithmic step that assigns every row (or column) object to a specific non-overlapping cluster, providing a clearer picture of the underlying structures. Additionally, clustering validates the visual patterns observed on the CA map, ensuring that the proximities seen visually correspond to statistically meaningful groupings and thereby strengthening the robustness of the interpretation.

Following this rationale, the analysis proceeded through the following steps. First, the coordinates of the first two principal axes of the CA were used as input to the hierarchical clustering algorithm. Ward’s method with Euclidean distance was then applied to derive the cluster solution. The resulting dendrogram is shown in Figure 5. Based on the largest increases in the dendrogram, solutions with 2, 3, or 4 clusters appear most plausible.

Next, to objectively determine the number of clusters, both the Silhouette scores and variance decompositions were utilized (see

Table 14. Silhouette scores and variance decomposition.

	2 Cluster Solution	3 Cluster Solution	4 Cluster Solution
Silhouette Score	0.503	0.473	0.494
Variance:
Within-class	69.51%	34.26%	21.46%
Between-classes	30.49%	65.74%	78.54%
Total	100.00%	100.00%	100.00%

). The Silhouette scores indicate moderate clustering quality, with 2 clusters performing slightly better overall. However, the 2-cluster solution shows a high within-class variance (69.51%). On the other hand, the 4-cluster solution has a slightly lower Silhouette score and better within-class variance. However, one of the clusters fails to include an integration mode, limiting its interpretability. The 3-cluster solution with a reasonable Silhouette score (0.473) and between-class variance (65.74%) aligns with the number of visual groupings identified in the CA plot. Also, the heat map presented in Figure 6 supports the 3-cluster solution (explained below), as the resulting clusters correspond to coherent, interpretable groupings of integration modes and criteria, which outweigh the small drop in silhouette score.

Once the clusters were defined, they were formally characterized in terms of the CA dimensions. To facilitate clear interpretation and reporting of each cluster’s distinct profile, a heat map was generated. The heat map visualizes how each integration mode and criterion is positioned along the two principal dimensions obtained from CA. The colors represent the relative magnitude and direction of each point’s F1 and F2 coordinates, with red shades indicating strong negative contributions and green shades indicating strong positive contributions (see Figure 6).

The first principal dimension (F1) reveals a pronounced separation among three distinct groups of integration modes and evaluation criteria. Cluster 1, located far on the positive side of F1 (bright green), includes the integration modes ML vs. MCDM and MCDM[ML vs. ML], together with the criteria Validation, Applicability, and Effectiveness. On the opposite end of F1 (strong negative values—red shades), Cluster 2 contains the sequential modes ML → MCDM and MCDM → ML, which cluster tightly with the criteria Sequential Processing and weakly with Subjectivity. A third, moderately negative group—Cluster 3—contains the mode MCDM + ML and the criteria Novelty, Dynamic Nature, Automation, and Flexibility. While also on the negative side of F1, this cluster is clearly distinct from the second cluster, reflecting a different conceptual orientation centered on adaptability, innovation, and integrated automation.

The second principal dimension (F2) appears to capture a slightly different conceptual contrast. It distinguishes the sequential modes (ML → MCDM and MCDM → ML, shown in green) from the fully integrated mode MCDM + ML (shown in red). The first group aligns with Sequential Processing and, to a moderate degree, with Efficiency and Complexity. In contrast, MCDM + ML is more strongly linked to Novelty and Dynamic Nature and moderately and weakly to Flexibility and Automation.

The combination of (i) distinct color gradients, (ii) clear separation between the three major branches in the dendrogram, and (iii) strong within-cluster similarity versus between-cluster contrast provides robust visual evidence in favor of a three-cluster structure. These three color patterns correspond exactly to the three clusters described above.

3.5.3. Evidence-Based Framework

The results of both CA and cluster analysis demonstrate the presence of three higher-level clusters, within which certain primary and secondary criteria emerge notably, while others remain consistently shared across all categories. In other words, while the five modes of integration differ in purpose, differences based on criteria are evident only among the three higher-level clusters formed by these modes:

• Cluster 1 consisting of sequential approaches.
• Cluster 2 consisting of hybrid approaches.
• Cluster 3 consisting of performance comparison approaches.

Based on these findings, and in response to Research Question 3, an evidence-based framework has been established, as illustrated in Figure 7, where each cluster offers unique perspectives and insights into the associations within and between the categories. Sequential Processing, Validation, and Novelty serve as highly distinguishing criteria for the clusters, as explored in the CA. In contrast, Knowledge Base, Consistency, and Explainability are positioned equidistantly from all three clusters and lie close to the center, indicating that they are relatively neutral and shared across all integration modes.

The proposed framework has several important implications for both practitioners and researchers. It enables a comparative assessment of integration modes and serves as a guide for selecting the appropriate approach tailored to specific purposes and problem characteristics. The primary and secondary distinguishing criteria outlined in the framework help researchers precisely position their work within the field.

For the integration modes in Cluster 1 (i.e., ML ↔ MCDM), the primary distinguishing criterion is Sequential Processing, which emphasizes the complementary interplay between ML and MCDM, where each method builds on the other’s output in a stepwise, mutually reinforcing manner. Beyond this distinction, this cluster also aligns closely with the criteria Efficiency, Complexity, Subjectivity, and Automation, and shows a weaker but notable association with Explainability. Therefore, a decision-maker may favor sequential approaches for their ability to optimize resource utilization, manage problem complexity, reduce individual biases, provide improved decision support with minimal human intervention, and enhance model transparency. For example, MCDM methods complement ML models by improving efficiency through structured evaluations that narrow down alternatives and by reducing complexity through breaking problems into manageable criteria (refer to Equations (8) and (13)–(16)). Conversely, ML algorithms strengthen MCDM methods by handling large-scale and complex data, reducing cognitive and computational burden (refer to Equations (2)–(7)). They also contribute to efficiency by processing vast amounts of data at speed, and to automation by enabling decision workflows to run with minimal human intervention. Moreover, ML uncovers hidden patterns and relationships among criteria that may be difficult for experts to detect manually (refer to Equations (3)–(6)). In particular, advanced ML and deep learning techniques serve as highly accurate predictive models that rely on large datasets and often require minimal expert input. However, a significant limitation of these models is their inherent “black-box” nature, which makes them difficult to interpret or explain [91]. Here, MCDM plays a critical role by evaluating and prioritizing features, identifying which features most strongly influence model predictions, and thereby enhancing explainability (refer to Equations (8) and (13)–(15)).

Cluster 2 involves hybrid approaches (MCDM + ML) with Novelty as their distinguishing criterion, indicating that these approaches introduce more innovative methodologies compared to other integration modes. Hybrid approaches are also associated with Flexibility, Automation, and Dynamic Nature, emphasizing adaptability and responsiveness to varying conditions and decision-making contexts, enabling certain tasks to be carried out with minimal human intervention and effectively handling changes over time (refer to Equations (18)–(22)). These abilities make Cluster 2 advantageous in dynamic, rapidly changing environments such as intelligent transportation, energy management, and smart manufacturing, where static decision rules can quickly become outdated. Therefore, problems requiring adaptive decision-making, minimal human intervention, and/or time-dependent modeling may benefit more from Cluster 2. It is worth noting that Automation is not unique to this cluster; rather, it is a shared characteristic of both hybrid and sequential MCDM–ML integration modes.

The integration modes in Cluster 3, described as performance comparison approaches, primarily focus on evaluating and benchmarking ML algorithms and MCDM methods. The distinguishing criterion for this cluster is Validation, while Applicability and Effectiveness also play a reasonable role in defining the contributions of the approaches. Researchers aiming to systematically evaluate methods and ensure both accuracy and practical relevance in their results may find Cluster 3 particularly valuable, as it offers a structured framework for comparing alternative models and identifying the most suitable approach for a given decision problem (refer to Equations (26)–(30) and (32)–(35)).

By examining integration modes according to distinguishing criteria, the framework reveals underexplored areas, such as developing methods that simultaneously improve explainability and computational efficiency or designing hybrid approaches that address the black-box limitations of learning models (see Section 5 for details).

The framework also illustrates that no single integration mode is universally superior; each has strengths and limitations. For example, sequential approaches offer structured and complementary decision flows but may lack novelty. Hybrid approaches drive innovation and flexibility but can increase methodological complexity. Performance comparison approaches contribute to benchmarking and validation but do not directly enhance problem-solving capacity (see Section 4.7 for details).

The following sections first demonstrate the validation of the proposed framework through statistical analyses and then elaborate on the key findings and their implications.

3.6. Validation of the Framework

Two complementary approaches have been employed to validate the findings of the previous analyses and the proposed framework: cluster validity (internal validation) and out-of-sample validity (external validation).

3.6.1. Clustering Validity

To assess cluster validity, discriminant analysis was conducted as a post hoc validation of the 3-cluster framework derived from the correspondence and cluster analyses. Using the principal coordinates and the assigned cluster memberships, this analysis tested the statistical robustness and interpretive clarity of the clustering solution.

The discriminant analysis first tested whether the clusters were statistically distinct by examining differences in their mean coordinate profiles, providing evidence of structural separation. The results offer strong empirical validation for the three-cluster framework. Both principal dimensions, F1 and F2, significantly and strongly discriminate the groups (Wilks’ Lambda < 0.40, p < 0.001), indicating that each dimension contributes substantially to the separation of the clusters (see

Table 15. Tests of equality of group means.

Dimension	Wilks’ Lambda	F	df1	df2	Sig.
F1	0.392	12.417	2	16	0.001
F2	0.326	16.514	2	16	0.000

). The multivariate tests (Wilks’ Lambda = 0.123, χ² = 32.48, p < 0.001) further confirm that the discriminant functions (i.e., linear combinations of the principal dimensions) jointly provide a significant and clear distinction among the three groups.

The analysis also evaluated how accurately the categories could be reclassified into their assigned clusters, thereby assessing the stability and robustness of the clustering solution.

Table 16. Classification results for original and cross-validated data.

		Cluster	Predicted Group Membership
Sample			1	2	3	Total
Original a	Count	1	9	0	0	9
		2	0	5	0	5
		3	2	0	3	5
	%	1	100.0	0.0	0.0	100.0
		2	0.0	100.0	0.0	100.0
		3	40.0	0.0	60.0	100.0
Cross-validated b	Count	1	9	0	0	9
		2	1	4	0	5
		3	2	0	3	5
	%	1	100.0	0.0	0.0	100.0
		2	20.0	80.0	0.0	100.0
		3	40.0	0.0	60.0	100.0

^a. 89.5% of original grouped cases correctly classified. ^b. 84.2% of cross-validated grouped cases correctly classified.

indicates that 89.5% of the original cases and 84.2% of the cross-validated cases are correctly assigned to their respective clusters. Note that cross-validation was performed using the leave-one-out procedure, in which each case is classified by discriminant functions derived from all other cases except itself [86]. This approach provides a more realistic estimate of predictive performance by ensuring that no case is classified using information from its own data. Cluster 1 shows perfect stability, with 100% correct classification in both the original and cross-validated results, indicating a clearly defined and distinct group. Cluster 2 also demonstrates strong robustness, with complete accuracy in the original classification and 80% in cross-validation. Cluster 3 exhibits moderate stability, with 60% accuracy in both analyses, reflecting limited overlap with Cluster 1 while maintaining a recognizable, distinct profile. The probabilities reported in

Table 17. Casewise statistics.

		Original				Cross-Validated
Integration Mode	Actual	Predicted	Probability			Predicted	Probability
/Criteria	Group	Group	1	2	3	Group	1	2	3
ML → MCDM	1	1	0.977	0.002	0.021	1	0.971	0.003	0.026
MCDM → ML	1	1	0.974	0.001	0.025	1	0.968	0.002	0.030
MCDM + ML	2	2	0.003	0.997	0.000	2	0.005	0.995	0.000
ML vs. MCDM	3	3	0.042	0.000	0.958	3	0.053	0.000	0.947
MCDM[ML vs. ML]	3	3	0.089	0.000	0.911	3	0.109	0.001	0.890
Novelty	2	2	0.000	1.000	0.000	2	0.000	1.000	0.000
Complexity	1	1	0.837	0.003	0.160	1	0.821	0.003	0.176
Validation	3	3	0.000	0.000	1.000	3	0.000	0.000	1.000
Subjectivity	1	1	0.860	0.011	0.129	1	0.836	0.014	0.150
Knowledge Base	1	1	0.728	0.012	0.260	1	0.679	0.017	0.304
Effectiveness	3	1 *	0.616	0.005	0.379	1 *	0.781	0.005	0.214
Efficiency	1	1	0.827	0.001	0.172	1	0.811	0.002	0.187
Applicability	3	1 *	0.568	0.002	0.430	1 *	0.693	0.002	0.305
Flexibility	2	2	0.056	0.941	0.003	2	0.074	0.921	0.005
Consistency	1	1	0.730	0.003	0.267	1	0.706	0.004	0.290
Automation	2	2	0.401	0.588	0.011	1 *	0.605	0.376	0.019
Sequential Process.	1	1	0.999	0.000	0.001	1	1.000	0.000	0.000
Dynamic Nature	2	2	0.000	1.000	0.000	2	0.000	1.000	0.000
Explainability	1	1	0.821	0.021	0.158	1	0.784	0.026	0.190

* Misclassified case.

confirm that these cases (Effectiveness and Applicability) lie closer to the Cluster-1 centroid. A similar pattern is observed for the criterion Automation in the cross-validation, where it is reassigned to Cluster 1. These criteria, therefore, require careful interpretation, as their positions suggest partial affiliation with both clusters and indicate that they may be shared across two groups of integration modes. On the other hand, no misclassifications occurred between Clusters 1 and 2 or between Clusters 2 and 3 in the original dataset. Overall, the accuracy levels are notably high—particularly given the small sample size—and demonstrate that the clusters are statistically stable and well-defined.

Note that, although the main assumption in discriminant analysis—namely, the equality of covariance matrices across groups, tested by Box’s M—is violated (p < 0.001), this outcome is expected given the small and uneven group sizes. Notably, the discriminant solution remains robust and interpretable despite this limitation.

Taken together, these findings demonstrate that the three clusters are statistically distinct, reasonably stable across validation, and well supported by the underlying structure of the data, thereby providing solid validation for the proposed framework.

3.6.2. Out-of-Sample Validity

To assess the external validation of the proposed framework, a separate CA was conducted on 22 holdout articles intentionally reserved during the literature review for validation (see Section 3.1). These articles were randomly selected from studies published in the last three years to ensure contemporary, unbiased coverage. In the holdout set (see Appendix F), MCDM → ML, ML → MCDM, and MCDM[ML vs. ML] are each represented by six studies, followed by ML vs. MCDM with five studies and MCDM + ML with one study. It is worth noting that three articles in the holdout set were assigned to more than one integration mode, reflecting the methodological hybridity and increasing sophistication of recent studies in this domain. Thus, the holdout set covers all ML–MCDM integration modes and exhibits a balanced distribution across the major categories identified in the main dataset, thereby providing a suitable basis for evaluating the generalizability of the proposed framework beyond the main sample. The key question was whether these newly examined articles exhibit patterns of association between integration modes and evaluation criteria that are consistent with the correspondence–cluster structure established using the initial set of 208 articles. The articles were independently evaluated by two experts using the same procedures followed for the original set of articles, ensuring consistency in coding and assessment across both samples (see Appendix F). All average-measure ICC values were 0.788 or higher, indicating an excellent level of interrater reliability between the two experts.

Consistent with the main analysis, the test of independence between the rows (integration modes) and the columns (criteria) confirmed the presence of statistically significant associations in the holdout set (${\chi }^2$ = 1820.864; df = 52; p < 0.001). Moreover, the first two dimensions, F1 (51.43%) and F2 (39.30%), together account for 90.73% of the variance, capturing most of the variance in the data structure.

It is important to note that the CA solutions derived from the 208-article sample and the 22-article holdout sample may differ in their origin, scale, and axis orientation due to sampling variability, meaning that the coordinates of the two maps are not directly comparable. To compare whether the geometry of the two CA solutions is structurally similar, Procrustes analysis was performed. The CA configuration from the 22-article set was rotated, translated, and scaled to optimally align with the 208-article configuration, and the goodness-of-fit was evaluated using the Procrustes sum of squared residuals and the Procrustes correlation (i.e., the overall similarity between the two configurations), with statistical significance assessed via 999-permutation testing [92]. The results of the Procrustes analysis indicate a strong congruence between the two CA solutions. A high Procrustes correlation ($R=0.9207$) combined with a low sum of squared residuals (${SS}_{Res}=0.1524$) suggests that the geometric structure of the holdout dataset closely matches that of the main dataset. Furthermore, a 999-permutation test—conducted by permuting the rows of the holdout configuration and recalculating the Procrustes correlation—yielded a permutation p-value of 0.001, confirming that the observed congruence is statistically significant at $\alpha =0.05$. Overall, these results provide preliminary evidence of the structural stability of the proposed evidence-based framework beyond the initial dataset. Figure 8 presents the aligned configurations from the Procrustes analysis along with the residual distances between corresponding points.

According to the plot, as observed in the primary analysis, F1 clearly separates ML → MCDM and MCDM → ML modes from the ML vs. MCDM and MCDM[ML vs. ML] modes, while F2 distinguishes MCDM + ML from both the ML → MCDM and MCDM → ML modes. Moreover, with respect to the principal coordinates, the pattern observed in the holdout set is consistent with the main analysis: F1 mainly contrasts criteria such as Validation (high positive score) with Sequential Processing, Dynamic Nature, Novelty, and Automation (high negative scores), while F2 mainly contrasts criteria such as Sequential Processing with Novelty, Dynamic Nature, and Flexibility.

To further test whether the clusters identified in the main dataset generalize, discriminant analysis was applied to the original coordinates of the holdout set. According to the results, both principal dimensions, F1 and F2, significantly and moderately discriminate the groups (F1-Wilks’ Lambda < 0.447, p < 0.002; F2-Wilks’ Lambda < 0.629, p < 0.025), indicating that each dimension contributes substantially to the separation of the clusters. The multivariate tests (Wilks’ Lambda = 0.280, χ² = 19.745, p < 0.001) further confirm that the discriminant functions jointly provide a significant and clear distinction among the three groups. The analysis also evaluated how accurately the categories could be reclassified into their assigned clusters (see Section 3.5.2), showing that 89.5% of the original cases and 78.9% of the cross-validated cases were correctly assigned to their respective clusters. Details of the classification accuracies and corresponding posterior probabilities are presented in Appendix G.

All these results provide supportive and exploratory evidence that the structural patterns identified in the original analysis remain consistent when applied to a small, independent, and recent set of studies. While the limited sample size constrains statistical stability and generalization, the findings suggest that the proposed framework exhibits a consistent internal structure beyond the initial dataset.

4. Findings and Discussion

4.1. Most Utilized Integration Mode

Identifying the most frequently utilized category offers valuable insights into the prevailing trends in the integration of ML algorithms and MCDM methods. By examining the distribution of these categories, we can better understand which integration strategies are most widely adopted and the underlying reasons driving their prominence. The bar chart in Figure 9 visualizes the distribution of the five integration modes. The MCDM → ML category has the highest usage, with 64 instances, followed by ML → MCDM with 56 instances, reflecting the complementary nature of these approaches, where one method addresses the limitations of the other. Performance comparison approaches, including MCDM[ML vs. ML] (39 instances) and ML vs. MCDM (38 instances), also demonstrate significant application. Finally, the hybrid approach MCDM + ML is the least frequently used, with only 18 instances. This relatively low number of studies may stem from challenges in integrating methods into hybrid forms and the need for novelty in such approaches. However, the adoption of this approach is steadily increasing as it offers more comprehensive and adaptive decision-making capabilities.

4.2. Usage Trends of Integration Modes

Figure 10 illustrates the trends in the use of five ML-MCDM integration modes from 2015 to 2025, highlighting the relative frequency of each mode over time. In response to Research Question 4, the findings reveal a general upward trend across integration modes, particularly after 2020, reflecting the growing adoption of these modes in the literature. The MCDM → ML category (purple line) has shown the highest growth in recent years, surpassing all other modes. ML → MCDM (green line) also demonstrates a steady increase, reaching high frequency by 2025. The two performance comparison categories (blue line: Comparison of ML algorithm performance using MCDM methods; red line: Comparing the performances of ML algorithms and MCDM methods) follow similar upward trends but grow more slowly. The hybrid ML + MCDM category (cyan line) shows the slowest growth and remains the least utilized mode. While its adoption has increased slightly in recent years, it remains less common than the other categories. Given the inherent characteristics of this category, such an outcome is to be expected. However, it is essential not to overlook that this area holds considerable potential and may exhibit further development in the future.

4.3. Frequency of MCDM Methods Across Integration Modes

Figure 11 illustrates the distribution of different MCDM methods across ML-MCDM integration modes. Among them, AHP and TOPSIS are the most frequently used. AHP is particularly dominant in the MCDM → ML category, highlighting its importance in defining criteria weights and guiding feature selection to improve ML models [79,93,94,95,96]. TOPSIS is also widely applied across multiple categories, especially in the MCDM[ML vs. ML] and ML → MCDM categories, as it directly utilizes quantitative performance data and reduces reliance on expert judgment, making it well-suited for data-driven decision-making.

Other methods, such as VIKOR, PROMETHEE, and Fuzzy MCDM methods, show comparatively lower usage across categories. VIKOR and PROMETHEE are less frequently employed in ML–MCDM integration due to their complexity and parameter requirements. VIKOR’s compromise ranking [97] and PROMETHEE’s preference modeling [98] are less directly compatible with quantitative ML performance data. Fuzzy MCDM methods, however, occur in multiple categories despite lower overall numbers, suggesting their unique role in addressing uncertainty and subjectivity within ML-MCDM integration.

It is observed that, while some methods dominate particular integration modes, almost all considered MCDM methods appear at least once across the categories, indicating a diversity of approaches rather than strict boundaries in method selection.

4.4. Frequency of ML Algorithms Across Integration Modes

Figure 12 presents the distribution of the most used ML algorithms across ML-MCDM integrations. Classification algorithms such as Random Forest, Decision Tree, Boosting Algorithms, SVM, Naïve Bayes, Neural Networks, K-Nearest Neighbors (KNN), and Logistic Regression play a crucial role in ML-MCDM integration. There are several reasons why classification-based ML algorithms are widely integrated with MCDM methods. For example, Logistic Regression frequently appears in MCDM[ML vs. ML], indicating its widespread use in benchmarking due to its ability to provide explainable results. Similarly, some classification algorithms, such as decision trees or tree-based models, retain a level of interpretability that aligns with MCDM’s structured nature. This transparency allows stakeholders to understand the reasoning behind predictions and decisions, including how different features are weighted in the model [99]. Additionally, algorithms such as Random Forest and Boosting are robust to overfitting and capable of handling high-dimensional spaces, making them suitable for integration into MCDM methods that may involve complex and heterogeneous criteria [100]. Furthermore, classification algorithms are well-suited for decision-making in finance, healthcare, and marketing industries, where labeled data is prevalent. They help translate complex decision-making frameworks into actionable classifications that can directly support business and operational strategies.

Notably, Neural Networks, SVM, and KNN are comparable in the MCDM → ML category (purple bars). Before the application of these algorithms, MCDM was mainly used for weighting [42,79,93,96,101,102,103,104,105,106] and labeling [107,108] purposes. These three algorithms also demonstrate broad applicability across diverse datasets and problem domains, and they are frequently integrated with MCDM approaches due to their versatility in addressing complex decision-making tasks [76,109,110,111,112].

In contrast, Linear Regression has a relatively lower presence across categories, suggesting that it is less commonly integrated into ML-MCDM frameworks. Similarly, clustering algorithms are less frequently used than classification algorithms, but they remain relevant in decision-support applications, particularly when dealing with unlabeled data. Clustering algorithms (such as K-means, Hierarchical Clustering, and Fuzzy C-means) have a balanced presence across all categories, suggesting their role in analyzing patterns and grouping similar decision alternatives.

4.5. MCDM Methods and ML Algorithms Commonly Used Together

Figure 13 presents a Sankey diagram that shows the MCDM methods and ML algorithms jointly used in ML–MCDM applications. By examining the thickness of the connections in the diagram, we can identify the most frequently used combinations of MCDM methods and ML algorithms. TOPSIS is the most commonly used MCDM method, often integrated with Boosting algorithms (38 times), Decision Trees (31 times), Random Forest (26 times), Support Vector Machines (22 times), Artificial Neural Networks (18 times), and K-Nearest Neighbors (17 times). AHP also shows considerable versatility, frequently integrated with Random Forest (24 times), Boosting algorithms (23 times), Artificial Neural Networks (20 times), Logistic Regression (16 times), Support Vector Machine (16 times), K-Nearest Neighbors (13 times), and Decision Trees (11 times). Furthermore, Fuzzy MCDM methods exhibit strong linkages with Random Forest (14 times), clustering algorithms (used 13 times), Artificial Neural Networks (10 times), SVM (10 times), Decision Tree (7 times), Boosting algorithms (10 times), and KNN (7 times). In contrast, VIKOR and PROMETHEE appear less frequently and have limited integrations across algorithms, suggesting they play more specialized roles in ML–MCDM applications. Other MCDM methods, although less commonly used, provide valuable specialized techniques for integrating with various ML algorithms, enabling the solution of complex decision-making problems that require customized approaches.

4.6. Dominant Application Areas

Integrating MCDM methods and ML algorithms has gained significant attention across a wide range of application domains. The extent of adoption varies between fields, depending on factors such as data availability, the complexity of decision-making processes, and specific industry needs. Understanding why a particular domain favors ML-MCDM integration can help clarify its influence on decision-making practices and the value it brings in different contexts. Figure 14 illustrates the distribution of application areas where integrated approaches have been utilized.

Regarding Research Question 5, the findings indicate that integrated ML–MCDM approaches have been most extensively adopted in the fields of Environmental Science and Healthcare, which account for the largest share of applications, indicating their significance in decision-making research [13,42,43,47,50,70,113,114,115,116]. The rapid advancement of medical technologies and environmental monitoring has led to the generation of massive volumes of both structured and unstructured data. While ML algorithms excel at processing these datasets by uncovering hidden patterns and producing predictive assessments, MCDM methods play a complementary role by interpreting and prioritizing these insights for practical decision-making. Both domains also require collaboration among multiple experts, including doctors, researchers, policymakers, environmental scientists, and public health officials. Integrated approaches are particularly valuable in these contexts, as they combine data-driven insights with structured, criteria-based reasoning to support complex, multidisciplinary decision-making. With rising global health crises and environmental challenges, the integration of data-driven and expert-based decision-making has become not only valuable but essential for building a sustainable future.

Supply Chain, Finance, and Sustainability also exhibit notable implementation [26,117,118,119,120,121,122,123,124]. Conversely, fields such as Manufacturing, Cybersecurity, Renewable Energy, and Tourism have lower levels of representation. Civil Engineering, Biology, Facility Location, Safety, and other application areas appear at the lower end of the tail, suggesting either limited application or emerging research interest in these areas.

Regarding the application areas of the different integration modes, the ML → MCDM category is versatile and broadly applied across Healthcare [125], Environmental Science [13], Supply Chain Management [126], and Marketing [127]. MCDM → ML and MCDM + ML are concentrated in Healthcare [47,128] and Environmental Science [42,129], indicating that these fields particularly benefit from structured expert input alongside ML. The integration modes ML vs. MCDM and MCDM[ML vs. ML] have more focused applications, implying that comparative or hybrid evaluation approaches are used primarily when domain complexity or decision-criticality demands careful method selection. The observed patterns suggest that certain integration strategies are applied more frequently in specific domains, potentially reflecting their suitability for the challenges of those fields.

4.7. Limitations of Integration Modes

MCDM → ML: One significant concern with this integration mode is the potential for subjectivity. The use of expert-driven criterion weights in MCDM can introduce bias, which in turn may compromise the objectivity of the ML model. This issue is compounded by methodological sensitivity, as the output of an MCDM method is heavily dependent on the chosen method and its specific parameters (e.g., normalization method) [130]. This instability can result in volatile inputs, making the subsequent ML models highly sensitive to arbitrary methodological choices. Additionally, scalability limitations present a practical barrier, as traditional MCDM methods struggle to manage the high-dimensional, large-volume datasets typical of modern ML, creating a critical preprocessing bottleneck.

ML → MCDM: The lack of interpretability in ML models raises ethical and practical concerns, particularly in contexts where transparency is essential for accountability and trust. While MCDM provides a structured framework to address such challenges [131], its inclusion also introduces methodological complexity. Furthermore, the reliance on large, well-labeled datasets in ML can introduce challenges when integrating it with MCDM methods, which themselves demand detailed criteria definitions and evaluations. Another limitation is the propagation of errors, uncertainties, or biases that are present in the output of ML models. These issues tend to be carried over into the subsequent MCDM analysis. Although the final ranking or prioritization produced by the MCDM process may appear precise and objective, it may be based on uncertain foundations inherited from the ML stage.

MCDM + ML: One of the foremost issues with the hybrid approach is the lack of standardized, proven methodologies for achieving genuine integration between the two methods. As a result, most implementations remain ad hoc, tailored to specific problems or datasets, and lack generalizability across domains. A further point to consider is that a tightly coupled hybrid model is inherently more complex than either of its components alone. This complexity introduces significant challenges in scalability and computational cost, particularly in real-time systems. Moreover, this integration faces a fundamental conceptual mismatch between the two approaches: ML is inherently data-driven, whereas MCDM generally focuses on decision-makers’ values [6]. Establishing a coherent connection requires careful theoretical grounding and methodological innovation.

MCDM[ML vs. ML]: A primary limitation is that the choice and structural characteristics of the MCDM method used to compare the performances of ML algorithms can strongly influence the final rankings [132]. Another challenge is that the final ranking is sensitive to the weights assigned to conflicting performance criteria. MCDM methods require trade-offs, for example, determining how much interpretability one is willing to sacrifice for a marginal improvement in accuracy. This sensitivity raises concerns about the robustness of the evaluation process, as even small changes in weighting can substantially alter the resulting algorithm rankings.

ML vs. MCDM: Finally, the comparative performance of ML and MCDM is inherently context-dependent, indicating that neither approach can be expected to perform effectively across all study areas, contexts, or dataset environments [77]. Many MCDM methods are designed for small-scale decision problems, making them computationally inefficient when applied to larger problems or high-volume datasets. In contrast, ML algorithms are generally better suited to handle such challenges. Additionally, MCDM methods perform effectively in static decision environments, while ML approaches can adapt to evolving conditions over time through model updates. This mismatch reduces the ability of this integration to support real-time or continuously updating decision environments and limits comparability across studies [20].

5. Future Research Suggestions

The integration of MCDM methods and ML algorithms has shown considerable promise, yet several methodological and practical challenges remain unresolved. As applications expand toward larger datasets, real-time decision-making, and complex dynamic environments, current approaches face limitations in scalability, interpretability, generalizability, adaptability, and in handling uncertainty and complexity. Addressing these challenges requires advancing both methodological innovations and domain-specific applications. In response to Research Question 6, this study has identified five research directions for future research, building on the findings from the literature review and expert evaluations.

Research Focus 1. Methodological Advancements in MCDM and ML Integration

This research focus addresses the fundamental computational and structural limitations of existing ML–MCDM integrations. The objective is to develop models that are scalable, less dependent on subjective manual input, and capable of handling complex internal relationships.

1.1. Improving Scalability and Efficiency: MCDM methods often face challenges with large and complex datasets due to computational demands and the manual effort required. Future research should therefore emphasize preprocessing techniques, such as clustering and dimensionality reduction, that preserve essential information while reducing data volume. Utilizing ML-based optimization approaches (e.g., genetic algorithms, particle swarm optimization) can further enhance accuracy and resource utilization throughout the weighting and selection of alternatives stages. As real-time and large-scale applications continue to grow, developing integrated approaches that balance accuracy, efficiency, and resource use will become increasingly important. Building on this line, future research could also explore MCDM-based evaluations of how advancements in ML—such as architectural innovations, parameter optimization, and emerging approaches like deep learning, preference learning, and transfer learning [4]—influence both the effectiveness and the computational efficiency of decision-making methods.

1.2. Modeling Complex Interdependencies: A further challenge lies in the interdependence among criteria or alternatives. While methods such as ANP and DEMATEL account for these relationships, they remain highly expert-dependent and time-consuming. Advanced approaches such as neural networks, structural equation modeling (SEM), or learning fuzzy cognitive maps may offer more efficient ways to address interdependencies and improve MCDM [15].

1.3. Automating Processes and Reducing Bias: Reducing reliance on expert judgment is another promising direction. Combining ML algorithms with objective MCDM weighting and aggregation methods (e.g., Entropy, CRITIC, Cumulative Belief Degree [133]) can significantly automate decision-making tasks, eliminate biases, and increase responsiveness. This approach not only allows systems to operate autonomously but also enables data-driven decisions to be made with minimal human intervention, enhancing the objectivity of the process.

Research Focus 2. Adaptability to Real-World Data and Dynamics

This research focus concentrates on enhancing the robustness of ML–MCDM models, enabling them to function effectively in environments characterized by changing conditions, uncertainty, and imbalanced data.

2.1. Adapting to Dynamic Environments: Rapidly changing environments, such as financial markets or social media, require models that evolve continuously with new data. Enhanced adaptability would allow MCDM methods to deliver more accurate, timely, and context-aware outcomes. Utilizing online learning, incremental learning, reinforcement learning, and adaptive neuro-fuzzy systems can enable MCDM methods to dynamically update their results in response to changing data over time [120,134,135]. Furthermore, combining non-linear ML models with MCDM methods [42,97] could better capture complex, time-varying behaviors that linear approaches fail to represent.

2.2. Managing Uncertainty: Many practical decision-making situations involve uncertainty due to incomplete or unreliable data. Advancing this area requires developing ML–MCDM frameworks that explicitly capture and incorporate uncertainty into decision models. Probabilistic ML models (e.g., Bayesian approaches) and ensemble learning can be adapted to MCDM methods to provide more robust evaluations under uncertainty. Such integrations can enhance both methodological reliability and improve confidence in decision outcomes by quantifying the credibility of alternatives.

2.3. Addressing Data-Specific Challenges: Another practical barrier in ML–MCDM research is the demand for large and carefully labeled datasets, particularly for deep learning. MCDM methods (e.g., sorting methods) can help reduce labeling costs by identifying the most informative samples and improving data efficiency.

Imbalanced datasets pose another challenge, as ML models often perform well on the majority classes but poorly on minority classes [124]. Various approaches, such as resampling techniques, algorithm-level methods, and anomaly detection, have addressed this issue in the literature.

Additionally, the careful selection of evaluation metrics is essential in imbalanced ML settings, as conventional measures often fail to capture performance across minority classes. Future research could investigate how MCDM methods can be applied to evaluate multiple performance metrics, enabling fairer model assessment. Furthermore, MCDM can help prioritize features that are particularly relevant to minority classes, thereby improving model robustness in imbalanced settings.

Research Focus 3. Enhancing Model Interpretability and Explainability

Beyond adaptability, real-world deployment also demands human interpretability and transparency.

3.1. Enhancing Explainability: Moving ML–MCDM research into real-world practice requires integrated models that are transparent, interpretable, and aligned with human decision processes. Transparent models are particularly critical in risk-averse domains such as medicine, where the reliance on black-box ML models poses significant barriers to adoption [4]. MCDM methods, particularly those that prioritize features, can enhance model explainability by identifying the most influential features in ML predictions.

Another promising direction for future study is to explore the integration of MCDM methods with explainability techniques, such as Shapley Additive Explanations (SHAP), Local Interpretable Model-Agnostic Explanations (LIME), and Anchors [136]. By aggregating and contextualizing the outputs of these techniques, MCDM methods can provide users with holistic and transparent insights into model behavior.

Beyond interpretation, MCDM can also be employed to simulate alternative decision scenarios, allowing users to examine how variations in input features or model parameters influence the resulting outcomes. Such simulation-based transparency can strengthen user understanding, increase accountability, and encourage ethical adoption of ML–MCDM systems.

Research Focus 4. Expanding Applications Across Sectors and Datasets

Building upon methodological advances and improvements in interpretability, this research focus concentrates on expanding the application of MCDM–ML integrations across diverse real-world sectors and datasets.

4.1. Generalizing MCDM–ML Integrations Across Diverse Sectors: A key research priority is examining the generalizability of MCDM–ML integrations across different sectors, datasets, and conditions [135]. Studies should test these integrations in diverse contexts such as finance, urban planning, and social media, as well as in high-impact areas such as emergency management and healthcare, to validate their robustness and adaptability [4,118]. Sector-specific evaluations will not only confirm applicability and adaptability but also encourage wider adoption.

4.2. Improving Performance Metrics for Real-World Applications:

In practice, performance metrics are commonly used as criteria when comparing ML algorithms through MCDM methods. However, relying exclusively on these metrics can be inadequate in real-world contexts, where broader factors such as scalability, interpretability, and complexity also play critical roles [131]. In sectors like healthcare, for instance, interpretability of an ML model can be just as important as its accuracy, as it provides insights into the decision-making process that medical professionals can trust and rely on in their actions. Because such criteria are often challenging to define and measure, fuzzy sets can be employed to represent them as linguistic labels, allowing for a more flexible evaluation process that accounts for subjectivity in assessments [65,137,138].

Research Focus 5. Leveraging Emerging Technologies

The final research focus highlights the potential of next-generation machine learning technologies to advance ML–MCDM integration.

5.1. Exploring Novel ML Technologies: Novel ML technologies such as generative pre-trained transformers (GPTs), federated learning, deep reinforcement learning, and metamodeling remain largely unexplored in the ML–MCDM literature. These methods could enhance decision support by enabling distributed learning, improving sequential decision-making, or capturing highly non-linear relationships in complex datasets. Comparative studies are needed to establish guidelines for selecting algorithms most relevant to specific problem domains.

Together, these five research focuses provide a comprehensive roadmap for advancing ML–MCDM integration, from foundational methodological improvements to practical applications and the adoption of emerging technologies. By addressing scalability, adaptability, interpretability, real-world applicability, and frontier technologies, ML–MCDM can evolve into a powerful tool for complex decision-making.

Table 18. Criteria and integration modes associated with the future research suggestions.

Suggestions		Involved Integration Mode(s)
Research Focus 1. Methodological Advancements in MCDM and ML Integration
1.1	Improving Scalability and Efficiency	ML → MCDM, MCDM → ML, MCDM + ML, MCDM[ML vs. ML]
1.2	Modeling Complex Interdependencies	ML → MCDM, MCDM → ML, MCDM + ML
1.3	Automating Processes and Reducing Bias	ML → MCDM, MCDM → ML
Research Focus 2. Adaptability to Real-World Data and Dynamics
2.1	Adapting to Dynamic Environments	MCDM + ML
2.2	Managing Uncertainty	ML → MCDM, MCDM → ML, MCDM + ML
2.3	Addressing Data-Specific Challenges	MCDM → ML, MCDM + ML, MCDM[ML vs. ML]
Research Focus 3. Enhancing Model Interpretability and Explainability
3.1	Enhancing Explainability	ML → MCDM, MCDM → ML
Research Focus 4. Expanding Applications Across Sectors and Datasets
4.1	Generalizing MCDM–ML Integrations Across Diverse Sectors	ML → MCDM, MCDM → ML, MCDM + ML, ML vs. MCDM
4.2	Improving Performance Metrics for Real-World Applications	MCDM[ML vs. ML]
Research Focus 5. Leveraging Emerging Technologies
5.1	Exploring Novel ML Technologies	ML → MCDM, MCDM → ML, MCDM + ML, MCDM[ML vs. ML]

links the suggested future research directions for ML–MCDM integration with the relevant modes.

6. Conclusions and Limitations

This study investigated the integration of ML algorithms and MCDM methods through a comprehensive approach that combines insights from a literature review, expert judgment, and statistical analysis to ensure both theoretical rigor and practical relevance. It establishes a new systematic categorization of integration modes based on their methodological characteristics and application contexts. It introduces a set of previously undefined criteria for standardized and reliable evaluation, enabling comparability and rigor. By employing CA and cluster analysis, the study maps the relationships between integration modes and evaluation criteria, revealing patterns, gaps, and distinguishing criteria among the integration modes, and demonstrates how the complementary strengths of ML and MCDM enhance decision-making while overcoming individual limitations. Building on these insights, a new evidence-based framework is proposed and validated to guide researchers and practitioners in selecting the most appropriate integration modes and aligning methodological choices with problem-specific requirements to address complex decision problems effectively.

Framed by the six research questions guiding the study, the key findings and their implications can be summarized as follows:

• First, the initial categorization from the literature review identified five main integration modes based on the purpose of integration; however, expert evaluations and subsequent analyses indicated that these could be consolidated into three broader clusters: (i) sequential approaches, (ii) hybrid approaches, and (iii) performance comparison approaches.
• Second, the results showed that specific criteria, such as Sequential Processing, Validation, and Novelty, act as key differentiators. In contrast, others, like Knowledge Base, Consistency, and Explainability, remain relatively neutral and are shared across all categories.
• Third, among the integration modes, MCDM → ML and ML → MCDM are the most widely used. The findings suggest that these categories are particularly suitable when the objective is to optimize resource utilization and computational efficiency, address complex decision-making scenarios involving multiple variables, reduce individual biases, and provide transparent, interpretable outcomes to support informed decisions. The study also revealed that most research has emphasized combining existing methods rather than introducing methodological innovations, leaving room for more original forms of integration. Although less common, hybrid approaches (MCDM + ML) are well-positioned to address this gap, offering strong potential for flexibility and dynamic modeling while reflecting the broader shift toward reducing human intervention in decision-making. When the goal is to validate model outputs and assess their practical applicability in real-world contexts, the findings showed that performance comparison approaches are the most appropriate. These approaches emphasize the critical role of comprehensive benchmarking in ML–MCDM integrations, providing a structured means to evaluate competing methods and generate reliable evidence to guide method selection.
• The fourth question regarding current trends in the joint use of MCDM and ML methods revealed that all categories exhibit a general upward trend, particularly after 2020, indicating a growing adoption in the literature. From a methodological perspective, the findings show that MCDM methods such as AHP and TOPSIS have emerged as the most dominant, often paired with classification-based ML algorithms such as Random Forest, SVM, and KNN. These pairings underscore a preference for interpretable, structured decision-making tools. While clustering and fuzzy methods were less frequent, their consistent presence across categories suggests an increasing focus on managing uncertainty and subjective judgments, particularly in complex decision environments.
• Fifth, the findings provide insights into the adoption patterns and opportunities across various application domains. In particular, environmental science and healthcare are the leading fields adopting ML–MCDM integrations, driven by their high data complexity and multi-criteria multi-stakeholder decision-making requirements. These domains particularly benefit from the synergy between data-driven insights provided by ML and expert-based prioritization enabled by MCDM. Other sectors, including finance, supply chain, and sustainability, also demonstrate considerable adoption, whereas areas such as cybersecurity, civil engineering, and tourism remain underrepresented, highlighting promising directions for future application and growth.
• Finally, this study has identified several promising directions for future research. The directions highlight key avenues for future research that can strengthen the effectiveness, robustness, and applicability of ML–MCDM integrations.

While this study offers valuable theoretical and practical contributions, it also has several limitations that should be acknowledged:

• The categorization and evaluation rely on the currently available literature, which may evolve as new integration approaches and application areas emerge. To address this limitation, future studies should update the classification with newly published research, extend the scope to additional domains, and incorporate longitudinal analyses to capture shifts in ML-MCDM integration.
• The definition of evaluation criteria and the assessment of articles are based on expert judgment. Although this enhances domain relevance, it may also introduce subjectivity and potential bias. Implementing structured consensus-building techniques or semi-automated evaluation tools using natural language processing (NLP) could help reduce subjectivity and improve the consistency of article assessments.
• Additionally, the evaluation was based on a limited number of experts, which may introduce bias due to the subjective judgment of a small group. Future studies could involve a larger and more diverse group of experts to improve reliability and reduce potential biases.
• The criterion of uncertainty was not directly considered in this study, although it is indirectly captured through factors such as subjectivity, complexity, and dynamic nature. Incorporating uncertainty-related evaluation criteria in future work could further enhance the robustness of the proposed framework and extend its applicability across diverse contexts.
• In terms of the criteria considered in this study, certain integration modes (e.g., MCDM → ML and ML → MCDM) were found to exhibit similar characteristics, making them difficult to distinguish clearly. Future research could incorporate additional or more fine-grained criteria to enable a more precise distinction among these modes and provide deeper insights into their unique contributions and applications.
• Another limitation of this study is that the relationships among the evaluation criteria were partially examined. Future research could address this gap by incorporating methods that explicitly model interdependencies between criteria, thereby providing a more comprehensive evaluation framework.
• Although the literature review was conducted using Elsevier’s Scopus database and Web of Science databases, these sources may not cover all relevant journals. As a result, specific articles, particularly from niche or emerging research fields, may have been overlooked, leading to an incomplete view of the literature. A more comprehensive review could be achieved by also including additional databases, such as IEEE Xplore and Google Scholar.
• The majority of the findings were derived from English-language journals, resulting in the exclusion of journals published in other languages. While this implies that the review is not exhaustive, the authors believe it offers a comprehensive overview and covers most of the relevant studies published in scholarly journals.
• Finally, applying and validating the framework across diverse domains would strengthen its generalizability and practical utility. Since the interpretation and relevance of certain criteria may vary by context, adapting the criteria definitions or developing domain-specific sub-frameworks could further enhance the accuracy and applicability of the evaluation process.

In conclusion, this study lays the groundwork for a systematic and comprehensive understanding of ML–MCDM integration. Providing a structured categorization and evaluation framework offers valuable guidance for researchers and practitioners seeking to improve decision-making in complex environments.