Development of a Sustainability-Oriented KPI Selection Model for Manufacturing Processes

Abstract

Modern manufacturing systems operate in a global and competitive environment, where sustainability has become a critical driver for performance. Performance measurement, as a method for monitoring enterprise processes, plays a central role in aligning operational efficiency with sustainable development goals. Recently, a number of different frameworks, systems, and methods have been proposed for small and medium enterprises. Key performance indicators (KPIs) are known to be powerful tools which provide accurate information regarding bottlenecks and weak spots in companies. The purpose of the current study is to develop an advanced KPI selection/prioritization model and apply it in practice. The initial set of KPIs are obtained based on a literature review. The expert’s knowledge, outlier methods, and optimization of the enterprise analysis model (EAM) are utilized for reducing the initial set of KPIs. A fuzzy analytical hierarchy process (AHP) is implemented for prioritization of the criteria. Five different MCDM (multi-criteria decision-making) algorithms are implemented for prioritization of the KPIs. The recently introduced RADAR method is extended to the fuzzy RADAR method, providing a flexible approach for handling uncertainties. An analysis and comparison of the rankings obtained by utilizing five MCDM algorithms is performed. The prioritized KPIs provide valuable input for improving KPIs with the highest impact in particular small and medium-sized enterprises (SMEs) when implementing sustainability-aligned performance metrics.

1. Introduction

For optimizing the manufacturing processes, it is essential to connect KPIs to the goals and production and maintenance activities of the company’s particular processes [1]. To avoid inappropriately selected technical performance indicators, it is important to focus on the implementation of the Enterprise Analysis Model and KPI Selection Model [2,3]. For prioritization of selected KPIs, the EAM integrated fuzzy AHP, TOPSIS approach, covering the vagueness and uncertainty, is developed. In [4], a methodology based on the theory of belief functions for evaluating the uncertainty of KPIs is proposed. Recently, the prioritization of KPIs for evaluating the greenness of products has been performed by utilizing the Pareto concept and the full consistency method [5]. The application of Lean principles and Q-methodology for evaluating KPIs in project-based organizations is studied in [6,7], respectively.

In the literature different types of methods are available for KPI prioritization, like MCDM, ML (machine learning), EC (evolutionary computing), etc. Herein is a focus on MCDM methods mostly due to their simplicity, low computational cost, and transparency. MDCM methods [8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,26,27,28,29,30,31,32,33,34,35,36,37,38,39,40] include commonly simple matrix operations (matrix normalization and weighted summation). ML algorithms [41,42,43] and evolutionary algorithms (EAs) [5,44,45] involve iterative computing and parameter tuning. ML algorithms require training, while EAs are population-based, making them more complex and time-consuming.

The AHP is a powerful decision-making methodology developed by Saaty in the 1980s to simplify the decision-making process [8]. It includes qualitative and quantitative techniques and makes it possible to decompose complex problems into simpler sub-problems where each level shows a set of objectives or criteria relative to each sub-group [8,9]. A number of review papers on the AHP are available [10,11,12,13]. The AHP research field development using social network analysis and scientometrics is studied in [13], and the applications of the AHP are overviewed in [12]. The AHP method allows the decision-maker to include a subjective attitude, experience, knowledge, and intuition in decision-making. Each level of the hierarchy consists of several elements, where the elements from the same level are independent of each other but comparable [14].

The AHP has been utilized successfully for wide-class complex MCDM problems covering civil engineering, renewable energy, production planning, etc., areas [15,16]. The nine-point scale simplifies the choice of criteria and provides information regarding the dominance of each element over others concerning the importance of each criterion of the higher levels of the hierarchy. Individual points of view are made in groups, considering the pertinent decision-maker, and are handled as a foundation for the analysis of the reasons for specific judgments; there is a one-week period that occurs during the setup of comparison matrices [9]. When the number of characteristics is rising in a hierarchy, more matchings between attributes need to be applied. Furthermore, by raising criteria and sub-criteria, the experts are dealing with physical and mental fatigue. As a result, the judgments are becoming unreliable, subjective, and imprecise. Therefore, the triangular fuzzy numbers (TFNs) are a valuable solution for handling subjective and imprecise judgments.

The fuzzy sets introduced in [17] provide a methodology for handling imprecision. The fuzzy AHP has been introduced as a combination of fuzzy sets and AHP in [18]. Replacing the fixed value judgments in AHP with interval judgments in fuzzy AHP simplifies the evaluations of the decision-makers. Fuzzy AHP can be considered a fuzzy extension of Saaty’s priority theory, providing a flexible approach for handling imprecise and inaccurate data. During the last decade, fuzzy AHP has become a popular fuzzy MCDM method in various areas like risk assessment [19], renewable energy [20], engineering, environmental problems, etc. A recent review of fuzzy AHP methods can be found in [21].

The basic idea of the TOPSIS method (Technique for Order Preference by Similarity to Ideal Solution) is that the most preferred alternative should have the shortest distance from the positive ideal solution (PIS) and the farthest distance from the negative ideal solution (NIS) [22]. In classical TOPSIS, the ratings and the weights of the criteria are given as crisp numbers. Such an exact evaluation may appear complicated for the decision-maker in real-life conditions [23].

In recent studies, the fuzzy AHP and TOPSIS methods are combined to solve various MCDM problems including supplier selection [24], metal matrix composite selection [25], customer-driven product design process [26], best plastic recycling method selection [23], performance evaluation of municipal services [27], risks prioritizing in self-driving vehicles [28], optimizing additive manufacturing parameters [29], and risk assessment for airborne network security [30]. In [31], an approach is proposed for manufacturing scheduling by combining a predictive schedule with an MCDM method. This combines the intelligent data-based models of random forest (RF) and the human knowledge-based multi-criteria structure of AHP in accordance with the objective and the subjective factors. In [32], the textile manufacturing process is optimized by combining reinforced learning, RF, and human knowledge-based MCDM methods. In [33], a methodology based on a fuzzy analytic network process and incomplete preference relations is proposed for sustainable supplier selection. In [34], a randomized weighted fuzzy AHP is introduced for stock portfolio selection using modified normalization with fuzzy TOPSIS. In [35,36], the fuzzy AHP approach is adapted for risk evaluation in gold and phosphorite mining industries. In [36], fuzzy AHP is combined with fuzzy TOPSIS.

Recently, a principally new RADAR (Ranking based on Distances and Range) method has been introduced in [37]. According to the method name, the ranking is based on both distance and range. The RADAR method emphasizes stability, tending to prefer alternatives that demonstrate consistent performance across all criteria [37]. In [38], the RADAR2 method was introduced and implemented for the selection of industrial equipment. The main difference between the RADAR2 method and the RADAR method is the modified normalization rule. The mathematical foundation of the RADAR methods is given in [39], demonstrating the robustness of the methods. By combining the fuzzy set theory and RADAR method, the fuzzy RADAR method approach is proposed in [40].

The main objective of this study is to develop an enterprise analysis model and implement it in practice within small and medium-sized enterprises (SMEs). One of the subtasks focuses on the prioritization of key performance indicators (KPIs). Selecting the appropriate metrics is a critical challenge for ensuring the sustainability of production processes and, ultimately, the survival of SMEs as a whole.

These goals are achieved through the application of recent multi-criteria decision-making (MCDM) methods, such as RADAR and RADAR2, as well as the development of a new fuzzy RADAR method, also integrating KPI prioritization within the enterprise analysis model. Prioritized KPIs help SMEs focus on the most important indicators, the improvement of which has the greatest impact on production processes. The KPI selection model operates as a cyclical process. After implementing the selected metrics into production processes, the entire procedure should be repeated to assess how the situation has changed and to evaluate the resulting impact on overall effectiveness.

The paper is organized as follows: Section 1 presents the literature review, outlines the objectives, and main activities undertaken to achieve these goals. Section 2 describes the enterprise analysis model. Section 3 explains the methodology, including the optimization of the EAM, fuzzy set theory, criteria prioritization using fuzzy AHP, and KPI prioritization using the fuzzy TOPSIS, RADAR, RADAR2, and fuzzy RADAR methods. Section 4 presents the numerical results. Section 5 concludes the results.

In recent years, sustainability has emerged as a central concern in manufacturing, where economic performance must be balanced with environmental and social responsibility. Performance indicators that consider energy usage, waste reduction, resource efficiency, and employee well-being are essential for aligning manufacturing strategies with the principles of sustainable development. Therefore, the development of a KPI selection model that reflects sustainability-related dimensions is critical for modern SMEs aiming to enhance their competitiveness while contributing to broader sustainability goals. This study integrates sustainability perspectives into KPI prioritization through structured and data-driven multi-criteria decision-making approaches.

2. Development of the KPI Selection Model as Part of an EAM

For optimizing different manufacturing processes it is important to monitor and collect real-time data from those processes to be able to make on-time management and optimization decisions. Collecting the economic and technical KPIs is the most time-consuming for analyzing different processes and determining the proper KPIs. To make the KPI selection process more effective, it is wise to integrate it with the company’s existing systems and production real-time monitoring tools [46,47], as presented in Figure 1.

Figure 1. Information movement in manufacturing systems and integration with the KPI selection model.

The developed KPI selection model can be considered as a practical tool for identifying critical weaknesses and guiding improvement actions based on future development perspectives [3], aligning with Technology Readiness Levels [48]. The metric-acquiring process is iterative. Here are a number of internal and external impact factors like the qualification of the staff, the level of digitalization, software and hardware solutions utilized, and their implementation time, etc. The KPI selection/prioritization model is shown in Figure 2.

Figure 2. KPI selection/prioritization model.

In order to identify the most critical problems in the company, in the first stage the data collection and analysis models are introduced. A cloud-based platform was developed to implement the survey. Over 70 research papers, covering management and design optimization of products and processes, are utilized for composing the initial survey. The questions are divided into 15 categories, shown in Table 1 [49].

Table 1. Construct a deviation over the questions.

Category	Constructs	Questions
Customer and supplier participation	5	9
Electronic data interchange	38	43
Enterprise name	1	1
Financial	1	2
General information	5	5
Human resource	31	42
Logistics	8	14
Mission and goals	7	12
NPI	21	28
Performance management	5	11
PLM implementation	13	14
Production	28	52
Quality management	8	15
Respondent information	6	6
Sales management	2	5
Total	179	259

The number of questions asked of the employees in the development process is based on the position that the specific employee occupies in the selected company. The interviewer gives each employee a separate package of questions, which covers the field where the employee has experience and competence. The 81 job positions were considered and divided into the following groups: Research and Development (R&D); Information Technology (IT); Business, Sales, and Marketing; Production; Quality; Human Resource (HR); Purchase and Logistics; and Chief executive officer (CEO) (high management). The questions were composed on the principle that they would describe different situations at a company. The questionnaire was divided into two groups: questions based on facts and questions based on situations or problems. Generally, more than 40 different scales in different answers were used [49].

The main goals of the EAM are to acquire information about the enterprise for further analysis and optimization tasks; to identify and define the weak spots in different production processes and company structure; and to indicate the important data that should be collected and the reason for collecting it (from the standpoint of identified critical points) [2]. In the first step all questions are connected to the KPIs, and because of that the right indicators that rely on the weight of the answer can be chosen in the responses. To eliminate misunderstandings and provide better effectiveness for the research, logical connections between the pairs of constructs and questions have been made: questions and KPIs were tested using a special web application [49]. The expert group of people (the expert group consists of 10 experts who have no connectivity to the study work but have enough knowledge and experience in this field) was required to do the matching of constructs with questions to be able to analyze whether different constructs can be described by the specific questions.

The pairs of the KPIs and the questions were tested in the same way, which means that each question was formulated in two different ways so that the main idea of both questions is the same. This process could help to eliminate the answers which were not understandable for employees. Parallel to the previous work, the KPIs were divided into the following three groups [49]: direct KPIs—obvious/direct relation to the responses; indirect KPIs—connection with more than one question; and suggested KPIs—the management based on the questions/situations.

The second step, data collection, is realized herein using a cloud server. This allows for the reduction in time for acquiring questions and simplifies the data processing. The database was integrated from one side with the Production Monitoring System (PMS) to be able to obtain real-time information about the production processes and from the other side with the Product Lifecycle Management (PLM) and Enterprise Resource Planning (ERP) systems to acquire the data of the whole lifecycle of the product [40]. Sensors are installed on the equipment to measure different data depending on the process, but in our case study the measured data are vibration, temperature, and voltage consumption, which in the turning process are used to predict the condition and lifetime of the working tool [50]. The main advantage of this process is to analyze the KPIs, e.g., the Overall Equipment Efficiency (OEE).

The third step is data analysis, where answers to duplicated questions are analyzed and grouped depending on the number of similar questions and their answers to simplify the Cronbach alpha analysis. The index of significance was applied with the 6-point scale to define the impact of the answer on the described situation [3,51]. Calculating the Cronbach alpha, the consistency test was performed.

Before entering the final step, the KPIs were selected for the study. The used approach was similar to that described in the optimization of the EAM process (the grouping process was performed based on the expert group decisions and on the following outlier methods: modified Z-score, Turkey’s method, and adjusted boxplot).

The selection of the evaluation criteria depends on different factors, like process-specific features of the company. Because of that, the following SMARTER criteria factors were selected based on the literature [52]:

• Specific: Detailed, clear, and specific goal;
• Measurable: Measurable target, process, or KPI;
• Achievable: Right level objectives and achievable KPI value;
• Relevant: Relevance to the role and main course of the team or individual person;
• Time-specific: Time frames for work or tasks;
• Explainable or Evaluated: The KPIs have been measured without understanding the reason for measuring. Everyone who is involved in the process should be aware of the proper goals and tasks;
• Relative or Reviewed: Relative and implemented KPIs, even if the company and volumes are growing.

3. Methodology

The methodology proposed is based on data analysis and optimization, utilizing SMARTER criteria, the prioritization of criteria using fuzzy AHP, and the prioritization of KPIs using the fuzzy TOPSIS method. First, the SMARTER criteria and fuzzy AHP are applied in order to determine the weights of the criteria [53]. Next, the fuzzy TOPSIS is utilized to derive a fuzzy decision matrix and the distances from fuzzy PIS and NIS. Finally, the KPIs are ranked based on the values of the similarities to an ideal solution.

The procedure proposed herein is based on combining fuzzy AHP-TOPSIS approaches found in the literature [23,54] with EAM optimization.

3.1. EAM Optimization

An effective EAM has been developed, which helps to perform analysis in a reasonable time frame without a remarkable reduction in quality. In the first step a thorough set of questions and KPIs was composed, and in the next step this set was limited to bounds, allowing resource-effective analysis. The optimization problem considered can be formulated as follows [55]:

$$\begin{gathered} \mathrm{Min} R_i \\ \text{subjected to} \\ \mathrm{K}\mathrm{P}\mathrm{I}\le {\mathrm{K}\mathrm{P}\mathrm{I}}^{\ast }, \mathrm{questions} \le {\mathrm{questions} }^{\ast }, \mathrm{P}\mathrm{I}>{\mathrm{P}\mathrm{I}}_{\ast }, \end{gathered}$$

(1)

where $R_i$ stands for resources (total time for analysis, working hours for completing the questionnaire, etc.), and ${\mathrm{K}\mathrm{P}\mathrm{I}}^{\mathrm{*}}$ and questions * are estimated upper limits for the number of KPIs and questions, respectively. In order to keep the model adequate, the information related to performance $PI$ should exceed the predefined value ${\mathrm{P}\mathrm{I}}_{\ast }$. The basic steps of the KPI selection/optimization procedure can be outlined as follows:

• Composing initial questionnaire and KPIs;
• Reducing questions and KPIs by utilizing expert groups;
• Reducing questions and KPIs by utilizing outlier methods;
• Estimating the final set of KPIs and questions.

The last three tasks can be implemented iteratively if the number of questions or KPIs is not small enough i.e., $\mathrm{K}\mathrm{P}\mathrm{I}>{\mathrm{K}\mathrm{P}\mathrm{I}}^{\ast }$ or questions > questions *.

Note that contrary to the standard approach, in the case of the posed optimization problem the initial solution selected is consciously infeasible. The first two constraints of Equation (1) are not satisfied due to the thoroughgoing set of questions and KPIs considered as candidates for the final set.

To reduce the time for completing the survey by investigating companies and concentrating more on production and its efficiency, optimization of the EAM was required. The first stage is the raw phase, and the initial data were taken from the previous study [3,51]. The next step, raw phase 1.1, was established. A new expert group was formed, and only questions with KPIs connected to the production were selected. As a result, 87 questions and 40 KPIs were selected. In the ranking stage, a new expert group of 10 people was employed (at least 5 years of experience in this research field or in an enterprise), and the importance of the questions on the 1-to-10-point scale was evaluated. The following was 10 points—the question is very important from the standpoint of production—and the following was 1 point—the question cannot provide the necessary information for productivity and efficiency evaluation. The average rank was calculated for each question, and the same process was applied to the group of 40 KPIs. The results of optimized phase I are shown in Table 2.

Table 2. Result of the optimization after phase I.

	Raw Phase 1.0	Optimized Raw Phase 1.1
Questions	259	87
KPIs	92	40

During the ranking phase, the ranked questions received from the experts may contain outliers that have a different impact on the dataset when compared with others. The outliers may have a critical impact on the data analysis. The goal was to optimize the questions by eliminating “faulty” answers from the total range set.

Four different and simple outlier detection methods, standard deviation method, modified Z-score method, Tukey’s method, and adjusted boxplot, were applied to the questions and ranked by the experts (a detailed description of these methods is omitted herein for conciseness’ sake, see details in [3,51]). The next step was the correlation of the weight average. After the ranking phase and optimization, the total number of questions was reduced from 87 to 40 and KPIs from 40 to 14.

The standard deviation shows that the answers from Expert 3 cannot be considered. The modified Z-score also shows that the results from Expert 3 cannot be considered. Similarly, Tukey’s method and the adjusted boxplot show that Expert 3 is an outlier. In the final phase of data optimization, the questions selected as outliers in the ranking phase were introduced to the expert group to make sure that the questions eliminated were unnecessary. The weights were applied to the questions, and the average values of each were calculated. The questions that received more than 6 points out of 10 were considered final data. According to these results, the final amounts of 61 questions and 13 KPIs were established.

3.2. Fuzzy Numbers

The fuzzy set theory was developed by Zadeh and has become an important methodology in pair-wise comparisons. The fuzzy number can be defined as a triple $M=\left(l, m, u\right)$ where its membership function is defined as follows [56]:

$${\mu }_M\left(x\right)=\left\{\begin{array}{c}{\displaystyle \frac{x}{m-l}}-{\displaystyle \frac{l}{m-l}}, x \in \left[l, m\right]\\ {\displaystyle \frac{x}{m-u}}-{\displaystyle \frac{u}{m-u}}, x\in \left[m, u\right]\\ 0, otherwise\end{array}\right.,$$

(2)

where $\left(l \le m \le u\right)$, $l$ and $u$ are lower and upper values, and $m$ is the middle value of $M$. When all three numbers are equal $\left(l=m=u\right)$ then we are dealing with non-fuzzy numbers. The main operations for two triangular numbers were described in [54] as follows:

$$\begin{gathered} M_1 \left(\oplus \right){ M}_2=\left(l_1+ l_2, { m}_1+ m_2, { u}_1+ u_2\right), \\ {M}_{1 } \left(\otimes \right) M_2\approx \left(l_1{l}_2, { m}_1{m}_2, { u}_1{u}_2\right), \\ {M}^{-1} \approx \left({\displaystyle \frac{1}{u_1}}, {\displaystyle \frac{1}{m_1}}, {\displaystyle \frac{1}{l_1}}\right). \end{gathered}$$

(3)

3.3. Criteria Prioritization Using Fuzzy AHP

For the pair-wise comparison of the criteria, the linguistic, crisp AHP scale and triangular fuzzy scale variables are introduced (see Table 3).

Table 3. Linguistic variables for the importance of the criteria, based on [8].

The Relative Importance in Terms of Linguistic Variables	Crisp AHP Scale	Fuzzy Triangular	Reciprocal Fuzzy
Equally preferred (EqP)	1	1, 1, 1	1, 1, 1
Equally to moderately preferred (Eq-MP)	2	1, 2, 3	1/3, 1/2, 1
Moderately preferred (MP)	3	2, 3, 4	1/4, 1/3, 1/2
Moderately to strongly preferred (M-SP)	4	3, 4, 5	1/5, 1/4, 1/3
Strongly preferred (SP)	5	4, 5, 6	1/6, 1/5, 1/4
Strongly to very strongly preferred (S-VSP)	6	5, 6, 7	1/7, 1/6, 1/5
Very strongly preferred (VSP)	7	6, 7, 8	1/8, 1/7, 1/6
Very strongly to extremely preferred (VS-ExP)	8	7, 8, 9	1/9, 1/8, 1/7
Extremely preferred (ExP)	9	8, 9, 9	1/9, 1/9, 1/8

For the evaluation of the criteria, a decision-making team should be formed. Herein, three academic and two industry experts were involved. The expert group has over 5 years of experience in the field of production and process optimization. The pair-wise comparison of the criteria is performed commonly in terms of the linguistic “grades” introduced in Table 3.

Next, the pair-wise comparison matrices of the individual decision-makers are aggregated by using the geometric mean as follows [54]:

$$r_{ij }={\left(\prod _{k=1}^K{c}_{ijk}\right)}^{{\displaystyle \frac{1}{K}}},$$

(4)

where $c_{ijk}$ is the fuzzy comparison value of criteria $i$ to criteria $j$ in the form of triangular fuzzy numbers given by the $k$-th decision-maker. The number of decision-makers is denoted by $K$.

Based on the aggregated pair-wise comparison matrix, the aggregated fuzzy comparison values $r_i=\left(l_i,{ m}_i,{ u}_i\right)$ of criteria $i$ can be computed as follows:

$$r_i={\left(\prod _{j=1}^N{r}_{ij}\right)}^{{\displaystyle \frac{1}{N}}},$$

(5)

where $N$ stands for the number of criteria (is equal to seven for SMARTER criteria). The fuzzy weight of the attribute $i$ can be calculated as the normalized value of $r_i$ as follows:

$$w_i=\left(l_i,m_i,u_i\right)=r_i\otimes {\left(r_1\oplus r_2\oplus \dots \oplus r_N\right)}^{-1},\hspace{1em}\hspace{1em}i=1,\dots ,N.$$

(6)

The triangular fuzzy weights can be defuzzified to a crisp number by applying the center of area method (an alternate approach, like graded mean integration, etc., is available) [57]. As a result, the crisp weight $w_i^{Crisp}$ can be determined for each criterion $i$ as follows:

$$w_i^{Crisp}=l_i+{\displaystyle \frac{\left[\left(u_i-l_i\right)+\left(m_i-l_i\right)\right]}{3}},$$

(7)

The normalized crisp weights read as follows:

$$w_i^{Cris{p}_{Normalized}}=w_i^{Crisp}{\left(\sum _{i=1}^N{w}_i^{Crisp}\right)}^{-1},$$

(8)

Finally, the criteria can be ranked based on the values of the normalized crisp weights. The consistency of experts as well as the consistency of the overall hierarchy should be checked. Herein, for this reason, Saaty’s classical consistency ratio process is applied as follows [54]:

$$Consistency Ratio={\displaystyle \frac{CI}{CR}}={\displaystyle \frac{0.025}{1.32}}=0.019.$$

(9)

where

$$CI={\displaystyle \frac{{\lambda }_{max}-N}{N-1}}={\displaystyle \frac{7.1512 - 7}{6}}=0.025.$$

(10)

where ${\lambda }_{max}$ is the largest eigenvalue of the aggregated decision matrix $r_{ij}$. Obviously, the value of the consistency ratio obtained, 0.019, falls well below the permissible threshold of 0.1

3.4. KPI Prioritization Using Fuzzy TOPSIS, RADAR, RADAR2, and Fuzzy RADAR

In the following, five algorithms are implemented for KPI prioritization (for the fuzzy TOPSIS method, two algorithms with different normalization rules are applied). The fuzzy TOPSIS, RADAR, and RADAR2 methods are found in the literature; the fuzzy RADAR method approach is introduced herein as an extension of the RADAR method. In Table 4, the crisp scale variables (RADAR and RADAR2) and the linguistic scale variables (fuzzy TOPSIS and fuzzy RADAR) for the estimation of the KPIs are introduced.

Table 4. Linguistic variables for the importance of the KPIs with respect to the criteria.

The Relative Importance of the KPIs with Respect to the Criteria in Terms of Linguistic Variables	Crisp AHP Scale	Fuzzy Triangular	Reciprocal Fuzzy
Very weak (VW)	1	1, 1, 1	1, 1, 1
Very weak to weak (VW-W)	2	1, 2, 3	1/3, 1/2, 1
Weak (W)	3	2, 3, 4	1/4, 1/3, 1/2
Weak to average (W-A)	4	3, 4, 5	1/5, 1/4, 1/3
Average (A)	5	4, 5, 6	1/6, 1/5, 1/4
Average to strong (A-S)	6	5, 6, 7	1/7, 1/6, 1/5
Strong (S)	7	6, 7, 8	1/8, 1/7, 1/6
Strong to very strong (S-VS)	8	7, 8, 9	1/9, 1/8, 1/7
Very strong (VS)	9	8, 9, 9	1/9, 1/9, 1/8

The expert group used for the estimation of the criteria was also employed for the estimation of KPIs. Each expert performed the evaluation of the KPIs with respect to the criteria in terms of the linguistic variables given in Table 5.

Table 5. Example of a pair-wise comparison matrix for criteria.

	Specific	Measurable	Available	Relevant	Timely	Explainable	Relative
Specific	EqP	1/MP	MP	1/MP	EqP	EqP	SP
Measurable	MP	EqP	EqP	EqP	MP	EqP	M-SP
Available	1/MP	EqP	EqP	EqP	EqP	1/Eq-MP	MP
Relevant	MP	EqP	EqP	EqP	Eq-MP	EqP	EqP
Timely	EqP	1/MP	EqP	1/Eq-MP	EqP	MP	EqP
Explainable	EqP	EqP	Eq-MP	EqP	1/MP	EqP	SP
Relative	1/SP	1/M-SP	1/MP	EqP	EqP	1/SP	EqP

The decision-maker’s results are converted into fuzzy triangular numbers and aggregated by using the arithmetic mean as follows:

$$x_{ij}={\displaystyle \frac{1}{K}}\sum _{k=1}^K{x}_{ijk},$$

(11)

where $K$ is the number of decision-makers and $x_{ijk}$ stands for the rating of KPI $i$ to criterion $j$ given by the $k$-th decision-maker.

3.4.1. KPI Prioritization Using Fuzzy TOPSIS

Let us start the KPI prioritization methods descriptions with one of the most popular methods, fuzzy TOPSIS. First, the normalization of the fuzzy decision matrix was performed. Herein, all SMARTER criteria are considered to be benefit type (higher value is better). The following approaches are available in the literature for normalization rules [23,54]:

$$Y_{ij}=\left(\begin{array}{ccc}{\displaystyle \frac{l_{ij}}{u^{max}}},& {\displaystyle \frac{m_{ij}}{u^{max}}}& {\displaystyle \frac{u_{ij}}{u^{max}}}\end{array}\right), \mathrm{with} u^{max}=\underset{\mathit{ij}}{\mathit{max}}{u}_{ij},$$

(12)

$$Y_{ij}=\left(\begin{array}{ccc}{\displaystyle \frac{l_{ij}}{u_j^{max}}},& {\displaystyle \frac{m_{ij}}{u_j^{max}}}& {\displaystyle \frac{u_{ij}}{u_j^{max}}}\end{array}\right), \mathrm{with} u_j^{max}=\underset{i}{\mathit{max}}{u}_{ij},$$

(13)

$$Y_{ij }=\left(\begin{array}{ccc}{\displaystyle \frac{l_{ij}}{\sqrt{\sum _{i=1}^M{l}_{ij}^2}}},& {\displaystyle \frac{m_{ij}}{\sqrt{\sum _{i=1}^M{m}_{ij}^2}}},& {\displaystyle \frac{u_{ij}}{\sqrt{\sum _{i=1}^M{u}_{ij}^2}}}\end{array}\right).$$

(14)

where $M$ is the number of KPIs and $u_j^{max}$ and $u^{max}$ stand for the maximum values through criteria $j$ and the entire decision matrix, respectively. The first two normalization rules represent linear transformations, and the third rule corresponds to the vector norm. Use of Equation (10) is not justified in cases where the values corresponding to different criteria differ substantially. Herein the fuzzy TOPSIS algorithm is implemented corresponding to normalization rules (13) (fuzzy TOPSIS_Linear) and (14) (fuzzy TOPSIS_Vector), respectively. Note that in the normalization rule (14) formulas corresponding to the benefit criteria are considered (herein, all criteria are assumed to be benefit type).

The weights of the criteria $w_j,$ obtained by applying fuzzy AHP, are utilized to compute the weighted normalized decision matrix $v_{ij}$ as follows:

$$v_{ij}=y_{ij} \ast w_j, i=1,\dots ,M, j=1,\dots ,N.$$

(15)

where $v_{ij}$ stands for elements of the weighted normalized fuzzy decision matrix, and $M$ and $N$ denote the number of KPIs and criteria, respectively.

In order to evaluate KPIs, the fuzzy PIS and NIS are determined, and the distances of each alternative to these points are computed. Herein, the following definitions of the PIS and NIS are employed [54]:

$$PIS=\left(v_1^{+},v_2^{+},\dots ,v_N^{+}\right), NIS=\left(v_1^{-},v_2^{-},\dots ,v_N^{-}\right),$$

(16)

where $v_j^{+}=\left(1, 1, 1\right)$ and $v_j^{-}=\left(0, 0, 0\right)$ for $j=1, 2,\dots ,N$. The distances of each KPI from PIS and NIS are computed as follows:

$$\begin{gathered} d_i^{+}={\sum }_{j=1}^{N}d\left(v_{ij},v_j^{+}\right), i=1,\dots ,M, \\ {d}_i^{-}={\sum }_{j=1}^{N}d\left(v_{ij},v_j^{-}\right), i=1,\dots ,M. \end{gathered}$$

(17)

where $d\left(x,y\right)$ denotes the distance between two fuzzy numbers and is given as follows:

$$d\left(x,y\right)=\sqrt{\left({\displaystyle \frac{1}{3}}\right)\left[{\left(l_x-l_y\right)}^2+{\left(m_x-m_y\right)}^2+{\left(u_x-u_y\right)}^2\right]}$$

(18)

The similarities to an ideal solution can be computed as follows:

$${CC}_i={\displaystyle \frac{d_i^{-}}{{d_i^{+}+d}_i^{-}}}, i=1,\dots ,M.$$

(19)

Final rankings of KPIs can be assigned based on the values of the similarity coefficients determined from Equation (19).

3.4.2. KPI Prioritization Using RADAR and RADAR2

The RADAR and RADAR2 methods are one of the newest MCDM methods introduced in 2024 in [37]. According to this new approach, the ranking of alternatives is based on distance and range [37,38,39].

Let us assume that pair-wise analysis of the alternatives and criteria is performed and the decision matrix $x_{ij}$ is composed of decision-makers. It is also assumed that the aggregation of the individual decision matrices is performed. Most of the MCDM methods, including fuzzy TOPSIS, considered above continue with the treatment of the aggregated decision matrix by applying normalization, weighting, etc. The RADAR and RADAR2 methods used a principally different approach. Based on the aggregated decision matrix, two new matrices are introduced: the maximum proportion matrix and the minimum proportion matrix. According to the RADAR method, the maximum proportion matrix is defined as follows:

$${\alpha }_{ij}={\displaystyle \frac{\frac{\underset{i}{\mathit{max}}{x}_{ij}}{x_{ij}}}{\left(\frac{\underset{i}{\mathit{max}}{x}_{ij}}{x_{ij}} + \frac{x_{ij}}{\underset{i}{\mathit{min}}{x}_{ij}}\right)}}, i=1,\dots ,M, j=1,\dots ,N,$$

(20)

and the minimum proportion matrix as follows [37]:

$${\beta }_{ij}={\displaystyle \frac{\frac{x_{ij}}{\underset{i}{\mathit{min}}{x}_{ij}}}{\left(\frac{\underset{i}{\mathit{max}}{x}_{ij}}{x_{ij}} + \frac{x_{ij}}{\underset{i}{\mathit{min}}{x}_{ij}}\right)}}, i=1,\dots ,M, j=1,\dots ,N.$$

(21)

Note that the formulas corresponding to the cost criteria are omitted for the sake of conciseness (all criteria used are assumed to be benefit type).

According to the RADAR2 method, the maximum proportion matrix is defined as follows [38]:

$${\alpha }_{ij}={\displaystyle \frac{\underset{i}{\mathit{max}}{x}_{ij}-x_{ij}}{\left(\left(\underset{i}{\mathit{max}}{x}_{ij}-x_{ij}\right) + \left(x_{ij}-\underset{i}{\mathit{min}}{x}_{ij}\right)\right)}}, i=1,\dots ,M, j=1,\dots ,N,$$

(22)

and the minimum proportion matrix as follows [37]:

$${\beta }_{ij}={\displaystyle \frac{x_{\mathit{ij}}-\underset{i}{\mathit{min}}{x}_{ij}}{\left(\left(\underset{i}{\mathit{max}}{x}_{ij}-x_{ij}\right) + \left(x_{ij}-\underset{i}{\mathit{min}}{x}_{ij}\right)\right)}}, i=1,\dots ,M, j=1,\dots ,N.$$

(23)

The relative relationship can be computed for both methods (RADAR, RADAR2) as follows:

$${RR}_{ij}={\displaystyle \frac{{\alpha }_{ij}}{\left({\beta }_{ij} + \left|{\alpha }_{ij}-{\beta }_{ij}\right|\right)}}, i=1,\dots ,M, j=1,\dots ,N,$$

(24)

and the weighted relative relationship matrix as follows [39]:

$$W{RR}_{ij }={RR}_{ij}{\omega }_j, i=1,\dots ,M, j=1,\dots ,N.$$

(25)

For the final ranking, the aggregated ranking index is computed as follows:

$${RI}_i={\displaystyle \frac{\underset{i}{\mathit{min}}\sum _{j=1}^N{WRR}_i}{\sum _{j=1}^N{WRR}_i}}, i=1,\dots ,M.$$

(26)

The last equations, Equations (24) and (25), hold good for RADAR and RADAR2 methods.

3.4.3. KPI Prioritization Using Fuzzy RADAR

In the current study, the RADAR method for KPI prioritization is extended to the fuzzy RADAR method (an approach similar to [37] with different rules for computing relative relationship and weighted relative relationship). The incorporation of fuzzy set theory and linguistic variables facilitates the use of interval estimates, thereby simplifying the judgment process for decision-makers. The decision matrix $x_{ij}$ is obtained by converting the linguistic grades to triangular fuzzy numbers according to Table 4, as follows:

$$x_{ij}=\left(\begin{array}{ccc}{l}_{ij},& m_{ij},& u_{ij}\end{array}\right), i=1,\dots ,M, j=1,\dots ,N.$$

(27)

For the fuzzy RADAR method, the maximum proportion matrix can be introduced as follows:

$${\alpha }_{ij}=\left(\begin{array}{ccc}{\displaystyle \frac{\frac{\underset{i}{\mathit{max}}{l}_{ij}}{u_{ij}}}{\left(\frac{\underset{i}{\mathit{max}}{u}_{ij}}{l_{ij}} + \frac{u_{ij}}{\underset{i}{\mathit{min}}{l}_{ij}}\right)}},& {\displaystyle \frac{\frac{\underset{i}{\mathit{max}}{m}_{ij}}{m_{ij}}}{\left(\frac{\underset{i}{\mathit{max}}{m}_{ij}}{m_{ij}} +\frac{m_{ij}}{ \underset{i}{\mathit{min}}{m}_{ij}}\right)}},& {\displaystyle \frac{\frac{\underset{i}{\mathit{max}}{u}_{ij}}{l_{ij}}}{\left(\frac{\underset{i}{\mathit{max}}{l}_{ij}}{u_{ij}} + \frac{l_{ij}}{\underset{i}{\mathit{min}}{u}_{ij}}\right)}}\end{array}\right), i=1,\dots ,M, j=1,\dots ,N,$$

(28)

and the minimum proportion matrix as follows:

$${\beta }_{ij}=\left(\begin{array}{ccc}{\displaystyle \frac{\frac{l_{ij}}{\underset{i}{\mathit{min}}{u}_{ij}}}{\left(\frac{\underset{i}{\mathit{max}}{u}_{ij}}{l_{ij}} + \frac{u_{ij}}{\underset{i}{\mathit{min}}{l}_{ij}}\right)}},& {\displaystyle \frac{\frac{m_{ij}}{\underset{i}{\mathit{min}}{m}_{ij}}}{\left(\frac{\underset{i}{\mathit{max}}{m}_{ij}}{m_{ij}} + \frac{m_{ij}}{\underset{i}{\mathit{min}}{m}_{ij}}\right)}},& {\displaystyle \frac{\frac{u_{ij}}{\underset{i}{\mathit{min}}{l}_{ij}}}{\left(\frac{\underset{i}{\mathit{max}}{l}_{ij}}{u_{ij}} + \frac{l_{ij}}{\underset{i}{\mathit{min}}{u}_{ij}}\right)}}\end{array}\right), i=1,\dots ,M, j=1,\dots ,N.$$

(29)

The remaining characteristics of the fuzzy RADAR method, ${RR}_{ij}$, $W{RR}_{ij}$ and ${RI}_I,$ can be computed based on Equations (24)–(26), taking account that $j=1,\dots ,3N$. Also, the values of the aggregated ranking index ${RI}_I$ should be defuzzified to implement it in the final ranking.

4. Results

The results concerning EAM optimization are introduced in Section 2; herein, the focus is on the results of fuzzy AHP, fuzzy TOPSIS, RADAR, and RADAR2, and fuzzy RADAR methods.

4.1. Criteria Prioritization Results Using Fuzzy AHP

Table 5 shows the pair-wise comparison between SMARTER criteria as a sample.

After the consistency check, the linguistic values in the pair-wise comparison matrix are transformed to the corresponding TFN according to Table 3. The values of the $r_{ij}=\left(l_{ij},{ m}_{ij}, { u}_{ij}\right)$, computed according to Equation (4), are presented in Table 6.

Table 6. Aggregated pair-wise comparison matrix.

	Specific	Measurable	Available	Relevant	Timely	Explainable	Relative
Specific	(1.00; 1.00; 1.00)	(0.34; 0.46; 0.66)	(0.64; 0.87; 1.25)	(0.26; 0.35; 0.57)	(0.70; 0.96; 1.30)	(0.61; 0.80; 1.08)	(1.43; 1.63; 1.89)
Measurable	(1.52; 2.19; 2.93)	(1.00; 1.00; 1.00)	(0.92; 1.29; 1.64)	(0.67; 0.94; 1.32)	(1.52; 2.22; 2.86)	(0.57; 0.71; 0.94)	(2.86; 3.98; 5.04)
Available	(0.80; 1.15; 1.55)	(0.61; 0.77; 1.08)	(1.00; 1.00; 1.00)	(0.68; 0.80; 0.90)	(0.80; 0.96; 1.25)	(0.32; 0.41; 0.57)	(2.00; 3.00; 4.00)
Relevant	(1.74; 2.83; 3.87)	(0.76; 1.06; 1.49)	(1.11; 1.25; 1.48)	(1.00; 1.00; 1.00)	(1.74; 2.46; 3.10)	(1.52; 1.93; 2.30)	(2.35; 3.02; 3.64)
Timely	(0.77; 1.05; 1.43)	(0.35; 0.45; 0.66)	(0.80; 1.04; 1.25)	(0.26; 0.35; 0.57)	(1.00; 1.00; 1.00)	(0.34; 0.46; 0.66)	(1.64; 2.05; 2.40)
Explainable	(0.92; 1.25; 1.64)	(0.74; 0.98; 1.32)	(0.92; 1.29; 1.64)	(0.44; 0.52; 0.66)	(1.52; 2.19; 2.93)	(1.00; 1.00; 1.00)	(2.30; 2.95; 3.57)
Relative	(0.53; 0.61; 0.70)	(0.20; 0.25; 0.35)	(0.25; 0.33; 0.50)	(0.27; 0.33; 0.43)	(0.42; 0.49; 0.61)	(0.28; 0.34; 0.44)	(1.00; 1.00; 1.00)

The fuzzy weights, crisp weights, and normalized crisp weights computed using Equations (6) and (7) are given in Table 7.

Table 7. Fuzzy and crisp weights of the criteria.

Criteria	Fuzzy Weights	Normalized Crisp Weights	Rank
Specific	(0.07; 0.10; 0.18)	0.107	5
Measurable	(0.12; 0.20; 0.32)	0.199	2
Available	(0.08; 0.13; 0.21)	0.131	4
Relevant	(0.14; 0.24; 0.37)	0.232	1
Timely	(0.06; 0.10; 0.17)	0.105	6
Explainable	(0.11; 0.17; 0.27)	0.168	3
Relative	(0.04; 0.06; 0.09)	0.058	7

In the last column of Table 7, the rank of the criteria is given based on the value of the normalized crisp weight. The results obtained allow us to conclude that the most valuable metrics for the considered problem are relevant, measurable, and explainable.

4.2. KPI Prioritization Using the Fuzzy TOPSIS Method

The sample of the evaluation results of one decision-maker is given in Table 8, where KPIs are evaluated against criteria.

Table 8. Performance evaluations of the KPIs.

	Specific	Measurable	Available	Relevant	Timely	Explainable	Relative
KPI1	S to VS	VS	VS	S	A to S	S	A to S
KPI2	S to VS	S to VS	S to VS	S	A to S	A to S	A to S
KPI3	VS	S	S	A to S	A to S	A to S	A
KPI4	W	S	S	A to S	S	A	A
KPI5	W	S to VS	S	S	A to S	A	A
KPI6	V-W to W	S	S to VS	S	A to S	W	S
KPI7	W to A	S	S to VS	A	S to VS	W to A	S
KPI8	A to S	VS	VS	S to VS	VS	A	VS
KPI9	A	S to VS	A to S	A to S	S to VS	S	S to VS
KPI10	A	S to VS	A to S	A to S	S to VS	S	S to VS
KPI11	VS	VS	VS	S	VS	A to S	S to VS
KPI12	W to A	S to VS	A to S	S	S	A	A to S
KPI13	W	S	A to S	A to S	S	W to A	A to S

The aggregated fuzzy triangular numbers $x_{ij}=\left(l_{ij}, { m}_{ij},{ u}_{ij}\right)$, representing the pair-wise comparison matrix criteria versus KPIs, are computed using Equation (11). Employing linear, criteria-wise normalization rule (13) and Equation (15), the weighted normalized fuzzy decision matrix can be evaluated as presented in Table 9.

Table 9. Weighted normalized fuzzy decision matrix using normalization rule (13).

	Specific	Measurable	Available	Relevant	Timely	Explainable	Relative
KPI1	(0.05; 0.08; 0.15)	(0.11; 0.19; 0.31)	(0.08; 0.13; 0.21)	(0.13; 0.22; 0.36)	(0.04; 0.07; 0.12)	(0.08; 0.13; 0.22)	(0.04; 0.05; 0.09)
KPI2	(0.04; 0.07; 0.14)	(0.07; 0.13; 0.24)	(0.06; 0.10; 0.18)	(0.10; 0.17; 0.29)	(0.04; 0.07; 0.12)	(0.08; 0.13; 0.21)	(0.04; 0.05; 0.09)
KPI3	(0.07; 0.10; 0.18)	(0.10; 0.18; 0.30)	(0.08; 0.12; 0.21)	(0.13; 0.21; 0.35)	(0.04; 0.07; 0.12)	(0.11; 0.17; 0.27)	(0.02; 0.04; 0.07)
KPI4	(0.03; 0.05; 0.10)	(0.08; 0.14; 0.26)	(0.07; 0.11; 0.20)	(0.13; 0.21; 0.35)	(0.05; 0.09; 0.16)	(0.07; 0.12; 0.21)	(0.02; 0.04; 0.07)
KPI5	(0.05; 0.08; 0.15)	(0.12; 0.20; 0.32)	(0.07; 0.11; 0.20)	(0.10; 0.17; 0.29)	(0.02; 0.05; 0.09)	(0.10; 0.16; 0.26)	(0.02; 0.04; 0.07)
KPI6	(0.03; 0.06; 0.11)	(0.06; 0.10; 0.20)	(0.05; 0.09; 0.16)	(0.11; 0.19; 0.32)	(0.04; 0.07; 0.12)	(0.07; 0.11; 0.19)	(0.03; 0.04; 0.07)
KPI7	(0.04; 0.07; 0.14)	(0.08; 0.14; 0.26)	(0.05; 0.09; 0.16)	(0.09; 0.16; 0.27)	(0.06; 0.09; 0.16)	(0.07; 0.12; 0.20)	(0.03; 0.04; 0.07)
KPI8	(0.05; 0.09; 0.16)	(0.10; 0.17; 0.29)	(0.05; 0.09; 0.16)	(0.10; 0.17; 0.30)	(0.06; 0.09; 0.16)	(0.09; 0.15; 0.25)	(0.04; 0.06; 0.09)
KPI9	(0.05; 0.08; 0.16)	(0.11; 0.18; 0.31)	(0.07; 0.11; 0.19)	(0.14; 0.24; 0.37)	(0.06; 0.09; 0.16)	(0.09; 0.15; 0.25)	(0.04; 0.06; 0.09)
KPI10	(0.05; 0.08; 0.16)	(0.09; 0.16; 0.29)	(0.07; 0.11; 0.19)	(0.13; 0.21; 0.35)	(0.06; 0.09; 0.16)	(0.08; 0.13; 0.22)	(0.04; 0.06; 0.09)
KPI11	(0.04; 0.07; 0.14)	(0.03; 0.07; 0.15)	(0.05; 0.09; 0.16)	(0.13; 0.22; 0.36)	(0.06; 0.10; 0.17)	(0.11; 0.17; 0.27)	(0.04; 0.06; 0.09)
KPI12	(0.05; 0.08; 0.15)	(0.08; 0.15; 0.27)	(0.05; 0.08; 0.15)	(0.13; 0.22; 0.36)	(0.06; 0.10; 0.17)	(0.07; 0.12; 0.21)	(0.03; 0.04; 0.07)
KPI13	(0.05; 0.08; 0.16)	(0.06; 0.10; 0.20)	(0.05; 0.08; 0.15)	(0.13; 0.21; 0.35)	(0.04; 0.07; 0.13)	(0.07; 0.12; 0.20)	(0.03; 0.04; 0.07)

The distances from PIS and NIS, the similarity coefficients, and the rankings of the KPIs obtained by utilizing the linear, criteria-wise normalization rule (13) are presented in Table 10.

Table 10. Final ranking of KPIs using normalization rule (13).

KPI	${\mathit{d}}_{\mathit{i}}^{+}$	${\mathit{d}}_{\mathit{i}}^{-}$	${\mathit{C}\mathit{C}}_{\mathit{i}}$	Rank
Inventory turnover (KPI1)	6.067	1.024	0.144	3
Percentage of additional freight costs (KPI2)	6.210	0.875	0.124	10
Product price/quality ratio (KPI3)	6.040	1.053	0.148	2
First pass yield/throughput yield (KPI4)	6.162	0.928	0.131	9
Defects per unit (KPI5)	6.126	0.963	0.136	6
Employees’ efficiency (KPI6)	6.266	0.816	0.115	13
Changes in implementation time (KPI7)	6.217	0.869	0.123	11
Actual production time (KPI8)	6.103	0.987	0.139	5
Overall equipment effectiveness (KPI9)	6.017	1.078	0.152	1
Net equipment effectiveness (KPI10)	6.072	1.024	0.144	4
On-time delivery (KPI11)	6.152	0.934	0.132	8
Tact time (KPI12)	6.136	0.955	0.135	7
Unit/line reliability (KPI13)	6.217	0.869	0.123	12

As can be observed from Table 10, in the case of the considered problem, the fuzzy AHP and fuzzy TOPSIS method-based solution leads to the results where the most important impact factors can be outlined as OEM (overall equipment effectiveness), product price/quality ratio, and inventory turnover.

The final ranking of KPIs obtained by utilizing the vector norm-based nonlinear normalization rule (14) is given in Table 11.

Table 11. Final ranking of KPIs using normalization rule (14).

KPI	${\mathit{d}}_{\mathit{i}}^{+}$	${\mathit{d}}_{\mathit{i}}^{-}$	${\mathit{C}\mathit{C}}_{\mathit{i}}$	Rank
Inventory turnover (KPI1)	6.697	0.325	0.0462	3
Percentage of additional freight costs (KPI2)	6.746	0.275	0.0392	10
Product price/quality ratio (KPI3)	6.689	0.332	0.0473	2
First pass yield/throughput yield (KPI4)	6.730	0.291	0.0415	9
Defects per unit (KPI5)	6.717	0.304	0.0433	6
Employees’ efficiency (KPI6)	6.765	0.256	0.0364	13
Changes in implementation time (KPI7)	6.747	0.274	0.0390	11
Actual production time (KPI8)	6.710	0.312	0.0444	5
Overall equipment effectiveness (KPI9)	6.682	0.340	0.0484	1
Net equipment effectiveness (KPI10)	6.700	0.323	0.0459	4
On-time delivery (KPI11)	6.728	0.293	0.0417	8
Tact time (KPI12)	6.721	0.300	0.0428	7
Unit/line reliability (KPI13)	6.749	0.272	0.0388	12

It can be observed from Table 10 and Table 11 that in the case of both normalization rules (linear criteria-wise (13) and nonlinear vector norm-based (14)), the same final ranking is obtained.

4.3. KPI Prioritization Using the RADAR, RADA2, and Fuzzy RADAR Method

Based on theoretical formulas, given in Section 3.4.2. for the RADAR and RADAR2 methods, and in Section 3.4.3. for the fuzzy RADAR method, the aggregated ranking indexes RI and final rankings are presented in Table 12 as follows.

Table 12. Final ranking of KPIs obtained using RADAR methods.

KPI	RI RADAR	RANK RADAR	RI RADAR2	RANK RADAR2	RI Fuzzy RADAR	RANK Fuzzy RADAR
KPI1	0.923	3	0.843	3	10.583	4
KPI2	0.615	11	0.255	13	7.831	12
KPI3	0.940	2	0.907	2	11.591	2
KPI4	0.709	9	0.384	9	9.698	7
KPI5	0.736	8	0.404	8	9.575	8
KPI6	0.567	13	0.269	12	7.111	13
KPI7	0.639	10	0.290	11	8.345	10
KPI8	0.817	5	0.498	6	9.150	9
KPI9	1.000	1	1.000	1	12.338	1
KPI10	0.844	4	0.592	4	10.783	3
KPI11	0.806	6	0.589	5	9.921	5
KPI12	0.767	7	0.420	7	9.848	6
KPI13	0.614	12	0.312	10	8.080	11

It can be observed from Table 10, Table 11 and Table 12 that the rankings of the KPIs obtained by applying the TOPSIS and RADAR methods are similar. Detailed analysis of the rankings is given in the next section.

4.4. Comparison and Analysis of the KPI Rankings

The rankings of the KPIs obtained by applying the five algorithms considered are summarized in Table 13.

Table 13. Final ranking of KPIs for five methods considered.

KPI	Fuzzy TOPSIS Rule (13)	Fuzzy TOPSIS Rule (14)	RADAR	RADAR2	Fuzzy RADAR
KPI1	3	3	3	3	4
KPI2	10	10	11	13	12
KPI3	2	2	2	2	2
KPI4	9	9	9	9	7
KPI5	6	6	8	8	8
KPI6	13	13	13	12	13
KPI7	11	11	10	11	10
KPI8	5	5	5	6	9
KPI9	1	1	1	1	1
KPI10	4	4	4	4	3
KPI11	8	8	6	5	5
KPI12	7	7	7	7	6
KPI13	12	12	12	10	11

It can be observed from Table 13 that in the case of all five algorithms the most important impact factors can be outlined as OEM (overall equipment effectiveness) and product price/quality ratio (in the case of four methods, inventory turnover is third). Also, the Employees’ efficiency has the least impact in the case of all five algorithms. In order to obtain a more precise picture of the similarities of the KPI rankings obtained, the WS coefficient is computed as follows [37]:

$$WS=1-{\sum }_{i=1}^N{2}^{-Rxi}{\displaystyle \frac{\left|Rxi-Ryi\right|}{\mathit{max}\left(\left|1-Rxi\right|,\left|N-Rxi\right|\right)}},$$

(30)

where $\mathit{R}\mathit{x}\mathit{i}$ and $\mathit{R}\mathit{y}\mathit{i}$ stand for the average rank position for alternative $\mathit{i}$ and $\mathit{N}$ is the total number of alternatives. In Table 14 the WS coefficient values are given for all available pairs of the five algorithms.

Table 14. WS coefficient matrix.

	Fuzzy TOPSIS (13)	Fuzzy TOPSIS (14)	RADAR	RADAR2	Fuzzy RADAR
Fuzzy TOPSIS (13)	X	1.000	0.9943	0.9896	0.9567
Fuzzy TOPSIS (14)		X	0.9943	0.9896	0.9567
RADAR			X	0.9936	0.9708
RADAR2				X	0.9748
Fuzzy RADAR					X

It should be mentioned that only the upper part of Table 14 is filled since the WS coefficient matrix is symmetric. As mentioned above, the KPI rankings corresponding to the two fuzzy TOPSIS algorithms coincide. In the case where WS coefficient values exceed 0.808, the similarity in rankings is considered excellent [37]. Obviously, all computed WS values exceed 0.808, and similarity can be considered very strong between all rankings obtained with the five algorithms considered. Thus, all results obtained are in good agreement.

In Table 15, the current TOPSIS (normalization rule 13) ranking is compared with rankings corresponding to maximal and minimal values of the expert estimates for the most influential KPI9.

Table 15. Fuzzy TOPSIS comparison.

KPI	Actual	Max	Lower
KPI1	3	3	3
KPI2	10	10	12
KPI3	2	2	5
KPI4	9	8	8
KPI5	6	6	2
KPI6	13	13	13
KPI7	11	11	11
KPI8	5	5	6
KPI9	1	1	1
KPI10	4	4	4
KPI11	8	9	9
KPI12	7	7	7
KPI13	12	12	10

It can be observed from Table 15 that even using maximal values of the expert estimates for the most influential KPI, KPI9, leads to marginal changes in the rankings, but using minimal values for the expert estimates received for KPI9 has remarkable changes. However, the similarity index value corresponding to actual and minimal ranking values is equal to 0.894, still exceeding the limit value of 0.808, meaning that the similarity between these rankings is strong [37].

5. Conclusions

An integrated EAM and fuzzy AHP-TOPSIS/RADAR methods-based approach for the prioritization of key performance indicators is developed.

An approach introduced for the selection and ranking of KPIs is implemented on a sample of EU private companies (SMEs) conducting business in the EU market. Thus, the most direct practical result is a list of ranked KPIs for a particular company.

A cloud-based platform was developed to implement the survey. This platform can be utilized for data collection and analysis for companies of various sizes, fields of activity, etc. The procedure, developed for KPI selection, is applicable to similar SMEs without remarkable changes.

The research contribution is as follows:

• Development of an integrated model for KPI prioritization (EAM and MCDM).
• Development of a fuzzy RADAR method approach for KPI prioritization.
• Implementation of two TOPSIS and three RADAR algorithms for KPI prioritization, performing similarity analysis.

The societal contribution is as follows:

• The cloud-based platform developed enables the democratization of access to sophisticated performance management tools for resource-constrained SMEs, representing a significant achievement that deserves explicit discussion.
• The experience gained from this research suggests that it is beneficial to make joint consensual decisions in collaboration between academia and industry.

Limitations of the study algorithms and models may need certain tuning/adaptation to consider the features of a particular company.

In future studies, refinement of the fuzzy RADAR method and development of the fuzzy RADAR method are foreseen, as well as the application and validation of these methods.

In addition, to improve operational efficiency the developed KPI selection methodology contributes to sustainability by enabling companies to identify and prioritize indicators that align with long-term environmental and socio-economic goals. The inclusion of SMARTER criteria, combined with fuzzy AHP and TOPSIS, allows SMEs to systematically assess which performance metrics best support sustainable production, resource optimization, and continuous improvement. This approach reinforces the role of KPI-based decision-making in the broader context of sustainable industrial development.