A Linguistic q-Rung Orthopair ELECTRE II Algorithm for Fuzzy Multi-Criteria Ontology Ranking

Abstract

In recent years, interest in the application of ontologies in various domains of knowledge has grown significantly. Ontologies are widely used in a myriad of areas, such as artificial intelligence, data integration, knowledge management, and the semantic web, to name but a few. However, despite the widespread adoption, there exist a range of problems associated with ontologies, such as the complexity and cognitive challenges associated with ontology engineering, design, and development. One of the solutions to these challenges is to reuse existing ontologies rather than developing new ontologies afresh for new applications. The reuse of ontologies that describe a knowledge domain is a complex task consisting of many aspects. One of the key aspects involves ranking ontologies to aid in their selection. Various techniques have been proposed for this task, but many of them fall short in their expressiveness and ability to capture the cognitive aspects of human-like decision-making processes. Furthermore, much of the existing research focuses on an objective approach to ontology ranking, but it is unquestionable that a wide range of aspects pertaining to the quality of an ontology simply cannot be captured in a quantitative manner. Existing ranking models fail to provide a robust and flexible canvas for facilitating qualitative ontology ranking and selection for reuse. To address the aforementioned shortcomings of existing ontology ranking approaches, this study proposes a novel algorithm for ranking ontologies that extends the Elimination and Choice Translating Reality (ELECTRE) multi-criteria decision-making method with the Linguistic q-Rung Orthopair Fuzzy Set (Lq-ROFS-ELECTRE II), allowing the expression of uncertainty in a more robust and precise manner. The new Lq-ROFS-ELECTRE II algorithm was applied to rank a set of 19 ontologies of the machine learning (ML) domain. The ML ontologies were evaluated using a set of seven qualitative criteria extracted from the Ontometric framework. The proposed Lq-ROFS-ELECTRE II algorithm was then applied to rank the 19 ontologies in light of the seven criteria. The ranking results obtained were compared against the quantitative rankings of the same 19 ontologies using the traditional ELECTRE II algorithm, and confirmed the validity of the ranking performed by the proposed Lq-ROFS-ELECTRE II algorithm and its effectiveness in the task of ontology ranking. Furthermore, a comparative analysis of the proposed Lq-ROFS-ELECTRE II against existing MCDM methods and other existing fuzzy ELECTRE II methods displayed its superior modeling capabilities that allow for more natural decision evaluation from subject experts in real-world applications and allow the decision-maker to have much flexibility in expressing their preferences. These capabilities of the Lq-ROFS-ELECTRE II algorithm make it applicable not only in ontology ranking, but in any domain where there exist decision-making scenarios that comprise multiple conflicting criteria under uncertainty.

1. Introduction

Ontology is a structured representation of knowledge within a specific domain that defines a set of concepts and the relationships between them. Ontologies play a crucial role in representing and managing domain knowledge to exploit big data and artificial intelligence. In a world of diverse and heterogeneous data sources, it becomes a major challenge to make sense of information that uses different terms or formats. Ontologies solve this by providing a common language and shared understanding in order to bridge the gap between heterogeneous and disparate data sources. Ontologies are widely used in a myriad of areas, such as artificial intelligence, data integration, knowledge management, and the semantic web, where they help to enhance machine understanding, support reasoning, and ensure interoperability between the different data sources. The core components of an ontology include classes or concepts, properties, individuals or instances, and axioms. Furthermore, there has been a proliferation of tools to aid the ontology development process, such as Protégé and the Web Ontology Language. In recent years, ontologies have seen widespread adoption in various applications in diverse fields, such as finance [1], mathematics [2], robotics [3], healthcare [4], and education [5]. However, despite the successful adoption and benefits, there exist a range of problems associated with ontologies. Some of the main challenges to the adoption and integration of ontology include the following [6,7]:

• The complexity and cognitive challenges associated with ontology engineering, design, and development.
• The steepness of entry for ontology adoption and the usability of their development tools.
• The lack of standardization and ambiguity within the domain terminology.
• Limited/lack of resources and resistance from stakeholders.
• Unfriendly user interfaces (UI) and UI interactions with ontology tools.

One of the solutions to address the aforementioned challenges is to reuse existing ontologies, possibly with some modifications, rather than developing new ontologies de novo [8,9]. This allows developers to leverage established domain vocabularies and accelerate the design process, ensuring improved consistency and interoperability. Reusing existing ontologies also promotes standardization and facilitates data integration within the same domain. However, ontology reuse in itself can be a complex and unintuitive process, requiring many stages. Oftentimes there may exist a vast number of ontologies applicable for reuse pertaining to a use-case. The task of evaluating and selecting the best ontology then becomes increasingly complex [10].

In order to reduce the complexity associated with selecting appropriate ontologies for reuse, many authors have developed strategies to rank ontologies based on their qualities, features, and characteristics [11,12,13,14,15]. In recent years, there has been interest in the applications of decision modeling and computational intelligence methods to aid in the selection of ontologies. This is considered a more viable technique because of the large amounts of data inherent in ontologies, along with the complexity and nuances associated with modeling the human decision-making process. Authors have made use of advanced techniques such as multi-criteria decision-making (MCDM) [9,10,16,17,18] for the task of ontology ranking and selection. These methods include the Technique for Order of Preference by Similarity to Ideal Solution (TOPSIS), ELECTRE, Weighted Sum Model, and the Weighted Product Model. But much of these works had the following shortcomings.

• The existing studies rank ontologies according to their quantitative metrics (numerical) as opposed to their qualitative metrics (in natural language). This is due to the inability of the models to process, capture, and exploit qualitative information. Very little work has been conducted on expressive decision-making that is able to capture qualitative criteria effectively.
• Almost all of the existing work conducted on ontology ranking is based on a singular perspective. However, in most real-world scenarios, there will be a group of stakeholders involved in the decision-making process. There is a lack of research focused on group decision-making for ontology selection.
• Most of the existing work does not take into account the users’ perspective. Accordingly, most techniques and current systems are not very user-friendly and users’ may often find it complex and tedious to express their opinions and judgments.

To address the aforementioned shortcomings, this research proposes a multi-criteria group decision-making algorithm based on the ELECTRE II method and the Linguistic q-Rung Orthopair Fuzzy Set (Lq-ROFS-ELECTRE II). The ELECTRE methods are a family of robust and widely applied MCDM methods. There are five variants of ELECTRE, which are ELECTRE I, II, III, IV, and Tri. Particularly, ELECTRE has been applied for the task of ontology ranking previously [10,16,18]. To further allow for extended modeling capabilities authors have integrated ELECTRE methods with fuzzy sets [19,20,21,22]. Distinctively, ELECTRE II is a robust and scalable method that has proven its viability for various tasks, especially ontology selection. On the other hand, the Linguistic q-Rung Orthopair Fuzzy Set (Lq-ROFS) is a powerful tool for enabling flexible and expressive decision-making, relaxing the constraints of other fuzzy sets such as Pythagorean fuzzy sets, hesitant fuzzy sets, and q-Rung Orthopair Fuzzy Sets. Furthermore, Lq-ROFS allows for decision-makers to provide their evaluations using a set of linguistic variables. It provides an extremely flexible structure for capturing the cognitive aspect of human-like decision-making. This enables decision-makers, such as stakeholders and domain experts, to express their judgments and perspectives in a manner that feels natural to them in an ontology selection process. The combination of ELECTRE II and Lq-ROFS results in an extremely powerful algorithm capable of efficiently and effectively solving the challenges of ontology selection explained above.

The contributions of this study are as follows.

• A novel Lq-ROFS-ELECTRE II algorithm is proposed that enables multiple decision-makers to evaluate decision-making problems in an effective manner, with the use of linguistic variables.
• A robust solution is proposed for the ontology ranking and selection problem by making use of multi-criteria group decision-making and fuzzy sets.
• A comprehensive survey and analysis is performed to extract the existing machine learning ontologies to promote their selection for reuse.

The rest of this paper is structured as follows. Section 2 reviews and discusses related literature. ELECTRE II is presented, and the various concepts relevant to the specification of the proposed Lq-ROFS-ELECTRE II algorithm are defined in Section 3. In Section 4, the detailed specification of the proposed Lq-ROFS-ELECTRE II algorithm is presented. The experimental results of the application of the Lq-ROFS-ELECTRE II algorithm to rank a set of ontologies are described in Section 5. Section 6 discusses the application of the proposed Lq-ROFS-ELECTRE II algorithm in real-world scenarios in other knowledge domains, and the paper is concluded in Section 7.

2. Literature Review

Many ranking methodologies have been proposed for ontology selection. A study by Lozano-Tello and Gomez-Perez [23] proposed the Ontometric framework to compare ontologies. This comprised a hierarchical taxonomy of features based upon five dimensions, namely content, language, methodology, tools, and costs. In Ref. [24] a content-based ranking system was proposed whereby the ontologies were evaluated based on their coverage of a particular domain of interest, using a calculated ontology score. Park et al. [25] developed a ranking model based on selection standards and metrics, enhancing semantic matching abilities. The model determines the specific metrics that should be used for ranking and assigns an appropriate weight to each metric. Another study by Butt et al. [26] introduced DWRank, a bi-directional graph walk ranking algorithm, that uses Learning-to-Rank methodologies to assign weights to the different ranking strategies it employs. Subhashini and Akilandeswari [14] developed the Onto-DSB method to rank ontologies according to their semantic web link as well as their internal structure.

Despite some success in the above-mentioned studies, there are various aspects to consider in an ontology ranking problem, and oftentimes these aspects conflict with each other. Some of these aspects include (i) the identification of suitable ontologies, (ii) the semantic compatibility, (iii) the licensing and legal considerations, and (iv) the documentation and provenance [8]. Accordingly, there has been an interest in applying multi-criteria decision-making algorithms to compare and select ontologies. A study by Esposito et al. [18] applied the ELECTRE I and III algorithms for the task of ontology ranking. Sooklall and Fonou-Dombeu [10] applied the ELECTRE I, II, III, and IV methods to rank a set of 200 biomedical ontologies according to 13 of their complexity metrics. Another study by Fonou-Dombeu and Viriri [17] proposed a framework known as C-Rank, which uses the Weighted Linear Combination Ranking technique (WLCRT) to rank ontologies. In ref. [16] a preference-disaggregation framework was applied to classify ontologies according to their complexity metric levels. A genetic algorithm was developed to determine the optimal thresholds in order to build an ELECTRE Tri method, which was then applied for classification of the ontologies. There have also been other MCDM methods applied to rank ontologies such as Technique of Preference by Similarity to Ideal Solution (TOPSIS), Weighted Sum Model (WSM), and Weighted Product Model (WPM) [9]. However, despite the success of the aforementioned MCDM models in ranking ontologies, they all make use of quantitative metrics only. Oftentimes it is equally important to evaluate ontologies according to the judgments of subject domain experts. This is achievable through the use of fuzzy-based MCDM. Therefore, this paper proposes a novel fuzzy-based MCDM algorithm, namely Lq-ROFS-ELECTRE II, that enables the evaluation of ontologies according to the judgments of subject domain experts. The ELECTRE methods are a family of outranking MCDM methods built on the concepts of concordance and discordance. There are five variants of ELECTRE, which are ELECTRE I [27], II [28], III [29], IV [30], and Tri [31]. The ELECTRE I method was the initial method and subsequent methods were developed to introduce additional modeling capabilities. ELECTRE versions I to IV perform ranking and ELECTRE Tri is an ordinal classification method based on predefined thresholds. The ELECTRE methods have been successfully applied for the task of ranking ontologies in the past and has proven to be viable, robust, and scalable [10,16,32]. To further increase the expressiveness of the methods, authors have proposed various extensions using fuzzy set theory. The Intuitionistic Fuzzy ELECTRE II method was proposed by Devadoss and Rekha [33], allowing for membership and non-membership degrees $u+v\le 1$, where u and v are the membership and non-membership degrees, respectively. This was extended with hesitant fuzzy sets by Chen and Xu [34] to allow membership degrees to comprise multiple values to capture decision-makers’ hesitance. The Pythagorean Fuzzy ELECTRE II method [19] was proposed in order to relax the constraint of the Intuitionistic Fuzzy ELECTRE II, such that $u^2+v^2\le 1$. Pinar and Boran [35] proposed the q-Rung Orthopair Fuzzy ELECTRE method which further generalizes the membership and non-membership constraints to $u^q+v^q\le 1$. This study contributes to the efforts by existing work to develop improved ELECTRE II versions by proposing a Linguistic q-Rung Orthopair Fuzzy variation to allow decision-makers to have enhanced modeling capabilities to capture both qualitative and uncertain information.

3. Preliminaries

In this section, the traditional ELECTRE II algorithm is presented, followed by the concepts of q-ROFS, Lq-ROFS, and Linguistic q-Rung Orthopair Fuzzy Numbers (Lq-ROFNs).

3.1. Traditional ELECTRE II

The ELECTRE II technique was developed by Roy and Bertier [28] in 1971 to handle ranking problems. The steps of ELECTRE II are as follows [28,36]:

• Step 1: Comparative Categories ${\mathcal{I}}^{-}$, ${\mathcal{I}}^{=}$, and ${\mathcal{I}}^{+}$

When considering alternatives $A_h$ and $A_k$, the criteria $C_j$ are sorted into sets according to the relations between those criterion pairs for $A_h$ and $A_k$, as in Equations (1)–(3). ${\mathcal{I}}^{+}$ contains those criteria where the $h^{th}$ alternative outperforms the $k^{th}$ alternative. ${\mathcal{I}}^{=}$ contains those criteria where two alternatives perform the same, and ${\mathcal{I}}^{-}$ contains those criteria where $A_h$ is outperformed by $A_k$. $g_j\left(A_h\right)$ represents the performance of the $h^{th}$ alternative in light of the $j^{th}$ criterion.

$$\begin{array}{cc}\hfill {\mathcal{I}}^{+}& =\{C_j\mid g_j\left(A_h\right)>g_j\left(A_k\right)\}\hfill \end{array}$$

(1)

$$\begin{array}{cc}\hfill {\mathcal{I}}^{=}& =\{C_j\mid g_j\left(A_h\right)=g_j\left(A_k\right)\}\hfill \end{array}$$

(2)

$$\begin{array}{cc}\hfill {\mathcal{I}}^{-}& =\{C_j\mid g_j\left(A_h\right)<g_j\left(A_k\right)\}\hfill \end{array}$$

(3)

The sets ${\mathcal{I}}^{+}$, ${\mathcal{I}}^{=}$, and ${\mathcal{I}}^{-}$ are aggregated, and the weights are summed up to form the sets $W^{+}$, $W^{=}$, and $W^{-}$ in Equations (4), (5) and (6), respectively. These sets represent the weights corresponding to the sets ${\mathcal{I}}^{+}$, ${\mathcal{I}}^{=}$, and ${\mathcal{I}}^{-}$, and $w_j$ is the importance weighting for the $j^{th}$ criterion.

$$\begin{array}{cc}\hfill W^{+}& =\sum _{j\in {\mathcal{I}}^{+}}{w}_j\hfill \end{array}$$

(4)

$$\begin{array}{cc}\hfill W^{=}& =\sum _{j\in {\mathcal{I}}^{=}}{w}_j\hfill \end{array}$$

(5)

$$\begin{array}{cc}\hfill W^{-}& =\sum _{j\in {\mathcal{I}}^{-}}{w}_j\hfill \end{array}$$

(6)

• Step 2: Index of Concordance

The index of concordance between every alternative pair is formulated. The degree of concordance $C(h,k)$ between $A_h$ and $A_k$ is quantified and calculated in Equation (7).

$$C(h,k)={\displaystyle \frac{W^{+}+W^{=}}{W^{+}+W^{=}+W^{-}}},\forall h\ne k$$

(7)

• Step 3: Discordance Index

The discordance index $D(h,k)$ for all alternative pairs $A_h$ and $A_k$ is given in Equation (8).

$$D(h,k)={\displaystyle \frac{\underset{j\in {\mathcal{I}}^{-}}{max}|g_j\left(A_h\right)-g_j\left(A_k\right)|}{\underset{j}{max}|g_j\left(A_h\right)-g_j\left(A_k\right)|}}$$

(8)

• Step 4: Concordance and Discordance Parameters

The decision-maker must define concordance parameters $\{p^{-},p^0,p^{*}\}$ such that the condition in Equation (9) is satisfied.

$$0\le p^{-}\le p^0\le p^{*}\le 1$$

(9)

They must also define discordance parameters $\{q^0,q^{*}\}$ such that the condition in Equation (10) holds.

$$0<q^0<q^{*}<1$$

(10)

• Step 5: Weak and Strong Outranking Relations

The strong outranking relation $A_h{S}_F{A}_k$ is defined in Equation (11).

$$A_h{S}_F{A}_k=\left\{\begin{array}{c}C(h,k)\ge C(k,h),\hfill \\ (C(h,k)\ge p^{*}\mathrm{and}D(h,k)\le q^{*})\hfill \\ \mathrm{or}(C(h,k)\ge p^0\mathrm{and}D(h,k)\le q^0)\hfill \end{array}\right.$$

(11)

The weak outranking relation $A_h{S}_f{A}_k$ is defined as in Equation (12).

$$A_h{S}_f{A}_k=\left\{\begin{array}{c}C(h,k)\ge C(k,h),\hfill \\ C(h,k)\ge p^{-}\mathrm{and}D(h,k)\le q^0\hfill \end{array}\right.$$

(12)

• Step 6: Exploit Outranking Relations and Determine Final Rank

Finally, the strong and weak outranking relations are exploited by applying a forward and backward distillation process [36], each of which results in a respective ranking of alternatives. These rankings are then combined in an appropriate manner to obtain the final ranking.

3.2. q-Rung Orthopair Fuzzy Set

Definition 1.

Let the set X be a non-empty finite set. A q-Rung Orthopair Fuzzy Set (q-ROFS) can be defined by Equation (13).

$$Q=\{x,u_Q\left(x\right),v_Q\left(x\right)\}$$

(13)

where $u_Q$ and $v_Q$ are mapping functions such that $u_Q:X\to [0,1]$ and $v_Q:X\to [0,1]$. These represent the membership and non-membership degrees of an element x with respect to X, respectively. The degrees must adhere to the constraint $0<u_Q{\left(x\right)}^q+v_Q{\left(x\right)}^q\le 1$ for $q\ge 1$. The indeterminacy degree of a q-ROFS can be expressed by Equation (14).

$${\pi }_Q\left(x\right)={(1-u_Q{\left(x\right)}^q-v_Q{\left(x\right)}^q)}^{{\displaystyle \frac{1}{q}}}$$

(14)

where ${\pi }_Q\left(x\right)\in [0,1]$ is an expression of the uncertainty that x belongs to Q. A q-Rung Orthopair Fuzzy Number (q-ROFN) can be used to express elements of a q-ROFS in the form $(u_Q,v_Q)$ where ${u_Q}^q+{v_Q}^q\le 1$ and $u_Q,v_Q\in [0,1]$.

3.3. Linguistic q-Rung Orthopair Fuzzy Set

Definition 2.

Let $X=\{x_1,x_2,\dots ,x_n\}$ be the universe of discourse. A Linguistic q-Rung Orthopair Fuzzy Set (Lq-ROFS) can be defined by Equation (15).

$$\widehat{\gamma }=\left\{(x,s_a\left(x\right),s_b\left(x\right))\right|x\in X\}$$

(15)

where $s_a\left(x\right)$ and $s_b\left(x\right)$ express the degree of linguistic membership and non-membership, respectively. For $x\in X$, $a^q+b^q\le l^q$ must hold for $q\ge 1$ and where l is the maximum linguistic term. The Linguistic q-Rung Orthopair Fuzzy Number (Lq-ROFN) is given as the pair $(s_a,s_b)$. The linguistic indeterminacy degree, which represents the expression of hesitation in each evaluation, is then expressed as $\pi \left(x\right)=s_{{(l^q-a^q-b^q)}^{{\displaystyle \frac{1}{q}}}}$.

3.3.1. Score of Lq-ROFN

In order to evaluate Lq-ROFNs the score can be determined using Equation (16). Let $\widehat{y}=(s_a,s_b)\in S_{[0,k]}$ be an Lq-ROFN. The score of $\widehat{\gamma }$ denoted by $Ls\left(\widehat{\gamma }\right)$ is given in Equation (16).

$$Ls\left(\widehat{\gamma }\right)={\left({\displaystyle \frac{k^q+a^q-b^q}{2}}\right)}^{{\displaystyle \frac{1}{q}}}$$

(16)

3.3.2. Accuracy of Lq-ROFN

Furthermore, to compare Lq-ROFNs, their accuracy values can be calculated using Equation (17). Let $\widehat{y}=(s_a,s_b)\in S_{[0,k]}$ be a Lq-ROFN. The accuracy of $\widehat{\gamma }$ denoted by $Lh\left(\widehat{\gamma }\right)$ is given by Equation (17).

$$Lh\left(\widehat{\gamma }\right)={\left(a^q+b^q\right)}^{{\displaystyle \frac{1}{q}}}$$

(17)

3.3.3. Comparing Lq-ROFNs

In order to compare different Lq-ROFNs with each other, their accuracy and score values can be used. Let $\widehat{{\gamma }_1}=(s_{a_1},s_{b_1})$ and $\widehat{{\gamma }_2}=(s_{a_2},s_{b_2})$ be two Lq-ROFNs where $\widehat{{\gamma }_1},\widehat{{\gamma }_2}\in S_{[0,k]}$. The following relations can be defined using their score and accuracy values, $Ls\left(\widehat{\gamma }\right)$ and $Lh\left(\widehat{\gamma }\right)$.

• $\mathrm{If}{L}_s\left({\widehat{\gamma }}_1\right)>L_s\left({\widehat{\gamma }}_2\right)\mathrm{then}{\widehat{\gamma }}_1>{\widehat{\gamma }}_2$
• $\mathrm{If}{L}_s\left({\widehat{\gamma }}_1\right)<L_s\left({\widehat{\gamma }}_2\right)\mathrm{then}{\widehat{\gamma }}_1<{\widehat{\gamma }}_2$
• $\mathrm{If}{L}_s\left({\widehat{\gamma }}_1\right)=L_s\left({\widehat{\gamma }}_2\right)\mathrm{then}{\widehat{\gamma }}_1={\widehat{\gamma }}_2$

  (a)

  $\mathrm{If}{L}_h\left({\widehat{\gamma }}_1\right)>L_h\left({\widehat{\gamma }}_2\right)\mathrm{then}{\widehat{\gamma }}_1>{\widehat{\gamma }}_2$

  (b)

  $\mathrm{If}{L}_h\left({\widehat{\gamma }}_1\right)<L_h\left({\widehat{\gamma }}_2\right)\mathrm{then}{\widehat{\gamma }}_1<{\widehat{\gamma }}_2$

  (c)

  $\mathrm{If}{L}_h\left({\widehat{\gamma }}_1\right)=L_h\left({\widehat{\gamma }}_2\right)\mathrm{then}{\widehat{\gamma }}_1={\widehat{\gamma }}_2$

3.3.4. Distance Measure

Definition 3.

Let $\widehat{{\gamma }_1}=(s_{a_1},s_{b_1})$ and $\widehat{{\gamma }_2}=(s_{a_2},s_{b_2})$ be two Lq-ROFNs, where $\widehat{{\gamma }_1},\widehat{{\gamma }_2}\in S_{[0,k]}$, and $({\pi }_1,{\pi }_2)$ are their corresponding degrees of indeterminacy, respectively. The Hamming distance between the Lq-ROFNs, $d_{H_d}(\widehat{{\gamma }_1},\widehat{{\gamma }_2})$, is given by Equation (18).

$$d_{H_d}(\widehat{{\gamma }_1},\widehat{{\gamma }_2})={\displaystyle \frac{|a_1^q-a_2^q|+|b_1^q-b_2^q|+|{\pi }_1^q-{\pi }_2^q|}{2k^q}}$$

(18)

3.3.5. Aggregation Operators

Let A be a set of n Lq-ROFNs such that $A=\{\widehat{{\gamma }_1},\widehat{{\gamma }_2},\dots ,\widehat{{\gamma }_n}\}$ and $\widehat{{\gamma }_1}=(s_{a_i},s_{b_i})$ for $i=1,2,\dots ,n$ and $(s_{a_i},s_{b_i})\in S_{[0,k]}$. The set A can be aggregated using the Linguistic q-Rung Fuzzy Weighted Averaging operator (Lq-ROFWA) as in Equation (19).

$$\begin{array}{ccc}\hfill \mathrm{L}q\text{-}\mathrm{ROFWA}(\widehat{{\gamma }_1},\widehat{{\gamma }_2},\dots ,\widehat{{\gamma }_n})& =\hfill & \left(s_{k{\left(1-{\prod }_{i=1}^n{\left(1-{\displaystyle \frac{a_i^q}{k^q}}\right)}^{{\omega }_i}\right)}^{{\displaystyle \frac{1}{q}}}},s_{k{\prod }_{i=1}^n{\left({\displaystyle \frac{b_i}{k}}\right)}^{{\omega }_i}}\right)\hfill \end{array}$$

(19)

4. Proposed L$\mathit{q}$-ROFS-ELECTRE II Algorithm

In this section, the novel Lq-ROFS-ELECTRE II algorithm is presented in the form of steps as follows.

Step 1: Let there be R decision-makers $(r=1,2,\dots ,R)$ providing their judgments over n criteria for m alternatives. Define a decision matrix for each decision-maker, as in Equation (20).

$$D_r={\left[{\widehat{\gamma }}_{ij}^r\right]}_{m\times n}=\left[\begin{array}{ccc}{\widehat{\gamma }}_{11}^r& \dots & {\widehat{\gamma }}_{1n}^r\\ ⋮& \ddots & ⋮\\ {\widehat{\gamma }}_{m1}^r& \dots & {\widehat{\gamma }}_{mn}^r\end{array}\right]$$

(20)

Step 2: Aggregate the R decision-makers matrices to form the final decision matrix D by applying Equation (21).

$$\begin{array}{ccc}\hfill D={\left[\mathrm{L}q\text{-}\mathrm{ROFWA}({\widehat{\gamma }}_{ij}^1,{\widehat{\gamma }}_{ij}^2,\dots ,{\widehat{\gamma }}_{ij}^r)\right]}_{m\times n}& =\hfill & \left[\begin{array}{ccc}{\widehat{\gamma }}_{11}& \dots & {\widehat{\gamma }}_{1n}\\ ⋮& \ddots & ⋮\\ {\widehat{\gamma }}_{m1}& \dots & {\widehat{\gamma }}_{mn}\end{array}\right]\hfill \end{array}$$

(21)

Step 3: Define a set of criteria importance weights, ${\overline{\omega }}^{*}$, for each criterion as in Equation (22).

$${\overline{\omega }}^{*}=\{\overline{{\omega }_1},\overline{{\omega }_2},\dots ,\overline{{\omega }_j}\}$$

(22)

Step 4: Determine strong ($C_{xy}^s$), medium ($C_{xy}^m$), and weak ($C_{xy}^w$) concordance sets, as per Equations (23), (24) and (25), respectively.

$$C_{xy}^s=\{j\mid Ls\left({\widehat{\gamma }}_{xj}\right)>Ls\left({\widehat{\gamma }}_{yj}\right),Lh\left({\widehat{\gamma }}_{xj}\right)>Lh\left({\widehat{\gamma }}_{yj}\right)\}$$

(23)

$$C_{xy}^m=\{j\mid Ls\left({\widehat{\gamma }}_{xj}\right)>Ls\left({\widehat{\gamma }}_{yj}\right),Lh\left({\widehat{\gamma }}_{xj}\right)\le Lh\left({\widehat{\gamma }}_{yj}\right)\}$$

(24)

$$C_{xy}^w=\{j\mid Ls\left({\widehat{\gamma }}_{xj}\right)=Ls\left({\widehat{\gamma }}_{yj}\right),Lh\left({\widehat{\gamma }}_{xj}\right)>Lh\left({\widehat{\gamma }}_{yj}\right)\}$$

(25)

Step 5: Determine strong ($Q_{xy}^s$), medium ($Q_{xy}^m$), and weak ($Q_{xy}^w$) discordance sets, as per Equations (26)–(28).

$$Q_{xy}^s=\{j\mid Ls\left({\widehat{\gamma }}_{xj}\right)<Ls\left({\widehat{\gamma }}_{yj}\right),Lh\left({\widehat{\gamma }}_{xj}\right)<Lh\left({\widehat{\gamma }}_{yj}\right)\}$$

(26)

$$Q_{xy}^m=\{j\mid Ls\left({\widehat{\gamma }}_{xj}\right)<Ls\left({\widehat{\gamma }}_{yj}\right),Lh\left({\widehat{\gamma }}_{xj}\right)\ge Lh\left({\widehat{\gamma }}_{yj}\right)\}$$

(27)

$$Q_{xy}^w=\{j\mid Ls\left({\widehat{\gamma }}_{xj}\right)=Ls\left({\widehat{\gamma }}_{yj}\right),Lh\left({\widehat{\gamma }}_{xj}\right)<Lh\left({\widehat{\gamma }}_{yj}\right)\}$$

(28)

Step 6: Determine the indifference set ($I_{xy}$) by applying Equation (29).

$$I_{xy}=\{j\mid Ls\left({\widehat{\gamma }}_{xj}\right)=Ls\left({\widehat{\gamma }}_{yj}\right),Lh\left({\widehat{\gamma }}_{xj}\right)=Lh\left({\widehat{\gamma }}_{yj}\right)\}$$

(29)

Step 7: Define the weights for the strong, medium, and weak concordance and discordance relations, and a weight for the indifference relation, as ${\omega }_C^s,{\omega }_C^m,{\omega }_C^w,{\omega }_Q^s,{\omega }_Q^m,{\omega }_Q^w,{\omega }_I$, where ${\omega }_C^x,{\omega }_Q^x,{\omega }_I\le 1,x\in \{s,m,w\}$.

Step 8: Determine the concordance matrix by applying Equation (30).

$$C={\left[c_{xy}\right]}_{m\times m}=\left[\begin{array}{ccc}{c}_{11}& \dots & c_{1m}\\ ⋮& \ddots & ⋮\\ c_{m1}& \dots & c_{mn}\end{array}\right]$$

(30)

where $c_{xy}$ is given by Equation (31).

$$c_{xy}={\displaystyle \frac{{\omega }_C^s{\displaystyle \sum _{j\in C_{xy}^s}}{\overline{\omega }}_j+{\omega }_C^m{\displaystyle \sum _{j\in C_{xy}^m}}{\overline{\omega }}_j+{\omega }_C^w{\displaystyle \sum _{j\in C_{xy}^w}}{\overline{\omega }}_j+{\omega }_I{\displaystyle \sum _{j\in I_{xy}}}{\overline{\omega }}_j}{{\displaystyle \sum _{j=1}^n}{\overline{\omega }}_j}}$$

(31)

Step 9: Determine the discordance matrix by applying Equation (32).

$$Q={\left[q_{xy}\right]}_{m\times m}=\left[\begin{array}{ccc}{q}_{11}& \dots & q_{1m}\\ ⋮& \ddots & ⋮\\ q_{m1}& \dots & q_{mn}\end{array}\right]$$

(32)

where $q_{xy}$ is given by Equation (33).

$$q_{xy}={\displaystyle \frac{max\left\{{\omega }_Q^s\underset{j\in Q_{xy}^s}{max}\left({\overline{\omega }}_j{d}_{H_d}({\widehat{\gamma }}_{xj},{\widehat{\gamma }}_{yj})\right),{\omega }_Q^m\underset{j\in Q_{xy}^m}{max}\left({\overline{\omega }}_j{d}_{H_d}({\widehat{\gamma }}_{xj},{\widehat{\gamma }}_{yj})\right),{\omega }_Q^w\underset{j\in Q_{xy}^w}{max}\left({\overline{\omega }}_j{d}_{H_d}({\widehat{\gamma }}_{xj},{\widehat{\gamma }}_{yj})\right)\right\}}{\underset{j}{max}{d}_{H_d}({\widehat{\gamma }}_{xj},{\widehat{\gamma }}_{yj})}}$$

(33)

Step 10: Determine the strong and weak outranking relationships as in Equations (34), (35) and (36), respectively.

$$x{S}^{F}y⟺\left\{\begin{array}{c}{C}_{xy}\ge {\alpha }^{+}\hfill \\ Q_{xy}\le {\beta }^{+}\hfill \\ C_{xy}\ge C_{yx}\hfill \end{array}\right.$$

(34)

$$x{S}^{F}y⟺\left\{\begin{array}{c}{C}_{xy}\ge {\alpha }^0\hfill \\ Q_{xy}\le {\beta }^0\hfill \\ C_{xy}\ge C_{yx}\hfill \end{array}\right.$$

(35)

$$x{S}^{f}y⟺\left\{\begin{array}{c}{C}_{xy}\ge {\alpha }^{-}\hfill \\ Q_{xy}\le {\beta }^{+}\hfill \\ C_{xy}\ge C_{yx}\hfill \end{array}\right.$$

(36)

where $\alpha$ and $\beta$ are the discordance thresholds defined by the decision-maker, such that $0<{\alpha }^{-}<{\alpha }^0<{\alpha }^{+}<1$ and $0<{\beta }^{+}<{\beta }^0<1$.

Step 11: Exploit Outranking Relations.

In order to generate the strong outranking graph, if alternative x strongly outranks alternative y, $\left(x{S}^{F}y\right)$, then a directed arc is drawn from x to y. Similarly, if alternative x weakly outranks alternative y, ($x{S}^{f}y)$, then a directed arc is drawn in the weak outranking graph. Thereafter, the forward and backward orders are performed, and finally, the results of the two orders are combined to produce a final ranking. The forward order is performed as follows:

• According to the strong outranking graph $G_1^F$, the set $N_1^F$ can be generated as the set of non-dominant alternatives. This set contains all alternatives that are not outranked by any other alternatives, and thus all alternatives that have no incoming arcs. The set $N_1^f$ is also generated in the same manner by considering the weak outranking graph $G_1^f$.
• The elements that lie in the intersection of sets $N_1^F$ and $N_1^f$, that is, $N_1^F\cap N_1^f$, results in the set $N_1$. The alternatives in $N_1$ are those not outranked in both the strong and weak outranking graphs. All alternatives in $N_1$ are assigned a forward rank of 1, ${\psi }_1\left(x\right)=1$, $\forall x\in N_1$, where ${\psi }_1\left(x\right)$ represents the forward ranking for alternative x.
• All nodes and edges corresponding to alternatives in $N_1$ are removed from both the strong and weak outranking graphs. After removing these nodes and edges, the resulting graphs are $G_2^F$ and $G_2^f$.
• Steps 1 to 3 are repeated iteratively until all alternatives are assigned a ranked, with each iteration producing a new set of strong and weak graphs $G_v^F$ and $G_v^f$. Eventually, all alternatives are assigned a forward rank.

Thereafter, the reverse ordering is performed as follows:

• All of the directed edges in the strong and weak outranking graphs, $G_1^F$ and $G_1^f$, are reversed to form the mirror image graphs.
• Each alternative is assigned a rank ${\psi }_2\left(x\right)$ using the same procedure as in the forward ranking (steps 1 to 3), where ${\psi }_2\left(x\right)$ represents the reverse ranking for alternative x.
• Due to the graph reversals, each rank is transformed using Equation (37), where the transformed rank is denoted by ${\psi }_3\left(x\right)$.

  $${\psi }_3\left(A_i\right)=1+\underset{A_v\in A}{max}{\psi }_2\left(A_v\right)-{\psi }_2\left(A_i\right)$$

  (37)

The final ranking for alternative x, denoted as $\overline{\psi }\left(x\right)$, is obtained by combining the forward and reverse order rankings. This is conducted by taking the midpoint of both rankings, as in Equation (38).

$$\overline{\psi }\left(A_i\right)={\displaystyle \frac{{\psi }_1\left(A_i\right)+{\psi }_3\left(A_i\right)}{2}}$$

(38)

The pseudocode for the Lq-ROFS-ELECTRE II algorithm is provided in Algorithm 1.

Algorithm 1 Lq-ROFS-ELECTRE II

Require: R decision-makers, m alternatives, n criteria

Ensure: Final ranking of alternatives

1: for $r=1$ to R do   2:       Construct decision matrix $D_r={\left[{\widehat{\gamma }}_{ij}^r\right]}_{m\times n}$   3: end for   4: Aggregate all $D_r$ into final decision matrix $D=\left[{\widehat{\gamma }}_{ij}\right]$ using Lq-ROFWA operator   5: Define criteria importance weights ${\overline{\omega }}^{*}=\{{\overline{\omega }}_1,\dots ,{\overline{\omega }}_n\}$   6: for all $x,y\in \{1,\dots ,m\}$, $x\ne y$ do   7:       Determine weak, medium, and strong concordance sets $C_{xy}^w$, $C_{xy}^m$, $C_{xy}^s$   8:       Determine indifference set $I_{xy}$   9:       Determine weak, medium, and strong discordance sets $Q_{xy}^w$, $Q_{xy}^m$, $Q_{xy}^s$ 10:       Compute concordance value $c_{xy}$ 11:       Compute discordance value $q_{xy}$ 12: end for 13: Define thresholds: ${\alpha }^{-},{\alpha }^0,{\alpha }^{+},{\beta }^{+},{\beta }^0$ 14: for all $x,y\in \{1,\dots ,m\}$, $x\ne y$ do 15:       if $c_{xy}\ge {\alpha }^{+}$ and $q_{xy}\le {\beta }^{+}$ and $c_{xy}\ge c_{yx}$ then 16:             $x{S}^{F}y$ (Strong outranking) 17:       else if $c_{xy}\ge {\alpha }^0$ and $q_{xy}\le {\beta }^0$ and $c_{xy}\ge c_{yx}$ then 18:             $x{S}^{F}y$ (Strong outranking) 19:       else if $c_{xy}\ge {\alpha }^{-}$ and $q_{xy}\le {\beta }^{+}$ and $c_{xy}\ge c_{yx}$ then 20:             $x{S}^{f}y$ (Weak outranking) 21:       end if 22: end for 23: Build strong outranking graph $G^F$ and weak outranking graph $G^f$ 24: Perform forward and backward ranking from $G^F$ and $G^f$ 25: Combine rankings to obtain final ranking of alternatives

5. Application of L$\mathit{q}$-ROFS-ELECTRE II for Ontology Selection

5.1. Experimental Design

5.1.1. Dataset

The dataset used in this study was obtained by performing a survey on the existing machine learning ontologies available. A thorough survey was conducted on both research articles and ontology repositories. Ten research databases were surveyed, viz. IEEE Xplore, ACM, Taylor & Francis, ScienceDirect, Springer, JSTOR, Emerald, MDPI, Wiley, and ArXiv. Furthermore, 6 popular ontology repositories were also surveyed, viz. BioPortal, AgroPortal, Ontology Lookup Service, Ontobee, OBO Foundry, and LOV. The first search of the 10 research databases using the search terms “Machine Learning” and “Ontology” resulted in 19 applicable papers in which an ML-related ontology was developed. The second search within the ontology repositories resulted in 8 ontologies. Thereafter, after a secondary analysis of the ontologies, 8 of the ontologies were eliminated and the remaining 19 were studied further. These reasons include duplicated ontologies, unavailability of source files, as well as dated and unmaintained source files. The final 19 ML-related ontologies along with a brief summary of each of the ontologies within the dataset is provided in

Table 1. Machine learning ontologies in the dataset.

i	Ontology	Class Count	Axioms	Thematic Area	Description
1	MLOnto [37]	433	1877	Machine Learning	Core ML concepts and workflows
2	Expose [38]	135	364	Data Mining	Tracks exposure events and context
3	HPC [39]	373	1931	High-Perf. Comp.	HPC resource and task modeling
4	EDAM [40]	3511	36,519	Bioinformatics	Bioinformatics operations and formats
5	VAIR [41]	424	5084	AI Risks	Models AI risk properties
6	FMO [42]	244	1860	Fairness in ML	Fairness in ML concepts and topologies
7	RAInS [43]	155	1471	AI Accountability	Responsible AI and computing
8	ITO [44]	16,283	1,048,656	General ML	Comprehensive IT process taxonomy
9	MLSO [45]	36	403	General ML	Light ML software pipeline ontology
10	Mex-Algo [46]	157	492	ML Algorithms	Metadata for ML algorithm components
11	Mex-Core [46]	65	218	Experiment Metadata	Core experiment metadata
12	Mex-Perf [46]	13	188	Performance Metrics	Performance indicators for experiments
13	FAIRnets [47]	77	516	Neural Networks	FAIR principles applied to Neural Nets
14	AIO [48]	443	3529	Deep Learning	Generic AI-related concepts
15	DSEO [49]	132	1072	Data Science	Data science education
16	OntoDM [50]	473	2350	Data Mining	Formalization of data mining tasks
17	MLLO [51]	345	2884	ML Lifecycle	Full ML model lifecycle ontology
18	SWeMLS [52]	65	3995	Semantic Web	Linking ML to semantic web resources
19	MLSchema  [53]	25	308	General ML	Schema for describing ML elements

.

5.1.2. Ontology Evaluation Criteria

The criteria used to evaluate the ontologies are extracted from the Ontometric framework [23]. Ontometric provides 5 categories of characteristics that can assist in the choice of the right ontology, including tools, language, content, methodology, and costs. Based on these categories, we constructed seven criteria in this study to run the proposed Lq-ROFS-ELECTRE II algorithm. The tools and costs categories were omitted because the ontologies in the dataset (listed in Table 1) are not specific to a project or system implementation, but rather a comparative analysis among the entire dataset was desired. Accordingly, in some cases, it may be appropriate to include criteria from these two categories as well. The seven criteria are presented in the Venn diagram in Figure 1, and are elaborated on as follows.

• Expressiveness of the knowledge domain by the ontology. How well does the ontology express ML concepts, relations, attributes, and their dependencies? This criterion aligns with the Language category of characteristics in Ontometric [23].
• Axiom design and logical expressiveness for reasoning support. Does the ontology include well-defined axioms that adequately formalize the logical relationships between concepts and instances, enabling potential reasoning tasks such as inference, consistency checking, and query answering? This criterion represents the Language category of metrics in Ontometric [23].
• Ontology evolution and maintenance practices. Does the ontology show evidence of systematic maintenance and evolution? Are there clear versioning practices, change documentation, and evidence of iterative improvement? This criterion corresponds to the Methodology category of characteristics in Ontometric [23].
• Documentation quality and usability support. How clear and comprehensive is the ontology documentation? Does it include complete usage examples, methodology explanations, and sufficient detail to enable effective use by domain experts and developers? This criterion covers the Methodology category of characteristics in Ontometric [23].
• Natural language alignment and concept clarity. How closely does the ontology’s formal representation align with natural language expressions of ML concepts? Are the concepts, labels, and definitions intuitive and clear to domain experts? This criterion aligns with the Language and Content categories of characteristics in Ontometric [23].
• Modularity and proper taxonomy use. How well structured is the taxonomy of the ontology? Are its taxonomy, modules and partitions in line with the ML domain requirements? This criterion addresses the Content category of characteristics in Ontometric [23].
• Alignment between formal ontology definitions and natural language descriptions. Does the ontology provide clear and accurate natural language documentation (e.g., rdfs:comment) for its formally defined classes, properties, and axioms? This criterion evaluates how well the ontology bridges the gap between formal specification and human-readable domain understanding. It aligns with three Ontometric categories [23] of Content, Language, and Methodology.

5.1.3. Computer and Software Environment

The experiments in this study were carried out using a 64-bit Microsoft^® Windows^® 10 device with 8 GB of RAM and a 512 GB SSD. The device had an Intel^® Core™ i-3 processor with a speed of 2.30 GHz. The Lq-ROFS-ELECTRE II algorithm was implemented using the Python 3 programming language together with the PyCharm 2025.2.4 development environment.

5.2. Experimental Results and Discussion

5.2.1. Decision Matrices

The first step is to obtain the expert opinions in the form of a decision matrix. In light of the 7 criteria defined above, 4 experts in the field of artificial intelligence and ontology engineering performed judgments on the 19 ontologies. These judgments, displayed in

Table 2. Evaluation given by expert 1.

Ontology	${\mathit{C}}_1$	${\mathit{C}}_2$	${\mathit{C}}_3$	${\mathit{C}}_4$	${\mathit{C}}_5$	${\mathit{C}}_6$	${\mathit{C}}_7$
MLOnto	($s_3$, $s_3$)	($s_3$, $s_3$)	($s_3$, $s_3$)	($s_5$, $s_1$)	($s_3$, $s_3$)	($s_5$, $s_1$)	($s_5$, $s_1$)
Expose	($s_3$, $s_3$)	($s_3$, $s_3$)	($s_3$, $s_3$)	($s_3$, $s_3$)	($s_3$, $s_3$)	($s_3$, $s_3$)	($s_3$, $s_3$)
HPC	($s_3$, $s_3$)	($s_3$, $s_3$)	($s_3$, $s_3$)	($s_3$, $s_3$)	($s_3$, $s_3$)	($s_3$, $s_3$)	($s_3$, $s_3$)
EDAM	($s_5$, $s_1$)	($s_5$, $s_1$)	($s_5$, $s_1$)	($s_5$, $s_1$)	($s_5$, $s_1$)	($s_5$, $s_1$)	($s_5$, $s_1$)
VAIR	($s_3$, $s_3$)	($s_3$, $s_3$)	($s_3$, $s_3$)	($s_3$, $s_3$)	($s_3$, $s_3$)	($s_3$, $s_3$)	($s_3$, $s_3$)
FMO	($s_3$, $s_3$)	($s_3$, $s_3$)	($s_3$, $s_3$)	($s_3$, $s_3$)	($s_3$, $s_3$)	($s_3$, $s_3$)	($s_3$, $s_3$)
RAInS	($s_3$, $s_3$)	($s_3$, $s_3$)	($s_3$, $s_3$)	($s_3$, $s_3$)	($s_3$, $s_3$)	($s_3$, $s_3$)	($s_3$, $s_3$)
ITO	($s_5$, $s_1$)	($s_5$, $s_1$)	($s_5$, $s_1$)	($s_5$, $s_1$)	($s_5$, $s_1$)	($s_5$, $s_1$)	($s_5$, $s_1$)
MLSO	($s_3$, $s_3$)	($s_3$, $s_3$)	($s_3$, $s_3$)	($s_3$, $s_3$)	($s_3$, $s_3$)	($s_3$, $s_3$)	($s_3$, $s_3$)
Mex-Algo	($s_3$, $s_3$)	($s_3$, $s_3$)	($s_3$, $s_3$)	($s_3$, $s_3$)	($s_3$, $s_3$)	($s_3$, $s_3$)	($s_3$, $s_3$)
Mex-Core	($s_3$, $s_3$)	($s_3$, $s_3$)	($s_3$, $s_3$)	($s_3$, $s_3$)	($s_3$, $s_3$)	($s_3$, $s_3$)	($s_3$, $s_3$)
Mex-Perf	($s_3$, $s_3$)	($s_3$, $s_3$)	($s_3$, $s_3$)	($s_3$, $s_3$)	($s_3$, $s_3$)	($s_3$, $s_3$)	($s_3$, $s_3$)
FAIRnets	($s_3$, $s_3$)	($s_3$, $s_3$)	($s_3$, $s_3$)	($s_3$, $s_3$)	($s_3$, $s_3$)	($s_3$, $s_3$)	($s_3$, $s_3$)
AIO	($s_4$, $s_2$)	($s_4$, $s_2$)	($s_4$, $s_2$)	($s_4$, $s_2$)	($s_4$, $s_2$)	($s_4$, $s_2$)	($s_3$, $s_3$)
DSEO	($s_3$, $s_3$)	($s_3$, $s_3$)	($s_3$, $s_3$)	($s_3$, $s_3$)	($s_3$, $s_3$)	($s_3$, $s_3$)	($s_3$, $s_3$)
OntoDM	($s_4$, $s_2$)	($s_4$, $s_2$)	($s_4$, $s_2$)	($s_4$, $s_2$)	($s_4$, $s_2$)	($s_4$, $s_2$)	($s_3$, $s_3$)
MLLO	($s_4$, $s_2$)	($s_4$, $s_2$)	($s_4$, $s_2$)	($s_4$, $s_2$)	($s_4$, $s_2$)	($s_4$, $s_2$)	($s_3$, $s_3$)
SWeMLS	($s_4$, $s_2$)	($s_4$, $s_2$)	($s_4$, $s_2$)	($s_4$, $s_2$)	($s_4$, $s_2$)	($s_4$, $s_2$)	($s_3$, $s_3$)
MLSchema	($s_3$, $s_3$)	($s_3$, $s_3$)	($s_3$, $s_3$)	($s_3$, $s_3$)	($s_3$, $s_3$)	($s_3$, $s_3$)	($s_3$, $s_3$)

,

Table 3. Evaluation given by expert 2.

Ontology	${\mathit{C}}_1$	${\mathit{C}}_2$	${\mathit{C}}_3$	${\mathit{C}}_4$	${\mathit{C}}_5$	${\mathit{C}}_6$	${\mathit{C}}_7$
MLOnto	($s_4$, $s_2$)	($s_4$, $s_2$)	($s_4$, $s_2$)	($s_5$, $s_1$)	($s_4$, $s_2$)	($s_4$, $s_2$)	($s_3$, $s_3$)
Expose	($s_2$, $s_4$)	($s_2$, $s_4$)	($s_2$, $s_4$)	($s_2$, $s_4$)	($s_2$, $s_4$)	($s_2$, $s_4$)	($s_2$, $s_4$)
HPC	($s_3$, $s_3$)	($s_3$, $s_3$)	($s_3$, $s_3$)	($s_3$, $s_3$)	($s_3$, $s_3$)	($s_3$, $s_3$)	($s_3$, $s_3$)
EDAM	($s_5$, $s_1$)	($s_5$, $s_1$)	($s_5$, $s_1$)	($s_5$, $s_1$)	($s_5$, $s_1$)	($s_5$, $s_1$)	($s_4$, $s_2$)
VAIR	($s_3$, $s_3$)	($s_3$, $s_3$)	($s_3$, $s_3$)	($s_3$, $s_3$)	($s_3$, $s_3$)	($s_3$, $s_3$)	($s_3$, $s_3$)
FMO	($s_3$, $s_3$)	($s_3$, $s_3$)	($s_3$, $s_3$)	($s_3$, $s_3$)	($s_3$, $s_3$)	($s_3$, $s_3$)	($s_3$, $s_3$)
RAInS	($s_3$, $s_3$)	($s_3$, $s_3$)	($s_3$, $s_3$)	($s_3$, $s_3$)	($s_3$, $s_3$)	($s_3$, $s_3$)	($s_3$, $s_3$)
ITO	($s_5$, $s_1$)	($s_5$, $s_1$)	($s_5$, $s_1$)	($s_5$, $s_1$)	($s_5$, $s_1$)	($s_5$, $s_1$)	($s_4$, $s_2$)
MLSO	($s_3$, $s_3$)	($s_3$, $s_3$)	($s_3$, $s_3$)	($s_3$, $s_3$)	($s_3$, $s_3$)	($s_3$, $s_3$)	($s_3$, $s_3$)
Mex-Algo	($s_3$, $s_3$)	($s_3$, $s_3$)	($s_3$, $s_3$)	($s_3$, $s_3$)	($s_3$, $s_3$)	($s_3$, $s_3$)	($s_3$, $s_3$)
Mex-Core	($s_3$, $s_3$)	($s_3$, $s_3$)	($s_3$, $s_3$)	($s_3$, $s_3$)	($s_3$, $s_3$)	($s_3$, $s_3$)	($s_3$, $s_3$)
Mex-Perf	($s_3$, $s_3$)	($s_3$, $s_3$)	($s_3$, $s_3$)	($s_3$, $s_3$)	($s_3$, $s_3$)	($s_3$, $s_3$)	($s_3$, $s_3$)
FAIRnets	($s_3$, $s_3$)	($s_3$, $s_3$)	($s_3$, $s_3$)	($s_3$, $s_3$)	($s_3$, $s_3$)	($s_3$, $s_3$)	($s_3$, $s_3$)
AIO	($s_4$, $s_2$)	($s_4$, $s_2$)	($s_4$, $s_2$)	($s_4$, $s_2$)	($s_4$, $s_2$)	($s_4$, $s_2$)	($s_3$, $s_3$)
DSEO	($s_3$, $s_3$)	($s_3$, $s_3$)	($s_3$, $s_3$)	($s_3$, $s_3$)	($s_3$, $s_3$)	($s_3$, $s_3$)	($s_3$, $s_3$)
OntoDM	($s_4$, $s_2$)	($s_4$, $s_2$)	($s_4$, $s_2$)	($s_4$, $s_2$)	($s_4$, $s_2$)	($s_4$, $s_2$)	($s_3$, $s_3$)
MLLO	($s_4$, $s_2$)	($s_4$, $s_2$)	($s_4$, $s_2$)	($s_4$, $s_2$)	($s_4$, $s_2$)	($s_4$, $s_2$)	($s_3$, $s_3$)
SWeMLS	($s_4$, $s_2$)	($s_4$, $s_2$)	($s_4$, $s_2$)	($s_4$, $s_2$)	($s_4$, $s_2$)	($s_4$, $s_2$)	($s_3$, $s_3$)
MLSchema	($s_3$, $s_3$)	($s_3$, $s_3$)	($s_3$, $s_3$)	($s_3$, $s_3$)	($s_3$, $s_3$)	($s_3$, $s_3$)	($s_3$, $s_3$)

,

Table 4. Evaluation given by expert 3.

Ontology	${\mathit{C}}_1$	${\mathit{C}}_2$	${\mathit{C}}_3$	${\mathit{C}}_4$	${\mathit{C}}_5$	${\mathit{C}}_6$	${\mathit{C}}_7$
MLOnto	($s_4$, $s_1$)	($s_3$, $s_2$)	($s_4$, $s_1$)	($s_4$, $s_1$)	($s_3$, $s_2$)	($s_3$, $s_2$)	($s_2$, $s_3$)
Expose	($s_2$, $s_3$)	($s_2$, $s_3$)	($s_2$, $s_3$)	($s_3$, $s_2$)	($s_2$, $s_3$)	($s_2$, $s_3$)	($s_1$, $s_4$)
HPC	($s_4$, $s_1$)	($s_3$, $s_2$)	($s_3$, $s_2$)	($s_3$, $s_2$)	($s_3$, $s_2$)	($s_3$, $s_2$)	($s_2$, $s_3$)
EDAM	($s_5$, $s_0$)	($s_5$, $s_0$)	($s_5$, $s_0$)	($s_5$, $s_1$)	($s_4$, $s_1$)	($s_5$, $s_0$)	($s_5$, $s_0$)
VAIR	($s_3$, $s_2$)	($s_3$, $s_2$)	($s_3$, $s_2$)	($s_4$, $s_1$)	($s_4$, $s_1$)	($s_3$, $s_2$)	($s_3$, $s_2$)
FMO	($s_4$, $s_1$)	($s_4$, $s_1$)	($s_3$, $s_2$)	($s_4$, $s_1$)	($s_4$, $s_1$)	($s_4$, $s_1$)	($s_2$, $s_3$)
RAInS	($s_2$, $s_3$)	($s_2$, $s_3$)	($s_2$, $s_3$)	($s_3$, $s_2$)	($s_2$, $s_3$)	($s_2$, $s_3$)	($s_1$, $s_4$)
ITO	($s_5$, $s_0$)	($s_5$, $s_0$)	($s_5$, $s_0$)	($s_5$, $s_0$)	($s_5$, $s_0$)	($s_5$, $s_0$)	($s_5$, $s_0$)
MLSO	($s_3$, $s_2$)	($s_3$, $s_2$)	($s_3$, $s_2$)	($s_3$, $s_2$)	($s_3$, $s_2$)	($s_2$, $s_3$)	($s_2$, $s_3$)
Mex-Algo	($s_3$, $s_2$)	($s_3$, $s_2$)	($s_3$, $s_2$)	($s_3$, $s_2$)	($s_3$, $s_2$)	($s_2$, $s_3$)	($s_2$, $s_3$)
Mex-Core	($s_3$, $s_2$)	($s_3$, $s_2$)	($s_3$, $s_2$)	($s_3$, $s_2$)	($s_3$, $s_2$)	($s_2$, $s_3$)	($s_1$, $s_4$)
Mex-Perf	($s_4$, $s_1$)	($s_3$, $s_2$)	($s_3$, $s_2$)	($s_3$, $s_2$)	($s_3$, $s_2$)	($s_3$, $s_2$)	($s_2$, $s_3$)
FAIRnets	($s_4$, $s_1$)	($s_3$, $s_2$)	($s_3$, $s_2$)	($s_3$, $s_2$)	($s_3$, $s_2$)	($s_3$, $s_2$)	($s_3$, $s_2$)
AIO	($s_3$, $s_2$)	($s_3$, $s_2$)	($s_3$, $s_2$)	($s_4$, $s_1$)	($s_3$, $s_2$)	($s_3$, $s_2$)	($s_2$, $s_3$)
DSEO	($s_2$, $s_3$)	($s_2$, $s_3$)	($s_2$, $s_3$)	($s_3$, $s_2$)	($s_3$, $s_2$)	($s_2$, $s_3$)	($s_1$, $s_4$)
OntoDM	($s_4$, $s_1$)	($s_4$, $s_1$)	($s_4$, $s_1$)	($s_4$, $s_1$)	($s_4$, $s_1$)	($s_4$, $s_1$)	($s_3$, $s_2$)
MLLO	($s_3$, $s_2$)	($s_3$, $s_2$)	($s_3$, $s_2$)	($s_3$, $s_2$)	($s_3$, $s_2$)	($s_3$, $s_2$)	($s_2$, $s_3$)
SWeMLS	($s_4$, $s_1$)	($s_4$, $s_1$)	($s_4$, $s_1$)	($s_4$, $s_1$)	($s_4$, $s_1$)	($s_4$, $s_1$)	($s_3$, $s_2$)
MLSchema	($s_2$, $s_3$)	($s_2$, $s_3$)	($s_2$, $s_3$)	($s_2$, $s_3$)	($s_2$, $s_3$)	($s_2$, $s_3$)	($s_1$, $s_4$)

,

Table 5. Evaluation given by expert 4.

Ontology	${\mathit{C}}_1$	${\mathit{C}}_2$	${\mathit{C}}_3$	${\mathit{C}}_4$	${\mathit{C}}_5$	${\mathit{C}}_6$	${\mathit{C}}_7$
MLOnto	($s_3$, $s_2$)	($s_3$, $s_2$)	($s_3$, $s_2$)	($s_4$, $s_1$)	($s_3$, $s_2$)	($s_3$, $s_2$)	($s_2$, $s_3$)
Expose	($s_2$, $s_3$)	($s_2$, $s_3$)	($s_2$, $s_3$)	($s_3$, $s_2$)	($s_2$, $s_3$)	($s_1$, $s_4$)	($s_1$, $s_4$)
HPC	($s_4$, $s_1$)	($s_3$, $s_2$)	($s_3$, $s_2$)	($s_3$, $s_2$)	($s_4$, $s_1$)	($s_3$, $s_2$)	($s_3$, $s_2$)
EDAM	($s_4$, $s_1$)	($s_5$, $s_0$)	($s_5$, $s_0$)	($s_5$, $s_1$)	($s_4$, $s_1$)	($s_5$, $s_0$)	($s_4$, $s_1$)
VAIR	($s_3$, $s_2$)	($s_2$, $s_3$)	($s_3$, $s_2$)	($s_4$, $s_1$)	($s_3$, $s_2$)	($s_3$, $s_2$)	($s_2$, $s_3$)
FMO	($s_3$, $s_2$)	($s_3$, $s_2$)	($s_3$, $s_2$)	($s_4$, $s_1$)	($s_4$, $s_1$)	($s_3$, $s_2$)	($s_2$, $s_3$)
RAInS	($s_2$, $s_3$)	($s_3$, $s_2$)	($s_2$, $s_3$)	($s_3$, $s_2$)	($s_2$, $s_3$)	($s_1$, $s_4$)	($s_1$, $s_4$)
ITO	($s_5$, $s_0$)	($s_5$, $s_0$)	($s_5$, $s_0$)	($s_5$, $s_0$)	($s_5$, $s_0$)	($s_5$, $s_0$)	($s_4$, $s_1$)
MLSO	($s_3$, $s_2$)	($s_2$, $s_3$)	($s_2$, $s_3$)	($s_2$, $s_3$)	($s_2$, $s_3$)	($s_1$, $s_4$)	($s_1$, $s_4$)
Mex-Algo	($s_3$, $s_2$)	($s_2$, $s_3$)	($s_2$, $s_3$)	($s_3$, $s_2$)	($s_3$, $s_2$)	($s_2$, $s_3$)	($s_2$, $s_3$)
Mex-Core	($s_3$, $s_2$)	($s_2$, $s_3$)	($s_2$, $s_3$)	($s_2$, $s_3$)	($s_2$, $s_3$)	($s_1$, $s_4$)	($s_1$, $s_4$)
Mex-Perf	($s_4$, $s_1$)	($s_1$, $s_4$)	($s_1$, $s_4$)	($s_1$, $s_4$)	($s_1$, $s_4$)	($s_1$, $s_4$)	($s_1$, $s_4$)
FAIRnets	($s_3$, $s_2$)	($s_2$, $s_3$)	($s_3$, $s_2$)	($s_2$, $s_3$)	($s_3$, $s_2$)	($s_2$, $s_3$)	($s_2$, $s_3$)
AIO	($s_3$, $s_2$)	($s_3$, $s_2$)	($s_3$, $s_2$)	($s_4$, $s_1$)	($s_3$, $s_2$)	($s_3$, $s_2$)	($s_3$, $s_2$)
DSEO	($s_2$, $s_3$)	($s_1$, $s_4$)	($s_1$, $s_4$)	($s_3$, $s_2$)	($s_2$, $s_3$)	($s_2$, $s_3$)	($s_1$, $s_4$)
OntoDM	($s_4$, $s_1$)	($s_4$, $s_1$)	($s_4$, $s_1$)	($s_4$, $s_1$)	($s_4$, $s_1$)	($s_4$, $s_1$)	($s_3$, $s_2$)
MLLO	($s_3$, $s_2$)	($s_3$, $s_2$)	($s_3$, $s_2$)	($s_4$, $s_1$)	($s_3$, $s_2$)	($s_3$, $s_2$)	($s_2$, $s_3$)
SWeMLS	($s_4$, $s_1$)	($s_4$, $s_1$)	($s_4$, $s_1$)	($s_3$, $s_2$)	($s_4$, $s_1$)	($s_3$, $s_2$)	($s_3$, $s_2$)
MLSchema	($s_2$, $s_3$)	($s_2$, $s_3$)	($s_1$, $s_4$)	($s_1$, $s_4$)	($s_2$, $s_3$)	($s_1$, $s_4$)	($s_1$, $s_4$)

, were expressed using a linguistic term set defined as $LTS=\{s_0=$ very poor, $s_1=$ poor, $s_2=$ average, $s_3=$ good, and $s_4=$ very good}. The 4 decision matrices were aggregated using the Equation (21) and resulted in the aggregated decision matrix in

Table 6. Aggregated evaluation matrix.

Ontology	${\mathit{C}}_1$	${\mathit{C}}_2$	${\mathit{C}}_3$	${\mathit{C}}_4$	${\mathit{C}}_5$	${\mathit{C}}_6$	${\mathit{C}}_7$
MLOnto	$(s_{2.61},s_0)$	$(s_{2.35},s_{1.19})$	$(s_{2.61},s_0)$	$(s_4,s_0)$	$(s_{2.35},s_{1.19})$	$(s_4,s_0)$	$(s_4,s_0)$
Expose	$(s_{1.35},s_{2.21})$	$(s_{1.35},s_{2.21})$	$(s_{1.35},s_{2.21})$	$(s_{1.82},s_{1.57})$	$(s_{1.35},s_{2.21})$	$(s_{1.26},s_{2.45})$	$(s_{1.16},s_{2.71})$
HPC	$(s_{2.61},s_0)$	$(s_2,s_{1.41})$	$(s_2,s_{1.41})$	$(s_2,s_{1.41})$	$(s_{2.35},s_0)$	$(s_2,s_{1.41})$	$(s_{1.82},s_{1.68})$
EDAM	$(s_4,s_0)$	$(s_4,s_0)$	$(s_4,s_0)$	$(s_4,s_0)$	$(s_4,s_0)$	$(s_4,s_0)$	$(s_4,s_0)$
VAIR	$(s_4,s_0)$	$(s_4,s_0)$	$(s_4,s_0)$	$(s_{2.61},s_0)$	$(s_{2.35},s_0)$	$(s_2,s_{1.41})$	$(s_{1.82},s_{1.68})$
FMO	$(s_{2.35},s_0)$	$(s_{2.35},s_0)$	$(s_2,s_{1.41})$	$(s_{2.61},s_0)$	$(s_{2.61},s_0)$	$(s_{2.35},s_0)$	$(s_{1.61},s_2)$
RAInS	$(s_{1.61},s_2)$	$(s_{1.82},s_{1.68})$	$(s_{1.61},s_2)$	$(s_2,s_{1.41})$	$(s_{1.61},s_2)$	$(s_{1.54},s_{2.21})$	$(s_{1.46},s_{2.45})$
ITO	$(s_4,s_0)$	$(s_4,s_0)$	$(s_4,s_0)$	$(s_4,s_0)$	$(s_4,s_0)$	$(s_4,s_0)$	$(s_4,s_0)$
MLSO	$(s_{2.35},s_{1.19})$	$(s_{2.22},s_{1.41})$	$(s_{2.22},s_{1.41})$	$(s_{2.22},s_{1.41})$	$(s_{2.22},s_{1.41})$	$(s_{2.02},s_{1.86})$	$(s_{2.02},s_{1.86})$
Mex-Algo	$(s_2,s_{1.41})$	$(s_{1.82},s_{1.68})$	$(s_{1.82},s_{1.68})$	$(s_2,s_{1.41})$	$(s_2,s_{1.41})$	$(s_{1.61},s_2)$	$(s_{1.61},s_2)$
Mex-Core	$(s_{1.82},s_{1.57})$	$(s_{1.61},s_{1.86})$	$(s_{1.61},s_{1.86})$	$(s_{1.61},s_{1.86})$	$(s_{1.61},s_{1.86})$	$(s_{1.26},s_{2.45})$	$(s_{1.16},s_{2.71})$
Mex-Perf	$(s_{2.51},s_0)$	$(s_{1.54},s_{2.06})$	$(s_{1.54},s_{2.06})$	$(s_{1.54},s_{2.06})$	$(s_{1.54},s_{2.06})$	$(s_{1.54},s_{2.06})$	$(s_{1.26},s_{2.45})$
FAIRnets	$(s_{2.35},s_0)$	$(s_{1.82},s_{1.68})$	$(s_2,s_{1.41})$	$(s_{1.82},s_{1.68})$	$(s_2,s_{1.41})$	$(s_{1.82},s_{1.68})$	$(s_{1.82},s_{1.68})$
AIO	$(s_4,s_0)$	$(s_4,s_0)$	$(s_4,s_0)$	$(s_{2.83},s_0)$	$(s_{2.35},s_{1.19})$	$(s_{2.35},s_{1.19})$	$(s_{1.82},s_{1.68})$
DSEO	$(s_{1.35},s_{2.21})$	$(s_{1.26},s_{2.45})$	$(s_{1.26},s_{2.45})$	$(s_{1.82},s_{1.57})$	$(s_{1.61},s_{1.86})$	$(s_{1.35},s_{2.21})$	$(s_{1.16},s_{2.71})$
OntoDM	$(s_3,s_0)$	$(s_3,s_0)$	$(s_3,s_0)$	$(s_3,s_0)$	$(s_3,s_0)$	$(s_3,s_0)$	$(s_{2.35},s_{1.19})$
MLLO	$(s_{2.61},s_1)$	$(s_{2.61},s_1)$	$(s_{2.61},s_1)$	$(s_{2.83},s_0)$	$(s_{2.61},s_1)$	$(s_{2.61},s_1)$	$(s_{2.07},s_{1.68})$
SWeMLS	$(s_4,s_0)$	$(s_4,s_0)$	$(s_4,s_0)$	$(s_{2.83},s_0)$	$(s_3,s_0)$	$(s_{2.83},s_0)$	$(s_{2.35},s_{1.19})$
MLSchema	$(s_{1.35},s_{2.21})$	$(s_{1.35},s_{2.21})$	$(s_{1.26},s_{2.45})$	$(s_{1.26},s_{2.45})$	$(s_{1.35},s_{2.21})$	$(s_{1.26},s_{2.45})$	$(s_{1.16},s_{2.71})$

. The q parameter was set to 2.

5.2.2. Importance Weightings

In this study, the mean weighting method was applied to obtain the criteria importance weights. This resulted in an equal weight of ${\displaystyle \frac{1}{7}}$ for each of the 7 criteria. The weights for the strong, medium, and weak concordance and discordance sets, and the indifference set were defined as $({\omega }_C^s,{\omega }_C^m,{\omega }_C^w,{\omega }_Q^s,{\omega }_Q^s,{\omega }_Q^s,{\omega }_I)$ = $(1,0.85,0.70,1,0.85,0.70,0.55)$.

5.2.3. Concordance and Discordance Matrices

In order to determine the concordance and discordance values, the score and accuracy values were determined using Equations (16) and (17). These are displayed in

Table 7. Scores for aggregated decision matrix.

Ontology	${\mathit{C}}_1$	${\mathit{C}}_2$	${\mathit{C}}_3$	${\mathit{C}}_4$	${\mathit{C}}_5$	${\mathit{C}}_6$	${\mathit{C}}_7$
MLOnto	3.38	3.17	3.38	4.00	3.17	4.00	4.00
Expose	2.54	2.54	2.54	2.90	2.54	2.41	2.24
HPC	3.38	3.00	3.00	3.00	3.28	3.00	2.87
EDAM	4.00	4.00	4.00	4.00	4.00	4.00	4.00
VAIR	4.00	4.00	4.00	3.38	3.28	3.00	2.87
FMO	3.28	3.28	3.00	3.38	3.38	3.28	2.70
RAInS	2.70	2.87	2.70	3.00	2.70	2.60	2.46
ITO	4.00	4.00	4.00	4.00	4.00	4.00	4.00
MLSO	3.17	3.08	3.08	3.08	3.08	2.88	2.88
Mex-Algo	3.00	2.87	2.87	3.00	3.00	2.70	2.70
Mex-Core	2.90	2.75	2.75	2.75	2.75	2.41	2.24
Mex-Perf	3.34	2.66	2.66	2.66	2.66	2.66	2.41
FAIRnets	3.28	2.87	3.00	2.87	3.00	2.87	2.87
AIO	4.00	4.00	4.00	3.46	3.17	3.17	2.87
DSEO	2.54	2.41	2.41	2.90	2.75	2.54	2.24
OntoDM	3.54	3.54	3.54	3.54	3.54	3.54	3.17
MLLO	3.30	3.30	3.30	3.46	3.30	3.30	2.95
SWeMLS	4.00	4.00	4.00	3.46	3.54	3.46	3.17
MLSchema	2.54	2.54	2.41	2.41	2.54	2.41	2.24

and

Table 8. Accuracy for aggregated decision matrix.

Ontology	${\mathit{C}}_1$	${\mathit{C}}_2$	${\mathit{C}}_3$	${\mathit{C}}_4$	${\mathit{C}}_5$	${\mathit{C}}_6$	${\mathit{C}}_7$
MLOnto	2.61	2.63	2.61	4.00	2.63	4.00	4.00
Expose	2.59	2.59	2.59	2.40	2.59	2.76	2.95
HPC	2.61	2.45	2.45	2.45	2.35	2.45	2.48
EDAM	4.00	4.00	4.00	4.00	4.00	4.00	4.00
VAIR	4.00	4.00	4.00	2.61	2.35	2.45	2.48
FMO	2.35	2.35	2.45	2.61	2.61	2.35	2.57
RAInS	2.57	2.48	2.57	2.45	2.57	2.69	2.85
ITO	4.00	4.00	4.00	4.00	4.00	4.00	4.00
MLSO	2.63	2.63	2.63	2.63	2.63	2.75	2.75
Mex-Algo	2.45	2.48	2.48	2.45	2.45	2.57	2.57
Mex-Core	2.40	2.46	2.46	2.46	2.46	2.76	2.95
Mex-Perf	2.51	2.57	2.57	2.57	2.57	2.57	2.76
FAIRnets	2.35	2.48	2.45	2.48	2.45	2.48	2.48
AIO	4.00	4.00	4.00	2.83	2.63	2.63	2.48
DSEO	2.59	2.76	2.76	2.40	2.46	2.59	2.95
OntoDM	3.00	3.00	3.00	3.00	3.00	3.00	2.63
MLLO	2.80	2.80	2.80	2.83	2.80	2.80	2.67
SWeMLS	4.00	4.00	4.00	2.83	3.00	2.83	2.63
MLSchema	2.59	2.59	2.76	2.76	2.59	2.76	2.95

. Using the score and accuracy values, the weak, medium, and strong concordance sets were determined. These were used to generate the global concordance matrix in

Table 9. Global concordance matrix.

	1	2	3	4	5	6	7	8	9	10	11	12	13	14	15	16	17	18	19
1	0.00	1.00	0.79	0.24	0.43	0.71	1.00	0.24	0.96	1.00	1.00	1.00	1.00	0.51	0.96	0.43	0.67	0.43	0.98
2	0.00	0.00	0.00	0.00	0.00	0.00	0.00	0.00	0.00	0.00	0.28	0.12	0.12	0.00	0.48	0.00	0.00	0.00	0.64
3	0.20	0.89	0.00	0.00	0.24	0.34	0.83	0.00	0.36	0.83	0.87	0.87	0.79	0.20	0.89	0.00	0.12	0.00	0.87
4	0.81	1.00	1.00	0.00	0.81	1.00	1.00	0.55	1.00	1.00	1.00	1.00	1.00	0.81	1.00	1.00	1.00	0.81	1.00
5	0.55	0.94	0.81	0.24	0.00	0.63	0.94	0.24	0.79	0.94	0.94	0.94	0.89	0.44	0.94	0.43	0.43	0.24	0.91
6	0.24	0.89	0.61	0.00	0.34	0.00	0.89	0.00	0.61	0.85	0.89	0.77	0.69	0.24	0.89	0.00	0.12	0.00	0.87
7	0.00	0.87	0.08	0.00	0.00	0.00	0.00	0.00	0.00	0.16	0.51	0.63	0.20	0.00	0.77	0.00	0.00	0.00	0.85
8	0.81	1.00	1.00	0.55	0.81	1.00	1.00	0.00	1.00	1.00	1.00	1.00	1.00	0.81	1.00	1.00	1.00	0.81	1.00
9	0.00	0.96	0.57	0.00	0.14	0.29	0.98	0.00	0.00	1.00	0.96	0.84	0.86	0.14	0.94	0.00	0.00	0.00	0.91
10	0.00	0.87	0.08	0.00	0.00	0.08	0.76	0.00	0.00	0.00	0.91	0.73	0.28	0.00	0.87	0.00	0.00	0.00	0.85
11	0.00	0.64	0.00	0.00	0.00	0.00	0.36	0.00	0.00	0.00	0.00	0.49	0.00	0.00	0.52	0.00	0.00	0.00	0.76
12	0.00	0.73	0.00	0.00	0.00	0.14	0.24	0.00	0.12	0.14	0.39	0.00	0.14	0.00	0.61	0.00	0.12	0.00	0.85
13	0.00	0.73	0.16	0.00	0.08	0.28	0.69	0.00	0.12	0.64	0.89	0.73	0.00	0.08	0.73	0.00	0.00	0.00	0.85
14	0.51	0.96	0.79	0.24	0.60	0.69	0.96	0.24	0.84	0.98	0.96	0.98	0.94	0.00	0.98	0.43	0.51	0.31	0.96
15	0.00	0.48	0.00	0.00	0.00	0.00	0.12	0.00	0.00	0.00	0.40	0.24	0.12	0.00	0.00	0.00	0.00	0.00	0.60
16	0.57	0.98	1.00	0.00	0.57	1.00	0.98	0.00	0.98	1.00	0.98	0.98	1.00	0.57	0.98	0.00	0.98	0.44	0.98
17	0.29	0.98	0.86	0.00	0.57	0.86	0.98	0.00	0.98	1.00	0.98	0.84	1.00	0.51	0.98	0.00	0.00	0.08	0.98
18	0.57	0.98	1.00	0.24	0.81	1.00	0.98	0.24	0.98	1.00	0.98	0.98	1.00	0.74	0.98	0.59	0.91	0.00	0.98
19	0.00	0.39	0.00	0.00	0.00	0.00	0.00	0.00	0.00	0.00	0.16	0.00	0.00	0.00	0.36	0.00	0.00	0.00	0.00

. After determining the global concordance, the sets of strong, medium, and weak discordance were determined. Using these sets, the global discordance matrix was determined and is presented in

Table 10. Global discordance matrix.

	1	2	3	4	5	6	7	8	9	10	11	12	13	14	15	16	17	18	19
1	0.00	0.00	0.01	0.14	0.12	0.01	0.00	0.14	0.00	0.00	0.00	0.00	0.00	0.12	0.00	0.04	0.01	0.14	0.00
2	0.14	0.00	0.14	0.14	0.14	0.14	0.14	0.14	0.14	0.14	0.14	0.14	0.14	0.14	0.14	0.14	0.14	0.14	0.00
3	0.14	0.00	0.00	0.14	0.14	0.14	0.00	0.14	0.14	0.00	0.00	0.00	0.00	0.14	0.00	0.14	0.14	0.14	0.00
4	0.00	0.00	0.00	0.00	0.00	0.00	0.00	0.00	0.00	0.00	0.00	0.00	0.00	0.00	0.00	0.00	0.00	0.00	0.00
5	0.14	0.00	0.00	0.14	0.00	0.02	0.00	0.14	0.02	0.00	0.00	0.00	0.00	0.14	0.00	0.08	0.04	0.14	0.00
6	0.14	0.00	0.10	0.14	0.14	0.00	0.00	0.14	0.10	0.00	0.00	0.05	0.04	0.14	0.00	0.14	0.14	0.14	0.00
7	0.14	0.00	0.14	0.14	0.14	0.14	0.00	0.14	0.14	0.14	0.14	0.14	0.14	0.14	0.02	0.14	0.14	0.14	0.00
8	0.00	0.00	0.00	0.00	0.00	0.00	0.00	0.00	0.00	0.00	0.00	0.00	0.00	0.00	0.00	0.00	0.00	0.00	0.00
9	0.14	0.00	0.13	0.14	0.14	0.14	0.00	0.14	0.00	0.00	0.00	0.07	0.08	0.14	0.00	0.14	0.14	0.14	0.00
10	0.14	0.00	0.14	0.14	0.14	0.14	0.00	0.14	0.14	0.00	0.00	0.14	0.14	0.14	0.00	0.14	0.14	0.14	0.00
11	0.14	0.06	0.14	0.14	0.14	0.14	0.14	0.14	0.14	0.14	0.00	0.14	0.14	0.14	0.06	0.14	0.14	0.14	0.00
12	0.14	0.05	0.14	0.14	0.14	0.14	0.08	0.14	0.14	0.12	0.03	0.00	0.14	0.14	0.05	0.14	0.14	0.14	0.00
13	0.14	0.01	0.14	0.14	0.14	0.14	0.03	0.14	0.14	0.06	0.00	0.07	0.00	0.14	0.01	0.14	0.14	0.14	0.00
14	0.14	0.00	0.01	0.14	0.08	0.01	0.00	0.14	0.02	0.00	0.00	0.00	0.00	0.00	0.00	0.06	0.02	0.14	0.00
15	0.14	0.11	0.14	0.14	0.14	0.14	0.14	0.14	0.14	0.14	0.14	0.14	0.14	0.14	0.00	0.14	0.14	0.14	0.05
16	0.14	0.00	0.00	0.14	0.14	0.00	0.00	0.14	0.00	0.00	0.00	0.00	0.00	0.14	0.00	0.00	0.00	0.14	0.00
17	0.14	0.00	0.02	0.14	0.14	0.03	0.00	0.14	0.00	0.00	0.00	0.04	0.00	0.14	0.00	0.14	0.00	0.14	0.00
18	0.14	0.00	0.00	0.14	0.00	0.00	0.00	0.14	0.00	0.00	0.00	0.00	0.00	0.00	0.00	0.02	0.00	0.00	0.00
19	0.14	0.14	0.14	0.14	0.14	0.14	0.14	0.14	0.14	0.14	0.14	0.14	0.14	0.14	0.14	0.14	0.14	0.14	0.00

.

5.2.4. Weak and Strong Outranking Relations

The global concordance and discordance matrices were used to determine the strong and weak outranking relations by comparing the values to the corresponding thresholds $({\alpha }^{-},{\alpha }^0,{\alpha }^{+},{\beta }^{+},{\beta }^0)$ = $(0.6,0.75,0.9,0.05,0.12)$.

Table 11. Strong and weak outranking relations.

	1	2	3	4	5	6	7	8	9	10	11	12	13	14	15	16	17	18	19
1	-	$S^F$	$S^F$	-	-	$S^f$	$S^F$	-	$S^F$	$S^F$	$S^F$	$S^F$	$S^F$	-	$S^F$	-	$S^f$	-	$S^F$
2	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	$S^f$
3	-	$S^F$	-	-	-	-	$S^F$	-	-	$S^F$	$S^F$	$S^F$	$S^F$	-	$S^F$	-	-	-	$S^F$
4	$S^F$	$S^F$	$S^F$	-	$S^F$	$S^F$	$S^F$	-	$S^F$	$S^F$	$S^F$	$S^F$	$S^F$	$S^F$	$S^F$	$S^F$	$S^F$	$S^F$	$S^F$
5	-	$S^F$	$S^F$	-	-	$S^f$	$S^F$	-	$S^F$	$S^F$	$S^F$	$S^F$	$S^F$	-	$S^F$	-	-	-	$S^F$
6	-	$S^F$	$S^f$	-	-	-	$S^F$	-	$S^f$	$S^F$	$S^F$	$S^F$	$S^f$	-	$S^F$	-	-	-	$S^F$
7	-	$S^F$	-	-	-	-	-	-	-	-	-	-	-	-	$S^F$	-	-	-	$S^F$
8	$S^F$	$S^F$	$S^F$	-	$S^F$	$S^F$	$S^F$	-	$S^F$	$S^F$	$S^F$	$S^F$	$S^F$	$S^F$	$S^F$	$S^F$	$S^F$	$S^F$	$S^F$
9	-	$S^F$	-	-	-	-	$S^F$	-	-	$S^F$	$S^F$	$S^f$	$S^f$	-	$S^F$	-	-	-	$S^F$
10	-	$S^F$	-	-	-	-	$S^F$	-	-	-	$S^F$	-	-	-	$S^F$	-	-	-	$S^F$
11	-	$S^f$	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	$S^F$
12	-	$S^f$	-	-	-	-	-	-	-	-	-	-	-	-	$S^f$	-	-	-	$S^F$
13	-	$S^f$	-	-	-	-	$S^f$	-	-	$S^f$	$S^F$	$S^f$	-	-	$S^f$	-	-	-	$S^F$
14	-	$S^F$	$S^F$	-	$S^f$	$S^f$	$S^F$	-	$S^F$	$S^F$	$S^F$	$S^F$	$S^F$	-	$S^F$	-	-	-	$S^F$
15	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	$S^f$
16	-	$S^F$	$S^F$	-	-	$S^F$	$S^F$	-	$S^F$	$S^F$	$S^F$	$S^F$	$S^F$	-	$S^F$	-	$S^F$	-	$S^F$
17	-	$S^F$	$S^F$	-	-	$S^F$	$S^F$	-	$S^F$	$S^F$	$S^F$	$S^F$	$S^F$	-	$S^F$	-	-	-	$S^F$
18	-	$S^F$	$S^F$	-	$S^F$	$S^F$	$S^F$	-	$S^F$	$S^F$	$S^F$	$S^F$	$S^F$	$S^f$	$S^F$	-	$S^F$	-	$S^F$
19	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-

shows the outranking relations $x{S}^{F}y$ (alternative x strongly outranks alternative y) and $x{S}^{f}y$ (alternative x weakly outranks alternative y). In other words, for every alternative pair $(i,j)$, if the $i^{th}$ ontology weakly outranks the $j^{th}$ ontology, then it is expressed as $x{S}^{f}y$, and if it strongly outranks the $j^{th}$ ontology, then it is expressed as $x{S}^{F}y$.

Figure 2 and Figure 3 display the outranking relations $S^F$ and $S^f$ graphically. The ontologies are represented as nodes and the directed edges show which ontologies strongly or weakly outrank other ontologies. The node of each ontology is expressed according to the color scale in-degree value, where green represents 0 and red represents the maximum number of incoming edges. In the strong outranking graph in Figure 2, it can be seen that ontologies 8 and 4 have the lowest node in-degree value, which means they are the strongest, whereas ontology 19 has the highest in-degree value, making it the weakest. In the weak outranking graph in Figure 3, ontologies 1, 3, 11, and 18 have the lowest in-degree value, and ontologies 2 and 6 have the highest values.

5.2.5. Exploitation of Outranking Relations

The final step is to calculate the forward and backward orders and then apply their midpoints to obtain a final ranking.

Table 12. Rankings obtained by forward and backward orders, and their midpoint.

Ontology	${\mathit{\psi }}_1\left(\mathit{x}\right)$	${\mathit{\psi }}_2\left(\mathit{x}\right)$	${\mathit{\psi }}_3\left(\mathit{x}\right)$	$\overline{\mathit{\psi }}\left(\mathit{x}\right)$
1	2	9	3	2.5
2	10	2	10	10
3	6	6	6	6
4	1	11	1	1
5	4	8	4	4
6	5	7	5	5
7	9	3	9	9
8	1	11	1	1
9	6	6	6	6
10	8	4	8	8
11	9	3	9	9
12	8	3	9	8.5
13	7	5	7	7
14	3	9	3	3
15	10	2	10	10
16	2	9	3	2.5
17	3	8	4	3.5
18	2	10	2	2
19	11	1	11	11

shows the ranks obtained by the forward order (${\psi }_1$), the backward order (${\psi }_2$), the transformed backward order (${\psi }_3$), and the midpoint ($\overline{\psi }$).

5.2.6. Final Ranking of Ontologies

The midpoint ($\overline{\psi }$) values in the rightmost column of Table 12 represent the final ranking of the ontologies. Some ontologies are shown to occupy the same position in the ranking. For example, the first position is occupied by ontologies 4 (EDAM ontology in Table 1) and 8 (ITO ontology in Table 1), ontologies 1 and 16 in the 2.5 position, ontologies 2 and 10 in the 10th position, etc. The complete final ranking of the ontologies in increasing order is as follows: $4, 8\succ 18\succ 1, 16\succ 14\succ 17\succ 5\succ 6\succ 3, 9\succ 13\succ 10\succ 12\succ 7, 11\succ 2, 15\succ 19$. That is, the EDAM and ITO ontologies in Table 1 are both ranked in first place. The second position is occupied by the SWeLMS ontology (see Table 1). In third position are the MLOnto and OntoDM ontologies. The MLSchema ontology in Table 1 is ranked last. The second and third last-ranked ontologies are the pair <Expose, DSEO> and <RAInS, Mex-Core>, ranked at positions 9 and 10, respectively.

The heatmap in Figure 4 expresses the extent to which each ontology in the dataset in Table 1 covers AI concepts. It can be observed that there are certain topics that have very poor coverage throughout almost all ontologies in the dataset. Only 3 out of 19 ontologies in Table 1 contain concepts regarding autoencoders: MLOnto, ITO, and AIO ontologies. Explainable AI is also represented very sparsely, with just 6 of 19 ontologies containing related content. A mere 2 of 19 ontologies contain concepts related to quantum computing; these are MLOnto and ITO ontologies. Transfer learning only appears in 1 of 19 ontologies, ITO ontology, and Transformer finds itself in just 4 of the 19 ontologies. This means that these concepts are present in just a few ontologies, and more work is required to develop further ontologies around these concepts.

The stacked bar graph in Figure 5 depicts the different quality metrics for each of the ontologies. It can be observed that the dataset in Table 1 contains both large and small ontologies. The largest ontology is the ITO ontology, with over 1 million axioms, 16,000 classes, and 93,000 individuals. The smallest ontologies are MEX-Perf, MLSchema, and Expose, with fewer than 500 axioms each. Regarding the density of the ontologies, SWeLMS has a very high average population density as well as a high relationship richness, suggesting a well-linked and densely populated ontology. Ontologies like EDAM and MLLO have many axioms and classes, but they lack individuals. Meanwhile, SWeLMS, VAIR, and FAIRnets have a more balanced level, showing both high class richness and large populations. The ranking achieved by the proposed Lq-ROFS-ELECTRE II (Table 12), along with the above analysis of the characteristics of the ML ontologies in the dataset in Table 1, can assist in the selection and reuse of these ontologies in the ML domain.

5.3. Comparative Analysis

This section compares the proposed Lq-ROFS-ELECTRE II algorithm to the traditional ELECTRE II method and other existing MCDM methods.

5.3.1. Comparison with Traditional ELECTRE II

In order to ascertain the validity of the ranking obtained with the proposed Lq-ROFS-ELECTRE II, the traditional ELECTRE II method is applied on the same dataset of ML ontologies in Table 1. However, traditional ELECTRE II cannot handle qualitative/linguistic variables and therefore it cannot make use of the 7 criteria used in the proposed Lq-ROFS-ELECTRE II algorithm. Instead, the 7 quantitative metrics displayed in Figure 5 were calculated and used as the criteria to run the traditional ELECTRE II method. Using concordance thresholds of $p^{-}=0.5$, $p^0=0.6$, and $p^{*}=0.7$, and discordance thresholds of $q^0=0.3$ and $q^{*}=0.4$, the traditional ELECTRE II method generated a ranking of 8 ≻ 18 ≻ 3, 14, 16 ≻ 1, 4, 5 ≻ 6, 17 ≻ 7 ≻ 13 ≻ 9 ≻ 19 ≻ 10 ≻ 2 ≻ 11 ≻ 15 ≻ 12.

It can be observed that the rankings from the proposed Lq-ROFS-ELECTRE II and the traditional ELECTRE II are quite similar, with some ontologies being assigned the same rank by both methods. Ontology 8 (ITO) was placed first by both methods. The graph in Figure 6 shows the ranks assigned to each ontology from either methods. The blue plot shows the ranks from the Lq-ROFS-ELECTRE II, the purple plot shows the ranks from the traditional ELECTRE II, and the red bars depict the rank difference for each ontology, that is, the difference between the rank given by the proposed Lq-ROFS-ELECTRE II and traditional ELECTRE II. When the rank difference is 0, it means the ontology was ranked the same by both methods. It can be seen that both plots follow the same trend. In $\mathrm{42\%}$ of instances, the same ranking is assigned to the ontologies, specifically, ontologies 8, 9, 10, 11, 15, 16, 17, and 18. The minimum non-zero rank difference is 1, and the maximum rank difference is 6. On average the difference in ranks between the methods is $2.9$.

To further evaluate and compare the rankings, two statistical metrics are calculated, that is, Spearman’s Rho and Kendall’s Tau correlation coefficients. The Spearman Rho value was $0.84$ and the Kendall Tau value was $0.67$. Both these values are relatively high which implies that there is a strong positive monotonic relationship between the rankings from proposed Lq-ROFS-ELECTRE II and traditional ELECTRE II. When the rank from one method increases, the rank from the other method tends to increase as well, and they largely agree on the order of ontologies. These findings attest to the validity of the ranking performed by the proposed Lq-ROFS-ELECTRE II algorithm and its effectiveness in the task of ontology ranking.

5.3.2. Comparison with Existing MCDM Methods

The proposed Lq-ROFS-ELECTRE II algorithm is compared with other MCDM methods that were applied for ontology selection in

Table 13. Comparison of Lq-ROFS-ELECTRE II with other MCDM methods for ontology ranking.

MCDM Method	Domain	No. of Ontologies	Quantitative Criteria	Qualitative Criteria
Traditional Methods
ELECTRE I [54]	Biomedical	70	8	No
ELECTRE I/III [18]	Academia	12	5	No
ELECTRE I/II/III/IV [10]	Biomedical	200	13	No
WLCRT [17]	Biomedical	100	8	No
TOPSIS/WSM/WPM [9]	Biomedical	70	8	No
Fuzzy Methods
ZPLTS-ELECTRE II [32]	Mental Health	9	5	5
L$\mathit{q}$-ROFS-ELECTRE II	Machine Learning	19	0	7

. It can be seen in Table 13 that only the proposed Lq-ROFS-ELECTRE II algorithm has been tested in ML ontologies, while most of the other MCDM methods used datasets of biomedical ontologies. The bottom of Table 13 also shows that the proposed Lq-ROFS-ELECTRE II algorithm has been tested on a larger dataset compared to the other fuzzy method implemented in an earlier study. Furthermore, the proposed Lq-ROFS-ELECTRE II algorithm implements the largest number of qualitative criteria than any previous study. These qualitative criteria implemented by the Lq-ROFS-ELECTRE II algorithm make it superior to other existing MCDM methods in Table 13 in that (1) it can deal with linguistic criteria that allow for more natural decision evaluation from subject experts in real-world applications of ontology selection, and (2) the individual concordance, discordance, and indifference weights, as well as outranking relation thresholds, allow the decision-maker to have much flexibility in expressing their preferences.

The proposed Lq-ROFS-ELECTRE II algorithm is further compared with other fuzzy variations of ELECTRE II in

Table 14. Comparison of proposed Lq-ROFS-ELECTRE II with ELECTRE II fuzzy variants based on key modeling capabilities.

ELECTRE II Variant	Structure	Membership	Non-Membership	Indeterminacy	Linguistic Support	Threshold Adaptability
Traditional ELECTRE II [28]	Crisp	×	×	×	×	×
IFS-ELECTRE II [33]	IFS	✓	✓	✓	×	×
PF ELECTRE II [19]	PFS	✓	✓	✓	×	×
PLTS-ELECTRE II [55]	PLTS	✓	×	✓	✓	✓
ZPLTS-ELECTRE II [32]	Z-PLTS	✓	×	✓	✓	✓
q-ROFS-ELECTRE II [56]	q-ROFS	✓	✓	✓	×	✓
L$\mathit{q}$-ROFS-ELECTRE II	L$\mathit{q}$-ROFS	✓	✓	✓	✓	✓

, specifically the Intuitionistic Fuzzy Set (IFS), Pythagorean fuzzy set (PF), Probabilistic Linguistic Term Set (PLTS), Z-Probabilistic Linguistic Term Set (Z-PLTS), and q-ROFS versions of ELECTRE II. Table 14 shows that, unlike other ELECTRE II-based algorithms that rely on fixed threshold values, the proposed Lq-ROFS-ELECTRE II algorithm offers threshold adaptability through the integration of a tunable q-parameter and linguistic evaluations, allowing context-sensitive outranking that aligns with the uncertainty and semantics of decision-maker inputs. It can be observed from Table 14 that the proposed Lq-ROFS-ELECTRE II algorithm is the only one that allows both membership and non-membership degrees, as well as indeterminancy degree, provides linguistic support, and allows for threshold adaptability. The combination of these capabilities leads to a powerful and expressive framework for decision-makers, whilst maintaining a balance between flexibility and accuracy.

5.3.3. Time Complexity of the Lq-ROFS-ELECTRE II Algorithm

The time complexity of the proposed Lq-ROFS-ELECTRE II algorithm was analyzed to further assess its efficiency. The 3 main parameters in the proposed Lq-ROFS-ELECTRE II algorithm outlined in Algorithm 1 are the number of decision-makers denoted as R, the number of alternatives denoted by m, and the number of criteria denoted as n. The time complexity for each step of the Lq-ROFS-ELECTRE II algorithm is shown in

Table 15. Time complexity analysis of Lq-ROFS-ELECTRE II algorithm steps.

Step	Time Complexity
1	$O(R\times m\times n)$
2	$O(R\times m\times n)$
3	$O\left(n\right)$
4	$O(m^2\times n)$
5	$O(m^2\times n)$
6	$O(m^2\times n)$
7	$O\left(1\right)$
8	$O(m^2\times n)$
9	$O(m^2\times n)$
10	$O\left(m^2\right)$
11	$O\left(m^3\right)$
Overall	$O(Rmn+m^{2}n+m^3)$

. The total time complexity is $Rmn+m^{2}n+m^3$. This time complexity may vary depending on the values of R, m and n. Specifically, there are 3 cases where the complexity of the proposed Lq-ROFS-ELECTRE II algorithm is reduced. The first case is when there are many alternatives and relatively few criteria, where $m\gg n$ and $m\gg R$. In this case $m^3$ dominates and the complexity becomes $O\left(m^3\right)$. The second case is when there are many criteria relative to alternatives, that is, $n\gg m$ and $m^2\times n\gg m^3$. Here the complexity becomes $O\left(m^{2}n\right)$. The third case is when there are many decision-makers, that is, $R\gg m$ and $R\times n\gg m^2$. The complexity becomes $O\left(Rmn\right)$ in this case. The surface plots representing the different time complexity cases are shown in Figure 7, Figure 8, Figure 9 and Figure 10. It can be observed in the general case (Figure 7) that there is nonlinear growth, combining contributions from all terms. When there are many alternatives (Figure 8), it can be seen that the surface is relatively flat along the number of criteria, indicating the minimal impact of n. The sharp increase in the third surface plot along n (Figure 9) indicates the effects of the dominance of the criteria. And when the number of decision-makers (R) dominates (Figure 10), a proportional growth can be observed with minimal curvature compared to other cases.

5.3.4. Limitations of the Proposed Lq-ROFS-ELECTRE II

Despite its superior capabilities over the existing MCDM methods and fuzzy ELECTRE II variants as presented above, the proposed Lq-ROFS-ELECTRE II algorithm has some limitations.

• The large number of weights required to be defined may sometimes be difficult for the user to select, especially if he/she does not want to specify importance weightings.
• The concordance and discordance thresholds require expert knowledge in order to define appropriate thresholds.

6. Applicability of L$\mathit{q}$-ROFS-ELECTRE II in Other Domains

Although the proposed Lq-ROFS-ELECTRE II algorithm has been implemented to rank ontologies, it can be applied in any domain where there are decision-making scenarios that comprise multiple conflicting criteria under uncertainty. The concept of Lq-ROFS enhances the algorithm’s ability to handle uncertainty, hesitation, and imprecise information, which are common challenges in real-world decision environments. The integration of Lq-ROFS with the ELECTRE II framework further strengthens the Lq-ROFS-ELECTRE II algorithm by incorporating both concordance and discordance analyses, which allow for robust ranking even in the presence of conflicting evaluations. Some critical areas where the proposed Lq-ROFS-ELECTRE II algorithm may be particularly useful are as follows:

• Medical and healthcare diagnosis tasks, such as selecting optimal treatment plans based on criteria such as effectiveness, side effects, cost, and patient preferences.
• Smart cities and urban planning tasks, such as evaluating and prioritizing urban development projects using criteria like cost, environmental impact, public acceptance, and scalability.
• Education and e-learning systems evaluation, such as the evaluation of different e-learning platforms or curricula according to factors like accessibility, pedagogical effectiveness, user experience, and flexibility.
• Finance and investment tasks, such as portfolio selection and optimization and investment decision-making under risk and uncertainty.

7. Conclusions and Future Work

In this paper, a novel decision-making algorithm, namely Lq-ROFS-ELECTRE II, was developed and applied to rank 19 ontologies of the ML domain in light of seven qualitative criteria. The criteria were extracted from the Ontometric framework and expressed in terms of its Language, Content, and Methodology categories. Four experts evaluated the criteria using the defined linguistic term set, which were then used as the decision matrices to the ranking problem. The ranking results show that the Lq-ROFS-ELECTRE II algorithm has successfully ranked all 19 ML ontologies. In order to ascertain the validity of the ranking, the traditional ELECTRE II method was applied to rank the 19 ontologies according to seven quantitative metrics. Spearman’s Rho and Kendall’s Tau correlation coefficients were calculated to compare the ranking obtained from the Lq-ROFS-ELECTRE II and the ELECTRE II methods. Despite the usage of different criteria, both methods have a fairly high level of correlation, indicating the validity of the rankings by the proposed Lq-ROFS-ELECTRE II. The proposed Lq-ROFS-ELECTRE II algorithm provides a more flexible and robust structure for experts to provide their judgments, making it better suited for the real-world application of ontology selection compared to existing MCDM methods. Furthermore, a comparative analysis of the proposed Lq-ROFS-ELECTRE II compared to other existing fuzzy ELECTRE II methods showed its superior modeling capabilities.

Future work would focus on applying the proposed Lq-ROFS-ELECTRE II algorithm to aid in ontology selection and ranking by considering a larger dataset of ontologies, as well as ontologies from other domains. Furthermore, a future direction would focus on the development of expressive criteria which can be used to evaluate and rank ontologies. Apart from the task of ontology ranking, the proposed Lq-ROFS-ELECTRE II algorithm may be applied to decision-making problems, both in academic and real-world environments. To further enhance the modeling capabilities of the Lq-ROFS-ELECTRE II algorithm, it can be integrated with other intelligent methods such as machine learning.