A Fuzzy DEMATEL-Based User-Centric Design Evaluation of Gamified Recommender Systems

Featured Application

This research presents a fuzzy DEMATEL-based evaluation methodology for gamified recommender systems, providing significant insights for adaptive system design in education, e-commerce, and healthcare. The framework emphasizes critical factors for adoption, such as usefulness, ease of use, and trust. It also provides advice on how to design: using rewards and competition in Points, Badges, and Leaderboards (PBL) design makes people more motivated in the short term. Acknowledgments, Objectives, and Progression (AOP) design makes things more open and helps things move along. Acknowledgments, Competition, and Time Pressure (ACT) design promotes productivity under competitive circumstances. Acknowledgments, Objectives, and Social Pressure (AOS) gets things done by using social responsibility and peer pressure. These results help developers to implement gamification elements with respect to domain-based objectives, enabling the active involvement of users.

Abstract

Gamified recommender systems, which mix game design with recommendation frameworks, are a new way to increase user involvement and satisfaction. Even though they have a lot of potential, there has not been any systematic research on how their design affects how people use them. This study introduces a fuzzy DEMATEL-based framework for the assessment and enhancement of gamified recommender systems. Four theoretically grounded gamified recommender system prototypes were developed as a novel contribution, as no readily available systems exist for these designs. The assessment utilized nine user-centric criteria—Effectiveness, Transparency, Persuasiveness, Satisfaction, Trust, Usefulness, Ease of Use, Efficiency, and Education—systematically derived from a PRISMA-guided literature review. This study integrates gamification theory, systematic review, and fuzzy decision-making to formulate a comprehensive framework for identifying the key factors influencing adoption. The fuzzy DEMATEL was applied to evaluate feedback from 25 end-users, and it was found that usefulness and ease of use were the most essential factors for satisfaction and system effectiveness. Analysis of design showed that competition in Points, Badges, and Leaderboards (PBL) design boosts short-term motivation, Acknowledgments, Objectives, and Progression (AOP) boosts progress and openness, Acknowledgments, Competition, and Time Pressure (ACT) boosts efficiency in competitive situations but might lower satisfaction, and Acknowledgments, Objectives, and Social Pressure (AOS) depends on social influence and accountability.

1. Introduction

Recommender systems (RS) are commonly used in today’s era. They recommend user-specific purchasing, learning, traveling, and entertainment [1,2,3,4,5,6]. By analyzing large amounts of data, RS improves decision-making. This boosts user satisfaction and engagement. Even though they are successful, keeping long-term use going is still a big problem, especially in situations where user motivation and tenacity are very important [7,8]. Digital platforms need recommender systems (RS) to guide users to shop, study, travel, and have fun [1,2,3,4,5,6]. Eliminating unnecessary information helps RS users make better decisions. It makes users satisfied and more engaged. Gamification elements also improve motivation, engagement, and behavior outside of games [9,10,11,12,13,14]. Trust, satisfaction, and system use were improved through the use of game elements in e-commerce, workplace technology, and healthcare and in education, learning performances, and engagement in gamification studies [13,15,16,17,18,19,20,21]. Gamified recommender systems (GRS) improve user adoption and user experience by combining personalization with incentive design [22,23,24,25,26,27,28]. GRS has been proposed for career promotion [29], tourism [22], adaptive learning [20,25], and consumer engagement [24]. Prior systematic reviews emphasize that explanations in recommender systems should not only enhance decision effectiveness and transparency, but also build trust, increase satisfaction, ensure usefulness and ease of use, improve efficiency, support persuasion, and even enable educational benefits [29]. These dimensions provide a comprehensive basis for evaluating the quality and impact of gamified recommender designs [16]. Recent reviews confirm their growing relevance and potential [30]. When designed well, gamification boosts user motivation and engagement [31].

However, most existing studies on GRS remain limited to conceptual domain-specific prototypes or narrative reviews [6,7,30,32,33,34]. Few works have examined the causal interrelationships among gamification elements that influence engagement and adoption. Prior research often employs descriptive or comparative approaches [10,11,24,32] with limited application of multiple-criteria decision-making (MCDM) techniques [34,35,36]. Prior reviews addressed the absence of analytical methods to inspect causal interrelations among gamified design frameworks [37]. Several multi-criteria decision-making (MCDM) techniques—such as the Analytic Hierarchy Process (AHP) [38], Analytic Network Process (ANP) [39], Technique for Order Preference by Similarity to Ideal Solution (TOPSIS) [40], and Decision-Making Trial and Evaluation Laboratory (DEMATEL) [41]—have been applied to gamification evaluation and system design. Recent studies have emphasized the growing role of intelligent and hybrid decision-making approaches in evaluating user-centric systems and complex learning environments. For instance, one study showed hybrid MCDM approaches as spherical fuzzy DEMATEL, ANP, and VIKOR (VIekriterijumsko KOmpromisno Rangiranje) can improve the modeling of evaluation criteria interdependencies under uncertainty, which is also helpful in analyzing complex relationships among gamification elements in recommender system design [42]. Researchers [43] examined decision-making frameworks in engineering that utilize artificial intelligence, with a focus on how multi-criteria evaluation can facilitate the creation of adaptive learning environments. These developments align with the current study’s objective to employ fuzzy-based causal modeling for the examination of gamified recommender systems. AHP, ANP, and TOPSIS are some of the methods that have been used to evaluate gamification. They generally help people decide what to do initially rather than making cause-and-effect connections. Gabus and Fontela developed DEMATEL to demonstrate and quantify complex cause-and-effect relationships between system variables [41]. This helps people comprehend how different factors affect each other while making technology–people decisions [44]. DEMATEL has advanced with the inclusion of fuzzy set theory, hybrid extensions like gray DEMATEL, rough DEMATEL, and amalgamations with other MCDM methods like AHP and TOPSIS [45,46,47,48,49].

These changes have made it much better at modeling uncertainty and taking into account both direct and indirect effects between evaluation criteria. This makes fuzzy DEMATEL a very useful tool for studying systems design and behavioral assessment. In addition, the fuzzy DEMATEL approach has proven effective in modeling interdependencies in fields such as education [50], retail [21], and gamification research [24], yet its application to GRS design evaluation remains scarce. Similarly in one study, a personalized gamified learning recommender system that dynamically adapts game elements to learner profiles was proposed [51], yet such systems rarely use DEMATEL to examine the causal relationships between gamification factors that drive engagement and adoption. This methodological gap restricts the ability to identify which gamification elements act as key drivers and how they interact to shape user outcomes.

To bridge this gap, this study introduces a fuzzy DEMATEL-based evaluation framework for gamified recommender systems. Three layers of novelty are offered. First, four JavaScript React-based prototypes were developed—Points, Badges, and Leaderboards (PBL); Acknowledgments, Objectives, and Progression (AOP); Acknowledgments, Competition, and Time Pressure (ACT); and Acknowledgments, Objectives, and Social Pressure (AOS)—since no readily available systems exist for these theoretically grounded designs [24,26,37,51]. Second, nine user-centric evaluation criteria—Effectiveness, Transparency, Persuasiveness, User Satisfaction, Trust, Usefulness, Ease of Use, Efficiency, and Education—were systematically derived through a PRISMA-guided review of 823 studies, refined to 19 relevant works published between 2013 and 2023. These criteria are widely supported in the prior literature as central to gamification and recommender system evaluation [13,15,17,18,19,20]. Third, fuzzy DEMATEL was applied to capture the causal structure among criteria, distinguishing between driving and dependent factors and thereby revealing the most influential determinants of adoption whereas other studies used DEMATEL to map causal links in a sustainability performance framework, showing its suitability for gamified recommender system evaluation [52].

The significance of this study lies in its integration of theoretical, methodological, and practical contributions. Theoretically, it advances gamification research by operationalizing four distinct GRS prototypes beyond discussions. This study uses a systematic literature review and fuzzy DEMATEL analysis to model complicated interdependencies and provide a reproducible evaluation methodology. The findings suggest design objectives for adaptive, user-centered gamified systems. These priorities include utility, usability, and trust. These ideas help education, e-commerce, and healthcare by encouraging people to use, like, and stay using the service.

2. Related Research

Gamification is the use of game-like elements in non-game contexts. It is often used to increase motivation, involvement, and behavioral adjustment [9,10,11,12,13,14,32]. Its positive effects on learning outcomes, involvement, and user satisfaction in education highlight its importance in boosting effectiveness and educational value [13,15,17,18,19,20]. Systematic evaluations show that gamification can boost motivation and performance, provided that game mechanisms are designed to provide clear feedback and progression routes [10,15,32]. Gamification in workplace technology, retail, and tourism has shown persuasive and trust-enhancing effects on system adoption and use [16,21,22].

Recommender systems (RS) have also become very important on digital platforms because they help users make content their own and deal with too much information [1,2,3,4,6,8]. Collaborative filtering, content-based filtering, and hybrid models have been widely employed in diverse fields such as e-commerce, education, entertainment, and healthcare [2,5,6,7]. Recent research has examined RS via ethical, explanatory, and cognitive perspectives, pinpointing transparency, trust, and efficiency as essential determinants for enduring adoption [1,2,4]. Even with these improvements, it is still hard to keep people interested and find a balance between personalization and user freedom.

Gamified recommender systems (GRS) are a new type of system that combines personalization with motivational design to encourage more people to use them. Proposed applications encompass career advancement systems [29], tourism [22], customized learning trajectories [17,25], and entertainment [26,27]. For example, researchers in [53] developed a gamified word-of-mouth RS to foster customer engagement [24], while researchers in [30] provided a systematic overview of GRS design trends and applications. To structure such designs, prior works have emphasized different gamification strategies: Points, Badges, and Leaderboards (PBL) as classic extrinsic motivators [24,37]; Acknowledgments, Objectives, and Progression (AOP) for transparent progression and competence-based feedback [26]; Acknowledgments, Competition, and Time Pressure (ACT) for efficiency and engagement under competitive conditions [24]; and Acknowledgments, Objectives, and Social Pressure (AOS) for leveraging social accountability and persuasiveness [51]. However, it is important to note that such comprehensive gamified recommender system designs are not readily available in practice, as most existing works remain limited to conceptual discussions or isolated case studies. Developing functional prototypes therefore represent a novel contribution of this study.

To provide more systematic evaluations, researchers have applied multiple-criteria decision-making (MCDM) techniques such as AHP, ANP, and TOPSIS [24,34,35,36,37]. Although these methods facilitate prioritization, they seldom account for the causal interdependencies among evaluation criteria, including trust, ease of use, and efficiency. While traditional multi-criteria methods such as AHP, ANP, TOPSIS, and DEMATEL have been extensively applied, more recent studies advocate integrating sustainability-oriented and knowledge-driven decision frameworks. A study improved DEMATEL by using a threshold-based approach to determine the most influential aspects, which helps recommender system evaluation prioritize key gamification criteria [54].

Building on this perspective, the current research adopts a fuzzy DEMATEL framework to uncover the interdependencies among gamification criteria, extending the methodological line of recent MCDM and AI-enhanced decision studies [1,2,3]. The Decision-Making Trial and Evaluation Laboratory (DEMATEL), particularly in its fuzzy extension, has proven effective in modeling such cause–and–effect structures [55,56,57,58,59]. It has been successfully applied in higher education [49], retail [21], and environmental management [44], but its use in GRS design remains limited. Most existing works either list gamification features or apply comparative frameworks without addressing how these nine critical dimensions interact with specific design strategies (PBL, AOP, ACT, AOS) to shape user experience and adoption [10,11,32].

This review reveals two evident gaps. There are not many complete frameworks on GRS, and most of the research is based on individual case studies. Second, although MCDM approaches have been employed, there has been insufficient emphasis on determining which criteria—such as usefulness, satisfaction, or trust—act as drivers and which function as dependent outcomes. This study introduces a fuzzy DEMATEL-based evaluation framework designed to rectify the deficiencies present in gamified recommender systems. The contributions include the development of four prototypes utilizing JavaScript React (PBL, AOP, ACT, AOS), informed by the existing literature [23,24,26,37], the systematic identification of nine user-centric criteria through a PRISMA-guided review, and the application of fuzzy DEMATEL to elucidate causal interrelationships. These contributions create a strong and repeatable framework for making user-centered gamified recommender systems that can be used in education, e-commerce, and healthcare.

This study aims to identify the principal user-centric factors that facilitate the adoption of gamified recommender systems.

• RQ1: What are the causal relationships among main user-centric criteria?
• RQ2: What elements act as principal catalysts for the implementation of gamified recommender systems?
• RQ3: In what ways do various gamified designs (PBL, AOP, ACT, AOS) affect these relationships?

3. Materials and Methods

This study followed a systematic multi-step methodology to evaluate gamified recommender system (GRS) designs using fuzzy DEMATEL. The approach combined a systematic literature review for criteria identification, prototype development for alternative designs, decision makers’ elicitation for evaluation, and fuzzy DEMATEL analysis for cause–effect modeling.

3.1. Research Design

A multi-criteria decision-making (MCDM) framework was adopted, as such an approach provides structured tools for evaluating complex socio-technical systems with multiple interacting factors. The fuzzy Decision-Making Trial and Evaluation Laboratory (fuzzy DEMATEL) was selected due to its ability to model interdependencies among criteria and distinguish between driving and dependent factors.

3.2. Identification of Criteria

To evaluate gamified recommender systems (GRS), this study adopted a systematic literature review approach guided by the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) framework [60]. A total of 823 records were initially identified from IEEE Xplore, ScienceDirect, Scopus, and Web of Science. After applying inclusion and exclusion criteria, duplicate removal, and language filtering, 19 primary studies published between 2013 and 2023 were retained for full-text analysis.

From these 19 investigations, recurrent characteristics of user experience and system quality were extracted using a systematic coding process. User engagement, system adoption, and gamification results were the targets of a thorough literature review. Effectiveness, Transparency, Persuasiveness, Trust, Usefulness, Ease of Use, Efficiency, and Education are the nine user-focused evaluation criteria that were identified during the investigation. All of the criteria were based on research that actually happened in the real world. To determine its efficacy, we consulted research that examined engagement with gamification features and personalization [26,35,53,61,62]. Clarity and interpretability in gamified recommendation system processes were the focus of studies that eventually gave rise to transparency [23,25,27,63,64].

Research on motivational and behavioral change mechanisms [16,29,65,66,67] acknowledges persuasiveness.

Feedback dashboard and educational system studies focused on user satisfaction [17,68].

Gamification frameworks identified ease of use and usefulness in large-scale, user-friendly system designs [68].

Research on motivational and behavioral change pathways identified persuasiveness [16,29,65,66,67].

Efficiency was supported by studies emphasizing algorithmic accuracy and learning optimization [17,22].

Education was confirmed by gamification in MOOCs and e-learning contexts [67].

The final selection of criteria thus represents a comprehensive synthesis of recurring evaluation dimensions across multiple contexts, including education, tourism, healthcare, and smart cities. These nine criteria were subsequently employed in the fuzzy DEMATEL framework to capture causal interrelationships and identify the most influential dimensions in the evaluation of gamified recommender systems.

3.3. Developing Four Gamified Recommender System Designs

From the prior literature on gamification design frameworks [23,24,26], four gamified recommender system designs were conceptualized, each combining different game mechanics that personalize gamification based on empirical data from learners’ real experiences. The designs use basic motivational ideas like competence, autonomy, relatedness, and social influence to create a theory-based framework for customizing gamified learning environments. These designs show that combinations of elements fit with established motivational constructs and improve personalization by connecting gamification design to learners’ intrinsic motivation and contextual characteristics [26]. The choice of English language learning as the focus area stems from its worldwide demand and the existence of explicit, quantifiable goals, such as vocabulary acquisition, grammatical proficiency, and reading comprehension. Students often have trouble staying motivated and practicing regularly, which makes this setting a good place to use gamification. Recommender systems tailor content to individual learner requirements, while gamification enhances motivation, feedback, and perseverance. This domain is useful in real life and fits well with the nine evaluation criteria used in this study. We used the JavaScript React environment to write code. The four designs of the gamified recommender system are as follows (see Appendix A for full design details):

Design 1: Points, Badges, and Leaderboards (PBL)—based on Game Design Theory, specifically traditional gamification mechanics—has demonstrated the ability to enhance extrinsic motivation (see Figure 1).

Design 2: Acknowledgments, Objectives, and Progression (AOP) places a strong emphasis on Self-Determination Theory (SDT) by recognizing achievement and setting goals (Figure 2).

Design 3: Acknowledgments, Objectives, and Social Pressure (AOS)—builds on Social Influence Theory, highlighting peer comparison and social accountability (see Figure 3).

Design 4: Acknowledgments, Competition, and Time Pressure (ACT) is based on Behaviorist Learning Theory, emphasizing that reinforcement and external stimuli, such as competition and urgency, influence behavior (Figure 4).

The designs show different ways to motivate people in gamified settings and can be used as different ways to add gamification to recommendation systems.

Table 1. Gamification comparisons across four recommender system designs.

Design	Theoretical Basis	Key Gamification Elements	Motivational Focus
PBL	Game Design Theory	Points, badges, leaderboards	Extrinsic rewards, competition, progress tracking
AOP	Self-Determination Theory	Objectives, acknowledgments, progression	Goal setting, competence, clear growth paths
AOS	Social Influence Theory	Objectives, acknowledgments, social pressure	Peer accountability, recognition, group motivation
ACT	Behaviorist Learning Theory	Acknowledgments, competition, time pressure	Reinforcement, rivalry, urgency

presents a comparison of four features of gamified recommender systems.

3.4. Data Collection

3.4.1. Participants

A group of 25 undergraduate students made the decisions. They had already used four gamified recommender systems in their classes and projects. Undergraduate participants were selected not as professional practitioners, but because they represent the end-users of educational recommender systems, aligning with the study’s context. The demographic characteristics of the 25 participants exhibit a combination of diversity and relevance to the study context. Of the 25 people who took part, 16 were males (64%) and 9 were females (36%). There were two groups of participants based on their academic backgrounds: Computer Information Systems, which had 14 participants (56%), and Management Information Systems, which had 11 participants (44%). Ten of the participants (40%) said they used recommender systems very often, and seven (28%) said they used them often. Five students (20%) used them less often, and three (12%) used them extremely rarely. Participants had different levels of platform experience, according to the data. Seven (28%) used Coursera and Khan Academy, six (24%) used Netflix and Amazon Prime Video, five (20%) used Spotify, four (16%) used Duolingo to learn a new language, and smaller groups used Strava (8%) and Fitbit (4%) to reach fitness goals.

This distribution highlights both the diversity of systems experienced by the participants and their practical awareness of recommender technologies, ensuring that their evaluations provide comprehensive and relevant insights for assessing gamified recommender systems.

3.4.2. Procedure

After ethical approval was granted by the researchers’ university ethical board on 25 April 2024 with the application number NEU/AS/2024/214, the decision makers tried four gamified recommender system designs throughout the coursework. They were presented with an evaluation survey for each gamified recommender system design. The decision makers provided pairwise comparisons of nine criteria using a linguistic questionnaire for each GRS design (e.g., “no influence,” “low influence,” “high influence”). Responses were aggregated into fuzzy direct-influence matrices using the geometric mean method to obtain the final cause–effect diagrams for each design.

3.5. Fuzzy DEMATEL Method

Let $\mathcal{C}=\{C_1,C_2,\dots ,C_n\}$ represent the evaluation criteria and $\mathcal{D}=\left\{\mathrm{PBL},\mathrm{AOP},\mathrm{AOS},\mathrm{ACT}\right\}$ the design alternatives.

Step 1—Define Criteria and Alternatives

Identify the nine evaluation criteria and the four designs (AOP, ACT, AOS, PBL).

Step 2—Collect Expert Evaluations

Experts provide pairwise influence judgments $C_i\to C_j$ using linguistic terms mapped to triangular fuzzy numbers (TFNs):

$$\mathrm{No}=(0,0,0.25),\mathrm{Very}\mathrm{Low}=(0,0.25,0.5),\mathrm{Low}=(0.25,0.5,0.75),\mathrm{High}=(0.75,1,1),\mathrm{Very}\mathrm{High}=(\mathrm{0.75,1},1)$$

For each decision maker $k=\mathrm{1,2},\dots ,K$:

$$\begin{array}{c}{\tilde{\mathbf{X}}}^{\left(k\right)}=[{\tilde{x}}_{ij}^{\left(k\right)}{]}_{n\times n},{\tilde{x}}_{ij}^{\left(k\right)}=({\mathcal{l}}_{ij}^{\left(k\right)},m_{ij}^{\left(k\right)},u_{ij}^{\left(k\right)}),{\tilde{x}}_{ii}^{\left(k\right)}=(\mathrm{0,0},0)\end{array}$$

(1)

Step 3—Aggregate Expert Opinions (Geometric Mean)

$$\begin{array}{c}{\tilde{x}}_{ij}=({\displaystyle \prod _{k=1}^K} {\mathcal{l}}_{ij}^{\left(k\right)}{)}^{1/K},{\tilde{m}}_{ij}=({\displaystyle \prod _{k=1}^K} m_{ij}^{\left(k\right)}{)}^{1/K},{\tilde{u}}_{ij}=({\displaystyle \prod _{k=1}^K} u_{ij}^{\left(k\right)}{)}^{1/K}\end{array}$$

(2)

Step 4—Defuzzification (Centroid Method)

$$\begin{array}{c}{x}_{ij}={\displaystyle \frac{{\mathcal{l}}_{ij}+m_{ij}+u_{ij}}{3}}\end{array}$$

(3)

This produces the crisp direct-relation matrix $X=[x_{ij}{]}_{n\times n}$.

Step 5—Normalize the Direct-Relation Matrix

$$\begin{array}{c}s=\mathrm{m}\mathrm{a}\mathrm{x}\{\underset{i}{\mathrm{m}\mathrm{a}\mathrm{x}}{\displaystyle \sum _{j=1}^n} x_{ij},\underset{j}{\mathrm{m}\mathrm{a}\mathrm{x}}{\displaystyle \sum _{i=1}^n} x_{ij}\},\mathrm{N}={\displaystyle \frac{1}{s}}X\end{array}$$

(4)

Step 6—Compute the Total-Relation Matrix

$$\begin{array}{c}\mathrm{T}=\mathrm{N}+{\mathrm{N}}^2+{\mathrm{N}}^3+\dots =\mathrm{N}(\mathrm{I}-\mathrm{N}{)}^{-1}\end{array}$$

(5)

Normalization guarantees $\rho \left(\mathrm{N}\right)<1$, so the series converges ($k\to \mathrm{\infty }$).

Step 7—Determine the Threshold Value

$$\begin{array}{c}\tau ={\displaystyle \frac{1}{n(n-1)}}{\displaystyle \sum _{i\ne j}} t_{ij},t_{ij}^{\left(\tau \right)}=\{\begin{array}{cc}{t}_{ij},& t_{ij}\ge \tau ,\\ & \\ 0,& t_{ij}<\tau \end{array},t_{ii}^{\left(\tau \right)}=0\end{array}$$

(6)

Step 8—Cause–Effect Analysis

Compute row and column sums:

$$\begin{array}{c}{D}_i={\displaystyle \sum _{j=1}^n} t_{ij},R_i={\displaystyle \sum _{j=1}^n} t_{ji}\end{array}$$

(7)

Then calculate prominence and relation:

$$\begin{array}{c}{P}_i=D_i+R_i,E_i=D_i-R_i\end{array}$$

(8)

Criteria with $E_i>0$ belong to the cause group, and those with $E_i<0$ to the effect group. The $(P_i,E_i)$ pairs are plotted to generate the cause–effect diagram.

4. Results

Following the fuzzy DEMATEL procedure described in Section 3.5, the analysis for each gamified recommender design—Points, Badges, and Leaderboards (PBL); Acknowledgments, Objectives, and Progression (AOP); Acknowledgments, Competition, and Time Pressure (ACT); and Acknowledgments, Objectives, and Social Pressure (AOS)—was carried out in eight computational steps aligned with Equations (1)–(8).

Step 1–2. Expert Evaluations and Linguistic Judgments

The experts were presented with the following rating scale for each of the four different gamified recommender designs: For four gamified recommender system that experts earlier tested, they evaluated each system by comparing the impact of following criteria (Effectiveness, Transparency, Persuasiveness, User Satisfaction, Trust, Usefulness, Ease of Use, Efficiency, and Education) over each other using the rating scale (No influence (NO), Very Low influence (VL), Low influence (L), High influence (H), Very High influence (VH), given in

Table 2. Rating scale for gamified recommender system design.

Criteria	C1	C2	C3	C4	C5	C6	C7	C8	C9
C1	NO/VL/L/H/VH	NO/VL/L/H/VH	NO/VL/L/H/VH	NO/VL/L/H/VH	NO/VL/L/H/VH	NO/VL/L/H/VH	NO/VL/L/H/VH	NO/VL/L/H/VH	NO/VL/L/H/VH
C2		NO/VL/L/H/VH	NO/VL/L/H/VH	NO/VL/L/H/VH	NO/VL/L/H/VH	NO/VL/L/H/VH	NO/VL/L/H/VH	NO/VL/L/H/VH	NO/VL/L/H/VH
C3			NO/VL/L/H/VH	NO/VL/L/H/VH	NO/VL/L/H/VH	NO/VL/L/H/VH	NO/VL/L/H/VH	NO/VL/L/H/VH	NO/VL/L/H/VH
C4				NO/VL/L/H/VH	NO/VL/L/H/VH	NO/VL/L/H/VH	NO/VL/L/H/VH	NO/VL/L/H/VH	NO/VL/L/H/VH
C5					NO/VL/L/H/VH	NO/VL/L/H/VH	NO/VL/L/H/VH	NO/VL/L/H/VH	NO/VL/L/H/VH
C6						NO/VL/L/H/VH	NO/VL/L/H/VH	NO/VL/L/H/VH	NO/VL/L/H/VH
C7							NO/VL/L/H/VH	NO/VL/L/H/VH	NO/VL/L/H/VH
C8								NO/VL/L/H/VH	NO/VL/L/H/VH
C9									NO/VL/L/H/VH

). The sample evaluation rating scale is given below:

C1: Effectiveness: is the Gamified Recommendation System effective in achieving its intended aims?

C2: Transparency: are Gamified Recommendation Systems visible and open in the way they work?

C3: Persuasiveness: does the Gamified Recommendation System influence and motivate users?

C4: User Satisfaction: are you satisfied with the performance of the Gamified Recommendation System?

C5: Trust: is the Gamified Recommendation System stable and consistent?

C6: Usefulness: is the Gamified Recommendation System a helpful system?

C7: Ease of use: is the Gamified Recommendation System easy to use?

C8: Efficiency: does the Gamified Recommender System effectively use resources to achieve its purpose?

C9: Education: does the Gamified Recommendation System foster successful learning outcomes?

Twenty-five decision makers (DMs) assessed the pairwise causal influence of each criterion $C_i$ on $C_j$ using the five-term linguistic scale defined as No, Very Low, Low, High, and Very High, represented by triangular fuzzy numbers (TFNs) $\left(\mathcal{l},m,u\right)$ shown in Table 2 which defines the linguistic-to-TFN mapping applied to all subsequent analyses. Linguistic fuzzy scale for expert judgments were used to construct the individual ${\tilde{X}}^{\left(k\right)}$ matrices.

Each expert’s judgments generated a fuzzy direct-relation matrix ${\tilde{X}}^{\left(k\right)}=\left[{\tilde{x}}_{ij}^{\left(k\right)}\right]$ as described in Equation (1), where diagonal elements were fixed to (0, 0, 0).

Because all designs share the same evaluation criteria, four parallel sets of matrices were created—one for each gamified design.

Step 3. Aggregation of Expert Opinions

The twenty-five individual matrices were aggregated through the geometric-mean operator (Equation (2)) to obtain the group fuzzy direct-relation matrix $\tilde{X}$.

After defuzzification via the centroid method (Equation (3)), the crisp direct-relation matrices $X=\left[x_{ij}\right]$ were generated for each design.

The mean causal influences of each criterion are computed in step 3.

Step 4. Normalization of the Direct-Relation Matrix

To guarantee comparability and convergence, each X was normalized according to Equation (4), resulting in the normalized fuzzy direct-relation matrices N.

The normalized matrices, essential for total-relation computation were performed in step 4.

Step 5. Computation of the Total-Relation Matrix

Utilizing Equation (5), $T=N(I-N{)}^{-1}$, each normalized matrix was expanded to encompass both direct and indirect influences. The fuzzy total-relation matrices are computed.After the defuzzification process, the resulting crisp total-relation matrices were generated in step 5.

Step 6. Threshold Determination and Network Relation Map (NRM)

For each design, the average of all off-diagonal $t_{ij}$ values was used as the threshold $\tau$ (Equation (6)).

Relations below $\tau$ were omitted to emphasize significant causal influences.

The thresholded total relation matrices for each design were calculated in step 6. The corresponding network relation maps (NRMs), which visually depict causal linkages among criteria, are illustrated in Figure 5 (PBL), Figure 6 (AOP), Figure 7 (AOS), and Figure 8 (ACT).

Step 7. Cause–Effect Computation

The sums of rows and columns of $T$-provided $D_i$ and $R_i$ (Equation (7)). Prominence ($D_i+R_i$) and relation ($D_i-R_i$) were subsequently derived (Equation (8)).

The final output matrices were computed using step 7.

Step 8. Comparative and Interpretive Analysis

Positive $E_i$ values show cause criteria. Negative $E_i$ values denote effect criteria.

In all four designs, Perceived Usefulness (C6) and Ease of Use (C7) consistently emerge as primary causal factors influencing system behavior, while Engagement (C8) and Satisfaction (C9) serve as outcome criteria.

Among the designs, AOP exhibits the strongest overall interdependence, suggesting that acknowledgment and progression mechanisms amplify systemic influence propagation, while ACT shows moderate causal density governed by competition and time pressure components.

4.1. Results for Design 1: Points, Badges and Leaderboards (PBL)

Table 3. The direct-relation matrix—Design 1.

	C1	C2	C3	C4	C5	C6	C7	C8	C9
C1	(0.000, 0.000, 0.000)	(0.300, 0.540, 0.760)	(0.310, 0.540, 0.750)	(0.430, 0.670, 0.860)	(0.360, 0.560, 0.770)	(0.270, 0.480, 0.710)	(0.320, 0.560, 0.780)	(0.390, 0.600, 0.780)	(0.290, 0.490, 0.720)
C2	(0.300, 0.540, 0.760)	(0.000, 0.000, 0.000)	(0.270, 0.480, 0.710)	(0.380, 0.620, 0.820)	(0.280, 0.480, 0.720)	(0.330, 0.540, 0.750)	(0.360, 0.600, 0.820)	(0.280, 0.460, 0.700)	(0.380, 0.630, 0.840)
C3	(0.310, 0.540, 0.750)	(0.270, 0.480, 0.710)	(0.000, 0.000, 0.000)	(0.400, 0.620, 0.810)	(0.320, 0.530, 0.740)	(0.350, 0.570, 0.780)	(0.290, 0.510, 0.710)	(0.330, 0.560, 0.770)	(0.240, 0.430, 0.670)
C4	(0.430, 0.670, 0.860)	(0.380, 0.620, 0.820)	(0.400, 0.620, 0.810)	(0.000, 0.000, 0.000)	(0.390, 0.590, 0.770)	(0.390, 0.620, 0.830)	(0.420, 0.650, 0.840)	(0.320, 0.550, 0.770)	(0.360, 0.560, 0.760)
C5	(0.360, 0.560, 0.770)	(0.280, 0.480, 0.720)	(0.320, 0.530, 0.740)	(0.390, 0.590, 0.770)	(0.000, 0.000, 0.000)	(0.450, 0.650, 0.800)	(0.280, 0.490, 0.700)	(0.400, 0.630, 0.840)	(0.290, 0.500, 0.740)
C6	(0.270, 0.480, 0.710)	(0.330, 0.540, 0.750)	(0.350, 0.570, 0.780)	(0.390, 0.620, 0.830)	(0.450, 0.650, 0.800)	(0.000, 0.000, 0.000)	(0.400, 0.600, 0.770)	(0.310, 0.540, 0.760)	(0.280, 0.490, 0.730)
C7	(0.320, 0.560, 0.780)	(0.360, 0.600, 0.820)	(0.290, 0.510, 0.710)	(0.420, 0.650, 0.840)	(0.280, 0.490, 0.700)	(0.400, 0.600, 0.770)	(0.000, 0.000, 0.000)	(0.330, 0.540, 0.720)	(0.300, 0.530, 0.750)
C8	(0.390, 0.600, 0.780)	(0.280, 0.460, 0.700)	(0.330, 0.560, 0.770)	(0.320, 0.550, 0.770)	(0.400, 0.630, 0.840)	(0.310, 0.540, 0.760)	(0.330, 0.540, 0.720)	(0.000, 0.000, 0.000)	(0.340, 0.540, 0.740)
C9	(0.290, 0.490, 0.720)	(0.380, 0.630, 0.840)	(0.240, 0.430, 0.670)	(0.360, 0.560, 0.760)	(0.290, 0.500, 0.740)	(0.280, 0.490, 0.730)	(0.300, 0.530, 0.750)	(0.330, 0.530, 0.730)	(0.000, 0.000, 0.000)

, below, indicates the direct-relation matrix, which is the same as the pairwise comparison matrix of the decision makers in Design 1.

Table 4. The normalized fuzzy direct-relation matrix—Design 1.

	C1	C2	C3	C4	C5	C6	C7	C8	C9
C1	(0.000, 0.000, 0.000)	(0.046, 0.084, 0.118)	(0.048, 0.084, 0.116)	(0.067, 0.104, 0.133)	(0.056, 0.087, 0.119)	(0.042, 0.074, 0.110)	(0.050, 0.087, 0.121)	(0.060, 0.093, 0.121)	(0.045, 0.076, 0.111)
C2	(0.046, 0.084, 0.118)	(0.000, 0.000, 0.000)	(0.042, 0.074, 0.110)	(0.059, 0.096, 0.127)	(0.043, 0.074, 0.111)	(0.051, 0.084, 0.116)	(0.056, 0.093, 0.127)	(0.043, 0.071, 0.108)	(0.059, 0.098, 0.130)
C3	(0.048, 0.084, 0.116)	(0.042, 0.074, 0.110)	(0.000, 0.000, 0.000)	(0.062, 0.096, 0.125)	(0.050, 0.082, 0.115)	(0.054, 0.088, 0.121)	(0.045, 0.079, 0.110)	(0.051, 0.087, 0.119)	(0.037, 0.067, 0.104)
C4	(0.067, 0.104, 0.133)	(0.059, 0.096, 0.127)	(0.062, 0.096, 0.125)	(0.000, 0.000, 0.000)	(0.060, 0.091, 0.119)	(0.060, 0.096, 0.128)	(0.065, 0.101, 0.130)	(0.050, 0.085, 0.119)	(0.056, 0.087, 0.118)
C5	(0.056, 0.087, 0.119)	(0.043, 0.074, 0.111)	(0.050, 0.082, 0.115)	(0.060, 0.091, 0.119)	(0.000, 0.000, 0.000)	(0.070, 0.101, 0.124)	(0.043, 0.076, 0.108)	(0.062, 0.098, 0.130)	(0.045, 0.077, 0.115)
C6	(0.042, 0.074, 0.110)	(0.051, 0.084, 0.116)	(0.054, 0.088, 0.121)	(0.060, 0.096, 0.128)	(0.070, 0.101, 0.124)	(0.000, 0.000, 0.000)	(0.062, 0.093, 0.119)	(0.048, 0.084, 0.118)	(0.043, 0.076, 0.113)
C7	(0.050, 0.087, 0.121)	(0.056, 0.093, 0.127)	(0.045, 0.079, 0.110)	(0.065, 0.101, 0.130)	(0.043, 0.076, 0.108)	(0.062, 0.093, 0.119)	(0.000, 0.000, 0.000)	(0.051, 0.084, 0.111)	(0.046, 0.082, 0.116)
C8	(0.060, 0.093, 0.121)	(0.043, 0.071, 0.108)	(0.051, 0.087, 0.119)	(0.050, 0.085, 0.119)	(0.062, 0.098, 0.130)	(0.048, 0.084, 0.118)	(0.051, 0.084, 0.111)	(0.000, 0.000, 0.000)	(0.053, 0.084, 0.115)
C9	(0.045, 0.076, 0.111)	(0.059, 0.098, 0.130)	(0.037, 0.067, 0.104)	(0.056, 0.087, 0.118)	(0.045, 0.077, 0.115)	(0.043, 0.076, 0.113)	(0.046, 0.082, 0.116)	(0.051, 0.082, 0.113)	(0.000, 0.000, 0.000)

, below, shows the normalized fuzzy direct-relation matrix of Design 1.

Table 5. The fuzzy total-relation matrix—Design 1.

	C1	C2	C3	C4	C5	C6	C7	C8	C9
C1	(0.036, 0.175, 1.863)	(0.078, 0.248, 1.964)	(0.079, 0.245, 1.915)	(0.103, 0.284, 2.068)	(0.089, 0.254, 1.954)	(0.077, 0.246, 1.961)	(0.083, 0.256, 1.960)	(0.092, 0.258, 1.953)	(0.076, 0.235, 1.913)
C2	(0.078, 0.248, 1.964)	(0.033, 0.168, 1.856)	(0.072, 0.233, 1.906)	(0.094, 0.274, 2.059)	(0.077, 0.239, 1.944)	(0.084, 0.250, 1.962)	(0.087, 0.258, 1.961)	(0.076, 0.236, 1.939)	(0.087, 0.250, 1.925)
C3	(0.079, 0.245, 1.915)	(0.072, 0.233, 1.906)	(0.032, 0.161, 1.760)	(0.096, 0.270, 2.008)	(0.082, 0.243, 1.899)	(0.086, 0.250, 1.918)	(0.077, 0.242, 1.899)	(0.082, 0.246, 1.900)	(0.067, 0.220, 1.856)
C4	(0.103, 0.284, 2.068)	(0.094, 0.274, 2.059)	(0.096, 0.270, 2.008)	(0.046, 0.207, 2.043)	(0.099, 0.273, 2.041)	(0.099, 0.279, 2.064)	(0.102, 0.283, 2.054)	(0.088, 0.267, 2.038)	(0.090, 0.259, 2.004)
C5	(0.089, 0.254, 1.954)	(0.077, 0.239, 1.944)	(0.082, 0.243, 1.899)	(0.099, 0.273, 2.041)	(0.038, 0.174, 1.833)	(0.103, 0.267, 1.957)	(0.079, 0.246, 1.935)	(0.095, 0.262, 1.946)	(0.077, 0.236, 1.901)
C6	(0.077, 0.246, 1.961)	(0.084, 0.250, 1.962)	(0.086, 0.250, 1.918)	(0.099, 0.279, 2.064)	(0.103, 0.267, 1.957)	(0.038, 0.178, 1.861)	(0.095, 0.262, 1.958)	(0.082, 0.252, 1.950)	(0.075, 0.236, 1.914)
C7	(0.083, 0.256, 1.960)	(0.087, 0.258, 1.961)	(0.077, 0.242, 1.899)	(0.102, 0.283, 2.054)	(0.079, 0.246, 1.935)	(0.095, 0.262, 1.958)	(0.036, 0.178, 1.841)	(0.084, 0.251, 1.934)	(0.078, 0.242, 1.906)
C8	(0.092, 0.258, 1.955)	(0.076, 0.236, 1.942)	(0.082, 0.246, 1.903)	(0.088, 0.267, 2.041)	(0.095, 0.262, 1.948)	(0.083, 0.252, 1.952)	(0.084, 0.252, 1.937)	(0.036, 0.172, 1.830)	(0.083, 0.240, 1.901)
C9	(0.076, 0.235, 1.910)	(0.087, 0.250, 1.922)	(0.067, 0.220, 1.854)	(0.090, 0.258, 2.001)	(0.077, 0.235, 1.898)	(0.075, 0.236, 1.911)	(0.077, 0.241, 1.904)	(0.081, 0.238, 1.894)	(0.031, 0.155, 1.762)

, below, shows the fuzzy total-relation matrix of Design 1.

Table 6. The crisp total-relation matrix—Design 1.

	C1	C2	C3	C4	C5	C6	C7	C8	C9
C1	0.465	0.54	0.528	0.584	0.543	0.537	0.545	0.546	0.52
C2	0.54	0.458	0.518	0.574	0.529	0.541	0.547	0.526	0.535
C3	0.53	0.519	0.436	0.564	0.526	0.535	0.525	0.528	0.5
C4	0.584	0.574	0.562	0.517	0.571	0.579	0.581	0.565	0.553
C5	0.544	0.53	0.525	0.571	0.459	0.555	0.534	0.548	0.519
C6	0.537	0.541	0.534	0.579	0.555	0.467	0.551	0.54	0.522
C7	0.545	0.547	0.524	0.581	0.533	0.551	0.463	0.538	0.525
C8	0.548	0.527	0.528	0.566	0.549	0.542	0.539	0.457	0.523
C9	0.521	0.536	0.499	0.554	0.519	0.522	0.525	0.521	0.432

, below, shows the crisp total-relation matrix of Design 1.

In this design, the threshold value is equal to 0.533 (see

Table 7. The crisp total-relationships matrix by considering the threshold value—Design 1.

	C1	C2	C3	C4	C5	C6	C7	C8	C9
C1	0	0.54	0	0.584	0.543	0.537	0.545	0.546	0
C2	0.54	0	0	0.574	0	0.541	0.547	0	0.535
C3	0	0	0	0.564	0	0.535	0	0	0
C4	0.584	0.574	0.562	0	0.571	0.579	0.581	0.565	0.553
C5	0.544	0	0	0.571	0	0.555	0.534	0.548	0
C6	0.537	0.541	0.534	0.579	0.555	0	0.551	0.54	0
C7	0.545	0.547	0	0.581	0.533	0.551	0	0.538	0
C8	0.548	0	0	0.566	0.549	0.542	0.539	0	0
C9	0	0.536	0	0.554	0	0	0	0	0

).

Table 8. The final output—Design 1.

	R	D	D+R	D − R
C1	4.814	4.808	9.622	−0.006
C2	4.772	4.768	9.54	−0.004
C3	4.655	4.664	9.318	0.009
C4	5.09	5.085	10.175	−0.006
C5	4.784	4.785	9.569	0.002
C6	4.83	4.826	9.656	−0.004
C7	4.81	4.808	9.618	−0.002
C8	4.769	4.778	9.548	0.009
C9	4.629	4.631	9.259	0.002

, below, shows the final output for Design 1.

The cause–effect diagram for Design 1 (PBL) is given below in Figure 5.

Figure 5. Cause–effect diagram—Design 1.

Figure 5. Cause–effect diagram—Design 1.

4.2. Results for Design 2: Acknowledgments, Objectives, and Progression (AOP)

Table 9. The direct-relation matrix—Design 2.

	C1	C2	C3	C4	C5	C6	C7	C8	C9
C1	(0.000, 0.000, 0.000)	(0.370, 0.590, 0.790)	(0.350, 0.600, 0.800)	(0.410, 0.650, 0.830)	(0.370, 0.600, 0.800)	(0.420, 0.640, 0.820)	(0.550, 0.800, 0.930)	(0.250, 0.430, 0.640)	(0.340, 0.560, 0.780)
C2	(0.370, 0.590, 0.790)	(0.000, 0.000, 0.000)	(0.230, 0.440, 0.680)	(0.290, 0.520, 0.750)	(0.390, 0.620, 0.790)	(0.340, 0.560, 0.780)	(0.280, 0.490, 0.700)	(0.210, 0.400, 0.630)	(0.320, 0.550, 0.780)
C3	(0.350, 0.600, 0.800)	(0.230, 0.440, 0.680)	(0.000, 0.000, 0.000)	(0.350, 0.560, 0.760)	(0.360, 0.580, 0.780)	(0.390, 0.640, 0.850)	(0.350, 0.590, 0.800)	(0.330, 0.560, 0.770)	(0.330, 0.570, 0.810)
C4	(0.400, 0.640, 0.820)	(0.290, 0.520, 0.750)	(0.350, 0.560, 0.760)	(0.000, 0.000, 0.000)	(0.280, 0.490, 0.710)	(0.330, 0.550, 0.750)	(0.410, 0.650, 0.840)	(0.370, 0.580, 0.770)	(0.310, 0.550, 0.770)
C5	(0.370, 0.600, 0.800)	(0.380, 0.610, 0.790)	(0.370, 0.590, 0.780)	(0.280, 0.490, 0.710)	(0.000, 0.000, 0.000)	(0.370, 0.600, 0.790)	(0.340, 0.570, 0.790)	(0.280, 0.500, 0.730)	(0.240, 0.440, 0.690)
C6	(0.420, 0.640, 0.820)	(0.340, 0.560, 0.780)	(0.390, 0.640, 0.850)	(0.330, 0.550, 0.750)	(0.370, 0.600, 0.790)	(0.000, 0.000, 0.000)	(0.400, 0.640, 0.840)	(0.280, 0.500, 0.700)	(0.270, 0.510, 0.720)
C7	(0.530, 0.780, 0.920)	(0.260, 0.470, 0.690)	(0.350, 0.590, 0.800)	(0.410, 0.650, 0.840)	(0.340, 0.570, 0.790)	(0.400, 0.640, 0.840)	(0.000, 0.000, 0.000)	(0.310, 0.510, 0.720)	(0.280, 0.500, 0.700)
C8	(0.250, 0.430, 0.640)	(0.210, 0.400, 0.630)	(0.330, 0.560, 0.770)	(0.370, 0.580, 0.770)	(0.280, 0.500, 0.730)	(0.280, 0.500, 0.700)	(0.310, 0.510, 0.720)	(0.000, 0.000, 0.000)	(0.290, 0.490, 0.700)
C9	(0.340, 0.560, 0.780)	(0.320, 0.550, 0.780)	(0.330, 0.570, 0.810)	(0.310, 0.550, 0.770)	(0.240, 0.440, 0.690)	(0.270, 0.510, 0.720)	(0.280, 0.500, 0.700)	(0.290, 0.490, 0.700)	(0.000, 0.000, 0.000)

, below, indicates the direct-relation matrix, which is the same as the pairwise comparison matrix of the decision makers in Design 2.

Table 10. The normalized fuzzy direct-relation matrix—Design 2.

	C1	C2	C3	C4	C5	C6	C7	C8	C9
C1	(0.000, 0.000, 0.000)	(0.058, 0.092, 0.124)	(0.055, 0.094, 0.125)	(0.064, 0.102, 0.130)	(0.058, 0.094, 0.125)	(0.066, 0.100, 0.128)	(0.086, 0.125, 0.146)	(0.039, 0.067, 0.100)	(0.053, 0.088, 0.122)
C2	(0.058, 0.092, 0.124)	(0.000, 0.000, 0.000)	(0.036, 0.069, 0.106)	(0.045, 0.081, 0.117)	(0.061, 0.097, 0.124)	(0.053, 0.088, 0.122)	(0.044, 0.077, 0.110)	(0.033, 0.063, 0.099)	(0.050, 0.086, 0.122)
C3	(0.055, 0.094, 0.125)	(0.036, 0.069, 0.106)	(0.000, 0.000, 0.000)	(0.055, 0.088, 0.119)	(0.056, 0.091, 0.122)	(0.061, 0.100, 0.133)	(0.055, 0.092, 0.125)	(0.052, 0.088, 0.121)	(0.052, 0.089, 0.127)
C4	(0.063, 0.100, 0.128)	(0.045, 0.081, 0.117)	(0.055, 0.088, 0.119)	(0.000, 0.000, 0.000)	(0.044, 0.077, 0.111)	(0.052, 0.086, 0.117)	(0.064, 0.102, 0.131)	(0.058, 0.091, 0.121)	(0.049, 0.086, 0.121)
C5	(0.058, 0.094, 0.125)	(0.059, 0.095, 0.124)	(0.058, 0.092, 0.122)	(0.044, 0.077, 0.111)	(0.000, 0.000, 0.000)	(0.058, 0.094, 0.124)	(0.053, 0.089, 0.124)	(0.044, 0.078, 0.114)	(0.038, 0.069, 0.108)
C6	(0.066, 0.100, 0.128)	(0.053, 0.088, 0.122)	(0.061, 0.100, 0.133)	(0.052, 0.086, 0.117)	(0.058, 0.094, 0.124)	(0.000, 0.000, 0.000)	(0.063, 0.100, 0.131)	(0.044, 0.078, 0.110)	(0.042, 0.080, 0.113)
C7	(0.083, 0.122, 0.144)	(0.041, 0.074, 0.108)	(0.055, 0.092, 0.125)	(0.064, 0.102, 0.131)	(0.053, 0.089, 0.124)	(0.063, 0.100, 0.131)	(0.000, 0.000, 0.000)	(0.049, 0.080, 0.113)	(0.044, 0.078, 0.110)
C8	(0.039, 0.067, 0.100)	(0.033, 0.063, 0.099)	(0.052, 0.088, 0.121)	(0.058, 0.091, 0.121)	(0.044, 0.078, 0.114)	(0.044, 0.078, 0.110)	(0.049, 0.080, 0.113)	(0.000, 0.000, 0.000)	(0.045, 0.077, 0.110)
C9	(0.053, 0.088, 0.122)	(0.050, 0.086, 0.122)	(0.052, 0.089, 0.127)	(0.049, 0.086, 0.121)	(0.038, 0.069, 0.108)	(0.042, 0.080, 0.113)	(0.044, 0.078, 0.110)	(0.045, 0.077, 0.110)	(0.000, 0.000, 0.000)

, below, shows the normalized fuzzy direct-relation matrix of Design 2.

Table 11. The fuzzy total-relation matrix—Design 2.

	C1	C2	C3	C4	C5	C6	C7	C8	C9
C1	(0.047, 0.220, 2.550)	(0.092, 0.273, 2.484)	(0.093, 0.292, 2.614)	(0.103, 0.298, 2.593)	(0.095, 0.286, 2.553)	(0.105, 0.301, 2.619)	(0.125, 0.327, 2.659)	(0.074, 0.245, 2.380)	(0.087, 0.270, 2.503)
C2	(0.092, 0.274, 2.481)	(0.030, 0.164, 2.209)	(0.068, 0.243, 2.424)	(0.077, 0.253, 2.410)	(0.090, 0.261, 2.382)	(0.085, 0.262, 2.439)	(0.078, 0.257, 2.454)	(0.060, 0.216, 2.219)	(0.077, 0.243, 2.336)
C3	(0.093, 0.290, 2.607)	(0.068, 0.241, 2.421)	(0.037, 0.193, 2.451)	(0.089, 0.273, 2.532)	(0.089, 0.270, 2.500)	(0.095, 0.287, 2.570)	(0.092, 0.286, 2.590)	(0.081, 0.250, 2.349)	(0.081, 0.259, 2.457)
C4	(0.101, 0.296, 2.581)	(0.076, 0.251, 2.402)	(0.089, 0.273, 2.529)	(0.038, 0.193, 2.398)	(0.078, 0.258, 2.463)	(0.088, 0.276, 2.529)	(0.101, 0.294, 2.566)	(0.087, 0.253, 2.323)	(0.079, 0.257, 2.425)
C5	(0.095, 0.285, 2.547)	(0.088, 0.259, 2.378)	(0.090, 0.272, 2.500)	(0.078, 0.258, 2.467)	(0.035, 0.182, 2.333)	(0.092, 0.277, 2.503)	(0.089, 0.277, 2.529)	(0.073, 0.237, 2.289)	(0.068, 0.237, 2.385)
C6	(0.105, 0.300, 2.612)	(0.085, 0.261, 2.435)	(0.096, 0.288, 2.570)	(0.088, 0.276, 2.533)	(0.092, 0.277, 2.503)	(0.040, 0.201, 2.455)	(0.100, 0.297, 2.597)	(0.075, 0.246, 2.342)	(0.074, 0.255, 2.448)
C7	(0.121, 0.322, 2.644)	(0.074, 0.252, 2.442)	(0.091, 0.285, 2.583)	(0.100, 0.293, 2.563)	(0.089, 0.276, 2.522)	(0.100, 0.295, 2.590)	(0.043, 0.210, 2.501)	(0.080, 0.250, 2.362)	(0.077, 0.257, 2.464)
C8	(0.073, 0.244, 2.374)	(0.060, 0.215, 2.215)	(0.081, 0.251, 2.349)	(0.087, 0.253, 2.326)	(0.073, 0.237, 2.289)	(0.075, 0.246, 2.342)	(0.080, 0.251, 2.368)	(0.028, 0.150, 2.050)	(0.071, 0.228, 2.243)
C9	(0.087, 0.269, 2.497)	(0.077, 0.242, 2.333)	(0.081, 0.260, 2.457)	(0.079, 0.257, 2.428)	(0.068, 0.237, 2.385)	(0.074, 0.255, 2.448)	(0.077, 0.258, 2.470)	(0.071, 0.228, 2.243)	(0.029, 0.164, 2.243)

, below, shows the fuzzy total-relation matrix of Design 2.

Table 12. The crisp total-relation matrix—Design 2.

	C1	C2	C3	C4	C5	C6	C7	C8	C9
C1	0.615	0.647	0.684	0.686	0.669	0.693	0.721	0.606	0.648
C2	0.651	0.516	0.617	0.623	0.625	0.635	0.634	0.559	0.602
C3	0.682	0.611	0.579	0.657	0.648	0.675	0.677	0.606	0.632
C4	0.683	0.617	0.657	0.57	0.633	0.659	0.68	0.605	0.626
C5	0.67	0.62	0.652	0.635	0.551	0.657	0.662	0.587	0.604
C6	0.692	0.63	0.675	0.659	0.655	0.586	0.687	0.601	0.627
C7	0.714	0.623	0.674	0.677	0.656	0.684	0.601	0.608	0.631
C8	0.61	0.56	0.612	0.611	0.591	0.607	0.616	0.478	0.575
C9	0.65	0.6	0.635	0.629	0.605	0.63	0.637	0.573	0.522

, below, shows the crisp total-relation matrix of Design 2.

In this design, the threshold value is equal to 0.629 (see

Table 13. The crisp total-relationships matrix by considering the threshold value—Design 2.

	C1	C2	C3	C4	C5	C6	C7	C8	C9
C1	0	0.647	0.684	0.686	0.669	0.693	0.721	0	0.648
C2	0.651	0	0	0	0	0.635	0.634	0	0
C3	0.682	0	0	0.657	0.648	0.675	0.677	0	0.632
C4	0.683	0	0.657	0	0.633	0.659	0.68	0	0
C5	0.67	0	0.652	0.635	0	0.657	0.662	0	0
C6	0.692	0.63	0.675	0.659	0.655	0	0.687	0	0
C7	0.714	0	0.674	0.677	0.656	0.684	0	0	0.631
C8	0	0	0	0	0	0	0	0	0
C9	0.65	0	0.635	0	0	0.63	0.637	0	0

).

The final output for Design 2 is shown in

Table 14. The final output—Design 2.

	R	D	D + R	D − R
C1	5.968	5.969	11.937	0.001
C2	5.425	5.461	10.886	0.036
C3	5.784	5.768	11.552	−0.017
C4	5.747	5.731	11.478	−0.015
C5	5.633	5.636	11.269	0.004
C6	5.825	5.812	11.636	−0.013
C7	5.916	5.869	11.785	−0.048
C8	5.222	5.26	10.482	0.038
C9	5.466	5.48	10.946	0.014

below.

The cause–effect diagram for Design 2 (AOP) is shown in Figure 6.

Figure 6. Cause–effect diagram—Design 2.

Figure 6. Cause–effect diagram—Design 2.

4.3. Results for Design 3: Acknowledgments, Objectives, and Social Pressure (AOS)

Table 15. The direct-relation matrix—Design 3.

	C1	C2	C3	C4	C5	C6	C7	C8	C9
C1	(0.000, 0.000, 0.000)	(0.430, 0.630, 0.770)	(0.330, 0.540, 0.720)	(0.320, 0.520, 0.730)	(0.220, 0.450, 0.680)	(0.420, 0.630, 0.800)	(0.560, 0.800, 0.930)	(0.180, 0.370, 0.620)	(0.400, 0.620, 0.780)
C2	(0.430, 0.630, 0.770)	(0.000, 0.000, 0.000)	(0.200, 0.390, 0.630)	(0.490, 0.710, 0.860)	(0.230, 0.440, 0.650)	(0.430, 0.670, 0.840)	(0.330, 0.530, 0.710)	(0.220, 0.440, 0.660)	(0.250, 0.470, 0.710)
C3	(0.300, 0.510, 0.700)	(0.230, 0.420, 0.650)	(0.000, 0.000, 0.000)	(0.440, 0.640, 0.800)	(0.360, 0.580, 0.790)	(0.440, 0.670, 0.850)	(0.390, 0.610, 0.780)	(0.150, 0.360, 0.610)	(0.190, 0.380, 0.610)
C4	(0.320, 0.520, 0.730)	(0.490, 0.710, 0.860)	(0.440, 0.640, 0.800)	(0.000, 0.000, 0.000)	(0.280, 0.480, 0.690)	(0.380, 0.630, 0.830)	(0.430, 0.640, 0.800)	(0.170, 0.390, 0.640)	(0.290, 0.510, 0.730)
C5	(0.250, 0.480, 0.700)	(0.230, 0.440, 0.650)	(0.360, 0.580, 0.790)	(0.270, 0.470, 0.690)	(0.000, 0.000, 0.000)	(0.330, 0.550, 0.770)	(0.380, 0.620, 0.790)	(0.220, 0.430, 0.660)	(0.230, 0.420, 0.640)
C6	(0.420, 0.630, 0.800)	(0.400, 0.640, 0.820)	(0.420, 0.650, 0.840)	(0.360, 0.610, 0.810)	(0.330, 0.550, 0.770)	(0.000, 0.000, 0.000)	(0.470, 0.700, 0.870)	(0.300, 0.520, 0.710)	(0.160, 0.370, 0.600)
C7	(0.560, 0.800, 0.930)	(0.350, 0.560, 0.740)	(0.370, 0.590, 0.770)	(0.430, 0.640, 0.800)	(0.380, 0.620, 0.790)	(0.480, 0.710, 0.870)	(0.000, 0.000, 0.000)	(0.300, 0.510, 0.710)	(0.240, 0.450, 0.680)
C8	(0.180, 0.370, 0.620)	(0.220, 0.440, 0.660)	(0.160, 0.370, 0.620)	(0.170, 0.390, 0.640)	(0.220, 0.430, 0.660)	(0.300, 0.520, 0.710)	(0.300, 0.510, 0.710)	(0.000, 0.000, 0.000)	(0.380, 0.580, 0.770)
C9	(0.400, 0.620, 0.780)	(0.250, 0.470, 0.710)	(0.190, 0.380, 0.610)	(0.290, 0.510, 0.730)	(0.230, 0.420, 0.640)	(0.160, 0.370, 0.600)	(0.240, 0.450, 0.680)	(0.380, 0.580, 0.770)	(0.000, 0.000, 0.000)

below indicates the direct-relation matrix, which is the same as the pairwise comparison matrix of the decision makers in Design 3.

Table 16. The normalized fuzzy direct-relation matrix—Design 3.

	C1	C2	C3	C4	C5	C6	C7	C8	C9
C1	(0.000, 0.000, 0.000)	(0.068, 0.100, 0.122)	(0.052, 0.086, 0.114)	(0.051, 0.083, 0.116)	(0.035, 0.072, 0.108)	(0.067, 0.100, 0.127)	(0.089, 0.127, 0.148)	(0.029, 0.059, 0.099)	(0.064, 0.099, 0.124)
C2	(0.068, 0.100, 0.122)	(0.000, 0.000, 0.000)	(0.032, 0.062, 0.100)	(0.078, 0.113, 0.137)	(0.037, 0.070, 0.103)	(0.068, 0.107, 0.134)	(0.052, 0.084, 0.113)	(0.035, 0.070, 0.105)	(0.040, 0.075, 0.113)
C3	(0.048, 0.081, 0.111)	(0.037, 0.067, 0.103)	(0.000, 0.000, 0.000)	(0.070, 0.102, 0.127)	(0.057, 0.092, 0.126)	(0.070, 0.107, 0.135)	(0.062, 0.097, 0.124)	(0.024, 0.057, 0.097)	(0.030, 0.060, 0.097)
C4	(0.051, 0.083, 0.116)	(0.078, 0.113, 0.137)	(0.070, 0.102, 0.127)	(0.000, 0.000, 0.000)	(0.045, 0.076, 0.110)	(0.060, 0.100, 0.132)	(0.068, 0.102, 0.127)	(0.027, 0.062, 0.102)	(0.046, 0.081, 0.116)
C5	(0.040, 0.076, 0.111)	(0.037, 0.070, 0.103)	(0.057, 0.092, 0.126)	(0.043, 0.075, 0.110)	(0.000, 0.000, 0.000)	(0.052, 0.087, 0.122)	(0.060, 0.099, 0.126)	(0.035, 0.068, 0.105)	(0.037, 0.067, 0.102)
C6	(0.067, 0.100, 0.127)	(0.064, 0.102, 0.130)	(0.067, 0.103, 0.134)	(0.057, 0.097, 0.129)	(0.052, 0.087, 0.122)	(0.000, 0.000, 0.000)	(0.075, 0.111, 0.138)	(0.048, 0.083, 0.113)	(0.025, 0.059, 0.095)
C7	(0.089, 0.127, 0.148)	(0.056, 0.089, 0.118)	(0.059, 0.094, 0.122)	(0.068, 0.102, 0.127)	(0.060, 0.099, 0.126)	(0.076, 0.113, 0.138)	(0.000, 0.000, 0.000)	(0.048, 0.081, 0.113)	(0.038, 0.072, 0.108)
C8	(0.029, 0.059, 0.099)	(0.035, 0.070, 0.105)	(0.025, 0.059, 0.099)	(0.027, 0.062, 0.102)	(0.035, 0.068, 0.105)	(0.048, 0.083, 0.113)	(0.048, 0.081, 0.113)	(0.000, 0.000, 0.000)	(0.060, 0.092, 0.122)
C9	(0.064, 0.099, 0.124)	(0.040, 0.075, 0.113)	(0.030, 0.060, 0.097)	(0.046, 0.081, 0.116)	(0.037, 0.067, 0.102)	(0.025, 0.059, 0.095)	(0.038, 0.072, 0.108)	(0.060, 0.092, 0.122)	(0.000, 0.000, 0.000)

, below, shows the normalized fuzzy direct-relation matrix of Design 3.

Table 17. The fuzzy total-relation matrix—Design 3.

	C1	C2	C3	C4	C5	C6	C7	C8	C9
C1	(0.045, 0.195, 1.596)	(0.104, 0.275, 1.663)	(0.087, 0.256, 1.639)	(0.091, 0.268, 1.705)	(0.068, 0.237, 1.605)	(0.107, 0.292, 1.764)	(0.128, 0.318, 1.780)	(0.057, 0.210, 1.524)	(0.091, 0.251, 1.577)
C2	(0.103, 0.274, 1.656)	(0.037, 0.174, 1.508)	(0.066, 0.226, 1.581)	(0.110, 0.281, 1.673)	(0.066, 0.225, 1.555)	(0.104, 0.286, 1.719)	(0.092, 0.271, 1.702)	(0.060, 0.210, 1.486)	(0.067, 0.222, 1.524)
C3	(0.084, 0.254, 1.639)	(0.071, 0.233, 1.593)	(0.034, 0.165, 1.483)	(0.102, 0.268, 1.657)	(0.085, 0.241, 1.566)	(0.105, 0.282, 1.712)	(0.100, 0.278, 1.703)	(0.048, 0.196, 1.472)	(0.057, 0.205, 1.503)
C4	(0.091, 0.269, 1.709)	(0.111, 0.284, 1.685)	(0.102, 0.268, 1.659)	(0.042, 0.189, 1.611)	(0.076, 0.239, 1.616)	(0.101, 0.290, 1.778)	(0.109, 0.295, 1.774)	(0.054, 0.211, 1.536)	(0.074, 0.234, 1.580)
C5	(0.073, 0.242, 1.613)	(0.066, 0.227, 1.567)	(0.084, 0.241, 1.568)	(0.074, 0.237, 1.616)	(0.027, 0.150, 1.430)	(0.085, 0.257, 1.675)	(0.094, 0.270, 1.677)	(0.057, 0.199, 1.455)	(0.060, 0.204, 1.483)
C6	(0.106, 0.289, 1.752)	(0.099, 0.280, 1.712)	(0.100, 0.275, 1.697)	(0.096, 0.283, 1.759)	(0.084, 0.254, 1.658)	(0.045, 0.206, 1.697)	(0.116, 0.310, 1.818)	(0.073, 0.232, 1.575)	(0.057, 0.220, 1.595)
C7	(0.129, 0.319, 1.784)	(0.096, 0.277, 1.718)	(0.097, 0.275, 1.703)	(0.109, 0.294, 1.774)	(0.093, 0.270, 1.676)	(0.119, 0.315, 1.835)	(0.051, 0.218, 1.713)	(0.076, 0.237, 1.589)	(0.071, 0.238, 1.620)
C8	(0.057, 0.210, 1.527)	(0.060, 0.211, 1.495)	(0.050, 0.196, 1.473)	(0.054, 0.210, 1.534)	(0.056, 0.199, 1.453)	(0.074, 0.235, 1.588)	(0.076, 0.237, 1.588)	(0.020, 0.123, 1.292)	(0.078, 0.213, 1.430)
C9	(0.091, 0.251, 1.577)	(0.067, 0.223, 1.531)	(0.056, 0.204, 1.501)	(0.074, 0.233, 1.576)	(0.060, 0.203, 1.478)	(0.057, 0.223, 1.606)	(0.071, 0.238, 1.616)	(0.078, 0.213, 1.428)	(0.025, 0.136, 1.350)

, below, shows the fuzzy total-relation matrix of Design 3.

Table 18. The crisp total-relation matrix—Design 3.

	C1	C2	C3	C4	C5	C6	C7	C8	C9
C1	0.435	0.508	0.489	0.509	0.467	0.539	0.562	0.432	0.473
C2	0.508	0.404	0.456	0.516	0.451	0.528	0.513	0.426	0.442
C3	0.489	0.464	0.392	0.504	0.466	0.524	0.519	0.413	0.426
C4	0.511	0.518	0.502	0.432	0.471	0.539	0.542	0.434	0.46
C5	0.475	0.455	0.467	0.472	0.371	0.499	0.509	0.413	0.422
C6	0.533	0.518	0.512	0.529	0.489	0.459	0.56	0.457	0.451
C7	0.562	0.517	0.512	0.54	0.504	0.567	0.472	0.463	0.468
C8	0.437	0.432	0.416	0.437	0.415	0.468	0.469	0.326	0.422
C9	0.478	0.447	0.427	0.463	0.422	0.46	0.473	0.421	0.346

, below, shows the crisp total-relation matrix of design 3.

In this design, the threshold value is equal to 0.472 (see

Table 19. The crisp total-relationships matrix by considering the threshold value—Design 3.

	C1	C2	C3	C4	C5	C6	C7	C8	C9
C1	0	0.508	0.489	0.509	0	0.539	0.562	0	0.473
C2	0.508	0	0	0.516	0	0.528	0.513	0	0
C3	0.489	0	0	0.504	0	0.524	0.519	0	0
C4	0.511	0.518	0.502	0	0	0.539	0.542	0	0
C5	0.475	0	0	0.472	0	0.499	0.509	0	0
C6	0.533	0.518	0.512	0.529	0.489	0	0.56	0	0
C7	0.562	0.517	0.512	0.54	0.504	0.567	0	0	0
C8	0	0	0	0	0	0	0	0	0
C9	0.478	0	0	0	0	0	0.473	0	0

).

The final output was given in

Table 20. The final output—Design 3.

	R	D	D + R	D − R
C1	4.428	4.414	8.843	−0.014
C2	4.262	4.245	8.507	−0.018
C3	4.172	4.195	8.367	0.024
C4	4.404	4.409	8.813	0.006
C5	4.055	4.083	8.138	0.028
C6	4.583	4.508	9.091	−0.075
C7	4.618	4.605	9.224	−0.013
C8	3.785	3.821	7.606	0.036
C9	3.911	3.937	7.848	0.026

below.

The cause–effect diagram for Design 3 is given in Figure 7 below.

Figure 7. Cause–effect diagram—Design 3.

Figure 7. Cause–effect diagram—Design 3.

4.4. Results for Design 4: Acknowledgments, Competition, and Time Pressure (ACT)

Table 21. The direct-relation matrix—Design 4.

	C1	C2	C3	C4	C5	C6	C7	C8	C9
C1	(0.000, 0.000, 0.000)	(0.390, 0.610, 0.780)	(0.300, 0.500, 0.670)	(0.390, 0.620, 0.800)	(0.370, 0.600, 0.810)	(0.290, 0.470, 0.680)	(0.490, 0.740, 0.890)	(0.180, 0.360, 0.590)	(0.350, 0.560, 0.750)
C2	(0.390, 0.610, 0.780)	(0.000, 0.000, 0.000)	(0.220, 0.420, 0.660)	(0.460, 0.680, 0.820)	(0.210, 0.410, 0.640)	(0.380, 0.600, 0.780)	(0.270, 0.450, 0.680)	(0.180, 0.390, 0.630)	(0.260, 0.470, 0.690)
C3	(0.300, 0.500, 0.670)	(0.220, 0.420, 0.660)	(0.000, 0.000, 0.000)	(0.390, 0.590, 0.760)	(0.270, 0.450, 0.690)	(0.390, 0.590, 0.760)	(0.350, 0.580, 0.760)	(0.250, 0.470, 0.690)	(0.180, 0.350, 0.570)
C4	(0.390, 0.620, 0.800)	(0.460, 0.680, 0.820)	(0.390, 0.590, 0.760)	(0.000, 0.000, 0.000)	(0.270, 0.450, 0.650)	(0.400, 0.620, 0.790)	(0.420, 0.660, 0.820)	(0.270, 0.490, 0.710)	(0.230, 0.420, 0.640)
C5	(0.370, 0.600, 0.810)	(0.210, 0.410, 0.640)	(0.270, 0.450, 0.690)	(0.250, 0.430, 0.650)	(0.000, 0.000, 0.000)	(0.370, 0.600, 0.810)	(0.300, 0.550, 0.750)	(0.270, 0.460, 0.710)	(0.230, 0.390, 0.600)
C6	(0.290, 0.470, 0.680)	(0.380, 0.600, 0.780)	(0.390, 0.590, 0.760)	(0.400, 0.620, 0.790)	(0.370, 0.600, 0.810)	(0.000, 0.000, 0.000)	(0.460, 0.690, 0.890)	(0.240, 0.450, 0.660)	(0.270, 0.490, 0.680)
C7	(0.490, 0.740, 0.890)	(0.270, 0.450, 0.680)	(0.350, 0.580, 0.760)	(0.420, 0.660, 0.820)	(0.300, 0.550, 0.750)	(0.470, 0.700, 0.890)	(0.000, 0.000, 0.000)	(0.360, 0.590, 0.780)	(0.270, 0.460, 0.650)
C8	(0.180, 0.360, 0.590)	(0.180, 0.390, 0.630)	(0.250, 0.470, 0.690)	(0.270, 0.490, 0.710)	(0.270, 0.460, 0.710)	(0.240, 0.450, 0.660)	(0.360, 0.590, 0.780)	(0.000, 0.000, 0.000)	(0.250, 0.400, 0.590)
C9	(0.350, 0.560, 0.750)	(0.260, 0.470, 0.690)	(0.180, 0.350, 0.570)	(0.230, 0.420, 0.640)	(0.230, 0.390, 0.600)	(0.270, 0.490, 0.680)	(0.270, 0.460, 0.650)	(0.250, 0.400, 0.590)	(0.000, 0.000, 0.000)

, below, indicates the direct-relation matrix, which is the same as the pairwise comparison matrix of the decision makers in Design 4.

Table 22. The normalized fuzzy direct-relation matrix—Design 4.

	C1	C2	C3	C4	C5	C6	C7	C8	C9
C1	(0.000, 0.000, 0.000)	(0.063, 0.098, 0.125)	(0.048, 0.080, 0.108)	(0.063, 0.100, 0.129)	(0.059, 0.096, 0.130)	(0.047, 0.076, 0.109)	(0.079, 0.119, 0.143)	(0.029, 0.058, 0.095)	(0.056, 0.090, 0.121)
C2	(0.063, 0.098, 0.125)	(0.000, 0.000, 0.000)	(0.035, 0.068, 0.106)	(0.074, 0.109, 0.132)	(0.034, 0.066, 0.103)	(0.061, 0.096, 0.125)	(0.043, 0.072, 0.109)	(0.029, 0.063, 0.101)	(0.042, 0.076, 0.111)
C3	(0.048, 0.080, 0.108)	(0.035, 0.068, 0.106)	(0.000, 0.000, 0.000)	(0.063, 0.095, 0.122)	(0.043, 0.072, 0.111)	(0.063, 0.095, 0.122)	(0.056, 0.093, 0.122)	(0.040, 0.076, 0.111)	(0.029, 0.056, 0.092)
C4	(0.063, 0.100, 0.129)	(0.074, 0.109, 0.132)	(0.063, 0.095, 0.122)	(0.000, 0.000, 0.000)	(0.043, 0.072, 0.105)	(0.064, 0.100, 0.127)	(0.068, 0.106, 0.132)	(0.043, 0.079, 0.114)	(0.037, 0.068, 0.103)
C5	(0.059, 0.096, 0.130)	(0.034, 0.066, 0.103)	(0.043, 0.072, 0.111)	(0.040, 0.069, 0.105)	(0.000, 0.000, 0.000)	(0.059, 0.096, 0.130)	(0.048, 0.088, 0.121)	(0.043, 0.074, 0.114)	(0.037, 0.063, 0.096)
C6	(0.047, 0.076, 0.109)	(0.061, 0.096, 0.125)	(0.063, 0.095, 0.122)	(0.064, 0.100, 0.127)	(0.059, 0.096, 0.130)	(0.000, 0.000, 0.000)	(0.074, 0.111, 0.143)	(0.039, 0.072, 0.106)	(0.043, 0.079, 0.109)
C7	(0.079, 0.119, 0.143)	(0.043, 0.072, 0.109)	(0.056, 0.093, 0.122)	(0.068, 0.106, 0.132)	(0.048, 0.088, 0.121)	(0.076, 0.113, 0.143)	(0.000, 0.000, 0.000)	(0.058, 0.095, 0.125)	(0.043, 0.074, 0.105)
C8	(0.029, 0.058, 0.095)	(0.029, 0.063, 0.101)	(0.040, 0.076, 0.111)	(0.043, 0.079, 0.114)	(0.043, 0.074, 0.114)	(0.039, 0.072, 0.106)	(0.058, 0.095, 0.125)	(0.000, 0.000, 0.000)	(0.040, 0.064, 0.095)
C9	(0.056, 0.090, 0.121)	(0.042, 0.076, 0.111)	(0.029, 0.056, 0.092)	(0.037, 0.068, 0.103)	(0.037, 0.063, 0.096)	(0.043, 0.079, 0.109)	(0.043, 0.074, 0.105)	(0.040, 0.064, 0.095)	(0.000, 0.000, 0.000)

, below, shows the normalized fuzzy direct-relation matrix of Design 4.

Table 23. The fuzzy total-relation matrix—Design 4.

	C1	C2	C3	C4	C5	C6	C7	C8	C9
C1	(0.041, 0.181, 1.380)	(0.094, 0.254, 1.430)	(0.080, 0.236, 1.392)	(0.100, 0.273, 1.498)	(0.089, 0.247, 1.430)	(0.085, 0.253, 1.495)	(0.115, 0.296, 1.556)	(0.058, 0.203, 1.338)	(0.083, 0.228, 1.318)
C2	(0.094, 0.254, 1.430)	(0.031, 0.151, 1.261)	(0.065, 0.211, 1.334)	(0.105, 0.265, 1.440)	(0.062, 0.208, 1.351)	(0.093, 0.254, 1.446)	(0.079, 0.241, 1.467)	(0.053, 0.193, 1.288)	(0.066, 0.203, 1.257)
C3	(0.080, 0.236, 1.392)	(0.065, 0.211, 1.334)	(0.030, 0.145, 1.216)	(0.094, 0.250, 1.408)	(0.070, 0.211, 1.335)	(0.094, 0.250, 1.419)	(0.090, 0.256, 1.453)	(0.064, 0.203, 1.274)	(0.054, 0.183, 1.220)
C4	(0.100, 0.274, 1.498)	(0.105, 0.266, 1.440)	(0.094, 0.251, 1.408)	(0.042, 0.186, 1.389)	(0.076, 0.230, 1.414)	(0.102, 0.276, 1.513)	(0.106, 0.289, 1.552)	(0.071, 0.223, 1.357)	(0.066, 0.211, 1.308)
C5	(0.089, 0.246, 1.430)	(0.061, 0.207, 1.351)	(0.070, 0.210, 1.335)	(0.072, 0.225, 1.414)	(0.028, 0.141, 1.255)	(0.089, 0.248, 1.446)	(0.081, 0.249, 1.473)	(0.065, 0.199, 1.295)	(0.060, 0.187, 1.242)
C6	(0.085, 0.253, 1.495)	(0.093, 0.253, 1.446)	(0.094, 0.250, 1.419)	(0.102, 0.275, 1.513)	(0.090, 0.249, 1.446)	(0.041, 0.184, 1.413)	(0.111, 0.291, 1.574)	(0.067, 0.217, 1.362)	(0.071, 0.219, 1.323)
C7	(0.115, 0.296, 1.556)	(0.079, 0.241, 1.467)	(0.090, 0.256, 1.453)	(0.106, 0.288, 1.552)	(0.082, 0.250, 1.473)	(0.113, 0.293, 1.574)	(0.045, 0.201, 1.486)	(0.085, 0.242, 1.410)	(0.073, 0.222, 1.351)
C8	(0.058, 0.203, 1.338)	(0.053, 0.193, 1.288)	(0.064, 0.203, 1.274)	(0.071, 0.222, 1.357)	(0.066, 0.200, 1.295)	(0.067, 0.217, 1.362)	(0.085, 0.242, 1.410)	(0.022, 0.122, 1.135)	(0.060, 0.179, 1.184)
C9	(0.083, 0.228, 1.318)	(0.066, 0.203, 1.257)	(0.054, 0.183, 1.220)	(0.066, 0.210, 1.308)	(0.060, 0.188, 1.242)	(0.071, 0.219, 1.323)	(0.073, 0.222, 1.351)	(0.060, 0.179, 1.184)	(0.022, 0.117, 1.063)

, below, shows the fuzzy total-relation matrix of Design 4.

Table 24. The crisp total-relation matrix—Design 4.

	C1	C2	C3	C4	C5	C6	C7	C8	C9
C1	0.386	0.45	0.429	0.477	0.445	0.461	0.504	0.394	0.409
C2	0.452	0.342	0.401	0.463	0.401	0.455	0.448	0.379	0.381
C3	0.432	0.401	0.33	0.446	0.402	0.449	0.458	0.385	0.36
C4	0.477	0.461	0.444	0.391	0.428	0.482	0.497	0.412	0.394
C5	0.447	0.401	0.401	0.427	0.333	0.451	0.455	0.385	0.367
C6	0.46	0.451	0.444	0.48	0.448	0.394	0.502	0.408	0.403
C7	0.503	0.445	0.454	0.496	0.452	0.504	0.418	0.435	0.409
C8	0.397	0.38	0.386	0.416	0.387	0.413	0.44	0.298	0.352
C9	0.415	0.384	0.362	0.399	0.37	0.409	0.415	0.353	0.282

, below, shows the crisp total-relation matrix of Design 4.

In this design, the threshold value is equal to 0.419 (see

Table 25. The crisp total-relationships matrix by considering the threshold value—Design 4.

	C1	C2	C3	C4	C5	C6	C7	C8	C9
C1	0	0.45	0.429	0.477	0.445	0.461	0.504	0	0
C2	0.452	0	0	0.463	0	0.455	0.448	0	0
C3	0.432	0	0	0.446	0	0.449	0.458	0	0
C4	0.477	0.461	0.444	0	0.428	0.482	0.497	0	0
C5	0.447	0	0	0.427	0	0.451	0.455	0	0
C6	0.46	0.451	0.444	0.48	0.448	0	0.502	0	0
C7	0.503	0.445	0.454	0.496	0.452	0.504	0	0.435	0
C8	0	0	0	0	0	0	0.44	0	0
C9	0	0	0	0	0	0	0	0	0

).

The final output for design4 is given in

Table 26. The final output—Design 4.

	R	D	D + R	D − R
C1	3.971	3.955	7.926	−0.016
C2	3.715	3.724	7.438	0.009
C3	3.651	3.663	7.314	0.012
C4	3.996	3.987	7.983	−0.009
C5	3.667	3.666	7.334	−0.001
C6	4.018	3.992	8.01	−0.026
C7	4.137	4.117	8.254	−0.021
C8	3.45	3.47	6.919	0.02
C9	3.356	3.388	6.744	0.032

below:

The cause–effect diagram for Design 4 is shown in Figure 8 below.

Figure 8. Cause–effect diagram—Design 4.

Figure 8. Cause–effect diagram—Design 4.

4.5. Validation

Validations was ensured through the following:

• Aggregation of expert judgments to reduce individual bias.
• Threshold sensitivity analysis.

Threshold sensitivity analysis looks at how changes to a model’s input parameters, especially those that are close to a set threshold, change the model’s output. It helps to find the values of the parameters that cause the model’s behavior or decision-making to change a lot. In short, it is about figuring out how much a model’s results change when something changes around a critical point. In fuzzy DEMATEL, the threshold determines which relationships are sufficiently robust to be represented in the causal diagram. Higher thresholds make networks that are simpler and less connected, while lower thresholds make networks that are more connected.

The thresholds for the four designs were as follows: Design 1 = 0.533, Design 2 = 0.629, Design 3 = 0.472, and Design 4 = 0.419.

Design 2 (0.629): a simpler map that only shows the most important connections.

Design 1 (0.533): a balanced view that shows both major and minor effects.

Design 3 (0.472): a denser network with more indirect effects.

Design 4 (0.419): most connections are still there, but they are less clear.

Results showed that thresholds were stable, which confirmed that criteria and causal groupings were always important.

Thresholds used were D1 = 0.533, D2 = 0.629, D3 = 0.472, D4 = 0.419; sensitivity checks showed stable (D + R) orderings across plausible ranges.

3.

Findings were validated through a comparison with previous applications of fuzzy DEMATEL in education, retail, and gamification. Researchers in [49] showed that it worked well to explain the main things that affect students’ choices of courses in school. In retail, [21] showed that it works well to find the most important things to consider when shopping in the metaverse. In gamification research, scholars in [23] emphasized its significance in examining intricate interrelations among gamification components. This study confirms the efficacy of fuzzy DEMATEL as a robust method for identifying cause–effect relationships in various domains, including the creation of gamified recommender systems.

5. Discussion

Fuzzy DEMATEL was used to evaluate four gamified recommender system architectures. The results helped demonstrate how things evolve and interact. Effectiveness, Transparency, Persuasiveness, User Satisfaction, Trust, Usefulness, Ease of Use, Efficiency, and Education were evaluated. The results show that various criteria have different effects and cause-and-effect correlations.

Two things made people satisfied, productive, and trustworthy: usefulness (C6) and ease of use (C7). This is consistent with previous research that demonstrated the significance of perceived usefulness and usability in technology adoption [21,24]. Trust (C5) functioned as a mediator, shaped by transparency and usability, thereby enhancing happiness and effectiveness, in alignment with prior research in education [51] and retail [21].

Satisfaction (C4), efficacy (C1), and education (C9) were the outcome criteria that were most affected by upstream causes. This pattern is in line with earlier research on gamification, which found that external factors boost short-term satisfaction but long-term effects depend on internal alignment [24]. C2 (transparency) and C8 (efficiency) had a moderate effect, especially on designs that are time-sensitive and deal with social issues. Persuasiveness (C3) affected designs that were based on competition and peers, which supports earlier research that showed that persuasive elements have a big impact on engagement [24].

Design-level comparisons showed that each group had a unique motivational pathway.

PBL (Design 1) was compelling but poor for learning and trust because it relied on external motivators. PBL increases short-term motivation, but this effect is not sustained, as shown in [23,24].

The most even construction was AOP Design 2. It was valuable and easy to use, improving teaching and learning. Its higher threshold (0.629) exhibited excellent causal clarity, supporting evidence indicating that explicit goals and controlled feedback improve long-term learning [51].

AOS (Design 3) employed social influence, so being honest and holding your friends accountable made people trust you and believe what you say. Education and retail research show that social approbation motivates people to try new things, reinforcing our understanding. It depended on teamwork, hence it did not work as well as prior methods.

ACT (Design 4) was more persuasive but less trustworthy and gratifying since it highlighted competitiveness and urgency. A previous study has shown that intense competition might lower intrinsic motivation [24]. The dense but unclear causal network with a threshold of 0.419 demonstrated this.

Overall, usefulness and user-friendliness are the most essential criteria for all designs, whereas satisfaction, efficacy, and education are the most important outcomes. Clarity and long-term attention make AOP the most pedagogically sustainable design. PBL and ACT may drive you temporarily, but they may damage trust and academic value. Working together helps individuals learn and stay interested, therefore AOS works best.

These results demonstrate that gamified recommender systems must go beyond points and competition. It needs goal setting, growth, and social transparency to keep people utilizing and making them work [24,51,53].

5.1. Limitations

The study has number of limitations. The study used undergraduate students as end-users to evaluate the gamified recommender systems, which may limit their applicability to younger learners, professional users, or varied cultural situations. Second, four gamification designs—PBL, AOP, AOS, and ACT—based on different theoretical frameworks were examined. These strategies are common, however, hybrid models and other designs may produce different results. The fuzzy DEMATEL approach involves subjective expert opinions. Even though group aggregation reduced individual bias, findings depend on participant perceptions. The study examined English language acquisition, however, applying the same paradigm to healthcare, e-commerce, or workplace training may provide different results. Despite their potential impact on real-world adoption, long-term engagement, cognitive load, and emotional responses were excluded from the analysis due to systematic review standards.

5.2. Implications of the Study

This work has various theoretical and practical implications. Game-based recommender system designs (AOP, ACT, AOS, PBL) are compared to show how different learning and behavioral theories affect user perception and system efficacy. This study shows that decision-making models can simplify complex design attribute interactions by integrating fuzzy DEMATEL with carefully stated criteria.

The findings can help educators, developers, and system designers integrate gamification into English language learning recommender systems. Finding causal and effect criteria can help you design better by ensuring that recognition, progress, competition, and social influence are leveraged to motivate learners and improve system usability. The methodology can be adjusted to numerous educational environments because the results are consistent across threshold levels.

5.3. Recommendations for Future Research

Based on the current results, several avenues for future research are proposed. The research could be augmented by assessing the suggested gamified recommender system designs with a broader and more heterogeneous learner demographic, inclusive of diverse educational levels and cultural contexts, to enhance generalizability. Subsequent research ought to include longitudinal studies to evaluate the enduring impacts of gamification elements on learner engagement, performance, and retention. Third, hybrid evaluation methods that combine fuzzy DEMATEL with other multi-criteria decision-making techniques, like ANP, TOPSIS, or fuzzy AHP, may give us better information about how important each criterion is compared to the others. Along with the fuzzy DEMATEL framework used in this study, future research could use other multi-criteria decision-making methods to confirm or build on these results. Distance-based methods, like the Technique for Order Preference by Similarity to Ideal Solution (TOPSIS), make it easier to rank gamification strategies based on how close they are to the best engagement results.

Utility-based methods like the Analytic Hierarchy Process (AHP) give you structured pairwise preferences that show what learners really care about. New hybrid and range-sensitive models, such as the Ranking based on the Distances and Range (RADAR) method, also make it possible to combine robustness with interpretability [69]. Using and comparing these methods on bigger datasets might make the analysis more reliable and give us a better picture of how causal–evaluative relationships work in the creation of gamified recommender systems. Generalizability may be improved by comprehensive empirical validation using learner datasets from varied educational contexts and longitudinal assessments. Using causal modeling and forecasting methodologies like machine learning-based recommender models helps us understand how gamification dynamics evolve over time and effect long-term learner engagement. The study’s methods show that making decisions based on more than one factor can help with complex rating problems. Fuzzy DEMATEL shows how different factors affect gamified system like how gray-correlation-based hybrid MCDM models help choose sustainable materials [70]. Future research should combine subjective user-centric elements with objective performance metrics like system usage statistics, response precision, and learning advancement indicators to improve fuzzy DEMATEL analysis of causal relationships’ reliability and external validity. To reduce subjectivity and improve causal inferences in gamified recommender system evaluations, future research should combine linguistic expert assessments with objective system interaction data like activity logs, completion rates, and engagement metrics. The paradigm could be effective in STEM education, job training, and workplace learning, not just in English. Future research should examine adaptive gamification, where system components dynamically adjust to learners’ profiles, preferences, and progress to improve personalized and effective instruction.

6. Conclusions

This research employed the fuzzy DEMATEL method to evaluate four sophisticated gamified recommender system designs for English language acquisition. A systematic review evaluated designs using nine criteria: Effectiveness, Transparency, Persuasiveness, Satisfaction, Trust, Usefulness, Ease of Use, Efficiency, and Education. Gamification strategies like Points, Badges, Leaderboards (PBL), Acknowledgments, Objectives, Progression (AOP), Acknowledgements, Objectives, Social Pressure (AOS), and Acknowledgements, Competition, Time Pressure (ACT) vary in clarity, motivation, and social influence.

This study used fuzzy DEMATEL to identify the main user-centric factors influencing GRS adoption. The proposed approach answered all three research questions.

The nine user-centric criteria in RQ1 showed unique causal linkages, demonstrating a hierarchy of causal and effective factors. Perceived Usefulness (C6) and Ease of Use (C7) consistently influenced Engagement (C8) and Satisfaction (C9) outcome variables. In all four gamified recommender system designs, Perceived Usefulness (C6) and Ease of Use (C7) were the main causal factors. These factors had minimal effects across configurations, but their relative importance and causal direction remained unchanged. This suggests that these parameters predict user engagement and adoption beyond design effects.

In response to RQ2, the analysis found factors C6 and C7 to be the main drivers of GRS adoption due to their predominance and positive correlation. This shows that usability and perceived value are essential for system engagement.

A comparison of four gamified designs—PBL, AOP, AOS, and ACT—showed that the Acknowledgments, Objectives, and Progression (AOP) design improved network connectivity and causal balance in response to RQ3. However, competition and time constraint in the ACT design had localized consequences.

Due to its high threshold, Design 2 had the greatest causal structure, but Design 1 was more egalitarian. The results are strong since causal network densities did not affect criteria importance across thresholds. Research shows that fuzzy DEMATEL efficiently analyzes gamification criteria’s interrelationships, making educational recommender systems easier to construct. Methodological comprehension is improved by providing fuzzy DEMATEL in gamified environments and providing practical guidance for adopting learner-centered gamification mechanics. As one of the first systematic attempts to combine gamification design with fuzzy DEMATEL in recommender systems, this study offers scholars and practitioners a new perspective. The study achieved its goals by (i) explaining causal interdependencies among user-centric criteria, (ii) identifying the most important adoption determinants, and (iii) showing how gamified design configurations alter these linkages. Fuzzy DEMATEL provides a structured approach for analyzing user-centric gamification strategies and improving system design. The study shows that fuzzy DEMATEL effectively investigates gamification criteria interdependencies, enabling educational recommender system creation. Methodological approaches are improved by testing fuzzy DEMATEL in gamified systems and offering practical guidance for choosing learner-centered gamification mechanics.

Importantly, this is among the first studies to systematically integrate gamification design with fuzzy DEMATEL for recommender systems, providing a novel perspective for both researchers and practitioners.